Quest for an Asset Viewer

Week 1

What a tough 2 week sprint!

The first task to pick off was getting screen saving and loading working. This was actually really easy to get going because serialization was built into my sprocket system. So in order to save and load all I had to do was serialize out the sprocket data to xml and then for loading I just had to write some code to place the loaded data into the right containers. I also ported over the code for throwing up the save/load dialogs. I kinda cheated when I originally setup the save/load dialogs. I just call the default win32 dialogs rather than create my own dialogs. I neatly packaged it up in my platform code so it should be easy to abstract it out when I do eventually get the code running cross-platform.

The next big thing was fixing the broken window drag functionality. I had a really tough time for this one for stupid reasons. It made for a really bad morning. Basically the problem was that all my mouse handling code was in local space. This works great for some things but when you're trying to move an object and the local space is changing everytime you move the object it creates some serious jitter! I didn't have a good way to expose the screen space deltas which is what I needed to move things properly. What I wound up having to do was calculate the delta based on the parent's coordinate space. I can't help but feel that's going to bite me down the road but for now the mouse drag is actually working better than before I started refactoring the UI system.

Once drag was working I was able to hook up the scale and translate buttons in the gizmo I created for manipulating UI Elements. I decided to leave rotation alone for now. Currently my UIClickable sprocket doesn't take into account rotation. I don't imagine it'd be a tough change, I already have a transformation matrix, it's just that I'm only using the translation element of it. Still I know it's going to be a lot more complicated then just multiplying by a rotation matrix.

I also spent some time just getting all the UI elements working with the UI Editor. While I was mucking around with that I needed to get arrays working with the protoform editor for certain create params. I managed to get them working the only problem is that they only work for simple types. So the problem is that I'm already using comma separated lists for the complicated types like Float3 which take multiple parameters for initialization and the way I'm parsing arrays is a comma separated list. This shouldn't be too tough to clean up in the future, just use brackets or something to separate items between commas.

Another big thing for this week was building a better way of viewing loaded assets as well as providing a means to select what textures to use in a UI element. The old way to select a texture for an element was to open up the xml package, check what I had set the id for on that particular asset and then type that id into the texture field in the protoform editor. I was about to start laying out the components in code when I realized this was a perfect test case for the editor! I always find when I actually start using a tool I find all sorts of bugs and come up with new feature ideas. The first bug/problem I ran into was that objects sorted in the order that they were placed. This was no good, I needed to be able to control the layering. For a simple solution to this problem I decided to use the z value I had been ignoring up until this point to sort the objects. I wrote some code that does a sort whenever an objects z value changed.

So once I had the sorting issue worked out I set about creating the asset entry ui element. This was easy, lay out a UISprite background, UISprite for a thumbnail placeholder and a UIText for the asset name and I was done. Then I realized that I had no way to load the object up and actually hook some code into it to replace the placeholder elements. I decided that the best thing to do was just start pounding out some code and see how it feels. I had the objects loaded into a UIContainer object and set up some code so I could clone that container so that I could create multiple objects using that template. I set up a UIAssetViewer class that held a bunch of these containers and did the logic for swapping out the placeholders. It works but I think it still needs some refinement.

Week 2

In the second week I managed to get the asset panels loaded and populating a UI container. There was another big cleanup operation, I needed to get scroll bars fixed up and switch my window class to use the UIComponentContainer for storing all the objects parented to it. This wasn't too bad of a change it was mostly copy and paste with a few little fix ups.  The scroll bar on the other hand.....

I spent the majority of the week working on getting this damn scroll bar working. Funny thing is it's still not working properly. The big problem was that I was trying to debug something that was a few layers deep. Getting all the math working properly was just turning out to be a huge pain. I managed to get something kind of working and that's going to have to do for now. The way it works is that it's designed to just move the components in the container around. It seems easy enough but for some reason I just couldn't get the math right for the life of me. I think it's one of those silly things where if I sit down and draw it out on paper it's going to become painfully obvious where the problem is coming from.

I also spent a little bit of time on cleanup. I cleaned up how I was calculating a lot of my element positions so that it's more data driven. This will make reskinning my UI elements a breeze. As it stood before I would had to have adjusted a lot of math but now it should just work.

WIP shot while I was getting the dragging working.

Showing off a few of the different UI elements.

First shot of the Asset Viewer working.

Adjusted the dimensions a bit to something that fit better on screen.

Showing off the scaled down UI. It makes better use of space but kind of loses the pixellated look I liked.

Fun In The Land Of UI

I spent the last 2 weeks working on a 2 week sprint to try and get the beginnings of a UI editor in place.

Week 1


The first step was cleaning up all the UI Elements I'd accrued over the last year or so. The big problem was that as my codebase evolved I didn't want to spend the time to go back and clean up old systems. On one hand this worked to my advantage because I was able to iterate much faster. On the other hand it left me with a lot of cruft in my codebase.

The first thing I wanted to clean up was the UIClickable. Maybe I should go into a bit of background though before I go into the details. Almost every UIComponent is made up of 4 core sprockets,  TransformSprocket, UIDimensions, UIClickable, and UIRenderable.

TransformSprocket: The transform sprocket is a common sprocket that just wraps a matrix, I use it in any component that needs a transform of some kind.

UIDimensions: This guy contains a width and height and is used to calculate the bounding box of a UI element.

UIClickable: Things get a little messy here, bad naming choice. This guy basically takes care of any input functions a UI component might need, these are currently mouse events and keyboard events.

UIRenderable: I haven't been a fan of this guy from the start, but basically any UI element that can be displayed has one of these and gets linked up through the IO system with a render callback which handles any necessary rendering.


Ok so where was I, ah yes the UIClickable. So previously I had an old message passing system which packed any necessary data into a message class so any function which took a message would have to pull out the data it needed from the Message object. The problem with this of course is that the person implementing the function doesn't actually have any idea what's actually in the message since it's really just a blob of data. Anyways over the course of the project this evolved into my Energon IO system which takes strongly typed messages so you know exactly what you're getting. The problem though was that I still had a lot of the old message handling going on and UIClickable had paths for both. So the first step was ripping out all the old message handling and clean up any of the UI elements that were still using that code path.

While I was cleaning up the code paths another problem I had with the UIClickable was that all of my coordinates for an object were based on their parents coordinate system, so a button at 0,0 that lived in a window, the 0,0 would refer to the center of the window. Now to accomplish this I had put all of the code in the mouse event callbacks in every UI element so that meant a lot of duplicated code, and in some cases the math was slightly different then other elements.

To clean this up I moved all of the conversion to local space calculations into the UIClickable. Since every sprocket can access it's parent and from there could access the TransformSprocket and UIDimensions it wasn't much work to refactor all this code. The downside to doing this though was that it meant not only did I have to rip out all the old codepaths in the UI elements, I also would have to refactor the current code paths to take into account that this was now being calculated inside of the UIClickable.

Another thing I put in that I didn't have before was mouse over events. I think this gives a really nice touch to the UI. Makes everything feel more alive when there's some kind of texture change or effect on mouse over. This led me to rework my buttons a little more. As it stood currently the buttons had support for textures on different states ie, mouse over, mouse down, idle. However I hadn't actually hooked those textures up to the UISprite which is a part of the button. So I set about adding a simple state machine for the button and setting up the sprite to take a different texture depending on the state that the button was in. I set up some safe guards so if a texture for a state isn't set in the create params it will just use the idle texture and of course if there isn't an idle texture set it will assert.

At this point I took a little break from the UI and had a look at revamping some of the code I had in place for hot swapping assets. What I had in place previously was a system which allowed you to specify a directory and a callback and if any files in that directory changed the callback would be called with the name of the file allowing you to perform any needed code to reload the asset.

This didn't really work very well for assets that were nested within multiple directories or if multiple asset types lived in one folder. What I decided to do instead was change my FileWatcher task to register with the AssetInventory callbacks based on asset types. So in this way there was a callback for Textures, Shaders, VoxelSprites, etc etc. I set up callbacks for all the main asset types I'm currently loading and plan on expanding it down the road with user specified type callbacks. So now it queries the AssetInventory every so often for the loaded assets. It then calculates an initial time stamp for any new assets. If the asset is already registered with the watcher it checks to see if the timestamp has changed. If the timestamp has changed then the asset is reloaded and swapped out of the AssetInventory. This way it's seamless to all the other systems, since all assets in the engine are passed around as handles which point back to the AssetInventory.

At least... all assets should be handles. The reality of it was that I had written a lot of my code that works with textures before the AssetInventory was in place. So this meant refactoring a lot of my material system to work with handles to textures instead of pointers. While there's a bit of a hit dereferencing the handles I think it's worth it in the long run. It'll help a lot when I get into streaming textures, so that I'll be able to seamlessly load in different mip levels by just swapping out textures in the AssetInventory.

Finally I started fleshing out the UIEditor. I set up a new game state that's selectable from the main screen. I also added in some of the components I had in the level editor, a protoform type list box for selecting what type of sprocket I'd like to create, and a protoform property window, which allows editing of the create params for a sprocket type. I also started to ponder how I would handle hooking up these UI screen's I create to actual gameplay logic. I settled on a solution where I would create an interface sprocket which would contain a number of inputs and outputs that I could link up in my IO Editor component to handle most of the logic that would be necessary for the screen. I also pondered setting up a scripting language to use with the front end. I'd like to get a scripting language in eventually for gameplay so I thought it might be as good a time as any. I put that on the back burner for now though because that comes with it's own set of problems that I'm sure will keep me busy for a few weeks.

Week 2


While I had added the protoform property window in the previous week, with all my changes to the UI nothing worked on it. It didn't even render! So I spent most of the day getting all the UI sprockets it uses working again. It was mostly easy work, just a lot of ripping out code and waiting for builds.

The next thing I wanted to get going was a gizmo for quick editing the UI sprocket. My plan was basically a border around the object with 3 buttons in 3 of the corners. A button for translation, rotation and scale. Clicking and dragging on the appropriate button would perform that transformation. I was able to get a button spawning but it's a bit of a hack for right now and currently the gizmo isn't hooked up to anything.

I also began pondering what my next sprint would be. I think I'd like to spend one more week fleshing out the basics of the UI Editor and then start working on an Animation Editor. It's kind of a big jump but I'd really like to get going on gameplay mechanics again and one of the things that was holding me back was getting in some placeholder animations. I think an animation editor would help get me going again.

Unfortunately this week was mostly all about the builds. I was changing a lot of core headers which caused 15-20 minute builds on this little netbook, that's a good chunk of my commute! I ran into a lot of problems with my protoforms which are essentially an xml tree of variables, inputs, and output definitions with state stored with them. I use protoforms as a way of serializing in/out of objects as well as editing the create params for a sprocket before I actually spawn it. All of this is done via a few handy macro's that do some code generation to create some functions that the protoforms query to get reflection data on all of the create params, inputs, and outputs an object can have.

I also started cleaning up another element of my UI Sprockets. Every UI Sprocket took in an FEContext as a create param. The FEContext had pointers to all the necessary systems that the sprocket would need to create itself things like memoryAllocators, renderContext, fontSystem, assetInventory. The thing is though that I added a ConstructionContext that gets automatically stored with every create param and this already has all the necessary systems in it. I added that a while after I started building my first set of UI Sprockets so none of them were really taking advantage of the ConstructionContext.

I spent a lot of time at the end of the week thinking about refactoring how the protoforms currently work. One thing that really bugs me about them is that they kind of encompass both the reflection data and the state data. I'd like to separate them so each class only has one set of reflection data and the actual state for each of those peices of data is stored separately in the protoform as a tree of state data. For example I have a VariableInfo, it stores things like the name, the type, the data offset from the beginning of the create params, the size, etc... however it also store the actual state of the variable inside of an object that manages converting to and from a string representation of the variable. I'd like to make these two concepts very separate. It'd be a big overhaul of the system as it stands now though so I think for now I'll just leave it how it is.

So that brings us up to date. I'm currently working on a one week sprint that cleans up some of the leftover things like addressing the gizmo a bit more in depth, saving/loading screens and properly handling multiple UI objects. I also need to address properly parenting a child UI Sprocket to a parent UI Sprocket, but that's all for another week.

Before Mouse Over

 After Mouse Over

First shot of my gizmo for editing, I know it looks awful it's just placeholder

 Showing a spawned button

 Illustrating a pressed button on the gizmo, when dragging that button the width and height of the UI Element gets changed.

 A bunch of spawned buttons of different sizes.