Adobe, Please Buy HaXe.

haxe_and_neko If you’re in the Flash space, you’ve probably seen a lot of coverage of HaXe lately. HaXe targets Flash 9 and 10, as well as JavaScript, PHP, NekoVM, and most recently C++. You can write code once and easily retarget it to different platforms, which is very exciting. This has been used to good effect by Hugh Sanderson, who is developing an iPhone path.

HaXe is cool not only for platform compatibility with things like the iPhone (and who knows – maybe other platforms like Android or Windows Mobile), but also for its language features. They deployed support for the new Flash 10 high-performance memory intrinsics before the official Adobe compiler. HaXe has generics/templates, decent conditional compilation, inlining, advanced typing a lot more. The biggest claim to fame HaXe has is a big performance margin over AS3, as well as better memory management options.

And performance is a big deal when you’re an interpreted language trying to run on Netbooks and televisions.

Since my day (and sometimes nights, too) job is working on a Flash game framework, HaXe looms large on my radar. AS3 and HaXe can already interoperate so if you want to use a HaXe physics library from your ActionScript 3 code, there’s no problem. But should we switch the core engine over? Having a seamless way to target iPhone would be big. Having an easy way to run your server code without requiring an integration with Tamarin would also be nice.

Ralph Hauwert over at unitzeroone summarizes the HaXe situation very well:

The language is great and has many features ActionScript lacks. It baffles me how one man army nicolas can single handedly write a language and a multi target compiler, and in many respects stay on top of the ActionScript compiler.

My first problem with Haxe is just that. The one man army. Should nicolas decide he doesn’t feel like it anymore, the platform is dead. Sure, there’s more people working on it, but in it’s core it seems to be that nicolas is the driving force. My second problem is that although there is a huge pool of ActionScript developers, which can work together within the industry standard called ActionScript. There’s not even near that many people who’ve even touched haxe, let alone the number of developers who are able to call themselves “Haxe Professionals”.

HaXe is very cool. But it doesn’t have the ecosystem and infrastructure to justify targeting it exclusively. As a small startup, we only get to target one platform for our technology, and AS3, although it leaves performance on the table, wins in terms of the pool of potential developers and the accessibility of the toolchain. How many people out there are using PHP or Java vs. x86 assembly?

If the basic technology is good enough, ease of use always wins out over performance. So for now we are going to stick with ActionScript 3 for PushButton Engine. You can write compelling content on it. If you really need HaXe for something, the option is there to link it in.

But I want to put the plea out to Adobe. I love you guys and all you have done in the Flash space. But please update your compiler technology. At least give us the compiler optimization wins like inlining and generics. It’s simply embarassing that one guy’s work can outperform you on your own VM!

Adobe, you have smart people working for you. You have smart people in your community. You have pretty decent tools. None of us want to jump over to a third-party technology, but if HaXe keeps beating you at your own game, it might be necessary. Buy HaXe if you must, or else match it with your in-house tech, but either way – I know you can solve this problem.

The web is catching up with you on the player front. You could dominate on the tools front and make the world better for everyone. Please. As a middleware developer, I really don’t want to have to support anything other than a single mainstream language and platform. 🙂

Technical Notes on O3D

Google released O3D, a web plugin for 3d rendering, today. It’s a pretty sweet piece of work. Definitely check it out if you have any interest in 3d rendering, the web, or JavaScript.

There are a couple of cool pieces to this puzzle, and I wanted to call them out to other people who might be reviewing this technology. The two sentence review: I am blown away. They got this right.

(FYI: This post isn’t meant to be a full walkthrough, just a quick indication of the cool jumping off points in the codebase.)

Stop number one is the graphics API abstraction. This is rooted in o3d/core/cross/renderer.h, which provides an abstract interface for performing render operations. This is sensibly designed, oriented for SM2.0 through SM4.0 level hardware. It will also deal with DX11 class hardware, although it won’t expose all of the niceties DX11 gets you. It wraps GL (o3d/core/cross/gl/) and D3D9 (o3d/core/win/d3d9). The purpose of all this is to provide a common ground for all the higher level code to issue draw commands from. It is not directly accessible from JavaScript (although some of the related classes like Sampler and State are.)

As an aside, writing a Pixomatic or Larrabee backend would not be hard. At GarageGames, we had a rendering API similar to this (unimaginatively called GFX), and it was fantastically useful. Last time I checked, there were DX8, DX9, DX10/11, GL, and Pixomatic backends for it in varying states of usefulness. We even had one guy write a backend that would stream draw commands over a socket, which was pretty cool. In the context of O3D, they have an implementation of Renderer that queues everything into command_buffer, then streams it to a command buffer server running in a separate thread, which issues the actual draw commands. It’s unclear whether this is used in the current version of the plugin based on casual inspection, but it’s a great example of what good design can get you.

The next piece is the DrawList, which is where most of the heavy lifting for drawing happens. JavaScript, of course, is not really desirable to have in your inner rendering loops, so you queue everything you want to draw (DrawElements) into a DrawList. This is wrapped by higher levels, of course, but it represents the lowest level API that’s available to JS code. All the sorting and state management to get stuff on screen happens in this area.

Alongside the DrawList stuff, you find the material system, which is pretty slick. You write your shaders in their shading language, and it converts to HLSL SM2.0 and Cg (arbvp1/arbfp1). This is enough to do almost anything you might want to (as fantastic as SM3.0+ is it’s not really necessary for most rendering). There’s a full SAS system so you can interface programmatically with your shaders.

Above this, you get into the scenegraph. Now, I subscribe to Tom Foryth’s views on scenegraphs, which is that they are basically a bad idea, but I think the Google guys were smart and set up their API so intelligent developers can avoid being screwed by the scenegraph. The scenegraph-esque stuff they do have (they call it a rendergraph) is powerful enough you can do most rendering without going nuts, and lets a lot of the heavy lifting stay in native code, where it will be fast. You end up doing retained mode-ish things, but since JS is slow, and most JS developers come from a DOM background, it works better than you might expect.

There are a lot of cool miscellaneous features, too. They embed the fast V8 JS engine right in the plugin so you can have consistently fast JS execution. They support falling back to an error texture (via Client.SetErrorTexture) when you fail to bind a sampler. You can group objects via Pack objects for easier management and stream things from archives, too. There are debugging aids and libraries for math and quaternions.

You should check out the samples. There’s a lot of impressive stuff. There’s the beach demo in the video at the top of this post, but they also do some cool stuff with animation, HUDs, text rendering, picking, and stenciled teapots. And everything is done in JavaScript right in the web page. Even the shaders are embedded in <script> tags!

O3D is a great piece of technology, and I hope it thrives. I’m excited to see what people build on this (and I’ve got a few ideas myself ;).

Tweaking your game with Google Spreadsheets

tweaksheet Our latest game, Grunts: Skirmish, has 200 tweakable parameters. There are 9 player units with three levels of upgrade, and another 9 enemy units. Each unit has between three and ten parameters that can be altered.

We tried a few approaches – hand-editing a large XML file (but it was too large and spread out) and an in-game tweaking UI (but it was too much work to get the UI to be friendly to use). The old standby of having the designer and artist file bug reports to have the programmer update the game wasn’t getting us very far, either.

“Well,” says I, “We’re some sort of Web 2.0 startup, right? And we’re developing a Flash game aren’t we? And Flash can talk to websites, can’t it? And don’t we use Google Docs for everything?”

It turns out there is an XML feed from public Google spreadsheets. And ActionScript 3 supports E4X, so you can directly manipulate XML without any extra work. Now we tweak our game using a shared spreadsheet up on Google Docs.

I wrote a parser for their format:

// Extract the entries. It's namespaced, so deal with that.
var xmlns:Namespace = new Namespace("xmlns", "http://www.w3.org/2005/Atom");
tweakXML.addNamespace(xmlns);

// Parse into a dictionary.
var cellDictionary:Dictionary = new Dictionary();
for each(var entryXML:XML in tweakXML.xmlns::entry)
{
   cellDictionary[entryXML.xmlns::title.toString()] = entryXML.xmlns::content.toString();
}

And wrote a quick component that would fetch the spreadsheet feed, parse it, and stuff it into the right places on named objects or template data. Now I have a little entry in our level file that looks like:

<object id="googleTweaker">
   <component class="com.pblabs.debug.GoogleSpreadsheetTweaker">
     <SpreadsheetUrl>http://spreadsheets.google.com/feeds/cells/0d8somekey848x/od6/public/basic</SpreadsheetUrl>
     <Config>
        <!--  Grunt Level 1 tweaks -->
        <_><Cell>B3</Cell><Property>#TheGruntProxy.creator.WarPointCost</Property></_>
        <_><Cell>C3</Cell><Property>!Grunt.health.mMaxHealth</Property></_>
        <_><Cell>D3</Cell><Property>!Grunt.ai.AttackSearchRadius</Property></_>
     </Config>
  </component>
</object>

Each line maps a cell in the spreadsheet to a property on a template or active game object. Some properties have to be set several places, which the system deals with automatically.

The biggest wrinkle was Google’s crossdomain.xml policy. Basically they do not allow random Flash apps to access their site. So I had to write a small proxy script, which sits on our development server next to the game and fetches the data for it. Figuring out I had to do this took more time than any other step.

The main difference between the snippet and the full code is the version in our repository is 220 lines long. I only have around 150 of the full set of 200 parameters hooked up, but after a hard afternoon’s work, the process for tweaking the game has become:

  1. Open Google Docs.
  2. Edit a clearly labeled value – like level 1 grunt health.
  3. Restart the game, which is running in another tab in your browser.

This takes you about a minute between trials. Not too bad. Before this, the process was:

  1. Get the game source from SVN.
  2. Find the right XML file – there are several.
  3. Find the right section in the XML – altogether we have 200kb of the stuff for Grunts!
  4. Change the value.
  5. Commit the change.
  6. Wait 5-15 minutes for the build system to refresh the live version of the game.

Ten minutes per tweak is not a good way to develop.

I’ve heard about developers using Excel spreadsheets for tweaking, but can’t find anything about using Google Docs to do it. But Google Spreadsheet is obviously a better choice. It has built-in revision tracking. You can edit it simultaneously with someone else. You can access live data in XML either publicly (like we did) or privately via their authentication API. It’s absolutely worth the half-day of your time it will take to add Google Spreadsheet-based tweaking to your game – even if it’s a non-Flash game, downloading and parsing XML is pretty easy with the right libraries.

I strongly suspect this feature will find its way into the next beta of the PushButton Engine. Which, by the way, you should sign up for if you are interested in developing Flash games. We’re bringing people in from the signup form starting this week. If you want more information, or just like looking at cool websites, click below to check out the new version of the PBEngine site, which has a bunch of information on the tech. Tim did an awesome job on the site design.

betashot

Edit: Patrick over on the GG forums asked about the proxy script. It’s actually ludicrously simple. Not very secure either so I wouldn’t recommend deploying it on a public server. I got my script from a post on Aden Forshaw’s blog. In the real world you would want to have some security token to limit access to your proxy script… but since this is for tweaking a game that is in development I didn’t sweat it.

Array Ordering in ActionScript 3

Update: I filed a bug about this issue.

coils This post starts out with a finite state machine, but it ends up talking about how iteration is implementing in ActionScript.

There I was – implementing a finite state machine. Each state had transitions, and the transitions are named and, additionally, processed in order. This is useful because you can order your transitions for consideration from most to least specific. Usually the last transition is unconditional.

Unfortunately, I began encountering issues with users who had the release player installed. Enemies would move erratically and transition between states inappropriately. Adding and removing log statements would sometimes make the problem go away, other times not. I suspected the VM or compiler.

As you may be aware the for and for each keywords in Flash are not guaranteed to iterate over keys or values in a consistent order. And of course, my loop for checking transitions looks like:


      	// Evaluate transitions in order until one goes.
      	for each(var t:ITransition in Transitions)
      	{
      		if(t.Evaluate(fsm) && fsm.SetCurrentState(t.GetTargetState()))
      			return;
      	}

The icing on the cake is the hashtable implementation in Tamarin, which is used internally for storing object properties as well as (partially) for Arrays. Whenever you store a value by key, the key is converted to an atom, and the atom is used as the key for the hash table look up. Atoms are (basically) pointers, so the location in the hash table is based on the memory allocation pattern of your SWF (as well as the allocations of the player binary you are using). (See HashTable::find for the implementation behind this.)

In other words, the order of iteration will be random, but consistent for a given combination of player binary and SWF (and execution pattern). In my case, the iteration order was what you’d expect (order of addition to the Array) for debug player, and backwards for release player. A case of good luck turning bad, since it led me to believe that everything was working correctly in the debug player.

How does this relate to the log statements I mentioned earlier? Because iteration order is tied to memory allocations, it looked like a VM or compiler bug, since adding or removing code would change memory allocation patterns.

From this experience, I have two pieces of knowledge to share.

First, if you need to have both non-integer keys and values, but want to iterate in a specific order, you can’t use any of the built-in ActionScript classes. Instead, use a class like senocular.com’s AssociativeArray, which subclasses flash.utils.Proxy to implement custom iteration logic.

Second, some parts of ActionScript do iterate in a consistent order:

  • “Dense” Arrays will iterate in order from first to last element. You can see the implementation for this in ArrayObject. “Dense” mode is heuristic but activated by having numeric indexes, especially starting at zero.
  • XML nodes. Children have a definite order based on the document and are iterated in this order.

The docs don’t describe this, to the best of my knowledge, which is why I post here. The best I can find is this from the online docs:

For example, you can use a for..in loop to iterate through the properties of a generic object (object properties are not kept in any particular order, so properties may appear in a seemingly random order)

Is there any more information out there? I’d like to think I’m overlooking something!

Thanks to Steven Johnson of Adobe – his feedback was essential in the process of sorting this out. Creative Commons picture by davebluedevil.

ActionScript 3 Reflection: list fields on an object with ClassFieldCache

Say you’re working on some sort of reflection/deserialization system in Flex, Flash, or another ActionScript 3 technology. It would be nice to know the type of all the fields on an object or class, including accessors, so that you can automagically infer the right way to parse your serialization data. You can use describeType to do this but it has some overhead – dumping a full XML description of a complex type can’t be something you’d want to do frequently, and then you have to parse it again.

I present the ClassFieldCache. It contains one static method, getFieldsOfClass, which will return a dictionary of types indexed by field name. It returns every kind of property you can set – both fields and accessors. (And it ignores constants and readonly accessors.)

Usage is like this:

	var dict:Dictionary = ClassFieldCache.getFieldsOfClass(myObject);
	trace("Field boo is of type " + dict["boo"]);

Which outputs “Field boo is of type flash.geom::Point” – or whatever that field happens to be.

And the class itself is as follows:

	import flash.utils.*;

	/**
	 * Utility class to get list of fields on an object or class.
	 */
	public class ClassFieldCache
	{
		/// Indexed by Class, this contains dictionaries mapping name to type (string).
		private static var smFieldInfoCache:Dictionary = new Dictionary(true);

		/**
		 * Return a dictionary describing every settable field on this object or class.
		 *
		 * Fields are indexed by name, and the type is contained as a string.
		 */
		public static function getFieldsOfClass(c:*):Dictionary
		{
			if(!(c is Class))
			{
				// Convert to its class.
				c = getDefinitionByName(getQualifiedClassName(c));
			}

			// Is it cached? If so, return that.
			if(smFieldInfoCache.hasOwnProperty(c))
				return smFieldInfoCache[c];

			// Otherwise describe the type...
			var typeXml:XML = describeType(c);

			// Set up the dictionary
			var typeDict:Dictionary = new Dictionary();

			// Walk all the variables...
			for each (var variable:XML in typeXml.factory.variable)
				typeDict[variable.@name.toString()] = variable.@type.toString();

			// And all the accessors...
			for each (var accessor:XML in typeXml.factory.accessor)
			{
				// Ignore ones we can't write to.
				if(accessor.@access == "readonly")
					continue;

				typeDict[accessor.@name.toString()] = accessor.@type.toString();
			}

			// Don't forget to stuff it in the cache. :)
			smFieldInfoCache[c] = typeDict;
			return typeDict;
		}
	}

Hope this is useful for you. You’re free to use it however you like. Attribution would be appreciated.

Forest Pack – Finally out the door

Trees With Shadows

It took a few years, but I finally released the fixed function GL forest renderer I was working on for Torque. You can find out all about it at my .plan on GarageGames. Various businessy things got in the way of releasing it (and perhaps a bit of apathy on my own part :) … but it’s nice to have it out! Done with? We’ll see. There’s always a lot of question/answer and trouble-shooting to go with these things.

On a personal level, I like to think this heralds a new chapter of my life, where things Just Get Done a lot more often. We’ll see how that pans out.

SpaceNavigator Support for Sauerbraten

SpaceNavigator I recently got a 3dconnexion SpaceNavigator after trying a friend’s. Really neat device. 6 degrees of freedom in a slick little package, and only $50. And it’s actually fun to use – not frustrating and cheap-feeling like many other “weird” input devices I’ve tried. Its big brother is the same knob with some extra buttons – for only $400. Yikes. Yet I’m beginning to see why you might want to upgrade…

Anyway, I did a lame integration with Sauerbraten, an open-source 3d engine, under Win32 against the 3dconnexion SDK. The integration is pretty simple – even crude. This is another code post, skip if you’re not a techy! :)


First, grab this file (spaceNav.cpp) and add it to the Sauerbraten project. Then add a COM reference to the TDxInput that ships with the 3d connexion SDK (which you can download here). Finally, go to engine/main.cpp and add:

extern void spaceNav_start();
extern void spaceNav_poll();

void updateCameraFromSpaceNav(float tX, float tY, float tZ, float rX, float rY, float rZ, float rAngle)
{
   // Construct a matrix from our cur yaw/pitch/roll
   matrix m(vec(1,0,0), vec(0,1,0), vec(0,0,1));

   m.rotate(-camera1->yaw * RAD,   vec(0,0,1));

   m.X.mul(tX / 10.f);
   m.Y.mul(-tY / 10.f);
   m.Z.mul(tZ / 10.f);

   camera1->vel.add(m.X);
   camera1->vel.add(m.Y);
   camera1->vel.add(m.Z);
}

Right before the main function (around line 434).

Right after the line that reads:

    SDL_ShowCursor(0);

Add:

    spaceNav_start();

And around line 630, inside the for(;;), add:

               spaceNav_poll();

You can bind to the SpaceNavigator buttons by setting up keymaps like…

keymap -131 SPACENAV_LEFT

…and similar to a cfg file that gets executed when Sauerbraten starts up – I used keymap.cfg. Then you can bind to SPACENAV_LEFT and execute whatever commands you like. The binding uses button ids -100 through -132. 31 and 32 are the two buttons on the SpaceNavigator. On other 3dconnexion products there are more buttons and thus you can make use of the other binds.

And that’s it. spaceNav_start() loads up the SpaceNavigator COM object, Connect()s to the driver, and sets some global. spaceNav_poll() queries state and calls updateCameraFromSpaceNav() with the current state. It also dispatches to keypress() when a button is hit so the console can deal with binds.

Known issues:

  • Not networked properly. I’m just stuffing into the camera1 entity’s velocity field. It’s good enough to edit with, which is what I want.
  • Not based on frame time, so will give different behavior based on framerate.
  • No rotation. Since you have to aim with the mouse anyway this didn’t seem like a big need, but if someone made clicking independent of heading it could be a really useful feature… It does take the yaw into account when translating so you can fly around pretty well.
  • Not configurable. You can tweak the code in updateCameraFromSpaceNav to change how/how fast you move but it would be better to have that all be console-controllable.

Conclusion

I hope this is useful to someone! Feel free to do what you like with this code. Sauerbraten is under zlib, and 3d connexion SDK probably has some restrictions, too.