Debugging Tamarin

I’m getting my feet wet in Tamarin, the open-source ActionScript 3 runtime from Adobe – same code that’s in Flash 9. It’s cool tech, and I’m ecstatic that they had the cajones to bring it out into the world – where it’ll definitely make the world a better place.

This gets kind of technical so I’ve hidden most of it behind the jump.

C++ And AS3 Compilation

So where am I? First step was to build the runtime, which was easy – compiles right out of the repo. Then I realized I couldn’t compile. That stalled me for a few months. Once we started working on the current project at work – which is in Flash 9 – it was a lot easier to figure out the compiler part. Flex 2 SDK for the win! It’s free, you see.

The relevant command to compile a .as file is:

java -jar SDK_PATH\lib\asc.jar myTest.as

Then you can run it with the binary you compiled earlier – just do:

avmplus_d.exe myTest.abc

(Paths are a little trickier than this in reality due to where the compiler sticks things, and if you did a release build the binary name might be different.)

Native Binding

The next thing I attempted was adding my own class. I’ll be cleaning up the code for this and posting it a little later – right now it’s sitting in AvmCore.cpp which is definitely not a good way to do it! But I will talk about the native-function-binding process, which is pretty slick.

Basically, there are two parts. First, some #defines that go in your C++ code and define some tables. They look like this:

BEGIN_NATIVE_MAP(TestClass)
   NATIVE_METHOD(TestClass_helloWorld, TestObject::helloWorld)
END_NATIVE_MAP()

BEGIN_NATIVE_CLASSES(AvmCore)
   NATIVE_CLASS(abcclass_TestClass, TestClass, TestObject)
END_NATIVE_CLASSES()

This matches up with some ActionScript code that looks like this:

package
{
   public final class TestClass extends Object
   {
      public native function helloWorld():void
   }
}

Now the weird thing here is that the C++ side references these magic files (see core/builtin.h and core/builtin.cpp for examples). builtin.h contains lines that look like this:

const int TestClass_helloWorld = 557;
const int abcclass_TestClass = 17;
extern const unsigned char builtin_abc_data[];

builtin.cpp is just a big array of bytes defining builtin_abc_data[] – AS3 bytecode, as it happens. Not too surprisingly, this code is one of the first thing the VM executes when it starts up. Where does it come from? What’s in it?

native keyword and ASC

There’s a file in core called builtin.as that references other .as files in the same directory, and in that is defined a bunch of stuff related to the core built-in classes like Function and Object and Boolean. Some of the methods are native and body-less – and sure enough there’s corresponding C++ code in the AVM – and others are populated with script code. (I assume that you have to have matching AS3 and C++ side functions or face all kinds of nasty bugs, but haven’t tested this yet.)

However – there’s no obvious way that the script code is mapped to C++ code except for these IDs defined in the builtin header file. Where do they come from? What tool made these files? They have all the marks of being machine generated – they’d have to be since there’s no way to manually specify a “native ID” for a method.

After some digging, I discovered the key to all this. It’s the AS3 compiler! If you compile a .as file with a native function in it, it automatically generates the C++ code to embed it in the AVM. That’s where all those weird constants come from.

Exception 0x80000001

So – I made a test class, regenerated the builtin scripts, and bound my own native functions. Neat! So I ran my test script to see if it would trigger the debugger (I set a breakpoint in my class implementation). And it did – but not in my code. Specifically, I got this weird exception:

First-chance exception at 0x00459272 in avmplus_d.exe: 0x80000001: Not Implemented.

Well, what the heck is that all about? It turns out that the JIT code generator, which produces machine code versions of your AS3 code, reserves a bunch of memory from the OS to build executable code in. On windows in particular, as part of the memory setup, it does it with a call to VirtualAlloc that looks like this:

void *res = VirtualAlloc(address,
o GCHeap::kNativePageSize,
o MEM_COMMIT,
o PAGE_GUARD | PAGE_READWRITE);

(This is in GCHeapWin.cpp line 317 or so.)

It turns out that PAGE_GUARD makes the first access to the specified page trigger an error. MSDN has more to say on the subject of guard pages. I haven’t fully traced this code yet but it appears that it’s part of a scheme to dynamically grow a buffer as the MSDN article implies. In any case, it’s not actually an error – just a side-effect of normal operation. This can be confirmed by running the AVM outside of a debugger – it functions correctly – and by continuing execution after the breakpoint is hit.

The details of how to suppress this error are different for every IDE, but in VC7, you go to Debug->Exceptions…, select Win32 Exceptions, then hit Add… – for number input 0x80000001, and description can be whatever you like. I’d recommend “page guard” or something like that. Then make sure that a breakpoint isn’t triggered by selecting it in the list and choosing “continue” from the options at the bottom.

Conclusion

Things we know how to do:

• Compile AS3 to ABC and update the builtins in the AVM.
• Compile the AVM from C++ source.
• Bind our own native C++ code to the AVM.
• Debug the AVM w/o extra exceptions causing problems under VC7. (The same steps should apply to VC8 as well.)

AVM is pretty cool. I look forward to working with it more. 🙂