This is why I always run a raster pipeline. The DisplayObject API is a trick played on the gullible and the trusting… — @bengarney
Everyone has their hot buttons. I work on libraries and architecture for a living, meaning that I am personally invested in those kinds of issues, meaning that I get excited about things like platform feature sets in ways that maybe most developers don’t.
Here are my assumptions: most Flash developers want to put art on the screen and move it around at a mostly decent framerate. Most of these developers are building interactive experiences and don’t have heavy coding backgrounds (although this is changing). Their priorities are influenced by this – they want ease of use, fast iterations, and don’t necessarily care about high performance (although this is changing, too).
Meanwhile, I inhabit an alternative universe. The last game I contributed to, Social City, had high thousands of frames of animation and hundreds of animated objects on screen at any given moment. In the whole scene there might be low thousands of visible objects. Because the world was isometric all these objects had to be drawn in the right order and most of them updated every frame.
In our initial technology tests, just having that many objects in the display list would make mouse interactions take up most of the CPU. Rendering was also extremely slow. Between these two issues we were entering the seconds-per-frame zone – and we hadn’t even added the gameplay or complex animation yet!
I am a big believer in finding the fast paths and sticking to them. Computers are fast. Really fast. My current low-end MacBook Pro, on a single core, has something like 1200 instructions per pixel to burn if I am trying to render 640×480 @ 24Hz. In the Social City case, all we wanted to do was copy opaque pixel data – which should be something like 5-10 ops per pixel, probably much less. So really, even taking into account that buildings often overlap, I should be able to push around a hundred frames a second if all I do is render.
All this is primarily to point out how awful a one second frame is. It means I am running a thousand times slower than I should be.
You might think that the most important thing in a game is how fast things can run. After all we always want to push the limit. So maximum speed is vital, right? That’s not necessarily true – we’re only looking at part of the picture. The human eye responds to rate of motion. It is really good at spotting speedups or slowdowns on the order of tens of milliseconds. It’s actually better to run at a lower, consistent framerate than it is to have a higher average framerate with occasional dips, because dips draw attention to performance or lack thereof. That’s why movies run at 24Hz, even though the TV in my living room can update at 120Hz, and why console games often target a fixed 30Hz or 60Hz.
A big part of building a rendering system for a game is identifying how to deliver a consistently good experience for the player, even when they have a fully built out complex level with tons of bad guys, interactive objects, and/or special effects.
How does all this relate to the Flash display list? Well, it is very powerful. But it is also unpredictable. It’s like the dark side of the Force. It’s a quick path to great power. Only later, do you realize that you’re wearing restricting black clothing, hanging out at weird parties with old men, and experiencing hard-to-diagnose performance issues.
Or more technically: the enemy of a system with consistent performance is black box code. Of course, I use the display list for lots of things, just like any Flash developer. It’s a useful, flexible, and powerful tool.
But when I have specific, performance oriented needs, I ditch it in favor of something I can control completely. That’s why I always use a raster pipeline – drawing all my own pixels – for my game rendering. Using BitmapData.copyPixels has a consistent, reliable cost, whereas the cost of drawing a DisplayObject hierarchy, either on stage or via draw(), is highly variable and difficult to predict in terms of both memory and CPU consumption. This is also why I avoid using Flash IDE created assets; it’s too hard to rely on the performance characteristics of art produced from it. If I have to, I always render everything to a BitmapData first and draw it later on my own terms.
(As an aside, Mobile Flash platforms lack CPU, so taking advantage of the GPU is important. In this case, I would still use bitmaps but let the GPU manage compositing when I can, for instance by using flash.display.Bitmap. I would also carefully measure the cost of off-screen Bitmaps to see if I need to manually add/remove stuff as it becomes visible. Some early tests also suggest that you can get pretty far on BitmapData on mobile if you are careful, too.)
I could write more on raster based rendering, but if you are interested in nitty gritty, you might enjoy looking over the Rendering2D library from PBE; it’s the basis of the rendering in Social City and other titles that I have worked on, and demonstrates a framework for efficient hybrid raster/vector rendering.
Ben Garney 
isnt this somehow the same as the new chapter in “cacheAsBitmap” -adobe is introducing just around the corner. where you now can choose to let flash in compile time render bitmaps from displayObjects and save them in local lib – and then access these images in run time fully automaticly as an improved and more static version of cacheAsBitmap…
PS I look forward to seeing your blog post!
I wonder if you were trying combining vector UI with raster rendering at background. We run at some problems with combining those two together which took some time and effort to figure out and optimize, worse that in two different games different same optimizations did not work or even made things worse 🙂 Speaking of DisplayList problems haha…And yeah biggest problems is that you need to do a lot yourself, things that were as easy as changing few properties or applying some tween now is very hard to achieve… Sometimes feels as if after running on ground you try running in water 😀
Hmm, I am not sure 90% is true. I do agree that in the end it's about what will get your project done to spec fastest and the display list can be a powerful ally in that battle.My quick rundown on your points:I don't use Flash IDE and none of our artists do, there is no workflow benefit. In fact having our own art pipeline and processing steps is a big benefit for download size and performance.Nesting is a waste of time IMHO. It just complicates rendering, updating, and hit detection.You can transform bitmaps just fine. (Although if you need to do high quality free rotate/zoom then vector is the way to go!)The timeline is a way for artists to break your game by introducing frame scripts, weird frame behaviors, high-overhead morphs/tweens, and so on. In the cases where I do use SWFs for animation/download reasons, I render them out to bitmap spritesheets first (either at runtime or before hand) so I have a fixed overhead render.Filters are best rendered to bitmaps.Filesize is always an issue; if you have vector friendly art it will be small, but all the Flash projects I have worked on have not – using vector assets would have increased file size and decreased rendering speed.Mouse events are a crutch, and worse, they are very difficult to reason about and debug. Why can't I put a breakpoint on a global mouse input hander then step through all the event dispatch? Instead I have to play games to get listeners in the right places and there's no convenient way to see the flow of activity.Dev time is a big one. Display list definitely solves a ton of problems right out of the gate. If you know the system you can be very effective. If you are on a 4 week project, then you need every reusable piece you can get. But more and more Flash games are going to be on bigger budgets, bigger timelines, higher art and technology standards. Will the 2 or 4 week game go away? Probably not. But I guarantee that Flash game budgets are going up and so are timelines. Maybe Social City is a front runner (but it only had 4 months, which is insanely short by most game dev standards), but it is not going to stand out front for long.Obviously I come from the background I come from (c++ game dev) and work on the projects I work on. I can only speak to what I know, and I don't pretend to have everything figured out. But for my little part of the development world, I feel pretty firmly. 🙂
The Social City footprint is primarily art. There are a lot of frames of animation and they take up space. :)People have gigs of RAM, when you are doing a game (not an ad) why not use it? Even my phone has enough RAM to run Social City. (Maybe not enough CPU, bandwidth, and screenspace, though. 🙂 )I agree that for some situations the display list is great. It is definitely preferable for building a UI. For mousing in a game, you often want custom behavior anyway so the mouse behavior built in tends to be a liability instead of a benefit.Probably the biggest trade off is development complexity. You might also get lucky and be doing something that maps well to the display list, in which case, go nuts!
Working in same space making similar games. For our main game rendering we use BitmapData.copyPixels too and for same reasons, for consistent and predictable behavior completely based on transparent code, not the black box which sometimes throws out surprises which take time to test and understand.But it has it's own pitfalls. Firstly if you have thousand of animated objects on screen your app will probably eat a lot of memory. Taking Social City as an example my browser memory jumps from 20mb to 250mb when game starts. And I have small city (haven't played much) and suspect that with larger one and neighbors it would get a bigger memory footprint. It's still fine, I noticed that some popular FB games take up to 400 and I seen some games in beta that were taking almost a GB of memory(probably they do something wrong) and most of them use raster pipelines. On other hands games that do use displaylist (as far as I can say Frontierville uses displaylist) are not that much smaller. Frontierville is 250mb too.Another problem is that you loose lot of flexibility and interactivity that is already built in to Flash rendering system like that it automatically detects what areas of screen to redraw. This is hard or even impossible to do with BitmapData approuch.So in my view you just exchange one things fro another:CPU for memory(may be not that much though)Transparency of code(no blackbox) for lack of features some of which probably are not impossible to do at all.
Hi Ben, I think that for 90% of cases, even games, display list is the way to go. There are too many reasons to list quickly, so I will follow up with my own blog post, but consider: workflow, nesting, transformations, timeline animation, filters, filesize, mouseEvents and dev time. I have made many Flash games, and very rarely is rendering speed an issue if you know what you are doing with the display list. Consider that many Flash games have to be turned around in 2-4 weeks. Big games like Social City are definitely the exception rather than the rule.
This is where the “gullible and trusting” part of my tweet comes in (I didn't really deeply explore it in the blog post). That is to say, the API temptingly lures you off the fast path, putting you in a situation where you have to start doing these exhaustive workarounds to suppress normally invisible behavior. The problem isn't any single issue, it's that you're working in an idiom where you have given up control. Certainly, you can deal with all the issues and get your app working, but I find it problematic that I'm even in a situation where the issues come up at all!I would greatly prefer a concise API that did one thing well and quickly (like presenting a framebuffer to the user). It would be harder to use but there are plenty of middleware developers to deal with it for people. Then you end up with the ability to write a system appropriate for what you are doing, rather than having to bend an existing complex and ad hoc system to fit.Realistically, we're both going to be developing display list oriented applications for years to come. 🙂 But sometimes it is fun to explore what could be.
I found that setting both mouseEnabled and mouseChildren on absolutely everything will do the trick for the mouse CPU issue. Just loop through every child of some common parent, and keep going down until you hit every InteractiveObject you can reach. You'd think mouseChildren on the common parent would do the trick, but it doesn't (or at least it didn't a few years ago when I actually had a use-case for that many objects displayed at once). To interact, put mouse events on a different display object and use the coordinates to figure out what was touched. I went from a mostly unusable SWF with 20,000+ display objects that maxed out CPU to one that ran nice and quick. In that case, only a few things would be animating at one time, though, and I can still imagine that rendering might be a problem for your particular use case with a lot more animation. However, I just wanted to point out that the mouse CPU issue can be addressed, if not quite so obviously.