Marbling It Up

@MarbleItUp has gone public. We will be launching soon on Switch. I wanted to talk a little bit about the game and how it came to be… and share some sweet GIFs (in high framerate black and white for download size reasons).

riding_dat_spline.gif
An extremely early baby shot of the game. Almost everything here – including the code – was later replaced. And that new stuff runs at 60hz on Switch in handheld mode.

Just over a year ago, Mark Frohnmayer came to me with a proposal – let’s build a sweet marble game and put it on Switch. The Switch at that time was a brand new platform, but the potential was immediately obvious to both of us. I agreed to partner with Mark and all of a sudden – after several years away – I was back in the game development world.

I didn’t know it on day one, but making Marble It Up! was going to be a rare privilege. I am incredibly proud of this game – the team, the levels, the visuals and the gameplay are all amazing, and I count myself privileged to be involved. This is really a great game.

As we near release, I feel confident it will be a hit. I am just hoping it will be blockbuster Michael Bay hit, not a indie niche Kevin Smith hit. Anyway, on with the story…

The Team

Like Danny Ocean, we had to get the old crew together. I already had Miha, an awesome coder, on my team from past projects, so he and I got rolling (hah) right away. Meanwhile, we immediately reached out to Alex, original level designer for Marble Blast, and Todd, one of the most talented (and hardest working) artists I know. We also pulled in the incredible Solovox to make an awesome soundtrack and Mike Jones to do the rest of the audio.

Along the way we ran into long time fans of our work on the original Marble Blast series. We brought in the team behind QuiVR, Jonathan and Fletcher, as well as long time community members, speed runners and developers Matan and Conor.

With the team thus completed, we went into full production. Development slowed down a little bit when a few months into development my Loom technology was acquired and I had to devote my focus to a roll (ho ho ho) with the acquiring company. This ended up being short lived and I was able to refocus my attention after about 10 months of distraction (so to speak).

All of our development was self funded. This gave us total control over direction, but also required all of us to really commit to the game. We’re using a studio model for the game itself – it has its own entity that holds the IP and pays out royalties. Maybe I’ll talk about the basic structure we used for that some day down the road.

Development Path

For this project, we used Unity 2017. It comes free with Switch developer access, and it looked like the best path forward on short notice. I briefly considered using Torque, the first commercial game engine I worked on. Because it was long in the tooth, it would require a solid porting effort before development could begin in earnest. We just didn’t have that luxury.

Although older versions of Unity were shaky, it has really matured in recent years and 2017 was easy to work with. Very quickly we had something up, and I started taking little progress GIFs (in black and white so they’d show up in Slack quickly). We got a ball rolling around in very short order:

InertialBall.gif
Very early rolling behavior.

The movement took a while to get right (more on that later). But ProBuilder made it super easy to throw a level together, and with a little work we got elevators going.

marble_on_elevator.gif
One of the first elevators.
collapsing_platform.gif
And collapsing platforms!

 

Being able to easily tie any geometry into a mover was a huge benefit for level design. It did require some discipline with prefabs to create consistent reusable elevators, but one-off showpieces like those found in Cog Valley or Staying Alive were super easy to make.

The physics simulation was very detailed, and one of the elements needed for both physics networking was a capacity for rewind. Instead of only allowing for a tick or two of rewind, I generalized to allow for arbitrary rewind. This made level testing MUCH easier:

rewind.gif
Check out the hook while my DJ revolves it.

And it also set the stage for two other features, ghost racing and replay. (I love ghost races way more than anyone else on the team, but I’m the lead programmer so they had to ship it.) Being able to watch and race against other players routes, as well as compete against yourself, adds loads of replay value to the game.

ghost_race.gif
What better challenger for you to face…. than YOURSELF!

Not to say everything was wine and roses. We quickly found that Unity’s physics were completely inadequate for our needs. In the end, about half of the game code was for our custom marble physics engine. I feel comfortable claiming that no other marble game comes close to our level of fidelity. Part of the reason we have our own physics is in order to make them unrealistic, too – sometimes bending the laws of physics in juuust the right way makes things way more fun.

GravitySurfaceRoll.gif
An extreme case of gravity surfaces.

We also found that the default Unity renderer was really not appropriate for the game. Shadows provide important depth cues, but Unity’s dynamic shadows were fuzzy and required very careful level design and lighting to be present in all gameplay situations. In the end, a simple projected blob shadow gave us a crisp look and perfect visibility to support the gameplay. It also saved us many man hours tuning levels.

Particles were also a huge pain – especially mesh particles. Unity does horrible slow things to these and cost us massive amounts of performance on the CPU-light Switch hardware. In the end, we had to make small systems to handle the costliest cases in order to meet our performance goal of 60hz in handheld mode.

rainbowMeshAnimation.gif
Without custom rendering, this effect could bring the Switch to its knees.

We also provide a full reflection on the marble. When I implemented this for Marble Blast Ultra, it took me a week or so to modify the scene graph to re-render the level in all six directions. Performance took some care, but ultimately it ran fine using more-or-less the default engine code. This was on the Xbox 360, which has way less RAM, a slower GPU and weaker CPU. Under Unity, on the more powerful Switch, we ended up having to write our own custom CommandBuffer based renderer and our own set of lightweight shaders in order to hit 60hz. This was an unexpected surprise late in development.

In general, Unity was excellent for prototyping and able to be coaxed into viability for production.

Conclusion

Man, I am stoked to see this game go out into the market. It has been a year of my life, some great challenges, some great fun, and hopefully, a game that brings a little bit of joy into the world. Let’s marble it up!

Game Articles Online

I wrote some game reviews/articles a while ago in collaboration with Blockland creator Eric Hartman.

All 12 are now online again, thanks to Eric. I’m especially proud of the history of every MAME-supported baseball game from 1976-1985, the article we did for GameDev.net titled Learning from the 3000 Classics, and our review of the 90s arcade version of Aliens.

Check ’em out!

I Wrote A Book: Video Game Optimization

More precisely, Eric Preisz and I wrote a book!

The book is called Video Game Optimization, and it covers everything you need to know to get maximum performance from any software project – but especially games. If you’re struggling with getting a great framerate out of your game, I highly recommend checking it out. 😉

Video Game Optimization goes all the way from high-level concepts like planning for performance in your project’s timeline, to determining which broad area of your system is a bottleneck, down to specific tips and tricks for optimizing for space, time, and interactivity. Based on the course that Eric Preisz taught at Full Sail University on optimization, it isn’t the only book you’d ever want to read on the subject, but we think it is a great introduction!

The journey from that initial conversation where our mutual friend Jay Moore introduced us and suggested I would be a good co-author, to the day when we finished and shipped the book was a long but rewarding trek. Eric moved across the country from Florida to Nevada, as he moved from teaching at Full Sail University to running the Tech and Tools group at InstantAction. He also became a father with the arrival of his son, Grant. I left after 5 years at GarageGames and helped build a new company, PushButton Labs.

A lot has changed while we wrote it, but it still felt really good to arrive at GDC, visit the book store outside the exhibition hall, and finding a big stack of Video Game Optimization sitting front and center. 🙂

Adobe MAX 2009 BYOL: Build a Flash Based Platformer in 90 Minutes

I will be doing a BYOL session at Adobe MAX 2009. It’s a lab where you learn how to build a platformer in Flash in 90 minutes, and it’s Wednesday at 4pm. It is titled “Build a Flash Based Platformer in 90 Minutes” in honor of its subject matter.

It is pretty cool stuff, and I’m excited to be sharing it! You get introduced to the PushButton Engine, get a preview copy of Clint Herron’s excellent Platformer Starter Kit, and (assuming things go smoothly), you end up building this platformer:

picture-1

(Click image to play demo)

We’re excited about it. 🙂

If you are a Flash game dev and at MAX or in the LA area, I’d love to talk games with you. Shoot me an e-mail (ben dot garney at gmail dot com) or DM me at @bengarney and let’s make it happen!

Also, with luck, Monday there should be a cool update on that secret project I was working on. 🙂

See you at MAX!

Technical Notes on O3D

Google released O3D, a web plugin for 3d rendering, today. It’s a pretty sweet piece of work. Definitely check it out if you have any interest in 3d rendering, the web, or JavaScript.

There are a couple of cool pieces to this puzzle, and I wanted to call them out to other people who might be reviewing this technology. The two sentence review: I am blown away. They got this right.

(FYI: This post isn’t meant to be a full walkthrough, just a quick indication of the cool jumping off points in the codebase.)

Stop number one is the graphics API abstraction. This is rooted in o3d/core/cross/renderer.h, which provides an abstract interface for performing render operations. This is sensibly designed, oriented for SM2.0 through SM4.0 level hardware. It will also deal with DX11 class hardware, although it won’t expose all of the niceties DX11 gets you. It wraps GL (o3d/core/cross/gl/) and D3D9 (o3d/core/win/d3d9). The purpose of all this is to provide a common ground for all the higher level code to issue draw commands from. It is not directly accessible from JavaScript (although some of the related classes like Sampler and State are.)

As an aside, writing a Pixomatic or Larrabee backend would not be hard. At GarageGames, we had a rendering API similar to this (unimaginatively called GFX), and it was fantastically useful. Last time I checked, there were DX8, DX9, DX10/11, GL, and Pixomatic backends for it in varying states of usefulness. We even had one guy write a backend that would stream draw commands over a socket, which was pretty cool. In the context of O3D, they have an implementation of Renderer that queues everything into command_buffer, then streams it to a command buffer server running in a separate thread, which issues the actual draw commands. It’s unclear whether this is used in the current version of the plugin based on casual inspection, but it’s a great example of what good design can get you.

The next piece is the DrawList, which is where most of the heavy lifting for drawing happens. JavaScript, of course, is not really desirable to have in your inner rendering loops, so you queue everything you want to draw (DrawElements) into a DrawList. This is wrapped by higher levels, of course, but it represents the lowest level API that’s available to JS code. All the sorting and state management to get stuff on screen happens in this area.

Alongside the DrawList stuff, you find the material system, which is pretty slick. You write your shaders in their shading language, and it converts to HLSL SM2.0 and Cg (arbvp1/arbfp1). This is enough to do almost anything you might want to (as fantastic as SM3.0+ is it’s not really necessary for most rendering). There’s a full SAS system so you can interface programmatically with your shaders.

Above this, you get into the scenegraph. Now, I subscribe to Tom Foryth’s views on scenegraphs, which is that they are basically a bad idea, but I think the Google guys were smart and set up their API so intelligent developers can avoid being screwed by the scenegraph. The scenegraph-esque stuff they do have (they call it a rendergraph) is powerful enough you can do most rendering without going nuts, and lets a lot of the heavy lifting stay in native code, where it will be fast. You end up doing retained mode-ish things, but since JS is slow, and most JS developers come from a DOM background, it works better than you might expect.

There are a lot of cool miscellaneous features, too. They embed the fast V8 JS engine right in the plugin so you can have consistently fast JS execution. They support falling back to an error texture (via Client.SetErrorTexture) when you fail to bind a sampler. You can group objects via Pack objects for easier management and stream things from archives, too. There are debugging aids and libraries for math and quaternions.

You should check out the samples. There’s a lot of impressive stuff. There’s the beach demo in the video at the top of this post, but they also do some cool stuff with animation, HUDs, text rendering, picking, and stenciled teapots. And everything is done in JavaScript right in the web page. Even the shaders are embedded in <script> tags!

O3D is a great piece of technology, and I hope it thrives. I’m excited to see what people build on this (and I’ve got a few ideas myself ;).

Tweaking your game with Google Spreadsheets

tweaksheet Our latest game, Grunts: Skirmish, has 200 tweakable parameters. There are 9 player units with three levels of upgrade, and another 9 enemy units. Each unit has between three and ten parameters that can be altered.

We tried a few approaches – hand-editing a large XML file (but it was too large and spread out) and an in-game tweaking UI (but it was too much work to get the UI to be friendly to use). The old standby of having the designer and artist file bug reports to have the programmer update the game wasn’t getting us very far, either.

“Well,” says I, “We’re some sort of Web 2.0 startup, right? And we’re developing a Flash game aren’t we? And Flash can talk to websites, can’t it? And don’t we use Google Docs for everything?”

It turns out there is an XML feed from public Google spreadsheets. And ActionScript 3 supports E4X, so you can directly manipulate XML without any extra work. Now we tweak our game using a shared spreadsheet up on Google Docs.

I wrote a parser for their format:

// Extract the entries. It's namespaced, so deal with that.
var xmlns:Namespace = new Namespace("xmlns", "http://www.w3.org/2005/Atom");
tweakXML.addNamespace(xmlns);

// Parse into a dictionary.
var cellDictionary:Dictionary = new Dictionary();
for each(var entryXML:XML in tweakXML.xmlns::entry)
{
   cellDictionary[entryXML.xmlns::title.toString()] = entryXML.xmlns::content.toString();
}

And wrote a quick component that would fetch the spreadsheet feed, parse it, and stuff it into the right places on named objects or template data. Now I have a little entry in our level file that looks like:

<object id="googleTweaker">
   <component class="com.pblabs.debug.GoogleSpreadsheetTweaker">
     <SpreadsheetUrl>http://spreadsheets.google.com/feeds/cells/0d8somekey848x/od6/public/basic</SpreadsheetUrl>
     <Config>
        <!--  Grunt Level 1 tweaks -->
        <_><Cell>B3</Cell><Property>#TheGruntProxy.creator.WarPointCost</Property></_>
        <_><Cell>C3</Cell><Property>!Grunt.health.mMaxHealth</Property></_>
        <_><Cell>D3</Cell><Property>!Grunt.ai.AttackSearchRadius</Property></_>
     </Config>
  </component>
</object>

Each line maps a cell in the spreadsheet to a property on a template or active game object. Some properties have to be set several places, which the system deals with automatically.

The biggest wrinkle was Google’s crossdomain.xml policy. Basically they do not allow random Flash apps to access their site. So I had to write a small proxy script, which sits on our development server next to the game and fetches the data for it. Figuring out I had to do this took more time than any other step.

The main difference between the snippet and the full code is the version in our repository is 220 lines long. I only have around 150 of the full set of 200 parameters hooked up, but after a hard afternoon’s work, the process for tweaking the game has become:

  1. Open Google Docs.
  2. Edit a clearly labeled value – like level 1 grunt health.
  3. Restart the game, which is running in another tab in your browser.

This takes you about a minute between trials. Not too bad. Before this, the process was:

  1. Get the game source from SVN.
  2. Find the right XML file – there are several.
  3. Find the right section in the XML – altogether we have 200kb of the stuff for Grunts!
  4. Change the value.
  5. Commit the change.
  6. Wait 5-15 minutes for the build system to refresh the live version of the game.

Ten minutes per tweak is not a good way to develop.

I’ve heard about developers using Excel spreadsheets for tweaking, but can’t find anything about using Google Docs to do it. But Google Spreadsheet is obviously a better choice. It has built-in revision tracking. You can edit it simultaneously with someone else. You can access live data in XML either publicly (like we did) or privately via their authentication API. It’s absolutely worth the half-day of your time it will take to add Google Spreadsheet-based tweaking to your game – even if it’s a non-Flash game, downloading and parsing XML is pretty easy with the right libraries.

I strongly suspect this feature will find its way into the next beta of the PushButton Engine. Which, by the way, you should sign up for if you are interested in developing Flash games. We’re bringing people in from the signup form starting this week. If you want more information, or just like looking at cool websites, click below to check out the new version of the PBEngine site, which has a bunch of information on the tech. Tim did an awesome job on the site design.

betashot

Edit: Patrick over on the GG forums asked about the proxy script. It’s actually ludicrously simple. Not very secure either so I wouldn’t recommend deploying it on a public server. I got my script from a post on Aden Forshaw’s blog. In the real world you would want to have some security token to limit access to your proxy script… but since this is for tweaking a game that is in development I didn’t sweat it.

Potpourri – Jul 22, 2009

A buddy of mine from GarageGames days, Orion Elenzil, posted some notes on how to get started with the free Flex SDK. Take a peek to see just how simple it can be to get started with Flex.

for those who may come across this post on their search to learning how to set up a command-line-only Flex environment, and who also know previously zilch about Flex/AS/Flash, i sketch the steps very roughly here: http://elenzil.com/flash/flash_1

— Orion

If you’re into Flash coding, you should also read Rob Sampson’s post on AS3 Math Optimization – int is the new floor(). Rob’s been doing ActionScript a lot longer than I have, and I love the historical perspective he can provide. You can tell he’s a graphic designer – his performance charts look awesome.

I have to quote this from Jeff Tunnell’s blog, Make It Big In Games:

I stumbled across this YouTube video of Gary Vaynerchuck of Wine Library TV giving a presentation at Web 2.0 Expo. At first I just thought the guy was a dick, and he even calls himself that at one point in the presentation, but it turned out to be a fascinating 15 minutes of video. Gary essentially says what I have been saying, i.e. make sure you love what you do, work hard, and things will work out, but he says it in a much more succinct, hard edged way than I do.

Check out the video here.

Blast From The Past: Blitz3D Models in Torque

I found some old screenshots on a backup, and uploaded them to my Flickr account.

As I was filing them away, I looked back in my .plans at GarageGames and realized that I had never talked about it publicly. At least, the site search didn’t turn anything up. Well, I think it’s probably safe to do so now. 🙂

Way back in April of 2005, I wrote a Blitz3D loader in Torque for Adrian Tysoe. It loaded Blitz3D files, with all their texture and material data, and it did polysoup collision, as well. This was several years before I released the OPCODE-based poly soup resource, which I think is now (in some form) in the official Torque codebase.

Montage from Blitz3D Loader Work

I love seeing how graphics code comes together, so I thought I would share this with the world – you might like clicking through the images and seeing how each step changed things. By far the biggest pain point was parsing the (sometimes obtuse) Bitz3D format. The format might be a pain, but in the hands of a good artist like Adrian, it gave really solid results. This was all before shaders were mainstream, of course, but still – the sand had a nice specular highlight, there was lots of clever texture layering, and the whole thing was pretty fast to render.

The more interesting aspect to doing the B3D loader was the prevailing opinion in the Torque community at the time that loading a non-DIF/DTS format was impossible – as was deviating from the collision models those formats used (convex polyhedra mostly). I’m not quite sure where they got this idea, but it was awfully frustrating at times to see people beat their brains out on those formats when there might be better options for their specific situation.

Partially, I did this project to prove to myself that I wasn’t just bullshitting people when I said that writing a loader/renderer was easy (certainly easier than canceling your project because another format wasn’t working for you!). It took me a month or so of part time hacking to get it all together. It didn’t animate, but it was just for environments – DTS is actually an awesome format for characters, and the 3Space runtime is very tight, one of the best parts of Torque in terms of features. (Aside: I hope they get a good Collada->DTS converter as the main pipeline sometime soon!)

Two years passed before anyone put polysoup in without rewriting the rest of the engine to use some other physics SDK. Actually, I ended up being the one to integrate polysoup with DTS. :-/ I was surprised no one else beat me to it! I brute-forced it – every triangle as its own convex. Torque ran this approach like a champ as long as you didn’t feed it tiny triangles. Did the same trick again when it came time to write Atlas1 collision a few months later, but that wasn’t really “polysoup” in a meaningful way for anyone since it only imported heightfields.

It’s interesting to me that the current Torque maintainer (and all around good guy) Matt Fairfax wrote loaders for .3DS, Quake3, and Unreal formats, and the maintainers that came before me (Rick Overman, Mark Frohnmayer, and Tim Gift) have also done things of this nature (in fact, they are at least partially responsible for DIF and DTS). I guess you don’t get to hold the reigns of TGE unless you’ve written a couple 3d renderers/loaders.

I think in total I’ve done 4 loaders with renderers – Blitz3D, Chunked LOD (for Atlas 1), Atlas2, and OBJ. Really, OBJ doesn’t count. Every graphics programmer in the universe has hacked up a OBJ loader – from Carmack on down.

In conclusion? I guess there are a few morals to draw from this nostalgia-inspired post:

  • Keep shots from your old projects. It’s fun to reminisce! I have a few hundred shots from Atlas development that might be fun to write about some day…
  • Don’t be afraid to buck common knowledge and go out in a technical direction no one thinks is feasible. Try it small and if it works, keep going.
  • Do projects for fun! Good stuff will always find an opportunity for reuse – be it some code, or a technique, or even a fresh perspective. The ideas and techniques I came up with for this project ended up helping me for years to come. Working with Blitz3D’s format helped me see Torque’s approaches in a fresh light.

the_more_you_know2

Edit: Adrian Tysoe posted on GarageGames about his side of this experience – here’s what he had to say:

Heh, I remember working on this with you. I figured they were the starting point for some of the later torque advancements. Just wish that the DTS had been updated to support more than 1UV. Was pretty fun to work with and performance was decent.

The worst part was trying to match invisible torque terrain collisions to .b3d meshes just to get the nice water blending around the edges. The terrain used the coldet collision for the actual character and weapons etc.

Was pretty neat and I enjoyed having 2UV’s to playwith so I could bake my own lightmaps in 3dsmax of gile[s], in many ways the most useful thing to make TGE rendering look a bit nicer.

Here’s a couple more pics the project Jeremy and I started using it.

oust-3a

oust-7a

Blockland Physics

I spent some of my downtime over the past few months working on client-side brick physics for Blockland. The feature has finally been announced, so I can talk about it. The video shows off most of the features; the main one that isn’t shown is the interaction between players/vehicles and bricks; they will push bricks around but aren’t affected themselves. Watch it in full screen, it’s full 720p HD video.

There were two big problems related to doing physics for Blockland. The first was the scale of the problem – there can be well over 100,000 bricks in a single server, which is beyond the capabilities of most physics SDKs to simulate as discrete objects. I started with PhysX, and moved to Bullet after realizing that the PhysX runtime is 80mb, far more than could be included in the Blockland download (which weighs in at 20 mb). Both of these libraries broke in different ways with large numbers of objects.

At first, I implemented a management system for brick proxies, so that it kept them under the hard limit in the SDK (around 2**16). PhysX accepted this, but Bullet’s broadphase has some stuff that’s O(# objects) or worse, so it fell down. Eventually, I moved everything into the same system used for static world geometry, which was a grid of static meshes. It turns out that Bullet is a bit faster at creating these mesh objects than PhysX was.

The static mesh cache took quite a bit work to get solid. Because the simulation is for aesthetic purposes, it can tolerate a fair amount of “fudge,” which I take full advantage of. Nearly every kind of update is timesliced so that only a little bit is done each tick. This keeps things smooth, even at the cost of the physical state being inconsistent for a tick or two. Most updates are lazy, as well, only done if a dynamically moving brick, player, or vehicle comes into the area.

The other problem is the wide variability of Blockland user’s computers and usage patterns. Not every user has enough CPU to run physics. And every user has the potential to build something that is very resource intensive to simulate. I spent a lot of time implementing a “physics diaper” – logic to detect when physics calculations were taking too much time, and scaling back the simulation until it’s fast again. This takes two forms. First, if physics ticks are too slow, the simulation is decimated – every other brick on the list of moving bricks is converted to a lighter weight parametric simulation that doesn’t consider collisions. If they remain too slow, then eventually the physics simulation is disabled entirely, until the user turns it back on. This can help with very complex builds or very slow computers.

Thanks to good physics middleware (I include both PhysX and Bullet under the “good” category, even though PhysX is a little on the bloated side), I was able to solve a pretty tough problem – simulating motion for hundreds of thousands of bricks on commodity hardware – in short order. And I have to thank my friend Eric for making such an awesome sandbox and letting me play in it. 🙂