Ray Tracing: Shoot Before You Fly

I'll take a break from iPad drivel for a few posts; at least one or two of you don't already own one. (Seriously, it's simply easier to blog about X-Plane for iPad because it is already released; a lot of cool things for the desktop are still in develpment.)

In response to my comments on water reflections in X-Plane 950, some users brought up ray tracing.

My immediate thought is: I will start to think more seriously about ray tracing once it becomes the main technology behind first person shooters (FPS).

Improvements in rendering technology come to FPS before flight simulators (and this is true for the combat sims and MSFS series too, not just us). Global shadows, deferred rendering, screens-space ambient occlusion...the cool new tricks get tried out on FPS; by the time they make it into a flight simulator the technology has moved from "clever idea" to "standard issue."

Consider that X-Plane now finally has per-pixel lighting. Why didn't we have it when the FPS first did? Well, one reason is that the FPS were cheating. If you look at the papers suggesting how to program per-pixel lighting, at the time there were all sorts of clever techniques involving baking specular reflections into cube maps and other such work-arounds to improve performance. These were necessary because titles at the time were doing per-pixel lighting on hardware that could barely handle it. X-Plane's approach (as well as other modern games) is to simply program per pixel lighting and trust that your GeForce 8800 or Radeon 4870 has plenty of shader power.

I believe that the reason for the gap between FPS and flight simulators come from two sources:

1. Viewpoint. You can put the camera quite literally anywhere in a flight simulator, and thus the world needs to look good from virtually any position. By comparison, if your game involves a six foot player walking on the ground (and sometimes jumping 10 feet in the air) you know a lot about what the user will never see, and you can pull a lot of tricks to reduce the performance cost of your world based on this knowledge. (This kind of optimization applies to racing games too.

   To give one simple example of the kind of optimization a shooter can make that a flight simulator cannot, consider "portal culling". A portal-culled world is one where the visibility of distinct regions have been precomputed. A trivial example is a house. Each room is only visible through the doors of the other rooms.* Thus when you are walking through a room, virtually no other room is being drawn at all. The entire world is only 20 by 20 meters. Thus the developers know that they have the entire hardware "budget" of computing power to dedicate to that one room and can load it up with effects, even if they are still expensive.

   (A further advantage of portal culling is a balance of effects. Because rooms are not drawn together in arbitrary combinations, the developers may find ways to cheat on the lighting or shadowing effects, and they know nothing will "clutter" the world and ruin the cheats.)**

2. Often the FPS will have pre-built content, rather than user-configurable content. Schemes like portal culling (above) only work when you know everything about the world ahead of time and can calculate what is visible where. The same goes for many careful cheats on visual effects.

   But a flight simulator is more like a platform: users add content from lots of different sources, and the flight simulator rendering engine has to be able to render an effect correctly no matter what the input. This means the scope of cheating is a lot smaller.

   Consider for example water reflections. In a title with pre-made content, the artists can go into the world in advance and mark items as "reflects", "doesn't reflect", reducing the amount of drawing necessary for water reflections. The artist simply has to look around the world and say "ah - this mountain is no where near a lake - no one will notice it."

   X-Plane can't make this optimization. We have no idea where there will be water, or airports, or you might be flying, or where there might be another multiplayer plane. We know nothing. Everything is subject to change with custom scenery. So we can't cheat - we have to do a lot of work for reflections, some of which might be wasted. (But it would be too expensive in CPU power to figure out what is wasted while flying.)

Putting it all together, my commentary on ray-tracing is this: the FPS will be able to integrate small amounts of ray tracing first, because they will be in a position to deploy it tactically, using it only where it is really necessary, in hybrid ray-trace + rasterized engines. They'll be able to exclude big parts of the scene from the ray tracing pass, improving performance. They'll be able to "dumb down" the quality of the ray trace in ways that you can't see, again improving performance. The result of all of this will be some ray tracing in FPS when the hardware is just barely ready.

For a flight simulator, it will take longer, because we'll need hardware that can do a lot more ray tracing work. We won't know as much about our world, which comes from third party content, so we won't be able to eliminate visually unimportant ray traces. Like deferred rendering, shadow mapping, SSAO and a number of other effects, flight simulators will need more computing power to apply the effects to a world that can be modified by users.

(Is ray tracing even useful, compared to rasterization? I have no prediction. Personally I am not excited by it, but fortunately I don't have to make a good guess as to whether it is the future of flight simulation. The FPS will be able to, by effective cheating, apply ray tracing way before us, and give us a sneak peak into what might be possible.)

* There never are very many windows in those first person shooters, are there?
** To be clear: there is nothing negative about the term "cheat" in computer graphics. A way to cheat on the cost of an algorithm means the developers are very good at their jobs! "Cheating" on the cost of algorithm means more efficient rendering. If the term cheating seems negative, substitute "lossy optimization".