The Dangers of Wandering in the Desert

I have been trying to put documentation on the X-Plane Wiki, and use this blog for announcements and "the inside story", rather than letting the blog turn into a poor-man's users manual. An aircraft developer asked me via email whether there was a blog entry on some of the pitfalls of the v9 panel lighting system. There is not, and the lighting system is under-documented. I will be working on improving the documents over the next few weeks, but the point of this blog entry is: "how did we get here?"

I am a huge fan of incremental software improvement. That's the subject of another blog post (perhaps on another blog), but for now I'll say this: all changes to the rendering engine since version 8.0 have been incremental ones, and yet if you were to look at the code, you wouldn't see a series of band-aids taped on top of each other. Each incremental change leaves the rendering code "fully updated", as if it had been written yesterday. I start each new scenery feature by first reshaping the existing code into the most useful form for what we want to have in the future, and then coding the new feature is relatively simple.

But this strategy has an Achilles heal; if the code being refactored has a public interface (whether it is a file format or programming API), then all of the intermediate steps in the journey become requirements for future products in order to maintain backward compatibility.

This is not a problem as long as the programmer knows where he is going. The danger comes when one of the intermediate steps is actually a step in the wrong direction, and becomes dead weight around a future design.

A Reasonable Progression: OBJ

The OBJ 800 file format has had a reasonable progression* since its birth in version 8. It has gained a number of new features, but each one has generalized and made more powerful previous ideas, such that "legacy behaviors" are not so painful. Some examples:

• Hard surfaces may now be decks (e.g. you can fly under them) or not, and you can specify what kind of hard surface you have. The original hard surface command was simply "it's hard" or "it's not". But viewed under the lense of the new scenery system, that old hard surface command implicitly implied "the surface is smooth" and "the hard surface is not a deck". So the new hard surface command is a more general version of the old one, which continues to work under the new system.
• Animations in version 9 can be key framed; in version 8 you simply specify start and end values. But start and end values are just like having two key frames. So viewed under the lense of the new scenery system, all animations are key framed; older objects always just have two key frames. The new key framed commands are a more powerful, more general version of the old ones.

I can't say that the relatively pain-free evolution of OBJ files over the last 4 years comes from good design or genius on my part - in truth it's probably just good luck. But I think one thing has helped me keep the new OBJ extensions relatively sane: most of them are conceived several months before they make it into X-Plane.

I have a scenery system to-do list that will last me at least another four years; most of it is filled with things that Sergio has asked for. This to-do list acts as sort of a road map for future scenery system extensions; for any possible OBJ change, I can look at it relative to the other todo items and ask: "is this extension going to play nicely with things to come?"

(As a side note, this is one of the reasons why there are not light maps in any of the X-Plane modeling formats. Light maps don't play well with a number of other scenery system extensions. I want to resolve the conflict between these future additions before they go into the sim.)

Wandering In the Desert

By comparison, the evolution of the panel system in version 9 has been more like wandering in the desert than a straight line toward a goal. Repeatedly, I put features into the panel system without a clear roadmap of where we would end up or how they would work together. The result is what you see now when looking over the panel documents: complexity and chaos.

Basically there are several major changes to the panel system that affect each other in strange ways:

1. A more complex lighting model on the 2-d panel in version 920. (That is, the 3 2-d spot lights, and generic instruments with back-lit, mechanical, or glass lighting.)
2. A more complex lighting model in the 3-d cockpit in version 930. (That is, 3-d spot lights, ATTR_cocpkit_region and generic instruments with back-lit, mechanical, or glass lighting.)
3. A separate panel used only to provide texture to the 3-d cockpit. (That is, the 3-d panel.)
   

The problem is the order that they were invented: first ATTR_cockpit_region, then the 3-d cockpit, then back-lit generic instruments, then 2-d spot lights, and then 3-d spot lights.

The result is two sources of confusion:

• Some combinations of features simply don't work together. Since all of the features appear to be independent, I sometimes get bug reports on these. For example, you can't use the 2-d spot lights on the 3-d panel. This is not a bug, it is by design! I will explain why some of these limits exist in future blog posts.
• Among the remaining combinations that do work together, there are a lot of choices about how to structure a plane - too many choices!

This second point is a tricky one: X-Plane has to continue to support whatever set of features was available for any given release (864, 900, 920, 930) so that older planes continue to work. But some of those combinations (e.g. the ones that exist in version 900) don't make a lot of sense for new planes made in 930.

I am open to ideas on how to solve this. I intend to document a "correct formula" for a modern plane, perhaps with tutorials, on the Wiki. I am also considering programming Plane-Maker to flag unusual combinations of features as a warning when saving 930 planes.

Either way, I fear I've learned my lesson from the panel system: incremental improvement of code is only a good idea if the programmer knows where he is going! Next time I will use Google Maps. :-)

* I suppose that whether you think the OBJ 800's evolution has been reasonable depends on your standards for file formats. OBJ 800 absolutely does show growing pains. I would only say: consider the number of revisions and the change in the hardware platform OBJ 800 feeds when you consider its stretch marks.