Scripting is really outside the domain of what I was focused on. For SL, it is only server side but as you point out, something like a windmill might be better on the client. My main focus was how to architect the geometry pipeline so that a particular format isn't built into the standard.

Stepping beyond the geometry question, for any geometric data on the server, you have to have some way to render it on the client. If you are animating the geometric data (i.e. via a client side script) some interface is needed to the "geometry format specific" renderer on the client.

At this point, you can't really say exactly what the interface to a "format specific geometry renderer" might be. You need an interface between the real renderer in the client and any "addin" renderers. If you have client side scripting, you may need another. You might also envision some server side interface for comunication back to the server. Or the renderer may be buried under other code (i.e. scripts may not even care about a "renderer", just the that object moves)

Exactly how these interfaces are implemented is yet another *huge* question!

Are local animations supported(ie non networked) like a windmill. Looking at the diagram I expect it might be handled by the renderer on the client? Ie if the language supported a runtime then it would be run there?

Is it the scripting mechanism which provides this or do you have local interpolators?

How does a client piece of code(ie say a car) that wants to talk with the network do this? Ie say I want to display the current time and weather in an in-car display? Typically you'd use web service, ajax etc to get this data. Can an object have access to the web to display local information?

http://wiki.secondlife.com/wiki/Geometry_Based_Architecture_View

While this is focused on some long term future Second Life, the general idea abstracts away the specific geometry formats. That is, it sets up a way of working so that any (most?) formats could be used.

This allows formats to compete within the standard rather than making a particular format a standard (kind of like how you can use HTML or PDF or Word as your document format in a browser).

In the VW system we are building we have made a conscious separation between assets and behavior, storing assets in X3D (because our Flux Player web plugin does a great job at rendering it) and dynamic behaviors in Javascript code and ancillary XML data (because a web browser does a much better job at those). As an aside, this separation will also allow us to deliver assets in COLLADA if we so choose in the near future— or OBJ or whatever. But X3D and COLLADA are the front runners and it's practically a coin-toss for us. We already ingest COLLADA into our Flux Studio tool and export X3D anyway.

But as Alan point out, we also have to be careful about what we are defining as "behaviors." I believe that predefined animations can be authored and stored in the DCC tools and preserved throughout the chain as part of the "mesh data," as jwatte calls it and that is far preferable to defining those in program code. i.e. I wouldn't want to animate key frames in a Javascript timer loop.

IMO the big win is in separating program logic from asset data and a good design should reflect that.

I come from X3D bias so I'll state that up front.

What I've liked about VRML/X3D systems is the packaging of visuals and behaviors. Ie when I want a dynamic object I typically need to refer directly to things inside the object. Let's take a car for instance. Let's say I have 3 behaviors. Open doors, move steering wheel and display turn signals. In order to activate the first two you need to know when the user "touchs" the door handle and steering wheel. This is handled in X3D by a Sensor(touch or drag in this case). The sensor is a marker for "when the user interacts with this do something". The other way I have seen it done is more direct. Ie the scripter controls a pick ray and is told what intersected. I like X3D approach in this better because the script is attached directly to the operation. Its not off in some other picking area. The other side of this, display the turn signal. Again this operation is fairly tightly coupled to the graphics picked. Do you swap textures, change emissive color, change a shader param? You can abstract the action, but its still tightly coupled to the choosen geometry.

Can you point me at behavior systems that you like better? I'm especially interested in declarative systems that other tools can make sense of. Just a wad of code for all the behavior system is very hard to read into many tools.

My thinking is that you need at least two formats: one for "static scenery," which supports things like mark-up for entity navigation, collision acceleration, on-demand paged downloading/loading, etc, and one for "dynamic objects," which means anything that animates or moves. Attempts in the past have been made to unify the two, but every successful, shipping virtual world actually splits these two.

For terrain, Forterra has taken steps in developing an open source, open specification format for the paged terrain use case; for more information see http://www.interopworld.com/members/ptf

For object meshes, however, there exist sufficient existing file formats. The two front-runner formats in my mind are X3D and COLLADA. Personally, I prefer COLLADA, because it has more robust support in authoring tools, and it does not over-extend itself in the level of specification. There may be others — MD5 and FLT come to mind.

I'd like to hear what experience you all have with different file formats that may fit the bill, and the pros/cons you've found with those file formats.