Short reflection

I've noticed a trend in my blogging, which is to relate a literal and deeply technical view of my work. This will likely continue for a while until I reach full speed here, but please check the blog out periodically if you find that the current level of discussion is not appropriate for you.

Cheers!
Josh

"Pulling on a thread"

The metaphor refers to one programming task leading to another. In this case, adding support for vertex buffer objects lead to writing a framework for managing OpenGL resource names/handles, and very nearly resulted in an all-out assault on the current content loading framework. Thanks be to Andreas Raab for advising against that (sometimes I can be hasty). But I digress...

What happened this week?

I finished the vertex buffer work alluded to in my last entry. Below, I'll discuss this in excruciating detail.

Work on Brie continues. Howard has been making progress with the coding, and we continue to discuss issues as they arise. A lot hinges on uncertainty about what TeaTime will actually look like. Now that I'm finished with the vertex buffer work, I'll be helping Howard with coding Brie. Fun!

So, tell me about those vertex buffers...

Such an interested audience!

I finished a prototype rendering path using vertex buffers shortly after my last blog posting. Preliminary results were very encouraging: for the previously mentioned large meshes, the frame rate jumped from approximately 5 fps to 35 fps. This encouraged me to implement this feature in a more robust fashion, suitable for general use in Croquet. This work had two major aspects: managing the low-level OpenGL names for the resources, and allowing the vertex buffer rendering path to be disabled when desired (currently, when the OpenGL implementation does not support vertex buffers, and whenever else desired).

In OpenGL, vertex buffers are named by an integer, as are many other types of resources such as textures and compiled shader programs. Recognizing the similarities between the way these resources are created, used, and destroyed, I designed a resource management framework that could support them all. I considered two major architectural approaches. The first was to generalize the functionality of OGLTextureManager, and the second was to create/destroy resource names within the scene graph objects as necessary (for example, this would involve modifying TTexture to directly hold an OpenGL name/handle, rather than referring to it indirectly via OGLTextureManager and OGLTextureHandle). The latter path was rejected because it would require significant reworking of TTexture, and more importantly because the handles are only valid in a single OpenGL context (disregarding resources shared between contexts, which is a can of worms that we can do without opening). Following the lead of OGLTextureManager, I created OGLResourceManager, which maps scenegraph objects (eg: TTexture, TVertexBuffer) to their low-level OpenGL counterparts (OGLTextureName, OGLBufferName).

Wow, that sounds pretty cool. How does this resource management fit in with rendering meshes?

With the resource manager in place, I turned to modifying TMesh to allow the use of vertex buffers. The cleanest way to do this was to make TVertexBuffer into a wrapper around the data (vertices, face indices, etc.). TVertexBuffer needs access to this data anyway, since it might have to create a new low-level vertex buffer object at any time (for example, if the Croquet image is saved and quit, and then restarted, an open TeapotMorph will create a new OpenGL context to render in, and new resources will have to be created in this context). It is also convenient because it makes it easy for the mesh to fall back to the vertex array render path whenever vertex buffers are not available or not desired.

Sounds good, but does it work?

First, the good news. There is a drastic improvement in the speed of rendering large meshes. This is excellent, since it was the motivation for this work.

Now, some indifferent news (not really bad). There was no speed improvement when rendering the default Croquet spaces using vertex buffers for all meshes. Some possible explanations are:

• we are overflowing the GPU memory (64MB on my laptop) and therefore OpenGL must send the data across the AGP bus anyway (essentially resulting in the vertex array model)
• for small meshes, the benefits of vertex buffers are outweighed by the overhead of managing resource names
  
• the bottlenecks are elsewhere for rendering the types of Croquet scenes that are currently typical
  

Preliminary benchmarking and profiling suggests that the last option is the most likely. However, I would have to study the issue more deeply before being willing to bet large sums of money on this opinion.

That sounds wonderful, but I really must go.

Me too. Thanks for listening.

Wisconsin Week Wun

Where to start? I just finished the first week at my new job as a Croquet hacker at the University of Wisconsin. What is Croquet, you ask? I'm glad you did. You might like to start with the project website, although non-techie friend have told me that they find it incomprehensible. My boss has been working on a redesign, which will hopefully be friendly to both end-users and hackers.

My move to Madison was eased by the Lombardis, who have graciously put me up at their house for a week while I found a suitable apartment. I have had a great time staying here, and am extremely grateful for their hospitality.

In between the inevitable adminstrivia, I have actually been able to get some work done this week. I've been working on two tasks, one big and one small. Small one first: increasing rendering performance of large models.

We have access to some high-poly models that were created by a laser range scanner (along the lines of the Digital Michaelangelo project. Such models are typically a) big (ours are over 200000 faces, and they can get much larger), and b) not structured for maximum rendering efficiency (for example, as long triangle strips). Loading the model into the scene drops the frame rate to about 3/second. Using Apple's OpenGL profiling tools, I was able to determine (as expected) that Squeak overhead is not the problem: approximately 95% of the time is spent in glDrawElements(). So, how can we fix this?

Two ideas immediately jumped to mind: using vertex buffers, and restructuring the mesh for faster rendering. The latter is a far more ambitious course, and I decided to try out VBOs first; if the performance improvement is not satisfactory, I can always try to reorganize the mesh later.

The current approach of using glDrawElements() is not a completely stupid way to do things. It is much better than the naive approach of calling glVertex() etc. for each vertex of the mesh. However, there is still the overhead of copying all of the vertex data over the AGP bus for each rendered frame. ARB_vertex_buffer_object is an OpenGL extension that addresses this problem by storing the vertex data in the GPU's memory. I expect to see a significant speedup, as many whitepapers from nVidia and ATI strongly recommend the use of this API for rendering static geometry. The downside is that it is an extension to OpenGL 1.4, which is beyond Croquet's current requirement of OpenGL 1.3 or better (I personally think that we should raise Croquet requirements to match those of Longhorn). With any luck, I will finish coding this approach this weekend.

The big thing that I am working on is the Brie UI framework, which has been designed over the last couple of months by Julian and Howard Stearns, our lead Croquet developer and all-round Smart Cookie. The overall ideas of Brie are best described in various posts in Howard's blog . Howard, Julian, and I spent a lot of time specifying the behavior of left/right mouse button clicks/drags/etc. in the virtual space. Where possible, we took advantage of the deep thought embodied in the Mac OS X UI. Sometimes it was not, since Croquet/Brie is not only a 3D environment, it is a 3D enviroment where (as Alan Kay says) "the authoring is always on". One can see an example of this philosophy in Squeak's Morphic UI. By Friday afternoon we ended up with a specification that we are all happy with. Hopefully we feel the same after reviewing our work on Monday morning :-).

Next, we will tackle the even thornier problem of attaching behaviors (reified as 3D objects) to other objects in the environment. In particular, there are deep questions about how objects inherit behaviors from parent objects (ie: objects higher up the scene graph), how objects customise default behaviors, and how parent objects can enforce certain behaviors in child objects. This is all still a bit of a whirl in my head as I try to catch up with Howard and Julian. Working with mouse button behaviors is so much more concrete! You'll be sure to hear more of this later.

That's my job! I can't imagine a better one.

Everyone else is doing it...

... so I thought I'd start a blog too.