Stereo Rendering with Data Explorer
Notes for OpenDX
dx -edit TestStereo
When DX comes up, put it in exec-on-change, and you'll see a simple image of my favorite sample data set. In this example, interaction modes are specified by keystrokes in the view window. If you type 'z' you'll enter zoom mode; button down motion up and down the imge zoom in and out. 'r' gets you rotate mode, and 'p' gets you pan mode.
Similarly, stereo modes are selected with keystrokes in the image window. If you hit the '2' key you will select a simple side-by-side stereo mode; you might have to zoom the image out to see it. I don't recall what '3' gets you, but '4' gets you a squashed over/under stereo mode.
There's more discussion of all this below, including how to set it up
appropriately for your own hardware environment.
Stereo System Modes
A default set of two stereo modes should, however, suit most needs. The first of these (index 0) implements stereo in side-by-side viewports in a single window (which is either passed into the DX Display modules from the SuperviseWindow module, or specified by the camera passed into Display, or which defaults to 640x480) and does not affect the hardware at all. This mode is quite useful for cross-eyed stereo. The second mode (index 1) implements what appears to be a standard SGI mode, in which a single window (which must be a full-screen window with no 'decorations' - eg. window borders and menus) is divided into separate over-and-under viewports, one filling the top 492 rows of pixels and the other, the bottom 492 rows. This mode allows you to specify the command used to place the hardware into stereo mode as environment variables. Thus, if you place the system into stereo mode by entering the command '/usr/gfx/setmon -n STR_RECT' at a command-line, by placing this command string into an environment variable named DX_INIT_STEREO_COMMAND and the corresponding command to revert to mono mode (on our SGI, '/usr/gfx/setmon -n 60HZ') into an environment variable named DX_EXIT_STEREO_COMMAND, DX will make the necessary calls via system() when the stereo mode 1 is entered or exitted.
I will include a sample a run-time loadable set of functions that illustrate
how you can supercede the default modes. If you have to do so, however,
I'd very much like to hear about it so that we can augment the built-in
modes prior to a formal release of this technology.
Stereo Camera Modes
The default stereo camera model implements stereo by providing look-to
and look-from separation (eg. parallel displaced viewing directions) and
asymmetric projection matrices. These are parameterized by three values:
the separation of the viewpoints as distance in model space (the 'ocular
separation'), the degree of asymmetry of the projection frustums - the
'frustum overlap', and the aspect ratio to be used. While the ocular separation
and aspect ratio are fairly obvious, the parameterization of frustum overlap
requires explanation. It is given as a floating point number, varying from
0 to 1, in which a value of 0.0 corresponds to zero overlap in the divergent
direction, and 1.0, to zero overlap in the convergent direction.
The Stereo Sample Net
To run the sample, install the new files, then set an environment variable named 'DXHWMOD' to 'DXhwddOGL.o' to cause DX to use OpenG. Then set the environment vaiables named DX_INIT_STEREO_COMMAND and DX_EXIT_STEREO_COMMAND environment variables to the appropriate command-line strings that will set your hardware into stereo mode - for our SGIs, it would be "/usr/gfx/setmon -n STR_RECT" and "/usr/gfx/setmon -n 60HZ", INCLUDING thr double quotes (I think that if you use csh, it would be something like "setenv DX_INIT_STEREO_COMMAND "\"string\"") and run the network by typing 'dx -edit stereo'. You will see the network editor open. Hit Execute On Change from the Execute pulldown and
a 800x400 window will appear containing an image of two isosurfaces of a test dataset. You'll see that the window is in rotate mode; click and drag to rotate the image. Now with the mouse in the image window, hit the '2' key. The image will go into side-by-side stereo mode, showing two 400x400 images; if you have the knack of it, you can see this in stereo by crossing your eyes. Hit '1' to return to mono mode. In either mode, pressing 'r', 'p' or 'z' will select rotation, panning or zooming from the default user interactors.
In model space, the geometry you are looking at fills a cube of side 2, centered at the origin. Default values are used for the separation and overlap of the default camera model. By pressing the 'w' key in the image window you can widen the separation, and by pressing the 'n' key you can narrow it. (Note: currently there is a negation problem here - the default separation is -1 and 'w' and 'n' are reversed). By pressing the '-' key you can increase frustum overlap, while by pressing the '=' key you lessen it.
Now, using the Windows / Open All Control Panels pulldown selection, open the control panel. You will see interactors for the default settings of the separation and overlap; you can revert to the default values by selecting 'reset separation' or 'reset overlap'. The amount that keypress events either increase or decrease these values is adjustable using the 'Separation stepsize' and
'Overlap stepsize' interactors, and you can revert to the defaults by pressing the 'Reset separation' and 'Reset overlap' interactors. You can also revert to the default camera by pressing the 'Reset camera' interactor.
Now change the 'Window size' interactor to 'full-screen', clear the 'window decoration' toggle in the control panel, reset the server (using the Connections / Reset Server command from the Execute pulldown on the visual program editor - this is required in the alpha version but will not be required in the next version) and re-enter Execute On Change mode. You should get a full screen undecorated image of the object. Now press '3' (rather than 2) to enable stereo, to use the over/under system mode rather than the side by side mode. If you installed the correct environment variable commands to cause your system to go into stereo mode, it will.
In this mode, you will find that the key-press still work - hitting
'1' will revert the image to mono mode. However, the window will overlay
your control panels. To retrieve them, you can hit 'c' in the image window
to close it, and later pop it back up using the "Pop window up" interactor
that you will find in the control panel.
Sample Network Details
The APIsSeparate APIs are used to implement user-defined stereo system modes and stereo camera models. These are handled in a manner very similar to the User Interactor API in vanilla DX.
The set of available modes (or camera models) is specified as a table, where each table entry consists of pointers to functions implementing the particular mode. For stereo system modes, table entries consist of instances of the structure typedeffed as 'StereoSystemMode' in hwStereo.h. In this structure, the entry points are:
int InitializeStereoSystemMode(Display *dpy, Window frame)
InitializeStereoSystemMode is used to place the system into stereo mode. The X display and image window are passed in as parameters.
int CreateStereoWindows(Display *dpy, Window frame,
Given an X display and a frame window, CreateStereoWindows specifies windows to contain the left and right views and viewports into those windows into which the views should be rendered. The leftWindow and rightWindow may either be independent windows, created (using standard X calls) as child windows of the frame, or may be set to the frame window itself. leftWindowInfo and rightWindowInfo are pointers to structures that define viewports into the leftWindow and rightWindow. Thus, two separate windows may be created and returned to DX, with viewports specifying zero offsets and full sizes in each, or the frame window may be used by passing its window ID back as leftWindow and rightWindow, and using leftWindowInfo and rightWindowInfo to specify separate viewports.
int ExitStereoMode(Display *dpy, Window frame, Window leftWindow, Window rightWindow)
ExitStereoMode is used to exit stereo mode. It should delete the left and right windows if they were created in InitStereoMode, and reset the hardware into mono mode.
Specifying your own set of default stereo system modes is accomplished in one of two ways - either by linking in an implementation of a function that installs the system mode table, or by loading in a function which will be called at runtime to load in a table. The first method involves implementing the function:
DefaultStereoSystemModes(int *nModes, StereoSystemModes **modeTable)
which sets *nModes to the number of modes being installed and **modeTable to point to a table of modes that is either allocated or statically present in the file. Alternatively, you can implement a file which implements the equivalent function:
DXEntry(int *nModes, StereoSystemModes **modeTable)
You can then compile and link this into an object file. Then you cause DX to load it at runtime by adding the directory containing the run-time loadable file to the list held in the DXMODULES environment and set the DX_STEREO_SYSTEM_FILE environment variable to the actual name of the loadable file. Both of these methods are illustrated in the sysmodes.c, makefile.link and makefile.runtime that I've included in this package. NOTE: in each case, you must edit the makefile and set BASE to the directory point where you've installed DX.
Similarly, the set of available stereo camera models is specified by a table of structures of type StereoCameraModel, also defined in hwStereo.h. The entry points therein are:
void *InitializeStereoCameraMode(void *block, dxObject)
InitializeStereoCameraMode is used to manage a parameter block for the stereo mode. Whenever a window is placed into stereo mode, this routine is called and passed the stereo camera mode argument (the 'args' member of the group passed in as the 'stereo camera model' attribute on the renderable object). InitializeStereoCameraMode can then allocate a parameter block and process the argument object to derive parameter values to place into the block. This block is then passed to subsequent calls to the remaining functions of the mode. If the argument object to the stereo mode changes without actually changing the mode itself, the current argument block is passed in so that the parameter values it contains can be updated.
int ExitStereoCameraMode(void *block)
Implementations of ExitStereoCameraMode should free up any space allocated in the parameter block.
int CreateStereoCameras(void *block,
CreateStereoCameras is passed the parameter block along with camera information, and should returns the left and right camera parameters and a pointer to projection matrices for each. The
camera information includes the projection type (1 implies a perspective transformation and 0 an orthographic transformation), the field of view (for perspective transformation) and width (for orthographic tranlformation), the current camera look-from, look-to and up vectors and the distance to the near and far clipping planes. The projection model used is very similar to that discussed in the OpenGL manuals, with the exception that a left-hand coordinate system is used.
int MapStereoXY(void *block, Window frame, Window view, WindowInfo viewInfo, int vx, int vy, int *fx, int *fy)
Given an event location (eg. button- or key-press event) in the view window passed in as the view window and/or viewport passed in as the view and viewInfo parameters, transform the point (vx, vy) to the corresponding point in the single-window model. This allows DX components external to the rendering process to handle event locations transparently to stereo rendering.
Again similar to the handling of stereo system modes, user-defined stereo camera models can be installed either at link- or run-time by implementing:
DefaultStereoCameraModes(int *nModes, StereoCameraModes **modeTable)
DXEntry(int *nModes, StereoCameraModes **modeTable)
and is illustrated in mymodes.c and makefile that I've included. To
install runtime loadable stereo camera modes you must add the directory
containing the loadable file to the list of directories in the DXMODULES
environment variable and set environment variable DX_STEREO_CAMERA_FILE
to the actual name of the loadable file.
Installing Custom Stereo System and Camera ModesThe use of custom stereo system and camra files is illustrated in the src subdirectory. There you will find mymodes.c and mycams.c along with a makefile that should work on your system (if you are running on the system on which the samples were installed). In src, build mymodes and mycams, and then return to the parent directory. Set the DX_STEREO_CAMERA_FILE environment variable to src/mymodes and X_STEREO_SYSTEM_FILE to src/mycams. Rerun the example. Now when you place it in stereo mode by striking '2' in the image window you will see the alternative diagonal stereo pairs.
Now enter side-by-side stereo by hitting the '2' key. You'll see the
side-by-side images, and the pick sphere will appear in both. Zoom out.
Now, instead of picking on the object itself, you pick on the virtual 3D
object which, in this case, is centered in between the two images (to actually
see the virtual 3-D image you'll need to look at it cross-eyed, or use
a stereo-opticon of some sort). You therefore can right button in the center
and you'll see a sphere glyph appear in both eye views.