Next Generation Immersive Visualization
Frederic I. Parke, Ph.D.
Visualization Sciences Program, Texas A&M University
College Station, Texas, USA, 77843-3137
This paper describes the development of a "next generation" high performance, flexible, relatively inexpensive, commodity based, spatially immersive visualization environment.
Current and near-future technologies and computational economics allow the development of better and more cost effective spatially immersive visualization systems. The systems being developed focuses on configurations utilizing a large number of identical modular components. For the most part these components are off the shelf, relatively inexpensive, commodity items.
KEYWORDS Spatial Immersive Environments, Fully Immersive Visualization, Virtual Reality
Today, immersive virual environment systems fall into two categories - head mounted displays and spatially immersive displays (SIDs) such as the CAVE developed at the University of Illinois at Chicago . Head mounted displays (HUDs) are designed to present two separate synthetic views of the virtual environment to the user. One view is for the right eye and the other view is for the left eye. These two stereo views are fused by the viewer's perceptual system in the same way as right and left eye views of natural scenes. HUD systems must, in addition to computing the synthetic displays, track the viewer's location and viewing orientation in the virtual world to create the correct right and left eye views.
In SIDs, such as CAVE installations, the right and left head mounted displays are replaced by multiple projected displays that form the walls of an environment cube. Current CAVEs have at most six planar surfaces, each representing a large portion of the possible field of view. Most CAVE installations have fewer than the maximum six display surfaces. Each of the projection surfaces is usually driven from an expensive, high performance graphics system such as SGI Onyx2 with one or more Infinite RealityTM graphics processors. Figure 1 illustrates the CAVE approach using three display surfaces and three image projectors. Figure 2 shows adding a fourth floor surface and a fourth projector utilizing a relecting mirror. Figure 3 shows a CAVE configuration with all six display surfaces.
In spatially immersive systems, the projected displays are usually presented as time sequential stereo images. The user is required to wear special glasses that have liquid crystal shutters whose operation is synchoronized to the diplay frame rate. As in the head mounted display version, the position and perhaps oreintation of the viewer must be tracked  if correct stereo views are to be projected on the CAVE walls.
An alternative to the active, time sequential, stereo presenation is a passive diplay approach that uses two projectors for each diplay screen. Polarized filters are placed in front of each projector. These filters are oriented at 90 degree angles. The viewer also wears
Figure 1 - A 3-sided CAVE
Figure 2 - A 4-sided CAVE
Figue 3 - A 6-sided CAVE
glasses with polarized lenses. These lenses are also oriented at 90 degree angles so that the right eye sees only the right eye projected image while the left eye sees only the left eye image.
While fully immersive visualization facilities are still relatively rare, they are becoming key facilitators for many research and industrial projects. This paper describes the development of "next generation" high performance, flexible, relatively inexpensive, commodity based, spatially immersive visualization systems.
THE IDEAL SYSTEM
It can be argued that the ideal spatially immersive environment would be one where the user is surrounded by a seamless spherical display surface that provides very high resolution, high update rate, 360 degree panoramic stereo views of extremely high complexity data. The current CAVE immersive environments are poor approximations to this ideal. Domed spatially immersive environments have been used for many years in flight training simulators  and dodecahedron approximations to spherical projections have been developed .
Current and near future technologies and computational economics allow the development of better and more cost effective spatially immersive visualization systems. The systems we are developing focus on configurations utilizing a large number of identical modular components. For the most part these components are off the shelf, relatively inexpensive, commodity items.
A spatially immersive visualization system consists of three major elements - 1) the computational infrastructure, 2) the surrounding display surfaces, and 3) the viewer tracking and interaction elements. We are exploring new approaches to both the computational infrastructure and the display surface geometries to be used. We have not focused on the viewer tracking and interactive elements and expect to use the approaches in current practice.
In our new approach, the computational infrastructure used is a visual computing extension of the Beowulf concept . A Beowulf system consists of a collection of commodity computers networked to form an inexpensive but powerful distributed parallel computing engine. The Beowulf concept generally makes use of extentions to the Linux  operating system.
The visual computing extension is to add a high performance commodity graphics processor to each of the computational nodes. The result is a powerful parallel distributed visualization system. Based on published performance benchmark results, collections of relatively low cost commodity visual systems compare very favorably in both cost and aggregate performance with the expensive high-end graphics systems typically used to support immersive systems . For example, an entry level SGI Onyx2 Infinite RealityTM system with an approximate cost of $170,000 has a measured DX benchmark performance of about 36. A high-end Pentium III based workstation costing about $13,700 has a DX performance of about 25. The cost ratio is about 12 to 1 while the performance ratio is only about 3 to 2, yielding a 8 to 1 cost perfomance advantage for the commodity workstation. While one must be cautious when using such single measure comparisons, the trend is clear.
Having the visual computing distributed over a collection of processors allows innovation in the structure of the display surfaces. In our approach, the aggregate display surface is composed of many display faces or facets. In such configurations, each facet need only display a relatively small portion of the total virtual environment. The graphics computation needed for each facet will fall within the capacity of today's, and certainly tomorrow's, high-end commodity graphics systems.
The envisioned faceted display elements could be arranged in a number of configurations. In fact, the display configurations could be tailored to meet specific applications. There are a number of polyhedral configurations whose faceted surfaces are good approximations to the ideal spherical display surface. Several of these are formed using many instances of only a single planar shape. These polyhedra require from 12 up to 60 or more planar faces . Among these is the 24 facet Trapezoidal Icositetrahedra illustrated below. In addition to the solid form, Figure 4 shows the 24 identical facets unfolded onto a plane.
Figure 4 A 24 facet Trapezoidal Icositetrahedra
(also known as a Deltoidal Icositetrahedra)
Figure 5 shows the array of projectors needed for a display environment using the Trapezoidal Icositetrahedra form. Compare this figure with Figures 1, 2, and 3 above. Each of these projectors would be driven from its own graphics computational node. The set of these 24 computational nodes would form a visually extended Beowulf system as discussed above.
Figure 5 - A 24 facet immersive environment with projector positions indicated.
Figure 6 shows an illustration of how such a system might be located within a high ceiling building. The human figure show the scale of this 15 foot diameter display structure. The building entrance shown is approximately 18 feet wide.
Figure 6 How a 24-facet immersive display structure might be located within a high ceiling room. The entrance shown is about 18 feet wide.
Figure 7 shows a cross-section through the 24 facet structure. The human figure is included to give scale to the illustration and to show where the viewer would be located relative to the display surfaces. A set of simulated molecular images is shown projected on the display surfaces.
Figure 7 A simulated cross-sectional view of a 15 foot diameter 24 facet immersive display environment.
We are focused on exploring and evaluating this new class of spatially immersive visualization systems. We are concerned with determining whether these new visualization environments are practical and effective.
The objectives in this research can be summarized as follow:
1) Explore and evaluate, at the conceptual level, possible geometric display structures and their implications for next generation spatially immersive environments.
2) Develop and evaluate software simulations of some of the more promising configurations identified in the conceptual phase.
3) Construct and evaluate operational prototype systems for the most promising geometric configurations.
4) Develop the software need to support simulated and operational prototype systems. Some of the issues to be addresses include; effective distribution of the graphic computation, dynamic data partitioning, synchronization of the displays, the required projection and image clipping algorithms, and automated display system calibration.
5) Develop a deeper understanding of the technical and effectiveness issues and trade-offs of these systems. How are technical considerations and user effectiveness related to geometric configuration?
6) Investigate the feasibility of mass replicated, modular, implementations of these systems.
7) Develop experimental designs to be used to evaluate prototype systems and experimentally measure the effectiveness of the systems.
We are currently in the conceptual development and initial simulation phases of this project. Simulation and small-scale prototypes will be used to verify conceptual results prior to committing to the construction of full-scale prototype designs.
Selection and development of the prototypes involves detailed simulation, physical structure design, supporting software design, physical fabrication and assembly, and software implementation.
Prototype evaluation will include both a technical evaluation of the system and an effectiveness evaluation. The technical evaluation will address issues such as computational complexity, computational loading, dynamic performance, and cost/performance.
In addition to the design and development of prototypes systems of this new class, we are concerned with the effectiveness of the immerse experiences they will provide. That is, do they empower the participant by affording new insights and deeper understandings, as well as by facilitating performance generally?
It remains the case that relatively little published research has attempted to characterize either the immersive experience in a simulated environment or establish whether an enhanced sense of immersion results in either improved or less effortful task performance .
Our approach to this evaluation will be to measure the concomitant cognitive, affective and physiological processes occurring during and after immersive experiences. Physiological activity such as heart rate and skin conductance have proven to be useful indicators of effort and attention in complex settings , and the electromyographic measurement of facial muscle activity has proven to be a robust indicator of affective processes . Continuous response measurement and secondary reaction time techniques have also proven to be useful measures of such ongoing psychological processes. Traditional measures of task performance, such as error rates, as well as somewhat novel measures of such performance, such as standardized performance trajectories, along with measures of the encoding, recall or recognition of information, will also be explored to assess the cognitive concomitants of immersive experiences.
We expect that this work will result in:
1) The development of an improved class of spatially immersive environments, including the construction and evaluation of several polyhedral prototypes utilizing low cost commodity components.
2) Specialized graphics software to support these prototypes.
3) System software to manage graphics data distribution, coordinated operation of many graphic computation nodes, display synchronization, and user interaction.
4) A much deeper understanding of the issues related to polyhedral immersive virtual environments.
5) Suggested designs for modular mass replicated immersive environments
This work has the potential to fundamentally influence the economics, availability, and pervasive-ness of spatially immersive environments. It has the potential to influence the design, development, and viability of future spatially immersive systems. It will contribute to the availability of these environments across a wide range of users in many disciplines.
In addition to the 24 facet polyhedra used above to illustrate the basic concepts, there are many other polyhedra formed from larger numbers of identical planar shapes. Two of these with 60 facets each are shown in Figure 8. These, and polyhedra with even larger numbers of facets, could be the basis for future spatially immersive systems. In fact, the larger the number of facets, the larger the number of computational nodes used. This increases the overall aggregate visual computational power of the system. It also decreases the size of each display facet for an environment of given size. More facets also result in a better approximation to the ideal spherical environment.
Figure 8 Two 60 facet polyhedra -the Deltoidal Hexecontahedron on the left, the Pentagonal Hexecontahedron on the right.
Faceted display configurations hold the promise of truly modular systems where the immersive environment is created by literally bolting together mass replicated modules. Each module containing the required structural, computational, and display elements. The display elements of these modules could possibly eventually be flat panel displays similar to those currently used in laptop computers.
 Cruz-Neira, C., Sandin, D, DeFanti, T., Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE, Proc. Siggraph 93, Aug. 1993, 59-66.
 Polhemus Navigation Sciences, 3Space Isotrack Users Manual, Colchester, VT, 1987.
 Reno, B. A., Full Field of View Dome Display System, Proc. AIAA/FSTC, 1989, 390-394.
 McCutchen, D., Method and apparatus for dodecahedral imaging system, U.S. Patent #5,023,725, June 11, 1991.
 Sterling, T. L., J. Salmon, D. Becker, D. Savarese, How to Build a Beowulf, Cambridge, MA, MIT Press, 1999.
 Hekman, J. and Staff, Linux in A Nutshell, Cambridge, MA, O'Reilly & Associates, 1997.
 Viewperf graphics performance benchmark results, available at http://www.specbench.org/gpc
 Holden, A., Shapes, Space and Symmetry, New York, Dover, 1971.
 Wenninger, M. J., Polyhedron Models, London, Cambridge University Press, 1971.
 Nemire, K. & Ellis, S.R., Calibration and evaluation of virtual environment displays. Proc. IEEE Symposium on Research Frontiers in Virtual Reality, 1993, 33-40.
 Lang, A., Ed., Measuring Psychological Responses to Media, Hillsdale, NJ, Lawrence Erlbaum Assoc., 1994.
 Tassinarry, L.G. & Cacioppo, J.T., Unobservable facial actions and emotion. Psychological Science, 3, 1992, 28-33.
Figures 5, 6 and 7 were created by Christina Garcia at the Texas A&M Visualization Laboratory.