Game Engine Virtual Reality with CaveUT
Jeffrey Jacobson and Michael Lewis, University of Pittsburgh
CaveUT, an open source freeware project, uses game technology to make immersive projection-based virtual reality affordable and accessible.
simple, the current public release of CaveUT works well for low-cost
displays. The most advanced version, which will be publicly available
in the fall of 2005, supports real-time spatial tracking and
stereographic imaging. It is currently installed and working in the SAS-Cube, a CAVE-like display.
Computer games with the most advanced simulation and graphics usually employ a game engine, a commercially available software package that handles basic functions. For example, the first-person shooter Unreal Tournament
for the PC employs the Unreal Engine to provide richly detailed
graphics, high-speed processing performance, a built-in physics engine,
a scripting language interpreter, and robust networking for shared
CaveUT modifies Unreal Tournament
to let it display in multiscreen enclosures suitable for immersive
virtual reality applications. VR applications developed with CaveUT
inherit all the Unreal Engine's capabilities along with Unreal Tournament's authoring support, open source code, content library, and large user community.
ORIGIN AND DEVELOPMENT
In 1990, researchers developed the original Cave Automatic Virtual Environment,
a partial cube approximately three meters per side, with each wall
functioning as a rear-projection screen illuminated by a projector. A
mainframe drives all of the CAVE's projectors, displaying a contiguous
visual image across all screens to produce a virtual landscape.
Stereographic imaging makes the virtual objects look more
three-dimensional, while real-time spatial tracking lets users interact
with the objects and navigate the space.
the 1990s, researchers developed many VR applications for the CAVE and
similar displays. They either programmed these applications directly,
starting from OpenGL or a similar graphics library, or wrote them using
advanced authoring kits. Despite the process's difficulty and expense,
it usually produced applications with poor and often primitive
graphics, low performance, and limited networking functionality.
In 1997, Paul Radjlich produced a version of Quake for CAVE that inherited the game's authoring support, networking, and other features. Unfortunately, Cave-Quake could not benefit from Quake's game engine, which was PC-based.
2000, the game industry had driven significant advances in graphics
hardware for the PC, and CAVE owners began replacing their mainframes
with PC networks thanks to the PC's low cost and increasing graphics
power. Further, the leading first-person shooters, Quake and Unreal Tournament,
surpassed the traditional CAVE-based applications in graphics quality,
performance, animation, and networking. These games also had
respectable authoring support, built-in physics, partially open source
code, a large base of existing content, and an active developer
Despite these developments, the
games still lacked the ability to perform real-time shape generation
for scientific visualization applications. Their interface and
animation support also proved limiting for applications using a
Thus, in 2000 we decided to adapt Unreal Tournament
to the BNAVE, a PC-based CAVE-like display. With Michael Lewis's
support and guidance, Jeffrey Jacobson and Zimmy Hwang invested a year
of careful study, then solved the multiscreen display problem by
inserting just six lines into the game's open source code.
packaged this as the first version of CaveUT, which we made into a
freeware project to attract collaborators. Since then, CaveUT has
become far more capable and complex through independent code
Today, most software
development for CAVE still uses the traditional VR authoring tools,
which have improved considerably but remain expensive. However, the
game engines have maintained their list of advantages and increased
their lead in graphics quality and performance, thanks to massive
funding by a game industry that has grown larger than Hollywood.
Already, many developers in science and industry take advantage of game technology for other uses,
with developers basing more and more immersive VR applications on
games. Meanwhile, CaveUT has attracted a growing list of collaborators,
which should lengthen with the release of the project's stereographic
HOW IT WORKS
multiscreen display based on CaveUT requires a server computer that
connects by a standard LAN to several client computers. Each client
drives one screen of the display, usually a projection screen
illuminated by a digital projector.
1. V-Cave schematic. In this simple CaveUT installation, each screen is
turned 45 degrees. The Virtual Theater is similar, but with four
screens instead of two.
The operator begins a multiplayer game of Unreal Tournament
with one normal player on the server and one spectator player on each
of the clients. Each spectator's view duplicates the view seen by the
player on the server. On each client, the CaveUT code rotates the
view—the screen's window—into the virtual world so that each screen
shows its part of the composite view. Figure 1 shows the view of one
screen turned 45 degrees to the left, while the other screen has been
turned 45 degrees to the right.
perspective correction, CaveUT employs Willem de Jonge's VRGL, an
OpenGL library modified for VR applications. For an installation with
no head tracking, the user must specify a single ideal viewing location
for the whole display. As long as the user's head stays at or very near
this point, the view will remain unified and undistorted. If the
installation does have a head tracker, CaveUT can correct the
perspective in real time to effectively follow the user's movement.
arrangement uses an unmodified Unreal Engine, with its internal
functioning unaffected by CaveUT and VRGL. This lets CaveUT remain open
source and easily upgradable to each new engine version.
provides CaveUT's good performance by ensuring that each machine's copy
maintains a complete instance of the virtual world and performs all its
own graphics rendering, physics, animation, and related operations.
Client-server communication is confined to fast and simple state-change
updates such as, "The player is now here and is moving in this
direction at this speed."
support at least 32 independent view screens for a single application,
in any configuration, and with multiple real players. For example,
developers could configure a six-walled enclosure with all its views
centered on the first user, two more four-walled enclosures with views
that center on the second and third users, and single-desktop
arrangements for eight more users. All 11 users could share a single
virtual environment. Some could be students, others instructors, and
yet others acting as intelligent scenery. In this scenario, only
imagination and budget limit the possibilities.
monoscopic nontracking CaveUT display can be configured cheaply from
common office equipment and simple hardware. Front-projection screens
can consist of any clean, white surface, while rear-projection screens
can be made from any translucent material. Each screen requires one
common DLP or LCD projector driven by a standard personal computer with
a good video card and an installed copy of Unreal Tournament, which currently retails for around $20.
2. Virtual Theater, University of Pittsburgh's Information Sciences
Department. The user can manipulate virtual objects or navigate using
standard game peripherals.
additional computer hosts the server and acts as the operator console.
The server connects to the clients through an Ethernet switch and the
appropriate cables. CaveUT/VRGL is currently limited to OpenGL and the
Windows OS, although porting it to Linux/Unix/MAC-OSX would not be
difficult. Configuring the Virtual Theater shown in Figure 2 cost only
$25,000 and required no special programming.
CaveUT, the installation developer can upgrade steadily from an
inexpensive display to a fully capable CAVE-like interface. The display
supports spatial trackers useful for head-tracking and advanced
controls. For example, the user can manipulate virtual objects or
navigate largely by pointing with the tracked controller.
also supports stereographic imaging at a per-screen cost of two
computers with specialized video cards and a stereographic projector
for active stereo or a pair of monoscopic projectors for passive
stereo. Although less expensive than active stereo, passive stereo is
limited to displays that array the screens horizontally around the
viewer because the illusion falls apart if the user's head tilts. When
used in combination with a head tracker, active stereo allows
unrestricted viewing angles.
A CaveUT installation could be used to interact with most content written for Unreal Tournament
and most applications built on the game. The highly localized code
changes that CaveUT introduces are unlikely to conflict with the UT-based application's code changes. This is an important advantage because the large community of Unreal Tournament
gamers and researchers produces a great deal of artistic content,
animation, and code modifications for the game engine. CaveUT
developers can use most of this material, in pieces or in whole
applications, and benefit from the Unreal development community's support and cooperation.
Several projects already use CaveUT, including the following:
- The ALTERNE project's artists use CaveUT for interactive art installations, storytelling, and information visualization.
- Researchers at the University of Pittsburgh's Visual Information Sciences Center use
CaveUT to prototype systems for first-responder emergency training,
virtual museums, way-finding applications, and architectural planning.
- Researchers at the University of Pittsburgh's Usability Lab (ULAB) are developing CaveUT projects for robotic simulations and virtual archeology.
- Stagecraft designers at the University of Southern California's Institute for Creative Technologies use CaveUT for interface prototyping.
- A low-cost portable CaveUT display has been demonstrated at conferences such as CHI 2002, HFES 2002, VR 2003, and I3D 2003.
- The artists at Elumenati are helping to develop CaveUT's close cousin, DomeUT, for applications in all-digital dome displays.
- Military researchers are using CaveUT as an immersive interface for their Unreal-Engine-based training simulators.
list will continue to expand because CaveUT's low initial cost makes it
accessible and attractive to students, educators, artists, developers,
gamers, and small businesses working in a wide range of disciplines.
with the powers of the Unreal Engine, CaveUT also inherits its biases.
The engine can support large virtual environments, but it works best
with relatively small worlds that are rich in detail and activity. For
example, the engine can support a dozen or so amazingly detailed and
lifelike humanoid agents more efficiently than a large number of simple
ones. All game engines work with pre-defined objects and cannot use a
data stream to continuously generate visual effects. However, they can
be used to display effects transmitted to the engine from other
software or generated from data in advance. CaveUT also inherits the
advantages and limitations of Unreal Tournament, but it could
easily be adapted to any game based on the Unreal Engine. Similarly,
CaveUT could be adapted for other graphics systems to provide more
options for the developer.
CaveUT now supports a wide range of applications. It continues to
develop through contributions of all kinds, such as the following:
Le Renard and Jean Lugrin will release their stereographic extension
for CaveUT this fall, and Le Renard will also release his spatial
tracking for CaveUT then.
Studios is adapting VRGL—and therefore DomeUT/ CaveUT—for use with
multiprojector curved screen and partial-sphere displays.
- Ivor Diosi is working on a method for streaming video from an external source onto a surface in the virtual environment.
- Jacobson and Demiurge are developing optimal ways to use commercially available game peripherals in an immersive display.
these new features will require documentation, tutorials, testing, and
extension to a variety of platforms, especially the low-cost ones. We
always welcome new users and collaborators as we build up the CaveUT
Jeffrey Jacobson is a graduate student and Michael Lewis is an associate professor, both in the University of Pittsburgh's Information Sciences Department. Contact Jacobson at email@example.com and Lewis at firstname.lastname@example.org.