Virtual Reality and Natural Eye Movement

First there were René Descartes thought experiments, then Tron’s light cycles and Videodrome.  Later, we had Star Trek’s holodeck, The Lawnmower Man, The Matrix, and Existenze.  The idea of virtual reality is nothing new.  I’ve certainly been fantasizing about it for just about as long as I can remember.  However, there hasn’t been much connection between the culture of fantastic virtual experiences and video games.  Sure, there have been crossover arcade experiments like this one, hippie programmers like this virtual reality pioneer, multiple kilobuck head mounted displays for your PC, and we won’t even mention this little gem, which had my adolescent head spinning with glee for months preceding its craptacular release.  However, I will mention this famous Nintendo Hoax video, which actually had me losing sleep pondering its many possibilities.

So why aren’t video games virtual reality yet?  One of the most obvious reasons is immersion.  We are still sitting placidly on our respective couches, eyes planted on a small viewing area and hands wrapped around a blob of plastic covered in protruding bumps and switches.  The problem has nothing to do with lack of graphics horsepower.  A cheap Nintendo 64, Dreamcast, or PS1 had plenty of computational cycles to provide a virtual environment.  I would argue that the problem is the way we use our eyes, or more accurately, the way we think we use our eyes.

Understanding how our eyes actually work is a challenging proposition because it falls into the realm of the automatic chores our brain takes care of without so much as a drip of conscious thought.  Our ocular functions are much like breathing or a heartbeat in this way.  However, scientific study can and does provide some of the answers we seek.

Ask a person how they see, and they will most likely respond with, “I see everything,” if they don’t just look at you funny and continue on with their day.  We imagine that our eyes are fantastic visual recorders which absorb the visual world with a stupendous bandwidth.  Our certainty in the trust of our optical nerves is soundly demonstrated in the breadth of our astonishment when our visual fallacies are presented to us through carefully designed experiments.

One of the common experiments used to demonstrate the way we use our eyes involves the playing of a recorded basketball practice session to a group of volunteers.  The volunteers are told to count the number of times the ball is passed from player to player.  Upon completion of the movie, participants are asked if they noticed anything strange about the presentation.  A very small percentage of the audience will respond with a yes, pointing out the appearance of a man dressed in a gorilla costume bounding through the scene.  The remaining majority of the volunteers are very good at counting the number of ball passes, but are completely astounded at not having noticed the gorilla.

The point of this experiment and others is to demonstrate that our eyes and ocular systems are only capable of focusing on very small areas, and after having this fact demonstrated to you; it is possible to observe yourself doing it.  When I actually concentrate on what my eyes are doing as I walk into a room I’ve never been in before, I notice that my eyes are in fact making many small movements, building an extremely rough representation of the surrounding individuals and objects.

So how does this all apply to video games?  A large percentage of the people in my generation have grown up with video games, and probably have a good portion of their neural/ocular systems mapped to the arm and wrist they use for “mouselook” in FPS games.  However, we are also trained to focus in on very small viewing areas, eyes locked on our televisions and VGA monitors for hours at a time.  This is all well and good, but provides a clear separation between the way we use our eyes to perceive video games (not very much natural eye movement) and the way we perceive real scenes (lots of natural eye movement).  This distinction represents a huge barrier to immersion.

After thinking this through carefully I decided to try out my own anecdotal and decidedly unscientific experiment using my ten foot widescreen projected display, and the Xbox 360 game known as Oblivion.  The hypothesis of my experiment was a simple one.  I predicted that if I sat close enough to my display screen to completely envelope my peripheral vision with rendered game world, my eyes would begin to move more naturally instead of tightly focusing like they did on my much smaller VGA monitor.  This distance turned out to be around four feet from my screen.

The first thing I noticed was that the walking speed in Oblivion was way too high for my experiment to work.  Even in sneak mode, the virtual world moving past me was an unidentifiable blur.  Then, inspiration!  Upon walking into the Mage Guild of Chorrol, suddenly and without warning, I was walking into a real building.  The immersion melted into my consciousness, smooth as butter.  My eyes darted naturally from object to object, noting the faces of my fellow guild mates then wandering on to the fully stocked and set banquet table in the main meeting room.  My stomach growled in appreciation of the spread of food items.  The key that triggered my epiphany?  I was now moving through the game world at about half of Oblivion’s sneak speed, using the analog walking speed control in a way that had never occurred to me before.

After my experiment, I expanded my hypothesis further to include the speed of video games as well as natural eye movement.  Traditional first person video games move at a fantastic speed compared to a natural human gate, with the standard excuse that anything slower would quickly bore the player into a coma.  I was no exception to this excuse.  I’d even complained on several occasions that Oblivion’s running speed was too slow.  However, the fantastic speed at which we play our video games encourages our brains to think very symbolically.  For example, when we are tearing by the intricately detailed and bepiped corridors of Unreal Tournament 2k4, for instance, our brains are thinking . . . (hallway) (left turn) (redeemer).  There simply isn’t any time for anything else.  The speed issue became even more apparent to me at last weekend’s LAN party where we played the old school Quake 3, and I made the now obvious observation that the substantial difference in detail level between Unreal 2k4 and Quake 3 added up to exactly zero in overall effect.

I feel that the repercussion of my simple experiment is the realization that the pursuit of virtual reality isn’t as far away as we might think.  In fact, it may be as close as a cheap projected display and a dark room.  Game developers might even take heart by this, start swimming upstream and reduce the walking speed in their first person games.  Why is this good news for game developers?  Because slower moving players means both that your game will require less art to create a cohesive world, while simultaneously creating a greater appreciation for the art in the players.  Of course, thanks to analog walking speed control in most of today’s console games, I don’t have to wait for the developers.  While some may continue to sprint through Oblivion at 25+ scale MPH, I’ll be the one slowly strolling the Imperial City streets, admiring the fantastic architecture in a fully realized world worthy of Descartes, Tron, or the Holodeck.