Toward Ultimate Reality - Electronic LSD and Virtual Displays
Abstract: This study explores various uses of virtual reality goggles in conjunction with consumer level items: color cameras, video effects, luminance keying and videotape, in an effort to ascertain what is feasible at the present time regarding consumer and industrial applications. As a measure toward eventual home theater display prospects, the viability of keying video into a see-through head-mounted display to form a virtual, wall-sized television will be determined. Another portion of the research will attempt to communicate aspects of the psychedelic experience into electronic form.
“When we apply the power of networking to the VR world, the potential for stretching the limits of human abilities becomes very powerful.” —Jaron Lanier (1)
INTRODUCTION
Virtual Reality (VR), its “newer” cousin Augmented Reality (AR), Enhanced Reality (ER), Simulated Environments (SE), Myron Krueger’s Artificial Reality (2) and other terms each attempt to describe unique sets of experiences and phenomenon that promise to radically alter the process of human communication. It is my belief that these various disciplines will continue evolving parallel with related technologies until a critical mass is achieved, resulting in a coalescence of unprecedented magnitude, which I have termed Ultimate Reality, in deference to Ivan Sutherland’s pioneering efforts described in The Ultimate Display. (3)
Mr. Sutherland’s first head-mounted display (HMD) was in fact a see-through design.(4) Through clever use of half-bright mirrors, a video display was mapped over the user’s field of vision. This was the first implementation of what is now known as Augmented Reality. I contend that this original incarnation will ultimately prove to be more useful to mankind.
1 - Byte, Vol. 22, No. 8, P. 32 “News and Views: Interviews.” (1997).
2 - While Mr. Krueger’s phrasing seems best suited to describe his own unique field of study into “other”environments, I fully believe his early research will prove instrumental in the coming technological convergence, particularly in the areas of telepresence and in making computers “aware” of human beings. Not only that, he constantly reminds us that such experiences should be enjoyable, even fun. This humanizing factor deserves far more attention than it currently receives.
3 - Sutherland, Ivan. "The Ultimate Display." Information Processing 1965: Proceedings of IFIP Congress 65, 2 (New York, May 24-29, 1965): 508.
4 - Pimentel, Ken and Teixeira, Kevin. (1995) Virtual Reality: Through The New Looking Glass. P. 44
Few fields have shown as much promise, while failing to deliver in the way of actual product, as Virtual Reality, thus thwarting the potential for rapid advances in human communication. While proving to be invaluable in the fields of science, medicine, and computing, what was once seen as a marketplace panacea has proven to be a commercial black hole. Various attempts to penetrate the U.S. consumer market have inevitably resulted in withdrawals and bankruptcies. From the fall of the pioneering VPL Research (5) to the 1997 bankruptcy filing of Virtual I-O, (6) VR has failed to attain a sizable niche in the marketplace beyond industrial and medical applications. Despite VR’s inability to gain a foothold in the consumer realm, the industry is still estimated to gross $1,000,000,000 in 1998 alone. (7)
There has yet to be a compelling use for VR/AR in daily life. Games apparently are not driving factors in moving this technology into homes. In true paradoxical fashion,we won’t have better displays until there is a large demand, and we won’t have a demand until we have better displays. Japan, however, has a healthy market for VR, and relies heavily on American research and development, primarily importing foreign gear for various consumer uses. As far back as 1993, the National Trade Data Bank Market Report painted a bright future for VR in Japan, somewhere in the neighborhood of $174-$435 million in projected sales and revenue from VR related fields, entirely within the realm of entertainment. (8)
It could be that the Japanese are culturally more ready for artificial reality experiences because of their preferred television viewing distance being half that of Western preferences. In other words, there is less reluctance to having a display near the eyes. While VR theme parks are a burgeoning industry in Japan, they have not caught on well in the United States,
5 - Rifken, Glenn. (1996) “Computerworld Inner Visions”. P. 1 (November 11, 1996) (September 12, 1998) www.elibrary.com Computerworld.
6 - Warren Publishing, Inc. Consumer Electronics (May 5, 1997) (August 13, 1998)
7 - Infonautics Corporation. “Virtual Reality Market to Reach Almost Billion Dollars in 1998”.www.elibrary.comPress Release
8 - The National Trade Data Bank Market Report. (Japan: 1993)
with few exceptions (Virtuality’s early success with Dactyl Nightmare and Chicago’s BattleTech center.) (9)
In the 1980’s, time base correctors (TBCs) (10) averaged a minimum cost of about $5000 each. With the advent of desktop video, in particular, Newtek’s Video Toaster, the demand for TBCs increased to such a level as to lower prices to the extent that today you can you can easily equip a computer with four TBCs, not to mention a video switcher, for less than $5000. If industry, by way of consumers, creates a demand for luminance and chrominance-based solutions, this pricing shift could occur with AR systems at a much higher rate, resulting in a product that remains affordable, but offers high quality images.
The ultimate artificial reality experience would be a direct mind link. Scientists are now able to view brainwave activity patterns as they occur. (11)
As the neuromagnetometers that perform this task become more sensitive, and the computers that interpret the data more sophisticated, it will be possible to think “apple,” and cause the representation of an apple to appear in virtual or augmented reality. Thus, I posit that it will one day be possible to record actual thoughts and dreams. It is at this point that things begin to get interesting, in the sense that we will be able to create other realities through our own thoughts.
Stereolithography is the process whereby 3-D objects are made into physical objects through the use of “object printers,” machines that convert the 3D data into solid objects by treating a volume of resin with lasers to harden it into specific shapes. (12) The future promises us a world in which solid objects can be transmitted via a 3D “fax machine,” eminently useful in design and prototyping work. Coupling this process with
9 - I find it interesting to see Commodore’s Amiga crop up repeatedly in this research. Both Dactyl Nightmare and BattleTech used Amiga 3000’s for image generation and control. Sense8’s first demo to Sun was also Amiga-based.
10 - Time base correctors correct the timing errors of videotape transports to provide a stable signal for switching and editing purposes.
11 - Berger, Bob “Mapping the Mindfields” Omni
12 - Pimentel, Ken and Teixeira, Kevin. (1995) Virtual Reality: Through The New Looking Glass. P. 269
MRI “mind-reading” could result in the ability of humankind to create solid objects out of pure thought. While not exactly “deus ex machina,” (13) it is a seductive prospect.
The technology that will tie these many disparate processes together into a unified whole comes from both academia and Hollywood. Engineers seeking to increase flexibility in film editing have developed a process whereby each element in a frame of film is digitized separately and is thereafter treated as a fully editable object. The coming technological convergence will utilize such processes, operating at faster than realtime speeds, to provide an experience that is visually indistinguishable from actual reality.
Literally, anything will be possible. The real and “other” worlds will begin to blur together, and form a synthesis that I refer to as Ultimate Reality. With ultra high-resolution cameras, a full field of vision (180 degrees) and a very high-resolution display (at least 9600 x 9600 pixel resolution), combined with accurate tracking and motion prediction, the interface will become transparent. It will then be possible to trick the human mind into believing that the virtual is actual. Indeed, it will become very difficult to distinguish between the two. In a realm worthy of Disney, broomsticks will dance across the room, you will converse with Alice in the tea room, and have the ability to instantly transport yourself to another point in time or space, whether across the country, or to a place that exists only in the ether. If the medium is indeed the message, we’re in for a wild ride.
LITERATURE REVIEW
While there is a wealth of information about Virtual Reality on library shelves,Augmented Reality has scarcely been given any attention there. It would seem the speed and immediacy of the World Wide Web has outpaced that of library database maintenance. The best data on the subject comes from the individuals who are developing
13 - God from a machine.
these technologies on a daily basis. The Web offers a close cadre of AR researchers making their findings available via University servers.
Virtual Reality - Scientific and Technological Challenges, from the National Research Council, is an in-depth examination of a wealth of issues related to the topic at hand. A weighty and technical tome, it is not directed towards the layman. Their recommendations, particularly in the area of promoting networked simulated reality development. (14) are heartening. They seek to actively encourage research and development in the area of networked VR applications.
A look at current trade journals indicates a growing interest in AR applications. E-Library retrieved some 20 popular press articles regarding augmented reality applications. Various sources for background include from The L.A. Times, and many trade journals in the field of video and 3-D animation. Metacrawler and Dogpile are meta-search engines, returning results from all of the other major search engines.
Best results were obtained with the University of Southwestern Louisiana’s licensed site access via the library’s homepage. A few academic search engines worthy of mention includes Lexis-Nexis, which didn’t have as many article references as I would have expected, given the sites weighty credentials. Web of Science is a research paper archive that provided a great deal of in-depth studies.
In 1995, Michihiro Uenohara and Takeo Kanade achieved real-time registration in AR at 30 hz via a method of computer vision (15) These researchers, as well as the scientists at the University of North Carolina at Chapel Hill, have initiated invaluable studies regarding the medical applications of augmented reality. X-ray vision is at hand, as physicians will one day soon be enable to peer within your various internal organs and
14 - National Research Council. Durlach, Nathaniel I. and Mavor, Anne S. (1995). Virtual Reality: Scientific and Technological Challenges
15 - M. Uenohara, T. Kanade, (1995). “Real-Time Vision Based Object Registration for Image Overlay”, Submitted to the
CVRMed '95, Nice, France.
systems, with the ability to view specific aspects such as the circulatory or respiratory systems alone. (16)
One book I found invaluable was The Virtual Reality Casebook, a collection of writings from a number of theorists, engineers, artists and other Artificial Reality denizens. Diverse and well rounded, the text was possibly the single-most consulted book in my research. It gives keen insight into the minds of the researchers and theorists that move the state of the art forward.
Despite a slight amount of product placement (the book is published in part by microprocessor giant Intel), Virtual Reality - Through the New Looking Glass (2nd edition), is a great starting point for anyone interested in the wide world of alternate realities. In all fairness, the authors are conscious of the possibility of perceived bias, and seem to go out of their way to be inclusive of the role of Apple’s Macintosh and even Commodore’s Amiga in the development of these technologies.
Virtual Reality Applications, edited by R.A. Earnshaw, J.A. Vince, and H. Jones,all from the U.K., also provides a well rounded perspective on the field, in a detailed manner that goes well beyond most American VR textbooks.
Myron Krueger, speaking of idealized hardware to come and the necessity of real-time interaction in an interview with Jas. Morgan, said: “Ultimately, a low-cost head-mounted display with the resolution of an OmniMax theater will be irresistible—if it’s as unencumbering as VIDEOPLACE. (17) It is my expectation that the two approaches will merge. The lightweight goggles will fit within ordinary eyeglasses. They will superimpose graphics on the real world. They won’t cut you off from your colleagues—you’ll be able to make eye contact with them.”
This is the crucial communication element that is missing from VR and,ironically, one that many people are currently working at adding to the experience. Until
16 -Virtual Reality Internet Resource Archive
17 - VIDEOPLACE is Krueger’s project that placed the participant’s silhouettes into a video display that allowed users to interact with each other, as well as with the computer-generated environment they inhabited.
we reach the point of networked VR experiences, immersion equals isolation. Will we come full circle and try to accurately represent our facial expressions using models? If the purpose is realism, then it becomes logical, and more efficient, to use one’s actual face via video. As video manipulation becomes more prevalent, we will gain the ability to alter our appearance slightly or dramatically, and do it in a photo-real environment.In a more abstract or representational framework, akin to today’s chat, non-realistic presentations of the self can be advantageous.
I expect in the interim to see polygon-based avatar chat capturing the subtleties of facial expression available in the next few years. In the same way object scanners have slowly migrated down to the desktop, technology that is now used for producing real-time cartoon actors for Saturday morning will be integrated into the coming networked virtual reality experience. The levels of interaction will be such that the future of entertainment as we know it may be headed the way of the drive-in movie. In the same respect, I fully believe that the future of mass and interpersonal communication will make television and the Internet seem like radio and black and white movies in comparison.
Augmented reality is approached in two diverse fashions: a digital display superimposed over clear glass, allowing a literally see-through display, and by use of various combinations of video cameras and HMDs. Each has proven to be tremendously empowering to the individual, and to a similar extent, the corporate entity. NASA’s work in correcting corneal defects via AR is a good example of what advantages the individual may gain. Telescopic sight, night vision, microscopic vision,and even more specialized abilities will eventually be commonplace, and currently exist on the market. How long before the technology becomes available in implant form?
Already, there is a growing gap between the computer literate and nonliterate. As technology increases exponentially, so increases the disparity between the rich and poor. Are we to become superhuman, able to exist in and see worlds the less fortunate cannot fathom? It would make the idea of “class warfare” outlandishly one-sided.
Boeing Corporation in particular has been the first major corporation to wholeheartedly embrace the concept of augmented reality, applying it to the field of aircraft production, which is imaginably a complex process. AR allows the Boeing worker to consult a vast array of manuals and documentation without having to leave the work area, or indeed, stop what he or she is doing at the time. The fuselage of the plane is located and tracked by the computer, allowing for the superimposition of construction details, such as the type that can tell a mechanic where to drill a hole, or an electrician how to wire a section. (18)
Idealized AR/VR is transparent, seamless. Integrating the power of computers with real-world displays presents us with a future reality in which all the world’s knowledge will be available to each of us, and delivered in such a fashion as to integrate into our lives as painlessly as a new cable box. Moore’s law being what it is, (19) the day of the “VR Man” (20) will soon be upon us. Are you prepared for the cultural shifts ahead?
Already, your computer can be considered an extension of your brain, or your self. Ongoing advances will soon empower you to actually augment your own mind through wearable computing. Do you think the Internet is useful? Can you imagine having it on a contact lens?
While the advantages seem obvious, the pitfalls remain hidden. Imagine if you will an age when such devices become ubiquitous as wristwatches, or at least laptop computers. The possibility that your very location could be tracked at all times has alarming implications, as does the concept of torture induced via AR/VR pseudo
18 - Pimentel, Ken and Teixeira, Kevin. (1995) Virtual Reality: Through The New Looking Glass. P. 260
19 - Moore’s law tells us that transitory capacity doubles in complexity every 2-3 years.
20 - The AR/VR equivalent of the Sony Walkman.
hallucinations. This type of application, perhaps administered along with conventional mind control techniques (drugs, sleep deprivation, etc.) could very well be the sort of reprogrammer sought by the CIA project group MK-Ultra, (21) and the “virtual torture” ominously predicted by George Orwell in 1984. (22)
Another fact that we must resign ourselves to is that VR and AR offer as much to the military and police as they do to industry, science, and consumers. The earliest research into telepresence, as in driving a vehicle remotely, began in the 1940’s. Remote viewing and telepresence will be one day result in persons being killed via a tank operated by a crew who remain removed from the actual battlefield. Janez Strehovec;characterizes an experience such as viewing the impact of a laser-guided, camera-equipped missile as “A paradigm of genuine techno-druggedness, caused by the identification with smart eyes’ view, and obsession with the success of a seductive action,fascinating also to the senses.” (23)
Even when we examine alternate realities in less metaphysical detail, a number of inevitable problems arise. The potential for not only viewing material others may deem offensive for a variety of reasons, but to actually become immersed in, will no doubt one day cause a furor, much in the way rock and roll, horror movies, and the Internet have in the past. The potential to now record experiences for others to view is a vast frontier for future friction between groups who would presumably restrict what others may experience immersively. Clearly another existential argument, ironically derived from what is in its simplest form, a hopped-up television set.
21 - Marchetti, Victor, John D. Marks The CIA and the Cult of Intelligence. (1984), Marks, John D. Search for the Manchurian Candidate. (1988).
22 - Orwell, George. 1984. (1949).
23 - “Vertigo--on purpose: Entertainment in simulators.” (1997). Journal of Popular Culture. Vol. 31 Issue 1, p.199
The exciting notion that William Gibson’s Cyberspace (24) is here, or around the corner, is one that I find to be demonstrably true. The science of data visualization allows for the graphic manipulation of large amounts of complex data in intuitive fashion, rendering the operator theoretically in charge of vast tokenized empires. Whether sensitive data should ever actually be open to manipulation via such methods is questionable, but remains a probability. In the end, there is little difference whether data is altered via a spreadsheet or a VR interface, each method maintains weak points open to malicious alteration. The programming interface offered by traditional data entry would likely never fully be replaced, but VR and AR offer an extremely favorable ratio between power and intuitiveness.
Some research that I feel will have a great deal of impact is occurring at Stanford University. There, researchers have created a virtual workbench area that is superimposed over the room they are in, allowing them to interact in 3D space with different objects of their choosing, such as the architecture of a building, or the construction of a molecule.
By locking the virtual and actual displays together and tracking them effectively, the result is one that can truly be considered approaching “Holodeck”-type reality. It is inevitable that we will one day view Shakespeare’s plays as they unfold around us, projected into the room. The obvious step beyond that would be interaction.
By combining this approach with the concept of telepresence, individuals will have the ability to “project” themselves to almost anywhere in the world. This branch of teleconferencing, still in its infancy, will one day make problems of time and distance largely irrelevant.When augmented reality merges with artificial intelligence, computers will be able to solve problems in the real world on their own. A plant supervisor could survey a
24 - “A shared, consensual hallucination…A graphic representation of data abstracted from the banks of every computer in the human system.” Gibson William. Neuromancer. (1984). New York, NY. ACE Books.
job-site while reviewing data overlaid onto their field of vision, while a computer working in conjunction with that person could potentially recognize patterns or phenomenon that might escape their own attention.By augmenting our own minds, and automating the augmentation process, we will be in a sense behaving as the brain itself does while learning. That is to say, when the brain learns a new task, the basal ganglia portion is utilized in this process. At some point, dependent upon the complexity of the concept to be learned, this knowledge is transferred into another part of the brain, the motor cortex, where it becomes more or less an automatic function, a learned response to a given situation. (25)
By one day using a computer as an extension of our own bodies, we will be able to solve and automate tasks by simply looking at them; Inventories controlled, factory floors monitored, and security operations supervised, by the electronic extensions of actual people.
One ongoing set of problems in all approaches remains that of display. Eyestrain, lack of resolution and field of vision, disorientation and other technical impediments to truly free flowing other reality immersion plague researchers the world over. While great strides have been made in identifying and overcoming potential health risks, a great deal of details remain to be worked out. Nevertheless, as technology reporter Karen Kaplan points out, “those obstacles don’t stop hard-core augmented reality fans from envisioninga wide range of applications.” (26)
RESEARCH QUESTIONS AND HYPOTHESIS
In 1991, I became convinced that the video approach to AR offered much more in the way of practical applications than did the heavily hyped field of VR. With all of the
25 - An oversimplification. For a detailed explanation of how the human mind is currently understood to function, please see Nadis Steve “The Energy Efficient Brain: PET Scans reveal how the brain delegates mental tasks.” Omni and Berger Bob “Mapping the Mindfields” Omni
26 - Kaplan, Karen. The Los Angeles Times. (1997) “3-D Technology Blends Fact and Fantasy” P. 3
complexity inherent in the real world, why then would you try to recreate such splendor using polygons? Even with the hundred fold advances in processor power, VR either remains at the Nintendo level of consumer usefulness, or stunningly cost prohibitive. As Simon Penny reminds us: “A criticism leveled at computer graphics…was that it was a cold space unable to persuasively represent the natural world.” (27)
While both approaches to AR offer solutions to specific problems, for the purposes of my study I postulate that a video approach is best suited for this research.Indeed, much of what I propose to investigate cannot be achieved via see-through optical displays. As is stated in Ronald T. Azuma’s “A Survey of Augmented Reality“: “Since both the real and virtual are available in digital form, video see-through compositors can,on a pixel-by-pixel basis, take the real, or the virtual, or some blend between the two to simulate transparency. Because of this flexibility, video see-through may ultimately produce more compelling environments than optical see-through approaches.” (28)
The same survey shows us that predictive motion algorithms must be employed to overcome the effect of system-induced delays inherent in both systems.One attractive aspect of some forms of AR is real-time updating. Superimposing textual data over your field of vision imposes no unnerving delays or lags, if it does not require registration. Registration error continues to be one of the biggest problems facing AR researchers. A millisecond of delay in updating the display can result in up to a millimeter of visual error. (29)
RESEARCH QUESTION #1: Can an affordable AR system utilizing luminance keying (30) deliver a satisfying experience to consumers in regards to Virtual Theater
27 - “Virtual Reality as the Completion of Enlightenment” (1995). Virtual Reality Casebook:. (P. 199)
28 - Azuma, R.T. “A survey of augmented reality“ (1997)
29 - Azuma, R.T. “A survey of augmented reality“ (1997)
30 - Luminance keying is a process whereby one video signal is blended with another based on brightness within one of the images. Video may be keyed (mapped) on a white surface, a black surface, or some shade between the two. It is superseded in quality by chroma keying, a more expensive color based process.
displays? While I recognize that at least one potential problem results from video’s full-frame nature, I feel this approach has merit mainly for the fact that it is affordable enough to be incorporated into consumer electronics in the near future.
While chroma keying provides a sharper key, its reliance on color is disadvantageous in the home setting. The somewhat cruder method of brightness based mapping allows for much more flexibility in regard to where video can be mapped, and to what extent. I do recognize the inherent value of chroma keyed video, however, and simply view it as a price/performance issue. This study will attempt to determine the value of mapped video as a form of mass communication delivery, as well as for use in industrial applications.
RESEARCH QUESTION #2: Through the use of simple visual devices, can AR deliver an experience not unlike the hallucinations associated with psychotropic drugs such as LSD and DMT? While it remains a touchy subject, the concept of electronic drugs, or electronics that mimic the effect of drugs, has established value in the field of psychotherapy. (31) There is an interest in certain segments of society in bringing such devices to fruition, but driven by two entirely different imperatives: the need to eliminate drug use versus the desire to share the psychedelic experience with others. Indeed, theorist Terence McKenna has stated in an interview with Mondo 2000, “The VR researchers have it all wrong. I want virtual DMT trips!” (32)
The study of perspective video (video filmed at human’s eye view), has been initiated by research such as the original work of Ivan Sutherland and NYNEX‘s Intelligent Interface Group’s Empathy Training. (33)
31 - Stevens, Jane E., “Next up for virtual reality: curing psychological ills.” Star Tribune,04-02-1995: 01E.
Dobson, Roger. “Conquering phobias with virtual reality.” (1996)
32 - McKenna, Terence. The Archaic Revival. (1991). Harper Collins Publishers. Reprinted from an article in Mondo 2000.
33 - Loeffler, Carl Eugene and Anderson, Tim. (1994) The Virtual Reality Casebook.
There is unquestionably a huge number of potential applications for video capture and playback utilizing HMDs. Education, training, not to mention entertainment, will soon be experienced via such methods. Given the eventuality, I submit that a see-through display will generally be preferred by end users. An example I use often: “Would you want your child’s babysitter wearing a helmet and being unable to see anything else?”
The added dimension offered by immersion, such as full peripheral vision(resulting in much more realistic feelings of motion) and enhanced feelings of identification with the camera/protagonist, will enable us to communicate experiences and emotional viewpoints in entirely new ways. Unfortunately, time constraints did not allow me to explore this usage of HMDs. Nevertheless, video is the language of AR/VR and its hybrids, and will become a chief focus when these products finally reach the consumer level.
METHODOLOGY
The hardware involved in these experiments was as follows. The HMD for display was a Virtual Research VR4 operating at the relatively modest resolution of 742 x230, displaying an S-VHS video signal. The input device was a Panasonic color video camera, small enough to sit comfortably on the head, or easily be held in the hand without fatigue. In this trial run, the camera was simply held in the user’s hand. The video signal from this camera, as well as a signal from video tape, was fed into a Amiga 2500/Video Toaster configuration, and the resultant signals sent out to the HMD for viewing.
The Video Toaster is a video card capable of luminance keying and numerous real-time digital video effects, allowing for the blending of four time base corrected video signals (videotape or camera). By manipulating the various effects, an array of
possibilities becomes apparent. It is not my intention to demonstrate an end product, but rather a proof of concept for further research.
The video camera served as the subject’s eyes, and the signal was fed into a TBC I from Digital Processing Systems in conjunction with the videotape signal. These synchronized elements were then fed into the Video Toaster inputs one and two. The program out from the Toaster was then fed into a Panasonic S-VHS deck to provide the signal necessary to drive the VR goggles’ Liquid Crystal Displays (LCDs). Once this configuration is wired, it is simply a matter of turning on the luminance keyer or digital effects to generate the desired effect.
Eight test subjects were brought in to demo the various applications, ranging in age from nineteen to sixty-five, six males and two females. A simple survey was conducted among them regarding the quality of the experience, the usefulness of the device, and their comments on the project.
RESULTS
Generally, the findings were encouraging. If nothing more, my research confirmed my beliefs that both virtual theater and electronic LSD are viable using essentially consumer gear. The VR4 goggle input being S-VHS or RGB was the biggest contributing factor regarding cost, requiring the use of an S-VHS deck that was otherwise unnecessary. A realistic estimation of the cost of the equipment involved (assuming the substitution of affordable goggles with composite inputs) would be $1500.
Several factors that detracted from the experience were derived from the nature of HMDs. Because of the size of the screens involved, and their distance from the eyes, an impression is given of having your eyes located four to six inches in front of your face.
The relatively low resolution coupled with the pixelated nature of LCDs diminishes the sense of realism necessary for an effective display. The participant’s criticisms of the HMD itself included a sense of isolation, the resolution/display problems, and an overall dissatisfaction with the bulkiness of the device. Regarding the luminance keying portion of the research, the video signal was mapped onto several closed sets of black venetian blinds. With very little adjustment being required, a more than adequate display was achieved. The users saw their own hands in front of them, a desk with a workstation, and behind the desk, the video-mapped window shades. As stated, several factors detracted from the overall effect, but the desired result was attained. As this technology migrates downward in price, it will be implemented more commonly in an industrial training capacity, such as the assembly of complex parts.
There exists an immediate market regarding turnkey installation of AR stations to aid in the inspection of circuit boards for the oilfield industry. When inspecting circuit boards using traditional analog tools (magnifying glasses), eye strain and failure are problems. By tightly focusing a camera from above onto a workbench area, problems of magnification are eliminated.
Contrary to my expectations, there was more interest regarding the psychedelic effects than there was in the television display, in both sexes and all age groups. Of the seven who participated in that portion of the research, all would consider wearing such a device on occasion, were it less obtrusive. This research has led to new theories on improving the experiences, in large part due to the response of participants.The next iteration of electronic LSD will be stereo 3D, the most common request, with the ability to combine digital trail and color effects. Enhanced Reality?
Likewise, the virtual theater display could be improved immensely by using two cameras to provide a stereo view, as well as by having a higher resolution display device.As one of the participants pointed out, the content displayed on the device plays a crucial role in determining the amount of immersion the user feels.
DISCUSSION
There currently seems to be a race between three competing technologies as the next-generation display device: Projectors and liquid crystal shutter glasses, VR/AR, and flat panel displays. New nanotube technology has the potential to make some virtual theater applications largely irrelevant. Perfectly formed glass crystals can now be quickly grown in arrays that are proving to be ideal for creating high-performance video displays. (34) The race is on, and we as human beings will ultimately win, regardless of the standards of delivery that arise.
Essentially, I see no reason for a lack of a VR “underground.” Currently the realm of researchers, VR has been an unassailable ivory tower to the public. Referred to in hushed tones but never experienced by most, it is the stuff of science fiction. Indeed, the C.A.V.E. (35) at the University of Chicago is the realization of Star Trek’s Holodeck, but is cost prohibitive for consumer use, apart from theme parks and theaters.C.A.V.E. immerses participants in a “room without walls,” using LCD shutter glasses and projected displays to achieve an experience unequaled elsewhere. If this system can be cost-reduced and used in conjunction with AR, we would be well on our way toward ultimate reality.As always, bandwidth and processor power determines the parameters of the experience. Considering the ongoing trend towards faster-than-real time video rendering,
34 - UPI. “Wafer-Thin TV Screens a Step Closer” (November 5, 1998)
35 - 1993 IEEE Computer Society Technical Committee on Computer Graphics’ “IEEE 1993 Symposium on Research Frontiers in Virtual reality” IEEE Computer Society Press. Los Alamitos, Ca. Bryson. Steve and Feiner, Steve. Co-Chairs.
it would seem that the power of video-based AR systems could soon surpass that of polygon rendering VR. When you consider that the two disciplines will one day be unified, such distinctions lose their relevance.
I seek to destroy the barrier that stands before the hobbyist and these new vistas of human communication. It is discouraging to consider waiting another ten years for such products to begin to appear on store shelves. Using a video camera, a VCR, and a HMD, people could begin trading experiences on tape for others to enjoy in the present term. Considering the potential benefits we may reap in the fields of entertainment, mass communication, interpersonal communication, education, medicine, and industry, I am curious as to why VR/AR research seems to be so localized in the U.S.
There is an enormous amount of basic research waiting to be done by adventurous researchers and hobbyists. HMDs have applications in each division of college, and must cease being thought of as solely the domain of computer scientists. I believe the industry would advance much more quickly with more input from visual artists, videographers,and end users. With the goals so clearly defined as to what VR/AR should ultimately resemble, I feel it is urgent to redouble our efforts to find workable solutions to the many problems facing this potential-laden, but under-utilized hardware setup.
“When we apply the power of networking to the VR world, the potential for stretching the limits of human abilities becomes very powerful.” —Jaron Lanier (1)
INTRODUCTION
Virtual Reality (VR), its “newer” cousin Augmented Reality (AR), Enhanced Reality (ER), Simulated Environments (SE), Myron Krueger’s Artificial Reality (2) and other terms each attempt to describe unique sets of experiences and phenomenon that promise to radically alter the process of human communication. It is my belief that these various disciplines will continue evolving parallel with related technologies until a critical mass is achieved, resulting in a coalescence of unprecedented magnitude, which I have termed Ultimate Reality, in deference to Ivan Sutherland’s pioneering efforts described in The Ultimate Display. (3)
Mr. Sutherland’s first head-mounted display (HMD) was in fact a see-through design.(4) Through clever use of half-bright mirrors, a video display was mapped over the user’s field of vision. This was the first implementation of what is now known as Augmented Reality. I contend that this original incarnation will ultimately prove to be more useful to mankind.
1 - Byte, Vol. 22, No. 8, P. 32 “News and Views: Interviews.” (1997).
2 - While Mr. Krueger’s phrasing seems best suited to describe his own unique field of study into “other”environments, I fully believe his early research will prove instrumental in the coming technological convergence, particularly in the areas of telepresence and in making computers “aware” of human beings. Not only that, he constantly reminds us that such experiences should be enjoyable, even fun. This humanizing factor deserves far more attention than it currently receives.
3 - Sutherland, Ivan. "The Ultimate Display." Information Processing 1965: Proceedings of IFIP Congress 65, 2 (New York, May 24-29, 1965): 508.
4 - Pimentel, Ken and Teixeira, Kevin. (1995) Virtual Reality: Through The New Looking Glass. P. 44
Few fields have shown as much promise, while failing to deliver in the way of actual product, as Virtual Reality, thus thwarting the potential for rapid advances in human communication. While proving to be invaluable in the fields of science, medicine, and computing, what was once seen as a marketplace panacea has proven to be a commercial black hole. Various attempts to penetrate the U.S. consumer market have inevitably resulted in withdrawals and bankruptcies. From the fall of the pioneering VPL Research (5) to the 1997 bankruptcy filing of Virtual I-O, (6) VR has failed to attain a sizable niche in the marketplace beyond industrial and medical applications. Despite VR’s inability to gain a foothold in the consumer realm, the industry is still estimated to gross $1,000,000,000 in 1998 alone. (7)
There has yet to be a compelling use for VR/AR in daily life. Games apparently are not driving factors in moving this technology into homes. In true paradoxical fashion,we won’t have better displays until there is a large demand, and we won’t have a demand until we have better displays. Japan, however, has a healthy market for VR, and relies heavily on American research and development, primarily importing foreign gear for various consumer uses. As far back as 1993, the National Trade Data Bank Market Report painted a bright future for VR in Japan, somewhere in the neighborhood of $174-$435 million in projected sales and revenue from VR related fields, entirely within the realm of entertainment. (8)
It could be that the Japanese are culturally more ready for artificial reality experiences because of their preferred television viewing distance being half that of Western preferences. In other words, there is less reluctance to having a display near the eyes. While VR theme parks are a burgeoning industry in Japan, they have not caught on well in the United States,
5 - Rifken, Glenn. (1996) “Computerworld Inner Visions”. P. 1 (November 11, 1996) (September 12, 1998) www.elibrary.com Computerworld.
6 - Warren Publishing, Inc. Consumer Electronics (May 5, 1997) (August 13, 1998)
7 - Infonautics Corporation. “Virtual Reality Market to Reach Almost Billion Dollars in 1998”.www.elibrary.comPress Release
8 - The National Trade Data Bank Market Report. (Japan: 1993)
with few exceptions (Virtuality’s early success with Dactyl Nightmare and Chicago’s BattleTech center.) (9)
In the 1980’s, time base correctors (TBCs) (10) averaged a minimum cost of about $5000 each. With the advent of desktop video, in particular, Newtek’s Video Toaster, the demand for TBCs increased to such a level as to lower prices to the extent that today you can you can easily equip a computer with four TBCs, not to mention a video switcher, for less than $5000. If industry, by way of consumers, creates a demand for luminance and chrominance-based solutions, this pricing shift could occur with AR systems at a much higher rate, resulting in a product that remains affordable, but offers high quality images.
The ultimate artificial reality experience would be a direct mind link. Scientists are now able to view brainwave activity patterns as they occur. (11)
As the neuromagnetometers that perform this task become more sensitive, and the computers that interpret the data more sophisticated, it will be possible to think “apple,” and cause the representation of an apple to appear in virtual or augmented reality. Thus, I posit that it will one day be possible to record actual thoughts and dreams. It is at this point that things begin to get interesting, in the sense that we will be able to create other realities through our own thoughts.
Stereolithography is the process whereby 3-D objects are made into physical objects through the use of “object printers,” machines that convert the 3D data into solid objects by treating a volume of resin with lasers to harden it into specific shapes. (12) The future promises us a world in which solid objects can be transmitted via a 3D “fax machine,” eminently useful in design and prototyping work. Coupling this process with
9 - I find it interesting to see Commodore’s Amiga crop up repeatedly in this research. Both Dactyl Nightmare and BattleTech used Amiga 3000’s for image generation and control. Sense8’s first demo to Sun was also Amiga-based.
10 - Time base correctors correct the timing errors of videotape transports to provide a stable signal for switching and editing purposes.
11 - Berger, Bob “Mapping the Mindfields” Omni
12 - Pimentel, Ken and Teixeira, Kevin. (1995) Virtual Reality: Through The New Looking Glass. P. 269
MRI “mind-reading” could result in the ability of humankind to create solid objects out of pure thought. While not exactly “deus ex machina,” (13) it is a seductive prospect.
The technology that will tie these many disparate processes together into a unified whole comes from both academia and Hollywood. Engineers seeking to increase flexibility in film editing have developed a process whereby each element in a frame of film is digitized separately and is thereafter treated as a fully editable object. The coming technological convergence will utilize such processes, operating at faster than realtime speeds, to provide an experience that is visually indistinguishable from actual reality.
Literally, anything will be possible. The real and “other” worlds will begin to blur together, and form a synthesis that I refer to as Ultimate Reality. With ultra high-resolution cameras, a full field of vision (180 degrees) and a very high-resolution display (at least 9600 x 9600 pixel resolution), combined with accurate tracking and motion prediction, the interface will become transparent. It will then be possible to trick the human mind into believing that the virtual is actual. Indeed, it will become very difficult to distinguish between the two. In a realm worthy of Disney, broomsticks will dance across the room, you will converse with Alice in the tea room, and have the ability to instantly transport yourself to another point in time or space, whether across the country, or to a place that exists only in the ether. If the medium is indeed the message, we’re in for a wild ride.
LITERATURE REVIEW
While there is a wealth of information about Virtual Reality on library shelves,Augmented Reality has scarcely been given any attention there. It would seem the speed and immediacy of the World Wide Web has outpaced that of library database maintenance. The best data on the subject comes from the individuals who are developing
13 - God from a machine.
these technologies on a daily basis. The Web offers a close cadre of AR researchers making their findings available via University servers.
Virtual Reality - Scientific and Technological Challenges, from the National Research Council, is an in-depth examination of a wealth of issues related to the topic at hand. A weighty and technical tome, it is not directed towards the layman. Their recommendations, particularly in the area of promoting networked simulated reality development. (14) are heartening. They seek to actively encourage research and development in the area of networked VR applications.
A look at current trade journals indicates a growing interest in AR applications. E-Library retrieved some 20 popular press articles regarding augmented reality applications. Various sources for background include from The L.A. Times, and many trade journals in the field of video and 3-D animation. Metacrawler and Dogpile are meta-search engines, returning results from all of the other major search engines.
Best results were obtained with the University of Southwestern Louisiana’s licensed site access via the library’s homepage. A few academic search engines worthy of mention includes Lexis-Nexis, which didn’t have as many article references as I would have expected, given the sites weighty credentials. Web of Science is a research paper archive that provided a great deal of in-depth studies.
In 1995, Michihiro Uenohara and Takeo Kanade achieved real-time registration in AR at 30 hz via a method of computer vision (15) These researchers, as well as the scientists at the University of North Carolina at Chapel Hill, have initiated invaluable studies regarding the medical applications of augmented reality. X-ray vision is at hand, as physicians will one day soon be enable to peer within your various internal organs and
14 - National Research Council. Durlach, Nathaniel I. and Mavor, Anne S. (1995). Virtual Reality: Scientific and Technological Challenges
15 - M. Uenohara, T. Kanade, (1995). “Real-Time Vision Based Object Registration for Image Overlay”, Submitted to the
CVRMed '95, Nice, France.
systems, with the ability to view specific aspects such as the circulatory or respiratory systems alone. (16)
One book I found invaluable was The Virtual Reality Casebook, a collection of writings from a number of theorists, engineers, artists and other Artificial Reality denizens. Diverse and well rounded, the text was possibly the single-most consulted book in my research. It gives keen insight into the minds of the researchers and theorists that move the state of the art forward.
Despite a slight amount of product placement (the book is published in part by microprocessor giant Intel), Virtual Reality - Through the New Looking Glass (2nd edition), is a great starting point for anyone interested in the wide world of alternate realities. In all fairness, the authors are conscious of the possibility of perceived bias, and seem to go out of their way to be inclusive of the role of Apple’s Macintosh and even Commodore’s Amiga in the development of these technologies.
Virtual Reality Applications, edited by R.A. Earnshaw, J.A. Vince, and H. Jones,all from the U.K., also provides a well rounded perspective on the field, in a detailed manner that goes well beyond most American VR textbooks.
Myron Krueger, speaking of idealized hardware to come and the necessity of real-time interaction in an interview with Jas. Morgan, said: “Ultimately, a low-cost head-mounted display with the resolution of an OmniMax theater will be irresistible—if it’s as unencumbering as VIDEOPLACE. (17) It is my expectation that the two approaches will merge. The lightweight goggles will fit within ordinary eyeglasses. They will superimpose graphics on the real world. They won’t cut you off from your colleagues—you’ll be able to make eye contact with them.”
This is the crucial communication element that is missing from VR and,ironically, one that many people are currently working at adding to the experience. Until
16 -Virtual Reality Internet Resource Archive
17 - VIDEOPLACE is Krueger’s project that placed the participant’s silhouettes into a video display that allowed users to interact with each other, as well as with the computer-generated environment they inhabited.
we reach the point of networked VR experiences, immersion equals isolation. Will we come full circle and try to accurately represent our facial expressions using models? If the purpose is realism, then it becomes logical, and more efficient, to use one’s actual face via video. As video manipulation becomes more prevalent, we will gain the ability to alter our appearance slightly or dramatically, and do it in a photo-real environment.In a more abstract or representational framework, akin to today’s chat, non-realistic presentations of the self can be advantageous.
I expect in the interim to see polygon-based avatar chat capturing the subtleties of facial expression available in the next few years. In the same way object scanners have slowly migrated down to the desktop, technology that is now used for producing real-time cartoon actors for Saturday morning will be integrated into the coming networked virtual reality experience. The levels of interaction will be such that the future of entertainment as we know it may be headed the way of the drive-in movie. In the same respect, I fully believe that the future of mass and interpersonal communication will make television and the Internet seem like radio and black and white movies in comparison.
Augmented reality is approached in two diverse fashions: a digital display superimposed over clear glass, allowing a literally see-through display, and by use of various combinations of video cameras and HMDs. Each has proven to be tremendously empowering to the individual, and to a similar extent, the corporate entity. NASA’s work in correcting corneal defects via AR is a good example of what advantages the individual may gain. Telescopic sight, night vision, microscopic vision,and even more specialized abilities will eventually be commonplace, and currently exist on the market. How long before the technology becomes available in implant form?
Already, there is a growing gap between the computer literate and nonliterate. As technology increases exponentially, so increases the disparity between the rich and poor. Are we to become superhuman, able to exist in and see worlds the less fortunate cannot fathom? It would make the idea of “class warfare” outlandishly one-sided.
Boeing Corporation in particular has been the first major corporation to wholeheartedly embrace the concept of augmented reality, applying it to the field of aircraft production, which is imaginably a complex process. AR allows the Boeing worker to consult a vast array of manuals and documentation without having to leave the work area, or indeed, stop what he or she is doing at the time. The fuselage of the plane is located and tracked by the computer, allowing for the superimposition of construction details, such as the type that can tell a mechanic where to drill a hole, or an electrician how to wire a section. (18)
Idealized AR/VR is transparent, seamless. Integrating the power of computers with real-world displays presents us with a future reality in which all the world’s knowledge will be available to each of us, and delivered in such a fashion as to integrate into our lives as painlessly as a new cable box. Moore’s law being what it is, (19) the day of the “VR Man” (20) will soon be upon us. Are you prepared for the cultural shifts ahead?
Already, your computer can be considered an extension of your brain, or your self. Ongoing advances will soon empower you to actually augment your own mind through wearable computing. Do you think the Internet is useful? Can you imagine having it on a contact lens?
While the advantages seem obvious, the pitfalls remain hidden. Imagine if you will an age when such devices become ubiquitous as wristwatches, or at least laptop computers. The possibility that your very location could be tracked at all times has alarming implications, as does the concept of torture induced via AR/VR pseudo
18 - Pimentel, Ken and Teixeira, Kevin. (1995) Virtual Reality: Through The New Looking Glass. P. 260
19 - Moore’s law tells us that transitory capacity doubles in complexity every 2-3 years.
20 - The AR/VR equivalent of the Sony Walkman.
hallucinations. This type of application, perhaps administered along with conventional mind control techniques (drugs, sleep deprivation, etc.) could very well be the sort of reprogrammer sought by the CIA project group MK-Ultra, (21) and the “virtual torture” ominously predicted by George Orwell in 1984. (22)
Another fact that we must resign ourselves to is that VR and AR offer as much to the military and police as they do to industry, science, and consumers. The earliest research into telepresence, as in driving a vehicle remotely, began in the 1940’s. Remote viewing and telepresence will be one day result in persons being killed via a tank operated by a crew who remain removed from the actual battlefield. Janez Strehovec;characterizes an experience such as viewing the impact of a laser-guided, camera-equipped missile as “A paradigm of genuine techno-druggedness, caused by the identification with smart eyes’ view, and obsession with the success of a seductive action,fascinating also to the senses.” (23)
Even when we examine alternate realities in less metaphysical detail, a number of inevitable problems arise. The potential for not only viewing material others may deem offensive for a variety of reasons, but to actually become immersed in, will no doubt one day cause a furor, much in the way rock and roll, horror movies, and the Internet have in the past. The potential to now record experiences for others to view is a vast frontier for future friction between groups who would presumably restrict what others may experience immersively. Clearly another existential argument, ironically derived from what is in its simplest form, a hopped-up television set.
21 - Marchetti, Victor, John D. Marks The CIA and the Cult of Intelligence. (1984), Marks, John D. Search for the Manchurian Candidate. (1988).
22 - Orwell, George. 1984. (1949).
23 - “Vertigo--on purpose: Entertainment in simulators.” (1997). Journal of Popular Culture. Vol. 31 Issue 1, p.199
The exciting notion that William Gibson’s Cyberspace (24) is here, or around the corner, is one that I find to be demonstrably true. The science of data visualization allows for the graphic manipulation of large amounts of complex data in intuitive fashion, rendering the operator theoretically in charge of vast tokenized empires. Whether sensitive data should ever actually be open to manipulation via such methods is questionable, but remains a probability. In the end, there is little difference whether data is altered via a spreadsheet or a VR interface, each method maintains weak points open to malicious alteration. The programming interface offered by traditional data entry would likely never fully be replaced, but VR and AR offer an extremely favorable ratio between power and intuitiveness.
Some research that I feel will have a great deal of impact is occurring at Stanford University. There, researchers have created a virtual workbench area that is superimposed over the room they are in, allowing them to interact in 3D space with different objects of their choosing, such as the architecture of a building, or the construction of a molecule.
By locking the virtual and actual displays together and tracking them effectively, the result is one that can truly be considered approaching “Holodeck”-type reality. It is inevitable that we will one day view Shakespeare’s plays as they unfold around us, projected into the room. The obvious step beyond that would be interaction.
By combining this approach with the concept of telepresence, individuals will have the ability to “project” themselves to almost anywhere in the world. This branch of teleconferencing, still in its infancy, will one day make problems of time and distance largely irrelevant.When augmented reality merges with artificial intelligence, computers will be able to solve problems in the real world on their own. A plant supervisor could survey a
24 - “A shared, consensual hallucination…A graphic representation of data abstracted from the banks of every computer in the human system.” Gibson William. Neuromancer. (1984). New York, NY. ACE Books.
job-site while reviewing data overlaid onto their field of vision, while a computer working in conjunction with that person could potentially recognize patterns or phenomenon that might escape their own attention.By augmenting our own minds, and automating the augmentation process, we will be in a sense behaving as the brain itself does while learning. That is to say, when the brain learns a new task, the basal ganglia portion is utilized in this process. At some point, dependent upon the complexity of the concept to be learned, this knowledge is transferred into another part of the brain, the motor cortex, where it becomes more or less an automatic function, a learned response to a given situation. (25)
By one day using a computer as an extension of our own bodies, we will be able to solve and automate tasks by simply looking at them; Inventories controlled, factory floors monitored, and security operations supervised, by the electronic extensions of actual people.
One ongoing set of problems in all approaches remains that of display. Eyestrain, lack of resolution and field of vision, disorientation and other technical impediments to truly free flowing other reality immersion plague researchers the world over. While great strides have been made in identifying and overcoming potential health risks, a great deal of details remain to be worked out. Nevertheless, as technology reporter Karen Kaplan points out, “those obstacles don’t stop hard-core augmented reality fans from envisioninga wide range of applications.” (26)
RESEARCH QUESTIONS AND HYPOTHESIS
In 1991, I became convinced that the video approach to AR offered much more in the way of practical applications than did the heavily hyped field of VR. With all of the
25 - An oversimplification. For a detailed explanation of how the human mind is currently understood to function, please see Nadis Steve “The Energy Efficient Brain: PET Scans reveal how the brain delegates mental tasks.” Omni and Berger Bob “Mapping the Mindfields” Omni
26 - Kaplan, Karen. The Los Angeles Times. (1997) “3-D Technology Blends Fact and Fantasy” P. 3
complexity inherent in the real world, why then would you try to recreate such splendor using polygons? Even with the hundred fold advances in processor power, VR either remains at the Nintendo level of consumer usefulness, or stunningly cost prohibitive. As Simon Penny reminds us: “A criticism leveled at computer graphics…was that it was a cold space unable to persuasively represent the natural world.” (27)
While both approaches to AR offer solutions to specific problems, for the purposes of my study I postulate that a video approach is best suited for this research.Indeed, much of what I propose to investigate cannot be achieved via see-through optical displays. As is stated in Ronald T. Azuma’s “A Survey of Augmented Reality“: “Since both the real and virtual are available in digital form, video see-through compositors can,on a pixel-by-pixel basis, take the real, or the virtual, or some blend between the two to simulate transparency. Because of this flexibility, video see-through may ultimately produce more compelling environments than optical see-through approaches.” (28)
The same survey shows us that predictive motion algorithms must be employed to overcome the effect of system-induced delays inherent in both systems.One attractive aspect of some forms of AR is real-time updating. Superimposing textual data over your field of vision imposes no unnerving delays or lags, if it does not require registration. Registration error continues to be one of the biggest problems facing AR researchers. A millisecond of delay in updating the display can result in up to a millimeter of visual error. (29)
RESEARCH QUESTION #1: Can an affordable AR system utilizing luminance keying (30) deliver a satisfying experience to consumers in regards to Virtual Theater
27 - “Virtual Reality as the Completion of Enlightenment” (1995). Virtual Reality Casebook:. (P. 199)
28 - Azuma, R.T. “A survey of augmented reality“ (1997)
29 - Azuma, R.T. “A survey of augmented reality“ (1997)
30 - Luminance keying is a process whereby one video signal is blended with another based on brightness within one of the images. Video may be keyed (mapped) on a white surface, a black surface, or some shade between the two. It is superseded in quality by chroma keying, a more expensive color based process.
displays? While I recognize that at least one potential problem results from video’s full-frame nature, I feel this approach has merit mainly for the fact that it is affordable enough to be incorporated into consumer electronics in the near future.
While chroma keying provides a sharper key, its reliance on color is disadvantageous in the home setting. The somewhat cruder method of brightness based mapping allows for much more flexibility in regard to where video can be mapped, and to what extent. I do recognize the inherent value of chroma keyed video, however, and simply view it as a price/performance issue. This study will attempt to determine the value of mapped video as a form of mass communication delivery, as well as for use in industrial applications.
RESEARCH QUESTION #2: Through the use of simple visual devices, can AR deliver an experience not unlike the hallucinations associated with psychotropic drugs such as LSD and DMT? While it remains a touchy subject, the concept of electronic drugs, or electronics that mimic the effect of drugs, has established value in the field of psychotherapy. (31) There is an interest in certain segments of society in bringing such devices to fruition, but driven by two entirely different imperatives: the need to eliminate drug use versus the desire to share the psychedelic experience with others. Indeed, theorist Terence McKenna has stated in an interview with Mondo 2000, “The VR researchers have it all wrong. I want virtual DMT trips!” (32)
The study of perspective video (video filmed at human’s eye view), has been initiated by research such as the original work of Ivan Sutherland and NYNEX‘s Intelligent Interface Group’s Empathy Training. (33)
31 - Stevens, Jane E., “Next up for virtual reality: curing psychological ills.” Star Tribune,04-02-1995: 01E.
Dobson, Roger. “Conquering phobias with virtual reality.” (1996)
32 - McKenna, Terence. The Archaic Revival. (1991). Harper Collins Publishers. Reprinted from an article in Mondo 2000.
33 - Loeffler, Carl Eugene and Anderson, Tim. (1994) The Virtual Reality Casebook.
There is unquestionably a huge number of potential applications for video capture and playback utilizing HMDs. Education, training, not to mention entertainment, will soon be experienced via such methods. Given the eventuality, I submit that a see-through display will generally be preferred by end users. An example I use often: “Would you want your child’s babysitter wearing a helmet and being unable to see anything else?”
The added dimension offered by immersion, such as full peripheral vision(resulting in much more realistic feelings of motion) and enhanced feelings of identification with the camera/protagonist, will enable us to communicate experiences and emotional viewpoints in entirely new ways. Unfortunately, time constraints did not allow me to explore this usage of HMDs. Nevertheless, video is the language of AR/VR and its hybrids, and will become a chief focus when these products finally reach the consumer level.
METHODOLOGY
The hardware involved in these experiments was as follows. The HMD for display was a Virtual Research VR4 operating at the relatively modest resolution of 742 x230, displaying an S-VHS video signal. The input device was a Panasonic color video camera, small enough to sit comfortably on the head, or easily be held in the hand without fatigue. In this trial run, the camera was simply held in the user’s hand. The video signal from this camera, as well as a signal from video tape, was fed into a Amiga 2500/Video Toaster configuration, and the resultant signals sent out to the HMD for viewing.
The Video Toaster is a video card capable of luminance keying and numerous real-time digital video effects, allowing for the blending of four time base corrected video signals (videotape or camera). By manipulating the various effects, an array of
possibilities becomes apparent. It is not my intention to demonstrate an end product, but rather a proof of concept for further research.
The video camera served as the subject’s eyes, and the signal was fed into a TBC I from Digital Processing Systems in conjunction with the videotape signal. These synchronized elements were then fed into the Video Toaster inputs one and two. The program out from the Toaster was then fed into a Panasonic S-VHS deck to provide the signal necessary to drive the VR goggles’ Liquid Crystal Displays (LCDs). Once this configuration is wired, it is simply a matter of turning on the luminance keyer or digital effects to generate the desired effect.
Eight test subjects were brought in to demo the various applications, ranging in age from nineteen to sixty-five, six males and two females. A simple survey was conducted among them regarding the quality of the experience, the usefulness of the device, and their comments on the project.
RESULTS
Generally, the findings were encouraging. If nothing more, my research confirmed my beliefs that both virtual theater and electronic LSD are viable using essentially consumer gear. The VR4 goggle input being S-VHS or RGB was the biggest contributing factor regarding cost, requiring the use of an S-VHS deck that was otherwise unnecessary. A realistic estimation of the cost of the equipment involved (assuming the substitution of affordable goggles with composite inputs) would be $1500.
Several factors that detracted from the experience were derived from the nature of HMDs. Because of the size of the screens involved, and their distance from the eyes, an impression is given of having your eyes located four to six inches in front of your face.
The relatively low resolution coupled with the pixelated nature of LCDs diminishes the sense of realism necessary for an effective display. The participant’s criticisms of the HMD itself included a sense of isolation, the resolution/display problems, and an overall dissatisfaction with the bulkiness of the device. Regarding the luminance keying portion of the research, the video signal was mapped onto several closed sets of black venetian blinds. With very little adjustment being required, a more than adequate display was achieved. The users saw their own hands in front of them, a desk with a workstation, and behind the desk, the video-mapped window shades. As stated, several factors detracted from the overall effect, but the desired result was attained. As this technology migrates downward in price, it will be implemented more commonly in an industrial training capacity, such as the assembly of complex parts.
There exists an immediate market regarding turnkey installation of AR stations to aid in the inspection of circuit boards for the oilfield industry. When inspecting circuit boards using traditional analog tools (magnifying glasses), eye strain and failure are problems. By tightly focusing a camera from above onto a workbench area, problems of magnification are eliminated.
Contrary to my expectations, there was more interest regarding the psychedelic effects than there was in the television display, in both sexes and all age groups. Of the seven who participated in that portion of the research, all would consider wearing such a device on occasion, were it less obtrusive. This research has led to new theories on improving the experiences, in large part due to the response of participants.The next iteration of electronic LSD will be stereo 3D, the most common request, with the ability to combine digital trail and color effects. Enhanced Reality?
Likewise, the virtual theater display could be improved immensely by using two cameras to provide a stereo view, as well as by having a higher resolution display device.As one of the participants pointed out, the content displayed on the device plays a crucial role in determining the amount of immersion the user feels.
DISCUSSION
There currently seems to be a race between three competing technologies as the next-generation display device: Projectors and liquid crystal shutter glasses, VR/AR, and flat panel displays. New nanotube technology has the potential to make some virtual theater applications largely irrelevant. Perfectly formed glass crystals can now be quickly grown in arrays that are proving to be ideal for creating high-performance video displays. (34) The race is on, and we as human beings will ultimately win, regardless of the standards of delivery that arise.
Essentially, I see no reason for a lack of a VR “underground.” Currently the realm of researchers, VR has been an unassailable ivory tower to the public. Referred to in hushed tones but never experienced by most, it is the stuff of science fiction. Indeed, the C.A.V.E. (35) at the University of Chicago is the realization of Star Trek’s Holodeck, but is cost prohibitive for consumer use, apart from theme parks and theaters.C.A.V.E. immerses participants in a “room without walls,” using LCD shutter glasses and projected displays to achieve an experience unequaled elsewhere. If this system can be cost-reduced and used in conjunction with AR, we would be well on our way toward ultimate reality.As always, bandwidth and processor power determines the parameters of the experience. Considering the ongoing trend towards faster-than-real time video rendering,
34 - UPI. “Wafer-Thin TV Screens a Step Closer” (November 5, 1998)
35 - 1993 IEEE Computer Society Technical Committee on Computer Graphics’ “IEEE 1993 Symposium on Research Frontiers in Virtual reality” IEEE Computer Society Press. Los Alamitos, Ca. Bryson. Steve and Feiner, Steve. Co-Chairs.
it would seem that the power of video-based AR systems could soon surpass that of polygon rendering VR. When you consider that the two disciplines will one day be unified, such distinctions lose their relevance.
I seek to destroy the barrier that stands before the hobbyist and these new vistas of human communication. It is discouraging to consider waiting another ten years for such products to begin to appear on store shelves. Using a video camera, a VCR, and a HMD, people could begin trading experiences on tape for others to enjoy in the present term. Considering the potential benefits we may reap in the fields of entertainment, mass communication, interpersonal communication, education, medicine, and industry, I am curious as to why VR/AR research seems to be so localized in the U.S.
There is an enormous amount of basic research waiting to be done by adventurous researchers and hobbyists. HMDs have applications in each division of college, and must cease being thought of as solely the domain of computer scientists. I believe the industry would advance much more quickly with more input from visual artists, videographers,and end users. With the goals so clearly defined as to what VR/AR should ultimately resemble, I feel it is urgent to redouble our efforts to find workable solutions to the many problems facing this potential-laden, but under-utilized hardware setup.
Comments
Post a Comment