We would like to advise you to read the glossary first, as you may encounter
some difficulties considering medical terminology.
This presentation is broken into the following sections:
Cortex
The cerebral cortex, the outermost layer
of the cerebral hemispheres of the brain, responsible for all forms of
conscious experience.
Retina
A multi-layered sheet of nerve cells at the back
of each eye which converts light into electrical signals that are transmitted
to the brain through the optic nerves and tracts. Below is a picture of
the anatomy of the retina showing layers, vessels and nerves.
Image courtesy of CIBAVision
Photoreceptor
One of the 125 million nerve cells in the retina
of each eye that emit electrical signals when activated by light of a particular
wavelength. There are two types of photoreceptor cells in vertebrates:
rods, used in low light, and cones, which respond in brighter light and
to color.
Image courtesy of Howard Hughes Medical Institute
The intricate layers and connections of nerve cells in the retina were drawn by the famed Spanish anatomist Santiago Ram?n y Cajal around 1900. Rod (a) and cone (b) cells are at the top. Optic nerve fibers leading to the brain may be seen at bottom right.
VRD
Is a new display technology that scans modulated
low energy laser light directly onto the viewer’s retina to create a perception
of a virtual image.
As shown in the above figure the reflected rays of the pencil pass through
its lens. The inverted image of the pencil is directly projected onto the
retina. The rod and cone cells of the retina are responsible of tranmitting
signals to the brain through the optical nerve.
Another important factor in seeing is the stereo view capability. The
images presented to the human vision must provide those characteristics
that make them appear stereoscopic. That way, the viewer can optically
calculate the distance of far seen objects. "The dumber the animal, the
smarter its retina," observes Denis Baylor of Stanford Medical School.
Below is a graphical explanation of how the human vision generates
stereoscopic images.
Image courtesy of Howard Hughes Medical Institute
The two eyes provide slightly different views of the same scene.
Information from the left visual field goes to the right side of
the retina in both eyes. At the optic
chiasm, half the nerve fibers from the left eye cross over
to the right hemisphere and the rest stay uncrossed, so that all
the information from the left visual field ends up in the right hemisphere.
In this way, a given hemisphere gets information from the opposite
half of the visual world—but each hemisphere gets input from both
eyes. Thus, the future VRD product must have stereoscopic capabilities.
Image courtesy of Microvision
As shown above, the source image is directed into the drive electronics
device where the green, blue, and red light are modulated. Then each pixel
passes through the horizontal and vertical scanners (one moves from left
to right and the other from up to down) and produces a rasterized image
onto the retina. The retina has no persistence like
phosphorus. Moreover, there is no flickering and the produced image is
very bright with very high resolution. The time each pixel is projected
onto the retina is very small (30-40ns). Furthermore, these devices consume
very little amount of energy. They also provide a wider field of view.
A very sophisticated representation of how the beam is projected through
the pupil can be found in the HIT Lab of Washington University (VRD
animations).
System Modules
Drive electronics
The drive electronics control the acousto-optic
modulators that encode the image data into the pulse stream. The color
combiner multiplexes the individually-modulated red, green, and blue beams
to produce a serial stream of pixels, which is launched into a singlemode
optical fiber to propagate to the scanner assembly. The drive electronics
receive and process an incoming video signal, provide image compensation,
and control image display. For VGA projection, the electronics process
over 18 Mpix/s. The virtual retinal display is capable of providing UXGA
resolution of 1600 x 1200 or 115 Mpix/s.
Light source
The light source module contains laser light sources,
acousto-optic modulators to create the pulse stream, and a color combiner
that multiplexes the pulse streams. To provide sufficient brightness, full-color
displays suitable for outdoor, daylight applications incorporate red diode
lasers (635nm), green solid-state lasers
(532 nm), and blue solid-state or argon gas lasers (450-470 nm range).
Systems designed for indoor use can incorporate LEDs; red, blue, and green
devices currently under development for such systems are being tested.
Generally, the energy levels are on the order of nanowatts to milliwatts,
depending on display requirements. The levels of light involved are well
within laser safety standards for viewing, as confirmed by analysis.
Scanner assembly
The scanner assembly contains two scanning mirrors.
One 24 mm x 6 mm x 6 mm scanning mirror sweeps the beam horizontally at
a high frequency, typically between 15.75 kHz and 18.9 kHz for 60-Hz non-interlaced
refresh rates. This corresponds to one-half the VESA monitor-timing standard
since the retinal scanning display can process and display pixels bidirectionally.
A second scanning mirror sweeps the beam of laser light vertically at 60
Hz to complete the raster image.
Microvision is currently developing a microelectromechanical system
(MEMS-) based full image scanner capable of bi-directional scanning. Unlike
MEMS-based spatial light modulators used for projection applications, the
millimeter-sized mirror in the virtual retinal display will provide the
scan angles and rates necessary to provide the full raster image. The need
to fabricate only one mirror significantly reduces the development effort
for higher resolutions and associated yield problems. It also eliminates
the potential for "dead" pixels due to inoperative mirror elements.
Pupil expander
Nominally the entire image would be contained in
an area of 2 mm2. The exit-pupil expander (not shown in Figure 2) is an
optical device that increases the natural output angle of the image and
enlarges it up to 18 mm on a side for ease of viewing. The raster image
created by the horizontal and vertical scanners passes through the pupil
expander and on to the viewer optics.
Viewer optics
The viewer optics relay the scanned raster image
to the oculars worn by the user. The optical system varies according to
the application. In the case of military applications such as helmet mounted
or head mounted display optics, the system incorporates glass and or plastic
components; for medical applications such as image-guided surgery, head-mounted
plastic optics are used. In industrial or personal displays, the optics
might be a simple plastic lens.
| Resolution | The VRD produces very high resolution images. Its resolution is limited only by diffraction and optical aberrations in the light source and not by how small one can make an individual pixel element in a large array of pixel elements. |
| Contrast Ratio | The brightness of the VRD can be increased to very high levels or decreased to minimal levels. As a result, its contrast ratio is inherently high and far greater than that of standard flat panel displays or even conventional CRT monitors. |
| Luminance | Conventional electronic displays do not emit (or transmit) substantial amounts of light energy. As a result, they are primarily used in controlled lighting environments, and it is difficult to see them under bright ambient light conditions such as exist outdoors. In a see-through mode, the VRD can be controlled to allow the user to see an image that matches the brightness levels of the ambient light conditions. |
| Color Range | CRTs and miniature FPDs are able to reflect only a portion of the total palette of colors visible to the human eye, and are limited in the degree of saturation they can achieve. Because red, green and blue light sources used in VRD technology emit highly saturated, pure color, the VRD can produce a range of possible colors and color fidelity superior to any other electronic display technology. |
| Power Consumption | Because they do not efficiently convert electrical energy into light energy, both back-lit FPDs and CRTs draw substantial power and are among the biggest battery consumers in portable devices that use them. As a result, their brightness is relatively low, because most of their energy input is wasted. VRD technology, by contrast, conveys virtually all of its generated light onto the retina, allowing bright displays with minimum power requirements. |
|
Scalability |
VRD technology is capable of improving performance more rapidly and with significantly lower research and development and tooling requirements than competing technologies. |
| Range of Application | Because the VRD delivers a truly general-purpose solution, without performance compromises or tradeoffs, it is appropriate for a broader range of applications than competing display technologies. This means that it can potentially achieve greater economies of scale necessary to drive costs downward and increase its appeal versus other display solutions. |
| Cost | Current flat panel displays use exotic and costly production techniques involving complex assemblies and critical tolerances. In contrast, the basic design of the VRD consists of subsystems that are very simple in their design and largely make use of established optical and electronic technologies. Investment in specialized manufacturing equipment is not required. Ultimately, VRD devices will be mass-produced at low cost. |
Image courtesy of HIT Lab, Washington University
The goals of the IVRD project are:
Image courtesy of HIT Lab, Washington University |
Image courtesy of HIT Lab, Washington University |
|
Prototype Characteristics: 640x480 60 Hz 50 degrees f-o-v RGB |
Emulator:
The emulator system requires a scanned light sources of the emulator to be modulated by passing the light through a 35mm slide. This provides us with the capability to vary the spot size at the retina, to vary the spacing between horizontal lines, to vary the spacing between pixels etc. More at HIT Lab |
Head Mounted Displays (HMDs)
"Helicopter pilots require information to support
time-critical (and often life-and-death) decisions," Swanson said. "If
that information is presented in a graphical and intuitive fashion, it
reduces the pilot's workload and can enhance visibility in degraded conditions.
A helmet-mounted display capable of presenting full-color graphical information
in both day and night flight operations has been the missing link to creating
an effective pilot-data interface. That ultimately could save both lives
and money."
Rick Rutkowski, Microvision's president and chief
executive officer, said, "The Army has a powerful vision: the ability to
overlay flight reference data, sensor imagery and weapons symbology on
[images from] the outside world." Such a versatile display capability is
expected to provide a significant performance boost to both aircraft and
pilot. "When you can also enable a pilot to see the normally invisible
'bloom' of a radar signature, or to project a 'pathway in the sky' in front
of him, and to superimpose wireframe or 3-D imagery onto the terrain, it
becomes even more powerful," he said.
Microvision delivered a monochrome green binocular
HMD system to the Army with a total field of view of 52° horizontal by
30° vertical with 1716 horizontal pixels by 960 display lines, and 1470
ft-Lm displayed raster luminance at the eye (see Figure 3). The system
weighs approximately 2 lb, including two display engines (one for each
eye), the helmet-mounting structure, and the optics to provide the large
display format. The system will project the images in a "see through" fashion
—the pilot will still be able to see the background scene, but has
the option of focusing in on the presented information.
According to Rutkowski, the Army's vision of the
virtual cockpit also includes a "what you see depends on where you look"
concept. As the pilot looks up and out of the cockpit, various types of
targeting, navigational or terrain overlays would appear. When pilots look
in a downward direction, they may see "virtual" instruments projected onto
the eye that literally replace many of the existing dials and multifunction
displays that are in cockpits today. Microvision has modularized the system
designed to permit delivery of a full-color 1280 x 1024-pixel biocular
head-mounted system three months later, as well as an HDTV (1920 x 1080
pixels) HMD to the Air Force by the year 2000 (see Figure 4).
 |
Wearable "augmented reality" displays Incorporated into eyeglasses, goggles or helmets, VRD technology will display an image that doesn't block the user's view but will instead superimpose a high-contrast monochromatic or color image on top of it. This ability can enhance the safety, precision and productivity of professionals performing complex tasks.
Interactive VRD
Wearable three-dimensional / interactive displays
VRD technology can be incorporated into eyeglasses, goggles or helmets
to create a stereoscopic, 3-d effect. These compact, high-resolution
displays can further enhance the visual realism of the interactive experience
to make the simulated environment more engaging.
Image courtesy of HIT Lab, University of Washington
Hand-held displays
Hand-held two-dimensional displays VRD technology
will be integrated into cellular phones and pagers, allowing users to tap
into business networks or the Internet to view e-mail, web pages, faxes
and files as if on a full-size desktop monitor. This ability affords greater
convenience and utility to users of these devices.
Image courtesy of Microvision
Medical applications (as referenced by Microvision Inc)
It has been estimated by the Community Services
for the Blind and Partially Sighted that 2% of the population in the United
States have low vision (20/400 or worse, legal blindness being 20/200).
People with low vision are not completely blind, but many are unable to
hold a drivers license and have a great deal of difficulty reading or watching
television. Possible causes include damage to the retina - central or peripheral
field degradation - and damage to the optical pathway - corneal damage/malformation
or lens damage. The HIT Lab develops these VRD that can provide more visible
images to people with low vision. Microvision capped 1998 by signing a
contract with the Wallace-Kettering Neuroscience Institute (Dayton, Ohio)
to collaborate on headgear for neurosurgery.
In its test screenings, "Patients without retinal
damage saw moving particles throughout their field of vision, but patients
with areas of retinal damage were actually able to paradoxically see their
own blind areas and localize them," a Microvision spokesman said.
"Using a pen-based computer program, patients were
asked to trace the borders of any visual disturbances," he said, "in effect
creating a digital map of their own retina, which accurately detailed the
locations of damaged tissue. The images were then compared with the 'gold
standard' — photographs taken from a fundus camera — with outstanding results."
A fundus camera captures images of the internal surfaces of hollow organs
like the eye.
The VRD-based workstation can "screen for retinal
damage in asymptomatic patients within virtually all of the visually significant
retina," William Freeman, professor of ophthalmology and administrative
director of the retina service at Shiley, reported at the meeting. "This
rapid, non-invasive screening test has the potential to be an excellent
addition to the clinical tool set of primary-care health providers to rapidly
screen for retinal damage and diseases. It will also allow for community
screening of underserved populations to detect vision loss." In the past
year, a Shiley spokesman said, the workstation screened 58 patients with
a variety of retinal diseases, including AIDS-related CMV retinitis, diabetic
retinopathy, ocular melanoma, blood-vessel disorders, retinal detachment
and macular degeneration.
Image courtesy of Microvision
The approach can also be adapted to image projection systems. The applications for VRD technology are varied—HUDs, color projections systems for entertainment or flight training simulators, etc. A key area for continued development is an image display system that can augment and enhance a person's task performance.