LCOS and AR/VR
Companies are delivering different renditions of augmented-reality and virtual-reality products to the market, including Google Glass, Microsoft Hololens, ODG R-7, Magic Leap One, Oculus Rift, HTC Vive, and the Dell Visor. All these products have different specifications and applications. This article will discuss the differences between VR and AR, and how LCOS microdisplays can play an important role in making these applications more viable.
by Po King Li
VIRTUAL-REALITY (VR) headsets have existed for more than 30 years, but major technical breakthroughs that provide a new experience to users have only arrived in the past few years. These breakthroughs include high-resolution thin-film transistor (TFT)-LCD and OLED panels, power graphics processing units (GPUs), cloud computing, and 3D rendering software.
Augmented-reality (AR) devices are cousins to VR devices, but with different DNA. There are several differences, but the main one is that VR provides an immersive experience for users, who experience the virtual world the system provides and are not able to see the environment around them (Fig. 1). AR involves a wearable device that allows users to view the surrounding environment with an overlap of digital content. For example, a user can see his friend’s face with a digitally generated hat; an architect is able to see her newly designed building at the construction site before construction has begun; a field technician can follow 3D instructions from the other side of the world to fix a copy machine; a doctor can perform heart surgery with directions from an expert in another country (Fig. 2).
Fig. 1: The Oculus Rift is a well-known VR headset that enables users to experience immersive applications in which they see nothing of their physical surroundings. Source: Oculus
Fig. 2: A popular AR device (which its maker refers to as a mixed-reality device) is the Hololens from Microsoft. The headset at left enables augmented-reality applications, including remote plumbing repairs, as shown at right. Source: Microsoft
AR glasses require a complex optical module to deliver the digital content overlap. We will briefly introduce these different optical architectures later in this article. The following are characteristics of AR systems:
• AR provides a see-through optical device for the user. When the AR glasses are off, the user is still able to see the environment surrounding her. AR glasses are not an immersive experience.
• AR glasses are a stand-alone device. They do not usually connect to any PC or game console. They have enough processing power to render the 3D imagery.
• The user experiences digital content as an overlap with the real environment.
• Currently, most AR applications are for enterprises, such as 3D models for architecture, warehouse management, and medical and educational applications. But more gaming applications are coming.
In summation, VR provides a virtual world to the user, whereas AR adds digital content to a real-world view. This is the key difference between these two product categories.
Liquid Crystal on Silicon
Liquid crystal on silicon (LCOS) is a well-known microdisplay technology that is widely used by AR headset designers due to several key advantages that we will discuss. The liquid crystal is sandwiched between a layer of glass and a silicon wafer (Fig. 3). The silicon wafer’s top metal layer has two key functions: First, it is a mirror to reflect the light, and second, the mirror’s voltage drives the liquid crystal, twisting it in order to create an image. When the polarized light reflects from the mirror, the light can project through the optical system so the user can see the image.
Fig. 3: A cross-section of LCOS includes, from top to bottom, a glass plate, the liquid-crystal materials, and a color-filter layer based on a silicon substrate.
LCOS has various applications in projectors, head-up displays (HUDs) for cars, and AR glasses. Other useful LCOS applications are in phase modulation for communication applications and holographic displays. (It would take another article to discuss phase modulation and LCOS.) LCOS projection systems offer one of the best image-quality visual systems available. Sony and JVC’s top-of-the-line home theater projectors both use LCOS as the display source. Since LCOS is based on a silicon design, there is no limit on resolution. Both 4K and 8K projection systems have been implemented with LCOS.
There are several different types of these systems (two appear in Fig. 4):
• The three-panel LCOS system is used mostly for home theater projectors. It uses three LCOS panels, each projecting in red, green, and blue light provided by the optical system. The system uses a lamp, LED, or laser as a light source.
• The color-sequential single-panel system has been used in some AR glasses. The system consists of one LCOS panel, and the system projects red, green, and blue color fields sequentially. The optical system sequentially provides red, green, and blue light to the LCOS panel.
• The color-filter single-panel system integrates a color filter on the top metal layer so that the optical system requires only white light as the input. This design uses the color filters on the LCOS panel to create the colored image.
Fig. 4: At left is a color-sequential LCOS optical system, and at right is a color-filter LCOS optical system.
Below are major reasons for adapting LCOS as an AR display solution:
Best image quality. As we mentioned in the last section, the best home theater projectors use LCOS. With LCOS, the user is not able to distinguish the pixels – there is no “screen-door” artifact. The picture is a smooth image.
Pixel size. LCOS offers the smallest pixel of all the displays; the pixel size can be as small as 2.5 um, which makes possible a 4K panel for each eye.
Flexibility. AR glass requires a complex optical system to combine the real-world image and the digitally generated image. There are various optical architectures on the market. Since optical architectures are distinctive, the requirements of the panel size, pixel size, and resolution are different for each design. LCOS offers the flexibility to fit a variety of designs. Other microdisplay technologies have limitations in terms of lead time, resolution, and pixel size. With regard to OLEDs, the smallest OLED pixel available is around 10 µm, but LCOS pixels can be as small as 2.5 µm.
Power consumption. Power usage is one of the most critical design parameters for wearable or portable devices, since the battery size is limited. LCOS is one of the lowest-power microdisplay technologies available.
Size. AR glass can be made into a wearable-sized device. LCOS is able to deliver a small panel with high resolution. Furthermore, Himax has offered alternative LCOS solutions called front-lit LCOS. Himax is the first company to create a front-lit LCOS optical imaging system of this type.
Luminance. The LCOS optical system for AR is a projection system. The luminance of the optical engine is dependent on the optical design. Some of the applications of AR glasses require outdoor environmental support, which require display luminance as high as 30K nits from the optical system (Fig. 5).
Fig. 5: The optical engine OE50 from Lumus uses 0.37 WXGA. Source: Lumus
Polarized light. Some of the optical architectures use a light guide as the eyepiece, and some of the light-guide designs use polarized light to govern the light direction (e.g., a polarizing beam splitter [PBS]). The LCOS display technology is suitable for polarized-light optical systems because of the liquid-crystal layer.
Optical Architecture for AR
There are several different optical architectures for AR glasses, summarized below into the following categories:
Prism type. This is a simple design that uses a polarizing beam splitter to reflect the image into the user’s eyes. The advantages of the prism design are its simplicity and high optical efficiency. One example of this architecture is Google Glass.
Projection system with combiner. An optical system projects the image onto a combiner. The user can see the digital content and the outside world through the combiner. The combiner has two functions: It reflects the projected image into the eyes and allows the eyes to see the outside world through the combiner as transparent glasses. Wearables company ODG makes products based on this architecture.
Light guide. The image is projected into a light guide, and the users see the image through the light guide. Examples of companies that use this approach are Lumus and WaveOptics.
Creating AR glasses is not a simple task, because many critical design parameters need to be considered, and they may conflict with each other. For example, in the case of the resolution of the display, higher is better, but will increase power consumption for the processor and the system, due to there being more pixels to render and calculate. Critical parameters for the optical module of AR glasses are: power consumption; size and weight; latency (motion to photon); color break-up; luminance; field of view (FOV); and eye box.
LCOS as a Solution
The color-filter LCOS used by Himax Display (the author’s company) provides simple solutions to some of the problems related to the above parameters; in particular, latency and color break-up. For any color sequential display, latency is a difficult problem to solve, because there is always a frame delay for converting the data from display data to color sequential data. This is achieved with an additional converter that converts the image data (RGB) into color-frame video data. (The applications processor [AP] cannot output display data sequentially yet, so the converter is needed between the LCOS and the AP.) For AR applications, latency (motion to photon) is very important, because humans turn their heads frequently. The user expects the contents to follow when and where her head is turned. Even one frame delay of latency at 120Hz refresh is too long for an AR application. Color-filter LCOS solves this difficult problem by taking away the need for conversion. It isn’t necessary to send color-sequential data to color-filter LCOS, and there are no data to convert. The display data will display onto the color filter LCOS immediately after they are received from the AP.
In the case of color break-up, since color-filter LCOS is not a sequential display, there is no risk of perceived color break-up. For normal color-sequential displays, designers need to increase the frame rates high enough to reduce the perceived color break-up, but in many cases, it can still be seen.
With regard to luminance, FOV, and eye-box issues, all these parameters are dependent on the optical system design. Since LCOS is very flexible for panel size and resolution, LCOS can support most of the optical architectures for different luminance, FOV, or ye-box requirements. Although the author cannot disclose the specification of current designs, since most of them are customers’ confidential information, it is fair to say that other microdisplay technologies are able to meet some of the results, but not all of them. For example, a typical AR system with an LCOS panel of .37 inches and a resolution of 720p can produce a 40-degree FOV at 30K nits luminance into an eye box measuring 10 mm × 10 mm.
For power consumption, LCOS occupies only a small percentage of the AR-system power budget. For example, a 1,080-ppi panel consumes less than 200 mW, compared to 2 W from the processor in the system. For comparison, a front-lit LCOS display can output 30K nits using 500 mW, whereas a micro-OLED requires nearly 1 W to achieve up to 1,000 nits.
In terms of size, the traditional optical engine for LCOS is not small (for example, 10-20 cc of volume, as shown in Fig. 6). This is why Himax has introduced a new front-lit LCOS optical system to reduce the optical engine size and weight. Front-lit LCOS integrates the biggest and heaviest items into the panel package – the PBS and the illumination system. This reduces the size of the optical engine by 40 percent, and the weight by 20 percent. Front-lit LCOS is based on Himax color-filter LCOS, which integrates the PBS and LED illumination system with the panel. Front-lit LCOS offers an improved solution for two of the most critical parameters for AR – weight and size. Users do not tolerate large and heavy glasses.
Fig. 6: The conventional LCOS optical engine with PBS (shown in the center of the photograph at upper left in a top view) is bulky compared to the Himax frontlit LCOS engine (shown in the lower left-hand corner of the photograph in a side view). The red-topped schematic image at lower left is a top view of the Himax engine architecture. The blue schematic at lower right shows the architecture of the conventional LCOS with PBS.
AR is likely to be an important new application in the near future. Every person may own a pair of AR glasses; the market could be similar to the smartphone market today. LCOS, and in particular, front-lit LCOS and color-filter LCOS, are display solution that will help solve AR design problems and help bring about the AR revolution. •
Po King Li is VP of marketing & sales, LCOS displays, at Himax Display. He can be reached at poking_li@himaxdisplay.com.