An Overview of Optical-Touch Technologies

Recent breakthroughs have led to a resurgence in optical touch systems, which were first introduced in the 1970s. Developers have been able to address issues of cost, performance in high-ambient light, and form factor, to name just a few. This article details how these problems have been overcome and what the future holds for this technology, including a look at several completely new approaches to optical-touch systems.

by Ian Maxwell

FIRST DEVELOPED at Carroll Touch (now part of Elo TouchSystems) in the 1970s and now sold by numerous suppliers, optical- touch systems offer many advantages when compared to other touch technologies. Many in the industry believe that, if not for two considerable drawbacks discussed below, optical- touch technology would today be the dominant touch technology. Recent technology developments in optical-touch screens could pave the way to the renaissance of optical- touch technology as the dominant touch-screen technology.

Introduction

The conventional optical-touch system uses an array of infrared (IR) light-emitting diodes (LEDs) on two adjacent bezel edges of a display, with photosensors placed on the two opposite bezel edges to analyze the system and determine a touch event. The LED and photosensor pairs create a grid of light beams across the display. An object (such as a finger or pen) that touches the screen interrupts the light beams, causing a measured decrease in light at the corresponding photosensors. The measured photosensor outputs can be used to locate a touch-point coordinate. Usually, the controller scans through the array of photosensors rather than measuring all of them simultaneously; thus, this touch technology is sometimes called "scanning IR." In more advanced versions of the technology, each photosensor measures light from more than one LED, which allows the controller to compensate for light blockage caused by non-moving debris on the screen (Fig. 1).

This traditional type of optical touch has been used primarily in niche applications of the touch market. Historically, its broader use has been hampered by two factors: the relatively high cost of the technology compared to competing touch technologies and the issue of performance in bright ambient light. This latter problem is a result of background light increasing the noise floor at the optical sensor, sometimes to such a degree that the touch screen's LED light cannot be detected at all, causing a temporary failure of the touch screen. This is most pronounced in direct-sunlight conditions where the sun has a very high energy distribution in the IR region.

 

p26_tif

Fig. 1: A schematic representation of conventional optical-touch technology. Illustration courtesy of Elo TouchSystems.

 

In addition, conventional optical touch has not been adopted for small handheld touch screens (such as in cell phones and PDAs) due to a number of other technical reasons, including power consumption, mechanical packaging constraints, and resolution limitations which limit the system's ability to detect small objects such as PDA-style pens. Because of their much lower cost, other technologies such as analog-resistive technology have dominated the mobile-device touch-screen market.

However, certain features of optical touch remain desirable and represent attributes of the ideal touch screen, including the option to eliminate the glass or plastic overlay that most other touch technologies require in front of the display. In many cases, this overlay is coated with an electrically conducting transparent material such as indium tin oxide (ITO), which reduces the optical quality of the display. This advantage of optical touch screens is extremely important for many device and display vendors because devices are often sold on the perceived quality of the user display experience.

Another feature of optical touch which has been long desired is the digital nature of the sensor output when compared to many other touch systems that rely on analog-signal processing to determine a touch position. These competing analog systems normally require continual re-calibration, have complex signal-processing demands (which adds cost and power consumption), demonstrate reduced accuracy and precision compared to a digital system, and have longer-term system-failure modes due to the operating environment.

Yet another key advantage of optical touch is that there is normally no direct impact of a finger, pen, or other object with the touch-recognition hardware. This reduces the possibility of failure modes typically caused by impact failure, wear, or fatigue of the touch screen. This is also related to the requirement for low-pressure touch. In an optical-touch system, only interaction with the light beams is required – no force needs to be applied to the system for detection or activation.

Finally, optical touch is capable of implementing multi-touch, something most other touch technologies cannot easily achieve. Although multi-touch has not been widely deployed in the past, there has recently been renewed interest in it, driven by new devices such as the Apple iPhone that make multi-touch an integral part of the user interface.

Recent Technology Enhancements

New Components and Improved Signal Processing: Since conventional optical-touch systems were first developed, key components such as LEDs, photodiodes, and CMOS chips have improved considerably in performance and reduced drastically in cost. Technologies used to produce molded optics and algorithms for signal processing have also been developed and improved. As a result, conventional optical-touch technology has improved and has at least maintained its competitive position against other touch technologies, which are also undergoing continuous improvement.

Improved Optical System Design. More recently, companies such as Elo TouchSystems and IRTouch have attempted to solve the background or ambient-light issue for optical touch, primarily by using a combination of improved bezel design (aperturing), optical filtering, and more sophisticated signal processing to enhance the signal-to-noise ratio. As an example of the latter, the infrared LEDs can be modulated with a specific frequency, while output of the photosensors can be demodulated at only that frequency, thus reducing the impact of the unmodulated infrared light in sunlight. The latest product specifications of manufacturers allow maximum ambient light in the range of 75–100 klx, indicating that these techniques havebeen largely successful in eliminating optical touch's sensitivity to sunlight.

 

p27_tif

Fig. 2: The Apple iPhone with projected-capacitive touch (left) and the Neonode N2 cell phone with conventional optical touch (right). Photo courtesy of PenComputing.com.

 

New Types of Optical-Touch Systems

New component technologies and reduced costs in key components have enabled the recent emergence of a number of entirely new optical-touch systems. Combined with cheaper and more-sophisticated optical-system design tools, this creates the perfect conditions for the current total re-examination of how optical-touch systems are designed and manufactured.

There are two broad types of new optical-touch systems: those that rely on a light source to provide the light, which is interrupted to detect touch, and those that use ambient light and do not incorporate a light source. Additionally, these new systems can be divided into those that rely on an interruption of defined light beams, and those that use complex signal processing to determine touch position from an image of the space above the display. The remainder of this article examines these new optical-touch systems.

Neonode

Neonode has taken conventional IR touch technology, using LEDs and photodiodes, and essentially miniaturized it for use in handheld devices. In addition to using the technology in its own N2 cell phone, Neonode is also marketing it to other device makers. However, it is currently unknown whether this technology is being adopted by any other cell-phone vendors. The key challenge for this technology likely will be the high bezel height. Many cell-phone manufacturers are continually trying to create devices that are flush or near-flush on the top cover, and they also prefer displays that extend as close to the side edges of the device as possible (in order to maximize both the display size and experience for multimedia functionality). For example, consider the Neonode N2 and Apple iPhone shown in Fig. 2. It is immediately obvious that the iPhone's screen surface is flush while the N2's screen surface is recessed. Based on the measurement of a sample product, the bezel height on the N2 is about 1.6 mm (including the thickness of the housing material) while the iPhone has a bezel height of zero (flush). Other issues that may hinder Neonode's touch-screen technology in the cell-phone market are cost and power consumption, both the result of the high number of optoelectronic components (LEDs and photodiodes) in the device.

Another potential challenge for this technology and also the Apple iPhone is the limitation for use as finger-touch only. Asian manufacturers of smart phones prefer to include stylus-based touch inputs to support character recognition. The light-beam spacing on the Neonode N2 is relatively wide at about 2.5 beams per centimeter, so that a finger touch covers about nine light-beam intersections. This conserves power but makes the touch screen unusable with a stylus. Even if a large stylus were used, handwriting recognition would still be impossible due to insufficient resolution. For comparison, the conductive-trace spacing on the iPhone is relatively narrow at about seven traces per centimeter, so that a finger touch covers about 25 trace intersections. However, even with its higher resolution, projected-capacitive technology inherently supports only finger touch, which eliminates use with a stylus or even with gloves, so the comparison is moot.

NextWindow, SMART Technologies, and Others

Camera-based optical touch has been implemented by NextWindow, SMART Technologies, and at least one new startup.

NextWindow's optical-touch-screen tech-nology uses two line-scanning cameras (Fig. 3) located at adjacent corners of a display. The cameras track the movement of any object close to the surface by detecting the interruption of an infrared light source. The light is emitted in a plane across the surface of the screen and is reflected back at the cameras by retro-reflecting strips located along three edges of the screen. (Retro-reflectors reflect light back along a path that is parallel to but opposite in direction from the angle of incidence.) When a finger (or any object) touches the screen, the controller analyzes the images from the cameras and triangulates the position of the touching object. SMART Technologies' optical-touch-screen technology does essentially the same thing, except that it uses four area-scanning cameras.

 

p28_tif

Fig. 3: A schematic representation of NextWindow's camera-based optical-touch technology. Illustration courtesy of NextWindow.

 

While it is technically possible to use an optical-touch-screen technology without a glass touch surface, neither supplier does so because of the need to protect the soft (2H) surface of the LCD. These technologies represent an advance over traditional optical- touch screens because they have fewer active components and, therefore, should be lower cost and have a longer mean time between failures (MTBF). NextWindow markets its touch screens in sizes ranging from 12 to 120 in.; most applications to date are found on monitor-sized displays (such as in the HP TouchSmart "family computer") and large displays used in interactive digital-signage applications.1 Although the technology has a sufficiently high resolution and data rate to support handwriting recognition with a stylus, it is unlikely to be offered for displays below about 10 in. where palm rejection is not needed due to border width, cost, and power-consumption concerns. In general, camera-based optical touch is unlikely to be used in mobile devices in the near future.

Perceptive Pixel

Researchers at New York University have developed a new approach to large multi-touch screens that can detect 10, 20, or even more fingers. A new company, Perceptive Pixel, has been formed to commercialize the technology – although it has yet to take place – in applications ranging from interactive whiteboards to touch-screen tables and digital walls, any of which could be manipulated by more than just one person.

Perceptive Pixel's technology works by introducing IR LED light onto a glass or plastic rear-projection screen. The technology uses frustrated total internal reflection (FTIR), which means that touch is detected when a finger touches the glass surface and light is scattered away from the finger and detected by a photosensor normal to the glass surface.2 In Perceptive Pixel's implementation, the photosensor is a camera located next to the projector (see Fig. 4). Because the tech-nology is designed to be used only with rear-projection displays, it is not applicable to mobile devices.

Sharp, Toshiba Matsushita Display (TMD), and Others

Sharp, TMD, and LG.Philips LCD recently have all demonstrated optical-imaging touch systems that use the display itself as an optical-sensing element. These new LCDs integrate a light-sensing element (photodiode or phototransistor) into each LCD pixel, which allows the display to act as a large-array photosensor; with appropriate image-analysis techniques, it can act as a touch sensor or even a card scanner. A recent demonstration by Sharp included a 3.5-in. LCD with a photosensor resolution of 320 x 480 pixels and a scanning rate of about 1 sec for the entire display. Because this is an inherently digital technology, it has the capacity to recognize multi-touch events (Fig. 5).

As a touch-screen technology for mobile devices, one challenge for this technology will be signal processing under a variety of ambient-light conditions. Unlike simple touch screens, this technology requires a complex image to be analyzed to determine if a touch event has occurred. This requires more sophisticated, expensive, and power-hungry processors compared with simple touch screens. In addition, varying background-lighting conditions further complicate the image analysis. Another concern is speed. For example, it is often quoted that hand-writing detection requires a minimum of 130 frames per second (fps) of touch recognition in order to avoid perceived user lag. This type of processing speed may be a challenge for image-array-based touch technologies implemented in low-power mobile devices.

 

p29a_tif

Fig. 4: A schematic representation of Perceptive Pixel's optical-touch technology based on frustrated total internal reflection. Illustration courtesy of Perceptive Pixel.

 

p29b_tif

Fig. 5: Sharp's 3.5-in. LCD with in-cell light-sensing optical touch. Photo courtesy of Nikkei Business Publications Tech-On!

 

The displays used in mobile devices vary greatly in size, aspect ratio, and resolution, with no real standardization by manufacturers. Therefore, in making a sensor built into each display, high costs associated with the design and NRE of the more complex touch-sensing LCDs are to be expected. In addition, these LCDs probably have lower pixel-aperture ratios, and hence may not be as bright as similar displays without the touch sensor built in.

RPO's Digital Waveguide Touch

RPO's Digital Waveguide Touch (DWT) is an optical-touch system that expands on the basic concept of the traditional IR system. This system uses one or two low-cost LEDs to provide a managed light source (effectively a planar sheet of IR light) projected from two adjacent bezel edges, and then utilizes polymer optical waveguides at the other two adjacent bezel edges to channel the light into separate 10-μm channels leading to a small photosensor array (Fig. 6).

This improvement on conventional IR touch effectively addresses all known shortcomings of the latter as follows.

Because the positioning of the optoelectronic components (LED and sensor) has been decoupled from the bezel of the display, the impact of the touch system on the bezel height and width is greatly reduced compared to conventional optical-touch systems. RPO has working demonstrators with only 2 mm of touch system outside of the active area of the display and a profile height of only 0.5 mm from the top of the protective display cover (lens) to the inside surface of the device's housing.

DWT has a much lower component cost due to having only one or two LEDs and one photosensor chip and a much higher resolution due to the "digitization" of the receive-side optical signal into separate optical waveguide channels, which are independently detected by individual pixels at the photo-sensor array. Hence, pen detection and handwriting recognition are possible.

Ambient-light conditions are not an issue due to the use of new approaches to filtering and aperturing and because of the small size of the optical waveguide receiving channels.

The key enabler for this technology is the availability of low-cost lithographically printed polymer optical waveguides developed by RPO. The company uses LCD-like process tools to deposit wet films and processes these films by using direct photo-patterning, followed by solvent development. While it sounds very simple, it took many years of developing the polymer materials and the manufacturing process in order to attain waveguides of sufficiently high resolution that also meet all the requirements of yield and durability. In addition, the optical system design used by RPO is quite sophisticated, but has resulted in a physical system that is simple and cheap to assemble.

This system was demonstrated by RPO at Display Week 2007, where multi-touch built into a PDA device was shown. DWT is now being placed into products in cooperation with a number of lead customers. In principle, this system can be utilized for a display of any size, but RPO is initially targeting small–to– medium-sized displays in consumer electronics and automotive applications.

Conclusion

It is likely that each of these new optical-touch technologies will address niches of the large and growing touch-screen market. Barring any technology shortcomings, we expect that these optical-touch technologies will provide key benefits over other competing touch technologies. Together, these new technologies, if grouped together as "optical-touch systems," could ultimately capture a large share of the total touch-screen market.

Of particular interest is the sudden growth of touch screens in handheld devices, driven by the Apple iPhone, other smart phones, GPS handheld devices, and personal media players. Of the above technologies, Neonode, Sharp, TMD, and RPO are explicitly targeting this space and wish to compete with the incumbent resistive and projected-capacitive touch technologies.

References

1"Introducing the NextWindow 1900 Optical Touch Screen: A NextWindow White Paper" (2007).
2J. Han, "Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection," Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST), (2005) (ACM 1-59593-023-X/05/0010). •

 

p30_tif

Fig. 6: A schematic representation of RPO's Digital Waveguide Touch.

 


Ian Maxwell is the Founder and Executive Director of RPO, Inc. He has founded a total of seven start-ups and is a Venture Partner with Allen & Buckeridge Venture Capital. He can be reached at 18 Bulletin Place, Sydney NSW 2000, Australia; telephone +61 (2) 6125-4968, e-mail: i.maxwell@rpo.biz.