LCD In-Cell Touch

Imagine being able to touch the surface of any display with your finger or a stylus and have the location of your touch instantly identified down to the exact pixels. Imagine this happening with no cover glass or special coatings or any other obstruction in front of the display, and with minimal change inside the display. That's the promise of in-cell touch. The problem is that the promise remains mostly out of reach. This article explores that promise and its current status in detail.

by Geoff Walker and Mark Fihn

THE term "in-cell touch" generally refers to the implementation of a touch sensor inside the cell of a liquid-crystal display (LCD). While the term and technology have been applied to touch sensors integrated into plasma-display panels, electrophoretic (electronic paper) displays, and OLEDs, this article examines only the application in LCDs.

LCD in-cell touch currently exists in three forms, only one of which is physically inside the LCD cell. The three forms are as follows:

• In-cell: The touch sensor is physically inside the LCD cell. The touch sensor can take the form of light-sensing elements, micro-switches, or capacitive electrodes.

• On-cell: The touch sensor is an X-Y array of capacitive electrodes deposited on the top or bottom surface of the color-filter substrate. Strictly speaking, when the electrodes are on the bottom surface of the substrate they are physically inside the cell – but this is still usually called "on-cell" because of the type of electrodes. (This is a good illustration of the fact that the terminology for in-cell touch is still evolving.)

• Out-cell: This new term, coined in 2009 by AU Optronics Corp., describes the configuration in which a standard touch screen (usually only resistive or projected capacitive) is laminated directly on top of the LCD during module manufacturing. Unlike the other two, this configuration typically requires an additional piece of glass – even though it is technically possible to use a film–film resistive touch screen in this case.

Because these terms and the technology that they describe are quite new, there is still quite a bit of variation in their use in technical and marketing documents. Caution is advised while reading any relevant material; "on-cell" may often be used to describe something that is actually "out-cell," and vice-versa.

Out-Cell Touch

Out-cell is basically just the integration of a touch solution at the LCD-module manufacturer. This is not fundamentally different than the touch integration that is often performed by third-party integrators today. The major difference is that it is likely to be lower cost, which means that out-cell is probably going to become a general trend, one most likely to occur with technologies that are easy to integrate. Table 1 categorizes all current touch technologies in terms of the difficulty of integrating them as out-cell touch.

As shown in Table 1, resistive and projected-capacitive touch screens are the most likely candidates for out-cell integration. These two most commonly used technologies accounted for over 95% of the total number of touch screens shipped in 2009. Both are often attached to LCDs by third-party integrators, so it is easy for the LCD-module manufacturer to do the same. Projected-capacitive sensors are increasingly being made on converted color-filter fab lines, so LCD manufacturers have easy access to the technology. All of these factors are causing a number of well-known resistive and projected-capacitive touch-screen manufacturers to begin to work closely with major LCD manufacturers on out-cell integration. Among the other touch technologies, only optical seems to be gaining any acceptance from the LCD manufacturers in terms of out-cell integration.

In-Cell and On-Cell Touch Technologies

There are currently three different touch technologies being used in in-cell and on-cell touch. They are summarized as follows:

• Light Sensing (In-Cell): This technology, also called "optical," uses the addition of a photo-transistor into some or all of the LCD's pixels. The screen can be touched with a finger, stylus, light-pen, or laser pointer. The touch-sensing array can also be used as a scanner. A cover-glass can be used to protect the LCD's surface.

• Voltage Sensing (In-Cell): This technology, also called "switch sensing," uses the addition of micro-switches for X and Y coordinates into each pixel or group of pixels. The screen can be touched with a finger or a stylus, within the damage limits of the LCD's surface. A cover-glass cannot be used to protect the LCD's surface.

• Charge Sensing (In-Cell): This technology, also called "pressed capacitive," uses variable-capacitor electrodes in each pixel or group of pixels. The screen can be touched with a finger or stylus, within the damage limits of the LCD's surface. A cover-glass cannot be used to protect the LCD's surface.

• Charge Sensing (On-Cell): This technology, also called "capacitive sensing," is basically the same as today's projected capacitive. It uses an X-Y array of capacitive-sensing electrodes on the top surface of the color-filter substrate. The screen can be touched only with a finger. A cover-glass can be used to protect the LCD's surface.

Table 2 shows which LCD manufacturers are working on each of the three in-cell/on-cell technologies. This list, based on investigation done by the authors, is undoubtedly both incomplete and inaccurate because not all manufacturers are forthcoming about their in-progress research. The authors take full responsibility for all errors and omissions.

The theoretical advantages of in-cell touch have always seemed very attractive. These include the following:

• Minimal or no added size, thickness, or weight (and therefore no effect on the end product's industrial design) in order to achieve the touch function.

• Theoretically unlimited (controller-dependent) multi-touch functionality, since each pixel or group of pixels should be individually detectable.

• Conceptually very high touch-performance, including low parallax error (assuming no cover-glass), very accurate and linear touch-point data (due to the unchanging underlying pixel matrix), and potentially higher resolution than the LCD (through inter-pixel interpolation when a sensor is present in each pixel).

• Theoretically much lower cost for the touch function, since the changes in an LCD's manufacturing cost should be minimal.

In reality, all of these advantages have turned out to be compromised to some degree. The next several sections of this article delve into each of the three technologies and their advantages and disadvantages in more detail.

Light Sensing

The concept of putting a light-sensing element into each pixel, announced first in a press release by TMD in 2003, was the first in-cell technology to grab the world's interest. TMD was also the first to issue a press release describing the concept of automatically switching between sensing the shadow of a finger in bright ambient light and sensing the reflection of the backlight from a finger in dim ambient light. In those early days, light-sensing in-cell touch seemed to be destined to take over the touch industry and make all conventional touch screens obsolete. By the end of 2007, most of the other major LCD manufacturers (AUO, LG Display, Sharp, etc.) had demonstrated similar technology. A conceptual illustration of light-sensing in-cell technology appears in Fig. 1.

The first commercial product using any form of in-cell touch was developed by Sharp in 2009. The product, the PC-NJ70A netbook shown in Fig. 2, uses light-sensing in-cell touch in a 4-in. continuous-grain (CG) silicon, 854 ´ 480 touchpad LCD. This LCD performs the same functions as a conventional opaque touchpad, with the addition of stylus support, two-finger multi-touch gestures, and limited scanning (shape recognition). The product, retailing at around $800, is available only in the Japanese domestic market. Sharp has made it clear that the PC-NJ70A is a "technology experiment" rather than a high-volume commercial product.

The development of this product by Sharp illustrated several fundamental issues with light-sensing in-cell touch. These issues, which have generally been acknowledged and/or confirmed by other LCD manufacturers, are as follows:

• The original concept of using reflected backlight to sense touch in low ambient light does not work if the on-screen image is black. Sharp's solution to this problem was to modify the netbook's LED backlight to emit more infrared (IR) light (which significantly increases power consumption) and to modify the in-pixel light-sensors to be more sensi-tive to IR. Because the LCD is transparent to IR, this solved the problem of being unable to sense touch on a black image.

• In bright ambient light, it is difficult to distinguish between the shadow of a touching object and the shadow of a proximate (non-touching) object. In dim ambient light it is difficult to distinguish between a reflection from the backlight and a reflection from an external light source. In essence, using a photo-sensor to reliably detect touch over the range of full sunlight to total darkness turned out to be much more difficult than expected.

• Putting a light-sensing element in every pixel turned out to be impractical because it consumed too much of the aperture (reducing efficiency) and required too much processing power. Sharp's solution to this problem was to use one light-sensing element for every nine pixels. This reduces the impact of the problems but has the disadvantage of also reducing the touch resolution to the point where (a) scanning an image of something placed on the display is no longer practical and (b) the quality of digital ink (when using a stylus) is not good enough.

• The display function and the touch function tend to interfere with each other. Expressed by Sharp as "severe electromagnetic interference (EMI) problems," this prevents the netbook's touch function from operating as fast as a normal opaque touchpad. One of the authors spoke with an engineer who had worked on the development of the product at Sharp; the engineer said that on average, the touchpad worked at about 25% of the speed of a normal touchpad, which made it quite annoying to use.

• The amount of processing power needed to operate the overall touch function (e.g., process multi-touch gestures, run the scanning function, etc.) turned out to be much higher than anticipated. This, along with the addition of IR LEDs to the backlight, resulted in high power consumption, which noticeably shortened the netbook's battery life.

Sharp was not the first to recognize the "can't touch a black image in low ambient light" problem. Planar observed the same problem and published a paper in 2007¹ which proposed a novel solution: inject IR light into the edge of a cover glass and use frustrated total internal reflection (FTIR) to provide the reflected IR that's sensed by the in-pixel light sensors. This eliminates dependence on ambient light and the backlight. In 2009, Planar sold the intellectual property for this idea to a company whose identity remains a closely held secret. Whether it will be available for licensing to LCD manufacturers remains to be seen.

One fundamental problem that Sharp avoided by using CG silicon is that of the mobility of the backplane. The level of mobility needed to implement light-sensing in-cell technology limits the practical implementation to CG or LTPS, which, in turn, limits the maximum size of a light-sensing in-cell touch screen to about 20 in.

Fig. 1: In this conceptual illustration of light-sensing in-cell touch, a photo-sensing element occupies a portion of the aperture of one subpixel; the element is connected to X and Y control lines so it can be read individually. Source: DisplaySearch.

The net effect of all the above-described problems is that the development of light-sensing in-cell touch has slowed down a great deal since initial demonstrations. The current consensus among the major LCD manufacturers seems to be that commercialization of light-sensing in-cell touch is still relatively far in the future.

Voltage Sensing

The basic concept of voltage-sensing in-cell touch is the same as that of the emerging "digital-resistive" touch technology. In essence, an X-Y switch matrix is overlaid on the LCD. In the case of an external digital-resistive touch screen, the matrix is formed by patterning the normally continuous transparent ITO conductors on the substrate and cover sheet of an analog-resistive touch screen into intersecting strips. When a finger or stylus forces an intersecting pair of strips together, a circuit is closed (i.e., the voltage measured between the pair goes from an open-circuit voltage of a few volts to a closed-circuit voltage of zero volts).

In the case of a voltage-sensing in-cell touch screen, micro-switches are added to each pixel to form the switch matrix. When a finger or stylus pressing on the surface of the LCD closes one or more micro-switches, the same voltage measurement is made. In both cases, the controller isolates and drives each column separately such that multiple row circuit closures can be detected on one column without interference from other columns, thus inherently providing multi-touch. A schematic illustration of a voltage-sensing in-cell touch design is provided in Fig. 3.

The advantages of the voltage-sensing form of in-cell touch include the following:

• The relative simplicity of the controller (compared with the much more complex controller required for light-sensing in-cell) potentially allows integration directly into the LCD driver IC.

• The voltage-sensing switch matrix is totally independent of ambient and backlighting.

• A voltage-sensing in-cell touch screen with one sensor per pixel should be optimal for use with a stylus, since subpixel resolution can be achieved by inter-pixel interpolation.

The disadvantages of voltage-sensing in-cell touch include the following:

• A cover glass cannot be used because the surface of the LCD must be depressed in order to actuate the micro-switches. Because the polarizer (top surface) on today's LCDs typically has a pencil hardness of only 2H or 3H, this is a significant limitation. For example, AUO's specification on one of its voltage-sensing in-cell touch screens is only 100K touches at less than 40 grams. This is radically less than the typical 30-million/80-gram specification on a five-wire resistive touch screen. While a harder polarizer is an obvious solution to this problem, until there is more demand for touch, the polarizer manufacturers have no motivation to increase hardness and the LCD manufacturers have no motivation to use more expensive, harder polarizers.

• Pressing the surface of most LCDs causes significant liquid-crystal pooling, which is visually distracting. Eliminating the pooling can be accomplished by changing the cell-spacer structure and/or changing to in-plane switching (IPS), but there are intellectual property (IP), cost, and other restrictions on doing so. Again, until there is more demand for touch, there is little motivation for the LCD manufacturers to make such changes.

• Adding micro-switches decreases the aperture, which makes the LCD less efficient.

• A finite pressure is required to activate the micro-switches, which means that multi-touch gestures can be more difficult to perform than with zero-pressure capacitive touch screens.

• The maximum size of a voltage-sensing in-cell touch screen is limited by the resistive and capacitive (RC) loading of the connecting traces, as well as by the space required for the traces. Currently, the practical size limit is about 26 in.

Even though there are some significant disadvantages, the advantages of voltage-sensing in-cell touch make it fairly compelling relative to light-sensing in-cell touch. Nevertheless, the number of LCD manufacturers who are working on voltage sensing is the smallest of all three in-cell technologies. The authors are not sure why this is the case; we speculate that it may be due to some IP considerations related to digital-resistive technology.

Fig. 2: Sharp's PC-NJ70A netbook is the first commercial product to use any form of in-cell touch. A light-sensing, in-cell touch screen is integrated in the touchpad LCD, circled in red in the photograph. Source: Sharp.

Charge Sensing

Table 2 clearly indicates that charge sensing is currently the most popular of the in-cell touch technologies. The basic reasons are that (a) it is closely related to projected-capacitive touch technology, which has rocketed from obscurity to the number two spot in the touch industry since the launch of the iPhone in 2007 and (b) light-sensing in-cell touch (the former and earliest favorite) has turned out to be much harder to implement than expected.

As previously described, charge sensing is being developed in two forms: in-cell and on-cell. The primary difference between the two is that in-cell charge sensing relies on a change in capacitance caused by the user pressing on a moveable electrode, while on-cell charge sensing relies on the user's body capacity changing the capacitance between a pair of fixed electrodes.

In the in-cell configuration (Fig. 4), conductive column spacers located on the underneath of the color-filter substrate are added into each pixel or group of pixels. Each spacer has a corresponding conductive electrode on the TFT-array substrate. An electric field is established between each pair of electrodes, which produces a base value of capacitance (stored charge) for each X-Y location. When pressure is applied to the surface of the display with a finger or stylus, the movement of the conductive spacer causes the value of the capacitance between the electrodes to change. This change is measured by a con-troller and used to determine the location of the touching finger or stylus. Because the conductive spacer is shorter than the main column spacer, the capacitive electrodes cannot contact each other and create a short circuit.

In the on-cell configuration (Fig. 5), two patterned layers of transparent ITO conductors are deposited at right angles to each other on top of the color-filter substrate (underneath the polarizer) with an insulating layer (dielectric) between them. An electric field is estab-lished between the two conductive layers, which creates a base value of capacitance (stored charge) between each X-Y intersection. The capacitance of the human body to ground causes a finger placed on top of the polarizer to change the value of the capacitance between the intersecting electrodes under the finger. This change is measured by a controller and used to determine the location of the touching finger. The number and spacing of the electrodes determines the touch resolution.

While both of these configurations use the same principle of measuring a change in capacitance between transparent electrodes, the controller and the interface it presents to the host system are typically unique to each LCD manufacturer and may even differ between in-cell and on-cell.

The advantages of the charge-sensing form of in-cell and on-cell touch include the following:

• The base technology of determining the location of a touch by measuring changes in small values of capacitance is well-understood and becoming increasingly common due to the recent extremely rapid growth of projected-capacitive touch.

• Charge-sensing touch is totally independent of ambient lighting or backlighting.

• A thin (typically 0.5 mm) cover glass can be laminated on top of the polarizer to protect the top surface of an LCD with charge-sensing on-cell touch; this is a significant advantage over either charge-sensing in-cell or voltage-sensing in-cell touch.

• Existing color-filter fabs can readily be modified to support manufacturing charge-sensing on-cell touch screens.

The disadvantages of charge-sensing in-cell and on-cell touch include the following:

• Because the capacitance values being measured are very small (typically less than 1 pF), charge-sensing touch is very sensitive to electromagnetic interference. It can be very difficult to make a charge-sensing system work properly, especially as the size of the LCD increases or with noisy LCDs.

• A significant amount of processing power is required in the controller for a charge-sensing touch system. The controller for on-cell charge sensing can be very similar to that for standard projected capacitive, while the controller for in-cell charge sensing has more unique requirements due to the higher level of integration with the LCD.

• The resolution that can be achieved with charge sensing is typically lower than can be achieved with either voltage-sensing or light-sensing in-cell touch. However, this is less significant with on-cell touch because the touch screen can only be activated by a finger (an inherently low-resolution pointing device).

• The conductive spacer electrodes used in in-cell charge sensing can cause some loss of aperture, which reduces efficiency. Similarly, the ITO electrodes used in on-cell charge sensing reduce the transmissivity of the LCD by a few percent, which reduces efficiency.

• The touch object in in-cell charge sensing can only be a finger, which in many applications (e.g., mobile phones in Asia) is a significant limitation.

• In-cell charge sensing will not work with a cover glass, so the LCD can easily be damaged.

• Pressing the surface of most LCDs causes significant liquid-crystal pooling, which is visually distracting. This is most evident when no cover glass is used (in-cell) but the use of a thin cover glass (on-cell) does not completely eliminate the problem.

• The maximum size of a charge-sensing touch screen is limited by the resistive and capacitive (RC) loading of the connecting traces. In-cell charge sensing is also limited by the space required for the traces. Currently, the practical size limit is in the range of 22–24 in.

The only LCD manufacturer who has announced actual available LCD products using charge-sensing touch is AUO. Sizes include 3.0 and 4.3 in.

Technology Comparison

Table 3 presents a comparison of the characteristics of the three in-cell touch technologies. The red-yellow-green ratings (worst, middle, best) are relative within the three in-cell technologies, not within all touch technologies.

Fig. 4: In the in-cell configuration for charge sensing, conductive column spacers for each pixel or group of pixels are located on the underside of the color-filter substrate, and there is a corresponding conductive electrode on the TFT-array substrate. Pressing the surface of the display causes the capacitance between the electrodes to change. Source: LG Display.

Opportunities

At least two areas of current research still hold promise for in-cell touch technologies, as follows:

• Applications such as tablet PCs that benefit from multiple input capabilities (e.g., touch and stylus) may drive solutions that combine multiple in-cell technologies, creating a hybrid technology.

• Recent developments in multi-color subpixel structures may create some interesting opportunities in light-sensing in-cell solutions. An RGBW structure, for example, could enable the photo-transistor to be located in the white subpixel, which would improve sensing performance while reducing some of the shadowing and power-consumption problems.

Fundamental Issues

Previous sections have described the advantages and disadvantages of each of the three technologies being used in in-cell and on-cell touch. However, there are several higher-level issues that affect the entire in-cell/on-cell touch picture as follows:

• The sensor portion of in-cell touch is almost certain to cost less in terms of both manufacturing materials and process than conventionally applied touch-screen sensors. (The controller portion may be comparable to that of conventional touch screens.) However, the cost of modifying the backplane and/or frontplane of an existing LCD design to add in-cell touch sensing is at least $1–2 million, due to masking. Given the very large number of different LCDs that exist, it is unlikely that an LCD manufacturer will make these modifications throughout an entire product line. It is more likely that only selected LCDs used in high-volume products with a high demand for touch will be modified for in-cell touch. In other words, it seems unlikely that in-cell touch is going to become the standard for touch in all LCDs.

• The lack of standards for the interface to in-cell and on-cell touch functionality could be a significant impediment to the spread of the technology in the future. If LCD manufacturers develop their own unique interfaces to the touch function (which seems to be the case thus far), it will greatly limit the ability of device OEMs to second-source LCDs with in-cell or on-cell touch.

• There is no perfect touch technology; each technology has advantages and disadvantages. This is the reason there are so many different touch technologies. It therefore seems unlikely that the three in-cell/on-cell touch technologies are going to dominate the touch industry and completely eliminate all other technologies.

Conclusions

Although in-cell touch has been eagerly anticipated for more than 7 years, it still has some distance to go to reach full commercialization. Light-sensing in-cell is probably the furthest away because it has the most un-resolved problems. Voltage-sensing in-cell has potential, but there are no announced LCDs or end-user products that incorporate it. Charge-sensing in-cell and on-cell are the closest to commercialization, with a few announced LCDs that will probably ship in mobile phones during 2010. The focus of most of the LCD manufacturers working on in-cell touch is now on mobile displays because sizes larger than 10 in. have proven to be quite difficult and there are no clearly identified high-volume touch applications.

References

¹A. Abileah and P. Green, "Optical Sensors Embedded within AMLCD Panel: Design and Applications," International Conference on Computer Graphics and Interactive Techniques," ACM SIGGRAPH (2007). •