A Flexible Display Enables a New Intuitive User Interface

The authors developed a prototype for a flexible-display system integrated with a bend-input function that enables users to interact with the display by flexing it. The display enables users to feel as if they are operating objects on the screen directly with their hands. This interactive intuitive interface is suitable for simple operations in application software and opens up new possibilities for flexible displays to be used as user-interface devices.

by Hajime Yamaguchi, Tsuyoshi Hioki, Shuichi Uchikoga, and Isao Amemiya

FLEXIBILITY is generally considered an important part of the future of displays. In recent years, a great deal of effort has gone into developing suitable modes and manufacturing methods for flexible displays. Reflective and self-emissive modes such as electro-phoretic^1,2 and organic light-emitting diodes (OLEDs)^3,4 have often been used for implementing flexible displays. But what is the actual potential value of flexibility?

Thinness and lightness are obvious positive attributes that also relate to the portability for mobile use. The ability to create a curved display through a flexible medium is also of value for design [Fig. 1(a)]. However, we suggest that input functionality through bending should be added to the potential value of flexible displays. Such functionality has three features – integrated input/output (I/O), analog input, and three dimensionality – that compares favorably to those of conventional pointing devices as shown in Fig. 1(b). The interactive operation of objects is achieved through the integrated I/O. Continuous changes in the radius of curvature of flexible displays enable analog input. And three-dimensional input means operation of the out-of-plane axis (z) of displays is possible compared to that of two-dimensional input operation of the in-plane axes (x,y). The addition of these features makes flexible displays into intuitive user-interface devices, not just display devices.

Fig. 1: (a) Some well-known values of flexible displays include portability and the capability of being made into a curved design. Input functions add usability. (b) Input functions with pointing devices (left) involve a separated I/O, digital input, and two dimensions compared to a more intuitive interface for flexible displays that incorporates integrated I/O, analog input, and three dimensions.

Concepts for input functions using bending have been presented in the past. But the bended parts were not displays but plastic plates or other materials attached to conventional displays. One such concept is Gummi,⁵ a bendable computer proposed by C. Schwesig et al. (Sony CSL, Japan) in 2003. Some interesting works were also reported in 2008 when J. Scott et al. (Microsoft Research Cambridge, UK) fabricated a force-sensing prototype using an augmented Ultra Mobile Personal Computer (UMPC).⁶ An intuitive page-turning interface for e-books⁷ was presented by T. Tajika et al. (Osaka University, Japan). And G. Herkenrath et al. (RWTH Aachen University, Germany) proposed twisting and bending in mobile devices.⁸

However, we created a prototype for a flexible-display system that is actually integrated with the bend-input function. We have been able to demonstrate zooming in/out in Google Earth and page up/down functionality in PDF files by bending the flexible display. The interface of the prototype is interactive and suitable for basic operations of applications software.

Methods

We used an 8.4-in. SVGA TFT-LCD, which has thin glass substrates (less than 0.1 mm in thickness) as a flexible panel.⁹ For convenience, thin glass substrates were used in developing our prototype, although flexible displays based on plastic substrates have also been developed.^10,11 The panel is sandwiched between two polarization films. We also developed a flexible backlight unit that consists of a thin light-guide plate (0.4 mm in thickness), 24 LED chips, and optical films including reflection, diffusion, and prism sheets. A flexible bending sensor, which is a commercially available resistive type, was augmented so that it could detect both convex and concave bending. The bending sensor was then attached to the flexible backlight unit. The interface between the bending sensor and the PC was accomplished using an microprocessing unit (MPU) and an AD converter. A firmware program for the MPU was developed so that the flexible backlight unit with a bending sensor was recognized as an human interface device (HID) by Windows.

Flexible Backlight Unit with a Bending Sensor

Figure 2 (left) shows a photograph of our developed unit. Bending the flexible backlight leads to changes in resistance of the sensor. The amount of change depends on the radius curvature of the display. Convex and concave bending increases and decreases the resistance, respectively. So, the unit can detect both the amount and the direction (convex and concave) of bending simultaneously as shown in Fig. 2 (right). The radius of curvature of the flexible backlight unit reaches 50 mm.

Fig. 2: The developed flexible backlight unit with a bending sensor is shown on the left On the right, changes appear in the resistance of the bending sensor with regard to the radius of curvature of the backlight unit.

Flexible-Display System

Figure 3 shows schematics and a photograph of the system, as well as specifications for a flexible-display system with bend-based input function.

Fig. 3: A schematic of the entire system appears on the left, with details of the display and bending sensor in the photograph on the upper right. On the bottom are the specifications for the flexible-display system.

The flexible LCD panel, the backlight unit with the bending sensor, and the polarization films are enclosed in a flexible case so they can be bended as one. Resistance of the sensor is digitized by the AD converter and compared with that of the flat state of the sensor, or the flexible display, in real time by the MPU. The threshold value of the radius of curvature for the operation of application software is programmed into the firmware of the MPU, and the resistance of the bending sensor is affected by ambient temperature and usage history. A reset button is installed so that the resistance of the bending sensor is reset to the flat state of the flexible display when the button is pressed.

Operation of Application Software with Flexible-Display System

Our developed prototype for a flexible-display system can be bent in two directions, convex or concave. In both directions, bending states are continuously measured. Figure 4 illustrates the mapping of bending to intuitive interface operations with Google Earth and PDF files.

Fig. 4: Bending can be mapped to intuitive interface operations in order to, for example, (top) zoom into the topography in an application such as Google Earth and (bottom) move forward and backward through pages of a PDF.

In the case of Google Earth, the screen can be zoomed in/out when the flexible display is bent in convex/concave state from a flat state. The zooming is stopped when the display is returned to a flat state. Customizable events triggered by the maximum level of bending can be programmed into the firmware of the MPU. Such an event might be the backlight switching off so that users stop bending the display. The speed of zooming corresponding to the radius of curvature of the display can be also set in the firmware.Figure 5 shows continuous zooming of Google Earth by bending the flexible display.

Fig. 5: Continuous zooming in Google Earth can be accomplished by bending the flexible display. Convex (top) zooms in and concave (bottom) zooms out.

For the operation of a PDF file, the bending sensor is installed at the rear side of backlight unit near the corner. Page up/down is achieved by bending the corner of the flexible display in the concave/convex state. The screen view is retained when the display is returned to a flat state. The speed of paging in correspondence to the radius of the curvature of the display can be also set in the firmware. Users can therefore read PDF files and e-books in a manner similar to printed media.

This input-output integrated system enables users to feel as if they are operating objects on the screen with their hands. Analog characteristics can be realized with continuous changes in the radius of curvature of the flexible display. These features make flexible displays into intuitive user-interface devices, not just display devices.

Flexible Displays in Cloud Computing

Even if a display is flexible, information devices, which have processing and storage functions, cannot be flexible. How might flexible displays still be useful in this context? In recent years, cloud computing, in which processing and storage functions remain on the network while being accessed remotely, has been changing the computing world. Previously, we developed a separate system¹² that uses a flat conventional (non-flexible) display and exhibited it as a detachable display at CeBIT 2005 and International CES 2006.¹³In this system, a user needs only a handheld interface, wirelessly connected to the network. Such a device could be a flexible display with an integrated input/output function and, as such, makes a promising candidate for a cloud computing user interface.

New Possibilities

The authors' prototype of a flexible-display system integrated with a bend-input function consists of a flexible display, flexible backlight unit with bending sensor, and MPU connected to a PC. Zooming in/out in Google Earth and page up/down in PDF files have been successfully executed by bending the flexible display. The interface of this prototype is interactive and suitable for the simple operation of application software. It opens up new possibilities for flexible displays as intuitive user-interface devices.

Some of these possibilities include a tablet-like display that could be manipulated in new ways or other types of mobile devices that would allow users to zoom in and out of maps, floor plans, etc. Intuitive page turning for e-readers might be another possibility. That is the case for web-browsing, in which users are allowed to easily move backward and forward through web pages. Another example is moving through layers in CAD or graphics, which have the layer structure. And, as with many new display technologies, the interface's very existence may drive new applications as yet unthought of.

References

¹I. French, "Flexible E-Books," SID Symposium Digest 40, 100–103, (2009).

²E. Huitema, F. Touwslager, E. van Veenendaal, N. van Aerle, and P. van Lieshout, "Rollable Displays: From Concept to Manufacturing," SID Symposium Digest 40, 104–107, (2009).

³D.-U. Jin, J.-S. Lee, T.-W. Kim, S.-G. An, D. Straykhilev, Y.-S. Pyo, H.-S. Kim, D.-B. Lee, Y.-G. Mo, H.-D. Kim, and H.-K. Chung, "World's Largest (6.5 in.) Flexible Full-Color Top-Emission AMOLED Display on Plastic Film and Its Bending Properties," SID Symposium Digest 40, 983–985, (2009).

⁴K. Arihara, M. Kano, K. Motai, Y. Naitou, M. Kadowaki, H. Nakajima, T. Tsuboi, C. Kato, Y. Kishimoto, and H. Maeda, "Fabrication of flexible 4.7-in. QVGA AMOLED panel driven by In-Ga-Zn-Oxide TFTs with a flexible color filter," Proc. IDW '09, 1613–1616 (2009).

⁵C. Schwesig, I. Poupyrev, and E. Mori, "Gummi: A Bendable Computer," Proc. CHI04, ACM, 263–270 (2004).

⁶J. Scott, L. M. Brown, and M. Molloy, "I Sense a Disturbance in the Force: Mobile Device Interaction with Force Sensing," MSR-TR-2008-57 (2008).

⁷T. Tajika, T. Yonezawa, and N. Mitsunaga, "Intuitive Page-turning Interface of e-books on Flexible e-paper based on User Studies,"Proc. MM08, ACM, 793–796 (2008).

⁸G. Herkenrath, T. Karrer, and J. Borchers, "TWEND: Twisting and Bending as new Interaction Gesture in Mobile Devices," Proc. CHI08, ACM, 3819–3824 (2008).

⁹T. Hioki, M. Akiyama, M. Nakajima, M. Tanaka, Y. Onozuka, Y. Hara, H. Naito, and Y. Mori, "A Flexible 8.4-in. Color Low-Temperature Poly-Si TFT-LCD," Proc. IDW '02, 319–322 (2002).

¹⁰T. Hioki, K. Miura, S. Abe, M. Tanaka, Y. Onozuka, Y. Hara, M. Akiyama, and S. Uchikoga, "A 5-in. Flexible TFT-LCD Using Transfer Technique," Proc. IDRC '05, 346–349 (2005).

¹¹K. Miura, T. Hioki, M. Tanaka, I. Amemiya, and S. Uchikoga, "A Curved TFT-LCD with a Curvature Radius of 10 mm," Proc IDRC '06, 135–138 (2006).

¹²H. Yamaguchi and S. Uchikoga, "New Platform for Mobile Display," Proc. IDW '05, 1309–1312 (2005).

¹³http://www.toshiba.co.jp/about/press/2005_03/pr1001.htm; http://www.toshiba.co.jp/ about/press/2006_01/pr0501.htm •