Developing Electronic Skin with the Sense of Touch
Although the concept may seem futuristic, research on electronic skin has wide-ranging practical impact.
by Ravinder Dahiya, William Taube Navaraj, Saleem Khan, and Emre O. Polat
MICROELECTRONICS technology and its subsequent miniaturization, which began almost immediately after the transistor was invented, have revolutionized computing and communication. However, miniaturization by itself is not the only way that electronics can evolve to add more value to our lives. Recent significant additions to the field include fabricating or printing electronics over large areas and on unconventional substrates such as plastics that flex, bend, and conform to 3D surfaces.1 Fuelled by a large number of applications, the pace at which flexible or bendable electronics is evolving is faster than ever before. This category includes an increasing list of applications ranging from foldable or conformable displays to smartwatches and wristbands to wearable electronics and even robotic skin.
Technological advances have inspired numerous multidisciplinary groups worldwide to develop artificial organs such as electronic skin and bionic eyes. These inventions could potentially bestow lost sensory feelings to disabled individuals, or provide useful sensory capabilities to machines to enable them to improve the quality of human lives, or enable innovative non-invasive means for early detection and monitoring of chronic diseases. Recent advances in electronic-skin technology have attracted increasing attention for their potential to detect subtle pressure changes, which may open up applications including health monitoring, minimally invasive surgery, and prosthetics.
Electronic sensory or “tactile” skin should help enable robots and similar machines (when these technologies are realized) to interact physically and safely with real-world objects. For example, a robot designed for healthcare could manipulate objects more effectively or help the elderly with greater safety, if its actions were based on feedback such as pressure or temperature coming from its body parts while it was physically in contact with the object or person in question.2 The recent incident at a Volkswagen plant in Germany, in which a worker was crushed by a robot,3 is similar to one reported in the early 1980s in Japan4 and reminds us of the importance of safety where robots are concerned.
Tactile sensing plays a fundamental role in providing action-related information such as sticking and slipping; vital control parameters for manipulation/control tasks such as grasping; and estimation of contact parameters such as contact force, soft contact, hardness, texture, temperature, etc. Tactile skin could also be indispensable for numerous medical diagnoses and surgical applications through haptic interfaces. For example, to feel the presence of tumors in underlying body tissue, the visual feedback of laparoscopic instruments is often insufficient; inserting a tool with tactile as well as visual feedback might prove more useful.2,5 In addition, although once a topic for science fiction, the notion of restoring sensory feelings to amputees can approach reality through solutions such as electronic or tactile skin enabled by sensitive electronics over flexible substrates (Fig. 1).2,5
Fig. 1: This conceptual image of a prosthetic device shaped like a human hand incorporates lightweight, ultra-flexible, high-performance, and cost-effective electronic skin.
Challenges Go Beyond Skin Deep
Realizing tactile skin is challenging, as this technology requires multiple types of sensors and electronics (e.g., to measure contact pressure, temperature, gas, chemical composition, etc.) on the flexible or conformable substrates and also over large areas.1,2 Many different sensors distributed over a wide area such as the entire body of a robot would require a complex signal-processing system capable of dealing with very large amounts of raw data.
One way to deal with big data is to create tactile skin with distributed computing, allowing information to be partially processed close to the sensing elements, then sending the smaller amounts of summary data results to higher-computational units. This would mean that in addition to distributed sensors, the tactile skin would require distributed electronics, and this is another challenge for developers of tactile skin. The miniaturization of electronics, which has followed Moore’s law since the 1960s, with more and more electronic components fitting into small die-sized areas, makes distributed electronics over large areas such as a robot’s body all the more difficult.
The recent trend of printed electronics may help meet this challenge.6 This includes printing transducer material directly on flexible substrates, as shown later in Fig. 3, or directly printing active electronic materials on the substrates.6 Large-area printed pressure sensors, radio-frequency identification tags (RFIDs), solar cells, light-emitting diodes (LEDs), transistors, etc. have been reported recently.
In the case of printed pressure or touch sensors, emerging trends include screen-printing or ink-jet printing of conducting or transducer materials including piezoelectric polymers such as P(VDF-TrFE) or composites of carbon nanotubes (CNTs) and PDMS (soft silicone) etc. While tools such as screen printers help bring sensor-related research closer to manufacturing, the printing of active electronics over large areas with performance at par with today’s silicon remains a challenge.
Factors that significantly contribute to the effectiveness of tactile skin in many applications including robotics are (a) sensor type and performance (e.g., sensitivity, ability to measure various contact parameters); (b) physical aspects (e.g., sensor placement, conformability, wiring); (c) data processing and hardware issues (data acquisition, signal conditioning, communication, power, compatibility with existing electronic and sensing hardware); (d) algorithms and software (processing data from tactile sensors distributed in 3D space, sensor representation, deciphering tactile information); and (e) engineering issues (integration of sensing structures with robots, maintenance, and reliability).
Touch Requirements
A rudimentary illustration of electronic skin can be seen in touch screens in wide use today. Touch screens detect contact location in the manner of a simple switch, i.e., ‘contact’ or ‘no contact.’ The physical act of touch is detected by measuring the change in resistance, capacitance, optical, mechanical, or magnetic properties of the material being touched. The various types of touch-screen technology include resistive, capacitive (self and mutual), acoustic, and optical.2
In terms of electronic skin, one of the more significant advantages of resistive-touch-screen technology – such as that used in the Nintendo DS game console – is that it can detect the touch or pressure of any object irrespective of the electrical property of the material and also offer protection against surface contaminants (both solids and liquids). In addition, resistive touch screens are cost-effective to fabricate and use very simple sensing circuitry. They require very low power. However, resistive touch screens need complex designs/architectures for implementing multi-touch – at a significantly high cost.
Resistive touch screens also have poor transparency and hence poor image clarity compared to that of capacitive touch screens. Capacitive touch screens allow multi-touch operations, albeit at the expense of increased per-cycle readout time and/or an additional processing block. Other touch-screen technologies such as acoustic touch screens and infrared touch screens also offer excellent transparency and durablity. However, they suffer from ambient light interference and are sensitive to liquids and contaminants.
Bendability Requirements
Touch screens based on resistive, capacitive, or acoustic technologies work for the most part on rigid and planar substrates, but these are giving way to bendable and foldable portable displays in the near future. One possibility for the future are ultra-thin bendable chips for drive electronics in displays and the bendable version of various touch-screen technologies.7 Replicating touch-sensing mechanisms in bendable displays will create research challenges. For example, the air gap that has to be maintained in resistive touch screens could lead to false inputs in flexible layers. Similarly, the ITO used to make conductive transparent films in capacitive touch screens does not lend itself to being flexed. It is a brittle ceramic-like material.
Graphene, which has the potential to revolutionize the technology, could offer another alternative for bendable touch screens. While graphene could power semiconductors and advance circuitry in the future, as technology improves, the most immediate implementation would be touch screens. Graphene is an excellent conductor of electricity and is especially strong for its light weight – it is estimated at 100 times more durable than steel. In addition to being strong and conductive, it is also flexible, transparent, and can be grown on large areas.8
Distributed Computing Requirements
Creating tactile skin involves more than just integrating or realizing sensors on flexible substrates. Unlike vision sensing, it is distributed and involves measurements of a multitude of touch parameters such as pressure, temperature, slip, etc. Given these requirements, it is unlikely to have a unique solution such as CMOS imagers, which revolutionized vision sensing and cameras. A number of different types of sensors would be needed to detect and measure the touch parameters. This also means that the data processing and computing requirements will be complex. In robots such as humanoids, the tactile data would come from different parts of the robot’s body, and schemes similar to the somatotopic maps in humans could be useful to effectively handle and utilize the tactile data. Such representations can also help in locating the tactile sensors’ three-dimensional space – which might otherwise be labor intensive and error prone because the positions of the tactile sensors change with the robot’s position. In terms of computing, a neuromorphic approach (circuitry that mimics the human neurological system) could be an interesting future development.
Prototype Efforts
A variety of approaches and designs are being pursued to develop an effective prototype of tactile skin. Early attempts to obtain bendable electronic skin followed the flexible printed circuit board (PCB) route, with off-the-shelf sensing and electronic components soldered to bendable PCBs that were made out of Kapton or thick polyimide. These approaches largely focused on measuring contact force or pressure. One such example was successfully used in a recently concluded European project dubbed ROBOSKIN, conducted by Professors Metta, Sandini, and colleagues (including the lead author of this article), which developed tactile skin for various robots including the iCub “humanoid” or human-style robot at the Istituto Italiano di Technologia in Genoa, Italy (Fig. 2).10,11 a
This semi-rigid skin is one of the most functional implementations of large-area touch sensors to date and was used to cover body parts with large curvatures, such as the arms of iCub. It was made of triangular modules, each having 12 capacitive touch sensors – obtained by placing a 5-mm-thick layer of soft and rubbery dielectric material on the electrodes (shown in Fig. 2), which were realized on the flexible PCBs. A conductive cloth covering the top of the soft dielectric material acted as the second electrode. When pressed by an object, the soft dielectric deformed, and the electrodes on its two sides came closer to each other, which led to increase in the capacitance. The change in capacitance was proportional to the contact force or pressure. Besides acting as the dielectric, the soft layer also provided the extra cushion, which led to improved safety. This capacitive skin on major body parts of i-Cub changed the research paradigm, whereby robotics research focus shifted from hands-based manipulation or exploration of objects to that involving exploiting multiple contacts with large parts of a robot’s body. This is similar to the manipulation of sandbags or big cardboard boxes by humans.
Fig. 2: The iCub robot (shown above) incorporates tactile skin on most of its body parts. Its e-skin was created with off-the-shelf electronic/sensing components integrated on a flexible printed circuit board. The images at the bottom show different sections of iCub’s body covered with flexible PCB-based skin. Images courtesy Giorgio Metta, IIT, Genoa, Italy.
Other examples of PCB-based semi-rigid robotic tactile skin include Hex-o-Skin, developed at TU Munich by Professor Cheng’s group12 and at the University of Tokyo by Professor Kuniyoshi’s group.13 These solutions also use off-the-shelf sensors and electronic components. The Hex-o-Skin has multiple sensors such as proximity, vibration, and temperature. The basic sensing module in Hex-o-Skin is implemented by an hexagonal PCB, which hosts a microcontroller for data preprocessing (e.g., clustering, feature extraction, etc.) and a transmission interface.
The printing of touch sensors on flexible substrates using screen printers or ink-jet printers is another route for obtaining tactile skin (Fig. 3).14 These printing tools have been employed to print conductive and transducer materials to develop various types of touch sensors, including resistive, piezoresistive, piezoelectric, and capacitive sensors.9
Fig. 3: Examples of screen-printed touch sensor cells include (a) P(VDF-TrFE) on polyimide and (b) MWCNT/PDMS sensors on PET.
With technology enabling ever more advanced robots, the electronic skin of the future needs to be far superior to what is possible today with off-the-shelf components. Interesting developments in this direction include electronic skin using organic semi-conductors with thin-film transistors based on active-matrix backplanes developed by Professor Someya’s group at the University of Tokyo,15 Professor Sekitani’s group at Osaka University, and Professor Bao’s group at Stanford University.16 Another group that has contributed to the field includes Dr. Stadlober from Joanneum Research. These organic semiconductors have favorable features such as low-temperature solution processing and inherent bendability. The pressure sensors based on organic semiconductors often integrate transducer material such as pressure conductive rubber with the transistors. The resistance of pressure conductive rubber when it is pressed results in a change in current through the transistors, which is then detected with associated electronics. However, transistors and sensors based on these materials are slow, due to low charge carrier mobility and the large channel lengths that are possible with current solution-processed technologies. They are also less stable. Effective utilization of electronic skin in applications such as robotics requires sensory data to be acquired and transmitted quickly (less than a msec), thus enabling a robot to react quickly.
In this context, electronic skin based on high-mobility semiconducting materials such as single-crystal silicon offers significant advantages. That is why our focus at the Bendable Electronics and Sensing Technologies (BEST) group at the University of Glasgow is on developing electronic skin using silicon and other high-mobility materials that can also be processed using existing micro/nanofabrication tools.
The major challenge in using silicon for conformable electronics is posed by its brittle nature – silicon cracks on bending. Furthermore, some of the fabrication steps for silicon-based devices require temperatures much higher than for flexible substrates such as plastics. We have overcome these challenges by using transfer-printing and contact-printing approaches. In the transfer-printing method, the silicon nanowires are carved out of bulk wafers using a top-down strategy and transfer printed to flexible plastic substrates. The high-temperature processing steps are carried out before transferring the wires to flexible substrates. The skin looks like a rubbery polymer that has tiny silicon nanowires on it; these lead to thin-film transistors and sensors. The transfer printing of silicon nanowires was motivated by research conducted at Professor Rogers’ group at the University of Illinois Urbana-Champaign.17 For the first time, at Glasgow, we have demonstrated the use of this technique to develop the wafer-scale transfer of ultra-thin chips onto flexible substrates. The ultra-thin chips are attractive for compact and integrated flexible electronics as well as future touch-enabled flexible displays.
Another alternative strategy is to grow silicon nanowires using a bottom-up approach and then carry out contact printing to realize electronic and sensing components on the flexible substrates. Professor Javey’s group at the University of California Berkeley has used this methodology to develop sensitive electronic skin.18 However, the actual use of these advanced tactile skin technologies in robotics has not yet been demonstrated. Perhaps this is because truly macroscale integration of electronic skin using the above advanced approaches is still a research challenge because uniformity over large areas is challenging.
The on-going EPSRC (Engineering and Physical Sciences Research Council) fellowship for the growth project “printable tactile skin” conducted by the BEST group at the University of Glasgow is currently working towards overcoming these challenges through a combination of novel approaches that involve the printing of electronics and sensors over large areas. Other alternatives for developing electronic skin include using low-temperature polysilicon technology, which has been explored for thin-film transistors for displays.
Although it may seem futuristic, research on electronic skin has wide-ranging practical impact. For example, today’s artificial hands have come a long way from the fictional (but based on real-world) example of Captain Hook. Using what’s called myoelectric linking, a prosthetic limb can pick up electrical impulses from any remaining muscle fibers on an arm, transmitting those impulses to articulating fingers and thumb. These prosthetic devices are continually being upgraded and remodeled to look and function as much like a real limb as possible.
As realistic as they may look, currently available prosthetic hands have physical properties that are still far from the characteristics of human skin – they are much stiffer, for example. Eventually, these advanced prosthetic devices must be up to the task of touching and being touched by other people, with as much realistic “feeling” as possible.
This goal is closer than ever with the sensitive synthetic skin being developed by the BEST group at the University of Glasgow as part of the EPSRC growth fellowship dedicated to printable tactile skin. The synthetic skin could lead to next-generation prosthetic arms with which users can feel a light touch, shake hands, and type naturally because the arm will send signals to the brain and in turn respond to brain signals. Further improving the experience will be smaller and more efficient batteries and lifelike materials that will more closely resemble real skin and be capable of more accurate communication between hand and brain. We are fortunate to be living in this exciting era and feeling the positive impacts of technology.
References
1R. Dahiya, “Electronic Skin,” AISEM Annual Conference Proceedings, XVIII, 1–4 (2015).
2R. S. Dahiya and M. Valle, Robotic tactile sensing: technologies and system (Springer, New York, 2013).
3http://www.theguardian.com/world/2015/jul/02/robot-kills-worker-at-volkswagen-plant-in-germany
4http://www.theguardian.com/theguardian/2014/dec/09/robot-kills-factory-worker
5R. S. Dahiya, “Epidermal Electronics: Flexible Electronics for Biomedical Application,” in Handbook of Bioelectronics (ISBN: 9781107040830), S. Carrara and Kris Iniewski eds. (Cambridge University Press, 2015).
6S. Khan, L. Lorenzelli, and R. S. Dahiya, “Technologies for Printing Sensors and Electronics Over Large Flexible Substrates: A Review,” IEEE Sensors Journal 15, 3164–3185 (2015).
7R. S. Dahiya and S. Gennaro, “Bendable Ultra-Thin Chips on Flexible Foils,” IEEE Sensors Journal 13, 4030–4037 (2013).
8E. O. Polat, O. Balci, and C. Kocabas, “Graphene-based flexible electrochromic devices,” Scientific Reports 4 (Oct 1, 2014).
9R. S. Dahiya, A. Adami, L. Pinna, C. Collini, M. Valle, and L. Lorenzelli, “Tactile Sensing Chips With POSFET Array and Integrated Interface Electronics,” IEEE Sensors Journal 14, 3448–3457 (2014).
10G. Metta, L. Natale, F. Nori, G. Sandini, D. Vernon, L. Fadiga, et al., “The iCub humanoid robot: An open-systems platform for research in cognitive development,” Neural Networks 23, 1125–1134 (Oct.–Nov. 2010).
11R. S. Dahiya, G. Metta, M. Valle, and G. Sandini, “Tactile Sensing – From Humans to Humanoids,” IEEE Transactions on Robotics 26, 1–20 (Feb 2010).
12R. S. Dahiya, P. Mittendorfer, M. Valle, G. Cheng, and V. Lumelsky, “Directions towards Effective Utilization of Tactile Skin – A Review,” IEEE Sensors Journal 13, 4121–4138 (2013).
13Y. K. Ohmura and A. Nagakubo, “Conformable and Scalable Tactile Sensor Skin for Curved Surfaces,” Proc. IEEE International Conference on Robotics and Automation, (May 2006).
14S. Khan, S. Tinku, L. Lorenzelli, and R. S. Dahiya, “Flexible Tactile Sensors Using Screen-Printed P(VDF-TrFE) and MWCNT/ PDMS Composites,” IEEE Sensors Journal 15, 3146–3155 (2015).
15M. Kaltenbrunner, T. Sekitani, J. Reeder, T. Yokota, K. Kuribara, T. Tokuhara, et al., “An ultra-lightweight design for imperceptible plastic electronics,” Nature 499, 458 (July 25 2013).
16M. L. Hammock, A. Chortos, B. C. Tee, J. B. Tok, and Z. Bao, “25th Anniversary Article: The evolution of electronic skin (e-skin): a brief history, design considerations, and recent progress,” Adv. Mater. 25, 5997–6038 (Nov 13, 2013).
17A. Carlson, A. M. Bowen, Y. Huang, R. G. Nuzzo, and J. A. Rogers, “Transfer printing techniques for materials assembly and micro/nanodevice fabrication,” Adv. Mater. 24, 5284–318 (Oct 9, 2012).
18K. Takei, T. Takahashi, J. C. Ho, H. Ko, A. G. Gillies, P. W. Leu, et al., “Nanowire active-matrix circuitry for low-voltage macroscale artificial skin,” Nat. Mater. 9, 821–826 (Oct. 2010). •