An empirical user study suggests that touch displays with haptic feedback reduce driver workload by making blind operation easier and faster.
by Thomas Vöhringer-Kuhnt, Kai Hohmann, Andreas Brüninghaus, and Ercan Tunca
INFORMATION DISPLAYS have become an indispensable element of the vehicle dashboard. In many vehicles, a center-console display is an essential part of the control functions for driver information systems. The size and visual quality of displays in instrument clusters are continually increasing, to the point where the display sometimes becomes the entire instrument cluster. There is no doubt that displays have become a core element of a vehicle’s human–machine interface (HMI).
Managing Distraction
Displays for the automotive HMI have to meet a long list of tough automotive-specific requirements, including a very wide range of ambient temperatures, sunlight readability, ruggedization, power efficiency, etc. Therefore, automotive display technology and vehicle integration are a special field within display technology. A different challenge relates to human distraction. Compared to smart phones (which are often used in vehicles but that is a separate area of concern), display technology in a car has to be designed in a way that minimizes driver distraction during a “secondary task” scenario, leaving enough visual, manual, and cognitive resources for the primary task of driving.
Successful utilization of display technology in the vehicle requires an appropriate control concept, high-performance hardware with suitable features, and ergonomics that facilitate “blind” operability to reduce eyes-off-the-road time. The question is how to optimize human interaction with the car through display ergonomics and the role of displays within a holistic HMI concept.1
Displays Support the Ideal of an Holistic HMI
The role of displays extends far beyond showing information. There would be no ergonomic way of controlling the wealth of functions in a modern car without display technology. In many cases, the displays are part of a control concept that combines hard keys and/or a central control element. In other cases, the display itself combines the two elements of information and control action. Touch displays in the center console, for instance, integrate the presentation of information and menu/button manipulation, which frees the driver of any mental transfer between separate places of visual information and command action. Any operation of touch displays has an immediate effect and maps to direct manipulation of objects in the analog world, while operating in-car devices indirectly (e.g., by rotary push buttons) involves a separation of action and reaction. Indirect operational paradigms also vary widely across car makes and models. Depending on the interior design and control concept of a specific car model, the element of touch can also be provided by a separate touch pad in combination with a display.
The benefits of displays as controller devices in vehicles reside in the high degree of freedom they offer with respect to interface design and in the experience users have with them in their daily lives. This level of freedom can be utilized within a holistic HMI concept. The term holistic refers to an approach across systems. This approach interconnects hardware and software components to dynamically support users‘ preferences, needs, conditions, and environment. A holistic HMI aims at a safer and more intuitive driving experience with enhanced joy of use. Drivers will be in constant interaction with their vehicle. They can enjoy intuitive ‘user-manual-free’ driving with display systems, control elements, and other components.
The Technical Side of Haptic Displays
Standard touch display technology requires constant visual control. In order for users to see that the desired function has indeed been activated, they depend on visual cues. Concepts requiring permanent manual-visual coordination are not state of the art for automotive applications, where looking away from the traffic situation is critically to be avoided.2 Therefore, developers at Continental have recently added haptic feedback to touch displays to provide an experience that users are familiar with from using conventional switches or buttons.
The haptic display shown in the cross section below (Fig. 1) was recently employed in a user study that tested the benefits of haptic feedback. Its actuators consist of an electromagnetic spool and a permanent magnet. As a finger moves across the display and passes the borders of soft buttons or presses a button, the buttons generate haptic feedback that can be clearly felt by the user. Also, the force applied by the user’s finger is measured constantly. The actuators are fitted behind the touch display and are located under the screen’s bonded layers (cover glass, capacitive sensor, display). The conditions of use in vehicles and the basic principle of active haptic feedback require an especially rigid structure. This technical solution can be scaled to larger display sizes to serve a variety of vehicle manufacturers´ requirements. An application of the haptic feedback display of 12.3 in. is already available as a customer demonstrator.
Fig. 1: This cutaway of a touch display shows components used to provide the haptic feedback.
As the user’s finger slides over the surface, search haptics enable the sensing of virtual borders between “buttons” (menu elements), without requiring the user to look at the display. Highly sensitive force recognition ensures that accidental touches can be distinguished from intentional operational commands. When the user presses his finger onto the selected “button,” a haptic signal will confirm that he has made a successful entry. The haptic signal is not visible to the eye because the deflection is only a tenth of a millimeter, using a very high acceleration to ensure reliable detection of the impulse. Type and intensity of the haptic feedback are freely configurable so it can be adapted to brand-specific OEM requirements. Personalization by the end-user is also theoretically possible. This specific demonstrator has four electromagnetic actuators underneath the moving display surface. The number of actuators depends on the display size and mass and the specified acceleration on the final cover glass; the minimum number of actuators is one. The user will sense the vibration/haptic impulse anywhere on the screen because the entire screen is actuated. The range of feedback elements is theoretically unlimited. It can vary from a soft single impulse up to a strong long-lasting vibration.
Usability and User-Centered Design
The above-mentioned haptic feedback application is the direct result of a user-centric design process that our researchers employ to ensure that an HMI supports drivers. Haptic feedback is a technical feature serving one purpose only – to make secondary task operation safer for the driver. The first promising observations with haptic feedback were gained with a touchpad input device.3 Adding haptic feedback to this control element reduced driver distraction while increasing user efficiency during control tasks.
To determine whether haptic feedback could improve the performance of drivers at the HMI level as well, we carried out a user study with a dedicated focus on blind operability of touch displays.4 A vehicle mockup, including driver seat, steering wheel and pedals, and an in-vehicle information system HMI presented on a Continental touch display system, was installed in the ergonomics laboratory at the Center of Competence HMI in Babenhausen. The primary driving task was to handle a vehicle on a country road, going at a constant speed of 70 km/h and facing oncoming traffic (instructions were given to keep the vehicle inside the right lane and follow the street accordingly). The traffic environment was provided by an open-source driving simulator software.5 We chose this setting because German accident statistics show that rural roads are one of the most dangerous driving environments.6 The simulated test duration was 45 sec for each trial, during which the driver was always given a secondary task. To complete the task, the driver had to choose one of four functions on a main menu, then activate one of six functions on a sub-menu, make a choice on a list of pre-filtered functional elements, and operate a numeric block with 12 keys. This action had to be carried out with haptic feedback activated and not activated in a randomized order parallel to the main driving task.
Haptic Feedback Study Results
Twenty-six test users (17 male, 9 female, average age 38.8 years, average annual mileage 20,000 km) carried out the 45-sec drive with and without haptic feedback, respectively. The driver performance (lane deviation from ideal path) was measured and the subjective assessment of the users themselves was scanned via a standardized questionnaire measuring intuitive use.7 As described above, the main criterion of driver performance was maintaining a safe trajectory throughout the simulation. In detail, the study revealed the following:
• There is a certain scattering in the control over the trajectory. Unfortunately, a few clear outliers among the test drivers watered down the results to some extent. However, as a tendency, it is still visible that haptic feedback reduces the amount of deviation from a safe trajectory (Fig. 2).
Fig. 2: Driving on a safe trajectory: Results are shown with haptic feedback (left) and without haptic feedback (right).
• The effectiveness of control operations with haptic feedback was higher than without. The number of correct actions was noticeably higher, while the number of erroneous actions was considerably lower. Haptic feedback reduced the error rate from 50% (without haptic feedback) to a mere 19.5% (Fig. 3).
Fig. 3: Effectiveness of carrying out the secondary task: Current operations are at left; wrong operations at right.
• Measuring the efficiency of carrying out the secondary tasks was also affected by a certain number of outliers, which made it difficult to pin down the actual effect (Fig. 4).
Fig. 4: Outliers mask the actual effect of haptic feedback on control efficiency: Results with haptic feedback are shown at left and without haptic feedback at right.
• Measuring the subjective workload showed a clear effect across all test drivers. They unanimously perceived the workload as much lower with haptic feedback. The scale ranged from 0 (0 no effort at all) to 150 (= maximum effort felt). Without haptic feedback, the average result was 40. With haptic feedback, the number dropped to 20. Statistically, this effect is highly significant (Fig. 5).
Fig. 5: Haptic feedback reduced the subjective workload by half: Results with haptic feedback (left) and without haptic feedback (right).
• The questionnaire results also advocate the use of haptic feedback. Without exception, details such as the low mental workload, easy achievement of goals, low perceived effort of learning, fast familiarizing with the control concept, and low perceived error rate confirm the satisfaction with haptic feedback across all test drivers. This effect is statistically also highly significant (Fig. 6).
Fig. 6: With haptic feedback (left), drivers are more satisfied with their control actions than without (right).
Adding up these findings – lower error rate, less stress, greater satisfaction – one can conclude that haptic feedback improves the performance of operating an automotive HMI and considerably reduces driver distraction.
Haptics Prove Helpful
The test users confirmed that the haptic feedback provided helpful guidance during the control action. In real-life driving conditions, the influence may be even greater as one can assume that test drivers will want to perform well “under observation” in a lab environment. In a more relaxed everyday driving situation where no one is observing the driver’s behavior, haptic feedback is likely to make more of a difference because one thing is sure – when a control action goes wrong, the driver will try to execute it again. To avoid making another mistake, the driver will likely pay more attention to the secondary task than during the first attempt. As a consequence, a lower error rate will translate into reduced driver distraction because it reduces the total number of control actions and therefore reduces the effort for secondary task completion.
Continental Interior Division has been examining and improving driver information technology for more than a century and has made advancing displays for the vehicle HMI a core focus since the 1990s. We plan to use the aforementioned research to continue our quest for optimization through haptic feedback designed to help minimize users’ distraction when they access displays for secondary tasks while driving. Continental has a haptic feedback application for a touchpad input device in mass production since 2014. It is currently built into the Mercedes C- and S-class vehicles.
Few things can be as annoying as a driver who approaches you not paying attention to the road ahead. In many cases, the cause will be a secondary task the driver is attempting to carry out. To minimize the visual share of a driver’s control actions, haptic feedback provides guidance through a sensory channel that is far less overburdened than the visual channel. In other words, haptic feedback confirms that “What you feel is what you get.”
References
1S. Rümelin, A. Butz, “How To Make Large Touch Screens Usable While Driving,” AutomotiveUI ‘13 Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, New York; ACM 48–55 (2013).
2M. Vollrath, A. K. Huemer, P. Nowak, O. Pion, “Ablenkung durch Informations-und Kommunikationssysteme.” Unfallforschung der Versicherer Forschungsbericht Nr. 26 (2014).
33M. Domhardt, E. Tunca, I. Zoller, P, Lotz, L. Schmidt, “Evaluation eines haptischen Touchpads für die Fahrer-Fahrzeug-Interaktion,“ E. Brandenburg, L. Doria, A. Gross, T. Günzler, H. Smieszek (Eds.), Grundlagen und Anwendungen der Mensch-Maschine-Interaktion: 10. Berliner Werkstatt Mensch-Maschine- Systeme (Berlin 2013) Berlin: Universitätsverlag der TU Berlin, 9–18 (2013).
44E. Tunca, T. Härder, L. Schmidt, T. Vöhringer- Kuhnt, D. Virant, “Blindbedienung auf einem Touch-Display mit und ohne haptische Rückmeldung in der Fahrzeuganwendung,” Gesellschaft für Arbeitswissenschaft e.V. (Eds.); Arbeit in komplexen Systemen. Digital, vernetzt, human?! 62. Kongress der Gesellschaft für Arbeitswissenschaft (Aachen 2016) (Dortmund, GfA-Press, 2016), (A.4.27).
5R. Math, A. Mahr, M. Moniri, C. Müller, “OpenDS – A new open-source driving simulator for research,” GMM-Fachbericht-AmE, 104–105 (2013).
6Statistisches Bundesamt. Fachserie 8 Reihe 7 - Verkehrsunfälle. DESTATIS (2015).
7J. Hurtienne, A. Naumann, “QUESI - A questionnaire for measuring the subjective consequences of intuitive use,” R. Porzel, N. Sebanz, M. Spitzer, M. (Eds.), Interdisciplinary College 2010. Focus Theme: Play, Act and Learn, Sankt Augustin: Fraunhofer Gesellschaft, p. 536 (2010). •
Thomas Vöhringer-Kuhnt, Kai Hohmann, Andreas Brüninghaus, and Ercan Tunca are with Continental (Interior Division) at Babenhausen (Germany). Thomas Vöhringer-Kuhnt can be reached at Thomas.Voehringer-Kuhnt@continental-corporation.com.