Display Imagery vs. Real Imagery
by Martin S. Banks
A review article on 3D displays, by Banks, Hoffman, Kim, and Wetzstein (Annual Reviews of Vision Science, 2016), asked the reader to imagine a Turing test for displays. In this test, a person would view input that comes either from a direct view of the real world or from a simulated view of that world presented on a display. Then the reader would have to decide: is it real or is it imagery from a display? The display would pass the Turing test if the viewer could not distinguish which was which.
Today’s displays would clearly fail this test because no one would be unable to distinguish real from display. Many displays would fail because of limitations in spatial and temporal resolution. Many would fail because of limitations in color and the range of displayable intensities. And many would fail because they would not create a realistic three-dimensional experience or would not stimulate oculomotor function (e.g., accommodation and eye movements) appropriately. But very significant progress has been and is being made in each of these areas.
Several disciplines are involved in the design, construction, evaluation, and use of displays including materials science, electrical engineering, computer graphics, and human-factors engineering. But an understanding of human vision is proving to be crucial to the enterprise because in the end the goal is to provide the desired perceptual experience for a human viewer. And display and computer-graphics engineers cannot know how to do this without incorporating what is known about the visual system’s capacities, particularly its limitations.
There are numerous areas in which an understanding of the human visual system has aided and continues to aid the design and construction of more-effective displays, as well as the development of better algorithms in computer graphics. In this issue of Information Display, we sample a small subset of these areas by focusing on three specific topics in which knowledge of human vision has been intimately involved. In “Visible artifacts and limitations in stereoscopic 3D displays,” Johnson, Kim, and Banks describe how previous research on temporal and spatial filtering in human vision has been used to minimize flicker, motion artifacts, and distortions of perceived depth in stereoscopic 3D displays. They show how one can best utilize a display’s temporal and spatial properties to enable realistic, undistorted visual experiences. In “Head-mounted-display tracking for augmented and virtual reality,” Gourlay and Held review the latest techniques for implementing head tracking in virtual- and augmented- reality displays. As the accuracy of head tracking improves in space and time, we can provide the viewer of a head-mounted display the compelling experience of a stable visual world. In “Accurate image-based estimates of focus error in the human eye and in a smartphone camera,” Burge reviews research on how the human eye accommodates to focus natural images. He then shows how the knowledge gained from understanding how the eye does it has led to a more efficient algorithm for focusing a camera.
It is an exciting time to be involved in the design, construction, and evaluation of visual displays. For instance, the development of head-mounted displays for virtual and augmented reality has created great challenges, but also breathtaking opportunities. I look forward to the time when the perceptual experience that devices provide will be sufficiently realistic to give the Turing test a run for its money.