Tablets and Other Handheld Display Devices for Medical Imaging: An Image-Quality Perspective
A research team predicts that the next generation of handheld displays might enable on-demand viewing of medical diagnostic images – available anywhere, anytime.
by Aldo Badano, Asumi Yamazaki, Peter Liu, and Wei-Chung Cheng
THE use of handheld viewing devices in medical-imaging applications has seen a tremendous increase in the last couple of years. Applications of current interest range from using the handheld as an aid in the acquisition of patient images in remote locations1 to the primary and secondary physician consultation of medical images from a variety of imaging modalities.2 Imagine being able to view and refer to diagnostic images of yourself or a family member on a handheld device while consulting with your physician. Despite the limited diagnostic utility of such a scenario, prompt access to images from your electronic medical record might contribute to your understanding of treatment options and thus help you make more informed decisions about alternative procedures. You would also be able to use your handheld device to share medical images with specialists in order to request second opinions. Yet another advantage of being able to view medical images on handheld devices might be when physicians need to make decisions within a limited time frame and do not have immediate access to a medical-grade workstation for image interpretation.
These scenarios, although not quite a reality as of today, will soon become the norm. The display industry is working toward providing handheld display technology that is capable of offering all the relevant medical imaging data. At this point, interpretation of the data will no longer be hindered by limitations in device image quality.
Interest in the above applications has been fostered by the availability of high-quality handheld display devices with higher pixel density, lower noise, and wider color gamut. Amidst these improvements, however, current users are faced with display characteristics that differ substantially from dedicated medical workstation displays. Moreover, and of particular relevance to this article, these characteristics differ substantially among handheld devices. It is then of great importance to the medical-imaging community to understand the benefits and limitations of handheld devices for the viewing of diagnostic medical images from the perspective of image quality and its effect on the detectability of disease conditions and abnormalities in a patient’s image data.
Recent Studies
Studies of diagnostic accuracy for handheld devices typically compare the diagnostic performance of a set of human readers on workstations against their performance on handheld displays. A number of recent reports suggest that for some devices and some visual tasks associated primarily with less-demanding areas of medical imaging, diagnostic performance with handheld displays is comparable to the existing practice of reading images on workstations or dedicated medical displays. For instance, McNulty et al. investigated the diagnostic accuracy of a tablet computer (first-generation iPad) in comparison with a Digital Imaging and Communications in Medicine (DICOM) calibrated or secondary-class LCD in the case of interpreting computed tomography (CT) and magnetic resonance images in emergency exams3 and suggested that tablets can be considered useful aids in the initial image interpretation stages when medical displays are not available. Another recent paper by Christopher et al. compared recommendations from ophthalmologists using a first-generation iPad with those made using a desktop display and found that the recommendations were similar.4
In a recent study by John et al., tablet computers with larger screens, high pixel counts, and touch-screen interfaces were found to be advantageous compared to mobile-phone devices for viewing radiological images.5 (The study also noted that tablets suffered from software instability and were of limited use for image manipulation such as zooming, panning, and annotating due to their small size.) McLaughlin et al. compared a tablet computer (first-generation iPad) with a diagnostic 2-Mpixel monochrome LCD and found no reporting discrepancies. Similar results were described by Johnson et al. on a similar experiment comparing radiologists’ interpretative performance of computer tomography (CT) images on the tablet to interpretation on a conventional PACS display.6 In addition, a similar recent study by Park et al. examined next-generation tablet computers (second-generation iPad) as teleradiology tools for evaluating brain CT7 and found that clinicians using tablets with a stable Internet connection could provide reliable remote evaluations.
As these previous studies demonstrate, experiments with human subjects and clinical evaluations seem to indicate that handheld medical image viewing can in some cases be as reliable as readings performed with dedicated medical monitors. However, many of these studies and their comparative findings are limited to specific device models and to specific viewing conditions that would not always be representative of actual ambient illumination conditions where these devices are utilized. Although the image-quality characteristics of medical workstation displays have been extensively documented (see, for instance, Ref. 8), handheld display devices have not yet been fully characterized in terms of spatial resolution, spatial noise, luminance response, and reflectance for various sizes and technologies, including LCDs and OLED displays using a consistent measurement methodology. Which of the many aspects of display device performance are more relevant for medical image viewing applications?
Image Quality: What Matters?
Among the display characteristics that need to be considered for evaluating image quality in a handheld display device, the ones with the most significant direct effect on performance are luminance response, spatial resolution, noise, and reflectance properties.
Luminance Response
The performance of a display device depends strongly on its ability to represent image values in a manner that is close to optimal and consistent for human reader interpretation of image data. Luminance performance is typically assessed using a photometric measurement device to compare the luminance output of the display device against the target model for image presentation, which in medical imaging is typically the expected contrast response given by the DICOM Grayscale Standard Display Function (GSDF) model based on a perceptually linear scale.
Table 1 shows the minimum and maximum luminance values and the luminance ratios because the handheld display brightness settings are fixed at maximum for a variety of devices. In general, we observe that OLED displays have higher luminance ratios due to the low minimum luminance values compared to that of LCD devices. It is worth noting that the medical monitors are calibrated to GSDF gray-scale mapping while the handhelds are calibrated to the original out-of-the-box settings. While the medical monitors comply with GSDF, all handheld devices exhibit a contrast response outside of the tolerance limit for secondary workstations given by Task Group 18 recommendations9 even at the maximum brightness settings. (TG18 was a national task force consisting of medical-imaging experts focused on the performance evaluation of electronic-display devices.)
Table 1: Specifications for the display devices tested in this study appear with their corresponding measured minimum and maximum luminance values and luminance ratios. The unit for luminance is cd/m2. LR is the ratio of maximum to minimum luminance.
The results of the analysis could significantly vary under the manually selected or automatic brightness setting.
Reflectance
Because handhelds are used in varying viewing conditions with differing amounts of ambient illumination, display reflectance is one of the most important features of the device performance that affects image quality. The deleterious influence of ambient light reflections has been documented for workstation medical monitors and has been dealt with by using a correction to the GSDF presentation look-up-table (LUT) that compensates the luminance scale.10 Reflectance is typically characterized by specular (Rs) and diffuse (Rd) components and measured under a hemispherical illumination geometry.11 Figure 1 shows diffuse reflection coefficients Rd for all devices as a function of the specular reflection coefficients Rs.
Fig. 1: Shown are the diffuse reflectance coefficients (left) and specular reflectance coefficients (right) for the devices tested in this study.
All handheld displays exhibit higher Rs than workstation displays, while some of them have relatively similar diffuse reflectance coefficients compared to workstation displays.
The reflectance measurements suggest that some handhelds suffer more in terms of image quality in the presence of higher ambient illumination. For instance, when used in a viewing environment with 50 lux at the face of the display, medical workstation devices will exhibit a diffuse reflected luminance in the range of 1.0–1.5 cd/m2, and some handheld devices will reflect up to 3 cd/m2, reducing (or negatively impacting) the useful range of luminance response.
However, these measurements rely on methods developed for workstation gray-scale monitors, and thus more studies are needed to fully understand the effects of ambient illumination on handheld image quality, including the effects of light-source spectral content and angular distribution.
Spatial Resolution and Noise
The description of the strength of signal-and-noise transfer at different spatial frequencies is a useful indication of the response of the display device to image content. Resolution and noise are typically characterized using the modulation transfer functions (MTFs) and noise power spectra (NPS) measured with an imaging photometer off a pattern on the screen. Using methods recommended in Ref. 9, we used a horizontal or vertical 1-pixel line pattern in a uniform background captured with high magnification by the photometric camera.
Figure 2 shows captured images displaying the 1-pixel line on each display. The subpixel shapes and layouts on the Phone2-OLED and Phone3-OLED with PenTile technology are different from those seen in LCDs. WS-5MPLCD, the only monochrome display in this study, has each subpixel at almost the same luminance value and the MTF is the closest to that of a square signal pattern.
Fig. 2: Each screen displays a 1-pixel line pattern that was captured by a photometric camera. Since these images are not exactly in the same scale, 0.1-mm-scale bars are indicated. Squares bounded by orange dotted lines show the 1-pixel region of the displays. The vertical lines reflect the horizontal resolution characteristics corresponding to the RGB direction.
Figure 3 shows MTFs as a function of absolute frequency in the horizontal and vertical directions for all displays. The horizontal MTFs of almost all tablet displays and the vertical MTFs of all handheld displays are higher than the MTFs of workstation displays. As seen in Fig. 1, the resolution characteristics of the display devices are affected by the symmetry of the pixel addressing scheme for the R, G, and B subpixels. In addition, the noise performance of handhelds is superior to that of the medical displays, as seen in Fig. 4, which shows 1D NPS in the horizontal and vertical directions for all displays.
Fig. 3: Modulation transfer functions are shown as a function of absolute spatial frequency in double-log-scale mode for the devices tested in this study. Solid (dashed) lines are for measurements in the horizontal (vertical) direction.
Fig. 4: One-dimensional noise power spectra are shown above for the devices tested in this study.
A detailed analysis of the results on resolution and noise should take into consideration the fact that handhelds are not seen at the same viewing distances as workstation medical monitors, nor are they seen in a “static” fashion, i.e., the angle of viewing is changing and can be adjusted by the viewer. All measurements reported in this article were taken at a normal viewing direction, perpendicular to the display face. This, along with possible degradation due to motion of the device while in use, are areas of current research in our laboratory.
Promising Performance, But Further Research Required
Although not covered in this article, other aspects of display performance are quite relevant for handheld medical image viewing, including temporal response, the effect of device motion on image quality, and the potentially rapidly varying viewing conditions. In summary, handheld displays can have good spatial resolution and noise characteristics compared to medical workstation displays. Since the luminance characteristics of the handheld display might not comply with the GSDF response, the displayed image contrast can be different from that of images radiologists and medical staff are familiar with from their workstations. Further investigations that rely on visual studies and take into account all relevant factors are needed to determine the reliability of the handheld device as an image viewing platform for demanding medical imaging applications.
Acknowledgments
Asumi Yamazaki and Peter Liu acknowledge funding by appointments to the Research Participation Program at the CDRH administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. DOE and the U.S. FDA. The mention of commercial products herein is not to be construed as either an actual or implied endorsement of such products by the Department of Health and Human Services. This is a contribution of the FDA and is not subject to copyright.
References
1C. J. Tuijn, B. J. Hoefman, H. v Beijma, L. Oskam, and N. Chevrollier, “Data and image transfer using mobile phones to strengthen microscopy-based diagnostic services in low and middle income country laboratories.” PLoS ONE 6: e28348 (2011).
2R. J. Toomey, J. T. Ryan, M. F. McEntee, M. G. Evano, D. P. Chakraborty, et al., “The diagnostic efficacy of hand-held devices for emergency radiological consultation. AJR 194, 469–474 (2010).
3J. P. McNulty, J. T. Ryan, M. G. Evano, L. A. Rainford, “Flexible image evaluation: iPad versus secondary-class monitors for review of MR spinal emergency cases; a comparative study,” Acad. Radiol. 19, 1023–1028 (2012).
4M. Christopher, D. C. Moga, S. R. Russel, J. C. Folk, T. Scheetz et al., “Validation of tablet-based evaluation of color fundus images,” Retina 32, 1629–1635 (2012).
5S. John, A. C. C. Poh, T. C. C. Lim, E. H. Y. Chan, and L. R. Chong, “The iPad tablet computer for mobile on-call radiology diagnosis? Auditing discrepancy in CT and MRI reporting,” J. Digital Imaging 25, 628–634 (2012).
6P. T. Johnson, S. L. Zimmerman, D. Heath, J. Eng, L. M. Horton, et al., “The iPad as a mobile device for CT display and interpretation: diagnostic accuracy for identification of pulmonary embolism,” Emergency Radiology 19(4), 323–327 (2012).
7J. B. Park, H. J. Choi, J. H. Lee, and B. S. Kang, “An assessment of the iPad 2 as a CT teleradiology tool using brain CT with subtle intracranial hemorrhage under conventional illumination,” J. Digital Imaging (2013).
8A. Badano, R. M. Gagne, and R. J. Jennings, “Noise in at-panel displays with subpixel structure,” Med. Phys. 31, 715–723 (2004).
9E. Samei, A. Badano, D. Chakraborty, K. Compton, C. Cornelius et al., “Assessment of display performance for medical imaging systems: Executive summary of AAPM TG18 report,” Med. Phys. 32, 1205–1225 (2005).
10M. J. Flynn, and A. Badano, “Image quality degradation by light scattering in display devices, J. Digital Imaging 12, 50–59 (1999).
11F. Zafar, M. Choi, J. Wang, P. Liu, and A. Badano, “Visual methods for determining ambient illumination conditions when viewing medical images in mobile display devices,” J. Soc. Info. Display 20, 124–132 (2012). •