New display applications for the interpretation of medical images include mobile image viewers, true-color modalities, and 3-D technologies for improved diagnostic performance.

by Aldo Badano and Wei-Chung Cheng

AS imaging technology evolves, so do the display devices used for image viewing. For instance, two-dimensional image acquisition techniques are being replaced with information-rich, three- or higher-dimensional schemes. This increased dimensionality for image sets is already occurring in breast imaging and needs to be considered when the performance of two- and three-dimensional display devices is being evaluated. In addition, modalities that were traditionally analog, such as optical microscopy and medical photography, are now migrating to the digital domain, following a path similar to that of radiology in the early 1990s. Finally, mobile-device technologies are prompting practitioners to consider using them to improve their workflow and availability. Most notably, mobile-display systems based on portable consumer-grade devices are now increasingly considered as complements to stationary desktop displays for review when there is no access to a dedicated workstation. The implications for pre-clinical regulatory evaluation of these emerging applications are considered and discussed in Part 2 of this article, "Pre-Clinical Assessment of Medical Displays for Regulatory Evaluation."

Displays for Mobile Devices

The introduction of a large variety of technologies and the improvements in the image quality of mobile display devices have led to the consideration of such devices as medical image viewers. From an evaluation perspective, the applicable methodology has to incorporate elements associated with the handling, orientation, and movement of the device with the possible interactions of software components with the operating system (e.g., power manager, network connectivity, and other mobile-device functions), and, most importantly, with the variability in ambient-illumination condi-tions in which the device can be used (see Fig. 1).

(a)

(b)

Fig. 1: The visibility of image features in five mobile display devices shown in (a) degrades under ambient illumination effects as shown in (b). Note the degradation introduced by specular reflections in the visualization of gray-scale steps in the test pattern and the significant contrast reduction in the CT image due to diffuse reflections.

In addition, the viewing of images in hand-held devices has raised concerns about the need to quantify the display characteristics not only under typical static laboratory conditions, but also in the presence of movement. In this case, resolution properties measured by the modulation transfer function (MTF) might have to be extended to include a component from move-ment while noise metrics might need to consider the presence of fingerprint marks in touch-sensitive screens. For mobile displays, physical size and pixel-array size are not the only technological changes (newer versions of mobile displays might soon overcome the 1k x 1k barrier). Instead, key differences in terms of image quality might be more associated with variable ambient illumination conditions, variable device-user interaction, and the impact of smudges and fingerprints in touch-screen devices.

Displays for True-Color Modalities

The emergence of medical-imaging modalities that rely on color scales in conjunction with gray-scale images has determined that display devices have to be capable of accurate color mapping with limited quantization or cropping of the color scales. This is typically achieved by tuning the subpixel look-up tables that map image values into pixel intensities. However, when these look-up tables are designed to map color, the gray-scale performance of the device can be compromised. Therefore, for modalities where color and gray-scale fidelity is relevant, additional testing needs to provide evidence that both scales are within appropriate tolerances. Several techniques have been reported for achieving proper gray scale and color calibration (criteria to guide these techniques have not yet been established). In addition, other aspects of technical performance might have analogous elements for color (e.g., angular color shifts at 30° and 45° in the diagonal, horizontal, and vertical directions at center and edge spots and uniformity.

Another aspect to consider is the migration from optical or light microscopy to digital that has raised issues related to the display of the massive amount of information that can be captured for a single tissue slide. In addition to colorimetric issues, it is also important to note that the pixel-array sizes of tiled digital microscopy slides might raise additional needs for cataloging issues related to large displays (XQVGA or even wall-sized). Also worthy of consideration are the perceptual elements of characterizing image quality when the screen size is several times the distance from the reader to the screen (see Fig. 2).

Fig. 2: As displays for viewing digital microscopy images grow larger (background), issues of scale, accuracy, and time for pan/zoom operations in the computer hardware and software come to the fore.

In this scenario, the temporal characterization might require more detailed methodology that incorporates not only transition times but also moving-target techniques such as those that are part of the SID-sponsored ICDM document.¹

Displays for 3-D Modalities

Several three-dimensional modalities are being considered and utilized for detecting breast cancer as adjuncts to or replacements for full-field digital mammography examinations. Among them, breast tomosynthesis,² a limited-angle tomographic modality, has demonstrated that the additional volumetric information acquired using multiple projections holds promise for improvements in the detection of masses, in part by removing the masking effect of normal anatomical structures that can hide or mimic lesions. Other technologies being currently developed include dedicated breast com-puted tomography³ and stereo-mammography.⁴

Display devices for these emerging three-dimensional breast-imaging modalities are partially characterized by the lists provided in Part 2 of this article in terms of their spatial and gray-scale performance. However, since the radiologist is now faced with a new read-ing paradigm that requires browsing over many high-resolution (several million pixel) images (slices from the reconstructed volume), the temporal response of the display becomes a relevant characterization topic. Detailed temporal characterization of the display devices is useful in understanding the potential limitations of different display technologies and products in their ability to accurately represent the images for the readers. For instance, a more complete set of gray-scale transitions and a more descriptive metric that characterizes limitations of proposed solutions (i.e., overshoots in overdrive) can prove useful in demonstrating new product capabilities.

Another approach to visualizing the three-dimensional image sets is the use of three-dimensional display devices that are becoming available with increased performance due in part to advancements geared toward the consmer markets. In that sense, existing work regarding the use of two-dimensional display devices in medical imaging has to be revisited wherever appropriate. This extension of the physical measurements to a third dimension raises challenges, not so much in the development of the measurement methodology, but in bridging the experimental physical quantities (e.g., stereo crosstalk and stereo acuity) to the task performance in diagnostic imaging. Figure 3 shows a stereoscopic display used to view digital microscopy. Such reconciliation is complicated by the widely different technologies that are now being considered for three-dimen-sional displays, including stereoscopic, autostereo-scopic, volumetric, and time-sequential implementations, among others. A tentative list of pre-clinical tests of relevance for 3-D medical display products is presented in Table 1 of Part 2 in this issue. The list of tests that would be relevant for a particular 3-D medical-display product will depend on the visual task performed with the device and the associated claims made by the manufacturer.

Fig. 3: Stereoscopic viewing of digital microscopy and other imaging modalities is a means of improving the visualization of 3D datasets.

Innovations in display technology are making possible new ways of reviewing medical images. How soon these devices become available to physicians will depend on the availability of validated methodologies that can be used to demonstrate their advantages and in what ways these devices contribute to the early detection of diseases and ultimately to improved patient outcomes.

Acknowledgment

This article was submitted from the Division of Imaging and Applied Mathematics, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration. The mention of commercial products herein is not to be construed as either an actual or implied endorsement of such products by the Department of Health and Human Services. This is a contribution of the Food and Drug Administration and is not subject to copyright.

References

¹International Committee for Display Metrology (January 2011). http://icdm-sid.org/

²L. T. Niklason, B. T. Christian, L. E. Niklason,D. B. Kopans, D. E. Castleberry, B. H. Opsahl-Ong, C. E. Landberg, P. J. Slanetz, A. A., Giardino, R. Moore, D. Albagli, M. C. DeJule, P. F. Fitzgerald, D. F., Fobare, B. W. Giambat-tista, R. F. Kwasnick, J. Liu, S. J. Lubowski, G. E. Possin, J. F. Richotte, C. Y. Wei, and R. F. Wirth, "Digital Tomosynthesis in Breast Imaging," Radiology 205 (2), 399–406 (1997).

³J. M. Boone, T. R. Nelson, K. K. Lindfors, and J. A. Seibert, "Dedicated breast CT: radiation dose and image quality evaluation,Radiology 221, (3) 657–667 (2001).

⁴D. Getty, C. D'Orsi, R. Pickett, M. Newell, K. Gundry, and S. Roberson, "Improved accuracy of lesion detection in breast cancer screening with stereoscopic digital mammog-raphy," Radiological Society of North America Annual Meeting (2007). •