The Spatial Standard Observer: A new tool for display metrology

In the design of displays, "beauty is in the eye of the beholder." But until recently, the industry has lacked tools to estimate quality as seen by the human eye. Here, the development and application of the Spatial Standard Observer (SSO), a new tool for estimating the visibility of spatial patterns, is detailed. Similar to measurements of luminance, or of CIE color coordinates, the SSO provides another means of measuring displays in units that are meaningful to the human observer.

by Andrew B. Watson

WE ARE in a time of explosive growth in digital display technologies, applications, and markets. During design and manufacture, displays are measured with instruments to quantify their visual quality. For this purpose, it would be useful to have an instrument that could mimic the performance of the human observer. We have developed an instrument of this kind – the Spatial Standard Observer (SSO), a software algorithm that incorporates a simple model of human-visual sensitivity to spatial contrast for use in a wide variety of display inspection and measurement applications.

The SSO began in an effort to account for the results of a research project known as ModelFest. This was a collaboration among an international consortium of vision-research groups, who sought to create a common set of benchmark data to describe human sensitivity to spatial patterns.^1-3 They designed a set of 43 standard stimuli, and collected contrast thresholds for each stimulus from a total of 16 human observers. A contrast threshold is a measure of the smallest amount of contrast that is required for the image to be visible. Contrast is the variation on luminance in the image, expressed as a fraction of the average luminance. The data showed large variations (about 1.5 log units) among the different spa-

Andrew B. Watson is the Senior Scientist for Vision Research at the NASA Ames Research Center, MS 262-2, Moffett Field, CA 94035-1000; telephone 650/604-5419, fax -3323, e-mail: andrew.b. watson@nasa.gov.

Fig. 1: Overview of the Spatial Standard Observer. The difference between test and reference images is filtered by a contrast-sensitivity function (CSF), windowed by an aperture function, and pooled non-linearly over space. The two graphs show the CSF (left) and the aperture (right).

tial patterns. Following collection of the data, the theoretical challenge was to account for these variations with a model of visual spatial processing. In our lab, we found that the data could be accounted for with a rather simple model.^2,3 That model formed the basis for the SSO, which is outlined in Fig. 1.

The input to the model is a pair of images: test and reference. The difference between test and reference images is filtered by a contrast-sensitivity function (CSF). The CSF is a measure of the visibility of different spatial frequencies at different orientations. Spatial frequencies are sinusoidal variations in contrast over space. This function is two-dimensional and reflects the decline in human-visual sensitivity at higher spatial frequencies and at very low frequencies, as well as the lower sensitivity at oblique orientations (the oblique effect).

The filtered image is then multiplied by an aperture function. This reflects the decline in human-visual sensitivity with distance from the point of fixation. The final step is to pool the resulting image over space, using a non-linear Minkowski metric, in which the absolute value of each pixel is raised to a power beta, summed, and then the beta root is taken. Beta is a parameter of the model, which here has a value of about 2.4. Because the SSO metric is calibrated against a large corpus of human data, the output is in units of just-noticeable difference (JND). This means that if two images differ by 1 JND, they should be just discriminable.

The SSO operates on digital images that subtend 2° or less, viewed from a specific distance, and whose pixels have a known relation to luminance. Extensions of the basic metric incorporate spatial masking, viewing of larger images, and color.

Mura Inspection

While flat-panel-display manufacturing is highly automated, most flat panels are examined for defects by human inspectors. This inspection stage is slow and costly, and becomes more difficult as panel sizes increase. Reliability and consistency of inspection are also generally unknown.

One important category of defect is called "mura," derived from the Japanese word for blemish.⁴ Mura are typically low-contrast spots, smudges, and streaks of various shapes and sizes that are visible when the display is driven at a uniform value. Different types of mura arise through different defects in the structure of the display. There have been previous efforts to define and quantify mura.^4,5 However, these definitions do not provide a clear method for measuring real mura, in part because the definitions are normative and do not provide general measurement methods.

To automate the process of display inspection, it is necessary to compute the visibility of the defect to a human. This requires a calibrated model of human sensitivity to spatial patterns such as the SSO.

Fig. 3: Motion-blur metric based on the Spatial Standard Observer. An ideal edge and the motion-blurred edge are subtracted and the difference is filtered by a contrast-sensitivity function and pooled nonlinearly over space. The result is a visible motion-blur measure (VMB) in units of JND.

To apply the SSO to mura detection, a single image of the display under test is acquired. This image is first preprocessed to remove signals that are not of interest. It may also be cropped and down-sampled. A reference image is then created from this image by removing mura-like signals. Test and reference images are then compared and their difference measured. The SSO produces measurements in units of JND. In a typical mode of operation, the SSO produces both an image showing the location of the mura, as well as a peak JND measure, defining the worst artifact in the image.

Fig. 4: Example of calculation of the Visible Motion Blur.

An example is shown in Fig. 2. On the left is an image captured from a 17-in. liquid-crystal-display (LCD) panel. The primary defect here is a bright blob in the upper right of the display. On the right is the SSO output, shown as an image, and thresholded at 2 JND. Measurements of this sort can be easily used for grading, selecting, or rejecting displays, as well as identifying the location of the major artifacts.

Motion Blur

Current LCDs excel in many ways, but they remain inferior to the best CRT displays in one respect. The image on the face of a CRT is produced by a beam that scans rapidly over the active area. As a result, each pixel is illuminated for only a fraction of the frame duration. In contrast, in the typical LCD, each pixel remains at the same brightness for the entire frame duration (a so-called sample-and-hold display). When combined with the relatively slow switching times of many LCDs, this results in a blurring of rapidly moving edges. This motion blur can be measured with a pursuit-camera system, to provide an image as seen by an eye tracking the moving edge.⁶ The motion blur can also be calculated from knowledge of the temporal response of the LCD.⁷

However, beyond measuring or calculating motion blur, it is important to know how visible the blur would be to a human observer. Recently, we have shown how the SSO can be used for this task as well.⁷

Motion blur is usually evaluated by observing a light–dark edge moving horizontally across the screen. As the human observer tracks this edge with his eye, the otherwise sharp edge is smeared by the slow temporal response. Figure 3 shows the cross section of

an ideal sharp edge, and the same edge as it would be rendered on the back of the eye of an observer tracking the edge, assuming a particular speed of motion and a particular LCD temporal response. In this example, the edge is blurred over a distance of several pixels. The next step is to take the difference between these two edges. The result is the artifact, but it is not yet expressed in visible units. We pass this difference signal through the contrast sensitivity function of the SSO, though in this case it is only one-dimensional because the edge varies only in one dimension. The filtered difference is then pooled in the standard manner using the Minkowski metric. Though not shown here, the aperture may also be included. The result is a measure of visible edge blur in units of JND. We call this the Visible Motion Blur (VMB).

This measure has several advantages over other proposed measures, which typically only look at the width of the blurred edge. First, it employs meaningful units of JND, as do all SSO measures. Second, the measure takes into account the contrast of the edge. This is important because not only black–white edges are used to test motion blur, but also so-called gray-to-gray transitions. Finally, because it looks at the entire blurred-edge waveform, this measure can take into account the effects of advanced techniques such as backlight strobing, overdriving, and black insertion.

Figure 4 shows an example of VMB calculation. We begin with a graph of the transition from dark to light over time as the LCD is switched from black to white [Fig. 4(a)]. In this example, we have exaggerated the slowness of the LCD and also given the function an overshoot, as might occur in the case of overdriving. The cross section of the resulting blurred edge is shown in Fig. 4(b). Subtracting that blurred edge from the ideal edge yields the artifact shown in Fig. 4(c). That artifact is then filtered by the CSF resulting in the waveform shown in Fig. 4(d) that is then integrated using the Minkowski metric to yield an output of 2.3 JND.

Measuring Screen Grain

Projection displays produce a large image at relatively low cost and are an important part of the current display market. Rear-projection display screens often employ optical elements such as spherical lenslets (glass beads) embedded in a dark light-absorbing material. This allows efficient transmission of the projected light, while rejecting most of the ambient light. This in turn ensures a high-contrast image, even in the presence of significant ambient illumination.

However, irregularities in the size and spacing of the beads lead to non-uniformities in appearance of the screen when uniformly illuminated. This particular artifact is often referred to as screen grain. In Fig. 5, a small 1.2-mm sample from one screen in which these irregularities are evident is shown. Note that this is a highly magnified view, and in typical use this screen would be viewed from some distance, 63.5 cm for example, where it would subtend only about 0.1°. Under these conditions, the individual beads would not be visible as shown in the figure, but the image on the screen would have a grainy appearance.

In the design of projection screens, it is important to minimize the visible screen grain. For this purpose, it would be useful to have an objective measure that accurately reflected the visibility of screen grain. In a recent project, I worked with Thomas Fiske, Louis Silverstein, and Sue Hodgson to evaluate a number of metrics for this purpose. A complete description of the project will be published in the Journal of the SID.⁸

Subjective data were collected from human observers for 10 different rear-projection screens, along with a standard, which was essentially uniform. The data were analyzed to express the screen grain of each display in units of JND. The units showed large differences ranging from 0 (indistinguishable from the standard) to over 4 JND.

Digital image samples were also taken from each screen, and these were analyzed by the SSO, which returned measures of the visible

Fig. 6: Comparison of human and SSO estimates of screen grain

artifact, in units of JND. A comparison of the human and SSO estimates is shown in Fig. 6. Not only is the correlation between the two estimates quite good (r = 0.934), but the absolute magnitude of the predictions is close to the actual values in JND. This absolute agreement might be even closer if the extended observation interval (several seconds) for the human observers was taken into account.

Conclusion

The Spatial Standard Observer offers a new tool for design, specification, and inspection of visual displays. The ability to automatically measure mura in flat-panel displays will improve the efficiency and thus lower the cost of automated manufacture of large flat-panel displays. A perceptually based measure of motion blur will allow rational design and selection of displays and display technologies, as well as rational specification of displays for the consumer. A perceptually based measure of screen grain will allow optimized design of projection screens. We have described three specific applications of the SSO, but there are many others. Just as the measurement of luminance is fundamental to the engineering of displays, so too should be the measurement of visible spatial contrast, through a device such as the SSO.

References

¹T. Carney, et al., "Modelfest: Year one results and plans for future years," Human Vision, Visual Processing, and Digital Display IX,3959, 140-151 (2000).

²A. B. Watson, "Visual detection of spatial contrast patterns: Evaluation of five simple models," Optics Express 6, 12-33 (2000).

³A. B. Watson and A. J. Ahumada, Jr., "A standard model for foveal detection of spatial contrast," J. Vision 5, 717-740 (2005).

⁴SEMI, " Definition of Measurement Index (Semu) for Luminance Mura in FPD Image Quality Inspection," SEMI D31-1102 (2002).

⁵VESA, "Flat Panel Display Measurements Standard Ver. 2.0," June 1, 2001 (2001).

⁶K. Oka, K. Kitagishi, and Y. Enami, "Motion Artifacts Measured by Using a Pursuit Camera," Proc. IDMC (2005).

⁷A. B. Watson, "The Spatial Standard Observer: A human vision model for display inspection, SID Symposium Digest Tech Papers 37, 1312-1315 (2006).

⁸T. J. Fiske, et al., "Visual Quality of High-Contrast Projection Screens Part I: Visibility of Screen-Based Artifacts and Noise," J. Soc. Info. Display (to be published). •