In order to conquer the issue of motion blur, it is important that a model be developed that simulates and visualizes the performance of an LCD screen together with the perception of the viewer. In this article, such a model – based on an earlier model used to predict PDP performance – is proposed.

by Carsten Dolar

ALTHOUGH flat-panel displays have almost completely replaced cathode-ray-tube (CRT) displays in television and video applications, in some aspects they still need further improvement to meet the performance expectations set by CRTs. Nowhere is this more apparent than in the display of moving images on liquid-crystal displays (LCDs). In order to fully understand the problem and characterize the effects of different components in a typical display system such as a television, it is desirable to model the flat-panel-display system together with the perception of the human observer. Such a model would make it possible to predict the perceived image quality.

Some years ago, a model had been proposed¹ to demonstrate the perceived image quality of a plasma-display panel (PDP). This model has now been adapted to the representation of moving images on LCDs to produce the descriptive simulation results²presented in this article.

Simulation Model

To simulate the perceived image, two models are actually required: a display model and a visual perception model, as shown in Fig. 1. Let us start with a model for the LCD that can be divided into two blocks: a model for the spatial behavior (the pixel intensity profile or spatial aperture) and a model for the temporal behavior (the impulse response or temporal aperture). This is in good agreement with other LCD models.³ An LCD pixel is rectangular in most cases and subdivided into three subpixels for the primary colors. For comparison, the intensity profile of the pixel of a CRT-based display is almost Gaussian.

Modeling the temporal behavior of the LCD is based mainly on concatenated step responses from one frame to the next, taking into account the hold-type behavior of the active-matrix addressing and driving scheme and using a first- or second-order low-pass filter to model LC dynamics. However, the LC material's switching properties depend on the current state and the input signal. There-fore, the LCD in general is a non-linear system, but it can be linearized in some cases. Due to the addressing and driving scheme, some space-dependent delays also need to be modeled, e.g., a certain delay per line since the lines are addressed sequentially, one after the other.

The perception model, as the second major part of the presented model, comprises basic reception mechanisms. The two important components in this model are smooth pursuit eye tracking and temporal integration of the eye. The first is simply modeled as a compensation of a linear movement; the latter is implemented as weighted temporal integration yielding a still image that represents a snapshot of the subjective impression of an observer watching an image sequence. The spatial filter in the perception model is used to simulate different viewing distances with a simple spatial low-pass filter.

Fig. 1: Model of LCD moving-image representation and perception.

The simulation tool based on the described model can be parameterized by identifying the single-pixel shape (monochrome situation) or the shape of the subpixels as well as the positions of the differently colored subpixels, and by defining the display's step response with either a rise and fall time or gray-to-gray response times to adjust the LC low-pass behavior. Moreover, the backlight switching can be parameterized (e.g., for a blinking backlight), as well as the addressing delays for each row of pixels (e.g., line-to-line delay). Further parameters comprise bit-depth and electro-optical transfer function (e.g., "gamma").

The current version of the simulation tool has been implemented in a MatLab environment; it has not been optimized for speedy performance and, thus, simulations are not carried out in real time.

In the following, we first describe static LCD behavior, i.e., the still-image representation, before the simulation results for moving images are presented. One important aspect therein will be the comparison between the presentation on a CRT and on an LCD. Later, the dynamic behavior of the LCD, i.e., the moving-image representation, is investigated and demonstrated. The most interesting point therein is the effect of motion blur in tracked objects in an image sequence.

Static LCD Behavior

The first interesting phenomenon to investigate with the model is the higher degree of error visibility on matrix displays.² Direct visual comparison of a CRT with an LCD shows that errors such as coding artifacts, block noise, etc., are more visible and thus more disturbing on an LCD than on a CRT. To investigate further, we simulated the perceived image impression of a still image with the described model. In this case, the display and perception model will reduce to spatial filtering, yielding the display's static behavior. It is assumed that the spatial display aperture is Gaussian for the CRT and rectangular for the LCD.

Figure 2 shows the output of the display model for an LCD and three CRTs with different beam diameters. (Note that the simulation results demonstrate the basic effect rather than the exact appearance since the shadow mask and subpixels are neglected in the simulations presented here.) The input image for all display simulations has been the same, i.e., having the same level of block artifacts. The output, however, looks very different. In the CRT representations, the blocking errors are concealed either by a lack of sharpness or by a visible line raster that masks the vertical block boundaries. In the LCD representation, however, these block boundaries are clearly visible. What are the reasons for the increased visibility of the errors? First of all, an LCD has a sharper image impression due to the fact that the pixels in a matrix display are separated from each other by a black-matrix structure. A step in intensity between two adjacent pixels can therefore be displayed ideally, contrary to CRT representation where the pixel profile leads to smooth transitions. Second, the LCD representation lacks a masking signal such as the line raster in CRT representation. A masking signal influences signal components that have the same direction and spatial frequency; in this scenario the line raster in the CRT representation affects the visibility of vertical block boundaries.

 (a)  (b)  (c)  (d)

Fig. 2: Error visibility on different displays: (a) LCD, (b)–(d) CRTs with different beam sizes (spot diameters 0.5, 1, and 2 lines).

 (a)  (b)  (c)  (d)

Fig. 3: Adding a masking signal to LCD simulation. (a) Vertical, (b) horizontal, (c) oscillation, and (d) masking noise.

To demonstrate the effect of masking on the image impression, artificial masking signals (vertical and horizontal oscillation, noise) were applied to the LCD simulation (see Fig. 3). Depending on the spatial frequency and the direction of the masking signal, horizontal and/or vertical blocking errors are concealed (which does not mean that the presented masking signals improve the overall image quality). Simply put, an LCD is able to present an image with very high subjective sharpness without interference by a masking signal. Therefore, steep signal transitions are perfectly displayed, positive at the edges of the image and negative at errors because it enhances the visibility of those artifacts. Thus, the image quality needs to be controlled very carefully in the editing process, in the coding for transmission, and, finally, in the signal processing in the display system (e.g., computer monitors or television sets).

 (a)  (b)  (c)  (d) 

Fig. 4: Influence of the viewing distance: (a) CRT and (b) LCD representation at long viewing distances, and (c) CRT and (d) LCD representation at short viewing distances.

 (a)

 (b)

Fig. 5: Influence of velocity on motion blur: (a) 60-Hz Panel, vx = 3 pixels per frame; (b) 60-Hz panel, vx = 6 pixels per frame.

This raises a question: How much of the mentioned error enhancement is perceived? To answer this question, the viewing distance and the acuity of human vision has to be considered. Short viewing distances will lead to a high spatial cut-off frequency in the perception model, yielding a distinct visibility of details. Long viewing distances will lead to the opposite, and the ability to perceive details will decrease. Figure 4 shows that the longer the viewing distance, the less the differences between the CRT and LCD representation. As the preferred (relative) viewing distance decreases with increasing screen size, the mentioned error enhancement is more visible on large-sized LCD-TV sets compared to that of common medium-sized CRT sets. Again, this emphasizes the point that the image quality needs to be checked with great care in each processing step.

 (a)

 (b)

Fig. 6: Influence of LC response time (60-Hz panel, vx = 6 pixels per frame). (a) Long LC response times and (b) zero LC response time (ideal hold-type behavior).

 (a)

Dynamic LCD Behavior

Now let us examine the dynamic behavior of the LCD in presenting a moving-image sequence. A widely noticed attribute of LCD moving-image representation is the introduction of motion blur to a tracked moving object in an image scene. The cause of this display artifact is now known; it is, namely, the length and the shape of the temporal impulse response of the display. To explain motion blur, imagine one pixel changing from black to white from one frame to the other and back to black in the next frame. The intensity of the pixel will be held for the entire duration of a frame (i.e., hold-type behavior). The gradual response of the LC material will introduce an additional low-pass behavior to the signal. Finally, the backlight modulates the light signal, which also affects the display's temporal impulse response. If the eye of the observer moves with a certain velocity relative to the screen, the position of the said pixel will move on the retina along the trajectory of the movement. Thus, the light intensity of the pixel will also be perceived along this trajectory. The temporal integration by the eye will transform the moving pixel to a spatial sensation along the motion trajectory, causing a blurring of the pixel or a point spread function that has the shape of the display's temporal impulse response. The higher the eye velocity and the longer the temporal impulse response, the more the pixel becomes blurred.

Using this linear display and perception model, the perceived image can be calculated with the (solely spatial) convolution of the image with the spatially transformed impulse response, known as the "motion-blur filter." The parameters of this motion-blur filter are given by the shape of the temporal impulse response of the display and by the velocity of the eye relative to the screen. Figure 5 demonstrates the influence of the object's velocity on the blur in the perceived image. The higher the velocity, the more blurred the perceived image becomes. The length of the display's temporal impulse response is determined by the duration of the hold-period and by the LC response time(s).

Let us now examine the influence of the response time of the display on the perceived image first. Figure 6 shows the simulation results of a display with a long response time for all gray-level transitions (around 20 msec) and of a display with zero response time, yielding a perfect hold-type behavior. The display with the long LC response time obviously presents a strongly blurred image. However, the ideal hold-type display also suffers from motion blur. This occurs due to the fact that the hold-type behavior itself also has an impact on the motion blur that cannot be neglected. Therefore, the response-time compensation by using faster LC materials or by applying signal preprocessing, e.g., overdrive, can only reduce the motion blur to a certain limit.^3,4 The shortening of the hold-time is thus an effective technique in motion de-blurring when the LC response times are short enough. Several ways have been proposed to do this: blinking the backlight, inserting black or gray images between two frames, or doubling the frame rate with motion-compensated frame interpolation. The latter is the preferred method in recent LCDs for TV applications. Another approach is to enhance the high spatial frequencies (details) that will be reduced by the motion blur by applying a motion-dependent signal pre-processing.³

Let us look at the simulation results of some of the measures that reduce motion blur. Figure 7 shows (a) simulations of a display operated at 60 Hz compared to (b) a 120-Hz display. This measure effectively reduces the amount of motion blur, but this will only hold in regions where the motion-compensated intermediate frame can be calculated exactly. Thus, the removal of the motion blur depends on the quality of the motion-estimation algorithm. The pre-emphasis of details prior to displaying a frame on the display is simulated in Fig. 8. Compared to the unprocessed version, more image details are retained in the perceived image. Because the amount of detail enhancement depends on the motion estimation as well, the reduction of motion blur will depend on the quality of the motion estimator in a display as well.

 (a)  (b)

Fig. 8: Motion de-blurring with motion-compensated inverse filtering. (a) Unprocessed and (b) preprocessed.

One question, however, remains to be answered: Is there a visible effect on the image due to the non-linear temporal behavior of the LCD introduced by the unequal response times for different gray-level steps? The answer is yes, but it depends on several factors. As is known, the LC-material response time depends on the starting condition and on the addressing voltage, and therefore on the image input signal. Thus, the temporal display model becomes non-linear, and common system theory cannot be applied to analyze the temporal behavior. However, the model can be considered linear for small temporal step sizes, and the system behavior can then be evaluated numerically. The simulation results shown in Fig. 9 demonstrate the differences between a display with a constant response time and a display with input-signal-dependent response times yielding non-linear behavior. Note that the input data is a downscaled version of Lena's face in order to emulate a low-resolution mobile-display device. The effects on the perceived image due to the non-linear behavior are geometric deformations and false colors. The first can be observed in the area around the root of the nose, the latter in the edge of the mirror. Those display artifacts depend on the differences in the response times of the display and the velocity of the eye movement. The smaller the differences in the response times for different gray level and color steps, the smaller the error in the perceived image. Because the differences in response times for different gray-level steps might be small in recent displays due to response-time compensation with overdrive, the differences in motion blur for those different gray-level steps will also be small. Therefore, the geometric distortions and color error will be less noticeable on large-sized home television sets and will more likely be visible in small-sized handheld TV devices because such display devices have a lower resolution and will be observed with a smaller relative viewing distance, which leads to a higher visibility of the mentioned artifacts.

Conclusions

So, how can this model be used? There is a variety of applications for such a model. For example, it can be used to optimize display-processing algorithms. With the right parameterization, an algorithm can be optimized for a certain display by evaluating the simulation results for this particular display. The current version of the simulation model has been implemented to yield qualitative illustrative results. It has been tested and verified by 10 subjects and various images in order to confirm the validity of the "frozen" moving images for illustration of motion-blur effects. In a further step, the model could be expanded to include more detailed physical properties of the display and a calibration in comparison to controlled subjective evaluations and ratings.

The simulator based on the presented model can also be used to investigate the perceived quality of new algorithms for flat-panel-display data processing. It can as well be used to quantitatively evaluate a display's properties, e.g., measuring the amount of motion-blur. Last, but not least, the presented model is good for demonstration of the display artifacts and the causes thereof. For instance, the simulator and its results were used several times in lectures regarding image processing and display techniques to demonstrate the LCD moving-image representations.

References

¹O. Franzen and U. Fischbeck, "Spatio-Temporal Modeling of the Moving Image Representation on PDP Displays," Proc. ISCE, L13-18 (2002).

²C. Dolar and H. Schröder, "Modeling Perceived LCD Moving Image Representation," Electronic Imaging," IS&T/SPIE (2008).

³M. A. Klompenhouwer, Flat Panel Display Signal Processing (Eindhoven University Press, The Netherlands, 2006).

⁴J. Ohwada, "Improving the moving-image quality of LCDs by using impulse driving," Information Display, No. 6 (2004). •

 (a)  (b)  (c)