WHY IS IMAGE QUALITY ASSESSMENT SO DIFFICULT?
Zhou Wang and Alan C.Bovik
Lab for Image and Video Engi.,Dept.of ECE Univ.of Texas at Austin,Austin,TX78703-1084 zhouwang@ieee,bovik@ece.utexas.edu
Ligang Lu
IBM T.J.Watson Rearch Center Yorktown Heights,NY10598
lul@
ABSTRACT
Image quality asssment plays an important role in various image processing applications.A great deal of effort has been made in re-cent years to develop objective image quality metrics that correlate with perceived quality measurement.Unfortunately,only limited success has been achieved.In this paper,we provide some insights on why image quality asssment is so difficult by pointing out the weakness of the error nsitivity bad framework,which has been ud by most image quality asssment approaches in the lit-erature.
Furthermore,we propo a new philosophy in designing im-age quality metrics:The main function of the human eyes is to extract structural information from the viewingfield,and the hu-man visual system is highly adapted for this purpo.Therefore,a measurement of structural distortion should be a good approxima-tion of perceived image distortion.Bad on the new philosophy, we implemented a simple but effective image quality indexing al-gorithm,which is very promising as shown by our current results.
1.INTRODUCTION
Image quality measurement is crucial for most image processing applications.Generally speaking,an image quality metric has three kinds of applications:
First,it can be ud to monitor image quality for quality con-trol systems.For example,an image and video acquisition system can u the quality metric to monitor and automatically adjust it-lf to obtain the best quality image and video data.A network video rver can u it to examine the quality of the digital video transmitted on the network and control video streaming.
Second,it can be employed to benchmark image processing systems and algorithms.Suppo we need to lect one from mul-tiple image processing systems for a specific task,then a quality metric
can help us evaluate which of them provides the best quality images.
我想相亲Third,it can be embedded into an image processing system to optimize the algorithms and the parameter ttings.For instance, in a visual communication system,a quality metric can help opti-mal design of the prefiltering and bit assignment algorithms at the encoder and the postprocessing algorithms at the decoder.
The best way to asss the quality of an image is perhaps to look at it becau human eyes are the ultimate receivers in most image processing environments.The subjective quality measure-ment Mean Opinion Score(MOS)has been ud for many years.
Original
signal Distorted signal
Qualtiy/
Measure Fig.1.Error nsitivity bad image quality measurement.
forms,discrete cosine transform(DCT),and Gabor decomposi-tions.The decompod signal is treated differently in different channels according to human visual nsitivities measured in the specific channel.The errors between the two signals in each chan-nel are calculated and weighted,usually by a Contrast Sensitivity Function(CSF).The weighted error signals are adjusted by a vi-sual masking effect model,which reflects the reduced visibility of errors prented on the background signal.Finally,an error pool-ing method is employed to supply a single quality value of the whole image being tested.The summation usually takes the form:
(1) where is the weighted and masked error of the k-th coefficient in the l-th channel,and is a constant typically with a value be-tween1and4.This formula is commonly called Minkowski error pooling.
2.2.Weakness of Error Sensitivity Bad Methods
The above error nsitivity bad framework can be viewed as a simplified reprentation of the HVS.Such simplification implies the following assumptions:
假如我是一只小鸟
1.The reference signal is of perfect quality.5岁女孩身高
2.There exist visual channels in the HVS and the channel respons can be simulated by an appropriate t of channel trans-formations.
3.CSF variance and intra-channel masking effects are the dominant factors that affect the HVS’s perception on each trans-formed coefficient in each channel.
4.For a single coefficient in each channel,after CSF weight-ing and masking,the relationship between the magnitude of the error,,and the distortion perceived by the HVS,,can be modelled as a non-linear function:.
5.In each channel,after CSF weighting and masking,the in-teraction between different coefficients is small enough to be ig-nored.
6.The interaction between channels is small enough to be ignored.
7.The overall perceived distortion is monotonically increasing with the summation of the perceived errors of all coefficients in all channels.
8.The perceived image quality is determined in the early vi-sion system.Higher level process,such as feature extraction, pattern matching and cognitive understanding happening in the hu-man brain,a
re less effective.
际恒9.Active visual process,such as the change offixation
points and the adaptive adjustment of spatial resolution becau of attention,are less effective.
Thefirst assumption is reasonable for image/video coding and
communication applications.The cond and third assumptions
are also practically reasonable,provided the channel decomposi-
tion methods are designed carefully tofit the psychovisual experi-mental data.However,all the other assumptions are questionable.
We give some examples below.
Notice that most subjective measurement of visual error n-
sitivity is conducted near the visibility threshold,typically using a
2Alternative Forced Choice(2AFC)method.The measurement results are not necessarily good for measuring distortions much
larger than just visible,which is the ca for most image process-外出报备制度
ing applications.Therefore,Assumption4is weak,unless more
弃儿convincing evidence can be provided.
It has been shown that many models work appropriately for simple patterns,such as pure sine waves.However,their perfor-
mance degrades significantly for natural images,where a large
number of simple patterns coincide at the same image locations.
This implies that the inter-channel interaction is strong,which is a contradiction of Assumption6.
Also,wefind that Minkowski error pooling(1)is not a good
choice for image quality measurement.An example is given in
Figure2,where two test signals,test signals1(up-left)and2
(up-right),are generated from the original signal(up-center).Test signal1is obtained by adding a constant number to each sample
point,while the signs of the constant number added to test signal
2are randomly chon to be1or.The structural information of the original signal is completely destroyed in test signal2,but prerved pretty well in test signal1.In order to calculate the
Minkowski error metric,wefirst subtract the original signal from
the test signals,leading to the error signals1and2,which have
very different structures.However,applying the absolute opera-tor on the error signals results in exactly the same absolute error signals.Thefinal Minkowski error measures of the two test sig-nals are equal,no matter how the value in(1)is lected.This example not only demonstrates that structure-prervation ability is an important factor in image quality asssment,but also shows that Minkowski error pooling(1)is very inefficient in capturing the structures of errors.By the obrvation that the frequency dis-tributions of the test signals1and2are very different,one might argue that the p
roblem can be solved by transforming the error signals into different frequency channels and measure the errors differently in different channels.This argument is emingly rea-sonable,but if the above example signals are extracted from cer-tain frequency bands instead of the spatial domain,then repeated
Fig.2.Minkowski error pooling.
channel transformation is needed to further decompo the trans-formed signal(possibly iterative transformations will be involved), andfinally the multiple time-transformed error signal will still be measured by the Minkowski error summation.In this n,the weakness of Minkowski error pooling still cannot be avoided.
There are some other weakness of the framework.For ex-ample,channel decompositions usually lead to very high compu-tational complexity,especially for well-designed visual channel transformations such as the Gabor tranforms.
3.STRUCTURAL DISTORTION BASED IMAGE
QUALITY MEASUREMENT
3.1.New Philosophy
We believe that one of the most important reasons that the error nsitivity bad methods cannot work effectively is that they treat any kind of image degradation as certain type of errors.Our new philosophy in designing image quality metrics is:
The main function of the human eyes is to extract
structural information from the viewingfield,and
the human visual system is highly adapted for this
purpo.Therefore,a measurement of structural dis-
tortion should be a good approximation of perceived
image distortion.
As exemplified by Figure2,large errors do not always result in large structural distortions.The key point of the new philoso-phy is the switch from error measurement to structural distortion measurement.
3.2.A New Image Quality Index炖鸽子汤的做法
Given the new philosophy above,the next problem is how to de-fine and quantify structural distortions.This is a challenging but interesting rearch topic that needs thorough investigations.
As afirst attempt,we developed a simple but effective quality indexing algorithm[11].Let x and y be the original and the test image signals,respectively.The propod quality index is defined as The dynamic range of is.The best value1is achieved if and only if for all.This quality index models any distortion as a combination of three different factors: loss of correlation,mean distortion,and variance distortion.In order to understand this,we rewrite the definition of as a product of three components:
index/demo.html.
4.CONCLUSIONS AND DISCUSSIONS
In this paper,we provide some insights on why image quality as-ssment is so difficult by showing the weakness of the tradi-tional error nsitivity bad image quality measurement approa-ches.A new philosophy is propod,which models image degra-dation as structural distortions instead of errors.A simple imple-mentation of the new philosophy exhibits very promising results.
深开头的四字成语
As pointed out by Watson in[12]:“Much of the theoretical and experimental work in spatial vision in the last thirty years has focusd upon spatial channels;on their existence and on their de-tailed shape and number.”We believe that the issues raid in this paper are more critical for the future development of successful image quality asssment methods.
(a)
(b)(c)
(d)(e)(f)
Fig.3.Evaluation of “Lena”images distorted by different means.(a)Original “Lena”image,512512,8bits/pixel;(b)Contrast stretched image,MSE =225,Q =0.9372;(c)Gaussian noi contaminated image,MSE =225,Q =0.3891;(d)Impulsive noi contaminated image,MSE =225,Q =0.6494;(e)Blurred image,MSE =225,Q =0.3461;(f)JPEG compresd image,MSE =215,Q =0.2876.
5.REFERENCES
[1]T.N.Pappas and R.J.Safranek,“Perceptual criteria for im-age quality evaluation,”in Handbook of Image and Video Processing (A.Bovik,ed.),Academic Press,May 2000.[2]VQEG,“Final report from the video quality experts group
on the validation of objective models of video quality asss-ment,”www.vqeg/,Mar.2000.[3] C.J.van den Branden Lambrecht,Ed.,“Special issue on im-age and video quality metrics,”Signal Processing ,vol.70,Nov.1998.[4] B.Girod,“What’s wrong with mean-squared error,”in Digi-tal Images and Human Vision (A.B.Watson,ed.),pp.207–220,the MIT press,1993.[5]J.Lubin,“A visual discrimination mode for image system
design and evaluation,”in Visual Models for Target Detec-tion and Recognition (E.Peli,ed.),pp.207–2
20,Singapore:World Scientific Publishers,1995.[6]S.Daly,“The visible difference predictor:An algorithm for
the asssment of image fidelity,”in Proc.SPIE ,vol.1616,pp.2–15,1992.
[7] A.B.Watson,J.Hu,and J.F.III.McGowan,“Digital video
quality metric bad on human vision,”Journal of Electronic Imaging ,vol.10,no.1,pp.20–29,2001.[8] A.B.Watson,G.Y .Yang,J.A.Solomon,and J.Villanor,
“Visibility of wavelet quantization noi,”IEEE Trans.Im-age Processing ,vol.6,pp.1164–1175,Aug.1997.[9]J.-B.Martens and L.Meesters,“Image dissimilarity,”Signal
Processing ,vol.70,pp.155–176,Nov.1998.[10] A.M.Eskicioglu and P.S.Fisher,“Image quality mea-sures and their performance,”IEEE Trans.Communications ,vol.43,pp.2959–2965,Dec.1995.[11]Z.Wang and A.C.Bovik,“A universal image quality index,”
IEEE Signal Processing Letters ,2002.[12] A.B.Watson,“Visual detection of spatial contrast patterns:
Evaluation of five simple models,”Optics Express ,vol.6,pp.12–33,Jan.2000.
