Multiresolution Gray-Scale and Rotation Invariant Texture Classification
with Local Binary Patterns
Timo Ojala,Matti PietikaÈinen,Senior Member,IEEE,and Topi MaÈenpaÈaÈ
AbstractÐThis paper prents a theoretically very simple,yet efficient,multiresolution approach to gray-scale and rotation invariant texture classification bad on local binary patterns and nonparametric discrimination of sample and prototype distributions.The method is bad on recognizing that certain local binary patterns,termedªuniform,ºare fundamental properties of local image texture and their occurrence histogram is proven to be a very powerful texture feature.We derive a generalized gray-scale and rotation invariant operator prentation that allows for detecting theªuniformºpatterns for any quantization of the angular space and for any spatial resolution and prents a method for combining multiple operators for multiresolution analysis.The propod approach is very robust in terms of gray-scale variations since the operator is,by definition,invariant against any monotonic transformation of the gray scale.Another advantage is computational simplicity as the operator can be realized with a few operations in a small neighborhood and
a lookup table.Excellent experimental results obtained in true problems of rotation invariance,where t
he classifier is trained at one
particular rotation angle and tested with samples from other rotation angles,demonstrate that good discrimination can be achieved with the occurrence statistics of simple rotation invariant local binary patterns.The operators characterize the spatial configuration of local image texture and the performance can be further improved by combining them with rotation invariant variance measures that characterize the contrast of local image texture.The joint distributions of the orthogonal measures are shown to be very powerful tools for rotation invariant texture analysis.
Index TermsÐNonparametric,texture analysis,Outex,Brodatz,distribution,histogram,contrast.
换届选举工作方案
拭目以待是什么意思æ
1I NTRODUCTION
A NALYSIS of two-dimensional textures has many poten-
tial applications,for example,in industrial surface inspection,remote nsing,and biomedical image analy-sis,but only a limited number of examples of successful exploitation of texture exist.A major problem is that textures in the real world are often not uniform due to variations in orientation,scale,o
r other visual appearance. The gray-scale invariance is often important due to uneven illumination or great within-class variability.In addition,the degree of computational complexity of most propod texture measures is too high,as Randen and Husoy[32]concluded in their recent extensive compara-tive study involving dozens of different spatial filtering methods:ªA very uful direction for future rearch is therefore the development of powerful texture measures that can be extracted and classified with a low-computa-tional complexity.º
Most approaches to texture classification assume,either explicitly or implicitly,that the unknown samples to be classified are identical to the training samples with respect to spatial scale,orientation,and gray-scale properties. However,real-world textures can occur at arbitrary spatial resolutions and rotations and they may be subjected to varying illumination conditions.This has inspired a collection of studies which generally incorporate invariance with respect to one or at most two of the properties spatial scale,orientation,and gray scale.
The first few approaches on rotation invariant texture description include generalized cooccurrence matrices[12], polarograms[11],and texture anisotropy[7].Quite often an invariant approach has been developed by modifying a successful noninvariant approach such as MRF(Markov Random Field)model or Gabor filtering.Examples of MRF-bad rotation invariant techniques include the CSA
R (circular simultaneous autoregressive)model by Kashyap and Khotanzad[16],the MRSAR(multiresolution simulta-neous autoregressive)model by Mao and Jain[23],and the works of Chen and Kundu[6],Cohen et al.[9],and Wu and Wei[37].In the ca of feature-bad approaches,such as filtering with Gabor wavelets or other basis functions, rotation invariance is realized by computing rotation invariant features from the filtered images or by converting rotation variant features to rotation invariant features[13], [14],[15],[19],[20],[21],[22],[30],[39].Using a circular neighbor t,Porter and Canagarajah[31]prented rotation invariant generalizations for all three mainstream paradigms:wavelets,GMRF,and Gabor filtering.Utilizing similar circular neighborhoods,Arof and Deravi obtained rotation invariant features with1D DFT transformation[2].
A number of techniques incorporating invariance with respect to both spatial scale and rotation have been prented[1],[9],[20],[22].[38],[39].The approach bad
.The authors are with the Machine Vision and Media Processing Unit,
Infotech Oulu,University of Oulu,PO Box4500,FIN-90014,Finland.
E-mail:{skidi,mkp,topiolli}@ee.oulu.fi.
Manuscript received13June2000;revid21June2001;accepted16Oct.
2001.
Recommended for acceptance by D.Jacobs.
For information on obtaining reprints of this article,plea nd e-mail to:
tpami@computer,and reference IEEECS Log Number112278.
0162-8828/02/$17.00ß2002IEEE
on Zernike moments by Wang and Healey [36]is one of the first studies to include invariance with respect to all three properties:spatial scale,rotation,and gray scale.In his mid-1990s survey on scale and rotation invariant texture classification,Tan [35]called for more work on perspective projection invariant texture classification,which has re-ceived a rather limited amount of attention [5],[8],[17].This work focus on gray-scale and rotation invariant texture classification,which has been addresd by Chen and Kundu [6]and Wu and Wei [37].Both studies approached gray-scale invariance by assuming that the gray-scale transformation is a linear function.This is a somewhat strong simplification,which may limit the ufulness of the propod methods.Chen and Kundu realized gray-scale invariance by global normalization of the input image using histogram equalizatio
n.This is not a general solution,however,as global histogram equalization cannot correct intraimage (local)gray-scale variations.
In this paper,we propo a theoretically and computa-tionally simple approach which is robust in terms of gray-scale variations and which is shown to discriminate a large range of rotated textures efficiently.Extending our earlier work [27],[28],[29],we prent a gray-scale and rotation invariant texture operator bad on local binary patterns.Starting from the joint distribution of gray values of a circularly symmetric neighbor t of pixels in a local neighborhood,we derive an operator that is,by definition,invariant against any monotonic transformation of the gray scale.Rotation invariance is achieved by recognizing that this gray-scale invariant operator incorporates a fixed t of rotation invariant patterns.
The main contribution of this work lies in recognizing that certain local binary texture patterns termed ªuniformºare fundamental properties of local image texture and in developing a generalized gray-scale and rotation invariant operator for detecting the ªuniformºpatterns.The term ªuniformºrefers to the uniform appearance of the local binary ,there are a limited number of transitions or discontinuities in the circular prentation of the pattern.The ªuniformºpatterns provide a vast majority,some-times over 90percent,of the Q ÂQ texture patterns in examined surface textures.The
most frequent ªuniformºbinary patterns correspond to primitive microfeatures,such as edges,corners,and spots;hence,they can be regarded as feature detectors that are triggered by the best matching pattern.
The propod texture operator allows for detecting ªuni-formºlocal binary patterns at circular neighborhoods of any quantization of the angular space and at any spatial resolu-tion.We derive the operator for a general ca bad on a circularly symmetric neighbor t of members on a circle of
radius ,denoting the operator as vf riu P
Y
.Parameter controls the quantization of the angular space,whereas determines the spatial resolution of the operator.In addition to evaluating the performance of individual operators of a particular ( Y ),we also propo a straightforward approach for multiresolution analysis,which combines the respons of multiple operators realized with different ( Y ).
The discrete occurrence histogram of the ªuniformº
手机格式化怎么弄
patterns (i.e.,the respons of the vf riu P
Y operator)
computed over an image or a region of image is shown to
be a very powerful texture feature.By computing the occurrence histogram,we effectively combine structural and statistical approaches:The local binary pattern detects microstructures (e.g.,edges,lines,spots,flat areas)who underlying distribution is estimated by the histogram.
We regard image texture as a two-dimensional phenom-enon characterized by two orthogonal properties:spatial structure (pattern)and contrast (the ªamountºof local image texture).In terms of gray-scale and rotation invariant texture description,the two are an interesting pair:Where spatial pattern is affected by rotation,contrast is not,and vice versa,where contrast is affected by the gray scale,spatial pattern is not.Conquently,as long as we want to restrict ourlves to pure gray-scale invariant texture analysis,contrast is of no interest as it depends on the gray scale.
The vf riu P
Y operator is an excellent measure of the spatial structure of local image texture,but it,by definition,discards the other important property of local image ,contrast,since it depends o
n the gray scale.If only rotation invariant texture analysis is ,gray-scale invar-iance is not required,the performance of vf riu P
Y can be further enhanced by combining it with a rotation invariant variance measure e Y that characterizes the contrast of local image texture.We prent the joint distribution of the
two complementary operators,vf riu P
Y
a e Y ,as a power-ful tool for rotation invariant texture classification.
As the classification rule,we employ nonparametric discrimination of sample and prototype distributions bad on a log-likelihood measure of the dissimilarity of histo-grams,which frees us from making any,possibly erro-neous,assumptions about the feature distributions.
The performance of the propod approach is demon-strated with two experiments.Excellent results in both experiments demonstrate that the propod texture opera-tor is able to produce,from just one reference rotation angle,a reprentation that allows for discriminating a large number of textures at other rotation angles.The operators are also computationally attractive as they can be realized with a
few operations in a small neighborhood and a lookup table.
The paper is organized as follows:The derivation of the operators and the classification principle are described in Section 2.Experimental results are prented in Section 3and Section 4concludes the paper.
2
G RAY S CALE AND R OTATION I NVARIANT L OCAL B INARY P ATTERNS
We start the derivation of our gray scale and rotation invariant texture operator by defining texture in a local neighborhood of a monochrome texture image as the joint distribution of the gray levels of b I image pixels:
t g Y g H Y F F F Y g ÀI Y
I
where gray value g corresponds to the gray value of the center pixel of the local neighborhood and g p p H Y F F F Y ÀI correspond to the gray values of equally
spaced pixels on a circle of radius b H that form a circularly symmetric neighbor t.
unrar
If the coordinates of g are (H Y H ),then the coordinates of g p are given by À sin P %pa Y os P %pa .Fig.1illustrates circularly symmetric neighbor ts for various ( Y ).The gray values of neighbors which do not fall exactly in the center of pixels are estimated by interpolation.
2.1Achieving Gray-Scale Invariance
As the first step toward gray-scale invariance,we subtract,without losing information,the gray value of the center pixel (g )from the gray values of the circularly symmetric neighborhood g p p H Y F F F Y ÀI ,giving:
t g Y g H Àg Y g I Àg Y F F F Y g ÀI Àg X解除劳动关系协议书
P
Next,we assume that differences g p Àg are independent of g ,which allows us to factorize (2):
%t g t g H Àg Y g I Àg Y F F F Y g ÀI Àg X
Q
In practice,an exact independence is not warranted;hence,the factorized distribution is only an approximation of the joint distribution.However,we are willing to accept the possible small loss in information as it allows us to achieve invariance with respect to shifts in gray scale.Namely,the distribution t g in (3)describes the overall luminance of the image,which is unrelated to local image texture and,conquently,does not provide uful in-formation for texture analysis.Hence,much of the informa-tion in the original joint gray level distribution (1)about the textural characteristics is conveyed by the joint difference distribution [28]:
%t g H Àg Y g I Àg Y F F F Y g p ÀI Àg X
R
This is a highly discriminative texture operator.It records the occurrences of various patterns in the neighbor-hood of each pixel in a Edimension l histogram.For constant regions,the differences are zero in all directions.On a slowly sloped edge,the operator records the highest difference in the gradient direction and zero values along the edge and,for a spot,the differences are high in all directions.
Signed differences g p Àg are not affected by changes in mean luminance;hence,the joint difference
distribution is invariant against gray-scale shifts.We achieve invariance with respect to the scaling of the gray scale by considering just the signs of the differences instead of their exact values:
%t s g H Àg Y s g I Àg Y F F F Y s g ÀI Àg Y
S
where
s x
I Y x !H H Y x `H X
&
T
By assigning a binomial factor P p for each sign s g p Àg ,we transform (5)into a unique vf Y number that characterizes the spatial structure of the local image texture:
vf Y
ÀI p H
s g p Àg P p X
U
The name ªLocal Binary Patternºreflects the function-ality of the ,a local neighborhood is thresholded at the gray value of the center pixel into a binary pattern.vf Y operator is by definition invariant against any monotonic transformation of the gray ,as long as the order of the gray values in the image stays the same,the output of the vf Y operator remains constant.
If we t ( V Y I ),we obtain vf V Y I ,which is similar to the vf operator we propod in [27].The two differences between vf V Y I and vf are:1)The pixels in the neighbor t are indexed so that they form a circular chain and 2)the gray values of the diagonal pixels are determined by interpolation.Both modifications are neces-sary to obtain the circularly symmetric neighbor t,which allows for deriving a rotation invariant version of vf Y .
2.2Achieving Rotation Invariance
The vf Y operator produces P different output values,corresponding to the P different binary pattern
s that can be formed by the pixels in the neighbor t.When the image is rotated,the gray values g p will correspondingly move along the perimeter of the circle around g H .Since g H is always assigned to be the gray value of element (H Y )to the right of g rotating a particular binary pattern naturally results in a different vf Y value.This does not apply to patterns comprising of only 0s (or 1s)which remain constant at all rotation angles.To remove the effect of ,to assign a unique identifier to each rotation invariant local binary pattern we define:
vf ri
Y min f y vf Y Y i
j i H Y I Y F F F Y ÀI g Y
V
沙河校区where y xY i performs a circular bit-wi right shift on the E it number x i times.In terms of image pixels,(8)
OJALA
ET AL.:MULTIRESOLUTION GRAY-SCALE AND ROTATION INVARIANT TEXTURE CLASSIFICATION WITH LOCAL BINARY PATTERNS 973
Fig.1.Circularly symmetric neighbor ts for different ( Y ).
simply corresponds to rotating the neighbor t clockwi so many times that a maximal number of the most significant bits,starting from g ÀI,is0.
vf ri Y quantifies the occurrence statistics of individual rotation invariant patterns corresponding to certain micro-features in the image;hence,the patterns can be considered as feature detectors.Fig.2illustrates the36unique rotation invariant local binary patterns that can occur in the ca of ,vf ri V Y can have36different values.For example,pattern#0detects bright spots,#8dark spots and flat areas,and#4edges.If we t I,vf ri V Y I corresponds to the gray-scale and rotation inv
账账相符ariant operator that we designated as vf y in[29].
2.3Improved Rotation Invariance withªUniformº
Patterns and Finer Quantization of the Angular
Space
Our practical experience,however,has shown that vf y as such does not provide very good discrimina-tion,as we also concluded in[29].There are two reasons: The occurrence frequencies of the36individual patterns incorporated in vf y vary greatly and the crude quantization of the angular space at RS intervals.
We have obrved that certain local binary patterns are fundamental properties of texture,providing the vast majority,sometimes over90percent,of all QÂQ patterns prent in the obrved textures.This is demonstrated in more detail in Section3with statistics of the image data ud in the experiments.We call the fundamental patternsªuniformºas they have one thing in common, namely,uniform circular structure that contains very few spatial transitions.ªUniformºpatterns are illustrated on the first row of Fig. 2.They function as templates for microstructures such as bright spot(0),flat area or dark spot(8),and edges of varying positive and negative curvature(1-7).
To formally define theªuniformºpatterns,we introduce a uniformity measure (ªpatternº),which corresponds to the number of spatial transitions(bitwi0/1changes)in theªpattern.ºFor example,patterns HHHHHHHH P and IIIIIIII P have value of0,while the other ven patterns in the first row of Fig.2have value of2as there are exactly two0/1transitions in the pattern.Similarly,the other27patterns have value of at least4.We designate patterns that have value of at most2asªuniformºand propo the following operator for gray-scale and rotation invariant texture description instead of vf ri Y :
vf riu P
Y
ÀI
p H
s g pÀg if vf Y P
I otherwi Y
&
W where
vf Y j s g ÀIÀg Às g HÀg j
ÀI
p I
j s g pÀg Às g pÀIÀg j X
IH
Superscript riu P reflects the u of r otation i nvariant ªu niformºpatterns that have value of at most2.By definition,exactly Iªuniformºbinary patterns can occur in a circularly symmetric neighbor t of pixels. Equation(9)assigns a unique label to each of them corresponding to the number ofª1ºbits in the pattern (H33 ),while theªnonuniformºpatterns are grouped under theªmiscellaneousºlabel( I).In Fig.2,the labels of the ªuniformºpatterns are denoted inside the patterns.In
practice,the mapping from vf Y to vf riu P
Y
,which has P distinct output values,is best implemented with a lookup table of P elements.
The final texture feature employed in texture analysis is the histogram of the operator ,pattern labels) accumulated over a texture sample.The reason why the histogram ofªuniformºpatterns provides better discrimi-nation in comparison to the histogram of all individual patterns comes down to differences in their statistical properties.The relative proportion ofªnonuniformºpat-terns of all patterns accumulated into a histogram is so small that their probabilities cannot be estimated reliably.
974IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,VOL.24,NO.7,JULY
2002
Fig.2.The36unique rotation invariant binary patterns that can occur in the circularly symmetric neighbor t of vf ri V Y .Black and white circles correspond to bit values of0and1in the8-bit output of the operator.The first row contains the nineªuniformºpatterns and the numbers inside them correspond to their unique vf riu P
V Y
codes.
Inclusion of their noisy estimates in the dissimilarity analysis of sample and model histograms would deteriorate performance.
We noted earlier that the rotation invariance of
vf y vf ri V Y I is hampered by the crude RS
quantiza-tion of the angular space provided by the neighbor t of eight pixels.A straightforward fix is to u a larger since the quantization of the angular space is defined by (QTH a ).However,certain c
onsiderations have to be taken into account in the lection of .First, and are related in the n that the circular neighborhood corresponding to a given contains a limited number of pixels (e.g.,nine for I ),which introduces an upper limit to the number of nonredundant sampling points in the neighborhood.Second,an efficient implementation with a lookup table of P elements ts a practical upper limit for .In this study,we explore values up to 24,which requires a lookup table of 16MB that can be easily managed by a modern computer.
2.4
Rotation Invariant Variance Measures of the Contrast of Local Image Texture
The vf riu P
Y operator is a gray-scale invariant ,its output is not affected by any monotonic transformation of the gray scale.It is an excellent measure of the spatial pattern,but it,by definition,discards contrast.If gray-scale invariance is not required and we wanted to incorporate the contrast of local image texture as well,we can measure it with a rotation invariant measure of local variance:
e Y
I ÀI p H
g p À" P Y where " I
ÀI p H
g p X
II
e Y is by definition invariant against shifts in gray
scale.Since vf riu P
Y and e Y are complementary,their
joint distribution vf riu P
Y
a e Y is expected to be a very powerful rotation invariant measure of local image texture.Note that,even though we in this study restrict ourlves to
using only joint distributions of vf riu P
Y
and e Y operators that have the same ( Y )values,nothing would prevent us from using joint distributions of operators computed at different neighborhoods.
2.5Nonparametric Classification Principle
In the classification pha,we evaluate the dissimilarity of sample and model histograms as a test of goodness-of-fit,which is measured with a nonparametric statistical test.By using a nonparametric test,we avoid making any,possibly erroneous,assumptions about the feature distributions.There are many well-known goodness-of-fit statistics such as the chi-square statistic and the q (log-likelihood ratio)statistic [33].In this study,a test sample was assigned to the class of the model w that maximized the log-likelihood statistic:
v Y w
f I
log w Y IP
where f is the number of bins and and w correspond to
the sample and model probabilities at bin ,respectively.
Equation (12)is a straightforward simplification of the q (log-likelihood ratio)statistic:
q Y w P
f I log
w P f I
log À log w Y
IQ
where the first term of the righthand expression can be
ignored as a constant for a given .
v is a nonparametric pudometric that measures like-lihoods that sample is from alternative texture
class,bad on exact probabilities of feature values of preclassi-fied texture models w .In the ca of the joint distribution
vf riu P
Y
a e Y ,(12)was extended in a straightforward manner to scan through the two-dimensional histograms.Sample and model distributions were obtained by scanning the texture samples and prototypes with the chon operator and dividing the distributions of operator outputs into histograms having a fixed number of f bins.
Since vf riu P
Y has a fixed t of discrete output values (H 3 I ),no quantization is required,but the operator outputs are directly accumulated into a histogram of P bins.Each bin effectively provides an estimate of the probability of encountering the corresponding pattern in the texture sample or prototype.Spatial dependencies between adjacent neighborhoods are inherently incorpo-rated in the histogram becau only a small subt of patterns can reside next to a given pattern.
Variance measure e Y has a continuous-valued output;hence,quantization of its feature space is needed.This was done by adding together feature distributions for every single model image in a total distribution,which was divided into f bins having an equal number of entries.Hence,the cut values of the bins of the histograms corresponded to the (IHH af )percentile of the combined data.Deriving the cut values from the total distribution and allocating every bin the same amount of the combined data guarantees that the highest resolution of quantization is ud where the number of entries is largest and vice versa.The number of bins ud in the quantization of the feature space is of some importance as histograms with a too small number of bins fail to provide enough discriminative information about the distributions.On the other hand,since the distributions have a finite number of entries,a too large number of bins may lead to spar and unstable histograms.As a rule of thumb,statistics literature often propos that an average number of 10entries per bin should be sufficient.In the experiments,we t the value of f so that this condition is satisfied.
2.6Multiresolution Analysis
We have prented general rotation-invariant operators for characterizing the spatial pattern and the contrast of local image texture using a circularly symmetric neighbor t of pixels placed on a circle of radius .By altering and ,we can realize operators for any quantization of the angular space and f
巨蟹座头像女or any spatial resolution.Multiresolution analysis can be accomplished by combining the information provided by multiple operators of varying ( Y ).
In this study,we perform straightforward multiresolu-tion analysis by defining the aggregate dissimilarity as the
OJALA ET AL.:MULTIRESOLUTION GRAY-SCALE AND ROTATION INVARIANT TEXTURE CLASSIFICATION WITH LOCAL BINARY PATTERNS 975
