Realtime 3D profile measurement by using the composite pattern bad on the binary stripe pattern
Jing Xu a ,n ,Bingtuan Gao b ,Jinhua Han a ,Jianguo Zhao b ,Shaoli Liu a ,Qiang Yi a ,Zhengda Zhao c ,Huabin Yin c ,Ken Chen a
a
巨蟹女摩羯男Department of Precision Instruments and Mechanology,Tsinghua University,Beijing 100084,China
b
Department of Electrical and Computer Engineering,Michigan State University,East Lansing,MI 48824,
USA c
Department of Manufacturing Engineering,AVIC Chendu Aircraft Industrial(Group)Co.,Ltd,Chendu,Sichuan 610091,China
a r t i c l e i n f o
Article history:
Received 23July 2011Received in revid form 24August 2011
跳远运动员Accepted 5September 2011
Available online 29September 2011Keywords:
Composite pattern Pha shifting
Binary stripe pattern
青莲山a b s t r a c t
In this paper,a one-shot composite pattern bad on three-step binary stripe pattern is developed to meet the demand of real-time 3D shape measurement for complicated objects.Theoretical analysis and numerical simulation prove that the propod pattern can provide higher accuracy than the composite pattern bad on the traditional three-step pha shifting.Furthermore,the significant advantage for the propod composite pattern is that it can provide the unique location of the stripe boundary,resulting in the elimination of the ambiguity which usually aris for measurement of complicated object with the pha shifting method.The propod composite pattern has been implemented to measure three different complicated objects.The results demonstrate that it is appropriate for the real-time 3D shape measurement.
&2011Elvier Ltd.All rights rerved.
1.Introduction
Over the past decade,vision-bad 3D profile measurement technology has been increasingly applied in manufacturing indus-tries.The 3D profile of a part,which can be reprented by a t of points (a point cloud),is needed in two areas:quality inspection or rever engineering.Among the existing vision-bad 3D measurement approaches,the structured light-bad measure-ment system,
including a projector and a camera,is more and more ud thanks to its capability of rapid-speed,den-resolu-tion,high-accuracy,and low-cost measurement performances.The above performance is determined by the pattern encoding strategy.The function of the encoded pattern is to build the accurate and reliable correspondence between the camera and the projector,which is the prerequisite of the 3D shape recon-struction by means of triangulation measurement technique.Therefore,the encoding strategy is of great importance for this technique.
According to the number of the patterns required for the correspondence establishment,existing pattern encoding meth-ods can be categorized into two types:multi-shot and one-shot pattern encoding methods.The one-shot pattern encoding method can be applied to the 3D profile measurement of the
moving objects such as the parts on the product line.Therefore,the one-shot pattern encoding strategy is the emphasis in this paper.
To achieve one-shot pattern,a direct-coding approach is a feasible solution,where each pixel has a unique color value in the single pattern,which means a mass of color is required.The captured color in the camera,however,does not only depends on the projected color from the projector,but also is aff
ected by the color property of the scanned object.Therefore,this approach is nsitive to the ambient illumination and the scanned surface property,leading to low reliability.As a result,it is not suitable for quality inspection.
Another solution is to encode the single pattern by using the spatial-neighborhood strategy,in which the primitive is the basic unit of the encoded pattern.The codeword of each primitive is determined by its own value,as well as the values of its neighboring primitives.The first kind of primitive is bad on the color,including colored stripe pattern,colored grid pattern,colored slit pattern,colored spot pattern,etc.[1–4].The major weakness of such a primitive is that the discrimination of the color depends on the colors of ambient illumination and the scanned surface.To improve the measurement reliability,the cond kind of primitive is bad on the geometry,such as circle,stripe [5],X-point [6],etc.However,the limitation of geometry-bad encoding strategy is the scattered resolution of the reconstructed point cloud.The primitives in the single pattern are arranged by the De Bruijn quence,pudorandom quence,
Contents lists available at SciVer ScienceDirect
journal homepage:/locate/optlastec
Optics &Lar Technology
0030-3992/$-e front matter &2011Elvier Ltd.All rights rerved.doi:10.1016/j.optlastec.2011.09.005
n
Corresponding author.Tel.:þ861062796126.E-mail address:jingxu@tsinghua.edu (J.Xu).
Optics &Lar Technology 44(2012)587–593
or M-arrays.It should be pointed out that the De Bruijn quence and pudorandom quence are usually ud as the one-dimen-sional encoding strategy,whereas the M-arrays is for two-dimen-sional encoding strategy.
地下拳击手At prent,the pha-shifting method(PSM)is the most com-monly ud pattern in3D profile measurement since it has pixel-level resolution,robustness to the variation of the surface property due to its pixel-independent reconstruction.For this technique,the minimum number of patterns required is three,which does not allow us to directly apply it to real-time measurement[7].To reduce the number of patterns,a straightforward way is to combine the multiple pha shifting patterns into a
single pattern to achieve the real-time measurement[8,9].The resulting composite pattern, however,still has the common drawback of the ,only the relative pha(the unique location in one sub-patch)can be determined,instead of the absolute pha(the unique location in the entire image).However,the absolute pha is required for3D profile reconstruction,especially for parts with a complicated shape. As a result,the conventional composite pattern bad on pha shifting is not appropriate for the3D shape measurement of complicated objects.
The Fourier Transform Profilometry(FTP)is another typical example of the one-shot pattern[10];however,the spectrum aliasing phenomena will intensively reduce the measurement accuracy.Besides,veral other pattern encoding methods for one-shot pattern have been reported[11,12].
To the best of our knowledge,a composite pattern with high accuracy and absolute pha is still missing from the literature, motivated by the facts,we develop a one-shot composite pattern bad on the binary(black/white)stripe pattern for real-time3D measurement in this paper.Compared with the traditional compo-site pattern bad on PSM,the propod composite pattern has two significant advantages:the higher accuracy and the unique location (absolute pha)in the entire image.In the following ctions,the conventional composite pattern bad on three-step pha shifting is referred
to as conventional composite pattern,while the propod composite pattern bad on the three-step binary stripe pattern is abbreviated to propod composite pattern.
2.Pattern encoding
2.1.Previous rearch
In our previous study,we propod a three-step binary stripe pattern which took advantage of both temporal and spatial encoding strategies such that the stripe boundary could be uniquely specified in the entire image[13].For this technique, the value of the stripe boundary is specified by the intensity variance of the three patterns in time domain,while the code-word of the stripe boundary is determined by its own value and the values of the stripe boundaries in space domain.The devel-oped patterns are the three top binary patterns as shown in Fig.3. The resolution of the propod pattern is768Â1024pixels, where the pitch of each stripe is6pixels,thus,there are128 stripes in total in the entire image.
To precily locate the stripe boundary,there exist at least two patterns containing two successive stripes with the opposite intensities as illustrated in Fig.1,where A,B,C,and D are the intensities of the successive pixels n and nþ1on the stripe boundary in two patterns.Therefore,the location P of th
e stripe boundary along the axis-y can be detected with a sub-pixel accuracy by
P¼nþ
AÀB
ðAÀBÞþðCÀDÞ
趣味素描ð1Þ
Compared with the PSM,there are two significant advantages for
the propod three-step binary stripe pattern:it can provide the
unique location of the stripe boundary in the entire image,leading
to the ambiguity elimination of the pha shifting;besides,it has a
high accuracy and the sophisticated anti-noi capability.However,
the propod three-step binary stripe pattern still cannot satisfy the
requirement of the real-time3D profile measurement due to the
number of the pattern is three.To solve this problem,this paper
aims to combine the three binary stripe patterns into a single
pattern without loss of accuracy.
2.2.Composite pattern
For the structured light-bad measurement system(including a
camera and a projector),the projector is considered as an inver
camera.Under the simplest ca,assume that the focal lengths of
the projector and the camera are equal;the image planes of the
projector and the camera are coplanar with both y-axes parallel to
the baline(the line connecting two optical centers).In this ca,
the epipolar line falls along the vertical scanline of the images as
shown in Fig.2.Thus,for the pixel in the projector,the correspond-
ing pixel must lie on the same vertical scanline of the camera.The
three binary patterns are encoded along the baline,and the three
carrier patterns(the sinusoid waveform with different frequencies)
are encoded perpendicular to the baline.
The modulated pattern in each channel is the production of the
binary pattern and the carrier pattern;and then the composite
pattern I P in the projector is the sum of the three modulated
patterns as shown in Fig.3.Therefore,we have:
I P¼AþBÁ
X3
n¼1
I b
n
Ácosð2p f p n x pÞð2Þ
y
冬天之美
1
n
Fig.1.Stripe boundary location.
Fig.2.Epipolar constraint for the composite pattern.
J.Xu et al./Optics&Lar Technology44(2012)587–593
588
where A and B are the scale factor ud to adjust the brightness and contrast of the projected composite pattern;I n b is intensity of the
binary pattern n ;f n p
is the modulation frequency in the channel n in the projector;x p reprents the image coordinate in the axis-x .
To simplify the image processing and improve the accuracy,the demodulation is performed row-by-row.If the scanned sur-face has the same height,then the captured image in the camera is same with that in the projector.Taking the modulated pattern in each channel into account,the intensity variation in each row of the image will be a standard sinusoid waveform with a certain frequency when the binary pattern is 1.Similarly,the intensity is constant when the binary pattern is 0.
On the other hand,the height variation of measured object will make the projected modulated pattern deform in the camera.Under the constraint of the epipolar geometry,the corresponding pixel is constricted along the vertical scanline.Conquently,the captured modulated pattern in each channel becomes ,the intensity along axis-x in some pixel will be abnt as the dashed line shown in Fig.4(a).Specifically,the solid line corresponds to 1in the square waveform while the dash line denotes 0in the square waveform along axis-x as shown in Fig.4(b).The are required to be specified to establish the correspondence between the camera and the projector.Thus,the intensity I c n ðx p ,y p Þin the each row of the captured modulated pattern for each channel in the camera can be reprented by
I c n ðx p ,y p Þ¼I A þI B ÁI s
n ðx p ,y p ÞÁcos ð2p f n x P Þ
ð3Þ
where I A and I B are the image brightness and contrast of the captured pattern;I s n ðx p ,y p Þis the square waveform reflecting the height variance of the measured object.
To measure the 3D shape of an object,the deformed square waveform (binary stripe pattern)has to be exacted from the captured composite pattern.To this end,the captured composite pattern in spatial domain is converted to that in frequency domain by Fourier
transform.The peak centered at the carrier frequency f n c
for each channel can be obtained as shown in Fig.7.It should be noted that
the carrier frequency f n c in the camera is different from f n p
in the projector,depending on the resolutions of the projector and the camera.A band-pass filter,which allows low frequency to pass well
but cuts off the high frequency around the carrier frequency f n c
,is ud to extract modulated pattern in each channel.Then,the Hilbert transform is applied to eliminate the negative frequency and double the positive frequency.Last,the inver Fourier transform is ud to demodulate the square waveform in the spatial domain,resulting in 3D shape recovery.
Now,we will discuss how the demodulated square waveform,containing a wide range of harmonics,determines the accuracy of the boundary location.The magnitude of the channel leakage in the frequency domain relies on the deformation level of the scanned surface.In other words,the higher frequency is con-tained in the square waveform,the more channel leakage exists.To minimize the channel leakage,a proper carried frequency should be chon to parate the three channels as far as possible in frequency domain.Theoretically,the three lected carrier frequencies should be uniformly distributed on the frequency domain.In practice,both camera and projector tend to attenuate high frequencies,leading to the reconstruction corruption.In this ca,the high carrier frequency should be discarded and the optimal frequency depends on the hardware in the experiment.However,the channel leakage cannot be avoided completely since the captured modulated pattern (the multiplication of sinusoid and the square waveform)in each channel is likely t
o cover the entire frequency domain due to the height variation of the measured object.Apparently,the high frequency portion of each deformed square waveform will be cut off during the demodula-tion process.The high frequency portion means that the sharp intensity gradient in a small region,whereas the low frequency portion has a constant intensity or a relative gentle gradient in a small region.Thus,the frequency specifies the width of the rising edge and falling edge of the square waveform.It is found out that the gradient of the edge of the square waveform demodulated becomes more gentle than the real one.
For our propod three-step binary stripe pattern,two pat-terns with opposite intensities exist in the neighboring area of a stripe boundary along the axis-x .Assume that pattern 1and pattern 2have the opposite intensities along the axis-x .In
other
Fig.3.Propod composite
pattern.
Fig.4.Deformed composite pattern:(a)modulated pattern;(b)square waveform.
J.Xu et al./Optics &Lar Technology 44(2012)587–593589
words,they have the same frequency spectrums along the axis-x in the frequency domain.Therefore,the channel leakage has equivalent effect to the demodulated square waveform 1and square waveform 2.Thus,the demodulated square waveforms in pattern 1and pattern 2still hav
e the opposite intensities,that is,the amplitudes of variation are equal as shown in Fig.5.For example,the intensity pairs of A and B in the same axis-x in Fig.1probably become A 0and B 0with the constraint of A ÀA 0¼B 0ÀB ¼a .Similarly,the equation C ÀC 0¼D 0ÀD ¼b is true.Therefore,the location detected by the demodulated stripe pattern can be reprented by
P 0
¼n þA 0ÀB 0A 0ÀB 0þC 0ÀD
0¼n þA ÀB À2a
A À
B þ
C À
D À2a À2b ð4Þ
The location error of the demodulated pattern is still constricted within one pixel,although the error exists,compared with Eq.(1).
3.Experimental results and discussion
The first experiment is to conduct the numerical simulation to compare the conventional composite patterns and the propod composite pattern.The deformation of the composite pattern is controlled by Fig.6,where the height denotes the deformation level of the composite pattern similar to [9].
To simulate the real condition of the 3D measurement,the same random noi is added to the two composite patterns.The resolution of the pattern is 768Â1024.The carrier frequency of three channels is 80,160,and 240in the projector,while the carrier frequency in the camera image is as shown in Fig.7.
For three-step pha shifting method,the patterns projected yield
I 1¼I ave þI mod cos j À2p 3 I 2¼I ave þI mod cos ðj ÞI 3¼I ave þI mod cos j þ
2p 8
>>>>><
>>>>>:ð5Þ
where I ave and I mod are the average intensity and intensity modulation.The pha can be solved by
j ¼arctan ffiffiffi3p ðI 1ÀI 3Þ
三到2I 2ÀI 1ÀI 3 !
ð6Þ
The intensities I 1,I 2,and I 3are inevitably changed due to the channel leakage.From Eq.(4),the calculated pha relates to I 1,I 2,and I 3.Furthermore,the maximum error cannot be constricted within 1pixel.
The simulated errors of two composite patterns are shown in Fig.8.The results show that the errors both exist in two composite patterns.For the propod composite pattern,error mainly aris on the boundary of the deformation region,with the maximum error about 0.8pixel;on the other hand,for the conventional composite pattern,the maximum error is around 1.5pixel.Therefore,the accuracy of the propod composite pattern is higher than that of the conventional composite pattern.Furthermore,the propod pattern allows us to directly measure the complicated object,that i
s,the pha difference between the adjacent pixels probably greater than 2p .The result agrees with the theoretical analysis that the propod composite pattern
can
Fig.6.Pattern deformation controller.
050100
150200250300
−1
012345678910x 10
7
frequency
m a g n i t u d e
Fig.7.Carried frequency in the camera image.
Fig.8.Error comparison:(a)propod composite pattern;(b)conventional
composite pattern.
1
Fig.5.Calculated unique pha through combination of step waveform.
J.Xu et al./Optics &Lar Technology 44(2012)587–593
590
reduce the influence of the channel leakage and improve the measurement accuracy in the last ction.
The parameter gamma of the optical components has to be corrected to reduce the nonlinear influence of the sinusoid waveform.The gamma relates the input voltage and the output brightness,which is usually reprented by a power function [14],who exponent is the gamma ðg Þof the projector.In this paper,the power function is improved by adding the ambient illumina-tion as:I c ¼a ðI p Þg þb
ð7Þ
where I c is the captured linear intensity value in the camera;I p reprents the nonlinear input intensity in the projector;a and b are the coefficient and constant factor.To be specific,a ¼0.002952,g ¼1:954,and b ¼7.255are calibrated in this study.Fig.9illustrates the relation between the intensity captured in the camera and the input intensity in the projector before and after gamma correction.
It implies that the intensity captured in the camera is propor-tional to that projected in the projector after gamma correction.The nonlinear influence has been greatly reduced by the gamma correction,resulting in the measurement accuracy improvement.
After gamma correction,the standard sinusoid waveform in the projector is illuminated onto a reference plane and generate the curve as shown in Fig.10,where the dashed line is the real
sinusoid waveform in the camera and the solid line is the ideal one.It is found out that the frequency of the captured sinusoid waveform is approximately equal to ideal one which means the gamma correction is done properly.
The cond experiment is carried out to verify the real-time measurements capability of the complicated parts,including a piece of car body,a bent pipe,and a round tank.
The dimension of the car body is around 700Â700mm with relatively smooth surface,but the hole in the car body makes the pha distribution discontinuous,which probably will cau incorrect 3D reconstruction with conventional composite pattern.Therefore,the car body is ideal for validating our propod composite pattern.The composite pattern is projected onto the scanned surface and the three binary stripe patterns are extracted from the captured composite pattern automatically,and then the unique stripe boundary is obtained to recover the 3D
profile,
Fig.9.Intensity relation between camera and projector:(a)before gamma correction;(b)after gamma correction.
200
300
400
500
600700800900
0.05
0.10.150.20.250.30.350.4x (pixel)
i n t e n s i t y
real ideal
Fig.10.Propod composite pattern.
Fig.11.Car body 3D shape measurement:(a)measured part;(b)point cloud.
J.Xu et al./Optics &Lar Technology 44(2012)587–593591
which is as shown in Fig.11.The result shows that the 3D shape of the car body can be reconstructed,including the boundary region.
The other two samples,the bent pipe and the round tank,have the common geometrical feature:round in shape with sharp contour.Figs.12and 13show the part and the corresponding recons
truction for the two parts,respectively.From the two figures,we can e that the smooth surface can be well rebuilt with a den point cloud.However,the density of point cloud near the contour is reduced.The reason is that the high frequency near the contour results in the frequency leak into other channels.In this ca,the intensities of some pixels in the demodulated three binary stripe pattern are decread.To guarantee the reliability of the 3D measurement,tho pixels are discard such that the resolution of the point cloud in the high frequency area is lower than that in the low frequency area.
The significant merit of the propod pattern is one-shot property,that is,the time of image acquisition approximates to that of camera exposure (around 10ms in our trial)in on-line mode.Besides,the data processing (pattern decoding and 3D reconstruction)takes much longer time (around 3s in our trial)in off-line mode.So,it costs approximately 3s in total.However,only the image acquisition will lead to errors for moving objects,which is the reason why our system can be applied to moving objects.
4.Conclusion
This paper aims to develop a one-shot composite pattern bad on binary stripe pattern for real-time 3D profile measure-ment.To this end,the Fourier transform is ud to integrate three-step binary strip
e pattern.However,similar to the compo-site pattern bad on pha shifting,the channel leakage tends to take place,resulting in the reconstruction error.We focus on the error analysis of the propod composite pattern bad on the binary stripe pattern through theoretical analysis,numerical simulation,and experimental verification.The experimental results demonstrate the propod composite pattern is superior to the conventional composite pattern in both accuracy and measurement performance of discontinuous surface since the pha shifting has the 2p ambiguity.In the future,we will study the accuracy improvement of the propod composite pattern.
Acknowledgments
摄影专业就业前景如何This work was supported by National Science Foundation of China (NSFC),Grant 51105218,50975148,51105219,
51005126.
Fig.12.Bent pipe 3D shape measurement:(a)measured part;(b)point
cloud.
Fig.13.Round tank 3D shape measurement:(a)measured part;(b)point cloud.
J.Xu et al./Optics &Lar Technology 44(2012)587–593
592
