I NSTITUTE OF P HYSICS P UBLISHING M EASUREMENT S CIENCE AND T ECHNOLOGY Meas.Sci.Technol.14(2003)1631–1639PII:S0957-0233(03)62381-7
Two-step calibration of a stereo camera system for measurements in large volumes M Machacek,M Sauter and T R¨o sgen
Institute for Fluid Dynamics,Sonneggstras3,ETH Z¨u rich,Switzerland
E-mail:machacek@hz.ch
Received17April2003,infinal form12June2003,accepted for
publication19June2003
Published29July2003
Online at stacks.iop/MST/14/1631
Abstract
Three-dimensional measurements with a stereo camera system are an
established standard practice and are ud in a wide range of different
applications.However,the calibration of the camera parameters,a crucial
step in the procedure carried out in order tofind the relation between the real
world and the digital image coordinates,is still associated with veral
difficulties.The include the accuracy,the implementation of the
algorithms and the calibration procedure.A new method is prented taking
into account in particular large measurement volumes where it is difficult to
find a suitable calibration target and cas where frequent recalibrations are
necessary becau of camera position changes.The method is bad on a
two-step lf-calibration with two different calibration targets,a planar
target and a two-point target.It is shown that this approach creates an
easy-to-implement algorithm and at the same time a simple calibration
procedure.A thorough asssment of the accuracy of the two-step
calibration method is prented.
Keywords:camera calibration,lf-calibration,two-point target
1.Introduction
Three-dimensional measurements with a stereo camera system are an established standard practice and are ud in a wide range of different Bruner et al2000,Liu et al2000).Before any measurement can be conducted the a priori unknown parameters of the camera model,describing the geometric relation between the scene and the camera images,have to be determined in some way.An accurate calibration is important forfinding reliable correspondence points and for an accurate three-dimensional reconstruction. However,a theoretically ideal camera calibration is often not feasible becau of circumstances such as tight time schedules,difficulties in accessing the measurement area and/or large measurement volumes.The dominant problem is that of the calibration target.If a photogrammetric calibration is employed a calibration target with points o
f known positions in the scene must be ud.The accuracy of the calibration depends directly on the characteristics of the target.A good calibration can only be ensured if the calibration point coordinates are accurately known and if the points are distributed inside the entire volume relevant for the measurement.While it is easy to construct a target for measurements in small volumes,considerable difficulties ari for measurement volumes where the size is larger than1m3(Machacek and R¨o sgen2003).An adequate calibration target for this ca is difficult to manipulate and expensive to produce with the accuracy needed for a reliable calibration.To overcome the problem of being dependent on a calibration target,advanced methods have emerged known as lf-calibration(Chen and Li2003).In the methods the information for the calibration is taken for example from known image point correspondences(the points can be from a calibration target or from the natural scene)in a multiple-view t-up or from a priori known features such as perpendicular or parallel lines.However,lf-calibration methods require more sophisticated algorithms and they are often subject to a compromi,such as regarding the choice of the camera parameters.In this paper it is shown that a flexible calibration method can also be obtained by a rather simple scheme,providing at the same time a camera model with no compromi regarding the parameters implemented.
0957-0233/03/091631+09$30.00©2003IOP Publishing Ltd Printed in the UK1631
M Machacek et al
First the inner parameters of the camera are calibrated with a photogrammetric method.Since the parameters do not depend on the camera positioning,the calibration can be done ‘off-line’with a calibration target of a suitable(small)size.In a cond step the external parameters are calculated using a lf-calibration.The natural features(corners,edges)of a scene generally vary strongly in quality and in number.In some cas there might be no natural features at all.As a conquence a certain calibration accuracy cannot be guaranteed.Therefore, to achieve independence of the environment in which the measurements are taken,a very simple mobile calibration target is ud for the lf-calibration.The reason that not all parameters are calibrated in a one-step lf-calibration is the resulting complexity of the optimization problem.A more sophisticated non-linear optimization algorithm and a solution with possibly less accuracy would be the conquence.An attempt at joint estimation of the inner and external parameters is described in Borghe and Cerveri(2000).
A fundamental assumption of the two-step calibration is that the inner parameters do not change after their calibration. This is precily fulfilled when the camera ttings,such as the focal length,the focus and the lens aperture,are not changed.However,becau the calibration object for the inner parameters is much smaller than the measurement volume,the focus has to be changed when the c党建工作制度
ameras are moved to the final t-up.In this ca the inner parameters do change.For a qualitatively good lens this change turns out to be small,so two-step calibration becomes practicable.
2.Camera model
The standard camera model ud in computer vision,the pinhole camera,is assumed for the prent ca.The model establishes the mathematical relationship between a real-space point X∈P3and its image x∈P2created in the image plane of an imaging device such as a digital camera.The pinhole camera model describes a perspective projection and can be formulated for homogeneous coordinates as a linear mapping. However,the linear approach does not offer enoughflexibility to accurately model a digital camera/lens configuration since the lens can introduce a significant distortion of the image. For a lens with a focal length of8mm as ud in the prent investigation,the radial distortion shift at the image corners is approximately nine pixels(figure1).Therefore the distortion has to be accounted for with a non-linear correction.The mapping relation from a point˜X=(X,Y,Z,1)T to pixel coordinates˜x p=(x p,y p,1)T becomes then
˜x =
R1−R1C1
01
˜X,(1)
x c=x+x(k1r2+k2r4)+p1(r2+2x2)+2p2xy,
劣的反义词y c=y+y(k1r2+k2r4)+p2(r2+2y2)+2p1xy,
(2)
˜x p= 1/p
h
0c h/2 0−1/p w c w/2
001
× −f s
x−sin(θ)x00
0−f cos(θ)y00
0010
˜x c,(3)
where˜x =(x,y,1)T are the distorted image coordinates and
x c=(x c,y c,1)T the distortion-corrected image coordinates1.
R1is the3×3rotation matrix containing the three angles
(ω,ϕ,κ)describing the camera orientation and C1=
赶集作文(X0,Y0,Z0)T the position of the camera centre.This t of
camera parameters is referred to as the external parameters.
The lens distortion is described by the coefficients k1and k2
of the radial distortion and by the coefficients p1and p2of
the tangential distortion;r=
x2+y2is the radius.The
parameter f denotes the focal length,s x is the scaling of the x-
axis,x0and y0are the offt between the principal axis and the
origin of the image plane andθis the skew angle between the
x-and y-axes.Thefive parameters are referred to as the inner
parameters and form the camera matrix.For the transformation
from image coordinates to pixel coordinates,the parameters
c w,c h describing the CCD chip width an
d height in pixel units
and p w,p h describing the pixel width and height in metric units
are introduced.This camera model gives a total of15a priori
unknown parameters,6parameters accounting for the camera
position and orientation(external or extrinsic parameters),5
parameters accounting for the specific properties of the camera
(internal or intrinsic parameters)and4parameters taking the
lens distortion into account.The pixel dimensions(p w,p h)
are assumed to be known.This t of15parameters provides
an accurate description of the mapping properties of a digital
camera.
The definition of the camera matrix as in equation(3)is
just one possible option among others.The important point
is that the matrix must contain a full diagonal left-hand3×3
sub-matrix to define a general affine transformation.Other
definitions either u fewer matrix elements,in which ca
a special affine transformation is described,or a different
parameter assignment of the matrix elements,in which ca
only the interpretation of the camera parameters is different.
For other possible camera matrix definitions e Hartley and
Zisrman(2000)or Faugeras and Luong(2001).
男男恋爱
The distortion model bad on four parameters
(k1,k2,p1,p2)works well for most lens but will produce
bad results for extremely wide-angle lens such asfish-eye
lens and an appropriate model should be ud in that ca.
More parameters can be taken into account;however,the
first coefficients of the radial and tangential distortion already
account for the major part of the distortion shift(figure1)
and taking more than four coefficients into account would
introduce a source of instability in the optimization routines
for the camera calibration.
3.Calibration method
3.1.First step—estimation of the internal parameters
While the photogrammetric calibration gives an estimate for
the entire camera parameter t,the external parameters are
not ud in the context of the prent method.This is becau
the camera position is changed after the photogrammetric
calibration and as a conquence the external parameters also
change.
1Herein,bold italic denotes a vector and bold sanrif a matrix,a tilde
indicates homogeneous coordinates,a numerical subscript the camera to which
the variable is related,the prime a variable in camera1coordinates and the
double prime a variable in camera2coordinates.
1632
Two-step calibration of a stereo camera system for measurements in large volumes
D i s t o r t i o n [p i x e l ]
Position [pixel]
Figure 1.Comparison of the different components of the
non-linear distortion depending on the position on the image
diagonal for the lens ud in the prent experiments (focal length 8mm,aperture 1.4):——:first-order radial component;
–––:cond-order radial component;–·–:first-order tangential component;······:cond-order tangential component.
The photogrammetric camera calibration us a t of points X i ,i =1,...,N ,with known global coordinates for determining the camera parameters in a one-step approach.The points are defined through the markers of a calibration object.The quality of the calibration is directly related to the properties of the calibration target (marker shape,marker size,number N of markers ud,distribution of the markers in the measurement volume)and to the scheme of marker coordinate extraction by image processing routines.For an accurate measurement,a large number of markers must be ud and the markers must be distributed over the entire measurement volume.It is important that the markers do not lie exclusively in a plane,becau a degenerate configuration would be created and the equations cannot then be solved for the camera parameters.
The points x i determined by the equations (1),(2)and (3)applied to the known global points X i and the measured points ˆx i are then ud to estimate the camera model parameters such that the projection error
12N i =1
f 2
i (4)
is minimized,where
f i =
(x p i −ˆx
p i )2+(y p i −ˆy p i )2.(5)
Here f i denotes the distance between the imaged control points
ˆx p i and the points x p i calculated with the camera model.
This non-linear minimization problem with 15unknown variables reprenting the internal and external camera parameters is solved with a standard Marquardt–Levenberg algorithm.The convergence of the solution depends on the initial guess of the camera parameters.A good estimate for the inner parameters is (x 0,y 0)=(0,0),s x =1,θ=0,k 1=0,k 2=0,p 1=0and p 2=0.For the focal length f the value given by the manufacturer is generally fairly clo to the true value and is a good initial estimate.For the external parameters it is more difficult to find intuitively a good guess (particularly for a new camera t-up).For that reason a pre-calibration with a linear camera model,which does not need any initial estimate of the parameters,can be ud.The linear
camera model is equal to the non-linear one,except that no distortion due to the lens is considered.
To stabilize the convergence,the optimization is initially performed with fixed internal camera parameters;thereafter one internal parameter is relead and the optimization is restarted.This procedure is repeated until all internal parameters are relead.The monitoring of the projection error (11)during this stepwi optimization showed that all internal camera parameters contribute significantly to a reduction of the projection error.
3.2.Second step—external parameters
For the calculation of the external parameters (X 0,Y 0,Z 0,ω,ϕ,κ)a lf-calibration method bad on the information on point correspondences between two views and a reference length is ud.This method is bad on the normalized eight-point algorithm described in detail by Longuet-Higgins (1981),Hartley (1997)and on the calibration target propod by Borghe and Cerveri (2000).The target consists of two markers mounted on a rod at a known relative distance from each other forming a pair of corresponding points required by the normalized eight-point algorithm.Such a target is straightforward to build without any restrictions due to the measurement volume size.Some specific aspects related to one-dimensional objects are given by Zhang (2001).
A fundamental assumption of the two-step calibration is that the inner parameters do not change after their calibration.This is precily fulfilled when the camera lens ttings,such as the focal length,the focus and the lens aperture,are not changed.However,becau the calibration object for the inner parameters is much smaller than the measurement volume,the focus has to be changed when the camera is moved to the final t-up.In this ca the inner parameters are likely to change.For a qualitatively good lens this change is assumed to be small,so the two-step calibration is practicable.The lf-calibration problem is stated as follows:for the calibration of the external parameters,2N point correspondences and N measurements of a known length L are given.The point correspondences are ud to estimate the esntial matrix and thereafter the scale ambiguity of the esntial matrix is resolved with the length measurements.
3.2.1.Esntial matrix estimation.The objective is to determine the esntial matrix given a t of image point correspondences between two cameras,{x n i ↔x n i },i =1,...,N ,such that the definition of the esntial matrix
˜x T n E ˜x
n =0,(6)
is fulfilled and the specific properties of an esntial matrix,that is five degrees of freedom and inclusion of a single unknown scale factor,are met.The properties translate into a matrix with two equal singular values and a third singular value equal to zero.A significant characteristic of the esntial matrix relation is that it is defined for normalized coordinates (indicated with the subscript ‘n’).Normalized coordinates are defined as the image of a point created with a perfect pinhole camera (as defined by the equation (1))with the focal length f = 1.Thus normalized coordinates only depend on the描写烟花的段落
1633
M Machacek et al
camera orientation and position.From (6)an epipolar line can be defined as
˜e 2=E ˜x n .(7)The epipolar line is esntially the projection of the point x n
from the first camera into the coordinate system of the cond camera.
Given the corresponding points,the coordinates are first normalized and then transformed through a translation and a scaling such that their mean value is equal to (0,0)and their standard deviation is e
qual to √2.For a more detailed motivation for this transformation,referred to as normalization,e Hartley (1997).
The problem of finding E can be formulated as
A e =0
(8)
where A contains the homogeneous coordinates of the N measurement points and e contains the esntial matrix coefficients.The solution is found by a singular value decomposition (SVD)of the matrix A .The derived matrix does not yet have the properties of an esntial matrix and this constraint must thus be impod afterwards on the matrix derived.Hartley and Zisrman (2000)propo replacing the initially found matrix E with a matrix E 1that minimizes the Frobenius norm E −E 1 subject to the condition det (E 1)=0.This can be conveniently done with a SVD E =UDV T where D =diag (a ,b ,c )contains the singular values.The new esntial matrix is E 1=UD 1V T with the replaced singular
values D 1=diag (a +b 2
,
a +b
2,0).After the esntial matrix conditions are enforced,the matrix E 1is de-normalized.
This method ud to impo the esntial matrix characteristics onto a matrix is the most straightforward one.More sophisticated procedures exist,such as the algebraic minimization algorithm.For further details,refer to Hartley and Zisrman (2000).
3.2.2.Esntial matrix decomposition.After the esntial matrix is found,the external parameters of both cameras are determined by an esntial matrix decomposition.Simultaneously the scale ambiguity of the esntial matrix is resolved with the N measurements of the target length L .
The esntial matrix is decompod into a rotational and a translatory part which correspond to the rotation matrix and the translation vector t describing the transformation between the two cameras.This decomposition is done by means of a SVD as described in detail by Faugeras and Luong (2001).It is assumed that the global coordinate system is located at the camera coordinate system origin of the first camera.As a conquence the rotation matrix of the first camera is equal to a unit matrix and the translation vector is equal to a zero vector.Therefore the decomposition of the esntial matrix will directly give the position of the cond camera.The decomposition gives four po
ssible solutions with a scale ambiguity,reprented by the factor λ,of the translation vector t .The correct solution from the four choices is found by means of the condition that a point reconstructed from both camera views must lie in front of both cameras.The scale ambiguity λis found by the minimization of the error in the distance estimate L between the two marker points of the calibration bar:
1
2
N i =1
δ2i (λ),(9)where
δi (λ)=L − X 1i (λ)−X 2i (λ) .
(10)
Here,X 1i and X 2i are the reconstructed calibration points of
the bar.This non-linear optimization problem can be solved with a Marquardt–Levenberg algorithm.
4.Accuracy
To asss the quality of the calibration method it is necessary to define suitable error criteria.In general the criteria ud in the optimization procedures to solve for the parameters are also ud as quality criteria.In the following,various calibration error definitions are given for a validation and comparison between different methods.
For the photogrammetric calibration,the quality can be estimated by the averaged Euclidean distance between the measured points ˆx and the projected points x calculated with the camera model and the known global coordinates X :
p =
1N N
i =1
ˆx i −x i .(11)
This error criterion,also called the re-projection error,is bad on the calibration of a single camera a
nd does not give any information about the accuracy of a multiple-camera t-up.Since the global coordinates of the calibration markers are known,an error criterion for a multiple-camera t-up can be defined as the averaged Euclidean distance between the given global coordinates X and the reconstructed global coordinates ˆX
: m =1N N
i =1 ˆX i −X i .(12)
This criterion is an estimate of the metric calibration accuracy.A different criterion for a two-camera t-up is bad on the definition of the esntial matrix (equation (6)):
e =1N N i =1
|˜x T
n i E ˜x n i |,(13)
or alternatively on the Euclidean distance between the epipolar
line ˜e =[e x ,e y ,e z ]T and the corresponding point x n i :
d =
1N N i =1
e x i ˜x n i +e y i ˜x n i +e z i
e 2x i
+e 2
y i
.(14)
The error criterion bad on the esntial matrix is an indicator of the projective reconstruction quality for a two-camera t-up.In the ca of the lf-calibration the three-dimensional coordinates of the calibration markers are not known;hence the re-projection error (11)and the metric reconstruction error (12)cannot be calculated.Instead,the error criteria characterizing the projective reconstruction (13),(14)are ud.If the
1634
Two-step calibration of a stereo camera system for measurements in large volumes
Figure2.The translation of the planar calibration target for the
photogrammetric calibration.
(Thisfigure is in colour only in the electronic version)
lf-calibration includes a length measurement,the metric
reconstruction accuracy can be assd using
δav=1
解渴的饮料N
N
i=1
(L−ˆL i),(15)
δdev=
1
2N
N
i=1
(L−ˆL i)2,(16)
where L is the known length andˆL is the measured length between two metric reconstructed points.
5.Calibration targets
5.1.Static target
Two different types of planar calibration target(a large and a small one)were ud.The large calibration target has a size adequate for the large volume where the stereoscopic measurements were conducted and is ud as a reference for the two-step calibration.The small calibration target is ud to calibrate the inner parameters for the two-step calibration and the target size is chon such that a fast and convenient calibration can be obtained.
The large target was a sandwich construction made of a 3mm aluminium sheet functioning as carrier,a retro-reflective foil layer and a black foil layer as cover.The cover had cut-out holes,made wi
th a high-precision plotter,to create the calibration markers.The plate had a width of1.55m,a height of1.1m and280markers with a diameter of10mm.
The cond,smaller calibration plate was a black anodized aluminium plate with the markers made by drilling0.001mm deep holes into the surface to remove the black anodized layer. The plate had a width of0.4m,a height of0.3m and680 markers with a diameter of1mm.
The calibration plates have to be translated during the calibration procedure in order to create a t of three-dimensionally distributed calibration markers(figure2).The translation increment d is directly related to the Z-coordinate scale and the translation direction t cal is related to the Z-coordinate direction.For a correct calibration the increment d must be precily known and the translation direction t cal must be perpendicular to the plate;henceα=90◦(efigure
2).
Number of planes
ϕ
[
r
a
d
日游i
a
n
]
Figure3.One of the camera orientation angles(ϕ)for various numbers of calibration plane positions and plane translation
increments.A small calibration plate(22×19.5cm)with532 circular markers of1mm diameter was ud.Translation
increments: :d=1mm;:d=2mm; :d=3mm;
•:d=5mm.
5.2.Mobile target
The calibration bar(figure2)was realized with two pinhole LEDs as calibration markers.To enable an accurate LED position measurement from a large range of viewing directions, the LED aperture must be small and the emission angle must be large.The LEDs ud(Cerled SMD)had an aperture of 60µm,a light emission angle of100◦,a luminous intensity of8mCd and a peak emission wavelength of660nm.The LEDs were operated in a puld mode with aflash duration of 270µs to ensure blur-free and well defined images.The pul frequency was controlled by a signal bad on the recording camera’s vertical drive(VD)signal for synchronization.The LEDs were mounted at a precily measured distance L on a fibre carbon bar.
6.Results
The calibration tests were performed for a measurement volume of approximately2m×2m width and height and 1m depth at a distance of2m from the camera pair.The cameras had a relative distance of0.4m from each other. To obtain a direct comparison and reference for the two-step calibration,the large planar calibration target was ud.The calibration of the inner parameters for the two-step calibration was established with the small planar target(ction5.1).
6.1.Photogrammetric calibration
A photogrammetric camera calibration strongly depends on the spatial calibration marker distribution.Since the calibration object is created by translating a planar calibration target,as shown infigure2,the translation increment d and the number of target positions are esntial information.Figures3and4 show the trend of the focus f and the camera orientation angle ϕfor different translation increments d and numbers of target positions.It can be en that with an increasing number of plate positions the parameters converge towards a constant value.For larger translation distances a faster convergence is obrved,whereas for too small translation increments no
1635
M Machacek et
al
Number of planes
f o c u s [m m ]
Figure 4.The calibrated focal length for various numbers of calibration plane positions and plane translation increments.A small calibration plate (22×19.5cm )with 532circular markers of 1mm diameter was ud.Translation increments: :d =1mm;:d =2mm; :d =3mm;•:d =5mm;◦
:d =10mm.
convergence is obrved.The fact of a convergence towards a constant parameter t does not guarantee that an optimal solution is obtained,but it is necessary for a reproducible calibration.
The quality of the calibration was estimated with the re-projection error as defined in equation (11).For a typical calibration with 4–5plane positions and a total of 2000–3000calibration markers,the re-projection error was on average 0.12pixels irrespective of the calibration plate size.For the large-measurement-volume test where a large calibration plate was ud,the metric reconstruction accuracy estimated with equation (12)was around 0.3mm in the plane perpendicular to the obrvation direction and 1.4mm for the depth measurement.This gives an averaged absolute error (Euclidean norm)of 1.5mm.If the measurement volume size is taken as reference,this corresponds to a relative accuracy of 0.02%for the in-plane measurement and 0.07%for the depth measurement.For a comparison with the two-step lf-calibration a photogrammetric calibration of the inner and external parameters with two cameras was conducted and the error d characterizing the quality of the point correspondences (as defined in equation (14))was calculated.The error was found to be on average d =0.36×10−3for normalized coordinates or d =0.3in pixel units.6.2.Self-calibration
The lf-calibration of the external parameters was conducted in the large measurement volume.Figure 5shows a typical example of the calibration bar marker positions as detected in the im
ages of both cameras.The calibration bar length ud in the experiments was 500.2mm.To estimate the effect of the lens adjustment (focus)on the calibration accuracy,the inner parameters were estimated once with the small calibration plate and again,as a reference,with the large calibration plate (ction 5).Since the large calibration target had adequate size for the measurement volume,the lens did not have to be re-adjusted (refocud).It turned out that the difference in the calibration accuracy between the two methods is only marginal.The averaged residual error of the esntial matrix relation was e =0.902×10−3with the small target and e
=Figure 5.Superpod images of the calibration bar from the two cameras.
100
200300400500
600
-4-3-2-101234D i f f e r e n c e L -L i [m m ]
^
Number of corresponding points
Figure 6.The deviation of the estimated calibration bar length from the true bar length for an adjusted (refocud)lens configuration.
0.848×10−3with the large target.For the two calibrations of the external parameters the same image t with 360point pairs of the stick was ud.The standard deviation of the calibration bar length estimation was δdev =2.59mm for the small target compared to δdev =1.38mm for the large target.
In the following the accuracy asssment of the two-step calibration with the small target is given.The calibration quality estimated with the matching error e as defined in equation (13)was on average e =0.6×10−3,with a data t containing on average 320image pairs.If an error above 1×10−3is obtained,it is an indication of a t of calibration images not sufficient for an accurate calibration.If there are one (or more)outliers in the data t with an error e >1×10−2,a poor estimate of the camera calibration parameters can be expected.A more intuitive estimate of the matching error is the distance d between the epipolar line and the corresponding point as given in equation (14).This error was found to be on average d =0.12×10−3(normalized coordinates)(or 0.11(pixels))using the normalized coordinates of 320image pairs.
The metric calibration accuracy of the lf-calibration method can be estimated by the accuracy of th
e calibration bar length measurement.Figure 6shows a typical result for the calibration bar length estimation error for the individual
不将就1636
