Omnidirectional stereo systems for robot navigation
Giovanni Adorni(*)Stefano Cagnoni(+) Monica Mordonini(+)Antonio Sgorbissa(*)
(+)Dept.of Computer Engineering(*)DIST University of Parma University of Genoa
Parma,Italy43100Genoa,Italy16145
Abstract分数
This paper discuss how stereo vision achieved through the
u of omnidirectional nsors can help mobile robot nav-igation providing advantages,in terms of both versatility
and performance,with respect to the classical stereo system
bad on two horizontally-displaced traditional cameras. The paper also describes an automatic calibration strat-egy for catadioptric omnidirectional nsors and results ob-tained using a stereo obstacle detection algorithm devid within a general framework in which,with some limitations, man
y existing algorithm designed for traditional cameras can be adapted for u with omnidirectional nsors. 1.Introduction
The need for robotic nsory systems that provide a global description of the surrounding environment is increasing. In mobile robotics applications,autonomous robots are re-quired to react to visual stimuli that may come from any direction at any moment of their activity,and to plan their behaviour accordingly.This has stimulated growing interest in omnidirectional vision systems[1].Such systems pro-vide the widest possiblefield of view and obviate the need for active cameras that require complex control strategies, at the cost of reduced resolution with respect to traditional cameras,that distribute a smallerfield of view on the same nsor surface.
Recent robotics and surveillance applications in which omnidirectional nsors have been ud effectively,as the only vision nsor or jointly with other higher-resolution non-omnidirectional ones,are described in[2,3,4]. Applications of mobile robotics in which robots rely on vision for safe and efficient navigation share a t of features and requirements,often conflicting with one another,that strongly influence application design criteria.Among the: robots are immerd in a dynamic environment that may change quite rapidly,within and beyond theirfield of action;
robots require high-resolution vision for accurate op-
eration within theirfield of action;
robots need wide-angle vision to be aware of what hap-
pens beyond theirfield of action and to react/plan ac-
cordingly.
Regarding the typical environment where virtually all indoor robotics and most outdoor robotics take place,fur-ther considerations can be made about the natural partial structuration of the space in which mobile robots operate.
Such a space is usually inferiorly delimited by the plane (floor/ground)on which robots move and extends vertically up to where robots can e or physically reach.Thefloor can therefore be assigned the role of reference plane in the main tasks in which mobile robots are routinely engaged during navigation,namely lf-localization,obstacle detec-tion,free-space computation.We could therefore call it a 2D augmented environment,to underline that the two di-mensions along which thefloor extends are privileged with respect to the third dimension.Even within the limita-tions,robots actually operate in a3D environment and their operation can take advantage of3D information.Stereo vi-sion is therefore appealing to veral navigation tasks.令人感动的一件事
However,traditional stereo vision tups,made up of two traditional cameras displaced horizontally,hardly satisfy the above-mentioned requirements of autonomous robotics applications.The u of omnidirectional nsors, besides providing the robot with obvious advantages in terms of lf-localization capabilities,can be extremely u-ful also to extract3D information from the environment us-ing stereo algorithms.
In ction2we introduce a nsor model,bad on the joint u of an omnidirectional nsor and a traditional one, with which powerful stereo algorithms can be implemented.
We then briefly compare such a model with traditional and fully-omnidirectional stereo tups.In ction3we propo
a framework within which a particular class of algorithms
for omnidirectional nsors can be easily developed,as an extension of traditional stereo algorithms,with almost no
1
extra overhead.Such a class of algorithms that are applica-ble to2D augmented environments,which
includes many if not most real-world applications,can be termed the quasi-3D(q3D)class.More precily,it compris algorithms that can exploit the prence,in the environment,of a ref-erence plane for which a transform(the Inver Perspective Transform)exists,which allows for the recovery of visual information through a remapping operation.A fast and sim-ple auto-calibration process that allows for such a mapping is described in ction4.In ction5,as an example,we eventually describe the basics of an efficient obstacle detec-tion algorithm developed within this framework.
2.Hybrid and fully-omnidirectional
stereo vision nsors
Using traditional stereo systems,typically made up of two traditional cameras aligned and displaced along the hori-zontal axis,has veral drawbacks in mobile robot applica-tions.Among them:
the constraints impod by the configuration of the two traditional cameras needed to obtain sufficient dispar-ity often conflict with the general requirements of the applications for which the stereo system is ud;
the resultingfield of view of the stereo system is much smaller than the,already limited,field of view of each of the two cameras.
Thefirst drawback mainly affects robot design,since it requires that a front and a rear side of the robot be clearly defined.This can be a vere limitation when holonomous robots are ud.With a traditional stereo tup,any recon-figuration of the(strongly asymmetric)vision system re-quires that both cameras be repositioned and might possibly call for structural modifications.
The cond drawback is particularly relevant in dynamic environments.If one considers that a robot movement should be ideally exclusivelyfinalized to performing the task of interest,it is immediately evident how penalizing it is for the robot having to move itlf just to cond its own perceptual needs.
Using omnidirectional nsors is beneficial with re-gard to both problems.Here,we consider two mod-els,a hybrid omnidirectional/pin-hole system and a fully-omnidirectional one.
In particular,it is clear that a symmetric coaxial fully-omnidirectional model as the one briefly discusd in c-tion2.1can solve both problems.However,the solution comes at the cost of a lower resolution in the farfield and of the loss of horizontal disparity between the two views, which may be also unacceptable in some
applications.
Figure1:A fully-omnidirectional nsor model(above)and the Inver Perspective Transform(e cti
on3)images of
a simulated RoboCupfield with four robots and a ball ob-
tained with such a mirror configuration(upper nsor below on the left,lower one below on the right).
A way to obtain stereo images,providing the robot with
both low-resolution omnidirectional vision in the farfield and high-resolution vision in the nearfield while keeping thefield of view as wide as possible,is to u a nsor made up of both an omnidirectional camera and a traditional one.
In the following,after showing the results of a simula-tion of a fully-omnidirectional system to provide a feeling of how images acquired by such systems may look like,we describe in details HOPS(Hybrid Omnidirectional/Pin-hole System),a stereo model that tries to achieve a good trade-off,with particular attention to mobile robot applications, between the features provided by omnidirectional and tra-ditional systems.
2.1.Fully omnidirectional model
A fully-omnidirectional stereo model us two omnidirec-
tional nsors for stereo-disparity computation.Infigure1 we show preliminary results of a simulated vision systems made up of two catadioptric omnidirectional nsors.We have taken into consideration a configuration in which the vision nsors are placed one above the other,and share a common axis(figure1,above on the right)perpendicular to the reference plane.
眼字组词2
Figure2:The two hybrid nsor prototypes:HOPS1and HOPS2.
The main drawback of such a coaxial configuration is that it provides no lateral stereo disparity(e ction5), which make obstacles recognizable only exploiting vertical stereo disparity.On the other hand,dealing with a stereo nsor having two nsors with parallel axes is more com-plicated,both in terms of construction,size and calibration.
2.2.Hybrid omnidirectional/pin-hole model HOPS(of which two prototypes are shown infigure2)is a hybrid vision nsor that integrates omnidirectional vision with traditional pin-hole vision,to overcome the limitations of the two approaches.If a certain height is needed by the traditional camera to achieve a reasonablefield of view,the top of the omnidirectional nsor may provide a ba for the traditional CCD-camera bad nsor that can lean on it,as shown infigure2.In the prototype shown i
nfigure2a the traditional camera isfixed and looks down with a tilt angle of about with respect to the ground plane and has afield of view of about.To obtain both horizontal and vertical disparity between the two images,it is positioned off the center of the device.The’blind ctor’caud by the upper camera cable on the lower nsor is placed at an angle of with respect to a conventional’front view’,in order to relegate it to the back of the device.If a lower point of view is acceptable for the traditional camera,it can also be placed below the omnidirectional nsor,provided it is out of the field of view of the latter.The top of the device is easily accessible,allowing for easy substitution of the catadioptric mirror.Conquently,also the camera holder on which the upwards-pointing camera is placed can be moved upwards or downwards,to adjust its distance from the mirror.In the prototype infigure2b,the traditional camera is positioned laterally above the omnidirectional nsor on a holder
that
Figure3:Example of images that can be acquired through the omnidirectional nsor(left)and through the CCD cam-era(right)of the HOPS1prototype.
can be manually rotated.
An example of the images that can be acquired through the two nsors of thefirst prototype is provided infigure3.
The aims with which HOPS was designed are accuracy, efficiency and versatility.The joint u of a standard CCD camera and of an omnidirectional nsor provides HOPS with different and complementary features:while the CCD camera can be ud to acquire detailed information about
a limited region of interest,the omnidirectional nsor pro-
vides wide-range,but less detailed,information about the surroundings of the system.HOPS,therefore,suits veral kinds of applications as,for example,lf-localization or obstacle detection,and makes it possible to implement pe-ripheral/foveal active vision strategies:the wide-range n-sor is ud to acquire a rough reprentation of a large area around the system and to localize the objects or areas of in-terest,while the traditional camera is ud to enhance the resolution with which the areas are then analyd.The different features of the two nsors can be exploited in both
a stand-alone way as well as in a combined u.In particu-
lar,as discusd in ction5,HOPS can be ud as a stereo nsor to extract three-dimensional information about the scene that is being obrved.
3.General framework for stereo algo-
rithm development
Images acquired by the cameras on-board the robots are af-fected by two kinds of distortions:perspective effects and deformations that derive from the shape of the lens through which the scene is obrved.Given an arbitrarily chon reference plane(typically,thefloor/ground on which robots move),it is possible tofind a function that maps each pixel in the image onto the corresponding point of a new image(with coordinates)that reprents a bird’s view of the reference plane.Limiting one’s interest to the reference plane,it is possible to reason on the scene obrving it with no distortions.The most ap-pealing feature,in this ca,is that a direct correspondence
3
between distances on the reconstructed image and in the real world can be obtained,which is a fundamental requirement for geometrical reasoning.This transformation is often re-ferred to as Inver Perspective Transform(IPT)[5,6,7], since perspective-effect removal is the most common aim with which it is performed,even if it actually reprents only a part of the problem for which it provides a solution. If all parameters related to the geometry of the acquisi-tion systems and to the distortions introduced by the camera were known,the derivation of could be straightforward. However,this is not always the ca,most often becau of the lack of an exact model of camera distortion.However, it is often possible to effectively(and efficiently)derive empirically using proper calibration algorithms,as shown in next ction.
The IPT plays an important role in veral robotics appli-cations in whichfinding a relevant reference plane is easy. This is true for most indoor Mobile Service Robotics ap-plications(such as surveillance of banks and warehous, transportation of goods,escort for people at exhibitions and muums,etc.),since most objects which the robot obrves and with which it interacts lie in fact on the same plane sur-face of thefloor on which the robot is moving.Since our system has been mainly tested within the RoboCup1envi-ronment,in the following we will take it as a ca study. In RoboCup everything lies on the playingfield and hardly rai significantly above,as happens,for example,with the ball.Therefore,the playingfield can be taken as a natural reference plane.
In the rest of the paper we will show how a general em-pirical IPT mapping can be applied,even more effectively, also to catadioptric omnidirectional nsors.The intrinsic distortion of such nsors,especially with respect to the typ-ical images with which humans are ud to dealing,makes direct image interpretation difficult,since a different refer-ence system(polar coordinates)is implicitly’embedded’in the images thus produced.However,their circular symme-try allows for a simplification of the IPT computation. Exploiting this feature in implementing the IPT for cata-dioptric omnidirectional nsors,we have devid an effi-cient automatic calibration algorithm that will be described in the next ction.
4.Omnidirectional nsor calibration
In computing,the generalization of the IPT for a cata-dioptric omnidirectional nsor,the problem is complicated by the non-planar profile of the mirror;on the other hand, the circular simmetry of the device provides the opportunity of dramatically simplifying such a procedure.
怎么考驾照
If the reflecting surface were perfectly manufactured,it would be sufficient to compute just the restriction of
Figure4:The pattern ud for calibrating a catadioptric om-nidirectional nsor(above).The fourth square from the center has a different color,to act as a landmark in auto-matically computing distances;below it the IPT image ob-tained after calibration is shown.The black circle hides the expansion of the area,roughly corresponding to the robot footprint,who reflection is removed in the original image by providing the mirror with a discontinuity in its center.
ing the pattern are known exactly,the only requirement is that one of the shapes,at known distance,be ,by its color)from the others.The shape should be possibly located within the highest-resolution area of the nsor.This permits to u the reference shape as a land-mark to automatically measure the distance from the cam-era of every shape on the reference plane.This also removes the need to accurately position the robot at a predefined dis-tance from the pattern,which could be a further source of calibration errors.
Operatively,in thefirst step of the automatic calibration process,the white stripe,as well as the center of every ref-erence shape,are easily detected.The reference points are inrted into the t of samples on which interpolation is then performed.The actual position of such points can be simply derived from the knowledge of the relative posi-tion of the square pattern to which it belongs with respect to the reference differently-colored shape.The process can be repeated for different heading
苹果手机激活查询s of the robot,simply turning the robot around its central symmetry axis.
In the cond step,interpolation is performed to compute the function from the point t extracted as described.
A look-up table that associates each pair of coordinates in
the IPT-transformed image to a pair of coordinates in the original image can thus be computed.
This calibration process is fast,can be completely auto-mated and provides good results,as shown infigure4. 5.Experiments with an IPT-bad ob-
stacle detection algorithm for om-
nidirectional nsors
As an example of algorithm porting from traditional stereo systems to omnidirectional ones using the generalized IPT, we report some sample results,obtained in a robot soccer environment,of a stereo algorithm for obstacle detection developed for traditional stereo systems[5]and adapted for u with HOPS.The algorithm is described in details el-where[9]:here we mainly aim at showing its potentials and evidentiating the role played by the generalized IPT.
Besides removing the distortion introduced by the omni-directional nsor using the IPT,the algorithm exploits the intrinsic limitation of the IPT to be able to provide undis-torted views only of the objects that lie on one reference plane.Everything that is above the plane is distorted dif-ferently as a function of its height and of the point of view from which it is obrved.Therefore,two IPT-transformed images of the same scene will differ only in tho regions that reprent ,any object located above the reference plane.In mobile robotics applications,the refer-ence plane is chon to be thefloor on which the robots are moving.
宅相Given two images of the same spatial region that in-cludes thefloor on which a robot is moving,the obstacle detection algorithm can be roughly summarid as follows:
plane identified by thefloor;
2.apply an edge extraction algorithm to the IPT-
transformed images;
3.skeletonize and binarize the contours using a ridge-
following algorithm;
tained in the previous step.
When the chromatic features of the two images obtained from the two nsors are virtually identical the steps2and 3of the algorithm can also be substituted by a thresholding algorithm by which objects that clearly stand out with re-spect to the background are highlighted.It is worth noting that the task to have”identical”chromatic features is not easy to achieve in hybrid systems,where one image is ac-quired directly while the other is acquired as a reflection on
a surface that may alter colors to some extent.
5
a)
b)
c)
d)
e)
Figure5:Obstacle detection:(a)images acquired by the
hybrid vision nsor;(b)the IPT of the spatial region in
(a)common to both images;(c)results of edge detection
applied to(b);(d)result of the ridge extraction from(c);(e)
difference between the two images in(d).
The white regions that can be obrved in the differ-
ence image,that reprent areas where an obstacle may be
prent,derive from two kinds of disparity that can be found
in stereo image pairs.If they derive from a lateral displace-
ment of the two cameras,they are located to the left and/or
right of obstacle projections in the IPT transformed images;
becau of this,both approaches ud to obtain binary dif-
ference images considered above provide very similar re-
sults.When a vertical displacement of the two cameras oc-
cur instead,such regions are located above and/or below the
孕妇喝柠檬水好吗obstacle
projections.
Figure6:Above:simulated results obtained by a coaxial
fully-omnidirectional system.The two IPT images(up-
per nsor on the left,lower on the right)of a simulated
RoboCup environment.Below:the difference image that
can be obtained with the coaxial configuration.The virtu-
ally null lateral disparity can be clearly noticed.
From the considerations and using other kinds of in-
lor)it is possible to tell regions that are
certainly free from regions that may be occupied by obsta-
cles.Figure5shows the results that can be obtained at the
end of each step.
To give aflavor of the potential of the algorithm when
applied to a fully-omnidirectional stereo device,infigure6
the difference images is shown,which was obtained by IPT-
transforming the(simulated)images taken from the two
nsors,and subquently computing and pre-processing
the difference between the two images.In particular,the
results of the difference between the lf-reflections of the
robot onto the two mirrors have been removed.
6.Discussion
In this paper we have described a Hybrid
冬凌草胶囊Omnidirectional/Pin-hole Sensor(HOPS)and a gen-
eral framework within which the IPT is ud to allow for
porting the”quasi-3D”(q3D)class of stereo algorithms
from traditional stereo systems to omnidirectional or
partially-omnidirectional ones.
The joint u of a standard CCD camera and of an omni-
directional nsor provides HOPS with their different and
complementary features:while the CCD camera can be
ud to acquire detailed information about a limited region
of interest(”foveal vision”),the omnidirectional nsor pro-
vides wide-range,but less detailed,information about the
surroundings of the system(”peripheral vision”).HOPS,
6
