An Introduction to Multinsor Data Fusion DA VID L.HALL,SENIOR MEMBER,IEEE,AND JAMES LLINAS
Invited Paper
Multinsor data fusion is an emerging technology applied to Department of Defen(DoD)areas such as automated target recognition,battlefield surveillance,and guidance and control of autonomous vehicles,and to non-DoD applications such as monitoring of complex machinery,medical diagnosis,and smart buildings.Techniques for multinsor data fusion are drawn from a wide range of areas including artificial intelligence,pattern recognition,statistical estimation,and other areas.This paper provides a tutorial on data fusion,introducing data fusion applica-tions,process models,and identification of applicable techniques. Comments are made on the state-of-the-art in data fusion.
I.I NTRODUCTION
In recent years,multinsor data fusion has received significant attention for both military and nonmilitary appli-cations.Data fusion techniques combine data from multiple nsors,and related information from associated databas, to achieve improved accuracies and more specific infer-ences than could be achieved by the u of a single nsor alone[1]–[4].The concept of multinsor data fusi
on is hardly new.Humans and animals have evolved the capability to u multiple ns to improve their ability to survive.For example,it may not be possible to asss the quality of an edible substance bad solely on the n of vision or touch,but evaluation of edibility may be achieved using a combination of sight,touch,smell, and taste.Similarly,while one is unable to e around comers or through vegetation,the n of hearing can provide advanced warning of impending dangers.Thus multinsory data fusion is naturally performed by animals and humans to achieve more accurate asssment of the sur-rounding environment and identification of threats,thereby improving their chances of survival.
While the concept of data fusion is not new,the emer-gence of new nsors,advanced processing techniques,and improved processing hardware make real-time fusion of data increasingly possible[5],[6].Just as the advent of symbolic processing computers(viz.,the SYMBOLIC’s Manuscript received April23,1996;revid October14,1996.
D.L.Hall is with the Applied Rearch Laboratory,The Penn-sylvania State University,University Park,PA16802USA(e-mail: dlh28@psu.edu).
J.Llinas is with the State University of New York,Buffalo,NY14260 USA(e-mail:llinas@acsu.buffalo.edu).
Publisher Item Identifier S0018-9219(puter and the Lambda machine)in the early1970’s provided an impetus to artificial intelligence[119],recent advances in computing and nsing have provided the ability to emulate,in hardware and software,the natural data fusion capabilities of humans and animals.Currently, data fusion systems are ud extensively for target tracking, automated identification of targets,and limited automated reasoning applications.Spurred by significant expenditures by the Department of Defen(DoD),data fusion technol-ogy has rapidly advanced from a loo collection of related techniques,to an emerging true engineering discipline with standardized terminology(e Fig.1),collections of robust mathematical techniques[2]–[4],and established system design principles.Software in the area of data fusion applications is becoming avavailable in the commercial marketplace[16].
Applications for multinsor data fusion are widespread. Military applications include:automated target recognition (e.g.,for smart weapons),guidance for autonomous vehi-cles,remote nsing,battlefield surveillance,and automated threat recognition systems,such as identification-friend-foe-neutral(IFFN)systems[14].Nonmilitary applications include monitoring of manufacturing process,condition-bad maintenance of complex machinery,robotics[129], and medical applications.Techniques to combine or fu data are drawn from a diver t of more traditional dis游戏闪退怎么办
ciplines including:digital signal processing,statistical estimation,control theory,artificial intelligence,and classic numerical methods[16],[12],[54].Historically,data fusion methods were developed primarily for military applications. However,in recent years the methods have been applied to civilian applications,and there has been bidirectional technology transfer[5].Various annual conferences pro-vide a forum for discussing data fusion applications and techniques[7]–[10].
In principle,fusion of multinsor data provides signifi-cant advantages over single source data.In addition to the statistical advantage gained by combining same-source data (e.g.,obtaining an improved estimate of a physical phenom-ena via redundant obrvations),the u of multiple types of nsors may increa the accuracy with which a quantity can be obrved and characterized.In the accompanying
0018–9219/97$10.00©1997IEEE
Fig.1.Table of terminology and
八拜之交H
definitions. Fig.2.FLIR and radar nsor data correlation.
Fig.2[2],a simple example is provided of a moving object, such as an aircraft,obrved by both a puld radar and an infrared imaging nsor.The radar provides the ability to accurately determine the aircraft’s range,but has a limited ability to determine the angular direction of the aircraft. By contrast,the infrared imaging nsor can accurately determine the aircraft’s angular direction,but is unable to measure range.If the two obrvations are correctly associated(as shown in the central part of thefigure), then the combination of the two nsors data provides an improved determination of location than could be obtained by either of the two independent nsors.This results in a reduced error region as shown in the fud or combined location estimate.A similar effect may be obtained in determining the identity of an object bad on obrvations of an object’s attributes.For example,there is evidence that bats identify their prey by a combination of factors that include size,texture(bad on acoustic signature),and kinematic behavior.
The most fundamental characterization of data fusion involves a hierarchical transformation between obrved energy or parameters(provided by multiple sources as
Fig.3.Inference hierarchy.
input)and a decision or inference(produced by fusion estimation and/or inference process)regarding the lo-cation,characteristics,and identity of an entity,and an interpretation of the obrved entity in the context of a surrounding environment and relationships to other entities (e Fig.3).The definition of what constitutes an entity depends upon the specific application under consid
,an enemy aircraft for a tactical air-defen application,or the location and characteristics of a tumor in a medical diagnosis application).The transformation between obrved energy or parameters and a decision or inference proceeds from an obrved signal to progressively more abstract concepts.In a target tracking application,for example,multinsor energy,converted to obrvations of angular direction,range,and range-rate may be converted in turn into an estimate of the target’s position and velocity (using obrvations from one or more nsors).Similarly, obrvations of the target’s attributes,such as radar cross ction,infrared spectra,and visual image,may be ud to classify the target,and allow a feature-bad classifier to declare an assignment of a label specifying target identity (e.g.,F-16aircraft).Finally,understanding the motion of the target and its relative motion with respect to the obrver,may allow a determination of the intent of the ,threat,no-threat,etc.).
The determination of the target’s position and velocity from a noisy time-ries of measurements constitute a classical statistical estimation problem[68],[62],[63]. Modern techniques involve the u of quential estimation techniques such as the Kalmanfilter or its variants.To establish target identity,a transformation must be made between obrved target attributes and a labeled identity. Methods for identity estimation involve pattern recognition techniques bad on clustering algorithms,
neural networks, or decision-bad methods such as Bayesian inference [107],Dempster–Shafer’s method[111],[130],[110],or weighted decision techniques[3].Finally,the interpreta-tion of the target’s intent entails automated reasoning us-ing implicit and explicit information,via knowledge-bad methods such as rule-bad reasoning systems[116]–[118]. Obrvational data may be combined,or fud,at a variety of levels from the raw data(or obrvation)level to a state vector level,or at the decision level.Raw nsor data can be directly combined if the nsor data are ,if the nsors are measuring the same physical phenomena such as two visual image
nsors Fig.4.Example of Monte Carlo evaluation of data fusion ben-
efits.
or two acoustic nsors).Techniques for raw data fusion typically involve classic detection and estimation methods. Converly,if the nsor data are noncommensurate,then the data must be fud at a feature/state vector level or decision level.
Feature-level fusion involves the extraction of repren-tative features from nsor data.An example of feature extraction is the u of characteristics of a human’s face to reprent a picture of the human.This technique is ud by cartoonists or political satirists to evoke recognition of famousfigures.There is evidence that humans utilize a feature-bad cognitive function to recognize objects.In feature-level fusion,features are extracted from multiple nsor obrvations,and combined into a single concate-nated feature vector which is input to pattern recognition approaches bad on neural networks,clustering algorithms, or template methods.
Finally,decision level fusion involves fusion of nsor information,after each nsor has made a preliminary deter-mination of an entity’s location,attributes,and identity.Ex-amples of decision level fusion methods include weighted decision methods(voting techniques),classical inference, Bayesian
inference,and Dempster–Shafer’s method. Qualitative advantages of data fusion for DoD systems have been cited by numerous authors.Waltz[1],for ex-ample,cites the following benefits for tactical military systems;robust operational performance,extended spatial coverage,extended temporal coverage,incread confi-,of target location and identity),reduced ambigu-ity,improved target detection,enhanced spatial resolution, improved system reliability,and incread dimensionality. Waltz performed Monte Carlo numerical studies to show the quantitative utility of data fusion for improved noncooper-ative target recognition(e Fig.4),leading to advantages in tactical air-to-air engagements.
Despite the qualitative notions and quantitative calcu-lations of improved system operation by using multiple nsors and fusion process,actual implementation of effective data fusion systems is far from simple.In practice, fusion of nsor data may actually produce wor results than could be obtained by tasking the most appropriate nsor in a nsor suite.This is caud by the attempt
Fig.5.DoD applications sumary.境遇是什么意思
to combine ,good data)with inaccurate or biad data,especially if the uncertainties or variances of the data are unknown.Quantitative evaluation of the effectiveness of data fusion system
must,in most cas,be performed by Monte Carlo simulations or covariance error analysis techniques[3],[46],[47].Fundamental issues to be addresd in building a data fusion system for a particular application include:
1)what algorithms or techniques are appropriate and
optimal for a particular application;
2)what architecture should be ,where in the
processingflow should data be fud);
3)how should the individual nsor data be procesd
年终奖个税税率表
to extract the maximum amount of information;
4)what accuracy can realistically be achieved by a data
闭上你的嘴打一字fusion process;
5)how can the fusion process be optimized in a dynamic
n;
6)how does the data collection ,signal
propagation,target characteristics,etc.)affect the
processing;
7)under what conditions does multinsor data fusion
improve system operation?
This paper provides a brief overview of multinsor data fusion technology and its applications.An introduction to data fusion techniques is provided along with a discussion of some fundamental issues.Some projections for the future of data fusion are provided along with an asssment of the state-of-the-art and state-of-practice.
II.M ILITARY A PPLICATIONS OF D ATA F USION
Two broad communities have focud on data fusion for specific applications:DoD and non-DoD.We will
address Fig.6.An example of multinsor ocean surveillance.
cpu风扇调速
each of the in turn,and provide examples of applications. The DoD community focus on problems involving the location,characterization,and identification of dynamic entities such as emitters,platforms,weapons,and military units.The dynamic data are often termed an Order-of-Battle databa or Order-of-Battle display(if superimpod on a map display).Beyond achieving an Order-of-Battle databa,DoD urs ek higher level inferences about the enemy ,the relationships among entities, their relationships with the environment,higher level en-emy entity organizations,etc.).Examples of DoD related applications include ocean surveillance,air-to-air defen, battlefield intelligence,surveillance,and target acquisition, and strategic warning and defen(e Fig.5).Each of the military applications involves a particular focus,n-sor suite,desired t of inferences,and a particular t of challenges.
Ocean surveillance systems are designed to detect,track, and identify ocean-bad targets and events.Examples
Fig.7.Overview of non-DoD application.
include antisubmarine warfare systems to support Navy tacticalfleet operations(Fig.6),and automated systems to guide autonomous vehicles.Sensor suites may include radar,sonar,electronic intelligence(ELINT),obrvation of communications traffic(COMINT),infrared,and synthetic aperture r
adar(SAR)obrvations[100].The surveillance area for ocean surveillance may encompass hundreds of nautical square miles,and a focus on air,surface,and subsurface targets.Multiple surveillance platforms may also be involved with numerous targets tracked.Challenges to ocean surveillance involve the large surveillance volume, the combination of targets and nsors,and the complex signal propagation environment—especially for underwater sonar nsing.An example of an ocean surveillance system is shown in Fig.6.
Air-to-air and surface-to-air defen systems have been developed by the military to detect,track,and identify aircraft and anti-aircraft weapons and nsors.The de-fen systems u nsors such as radar,passive elec-tronic support measures(ESM),infrared,identification-friend-foe(IFF)nsors,electro-optic image nsors,and visual(human)sightings.The systems support counter-air,order-of-battle aggregation,assignment of aircraft to raids,target prioritization,route planning,and other activi-ties.Challenges to the data fusion systems include enemy countermeasures,the need for rapid decision making,and potentially large combinations of target-nsor pairings.A special challenge for IFF systems is the need to confi-dently and noncooperatively identify enemy aircraft.The proliferation of weapon systems throughout the world,and the subquent lack of relationship between the nationality of weapon origin and combatants who u the weaponry, caus incread IFF challenges.
Another application,Battlefield Intelligence,Surveil-lance,and Target Acquisitions systems attempt to detect and identify potential ground targets.Examples include the location of land mines and automatic target recognition of high value targets.Sensors include airborne surveillance via Moving Target Indicator(MTI)radar,synthetic aperture radar,passive electronic support measures,photo reconnaissance,ground-bad acoustic nsors,remotely piloted vehicles,electro-optic nsors,and infrared nsors. Key inferences sought are information to support battlefield situation asssment and threat asssment,and cour-of-action estimation.
A detailed discussion of DoD data fusion applications can be found in the collected annual Proceedings of the Data Fusion Systems Conference[7],Proceedings of the National Symposium on Sensor Fusion[9],and various strategic documents[15].
III.N ONMILITARY A PPLICATIONS OF D ATA F USION摇一摇怎么关闭
A cond broad community which address data fusion problems is the academic/commercial/industrial commu-nity.This diver group address problems such as the implementation of robotics,automated control of industrial manufacturing systems,development of smart buildings, and medical applications(e Fig.7),among other evolving applications.As with the
店庆活动方案military applications,each of the applications has particular challenges,nsor suites,and implementation environments.
Remote nsing systems have been developed to identify and locate entities and objects.Examples include systems to monitor agricultural ,the productivity and health of crops),to locate natural resources,and to monitor weather and natural disasters.The systems rely primarily
