Jointly published by Akadémiai Kiadó, Budapest Scientometrics, Vol. 72, No. 1 (2007) 117–147 and Springer, Dordrecht DOI: 10.1007/s11192-007-1700-5
Measuring rearcher interdisciplinarity
A LAN L. P ORTER, A LEX S. C OHEN, J. D AVID R OESSNER, M ARTY P ERREAULT*想自己创业
National Academies Keck Futures Initiative (NAKFI), Irvine, CA (USA)
We offer two metrics that together help gauge how interdisciplinary a body of rearch is. Both draw upon Web of Knowledge Subject Categories (SCs) as key units of analysis. We have
asmbled two substantial Web of Knowledge samples from which to determine how cloly
individual SCs relate to each other. “Integration” measures the extent to which a rearch article
cites diver SCs. “Specialization” considers the spread of SCs in which the body of rearch (e.g.,
the work of a given author in a specified time period) is published. Pilot results for a sample of
rearchers show a surprising degree of interdisciplinarity.
昆明陆军讲武堂
Background
Rearch spirals inward; science cascades outward. This dual image supports a vital realization – scientific advance requires knowledge transfer among rearchers who may focus tightly on extremely circumscribed rearch issues. The typical rearch * Alan Porter is an Evaluation Consultant with the National Academies Keck Futures Initiative (NAKFI), and he co-directs the Technology Policy and Asssment Center, Georgia Tech, and is Director of R&D, Search Technology. Alex Cohen, Evaluation Rearch Associate / Programmer, and Marty Perreault, Program Director, are staff with the U.S. NAKFI. David Roessner is the NAKFI Senior Evaluation Consultant, and also co-directs the Technology Policy and Asssment Center at Georgia Tech and is with SRI International as well. Address correspondence relating to the paper to Alan Porter: alan.porter@isye.gatech.edu.; correspondence pertaining to the NAKFI to Marty Perreault, the Program Director: MPerreault@nas.edu.
Received November 16, 2006
Address for correspondence:
A LAN L. P ORTER
Technology Policy and Asssment Center, Georgia Tech, Atlanta, GA (USA)
E-mail: alan.porter@isye.gatech.edu
0138–9130/US $ 20.00
Copyright © 2007 Akadémiai Kiadó, Budapest
朵的笔画
All rights rerved
A. L. P ORTER et al.: Measuring rearcher interdisciplinarity
investigation burrows into detail, cumulating information, probing for nuances. Once completed, the challenge shifts abruptly to communicating the rearch findings. Rearch knowledge needs to be suitably conveyed to others working on the same immediate problem – and to a wider circle of colleagues working on related problems – and beyond, to tho who may gain usable knowledge for different applications from the findings.
Go back for a cond to the time one of us (Porter) was working on his disrtation in Psychology (1970–1972). Most scientists (using the term inclusively to cover social scientists, engineers, and oth
er rearchers) published in the same half-dozen or so journals that they regularly read. Publication process might take a couple of years. The limited quantity of information that one could digest prompted keeping clo track of developments only in one’s immediate rearch area. One didn’t keep up with the Psychology literature; one followed developments in the study of memory, favoring particular organisms as subjects, for certain memory forms, affected by particular stimuli & conditions, etc. The notion of a “rearch trail,” narrowly tracked, made n. At its best, this efficiently directed rearcher efforts to resolving the problem at hand. At its worst, this led to inbred patterns of rearch leading to no lasting value.1 The inherently opposing tendencies – focus vs. outreach – now play out in a richer context than in earlier scientific eras. Wider and faster information access is the pivotal change force. Electronic databas, the internet, and networked computers, accesd daily by esntially every scientist in the world, provide dynamic rearch knowledge resources. Yet, scientific norms suited to earlier eras linger. The scientific disciplines that aro in the 19th and 20th centuries still dominate academic life (somewhat less so for rearchers in government or industry). One’s professional identity, institutional home (department), and external peer group (professional societies) are discipline-centered. However, even in the early 1970’s, the Psychology Department at UCLA did not reflect a unified field. Rather, it was home for, among others, clinical psychologists and neuroscientists who did not share the same paradigm, subject matter, or rearch methods.
The discrepancies between disciplinary roots and rearch realities loom large. In the 1970’s and 1980’s interest in interdisciplinary rearch process picked up momentum. The notion was to identify and overcome barriers to more effective rearch knowledge transfer across domains. Good insights into issues, influences, and mechanisms emerged.2 For whatever reasons (especially decline of rearch funding), the study of interdisciplinary rearch process waned just as funding of major interdisciplinary units expanded (e.g., U.S. National Science Foundation Engineering Rearch Centers and Science & Technology Rearch Centers programs).
Moving to the prent, we find explosive growth in emerging rearch arenas, including genetics & biotechnology, information science & technology, and advanced materials and nanotechnology. Obviously the rearch frontiers pay little heed to 19th 118Scientometrics 72 (2007)
A. L. P ORTER et al.: Measuring rearcher interdisciplinarity century disciplinary boundaries. How can we expedite progress in rearch and development? Various parties at interest e the lowering of disciplinary barriers as vital. This has triggered renewed interest in promoting interdisciplinary rearch and its counterpart, interdisciplinary teaching. In particular, our own efforts derive from the National Academies Keck Futures Initiative (NAKFI) – a $40 million, 15-year program to boost interdisciplinary rearch in the U.S. [www.keckfutures].
NAKFI evaluators have dual interests in measuring interdisciplinarity. At the macro scale, we ek to estimate and track the extent of interdisciplinary rearch (“IDR”) in the U.S. At the micro scale, we want to evaluate whether NAKFI conferences and ed grant programs increa the interdisciplinarity of participants’ rearch (and teaching). But as Morillo et al. note, “we do not have appropriate indicators to measure interdisciplinarity.”3 This paper prents our approach to measuring how inter-disciplinary a particular body of rearch is – whether that of one rearcher, a group of rearchers, or a rearch domain.
Premis in measuring interdisciplinarity
What is interdisciplinarity? Terminology varies and definitions abound. We apply the following definition, bad on a National Academies report:4无花果花
Interdisciplinary rearch (IDR) is a mode of rearch by teams or individuals that integrates
x perspectives/concepts/theories and/or
x tools/techniques and/or
x information/data
from two or more bodies of specialized knowledge or rearch practice. Its purpo is to advance fundamental understanding or to solve problems who solutions are beyond the scope of a single field of rearch practice.
Examples of bodies of specialized knowledge or rearch practice include: low temperature physics, molecular biology, developmental psychology, toxicology, operations rearch, and fluid mechanics.
As Morillo et al. state:3 “we consider ‘multidisciplinarity’ as a basic situation in which elements from different disciplines are prent, whilst ‘interdisciplinarity’ is a more advanced stage of the relationship between disciplines in which integration between them is attained.” Integration is the key concept here – distinguishing the “amless cloth” of IDR, from the “patchwork quilt” of multidisciplinary rearch, and the more restricted focus of disciplinary rearch. We don’t emphasize the creation of new “interdisciplines” per – i.e., transdisciplinary rearch. Elwhere we review earlier findings on IDR, explore the distinctions, and consider the variety of related caus and effects.2,5
Scientometrics 72 (2007) 119
A. L. P ORTER et al.: Measuring rearcher interdisciplinarity
We focus first on the body of rearch of a given rearcher. Specifically, we want to compare the outputs of Rearcher A for a time period (e.g., 3 years) just before participating in a NAKFI program to that of his/her work in a corresponding time period after. That said, the main exemplars of that body of rearch will usually be scientific papers. In most rearch arenas, journal articles and/or conference papers are important. In others, patents are the exemplary outputs. And in some arenas, various other forms reflect the esntial outputs of rearch activity: reports (sometimes confidential, sometimes in “gray literatures”), software, and process improvements (perhaps kept as trade crets).
We acknowledge that rearch is not solely measured by its outputs. Interdisciplinarity can also be gauged in terms of: proposals (e.g., topical emphas, collaboration) or projects (e.g., teaming arrangements, mixed funding sources). In addition, asssment may focus on other interdisciplinary professional facets: teaching (e.g., cross-departmental, co-teaching) or affiliation (e.g., joint appointments in multiple departments, participation in organized rearch units). But for now, our attention is directed to rearch outputs in the form of ts of papers. We will draw heavily on citations (referencing) among papers; this can be considered a form of rearch outcomes.
Our strategy to measure interdisciplinarity (elaborated in succeeding ctions) is bad on the follow
ing premis:
x Integration of knowledge not routinely found within a rearch field equates to greater interdisciplinarity.
x Such integration can be measured by examining the spread of a paper’s references.
x We devi measures by relating papers’ cited journals to their corresponding Subject Categories (rearch fields, as assigned by Thomson
ugly是什么意思
Institute for Scientific Information – ISI – accessible through their Web of
Knowledge – WoK – site).
x The interdisciplinarity metrics take into account the inter-relatedness of Subject Categories.
x Additional measures can be derived from the degree to which cited Subject Categories relate to that of the journal of the paper, and from the spread of
publications across Subject Categories.奇迹的拼音
Empirical aspects in measuring interdisciplinarity
Many information resources are interesting candidates in gauging interdisciplinarity. We briefly note a few. Our focus on rearchers leads us to begin with their CVs. Recent studies of science highlight the wealth of information to which CVs provide entrée.6 On a pilot group of rearchers from two rearch universities, we explored 120Scientometrics 72 (2007)
A. L. P ORTER et al.: Measuring rearcher interdisciplinarity whether some aspects, including educational background (e.g., whether cross-disciplinary or not) and position, correlated with interdisciplinary publication. For example, educational background from
疫情防控先进
B.S. to Doctorate to current position for 6 of the 17 rearchers was cleanly disciplinary (e.g., all degrees and affiliation in the same field). For the other 11, it was more diver (e.g., degrees in Chemistry and Physics, current affiliation in Computer Science/Math department). Surprisingly, papers by tho with relatively pure disciplinary backgrounds emed at least as interdisciplinary. In addition, departmental affiliation aligns with the top ISI publication Subject Categories for only about 10 of 17. So, for now, we t aside the “influence factors” as interest-ing in their own right, but not a strong indicator (or predictor) of IDR rearch output.
Let’s turn to rearch papers. The are probably the most accessible, most clear-cut entities on which to gauge interdisciplinarity of the underlying rearch. Papers can be treated at various degrees of detail, including:
x Bare esntials – #, co-authoring, where published (journal)
x Key text – add information from titles, abstracts, keywords
x Full text – add information from the entire paper
We t aside co-authoring for now. Tho who key on collaboration as the esnce of IDR would likely avail themlves of co-authoring information as the primary measure. We key on integration, and note that this can be accomplished by a single author as well as by a team; so, given our conceptualization of IDR, co-authoring becomes condary.
We considered trying to categorize papers bad on their keywords or classification codes (e.g., as assigned by databa indexers to identify what the paper address), as others have done to good effect.7 In other applications, we have ud such information to cluster papers, identify rearch themes inductively, define rearch domains, and track R&D advances.8,9 We also have the capabil
ity to enrich available keyword ts by extracting noun phras from titles and abstracts – an approach Kostoff and colleagues have ud to locate papers.10 We could also u keyword ts (ontologies) imported from outside sources.11 Such approaches would offer advantages in asssment of papers and patents wherever they appear or are indexed. That said, we prently back off from the approaches becau of their complexity and cost. Much work would be needed to devi a viable methodology to gauge the interdisciplinarity of ts of papers reflecting diver rearch areas. In contrast, the approach we pursue is restricted to content indexed by one key information resource – ISI’s WoK.
Our approach relies on the WoK, which includes the Science Citation Index, Social Science Citation Index, and the Arts and Humanities Citation Index. For a given rearcher, one can arch WoK in two ways:
x general arch – retrieves summary information on papers by that rearcher, including co-authors, co-author affiliations, journal, subject Scientometrics 72 (2007) 121
A. L. P ORTER et al.: Measuring rearcher interdisciplinarity
category, year, and references (first author, year, journal, volume, first
page)
x citation arch – retrieves summary information on each paper that CITES papers authored or co-authored by our rearcher, including authors, title,
journal, subject category, year, keywords, and cited references Key limitations in using WoK include:
x it does not generally include conference papers and does not include patents at all [there is a parate ISI databa containing conference papers] x coverage varies considerably among rearch areas
x keywords are not well controlled
陈妃平Our pilot asssment concluded that coverage was quite good, but not uniform across a wide range of science, engineering, and bio-medical domains. We compared publications between CVs and arch results in WoK. This is not as exact as it might em as CV inclusion is not always complete and variations in citation leave some ambiguity. We deemed coverage to be excellent or very good for 12 of 17 rearchers in our pilot sample. We consider coverage “good” for 4 others (WoK containing from 65-85% of their CV’s journal papers). We assd coverage as “bad” for 1 Sports
Medicine specialist. Medical rearch was quite well-covered (2 judged “excellent” and 1 “good”). Engineering is not covered as extensively by WoK, but it is acceptable – 3 judged “very good” (Chemical Engineering, Biomedical Engineering, Industrial Engineering) and 2 “good” (Electrical and Mechanical Engineering). The pilot results caution against cross-field comparisons without explicit checking of WoK coverage.
We recognize that interdisciplinarity is not a unitary construct with a clear, single indicator. At one extreme, one can conceive of rearchers who always publish in the same rearch field. Even among such scientists, we would like to differentiate tho who integrate source materials from multiple fields from tho who draw on a narrower range of sources. At the other extreme, imagine rearchers who publish each of their papers in a different field. Again, we might deem some of them as more interdisciplinary than others.
For instance, one of the 17 pilot rearchers who kindly provided CV information to us publishes in veral rearch fields (Table 1 shows the Subject Categories reprented). Perusal of the fields suggests strong common threads of this polymer chemistry work among materials science, chemistry, physics, and more or less related fields. Tho other fields include: crystallography, which could reasonably reflect techniques being applied; biomedical engineering, which could be an applic
ation domain; education, which could dovetail with teaching of the principles being rearched. Such empirical findings suggest the desirability of a metric to get at how cloly the rearch fields reprented are associated.
Table 1 also includes information on the Subject Categories (“SCs”) cited (referenced) by this rearcher’s 116 papers (published from 1988–2004) and the 876 122Scientometrics 72 (2007)
