Impact Asssment Methodologies for Micro®nance: Theory,Experience and Better Practice
DAVID HULME*
University of Manchester,Manchester,UK
Summary.ÐMicro®nance programs and institutions are increasingly important in development
strategies but knowledge about their impacts is partial and contested.This paper reviews the
methodological options for the impact asssment(IA)of micro®nance.Following a discussion of
the varying objectives of IA it examines the choice of conceptual frameworks and prents three
paradigms of impact asssment:the scienti®c method,the humanities tradition and participatory
learning and action(PLA).Key issues and lessons in the practice of micro®nance IAs are then
explored and it is argued that the central issue in IA design is how to combine di erent
methodological approaches so that a``®t''is achieved between IA objectives,program context and
the constraints of IA costs,human resources and timing.The conclusion argues for a greater focus
on internal impact monitoring by micro®nance institutions.Ó2000Elvier Science Ltd.All
rights rerved.
Key wordsÐmethods,micro®nance,credit,impact asssment,monitoring and evaluation,
poverty reduction
1.INTRODUCTION
In recent years impact asssment has become an increasingly important aspect of development activity as agencies,and particu-larly aid donors,have sought to ensure that funds are well spent.As micro®nance programs and institutions have become an important component of strategies to reduce poverty or promote micro and small enterpri develop-ment then the spotlight has begun to focus on them.But knowledge about the achievements of such initiatives remains only partial and is contested.At one end of the spectrum are studies arguing that micro®nance has very bene®cial economic and social impacts (Holcombe,1995;Hossain,1988;Khandker, 1998;Otero&Rhyne,1994;Remenyi,1991; Schuler,Hashemi&Riley,1997).At the other are writers who
caution against such optimism and point to the negative impacts that micro-®nance can have(Adams&von Pischke,1992; Buckley,1997;Montgomery,1996;Rogaly, 1996;Wood&Sharrif,1997).In the``middle'' is work that identi®es bene®cial impacts but argues that micro®nance does not assist the poorest,as is so often claimed(Hulme& Mosley,1996;Moly&Hulme,1998). Given this state of a airs the asssment of micro®nance programs remains an important ®eld for rearchers,policy-makers and devel-opment practitioners.1This paper reviews the methodological options for asssing the impacts of such programs drawing on writings on micro®nance and the broader literature on evaluation and impact asssment.Sub-quently it explores ways in which impact asssment practice might be improved.It views impact asssment(IA)as being``...as much an art as ''(a phra lifted from Little,1997,p.2).Enhancing the contri-bution that impact asssment can make to developmental goals requires both better World Development Vol.28,No.1,pp.79±98,2000Ó2000Elvier Science Ltd.All rights rerved
Printed in Great Britain
0305-750X/00/$-e front matter
PII:S0305-750X(99)00119-9
/locate/worlddev
*An earlier version of this paper was prepared for the
Consultative Group to Assist the Poorest(CGAP)in
association with Management Systems International and饥荒锅怎么做
USAIDÕs AIMS Project.It was prented at a virtual
meeting of the CGAP Impact Asssment Working
Group,April7±19,1997.I should like to thank the
participants in that meeting for their comments,tho
who prepared the background papers(Renee Chao-
Bero ,Osvaldo Feinstein,Gary Gaile,Peter Little,
Linda Mayoux,Paul Mosley and Arne Wiig)and
内存16g够用吗
Carolyn Barnes,Monique Cohen,Jennifer Sebstad and
three anonymous reviewers for comments.Particular
thanks are due to Richard Montgomery for the ideas
that he has given me about impact asssment.Final
revision accepted:25May1999.
79
science and better art.The scienti®c improve-ments relate to improving standards of measurement,sampling and analytical tech-nique.Econometricians and statisticians are particularly concerned with this®eld. Improving the``art''of impact asssment has at least three strands.One concerns making more systematic and informed judgements about the overall design of IAs in relation to their costs,speci®c objectives and contexts. The cond is about what mixes of impact asssment methods are most appropriate for any given study.The third relates to increas-ing our understanding of the ways in which the results of IA studies in¯uence policy-makers and micro®nance institution(MFI) managers.
2.IMPACT ASSESSMENT:OBJECTIVES
Impact asssment studies have become increasingly popular with donor agencies and, in conquence,have become an increasingly signi®cant activity for recipient agencies.In part this re¯ects a cosmetic change,with the term IA simply being substituted for evalua-tion.But it has also been associated with a greater focus on the outcomes of interventions, rather than inputs and outputs.While the goals of IA studies commonly incorporate both ``proving''impacts and``improving''interven-tions,IAs are more likely to prioritize the proving goal than did the evaluations of the 1980s.A t of factors are associated with the extreme``pole''positions of this continuum and the underpin many of the issues that must be resolved(and personal and institutional tensions that ari)when impact asssments are being initiated(Figure1).
Behind the shift from``evaluation''to``IA'' are a number of factors.The are not explored in any detail in this paper but they form an esntial element for the understanding of IA and its potential contributions.Explicitly,IAs are promoted by both the sponsors and implementers of programs so that they can learn what is being achieved and improve the e ectiveness and e ciency of their activities. Implicitly,IAs are a method by which sponsors ek to get more information about program e ectiveness than is available from the routine accountability systems of implementing orga-nizations.I
As are also of signi®cance to aid agencies in terms of meeting the ever increasing accountability demands of their governments (in this era of``results''and``value for money'') and for contesting the rhetoric of the anti-aid lobby.While recipient agencies bene®t from this,they are one stage removed,and many are likely to e donor-initiated IA as an activity that has limited practical relevance for program activities.To quote the director of a large Asian micro®nance institution that has received substantial amounts of aid®nanced IA consultancy and internal IA-capacity building ``...impact asssment studies keep we donÕt u them very
much.''
Figure1.The goals of impact asssment.
WORLD DEVELOPMENT
80
A®nal issue to rai in this ction is whether the expectations of OECD-bad agencies about the feasibility of the accurate measure-ment of impacts in the di cult contexts of developing countries(limited numbers of professional rearchers,few written records, illiteracy,communication problems etc.)are higher than in their own countries.My professional experience of EU-®nanced``small enterpri development''projects in Manches-ter has revealed a startling lack of concern with impacts:this is in marked contrast to my consultancy work in Bangladesh where donors criticize nongovernment organizations(NGOs) for failing to make impact asssment a prior-ity!If recipients perceive that the IA standards expected of``them''are higher than donors expect of themlves then IA will be en as an external imposition rather than a shared opportunity.
3.ASSESSING IMPACT:THE CHOICE OF
CONCEPTUAL FRAMEWORKSPhm
All impact asssment exercis have a conceptual framework at their heart.In well-planned and well-resourced IAs with long ``lead-in''times such frameworks are usually explicitly identi®,Khandker,1998; Sebstad,Neill,Barnes&Chen,1995;Schuler& Hashemi,1994).By contrast,in many smaller scale exercis the framework is implicit and may be en as``common n.''There are three main elements to a conceptual frame-work:
Ða model of the impact chain that the study is to examine,
Ðthe speci®cation of the unit(s),or levels,at which impacts are assd,and
Ðthe speci®cation of the types of impact that are to be assd.
(a)Models of impact chains
Behind all micro®nance programs,and indeed virtually all aid®nanced initiatives,2is the assumption that intervention will change human behaviors and practices in ways that lead to the achievement(or rai the probability of achievement)of desired outcomes.IAs asss the di erence in the values of key variables between the outcomes on``agents''(individuals, enterpris,houholds,populations,pol
icy-makers,etc.)which have experienced an inter-vention against the values of tho variables that would have occurred had there been no intervention.The fact that no agent can both experience an intervention and at the same time not experience an intervention generates many methodological problems.All changes are in¯uenced by mediating process(speci®c characteristics of the agent and of the economic,physical,social and political envi-ronment)that in¯uence both behavioral chan-ges and the outcomes in ways that are di cult to predict(Sebstad et al.,1995).
The impact chain is very simply depicted in Figure2.A more detailed conceptualization would prent a complex t of links as each ``e ect''becomes a``cau''in its own right generating further e ects.For example,in
a Figure2.The conventional model of the impact chain.
IMPACT ASSESSMENT METHODOLOGIES FOR MICROFINANCE81
conventional micro®nance project a package of technical assistance and capital changes the behavior(and products)of a micro®nance institution(MFI).The MFI subquently provides di erent rvices to a client,most commonly in the form of a loan.The rvices lead to the client modifying her/his microenter-pri activities which in turn leads to incread or decread microenterpri income.The change in microenterpri income caus chan-ges in houhold income which in turn leads to greater or lesr houhold economic curity. The modi®ed level of houhold economic curity leads to changes in the morbidity and mortality of houhold members,in educa-tional and skill levels and in future economic and social opportunities.Ultimately,perhaps, the changes lead to modi®cations in social and political relations and structures.The complexity of such chains provides the asssor with a range of choices about which link(or links)to focus on.For micro®nance,it is uful to distinguish between two main schools of thought with regard to which link(s)in the chain to focus on.For convenience,the are termed the``intended bene®ciary''3school and the ``intermediary''school.
The intended bene®ciary school,building on the ideas of conventional evaluation,eks to get as far down the impact chain as is feasible(in terms of budgets and techniques)and to asss the impact on intended bene®ciaries(individuals or houholds).The intermediary school focus purely on the beginning of the chain and in particular on changes in the MFI and its oper-ations.Its roots are cloly associated with the Ohio State University SchoolÕs analys of rural ®nance.Generally,two key variables are focud on:institutional outreach and institu-tional sustainability4(Yaron,Benjamin& Piprek,1997).If both outreach and sustaina-bility have been enhanced then the intervention is judged to have a bene®cial impact as it has widened the®nancial market in a sustainable fashion.This is bad on the assumption that such institutional impacts extend the choices of people looking for credit and savings rvices and that this extension of choice ultimately leads to improved microenterpri performance and houhold economic curity.While this assumption can be supported by theoretical frameworks(if a t of further assumptions are made about perfect competition and other factors)it is an assumption which has proved invalid in a number of experiences.5In addi-tion,it will not reveal borrower``cross®nanc-ing''of loans(Wiig,1997)which may threaten the long-term viability of an MFI.
While the choice between the two schools can ultimately be en as an ideological choice (does o
ne prioritize contributions to improved welfare or to more e cient markets?)it is possible to recognize di erent strengths and weakness.The intended bene®ciary school makes fewer assumptions about the impact chain and is better able to distinguish``who'' bene®ts and``how.''It is,however,demanding in both methodological and cost terms.The intermediary school ufully incorporates notions of sustainability and provides an IA methodological framework that can be oper-ated largely with pre-existing data.It is, though,very weak on``who''bene®ts and ``how''(as illustrated by asssments of the USAID-®nanced APPLE program).6Possible ways of strengthening the intermediary school approach have been suggested by Feinstein (1997)through the analysis of borrower trans-action costs.He propos the collection of longitudinal data on borrowers transaction costs(p.5)to asss whether an MFI has bene®ted ,has reduced their total costs for accessing®nance.This o ers a potential``bridge''between the two main ``schools,''if data on``who''borrowers are also collected.
(b)Units of asssment
Following on from the design of a model of the impact path comes the choice of the unit(s) of asssment(or levels of asssment).Com-mon units of asssment are the houhold,the enterpri or the institutional environment within which agents operate.Occasionally studies have att
empted to asss impact at an individual ,Goetz&Sen Gupta,1996; Peace&Hulme,1994),but this is relatively rare and has to take a qualitative focus.More recently some studies have attempted to asss impacts at a number of levels,such as Hulme and Mosley(1996)who looked at microenter-pri,houhold,community and institutional levels and USAIDÕs Asssing the Impact of Microenterpri(AIMS)Project.Through a houhold economic portfolio model(HEPM) the latter eks to asss impacts at houhold, enterpri,individual and community levels and thus produce a fuller picture of overall impacts(Chen&Dunn,1996).
The relative advantages and disadvantages of di erent units of asssment are summarized in
WORLD DEVELOPMENT 82
Table 1.As can be en,a focus purely on the ``individual''or the ``enterpri''has such drawbacks that they could be viewed as discredited.The houhold economic portfolio model has much to recommend itÐespecially if institutional impacts are incorporated in the community level analysis.It does have the profound disadvantage,though,of making asssment demanding in terms of costs,skilled personnel and time.If ud with limited resources it risks sacri®cing depth for breadth of coverage of possible impacts.
(c)Types of impact
An almost in®nite array of variables can be identi®ed to asss impacts on di erent units.To be of u the must be able to be de®ned with precision and must be measurable.Conventionally,economic indicators have dominated micro®nance IAs with asssors particularly keen to measure changes in income despite the enormous problems this prents.Other popular variables have been levels and patterns of expenditure,consumption and asts.A strong ca can be made that asts are a particularly uful indicator of impact becau their level does not ¯uctuate as greatly as other economic indicators and is not simply bad on an annual estimate (Barnes,1996,p.v).
腥红假期
The social indicators that became popular in the early 1980s (e.g.,educational status,access to health rvices,nutritional levels,anthropo-metric measures and contraceptive u)have recently been extended into the socio-political arena in an attempt to asss whether micro®-nance can promote empowerment (Mayoux,1997;Goetz &Sen Gupta,1996;Schuler &
Table 1.Units of asssment and their advantages and disadvantages
Unit Advantages
Disadvantages
Individual
ÐEasily de®ned and identi®ed
ÐMost interventions have impacts beyond the individual
新闻选题ÐDi culties of disaggregating group impacts and impacts on ``relations''Enterpri
ÐAvailability of analytical tools
(pro®tability,return on investment etc)
ÐDe®nition and identi®cation is di cult in microenterpris
ÐMuch micro®nance is ud for other enterpris and/or consumption
童年趣事200字ÐLinks between enterpri performance and livelihoods need careful validation Houhold ÐRelatively easily de®ned and identi®ed ÐSometimes exact membership di cult to gauge
ÐPermits an appreciation of livelihood impacts
ÐThe assumption that what is good for a houhold in aggregate is good for all of its members individually is often invalid
ÐPermits an appreciation of interlink-ages of di erent enterpris and consumption
Community
ÐPermits major externalities of interventions to be captured ÐQuantitative data is di cult to gather ÐDe®nition of its boundary is arbitrary Institutional impacts
ÐAvailability of data
与有情人ÐHow valid are inferences about the outcomes produced by institutional activity?
ÐAvailability of analytical tools
(pro®tability,SDIs,transaction costs)
Houhold
economic portfo-lio (i.e.hou-hold,enterpri,individual and community)
男生卡通ÐComprehensive coverage of impacts ÐComplexity ÐAppreciation of linkages between di erent units
ÐHigh costs
ÐDemands sophisticated analytical skills ÐTime consuming
IMPACT ASSESSMENT METHODOLOGIES FOR MICROFINANCE 83
