Oxfam GBResults-Based Management: Friend or Foe? Author(s): Michael J. Hatton and Kent Schroeder Reviewed work(s): Source: Development in Practice, Vol. 17, No. 3 (Jun., 2007), pp. 426-432 Published by: Taylor & Francis, Ltd. on behalf of Oxfam GB Stable URL: http://www.jstor.org/stable/25548228 . Accessed: 31/07/2012 06:27 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp . JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. . Taylor & Francis, Ltd. and Oxfam GB are collaborating with JSTOR to digitize, preserve and extend access to Development in Practice. http://www.jstor.org and only half-jokingly. energy. Key Words: Aid. managing. Cummings 1997) or it reviews themedium-term experience with RBM from the 426 ISSN 0961-4524 Print/ISSN DOT 1364-9213 Online 030426-07 10. It is a fact of life formany of us working in this field. RBM has focused the work of numerous bilateral and multilateral agencies on defining. and resources away from actually doing work. The varying opinions with respect to RBM clearly imply the need for increased efforts to explore its effectiveness as a development tool.Yet discussions with colleagues regularly take on a negative tone when the topic of RBM arises. for example. RBM is referred to as part of the problem. Often. measuring outcomes. Number 3. Methods Results-based Over the past indevelopment management decade. June 2007 Results-based foe? Michael Ta?VLaJ?rouP |\ friend or management: J.1080/09614520701337160 ? 2007 Oxfam GB Routledge Publishing . and impacts have become a familiar refrain. development This article some provides broad reflections on RBM from a distinctive vantage point: theperspective of theproject (or programme) evaluator.in particular by many of those directly involved in project implementation. a that requirement consumes time. many development practitioners view RBM in a negative light. Donors and at least some international implementing agencies tend to be more supportive ofRBM as a management and evaluation tool. The article reflects on challenges associated with RBM and draws from these reflections a number of suggested strategies to improve its use. outputs. local govern ments and beneficiaries are less enthusiastic. it is a view not infrequently adopted . and results. It concludes that development practitioners need to be more aggressive in implementingRBM. Broadly speaking. While this is not a firmposition taken by all players or at all levels in development work. energy. Since the mid-1990s. In the words of one colleague: until the next management fad comes and obstructs the 77/ just bide my time along'.Development inPractice. considering it to be a donor requirement that diverts time. Inputs. A considerable amount has been written about the use of RBM by development agencies to date. results-based (RBM) management has become well entrenched as a man agement tool for development. Much of it is descriptive (see.Hatton and Kent Schroeder Results-based management (RBM) is well entrenched as a management toolfor international development practice. and resources actual doing of development work. Volume 17. Yet after a decade of its use. often New This Public referred reform.Results-based management: friend or foe? perspective of donor agencies (see. RBM further trickled down to the management approaches of non-government organisations (NGOs). the power relationship among the key players donors. it is an appropriate time to reflectmore deeply on RBM as a development tool. compared with the long-term results that it intends to generate. decentralised services where appropriate. Number 3. promoted a client-focused orientation to services. and emphasised accountability based on the effectiveness and relevance of results. was driven early Management more concerns demands for about efficient and responsive services. the key question remains: what does our actual experience tell us about RBM? Development inPractice. June 2007 427 . This strategy focuses on the achievement of results. It injected market strategies into public management. RBM affectsmost of us involved in development work in some manner. private companies. the scope and breadth of the reformswere so broad that the OECD claimed thatNPM represented a paradigm shift in public management (OECD 1995: 81). as a management strategy. Volume 17. and Danida in Denmark. The article reflects on RBM challenges and seeks to draw from these reflections some insights for improving the use of RBM tomanage for results. as some development agencies have been using RBM for ten or more years. Given thiswidespread use. a fundamental activities re-orientation long term (impacts). DFID in theUK. In particular. in themedium. the perceived need for increased accountability. by spiralling budget defi cits. and so on . New Public Management offered a response. some deride it. These reforms extended to the bilateral development programmes of OECD countries. and why is it so widespread? RBM has its roots in the wave of public-sector reform that swept many of the OECD countries in the as to 1990s. One aspect of these reformswas the incorporation of results-based management. approaches therefore This rep that were dominated by an emphasis on inputs and activities. Why RBM? Where did results-based management come from. and citizens' general discontent with their governments. for example. Results-based CIDA management became the management strategy of choice for agencies such as in Canada. AusAID inAustralia. assessing therisks thatmay impede expected results. USAID in theUSA. recipients. (NPM). the assumption being that results would follow if the inputs and activities were appropriately robust. given the relatively short time since RBM came into general use. all management guides resents and (outcomes).Complicating things further. This is not surprising. of defined management RBM results. However. Taken in total. implementers.is clearly asymmetrical. Office ofthe Auditor General of Canada 2000). reporting on performance in achieving results. Binnendijk 2000. sometimes called performance management. the ultimate achievement in the towards away from previous At its heart. and acting on performance infor is at the core of this process: human and financial resources (inputs) generate activities that produce results in the short term (outputs). In thismixed context. mation. and some just try to ignore it. also incorporated RBM as a management strategy. This article provides some broad reflections on RBM from a distinctive vantage point: the perspective of the project (or programme) evaluator. Some embrace it. including theWorld Bank and a variety of UN agencies.monitoring programmes designed to achieve these results through theuse of appropriate indicators. A 'results chain' term end-of-project. RBM emphasises the importance of defining expected resultswith the involvement of key stakeholders. and higher-education institutionsacting as the implementing agencies for bilateral and multilateral development organisations. Multilateral organisations. different recipients. There is much greater convergence with respect to the opinions about and commitment to RBM today than in thepast. practice. and this trendwill continue to grow. Breaking with the past approach of managing for inputs and activities instead of for results is a key challenge that is still poorly understood by some. and under standing remain significant. Not surpris ingly. different donors. how it is described. And all this is shifting. including developing relationships with them. how it is employed. different donor managers. Hatton RBM challenges: Schroeder reflections fromevaluation practice The work of monitoring and evaluation typically involves critically assessing the process and results of a project or programme and offering a written analysis. in the same way. It is easy to imagine the confusion and frustrationof an executing agency thatworks with one donor forwhich RBM is the focus. after a and action taken with respect to the employment of RBM vary tremendously. It also provides the evaluator with a potentially key role in achieving project results by contributing to learning. best RBM reporting flows from reports thathave included active participation by the develop ing-countrypartner. Volume 17. It is also not uncommon for reports in the first two to three years of a project to emphasise that it is too early for results. Inclusion of the developing-country partner inRBM planning and reporting cannot be used effectively unless the developing-country partner also understands the The paradigm. Many project reporters prefer to work on page after page of activities. how it should be employed. is well trained in it. But differences of opinion. value. and the second seems not to grasp its value. This gives the evaluator a unique perspective on the gaps and successes in programme management. depending on the players. and its value. including at the output level. Diverse perspectives on RBM It is not easy to find two people who will describe RBM decade or more of its use. and with another donor for is an afterthought. the emphasis. importance. decision making. Evaluation does not get past such elements as the qualifications and capacity of the fieldmanager or project manager.Number 3. thoughmuch more slowly than has been the case in the past. That partnership will not develop unless the developing-country partner is full RBM 428 Development inPractice.must come from a a partnership.Or consider working with two managers from the same which RBM donor. Lack of understanding of RBM as a results-focused approach In addition to diverse perspectives on RBM.Michael and Kent J. RBM at a mid-level of effective application. June 2007 . Even today. So what can the view from the evaluator's perch tell us about the challenges associated with the use of results-based management? Our experience suggests that the following challenges are common. and the logistics of flowing financial orHR capacity. Individuals. and finding that the firstdirects the executing agency to use RBM as the cornerstone inmanagement and reporting. It is not uncommon for a 60-page report to have only three pages that describe results past the activity stage.For many projects the operational challenges are such that outputs and outcomes are simply not on the radar screen. or higher. reporting on outputs and outcomes is an area of weakness. and accountability. and different recipient countries bring different perspectives to what RBM means. and has made an explicit commitment to the approach. or phys ically reaching the beneficiaries. it is striking how often within implementing agencies there is a failure to understand RBM as a results-focused strategy. But if the identifiedand tar geted results are not realistic. And when it does happen with the partner. outcomes. not simply on how ambitious they are. In some cases this is due to a lack of understanding of RBM within the executing agency. theywill speak of itas 'our plan'. Ifwe were to ask all development stakeholders. In the end. The need to define expected outputs. Executing agencies and their implementing partners on the ground also regularly ignore the identification of unexpected results. June 2007 429 . Identifying realistic results and unexpected results Results thatare realistically achievable may not always be at the frontofmanagers' minds. and this element alone is interpreted inmany differentways. it may also be that different results are valued differently by different players. at least in part. If 'our' plan includes RBM. 'Our' plan comes with a defence of how the resources are allocated.much else falls into place. RBM is perceived by implementing on partners.do not any of thekey stakeholders beneficiaries. Results identified at this stage are not necessarily ramped back to the real world as theproject unfolds. but it is also one of thehardest things to operationalise effectively. Stakeholder participationmust be meaningful. In other cases.This is perhaps best explained by work plan. Otherwise. You cannot effectivelymanage for results if . the implementing agency is left to negotiate a solution and report accordingly. a management strategy involving only half of a partnership will not generate relevant and sustainable results. agencies as a management practice from the developed world. Further. 'Our' plan is typically described in detail with glowing pride. Effective RBM practice does not happen without the partner. with less knowledge about the detail and little commitment to the core. Confusion at this level quickly translates into confusion and lack of inter est in the developing-country partner. Grappling with the challenges of defining. confidentially. then theproject cannot be managed for successful results. Yet commitment to full inclusion of the developing-country partner inRBM planning and reporting is often weak.That is an essential step. development partners. If assessment annual of the for their to ask developing-country partners interesting itsdevelopment was a shared task. positive replies would probably not reach 75 per cent. Number 3. and measuring meaningful stakeholder participation is vital toRBM. This is especially the case when you ask people what can be achieved when they are in the process of bidding on a project or tryingto sell an unsolicited proposal. the partnerwill probably be committed to it. it is importantfor those results tobe eval uated. Meaningful stakeholder participation Stakeholder participation is a core element of RBM.which may be very significant.Unexpected results. This is not easy. implementing. Even the word 'meaningful' is open to challenge and confusion at the best of times. These are impossible to measure. if theirparticipation in the allocation of project expen ditures had been 'satisfactory' or better. by how achievable they are. often so subjective thatmaking realistic assessments can be very challenging. 'The' plan is described with little or no information on the costs of various elements and the options that have been considered. 'The' plan ismost often described in a cursory fashion. Some project documentation speaks about equal partnerships or equal participation. Volume 17. not to be imposed RBM monitoring and reporting therefore remain the sole domain of the implementing agency. Development inPractice. and impacts at theplanning stage can lock the focus of those implementing a project into achieving those specific results.When a donor requires targeted results to be part of a proposal. it is 'theplan'. It is participant in both planning and reporting.When the local NGO's needs or expectations differ from the donor's. and donor agencies consider the expected results to be relevant.Results-based management: friend or foe? an example. end up being ignored or downplayed if they do not neatly fit into the original results framework. the partner and the donor agency entrepreneurial enough tomake changes in project design and capacity as needs and the environment change over a multi-year period? If not. For example. or both. especially at the outcome level. a mark. and maybe not even so problematic as to affect behaviour significantly after the fact. irrelevant. Similarly. Hatton and Kent Schroeder and effective indicators Appropriate Appropriate and effective indicators are critical formeasuring success and feeding project learning.Yet the connection between data collection on theone hand. incentives and consequences Does an executing agency that reportspoorly. Opportunities for learning One of thekey roles ofRBM reporting is toprovide informationthatcan be acted upon. Managing risks All project partners need to be able to take informed and timely action tomanage risks. While this is important. are projects rewarded appropriately for results that have been defined. Some projects and programmes are of a project is also evaluated annually. An external evaluation the 60 per cent point lenge in maintaining consistency tomake hard-pressed in evaluation. a five-year there project is often a chal or programme could have three differentdonor-agency project officers and two differentmonitors over that period.Managing and reporting performance requires ongoing learning. Projects must be nimble and flexible enough to adapt to changing conditions over the duration of their life. Maintaining Performance evaluation consistency in such a context is a considerable challenge. easily gathered quantitative indicators that do not measure results as deeply as they could. but many are others not. the potential of a results-focused strategy is greatly diminished. Volume 17. the mainstreaming of gender equality has made the inclusion of gender issues and accompanying indicatorsmandatory inmany projects. with littleapparent commitment to or knowledge of RBM. RBM risks being thwarted.does it really measure change in gender relations or power imbalances in anymeaningful sense? Simplified indicators make itmuch more difficult tomeasure meaningful relationships between inputs and results. Further. Limited focus on evaluation In many cases there is a limited focus on external evaluation within project and programme activity. Yet constraints on time and resources often lead to the selection of simplified. The demands of day-to-day operations frequently rob organisations of time to reflect on 430 Development inPractice. An end-of at project evaluation has no influence on project or programme delivery. despite the critical role of evaluation inmeasuring results and generating learning. and reported through the participation of all stake holders? Without real incentives for achieving results or consequences for poor reporting and management. Are the executing agency. is often not made. and the incorporationof learning arising from these data on theother. It is common forprojects to select indicators thatmeasure nothingmore than the percentage of project staffand participants who are women. For example.Michael J. achieved. leading to results that are inap propriate. June 2007 . This should not be confused with manipulating project ends to meet the capacity or interestsof the executing agency or beneficiary. suffersignificant consequences? Often the results of not being committed toRBM are not disastrous enough tomodify behaviour in advance. Number 3. or partners. Development inPractice. evaluate. and to incorporate this learning into project management. the course requires with RBM. and how thiswill be measured. to draw lessons from these reflections. efficient. Many projects simply cannot find the time to do this. The best projects incorporate learning throughouttheir lifespan. Management Paris: DAC in the Development Working Party on Aid www. To write learning activities directly into project work plans that explicitly put aside time for project personnel to analyse. But a decade of use does notmean thatwe can afford to become complacent in our implementation of RBM. Results-based management is an important tool for development work. organisations. more There are aggressive. We need to redouble our efforts tomeet the challenges of RBM head-on and adopt its blueprint for success. Number 3. and write effectivelywith an RBM focus. To compel executing agencies to include developing-country partners in all facets of project development and implementation.Results-based management: friend or foe? informationgathered throughmonitoring and evaluation. Volume 17. It has not been as easy to understand and implement as was first expected. References Annette Binnendijk. We Being more not only to stay need aggressive. leading to effective. To engage all partners inmutually defining the depth and breadth of participation.oecd. June 2007 431 Cooperation Evaluation. To place greater on emphasis evaluating executing based agencies on results. and specifically undertake more post impact studies to inform longer-termproject development. some key areas of focus thatwould greatly improve RBM within our development projects. (2000) 'Results-based Background Report'. It is in fact a very powerful tool. and incorporate the lessons learned from RBM reporting. Agencies: available A at . as some might suggest? Is it a faddishmanagement strategy simply to be tolerated while it lasts? Despite the challenges. We to strive for the following: need To oblige executing agencies of all types to report effectively through an RBM framework that includes identifying realistic results and appropriate indicators at the risk of losing the project if theyfail to comply. The agencies. But isRBM the enemy thatdetracts from development work. To increase the evaluation level of most projects. it is very tough to plan. and corporations thatembrace RBM typically produce strong results. both positive and negative. reflect on. analyse. addressing we the to become need challenges. In fact. Conclusion: furtheringsuccess The challenges outlined above illustrate thatRBM can indeed be complex.and rel evant results forbeneficiaries. Review of Experience. To ensure thatall reporting includes both intended and unintended results. used effectively. at the risk of losing the project. including revised results that arise from the successful management of risks during the life of the project. however. our experience would suggest thatRBM. not just on activities. To provide greater incentives and rewards to projects for achieving results.pdf (retrieved1February 2006). Over the past decade ithas proved to be both successful and challenging. including reporting for results. The result is lost opportunities. provides a framework that guides the delivery of real results that improve the lives of thosemost directly affected by development initiatives.org/dataoecd/17/l/1886527. Canadian OECD Office Governance (1995) of the Auditor from the Literature'. Hatton Michael and Kent (1997) Harry Cummings. College Schroeder author) (corresponding of the Humber Institute of Technology and Kent in the Business School Project Director for with NGOs and academic institutions in designing. Contact programme is the International Advanced 205 Humber Ontario. he has provided technical assistance in Africa opment programmes details: level in the areas of teacher education. of Canada Results-based (2000) 'Implementing Management: at www. Institute Canada of Technology and Advanced Learning. Reforms General available 29 January2006). logical frameworks management: Journal of Development Studies 28(Special Issue): 587-96. 5L7. strategic planning. June 2007 . 5L7. M9W managing.schroeder@humber. in Transition: Public Management inOECD Paris: OECD. In addition. Volume 17. Countries.html Lessons (retrieved The authors at the Humber is Vice President Academic Institute of Technology and Advanced as a project director. and evaluator for international devel 15 years' experience at the project and Asia.ca/domino/other. eight and of managing international development and evaluating projects.ca> 205 Humber College Boulevard. Schroeder contrasts and results-based 'Logic models. Hatton He has Institute Canada and Advanced of Technology Learning.gc.J. Michael Learning. Number 3.hatton@humber. J. Boulevard. <michael. evaluating School.nsf/html/OOrbm_e. He also has experience a capacity-building The Business details: in the Canadian Arctic. M9W 432 Development inPractice.oag-bvg. Toronto. management and Contact training. Humber Ontario. Humber Toronto.ca>. monitor. has worked He years Learning. and comparisons'. <kent.
Report "1- Results-Based Management - Friend or Foe.pdf"