 Sixty people from 33 organizations worldwide, almost half of them women and from outside the 16 centres supported by the Consultative Group on International Agricultural Research (CGIAR), convened this March in Cali, Colombia, to rethink the way agricultural researchers go about assessing their impacts on reducing poverty and economic, social and gender inequities.
Traditional assessment methods unsuitable
This group thinks that traditional economic impact assessment methods such as ‘rate of return studies’ are unsuited for evaluating research activities aimed at sustainable poverty reduction. Indeed, for the last 5-10 years, many CGIAR centres have been widening their range of methods to assess their diverse outcomes and impacts. The impact experts and local partners gathered at this meeting recommended that CGIAR management take immediate steps to acknowledge the legitimacy of this diversity and the broad range of impact assessment methods needed to evaluate it.
New ‘linking’ role for researchers
The group also stressed the central, rather than peripheral, role that today’s researchers must play in linking researchers, academics, farmers, marketers, policymakers and representatives of civil society in creating and sharing knowledge. To do this, the group warned, will require that research organizations change the way they do business. Specifically, these organizations will have to recognize the legitimacy and challenges of such boundary-spanning work by dedicating substantial time and resources to it and rewarding those who do it well.
Learning through participatory research
Learning organizations that are effective at innovation are also likely to be effective in engaging end-users. Thus, participants at this meeting highlighted the need to find more thoughtful ways of assessing who to involve in a given research project and how to involve them. Participatory tools demonstrably effective at this kind of ‘action research’ were discussed and evaluated. What was clear to all was a continuing need to better engage farmers and other end-users of research for development, or the civil society organizations that represent them, in meaningful ways at appropriate points throughout the research process. They also recommend that scientists managing complex research projects spanning the policy, civil society, agricultural, local development and private sectors be recognized and rewarded for taking on such complexity.
Excerpts from Workshop Brief No 2 states:
| It is time for the CGIAR to present a clear strategy and code of conduct for engaging users (including farmers, the poor and the civil society organizations that represent them) in on-the-ground research processes. . . . The CGIAR guidelines for impact assessment currently being finalized (based on a rate of return methodology alone) are not adequate for much of the research it conducts.
We urge management to support the rapid development of another set of impact assessment guidelines specifically for evaluating complex collaborative research, and to adapt the performance measurement and other systems to reflect these new approaches. Without them, we risk inappropriately assessing the work we are doing that is most likely to lead to sustainable solutions to poverty, and possibly even driving it out of the CGIAR research portfolio.
More realism needs to be applied to the concept of attribution and causation within complex collaborative research, where impacts are not likely to be attributable to the CGIAR or single causes. Knowing that different collaborators play different roles over time and multiple causal strands contribute to impact, we should focus assessments on contribution rather than attribution. Over-emphasis on attribution may damage the trust needed for effective collaboration. In addition, greater emphasis needs to be placed on understanding adaptation processes rather than adoption per se of finished technology. |
Principles for linking knowledge with action
ILRI agricultural economist, impact assessment expert and innovations leader Patti Kristjanson is committed to developing a set of principles for linking knowledge with action and to further linking the principles identified with tools, methods, approaches and strategies. Kristjanson says ILRI is collaborating with Harvard University’s Sustainability Science Program in development of a training course on this for CGIAR research managers and their partners.
The workshop participants agreed on the following four key messages.
Mission-oriented scientists need to rethink how they do research to have sustainable impacts on reducing poverty as well as how to evaluate those research impacts.
How scientists do research is key to achieving pro-poor, gender-sensitive and socially
inclusive results. Working more thoughtfully with, and helping to bridge boundaries
between, strategically chosen partners can help increase the probability of linking the
knowledge generated by the research to actions that lead to sustainable poverty
reductions.
Mission-driven researchers need to continue to bring other (existing) evaluation methods and approaches into more regular practice.
A wide array of evaluation methods and approaches already exists that is not fully
used by the agricultural and natural-resource management R&D communities. Members of those communities should review the available options and try out methodologies they are not familiar with.
Many scientists still view non-economic assessment methods as ‘illigitimate’.
There is still a high degree of skepticism among agricultural and natural resource management researchers, particularly within the CGIAR, about using non-economic and non-statistical data in evaluations. More empirical evidence of the validity and value of approaches other than economic (e.g. ex-post assessments) is needed.
Methodology gaps still exist.
It can be argued that the CGIAR are employing inadequate evaluation methods for the 75% of its research unrelated to germplasm improvement (e.g. policies, institutions, natural resource management, gender and social inclusion). CGIAR research evaluators need to refine existing, as well as often employ multiple, methods to fill this important gap.
For more information, see www.prgaprogram.org/riw
“Workshop on Rethinking Impact: Understanding the complexity of poverty and change: Summary”, 26–28 March 2008, Cali, Colombia, ILRI Innovation Works Discussion Paper 4, ILAC (Institutional Learning and Change) Working Paper 7 and PRGA (CGIAR Systemwide Program on Participatory Research and Gender Analysis) Working Document 26, September 2008.
The Challenge Paper, Initial Synthesis of Feedback, and Workshop Workbook
Further Information contact:
Patti Kristjanson
Innovation Works Leader
International Livestock Research Institute (ILRI)
Nairobi
Kenya
p.kristjanson@cgiar.org |
