Making Evaluation Useful and Useable.
Using Evidence is a research and evaluation consultancy that focuses on development and research evaluation. Using Evidence is concerned not only with high quality research, but in exploring ways to enhance the use of results.
What we do
Evaluation Methods and Research
I have had a strong focus on evaluation methods for many years because I think we need new approaches that take account of complexity and that also look at systems not only at projects and programs. I worked with colleagues on the development of Outcome Mapping as well as an approach to Organizational Assessment. Both of these methods integrate context directly into the assessment approach and treat it as a key variable. I have also worked with a team in an action research project on the development of an approach to Evaluating Capacity Building. In collaboration with ORS Impact we developed and validated, Informing Advocacy and Communications Capacity Building Efforts. In addition, I am working with the Bill & Melinda Gates Foundation’s Maternal, Neonatal and Child Health team on an approach to evaluating implementation research. In 2017 I published a working paper with the Knowledge Sector (Indonesia) on How to Evaluate a Mental Revolution. In 2018, I published a working paper, Policy and Causality, on the Knowledge Sector Initiative (Indonesia) website.
Major evaluations can benefit from an external perspective to review the design and draft report. I have carried out quality assessments for Universalia Management Group on evaluations they have conducted for the World Food Program, and for IUCN on Value for Money assessments. Since 2014 I have advising DFID on the evaluation of the International Growth Centre. As many of my evaluation colleagues, I also carry out pro bono peer reviews for a number of evaluation journals.
Enhancing the Use of Evidence
Evidence from research and evaluation needs to be used if it is going to contribute to social improvement. Too often we focus on the methods and the study itself, and forget about use until the very end. Use needs to be thought about from the beginning. This focus was reinforced by a study I did for the International Development Research Centre in which we looked at how the research the Centre supports influences public policy – Knowledge to Policy. The findings of that study on the mechanisms of influence has been an important influence on my work over the past decade. I was involved for some years with the Knowledge Sector Initiative in Indonesia where we worked across the policy system to strengthen both supply and demand for policy evidence and its use, as well as working on the legislative and regulatory barriers that inhibit the effective conduct and use of evidence in policy processes. Currently I am working with the Agricultural Science and Technology Indicators program of the International Food Policy Research Institute on a project to enhance the use of evidence gathered through this program, particularly at the national level. In 2018, with colleagues in Indonesia, I published Local Knowledge Matters in English and Indonesian, examining the role of local knowledge in public policy processes. As well, I co-edited Knowledge, Politics and Policymaking in Indonesia, which examines key questions in evidence-informed policymaking in Indonesia.
Evaluation and Knowledge Systems and Frameworks
Many organizations struggle to develop effective and use-oriented evaluation systems that help the organization learn and improve. While at the International Development Research Centre (IDRC) I was involved in the design of evaluation systems and processes that helped the organization and its programs learn from experience. I designed and implemented the first of several rounds of program review at the Centre, worked with a cross Centre team to revise our project completion reporting process, and worked with programs on building evaluation of their strategies. Since leaving IDRC, I have worked with the MasterCard Foundation and the Rideau Hall Foundation in providing support on their evaluation and learning systems. I am working with an IDRC global project on thinking about its evaluation approach in grant making. I am also supporting the Bill & Melinda Gates Foundation work with the Maternal, Neonatal and Child Health team on strengthening its evaluation systems. At the systems level, I am exploring how evaluation can look at more than an organization or a program and take account of whole system change. I have written on this in a working paper published by the Knowledge Sector Initiative as well as joint discussion paper with colleagues on the role of knowledge systems, State Capability, Policymaking and the Fourth Industrial Revovlution: Do knowledge systems matter? With the Global Development Network, I am involved as an advisor on their Doing Research program to assess the state of social sciences research in Low and Middle Income countries. With a small team based in Africa, I am working with the MasterCard Foundation on strengthening African Leadership in research.
I am actively involved in evaluations as well. While at IDRC I was involved in a wide range of evaluations. More recently I have been involved in organizational assessment of UNWomen (regional architecture) in collaboration with EnCompass, and at the Rideau Hall Foundation, with Cindy Weeks Consulting. With Global Affairs Canada, I integrated a Principles-Focused approach to a country program evaluation to evaluate coherence in development, trade and diplomacy efforts. I worked with the Ontario Brain Institute on design of evaluation for some of its programming and also with Sambodhi Research and Communications on process and health systems evaluation of programming in maternal and child health in Uttar Pradesh, India.
Evaluation should leave behind improved capacity to do evaluation. For many years, I conducted training around the world in organizational assessment and Outcome Mapping. More recently I have been working with the UNESCO Regional Office for Sciences in Latin America and the Caribbean, on a Massive Online Open Course on “Inequities in Latin America and the Caribbean. Research and policy for social transformations”, delivering three modules on evaluation. The course was piloted in 2017 and is running in both Spanish and English in 2018 and 2019. I published a chapter in the June 2017 issue of New Directions for Evaluation, ‘Building Evaluation Capacity to Address Problems of Equity.’ I am working with colleagues on thinking through how to address issues of culture in evaluation and how to promote a more active integration of culture into evaluation thinking and methods. In November 2017, I delivered an Outcome Mapping Workshop for the Higher Education department of the Ministry of Research, Technology and Higher Education in Indonesia, as well as keynoting a conference on impact evaluation the Ministry held in Bandung. In February 2018, I delivered an introductory Outcome Mapping workshop to the Atlantic team of the First Nations and Inuit Health Branch (FNIHB) of Indigenous Services Canada.
Fred Carden is the author or co-author of 8 books published in a dozen languages, as well as numerous chapters, articles and presentations.
State Capability, Policymaking and the Fourth Industrial Revolution: Do Knowledge Systems Matter?
The challenges of the Fourth Industrial Revolution are as huge and varied as opportunities: are we ready for them? How can knowledge systems contribute? There is no simple answer. This is an incredible window of opportunity to sense how knowledge can play a significant role in making public policy decisions to face the “unknown” embracing complexity, and maybe even to re-define what knowledge is and how it can be co-produced. This paper, “State Capability, Policymaking and the Fourth Industrial Revolution: do Knowledge Systems matter?” is co-produced by Politics & Ideas, Capability, Demos Helsinki, Helvetas, Southern Voice, UNESCO Montevideo and Using Evidence. Here, we launch a collective exploration on how knowledge systems could become even more relevant in the future to contribute to new ways of making good policy decisions that capitalize the opportunities of this revolution and diminish its potential risks, especially in middle income countries. Published by Southern Voice.
Informing Advocacy and Communications Capacity Building Efforts
This assessment framework aimed at strengthening advocacy and communications capacity was developed and validated through a pilot program with 19 grantees of the Bill & Melinda Gates Foundation, spread across 11 countries of Africa, Asia and South Asia. The brief describes the methodology, presents the tools, and includes insights from initial implementation that might be useful to others. Published by ORS Impact in Seattle. (download PDF, 2.2 MB)
Informing policy with agricultural R&D evidence: An ASTI pilot project in Ethiopia, Nigeria, and Tanzania
For nearly two decades ASTI has filled the unique role of providing relevant, high-quality data and analyses on investments, human resource capacity, and institutional developments in national agricultural research systems (NARS) in low- and middle-income countries. Despite its global profile as high quality evidence the incorporation of ASTI evidence into national policymaking is mostly ad hoc and often indirect. Moreover, interventions to influence the uptake of ASTI evidence for this purpose have been limited. Given the important role of agricultural R&D in increasing agricultural productivity, economic growth, and poverty reduction, ASTI initiated a pilot study in Ethiopia, Nigeria, and Tanzania with the intention of developing a clearer understanding of how ASTI evidence could be used more effectively and consistently used by national governments and other stakeholders to promote the allocation of sustainable resources to agricultural research.
Knowledge to Policy
Making the Most of Development Research The inability of development agencies to understand and improve the performance of the organizations they support continues to impede progress in the developing world, even after a decade of reforms. Strengthening the institutions that receive those grants and loans — including government ministries and executing agencies as well as nongovernmental organizations — has become the key to improving the efficiency and effectiveness of development assistance. Read now. (2.7MB PDF)
Whose knowledge matters?
Evaluation for Development, 3 August 2018
Learning about learning in an adaptive program.
With Arnaldo Pellini. Better Evaluation, 21 March 2017.
Indonesia’s knowledge sector is catching up, but a large gap persists.
With Arnaldo Pellini and Helen Tilley. The Conversation. 16 November 2016.
Science and Research in the Public Interest – the Role of National Science Academies.
Research to Action. 10 November 2016.
Remarks to the Indonesian Young Academy of Scientists – ALMI
AIPI Silver Jubilee 29 May 2015
Whose Development Results Count?
Norrag News. August 2, 2012
2017 & 2018. UNESCO Montevideo. MOOC (Massive Open On-line Course).
Desigualdades en América Latina y el Caribe: Investigación, Políticas, y Gestión para las Transformaciones Sociales.
Module 1: Experiences from the Field: https://www.futurelearn.com/courses/desigualdades-america-latina/1/steps/266622
Module 3: Evaluating for Equity.
Module 4: From Knowledge to Policy
2016. Rethinking Research Policies and Practices. (at 43:30). Centre for Innovation, Policy and Governance, Jakarta.
2012. Emerging Practices in Evaluating Policy Influence. My M&E eLearning Series on Emerging Practices in Development Evaluation.
2010. Development is Not an Intervention. Presentation to the Virtual Conference on Evaluation, Witwatersrand University.
2009. Evidence-Based Policy Making. IFPRI, Washington. DC.
Fred is an evaluation specialist with over 30 years’ experience in evaluating development research and programming, and the development of evaluation methodologies. In addition to his focus on methods, Fred has interest and expertise in the use of evidence in public policy. In 2015 Fred established Using Evidence Inc., a global consultancy that focuses on efforts to improve the use of research and evaluative evidence in decision making.
Prior to establishing Using Evidence, Fred spent two years in Jakarta as Lead Technical Advisor to the Knowledge Sector Initiative (KSI), a 15-year Australian funded project to support the knowledge to policy cycle in Indonesia.
Before joining KSI, Fred was Director of Evaluation at the International Development Research Centre in Canada, where he worked from 1993-2013. He is the author and co-author of numerous publications including Local Knowledge Matters, Knowledge to Policy: Making the most of development research, Outcome Mapping, and Enhancing Organizational Performance. He has published articles on other evaluation challenges, such as evaluating equity, and strategy evaluation. He continues to write, with primary interests in the use of evidence in public policy, implementation evaluation, and the evaluation of change at the macro level. See Publications for a complete list.
Fred’s current work includes research on the use of agricultural research evidence at the national level in Africa, evaluating implementation research, exploring the implications of the Fourth Industrial Revolution in middle income countries, and integrating learning and evaluative thinking more effectively into organizations.
Fred serves on the Board of the Partnership for Economic Policy and is an Advisor to he Doing Research project of the Global Development Network. He sits on the editorial boards of the American Journal of Evaluation, the Journal of Evidence and Policy, and New Directions for Evaluation.
Fred holds a PhD from the University of Montreal (1990) and a Fellowship in Sustainability Science at Harvard (2008). He lives in Ottawa, Canada.