613 252 8642 fred@usingevidence.com

Using Evidence

Making Evaluation Useful and Useable.

Using Evidence is a research and evaluation consultancy that focuses on development and research evaluation. Using Evidence is concerned not only with high quality research, but in exploring ways to enhance the use of results.

What we do

Evaluation Methods and Research

I have had a strong focus on evaluation methods for many years because I think we need new approaches that take account of complexity and that also look at systems not only at projects and programs. I worked with colleagues on the development of Outcome Mapping as well as an approach to Organizational Assessment. Both of these methods integrate context directly into the assessment approach and treat it as a key variable. I have also worked with a team in an action research project on the development of an approach to Evaluating Capacity Building. My current work in this area is a collaboration with ORS Impact to develop an approach to evaluating advocacy capacity. We are piloting a 360 approach in a number of Bill and Melinda Gates Foundation projects. In addition, I am working with the Foundation’s Maternal, Neonatal and Child Health team on an approach to evaluating implementation research.

Quality Assurance

Major evaluations can benefit from an external perspective to review the design and draft report. I have carried out quality assessments for Universalia Management Group on evaluations they have conducted for the World Food Program, and for IUCN on Value for Money assessments. As many of my evaluation colleagues, I also carry out pro bono peer reviews for a number of evaluation journals.

Enhancing the Use of Evidence

Evidence from research and evaluation needs to be used if it is going to contribute to social improvement. Too often we focus on the methods and the study itself, and forget about use until the very end. Use needs to be thought about from the beginning. This focus was reinforced by a study I did for the International Development Research Centre in which we looked at how the research the Centre supports influences public policy – Knowledge to Policy. The findings of that study on the mechanisms of influence has been an important influence on my work over the past decade. I was involved for some years with the Knowledge Sector Initiative in Indonesia where we worked across the policy system to strengthen both supply and demand for policy evidence and its use, as well as working on the legislative and regulatory barriers that inhibit the effective conduct and use of evidence in policy processes. Currently I am working with the Agricultural Science and Technology Indicators program of the International Food Policy Research Institute on a project to enhance the use of evidence gathered through this program, particularly at the national level.

Evaluation Systems and Frameworks

Many organizations struggle to develop effective and use-oriented evaluation systems that help the organization learn and improve. While at the International Development Research Centre (IDRC) I was involved in the design of evaluation systems and processes that helped the organization and its programs learn from experience. I designed and implemented the first several rounds of program review at the Centre, worked with a cross Centre team to revise our project completion reporting process, and worked with programs on building evaluation of their strategies. Since leaving IDRC, I have worked with the MasterCard Foundation providing support on their evaluation system. I am currently working with a team to support the development of an evaluation and learning system at the Rideau Hall Foundation in Canada. I am working with an IDRC global project on thinking about its evaluation approach in grant making. I am also supporting the Bill & Melinda Gates Foundation work with the Maternal, Neonatal and Child Health team on strengthening its evaluation systems. At the systems level, I am exploring how evaluation can look at more than an organization or a program and take account of whole system change. I have written on this in a working paper published by the Knowledge Sector Initiative and intend to develop further work in this area.

Evaluation

I am actively involved in evaluations as well. While at IDRC I was involved in a wide range of evaluations. More recently I have been involved in organizational assessment of UNWomen (regional architecture) in collaboration with EnCompass, and at the Rideau Hall Foundation, with Cindy Weeks Consulting. With Global Affairs Canada, I am integrating a Principles-Focused approach to a country program evaluation.

Evaluation Capacity

Evaluation should leave behind improved capacity to do evaluation. For many years, I conducted training around the world in organizational assessment and Outcome Mapping. More recently I have been working with the UNESCO Regional Office for Sciences in Latin America and the Caribbean, on a Massive Online Open Course on “Inequities in Latin America and the Caribbean. Research and policy for social transformations”, delivering three modules on evaluation. The course was piloted in 2017 and is running in both Spanish and English in 2018. I published a chapter in the June 2017 issue of New Directions for Evaluation, ‘Building Evaluation Capacity to Address Problems of Equity . I am working with colleagues on thinking through how to address issues of culture in evaluation and how to promote a more active integration of culture into evaluation thinking and methods. In November 2017, I delivered an Outcome Mapping Workshop for the Higher Education department of the Ministry of Research, Technology and Higher Education in Indonesia, as well as keynoting a conference on impact evaluation the Ministry held in Bandung. In February 2018, I delivered an introductory Outcome Mapping workshop to the Atlantic team of the First Nations and Inuit Health Branch (FNIHB) of Indigenous Services Canada.

Featured Publications

Fred Carden is the author or co-author of 8 books published in a dozen languages, as well as numerous chapters, articles and presentations.

Knowledge, politics and policymaking in Indonesia

This edited volume examines key questions about evidence-informed policy making in Indonesia. It draws on insights and evidence acquired through the implementation of the Knowledge Sector Initiative, a joint programme between the governments of Indonesia and Australia which aims to strengthen demand for and use of evidence in policy making in Indonesia. With contributions from academics, policy researchers, policymakers and development practitioners, the volume deepens readers’ understanding of how knowledge and politics shape the policy making process in Indonesia. This volume will be of interest to Indonesian and international researchers, academics, students, practitioners and policy makers concerned with different aspects of evidence-informed policy making research and processes. In particular, regional and international development practitioners and development partners interested in learning from Indonesia’s efforts to improve how evidence is used to address key development challenges, will find this volume valuable.
Publication Link

 

Local Knowledge Matters

This book explores the critical role that local knowledge plays in public policy processes as well as its role in the co-production of policy relevant knowledge with the scientific and professional communities.

The authors consider the mechanisms used by local organisations and the constraints and opportunities they face, exploring what the knowledge-to-policy process means, who is involved and how different communities can engage in the policy process.

Ten diverse case studies are used from around Indonesia, addressing issues such as forest management, water resources, maritime resource management and financial services. By making extensive use of quotes from the field the book allows the reader to ‘hear’ the perspectives and beliefs of community members around local knowledge and its effects on individual and community life. Download ebook at Publication Link or with Open Access

 

 

Pentingnya Pengetahuan Lokal

Blue Cover includes image of long boat drifting down secluded river with large family in boat

Buku “Pentingnya Pengetahuan Lokal” membahas tentang peran penting pengetahuan lokal dalam proses penyusunan kebijakan publik dan sekaligus peran pengetahuan lokal dalam proses kreasi bersama dengan jenis pengetahuan ilmiah dan profesional untuk menghasilkan pengetahuan yang relevan untuk pengambilan kebijakan. Buku ini menguraikan mekanisme, tantangan dan kesempatan yang dialami oleh organisasi lokal dalam mengeksplorasi apa yang dimaksud dengan proses dari pengetahuan menuju ke kebijakan, siapa saja yang terlibat dan bagaimana unsur masyarakat yang berbeda terlibat didalam prosesnya. 

Terdapat sepuluh studi kasus yang dipaparkan dari seluruh Indonesia dengan beragam isu seperti manajemen kehutanan, sumber air, manajemen sumber daya maritim, dan asuransi kesehatan. Dengan penyajian bukti yang mumpuni dari lapangan, buku ini memampukan kita dalam menganalisa hubungan dari perspektif dan kepercayaan anggota masyarakat terhadap pengetahuan lokal dengan kehidupan individu maupun dalam kehidupan bermasyarakat.

Buku ini adalah terjemahan dari “Local Knowledge Matters: Power, Context and Policy Making in Indonesia”, Bristol: Policy Press. 2018.

Read Now (4 MB PDF)

Knowledge to Policy

Making the Most of Development Research The inability of development agencies to understand and improve the performance of the organizations they support continues to impede progress in the developing world, even after a decade of reforms. Strengthening the institutions that receive those grants and loans — including government ministries and executing agencies as well as nongovernmental organizations — has become the key to improving the efficiency and effectiveness of development assistance. Read now. (2.7MB PDF)

Blogs

Whose knowledge matters?
Evaluation for Development, 3 August 2018

Learning about learning in an adaptive program
With Arnaldo Pellini. Better Evaluation, 21 March 2017.

Indonesia’s knowledge sector is catching up, but a large gap persists.
With Arnaldo Pellini and Helen Tilley. The Conversation. 16 November 2016.

Science and Research in the Public Interest – the Role of National Science Academies.
Research to Action. 10 November 2016.

Remarks to the Indonesian Young Academy of Scientists – ALMI
AIPI Silver Jubilee 29 May 2015

Whose Development Results Count?
Norrag News. August 2, 2012

Videos

2017 & 2018. UNESCO Montevideo. MOOC (Massive Open On-line Course).
Desigualdades en América Latina y el Caribe: Investigación, Políticas, y Gestión para las Transformaciones Sociales.

Module 1: Experiences from the Field: https://www.futurelearn.com/courses/desigualdades-america-latina/1/steps/266622

Module 3: Evaluating for Equity.
https://www.futurelearn.com/courses/desigualdades-america-latina/1/steps/275690#fl-comments

Module 4: From Knowledge to Policy
https://www.futurelearn.com/courses/desigualdades-america-latina/1/steps/276927

2016. Rethinking Research Policies and Practices. (at 43:30). Centre for Innovation, Policy and Governance, Jakarta.

2012. Emerging Practices in Evaluating Policy Influence. My M&E eLearning Series on Emerging Practices in Development Evaluation.

2010. Development is Not an Intervention. Presentation to the Virtual Conference on Evaluation, Witwatersrand University.

2009. Evidence-Based Policy Making. IFPRI, Washington. DC.

About Fred

Fred is an evaluation specialist with over 30 years’ experience in evaluating development research and programming, and the development of evaluation methodologies. In addition to his focus on methods, Fred has interest and expertise in the use of evidence in public policy. In 2015 Fred established Using Evidence Inc., a consultancy that focuses on efforts to improve the use of research and evaluative evidence in decision making.

Prior to that, for two years, he was based in Jakarta as RTI’s Lead Technical Advisor for the Knowledge Sector Initiative, a 15-year Australian funded project to support the knowledge to policy cycle in Indonesia.

Before joining RTI, Fred was Director of Evaluation at the International Development Research Centre in Canada, where he worked from 1993-2013. He is the author and co-author of several publications including “Knowledge to Policy: Making the most of development research,” and “Evaluating Capacity Development”. In his methods work, he has co-published “Outcome Mapping,” “Enhancing Organizational Performance,” and “Organizational Assessment,” as well as numerous articles on other evaluation challenges, such as evaluating equity, and strategy evaluation. He continues to write, with primary interests in the use of evidence in public policy, and the evaluation of change at the macro level. Forthcoming books include Local Knowledge Matters: Power, context and policy making in Indonesia (with K. Nugroho and H. Antlov, Policy Press 2018) and Knowledge, Politics and Policy Making in Indonesia, edited with Arnaldo Pellini, B. Prasetiamartati, K. Nugroho, and E. Jackson. Springer. 2018.

His current work includes, among others, quality control of evaluations, the development of tools for advocacy capacity evaluation, research on the use of agricultural research evidence at the national level in Africa, evaluating implementation research, and integrating learning and evaluative thinking more effectively into organizations.

Fred serves on the Board of the Partnership for Economic Policy; he sits on the editorial boards of the American Journal of Evaluation, the Journal of Evidence and Policy, and New Directions for Evaluation.

Fred holds a PhD from the University of Montreal (1990) and a Fellowship in Sustainability Science at Harvard (2008).