About the Book
Recognizing that complexity calls for innovative, conceptual, and methodological solutions, this unique book offers practical guidance to policymakers, managers, and evaluation practitioners on how to design and implement complexity-responsive evaluations that can be undertaken in the real world of time, budget, data, and political constraints. Introductory chapters present comprehensive, non-technical overviews of the most common evaluation tools and methodologies, and additional content addresses more cutting-edge material. The book also includes six case study chapters to illustrate examples of various evaluation contexts from around the world.
Table of Contents:
Part 1: Dealing with Complexity in Development Evaluation: A Framework
Chapter 1: Complexity in Development Evaluation: The Framework of the Book - Michael Bamberger, Jos Vaessen, and Estelle Raimondo
Chapter 2: Toward More Complexity-Responsive Evaluations: Overview and Challenges - Estelle Raimondo, Jos Vaessen, and Michael Bamberger
Chapter 3: Management of Complexity-Responsive Evaluations - Michael Bamberger
Part 2: Dealing with Complexity in Development Evaluation: Methodological Approaches
Chapter 4: Impact Evaluation Approaches and Complexity - Jos Vaessen, Estelle Raimondo, and Michael Bamberger
Chapter 5: Understanding What is Being Evaluated: Theory-Based Evaluation - Frans L. Leeuw
Chapter 6: Five Practical Evaluation Problems to Which TBE Can Contribute - Frans L. Leeuw
Chapter 7: Dealing with Complexity by Unpacking and Reassembling Elements of a Complex Program - Michael Bamberger, Estelle Raimondo, and Jos Vaessen
Chapter 8: The Importance of a Mixed Methods Approach for Evaluating Complexity - Michael Bamberger
Part 3: Emerging Data and Innovative Techniques to Deal with Complexity in Development Evaluation
Chapter 9: Complexity in Review and Synthesis Studies - Jos Vaessen
Chapter 10: Emergent Technologies and Creative Use of Multiple Sources of Information - Susan van den Braak, Sunil Choenni, and Michael Bamberger
Chapter 11: Applying Emergent Technologies to Complex Program Evaluation From the INGO Perspective - Kerry Bruce and Alison E. Koler
Chapter 12: The Evaluation of Complex Development Interventions in the Age of Big Data - Emmanuel Letouzé, Ana Areias, and Sally Jackson
Part 4: Dealing with Complexity in Development Evaluation: The Institutional Challenges
Chapter 13: Dealing with Institutional Complexity: Implications for Evaluation Design, Process, and Use - Estelle Raimondo
Chapter 14: Gender Equality in Development Evaluation: The Intersection of Complexities - Estelle Raimondo and Michael Bamberger
Part 5: Complexity of Evaluation in Practice: Case Studies
Chapter 15: A Case Study in Complexity: Evaluating a Long-Term Effort to Prevent Gender-Based Violence in El Salvador - Allison R. Davis and Melida Guevera
Chapter 16: Microcredit and Women′s Empowerment: Complexity in Systematic Review - Jos Vaessen, Ana Rivas and Frans L. Leeuw
Chapter 17: Evaluation of Coordination Against Trafficking in Persons: A Case Study of a Complexity-Responsive Evaluation - Kim Forss
Chapter 18: Complexity from the Perspective of Philanthropic Foundations and Their Evaluation Practices - Leny van Oijen
Chapter 19: Evaluating General Budget Support - Antonie de Kemp and Geske Dijkstra
Chapter 20: Dealing with Complexity in a Realist Synthesis: Community Accountability and Empowerment Initiatives - Gill Westhorp, Bill Walker, and Patricia Rogers
Glossary
Index
About the Author :
Michael Bamberger has been involved in development evaluation for fifty years. Beginning in Latin America where he worked in urban community development and evaluation for over a decade, he became interested in the coping strategies of low-income communities, how they were affected by and how they influenced development efforts. Most evaluation research fails to capture these survival strategies, frequently underestimating the resilience of these communities – particularly women and female-headed households. During 20 years with the World Bank he worked as monitoring and evaluation advisor for the Urban Development Department, evaluation training coordinator with the Economic Development Department and Senior Sociologist in the Gender and Development Department. After retiring from the Bank in 2001 he has worked as a development evaluation consultant with more than 10 UN agencies as well as development banks, bilateral development agencies, NGOs and foundations. Since 2001 he has been on the faculty of the International Program for Development Evaluation Training (IPDET). Recent publications include: (with Jim Rugh and Linda Mabry) RealWorld Evaluation: Working under budget, time, data and political constraints (2012 second edition); (with Marco Segone) How to design and manage equity focused evaluations (2011); Engendering Monitoring and Evaluation ( 2013 ); (with Linda Raftree) Emerging opportunities: Monitoring and evaluation in a tech-enabled world (2014); (with Marco Segone and Shravanti Reddy) How to integrate gender equality and social equity in national evaluation policies and systems (2014). Jos Vaessen (Ph.D. Maastricht University) is Principal Evaluation Specialist at the Internal Oversight Service of UNESCO in Paris and lecturer at Maastricht University, The Netherlands. After completing his M.Sc. in 1997 (Wageningen University) and prior to starting his current position at UNESCO in 2011, he has been involved in research, teaching and evaluation activities in the field of international development at Antwerp University and, more recently, Maastricht University. Over the last fifteen years or so, he has worked for several multilateral and bilateral international organizations mostly on evaluation-related assignments. His fields of interest include: theory and practice evaluation, impact evaluation, rural development and environment. In addition to managing and conducting evaluations Jos regularly serves on reference groups of evaluations of different organizations.
He has been (co-) author of more than 30 publications, including three books. Recent publications include: Impact evaluations and development – NONIE guidance on impact evaluation (2009, co-author, with F. Leeuw), Mind the gap: perspectives on policy evaluation and the social sciences (2009, co-editor,with F. Leeuw), The effects of microcredit on women’s control over household spending in developing countries (2014, coordinator and first author, with A. Rivas, M. Duvendack, R. Palmer Jones, F. Leeuw, G. van Gils, R. Lukach, N. Holvoet and J. Bastiaensen, J.G. Hombrados, and H. Waddington). Estelle Raimondo is a Research Associate and a Ph.D. candidate at the George Washington University’s Trachtenberg School of Public Policy and Public Administration where she specializes in development evaluation. Prior to joining GWU, Estelle served as an Associate Evaluation Specialist for the United Nations Educational, Scientific and Cultural Organization (UNESCO). In this capacity, she conducted a number of evaluations and policy reviews especially in Africa and worked closely with the United Nations Evaluation Group on integrating gender into evaluation practice. Over the last five years or so, Estelle has worked on a number of evaluation and research assignments for a range of organizations (e.g., the World Bank, the Independent Evaluation Group, UNEG, the Polish Government, the NSF, The GW Regulatory Center). Her current research focuses on how monitoring and evaluation can contribute to organizational and strategic change in international development agencies. She is exploring issues of evaluation culture, knowledge brokering and result-based management’s adequacy for organizational learning. Estelle is also researching how to adequately combine a range of methodologies stemming from different fields (e.g., quasi-experimental design, Case-based methods and systems-thinking) to address complex issues, such as gender equality. Estelle graduated summa cum laude from the Paris School of International Affairs with an M.A. in International Economic Policy. She also holds a Master of International Affairs in Economic and Political Development from Columbia University’s School of International and Public Affairs (SIPA) where she studied as a Fulbright scholar.
Review :
This volume provides useful guidance to development evaluation specialists to deal with their increasingly complex contexts and tasks.
The book shows, through a meticulous review of academic studies and some specific cases of real evaluations, that complex development interventions can be evaluated effectively and some of the techniques that make this possible.
This is the ′nuts and bolts′ book that evaluation practitioners have been waiting for. Authored by highly experienced evaluators and grounded in real life examples from all parts of the world it illuminates the core ideas of complexity theory, makes sense of abstruse systems concepts and guides the reader through the contemporary maze of program theory, mixed methods and web based information technologies. A lucid and comprehensive treatment of the evaluation state of the art it is a must-have text for emerging evaluators, advanced evaluators and evaluation managers operating in uncertain, turbulent and conflict ridden environments.
This is an extremely useful book when managing and carrying out evaluations of the increasingly complex nature of international development. It should be required reading for everyone involved in assessing results of development cooperation.
This book is impressive in its breadth and depth of coverage of a topic increasingly on the minds of evaluation practitioners. It provides much needed practical guidance and valuable conceptual insights in an area that’s often discussed but rarely well understood. The book will appeal to a wide range of experienced evaluation professionals, emerging evaluators and students of evaluation.
This is one of the most important books ever published in the field of development evaluation. Dealing with complex systems is a premier challenge for evaluators working in real life situations. As a practitioner in the field I cannot emphasize the value of this book enough. The authors tackle the issue of complexity and its critical significance from both theoretical and practical points of view. They offer concrete ways in which evaluators and organizations can deal effectively with complexity. The editors, each one of them an eminent authority in the field, have put together a group of authors who bring a wealth of knowledge and experience to the table. Highly recommended to evaluation commissioners and practitioners, as well as those with an interest in the scientific aspects of evaluation approaches and methodologies.
This is the first book that seeks to operationalize a complexity perspective into mainstream development evaluation, focusing on the specificities of this broad based sector, encompassing multiple objectives, institutions, contexts, constituencies and beneficiaries. The main conclusions can speak then to larger audiences of evaluators, program designers, managers, and researchers throughout the world, making a very useful contribution to evaluation culture and practice.
