Developing Monitoring and Evaluation Frameworks

This practical book provides clear, step-by-step guidance on how to develop a monitoring and evaluation framework in a participatory, logical, systematic, and integrated way. The authors outline the key stages and steps involved, including: scoping the framework; identifying planned results; using program theory and program logic; developing evaluation questions; identifying processes for ongoing data collection and analysis; determining means to promote learning; reporting; and dissemination of results. A final chapter focuses on planning for implementation of the framework, with reference to the broader program and organizational context. The authors draw on their extensive experience in developing monitoring and evaluation frameworks to provide examples of good practice that inform organizational learning and decision making, while offering tips and guidelines that can be used to address common pitfalls.

Table of Contents:
Preface Acknowledgments About the Authors Chapter 1 Introduction to Developing Monitoring and Evaluation Frameworks What Is a Monitoring and Evaluation Framework? Why Invest? What Functions Does a Monitoring and Evaluation Framework Serve? What Gap Does This Text Intend to Fill? Key Features of the Approach Program-Level Focus What Are the Differences Between Monitoring and Evaluation? What Are the Complementaries Between Monitoring and Evaluation? What is Included in a Monitoring and Evaluation Framework? Format and Layout of the Text Chapter 2 Foundation Concepts 1. Multiple Purposes for Monitoring and Evaluation 2. Results-Based Management Approach 3. Theory-Based Approach 4. Evaluation-Led Monitoring and Evaluation 5. Participatory Orientation Chapter 3 First Steps: Scoping the Monitoring and Evaluation Framework, Stakeholder Mapping and Evaluation Capacity Building 1. Introduction to Scoping the Framework With Key Stakeholders 2. Key Steps in Scoping the Framework Step 1: Identify Requirements Step 2: Determine Participation Arrangements Step 3: Identify Possible and Preferred Approaches Step 4: Review Resource Parameters Step 5: Confirm Purpose and Parameters of the Framework 3. From First Steps Onwards SUMMARY CHECKLIST CHAPTER REVIEW QUESTIONS Chapter 4 Program Theory and Program Logic as a Foundation for the Monitoring and Evaluation Framework 1. Introduction to Developing Program Theory and Program Logic 2. Steps Involved in Developing Program Theory and Program Logic Step 1: Plan Stakeholder Engagement Strategy Step 2: Develop Program Theory Step 3: Develop Program Logic Step 4: Confirm Program Theory and Program Logic with Key Stakeholders SUMMARY CHECKLIST CHAPTER REVIEW QUESTIONS Chapter 5 Evaluation Questions - Determining What We Want to Know 1. Introduction to Developing Evaluation Questions 2. Using Evaluation Domains to Guide Selection of Questions 3. Using Program Theory and Logic to Determine Evaluation Questions 4. Steps Involved in Developing Evaluation Questions Step 1: Develop Draft Evaluation Questions Step 2: Facilitate Stakeholder Participation Step 3: Scope Number and Range of Questions Against Data and Resources Available Step 4: Present Questions to Stakeholders for Final Endorsement Step 5: Finalize Evaluation Questions SUMMARY CHECKLIST CHAPTER REVIEW QUESTIONS Chapter 6 The Monitoring Plan 1. Introduction 2. Introduction to Program Monitoring 3. The Monitoring Plan 4. Steps in Developing Monitoring Plan Step 1: Identify Focus Step 2: Develop Performance Indicators and Targets Step 3: Identify Data Collection Processes and Tools Step 4: Determine Responsibilities and Time Frames 5. The Future of Program Monitoring SUMMARY CHECKLIST CHAPTER REVIEW QUESTIONS Chapter 7 The Evaluation Plan 1. Introduction 2. Focusing on Evaluation 3. The Evaluation Plan 4. Steps Involved in Developing the Evaluation Plan Step 1: Determine Overall Evaluation Approach Step 2: Identify Evaluation Questions Requiring Criteria and Standards Step 3: Identify Focus of Evaluation and Methods for Each Questions Step 4: Determine Responsibilities and Time Frames Step 5: Review the Monitoring and Evaluation Plans SUMMARY CHECKLIST CHAPTER REVIEW QUESTIONS Chapter 8 Collecting, Managing, Analyzing and Synthesizing Data to Reach Evaluative Conclusions 1. Introduction to Data Processes That Support Sound Conclusions 2. Organizational Context for Data Collection, Management, and Analysis 3. Data Quality 4. Steps Involved in Data Collection, Management and Analysis Step 1: Develop Data Collection Plan Step 2: Develop Data Management Plan Step 3: Consider Approach to Data Synthesis Step 4: Consider Approach to Making Evaluative Judgments and Reaching Evaluative Conclusions SUMMARY CHECKLIST CHAPTER REVIEW QUESTIONS Chapter 9 Learning, Reporting and Dissemination Strategies 1. Introduction 2. Steps Involved in Learning, Reporting and Dissemination Step 1: Consider Developing or Refining a Learning Strategy That Maximizes Use of Conclusions, Recommendations and Lessons Step 2: Consider the Identification of Recommendations and Lessons Step 3: Provide Guidance on Developing a Reporting and Dissemination Strategy SUMMARY CHECKLIST CHAPTER REVIEW QUESITONS Chapter 10 Planning for Implementation of Monitoring and Evaluation Frameworks 1. Introduction to Planning and Implementation 2. Key Steps in Implementation of Monitoring and Evaluation Frameworks Step 1: Confirm Program Management Arrangements Step 2: Work Planning for Implementation Step 3: Plan for Monitoring and Review of Framework SUMMARY CHECKLIST CHAPTER REVIEW QUESITONS Chapter 11 Conclusion Appendix References Index

About the Author :
Anne Markiewicz is an independent evaluation consultant and the Director of Anne Markiewicz and Associates. Commencing her professional career a social worker, Anne held academic positions in Social Work at two Australian Universities. In 1997, Anne pursued her interest in evaluation and commenced working as an independent consultant. In 2000, she completed a Master of Education in Program Evaluation at the University of Melbourne. Since that time, Anne has completed a significant number of evaluation projects for a range of Australian government departments, non-government organizations and international agencies. She has specialized in evaluating programs with social justice objectives. Anne’s practice as an evaluator has highlighted the need for early evaluation planning to provide a clear and agreed focus, ensure data availability and support the production of credible evaluations. She has increasingly focused on the development of Monitoring and Evaluation Frameworks for a range of clients. She has designed and delivered an array of related training programs, which also encompass areas such as stakeholder engagement, negotiation and evaluation reporting. This training has been delivered extensively in Australasia, particularly within Australia and Papua New Guinea.  Anne has a commitment to evaluation capacity building and has provided mentoring support to a range of government and non-government organizations. Anne is a past board member of the Australasian Evaluation Society, holding positions of Secretary and Vice President. She has received two awards for excellence in evaluation from the Australasian Evaluation Society. These were the ‘Indigenous Evaluation Award’ (2008) for an evaluation completed with the Australian Department of Finance and Deregulation, and the ‘Outstanding Contribution to Evaluation Award’ (2013). Anne has published articles on the political context to evaluation, balancing stakeholder interests in evaluation and the evaluation of programs for Indigenous Australians. Dr. Ian Patrick is an independent evaluation consultant and Director of Ian Patrick and Associates. Ian has a number of threads in his career which have supported his move into evaluation. Building on a background in teaching and radio and television production, Ian shifted his focus to international program management. He became aware of the challenges involved in evaluation of complex programs operating in the Asia-Pacific region while working for several Australian universities and managing contractors. Ian completed a PhD in Asian Studies in 1997, and commenced working in academic roles in international development for both the RMIT University and Deakin University, Australia. Ian is currently an Honorary Senior Fellow with the Development Studies Program at the University of Melbourne. Ian commenced working as an independent consultant in 1998 with roles encompassing evaluation, program design, strategy development and organizational review. The focus of this work has progressively shifted to concentrate on the evaluation area and has also achieved a balance between Australian and international contexts. He has undertaken evaluation related assignments for a range of international agencies, Australian government departments and non-government organisations. He has interests in evaluation practice in both public sector and civil society contexts with sectoral experience crossing areas such as law and justice, health, education, rural and urban development, environment, human rights, disaster management, media and the arts, and migration and Indigenous issues. Ian is a member of the Australasian Evaluation Society and the American Evaluation Association. In 2012, he received the Best Evaluation Policy and Systems Award from the Australasian Evaluation Society for a Monitoring and Evaluation Framework developed for the Mongolia Australia Scholarship Program.

Review :
"The text provides an excellent resource for developing monitoring and evaluation frameworks for a wide range of organizations and programs. Students and practitioners are provided with a great resource that can help organizations better society, and maximize evaluation efficiency by integrating monitoring with evaluation." This book does a brilliant job of integrating program monitoring concepts into the broader field of evaluation.  It fills a glaring gap in the evaluation literature, and I highly recommend it for practicing evaluators, evaluation scholars, and evaluation faculty and students. At last--a step-by-step guide to realizing the potential of monitoring and evaluation by building them in from the beginning. This book provides a systematic and well-tested approach to planning and successfully implementing an integrated monitoring and evaluation framework.