This practical book provides clear, step-by-step guidance on how to develop a monitoring and evaluation framework in a participatory, logical, systematic, and integrated way. The authors outline the key stages and steps involved, including: scoping the framework; identifying planned results; using program theory and program logic; developing evaluation questions; identifying processes for ongoing data collection and analysis; determining means to promote learning; reporting; and dissemination of results. A final chapter focuses on planning for implementation of the framework, with reference to the broader program and organizational context. The authors draw on their extensive experience in developing monitoring and evaluation frameworks to provide examples of good practice that inform organizational learning and decision making, while offering tips and guidelines that can be used to address common pitfalls.
Table of Content
Preface
Acknowledgments
About the Authors
Chapter 1 Introduction to Developing Monitoring and Evaluation Frameworks
What Is a Monitoring and Evaluation Framework?
Why Invest?
What Functions Does a Monitoring and Evaluation Framework Serve?
What Gap Does This Text Intend to Fill?
Key Features of the Approach
Program-Level Focus
What Are the Differences Between Monitoring and Evaluation?
What Are the Complementaries Between Monitoring and Evaluation?
What is Included in a Monitoring and Evaluation Framework?
Format and Layout of the Text
Chapter 2 Foundation Concepts
1. Multiple Purposes for Monitoring and Evaluation
2. Results-Based Management Approach
3. Theory-Based Approach
4. Evaluation-Led Monitoring and Evaluation
5. Participatory Orientation
Chapter 3 First Steps: Scoping the Monitoring and Evaluation Framework, Stakeholder Mapping and Evaluation Capacity Building
1. Introduction to Scoping the Framework With Key Stakeholders
2. Key Steps in Scoping the Framework
Step 1: Identify Requirements
Step 2: Determine Participation Arrangements
Step 3: Identify Possible and Preferred Approaches
Step 4: Review Resource Parameters
Step 5: Confirm Purpose and Parameters of the Framework
3. From First Steps Onwards
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 4 Program Theory and Program Logic as a Foundation for the Monitoring and Evaluation Framework
1. Introduction to Developing Program Theory and Program Logic
2. Steps Involved in Developing Program Theory and Program Logic
Step 1: Plan Stakeholder Engagement Strategy
Step 2: Develop Program Theory
Step 3: Develop Program Logic
Step 4: Confirm Program Theory and Program Logic with Key Stakeholders
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 5 Evaluation Questions – Determining What We Want to Know
1. Introduction to Developing Evaluation Questions
2. Using Evaluation Domains to Guide Selection of Questions
3. Using Program Theory and Logic to Determine Evaluation Questions
4. Steps Involved in Developing Evaluation Questions
Step 1: Develop Draft Evaluation Questions
Step 2: Facilitate Stakeholder Participation
Step 3: Scope Number and Range of Questions Against Data and Resources Available
Step 4: Present Questions to Stakeholders for Final Endorsement
Step 5: Finalize Evaluation Questions
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 6 The Monitoring Plan
1. Introduction
2. Introduction to Program Monitoring
3. The Monitoring Plan
4. Steps in Developing Monitoring Plan
Step 1: Identify Focus
Step 2: Develop Performance Indicators and Targets
Step 3: Identify Data Collection Processes and Tools
Step 4: Determine Responsibilities and Time Frames
5. The Future of Program Monitoring
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 7 The Evaluation Plan
1. Introduction
2. Focusing on Evaluation
3. The Evaluation Plan
4. Steps Involved in Developing the Evaluation Plan
Step 1: Determine Overall Evaluation Approach
Step 2: Identify Evaluation Questions Requiring Criteria and Standards
Step 3: Identify Focus of Evaluation and Methods for Each Questions
Step 4: Determine Responsibilities and Time Frames
Step 5: Review the Monitoring and Evaluation Plans
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 8 Collecting, Managing, Analyzing and Synthesizing Data to Reach Evaluative Conclusions
1. Introduction to Data Processes That Support Sound Conclusions
2. Organizational Context for Data Collection, Management, and Analysis
3. Data Quality
4. Steps Involved in Data Collection, Management and Analysis
Step 1: Develop Data Collection Plan
Step 2: Develop Data Management Plan
Step 3: Consider Approach to Data Synthesis
Step 4: Consider Approach to Making Evaluative Judgments and Reaching Evaluative Conclusions
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 9 Learning, Reporting and Dissemination Strategies
1. Introduction
2. Steps Involved in Learning, Reporting and Dissemination
Step 1: Consider Developing or Refining a Learning Strategy That Maximizes Use of Conclusions, Recommendations and Lessons
Step 2: Consider the Identification of Recommendations and Lessons
Step 3: Provide Guidance on Developing a Reporting and Dissemination Strategy
SUMMARY CHECKLIST
CHAPTER REVIEW QUESITONS
Chapter 10 Planning for Implementation of Monitoring and Evaluation Frameworks
1. Introduction to Planning and Implementation
2. Key Steps in Implementation of Monitoring and Evaluation Frameworks
Step 1: Confirm Program Management Arrangements
Step 2: Work Planning for Implementation
Step 3: Plan for Monitoring and Review of Framework
SUMMARY CHECKLIST
CHAPTER REVIEW QUESITONS
Chapter 11 Conclusion
Appendix
References
Index
About the author
Dr. Ian Patrick is an independent evaluation consultant and Director of Ian Patrick and Associates. Ian has a number of threads in his career which have supported his move into evaluation. Building on a background in teaching and radio and television production, Ian shifted his focus to international program management. He became aware of the challenges involved in evaluation of complex programs operating in the Asia-Pacific region while working for several Australian universities and managing contractors. Ian completed a Ph D in Asian Studies in 1997, and commenced working in academic roles in international development for both the RMIT University and Deakin University, Australia. Ian is currently an Honorary Senior Fellow with the Development Studies Program at the University of Melbourne. Ian commenced working as an independent consultant in 1998 with roles encompassing evaluation, program design, strategy development and organizational review. The focus of this work has progressively shifted to concentrate on the evaluation area and has also achieved a balance between Australian and international contexts. He has undertaken evaluation related assignments for a range of international agencies, Australian government departments and non-government organisations. He has interests in evaluation practice in both public sector and civil society contexts with sectoral experience crossing areas such as law and justice, health, education, rural and urban development, environment, human rights, disaster management, media and the arts, and migration and Indigenous issues. Ian is a member of the Australasian Evaluation Society and the American Evaluation Association. In 2012, he received the Best Evaluation Policy and Systems Award from the Australasian Evaluation Society for a Monitoring and Evaluation Framework developed for the Mongolia Australia Scholarship Program.