Kylie Hutchinson 
Evaluation Failures [PDF ebook] 
22 Tales of Mistakes Made and Lessons Learned

Foreword – Michael Quinn Patton
Acknowledgments
Introduction
PART I. MANAGE THE EVALUATION
Chapter 1. It’s Not Me, It’s You: The Value of Addressing Conflict Head On – Corey Newhouse
Chapter 2. The Scope Creep Train Wreck: How Responsive Evaluation Can Go Off the Rails – E. Jane Davidson
Chapter 3. The Buffalo Jump: Lessons After the Fall – Gail Vallance Barrington
Chapter 4. Evaluator Self-Evaluation: When Self-Flagellation Is Not Enough – Emma Williams
PART II. ENGAGE STAKEHOLDERS
Chapter 5. That Alien Feeling: Engaging All Stakeholders in the Universe – Hallie Preskill
Chapter 6. Seeds of Failure: How the Evaluation of a West African Agricultural Scale Up Project Went Awry – Thomas Archibald
Chapter 7. I Didn’t Know I Would Be a Tightrope Walker Someday: Balancing Evaluator Responsiveness and Independence – Rakesh Mohan
Chapter 8. When National Pride Is Beyond Facts: Navigating Conflicting Stakeholder Requirements – Felix Muramutsa
PART III. BUILD EVALUATION CAPACITY
Chapter 9. Stars in Our Eyes: What Happens When Things Are Too Good to Be True – Jara Dean-Coffey
PART IV. DESCRIBE THE PROGRAM
Chapter 10. A “Failed” Logic Model: How I Learned to Connect With All Stakeholders – Chris Lovato
Chapter 11. Lost Without You: A Lesson in System Mapping and Engaging Stakeholders – Kylie Hutchinson
PART V. FOCUS THE EVALUATION DESIGN
Chapter 12. You Got to Know When to Hold ’Em: An Evaluation That Went From Bad to Worse – Robert P. Shepherd
Chapter 13. The Evaluation From Hell: When Evaluators and Clients Don’t Quite Fit – Marla Steinberg
PART VI. GATHER CREDIBLE EVIDENCE
Chapter 14. The Best Laid Plans of Mice and Evaluators: Dealing With Data Collection Surprises in the Field – Jan Noga
Chapter 15. Are You My Amigo, or My Chero? The Importance of Cultural Competence in Data Collection and Evaluation – Isaac D. Castillo
Chapter 16. OMG, Why Can’t We Get the Data? A Lesson in Managing Evaluation Expectations – Jennifer Bisgard and Mary Pat Selvaggio
Chapter 17. No, Actually, This Project Has to Stop Now: Learning When to Pull the Plug – Karen Snyder
Chapter 18. Missing in Action: How Assumptions, Language, History, and Soft Skills Influenced a Cross-Cultural Participatory Evaluation – Susan Igras
PART VII. JUSTIFY CONCLUSIONS
Chapter 19. “This Is Highly Illogical”: How a Spock Evaluator Learns That Context and Mixed Methods Are Everything – Benoît Gauthier
Chapter 20. The Ripple That Became a Splash: The Importance of Context and Why I Now Do Data Parties – Diana Tindall
Chapter 21. The Voldemort Evaluation: How I Learned to Survive Organizational Dysfunction, Confusion, and Distrust – Lisa O’Reilly
PART VIII. REPORT AND ENSURE USE
Chapter 22. The Only Way Out Is Through – Stephanie Evergreen
Conclusion

Kylie Hutchinson took her first course in evaluation in 1988 and was hooked. She won the Canadian Evaluation Society’s Student Competition in 1990, made the leap into evaluation consulting in 1998, and has never looked back. In 2005 she was asked to deliver the Canadian Evaluation Society’s Essential Skills course in British Columbia which she did until 2011. This focus on training sparked a new interest in evaluation capacity-building and she enrolled in Vancouver Community College’s Instructor Development Program. By 2007 approximately 40% of her work was evaluation capacity-building. She continues to this day providing both her own private and sponsored workshops and webinars for clients such as the American Evaluation Association, Canadian Evaluation Society, and U.S. Centers for Disease Control. Since 2008, she has become increasingly interested in evaluation reporting, and has run a pre-conference workshop on the topic at AEA. Kylie acknowledges her roots are firmly rooted in the muddy trenches of evaluation practice versus academia. She has a passion for taking evaluation theory and making it accessible for evaluators to use immediately in their practice: this is evidenced by some of the free evaluation resources she has produced over the years including one-page reference sheets, a podcast, two videos, an e Learning course, a mobile learning course for senior decision-makers, and an evaluation glossary mobile app. She is also the author of An Innovative Guide to Evaluation Reporting and Survive and Thrive: Three Steps to Securing Your Program Sustainability