Marlene Laubli Loud & John Mayne 
Enhancing Evaluation Use [EPUB ebook] 
Insights from Internal Evaluation Units

Enhancing Evaluation Use: Insights from Internal Evaluation Units offers invaluable insights from real evaluators who share strategies they have adopted through their own experiences in evaluation. Readers will learn about the challenges, solutions, and lessons drawn from the experience of evaluators working in a wide range of organizations. Referencing the latest literature, contributors discuss factors that help or undermine attempts to foster an evaluative thinking and learning culture within an organization. Applicable in a wide range of situations, their accounts demonstrate the initiative and innovative thinking they use to address challenges in various, sometimes complex, evaluation settings. Questions at the end of each chapter stimulate thought and discussions about the issues raised and allow readers to apply their findings to their own situations.

‘This book speaks to a cutting-edge topic, that is, the potential to generalize program evaluation expertise to larger organizational questions, and the cases from multiple international contexts represent a unique feature.’
—John Clayton Thomas, Georgia State University

 

‘The use of actual cases to highlight major concepts in evaluation in the public sector is a great feature.’
—Danica G. Hays, Old Dominion University

 

‘The text provides practical information from a variety of organizational contexts and the integration of international experiences provides for expanded discussion of evaluation theory and practice.’
—Kathleen Norris, Plymouth State University

 

‘The key strengths of this book lie in its national, supra-national and international organizational contexts, its consistency in insider perspectives, and the detailed examples provided.’
—Donna Haig Friedman, University of Massachusetts, Boston

‘The book of essays reviewed here was edited by two eminent evaluators. It fills an important gap in the literature: in pursuit of improved quality of evaluation products, evaluation thinkers have lavished attention on evaluation methods, ethics and use but they have sorely neglected evaluation governance issues and have largely failed to probe the workings of evaluation within organizations.



Yet, most evaluations are commissioned by (or undertaken within) organizations. The choices organizations make in structuring evaluation functions and designing evaluation processes have a major bearing on the relevance, validity and usefulness of evaluations. Refreshingly, the book offers fresh mental models and practical lessons about evaluation systems and practices adopted in diverse organizational settings.



All contributors to the book are seasoned practitioners. They hail from national, supranational and international organizations and many of them have trespassed across these thematic and organizational boundaries. They all are equipped to draw on a vast reservoir of hands—on experience as evaluation commissioners, managers, internal evaluators or external practitioners. Remarkably, they also display considerable familiarity with relevant themes treated by the organization management and evaluation literature.



Given its pragmatic focus the book is bound to elicit broad based interest among evaluation practitioners. While it addresses familiar dilemmas and challenges (evaluation independence, evaluation utilization, organizational learning, nurturing of an evaluation culture, etc.) it does so from the distinctive perspective of ‘insiders’ who have had to contend with a variety of organizational constraints and management pressures. Revealingly most contributors find good reasons to be optimistic about the possibility of positive change.



The stage is set by John Mayne whose introductory and authoritative chapter identifies the critical importance of organizational factors in the utilization of evaluation results. This is followed by Bastiaan de Laat’s chapter which puts forward an ingenious analytical construct: the ‘tricky triangle’ of relationships that links commissioners, evaluators and evaluands. Specifically De Laat unmasks and assesses the different and complex configurations that result from this triangular interplay.



Penny Hawkins’ elegant contribution underlines the importance of evaluation independence. It highlights the considerable impact that the electoral cycle may have on evaluation approaches. It also reveals that indigenous cultures may present formidable obstacles to the very notion of external evaluative oversight. The next chapter penned by Marlene Laubli’s is equally perceptive in tracing the creative adaptation that the evaluation function underwent in response to changes in the organizational force field of the Swiss Federal Office of Public Health’s Evaluation Unit.



In a similar vein Erica Wimbush provides a lucid exposition of the strategies used to bridge evaluative knowledge and management action in Scotland’s public health agency. Still within the health field, Nancy Porteous and Steve Montague show convincingly that transformative organizational change in Canada’s Public Health Agency was facilitated through judicious evaluation programming and what has come to be called ‘double loop learning’. Finally, Bastiaan de Laat and Kevin Williams summarize important findings from of two empirical studies of evaluation use in the European Commission that go a long way in identifying good practices for evaluation commissioning in a supra— national organizational context.



The next two chapters of the book focus on two specialized agencies of the United Nations. They tread gently in what is admittedly a highly sensitive and contested terrain. Maria J. Santamaria Hergueta, Alan Schnur and Deepak Thapa discuss how the World Health Organization’s evaluation function evolved towards greater independence over several decades while Janet Neubecker, Matthew Ripley and Craig Russon focus on the innovative techniques used within the International Labour Organization to enhance utilization of self evaluation findings.



The final chapter by Marlene Laubli Loud misses the opportunity to compensate for the ‘good news’ bias that understandably characterizes the previous contributions. Surprisingly, it displays ambivalence towards evaluation independence which it equates with externality, a concept soundly rejected by de Laat. But setting this dimension aside, the chapter is interesting and valuable since it reflects the co—editor’ vast experience as an evaluator, evaluation trainer and evaluation manager in diverse national and international contexts.



Thus the chapter provides a useful compilation of ‘take away’ messages focused on good evaluation management practices. It is on solid grounds when it describes how to make effective use of advisory committees, motivate potential evaluation users, design good terms of reference, select the right evaluators, craft judicious evaluation policies and keep an eye out for political ‘windows of opportunity’.



Each chapter of the book concludes with a crisp exposition of overarching findings, discussion topics and references. Thus the volume should be of practical value to teachers, students, professional evaluators as well as evaluation commissioners and programme managers. All in all, this is a book that belongs on your shelf if you are intent on enhancing the role that evaluation plays in your organization.’

—Robert Picciotto,
UKES Council Member

Mục lục

Preface – Marlène Läubli Loud

1. Issues in Enhancing Evaluation Use – John Mayne

2. Evaluator, Evaluand, Evaluation Commissioner: A Tricky Triangle – Bastiaan de Laat

3. Evaluation Management: Lessons From New Zealand and International Development Evaluation – Penny Hawkins

4. Institutionalization and Evaluation Culture—Interplay Between the One and the Other: Lessons From the Swiss Federal Office of Public Health (FOPH) – Marlène Läubli Loud

5. Reshaping Evaluation to Enhance Utilization in Scotland: The Role of Intermediary Bodies in Knowledge-to-Action Strategies – Erica Wimbush

6. From Discrete Evaluations to a More Holistic Organizational Approach: The Case of the Public Health Agency of Canada – Nancy Porteous and Steve Montague

7. Evaluation Use Within the European Commission (EC): Lessons for the Evaluation Commissioner – Bastiaan de Laat and Kevin Williams

8. Evaluation Policy and Practice in a Changing Environment: Evolution of the Evaluation Function in the World Health Organization (WHO) – Maria Santamaria, Alan Schnur, and Deepak Thapa

9. Building for Utilization: The Case of the International Labour Organization (ILO) – Janet Neubecker, Matthew Ripley, and Craig Russon

10. What Can We Learn From Practitioners? Some Thoughts and Take-Home Messages for Internal Evaluation Units – Marlène Läubli Loud

Index

About the Editors

About the Contributors

Giới thiệu về tác giả

John Mayne (Ph D) is an independent advisor on public sector performance. He has been working with a number of organizations and jurisdictions around the world, including several agencies of the UN and development banks, a number of governments and international NGOs, the European Union, the OECD and several Canadian federal departments on results management, evaluation, and accountability issues. Until 2004, he was at the Office of the Auditor General where he led efforts at developing practices for effective managing for results and performance reporting in the government of Canada, as well as leading the office’s audit efforts in accountability and governance. Prior to 1995, John was with the Canadian Treasury Board Secretariat and Office of the Comptroller General. He has authored numerous articles and reports on results management, evaluation, and evaluation methodologies and edited five books in the areas of evaluation, public administration, and performance monitoring. In 1989 and in 1995, he was awarded the Canadian Evaluation Society Award for Contribution to Evaluation in Canada. In 2006, he became a Canadian Evaluation Society Fellow.