Performance Practice Introduction: External Evaluation

Introduction: External Evaluation

Performance Practice Learning Module

How can you use external evaluation to ensure mission effectiveness?

First, determine whether your organization is ready for external evaluation, and if it is, what type of evaluation is most appropriate at its stage of development.

This learning module is one of seven Performance Practice modules that help you and your team reflect on the extent to which your organization’s behaviors and practices align with the principles of high performance. Review the questions below—and share them with your colleagues—for an introduction to the principles and practices of external evaluation for mission effectiveness.

What can you learn about external evaluation by discussing these topics?

Do your leaders complement the organization’s internal monitoring with external evaluations conducted by highly skilled, independent experts?

  • To help drive improvements in your organization, do you periodically arrange for external evaluations conducted by experts with credibility in the field?

Do your leaders commission external assessments to learn more about how well programs are being run, what these programs are or aren’t accomplishing, who is or isn’t benefiting, and how the programs can be strengthened? Leaders don’t use external assessments as a one-time, up-or-down verdict on the organization’s effectiveness.

  • Are your organization’s external evaluations designed to assess the reliability and validity of your internal performance data, the quality of your implementation, and the overall effectiveness of your efforts?

Do your leaders recognize that there are many different types of external assessments and that no one type is right for every organization or for every stage of an organization’s development? Independent evaluators who understand how different methodologies fit different contexts can help leaders match the tool to the task.

  • Has your organization adopted a formal external evaluation plan that spells out the different types of evaluations that are relevant for you at different stages of your development? Do you update the plan periodically?
  • Does your organization’s plan include formative (implementation) evaluation to help you determine:
    • the quality of your internal data and program implementation?
    • whether you are delivering programs with fidelity to your model?
    • how well you are doing at recruiting and enrolling the population for which your programs are designed?
    • your program utilization, program completion, and participant engagement?
    • which of your clients achieve the intended outcomes, which do not, and which exit the program prematurely?
  • Does your organization’s evaluation plan include summative (impact) evaluation of programs that have been running as intended for several years to help you determine whether you’re making a difference beyond what would have happened anyway?

Do your leaders draw a clear distinction between outputs (e.g., meals delivered, youth tutored) and outcomes (meaningful changes in knowledge, skills, behavior, or status)? If you are working to improve outcomes, do you commission evaluations to assess whether they are having a positive net impact? In other words, you want to know to what extent, and for whom, you’re making a meaningful difference beyond what would have happened anyway.

  • Does your organization’s internal performance data clearly distinguish between outputs and outcomes—and has it been validated by independent experts?
  • Do your organization’s external evaluators use output data to help you learn about program quality and fidelity?
  • Do your organization’s external evaluators use outcome data to help you determine whether you’re making a difference beyond what would have happened anyway? This requires using a reliable research design to compare data from your participants with data from similar people who did not receive your services.

If your leaders have plans to expand significantly any programs aimed at improving outcomes, do they understand the special obligation of commissioning a rigorous evaluation that can assess net impact?

  • If your organization plans to grow significantly, are you conducting (or have conducted) both rigorous formative (implementation) and summative (impact) evaluations—with enough lead time to allow you to make critical adjustments and ensure that expanded programs will have the best chance of achieving net impact for those you serve?
  • Has (or will) your organization put growth plans on hold—with plans to redesign them before resuming growth—if/when evaluation findings show that you’re having significant trouble with implementation or your clients are not benefiting in the ways you had expected?

Even leaders who commission the most rigorous of impact evaluations don’t stop there. They commission additional assessments to gauge their impact in new settings (or for new populations) and achieve greater positive impact for the money they spend.

  • Does your organization conduct new external evaluations (formative or summative) whenever you make significant program changes, operate programs in new contexts, and/or enroll different target populations?
  • Because the societal context in which your organization and programs operate constantly changes, does your organization periodically conduct new summative evaluations?

Do your leaders share the methodology and results of external assessments to help others learn and avoid mistakes?

  • Does your organization share your evaluation plans throughout the organization and with interested stakeholders?
  • Does your organization share your evaluation findings throughout the organization as the basis for strengthening your programs and with external stakeholders who can benefit from the knowledge?
Download Printer-Friendly Version of This Introduction

Need Some Inspiration? Watch These Videos