Module 3 – Programs

Programs

Performance Practice Learning Module

How well-designed and well-implemented are your programs? Get together to make them even better.

If you need to define, measure, and produce better outcomes, this module helps you take the right next steps for your organization. It addresses program design within a larger context, the evidence base, and feedback from beneficiaries.

Proof points include using the right data to support high-quality implementation, building strong client relationships (for direct-service organizations), and resisting the temptation to introduce practices or programs that don’t align with the organization’s mission. Executive leadership, program directors, managers, and staff can work through this module together to make progress in the quest for better results.

1 LEARN

User Guide

Introduction to the Performance Practice, acknowledgments, application, and development methodology

Download File

2 COLLECT DATA

Worksheet

Use this worksheet to complete the self-assessment.

Download File

3 REVIEW RESULTS

Reporting App

The reporting app compiles your results - no manual aggregation required!

Windows Instructions
Windows Reporting App

MacOS Instructions
MacOS App, up to v10.12/Sierra
MacOS App, v10.13+/High Sierra

Watch: Reporting App Guide

Programs Principles and Proof Points

Principle 3.1: Leaders and managers who run programs that are intended to produce meaningful life changes hold themselves accountable for helping participants achieve desired outcomes. Programs intended to provide important but not life-changing products or services, such as food for homeless families, are not accountable for outcomes but must deliver high-quality outputs.

3.1.1: My organization’s leaders and managers ensure that we track the quality of our outputs in an ongoing manner using a clear set of appropriate and measurable indicators. We review output quality regularly and work continually to deliver high-quality outputs.

3.1.2: In cases in which program outputs are intended to drive participant outcomes, my organization’s leaders and managers track outcomes using a clear set of measurable indicators. Leaders and managers involve staff in monitoring outcome progression and working to improve results.

Principle 3.2: Leaders, managers, and staff treat constituents with respect, authenticity, and empathy.

3.2.1: My organization selectively hires those who have a deep understanding of the people and causes we serve and have demonstrated a strong ability to connect with people in a compassionate, accepting, and collaborative manner.

3.2.2: My organization cultivates these abilities through ongoing staff development.

3.2.3: My organization’s leaders and managers hold staff accountable for treating those we serve with respect, authenticity, and empathy.

Principle 3.3: Leaders and managers select or design their programs and strategies based on a sound analysis of the issues and evidence-informed assumptions about how the organization’s activities can lead to the desired change (often referred to as a “theory of change”).

3.3.1: My organization has assembled and regularly reviews the best available evidence as part of selecting, designing, and developing its key programs and strategies. (For service organizations, the continuum of evidence usually consists of the following, from weakest to strongest: 1) ideas put forward by credible practitioners; 2) ideas acknowledged and validated over time by practitioners in the field; 3) field-wide reviews of programs for which evidence of effectiveness exists; 4) knowledge that has been developed by researchers studying similar target populations; 5) applying data science techniques that can rigorously evaluate outcomes using longitudinal program data; 6) borrowing core elements from similar programs that have benefited from rigorous impact evaluations; and 7) research on the organization’s program(s) validated through rigorous impact evaluations.)

3.3.2: My organization has a theory of change that includes a target population/audience and a detailed service/program model with aligned outputs, outcomes, and measurable indicators.

3.3.3: My organization’s theory of change is:

  • plausible (makes sense to the informed reviewer)
  • doable (can be executed with available resources)
  • measurable (key elements can be monitored using qualitative and quantitative data)
  • testable (program model or advocacy strategies are codified in ways that allow for internal monitoring and external evaluation)
  • socially significant (success would have high value for our target population or cause).

3.3.4: My organization is committed to maintaining fidelity to our theory of change. We have instilled it in our culture and manifest it in the way we implement our programs; conduct our daily operations; and assess the quality and effectiveness of our programs.

Principle 3.4: Once programs and strategies are up and running, leaders and managers continually ask: “Are we collecting the information we need to ensure we’re effectively meeting the needs of our participants? Are we reviewing and acting upon the latest evidence in our field? Are we open to counter-evidence that suggests we should be doing things differently?”

3.4.1: To deliver highly effective programs and services, my organization periodically reviews what data it collects, the reasons for collecting these data, the quality standards for these data, and the uses for these data.

3.4.2: My organization designates an individual or team to review and report on relevant research in the field and flag findings that support or challenge our assumptions about program delivery, what works, and why.

Principle 3.5: Leaders and managers implement their programs in a high-quality manner using rigorous implementation standards.

3.5.1: My organization’s program teams implement our services based on codified program models that address:

  • intended outputs and outcomes
  • key implementation indicators and data-collection standards
  • phasing, dosage, and duration of activities
  • professional requirements for staff.

3.5.2: My organization holds an individual or team accountable for monitoring whether we are implementing our programs with fidelity.

3.5.3: My organization conducts frequent reviews of our implementation data and makes corrections to our activities in real time to improve quality and effectiveness.

Principle 3.6: Leaders and managers are sensitive to the cultural, racial, and political dynamics in the communities they serve and are open to making adjustments to their programs and strategies when these dynamics shift.

3.6.1: My organization invests time and other resources to study the local dynamics that affect our ability to deliver highly effective programs and services. (Depending on the type of organization, this could include identifying key influencers/power centers in a community, studying the historical roots underlying present-day attitudes, and/or mapping relevant programs or efforts engaging the same population or audience.)

3.6.2: My organization intentionally and routinely works to build strong relationships and productive collaborations with relevant organizations and influencers whose actions and decisions affect our target population/audience and our ability to succeed.

Principle 3.7: Leaders and managers establish and rigorously apply clear criteria for who is in their target population.

3.7.1: My organization has defined and made clear to all staff and stakeholders our target population (clients at the core of our mission with whom we work to achieve improvements in measurable outcomes) and/or our target audience (groups we need to influence if we are to create our intended knowledge, attitude, behavior, or policy change).

3.7.2: My organization collects data on how each enrollment aligns with our target-population criteria.

3.7.3: My organization actively applies the criteria in the process of enrolling new participants.

Principle 3.8: Leaders and managers do a good job of recruiting, retaining, motivating, listening to, and learning from their participants.

3.8.1: My organization is relentless about recruiting people in our target population.

3.8.2: My organization actively seeks feedback from members of our target population or target audience—those closest to the problems we’re addressing—and uses this information to help us design and improve our programs and strategies.

3.8.3: My organization is relentless about helping participants stay engaged until they achieve the intended outcomes and about learning why some drop out despite our best efforts to retain them.

Principle 3.9: In the case of direct-service organizations, all management and staff seek to build strong relationships with those they serve. These relationships are often the single biggest determinant of whether participants will stay engaged in programming and thereby achieve the desired results.

3.9.1: My organization’s managers and staff engage with participants in ways that make participants feel heard and understood.

3.9.2: My organization systematically uses data on staff-participant relationships to inform staff recruitment, training, coaching, and development—as well as drive program improvement.

Principle 3.10: Leaders and managers guard against the temptation to veer off course in search of numbers that look good in marketing materials or reports to funders.

3.10.1: My organization has checks and balances to ensure that the organization does not engage in corner-cutting measures (e.g., cherry-picking participants, biasing data) in pursuit of misleadingly impressive results.

3.10.2: My organization has checks and balances to ensure that we accurately report both the number and percent of enrolled participants who achieve intended outcomes.

3.10.3: My organization has checks and balances to protect against “mission creep”—chasing funding opportunities by tacking on new programs that stretch beyond our core purpose.


Note: These principles and proof points were updated in October, 2019 to align with release 2.0 of the Performance Imperative.

Download Principles & Proof Points for Programs

Sample Reports

With the free reporting app, you can create a report like the sample below in minutes. To create your own report, you will first need to collect your data. Go to any of the modules to get started.

Summary:
The Summary report shows the distribution of ratings for each proof point, gives a picture of the level of consensus, and opens the door to productive conversations about ways to move forward.

Details:
For a thorough understanding of individual perspectives, drill down to see each respondent’s ratings and comments per proof point.

Programs Module Principles & Proof Points
Principles and proof points for this module to provide overall perspective and context before completing the Workbook

Programs Module Worksheet
File to use to complete the self-assessment, includes instructions; open on laptop/desktop rather than mobile device

User Guide
Introduction to the Performance Practice, acknowledgments, application, and development methodology

Reporting App Downloads
The reporting app compiles data from all the worksheets into one spreadsheet.
Reporting App: Windows
Reporting App: MacOS (v10.13 + newer)
Reporting App: MacOS (v10.12 + older)

Got questions or feedback? Contact us at PerformancePractice@leapambassadors.org

Do NOT follow this link or you will be banned from the site!