Two years ago, nonprofit leaders across the Pacific Northwest shared that evaluating their organization's impact was the skillset they felt least equipped for. It showed up as the lowest-rated capacity in four of five states in the 2024 Nonprofit Capacity Survey, administered by the state nonprofit associations serving Alaska, Idaho, Montana, Oregon, and Washington*. A gap that consistent led these nonprofit association leaders to ask: what’s really getting in the way of evaluation, and what needs to change in the sector to better support this critical practice?
In partnership with Dialogues in Action and with funding from the Murdock Trust, the five state nonprofit associations conducted a qualitative study of the practices, dynamics, context, and opportunities for developing the nonprofit ecosystem of evaluation. The results draw on in-depth, one-on-one interviews with 110 leaders of 501(c)(3) organizations across all five states.
Key Findings on Nonprofit Evaluation Capacity #
The report identified 12 key findings. Here are just a few:
- Evaluation has often been treated as a compliance requirement or measuring stick for funders and donors. But some nonprofits are shifting toward evaluation as a tool for continuous improvement, using data to celebrate their staff when their work is making a difference, and changing strategy and methodology when the evidence calls for it.
- The sector lacks resources for evaluation literacy. Many organizations already have data and dashboards, but it takes training to turn these numbers into meaningful insights.
- Evaluation works best when it isn't owned by one person or one team, but is built into how staff at every level do their jobs.
- The qualitative evidence nonprofit leaders find most convincing (the stories and context behind the numbers) is often the hardest to capture in formal evaluation.
- Data requests across funders and government systems often do not connect, requiring duplicate data. There is great opportunity for coordination across the funding ecosystem.
Mental Modes Behind Nonprofit Evaluation #
Interviews surfaced a few common assumptions about the purpose and benefit of evaluation, some of which are getting in the way, and some of which are strengthening evaluation culture across the sector. Here are a few of each from the report:
Assumptions That Hold Nonprofits Back #
- “Measurement means numbers.” When leaders assume measurement is always quantitative, it becomes easy to conclude that the parts they care about – the stories and people – are impossible to evaluate.
- “Evaluation is mostly a technical specialty.” Believing this means you’re probably missing out on the observations and judgement of your staff, and their deep knowledge of your work, even if they aren’t trained analysts.
- “Evaluation is an event.” In reality, evaluation is most effective when it’s seen as part of a broader culture of learning, and an ongoing practice.
Assumptions That Strengthen the Social Sector #
- “Evaluation should be curious and brave, not just confirmatory.” Leaders that can use evaluation as data to improve impact rather than for confirmation bias are well positioned for success.
- “Claiming less makes the claim stronger.” Only claim what you’re confident about, and acknowledge that your work is one piece of the puzzle.
- “The perspectives of the people served are needed to understand what the data mean.” Bringing the data back to participants and partners strengthens evaluation, and treats community judgement as part of the analysis.
- “Choosing what not to measure is part of the design.” Being selective in what you evaluate strengthens organizational focus.
What Should Nonprofits, Capacity-Builders, and Funders Do Next? #
The report concludes with recommendations for nonprofits, capacity-building organizations, and funders. Here are a few steps forward for each:
Nonprofits #
- Nonprofits can strengthen their evaluation muscle by building reflection into every aspect of their work. Post-program debriefs, quarterly reporting meetings, and ongoing data-collection in staff meetings are all evaluation.
- Nonprofits can use the data they already have, before investing in anything new.
Capacity-Builders #
- Capacity-building organizations can convene peer cohorts and create structured time for nonprofits to learn from each other as they evaluate together.
- Capacity-building organizations can provide right-sized tools for different organizational sizes, missions, and stages, knowing that evaluation changes based on sector and scale.
Funders #
- Funders can fund evaluation itself, including supporting the shared infrastructure and networks that help nonprofits evaluate together.
- Funders can coordinate and simplify reporting. Fewer substantive reports and common applications would give nonprofits more time to focus on what they’ve learned from evaluations.
- Funders can share aggregated findings back with nonprofits, making reporting a two-way exchange and fostering learning across the sector.
Explore expanded insights, two diagnostic tools to evaluate your nonprofit's own evaluation practices, and download the full report at the Northwest Nonprofit Capacity Research website.
This research would not have been possible without the willingness of 110 nonprofit leaders to engage in this conversation, or the five state associations taking the 2024 survey findings seriously, or Dialogue in Action’s expertise and leadership. Thank you to all of them for equipping the sector to see itself more clearly and find a productive way forward.
*The Foraker Group in Alaska, Idaho Nonprofit Center (a program of the Idaho Community Foundation), Montana Nonprofit Association, Nonprofit Association of Oregon, and Nonprofit Association of Washington