Let’s face it. Evaluation can be overwhelming.
The time and expense to do a formal evaluation can be daunting, not to mention the access to skilled evaluators who are able to understand our organizations and what we really need from an evaluation.
And yet, how do we know if what we’re doing is truly meeting our organizations’ strategic goals if we don’t stop to assess it? Maybe we’re afraid of what an evaluation will tell us – or that the traditional mechanisms won’t capture what was amazing about our program. Without evaluating what is happening, however, we can’t see deeper reasons why some things worked while others did not. We can’t learn about what to do differently. We also miss the undiscovered stories of what it meant to our stakeholders, even if the outcome wasn’t perfect.
Perhaps our programs are full and we don’t see the point of evaluating since everything is going so well. But do we really understand the impact our program is having from the perspective of our stakeholders? Do we see the emerging needs? What if we could make our programs even better?
Luckily there are resources and tools available to help us on our way. It may be surprising to learn that most tools put their focus on the steps we need to take before doing an evaluation. Steps include deciding what we want to accomplish with evaluation, who needs to be involved and how we will share what we find. Thinking about evaluation in these terms helps to break down the process into bite sized pieces.
Here are some tips that will help us get ready to evaluate.
Build an evaluation mindset
When thinking about new program ideas, it can be very useful to dedicate time to consider how we will evaluate it. Weaving evaluation into all aspects of our operational activities pumps up our evaluation muscles. It helps to ensure that our activities are geared to outcomes that support our organizations’ strategic goals. Keep in mind that what we want out of evaluation could be in addition to what our funder requires at project end.
Become familiar with different kinds of evaluation
Doing an evaluation can seem out of reach if what comes to mind is an intensive process that requires expert knowledge. While this kind of evaluation is definitely part of the picture, there are other less formal ways to measure that will help evaluate our programs and inform what we need to do next. Bringing program participants or staff together for some honest discussion about what went well and what didn’t go well is considered a form of evaluation. If we come with an open mind and questions intended to stimulate discussion and reflection, this can be one way to gather feedback that feels attainable. (Supplying tasty snacks as an incentive doesn’t hurt, either.)
Thinking in practical terms about our capacity and appetite for evaluation will help us avoid an evaluation effort that is disappointing and leaves our organizations with little insight to inform future direction.
Here is a chart that can help us become familiar with the various kinds of evaluation activities and where they are best used.
Get clear on your purpose
What’s the difference between an indicator and an outcome? There is a lot of jargon-y language in evaluation and unless you do evaluation regularly, it can be difficult to remember. Fortunately, there is an app for that – the Evaluation Glossary Mobile App is a virtual dictionary of terms used in evaluation. In addition, this handy resource provides a quick overview of some commonly used terms in evaluation that overlap in their meaning.
No matter what terms we use, it’s important to get clear on our overall purpose for evaluation. Do we want to evaluate the way a program is operating? Do we want to measure our achievements after a project is complete? Do we want to evaluate the overall effectiveness of a project or program? Understanding what we want to get out of an evaluation will help to determine the approach. Here’s a quick primer on the various reasons for evaluation to get us thinking about how we intend to use our evaluation results.
The ONN also has a discussion guide that can help stimulate dialogue with our boards, funders or other stakeholders when planning an evaluation.
Don’t do it alone
Effective evaluation relies on inviting the help and feedback of a variety of stakeholders: people who care about what we are doing and can share their perspective. This could include service recipients, staff, volunteers, board members, peer organizations, and funders. Engaging them in the planning process helps to ensure their buy-in and support during the evaluation activities. Their input can also give us valuable perspective on how to get the most out of our evaluation efforts. Here is a tool to help identify our stakeholders in evaluation.
Include story in your strategy
Once we decide on our goals for evaluation and select an approach that fits, we will need to consider which evaluation tools will help us gather the information we need. There are many tools available; surveys, interviews, focus groups and documentation of project activities are among the most popular. Other tools include journal recordings of project participants, staff reports on daily activities, and stories. Stories for evaluation purposes can be narratives about an event, but the most powerful kind of story is personal. Collecting stories from people who have been impacted by our work is one of the most effective ways to share our outcomes. Stories can communicate the real and human difference we are making that data alone cannot convey. Impact stories can be collected and shared individually. A group of stories can also be analyzed for evaluative purposes to expose their common themes. Stories are an amazing way to gather evidence, as they can be told by anyone with experience of your organization. There is no special knowledge of nonprofits or of evaluation required. The Most Significant Change approach is one way that stories have been used for evaluative purposes. You can read more about it here.
Taking the time to plan an evaluation allows us to get clear on how it will support our strategic goals and will guide our choice of approaches and tools. Having an evaluation strategy that is geared to our organizations’ needs and capacity will make the process more relevant and interesting for everyone involved. This process can make the whole idea of evaluation less overwhelming and fuel our curiosity to discover how our organizations are really making a difference.
This week’s blog was written by Liz Dennis. As PLC’s Communications and Evaluation Manager, Liz brings her passion and 10 years’ experience in the nonprofit sector to PLC as the first point of contact with clients. She also works with clients to help them understand the impact of their work with PLC through evaluation and story. Connect with Liz at firstname.lastname@example.org.