Measuring Impact: Notes from a True Charity Chat

AVERY WEST
Member Engagement Director
Read more from Avery

Listen to this article:

In November, True Charity Network members from around the country gathered virtually for a casual “True Charity Chat” to brainstorm and share experiences. This quarter’s topic was “Outcomes Workshopping.” 

Nathan Mayo, VP of Programs, highlighted resources in the Outcomes Toolkit and shared best practices he’s learned from helping members implement them. 

Here are four tips from that conversation:

• Ask, “What am I going to do with this information?”

Ask this before creating documents, incentives, and systems to track data. Otherwise, you may end up with information that’s not useful. 

For example, True Charity used to track Summit attendees’ program changes to guide us in making Summit decisions. Eventually, we realized we were using another measure: attendees’ feedback on topic preferences. We also discovered program changes are extremely hard to track; and we learned our annual “Share Your Success Campaign” is a more streamlined way to capture the data were were looking for. That said, if you believe data you’ve collected is useful and suggests change is needed, go for it! If at some future point it’s no longer valuable, you can always drop it.  

• Learners’ confidence is a key measure of success.

To illustrate, suppose your organization is committed to teaching people how to change car tires. You’d probably assess their increased knowledge by asking, “What is the first step in changing a tire?” Likewise, if you’re all about helping people prepare for a job interview, you might ask, “Which of the following outfits should you wear?” 

At best, these questions sample only a small amount of the knowledge you shared. A better question is, “How confident do you feel in changing a tire?” or “in choosing an outfit for your interview?” An assessment of confidence levels before and after the class will be a better indicator of the success of the class than whether a single question was answered correctly.

• Differentiate between outcomes for different groups of people.

Nathan shared that organizations often mistakenly track the same outcomes for different clients. For instance, a shelter might measure how many people go through their case management program and then find a job. However, a job might not be the right next step for elderly or severely disabled individuals. Instead, measuring their sense of hope about life or their social capital would be a better indicator of success. 

• Keep in touch with your graduates through events, gift cards, and more!

Much of our conversation focused on the challenge of keeping in touch with clients long enough to see if a program made a difference. Nathan encouraged leaders to follow up with a representative sample of clients rather than attempt to track down every one. 

For instance, six months after the completion of training, an organization might speak with every 10th (or 100th) member who went through their car maintenance program. Focusing on individuals from a smaller group also gives you the opportunity to offer a gift card or take that person out to lunch. 

Jim Ott, a Network member in Dubuque, Iowa, shared that previous graduates of their program come back to lead cohorts of new participants. They build relationships and reach out to those members after graduation. He feels a more personal, “How are you doing?” text from a friend leads to greater response than a, “Please fill out this survey” link from their organization. 

The Georgia Center for Opportunity (GCO) hosts post-graduation follow-up events. Graduates are invited to participate in fun, light-hearted professional development and meet with their mentors. This gives the GCO staff an opportunity to check in with them and monitor their progress. 

If you missed this conversation, don’t worry! In our Outcomes Toolkit, Nathan and the True Charity team have compiled a step-by-step guide (complete with video training, documents, and spreadsheets) to help you get started measuring the right kind of outcomes for your organization. 

Join our True Charity Network community to access more Outcomes material and participate in future “True Charity Chats.”