I had the good fortune to attend the “Measuring Performance in the Social Sector: Essential or Impossible?” panel discussion called hosted by the Schulich School of Business last month. Forums such as this one truly demonstrate the value of the “Schulich Community.” Brenda Gainer curated a fascinating panel of practitioners in socially-minded organizations. The well-attended evening also featured Dr. Alnoor Ebrahim, a former Torontonian, who is an Associate Professor at Harvard Business School.
The following are my big “take-aways” from the discussion.
As a consultant to practitioners, I found the discussion on selecting measures most applicable to client situations that I have seen. Dr. Ebrahim suggested that there are two fundamental questions to consider in selecting an appropriate measurement by which to judge your organization:
- How certain isf the causality you assign to your activities achieving the desired impact?
- What are the limits of your control in affecting that impact?
These are not easy questions to answer, but, if addressed sincerely, will spur a valuable discussion in any organizational context.
The “limit of control” part of the model mirrors the elements present in a logic model: inputs, outputs, outcomes and impact. For example: If the impact I am after is a cleaner environment in an urban centre, I could pursue an outcome of less landfill waste by running blue box programs (outputs). For this I would need money (inputs) to buy boxes, to promote use and enable distribution.
That example is oversimplified to illustrate the components. There could be violent disagreement as to whether something is an “outcome” or an “impact,” which transitions us into the “causal uncertainty” discussion. This is where the “theory of change” thinking is reflected. For example, Plan Canada’s “Because I am a girl” initiative links providing education for females in the developing world to a myriad of issues. According to The Public Health Agency of Canada, there is a strong link between “childhood vaccination” and “reduction of certain preventable illnesses.” If I understand the approach correctly, asking the question is not disputing the assumptions, but merely illustrating that some links have more empirical backing than others. Again, you can imagine the type of colourful conversation that could ensue with different stakeholder groups or even within one organization.
Once the dust settles on these discussions, you many find your organization “fits” in one of four general areas.
General Area #1 – Raise awareness to the problem even though there’s no clear solution
- Grid position: HIGH causal uncertainty/Control stops at Input and Output
- Description: “We can’t be sure that it works directly, but we think it is worth doing what we are doing to get the impact we want.”
- Useful measure: Measure your degree of influence over powerful stakeholders
General Area #2 – We will do more good by better doing what we do
- Grid position: LOW causal uncertainty/Control stops at Input and Output
- Description: “We know what we are doing works in delivering the impact that we want. We control how the resources are deployed.”
- Useful measure: Measure your output and your efficiency.
General Area #3 – Share information to figure out our impact
- Grid position: HIGH causal uncertainty/Control continues to Outcomes
- Description: “We have the means to control outcomes, but connecting what we do to the final impact is difficult.”
- Useful measure: Collaborate with others to find meaningful measures of impact.
General Area #4 – The goal is clear and we need to establish a common scoreboard
- Grid position: LOW causal uncertainty/Control continues to Outcomes
- Description: “The impact is clear. There are others involved in helping us get there.”
- Useful measure: Use integrated measures and an integrated approach.
CAUTION: In reading this article you are getting my thoughts on a framework that is currently under development. That said, the conversations about causality and control are very good to have. I will suggest that they may also be difficult to have because the questions challenge what might be institutional truths. Don’t be shy about asking for help!
Comments are closed