EVALUATION AND IMPACT MEASUREMENT
Life, people, families, communities and villages are messy. Even when it looks simple, the chances are that isn’t once you scratch the surface. We believe that no single organisation or programme, creates any social impact in complete isolation. So any attempt to seek attribution, should start from a desire to understand and improve rather than prove and must be tempered with a realistic understanding of the context.
During the summer of 2014 Louise Erskine PhD, Head of Programmes & Research @ Career Volunteer very kindly evaluated our two main activities, school building and “Clean Hands Saves Lives” projects. Her report is downloadable: Impact Study 2014
In addition to having a realistic understanding of the context, we need to explore impact measurement without breaking the bank. When possible we compare the outcomes of a given village we are currently working with to a similar village we have yet to work with. In practise this means we create a control group by selecting a community from our waiting list of villages needing support and compare the differences, before and after our project intervention. This is supported by secondary research on national averages across Ghana compared to the villages we are working in. Lastly we look at reductions of illness via the stats held at a very local clinic level.
If we want really to understand how change happens, then we need to embrace its complexity and not seek to isolate and attribute impact, but instead to understand our own contribution, and role, within the system. Everything we do is heavily influenced by the behaviour and practices of the participants: teachers, village elders, school children so we try and work with them and seek evidence of the impact of coalitions and groups, not specific projects in isolation.
Ultimately, if we’re trying to bring about change within a system, doesn’t it make sense to seek out this rich, complex, messy understanding of the world and stop pretending everything’s simple, and can be reproduced under experimental conditions? If we don’t, we may end up lying to our funders, lying to our beneficiaries, and lying to ourselves.
We believe leaders of NGO’s and charities need to stop obsessing about proving their impact, and instead focus on improving it.
(A lot of our thinking on M&E is directly from Tris Lumley, the Head of Development at NPC.)