Measuring Social Impact: Truly Creating Value or Fooling Yourself?

Full of ideas on how to innovate and solve world problems: designers and organizations come up with the most promising ideas to change the world and lives of people. But, how do we know if value is created or if opportunities are wasted?

Social impact design, a term that is referring to ‘the practice of design for the public good, especially in disadvantaged communities’ (Smithsonian Institution, 2013), has gained the interest of designers in recent years. This could be due to the recent studies showing that our world is in desperate need of change. One in six people live in extreme poverty (Olinto and Uematsu, 2013) and according to UNICEF (2009) around 24,000 children under the age of 5 died every day as a consequence of poverty in 2008. This is one child about every 3.5 seconds.

Simultaneously, the belief that effectiveness should be measured at the level of impact has grown, “for how can programs know whether what they are doing is working unless such tracking is done?” (Hatry, 2010). Proving that you are making a difference is becoming crucial to allocate money to the most effective strategies and to help in making choices about the actual implementation of products and services.

Though, according to the findings of the Smithsonian Institution (2013), the biggest gap existing here is that the measurement of the perceived value of social impact design is limited or even absent. Designers seem to have only limited data to persuade organizations such as nonprofits and government agencies to implement the design.

However the aim to ‘do good’ is present, organizations and designers seem to frequently support or continue with interventions of which the effectiveness is unknown (Liket, 2014). Furthermore, having no data about social value means that you are unable to improve the effectiveness. Convincing good intentions seem to be more important than actual cost-benefit analysis.

An example of how decision-making should have been influenced by measuring the actual impact of a product is the Playpumps project in which water pumps are powered by a children’s roundabout, designed for developing countries. Sounds great, but happens to be counterproductive after having over 4000 PlayPumps put in African villages. A quarter of the pumps had to be repaired quickly, children missed school as they had to pump water, it was too intensive and humiliating for older people to use and the pumps cost $14,000 each, over four times as expensive as a normal pump (Georg, 2015). The waste of time and money and the damage could have been prevented if an impact was scientifically measured in the pilot phase.

Measuring value ultimately maximizes the cost-benefit ratio and at the same time minimizes risk. A question that arises is to what extent one should minimize risk, as it can be seen as an essential part of innovation and the creative process. Ultimately, innovation is ‘the art of welcoming risk’ according to Kuczmarski (1996). The OECD (2006) proposes to measure impact during the pilot phase, especially for programs that are due to be substantially scaled up. This would allow for risk-taking up and until the pilot phase and would minimize the risk of implementing an ineffective application. Furthermore, to measure real impact, a counterfactual should be present to compensate for what would have happened anyway (Clark et al., 2004).

To measure impact, consider the Impact Value Chain (Clark et al., 2004). Start with a mission and measure mission-related impact before implementation. A product that illustrates how impact measurement is well and effectively used is the Malaria bed net. The mission related to the nets is to prevent Malaria infection. Looking at the impact value chain, the inputs are resources such as the funds, time and equipment, and the activity is distributing free bed nets.

The output is the number of nets distributed and finally, the outcome is measured as the bed net usage amongst beneficiaries. Agencies mainly stop measuring at this stage of the value chain: they have been collecting data about input and output levels for decades, but the mission-related impact is still unknown (Liket, 2009). In the example, the mission-related impact is measured as a change in infection rates linked to the bed net usage. The nets are proven effective before they were widely used by collecting detailed information from before and during distributions (Givewell, 2016) and so money is invested efficiently and ultimately the lives of people are actually improved. If you are not measuring social value, are you getting it? Not measuring the impact means fooling yourself that you are making a difference in the world and could even mean being counterproductive. Designers and organizations should take their social responsibility seriously. Money should be allocated efficiently and choices about the implementation of design should be made according to the outcomes of impact measurements. Strive to make the world a better place, create value and innovate by taking risks, but implement based on reasoning and scientific evidence.

 

Article by Celine Hoogendijk

For more information and references, please contact 1730516@brunel.ac.uk