Podcast: How to measure resilience capacities – Experience from Oxfam’s impact evaluations

james Methodology, Real Geek

[buzzsprout episode=’2559154′ player=’true’]

Oxfam defines resilience as ‘the ability of women and men to realize their rights and improve their wellbeing despite shocks, stresses and uncertainty’ (The future is a choice, Jeans et al., 2016). So… Can resilience be “measured”?!

Well, we have been trying since 2012! Inspired by John Twigg’s (2009) characteristics approach, our measurement approach relies on identifying characteristics of resilience or resilience capacities. In other words, “hypothesising that there are particular characteristics of households (and even communities, organisations, governments, etc.) that affect how well they are able to cope with shocks, positively adapt to change and demand their rights” (Bushell and Hughes 2013, p.4). In our current understanding of the three capacities of resilience, we go beyond the ability to cope with shocks and positively adapt to change by integrating transformative change e.g. tackling the root causes of inequalities and vulnerabilities (see Absorb, Adapt, Transform: Resilience capacities, Jeans et al., 2017). And we understand these three capacities are intermediate outcomes towards realization of rights and improvement of well-being.

The characteristics approach has been a core aspect of the measurement framework we adopted in our impact evaluations, as part of the Effectiveness Reviews on resilience-building initiatives. So is the multi-dimensional approach, inspired by the Alkire-Foster method used by the Oxford Poverty and Human Development Initiative (OPHI). As well as this, the holistic approach, where some indicators used to characterize resilience will be related to the project logic, others won’t. All will be context-specific. This is in recognition of resilience being multi-dimensional and a broad concept (resilience-building initiatives usually focus on some aspects of it). Former advisers in impact evaluations Rob Fuller and Jonathan Lain reflected on these points here.

While shifting the measurement framework to focus on the three capacities of resilience (Fuller and Lain 2015 and 2017), we have also been more intentional at acknowledging the “multi-layered nature of vulnerability: shocks and stressors themselves occur at multiple scales and impact people at different levels” (Bene 2018, p.1) as well as ‘recognizing that women, men, girls and boys have differentiated vulnerabilities, i.e. that they are exposed differently to risks and uncertainties and are affected differently by them’ (Sotelo Reyes, 2017, p. 4). A few examples – mentioned in the podcast and integrating both of these points are:

  • Individual level index, for women and men, taking into account household level characteristics, varying the gender of the respondent in each household: Resilience in the West Bank: Impact evaluation of the ‘From emergency food security to durable livelihoods: building resilience in the Occupied Palestinian Territories’ project (forthcoming)

The individual level index taking into account household level characteristics is the approach we are currently developing further. This means including indicators at household and individual levels in the index construction, as both levels matter. For example, in a given context ownership of livestock at the household level may be considered as savings which can be sold to start new income-generating activities (as opposed to raising income), hence a characteristic of adaptive capacity. However, who has control over the decision to sell the livestock will also matter for the capacity of this individual to positively adapt to stresses and uncertainties, therefore it is also a characteristic of adaptive capacity.

The gender justice integration entailed two main changes at the design stage. First, thinking through sampling strategies carefully (discussed here and here soon), and second, changing what is taken into account in our measurement framework. For example, participating in community groups may provide a forum for voicing concerns and engaging in collective action for everyone. This may have some transformative potential. However, transformative capacity is also about social norms. People’s opinion on violence within the household is key to reducing vulnerability and long-term inequality.

So… This is some of the journey we have embarked on since 2012, to measure resilience (capacities) in our quantitative impact evaluations! The measurement approach is always evolving as we take a wider perspective of resilience, and as we grapple with its specificity in each context. If you want to hear more about this measurement question, but also about why resilience matters (2 minutes in), the developments and limitations of the approach developed in the Effectiveness Reviews (15 minutes in), how this applies (or not) to the MEAL framework for resilience more broadly (22 minutes in), how to strengthen accountability (20 minutes in) or how evaluation and measurement questions are political and about power (11 minutes in and 18 minutes in), tune into our podcast which covers all of these points in more detail.

Feel free to provide feedback and let us know what you think!

And here are the main references behind the discussion in the podcast:

Author

Alexia Pretari

Author

Ania Gaboune

Ania Gaboune is Accountability Lead at Oxfam GB

Photo credit: Aurélie Marrier d’Unienville/Oxfam
OXFAM See For Yourself donor Jacqui stands in Sandhikharka municipality, Arghakhanchi district, Nepal where OXFAM is building economic resilience of migrant families