Karama Ltaif, participant in the AMAL programme training in Tunisia on youth leadership; civil and political society; advocacy for women's rights; humans rights. Credit: Ellie Kealey / Oxfam

How to measure women’s empowerment so that it reflects the views and opinions of all women interviewed in a study

and

Gender, General, Real Geek 2 Comments

Ensuring all voices are heard in evaluation processes is challenging but essential. Natalie Naïri Quinn from Oxford University and Oxfam’s Simone Lombardini present their analysis from a discrete choice experiment conducted in Tunisia.

Karama Ltaif, participant in the AMAL programme training in Tunisia on youth leadership; civil and political society; advocacy for women's rights; humans rights. Credit: Ellie Kealey / Oxfam

Karama Ltaif, participant in the AMAL programme training in Tunisia on youth leadership; civil and political society; advocacy for women’s rights; humans rights. Credit: Ellie Kealey / Oxfam

Measuring women’s empowerment has become more and more important for assessing development projects aimed at supporting women. Different measurement tools are available; but while most of them are designed to be as inclusive as possible, none has yet been developed to reflect the views and opinions of all the women involved in a study.

Oxfam GB is concerned with understanding the extent to which its projects have contributed to women’s empowerment, and we have made significant investments in developing a measurement tool for use in evaluations of this hard to measure concept.

Building on the experience from the Women’s Empowerment in Agriculture Index (WEAI) we have developed a measurement tool based on a composite index. The measurement tool is founded on the recognition that empowerment is a multidimensional and context-specific concept. While the framework remains constant across evaluations, the characteristics and indicators that make up the index change from context to context, aiming to capture the characteristics of an ‘empowered woman’ in the socio-economic context of analysis.

Last year Oxfam published A ‘How To’ Guide to Measuring Women’s Empowerment: Sharing experience from Oxfam’s impact evaluations, and back in 2014 we published critical reflections on the measurement approach, in the Gender & Development journal.

Oxfam’s measurement approach recognises the importance of including all the relevant stakeholders – including partner organisations and project participants – in efforts to understand the characteristics of an ‘empowered woman’ in different socio-economic contexts. In practice, this has taken the form of conducting focus group discussions or in-depth interviews with some representatives of women involved in the project, to help to identify what empowerment means in the context of the analysis. However, there is the question of how representative of the larger group are the women who can engage in these discussions. Frequently, given time and budget constraints, these are not randomly selected women, but rather the most involved in project activities.

Piloting a discrete choice experiment in Tunisia

In order to give greater voice to all the women involved in the study, as part of an Effectiveness Review of the AMAL project in Tunisia, we implemented a discrete choice experiment (DCE) to assign implicit weights to the empowerment indicators which reflect the views and the perceptions of the women interviewed in the survey.

A DCE is a stated-choice method that consists of presenting hypothetical scenarios to respondents in order to reveal their preferences or perceptions, which we then used to determine the indicators’ weights for constructing an overall composite empowerment index.

 We used the responses to estimate implicit weights for the individual indicators, reflecting respondents’ revealed preferences and opinions on what an empowered woman looks like.
During a workshop with programme staff and partner organisations to design the impact evaluation, we identified 14 characteristics (and associated indicators) that describe an empowered woman in the context of the project. We then developed a questionnaire measuring those empowerment indicators, which was used with a representative sample of project participants and a group of comparison women. At the end of the questionnaire, respondents were shown seven pairs of hypothetical women with different characteristics (based on the 14 characteristics), and were asked to indicate which woman in each pair they considered to be more empowered.

We used the responses to estimate implicit weights for the individual indicators, reflecting respondents’ revealed preferences and opinions on what an empowered woman looks like. The table and figure below provide a summary of the indicators describing what an empowered woman would look like in Tunisia, and the relative weights assigned to each indicator using the DCE.

 

Weight

From DCE

Personal
Self-confidence and self-esteemShe feels that she has a number of good qualities11%
Ability to make decisions for herselfShe can personally take decisions regarding herself12%
Recognizing that violence is not acceptableShe considers it unacceptable for a man to beat his wife7%
Awareness that collective action is more effectiveShe believes that acting as a group is effective to solve issues19%
Knowledge and awareness of women’s rights (ability to recognize problems)She thinks that men and women should have the same rights10%
Relational
Ability to make decisions in the householdShe is able to make decisions within the household9%
Participation and ability to make decisions in the public sphereShe actively participates in civil society and associations1%
Participation and ability to influence or make decisions political sphereShe actively participates in political parties0%
Taking action to stop violenceIn cases of experience of violence, she is able to report it18%
Independent incomeShe has an independent source of income2%
Control over resources in her householdShe has control over assets and resources in her household4%
Environmental
Equality of opportunityShe lives in a community that ensures that women have equal political opportunities with men0%
Social normsShe lives in a society that allows her to be free2%
Legislative protection for women’s rightsShe lives in a society where women’s rights are enshrined in law6%

 

Results

The results suggest that according to the respondents, the most important indicator for describing an empowered woman in Tunisia is ‘being aware of the benefits in participating in collective actions’, accounting for 19 percent of the total index, followed by ‘being able to take action to stop violence’ (18 percent of the total index). ‘Self-confidence’ and ‘ability to make decision for herself’ are also considered important characteristics, accounting respectively for 11 and 12 percent. Finally, ‘knowledge and awareness of women’s rights’, which accounts for 10 percent of the index. These five indicators represent 70 percent of the total weights for all the 14 indicators identified.

 

Results of the discrete choice experiment

Results of the discrete choice experiment

 

When we employed the empowerment index using the weighting from the DCE for the impact evaluation, we found that the average weighted empowerment index is reduced for both the intervention and comparison groups, compared with the un-weighted empowerment index. However, in both cases, the difference between the intervention and comparison group is still statistically significant.

If you are interested in discussing more, please get in touch. We also recently presented a technical explanation of the methodology at the CSAE.

Download the Effectiveness Review
Author
Natalie Naïri Quinn

Natalie Naïri Quinn

Natalie is Senior Research Officer for the Oxford Poverty and Human Development Initiative as well as Domus Fellow and Tutor in Economics at Lady Margaret Hall. Her core research interest is the meaningful aggregation of information over people, over time and over dimensions to measure socioeconomic concepts including poverty, inequality and wellbeing. The broad objective of her theoretical work is characterisation of aggregate measures on the basis of clear intuitive ethical principles and empirical regularities. Her empirical work focusses on applications in low and middle income countries: including measuring women's empowerment in a way that respects subjects' perceptions in Tunisia, and examining the distributional impact of a programme of targeted cash transfers in Kenya.

Author
Simone Lombardini

Simone Lombardini

Simone is a global impact evaluation adviser at Oxfam GB. He provides specialist advice on tools, methods and process for undertaking rigorous impact evaluation on Oxfam's projects. He is currently working on a range of focused evaluations of a random sample of Oxfam's projects in order to capture and communicate our effectiveness as an organisation and promote effective learning.