Ensuring all voices are heard in evaluation processes is challenging but essential. Natalie Naïri Quinn from Oxford University and Oxfam’s Simone Lombardini present their analysis from a discrete choice experiment conducted in Tunisia.
Measuring women’s empowerment has become more and more important for assessing development projects aimed at supporting women. Different measurement tools are available; but while most of them are designed to be as inclusive as possible, none has yet been developed to reflect the views and opinions of all the women involved in a study.
Oxfam GB is concerned with understanding the extent to which its projects have contributed to women’s empowerment, and we have made significant investments in developing a measurement tool for use in evaluations of this hard to measure concept.
Building on the experience from the Women’s Empowerment in Agriculture Index (WEAI) we have developed a measurement tool based on a composite index. The measurement tool is founded on the recognition that empowerment is a multidimensional and context-specific concept. While the framework remains constant across evaluations, the characteristics and indicators that make up the index change from context to context, aiming to capture the characteristics of an ‘empowered woman’ in the socio-economic context of analysis.
Last year Oxfam published A ‘How To’ Guide to Measuring Women’s Empowerment: Sharing experience from Oxfam’s impact evaluations, and back in 2014 we published critical reflections on the measurement approach, in the Gender & Development journal.
Oxfam’s measurement approach recognises the importance of including all the relevant stakeholders – including partner organisations and project participants – in efforts to understand the characteristics of an ‘empowered woman’ in different socio-economic contexts. In practice, this has taken the form of conducting focus group discussions or in-depth interviews with some representatives of women involved in the project, to help to identify what empowerment means in the context of the analysis. However, there is the question of how representative of the larger group are the women who can engage in these discussions. Frequently, given time and budget constraints, these are not randomly selected women, but rather the most involved in project activities.
Piloting a discrete choice experiment in Tunisia
In order to give greater voice to all the women involved in the study, as part of an Effectiveness Review of the AMAL project in Tunisia, we implemented a discrete choice experiment (DCE) to assign implicit weights to the empowerment indicators which reflect the views and the perceptions of the women interviewed in the survey.
A DCE is a stated-choice method that consists of presenting hypothetical scenarios to respondents in order to reveal their preferences or perceptions, which we then used to determine the indicators’ weights for constructing an overall composite empowerment index.
We used the responses to estimate implicit weights for the individual indicators, reflecting respondents’ revealed preferences and opinions on what an empowered woman looks like.During a workshop with programme staff and partner organisations to design the impact evaluation, we identified 14 characteristics (and associated indicators) that describe an empowered woman in the context of the project. We then developed a questionnaire measuring those empowerment indicators, which was used with a representative sample of project participants and a group of comparison women. At the end of the questionnaire, respondents were shown seven pairs of hypothetical women with different characteristics (based on the 14 characteristics), and were asked to indicate which woman in each pair they considered to be more empowered.
We used the responses to estimate implicit weights for the individual indicators, reflecting respondents’ revealed preferences and opinions on what an empowered woman looks like. The table and figure below provide a summary of the indicators describing what an empowered woman would look like in Tunisia, and the relative weights assigned to each indicator using the DCE.
| Weight
From DCE |
||
| Personal | ||
| Self-confidence and self-esteem | She feels that she has a number of good qualities | 11% |
| Ability to make decisions for herself | She can personally take decisions regarding herself | 12% |
| Recognizing that violence is not acceptable | She considers it unacceptable for a man to beat his wife | 7% |
| Awareness that collective action is more effective | She believes that acting as a group is effective to solve issues | 19% |
| Knowledge and awareness of women’s rights (ability to recognize problems) | She thinks that men and women should have the same rights | 10% |
| Relational | ||
| Ability to make decisions in the household | She is able to make decisions within the household | 9% |
| Participation and ability to make decisions in the public sphere | She actively participates in civil society and associations | 1% |
| Participation and ability to influence or make decisions political sphere | She actively participates in political parties | 0% |
| Taking action to stop violence | In cases of experience of violence, she is able to report it | 18% |
| Independent income | She has an independent source of income | 2% |
| Control over resources in her household | She has control over assets and resources in her household | 4% |
| Environmental | ||
| Equality of opportunity | She lives in a community that ensures that women have equal political opportunities with men | 0% |
| Social norms | She lives in a society that allows her to be free | 2% |
| Legislative protection for women’s rights | She lives in a society where women’s rights are enshrined in law | 6% |
Results
The results suggest that according to the respondents, the most important indicator for describing an empowered woman in Tunisia is ‘being aware of the benefits in participating in collective actions’, accounting for 19 percent of the total index, followed by ‘being able to take action to stop violence’ (18 percent of the total index). ‘Self-confidence’ and ‘ability to make decision for herself’ are also considered important characteristics, accounting respectively for 11 and 12 percent. Finally, ‘knowledge and awareness of women’s rights’, which accounts for 10 percent of the index. These five indicators represent 70 percent of the total weights for all the 14 indicators identified.
When we employed the empowerment index using the weighting from the DCE for the impact evaluation, we found that the average weighted empowerment index is reduced for both the intervention and comparison groups, compared with the un-weighted empowerment index. However, in both cases, the difference between the intervention and comparison group is still statistically significant.
If you are interested in discussing more, please get in touch. We also recently presented a technical explanation of the methodology at the CSAE.
Download the Effectiveness ReviewAuthor
Author
