Photo credit: Kieran Doherty/Oxfam

How we’ve put into practice GDPR and conducting impact evaluations

Methodology, Real Geek

Photo credit: Kieran Doherty/Oxfam
Photo credit: Kieran Doherty/Oxfam

We have been thinking a lot about the right to privacy lately, in relation to the General Data Protection Regulation (GDPR) and our role in Oxfam as impact evaluation advisers. The incorporation of this right in the EU Charter of Fundamental Rights reflects a broad recognition that privacy is fundamental to dignity and freedom. We know that when individuals are not empowered and do not have choice regarding the way their data is used, the side-effects can include distress, loss of other rights, and loss of dignity and respect.

Under GDPR, can we still do an impact evaluation if we cannot share data across borders, including into and out of the EU? What if we need to use data our partners have collected? How about working with consultants on impact evaluations? Perhaps you have had similar questions. In our latest paper from the Going Digital Series, Privacy and data security under GDPR for quantitative impact evaluation, we share the protocols we have developed within the context of GDPR, as well as the debates we have faced along the way. 

We fundamentally believe in the right to privacy. On top of our legal and compliance obligations, Oxfam adopted a rights-based approach to collecting program data in 2015 with its Responsible Program Data Policy (RDP). The RDP sets out our commitment to upholding the rights and dignity of those whom data is about. We see GDPR as complementary to our work and our principles, but at the same time we have faced many practical dilemmas in our day-to-day activities.

For quantitative impact evaluations, Oxfam often conducts household or individual surveys. During these surveys, we use the personal information people share with us to understand the impact of our programmes. In the paper we review system-wide controls and privacy throughout the data lifecycle. Along the way, we discuss our biggest dilemmas and how we have sought to address them. We also provide examples of an informed consent protocol, device setup guidelines, a personal data processing agreement, and a pseudonymization and anonymization protocol.

One of the dilemmas we faced centres around informed consent and how that relates to transferring data internationally, which is restricted by GDPR. At a certain point it seemed that GDPR’s stance on international transfers meant that impact evaluations using personal data was no longer feasible unless all analysis was done in the country where the data was collected. We would not be able to transfer data into the EU and then back to that country. While we do this as much as we can, it’s not always possible. After much discussion, we came to the crux of the issue. Once we have the right legal protections in place, is it possible to be transparent enough for people to consent, in an informed way, to their personal information being shared with Oxfam in other countries?

So, we have added a point on this in our informed consent protocol, which also covers other key points like the purpose of the survey, what it involves including benefits and risks, how the information they share will be used, ethical considerations and contact details for making a complaint or withdrawing consent. However, this protocol is purely an example that needs to be carefully adapted for the context of each evaluation.

Another dilemma we had is about using lists of project participants to recruit survey respondents when these lists are maintained by our partners. This one has probably been the most challenging to understand and address. Ideally, the informed consent protocol of our partner mentioned sharing data with Oxfam. To be transparent, if that is not the case for any reason and the data is shared with us, we then have to notify the individuals about the fact that we have their data and what we will do with it.

Doing this would be extremely challenging in many cases, especially if we are randomly sampling respondents to survey from a much larger population. Again, after much discussion, we have arrived at a tiered approach where our first choice is to work with the partner to carry out the sampling ‘pseudonymously’. For example, if we use a list of identification numbers only, the partner can share only the contact information for those selected for an interview. Then, when inviting them to participate in the survey, we first explain how we got their contact details and check whether we can use it in relation to the survey or if we should delete it.

The two dilemmas summarised here are key illustrations of the challenges we have faced implementing GDPR to improve on the privacy protection practices we already had in place. The latest Going Digital paper goes into more detail on these challenges and others. We hope it will be a useful reference for others. At the same time, we recognize there are many other ways to approach privacy and GDPR. We would love to hear more about the dilemmas you have faced, the debates you have had and any solutions and protocols you are able to share.

Author
Jaynie Vonk

Jaynie Vonk

Jaynie conducts Effectiveness Reviews and other impact evaluations with Oxfam. She promotes evidence-based learning, accountability, and understanding of programme results, leading on the development of sustainable water and sanitation measurement tools.