Data collection in Zambia. Photo: Bryan/Oxfam

Improving data quality with digital data collection

, and

ICT4D, Methodology, Real Geek 1 Comment

Emily Tomkys Valteri, Alexia Pretari and Simone Lombardini share practical tips to help improve quality in survey data collection, and introduce the latest case study in the ‘Going Digital’ series.

Data collection in Zambia. Photo: Bryan/Oxfam

Data collection in Zambia. Photo: Bryan/Oxfam

Sometimes survey data doesn’t add up: two household members are married to each other and yet have different marital statuses, consent statements have not been fully read, percentages of income add up to 131%, data from the same respondents across different surveys cannot be linked up – just a few common issues you may have encountered.

In our latest paper from the Going Digital Series, Improving data quality with digital data collection, we share some of the survey features we have employed to improve the quality and the accuracy of data collected.

Oxfam’s Responsible Program Data Policy states that respondents have the right to make an informed decision as to whether they give consent to participate in the survey or any data collection exercise. It has, at times, been evident that consent forms are not receiving enough attention from enumerators at the beginning of an interview. To tackle this issue, we piloted uploading a pre-recorded audio consent statement which enumerators played before the beginning of the survey, and set up speed violations which flagged, if and at what point in the file, the audio file was skipped.

Whilst working to improve questionnaire design, Oxfam’s Impact Evaluation Advisers have conducted A/B testing of survey components testing different designs of time modules and consent forms. We also set up a range of both hard checks and soft checks which flag potential errors in the survey, allowing for follow-up with specific enumerators and re-training if needed.

These are some of the examples described in the paper, alongside the corresponding coding in SurveyCTO, a mobile survey software, and Stata, a statistical analysis software. Adding these types of features has positive implications for the quality of the data, and therefore the validity of the study. It also helps reduce the amount of time and resources spent on data cleaning which can instead be invested in other activities, for example sharing results with communities (see Going Digital: Using and sharing real-time data during fieldwork). 

Technology alone is not the solution, quality checks ultimately rely on survey supervisors and managers having enough time

Technology alone is not the solution, quality checks ultimately rely on survey supervisors and managers having enough time, expertise and resources to follow-up with enumerators and respondents, and invest time in the re-training of enumerators if needed. We therefore suggest planning which checks are a priority for the study validity, and carefully identifying a line between adding value and creating an extra burden for the survey team and the respondents.

Quality checks rarely replace human interaction, and discrepancies in the data should be investigated as a team.

Using digital data collection technologies responsibly has the potential to increase data quality, even when resources are limited. Our hope in releasing this paper and making these simple examples available alongside SurveyCTO and Stata coding, is that our colleagues will build on it, leading ultimately to greater knowledge, learning and impact.

Download “Improving data quality with digital data collection”
Author
Emily Tomkys Valteri

Emily Tomkys Valteri

Emily is ICT in Programme Officer at Oxfam GB. she supports staff with applications of ICTs within their work. She specialises in mobile data collection, and coordinating Oxfam's work on how ICTs can be used within monitoring, evaluation, accountability and learning (MEAL). Emily also co-organised the MERL Tech conference in London in 2017.

Author
Alexia Pretari

Alexia Pretari

Alexia leads the measuring resilience work for Oxfam Great Britain, developing new tools and methods for assessing resilience capacities; she works primarily on impact evaluation design and implementation. Alexia is passionate about finding ways for impact evaluations to better reflect how power, and its different dimensions and intersections, play out and affect communities, households and individuals (including taking into account intra-household dynamics).

Author
Simone Lombardini

Simone Lombardini

Simone is a global impact evaluation adviser at Oxfam GB. He provides specialist advice on tools, methods and process for undertaking rigorous impact evaluation on Oxfam's projects. He is currently working on a range of focused evaluations of a random sample of Oxfam's projects in order to capture and communicate our effectiveness as an organisation and promote effective learning.