2017 Highlights

It’s been a busy year for us here at mVAM, but some things stood out among all the rest. Here, we take you through some of our highlights from 2017:

KOICA pic 2

Staff from several countries take part in an mVAM workshop in Kigali, August 2017

1: mVAM for everyone! Our free and open online course

After four years of testing, designing and deploying remote data collection projects, we partnered with Leiden University to develop an online course to share what we’ve learned so far. Our Remote Food Security Monitoring online course was launched in May, and aims to provide a clear understanding of what remote food security monitoring entails, when it is a useful tool, and how to implement a remote food security monitoring project. The course is free and self-paced, and open to anyone who is interested in setting up a remote data collection project.

mudasair

WFP/Jean-Martin Bauer

2: Expanding across Asia and the Pacific

During 2017, we kept growing, scaling up in the Asia/Pacific region. WFP’s Nepal and Sri Lanka country offices collaborated with their respective national government partners to launch  mobile-based food security monitoring systems. Nepal’s mNekSAP was the first to use an innovative dual-mode approach to collect data from a panel of households previously surveyed during a baseline assessment, combining remote mobile data collection with traditional face-to-face methods so as to not miss out on following up with those households without a phone. This means that the data gathered through mNEKSAP is not only representative (ensuring coverage of non-phone owners), but through re-interviewing the same individuals, it also provides us with a rare panel data set, which is optimum.      

Afghanistan, Myanmar and Papua New Guinea kept busy with ongoing mobile data collection. Afghanistan now uses mVAM to conduct several different types of surveys, from conflict rapid assessments, to market monitoring, to post-distribution monitoring. Most recently, they launched their first round of nutrition data collection for the Minimum Dietary Diversity for Women (MDD-W) indicator – stay tuned for results!

Meanwhile in PNG, their 4th nationwide survey introduced the Food Insecurity Experience Scale – an official SDG 2.1.2 indicator. Our hope is that we can use mVAM to help measure progress in this area.  Also in the region, we’ve been looking at ways to use the PRISM system to better visualize mVAM data and link it to other information sources. More on that in 2018!

WFP/Maria Muraskiewicz

WFP/Maria Muraskiewicz

3: Keeping up with remote nutrition data collection

We’re also expanding in terms of the type of data we use mVAM to collect. Following the success of last year’s remote nutrition data collection pilot in Kenya, we’ve moved on testing whether this is also feasible in Malawi and Niger, and which technologies we can use to collect the data.

From October 2016 to April 2017, we worked with GeoPoll in Malawi to develop a tool and methodology for collecting MDD-W data using SMS surveys. We conducted five rounds of surveys, during which we constantly adapted the indicator to make sure it was suitable for SMS surveys. We learned that the design of the questions was especially important – simple questions, a mix of open-ended and list-based questions, and the option to take the survey in the respondent’s preferred language proved particularly helpful.

In Niger, we tested the feasibility of using CATI to collect MDD-W data in IDP camps in the conflict-affected Diffa region. Through focus groups and in-depth interviews, it became evident that despite low phone ownership rates among women, most women do have access to phones through sharing with household members or neighbours. Men had little hesitation to women in their families being called when they were informed in advance, when female operators were used, and when the operators identified themselves as calling from WFP.. We’re now analysing the data we collected through both F2F and CATI, in order to understand potential mode effects and selection bias.

(For a full overview of our nutrition work, check out Episode 12 of VAM Talks!)

IMG_9645

4: Responsible data (collection, storage, sharing and distribution!)

Mobile data projects come with their own particular set of risks and challenges with regards to data privacy and protection. In a time when reports of data breaches seem to occur more and more frequently, what steps should we take to ensure that we aren’t accidentally putting the very people we are trying to assist at risk? Working with the International Data Responsibility Group (IDRG) and Leiden University’s Centre for Innovation, we developed a field book for Conducting Mobile Surveys Responsibly, which outlines the main risks of mobile data collection and provides guidelines for responsible data collection, storage, processing and distribution in complex humanitarian contexts. In December, we brought together experts on three different continents for a webinar on Responsible Mobile Data Collection, in which they discussed the challenges of remote data collection projects and shared best practices, tools, and tips for adhering to privacy and protection guidelines – from the field level to the WFP context and across the broader humanitarian and development sphere.  

Testing the chatbot in Nigeria

WFP/Seokjin Han

5: Communicating both ways: WFP speaks to …

As mobile technology continued to develop, we looked at ways to use new tools to allow the people we serve to start conversations with us about their own food security situations. In addition to getting information that we can use to improve the design of food assistance programmes, we want to ensure that the line is open so that people in the communities we serve can contact us and access information that is useful to them. In 2017, we continued the development of our two 2-way communications tools – a food security chatbot, and Free Basics, a platform which allows people to access certain sites on the internet at no data cost.

The start of the year saw us in New York where one of our partners, Nielsen, organized a hackathon to design a chatbot that could help collect information during a humanitarian response. Over the course of the year, we worked on developing use cases in different contexts – in Haiti , Nigeria and Kenya – and are now developing a chatbot builder with another partner of ours, InSTEDD. We look forward to deploying the bot in the new year.

Simultaneously, we expanded Free Basics after successfully piloting it in Malawi in November 2016; sites will soon go live in Rwanda, DRC and Niger. Back in Malawi, the original site, which started out as a free website to share weekly staple food prices, is now shifting its focus to address the needs of the more than 30,000 refugees and asylum seekers hosted in the country. The majority of the group lives in two camps where WFP provides food assistance in the form of monthly in-kind distributions and cash-based transfers. As their ability to move outside of the camps where they currently live is quite limited, having information not only about food prices in their immediate area but also food stocks is incredibly helpful.

Thank you to our partners and donors, without whose support none of this would be possible, and to you – our readers – for following along! See you in 2018!

Collecting Mobile Data Responsibly: webinar recording & takeaway messages

Thank you to everyone who tuned in to our live webinar on Responsible Mobile Data Collection last week! Five panelists on three continents discussed the challenges of remote data collection projects and shared best practices, tools, and tips for adhering to privacy and protection guidelines – from the field level to the WFP context and across the broader humanitarian and development sphere.

But don’t worry if you missed the event – a recording is available above and here, and we have summarised the key takeaways messages for a quicker read below. We’re also sharing at the end of this page additional resources and answers to the questions that we were not able to answer during the webinar due to time constraints.

Thank you very much to our five great panelists:

  • Asif Niazi and Raul Cumba, Vulnerability Analysis and Mapping Officers, WFP Iraq Country Office
  • Angie Lee, Food Security Analyst, WFP mVAM
  • Jos Berens, Data Policy Officer, UN Office for the Coordination of Humanitarian Affairs (UN OCHA)
  • Michela Bonsignorio, Global Advisor on Protection and Accountability to Affected Populations, WFP
  • And moderator Maribeth Black, Food Security Analyst, WFP mVAM

What you should know about Responsible Mobile Data Collection:

Greater risks and challenges: In a pre-digital era, there was more direct control over data. Now, as data collection for humanitarian action relies increasingly on digital tools and automated processes, there is a real need to raise awareness of the risks and harms that can occur at every stage of the humanitarian life cycle as well as  the methods for reducing these risks. Challenges include: data falling into the wrong hands, risks related to the IT infrastructure and outsourcing, bias and discrimination, and risks to the rights of data subjects.The example from Iraq highlights some of the challenges with remote data collection: In ISIL-controlled Mosul, people were afraid to answer calls, as it was illegal for the public to use mobile phones. When people responded, the length of the questionnaire and the short time available to ask the questions affected the data quality and response rate.

Step One: an even better understanding of the potential risks. Despite the recognition of the data protection risks and the development of ways to mitigate these, we, humanitarian actors, need to develop a better understanding of where these dangers lie. Sometimes we just don’t know how sensitive a dataset can be; in these cases, it is better to err on the side of caution. UN OCHA’s Centre for Humanitarian Data works on data policy with the aim of supporting a responsible growth of the use and impact of data in the humanitarian sector.

Step Two: mitigating the risks. Responsible mobile data collection has humanitarian principles at its core, such as “do no harm.” At every stage of the data collection and analysis lifecycle, these principles must be adhered to. In a first instance, understanding the local context and engaging in sensitisation campaigns and digital literacy trainings are important. The survey design should strive to minimise bias and ensure that no information other than the information that is really needed is collected. Just as important is the the transmission and storage of data using state-of-the-art security means. Finally, the publication of results ought to be anonymised and protection-sensitive, and there must be functioning and safe mechanisms for participants and others to report problems.In the Iraq case, the survey questionnaire was shortened, and the food security and market components were put together so as to minimize the time respondents had to use their phones.The WFP Data Responsibility Field Book offers both guidelines for the daily work of staff involved in mobile data collection and forms a basis for WFP’s internal dialogue on data responsibility.WFP’s corporate data privacy and security policies are contained within the Guide to Privacy and Personal Data Protection.

Several international collaborations already exist to address the issue of data privacy in humanitarian response. Examples are the Harvard Humanitarian Initiative; the framework developed by UNHCR and the Danish Refugee Council for data-sharing in practice, introducing a common language among humanitarian actors as well as a set of principles and shared processes; the ICRC’s work in collaboration with a Brussels privacy hub; and the International Data Responsibility Group, constituted of research institutes, think-tanks, and the international public sector.

 

Q & A

1. How do you ensure the authenticity of the interviewee, do you monitor the location of the mobile phones?

Firstly, the interviewer has the name and phone number of the respondent and will check with the person answering the phone whether they are talking to the same person.

Secondly, the service provider has series of towers and knows where a particular mobile phone is calling/answering from. That way, service providers can programme their computers to only call people from particular areas and we can ensure that interviewees are actually coming from a particular location.

Thirdly, we closely scrutinise the output of our data and analyses and make sure that, where the data does not seem to make sense, we investigate all possible sources of bias and error.

More generally, mVAM identifies respondents in three ways:

  • by asking respondents of traditional face-to-face surveys to agree to a follow-up phone survey;
  • by randomly calling people through mobile phone user rolls who have volunteered to take phone surveys. Telecom companies maintain a list of phone numbers of subscribers who agree to participate in surveys. Randomly selected mobile phone users in the areas of interest to WFP are then contacted, as per our sampling instructions.
  • by calling numbers generated through random digit dialing. Respondents are always given the choice to opt in to the survey or decline. Whether it’s WFP or third party providers that conduct phone surveys, the list of contacts (names, phone numbers, locations) are stored and managed in a safe and secure environment; only processed and aggregate data are shared for monitoring purposes – no individual’s statistics or geographic coordinates are released.

2. How do you ensure accuracy and validity of the information through phone call interviews?

In order to ensure the reliability of data, mVAM phone surveys are designed on the basis of representative sampling and using stratification techniques where possible. Results are reported by drawing inferences from large enough samples, complemented by thorough identification of key informants. The quality of data collected through phone surveys is also evaluated with reference to data from concurrent face-to-face surveys and/or secondary baseline data whenever feasible. For more information on representativity and how to account for bias, please refer to the methodology section of the mVAM blog: http://mvam.org/info/methodology/

3. What mechanisms are put in place to ensure the reliability of crowd-sourced data?

Data is triangulated with existing secondary sources of information including face-to-face assessments, field monitoring and key informant reports.

4. How can we integrate information security considerations during the early phases of a survey (especially during survey design and data collection)?   

The advice given in the Data Responsibility Field Book is to:

Understand and engage with local context – Engage with the community about major risks related to the proposed data collection. This can be done by interviewing members of the community and through a quick literature review on the mobile phone landscape (e.g. mobile phone ownership and usage rate, social and gender norms) in the country. Work with a community-based organization (CBO) or NGO in the community that can sensitize people about the activity. It is vital to engage with the community before collecting data. If there are protection risks, these need to be communicated. Explore opportunities with ‘self-organizing’ groups, whereby respondents set up management committees themselves.

Choose the right provider – When outsourcing phone surveys to commercial centers, ensure providers are scrutinized and vetted. Undertake due diligence on candidate companies and assess their compliance with best practices in terms of data security and privacy.

Conduct a Privacy Impact Assessment – as outlined above. It is important that, prior to any intervention, WFP conducts a Privacy Impact Assessment (PIA). The purpose of a Privacy Impact Assessment is to identify, evaluate and address the risks arising from the processing of personal data within an activity, project, programme or other initiative. It is important to note that such risks are not only related to IT aspects; they necessarily span across social, political, protection and legal considerations. A PIA framework is available to guide country offices in conducting a PIA. Please contact michela.bonsignorio@wfp.org for further assistance.

Data minimization: collect data on a need-to-know basis only – Collected data must be limited to the minimum necessary to achieve the objective in order to avoid unnecessary and potentially harmful intrusion into people’s private lives. In particular, information about people’s ethnicity, political opinions, religious beliefs or health or sexual orientation/choices should be strictly avoided unless absolutely necessary to the purpose of the survey. This information is not usually collected in WFP’s food security surveys.

Ensure your data collection has a specified purpose – Given the sensitivities and risks of collecting, storing and sharing data, personal and demographic data should never be collected indiscriminately. The purpose of data collection and processing must be clear and unambiguous and must be defined prior to data collection.

Review existing domestic legislation – Local legislation may pose challenges when collecting sensitive data. For example, applicable domestic laws may contain provisions that could force your local partners to disclose personal data in their possession to the government. Under such circumstances, you should only collect data if it is comfortable with the data being shared with the government.

Furthermore, it is advisable to conduct an assessment of the data landscape (including a check of whether the desired data is already (being) collected by other organizations, and whether it would be possible to gain access to, or use that data?).

5. With the rapid increase in datasets shared through the HDX platform, is there any mechanism established to check the data quality and authenticity of these datasets?

Organizations joining the HDX platform are vetted by our team. Every dataset uploaded to the platform is subjected to a quality assurance process, including a data-sensitivity check. The HDX team is not in a position to verify the authenticity of all datasets –this is the responsibility of the contributing organizations.

6. What about the level of dropout of respondents in mobile surveys?

Non-response and attrition rates vary across countries and can be attributable to different reasons (e.g. insecurity, displacement, survey fatigue). Since the inception of the project, mVAM has been following the best practice of providing a modest amount of airtime credits to survey respondents as an incentive for continued engagement. However, more than material incentives, we found that altruism is the biggest driver of response. The respondents must, however, feel that their identity will be protected and they have no need to worry about any negative repercussions. Additionally, we are exploring ways to leverage mobile technology to empower vulnerable communities by increasing their access to information on food prices, nutrition and feedback mechanisms.

7. Is the information from the service provider input into your organization’s database?

Yes. Raw data is sent to WFP in a CSV file at the end of each data collection round which is then stored in a dedicated database for cleaning and processing prior to analysis. Phone numbers or any personally identifiable information are anonymized to ensure sensitive data remains confidential.

8. Thanks to all panellists! Michela mentioned the best practices of having mechanisms for research participants or survey respondents to access findings and/or have a say in how their data is used. Do you have any examples of good mechanisms that have been established for that, especially in areas where access is an issue or with hard to reach populations?

Consultations with the affected population prior to designing an intervention is commonly considered a good practice. Where the population is accessible, it is recommended to hold focus groups and interviews with key informants to gather a representative picture of the reality on the ground. This can also be part of a PIA (see above). In the case of mVAM, such consultations are aimed at understanding issues like effective access to mobile phones and technology, digital literacy among the population, possible social and cultural obstacles affecting individuals’ free participation in surveys, perception issues, security threats. The mVAM team is particularly committed to engaging with the local population at all stages of its interventions. Feedback from the people is regularly gathered by field monitors and through ad hoc field missions.

It would be equally important to ensure that people participating in mVAM two-way communications can contact us at any time to request clarifications and/or express concerns about the utilization of their personal data. This might be built as an ad hoc option into the same mVAM communication system or may be achieved through the establishment of dedicated communication channels (e.g. hotline, email, etc.). Existing complaint and feedback mechanisms previously set up for programmatic purposes can also be used to that end. For example, in Lebanon efforts are underway to set up an interagency common call center as a mechanism to address queries specifically related to the assistance channeled through a Common Card. The call center will be also used as a receptor of concerns and requests related to personal data protection.

When the population resides in a hard-to-reach area, WFP should still conduct proxy consultations with humanitarian partners who have access to the population, if possible. A soft preliminary survey could be also launched via mobile phones, aimed at reaching people and understanding any possible risk affecting the roll out of the mVAM initiative. The survey itself may be sensitive and potentially dangerous, so it is recommended to avoid highly sensitive topics and utilize neutral, soft scripts. The assistance of a protection officers/advisors to that end is recommended.


Resources referenced during the webinar:

#data4food: Join us for the first mVAM Webinar!

webinar1-twitter

Join us on Monday for the first of our webinar series! We’ll be hosting a discussion with experts in the fields of mobile data collection, gender, and data analysis:

Addressing Gender-related Challenges in Remote Mobile Data Collection


12 December 2016
9am EST/2pm Dakar/3pm Rome/5pm Nairobi 

 

The discussion will explore some key issues that arise in remote mobile data collection, such as:

  1. Women’s Participation: How can we engage more women when conducting surveys via mobile phone? How can qualitative research help improve female participation rates?
  1. Analyzing Data for Zero Hunger: How do low female participation rates bias our data and thus our ability to design effective, evidence-based programmes? Given the barriers to women’s participation, what can we do right now to analyze our data in a way that better represents women’s experiences? Are we even asking the right questions?
  1. Mobile’s Potential: What are the untapped possibilities for using remote mobile data collection to collect information on both men’s and women’s experiences (e.g. protection issues like anonymous reporting of gender-based violence)? What are the limitations?

Panelists:

  • Joyce Luma, Country Director, WFP South Sudan (former head of WFP Trends and Analysis Service): Gender, mobile phone surveys, and data analysis for Zero Hunger
  • Sangita Vyas, Managing Director, r.i.c.e. (Research Institute for Compassionate Economics): Methodologies for capturing women’s experiences in mobile phone surveys in India
  • Micah Boyer, University of South Florida: Women, markets, and mobile phones in the Sahel
  • Kusum Hachhethu, WFP mVAM Team and Nutritionist, Qualitative research for using mVAM to reach rural Kenyan women

 

To join the webinar, connect via this link:

https://www.bigmarker.com/world-food-programme/Gender-Remote-Mobile-Data-Collection

 

Are you on Twitter? Participate in the discussion on Monday with the hashtag #data4food