New places, new tools: what’s up next for mVAM?

KOICA pic 2

We’ve just got back from Rwanda where we were holding a workshop on using mVAM to expand real-time food security and nutrition monitoring with Internally Displaced Persons (IDPs) and refugee populations. The project, which is made possible by the support of the Korean International Cooperation Agency (KOICA), will be implemented in ten countries in sub-Saharan Africa where WFP works.

What’s the project?

The KOICA project has two aims. First, it aims to empower information exchange with marginalized populations, specifically IDPs and Refugees. Secondly, it supports the collection of food security and nutrition data using the latest mobile and satellite technologies. This will happen in ten countries in Sub-Saharan Africa: the Central African Republic (CAR),The Democratic Republic of Congo (DRC), Kenya, Malawi, Niger, Nigeria, Rwanda, Somalia, South Sudan and Uganda.

How are we going to do this?

As you know, two-way communication systems are an important part of our work. As well as getting information that we can use to inform WFP programmes, we want to ensure that the line is open so that people in the communities we serve can contact us and access information that is useful to them. We’ve already been using Interactive Voice Response and live calls to share information with affected populations, and are now expanding our toolbox to include new technologies: Free Basics and a chatbot.

Remote data collection isn’t just done by mobile phones – VAM already uses other sources, such as  satellite imagery analysis – to understand the food security situation on the ground.  Under this project, we’ll also help countries incorporate similar analysis which will complement two-way communication systems to provide a fuller picture of the food security situation.

Finally, we’re going to harness our knowledge of Call Detail Records analysis: de-identified metadata collected via cell phone towers about the number of calls or messages people are sending and which towers they are using. We have already used this technique in Haiti to track displacement after Hurricane Matthew, and we’re really excited to transfer these ideas to another context to ensure we get up-to-date information on where affected communities are so we can better target food assistance in the right locations.

What happened at the workshop?

Representatives from all 10 country offices, three regional bureaus and staff from HQ came together to discuss the three main project components. During the workshop, the different country offices had the chance to learn more from members of the mVAM team about the specific tools they can harness and ensure their collected data is high quality, standardised and communicated effectively. However, the best part about bringing everyone together was that country teams could share their experiences about how they are already using mVAM tools. We heard from the Malawi country office about their Free Basics pilot, and Niger and Nigeria explained how they’re implementing IVR so affected communities can easily contact WFP, even after work hours. Sharing their different experiences and learning about how different tools have worked in each context not only gave everyone an overview of what mVAM is doing so far, it also helped everyone understand the implementation challenges and how to overcome them.

What’s next for the KOICA project?

We’re really excited for the next stage of the project. Each country office has now planned what tools they’re going to use to increase their communications with affected communities and how they will improve their existing data collection systems. It’s going to be great to see the impact these tools will have not only on WFP’s response, but also how they will empower the communities we’re serving. 

Our 5 mVAM Highs from 2016

collage

1. Awards for Remote Mobile Data Collection Work

At the Humanitarian Technology 2016 conference, our paper Knowing Just in Time Knowing Just in Time’ won Best Paper for Outstanding Impact. In the paper, we assessed mVAM’s contribution to decision-making by looking at use cases for mVAM in camps, conflict settings and vulnerable geographies. Check out our blog Tech for Humanity for more on it and our other conference paper  mVAM: a New Contribution to the Information Ecology of Humanitarian Work

To close the year, we had a nice surprise from Nominet Trust, the UK’s leading tech for good funder. We made their 100 most inspiring social innovations using digital technology to drive social change around the world.  

2. New Tech

In this day and age there’s a lot of buzz around data visualization. We’ve been honing our skills with Tableau. Check out the data visualizations we did for Yemen and Haiti.

We’re also in the era of Big Data. We partnered with Flowminder, experts in analyzing call detail records, to track displacement in Haiti after Hurricane Matthew.  Find out more in ‘After the storm: using big data to track displacement in Haiti

We’re also super excited about the chatbot we started developing for messaging apps and our roll out of Free Basics in Malawi which is allowing us to share the food prices we collect in mVAM surveys with people in Malawi With mVAM, our main focus has been reaching people on their simple feature phones. But we know that smartphone ownership is only going to increase. Contacting people through internet-enabled phones opens up loads of new forms of communication and data collection. is still reaching people on their -free basics

3. Expansion!

mVAM expanded to 16 new countries facing a wide set of challenges: conflict, El Nino drought, hurricanes, extremely remote geographies. We’ve been tracking and learning about what remote mobile data collection can add to food security monitoring systems and what its limits are in different contexts. For some of the highlights, check out our blogs on Afghanistan, Democratic Republic of Congo, Haiti, Nigeria, Papua New Guinea, and  El Nino in Southern Africa,

4. Dynamic Partnerships

To have a lasting impact, we need to work with governments. We are really proud of our partnership with CAID, the Cellule d’Analyses des Indicateurs du Développement  under the Prime Minister’s Office in the Democratic Republic of Congo. We collaborated on setting up a national market monitoring system- mKengela that they are now running. We’ve had intensive technical sessions with the CAID team in Rome and Kinshasa to work on solutions that will fit their data management and analysis needs. The CAID team even traveled to Johannesburg to share their remote mobile data experience with other African countries and help other governments use this technology.

We’re also working with Leiden University. Bouncing ideas off of their team at the Centre for Innovation helps us move forward on tricky challenges. We’re also collaborating with them to develop an online course where we’re going to share our methodologies and how to use remote technology to monitor food security. Check out Welcome to Vamistan for more.

We are in the field of tech. So we can’t do our job well without partnering with the private sector. It’s definitely a dynamic area, and also one where we at mVAM are learning what works best in melding our humanitarian goals with the exciting private tech potential out there. Check out our blog From the Rift Valley to Silicon Valley and our hackathon with Data Mission for more.

5. Learning- the neverending process

In addition to trying out new technology, we’ve been trying to answer some important questions about the live calls, SMS, and IVR surveys which make up the bulk of mVAM data collection.  We’re also doing mode experiments to understand how people answer differently based on which mode we use to contact them. Check out our first Mind the Mode article with more coming in 2017. In Kenya, we are looking into whether we can ask nutrition indicators through mVAM methods. A major challenge is reaching women through phone surveys so we organized a gender webinar with partners to learn from what they are doing- check out our key gender takeaways. These are key questions and they can’t be resolved overnight. But we’re making steady progress in understanding them, and we’re excited for what more we’ll find out in 2017.

Thanks to everyone who has supported our work this year and kept up with our blog!

DRC Data Challenge: Brainstorming Solutions with CAID

image_mkengela discussion

We just spent three days with the fabulous CAID team from the DRC government.  We’ve been working with CAID, DRC’s Center for the Analysis of Development Indicators, since January to establish mKengela, a national market monitoring system using mobile technologies. We contracted a professional call center in Kinshasa to call traders twice a month to ask about food prices. For more info check out our mKengela blog entry and our first market reports:

Bulletin_Information _marchés_RDCongo_mKengela_n°2_30May_2016_JBB_2_cropped

But CAID has even bigger ambitions for data collection and analysis in DRC. So the WFP Country office and our team here in Rome decided to organize a technical visit in Rome to collaborate further on data systems. Their coordinator, Grégoire Mwepu and two technical staff, Marc and Bertrand, came to explore solutions for data management and sharing. Sib, our VAM Officer on the ground, attended from our office in Kinshasa.

The result- great brainstorming! Everyone was throwing out ideas.

API and Data Management

image_bertrand discussionCAID is very interested in setting up their own API- we’ve set up an API to share our data. But the rule for an API is get your data management straight first. Once your data is properly structured, then you can throw on an API on top- think of it as the icing on the cake. So we had a lot of discussions on what would be the best solution, not only to manage mKengela market data but all the development data that CAID is collecting across 145 territories on agriculture, health, energy, etc. While CAID works on data management, they can connect to our API for any data we have on DRC.

Automated Processing

CAID has set up an impressive data collection system in 145 territories. But that’s a lot of data. So they were also interested in how to automate as much of the cleaning and analysis process as possible. We discussed what we are doing the Stats Engine we set up to automatically run data quality checks and calculate statistical tests.

Arif (Chief Economist- WFP) and Grégoire (Coordinator- CAID) examine visualizations of DRC data

Arif (Chief Economist- WFP) and Grégoire (Coordinator- CAID) examine visualizations of DRC data

Data Visualization

Like us, CAID is interested in anything that makes it easier for people to understand, especially decision-makers. We also discussed data visualization and showed them our work with Tableau (see our blog entry on experimenting with Data Visuals and Tableau).

mKengala Revisions

After two rounds of market data collection, it was also time to review mKengela. In our previous blog post on mKengala, we were excited that CAID had diligently collected so many local measurements. But it turns out that having so many local measurements of varying sizes is hard to manage. So we’d recommend starting with a longer list and then shortening it to the most commonly used units of measurement after a couple rounds. CAID’s representatives in the 145 territories are also collecting more trader phone numbers since some numbers they had were always turned off or no longer working.

These three days of brainstorming technical solutions were very exciting for us. With mVAM, our priority is to partner with national governments and see how mobile technologies can help their humanitarian and development response.

A huge thanks to Grégoire, Marc, and Bertrand for traveling all the way to Rome. Stay tuned as CAID puts some of the ideas discussed into action and we hopefully have a follow up technical session in Kinshasa later this year.

CAID and mVAM Technical Visit Participants. Thanks CAID for coming all this way!

CAID and mVAM Technical Visit Participants. Thanks CAID for coming all this way!

How ICTs Help WFP Increase Food Security in DRC

ICTworks Blog

By Arjun Puri – ICTworks – 9 May 2016

ICTworks_Image

Upheaval, uncertainty and instability are common in conflict stricken countries like the Democratic Republic of the Congo (DRC). The World Food Programme (WFP) reports more than 60 percent of the population lives below the poverty line in DRC, with national food scarcity and chronic malnutrition running rampant in these settings. In this type of situation, the humanitarian community steps in to help fight hunger and provide access to food for vulnerable civilian populations.

Better Data Collection

WFP’s mobile Vulnerability Analysis and Mapping (mVAM) team is leveraging open source mobile and voice technologies to improve timely data collection regarding food supply and access in collaboration with the population which is facing food scarcity themselves. One of the ICT tools being deployed by the WFP mVAM team is Verboice, a low-cost, open source interactive voice response system (IVR) technology created by InSTEDD…

Read full article here.

Filling in the blanks in DRC: Introducing mKengela

Aerial view of the Mugote IDP camp View from a UN flight in DRC. WFP/Leonora Baumann

Data collection in the Democratic Republic of Congo can be daunting. We lack data for much of a country that is 2/3 the size of Western Europe. See the map below with the big grey space- that’s an area where we have no data and it’s probably about the size of France. The last census was in 1984 and a proposal for the next census has landed in the middle of a serious political controversy.

Map of DRC- the grey area has no data.

Map of DRC- the grey area has no data.

But things are moving- especially on the mobile network side. Network coverage is expanding- a private sector source told us that his company is building 1,000 cell phone towers per year. There are four major cell phone companies operating in DRC, and all territories now have at least some coverage. Mobile penetration remains low but is expanding- currently around 35% of people are estimated to have phones.  We started our first mVAM pilot in DRC- in Goma collecting data from Mugunga III IDP camp (blog, video) and market prices in Goma (blog).

CAID- A promising government collaboration

But we thought, DRC is also an exciting case for national mobile data collection. It turns out a new, young, dynamic government team under the Prime Minister’s office thinks so too.  The Cellule d’Analyses des Indicateurs du Développement (CAID), or Center for the Analysis of Development Indicators has the atmosphere of a fast-paced Congolese start up. And innovation is necessary if CAID is going to fulfill its crucial mission: addressing DRC’s vast data collection challenges and ensuring the various data that is collected is available openly on a central website.

For the CAID, the first priority is filling in that grey emptiness. They set up an impressive data collection system, recruiting agents in each of the 145 territories to collect data. And they were already using mobile technology!  As their agents went out to collect data, they often used an SMS system to send back results.

CAID-WFP brainstorming a mobile monitoring system for DRC. Top: Bertrand and Didier (CAID), Sib (WFP). Bottom: Jean-Martin (WFP) and Max (CAID)

CAID-WFP brainstorming a mobile monitoring system for DRC. Top: Bertrand and Didier (CAID), Sib (WFP). Bottom: Jean-Martin (WFP) and Max (CAID)

So for us at WFP, working with CAID was a perfect match. The WFP Country Office in DRC has made partnering with the government to enhance technical capacity a priority. So we at WFP were thrilled to meet with CAID early this year and brainstorm ideas of how remote mobile data collection could support their efforts to put in place a data collection system covering the whole country. Their agents have lots of work as they try to collect data for all the different development sectors for their territories- and territories are huge.

Designing mKengela- a national mobile monitoring system

As we brainstormed, the idea for a national mobile monitoring system, mKengela or mobile “alert” in Lingala,  started to take shape. To lighten their workload and ensure a steady flow of data, CAID liked the proposal to monitor market prices and gather household information by phone calls.  At WFP, we thought great- we can monitor 145 markets or one per territory throughout the country. But CAID sent their agents out to collect the phone numbers from 1350 traders in 435 markets! CAID is ambitious and ready to try new things, including 3 markets per territory to get a more meaningful national price monitoring system.

If you read our blog on collecting market prices in Goma, you’ll remember that units of measurement can make or break your price data collection. If you don’t ask people in local units, you won’t get good information. But you also then have the difficult task of converting various local units to the metric system. Well CAID solved this problem. Max, the statistician at CAID, looked through all the price data that CAID had. Their agents had collected prices in local units from traders and then weighed the amount in kg or liters. So Max had the list of almost all local units used and the corresponding metric conversion. When there was some variation in the metric weight for a local unit, Max took the mean weight. Voilà- about the best solution to the local units of measurement challenge that you could ask for.

mKengela will also have a household survey component. We are planning on collecting information on food security indicators- household consumption and coping- from 5,200 households in 26 provinces. For the moment, the network and cost constraints make a household survey at the territory level a little out of reach. So we plan to pilot the survey at the province level first.

Strong Partners, Strong System

This of course would not be possible if we didn’t have tech-oriented donors and an excellent call center to work with. We received funding from USAID and the Belgian Development Cooperation for scaling up mobile data collection in DRC.  In terms of a call center partner, we have struggled to find a good private sector partner in some countries. But in Kinshasa, Congo Call Center, created by two entrepreneurial women, is extremely professional and experienced. Their operators asked us thoughtful questions during the training, helping us further improve the questionnaires.

So momentum is growing around mKengela in DRC. We’re excited about this new partnership with CAID and all the opportunities that come from setting up a national monitoring system.  We’re planning to keep partnering with CAID- next priority, working together to improve data management.

Operators at Congo Call Center in Kinshasa call traders throughout the country.
Operators at Congo Call Center in Kinshasa call traders throughout the country.

Mobile Tech for Mobile IDPs in DRC

WFP food distribution in Mugunga camp

IDPs in Mugunga. Photo: WHO/Christopher Black

We’ve been writing a lot about how mobile technologies give us new opportunities to track food security. As WFP, we provide food assistance to many refugee and IDP camps. But right now, our knowledge often stops at the camp border.  What happens to refugees or IDPs when they leave the camp? And importantly for WFP, what happens to their food security situation? Mobile surveys could provide a key to this mystery.

Mobile Surveys and IDP Flows

jean baptiste_filtered

Jean-Baptiste Pasquier

Jean-Baptiste, our brilliant young colleague, did his Master’s Thesis on precisely these questions. He looked at almost two years of data, collected by mVAM since December 2013 in Mugunga III camp- 10km from Goma, DRC. Approximately 4,664 people live in Mugunga III, and since many people didn’t have phones, we distributed phones to 340 randomly selected households so they could participate in a phone survey.

Every month, our WFP operators, Mireille and Jean-Marie, have been diligently calling these same households. They’ve been asking households about their food consumption and any coping strategies that they’ve had to resort to if they were short of food. These questions let us calculate two key food security indicators- a household’s food consumption score (FCS) and reduced coping strategies index (rCSI). It’s also gotten Jean-Baptiste some pretty good data to play with. (For more on our work in DRC, see our blogs on our DRC launch, our market monitoring, and our 2-way communication system with camp residents).

Mireille and Jean Marie_cropped

Mireille and Jean-Marie review a call script. Photo: WFP/Marie Enlund

In Mugunga III, like most IDP camps, the population is always in flux and hard to track. People come and go without officially notifying the camp administration. In IDP speak, a “returnee” is someone who has left the camp (and in theory “returned” home though in practice the person might just have gone to live somewhere else). In March 2015, we started asking people about whether they were “returnees” and if so, where they had gone.

By using our mVAM data, Jean-Baptiste was able to pick up on changes in camp population not picked up by official figures and track where people went.  Most returnees reported staying in areas nearby to camp. Few were returning home to Masisi where over half of the IDPs in Mugunga III were from but where there still was conflict.

 IDP Flows and Food Security

We don’t just want to know where returnees go; we want to know how they are doing. However, usually, once IDPs leave the camp, they fall off our radar screen and we have no more information. But with mVAM surveys, returnees continued to respond to our calls asking about their household food security situation. Jean-Baptiste decided to see whether there were any difference in the food security situation between returnees and IDPs who remained in the camp

Sure enough, there was a difference. Returnees had better food consumption on average than IDPs who were in the camp.

But you might be wondering whether returnees were doing better even before they left the camp. Jean-Baptiste found that yes- on average, returnee food consumption climbed in the months before departing. It also improved more over time than IDPs who stayed in the camp; maybe their situation was improving so much that it allowed them to leave the camp.

graph for blog

Then, Jean-Baptiste went a step further. Maybe returnee households were just plain old different than IDPs who stayed in the camp. But he found that even by controlling for differences (for stat geeks- using a fixed effects model), leaving the camp had an estimated food consumption score increase of 7.64, which would be the equivalent of a 27% increase in the average food consumption score of an IDP currently in the camp.

Needless to say, all these findings could have a lot of implications for our programmes. Our office in DRC is looking into it.

Also, if we’ve peaked your interest, read Jean-Baptiste’s excellent thesis here.

Our 5 hacks for mobile surveys for 2015

WomanPhoneGoma

An mVAM respondent in Mugunga III camp, DRC.

  1. Gender matters. Design and run your survey in a way that promotes women’s participation. With mobile surveys, it’s hard to get as many women to respond as men. Make sure you’re calling at the right time and that you provide incentives. We also recommend having women operators. For more of our thinking on gender in mobile surveys, check out our blog entry on gender issues in West Africa.
  1. Validate mobile data against face-to-face data. Your mobile survey results may differ significantly. In many contexts, cell phone penetration has not reached the most vulnerable groups. In DRC, we had to provide phones to Internally Displaced Persons (IDPs) and access to electricity- to learn more check out our video and our blog entry. But it’s not always possible to distribute phones so it’s important to check your results against other data sources. Also, people get tired of answering their phones all the time so attrition and low response rates will affect your results.
  1. Mind the mode!  Your results will differ according to whether the survey is done through SMS, IVR, or live calls by an operator. Live calls have the highest response rates, but you have to be ready to pay. For simpler data, we have found that SMS is effective and cheap. Just remember- the context matters. SMS is working well with nationwide surveys, even in countries where literacy rates are not that high- check out our recent results in Malawi. However, SMS can be a problem in communities where literacy rates are very low or familiarity with technology is low as we found in DRC IDP camps. For Interactive Voice Response (IVR) that use voice-recorded questions, the jury is still out on its usefulness as a survey tool.  IVR didn’t work as well as SMS in Liberia, Sierra Leone, and Guinea during the Ebola crisis (HPN June 2015). But IVR has potential as a communication tool to push out information to people. Check out our entry on our two-way communication system where we use IVR to send distribution and market price information to IDPs in DRC.
  1. Keep the survey user friendly and brief. Always keep your survey short and simple. Stay below 10 minutes for voice calls, or people will hang up. If you are texting people, we don’t recommend much longer than 10 questions. Go back to the drawing board if respondents have trouble with some of your questions. With mobile surveys, you don’t have the luxury of explaining everything as with in person interviews. It might take a few rounds to get it right. When we want food prices, we’ve found we need to tweak food items and units of measurement in Kenya and DRC to best capture what people buy in local markets. Again, short and sweet should be the mobile survey mantra.
  1. Upgrade your information management systems. There is nothing as frustrating as collecting a lot of great data – without being able to manage it all! Standardize, standardize, standardize! Standardize questions, answer choices, variable names, and encoding throughout questionnaires. Automate data processing wherever possible. Also, you’ll be collecting phone numbers. This is sensitive information so make sure you have the correct confidentiality measures in place. Check out our Do’s and Don’ts of Phone Number Collection and Storage and our script for anonymizing phone numbers. Finally, share your data so others can use it! We’re posting our data in an online databank.

 

 

Will IVR work for food security surveys in a Somalia IDP camp?

As the mVAM pilot project enters its final quarter, the team is focusing on finalizing all planned activities, while documenting learning that will allow us to scale up with a strong evidence base. This month’s highlights include some hands-on work with the team in Somalia, and the launch of a comprehensive review of our activities.

The Somalia IVR coming along
A key question we have is whether interactive voice response (IVR) surveys are user friendly enough to be used in Somalia with the vulnerable groups that WFP works with. The major issue to resolve was ensuring the IVR system Verboice in our Galkayo field office was fully operational. Although we had been able to place some IVR calls, the system required dedicated attention to be fully operational In mid-January, Marie and Lucia headed to Galkayo to meet with the team for a troubleshooting mission.
Thanks to late night remote support from Gustavo at INSTEDD, bugs were ironed out, and we were soon able to get our first complete IVR surveys using a Somali language questionnaire. The team in Galkayo was trained on how to place the calls and will be following a plan to scale up IVR calls in February. Meanwhile, we will continue collecting food security data through calls placed by our operators, a modality that has worked well to date.

Featured image

Making the IVR operational: training underway

During the visit, a key discussion took place regarding appropriate incentive rates. In both DR Congo and Somalia, respondents receive a token of appreciation from WFP in order to promote participation in surveys. The amount we provide –USD 0.50 per call – is equivalent to 5 minutes of airtime. While our respondents in DR Congo seem thrilled to receive this amount of airtime, the question of increasing the incentive has come up in Somalia, where it is perceived as too small.

There seem to be three schools of thought in the team. Some believe the incentive should increase in Somalia. Others think that increasing call attempts and better sensitizing respondents should be sufficient to ensure good response to our surveys. Others still question the principle of providing an incentive to people who might already receive food assistance from WFP.

In coming months, we will be making sure respondents are called more often and that the messages they receive tell them about the importance of their participation. We would then consider working with a larger incentive in the future should response rates not improve.

Launching the mVAM review

A critical milestone of the project is capturing and sharing learning. In order to proceed with scale-up strategically and responsibly, the review of the mVAM pilot in Somalia and DRC is now ongoing. Professor Nathan Morrow, who teaches at Tulane University’s Payson Center, is leading the review. Nathan has written extensively about technology in the humanitarian world, including a review of Ushaidi’s contribution to the 2010 earthquake response in Haiti.

In January, Nathan traveled to Goma, DRC, to meet with WFP staff, key stakeholders, and beneficiaries residing in the Mugunga 3 IDP camp to hear from them how the pilot was going, document their questions and concerns. He will also be chatting with staff in Somalia and the three-EVD affected countries to learn how they view the project.

The review will include documenting the demonstrated potential of mVAM at a larger level; noting areas of improvement that can ameliorate our technology; and explore how mVAM’s technology fits within the larger humanitarian sector’s work. Results will be available in the spring.

Can we use SMS for food security surveys in a Congolese IDP camp?

Blog entry originally posted in December 2014 on the Humanitarian Innovation Fund website.


Featured image

Response rates to voice calls, DR Congo Source: WFP

Almost one year into data collection, we are now fairly confident that live voice calls, placed by operators, are a good way to stay in touch with people in the extremely vulnerable communities we work with.  Since January 2014, we have been able to conduct monthly rounds of phone surveys typically reaching between half and two-thirds of selected respondents, while collecting data of good quality. However, it’s not yet clear if either IVR or SMS offer the same advantages in our pilot contexts.

SMS: cool tool, wrong setting?

This month, we attempted to understand whether SMS surveys would work in an IDP camp. Using SMS is attractive, because it is low-cost and easy to automate using free software.  While we have had good results with SMS(link is external)when running simple national or province-level food security surveys, we have yet to evaluate the tool’s suitability in a high-vulnerability refugee camp setting.  In November, two Rome-based mVAM team members, Marie and Lucia, travelled to Goma to attempt to do just that. They helped the team in Goma organize a simple food security survey involving face-to-face interviews, live voice calls and SMS. The data collected from this exercise will allow us to understand the strengths and weaknesses of these different survey tools.

Featured image

Residents in a Congolese refugee camp responding to text messages

In order to run the SMS survey, we used Pollit(link is external) a free, open-source tool. It is easily accessible with an internet connection and requires only minimal hardware to function—a computer, a mobile phone and an internet connection. The tool was developed by In(link is external)STEDD, the same company that developed Verboice(link is external), the programme we are using for IVR calls. In the future, Pollit may allow us to periodically run short SMS surveys in-house, bringing a lot of flexibility to our field teams.  During the Goma test, Pollit proved to be a simple and flexible tool. It was easy to set up and worked smoothly during the six days of data collection.

However, response rates to SMS surveys turned out to be low, particularly compared to voice calls and face-to-face surveys. Our enumerators reported that people in the camp are not used to using the SMS function on their phones. They typically communicate using voice calls, due to low literacy and habit. In some cases, the phones people owned were broken or had dirty screens, making it difficult to read and reply to the messages we were sending.  These issues, however, do not prevent us from using voice calls, which seem to be the preferred modality amongst respondents in DR Congo. This seems to suggest that we should stick to live calls for Mugunga 3 camp, and use SMS questionnaires in other settings. We are now analyzing the data we collected in Goma in order to answer other questions we have, which includes comparing data quality for the different survey modes. We’ll be sure to share those insights later.

Getting to know Goma

Blog entry originally posted in November 2013 on the Humanitarian Innovation Fund website.


In this month’s entry we explore the results of the face-to-face survey in the eastern Democratic Republic of Congo, and provide an update on the process of getting our call centres set up.

Earlier this month, our colleagues in the Democratic Republic of Congo finished the face-to-face assessment that serves as the mVAM baseline. These face-to-face assessments allow us to understand the profiles of our respondents, and request permission to call prospective survey respondents in the future.  The assessment was carried out in a camp hosting internally displaced persons near Goma. A total of 333 households were visited. The methodology followed standard WFP guidelines for camp settings.

In addition to the core set of questions on household demographics, food consumption, and coping strategies, the respondents were asked if they would like to participate in the monthly mVAM voice surveys. Some 90% (300 households), expressed their willingness to receive phone calls from WFP.  This is the same proportion of households that agreed to participate in central Somalia, the other location where this survey method will be piloted.

According to the GSMA’s report on mobile telephony, unique mobile phone penetration is below 20% in the Democratic Republic of Congo – well below the 30% average for Sub Saharan Africa, and below Somalia.

The findings of the face-to-face survey show that only 24% of respondents in the camp near Goma own a mobile phone. Another 33% reported not having a phone, but that another member of the household owned one. These figures explain why basic mobile phones should be distributed to all who had signed up for the mVAM surveys in Goma. Note that this is different to the approach in Somalia, where high mobile phone ownership rates make phone distribution unnecessary. During the monthly data collection rounds, it will be interesting to compare response rates under these two scenarios; how much of an issue does using WFP-provided phones become (re-charging, lost/stolen phones etc.).  Further insights to the results from the face-to-face survey in the Democratic Republic of Congo follow below.

Featured image

‘Can we call you and what’s your phone number?’

Household characteristics

Households participating in the mVAM surveys have an equal number of female and male headed households, 50.3% and 49.7% respectively, with an average of 5.5 members in the household. The great majority (89%) of mVAM respondents have been displaced and in the camp since 2009, with 41% arriving in 2012 when conflict in North Kivu increased. Almost all (94%) of respondents said they had received food assistance during the past month.

Coping behaviour

As in Somalia, we are curious to know whether the statistics we will collect through voice calls reflect the situation of the population at large. One of the key indicators we will collect through our calls is the reduced coping strategies index (rCSI), a quick and simple indicator that reveals how households manage or cope with shortfalls in food consumption.

The mean rCSI for the households that volunteered to participate in mVAM was 24.57, compared to 25.42 for the general population. A difference in means test, shows that there is no statistically significant difference (p=0.49) between the two groups.  The rCSI result of the mVAM respondents is also reflective of the overall IDP population that was surveyed, thanks to the high percentage of households that agreed to participate. We realise that it will be a challenge to ensure that all of these households respond to our calls.

A mean rCSI of almost 25 is high, in keeping with high rCSI values commonly observed during previous surveys in the same area. A high CSI indicates that people are using a lot of coping strategies to deal with the lack of food or resources to purchase food. The most frequent strategies used were consuming less preferred or less expensive foods, reducing the number of meals in a day and eating smaller quantities of food.

Expenditures on food and debt

The respondents’ main income sources are daily labour (25%), petty trade (15%) and agriculture (14%). A huge amount, 83% on average, of monthly household expenditure is spent on food. The amount is even higher than with the case of our mVAM survey respondents in Somalia, who spent an average of 76% of their monthly expenditure on food. At the time of the survey, 65% of the respondent households were in debt and 85% of households had accumulated debt in the past 12 months. In three times out of four, the reason for getting into debt was to purchase food.

Household assets

Similar to the baseline in Somalia, this assessment showed that the prospective mVAM survey respondents generally owned very few assets, the most common ones being a hoe/axe/machete (42%), mobile phone (24%) and radio (14%).  Only very few respondents owned productive assets such as chicken (8%) or goats (3%). It’s acknowledged that the distribution of mobile phones in asset-poor settings raises challenges – this was discussed with the community and WFP partners at the camp and will be monitored during the project.

Updates on other developments

This month we also conducted three training sessions, training a total of 14 WFP staff in headquarters, the Democratic Republic of Congo and Somalia. The training involves showing users how to use Verboice and set up a SIP channel to do calls. After about an hour, colleagues in Rome and in the field were able to place phone calls through a SIP channel, using Verboice. This confirms our earlier insights about the user friendliness of the software.

The recruitment of our call centre operators is still ongoing in both countries; operators should be on board by January. In the meantime over the next few weeks, we will be drafting the operator’s manual. The document will describe how to manage the process of placing live and automated calls.  We’ve also just finished buying the modems and servers that we need to set up for the Interactive Voice Response system for both countries.