It’s been a tough past year for Papua New Guinea (PNG). Since April 2015, El Nino has hit the country hard with both frost and drought. With damaged crops and dried up creeks, people are struggling with both water shortages and getting enough to eat. WFP supported the National Disaster Center (NDC) in Papua New Guinea by launching a mobile phone survey to track the deteriorating food security situation and identify hotspots. In January, we started calling households to collect indicators on their food security at both the household and community level.
Why mobile phone surveys in PNG?
PNG is an extremely diverse country. It is spread out over 600 islands and has more indigenous languages spoken than islands- the current estimate is 800 different languages. Also, the islands are mountainous, with peaks as high as 4,500 meters, and covered in dense, tropical rainforests. The largest island, New Guinea, houses the third largest remaining block of tropical forest in the world after the Amazon and Congo basins. This makes PNG one of the richest countries in the world in terms of biodiversity, ecosystems, landscapes, and indigenous cultures.
However, these remarkable characteristics also make traditional face-to-face food security assessments challenging to say the least. The rugged nature of the territory, coupled with poor transport networks, makes it difficult for people to move around. Many villages can only be reached by foot, by boat, or if you can afford it, by helicopter. To do a proper face-to-face survey, enumerators would need to travel for days or use helicopters – a time-consuming or very expensive task.
Learning about mobile phone culture in PNG
As we were doing our homework on mobile phone use in PNG, we learned that mobile penetration has increased substantially in the past ten years, expanding from 1.6% in 2006 to 35% in 2015. Yet, this is still a low penetration rate. So we decided to try something new and added community level questions to our household questionnaire. PNG seemed like the right place to test a community-oriented approach as 87% of the population in PNG is rural, mostly living in very small communities of a few hundred residents. Communities are also very tight knit; members are aware of what is going on in each other’s lives. With over 800 languages, we were wondering how we would actually communicate with everyone. We were reassured that even if 800 languages are spoken, the majority of the people are able to speak Tok Pisin.
To learn more about ‘mobile phone culture’ in Papua New Guinea, we contacted Dr. Amanda Watson, an expert in communication technologies in the country who has been working for several years on the Coffey-managed, Australian Government-funded Economic and Public Sector Program (see case study here). She explained that traditional methods of remote communication were public (striking drums, blowing into shells, and singing from mountaintops). Mobile phones introduced private remote communication for the first time in PNG. This new innovation far from society’s eyes raised some concerns in the population about mobile phones being used to foster illicit relationships like extra-marital affairs or organize criminal activities. Yet, the impact of mobile communication has generally been viewed positively, and Amanda herself already advocated for the use of mobile phones in collection of drought-related information.
Partnering with Digicel’s call center
Digicel has the best network coverage of the three network carriers in PNG, with coverage in areas where 94% of the population lives. So WFP contacted Digicel who put together a team of very motivated phone operators, coming from all over the country, and with previous experience in conducting phone campaigns for other organizations.
The team participated in a 3 day mVAM workshop which included training in food security issues, focus group discussions, and test calls. Training operators on the objectives and content of the questionnaire was crucial to minimize operator bias and increase the reliability of data, particularly because the questionnaire used in PNG is more complex than the typical mVAM questionnaire.
Guess what- we learned that people in PNG love to talk! Initial interviews lasted up to 25 minutes – which is both a long time to keep people on the phone and quite expensive! So our operators did a great job learning what to say to keep people on point and reduce the call length. By the end of the third week of phone calls, we were able to reach our target: 3,709 completed surveys from 233 Local Level Government (LLG) areas.
Stay tuned to learn about the results!
By Elena. L. Pasquini – Devex – 19 January 2015
Humanitarian organizations delivering food assistance in conflict-affected areas or regions plagued by natural disasters or outbreaks of epidemic diseases cannot do their job blindly. Practitioners operating on the ground need a steady flow of accurate, up-to-date information to remain effective — and safe.
In order to design, implement and monitor programs when access is limited and risky, critical questions must be answered to understand the cause of the crisis, ascertain the number of people affected, what type of assistance they require and where, and whether they have successfully received that assistance…
Read Full Article
Have you missed some of the mVAM blog posts this year? No worries, here we’ve gathered our top 5 blog posts of 2015.
Have a look!
- Crowdsourcing Food Prices in Kakuma, Kenya. How mVAM is using SMS to monitor food prices in a refugee camp in northwestern Kenya.
- Hello Operator. Some thoughts to share on quality control for phone surveys, as well as some insights from our call center in Kenya.
- Ladies – We can’t hear y’all! Addressing Gender Issues in Mobile Data Collection in West Africa. Challenges generating data and analysis on gender dynamics using mobile data collection tools and how we are addressing them.
- Iraq: learning under pressure. Lesson learned while deploying mVAM remote mobile data collection system in the Iraq emergency.
- Our 5 hacks for mobile surveys for 2015. Tips for successful mobile surveys.
- Gender matters. Design and run your survey in a way that promotes women’s participation. With mobile surveys, it’s hard to get as many women to respond as men. Make sure you’re calling at the right time and that you provide incentives. We also recommend having women operators. For more of our thinking on gender in mobile surveys, check out our blog entry on gender issues in West Africa.
- Validate mobile data against face-to-face data. Your mobile survey results may differ significantly. In many contexts, cell phone penetration has not reached the most vulnerable groups. In DRC, we had to provide phones to Internally Displaced Persons (IDPs) and access to electricity- to learn more check out our video and our blog entry. But it’s not always possible to distribute phones so it’s important to check your results against other data sources. Also, people get tired of answering their phones all the time so attrition and low response rates will affect your results.
- Mind the mode! Your results will differ according to whether the survey is done through SMS, IVR, or live calls by an operator. Live calls have the highest response rates, but you have to be ready to pay. For simpler data, we have found that SMS is effective and cheap. Just remember- the context matters. SMS is working well with nationwide surveys, even in countries where literacy rates are not that high- check out our recent results in Malawi. However, SMS can be a problem in communities where literacy rates are very low or familiarity with technology is low as we found in DRC IDP camps. For Interactive Voice Response (IVR) that use voice-recorded questions, the jury is still out on its usefulness as a survey tool. IVR didn’t work as well as SMS in Liberia, Sierra Leone, and Guinea during the Ebola crisis (HPN June 2015). But IVR has potential as a communication tool to push out information to people. Check out our entry on our two-way communication system where we use IVR to send distribution and market price information to IDPs in DRC.
- Keep the survey user friendly and brief. Always keep your survey short and simple. Stay below 10 minutes for voice calls, or people will hang up. If you are texting people, we don’t recommend much longer than 10 questions. Go back to the drawing board if respondents have trouble with some of your questions. With mobile surveys, you don’t have the luxury of explaining everything as with in person interviews. It might take a few rounds to get it right. When we want food prices, we’ve found we need to tweak food items and units of measurement in Kenya and DRC to best capture what people buy in local markets. Again, short and sweet should be the mobile survey mantra.
- Upgrade your information management systems. There is nothing as frustrating as collecting a lot of great data – without being able to manage it all! Standardize, standardize, standardize! Standardize questions, answer choices, variable names, and encoding throughout questionnaires. Automate data processing wherever possible. Also, you’ll be collecting phone numbers. This is sensitive information so make sure you have the correct confidentiality measures in place. Check out our Do’s and Don’ts of Phone Number Collection and Storage and our script for anonymizing phone numbers. Finally, share your data so others can use it! We’re posting our data in an online databank.
Guestblog from our colleagues at the WFP Regional Bureau for West Africa in Dakar.
Generating data and analysis on gender dynamics using mobile data collection tools is admittedly a challenge, but one which we know is worth the extra effort. To date, we’ve noticed two major challenges in capturing gender dynamics in our mobile surveys. Firstly, women are not participating in very high numbers. Secondly, we need to keep our questionnaires short to be effective, but this means that we cannot ask all the questions on gender issues that we would like.
Women make up less than 25% of mVAM survey respondents in West Africa
It’s been hard to get as many women to participate in mobile surveys as in face-to-face surveys. In Guinea, Sierra Leone, and Guinea, women make up between 16-24% of total mVAM respondents. However, in Liberia in a face-to-face Emergency Food Security Assessment conducted in May 2015 (EFSA), women made up 50% of the respondents. We are of course worried about bias in our mobile data from the underrepresentation of women and what it means for the accuracy of our results. To design the best interventions, we need to hear from women as well as men.
In West Africa, the mVAM team is planning on testing ways to increase female engagement in our surveys. We are not sure what will work best, but we want to explore using radio ads to target women, employing female operators to make female respondents more comfortable participating, and recruiting men to encourage their female household members to participate. We’ll let you know what works, and if you have ideas for us, leave them in the comments!
It’s not just who you ask but what you ask.
For mobile data collection to work well, we need to keep it brief. If we call or text people too many questions, they’ll get tired and hang up on us or stop texting back. With remote data collection, we also cannot ask very complicated questions. This means that we cannot ask everything we would want to better understand gender dynamics in the communities where we work.
Currently, we do however ask respondents about the sex of head of household. During the ebola epidemic, our mVAM data showed that female-headed households were generally more food insecure than male-headed households. However, we can sometimes face a double problem- if fewer women are answering our calls or texts, not only are we missing their perspective but we are also likely missing many female headed households.
Our team in Dakar, Senegal, is also looking at ways to redesign questions in West Africa that better represent women as well as men. For example, a current mVAM survey question to gauge employment reads: “Currently, how much are people paid per day for manual labor in your community?” It’s a simple measure that has been shown to be a good predictor of community food security trends in Sub-Saharan Africa. But it also disproportionately reflects men’s lives, as manual labor tends to be a male-dominated sector. So our colleagues in Dakar are thinking about tweaking existing questions or creating new ones that are good socioeconomic indicators for both women and men.
We also are looking at designing new surveys to better capture the realities of women, men, girls, and boys. In West Africa, we plan to conduct several country pilots in 2016 using mobile technologies to collect gender-sensitive information on market dynamics that can affect food security. For example, women traders are the primary suppliers of certain commodities; in some contexts, they are the sellers of palm oil, green leafy vegetables, and local rice. If an emergency occurs and suddenly women traders stop traveling to markets, there may be shortages of these products. Our team in West Africa is also looking at developing a Women’s Empowerment in Markets Index (WEMI).
These efforts are part of WFP’s larger effort to expand available information on gender roles and disparities in food markets, and apply appropriate gender analysis to inform market-based humanitarian interventions and beyond. Stay tuned for further blog entries on our progress in West Africa and also on gender and mVAM in other regions.
Crowdsourcing– it’s one of the buzzwords of the moment. It’s been used to raise money, to build tech solutions, for social activism. So, why not crowdsource food price information using the SMS tools we have developed for mVAM? We could text people, lots of people, in communities and ask them about food prices. It sounds better than the alternative of collecting data in person, which can mean traveling for hours to remote or insecure areas on bad roads. After all, food prices are a short and simple piece of information that should be easy to text.
Kenya, a dynamic, tech savvy country, seemed like a great place to test crowdsourcing food prices. We decided to start in August with Kakuma refugee camp in northwestern Kenya. Kakuma camp has been there since 1991, and WFP currently provides food assistance to 148,000 refugees or 33,000 households. 60% of households have a phone number, giving us a large pool of people to contact by SMS.
Crowdsourcing food prices- how it works
We are currently working with GeoPoll, a mobile survey company, to send out around 1600-1700 text messages to random numbers in Kakuma camp each week. Our target is to get 100 fully completed responses back.
Refugees receive an SMS asking them to text back to ‘opt in’ to take the survey. If they send us back ‘1’ for ‘yes,’ we text them questions about 7 food items. They can text us back the prices without being charged.
At the end of the survey, we also ask people to describe
their main concern about living in the refugee camps. We get really rich information through this – see our word cloud above. Often, refugees’ main concern is their family’s food security: “The food is not enough for us.”
We currently have around a 6% response rate. While this might seem low, it’s fair by industry standards for SMS surveys. For us, it’s sufficient for monitoring food prices that do not require an exhaustive sample. We have found that incentives really help participation. The respondents receive an airtime credit of 10 free texts as a thank you for completing the survey. The airtime incentive has been welcomed by the respondents – some are even calling our WFP Operator saying they didn’t get the survey that week!
Basically, practice makes perfect. We’ve found that by tweaking some questions to make them more relevant to the people you are asking, we can increase our data quality. For instance, we were asking refugees about the price of fresh milk, but it’s powdered milk that is usually available on the market. So we switched to powdered milk to get more meaningful data. We also started letting refugees select ‘don’t know’ and skip certain commodities if they don’t know the price. Before, they were having to make a wild guess if they didn’t know just to move onto the next question.
With SMS, we get price data cheaply and quickly, allowing us to understand how prices in Kakuma are changing week to week. It’s helping WFP as we deploy our mobile money program in the camp.
IT-VAM Collaboration: Meet Rosemary.
Behind every great program, there’s a great woman. For us in Kenya anyway, this tweaked maxim is true. Rosemary is our mVAM Project Manager in Nairobi. She’s been working since 2007 with WFP as an IT Specialist. mVAM really is at the nexus of IT technology and data analysis. So, when we rolled out mVAM in Kenya, IT and VAM, our food security data analysis unit, joined forces. Rosemary moved to sit with her colleagues in VAM and has been doing a great job.
Rosemary’s liking running mVAM on the ground:
“Working on mVAM is fantastic because it’s so new. I like the challenge. It has made me really appreciate IT’s role in WFP’s operations and the importance of our work to our beneficiaries. Our work in IT has a direct impact.”
Rosemary and her VAM colleagues’ spirit of cooperation has made mVAM an innovative, successful project thusfar in Kenya.
In Goma, WFP sends out automated SMS surveys to student key informants to collect food prices in the local markets. The information is then recorded into an Interactive Voice Response system that beneficiaries can call to listen to information on food prices, WFP distribution dates, and phone survey dates free of charge.
Generally, food price data collection is the domain of grizzled enumerators, who scour markets, pencil and clipboard in hand. The use of SMS technology helps us deal with price data collection in a way that is quick, cheap and fun. Here’s a flavor of what we have tried in Goma, DRC.
Youthsourcing in DR Congo
When we were setting up the 2 way communication system that involves sharing food prices with the community by cell phone, we faced the problem of finding cell-phone savvy people to help us out. Our local staff told us to consider working with youth.
Enter Alain, a statistics student, and Pauline, a community development student at the University of Goma. We were impressed by their CVs and their eagerness. So we asked them if they would be interested in working with mVAM. Note: we are not using their real names here because they are anonymously collecting prices.
Soon, the students were trained in SMS (the training was not long – it’s just like texting one of your friends!). Once a week, they go to Goma’s three biggest markets to check prices from different vendors. We text them an SMS price questionnaire, and they text us back prices of 15 commodities. They receive a modest amount of airtime credit as a token of appreciation. To make sure they are getting the right prices, they ask both vendors and people who just bought food how much they paid.
We also check the data quality, and bottom line, the students did a great job! At the beginning, we had a little problem with our fish price question. There are different types and sizes available on the Goma market…arias, lazeria, capitaine, anguille, tilapia…so we were getting back huge, confusing price ranges. When starting market price monitoring by SMS, it’s very important to use clear, local units of measurements. To standardize, we picked tilapia and asked the students to monitor the price for a 300g fish. This did the trick, and from then on, the fish price data has been reliable.
We’re pleased to be working with motivated and tech-savvy Congolese youth!
After getting back the prices by SMS every week, we average them and post the updated market prices in Swahili to our IVR (Interactive Voice Response) server in Goma. People can call in for free and navigate menus to find out food prices, listen to food distribution information or leave us a message. (For more details on how the 2 way communication system works, check out this previous blog entry.)
This fall, we are exploring the feasibility of expanding market prices monitoring to the Beni, a conflict area north of Goma. There are about 45,000 IDPs living with host families in Eringite, Mbau, and Mvivi in the Beni territory. If we can get youth or other people in the Beni community involved in market price SMS collection, hopefully the IDPs could go to the market with a better idea of food prices.