Dust, sand, hospitality and technology

In her position as a Monitoring and Evaluation Officer in Niamey, Niger, Marisa Muraskiewicz thrives on the opportunity to make a positive impact on women’s lives through mobile technologies… and in the process she quickly discovered that there is a lot more than meets the eye in Niger!

marisa1

WFP/Marisa Muraskiewicz

My first time in Niger was in 2015 when I went on a three-week mission to the country as part of my work with the global Food Security Cluster based at WFP’s Headquarters in Rome. My first impressions of the country included a lot of dust and sand, but I equally took home with me fond memories of my colleagues and the dynamic work environment there.

About a year later, an opening for a Junior Professional Officer position came up in Niger and I jumped at the chance. For the past one and a half years I have been working on strengthening WFP’s mobile food security monitoring in the country and my work is varied and fulfilling.

We conduct bi-monthly phone surveys on household food consumption and coping strategies, and ask traders about the availability and prices of products on the markets. We have also completed three rounds of data collection on nutrition indicators. My job is to design the questionnaires, supervise data collection by the call center, analyse data, and produce reports in which we share our key findings and which enable WFP to respond efficiently and effectively.

But our goal is not only to collect data from communities that are experiencing food insecurity – we also want to share information that is useful for them. To that end, we’re currently setting up a two-way communication system using Interactive Voice Response (IVR) technology and a Free Basics website, through which we will be able to share, for instance, the market prices of various goods back to the communities.

Coordination between different WFP offices, government organisations, and companies is another key feature of my work here. For example, I’m currently working on creating a partnership between WFP, the government of Niger, and Airtel, a mobile network operator. Our goal is to be able to conduct analyses of call detail records (CDR) – key metadata from phone calls – which will help us to map the directionality and duration of migration events driven by conflict or drought. This will allow WFP to efficiently allocate resources and target assistance to the most vulnerable areas.

WFP/Marisa Muraskiewicz

WFP/Marisa Muraskiewicz

But it is speaking directly with people which has brought the most meaning to my work. Before we implement our mobile data collection technologies, it is important to have a comprehensive understanding of how affected populations can be reached and what their information needs are. To this end, I have traveled widely within the country, including to the volatile Diffa region. Visiting some of the remote villages where mVAM activities are in place always leaves a lasting impression.

Recently, I interviewed 20 women in the Diffa region in their houses – each woman warmly welcomed us into her home, usually a single room house without electricity or effective protection from rainfall. Back in Niamey, the impressions of these encounters motivate me to work towards providing a service that can make a positive contribution in these women’s lives.

There are of course plenty of challenges along the way, including low mobile phone ownership rates and limited access to the internet. But these technologies are becoming increasingly affordable in Niger and offer a huge potential for people. They offer access to important information, such as distribution dates, entitlements, nutrition, and food prices, which can empower people to make informed decisions on food purchases and consumption. Internet access also increases opportunities for employment as people are, for example, able to set up websites to sell their goods. It is fulfilling for me to be able to contribute to the benefits that people can reap through increased access to mobile phones and the internet.

“I experienced how much hunger can affect you”

Venkat Dheeravath, VAM Programme Policy Officer in Papua New Guinea, talks about implementing mVAM in a country where 850 languages are spoken, his journey with WFP, from South Sudan to Southeast Asia via Iraq, and a moment in the field that changed him: being stranded without food rations and with no means of communication

Venkat leading a food distribution in the remote Highlands region of Papua New Guinea, for a community affected by the El Niño-induced drought

Venkat leading a food distribution in the remote Highlands region of Papua New Guinea, for a community affected by the El Niño-induced drought

I grew up on a family farm in Andhra Pradesh, India. We grew vegetables for sale and I experienced the joys and hardships of farming while attending school. Little did I think then that I might one day be leading efforts to assess the food needs of vulnerable communities!

I studied Civil Engineering in Hyderabad City and worked in this field for several years before moving to GIS and Remote sensing, mapping croplands and completing my doctoral degree. Having also fulfilled my dream of working with NASA and the US Geological Survey, I asked myself “What next?”

I’ve long had a desire to serve humanity, and so my humanitarian journey with WFP started in South Sudan. As a GIS officer in Juba, I was meant to stay only for a short while – but in the end it turned out to be a five year stint! During that period, I assessed and mapped the entire South Sudan road network to assist the humanitarian community and the Government of South Sudan. There were countless times when while on mission, I had to sleep in the car on the middle of a remote road because our car got stuck in the mud – sometimes I had to survive only on muddy water!!

From East Africa, I moved to Iraq, where I helped set up and implement the country’s first mobile-based (mVAM) food security and market monitoring system. Then my journey took me, via Indonesia, to Papua New Guinea. Again, I was only supposed to stay for two weeks to support WFP’s response to the El Niño drought – but I’ve now been here for almost two years!

Since coming, I have successfully implemented mVAM in Papua New Guinea – even though many people did not believe it would work in a country where there are over 850 languages spoken. The context for WFP’s work here couldn’t be more challenging: data is scarce, the health, transport, and communication facilities are very basic, and accessibility and security problems make large regions of the country a very expensive place to operate any programmes. With 80% of the population living in very remote areas that are difficult to access, conducting food security monitoring through traditional face-to-face data collection methods would have been close to impossible. mVAM’s remote food security monitoring approach offered an alternative, viable option.

Digicel Call Center in Port Moresby, Papua New Guinea, from where the mVAM survey interviews are conducted

Digicel Call Center in Port Moresby, Papua New Guinea, from where the mVAM survey interviews are conducted

But we as the mVAM team also had to make sure that we would be able to effectively reach the people. Because of the large number of languages spoken in the country, we created our survey in two of the most common languages (English and Tok Pisin) and hired operators from different regions who could also speak various dialects. The second problem – no network coverage in some parts of the country – initially seemed hard to overcome, but, upon closer inspection, people in these regions are used to traveling across wards in order to catch a signal and communicate with relatives and traders pass by, so it was in fact possible to reach people who lived in areas not covered by a mobile signal. Our cooperation with the mobile network operator Digicel, which has solid network coverage and close to 100% of the market share, further helped us to reach a decent sample from the most drought-affected areas.

In February 2016 mVAM was first implemented in Papua New Guinea. In cooperation with the country’s National Disaster Centre, WFP launched a telephone-based survey to assess the effects of the El Niño-induced drought on food security and livelihoods. Our survey became the most comprehensive assessment of food security in the country. The findings then formed the basis for the design of WFP’s emergency response, helping us to provide food assistance to 268,107 of the country’s most vulnerable, food-insecure people.

For almost a week during the El Niño crisis, I travelled the ocean on a small dinghy with a life jacket to see the food insecurity situation on the remote islands of Milne Bay and subsequently led the distribution of food assistance with the Provincial Government. I am proud to say that I did not leave even one family behind on the outer islands and atolls, of which there are 110!

However, my dinghy trip was by no means my greatest adventure Papua New Guinea held in store for me. I recently travelled to a very remote area called Kira Station in Oro Province, located on a steep mountain in Waria Valley to validate the findings from our most recent mVAM survey, which classified the area as one of severe food insecurity. The only way to reach Kira Station is to use a private airline, which flies twice a week – provided there are enough passengers.

Our journey there went smoothly, but after two days, when we were supposed to fly back to Lae city, no plane came to pick us up. We were stranded with no means of communication. My satellite phone did not work because of technical issues, and there was no mobile signal in Kira Station. We had to walk through mountains for a day and a half before we were able to catch a very weak signal in one of the wards which borders Morobe Province, which allowed me to send a text message to the WFP regional office during a night of thick clouds and heavy rain. Every day, we looked up at the sky waiting for the plane only to see other planes flying over us.

On the mission to Kira Station to validate the mVAM survey findings

On the mission to Kira Station to validate the mVAM survey findings

We ran out of food rations. Most of the communities around us were consuming only one partial meal a day since the crops had failed. So I also ended up surviving on greens (Choko leaves usually grown in the wild bush), poisonous nuts (which have to be processed carefully before consumption and are only eaten when no other food is available), spring water, and a few coconuts. In the ten days I spent stranded without rations, I truly experienced how much hunger can affect you!

Finally, we decided to walk to reach the nearest airstrip in Garesa in the neighboring Morobe Province, assisted by four local community leaders from Kira Station. We hiked through mountains, rivers, valleys, swamps, and steep cliffs, for another day and a half, during which we survived on greens and river water. The mountain paths were very slippery, but happily the rivers were not flooded so we managed our journey without any incidents except for a few falls on slippery tracks. On arrival at the Garesa airstrip, we were lucky that a plane landed shortly afterwards and the pilot agreed to take us back to Port Moresby although we would only be able to pay for the fare on arrival.

We continue our commitment to ensure that vulnerable communities get the support that they need, currently we’re focusing on establishing a two-year food security surveillance and analysis programme in partnership with the National Disaster Centre, the Department of Agriculture and Livestock, and the National Statistics Office. A lot remains to be done in Papua New Guinea, but I strongly feel that technology can play a major role in connecting and ensuring the food security of remote vulnerable communities.

2017 Highlights

It’s been a busy year for us here at mVAM, but some things stood out among all the rest. Here, we take you through some of our highlights from 2017:

KOICA pic 2

Staff from several countries take part in an mVAM workshop in Kigali, August 2017

1: mVAM for everyone! Our free and open online course

After four years of testing, designing and deploying remote data collection projects, we partnered with Leiden University to develop an online course to share what we’ve learned so far. Our Remote Food Security Monitoring online course was launched in May, and aims to provide a clear understanding of what remote food security monitoring entails, when it is a useful tool, and how to implement a remote food security monitoring project. The course is free and self-paced, and open to anyone who is interested in setting up a remote data collection project.

mudasair

WFP/Jean-Martin Bauer

2: Expanding across Asia and the Pacific

During 2017, we kept growing, scaling up in the Asia/Pacific region. WFP’s Nepal and Sri Lanka country offices collaborated with their respective national government partners to launch  mobile-based food security monitoring systems. Nepal’s mNekSAP was the first to use an innovative dual-mode approach to collect data from a panel of households previously surveyed during a baseline assessment, combining remote mobile data collection with traditional face-to-face methods so as to not miss out on following up with those households without a phone. This means that the data gathered through mNEKSAP is not only representative (ensuring coverage of non-phone owners), but through re-interviewing the same individuals, it also provides us with a rare panel data set, which is optimum.      

Afghanistan, Myanmar and Papua New Guinea kept busy with ongoing mobile data collection. Afghanistan now uses mVAM to conduct several different types of surveys, from conflict rapid assessments, to market monitoring, to post-distribution monitoring. Most recently, they launched their first round of nutrition data collection for the Minimum Dietary Diversity for Women (MDD-W) indicator – stay tuned for results!

Meanwhile in PNG, their 4th nationwide survey introduced the Food Insecurity Experience Scale – an official SDG 2.1.2 indicator. Our hope is that we can use mVAM to help measure progress in this area.  Also in the region, we’ve been looking at ways to use the PRISM system to better visualize mVAM data and link it to other information sources. More on that in 2018!

WFP/Maria Muraskiewicz

WFP/Maria Muraskiewicz

3: Keeping up with remote nutrition data collection

We’re also expanding in terms of the type of data we use mVAM to collect. Following the success of last year’s remote nutrition data collection pilot in Kenya, we’ve moved on testing whether this is also feasible in Malawi and Niger, and which technologies we can use to collect the data.

From October 2016 to April 2017, we worked with GeoPoll in Malawi to develop a tool and methodology for collecting MDD-W data using SMS surveys. We conducted five rounds of surveys, during which we constantly adapted the indicator to make sure it was suitable for SMS surveys. We learned that the design of the questions was especially important – simple questions, a mix of open-ended and list-based questions, and the option to take the survey in the respondent’s preferred language proved particularly helpful.

In Niger, we tested the feasibility of using CATI to collect MDD-W data in IDP camps in the conflict-affected Diffa region. Through focus groups and in-depth interviews, it became evident that despite low phone ownership rates among women, most women do have access to phones through sharing with household members or neighbours. Men had little hesitation to women in their families being called when they were informed in advance, when female operators were used, and when the operators identified themselves as calling from WFP.. We’re now analysing the data we collected through both F2F and CATI, in order to understand potential mode effects and selection bias.

(For a full overview of our nutrition work, check out Episode 12 of VAM Talks!)

IMG_9645

4: Responsible data (collection, storage, sharing and distribution!)

Mobile data projects come with their own particular set of risks and challenges with regards to data privacy and protection. In a time when reports of data breaches seem to occur more and more frequently, what steps should we take to ensure that we aren’t accidentally putting the very people we are trying to assist at risk? Working with the International Data Responsibility Group (IDRG) and Leiden University’s Centre for Innovation, we developed a field book for Conducting Mobile Surveys Responsibly, which outlines the main risks of mobile data collection and provides guidelines for responsible data collection, storage, processing and distribution in complex humanitarian contexts. In December, we brought together experts on three different continents for a webinar on Responsible Mobile Data Collection, in which they discussed the challenges of remote data collection projects and shared best practices, tools, and tips for adhering to privacy and protection guidelines – from the field level to the WFP context and across the broader humanitarian and development sphere.  

Testing the chatbot in Nigeria

WFP/Seokjin Han

5: Communicating both ways: WFP speaks to …

As mobile technology continued to develop, we looked at ways to use new tools to allow the people we serve to start conversations with us about their own food security situations. In addition to getting information that we can use to improve the design of food assistance programmes, we want to ensure that the line is open so that people in the communities we serve can contact us and access information that is useful to them. In 2017, we continued the development of our two 2-way communications tools – a food security chatbot, and Free Basics, a platform which allows people to access certain sites on the internet at no data cost.

The start of the year saw us in New York where one of our partners, Nielsen, organized a hackathon to design a chatbot that could help collect information during a humanitarian response. Over the course of the year, we worked on developing use cases in different contexts – in Haiti , Nigeria and Kenya – and are now developing a chatbot builder with another partner of ours, InSTEDD. We look forward to deploying the bot in the new year.

Simultaneously, we expanded Free Basics after successfully piloting it in Malawi in November 2016; sites will soon go live in Rwanda, DRC and Niger. Back in Malawi, the original site, which started out as a free website to share weekly staple food prices, is now shifting its focus to address the needs of the more than 30,000 refugees and asylum seekers hosted in the country. The majority of the group lives in two camps where WFP provides food assistance in the form of monthly in-kind distributions and cash-based transfers. As their ability to move outside of the camps where they currently live is quite limited, having information not only about food prices in their immediate area but also food stocks is incredibly helpful.

Thank you to our partners and donors, without whose support none of this would be possible, and to you – our readers – for following along! See you in 2018!

Postcard from Niamey

WFP/Maria Muraskiewicz

WFP/Marisa Muraskiewicz

What you might have missed since our last report

We are back in Niamey, the capital of Niger, where the Harmattan wind is raging through the desert landscape. Although this is the ‘cooler’ season of the year, temperatures easily reach upwards of 38/39 degrees Celsius (100+ degrees Fahrenheit) at the height of the day.

Quite a few things have changed since we last reported on Niger. Moustapha, the VAM Officer, transferred to Nigeria, leaving the mVAM endeavours in Niger in the capable hands of Marisa, Herizo, and team. And boy have they been keeping busy! Thanks to their diligent efforts, three types of mVAM surveys are being implemented today: (1) a bi-monthly household survey; and (2) a key informant trader survey, both of which collect data in the volatile Diffa Region, which has been affected by the Boko Haram crisis; and (3) a nationwide household food security survey that covers hotspot sentinel sites. In addition, the team recently completed its first trial round to collect data for two nutrition indicators in the Diffa Region – the Minimum Dietary Diversity for Women (MDD-W) and the Minimum Acceptable Diet (MAD) indicators – to examine the feasibility of collecting nutrition data through mobile surveys in the Niger context (more on this will be shared in a separate blog entry in the future).

But perhaps the best way to appreciate the progress the Niger team has made while acknowledging the lingering challenges for mVAM in the country, is to pick up the discussion where we left off last time.

Connectivity, still a major challenge

While there is 3G in Niamey and the surrounding urban areas and calls can be placed in remote rural areas, poor connectivity compounded by frequent power cuts remains a big challenge in Niger. The call center that carries out the CATI (Computer Assisted Telephone Interviewing) surveys often has to call the same number at least ten times before it reaches the respondents. They’ve even installed a generator that can serve as a back-up in the event of sudden electricity outages. Meanwhile, the IT team within the WFP Niger Country Office has been in discussions with major mobile network operators in the country to identify solutions for better coverage, including the use of satellite channels. Whilst this expensive alternative is not available to the poorest and most vulnerable communities, we are hoping that more public and private investments will be made to improve overall connectivity in the future.

Marisa Niger2

WFP/Marisa Muraskiewicz

Connecting to women, no small feat

We also reported last time that very few women own their own mobile phone in Niger, and some don’t even have the right to receive a call without their husband’s permission. Following best practices from other countries that are facing similar challenges, the call center conducted the last round of CATI surveys employing only female operators and witnessed a slight improvement in female response rates. Nonetheless, the average female response rate is still less than ten percent, so we need to continue to step up our sensitization and outreach efforts.

New mVAM tools coming to Niger: Numero verte & IVR AND Free Basics

On the bright side, we have been able to configure the IVR (Interactive Voice Response) software (Verboice) and connect it to a Numero verte – a four digit toll-free phone number – which can handle multiple incoming calls from various local network operators simultaneously. This hotline number will boost WFP Niger’s capacity to receive complaints and feedback from beneficiaries and take action when needed, bringing us closer to the communities we are supporting. Meanwhile, a new Free Basics site is in the making, which will allow us to share up-to-date market price information and tips on good nutrition and health practices with families and communities. So we are happy to admit that we were wrong last time when we said we didn’t think we would be using any of our ‘fancy’ tools in Niger any time soon!

Angie Niger

WFP/Angie Lee

A bright future for mVAM in Niger

As remarkable as the achievements of Niger’s team have been over the past year, there are no plans to stop! They are working on new activities that will make mVAM even more relevant for reaching the goals of WFP in Niger and our partners. In the coming months, the team will focus on working closer with the government, which has a keen interest in deploying mobile technologies for food security monitoring and early warning, as well as scaling up mVAM to expand our market monitoring activities.

Qu’est-ce qui se passe au Burundi?

IMG_0093

WFP/Silvia Calo

This week, we were in Burundi to improve how we collect, manage, and visualize data. Specifically, we wanted to work on two surveys that we conduct in the country using mVAM: an early warning survey – “Systeme d’Alerte Précoce au Burundi” (SAP) – and price data collection, known as mMarket.

mVAM has been active in Burundi since October 2016 and has collected information on different early warning indicators every month since then. Given the very low mobile phone ownership rates in the country, it is not feasible to conduct household food security surveys using mVAM. However, we have been able to gather useful information by regularly calling 55 Burundi Red Cross volunteers there who make up the SAP. These volunteers are Burundian citizens who work closely with the communities we’re trying to reach. They organize weekly meetings with local community focal points, which gives them a good understanding of the food security situation.

Gathering information about food security in the communities through key informants has its challenges of course. Finding out how households are coping without interviewing them directly can sometimes be difficult.

We visited WFP’s Country Office in Burundi in order to combine the local team’s knowledge of the Burundian context with our experience of conducting phone surveys. The result?  A new questionnaire that is shorter than the previous one, but still contains all the indicators needed for a meaningful early warning survey. Although the additional indicators we collected in the longer survey provided valuable information, very long questionnaires conducted over the phone have their own set of risks – the length may lead to key informants dropping out or not being willing to participate in the survey at all. Even worse, key informants may want to speed up the survey and don’t think carefully about their answers. We need to remember that volunteers who provide information are often very busy providing assistance to the local communities and may not have much time to speak over the phone!

Burundi pic

WFP/Silvia Calo

The second objective of our trip was to improve the mMarket data collection, which uses information from traders in different geolocated markets in the country. We added some commonly consumed food items to the questionnaire, as well as some non-food items, such as the cost of fuel, which serves as an early warning indicator for a rise in food prices.

Both SAP and mMarket yield large amounts of data at a high frequency. Since the added value of mVAM is providing valuable information in as close to real-time as possible, we always try to find new ways of speeding up the data analysis process and the publication of bulletins. As key informant surveys like SAP and mMarket deliver qualitative, rather than quantitative information, there is no magic statistical formula that can be used to make sense of the data. Hence, the only way to build a story around the data is to look at the data itself. We go about doing this by using data visualization tools like the software Tableau. Rather than simply looking at tables, we used dashboards and triangulated the indicators we collected, which enabled us to track how the food security situation is evolving. In the future, we might also use Tableau to produce interactive bulletins, so that users can explore the data we collect in more depth.

Starting in November 2017, our revamped questionnaires will be used and we will publish new bulletins, which will include interactive data visualizations. Stay tuned!

A new mVAM baby in Mali, weight: 7800 respondents!

WFP/Sebastien Rieussec

WFP/Sebastien Rieussec

This week we’re reporting on our latest news from mVAM in Mali. In this landlocked country in the Sahel chronic food insecurity and malnutrition is widespread – WFP has been present in Mali since 1964. In the last few years Mali has been coping with numerous shocks – such as droughts, floods and a military coup – that led to a political and security crisis and increased food insecurity in the country: by 2016 around 3.1 million people in Mali were food insecure. Households are particularly affected during the lean season, between June and September; and this year WFP estimated 3.8 million people affected by food insecurity, of which 601,00 people in urgent need of food assistance.

To monitor the food security situation, the Government of Mali, with WFP support, does two nationwide face-to-face surveys, in February and September each year. However, in between these times and especially during the lean season that takes place during the summer in Mali there was no data collection – so mVAM was there to fill the ‘data gap.’ We’ve previously blogged about the Mali mode experiment we did comparing data collected by live calls and face-to-face data. As the results showed that there was little difference between the modes, in August the Country Office rolled out mVAM nationwide so that they could get food security information from households affected by this particularly difficult period of the year. During the previous face-to-face survey phone numbers were collected…out of the 13,400 numbers we collected we reached over 7,800 households – mVAM’s largest-ever survey!

With each survey comes different country-specific ‘problems’. There are many different reasons why people might not want to take part in a phone survey – but in Mali, we found one of the biggest was mistrust. People are not used to doing surveys via mobile phones and are sure that there is some form of trick behind them. Many reported that they know that there are lots of mobile phone scams and worry that the call from an unknown number purporting to be from WFP is just another one of these. One of the reasons why they were suspicious  was due to the fact that there was a long time gap between the number collection and the phone survey. This was actually a deliberate choice by the Country Office to ensure that the survey was not just a ‘follow up’ survey to face-to-face data collection like our mode experiment and was getting new information during this specific time period. What wasn’t foreseen was that this meant people forgot that they had given WFP their number and may have not fully understood why they did so in the first place.

Mali blog Edith 2

WFP/Nanthilde Kamara

To get around this issue, the Country Office is planning to use several tactics. As well as using SMS and national radio to advertise the survey, the next time that phone numbers are collected, there will be more time spent on explaining exactly what the purpose of the survey is. The annual September face-to-face food security survey is currently ongoing, so enumerators are now explaining that they might be called by WFP later on this year. The call centre that supports mVAM in Mali calls everyone with the same unique number, this number will be shared with community leaders just before the survey so that they can inform people that they will be rung by this specific number and that it’s an official call from WFP. Respondents will then be able to save the number in their phone so they know when they get the call exactly who it is and it won’t be just an unknown number.

The analysis is still ongoing: We’re looking forward to the results!

 

New places, new tools: what’s up next for mVAM?

KOICA pic 2

We’ve just got back from Rwanda where we were holding a workshop on using mVAM to expand real-time food security and nutrition monitoring with Internally Displaced Persons (IDPs) and refugee populations. The project, which is made possible by the support of the Korean International Cooperation Agency (KOICA), will be implemented in ten countries in sub-Saharan Africa where WFP works.

What’s the project?

The KOICA project has two aims. First, it aims to empower information exchange with marginalized populations, specifically IDPs and Refugees. Secondly, it supports the collection of food security and nutrition data using the latest mobile and satellite technologies. This will happen in ten countries in Sub-Saharan Africa: the Central African Republic (CAR),The Democratic Republic of Congo (DRC), Kenya, Malawi, Niger, Nigeria, Rwanda, Somalia, South Sudan and Uganda.

How are we going to do this?

As you know, two-way communication systems are an important part of our work. As well as getting information that we can use to inform WFP programmes, we want to ensure that the line is open so that people in the communities we serve can contact us and access information that is useful to them. We’ve already been using Interactive Voice Response and live calls to share information with affected populations, and are now expanding our toolbox to include new technologies: Free Basics and a chatbot.

Remote data collection isn’t just done by mobile phones – VAM already uses other sources, such as  satellite imagery analysis – to understand the food security situation on the ground.  Under this project, we’ll also help countries incorporate similar analysis which will complement two-way communication systems to provide a fuller picture of the food security situation.

Finally, we’re going to harness our knowledge of Call Detail Records analysis: de-identified metadata collected via cell phone towers about the number of calls or messages people are sending and which towers they are using. We have already used this technique in Haiti to track displacement after Hurricane Matthew, and we’re really excited to transfer these ideas to another context to ensure we get up-to-date information on where affected communities are so we can better target food assistance in the right locations.

What happened at the workshop?

Representatives from all 10 country offices, three regional bureaus and staff from HQ came together to discuss the three main project components. During the workshop, the different country offices had the chance to learn more from members of the mVAM team about the specific tools they can harness and ensure their collected data is high quality, standardised and communicated effectively. However, the best part about bringing everyone together was that country teams could share their experiences about how they are already using mVAM tools. We heard from the Malawi country office about their Free Basics pilot, and Niger and Nigeria explained how they’re implementing IVR so affected communities can easily contact WFP, even after work hours. Sharing their different experiences and learning about how different tools have worked in each context not only gave everyone an overview of what mVAM is doing so far, it also helped everyone understand the implementation challenges and how to overcome them.

What’s next for the KOICA project?

We’re really excited for the next stage of the project. Each country office has now planned what tools they’re going to use to increase their communications with affected communities and how they will improve their existing data collection systems. It’s going to be great to see the impact these tools will have not only on WFP’s response, but also how they will empower the communities we’re serving. 

Mind the mode:

Who's texting & who's talking in Malawi?

Malawi mVAM respondent WFP/Alice Clough

Malawi mVAM respondent
WFP/Alice Clough

It’s time for another installment of our Mind the Mode series. For those of you who follow this blog regularly, you know that the mVAM team is continually evaluating the quality of the data we collect. Past Mind the Mode blogs have discussed our work in Mali looking at face-to-face versus voice calls, our comparison of SMS and IVR in Zimbabwe and the differences in the Food Consumption Score (FCS) for face-to-face versus Computer-Assisted Telephone Interviews (CATI) interviews in South Sudan.

This month, we turn our attention to Malawi, where we recently completed a study analyzing the differences in the reduced Coping Strategies Index (rCSI) when it’s collected via CATI and SMS. This indicator helps measure a household’s food security by telling us what actions they might be taking to cope with any stresses such as reducing the number of meals a day or borrowing food or money from friends or family. From February to April 2017, around 2,000 respondents were randomly-selected for an SMS survey and 1,300 respondents were contacted on their mobile phones by an external call centre to complete a CATI survey.

People Profiling: who’s Texting and who’s Talking? 

Across all three rounds, a greater proportion of respondents in both modalities were men who lived in the South and Central Regions of the country and came from male-headed households. However, the respondents taking the SMS survey were much younger (average age 29) than those who took the CATI survey (average age 40). This probably isn’t surprising when you consider that young people across the world tend to be much more interested in new technologies and in Malawi are more likely to be literate.

The results from our mode experiment in Zimbabwe showed that IVR and SMS surveys reached different demographic groups so we figured we might see the same results in Malawi. However, this was surprisingly not the case: both CATI and SMS participants seemed to come from better-off households. In our surveys we determine this by asking them what material the walls of their home are made from (cement, baked bricks, mud, or unbaked bricks).

better off-worse off wall type malawi

More respondents (60%) said they have cement or baked brick walls as opposed to mud or unbaked brick walls, an indicator of being richer.

Digging into the rCSI

So what about the results observed for the rCSI between the two modes? The CATI rCSI distribution shows a peak at zero (meaning that respondents are not employing any negative coping strategies) and is similar to the typical pattern expected of the rCSI in face-to-face surveys (as you can see in the two graphs below).

Density plot for CATI Feb-April 2017

 

SMS rCSI

The SMS results, on the other hand, tend to have a slightly higher rCSI score than in CATI, meaning that respondents to the SMS survey are employing more negative coping strategies than households surveyed via CATI. This is counter-intuitive to what we might expect, especially since the data illustrates that these households are not more vulnerable than CATI respondents. Presumably, they would actually be better educated (read: literate!) to be able to respond to SMS surveys. We’re therefore looking forward to doing some more research in to why this is the case.

Box plot cati rcsi

It’s All in the Numbers

Some interesting patterns in terms of responses were also observed via both modalities. SMS respondents were more likely to respond to all five rCSI questions by entering the same value for each question (think: 00000, 22222…you get the idea!). At the beginning of the survey, SMS respondents were told that they would earn a small airtime credit upon completion of the questionnaire. We conjecture that some respondents may have just entered numbers randomly to complete the questionnaire as quickly as possible and receive their credit. Keep in mind that entering the same value for all five rCSI questions via CATI is a lot more difficult, as the operator is able to ask additional questions to ensure that the respondent clearly understands the question prior to entering the response.  For SMS, there’s no check prohibiting the respondent from dashing through the questionnaire and entering the same response each time.

We also saw that the percentage of respondents stating that they were employing between zero and four strategies was much lower among SMS respondents than CATI respondents across all three months of data collection. Conversely, more respondents (three out of five) in the SMS survey reported that they were using all five negative coping strategies than in the CATI survey. Again, this is counter-intuitive to what we would expect.  It might mean that SMS respondents didn’t always correctly understand the questionnaire or that they didn’t take the time to reflect on each question, completing questions as rapidly as possible to get their credit; or simply entered random numbers in the absence of an operator to validate their responses.  The graphs below illustrate the differences in rCSI responses between CATI and SMS.

Figure 3: Distribution of the number of coping strategies reported by SMS and CATI respondents by months

Figure 3: Distribution of the number of coping strategies reported by SMS and CATI respondents by months

From these results, you can see that we still have a lot to learn on how survey modality affects the results. This is just the start of our research; so expect more to come as the team digs deeper to better understand these important differences.

Mind the mode …. and the non-response

How voice and face-to-face survey data compares in Mali

This is the third entry in our ‘Mind the Mode’ series on the mVAM blog. We are constantly assessing our data collection modalities to better understand what produces the most-accurate results and what biases may be present. One of our recent experiments took us to Mali, where we were comparing the food consumption score between face-to-face (F2F) interviews versus mVAM live calls.

It’s all in the details
To do this, in February and March, the WFP team first conducted a baseline assessment in four regions of the country. As part of the baseline, we collected phone numbers from participants. Approximately 7-10 days later, we then re-contacted those households who had phones, reaching roughly half of those encountered during the face-to-face survey. We weren’t able to contact the other households. To ensure the validity of the results, we made sure the questionnaire was the exact same between the F2F and telephone interviews. Any differences in wording or changes in the way in which the questions were asked could adversely affect our analysis.

The findings from our analysis were quite interesting. We found that food consumption scores (FCS) collected via the mVAM survey tended to be slightly higher than those collected via the face-to-face survey. The graph below illustrates this shift to higher scores between the two rounds. Higher FCS via mVAM versus F2F surveys is not atypical to Mali. We’ve observed similar outcomes in South Sudan and other countries where mVAM studies have taken place.

mali dist

 

Why could this be? There are two main reasons that could explain this difference. Either it might be due to the data collection modality (i.e., people report higher food consumption scores on the phone)? Or, a perhaps a selection bias is occurring? Remember that we were only able to contact roughly half of the participants from the F2F survey during the telephone calls. So, it’s possible that people who responded to the phone calls are less food insecure, which could make sense, since we often see that the poorest of the poor either don’t own a phone or have limited economic means to charge their phone or purchase phone credit.

To test these hypotheses, we dug a bit deeper.

Same same…
Are people telling the same story on the phone versus face-to-face? Based on our results, the answer is yes! If we compare the same pool of respondents who participated in both the F2F and telephone survey rounds, their food security indicators are more or less the same. For example, the mean mVAM FCS was 56.21 while the mean F2F FCS was 55.65, with no statistically significant difference between the two.

But different…
So what about selection bias? In the F2F round, there are essentially three groups of people: 1) those who own phones and participated in both the F2F and mVAM survey; 2) people who own phones but didn’t participate in the mVAM survey, because they either didn’t answer the calls or their phone was off; and 3) people who do not own a phone and thus couldn’t participate in the mVAM survey.

People who replied to the mVAM survey have overall higher FCS than those that we were unable to contact. What we learned from this experiment is that bias does not only come from the households that do not own a phone but also from non-respondents (those households who shared their phone number and gave consent but then were not reachable later on for the phone interview). Possible reasons why they were not reachable could be that they have less access to electricity to charge their phone or that they live in areas with bad network coverage. The graph below illustrates the distribution by respondent type and their respective FCS.

mali boxp

When you compare the demographics of people in these three groups based on the data collected in the baseline, you can see that there are significant differences, as per the example below. Notice that the education levels of respondents varies amongst the three groups—those without a phone tend to be less educated than those who own a phone and participated in the mVAM survey.

mali profile

This study taught us a valuable lesson. While we are confident that there is no statistically significant difference between face-to-face and phone responses within the Mali context, there is a selection bias in mVAM-collected data. By not including those without phones as well as those who did not respond, we are missing an important (and likely poorer) subset of the population, meaning that the reported FCS is likely higher than it may be if these groups were included. One way to account for this bias is to ensure that telephone operators attempt to contact the households numerous times, over the course of several days. It’s important that they really try to reach them. The team is also studying how to account for this bias in our data analyses.

mVAM for nutrition: findings from Kenya

2WFP-Kusum_Hachhethu

Photo: WFP/Kusum Hachhethu

We’ve used mVAM to collect data on a range of things that impact food security – so what about information on nutrition? Back in October, we went to Kenya to conduct a study on whether we could use remote mobile data collection to gather information on women and children’s nutrition.

The summary of our findings from the case study are now available in a new report from mVAM and our partners in the study, WFP’s Nutrition Division and the World Agroforestry Centre (ICRAF).

Read more:

kenya-report