New places, new tools: what’s up next for mVAM?

KOICA pic 2

We’ve just got back from Rwanda where we were holding a workshop on using mVAM to expand real-time food security and nutrition monitoring with Internally Displaced Persons (IDPs) and refugee populations. The project, which is made possible by the support of the Korean International Cooperation Agency (KOICA), will be implemented in ten countries in sub-Saharan Africa where WFP works.

What’s the project?

The KOICA project has two aims. First, it aims to empower information exchange with marginalized populations, specifically IDPs and Refugees. Secondly, it supports the collection of food security and nutrition data using the latest mobile and satellite technologies. This will happen in ten countries in Sub-Saharan Africa: the Central African Republic (CAR),The Democratic Republic of Congo (DRC), Kenya, Malawi, Niger, Nigeria, Rwanda, Somalia, South Sudan and Uganda.

How are we going to do this?

As you know, two-way communication systems are an important part of our work. As well as getting information that we can use to inform WFP programmes, we want to ensure that the line is open so that people in the communities we serve can contact us and access information that is useful to them. We’ve already been using Interactive Voice Response and live calls to share information with affected populations, and are now expanding our toolbox to include new technologies: Free Basics and a chatbot.

Remote data collection isn’t just done by mobile phones – VAM already uses other sources, such as  satellite imagery analysis – to understand the food security situation on the ground.  Under this project, we’ll also help countries incorporate similar analysis which will complement two-way communication systems to provide a fuller picture of the food security situation.

Finally, we’re going to harness our knowledge of Call Detail Records analysis: de-identified metadata collected via cell phone towers about the number of calls or messages people are sending and which towers they are using. We have already used this technique in Haiti to track displacement after Hurricane Matthew, and we’re really excited to transfer these ideas to another context to ensure we get up-to-date information on where affected communities are so we can better target food assistance in the right locations.

What happened at the workshop?

Representatives from all 10 country offices, three regional bureaus and staff from HQ came together to discuss the three main project components. During the workshop, the different country offices had the chance to learn more from members of the mVAM team about the specific tools they can harness and ensure their collected data is high quality, standardised and communicated effectively. However, the best part about bringing everyone together was that country teams could share their experiences about how they are already using mVAM tools. We heard from the Malawi country office about their Free Basics pilot, and Niger and Nigeria explained how they’re implementing IVR so affected communities can easily contact WFP, even after work hours. Sharing their different experiences and learning about how different tools have worked in each context not only gave everyone an overview of what mVAM is doing so far, it also helped everyone understand the implementation challenges and how to overcome them.

What’s next for the KOICA project?

We’re really excited for the next stage of the project. Each country office has now planned what tools they’re going to use to increase their communications with affected communities and how they will improve their existing data collection systems. It’s going to be great to see the impact these tools will have not only on WFP’s response, but also how they will empower the communities we’re serving. 

Our experiment using Facebook chatbots to improve humanitarian assistance

Testing the chatbot in Nigeria

Testing the chatbot in Nigeria

It must have been above 40 degrees Celsius that afternoon in Maiduguri, Nigeria. Hundreds of people were waiting to cash the mobile money they receive from the World Food Programme (WFP), sitting under tarps that provided some protection from the sun – in other words, the perfect time to sit and chat.

“How many of you have smartphones?” we asked. We waited for the question to be asked in Hausa, and out came mobile devices of all shapes and sizes. “How many of you have Facebook accounts?” Even before the question was translated, we saw nods all around.

“Of course we’re on Facebook – it’s the way we can message friends and family”

Displaced people in Nigeria, even those facing famine and urgently need aid, are connected and rely on messaging apps.

A leap of faith: from SMS to chatbot surveys

Collecting information in communities on the humanitarian frontline is dangerous, cumbersome and expensive, particularly in conflict settings. In north-east Nigeria, our assessment teams travel by helicopter or in convoys, and some locations are simply too insecure to visit at all. This means that decisions about emergency food assistance are sometimes made with very limited information.

But increasing access to mobile phones is changing this. WFP’s mobile Vulnerability Analysis and Mapping (mVAM) project has adopted SMS, Interactive Voice Response and call centres to collect food security information from communities enduring crises like Ebola or the Syrian civil war. Nelsen, a global information and measurement company, found that using SMS we are able to run our surveys 50% cheaper and 83% faster than we would have for face-to-face surveys, while putting no enumerators in harm’s way. The system’s success means we’re now using mobile tools to collect and share information in 33 countries.

Our successes with automated surveys meant we were keen to look into using chatbots (automated assistants that are programmed into messaging apps) to collect food security data. We were especially curious about the fact that a bot could help us ‘chat’ with thousands of people simultaneously and in real-time, like others have.

chatbot interaction

A sample chatbot interaction

To reach as many people as possible, we decided to create a bot that would operate on a popular messaging app, like Facebook Messenger or Telegram, so people could take our surveys on a platform they already use.

You might think it’s unreasonable to expect people in conflict settings to be connected at all. But, as our Nigeria example shows, their connection is a lifeline to normality. We also found that in many countries operators sell ‘social bundles’ that offer unlimited Facebook, WhatsApp or other social media for a single low price.

Where ‘Facebook Lite’ is available, people can even connect for free. All this means that communicating with vulnerable communities could happen in real time and at little to no cost to the respondent or WFP.

Introducing Food Bot

Last summer, we decided to try it out. InSTEDD developed a chatbot prototype that we demoed with Sub-Saharan African migrants in Rome. The demo asked the respondent to share information about food security in their community and allowed them to look up updated food prices.

Our testers liked the fact that talking to our bot felt like having a conversation with a real person. We felt like we were on to something! Earlier this year, Nielsen helped us further develop a chatbot design that calls for multiple gateways, natural language processing capabilities, and a reporting engine.

The current version of Food Bot is programmed to ask a predefined set of questions to the user – it does not rely on artificial intelligence yet. Food Bot goes through a simple questionnaire and saves the answers so that our analysts can process them.

The chatbot format also lets users ask us questions and is a channel for us to give useful information we’ve collected back to these communities. These include messaging on WFP programmes, food prices, weather updates, nutrition and disease prevention. The version we are using for testing currently runs on Facebook Messenger, but we want to make sure it works on all the relevant messaging apps.

No walk in the park

Before we get carried away, we need to consider some of the very real challenges. A timely report by the ICRC, Block Party and the Engine Room emphasizes the new responsibilities that humanitarian agencies assume as they make use of messaging apps to communicate with affected populations. Notably, the use of chat apps to collect information from people who have fled their countries or home raises the important issue of responsible data practices. If we are ever hacked, people’s personal details could be put at risk, including names and pictures. We will certainly have to review our existing data responsibility guide and continue obtaining advice from the International Data Responsibility Group (IDRG), as well as build an understanding of data responsibility principles in the field.

We also suspect that the audience we reach through Food Bot will be younger, better off, more urban and more male than the general population. The convenience of collecting data through a bot does not dispense with the hard task of seeking out those who are not connected and who are probably the most vulnerable. We want to explore ways to make our bot as accessible as possible like translating text into local languages, using more icons in low literacy settings and working with civil society organisations that specialize in digital inclusion.

Finally, we realize that we must prepare to manage all of the unstructured information that Food Bot will collect. Colleagues in the field are already weary of collecting yet more data that won’t be analysed or used. As a result, the team is working on setting up the infrastructure that is needed to process the large volumes of free text data that we expect the bot to produce. This is where our work with automated data processing and dashboards should pay dividends.

This post was originally published on ICT Works as part of a series on humanitarian chatbots.

Postcard from Dakar

mVAM workshop participants all smiles after learning more about IVR WFP/Lucia Casarn

mVAM workshop participants all smiles after learning more about IVR WFP/Lucia Casarn

During the last week of June, staff from WFP HQ’s mVAM team, the West and Central Africa Regional Bureau, and Nigeria and Niger Country Offices met in beautiful Dakar to work together on Interactive Voice Response (IVR) systems for two-way communication. (If you want to dig deep into all details IVR-related, check out the lesson in our mVAM online course!)

We’ve previously blogged about how WFP is responding to the needs of people who have been displaced due to Boko Haram insurgencies in both Nigeria and Niger. When we implemented these operations we also put communication channels in place so beneficiaries are able to contact WFP. In Nigeria, the Maiduguri Field Office created a hotline. Their operators receive an average of 100 calls per day from beneficiaries asking food security-related questions and providing feedback on the operations. The problem is the hotline is only available during working hours and has a limited number of people who can call in at the same time. To work around this they’re therefore looking at how an IVR system can support the call operators who are dealing with high volumes and better manage calls that take place outside of normal office hours. WFP Niger wants to set up a similar hotline system but without full time phone operators. Beneficiaries will call in to an automated IVR system and their queries and feedback recorded by the system and followed up by the Country Office. 

A Nigeria IT Officer working to install a GSM gateway for IVR usage in Maiduguri WFP/Lucia Casarin

A Nigeria IT Officer working to install a GSM gateway for IVR usage in Maiduguri WFP/Lucia Casarin

During the workshop participants were trained by InSTEDD on how to physically deploy IVR using a GSM gateway (a fancy tool that automatically places phone calls) and Verboice, the free open source software they’ve developed to manage these systems. The team also discussed the nitty gritty technical aspects of the system, including creating and modifying call flows (the sequencing of questions), scheduling calls and downloading collected call logs and recordings. Most importantly, participants had the opportunity to share their experiences and challenges with experts in this field and discuss best practices, alternative deployments and technical solutions.

The Country Office staff have now returned to Niger and Nigeria and they’ve already started testing the use of the IVR machines. We’re eager to begin logging data and hearing more from our beneficiaries. So stay tuned!

 

 

 

If you’re not human then who are you?

Experimenting with chatbots in Nigeria and Haiti

WFP/Lucia Casarin

Testing the bot in Haiti – WFP/Lucia Casarin

Readers of this blog know that the team has been experimenting with chatbots to communicate with disaster-affected communities – read our previous posts about our prototype and the Nielsen Hackathon.

As part of this effort, during recent missions to Haiti and Nigeria, our team went out to talk to communities to find out whether a chatbot would be right for them.

Would a chatbot be a stretch in these communities?

Well it’s not that much of a stretch.

In North East Nigeria, most displaced people live in Maiduguri, a city of over 1 million people. In this ‘urban’ setting connectivity is good, most people own cell phones and many young people use social media and messaging apps. Mobile operators have been offering services that allow people to access the internet by selling ‘social bundles’ (unlimited social media access sold in very small increments) and offer some services for free, including Facebook Light and Facebook Messenger.

In Haiti, three-quarters of the population live in the capital, Port-au-Prince, where 3G connectivity is good and most people use messaging apps to communicate with friends and family. Even in rural and difficult-to-reach communities, leaders and young people own smartphones and connect to the internet. There is a lot of competition between mobile operators so the prices for mobile data are very low. This means that most people can afford to access the internet either via their own smartphone or from shared smartphones.

A

Mobile phones charging station on the road from Léogane Peri to Port-au-Prince WFP/Lucia Casarin

A bare-bones demo

In both countries we tested a simple chatbot that asks people about food prices and what the food security is like in their community. The survey we used was much more basic than our usual mobile questionnaires as we felt it was important to keep things simple at this stage.

For Nigeria, the bot demo was initially in English but we soon translated it into Hausa, the primary language spoken by displaced persons in Maiduguri. In Haiti we made it available both in Creole and French. The chatbot was very responsive on 3G and it even worked with slower 2G connections so the technology works in these contexts. But this was only the starting point, what we really wanted to know was what ‘real’ people thought about the bot.

We organized focus group discussions with displaced people in Maiduguri and with community representatives in Haiti. We helped people access the WFP bot via their Facebook accounts, and they began chatting away.

Sounds cool, but what are the limitations?

Here’s what people said:

First of all, people thought the bot is a convenient, quick, and easy way to get in touch directly with WFP and they really liked that the bot allows them to speak to WFP without intermediaries. They had lot to tell us particularly through the open-ended question where they typed out detailed responses.

In Nigeria, they did tell us that our (somewhat wordy) English-language demo should be translated into Hausa because it would make it easier for everyone to use. Our first group of testers were young people who were already Facebook users and so were familiar with Messenger. It was therefore no surprise that they were interacting smoothly with the bot and able to go through our questionnaire in minutes.

WFP/Jean-Martin Bauer

Testing the bot in Nigeria – WFP/Jean-Martin Bauer

In Haiti, people started interacting with the bot as if it was a human rather than an automated questionnaire so they got stuck pretty fast when it wasn’t as naturally responsive as they’d expected. This means that either we give clearer instructions to people or we add Natural Language Processing capabilities to our bot.

There are of course other barriers. In both countries women appeared to be less likely to own a smartphone. This means that bot users will likely be overwhelmingly young, male and better educated than other people – hardly ‘representative’ of WFP’s target affected population. The free version of the bot is also not always available: in Nigeria only Airtel subscribers can access it, while in Haiti the free service doesn’t exist yet.

This means that the bot would need to be a complement to the other tools we have. We might use data from the bot to obtain a quick situation update, but we will continue relying on other sources for more representative data.

Hearing from those who are #FacingFamine

Photo: WFP/Amadou Baraze

Photo: WFP/Amadou Baraze

 

In early March, Stephen O’Brien, the United Nations’ Emergency Relief Coordinator, reported that 20 million people across four countries face starvation and famine.  The famines looming in Yemen, South Sudan, Somalia and Nigeria represent the largest humanitarian crisis since the UN’s creation. “Without collective and coordinated global efforts,” O’Brien said, “People will simply starve to death, and many more will suffer and die from disease.”

One of the components that complicates these particular emergencies is access to the areas in crisis. Without safe and unimpeded access for humanitarian aid workers, it’s difficult to get a picture of what’s going on in the affected areas, which adds another dimension to an already challenging response. In Northeast Nigeria, the threat of violence made it difficult for WFP’s food security analysts to visit vendors in local markets or speak with people in their homes – all part of their usual food security monitoring routine.

In order to continue gathering information needed to understand the situation in the affected areas, WFP used remote mobile data collection to get a picture of what was happening in the communities they could no longer speak to in person. With an overwhelming amount of responses, we turned to Tableau , who had already helped us create data visualizations for other countries which use mVAM, to help us visualize the results in a way that could be easily understood by everyone.

mVAM hears directly from people in affected communities in the northeast of Nigeria

mVAM hears directly from people in affected communities in the northeast of Nigeria

 

Our latest interactive data visualization of the food security situation in Northeast Nigeria is now online, and the story of how it came to be can be found on Tableau’s blog. Make sure to check out the free response section, where you can hear from 5,500 households on what should be done to improve the food security in their community.

 

6,000 degrees of mVAM

nigeria-assistance

For the last six years Northern Nigeria and the surrounding countries of Niger, Chad and Cameroon have been suffering under Boko Haram insurgency. Across the four countries affected, security and humanitarian conditions are still deteriorating as populations continue to flee the systemic violence and conflict. We’ve previously written about how WFP is using mVAM in Niger to get dynamic data to complement their face-to-face surveys but we also wanted to blog about what we are doing in Nigeria itself.

Recent offensives by the Nigerian government have meant that many areas of northeastern Nigeria have recently become accessible – ‘showing’ the depth of the humanitarian crisis. In the worst affected areas of Borno and Yobe states famine-like conditions may be occurring. It’s now estimated that 2.1 million people are displaced, 81% of whom are living in local communities. This influx of people, coupled with successive poor harvests and a worsening economy has also put a strain on the host communities, there are now 4.4 million people who are food insecure.

nigeria-situation-map

Security constraints in northeastern Nigeria continue to limit the ability to conduct traditional face-to-face surveys, especially in Borno state. As mVAM has proven itself a useful tool in conflict settings and gathering information in difficult to access areas, the Nigerian National Emergency Management Agency (NEMA) and WFP have opted to use remote data collection to collect basic food security and market data.

The scale of the crisis and affected population meant that we wanted to try and reach even more people than our normal sample sizes of 1500. In our June/July round, we managed to reach slightly over 6,000 households in Adamawa, Borno and Yobe States, greatly increasing the reach and precision of our estimates.

Our findings showed that household purchasing power has deteriorated and more families are food insecure.  In the Local Government Areas (LGA) of Potiskum in Yobe State and Maiduguri/Jere in Borno State, the percentage of severely food insecure households effectively doubled since February-March. In the same time period, prices for local rice and local maize have risen but manual labour wage rates did not increase, severely reducing household purchasing power.  We also found that, despite this, only 11% of the surveyed population report that they received food assistance in the last 30 days.

Alongside collecting traditional food security indicators, this large sample size means that we had the chance to ask 6,000 households to express in their own words what the food security situation in their community.  They told us:

“There is no food in the community.  Because of the insurgency people have stopped farming” – Male Resident from Shelling, Adamawa

“The food situation over here is so critical…not only the IDPs, even the residents are suffering themselves“-Male IDP in Gujba, Yobe

“Food are scarce, even the middle spend all their income on food because of how difficult the situation is here” Male Resident in Nguru, Yobe  

As we prepare to call back these same households in November, we’d like help.  If you had the chance to reach  6,000 households in Northern Nigeria – what would you ask?
Here’s the questionnaire we used last round. Please tell us what you would like to ask – just fill in the contact form below.

Chatbot: back to the drawing board

alice-hand-model

We’ve recently developed a prototype of a chatbot to communicate with people via the Telegram messaging app, but it will eventually work on any messaging app. The purpose of a prototype is to test our approach thus far in the real world and then go back to the drawing board to improve it. Before this month, our testing had been limited to our colleagues here in Rome and our partners at InSTEDD.  However, we really needed feedback from people in the communities we are actually trying to reach.

Eventually we’ll test a later version of the chatbot in the countries where we work. But for some initial feedback, we were able to get in contact with people right on our doorstep, who had completed a difficult journey to Rome. UNHCR recently estimated that there are roughly 65.3 million people currently forcibly displaced worldwide. Instability and conflict in the Middle East and Africa has led many to flee to Europe in the hope of a safer, better life. One of the most used and most dangerous routes is via Libya and then across the Mediterranean Sea to southern Italy. To give you an idea of the scale, between 29 August and 4 September this year, Italy averaged over 2000 arrivals every day. Of those who reach the mainland, many make their way north to Rome. There are now many centres across the city that provide refuge, often in the form of meals, language learning and legal support.  

UN photo/UNHCR/Phil Behan

We went to one of these migrant centres to speak with people and get their feedback on whether the chatbot would be useful in their home communities. We can hear migration statistics but listening to people’s stories really made these statistics come alive. One person we talked to described being saved by the Italian Coast Guard as the ship transporting him sunk in the Mediterranean. He said he will always be grateful to the Italian government for his rescue.

So needless to say, we were very grateful that people would take time and test the bot. Its is currently in English so we were only able to test it with English speakers right now. The goal is to get it running well in English and then translate and adapt it to other languages.

First we asked people a couple of questions about smartphone ownership in their country of origin. They told us that while the poorest people in rural areas don’t have smartphones around 70% of the population does – meaning that we can still communicate with a lot of people via smartphone. They then had a go at using our chatbot, first answering the food security survey and then trying out the price database. Here are a few things that speaking with them helped us realize:

Simplify our questions and build up to them more. We know we spend a lot of our time working with food security surveys and we know our food security questions by heart. We can forget how weird they can sound to everyone else, especially over a chat. For our participants, it was the first time they’d seen something like this so they were at times confused about how to respond to the questions about their diet or their coping strategies. They were especially confused because the questions seemed to come out of nowhere, with no build up or putting them in context. By the second time around, they went through the survey much quicker, but we need to make sure to get the best first time responses. We need to speak normal language, not make everyone else try to speak our specialized jargon.

No one wants to interact with a robot: make the chatbot as chatty and friendly as possible. Our participants also advised us that it would be good to add some slang and colloquial language. But it is important to have it to feel like as natural an interaction as possible: As one of them said: “ If I want to say something or someone to talk to, I can write, and the chatbot can help and I can relax.’

Make it as intuitive as possible. The chabot users will have different backgrounds and tech literacy. Right now, as one of our participants put it, it’s accessible for “any educated person’, but we don’t want to limit our target audience. Our users might not even have secondary education so we want anyone who can use facebook to find it straightforward.img_2350

Make sure the bot recognises typos! Everyone knows how easy it is to make a typo on your smartphone so it’s essential that our chatbot recognises a few of the easy ones. When we ask people how many days in the past week that they ate vegetables for example, it’s pretty easy to give ‘3 days’ ‘three days’ ‘three’ and ‘3days’ and all mean the same thing! Even potentially typos like “theee days” or ‘three dyas”. We need to integrate these differences in text and typos as acceptable responses, asking for confirmation when needed, so we get the best results.

Put the bot on different messaging apps. One of the reasons why they were a bit hesitant with the bot at the beginning was the fact that they were not used to the Telegram app. It’s important for the bot to run on the app people use most. This can vary depending on the country, so when we do our pilot, we need to put the chatbot on the most commonly used messaging app.

Give people food price information for their areas. At the moment, our bot automatically reads the general WFP food price database. Whilst this is a cool way of looking at food prices all over the world, it’s not actually that useful on a day-to-day basis. Our participants said that knowing up-to-date regional prices would be great – as it would allow them to go to different parts of the country to buy food if the price was particularly low there. As we are already collecting high frequency price data, we want to be able to integrate this into the chatbot. This would be a great way to use the real-time data that we collect to give directly back to the people we collect from.

Overall the participants were positive to the bot as it stands, even saying that they think it’s ‘really cool’. However, there’s a lot of work to be done to make it more user friendly. Using this feedback we are going back to the drawing board. We hope to have an even better version for our official pilot in sub Saharan Africa later this year. We are very grateful to the people who helped us test this last week. They have much more pressing things to worry about so we thank them again for generously giving us a bit of time.