If you’re not human then who are you?

Experimenting with chatbots in Nigeria and Haiti

WFP/Lucia Casarin

Testing the bot in Haiti – WFP/Lucia Casarin

Readers of this blog know that the team has been experimenting with chatbots to communicate with disaster-affected communities – read our previous posts about our prototype and the Nielsen Hackathon.

As part of this effort, during recent missions to Haiti and Nigeria, our team went out to talk to communities to find out whether a chatbot would be right for them.

Would a chatbot be a stretch in these communities?

Well it’s not that much of a stretch.

In North East Nigeria, most displaced people live in Maiduguri, a city of over 1 million people. In this ‘urban’ setting connectivity is good, most people own cell phones and many young people use social media and messaging apps. Mobile operators have been offering services that allow people to access the internet by selling ‘social bundles’ (unlimited social media access sold in very small increments) and offer some services for free, including Facebook Light and Facebook Messenger.

In Haiti, three-quarters of the population live in the capital, Port-au-Prince, where 3G connectivity is good and most people use messaging apps to communicate with friends and family. Even in rural and difficult-to-reach communities, leaders and young people own smartphones and connect to the internet. There is a lot of competition between mobile operators so the prices for mobile data are very low. This means that most people can afford to access the internet either via their own smartphone or from shared smartphones.

A

Mobile phones charging station on the road from Léogane Peri to Port-au-Prince WFP/Lucia Casarin

A bare-bones demo

In both countries we tested a simple chatbot that asks people about food prices and what the food security is like in their community. The survey we used was much more basic than our usual mobile questionnaires as we felt it was important to keep things simple at this stage.

For Nigeria, the bot demo was initially in English but we soon translated it into Hausa, the primary language spoken by displaced persons in Maiduguri. In Haiti we made it available both in Creole and French. The chatbot was very responsive on 3G and it even worked with slower 2G connections so the technology works in these contexts. But this was only the starting point, what we really wanted to know was what ‘real’ people thought about the bot.

We organized focus group discussions with displaced people in Maiduguri and with community representatives in Haiti. We helped people access the WFP bot via their Facebook accounts, and they began chatting away.

Sounds cool, but what are the limitations?

Here’s what people said:

First of all, people thought the bot is a convenient, quick, and easy way to get in touch directly with WFP and they really liked that the bot allows them to speak to WFP without intermediaries. They had lot to tell us particularly through the open-ended question where they typed out detailed responses.

In Nigeria, they did tell us that our (somewhat wordy) English-language demo should be translated into Hausa because it would make it easier for everyone to use. Our first group of testers were young people who were already Facebook users and so were familiar with Messenger. It was therefore no surprise that they were interacting smoothly with the bot and able to go through our questionnaire in minutes.

WFP/Jean-Martin Bauer

Testing the bot in Nigeria – WFP/Jean-Martin Bauer

In Haiti, people started interacting with the bot as if it was a human rather than an automated questionnaire so they got stuck pretty fast when it wasn’t as naturally responsive as they’d expected. This means that either we give clearer instructions to people or we add Natural Language Processing capabilities to our bot.

There are of course other barriers. In both countries women appeared to be less likely to own a smartphone. This means that bot users will likely be overwhelmingly young, male and better educated than other people – hardly ‘representative’ of WFP’s target affected population. The free version of the bot is also not always available: in Nigeria only Airtel subscribers can access it, while in Haiti the free service doesn’t exist yet.

This means that the bot would need to be a complement to the other tools we have. We might use data from the bot to obtain a quick situation update, but we will continue relying on other sources for more representative data.

Hearing from those who are #FacingFamine

Photo: WFP/Amadou Baraze

Photo: WFP/Amadou Baraze

 

In early March, Stephen O’Brien, the United Nations’ Emergency Relief Coordinator, reported that 20 million people across four countries face starvation and famine.  The famines looming in Yemen, South Sudan, Somalia and Nigeria represent the largest humanitarian crisis since the UN’s creation. “Without collective and coordinated global efforts,” O’Brien said, “People will simply starve to death, and many more will suffer and die from disease.”

One of the components that complicates these particular emergencies is access to the areas in crisis. Without safe and unimpeded access for humanitarian aid workers, it’s difficult to get a picture of what’s going on in the affected areas, which adds another dimension to an already challenging response. In Northeast Nigeria, the threat of violence made it difficult for WFP’s food security analysts to visit vendors in local markets or speak with people in their homes – all part of their usual food security monitoring routine.

In order to continue gathering information needed to understand the situation in the affected areas, WFP used remote mobile data collection to get a picture of what was happening in the communities they could no longer speak to in person. With an overwhelming amount of responses, we turned to Tableau , who had already helped us create data visualizations for other countries which use mVAM, to help us visualize the results in a way that could be easily understood by everyone.

mVAM hears directly from people in affected communities in the northeast of Nigeria

mVAM hears directly from people in affected communities in the northeast of Nigeria

 

Our latest interactive data visualization of the food security situation in Northeast Nigeria is now online, and the story of how it came to be can be found on Tableau’s blog. Make sure to check out the free response section, where you can hear from 5,500 households on what should be done to improve the food security in their community.

 

6,000 degrees of mVAM

nigeria-assistance

For the last six years Northern Nigeria and the surrounding countries of Niger, Chad and Cameroon have been suffering under Boko Haram insurgency. Across the four countries affected, security and humanitarian conditions are still deteriorating as populations continue to flee the systemic violence and conflict. We’ve previously written about how WFP is using mVAM in Niger to get dynamic data to complement their face-to-face surveys but we also wanted to blog about what we are doing in Nigeria itself.

Recent offensives by the Nigerian government have meant that many areas of northeastern Nigeria have recently become accessible – ‘showing’ the depth of the humanitarian crisis. In the worst affected areas of Borno and Yobe states famine-like conditions may be occurring. It’s now estimated that 2.1 million people are displaced, 81% of whom are living in local communities. This influx of people, coupled with successive poor harvests and a worsening economy has also put a strain on the host communities, there are now 4.4 million people who are food insecure.

nigeria-situation-map

Security constraints in northeastern Nigeria continue to limit the ability to conduct traditional face-to-face surveys, especially in Borno state. As mVAM has proven itself a useful tool in conflict settings and gathering information in difficult to access areas, the Nigerian National Emergency Management Agency (NEMA) and WFP have opted to use remote data collection to collect basic food security and market data.

The scale of the crisis and affected population meant that we wanted to try and reach even more people than our normal sample sizes of 1500. In our June/July round, we managed to reach slightly over 6,000 households in Adamawa, Borno and Yobe States, greatly increasing the reach and precision of our estimates.

Our findings showed that household purchasing power has deteriorated and more families are food insecure.  In the Local Government Areas (LGA) of Potiskum in Yobe State and Maiduguri/Jere in Borno State, the percentage of severely food insecure households effectively doubled since February-March. In the same time period, prices for local rice and local maize have risen but manual labour wage rates did not increase, severely reducing household purchasing power.  We also found that, despite this, only 11% of the surveyed population report that they received food assistance in the last 30 days.

Alongside collecting traditional food security indicators, this large sample size means that we had the chance to ask 6,000 households to express in their own words what the food security situation in their community.  They told us:

“There is no food in the community.  Because of the insurgency people have stopped farming” – Male Resident from Shelling, Adamawa

“The food situation over here is so critical…not only the IDPs, even the residents are suffering themselves“-Male IDP in Gujba, Yobe

“Food are scarce, even the middle spend all their income on food because of how difficult the situation is here” Male Resident in Nguru, Yobe  

As we prepare to call back these same households in November, we’d like help.  If you had the chance to reach  6,000 households in Northern Nigeria – what would you ask?
Here’s the questionnaire we used last round. Please tell us what you would like to ask – just fill in the contact form below.

Chatbot: back to the drawing board

alice-hand-model

We’ve recently developed a prototype of a chatbot to communicate with people via the Telegram messaging app, but it will eventually work on any messaging app. The purpose of a prototype is to test our approach thus far in the real world and then go back to the drawing board to improve it. Before this month, our testing had been limited to our colleagues here in Rome and our partners at InSTEDD.  However, we really needed feedback from people in the communities we are actually trying to reach.

Eventually we’ll test a later version of the chatbot in the countries where we work. But for some initial feedback, we were able to get in contact with people right on our doorstep, who had completed a difficult journey to Rome. UNHCR recently estimated that there are roughly 65.3 million people currently forcibly displaced worldwide. Instability and conflict in the Middle East and Africa has led many to flee to Europe in the hope of a safer, better life. One of the most used and most dangerous routes is via Libya and then across the Mediterranean Sea to southern Italy. To give you an idea of the scale, between 29 August and 4 September this year, Italy averaged over 2000 arrivals every day. Of those who reach the mainland, many make their way north to Rome. There are now many centres across the city that provide refuge, often in the form of meals, language learning and legal support.  

UN photo/UNHCR/Phil Behan

We went to one of these migrant centres to speak with people and get their feedback on whether the chatbot would be useful in their home communities. We can hear migration statistics but listening to people’s stories really made these statistics come alive. One person we talked to described being saved by the Italian Coast Guard as the ship transporting him sunk in the Mediterranean. He said he will always be grateful to the Italian government for his rescue.

So needless to say, we were very grateful that people would take time and test the bot. Its is currently in English so we were only able to test it with English speakers right now. The goal is to get it running well in English and then translate and adapt it to other languages.

First we asked people a couple of questions about smartphone ownership in their country of origin. They told us that while the poorest people in rural areas don’t have smartphones around 70% of the population does – meaning that we can still communicate with a lot of people via smartphone. They then had a go at using our chatbot, first answering the food security survey and then trying out the price database. Here are a few things that speaking with them helped us realize:

Simplify our questions and build up to them more. We know we spend a lot of our time working with food security surveys and we know our food security questions by heart. We can forget how weird they can sound to everyone else, especially over a chat. For our participants, it was the first time they’d seen something like this so they were at times confused about how to respond to the questions about their diet or their coping strategies. They were especially confused because the questions seemed to come out of nowhere, with no build up or putting them in context. By the second time around, they went through the survey much quicker, but we need to make sure to get the best first time responses. We need to speak normal language, not make everyone else try to speak our specialized jargon.

No one wants to interact with a robot: make the chatbot as chatty and friendly as possible. Our participants also advised us that it would be good to add some slang and colloquial language. But it is important to have it to feel like as natural an interaction as possible: As one of them said: “ If I want to say something or someone to talk to, I can write, and the chatbot can help and I can relax.’

Make it as intuitive as possible. The chabot users will have different backgrounds and tech literacy. Right now, as one of our participants put it, it’s accessible for “any educated person’, but we don’t want to limit our target audience. Our users might not even have secondary education so we want anyone who can use facebook to find it straightforward.img_2350

Make sure the bot recognises typos! Everyone knows how easy it is to make a typo on your smartphone so it’s essential that our chatbot recognises a few of the easy ones. When we ask people how many days in the past week that they ate vegetables for example, it’s pretty easy to give ‘3 days’ ‘three days’ ‘three’ and ‘3days’ and all mean the same thing! Even potentially typos like “theee days” or ‘three dyas”. We need to integrate these differences in text and typos as acceptable responses, asking for confirmation when needed, so we get the best results.

Put the bot on different messaging apps. One of the reasons why they were a bit hesitant with the bot at the beginning was the fact that they were not used to the Telegram app. It’s important for the bot to run on the app people use most. This can vary depending on the country, so when we do our pilot, we need to put the chatbot on the most commonly used messaging app.

Give people food price information for their areas. At the moment, our bot automatically reads the general WFP food price database. Whilst this is a cool way of looking at food prices all over the world, it’s not actually that useful on a day-to-day basis. Our participants said that knowing up-to-date regional prices would be great – as it would allow them to go to different parts of the country to buy food if the price was particularly low there. As we are already collecting high frequency price data, we want to be able to integrate this into the chatbot. This would be a great way to use the real-time data that we collect to give directly back to the people we collect from.

Overall the participants were positive to the bot as it stands, even saying that they think it’s ‘really cool’. However, there’s a lot of work to be done to make it more user friendly. Using this feedback we are going back to the drawing board. We hope to have an even better version for our official pilot in sub Saharan Africa later this year. We are very grateful to the people who helped us test this last week. They have much more pressing things to worry about so we thank them again for generously giving us a bit of time.