How many pizzas does it take to build a chatbot?

Hackers are hungry Photo: WFP/Pia Facultad

Hackers are hungry
Photo: WFP/Pia Facultad

This week, government, business, academia and civil society leaders will gather at Davos to discuss solutions to the world’s biggest challenges – including how new technologies can be leveraged to solve some of the most serious problems we face. At mVAM, we continue to explore how some of these technologies could be used to help eliminate chronic hunger, malnutrition and food insecurity – most recently looking at how chatbots could help collect important information during a humanitarian response.

Last week, our collaborators at Nielsen – one of the early supporters of mVAM – organized a 24-hour hackathon at the Nielsen Tech Hub in New York City. As part of ongoing efforts through Nielsen Cares, the hackathon aimed to develop an open-source humanitarian chatbot that can collect real-time information about food security. This came at the right time for WFP – we’d developed and tested a prototype of the chatbot with InSTEDD, and Nielsen’s technology and development input helped bring in important new capabilities. Ultimately, our goal is to field-test a chatbot in Haiti in the next few months to help us track food security conditions as people recover from the impacts of Hurricane Matthew.

The event was open to the public. A diverse group of students, volunteer hackers, and Nielsen staff showed up to take on the challenge, despite the wintry weather. InSTEDD’s Director of Platform Engineering, Nicolás di Tada also participated.

Much more than a chatbot

What the hackers built is much more that a chatbot: it is a bona-fide chat-based data collection and reporting system. Rather than attempt to outdo each other (as is the case in most hackathons), the teams split up to build the different components of the system. The different teams, made up of perfect strangers, communicated during the hackathon through Slack. After 24 hours, most components were fully coded up, but there were still bugs with the orchestrator and the gateway that additional post-hackathon work will resolve.

The architecture of the system, as defined by Nielsen, includes:

  • a management interface that allows an analyst to set up a questionnaire, including and skip logic, and validation rules that prompt the user when they enter a wrong answer. The interface was built using the Angular 2 JavaScript framework;
  • a gateway that is able to interact with respondents through Facebook Messenger and potentially other chat applications. The Facebook gateway was built on top of the AWS Lambda service;
  • a natural language processing engine that analyzes text on the fly. It allows the chatbot to ‘interpret’ a user’s answers. For now, the NLP engine processes English language text, although the engine includes a translation service and, by default, translates all languages to English for more advanced NLP tasks. The engine was built using the AWS Lambda service and leverages IBM Watson’s AlchemyLanguage service for text processing.;
  • a set of ‘backend APIs’ that manage respondent and survey data, route respondents from each response to the next question, and provide data to user interfaces .  The APIs were built using the Django framework for python and deploys on the AWS Elastic Beanstalk service;
  • an ‘orchestration layer’ that maintains survey status and routes messages between the end user and the various backend services. The orchestration service is built on top of the AWS Lambda service; and
  • a “reporting and data visualization engine”. Data vizzes were built using Highcharts, a JavaScript-based application. This allows an analyst to instantly see the results of the chatbot surveys.

 

chatbot

 

Leveraging cloud services from the Amazon Web Services product catalog, the teams were able to build a scalable, cost effective platform that can be deployed quickly to multiple locations globally.

Remember the humans

We also received tips from a chatbot specialist, Alec Lazarescu from Chatbots Magazine. He encouraged us to ‘onboard’ users with an initial message that gives people a clear idea of what the chatbot is for. He told us to avoid ‘dead ends’ and allow users to speak to a human being in case they get stuck.

We’re very grateful to Nielsen for their support and to all the participants for their energy and creativity. The next steps involve WFP and InSTEDD accessing the code and work on ironing out the kinks. We expect challenges with the natural language processing in Haitian Creole, a language that is probably under-researched. Making the different parts of the chatbot work together seamlessly also appears to be an area we will still have to work on.  And, of course, the final test will be to see whether our target group – people living in Haiti – find the chatbot engaging.

Our 5 mVAM Highs from 2016

collage

1. Awards for Remote Mobile Data Collection Work

At the Humanitarian Technology 2016 conference, our paper Knowing Just in Time Knowing Just in Time’ won Best Paper for Outstanding Impact. In the paper, we assessed mVAM’s contribution to decision-making by looking at use cases for mVAM in camps, conflict settings and vulnerable geographies. Check out our blog Tech for Humanity for more on it and our other conference paper  mVAM: a New Contribution to the Information Ecology of Humanitarian Work

To close the year, we had a nice surprise from Nominet Trust, the UK’s leading tech for good funder. We made their 100 most inspiring social innovations using digital technology to drive social change around the world.  

2. New Tech

In this day and age there’s a lot of buzz around data visualization. We’ve been honing our skills with Tableau. Check out the data visualizations we did for Yemen and Haiti.

We’re also in the era of Big Data. We partnered with Flowminder, experts in analyzing call detail records, to track displacement in Haiti after Hurricane Matthew.  Find out more in ‘After the storm: using big data to track displacement in Haiti

We’re also super excited about the chatbot we started developing for messaging apps and our roll out of Free Basics in Malawi which is allowing us to share the food prices we collect in mVAM surveys with people in Malawi With mVAM, our main focus has been reaching people on their simple feature phones. But we know that smartphone ownership is only going to increase. Contacting people through internet-enabled phones opens up loads of new forms of communication and data collection. is still reaching people on their -free basics

3. Expansion!

mVAM expanded to 16 new countries facing a wide set of challenges: conflict, El Nino drought, hurricanes, extremely remote geographies. We’ve been tracking and learning about what remote mobile data collection can add to food security monitoring systems and what its limits are in different contexts. For some of the highlights, check out our blogs on Afghanistan, Democratic Republic of Congo, Haiti, Nigeria, Papua New Guinea, and  El Nino in Southern Africa,

4. Dynamic Partnerships

To have a lasting impact, we need to work with governments. We are really proud of our partnership with CAID, the Cellule d’Analyses des Indicateurs du Développement  under the Prime Minister’s Office in the Democratic Republic of Congo. We collaborated on setting up a national market monitoring system- mKengela that they are now running. We’ve had intensive technical sessions with the CAID team in Rome and Kinshasa to work on solutions that will fit their data management and analysis needs. The CAID team even traveled to Johannesburg to share their remote mobile data experience with other African countries and help other governments use this technology.

We’re also working with Leiden University. Bouncing ideas off of their team at the Centre for Innovation helps us move forward on tricky challenges. We’re also collaborating with them to develop an online course where we’re going to share our methodologies and how to use remote technology to monitor food security. Check out Welcome to Vamistan for more.

We are in the field of tech. So we can’t do our job well without partnering with the private sector. It’s definitely a dynamic area, and also one where we at mVAM are learning what works best in melding our humanitarian goals with the exciting private tech potential out there. Check out our blog From the Rift Valley to Silicon Valley and our hackathon with Data Mission for more.

5. Learning- the neverending process

In addition to trying out new technology, we’ve been trying to answer some important questions about the live calls, SMS, and IVR surveys which make up the bulk of mVAM data collection.  We’re also doing mode experiments to understand how people answer differently based on which mode we use to contact them. Check out our first Mind the Mode article with more coming in 2017. In Kenya, we are looking into whether we can ask nutrition indicators through mVAM methods. A major challenge is reaching women through phone surveys so we organized a gender webinar with partners to learn from what they are doing- check out our key gender takeaways. These are key questions and they can’t be resolved overnight. But we’re making steady progress in understanding them, and we’re excited for what more we’ll find out in 2017.

Thanks to everyone who has supported our work this year and kept up with our blog!

Chatbot: back to the drawing board

alice-hand-model

We’ve recently developed a prototype of a chatbot to communicate with people via the Telegram messaging app, but it will eventually work on any messaging app. The purpose of a prototype is to test our approach thus far in the real world and then go back to the drawing board to improve it. Before this month, our testing had been limited to our colleagues here in Rome and our partners at InSTEDD.  However, we really needed feedback from people in the communities we are actually trying to reach.

Eventually we’ll test a later version of the chatbot in the countries where we work. But for some initial feedback, we were able to get in contact with people right on our doorstep, who had completed a difficult journey to Rome. UNHCR recently estimated that there are roughly 65.3 million people currently forcibly displaced worldwide. Instability and conflict in the Middle East and Africa has led many to flee to Europe in the hope of a safer, better life. One of the most used and most dangerous routes is via Libya and then across the Mediterranean Sea to southern Italy. To give you an idea of the scale, between 29 August and 4 September this year, Italy averaged over 2000 arrivals every day. Of those who reach the mainland, many make their way north to Rome. There are now many centres across the city that provide refuge, often in the form of meals, language learning and legal support.  

UN photo/UNHCR/Phil Behan

We went to one of these migrant centres to speak with people and get their feedback on whether the chatbot would be useful in their home communities. We can hear migration statistics but listening to people’s stories really made these statistics come alive. One person we talked to described being saved by the Italian Coast Guard as the ship transporting him sunk in the Mediterranean. He said he will always be grateful to the Italian government for his rescue.

So needless to say, we were very grateful that people would take time and test the bot. Its is currently in English so we were only able to test it with English speakers right now. The goal is to get it running well in English and then translate and adapt it to other languages.

First we asked people a couple of questions about smartphone ownership in their country of origin. They told us that while the poorest people in rural areas don’t have smartphones around 70% of the population does – meaning that we can still communicate with a lot of people via smartphone. They then had a go at using our chatbot, first answering the food security survey and then trying out the price database. Here are a few things that speaking with them helped us realize:

Simplify our questions and build up to them more. We know we spend a lot of our time working with food security surveys and we know our food security questions by heart. We can forget how weird they can sound to everyone else, especially over a chat. For our participants, it was the first time they’d seen something like this so they were at times confused about how to respond to the questions about their diet or their coping strategies. They were especially confused because the questions seemed to come out of nowhere, with no build up or putting them in context. By the second time around, they went through the survey much quicker, but we need to make sure to get the best first time responses. We need to speak normal language, not make everyone else try to speak our specialized jargon.

No one wants to interact with a robot: make the chatbot as chatty and friendly as possible. Our participants also advised us that it would be good to add some slang and colloquial language. But it is important to have it to feel like as natural an interaction as possible: As one of them said: “ If I want to say something or someone to talk to, I can write, and the chatbot can help and I can relax.’

Make it as intuitive as possible. The chabot users will have different backgrounds and tech literacy. Right now, as one of our participants put it, it’s accessible for “any educated person’, but we don’t want to limit our target audience. Our users might not even have secondary education so we want anyone who can use facebook to find it straightforward.img_2350

Make sure the bot recognises typos! Everyone knows how easy it is to make a typo on your smartphone so it’s essential that our chatbot recognises a few of the easy ones. When we ask people how many days in the past week that they ate vegetables for example, it’s pretty easy to give ‘3 days’ ‘three days’ ‘three’ and ‘3days’ and all mean the same thing! Even potentially typos like “theee days” or ‘three dyas”. We need to integrate these differences in text and typos as acceptable responses, asking for confirmation when needed, so we get the best results.

Put the bot on different messaging apps. One of the reasons why they were a bit hesitant with the bot at the beginning was the fact that they were not used to the Telegram app. It’s important for the bot to run on the app people use most. This can vary depending on the country, so when we do our pilot, we need to put the chatbot on the most commonly used messaging app.

Give people food price information for their areas. At the moment, our bot automatically reads the general WFP food price database. Whilst this is a cool way of looking at food prices all over the world, it’s not actually that useful on a day-to-day basis. Our participants said that knowing up-to-date regional prices would be great – as it would allow them to go to different parts of the country to buy food if the price was particularly low there. As we are already collecting high frequency price data, we want to be able to integrate this into the chatbot. This would be a great way to use the real-time data that we collect to give directly back to the people we collect from.

Overall the participants were positive to the bot as it stands, even saying that they think it’s ‘really cool’. However, there’s a lot of work to be done to make it more user friendly. Using this feedback we are going back to the drawing board. We hope to have an even better version for our official pilot in sub Saharan Africa later this year. We are very grateful to the people who helped us test this last week. They have much more pressing things to worry about so we thank them again for generously giving us a bit of time.

Introducing our Chatbot

bot pictureAn important part of our job at mVAM is to stay tuned into the developments in the rapidly evolving mobile technology sector. Lately we noticed two main trends: first, more and more people in the places we work are using smartphones and chat apps to communicate, leading us to think about how to better reach out to this segment of the population. Second, chatbots, robots that live in chat applications, are all the rage and have a big potential to contribute to our work.

What is a chatbot?

A chatbot is a computer programme that uses artificial intelligence to interact with users through a messaging service in a way that is designed to seem like a conversation. We’ve been experimenting with ways to expand our capacity for two-way communication, i.e. contacting local communities but also hearing back from them. A chatbot provides a friendlier, more responsive way to interact with people by letting them communicate more naturally, in a “chat” as the name implies. As well as answering the chatbot’s questions, users can also ask the chatbot simple questions.

Since we piloted mVAM in 2013, we’ve collaborated with InSTEDD, a nonprofit design and technology company that develops innovative open source tools for social impact. For three years, we used their SMS and IVR software to collect food security information. So when we wanted to delve into using chatbots, it was only natural that we reached out to them. Of course, not everyone we want to survey will have access to a smartphone. A large proportion of people using messaging apps at moment are young, urban, and male, introducing a bias to our surveys. But as smartphone ownership becomes more prevalent this won’t always be the case. This technology is really promising so we want to stay on top of it and see how it can be used for humanitarian purposes. As a first step, we want to use a chatbot to conduct a mobile food security survey on a messaging app. At the moment we are using Telegram because they have an API, which allows developers to easily build customized tools, but we are designing the bot so that it can be used on other messaging apps.

Here’s what our chatbot with InSTEDD would look like. Respondents are contacted on Telegram via their smartphones and asked a series of questions, about their food security and livelihood situations just like they would be by phone, SMS, or on our other mVAM modalities.

Check out our chatbot demo:

Why are we so excited about chatbots?

Chatting on a messaging app lets us collect new types of information. People can send our chatbot pictures, voice notes and geolocations that would enrich our food security analysis. As part of our analysis we ask people socio-demographic questions about things like their roof type, which give indications about a household’s economic status. Using the chatbot we can actually get pictures of these answers! We’ll literally see and hear about the situation on the ground and get to double check where these pockets of food insecurity actually are.WhatsApp-Image-20160721

It’s cheap! Not only does the chatbot have the potential to reach more people, the format is also cheaper than SMS, IVR and Live Calls.

It’s way more fun. The chatbot can process more complex sentences and respond more dynamically, letting the user drive the conversation. There’s a whole spectrum of things a chatbot can be programmed to do anything from a stilted, regimented conversation where users can only answer in a certain way, to natural language processing where users can chat as they would with a human. We think the technology just might not be there yet to meet our needs for a completely natural chat- we followed the Microsoft chatbot problem closely. However since we are only focusing on a specific topic we are opting for something in the middle- that allows us to get the food security information we need but also give users a natural, fun experience.

It lets us share more information. The chatbot can automatically read WFP’s food price database and tell people about the food and commodity prices where they live and give information about any big changes in the last few months. This database is so detailed that we can actually provide this information down to the market level in many countries!  Every time a new dataset is added to the original database, the chatbot automatically updates its price information, ensuring that local communities can access the latest information.

It’s flexible. The chatbot doesn’t have to just be used for prices. Users could ask WFP questions about our food distributions or programmes – whatever information we are able to insert in our database. This way we can provide a great incentive for people to complete our surveys, giving our beneficiaries a chance to give us feedback on the services we provide, and sending them a variety of information at a low cost.

Our chatbot is still a prototype, but we will let you know how our testing goes before we roll out our first pilot.