Introducing our Chatbot

bot pictureAn important part of our job at mVAM is to stay tuned into the developments in the rapidly evolving mobile technology sector. Lately we noticed two main trends: first, more and more people in the places we work are using smartphones and chat apps to communicate, leading us to think about how to better reach out to this segment of the population. Second, chatbots, robots that live in chat applications, are all the rage and have a big potential to contribute to our work.

What is a chatbot?

A chatbot is a computer programme that uses artificial intelligence to interact with users through a messaging service in a way that is designed to seem like a conversation. We’ve been experimenting with ways to expand our capacity for two-way communication, i.e. contacting local communities but also hearing back from them. A chatbot provides a friendlier, more responsive way to interact with people by letting them communicate more naturally, in a “chat” as the name implies. As well as answering the chatbot’s questions, users can also ask the chatbot simple questions.

Since we piloted mVAM in 2013, we’ve collaborated with InSTEDD, a nonprofit design and technology company that develops innovative open source tools for social impact. For three years, we used their SMS and IVR software to collect food security information. So when we wanted to delve into using chatbots, it was only natural that we reached out to them. Of course, not everyone we want to survey will have access to a smartphone. A large proportion of people using messaging apps at moment are young, urban, and male, introducing a bias to our surveys. But as smartphone ownership becomes more prevalent this won’t always be the case. This technology is really promising so we want to stay on top of it and see how it can be used for humanitarian purposes. As a first step, we want to use a chatbot to conduct a mobile food security survey on a messaging app. At the moment we are using Telegram because they have an API, which allows developers to easily build customized tools, but we are designing the bot so that it can be used on other messaging apps.

Here’s what our chatbot with InSTEDD would look like. Respondents are contacted on Telegram via their smartphones and asked a series of questions, about their food security and livelihood situations just like they would be by phone, SMS, or on our other mVAM modalities.

Check out our chatbot demo:

Why are we so excited about chatbots?

Chatting on a messaging app lets us collect new types of information. People can send our chatbot pictures, voice notes and geolocations that would enrich our food security analysis. As part of our analysis we ask people socio-demographic questions about things like their roof type, which give indications about a household’s economic status. Using the chatbot we can actually get pictures of these answers! We’ll literally see and hear about the situation on the ground and get to double check where these pockets of food insecurity actually are.WhatsApp-Image-20160721

It’s cheap! Not only does the chatbot have the potential to reach more people, the format is also cheaper than SMS, IVR and Live Calls.

It’s way more fun. The chatbot can process more complex sentences and respond more dynamically, letting the user drive the conversation. There’s a whole spectrum of things a chatbot can be programmed to do anything from a stilted, regimented conversation where users can only answer in a certain way, to natural language processing where users can chat as they would with a human. We think the technology just might not be there yet to meet our needs for a completely natural chat- we followed the Microsoft chatbot problem closely. However since we are only focusing on a specific topic we are opting for something in the middle- that allows us to get the food security information we need but also give users a natural, fun experience.

It lets us share more information. The chatbot can automatically read WFP’s food price database and tell people about the food and commodity prices where they live and give information about any big changes in the last few months. This database is so detailed that we can actually provide this information down to the market level in many countries!  Every time a new dataset is added to the original database, the chatbot automatically updates its price information, ensuring that local communities can access the latest information.

It’s flexible. The chatbot doesn’t have to just be used for prices. Users could ask WFP questions about our food distributions or programmes – whatever information we are able to insert in our database. This way we can provide a great incentive for people to complete our surveys, giving our beneficiaries a chance to give us feedback on the services we provide, and sending them a variety of information at a low cost.

Our chatbot is still a prototype, but we will let you know how our testing goes before we roll out our first pilot.

Will IVR work for food security surveys in a Somalia IDP camp?

As the mVAM pilot project enters its final quarter, the team is focusing on finalizing all planned activities, while documenting learning that will allow us to scale up with a strong evidence base. This month’s highlights include some hands-on work with the team in Somalia, and the launch of a comprehensive review of our activities.

The Somalia IVR coming along
A key question we have is whether interactive voice response (IVR) surveys are user friendly enough to be used in Somalia with the vulnerable groups that WFP works with. The major issue to resolve was ensuring the IVR system Verboice in our Galkayo field office was fully operational. Although we had been able to place some IVR calls, the system required dedicated attention to be fully operational In mid-January, Marie and Lucia headed to Galkayo to meet with the team for a troubleshooting mission.
Thanks to late night remote support from Gustavo at INSTEDD, bugs were ironed out, and we were soon able to get our first complete IVR surveys using a Somali language questionnaire. The team in Galkayo was trained on how to place the calls and will be following a plan to scale up IVR calls in February. Meanwhile, we will continue collecting food security data through calls placed by our operators, a modality that has worked well to date.

Featured image

Making the IVR operational: training underway

During the visit, a key discussion took place regarding appropriate incentive rates. In both DR Congo and Somalia, respondents receive a token of appreciation from WFP in order to promote participation in surveys. The amount we provide –USD 0.50 per call – is equivalent to 5 minutes of airtime. While our respondents in DR Congo seem thrilled to receive this amount of airtime, the question of increasing the incentive has come up in Somalia, where it is perceived as too small.

There seem to be three schools of thought in the team. Some believe the incentive should increase in Somalia. Others think that increasing call attempts and better sensitizing respondents should be sufficient to ensure good response to our surveys. Others still question the principle of providing an incentive to people who might already receive food assistance from WFP.

In coming months, we will be making sure respondents are called more often and that the messages they receive tell them about the importance of their participation. We would then consider working with a larger incentive in the future should response rates not improve.

Launching the mVAM review

A critical milestone of the project is capturing and sharing learning. In order to proceed with scale-up strategically and responsibly, the review of the mVAM pilot in Somalia and DRC is now ongoing. Professor Nathan Morrow, who teaches at Tulane University’s Payson Center, is leading the review. Nathan has written extensively about technology in the humanitarian world, including a review of Ushaidi’s contribution to the 2010 earthquake response in Haiti.

In January, Nathan traveled to Goma, DRC, to meet with WFP staff, key stakeholders, and beneficiaries residing in the Mugunga 3 IDP camp to hear from them how the pilot was going, document their questions and concerns. He will also be chatting with staff in Somalia and the three-EVD affected countries to learn how they view the project.

The review will include documenting the demonstrated potential of mVAM at a larger level; noting areas of improvement that can ameliorate our technology; and explore how mVAM’s technology fits within the larger humanitarian sector’s work. Results will be available in the spring.

Can we use SMS for food security surveys in a Congolese IDP camp?

Blog entry originally posted in December 2014 on the Humanitarian Innovation Fund website.


Featured image

Response rates to voice calls, DR Congo Source: WFP

Almost one year into data collection, we are now fairly confident that live voice calls, placed by operators, are a good way to stay in touch with people in the extremely vulnerable communities we work with.  Since January 2014, we have been able to conduct monthly rounds of phone surveys typically reaching between half and two-thirds of selected respondents, while collecting data of good quality. However, it’s not yet clear if either IVR or SMS offer the same advantages in our pilot contexts.

SMS: cool tool, wrong setting?

This month, we attempted to understand whether SMS surveys would work in an IDP camp. Using SMS is attractive, because it is low-cost and easy to automate using free software.  While we have had good results with SMS(link is external)when running simple national or province-level food security surveys, we have yet to evaluate the tool’s suitability in a high-vulnerability refugee camp setting.  In November, two Rome-based mVAM team members, Marie and Lucia, travelled to Goma to attempt to do just that. They helped the team in Goma organize a simple food security survey involving face-to-face interviews, live voice calls and SMS. The data collected from this exercise will allow us to understand the strengths and weaknesses of these different survey tools.

Featured image

Residents in a Congolese refugee camp responding to text messages

In order to run the SMS survey, we used Pollit(link is external) a free, open-source tool. It is easily accessible with an internet connection and requires only minimal hardware to function—a computer, a mobile phone and an internet connection. The tool was developed by In(link is external)STEDD, the same company that developed Verboice(link is external), the programme we are using for IVR calls. In the future, Pollit may allow us to periodically run short SMS surveys in-house, bringing a lot of flexibility to our field teams.  During the Goma test, Pollit proved to be a simple and flexible tool. It was easy to set up and worked smoothly during the six days of data collection.

However, response rates to SMS surveys turned out to be low, particularly compared to voice calls and face-to-face surveys. Our enumerators reported that people in the camp are not used to using the SMS function on their phones. They typically communicate using voice calls, due to low literacy and habit. In some cases, the phones people owned were broken or had dirty screens, making it difficult to read and reply to the messages we were sending.  These issues, however, do not prevent us from using voice calls, which seem to be the preferred modality amongst respondents in DR Congo. This seems to suggest that we should stick to live calls for Mugunga 3 camp, and use SMS questionnaires in other settings. We are now analyzing the data we collected in Goma in order to answer other questions we have, which includes comparing data quality for the different survey modes. We’ll be sure to share those insights later.