Who You Callin’ Cupcake?

We did something very fresh this month with the open ended question we have been asking in our surveys. Typically we do simple word counts and sift through to find responses that are ‘extra-juicy.’ This time we did all that too, but also gauged perceptions using the Pattern sentiment analysis algorithm developed by the Computational Linguistics & Psycholinguistics Centre at University of Antwerp. For a given sentence, the algorithm returns the ‘polarity’: a measure of how positive/negative the statement is on a scale of -1.0 to +1.0. You can learn more about it here.

For example the statement “The food situation is a bit encouraging as we move positively out of Ebola’s way” has a polarity of 0.215—a modestly positive sentiment given a modestly optimistic statement. Conversely, the highly pessimistic statement, “The food situation is very bad. People need food, People always begging for food.,” is scored to be a -0.91. The algorithm isn’t perfect, as its mainly picking up on keywords and unable to understand the nuance, but on an aggregate it at least words directionally, i.e. it tends to at least classify negative/positive with decent accuracy.

Histograms of the results are displayed below.

Figure 1_Sentiment Polarity Distribution for Liberia

Figure 2_Sentiment Polarity Distribution for Sierra Leone

From the histograms you can see that slightly more positive responses were received in May for both countries. An increase in the mean is observed in both Liberia (+0.016) and Sierra Leone (+0.034), but the improvement is only significant in Sierra Leone (p<0.05) as detected by a Mann-Whitney test.

What is fascinating is that this totally corroborates with other indicators we have been tracking! Both countries experienced average net improvements in rCSI—our main indicator of food resource hardship—of 0.5 points. But Liberia is relatively worse off than Sierra Leone, with slightly higher levels of hardship. Hence for Liberia experienced a 3.4% drop for May whereas Sierra Leone had a 3.65% drop. It may just be coincidence–without further examples, we simply cannot tell—but the sentiment improvement dynamic, at least in this case, directly connects to the trend and magnitude of actual underlying indicators.

The same statistical test run by region reveals a statistically significant improvement in sentiment for Bong County, Liberia (p<0.05).  Far and away Bong, County Liberia experienced the largest decrease in rCSI for May amongst all places surveyed, going from 15.53 to 13.8, a 11.14% drop! This should at least makes you pause and think…we may be on to something….

Leave a Reply

Your email address will not be published. Required fields are marked *