Test Complete: 6 things we learned about online surveys

IMG_3030

Photo: WFP/Catherine Clark

We told you in a recent blog post that we were testing online surveys in Haiti to get an idea of urban food security perceptions. This was very new for us so here’s a quick follow up on what we learned:

1. Get ready to experiment! For online Surveys (and pretty much anything else we do), this is our approach. We don’t have that much experience designing community level questions so we decided to throw in a variety to see what worked and what didn’t. Our question asking respondents to name two main difficulties faced by poor people probably worked the best so we’ll keep that in. However, we also asked if poor people’s quality of life had improved, worsened, or stayed the same in the past 3 months. Almost everyone said “worsened”  but we weren’t sure if we were really capturing a trend. Given that our respondents were generally pretty wealthy, they might automatically put this response when they are asked about poor people.

2. Re-iterate! Your questions will need tweaking. In our case, street food is an important part of urban Haitians’ diet so we asked a question on the price of a plate of spaghetti. The survey format meant respondents couldn’t free type any answers so we gave them price ranges. It turns out these were a bit too wide as almost all the responses were in one range. However, now that we know the approximate range, in the next round we can put specific numbers in and allow respondents to select an exact value.

3. Tailor your questions to your respondents. You might not actually know that much about them the first time so think about who your respondents will be and what they can realistically answer. Again, a lot of this comes through trial and error. We asked about wage rates for a male manual worker and for a female domestic worker, but our
respondents were mostly young, male and well-educated so they had no idea about how much these people get paid. Age gender

We’d also really like to know about migration so we tested a question. But daily migratory flows in Haiti aren’t very big so back to the drawing board with this idea.

4. Use partial responses. People will drop out of the survey as it goes on. However, we still wanted to use all of the information they did give us, especially since this was a qualitative survey.

5. Be patient. Expect a steady volume of responses, but it could take some time to reach your target. We averaged about 30 completed questionnaires a day. Our target was 750 completes so it took about 5 weeks.

IMG_3242

6. Reaching poor areas is always a challenge. It’s still difficult to get enough responses from poor areas. We knew that going in, and sure enough, in Haiti’s biggest slum, we did not have many responses.

As always, we’ve learned a lot while experimenting. We’re looking forward to revising our questionnaire and giving online surveys another go!

Leave a Reply

Your email address will not be published. Required fields are marked *