Are chatbots better than FAQs? New vaccine research sheds light

By Tanya Newhouse, Co-founder at Clevertar

I enjoy telling the story of how a chatbot helped resolve a family dispute (in my favour, of course). Late last year, when my husband was sick with a cold one morning, SA Health’s COVID-19 Assistant convinced him to 1. stay home, and 2. get a COVID test, and 3. don’t even think about going to work.

I still remember him sitting at the breakfast table, snuffly, coughing, and complaining of a sore throat, but dressed and ready to leave the house (as we Gen X’s are enculturated to do), even in the midst of a pandemic. All while I pleaded with him to stay put. Unfortunately, none of my protests or attempts at smooth persuasion worked to change his mind.

Finally, we resorted to the internet and found Zoe, a trusted presence on the SA Health website. After a short conversation she made it pretty clear that my husband wasn’t going anywhere until a COVID test came back negative.

Nice one, Zoe.

Recently an industry colleague sent me this French research from January 2021 showing the effectiveness of a chatbot to promote positive attitude towards COVID-19 vaccination, increase people’s intention to vaccinate, reduce hesitancy, and even recommend positive action to their friends. This confirmed my personal experience. It seemed that a COVID-19 assistant could do more than answer queries about getting tested; it could even increase people’s willingness to get vaccinated, reducing burden on phone lines while having a positive population effect.

But why employ a vaccination chatbot in the first place? According to the authors, this tool was chosen in response to a literature review showing that traditional information sources like leaflets and FAQs aren’t persuasive compared with a discussion group for vaccination messages. They believe that this is because there’s no ‘interlocutor’ to answer the very specific questions and objections people have, leaving them with similar views to where they started. Worse, it may even ‘backfire’ by reinforcing their views or doubts, creating stronger negative views than existed in the first place.

So although leaflets and FAQs are easy to disseminate, they don’t do their job very well. As the authors note,

‘changing minds at scale is a difficult endeavour’.

But changing minds has become fundamental in the fight against COVID-19. Government authorities around the world are aiming for herd immunity, and since most countries are not mandating the vaccine, a very large proportion of their populations needs to be willing to take the jab. Twice, in most cases.

At the same time, authorities are fighting against misinformation, vaccine hesitancy, or even a lack of motivation. It’s crucial that their public communication strategies work.

To test the hypothesis that a chatbot could play the role of interlocutor, engaging in fruitful discussions with people about the COVID-19 vaccination to reduce hesitancy and promote positive intentionality, the researchers created a chatbot with 51 questions to address the main concerns identified by the public.

They split 614 participants into a control and intervention group, identifying three predominant attitudes:

1. ‘No, I will not get vaccinated’
2. ‘Yes, I will wait some time before getting vaccinated’
3. ‘Yes, as soon as vaccine is available for me’

The idea was to test which, if any, groups changed their attitude. It was important that they tested the chatbot for each cohort, not just as a whole.

They found their hypothesis was confirmed: participants held more positive attitudes towards the COVID-19 vaccine after the experiment task in the Chatbot condition than in the Control condition. Perhaps even more importantly, ‘this relation held among the third of the participants initially holding the most negative attitudes towards the COVID-19 vaccines’.

For those in the chatbot group, there were a few other results worth noting:

• 37% increase in participants holding positive attitude
• 20% decrease in participants saying they wouldn’t get vaccinated
• 45% of participants reported having tried to convince other people (typically, between two and five) of their position on the COVID-19 vaccines
• No backfire effect

These results were statistically significant, and have led the authors to conclude that:

‘a properly scripted and regularly updated chatbot could offer a powerful resource to help fight hesitancy towards COVID-19 vaccines. Besides its direct effect on vaccine hesitant individuals, the chatbot could prove invaluable to pro-vaccination individuals, including professionals looking for information to use in interpersonal communication with vaccine hesitant individuals.’

So how do you change minds at scale? The evidence for COVID-19 vaccine suggests to try a chatbot. See how we built SA Health’s chatbot in 6 days to respond to COVID-19.