How do your customers feel about AI chatbots?

What do people really think of customer service chatbots and the new generation of AI bots transforming the space? We conducted user research to find out what people really think.

We’re in the middle of an incredible wave of AI innovation, and as a result we hear lots about what ChatGPT can do, and which industry is going to be upended first.

While the world focuses on the technology itself, far less attention has been given to the way the users of these chatbots feel about them.

There was a Gartner study, conducted between December 2022 and February 2023, before LLM customer service chatbots became widely available, which revealed that a mere 8% of nearly 500 B2B and B2C customers had used a chatbot during their most recent customer service experience. Of those, just 25% said they would use that chatbot again in the future.

Now that the world of chatbots is transforming, we wanted to answer the important question: “How are people’s feelings towards customer service bots changing with the arrival of LLM-powered chatbots?”

Uncovering attitudes to chatbots

We conducted a lot of user research as part of building our new AI chatbot, Fin, and spent some time considering the issue of user attitudes to these bots. Along with our teammates Maia Bridi and Sophie Woods, we wanted to discover the most common attitudes to chatbots, and how those perceptions are changing now that AI chatbots have arrived.

For this initial research study, we interviewed a number of end users to get a sense of their attitudes to chatbots and examined their reactions as they interacted with Fin.

“People have encountered chatbots of wildly varying quality from various companies, and those poor experiences have only increased the general preference to connect with a human support representative”

We discovered that:

• The chatbots people have dealt with up to now are quite unpopular (as the Gartner study appears to confirm). End users tend to hold negative sentiment towards bots, mainly because bots are seen as an obstacle that needs to be overcome in order to reach a human agent, who will actually be able to resolve their issue.
• People have encountered chatbots of wildly varying quality from various companies. Those poor experiences have only increased the general preference to connect with a human support representative straight away. In this sense, many chatbots are considered similarly to phone trees – a hurdle to be navigated, rather than a source of assistance.
• Until now, bots’ limited natural language skills have made them frustrating and clunky to interact with – people felt they had to use the right keywords to be understood by the chatbot.

AI is changing perceptions of chatbots

The arrival of ChatGPT is quickly changing this perspective among end users. The wave of hype has cut through with most people, and dramatically changed expectations around the performance of chatbots.

“I would love for all the chatbots to be like this because I don’t need to speak to humans”

Already, people are much more confident that an AI chatbot will understand their queries, and ultimately will be able to help them. As one of the research participants put it to us after interacting with Fin, “I would love for all the chatbots to be like this because I don’t need to speak to humans. I get the answer that I want and that’s it.”

It’s not all entirely positive though – there are concerns that new AI chatbots will make it impossible to access human support, which underlines the importance of clarifying that customers can still ultimately reach a human support representative if required.

Being able to convey emotions and frustrations is important for end users, and they want to know there’s an option to deal with a person – often there is an underlying need to feel heard. Frequently, people feel like a human support agent will be able to show flexibility in resolving their issue in a way that a chatbot can’t.

AI improves conversational flow

The messenger format is inherently conversational, but existing chatbots have never been able to hold convincing conversations – no matter how well-crafted, their understanding of natural language queries is limited and their responses are always pre-scripted. So while interacting with a chatbot has all the dynamics of a conversation, the fluency is going to be slightly off.

“While new AI chatbots don’t always match the feeling of conversing with an actual person, they feel far more natural than what preceded them”

ChatGPT has transformed that dynamic – the stilted patterns of existing chatbots has been replaced by the comparative linguistic naturalness of generative AI.

We suspect this is one of the key reasons behind the change in perceptions – while new AI chatbots don’t always match the feeling of conversing with an actual person, they feel far more natural than what preceded them.

AI bots are exceeding expectations

As a result, AI chatbots such as Fin are exceeding customer expectations around bot interactions – having been conditioned by the limitations of prior chatbots, AI bots are providing better resolutions in a more natural way.

That’s not to say everyone is ready to go with a bot-only customer service experience. The comfort in knowing that there are humans who will be available to assist with your issue is very real. Another participant described it like this: “I guess the apprehension going into it is whether it is actually going to be able to help me, or how do I get to a live person?”

“This is a fast-moving environment, and people are just getting used to interacting with these AI chatbots”

It was clear from the research that knowing that Fin can put you through to a human if your question is not answered increases customer trust – it provides a sense of confidence and reduces the risk of frustration.

This is a fast-moving environment, and people are just getting used to interacting with these AI chatbots, so no doubt customer expectations will continue to adapt and develop as use of these bots become more common. One thing is very clear – this is going to be a fascinating area of research for us over the coming months and years.