Intercom’s Fergal Reid and Ciaran Lee on the making of Resolution Bot
To celebrate the launch of our most ambitious machine learning project to date, this week's podcast features a conversation with one of the brains behind the operation.
Anyone who’s tangled with customer support knows: getting an answer to your question is one thing, but actually landing on a useful resolution to your problem can be another.
Two years ago, we introduced Answer Bot to address our customers’ most important questions. It did a terrific job, but we knew we could do even better. This week, we’re proud to unveil its successor: Resolution Bot, Intercom’s upgraded support chatbot that scales your team by answering 33% of common questions. For more info on what Resolution Bot can do for you, check out our launch page.
With such a game-changing product to share, we also wanted to bring you the backstory from the folks who have been in the trenches, bringing Resolution Bot to life. We sat down for a chat with our own Fergal Reid, Principal Machine Learning Engineer, to learn why Answer Bot had to evolve past simply answering questions to focus on solving problems at scale. We also talked through everything from the rise of deep learning to the future of automated support to the reasons why automation is a little like washing clothes.
Short on time? Here are five quick takeaways.
- The technology for Resolution Bot has been waiting in the wings, but the user experience has been risky. Until now. With a bot that can jump in and proactively give users the answers they’re looking for, we think there’s something magical here.
- In the past two years, there’s been a huge leap forward in the accuracy and the predictive power of the neural networks. Additionally, we have more data available to us that allows Resolution Bot to hone in on subtle distinctions in wording that a customer might throw at it.
- Companies often have a good idea about what they want their bot to answer, but they’re not always sure where to start. That’s why we built a curation tool to help you search how your customers have asked questions in the past.
- Resolution Bot is available in 7 different languages. To avoid the large maintenance burden that can come with multilingual ML systems, we’re relying on a neural network that can learn multiple languages in an end-to-end way.
- Making predictions about the future of automation can be tricky, but we aim to move more and more towards personalizing these automated support experiences and predicting what the users want as early as we can.
If you enjoy the conversation, check out more episodes of our podcast. You can subscribe on iTunes, stream on Spotify or grab the RSS feed in your player of choice. What follows is a lightly edited transcript of the episode.
Ciaran Lee: Fergal, welcome back to Inside Intercom. This time, it’s all to mark the launch of Intercom’s Resolution Bot. Before we delve into that, can you give a really quick background as to your role at Intercom?
Fergal Reid: I lead the Machine Learning team at Intercom. I joined Intercom about two and a half years ago. I think I was our first dedicated ML hire. Happily, we’ve brought several machine learning products to market and we’ve had a good response, and the team has grown, and we have four full-time people from a range of disciplines.
One thing we’ve really tried to do is to be able to do the full product loop ourselves. It’s really important for us to not get blocked by other teams pushing stuff to production and to be able to have tight feedback loops as we build machine learning products. That’s where it’s great to have a senior production engineer on the team helping us all the time.
Introducing Resolution Bot
Ciaran Lee: On to Resolution Bot, this is the next generation of our Answer Bot (which we introduced in 2018), our intelligent chatbot that automatically and instantly tackles common customer questions. Tell us what’s changed there.
Fergal Reid: It was a lot of work building the first generation of Answer Bot, because when we started that project, we really didn’t know what we were going to get ourselves into. We saw that there were a lot of bots across the industry that were starting to get good.
Intercom, on record historically, has been a little bit skeptical of bots, right? Because maybe five years ago, the first generation of chatbots were very over-promised. We had all this media hype about how bots are going to do everything. But the tech just wasn’t there yet – and maybe more importantly, the product experiences weren’t there yet. Even if we amazingly had the tech five years ago, there was a lot to figure out in terms of what actually makes a good bot product or a good automation product. It’s been easy for about 30 years to make something that would match keywords and try to give an answer from a bank of answers. That’s really old tech, but we haven’t had chatbots and automation actually answering people’s questions for very long. Why is that?
“The first few times we saw this in our alpha test, we thought, ‘Wow, there’s something magical here.'”
One reason is the rise of messengers. As an end user, I love using a messenger for support. I don’t like being on the phone, having to wait for support. I can’t do anything else when I’m waiting. And if I send an email, maybe I’ll never receive an email back – or it’ll take 24 hours, they’ll be a clarifying question, and I’ll ask something else. Messengers are good like this, and automation works really well with messengers, because you can have that rapid back and forth. Putting a bot in the user interface space is relatively new. And we have the machine learning technology to build a bot that can come in and can help and not look stupid.
All these things seemed to be converging about two years ago when we started to build Answer Bot, but as we actually built it, we had a whole ton of unknowns. We didn’t know whether this was actually going to be able to help anybody. And we had to build a whole lot of prototypes and really fast development cycles to put very early alpha versions of what eventually became Answer Bot in front of end users.
And once again, what I thought went well for us in that project was that we were literally about six or eight weeks into development when we actually had a scrappy, early version of Answer Bot answering people’s questions on Intercom. And we were like, “Wow, this is really working!” Someone came along and asked a fairly complex question that maybe the customer support person wouldn’t know without talking to a specialist. And the bot just jumps in and gives the end user their answer. The first few times we saw this in our alpha test, we thought, “Wow, there’s something magical here.”
So we built this product that eventually became Answer Bot. It took us about 10 months to build it and ship it. And when we got maybe three or four months in, it became obvious to us that the end user experience here isn’t really risky anymore. So we launched it, but then we got to these progressive waves of uncertainty. We weren’t sure if it was going to deliver real ROI for people. Is it just going to come in and answer their questions, or is it actually going to resolve end user issues at scale? So we spent a lot of time toward trying to figure out how we could really measure the success of a product like this.
Ultimately we developed what we called a “resolution metric.” The idea was: “Okay, the user has asked the question, and the bot is giving them an answer. And the bot will give them buttons like, ‘That helped!’ or ‘Wait for the team.’” That allows us to look at the times when the user explicitly says it helped or when they’ve just gotten their answer, and they left. Then the bot will prompt them and say: ”It looks like you’ve gotten your answer. Do you need any follow-up support with the team?”
“In the past two years, there’s been a huge leap forward in the accuracy and the predictive power of the neural networks that the process, natural language”
That’s how we define the resolution metric. Since then, in production, the actual number of resolutions this bot has generated has exceeded our early expectations. We’ve done more than 600,000 resolutions across a wide range of customers. Each one of those is an actual time when an end user has come along and asked a complicated, hard question, and the bot has given them an answer that resolved it, and they haven’t had to wait anymore for the team to help them. We’re pretty happy with that.
That’s why we’re starting to evolve the product and focus it more and more on resolutions. We’re even changing the name of what used to be Answer Bot. We’re now calling it Resolution Bot, front and center, to really focus on resolutions and times when it specifically helped end users.
The rise of deep learning
Ciaran: One really interesting thing that’s different now is that for this next major iteration of the product, we have so much more data. Previously, when we used it for ourselves and a few beta customers, it was a really limited dataset. But now we can A/B test different strategies against one another. Having all this rich data and context and history allows us to make more informed decisions about where we should go next.
Fergal: Absolutely. One lovely thing about getting to work at Intercom and build machine learning products here is that we have a huge number of customers. We care a lot about them, and we get a ton of data. So when we build a product, what makes it hard is that it can’t work for just one or two customers, it’s got to work for a broad section of our customer base. But the upside is that we get data on a whole lot of different businesses, and we can really go and use that data to tune these bots.
You mentioned A/B testing. With this shift to Resolution Bot, we’ve moved to an entirely new backend that actually powers the core algorithms of the bot. Two and a half years ago, we were watching the rise of deep learning, and we took all the top cutting-edge networks that were out there, and we benchmarked different network architectures, and none of them were just quite there yet. We could use them as components, and we got a lot of mileage out of using word vectors, which tells you which words are similar to each other. We got tons of mileage out of using them as components of Answer Bot, but none of the neural networks were really ready for prime time at that point.
But in the past two years, there’s been a huge leap forward in the accuracy and the predictive power of the neural networks that the process, natural language. So one of the things we’ve done for Resolution Bot is we’ve switched over to a neural network-based architecture. And it’s really good. It’s more accurate across a wider range of customers. It’s able to do things like look at parts of a conversation. In the past, if someone said, “I have a question about Salesforce,” the product would focus on the word Salesforce and see if that matched. But now we’re able to parse the difference between someone saying, “I have a specific question about Salesforce,” versus “I need some general information about Salesforce.”
“We have a huge focus on design, on ease of use, and on building products our customers can actually use”
These are very subtle distinctions, but it’s a big change in machine learning that now we can hone in on those subtle distinctions. That’s very exciting. Part of the reason we can build these products and be confident shipping them is that we have data that can help us fine-tune these machine learning models, and we can be confident they’re actually going to move the needle for our customers.
Over the past few months as we geared up to this launch, we’ve been doing a large-scale testing program to actually check how well the new engine will perform against the old one. And the amount of data has enabled us to really become confident in identifying individual use cases and domains where the new model needed tuning to exceed the performance of the existing model. It’s great to be in this data-rich environment to be able to quantitatively understand those product features.
Ciaran: Can you tell us about some of the implications of being able to simply do some of this backend work more quickly and efficiently?
Fergal: One of the great things about these neural network architectures is they take a ton of time to train, but once you’ve trained them, they’re computationally fast. Being able to look at “What are the answers that match the question the end user has asked?” has a bunch of benefits. It reduces any latency of how long it takes the bot to respond. That’s a small deal (because computers are pretty fast) but it means the new neural network approach is literally hundreds of times faster than the more traditional ML approach we had before. And that makes it way faster on the backend.
We have this feature in Answer Bot, which we internally call “suggested common questions.” When you’re in the Answer Bot curation tool, we try and find the biggest groups of questions that people are asking about. So maybe one of your biggest issues is, “I need to reset my credit card.” Or “My credit card was stolen, and I need to reset my password.” We want to be able to put those answers in front of our customers. Finding questions that are similar to each other and clustering them together works way more efficiently for us. Then we can use it on more and more of our customers’ historical questions and give them better recommendations. It’s exciting.
Products customers can use
Ciaran: Before we built Resolution Bot, if our customers wanted these types of capabilities, they would have often had to build them themselves. But there may have been barriers to that: maybe lack of skills or fear of the unknown or just data quality. How do you feel Resolution Bot mitigates against these for our customers?
Fergal: This cuts to the heart of one of the reasons why I love working at Intercom, which is that we have a huge focus on design, on ease of use, and on building products our customers can actually use. That’s not to say that our competitors don’t think about these things, too, but Intercom traditionally has had a product that works for you if you’re a very small one-person company and also works for you if you’re a large company. That has really forced us to get good at design. We haven’t been able to assume that our customer is going to have a team of data scientists to set up a bot. And if you look at a lot of the other tools you can use to build bots, a lot of them involve setting up rules: if the end user says X, then the bot says Y.
The problem with rules-based approaches like that is that, once you have 10 or 20 things your bot can do, and it starts to get a lot like programming. One thing we’ve tried to with our Resolution Bot is to enable that natural language processing without needing those programming and data science skills. It’s taken a lot of time and a lot of work to build that sort interface that really works for our customers.
We did several rounds of user research – with some painful times on of the design and machine learning ends – to really try and make a product that works there. But now we have the data that actually shows our customers are managing to set up loads of answers and get resolutions. When we talk to them, it’s somebody from customer support or customer success who’s doing this frequently, someone a little more on the technically sophisticated end of the team. It’s not a programmer, and it’s not a data scientist. And that’s something where I’m really proud of.
“It’s not enough that the bot needs to know that one of the common questions is about resetting your credit card. It needs to be told all the different ways people can ask”
Ciaran: I remember it being a bit of an emotional roller coaster the first time we were shipping this product: how hard it was to make something that is easy to use in this space. Initially, we built this bot that would answer customer’s common questions, and we assumed people would know what their common questions were (and that we would know those common questions). But we found out that we didn’t. It’s hard.
Fergal: This is quite a controversial time internally in the development of the product. When we built the first alpha version, it was just for Intercom’s own customer support team. We built this super crude, basic, ugly clustering tool that would attempt to figure out what Intercom’s common support questions were.
If you have a big enough hammer, everything looks like a nail. We decided to throw some data at the problem and then see if we could use machine learning to try and discover those questions. It kind of worked, but it was just something that was suitable for use as an internal tool. When the time came to actually develop the product in full production for real, there were a lot of different opinions internally about whether we actually needed to build something like this. Did we need to give people that leg up?
The story of automation always starts off slowly, and it feels really weird at the start”
Some people at Intercom were like, “Well, our customers are going to know their top 10 questions.” Our customers are answering questions all the time, or they’re managing a support team, and they have a pretty good idea what their most common questions are, but they can’t necessarily write down – if you give them a blank screen – all the different ways their users ask those questions. But that’s what the bot needs.
It’s not enough that the bot needs to know that one of the common questions is about resetting your credit card. It needs to be told all the different ways people can ask. If someone says, “Hey, my account is locked, and I can’t spend any money,” you as a company have to recognize that’s a question about credit cards. Finding all the different ways people ask those common questions takes time.
So we built this curation tool that would enable you to search how your users have asked those questions in the past. That was really the breakthrough moment in the development of the curation experience of our NLP bots, and it really enabled people to actually go from staring at a blank canvas with a pretty good idea of what they wanted the bot to answer – but not really sure how to proceed from there – to actually getting started and configuring big bot installs that would do a lot of resolutions and answer a lot of people’s questions quickly.
Making Resolution Bot work for everyone
Ciaran: Another thing I find fascinating is that Intercom may not be an ideal candidate for Resolution Bot. We have a broad product with lots of features, but we have customers with simpler products. I’m thinking of airlines, power companies, and the like – more well-understood business with a smaller surface area, at least in terms of the product their customers interact with. I’m sure we see different Resolution Bot performance across these different types of customers.
Fergal: This is absolutely the case, and it’s something that risked misleading us a little bit at the start. Our first customer is often ourselves, because it’s just easy to run the beta, and we can have a really close relationship with our support team when we’re doing the earliest, riskiest product development. One thing to be careful of is that Intercom is a big product. And our customers are pretty sophisticated. They’re spending a lot of time on that product. I might have a relationship with my utility company, but I’m not spending my working day in that electricity company’s product. I probably don’t even know it that well. It’s very easy for me to get lost in their web interface. And so if I’m an end user, I might say, “I have a meter reading, or I have a problem with my bill.”
“We’ve heard from our customers that they see a big opportunity with bots to help their customers help themselves”
The electricity company probably gets that same question tens of thousands of times a day. But at Intercom, we don’t get many questions tens of thousands of times a day. When we were doing this internal version of the first round of our bots, we were like, “Gosh, it takes an awfully long time to get high coverage to the point where you’ve curated 30% or 50% of all your incoming questions.” And then we discovered, when we did beta with some of our customers, that the conversational space is different, because you have to really think about the biggest groups of questions in the space. Does the single biggest topic people ask about cover 1% of your total inbound volume of questions? Because the return investment of a bot is going to completely depend on those dynamics.
Internally, we’ve actually quantified for a lot of our different customers, looking at their data and its distribution of that space. It’s absolutely something that varies from customer to customer. We always see that creating more answers will always increase your coverage, so we always encourage our customers to do that. But the reality is that the level of return of each answer you create is just going to vary from customer to customer.
“I really care about giving end users the answers they want in a hurry”
Some customers have relatively easy domains, and some customers have harder domains. And it’s really been a journey for us to understand that. And we have to be careful: anytime we make a change to our bot technology, we have to make sure it works for all of our customers and it never degrades the performance of any one of these customers who has a very different domain.
We’ve discovered that some customers have an easier space in terms of natural language processing. There are some customers where particular words strongly identify what the end user is asking about. Let’s say your product is called Resolution Bot, and you have a customer that comes along and asks a question about a Resolution Bot, you’re pretty damn sure what they’re asking about. But another Intercom product is Messages. When someone comes along and asks an Intercom conversation about “messages,” that’s harder because a lot of people talk to us about messages in a lot of different contexts.
Some of our customers have these relatively easy spaces that are just a great fit, and our bots will work almost out of the box. But for other customers, it’s just harder. They’re just going to require more time and more effort. You can still get there, but you need a bigger volume of questions, a bigger volume of inbound conversations each month for the ROI calculation to make the same amount of sense. So it absolutely varies from customer to customer.
Helping customers help themselves
Ciaran: We’ve heard from our customers that they see a big opportunity with bots to help their customers help themselves. What kind of customer problems where you focused on solving with Resolution Bot
Fergal: There are a whole lot of answers to that. To be completely honest here, I really care about giving end users the answers they want in a hurry. I’m still really fascinated by the magic we saw in the first round of alpha what was then Answer Bot two years ago – which is just that experience where someone comes along and they ask a question, they get the answer, and it just solves the problem for them. I think that’s amazing. We’re on this big, long journey to be able to help more and more people in a quick, snappy way.
At the same time, there is just an economics argument here. We want to help our customers scale. We want them to be able to give the high-quality support people associate with Intercom on a bigger and bigger scale. Automation is a key lever to do that. We’re not there yet, but we’re on a journey there and we’re investing in it. If we can get to the point where, for every question you’ve been asked more than five times, a bot can come in and take a shot at it, that will be amazing. Your support team is going to be much more efficient. It’s going to spend way less time answering the same rote stuff again and again, and the nature of support is going to change.
“It’s about removing that drudgery, and it’s going to make messengers more cost effective than email”
It’s going to change from a world where your support team is sitting there, getting the same questions over and over, doing a mechanical job – and instead, it’s going to shift to a role where the support team is about automation. It’s about removing that drudgery, and it’s going to make messengers more cost effective than email. It’s going to make messengers more cost effective than a phone conversations, because that messenger is just really well-suited for fast interaction with an end user to try and resolve their problem. I’m really motivated, personally, by the end-user experience, but there’s something that’s good there for both the end users and the businesses on the support team.
Ciaran: It’s so interesting that when automation like this works well it’s actually better for everyone. An end user can ask a question and get an answer instantly (or quicker than a human would be able to do it). And the support team can concentrate on the longer tail of conversations that actually really require human interaction.
It’s like how we used to go into the bank, queue up, wait to see a teller, and provide documentation before you could get your cash out. Now, of course, there are ATMs everywhere, open 24 hours a day. They’re quick, and it’s simpler and better for everyone. And even more interesting is that ATMs are becoming obsolete because we can pay with our phones, and it’s becoming a better, more frictionless experience all around. Done right, automation can be a big win for everyone.
“Automation of any sort is really hard and time consuming and difficult to get right at the start”
Fergal: I was worried that when you were telling the story about the bank teller and the ATM, there’s going to be some section of our audience that’s thinking: “What’s an ATM? What’s cash?” You’re touching on something interesting there, which is that the story of automation always starts off slowly, and it feels really weird at the start. But then it gets to the point where it’s just so ingrained that you don’t really see it anymore.
Take your washing machine. The first generation of washing machines was really dangerous because electricity was mixing with water. It just wasn’t good. And now the washing machine is just part of every household in the developed world, and you just put your clothes in, and it makes your clothes clean. That used to take hours of labor. Automation of any sort is really hard and time consuming and difficult to get right at the start. You see this adoption curve where it always takes longer to really nail that end user experience than you initially think.
But then you just get to some inflection point in the curve, and suddenly it’s everywhere. I’ve no reason to think this won’t be the case with these automation products that we’re building here. It’s a long, hard fight for us and for our customers to increase the percentage of conversations – the percentage of end users – that an automated product can help. It takes time, and it takes effort. But then in the end, you just get to some critical inflection point where you’re like: “Gosh, this is now resolving 30%, 40% of my questions. I couldn’t live without this.”
“That’s been the history of automation everywhere. It’s a curve that takes much longer than you think”
We hope we’re leading the industry here – and we’ve certainly got claim to be a leader here – but there’s a whole industry working on this on in different ways, and it’s working. It’s been successful, and consumer behavior is going to change. We’re going to move from a world where people are looking at this bot and thinking: “Oh yeah, that looks useful. Maybe I should interact with that,” to a world where people are like: ”Why do I have to wait? This is an easy question. Where’s the bot?”
I believe in that, and that’s been the history of automation everywhere. It’s a curve that takes much longer than you think. People are like, “The system is going to break.” They’re dealing with a customer, and something super unexpected that’s never been seen before is going to happen. Or one of their customers has a terrible bug and requires per customer resolution. We’re never going to get to the point where we have 100% of questions automated, but I do think we can get to the point where the majority of common things are just answered immediately. And we’re getting better and better at that.
Ciaran: A great thing that Resolution Bot has to its advantage is a free shot at the start of the conversation to see if it can help. And if it doesn’t, if the person presses “Wait for the team,” or they continue typing, which simply cascades through to a human-to-human conversation, just as you would expect and doesn’t take any longer. That’s a really nice thing to have. It allows us to push the envelope. Take a few more risks knowing that there’s this great fallback.
Fergal: That’s a nice segue into one of the product features we’ve recently launched: this idea of auto suggested answers before the end user even hits return. If we detect that we can answer the question if they’re still just writing, we’re going to start prompting the user with these answers that appear just above the composer.
We’re really taking our inspiration here from a few years back, when you’re typing on your Android or iPhone. You see these suggestion words and sometimes even phrases now. If you use Gmail, it actually offers autocomplete phrases before you finish typing. That’s fast becoming what end users expect. It’s fast becoming the norm of any typing interface that, “If you can help me before I even finish what I’m typing, please do that.”
“That is the gold standard for where to put an automated system. We’re looking for somewhere where it doesn’t hurt too much if we get it wrong, but it helps when we get it right”
We really see that users are expecting and using it more and more across a whole lot of different tools. With autosuggest, we’re bringing that to our bots. Resolution Bot now features this. Hopefully you’ll see that and you’ll think: “I don’t even need to ask this question. There’s this little thing here telling me about reset password. I’m just going to click that.” They click that, and then suddenly they’re in this automated support flow. That feels good to us.
We also hope we can suggest answers more aggressively here than we would if we have to wait until after they press return. Part of that is just screen real estate. We have the opportunity of giving them a little inline menu of two or three answers. Maybe they change their query. If you’ve asked a question and then you press return, the bot has to be pretty confident in what it presents you as the reply. We’ve always wanted to make sure it is a good user experience, so we’ve always been reluctant to dial the bot up so that it just gives you the best thing it has, even if it’s not that useful. We think that if bots and automation interfaces just to make their best guess, even if it’s completely irrelevant, they reduce user trust over time.
Ciaran: It can feel very clunky.
Fergal: Exactly. We don’t want to do that, because then the next time the user comes along and asks a question and the bot has an answer that actually will help them, they will ignore it. We don’t want to do that. But with Resolution Bot and with autosuggest, these are suggestions. They’re not taking as much of your attention. We did a lot of user research on this before we shipped, and we discovered that end users – just like those suggestions on your Android keyboard – will ignore them if they’re not that relevant, but they’ll see them if they are relevant. That is the gold standard for where to put an automated system. Because machine learning systems are never perfectly accurate, so we’re looking for somewhere where it doesn’t hurt too much if we get it wrong, but it helps when we get it right. We really think we’ve done this with autosuggest for Resolution Bot.
Navigating multiple languages
Ciaran: It’s exciting. We have global customers who understandably want to provide support in their native languages. We added six new languages, which means Resolution Bot is now fluent in seven. Tell us how you approached building this. What were some of the challenges you faced?
Fergal: Right. Yeah, that’s a good question. One thing we were really worried about was, if we build this multilingual version, are we going to have to maintain that? Let’s say we support 20 or 30 languages eventually. Are we going to have to maintain a different machine-learning system for each of those 20 or 30 languages? We could build it. It would take us time, but the hard part isn’t building it. The hard part is maintaining the system, because each language is going to have its own idiosyncrasies. Then customers are going to ask: ”Hey, why is the bot doing this? This doesn’t quite make sense in my native language.” We’re going to have to iterate on that and patch it and improve it. Over time, these systems are going to diverge from each other. That’s a very big maintenance burden, and if we go into that world, we have to do it with our eyes wide open because machine learning systems are not like normal products. The maintenance burden can get very severe.
“Instead of us having to manually code in the intricacies of each individual language, we have a neural network that encodes them end-to-end”
For Resolution Bot, one of the things that we did was switch over to a neural network-based approach. The particular family of neural networks we’re using actually can learn multiple languages in an end-to-end way. That just makes it much easier to maintain multiple languages going forward. It also means that, instead of us having to manually code in the intricacies of each individual language and each individual idiom, we have a neural network that encodes them end-to-end. Much more of the complexity and the maintenance burden is hidden in the neural network. We can A/B test, and we can make sure it’s working well for all our languages, but we don’t have to individually code each individual language in great detail.
We still have to patch small things that go wrong in any language. We had to do that during our beta to get the accuracy standard up to where we wanted it, but it’s just easier to maintain it on an ongoing basis. The languages we selected and decided to support are the next six most common languages that Intercom customers get questions in after English. We will likely add more over time.
Ciaran: I can’t wait to see how it performs. One thing I found really interesting is looking at German customers and some of the idiosyncrasies of their language space. One thing that struck me was just how it seems to me there are a lot more different words in German than we have in English. I actually don’t know if that’s true, but from just visually looking at questions, it looks there are a lot of unique, complex phrases that are constructed perhaps out of a few different words. I can’t help but wonder, is German a language that is well suited towards automation like this?
“There are little steps we can take along the way for more automation. As we show we can produce good results and good experiences and solve people’s problems, we can start dialing that up a little bit more”
Fergal: Yeah. I did a few years of German in school. I wouldn’t claim to be a speaker, but I definitely studied the language for a few years. You get these large compound words to describe concepts in German slightly more than you get overloaded words in a language like English. It’s been really interesting in our testing that sometimes it actually improves the accuracy. Big words that identify distinct concepts are a relatively easy place for machine learning algorithms. There’s been a lot of learning for us as we’ve built the multilingual version of our bots. One of the things we’ve noticed is that there are some languages where NLP works even better.
The future of automated support
Ciaran: Looking to the future, then, this feels like a pretty important juncture in how we use and build for machine learning. What do you think the next 10 years hold for automated support?
Fergal: The nature of this field is that any speculation is fraught. We touched on it a little bit earlier in the conversation. We have a certain resolution rate, a certain self-serve percentage that we’re doing at the moment. We want to increase that for our customers and for our customers’ users. There’s a whole ton of stuff we’re prototyping and experimenting with all the time (which I can’t really go into in detail because it’s just so early), and 70% of these things never make it out of prototype stage. But we’re always looking for ways to help our customers discover what it is that will move the dial for them if they teach it to the bot.
“We wanted to layer in some automation on top of existing human-to-human conversational conventions. We want to be careful that no one ever gets stuck in bot jail”
We’re always doing more work on integrating the different parts of our product. We want to make it so that you can have bots that use ever-increasing amounts of user context to figure out what you can help your customers with. We’re going to dig into all those areas more and more. We’ve just shipped our most advanced targeting ever for Resolution Bot out of any of our automated bot products. We want to move more and more towards personalizing these support experiences and predicting what the users want as early as we can. We’re doing a lot of work in that area at the moment.
Ciaran: It’s so interesting. For me, if I were to think about a general trend that I can predict for us, I think of a couple things. One is just that we started off very deliberately conservative. We wanted to layer in some automation on top of existing human-to-human conversational conventions. We want to be careful that no one ever gets stuck in bot jail.
If the bot replies, and the end user just continues typing a message, that then cascades through to a human-to-human conversation. There are little steps we can take along the way for more automation. As we show we can produce good results and good experiences and solve people’s problems, we can start dialing that up a little bit more.
“It would be great to live in a world where we detect not just your common questions, but actually your common flows, and we suggest those to you…That’s the direction we’re going, but it’s going to take work”
When we built this initially, we thought, “Okay, we’re going to automate frequently asked questions, simple things that can be questions that can be answered with a piece of text.” But actually, the more you look at support interactions, the more you realize that those frequently asked questions are actually just a subset. They’re the first layer of things you can automate with a piece of text response. I’m really interested in thinking about what if Resolution Bot could not just reply with a piece of text or link to an article but kick off a little app or a custom bot workflow or automate a high-level process that typically does still require two people to interact with.
Fergal: Absolutely. We have this Resolution Bot product at the moment, which will give you a single answer. As you say, that the answer can be a product tour, it can have an app inside it – but it is still very much like, “There’s a single issue here, and we’re answering that single issue.” We have other products (like our custom bots), where if you want to build this branching flow, you can do that. It’d be great to unify these two things. It would be great to live in a world where we detect not just your common questions, but actually your common flows, and we suggest those to you. But there’s a whole lot of complexity there around inferring this from people’s data and then getting them set up with that. That’s the direction we’re going, but it’s going to take work.
Ciaran: It’s exciting. I can’t wait to see what we do in this space. Fergal, this has been a really fun chat. Thank you so much. Really appreciate your time.
