Built for you: Behind conversation topics and custom reports
Businesses are having more and more conversations with their customers, with lots of positive benefits – but the volume of conversation data can be hard to process.
In previous episodes, we’ve talked about how customer feedback and cross-team collaboration play a crucial role in the features and updates we build here at Intercom. This week, we see how they tie together as we go behind the scenes into one major launch from the past month. Or rather, two – conversation topics and custom reports.
To understand a bit more about the launch, we’ve gathered some of the folks behind it:
- – Senior Product Education Producer
- – Senior Product Marketing Manager
- Rati Zvirawa – Product Manager
- Tanya Sivo – Product Designer
They’ll walk you through the process of building these features and the guiding light that informed every decision along the way – an obsession over what success looks like for our customers.
If you enjoy our discussion, check out more episodes of our podcast. You can subscribe on iTunes, stream on Spotify or grab the RSS feed in your player of choice. What follows is a lightly edited transcript of the episode.
Thomas Creighton de Farias: Thank you for joining us today on this special episode of Inside Intercom. This is the third of the Built For You podcasts. Now, here at Intercom, one of our company values is obsessing over our customer’s success, and that means deeply understanding what our customers are trying to achieve.
In previous episodes, we’ve heard how many of the features and updates we build are based on customer feedback. We talked about the collaboration between various teams to bring those features from an ideation stage right through to shipping them and following up with fast iterations or additions.
“How do you make sense of and identify the trends in the many conversations your team is having with your customers?”
One of the things we’ve consistently heard through each episode is that great things can happen when you have a conversation. And while a single conversation or a feedback group might be relatively easy to handle, how do you make sense of and identify the trends in the many conversations your team is having with your customers?
Well, my panel today are no strangers to asking that same question in conversations they have with each other, as they have been instrumental in our recent release of custom reports and conversation topics. Today I’m joined by Orinna Weaver, Rati Zvirawa, and Tanya Sivo. Let’s start by going around the table and hearing about what you do here at Intercom.
Orinna Weaver: Hi, I’m Orinna Weaver, and I’m a senior product marketing manager here at Intercom, which can mean different things to a lot of people and companies. But I think, at its core, it’s bringing our new features and products to market.
Rati Zvirawa: Hi, I’m Rati, product manager at Intercom. I focus on the reporting area where we report on all things that happen at Intercom for you.
Tanya Sivo: Hi, my name is Tanya Sivo and I’m a product designer. I work with a cross-functional team that includes product managers, engineers, and also partners from research and analytics on different customer problems like conversation topics.
What does success look like?
Thomas: Awesome. Well, you’re all very welcome. I mentioned at the start our company values: obsessesing over our customer success. I’m wondering, what does that mean on a day-to-day basis for you all? What does it look like in practice? Tanya, we’ll start with yourself.
Tanya: Sure. I’d say it plays a huge role because I think the core thing I’m doing as a product designer is solving our customer’s problems so they can be successful. On a day-to-day basis, at every project, we always start with a problem. We gather as much customer feedback, qualitative and quantitative, as we can, to understand the problem space and define it right. This is the most important part, I think. Because if you don’t define the problem right, you won’t solve it.
“Shipping is the beginning of a conversation with our market. We want to listen to what the market is saying back to us”
Once we understand the problem clearly enough, we start exploring the solutions space, we test directly with customers, show them solution prototypes, et cetera. Once the solution is released, we again rely on feedback and data to see whether we solved their problem. And as our leaders always tell us, shipping is the beginning of a conversation with our market. We want to listen to what the market is saying back to us. And after that, we just start again with the next problem. Also, I’d say customer feedback is a common thread, and customer success is our unified goal at every stage of the project.
Thomas: How about yourself, Rati?
Rati: Yeah, I’d echo the exact same thing. As a product manager, I’d be working very closely with product design to make sure that we understand the problem. It’s really where we start everything at Intercom, to make sure we know that what we’re solving for is relevant for our teams. And from there, making sure that we’re validating them with customers all the way through the design process.
“I can’t imagine going through the process without talking to customers”
I can’t imagine going through the process without talking to customers both in the very beginning, just to get a sense of what the different pain points are, and then showing them the different approaches that we’re trying to take to solve those problems. Again, that’s one of the things that resonates with me when we do ship – it’s not the end, it’s almost like a new beginning, and we start iterating again over it. It really is a thread that goes through for us. It’s super key for us to have confidence that what we’re shipping to customers is adding value.
Thomas: And Orinna, you’re wearing a slightly different hat in terms of marketing. What does it mean for you on a day-to-day basis?
Orinna: For me, as a product marketer, this value comes out when I’m working on messaging. That’s kind of what comes to mind for me, thinking how a new product or feature makes our customer’s lives better, and if it solves their problems and leads to a better day-to-day experience. In crafting messaging, I’m trying to channel our customers and really dig into what success looks like for them.
Opening new possibilities with custom reports
Thomas: Awesome. Rati, your team has been working on custom reports, and I have to confess that anytime I hear the word custom being used to describe something, my general reaction is to think how much harder it must be to create something flexible or a bit of a blank canvas for customers. Can you tell us about the problem you’re trying to solve and how long your team has been working on it?
Rati: Definitely. Yeah, the custom word can be a bit scary when you hear it coming into the product. The problem we were trying to solve with custom reports, and I feel like there’s always been a thread of reporting feedback at Intercom over the past year, is that people just want more flexibility with the data that they have. As teams are growing more and more – more conversations, more teammates –, data becomes a key part of operating your support team at scale. And it was just becoming increasingly difficult for customers to do that at Intercom. We were seeing a lot of workarounds from support managers and teammates needing to export the data or feeling stuck with the capabilities we offered.
“It was a process of trying to find the right in-between, between custom and bespoke to give you the metrics you wanted with more flexibility”
And so we started to zoom in and understand when teammates are asking for more flexibility, are custom reports the right solution? Is it us adding in a little bit of data? So we were trying to explore the range of solutions from what we currently have, which is the bespoke reports where you can see the metrics that matter to you, or, going the other end, allowing you to define those metrics and expose them in the way that matters to you. And so it was a process of trying to find the right in-between, between custom and bespoke to give you the metrics you wanted with more flexibility.
Thomas: And of course, you mentioned data there, but people like to splice their data and present it in so many different ways to tell different stories, don’t they? How did you decide which particular chart types to add?
“Even though reporting is an established tool of sorts, we didn’t want to make assumptions on the value that we were going to add to customers”
Rati: Yeah, the chart types one was a big conversation with the team. With some reporting platforms you see, some can have a range of like 20 different chart types. And then some can have four chart types that you have to pick and choose between. We wanted to start small and learn, going back to this thread of listening to customers. Even though reporting is an established tool of sorts, we didn’t want to make assumptions on the value that we’re going to add to customers.
And so we thought a good baseline was to start with a few charts in our beta and hear feedback from customers to see where the key pain points were and what additional charts they wanted. And not just specific chart types, but understanding what they wanted to solve for with that view. And that really helped us narrow down the charts we began with in our first version, and over time, we can start growing those as we understand the value they’ll add to customers.
Thomas: For sure. I just want to come back to a point you mentioned earlier because there’s a certain expectation in terms of reporting. You kind of expect any software to have off-the-box reports. And for some of the more complex, you might expect to use maybe a third-party tool, using an API to get those, as you said, bespoke reports. On a scale of out-off-the-box and third-party reporting tools, where do you think custom reports sit?
Rati: Oh, that’s a good question. I think we’re somewhere in-between. We’ve got us off-the-box, which will always be valuable because you want to come into a tool and we make it easy for you to get started and immediately started getting value from Intercom as soon as you start having your conversations. Where custom reports sit is: when you’re starting to have a view that matters to you and your team, because we understand every team is a little bit different, some people care about channels, some people care about geo, some people care about both, and so custom reports allow you to continue using Intercom off the box at ease within the product, but with a lot more flexibility and access to your data.
This doesn’t replace the place of third-party tooling like Tableau and GoodData. Those products are fit-for-purpose tools built for you to combine not just Intercom data, but data from multiple sources. And that’s super powerful and still very relevant. Custom reports are another tool in your toolbox that you can use to get more insight.
Finding actionable insights with conversation topics
Thomas: Awesome. Tanya, I’ll jump over to you here. At Intercom, we have tags, we have conversation data, and now we have conversation topics. Firstly, could you tell us about conversation topics?
Tanya: Sure. I’d like to start with a problem we solved with conversation topics. We started with a very real customer problem. Support managers wanted to know the topics and trends of what their customers were saying to improve customer experiences by allocating their support teams’ efforts effectively. And before our conversation topics, they either didn’t do this at all or used manual tagging, which means that people on their team had to tag every conversation that came in. But there are a few problems with tagging – it takes a lot of time, there is a lot of human error, teammates tagging constantly, changes are cumbersome and forward-facing only, and categories are too broad and barely provide actionable insights.
Some of them also did ad hoc deep-dive analysis, which can be effective but are pretty labor-intensive and not scalable. So, as the solution, we made conversation topics. It uses machine learning to surface insights from your conversations while managing them at scale. And unlike other analysis platforms, topics are fully backed into Intercom, which allows you to proactively and retroactively apply topics to your conversations based on our different keywords in the messages. This saves teammates a lot of time tagging each conversation in the inbox, and you also can track granular trends accurately. You can analyze specific patterns easily and quickly. Using email, we can suggest trends you didn’t know about and help you uncover blind spots.
“Tags, conversation data, and now topics are just different ways to categorize and label conversations”
Thomas: Amazing. Is there a hierarchy of how tags, data, and topics work together?
Tanya: Yeah, it’s a great question. All three – tags, conversation data, and now topics – are just different ways to categorize and label conversations. Each method has its benefits, depending on what’s important to you, but in general, we recommend using a combination of all three. Topics are ideal for trend reporting, like following what your customers are talking to you about. Conversation data attributes capture more granular details that require humans to understand, such as how urgent a conversation is. Plus, they’re perfect for workflow management. And tags are great for marking specific messages in the conversation and for an individual’s categorization needs. So, by using all three, you can create this comprehensive categorization system, which opens all kinds of insights for you.
Thomas: And of course, the striking thing about topics and suggested topics is that visual chart. For those of you who haven’t seen it, the chart is, I want to say, a bubble graph of your top topics based on the number of conversations that your customers are having about each. Then, you can apply what I would call a heat map, in the sense where you can look at your CSAT scores and responsiveness for each. What was behind that visual representation?
Tanya: Topic visualization, or this bubble chart, is just basically visualizing your topics for you. And it’s supposed to help customers to quickly read insights because you can easily compare your topics with each other or with your suggestions using different metrics you just listed. You can compare them by volume or you can break them down by first response time, time to close, and customer satisfaction. So we just help you understand your topics and get insights more quickly.
Thomas: I’m always interested in features like this where it’s super creative. It’s not directly born from customer feedback. And I mean that in the sense that nobody has asked for a bubble graph. People may have described a problem in a certain way. When you create something like this, do you create a whole new set of problems that you might have to solve down the line? Does it open a can of worms?
Tanya: Yeah. I think topic visualization was kind of bad for us. We just wanted to help customers get insights quicker. Initially, topics were just like this table of topics and KPIs. Under our topic visualization, on this bubble chart, there is a table with your topics and different KPIs like volume response time and time to close, et cetera. And what we wanted to achieve with this visualization was helping customers understand and get insights faster. So there weren’t any particular requests for that since the whole thing was new. It just like us also we wanted to help customers here. Does it make sense?
Getting the messaging right
Thomas: Orinna, customer reports and conversation topics were released alongside each other. On the one hand, you have reports that are very obviously being built on customer feedback, as described by Rati there, and conversation topics are born out of a bit of creativity. Are there challenges trying to marry the two of those together in your messaging?
Orinna: Yeah, absolutely. I mean, they’re definitely worse than challenges. The R&D team here ships really fast and really frequently, as anyone listening to this is, I’m sure, well aware by now. But we on the product marketing team need to be careful not to over-message our customers. Of course, we love to tell everybody about all the things that we ship, but we need to give our customers some kind of breathing room to focus on just the most important things.
“The main consideration for me was giving each feature some of the limelight, but not getting lost in an overarching reporting story”
Custom reports and conversation topics were built by different teams, but they were going to be ready around the same time. So this gave us a really great opportunity to bundle these two features together into a single announcement. As you’ve heard, these are two different features solving distinct customer problems, but they were related enough. They’re both about reporting, they’re both about getting more insights. Some from your conversation, some from other parts of your business like team performance or whatever it might be. But the main consideration for me was giving each feature some of the limelight, but not getting lost in an overarching reporting story.
As you’ve talked about, we had a great opportunity with these two, in particular, because custom reports were a really long-standing and top feature request for a long time. But conversation topics were a bit more unique and innovative to our space. So this joint announcement was an opportunity to deliver customer value with a commonly requested feature and bringing something unique to the table alongside it. It ended up being a great opportunity.
Thomas: In your messaging, there’s always a balance between focusing on the problems that are being solved versus stating what the feature actually does. How much do you focus on the problems that can be solved? Or does that end up highlighting them in the first place?
“From the messaging front, we’re focused on the why rather than the how”
Orinna: We really do focus on the problems to be solved here at Intercom. And that starts before the feature is even built, with the product manager. That’s what they’re rooted in. And so when we start on our messaging on the product marketing side, we’re rooted in the customer problems. It’s hard not to focus on that, to be honest, but we want to help folks understand why they should be interested and what adopting a certain product or feature will help them achieve, not just how it works. There are help docs and places that they can go to get a better understanding of how it works or what it does, but from the messaging front, we’re focused on the why rather than the how.
Thomas: And of course, when you’re getting to the stage where you’re getting feedback and feature requests at scale, it’s not always possible to build all of them. Is there an element of managing expectations that needs to happen?
Orinna: Yeah. To some degree, you’re right. Unfortunately, we can’t build everything our customers want, even if we wanted to. We just try to center around our target customer here and balance things, like I mentioned, to include some feature requests and some things that we truly believe our customers would benefit from, even if they’re not saying the exact words.
Starting small and building up
Thomas: Rati, I’m just thinking in terms of right now, what kind of feedback are you getting about reports?
Rati: We launched two and a half weeks ago or three weeks ago, and the feedback’s been positive, which is great, but then digging a little deeper as more customers have started getting into the product and feeling it out and living with it for the past two and a half weeks, we’re starting to see feedback trickling in, in terms of just interaction with the tool, for example, filters. It was one of the early things we were scoping to figure out how far we should take filters and how complex they should be to begin with.
We started with more basic filters, and we’re starting to hear customers wanting more from our filter capability. And, of course, more data. I feel like that’s the core of reporting is always asking for more data. It’s been good to start to see trends of what kind of data customers are asking for.
“One key thing we do as part of starting with the problem is defining what success means for the product”
Thomas: For sure. I’m always curious about how you measure whether reports are solving the issues that initially led to the development of custom reports.
Rati: That’s a great question. At the beginning of this chat, we talked about starting with the problem. And one key thing we do as part of starting with the problem is defining what success means for the product. And that’s a mix of both quantitative, so you can measure it independently of opinions or what customers may say, and then qualitative, on actual feedback we get for customers.
Throughout the beta, and even now as we’ve launched, we’re both looking at the metrics that we’ve set out in terms of customers adopting the product very early on, and at engagement as well. There’s a certain level of engagement that we baselined from the beta we’re also expecting when we go live.
And then, to give those numbers color, we’re now listening in from customers for feedback. Can they actually find the answers they were looking for? A big one on why we even embarked on customer reports was so we could help support teams with large volumes and larger teams activate questions and see the metrics that matter to them. Asking customers directly, can they see the metrics that matter to them? Are they able to answer their questions?
“It’s the balance between both qualitative and quantitative data. That’s how we’re validating if we’ve solved the problem”
And it’s still the guiding question that directs us for feature requests coming in. When someone’s asking for data, we try to connect that to understand if it’s for solving the key problem of trying to help them manage their teams? It’s both, it’s the balance between both qualitative and quantitative data. That’s how we’re validating if we’ve solved the problem.
Thomas: For sure. Tanya, are there any problems or customer feedback that your team is working on right now?
Tanya: With conversation topics, we are changing our customers’ already-established workflows. And even if it is an improvement for them, they still need time to try, test, and adopt topics. So now, we are gathering feedback on adoption, mainly quantitative data, and figuring out what’s next for topics. Actually, I would like to use this as an opportunity to highlight we really value our customers’ feedback. So please share with us your thoughts, requests, questions, and comments about topics. We really appreciate it.
Coming up soon
Thomas: Orinna, are there broader themes of problems that you’re digging into right now in marketing that you can tell us about?
Orinna: Yeah, I can. On product marketing, the work we’re doing is often a bit more forward-looking. Right now, I’m thinking through messaging and how we’re going to bring something new to market, not related to reporting, but I can give you a little teaser on some of the messaging and go-to-market plans.
“I’ve been working with customers to get a better understanding of their needs and their pain points and ultimately, what a good integration looks like for them”
We’ve heard from our customers that our existing Salesforce integration could use a little bit of extra love. This is a feature born out of feature requests, as we were talking about before. And so we’ve been making space to dig more into that. We have an integration now but we’re working to make it better. I’ve been working with customers to get a better understanding of their needs and their pain points and ultimately, what a good integration looks like for them. That’s what I’m digging into now. You will see a lot more about that coming in the coming quarters. So stay tuned.
Thomas: I am sure you have just made lots of people very happy, talking about that.
Orinna: I hope so.
Thomas: Orinna, Tanya, Rati, thanks again for joining us on this podcast. I’ve really enjoyed our conversation. We talked about Built For You and how Intercom builds features based on feedback. It’s really exciting.
But here, we have tools that will help you identify the trends and the reasons why customers reach out to you. And everyone wins when you listen to that feedback. It’s a source of inspiration for your product teams and their roadmap. And for your customers, they’ll continue to realize the value of your services and products if you’re constantly evolving and chipping away at those issues. Yes, great things can happen from a conversation. And when it comes to scale, even bigger things can happen when you get the insights into those conversations your team and customers are having daily.
