Built for You: How customer feedback informs what we ship
This week’s show is a very special panel episode where we’re hearing how conversations with our customers have shaped our product.
At Intercom we believe great things can happen when you have a conversation. Which is why customer feedback is a big deal to all of our teams – it’s really integral to what we do and how we build. So if you’ve ever asked yourself – how does my feedback add up to the features and updates that we build? Then this podcast has the answers.
Taking part in the panel are (in order of appearance):
- Thomas Creighton de Farias – Senior Product Education Producer
- Mathew Cropper – Senior Product Manager
- Jen Murphy – Senior Product Researcher
- Maria Gutierrez – Senior Director of Engineering
Between them they break down the process and share their collaborative way of working and how the customer voice remains central throughout the process – from initial feedback to a feature launch.
If you enjoy the conversation, check out more episodes of our podcast. You can subscribe on iTunes, stream on Spotify or grab the RSS feed in your player of choice. What follows is a lightly edited transcript of the episode.
Thomas Creighton De Farias: Thanks for joining us today on this special episode of Inside Intercom. This is the first in a series of quarterly panels that we’ll be hosting as part of Built For You.
Customer feedback is a big deal here at Intercom. It’s integral to what we do, and oftentimes it may not be apparent how big of a role it has played when people see a launch or features are released. So if you have ever asked yourself, “How does customer feedback manifest into the features and updates that we build?” we’ll be answering that by chatting with the people that make it happen: the R&D team members who take your feedback and turn it into the tools that you use. So let’s hear from our panelists about who they are and the work they do at Intercom. Maria, let’s start with you.
Maria Gutierrez: Hi, I’m Maria Gutierrez, and I’m senior director of engineering and co-site lead for our London office. In London, we focus on improving our support solutions. That means everything from leveling up our teammate inbox experience to the evolution of our bots. I’m also part of the R&D leadership team, so a big focus for me is how we can improve our operations as an organization and how we continue to invest in the growth of engineers across the board.
“We’re a company that’s really all about conversations”
Mat Cropper: Hi, I’m Mat Cropper, and I’m a senior product manager. I work with designers and engineers on our mobile SDKs. Those bring Intercom’s features into our customers’ own mobile apps. We also look after Intercom’s own conversations apps for Android and iOS.
Jen Murphy: Hi, I’m Jen Murphy, and I am a senior product researcher. I work with the teams that build Intercom’s support products. I’m based in our Dublin office, and I’ve been at Intercom about three years now. In that time, I’ve had the opportunity to work with many different product teams – from the teams that built our messenger to those that work with our marketing features.
How we get feedback
Thomas: There’s lots of experience in this virtual room! How do problems land on your desk in the first place? Mat, maybe we’ll start with you. How do you get feedback from customers?
Mat: We get a lot of feedback. As you can imagine, we’re a company that’s really all about conversations. We have a lot of them. Part of the way that we uncover our customer’s needs is through things like direct feedback that we get through our messenger and the one-to-one conversations that our sales team might have. There’s our forum, social media, all kinds of different places. We’ve got things set up so that teams see all that feedback in real time when it comes in, and we regularly create customer voice reports. And that’s basically where we take all of those conversations and spend time trying to understand them. Then we try to identify trends and insights that we can use.
When we analyze all those conversations, what we’re looking out for is a bunch of things:
- What is it our customer is trying to achieve?
- What products and features are they trying to use to achieve it?
- Where’s the friction?
- What’s stopping them from being able to do that?
- What is the impact of solving that particular problem or piece of feedback?
And we tag all of that and aggregate it so we can see all those trends and themes. And then aside from all of the direct feedback people volunteer, we also do a bunch of qualitative and quantitative research when we want to go really deep into a problem. We’ll also proactively look at features we’ve built and say: “Did we really solve the problem when we did this? If not, why didn’t we do that?”
Mat: A good example of this would be Mobile Carousels, which is a product we launched back in June. It’s a really unique way to communicate with people who use your mobile app. And that actually didn’t come from feedback people were giving us around the features we have in our mobile apps and SDK today; it was actually feedback we got for Product Tours, which is a product that we launched the year prior and lets you show people around your website, your web app, that kind of thing. We basically got loads of direct feedback where people were asking for Product Tours to be made available for mobile apps.
“All of that feedback and data tells us that we’re delivering on the actual needs we uncovered when we originally explored the problem”
So you’ve got to ask yourself, “How did we get the leap from that to Mobile Carousels?” Our process at Intercom always starts with really, deeply understanding the problem. Here, we’ve got a lot of feedback about Product Tours and mobile apps. There’s definitely a strong signal that there’s something people need. What we do next is we spend a lot of time talking to those customers to understand more: What is it they’re trying to achieve with this? And if they had Product Tours for mobile, how would they use it? And what would they expect to see from it? Those kinds of things.
When we did that, what we learned was that people have real needs around onboarding new users and better in-product messaging. And they thought that Product Tours was going to be the answer to that. But in reality, Product Tours was only ever going to solve a small portion of the problem for those customers. So what we did from there was say, “Well, what can we do better?”
We did a bunch of research, prototyping, evaluation, exploration, testing – all of those kinds of things. And then that led us, in the end, to Mobile Carousels, which is the product we launched. And there was a really close collaboration with a group of customers all the way through that. They were involved in the early exploration work, and they were involved in taking a look at our prototypes and the experimentation. A bunch of them did a beta with that product before we launched it as well. Now that we’ve launched it, all of that feedback and data tells us that we’re delivering on the actual needs we uncovered when we originally explored the problem. They’re seeing things like better retention, better engagement, those kinds of things.
And how do we take in that feedback about product? Or is it face value? We would have built Product Tours for mobile. But actually through really digging into it and spending time with people, what we got was a much more positive impact in the long run for all of those customers that gave us that feedback.
Understanding the customer’s voice
Thomas: Jen, I might bring you in here as well, because Mat has mentioned they’re working on qualitative and quantitative research to get deeper into a problem, and I have this image of the research team, whose skillset is a mix between a caring therapist on one side or maybe a seasoned investigative journalist on the other. Because we get this real mixed bag of feedback from our customers: some that is quite specific and others that might be more of a general commentary on the world at large. Where do you even start with the feedback that you see?
Jen: A researcher’s role in this whole feedback process is almost like a facilitator of questions so that we’re ensuring we’re investigating a problem and answering the right questions. Have we prioritized the right lines of investigation? Is this a problem we already know exists? Is this a new problem? Is this even the right problem for us to understand right now?
I wonder what questions we should be asking that will help us define the problem well enough so we can solve it. As Mat mentioned, we get signals about problems in specific areas all the time from inputs like our customer voice report. One of the challenges here – and where research can help – is asking if there are themes and patterns across all of those signals. Is there an underlying problem that is triggering requests for different features? How do all of these needs tie together? What’s actually the real source of this feedback? Is there something bigger happening?
One thing we saw recently that pointed to something bigger was actually triggered by the pandemic. The world we live in now obviously has changed – we all know this – but we thought how the pandemic was affecting our customers. We observed support teams that were experiencing spikes in conversation volume and really struggling to scale their human support to meet growing expectations. So we had questions about how it was impacting support teams and how they work, and we wanted to identify opportunities for Intercom to help.
“It gave our product teams a deep understanding of what they aspire to be and maybe what role Intercom can play in getting them there”
We worked with our customers to deeply understand their environment, the challenges they’re facing as a support organization working remotely, how their team was performing, how their team was feeling when they’re working like that and how they’re thinking of solving some of those problems in the future. What we realized from these conversations was that support teams were actually so overwhelmed. They were just trying to keep afloat. They weren’t really sure where to start to reduce that workload on their team. This was really impacting their team morale and their team performance. They’re trying to balance team efficiency with great customer service and are faced with this choice between those two.
We heard from support leaders that wanted to shift from this reactive, unpredictable way of providing support to a more proactive and controlled approach – to providing that exceptional customer-support service without sacrificing speed, efficiency and even your team’s morale. We talked to customers that we could see were already making moves to be more proactive. We learned from them, and this ultimately led to the building of our conversational support funnel.
Not only did listening to our customers’ challenges – their worries, their struggles, all the concerns that they have currently – help us build a framework for them to use, but it also gave our product teams a deep understanding of what they aspire to be and maybe what role Intercom can play in getting them there. It helped us figure out what solutions we should prioritize – like ticketing workflows to drive team efficiency or helping them to set clear expectations with their end users through more prominent reply times in the messenger.
Really understanding the real underlying problem behind all those signals helps you understand how everything needs to work together, and it helped us figure out what we should prioritize to help our customers be successful.
Mat: One of the really interesting things that Jen alluded to there is actually the impact of being able to scale those insights. So you have Jen and the research team who were doing all this awesome stuff to understand and rationalize all of the signals that we get from our customers, their problems and what they need. But the great thing that they do as well is make sure it’s not siloed in one place.
The fact that every team in Intercom benefited from that particular bit of research is one of the real superpowers of a great piece of research. Something that an R&D team might do is make sure that people can absorb that widely, because it means that people have a much more holistic view of the feedback and how it might impact many different areas of products. That’s a really fantastic thing we do.
Engineering shapes the solution
Thomas: Maria, I’d like to bring you in here at this stage, because your team is tasked with building the solutions. Presumably that means that your team needs to collaborate a lot with both Jen and Mat’s team. Where does engineering fit into this process?
Maria: Absolutely. Alluding to what Mat was saying there, one of our principles that we follow in engineering is that we should help shape the solution. We never blindly, as engineers, execute on requirements that are defined by others. What is expected of me and of all the engineers in our organization is that we deeply understand the value of our work and to do so is critical.
We partner from the beginning with research, with product managers, with designers and with analysts to help really understand those customer problems and to design those solutions that we then know can implement effectively and that will ultimately deliver value to our customers and solve those problems that Jen is making reference to. Sometimes engineers participate in research calls; we only have so many researchers. If, sometimes, we really need to answer some questions, you will see engineers jumping in, and – obviously with some guidance from our experts and our researchers – they will be able deal with some of those pieces of trying to capture some of that customer insight. But they may also lead some of the betas if the product manager is too busy, or they will work with our customer service team as well when customers raise issues with assistance solutions to really understand what the problems are.
This is important for us because it helps us make more informed technical decisions and trade-offs. When we know the implications of a problem for our customers, it’s a lot easier for us to know what we need to prioritize and what’s going to be most impactful for us. It also allows us to assess risks better and earlier if we understand those problems from the beginning and we are involved. Later on in the process, we can see from far away, with a little bit more time, what problems we may have to deal with as we start implementing those solutions.
“It’s easier to make commitments as a team when we are all involved in coming up with a plan”
It also builds empathy with our customers, and it reinforces the fact that we are all in this together. And it’s easier to make commitments as a team when we are all involved in coming up with a plan. And everybody in the team is excited about the problems that we’re solving because we all understand them very well. It also allows us to leverage everyone’s expertise and experiences in the team when it comes to solving those problems, and that’s always a win.
Thomas: That’s really interesting. I would have thought that the process was very much about each team handing over to the next team as soon as they’ve contributed or looked at a problem with a particular lens. But by the sounds of it, from the get-go, everybody is involved and trying to look at that problem and understand that problem from the outset. Does that cause particular challenges?
Maria: It does cause a lot of challenges. Intercom is a platform, and its strength is how any solution we build works well with other parts of the product or even solves different use cases. Achieving that is not as straightforward when we have a lot of people involved. Especially as the product is becoming more powerful and sophisticated, it really requires really strong collaboration across teams and across different organizations as well. So to guarantee that we deliver joined-up workflows and really excellent quality and operational availability, it’s critical that everybody works well together and provides the relevant context to other folks proactively – and that they loop in peers before making some decisions and commitments.
We really need to lean into some of our principles and processes to achieve this. You could argue that you have some overhead, but we believe that having some overhead ultimately results in much better solutions for our customers and for us as an organization. To mitigate those problems, in engineering, for example, we like familiar solutions and technologies that we understand well. So we consider ourselves quite conservative when it comes to using different technologies, and we reuse different solutions as much as possible to remove that complexity.
“What we’re really purposeful in doing is making sure that all the way through everything that we’re doing, the voice of the customer is really there and central to what we’re thinking about”
We try to always keep things simple because we know that there are multiple people that maybe haven’t been that close to the implementation of the solutions, and they need to get up to speed and really understand what’s going on easily. And we need to be able to maintain those things in an easy way. But obviously, sometimes things do go wrong.
We have great examples of where that collaboration has been really, really excellent. Everything has worked really well, but sometimes – we’re all human – we miss parts, or we make assumptions that maybe don’t turn out to be right. And that has an impact on all of the teams, whether it is our peers with implications for product managers, designers; or the teams that are working on similar parts of the product to the ones that we are changing. So we need to make sure we’re always supporting each other and learning from each other at all times. And that we always assume the best intentions in the work that everybody’s doing.
How research overcomes challenges
Thomas: Mat, you want to jump in there?
Mat: What Maria’s saying is 100% right. To extend the thought a little bit, what we’re really purposeful in doing is making sure that all the way through everything that we’re doing, the voice of the customer is really there and central to what we’re thinking about.
In the very beginning, it’s not just about engineering, design, product and research getting together. We’re talking to customers. But on top of that, we bring in voices and collaboration from folks on our customer support team or from the sales team – people who are talking to our customers day in, day out. They’ve got a real insight that’s really valuable. On a recent project, we were exploring a problem and trying to start thinking about solutions. We had folks from all of those different parts of Intercom all in one place, collaborating and sharing their insights and helping us to refine things down. And that’s one of the key things.
Like Maria said, there’s the collaboration between the various teams, the various disciplines, but also making sure you’re bringing in those insights from different perspectives of people who we’re talking to our customers day in and day out to make sure that it continues to stay real for us when we’re thinking about these kinds of things.
“When we do this kind of research, we’re balancing priorities that sometimes might actually conflict with each other”
Thomas: What would you say are the biggest challenges for you and your team as part of this process?
Mat: One of the really unique challenges we have at Intercom is that our products fall into two categories of stuff that our customers use. There are things like our conversations inbox. And then there are things that end users use, like the messenger on your website.
When we’re exploring a problem, we can speak to our customers to help us refine our understanding of a problem, but that’s a lot more challenging to do with end users. We generally can’t speak to them easily and directly. So we need to understand both the outcomes that our customers want to achieve and those that end users are trying to achieve as well. We need to balance all of that to build something that’s really effective.
For example, with Mobile Carousels, one of the things that our customers wanted was to be able to increase retention through better onboarding of new users in the mobile apps. And that’s great, but what do the users of their app want to achieve? And how might solving that problem actually be damaging to the experience of an end user? When we do this kind of research, we’re balancing priorities that sometimes might actually conflict with each other. That’s a really challenging part of responding to feedback, I would find.
Thomas: Jen, can I put the same question to you? Does the research team have any big challenges as part of working as a larger group involved in this type of project and listening to customer feedback?
Jen: One of the challenges for us is that we have a really broad range of customers from different industries. Maybe they have different business models. And they use older tools alongside Intercom and have different ways of working and different approaches to their business. This means they’ll have different needs, which can be sometimes hard to understand from just one piece of feedback or just one feature request.
When it comes to understanding the feedback we get, we really have to put in time and effort to get that context of our customer’s world. We need to dig in a bit deeper to understand their environment and the context around a problem to be able to identify how to solve it. And that takes a lot of time.
For example, we got a lot of feature requests related to ticketing. We thought, “Okay, maybe every support team needs ticketing.” But when we actually dug in a bit deeper, we found that this need was more prevalent for customers that handle more complex support questions, where ticketing workflows are actually best suited for issues that might need to be investigated or handled by multiple teams.
And that actually took a lot of time for us to figure out. So when it comes to taking on feedback or looking at feature requests, it can sometimes be a challenge to get the full picture of that actual need from just that one input without having to go deeper and getting the context behind it – which takes time, and time is always a challenge.
Solving problems with multiple teams
Thomas: Maria, maybe I can present that question in a different way to yourself because this quarter, we released conversation data. And it really looks this solved quite a number of different issues, but I can imagine the output of all of those conversations is conversation data, which looks to solve all of those things very neatly – but it must’ve been very complex to see all of those problems initially presented and trying to find that one silver bullet that would answer all of those things. What are the challenges for the engineering team in trying to solve multiple problems all at once?
Maria: As Jen was saying, we knew we needed to introduce better ticketing workflows to help solve this problem. And it would have been easy for us to introduce the concept of a ticket, for example, to track those complex queries. It wasn’t that easy under the hood. But still, instead of going with what seems like the straightforward solution, the team that was leading this effort in collaboration with research tried to really, fully understand better the problem – and how we could evolve what was really working for our customers already in our product to solve those specific, complex conversational use cases.
“To really solve the problem, we needed to make that information valuable across the whole platform”
As a result, we ended up introducing the new conversation data attributes into our system to structure our conversation data more rigorously. But to really solve the problem, we needed to make that information valuable across the whole platform. There is not a lot of value if that information is available only in the inbox. And the only way you can get it there is by the teammate inputting that information for each conversation. But then if you can report, is that really valuable? Or how about giving users an opportunity to pre-fill that information? And if we are getting that information from the customers indirectly – if we are not using that information to maybe do some smart routing of those conversations to the right teammate in an automated fashion – we are really diminishing the value of that feature.
So as you can imagine, this project ended up impacting multiple teams because we really wanted to provide that end to end experience on workflow, so that we made the most out of solving this problem. The way we ended up implementing this work in a very short period of time, to be fair, was by having engineers from different teams collaborate between themselves and obviously collaborate with product managers and designers to both share the context of the problem to solve. That’s because there was mostly one team that had been really deep into understanding the customer problem. And we needed to pass that context and that knowledge by leveraging their expertise in the different parts of the product and the code base.
We have some engineers that are very proficient in how our bots and automated solutions work: engineers that are working in our messenger, engineers that are experts in the inbox. Having them all together, working to really accelerate the way that we were able to implement this, they had good insight into what was possible and any potential issues delivering that cohesive solution. It was pretty incredible to see how much ownership they demonstrated and how quickly the solutions came together, once we were very clear on the problem that we were trying to solve once we agreed the direction we wanted to implement this solution.
Enabling close collaboration
Thomas: Amazing. So then looking ahead to the future. Is there anything that you are all working on for the next quarter that you’re really excited about?
Jen: It’s going to come across a bit cliche for a researcher, but it’s the close collaboration that we’ve started to have between our product teams and our customers. Intercom has always had a pretty close relationship with our customers, and we always love to hear what we can do better. But during the last couple of months working remotely and with the pandemic, the gap between the folks that build products and the customers that use products is closing. And that’s really exciting to me as a researcher.
All the conversations we have with customers are super valuable, and we need a way for that to be effective at scale. Part of my role at the moment and for the next quarter is actually enabling others to have these conversations with designers, engineers, product managers; getting them chatting with our customers; and listening to their feedback. So I’m really looking forward to helping to facilitate those conversations and give our teams a deeper look into the world of our customers.
“During the last couple of months working remotely and with the pandemic, the gap between the folks that build products and the customers that use products is closing. And that’s really exciting to me as a researcher”
Thomas: Awesome. How about you, Mat?
Mat: I’m really excited about the next few months. I’m going to sound like I can only ever talk about Mobile Carousels, but I’m going to run with it. One of the main problems we originally set out to solve was first-time onboarding for mobile app users. And what we’ve heard a lot from folks since launching the product is that they’re really, really keen to use carousels in the context of proactive support and helping people self-serve. They want to find answers themselves before starting a conversation in the messenger or something like that.
So we’re working on those problems at the moment. I can show one thing that we’re doing, because development is well underway, and that’s that we’re making it possible for people to trigger specific carousels themselves without relying on Intercom. That means users will be able to open a Mobile Carousel from a button in your app. You’ll be able to do things like have a list of tutorials on the help screen of your app – or open a carousel if someone is having trouble with a specific feature. So that’s going to be really powerful.
Thomas: Sounds like fun, Maria, how about you?
Maria: We’ve made a lot of progress in the past few months improving our support solutions. We had a huge launch only a few weeks ago, but we’re just getting started. We have learned a lot recently, as Jen was saying, about what support teams really need to be much more effective and productive while providing the best experience for their customers.
We are continuing to invest in that, and I’m really excited about the work that we’re doing to continue improving our automated support and self-serve capabilities so that we can provide the most relevant content to customers immediately – when that’s appropriate, of course. I’m also excited about some of the upcoming improvements to how teammates can handle any requests more effectively in the inbox when their users do need a little bit of extra help.
Thomas: Sounds awesome. Listen: Maria, Mat, Jen: thanks again for joining us on this podcast. It has been hugely insightful and really a bit of an eye-opener to hear about our process. I have to say, it feels a lot more dynamic and flexible than I imagined it would be. And it’s incredible that your teams get access to that unfiltered feedback from the outset and are able to work on those problems as a team. But it’s creative, with everyone looking at that problem with their expertise in mind. Thanks again for sharing with us today.
Want to see these features in action? Watch the recording of our latest Built for you webinar.
