Intercom on Product: How we unlock the power of feedback

Launching a new product is the beginning of the conversation, not the end.

When we ship, we of course have done our diligence with research and design, but we’re also keenly aware that there’s so much we don’t know. As a result, we’re looking to start rich dialogues with lots of different people: prospective customers who are talking to Sales, existing customers who are solving issues with Support, and customers who have left for alternative solutions or for other reasons.

Luckily, we have a few tools for facilitating great conversations with those groups and acting on that feedback – which I unpack with Paul Adams (SVP of Product) in the second episode of Intercom on Product, our new series where we share our latest thoughts on how to build a great product.

You can listen to our full conversation above, or read some highlights of our conversation below.

If you enjoy the conversation and don’t want to miss the rest of the series, you can subscribe on iTunes or Google Podcasts, stream on  Spotify or Stitcher, or you can grab the RSS feed in your player of choice.

Unlocking the power of feedback

Des: We’ve just launched a product recently in Product Tours. Product Tours was interesting in that it was a long-standing request – our sales team wanted us to build it, our customers wanted to build it, people left Intercom because we didn’t do it. So, it’s out there in the wild now. What are you looking for in the responses from customers who are trying it out?

“We have a principal here called ‘Ship to Learn’, which is that when we ship something, our intent is to learn”

Paul: Obviously, when you launch a product, a brand new product, you want to learn what’s good, what’s bad. We have a principal here called “Ship to Learn”, which is that when we ship something, our intent is to learn, and when we ship something like Product Tours, which is a big project internally, we ship with confidence that we’ve done a good job, because we would’ve done a lot of research and analysis, and thinking, and design work and iterations, and so on, all the way through build. But humility too. We often talk about this idea that when we ship at Intercom, we want to ship with a combination of confidence and humility.

Des: Is that like, “We’re sure we’ve done our best, but we’re open to the idea that we’re wrong,” kind of thing?

Paul: Exactly. But the humility thing is the bigger piece of that, and the humility is simply assuming we’re going to get something wrong. We always assume we’ll get something wrong. We’ve never not launched something and learned straight away, out the door, day one, “Oh, interesting. Hadn’t thought about that before.”

How to combat organizational amnesia

Paul: For the launch of Product Tours, the first job of the Tours team is to stay together and listen and talk and listen further, and then start working on better and better and better versions of the product.

Des: I think in the earlier days of Intercom, when we had like, much, much fewer engineers and designers than we had even products or product ideas, it was more tempting for us to jumble people around. And the one tax I always felt we paid was sort of organizational amnesia. The person who built the thing is no longer the person talking to the customers about why the thing doesn’t do what they thought it would do.

“There’s a real benefit or value in that, like longitudinal ownership of having somebody experience it from idea through to execution”

And as a result, you have a lot of people learning this stuff for the first time. There’s a real benefit or value in that longitudinal ownership of having somebody experience it from idea through to execution, through to feedback and course correction. This lets them realize, actually, it turns out what you thought in October is no longer true here in May, when people are actually trying to do the thing. And I think some of our bigger mistakes internally have come from shuffling people off things too quickly.

Listening to your unmet customers

Paul: It’s very fashionable to talk to customers, “Hey, you gotta be out there talking to customers. Make sure you’re talking to customers.” And so you end up with loads of people just talking to customers, and it’s not structured, and they don’t necessarily have the right skills or research background or training to pull the signal out of the noise. We’ve put in place a structure designed to do exactly that. We needed something systematic and scientific, frankly.

The “Problems to be Solved” report then came about initially from the Sales team having all these conversations that were happening every day, and coding them in Salesforce, which is the tool that they use. And they’d code the conversation with a bunch of different tags, like this type of feature request. And then we would pull it all out into like a big list, like a spreadsheet, basically, stack rank it, and then the Sales team, with the Marketing team’s help, would then put opinion over it, you know?

“This is a lesson I’ve learned the hard way through a lot of startups I’ve met with – you’re not talking to the people who can’t buy your product”

Des: This is a lesson I’ve learned the hard way through a lot of startups I’ve met with – you’re not talking to the people who can’t buy your product.

Unless your sales team are giving you feedback, those people have no way of offering feedback. Sure, they can sign up, give you feedback, and then quit, but they’re not going to do that, because they have an other things to be doing. So the biggest realization for me is the importance of the voice of the unmet customer, the people who want to use your product, but they can’t. This is the stuff you need to build to enable them.

Deciphering the ‘Customer Voice Report’

Paul: The Customer Voice Report, then, comes from the other side, from our support team. So it’s the voice of the customer. It’s a little bit self-explanatory. Just as our sales team talked to all these prospective customers, tagged the conversation, providing structure on all these conversations that they have, the support teams do exactly the same thing.

The support team is solving all these problems our customers have, and as a result, either the conversation they’re having is an actual feature request or product gap. And then they’ll often get more stuff like, “Oh, and by the way, while I’m here, I’d love X, I’d love Y, this is hard, I’m confused.” We again extract a data report, stack rank it, order it, and then that comes into the roadmap process as well.

Inputs into our roadmap

Des: When we look at our churn, so people who have genuinely quit using Intercom for a feature gap or a feature quality issue, how do we tally that against Customer Voice Report? I feel like what they’re saying when they churn, may be a bit closer to what they’re doing, or at least it certainly turns the dial on severity?

“We have four inputs to our roadmap, and the Customer Voice Report is part of one input, and the Problems to be Solved doc is part of another input”

Paul: Yeah, very much. And actually there’s an important point there, which is you should not blindly follow these lists. We’ve four inputs to our roadmap, and the Customer Voice Report is part of one input, and the Problems To Be Solved doc is a part of another input. They’re just inputs. Sometimes we never do things on both. Sometimes the team will do lots of things on both. But yeah, churn is what people do. We run a survey when people quit. We basically ask them, “Hey, can you please tell us why you’re quitting?” And the goal there, of course, is for us to understand why they’re quitting because we don’t want them to quit. And that’s another input into our roadmap.

Finding a balance between the different feedbacks

Paul: If you launch your product, you tend not to have a lot of input from sales because they haven’t started selling it yet, but I often think you want to work backwards through the funnel, fix the things that are causing the product to be churn-y first, then fix the frequently requested features that maybe aren’t total churn causes, but are definitely stumblers, or make the product less attractive.

When you genuinely feel like you’re on good ground, like if you’ve good net retention, et cetera, then you can start saying, “Okay, well let’s cast the net wider. Let’s now take on sales feedback, and let’s bring in more people into the mix,” who themselves may well, like we might fix the sales feature, the sales blocker, and that might bring them in, but then they might just churn it to the side because of something else that they forgot to ask for during procurement. And then we’re back to the start again.

We’re like, “All right, let’s work from the back forwards again.” Advice I’ve always often given in marketing teams is work from the end of your funnel backwards, as in fix your product marketing before you throw a million dollars of ads at it.

But similarly, you fix your product before you try and expand the addressable market of your product, right?

Des: It resonates strongly with me. Like that’s I think how we’ve always thought about it for a long time. But, it reminds me as well … And you mentioned earlier, like the pendulum swing between like qualitative and quantitative data over the years, we’ve had a little bit of a swing here, too. And that’s not what you want. You want a balance. And the balance doesn’t have to be like a third, a third, a third.

“It’s very important for us that we have a balance, that we are equally considering all the constituents you mentioned”

It might differ depending on your company, but you want a true balance that feels balanced for you, and then you want to do it in that kind of reverse order. There were times, in our early days, we were very heavy on the support side and our Customer Voice Report. Only later did our sales voice grow with Problems to be Solved. And now it’s very important for us that we have a balance, that we are equally considering all the constituents you mentioned; people who are quitting, people who are frustrated, who are existing customers, people who just get started, are trying it.

Customer feedback generally tends to correlate with customer satisfaction. If most people have to ask for 10 features, they’re probably not loving the tool.

Lastly, you’ve got commercial success with a product, which is if no one can procure your product because you lack certain features, then your revenue targets are going to be not as crisp as you might like.

Each of them do map to a metric, and I think it’s worth keeping that in mind when you’re looking at a roadmap.