Research at Intercom: thinking beyond product
User research at a tech company typically lives within the product team.
Researchers work with product managers and designers to evaluate future product concepts with users, test pre-launch prototypes, and gather user feedback after launch to improve the product. It’s a discipline that’s exploded in recent years and companies of all kinds realize that research is essential if you want to design an amazing customer experience.
But there’s a problem with this approach. Restricting the role of user research to collaboration with just your product team means critical aspects of your customers’ experience are never evaluated. You’ll miss out on studying relevant parts of the customer experience that flank your core product, such as the marketing site where people first learn about your product, or your process for handling customers who have billing issues months later.
At our recent Inside Intercom event in San Francisco, I outlined how transformative user research can be when you introduce the discipline early to a company and apply it throughout the organization. I share details of how we conduct “classic” user research, such as mobile usability testing and eye-gaze tracking, to design our products. The talk also highlights how we’ve expanded the role of research to improve our entire customer journey, from evaluating user comprehension of our marketing material at the top of the funnel to creating a process to act upon feature requests from valued customers.
Transcript:
At Intercom we run a classic kind of product research that fits in with the product development cycle. Researchers will work with the product team to evaluate early concepts and prototypes before launch, and then gather customer feedback after launch to improve the products. That’s all classic stuff, but what’s unique about Intercom is we really pay a lot of attention to detail for a company of our size and age. What follows is two examples of that.
The first is a really fun study that we did around our Messenger product using a technique called Eye Tracking. When you launch the Intercom Messenger, there’s some text. One of the things we were curious about was what text are people actually reading. That’s a really hard thing to figure out in research because people are really bad at telling you accurately what they’ve read – and it’s just hard to research – but there is one technique that can tell you exactly what they’ve looked at and that’s Eye Tracking.
This is how it works: You can rent this rather clunky looking monitor, the Tobii Eye Tracker, which is actually very expensive and has this kit built into it. It beams infrared light into the participant’s eye, into their retina, and that bounces back out and is picked up by these cameras in the hardware of the monitor. You can then start to track exactly where someone is looking. This is super satisfying for a researcher because you can actually pinpoint things that a person could never actually tell you.
We took an example consumer site, Bellroy leather goods, and we looked at different ways that we could attract attention to the Messenger. This was actually a precursor to the product we recently launched called Acquire. It’s a way of talking to visitors on your site. We were curious to see just how much attention this kind of visual treatment gets.
In one version, which just shows text, you can see that actually there’s not much attention. But in another version we included some faces from the people at the company that you’re talking to and some slightly different text. What you can see, within the first ten seconds that someone is looking at this page, is this really caught people’s eye in a much more obvious way. This research actually helped us refine our designs. In the Acquire product you can see that the message that sits in your site does have the faces of the people at your company.
The second example is mobile research. If you’re not familiar, we have some products that allow you to communicate with your users in your mobile apps. We looked at a chat style message, a big announcement in the center of the screen so you can, for example, announce new product launches, and a push notification.
The way that we researched this particular question was using this really cool app called Lookback. It’s free actually, and not only can you see exactly what’s happening on the phone screen, but you can see the participant’s face as well. Often what someone’s face is doing compared to what they’re actually telling you can be very different. It’s really very important additional info.
We had some very detailed questions like, when we send the message is this the right position to place the message on the screen? If you think of someone using the phone with one hand, can they actually open it with one hand ergonomically? We’re giving this little snippet of the message. Is that actually enough text to give you a clue about what this message is about? How many seconds should that be displayed for on-screen? These are the kinds of things that we were thinking about and able to study.
One fun thing we observed is that people kept swiping a message away to the left. I’m like, “What’s going on there?” People said, “Oh yeah, that’s just because I use Tinder and I’m always swiping left.” It’s pretty funny, but at the same time really important because if you can identify these kind of established patterns then those are probably things that are going to carry over to your own products as well. These established interactions are very important to know about.
That’s just the flavor of product research that we run. That’s all well and good, and pretty traditional to be honest; user research is working with the product team. But there’s so much else to customer experience. There are so many other touch points of your product and your whole company that shape someone’s view of your products.
What tends to happen is organizational walls appear within the company as it gets larger. That’s also the stage when people typically bring in researchers. It’s when the company is much more established and these divisions are already there. That’s such a shame because we’ve seen if you bring in researchers early they can actually inform every aspect of your business. Those kind of walls just don’t exist in a small company. You’re all chatting over lunch, talking about the problems that you have and research can help.
Here’s some of the ways that research has helped other parts of the business beyond product. You may be familiar with User Testing. Again, this is a really cost-effective tool anyone can use easily.
We do the simplest thing, but it’s so effective. We let people who work for small tech companies play around on our site for maybe a minute and then ask them, what do you think Intercom does? The kinds of answers you get back are so revealing in terms of the misunderstandings people have and the language you’re using that isn’t quite communicating clearly. That’s really allowed us to fine tune our marketing.
Second, we write articles about how to do great research using Intercom or other tools for our blog. That really helps people, and it primes users to get the most out of the product.
We’ve also done some research into who reads the blog and what they actually want to try, and we constantly improve the content that we publish. Invoices are another detail where research really shows some love. It’s so important to us.
Finally, feature requests. In a lot of companies you end up with some feedback from customers that is really interesting, and it would probably be good to do something with that feedback – but there’s no system to do that. We realized that Intercom supports these incredibly nutritious customer conversations, and we helped create a tagging system for them. Then the research team analyzes the feature requests coming in, turns them into a stack rank of popular requests per team and feeds those into our roadmap. It’s really satisfying. We’ve closed this loop between the customer and the roadmap.
Research is valuable to every part of your business, and if you invest early as Eoghan did, you get all these immense benefits. You end up with better marketing and customers who are primed to get the most out of the product, you’re getting a better customer experience, and hopefully you’re also getting customer retention and lower support cost as a result of all this work.
My advice is if you don’t have a researcher already, go and hire one. Then encourage them to really think beyond the walls of the product team.
