What 2017 taught us about product
If you're an avid reader of our blog or caught us on our World Tour, you know we're strong advocates for sharing our lessons learned.
https://art19.com/shows/inside-intercom/episodes/9fb7a7c7-e4b3-451b-832d-15e7a1952db3So before we turn the page into 2018, I sat down with the leaders of our product team, VP of Product Paul Adams and Director of Product Design Emmet Connolly, for a debrief on the past year. We discuss what shipping nearly 150 customer-facing product changes in a single year meant for the way we work, how our views on bots evolved after releasing our own, why we built our own design system, why product teams should get to know their sales colleagues, and much more.
If you enjoy the conversation, check out more episodes of our podcast. You can subscribe to it on iTunes or grab the RSS feed in your player of choice.
Below is a lightly edited transcript of the conversation. Short on time? Here are five key takeaways:
- In 2017 we made a conscious decision to break projects into smaller pieces that we could get into customer’s hands faster. Embracing program management was crucial to doing this successfully.
- The sweet spot for bots is building them to tackle tasks they can handle better or faster than a human, as opposed to those that require empathy or problem solving skills.
- Intercom at its core is a series of workflows made up from the same pieces reused again and again. Developing our own in-house design system has proven crucial to keeping things consistent.
- When splitting departments across offices, optimize for teams to be independent of one another. That doesn’t mean limiting cross-team collaboration, but rather freeing them to get work done and make decisions.
- Want to learn more about your product’s shortcomings or your customer’s problems? Talk to sales or sit in on a sales call, one of the most under-appreciated resources at a product team’s disposal.
Des Traynor: We’re going to do a year in review of sorts, in terms of what we learned. Paul, 2017 was a counter-year to 2016 in some senses. What were the big changes you made and why?
Paul Adams: The biggest change was number of projects. In 2015 we had close to 50 people working across product, design and engineering, and we made just north of 100 customer-facing changes. In 2016, with more people, you’d expect that we would have more changes, but we didn’t, we had less, between 50 and 60.
So at the end of 2016, with more people, we did less on paper. It’s more complicated than that. There were three big projects in 2016: we built our Educate product, we redesigned our Messenger from the ground up, and we shipped Smart Campaigns. The problem with big projects is that your customers don’t get them for a long time. Educate and Messenger took about a year, Smart Campaigns took three to six months. With the Messenger, that was a year where we had the same Messenger, so our customers weren’t seeing changes and it was less clear that we were building something they would love and value.
We made scoping a much more deliberate early action.
So in 2017, we said we wouldn’t do that. We would try to break projects up into smaller pieces and get them to the customers earlier, and we really refined our process to do that. We made scoping a much more deliberate early action, and this year’s been amazing. We have nearly 150 things on our list of product changes.
Des: From a design point of view this is a big shift from a small number of big projects. As a designer on a big project you can conceive the whole thing at once and design it holistically. 150 changes, each done on its own sort of scope and on its own timeline, how does that happen with a holistic design? Did we pay any tax there?
Emmet: It certainly wasn’t 150 completely random, disconnected things. So rather than saying we have 150 things to do, the approach is to start with the big themes then break them down really into 150 releases. Each one of those releases needs to be able to stand on its own, and a lot of that is around planning and scope, rather than design.
Des: Is there a sense of a big design up front and then from there you break it up into smaller pieces?
Emmet: One of our core principles is to think big and start small. That means doing a lot of design thinking and trying to be expansive in your vision and where you want to get to, but then getting there in many small steps. So, “think big” certainly doesn’t mean do tons and tons of design work up front and then build it in small steps. Instead, have a fuzzy picture of where you want to get to, and then start designing in small steps, because you’re almost certainly going to course-correct anyway as a result of the feedback loop you get from each of those steps shipping. It’s more of an exercise in keeping the fidelity of your early-stage work quite low and then being able to iterate your way along, while roughly staying on that path you planned at the start.
Des: There’s an interesting principle there, which is keeping the fidelity appropriate. The closer something gets to a fully polished design, the more attachment people have to it, whereas I think it’s worth being honest about the fact that, “Hey, we’re going to ship this and not everyone’s going to love it and we’re going to learn a lot.” We need to be open to that.
Emmet: It’s a bit like correcting the common misconception around a minimum viable product as well. Launching something that is a bit shit is how it’s often misperceived. The thing that we had to keep in mind is that for each of those tiny steps, the quality needed to be really high so that thing had a chance of surviving on its own, but it’s part of this longer story that we’re trying to string together.
Paul: Another interesting component here was that we embraced program management this year in a big way. For example, towards the end of the summer we had a new Live Chat for Sales solution for customer acquisition, which was actually 15 features. At the start of the project we had a “program” of work, and these programs have a goal, which in this case was to launch improvements to a product we already had called Acquire.
We had 15 things and they were worked on independently as part of this program, but when they were ready for customers, they’d go to a beta – some of them we even announced or released to all people. Then we’d progress. We’d learn how to improve them as the others were getting built. We iterated, iterated and iterated so by the time we got to the announcement, it was more refined.
Bringing our Operator bot to life
Des: Last year we were saying bots generally were still at the early stage of the hype cycle, that the sizzle was better than the steak in some sense. Do you think that’s changed in 2017? Obviously, we released our own bot, Operator, within this year. Are the limitations still in play? Do you think bots are supposed are best for simple, automatable, basic tasks? Is that still the take?
Paul: I think that’s still a take. A lot of the things a year later are still the same in many ways, but the hype has died down for sure, which is good. This is the classic Gartner hype cycle, where something is overhyped and then enters the trough of disillusionment.
The Gartner hype cycle. Learn more about what building Operator taught us here.
We’re definitely in that stabilization phase of bots. I personally think bots are a huge deal for the future. They’re going to be a very commonplace thing. I think we’ll look back in years to come at today’s world where we didn’t interact with bots as much as we do or will do in the future and not really understand how we did things in a different way. They’ll be so prevalent. It will be predominantly for the same, simple repetitive task – things computers are much better at than humans.
Des: Is that excitement because you believe there’s so many of these low-level, repetitive tasks that when we have bots taking care of them all it’ll free us up, or is it some net new capability?
Paul: It’s definitely new capabilities. Yes, those tasks are low level, they’re often repetitive, and computers could do them faster and better, but the reason computers can do them faster and better is often because there’s huge permutations happening in the background. There’s really complex code running. Think about self-driving cars. If you really wanted to go there, you could say that a self-driving car is a bot in a way, or that driving is a repetitive task.
A lot of these bots are simple on the face and pretty complicated at the back. There’s loads of life that we can automate and people can go out and do things that are more meaningful.
Des: Operator was a significant release, and we touched on all sorts of new challenges. Obviously our mission is to make business personal, yet we released an actual robot into conversations. How did that all play out within the design team?
Emmet: There was this initial period of hype. Then, once we got into experimenting a lot, which we were doing in late 2016 in the run up to releasing Operator, we realized there are some clear things that it does well and some things that it certainly isn’t ready for at a technology level.
There’s a certain category of customer conversations that are more suited to a human dealing with them – things that require empathy and problem solving. But if you asked a lot of customers, “Would you like to wait for a human to answer your question?”, even if the response time was really good, “or would you like to get the correct answer to your question immediately?”, a lot of people would choose the latter.
Forms are one way Operator automates tasks that are slow and mundane for sales teams. Read more about the concept here.
More than thinking about how to program the bot and how to get the bot to say the correct things, a lot of the design problem for us became looking at the question itself and deciding, is this even appropriate? Internally, we had this concept of answerability. Should we consider this something that a bot could or should answer? It suddenly becomes more of a question of rooting. Of course, when you have an answerable question, you need the bot to service that answer in a good way and there’s a lot of work still to be done around that, and a lot of design discovery we still need to do to find out what works and what doesn’t. It’s more of a nuanced view than all humans all the time or bot all the things, which I think was what really led to the crash in the hype cycle. A lot of the earlier approaches in late 2016 were very naive. People were trying to apply bots to problems that they weren’t very well suited to.
Des: In the case of Operator there are some frankly transactional questions we might receive like, “Do you have an iOS SDK?” Versus something like, “Hey, I’m running a program of work where we’re considering improving our business, but…” Operator can answer the former category because the user’s not bringing a lot of unique context with them. We don’t need to hear more. Do you think that’s where we are for the moment, or do you think we’ll be able to go further?
People were trying to apply bots to problems they weren’t very well suited to.
Emmet: Like most technology, they’re in this eternal stage of being born, so there’s always more room for them to continually get better and they will. A lot of what we learned about the mechanics of putting Operator together is when you’re building a machine learning system, a lot of the magic and challenge in putting that together is in training the system and helping the system to bootstrap itself. If you’re a smaller business and you have a low throughput of questions coming in on which to train the bot, that’s a challenge. We found ourselves having to design the back end of our tool interfaces to allow the bot to gradually get trained, to start to get to the point where it’s learned enough about questions and can start stepping in and providing answers itself.
Des: Paul, do you wanna make another proclamation about bots in 2018? More of the same?
Paul: They’re going to grow up more next year than they did this year. What I mean by that is people will encounter them more and there’ll be more uses of them. Like Emmet said, a lot of this is about whether the bots have data sets they can learn over. It’s hard to know what’s going on inside other companies, but we’ve spent a lot of this year figuring out how to make them work. Now we’re at a point, especially with Operator, where it’s working in different contexts and we have lots of ideas for how it might work better in the future. Next year I think we’ll see those things come to fruition, and I imagine the other companies are doing similar. Even some little things we’re seeing here and there towards the end of this year suggest there’s new use cases opening up where bots are providing better or faster service than humans could.
A homegrown design system
Des: Operator contained a lot of new design components, and as you said earlier Paul, we also released 150 other things, which were held together by the design process that Emmet described. This year we released our first proper internal design system, Full Stack Design. Why did we go ahead and formalize all this? Was it yak shaving or was there actual value there?
Emmet: Intercom is, at some level, a series of workflows made up from the same pieces reused again and again. That’s the path we’re attempting to take from a design point of view towards building something that’s very powerful but simple at the same time. Design system seemed very well suited towards achieving that.
I fully believe in trying to get in touch with other people in the industry and asking them, “You might be a year or two in scale or in growth ahead of where we’re at, but tell me about all the mistakes you made over the last couple of years so I can learn from them.” A lot of those people were all building design systems, but they looked a lot like the things that you saw publicly available like Twitter Bootstrap or Material Design. I was really struck with the fact that the purpose of something like that, which is to allow tens or hundreds of thousands of developers to build using the same set of tools, is very different to what an in house design system should be used for, which is to allow tens of designers to build one product. Not thousands of designers to build thousands of products.
Des: Were they over solving the problem or were they not being specific enough to themselves?
Emmet: More of the latter, because you take inspiration from what you can see publicly. I’ve never seen Twitter’s internal design system directly, but I’ve seen what they’ve shared publicly, so that’s what you take your inspiration from.
For us, what really held a lot of promise – and this is what I was aiming for with the Full Stack Design System – was seeing if you can infuse the system throughout all of the different layers of work that your product is made up of. The APIs, the code and the design objects that you use to design things, and even the names that you give things inside your product and inside your help docs.
Within the Full Stack Design System, a “conversation” is one of our core objects.
If you can make that system a lot more specific rather than about a bunch of generic buttons and dropdowns, then you can speed up the workflows. It stands for reason because they’re very custom built. They’re very bespoke for purpose.
London calling: adding a second R&D office
Des: This year we went from one R&D office in Dublin to a second in London. Going from one floor to two floors is a big change, but going from office to two offices is a very big change. Why did we do that and what have we learned?
Paul: The reasons we had are the reasons you’d expect to hear. We’re growing really fast and in order to fulfill our potential, we need to hire lots of great people. They’re not all in Dublin, nor are they going to relocate to Dublin. Opening an R&D office outside of Dublin where we build product was somewhat inevitable if we continued to grow as fast as we have done.
We looked at all the different places we could go and London became the clear winner. For one, there’s amazing talent there. Eight years ago I worked in London at Flow and Google, and I knew from my own experiences the fantastic people you could hire there. It’s also so close to Dublin. We can fly over and back and can actually have the best of both worlds.
Des: We’ve since gone and opened an R&D wing in SF, so we’ll learn even more along the way. How consistent do we want the London product team to be? Is it the same process worldwide? Would you be confident that you could swap engineers and designers around and they’d know exactly how to work?
Paul: You need to think about the things people talk about. So, if you go to London or go to San Francisco or you’re in Dublin and you find everyone talking about process, that’s a pretty bad outcome, because they’re not talking about product. Or even worse, they’re talking about company politics.
Our cozy London office, now the home of our Educate and Operator product teams.
In large parts of my experience at Google in London working with Mountain View, which is where the headquarters is and where I subsequently moved, there was a lot of politics. There was a lot of talk about how teams didn’t work together. One of the reasons for that was that teams were split across location, so when I was working in the UX team in London, I was working with engineers in Canada and Mountain View. We weren’t setup to do that well like a totally remote company.
For us, we’re optimizing for teams to be independent of each other and of the location. Educate has a team in London. Operator has a team in London. Of course, they work and collaborate with teams in Dublin, but they’re designed to be as independent as possible. They’re not talking about politics as much, we hope, because they are free to get their work done.
Des: Emmet, you also worked remote. How do designers collaborate remotely?
Emmet: I worked in a satellite office, but I worked at Google in the design team at the same time as Paul, and I experienced it on both sides of the divide. I was in the Zurich office initially and collaborating with a team over in Mountain View. Then I relocated myself to Mountain View and certainly the center of gravity in terms of power and decision making naturally is in the head office there. It was incredibly frustrating when you’re on the wrong side of that divide.
To answer the question, the absolute, number-one way, even in one of the most technologically advanced companies in the history of the world, is to get on a plane and go speak to the person face to face. There’s just no substitute for that. In fact, we had several instances in projects where things came to a head and you’re saying, “You know what? Someone has to get on a plane, and we need to work this out in person.” As soon as you can do that, so many tensions dissolve, and you see that person on the other end of the email thread as a human again. That was part of the reason why we also chose London initially– it’s very easy to maintain. It’s the same time zone and a short flight away. We’ve taken day trips over and back.
Emmet: We will have lots of those little teething problems to iron out, but we’ll have ample opportunity to do so. That experience that both of us had makes me hypersensitive to it. There’s so many little things that you have to keep in mind, like not forgetting to invite people in that other office to meetings. We’re organizing our Christmas party; how do you include the people from that office? It really is important to establish them not as some kind of second-class office, not as a remote office, but as a sister office. That’s really important.
What product teams can learn from sales
Des: Paul, you had a recent tweet that got a surprising amount of engagement given it was talking about sales. You basically said a lesson you learned the hard way and the wrong way, frankly, was that you need to talk to your sales team, both your sales leaders and your front people on the ground. You also recently did a sales day where you sat with our frontline account executives and sales development representatives. It all seems to have been a Road to Damascus type moment for you. What happened?
Many Product people don’t talk to their Sales team. This is a critical mistake. Your Sales team knows your product’s failings better than you do. I learned this too late in my career, and I still don’t do it enough. Don’t be me.
— Paul Adams (@Padday) November 30, 2017
Paul: Throughout most of my career, there are two areas of the company that many, many product teams ignore. That’s sales and support. Lots of people will say they don’t ignore them, but when did you last talk to somebody in those teams? Many people pay them lip service, but they never talk to them. The irony is that these are the two teams that talk to customers everyday all day. We’ve had a longstanding relationship with our support team here, where they input heavily into our roadmap process. We’ve had a growing one with the sales team over the past year or two, and it’s matured as our sales team has grown because sales is a relatively newer function at Intercom.
There’s research happening in your company every day, and most product teams are not tapping into it.
These things just get real when you end up on a sales call and talking to the people on the sales team. I was listening in on a couple of sales calls and the amount of insight, the subtleties and nuances that customers are talking about, how the sales team had to talk to them…I just found myself thinking, “How do we turn these hundreds of sales calls we’re having into product insight that we can turn into actual product?” I don’t have any answers to how we do that. We’re working on it. There’s research happening in your company all day, every day, and most product teams are not tapping into it.
Des: Not that long ago myself and Senior Director of Marketing Matt Hodges came to Dublin and gave a big presentation about why product might listen to marketing every now and then. Why is it this doesn’t come naturally to the product teams who would otherwise see themselves as empathetic and research-driven?
Paul: I don’t know. One of the causes in most companies is that the organizations are siloed. Organizational design. Even at Intercom, we split up over two floors this year, and we’ve actually got four wings basically. Guess what, the sales team sits on a different wing to the product teams. I’m only realizing that now. Maybe that’s part of the problem. If you had people sitting beside each other, overhearing sales calls, and talking in the same micro-kitchens, you might actually start to break down these barriers. That’s the eureka moment that I had.
Getting people right
Des: We spend a lot of time at Intercom talking about lessons learned. I’m gonna do the horrible thing and distill it down to a single one. What’s the one thing key thing you learned in 2017?
Emmet: I realized there’s almost nothing massively new under the sun, just different flavors of the same problems – and maybe even different flavors of the same solution as well, when you get down to what you actually design and ship. I realized that I still had an awful lot to learn about some of the basic stuff like managing and building teams. You don’t necessarily need to reinvent from first principles when it comes to providing a good, healthy, working environment and processes for the people on your teams. I found that there was a ton I could learn from talking and reading about how other people solve those problems.
Paul: Mine is very similar to Emmet’s. I learned about people. What I mean by that is that I taught myself about how people work. This year I think we grew up. A lot of other companies that are fast growing like ours might have similar stories, but I think we grew up in a bunch of ways. For me, all the books I read this year were about people – managing people, teams, how you build teams, teamwork, etc. There was no product or design books in there. I lost my design nerd brain and instead adapted this new people nerd brain.
I talk to people way more. I have 1:1s religiously. I don’t skip them or cancel them or decide that the design review is more important. I listen way more, and I try and learn. I also realized how little I know about a lot of this stuff. To Emmet’s point, you don’t need to reinvent the wheel. There’s fantastic resources out there. Psychology is an old field. There’s lots to learn from things that are 50 or even 100 years old.
