Josh Seiden on why product teams should focus on outcomes over output
Deciding what your engineers should do next can be a lot like climbing a ladder. On the lowest rung is a problem to be solved. At the top is an impact on the business, a change in the bottom line.
But the rungs in between can often be quite flimsy. Instead of trying to jump straight from the theoretical business impact to a directive for your R&D teams, it’s essential to stop for a moment and consider: “What is the desired outcome? If we implement a new feature, how will customer behavior change in a meaningful way?”
At Intercom, we’re in the thick of working through how to help our teams become more outcome-focused. That’s why we asked Josh Seiden, a well-known product consultant and author, to join us on the podcast. He’s just released a new book called Outcomes Over Output.
We wanted to jump into this hot topic to see if we could marry our practical experience with Josh’s framework. In this episode, we cover everything from how to properly scope problems to why change is sometimes harder for leaders than it is for teams.
Short on time? Here are five quick takeaways:
- Narrow your definitions. You want to reduce the scope of your question to: “What are the things we can change in our behavior or our users’ behavior that might make our customers more satisfied?” Scoping down to a tight focus clarifies what teams should be working on.
- When developing a new feature, the first question you should be asking is, “What will people be doing differently as a result of this feature?” Too often, we just build a feature and move on without analyzing how it’s performing.
- Everybody in the company should understand the business model. That way, they make logical jumps and understand how the work they’re doing could conceivably impact the bottom line.
- Change is much easier for teams and much harder for leaders. Leaders have to get out of the business of saying: “I have this solution in my head. Go do it.” Instead, they have to do the work of saying, “Actually, what problem am I asking people to solve?”
- Change starts small. Pick one project or one set of stories where you’ll use the language of agile. Expect it to go okay, not great. Expect to use retrospectives and to continuously improve your process as you try to implement this, because you’ll learn a lot.
If you enjoy our conversation, check out more episodes of our podcast. You can subscribe on iTunes, stream on Spotify or grab the RSS feed in your player of choice. What follows is a lightly edited transcript of the episode.
Brian: Josh, welcome to Inside Intercom. Thanks for joining us on your book tour. Outcome versus output: it’s a hot topic in the industry these days, and it’s actually a hot topic for us in R&D to Intercom. Let’s start by hearing about your background and what made you write this book in the first place.
Josh: I’ve spent about 30 years in the technology industry, most of that time working as a user experience designer. In the last 10 or so years, I’ve spent a lot of time writing about design and about how product teams can be more effective together. With my co-author Jeff Gothelf, I wrote a book called Lean UX, which was a book about how product teams can work together in a more agile way. I wrote a second book with him called Sense and Respond, which was a book for business leaders to understand the importance of re-organizing around software as the beating heart of our businesses. One of the topics that has come up recently in my consulting is this idea of outcomes. It just was a useful idea in so many contexts that I felt like I had to write down and share what I had been learning.
An outcome is a change in human behavior that drives business results
Brian: It seems like it’s one of these things that’s definitely bubbling up in so many different places at the same time. First of all, I wanted to just thank you for writing a short business book. You’ve got to be applauded for that. Instead of figuring out how to beef up a book and turn it into 300 pages so it lands with a thud on your desk, you kept it concise. It was about an hour to read. When it’s short, you give yourself permission to stop and think, which is half the value of these things, to digest it. Did you have to actively resist the temptation to write a longer book here?
Josh: The book is published by Sense and Respond Press. Full disclosure: I’m a co-founder of that also with Jeff. One of our value propositions was that we’re a press that publishes short, practical business books. When we did our user research before we started the press, we met all these people — busy leaders who had stacks of what we came to call aspirational books, books that were sitting on the corner of their desk that they wanted to read, that they knew they should read, but that were just too long. We’ve all had the experience of reading a book that should have been an article or reading an article that should have been a tweet, right? We thought this was an opportunity for us to publish the heart of the idea and not fill it out with fluff. That’s been very successful for us.
Narrow your definitions
Brian: On the surface of it, outcome versus output almost seems self-explanatory. It’s not about what you ship so much as about the value you create. Let’s cut straight to the TLDR here: what’s the main thing you’re trying to say about this?
Josh: Yeah, I think you’re right. I think the notion is kind of self-explanatory. That’s made it almost a slogan for people, but my opinion in working with teams has been that it’s actually really hard to get beyond the slogan. Conceptually, yeah, there’s nothing to disagree with there, but how do you make it practical? The main premise of the book is that by defining outcomes really, really precisely – and in the book, I say an outcome is a change in human behavior that drives business results – you can use that really narrow definition of outcome and start to make it possible to really apply this idea of outcomes in our work.
Brian: I think that definition was probably the biggest aha moment for me, reading through this. What do you think this was in contrast to? Why do you think that narrow definition is helpful here?
Josh: For me, a lot of this got unlocked. There’s a paper from the late ’90s that was published by the Kellogg Foundation, which describes this thing called the Logic Model Framework. I talk about that a little bit in the book, but basically, they break down program planning from the resources and the activities you do, the outputs you create, the outcomes that those create, all the way up to the impact they create. By having all of those different levels in the model, it helps you specify the thing you are working on more precisely. I think a lot of times, we use the word outcome. We just mean result. That’s fine in casual conversation, but it’s really hard when we’re actually planning work, when we’re trying to write user stories, for example, to just say, “What is the result we want?” It’s a super open question. To have to have the answer for every story is really difficult. You ask, what’s it in contrast to? It’s in contrast to a really broad and undefined definition.
The Project Logic Model, adapted from Kellogg Foundation
Brian: That resonates a lot. Paul, one of our design managers here, was writing a document and referencing that model you mentioned. I was actually struggling to differentiate. Outcomes, impact – aren’t these the same thing? What’s the difference? In your book you discuss the narrowness of that definition. Definitions are arbitrary. Let’s accept that, but they’re helpful when they’re arbitrary. The aha moment for me was separating the customer outcome, the business impact, etc. Talking about it here internally, what has really resonated is saying, “Business impact is at the top.” Of course, that’s what we all want to do, but we’ve got to map it down to a lower level for what the teams can focus on.
By focusing on customer outcome, that’s where this starts to get achievable. Making customer outcome measurable is the key to actually being able to translate business impact into something that’s more relevant for the teams and something they can actually hope to directly change.
Josh: I’ll tell you: my first aha moment with this was a couple of years ago. I write a little bit about this in the book, but I was working with a team whose mission for the year was to increase net promoter score on their service, which is very clear and also left them with this giant set of questions like, “How in the world are we going to do that?” In working with them to ask and answer that question, we understood what increasing net promoter score means, but how would we do it? That “how do we do it” question led to us asking, “What are we going to do to make people more satisfied and more likely to recommend us?” The next question was, “What are the things they’re doing or that we’re doing that make them satisfied or dissatisfied?”
What are the things we can change in our behavior or our users’ behavior that might make our customers more satisfied?
When you just start asking that series of questions and really focusing it down to “What are we doing, and what are they doing?”, you reduce the fuzziness of the question from net promoter score — which, by the way, is something that no single team can change. It’s a scoping problem at that point. You reduce the scope of the question to, what are the things we can change in our behavior that might make our customers more satisfied or what are the things we can do that will help our customers change their behavior so that they’re more satisfied? Just by scoping down to that very tight focus, for me, it really clarifies for teams what they should be working on. I think that’s the advantage of having this very specific definition.
Moving the focus from problems to outcomes
Brian: That totally makes sense. When we’re trying to map that down to customer outcomes, how confident do you think teams need to be about that mapping? How much is that speculative versus saying, “We’ve got some solid data here that really backs up this claim”?
Josh: You need to at least have a hunch that if we change this it will positively change customer behavior, but you don’t have to know for certain. All of this work is set in the context of the complexity of making software products and the uncertainty that complexity brings. We don’t know. If we ship a new feature, is it going to create value for the customer? Sometimes we know, and sometimes we don’t know. If we know for certain, then there’s really no reason in even using an outcome focus to begin with.
For example, I want to get customers to visit my site more often. How do I get them to visit my site more often? I might have a dozen ideas. I might have 100 ideas. How do I know which one I should be working on? All of this, then, comes back to the lean startup notion of having a hypothesis and testing your hypothesis with an experiment or a minimum viable product or something like that. Teams can say: “We think that if we make this feature, it will generate this outcome. Honestly, we’re not sure, so our next step is not just to go all in and make the feature, our next step is actually to test our hypothesis by running some experiment.” That experiment might be to make a minimum version of the feature. It might be to talk to people. It might be some kind of research. I don’t think you can ever be 100% certain, but you want to make an effort. This is specifically a technique that, when you mix it with hypotheses and experiments, lets you work your way through that uncertainty.
Brian: Uncertainty is such a powerful way to frame this. I think this is another aha moment: it’s not all or nothing. Sometimes, just building the damn feature is the right thing to do because there’s sufficient certainty of the value. It’s well enough understood. The risk, of course, is that Product Management 101 says that shipping a feature without understanding the problem is pretty risky. Normally, you try to go over to the problem space and really get more confident in it, and then you can start to progress with more confidence in solving it.
What’s interesting here — and I’m keen to get your perspective on this — is a problem-focused approach versus an outcome-focused approach.
At Intercom, 40% of our time is spent defining the problem before we start designing anything.
Let me use one example we’re literally working on right now, which I think hopefully is easy for anyone to get their heads around. We’re improving Inbox search. We’ve got a team inbox, which Intercom customers use to respond to their end users. There are a bunch of people working in this inbox. For years we’ve had a really basic search option, and we’ve got heaps of years of feedback of people telling us of how it’s not doing the job. They can’t find the conversations they’re looking for. We’re pretty dialed in on the problem we’re trying to solve.
But when we move to the outcome, it gets a little shakier. It gets less certain. We could say we’re trying to get people to do more searches, maybe. Or maybe we’re just trying to get them to get accurate search results, which is hard to exactly measure. We think we’ve got some proxies for it there, but we’re a little shakier as we go out. If we go from problem to outcome, it’s a little shaky, but we think we can get reasonably firm there. Then when we try to go to business impact where it’s a huge leap. When we’re working problems-up, going to business impact (where we hope this will affect retention, and we’ve tried to do some models where you can actually try to get some element of a sense of the scale of what the business impact is), it feels really loose and shaky.
And that’s a problem-focused one where you’re solid there, but then shakier as you go out as compared to an outcome-focused approach, like trying to improve activation. If you’re able to improve new customer activation, you’re confident that’s going to change your business impact, but you’re actually not sure what the problem is in the first place, so you’re way shakier on the problem. The lens of uncertainty and different approaches to building and prioritizing projects come with different risks of uncertainty at different levels. I’m worried I’m way too much in the abstract, here. Tell me if I’m making any sense.
Just shipping features and moving on is a huge problem for product teams
Josh: To go back to your example, you’ve got this search feature in the product. When you frame that, you know you want to improve it. That’s problem-focused framing: “We have lots of evidence that the feature is less good than it should be. We want to improve it.” The way you add maybe an outcome focus to that is by asking, how will you know? How will you know that you’ve improved the feature? Specifically what will people be doing differently after you’ve improved the feature? That’s a user behavior that is the outcome question. After we’ve improved inbox search, people will get to search results more quickly. People will not run three searches in a row, but they’ll search, find and go. That’s the first question: what will be people be doing differently as a result of this feature?
That’s really important, and I want to stay on that question for a second before I get to your next question. That’s really important for product teams, because it’s often the case that we ship a feature and then move on. Somebody asks for a feature. We ship it. We’re done. We move on to the next one. We never go back and ask: “Are people using this feature? Are people getting the value that we expect them to get from this feature? Have they adopted the feature?” After we’ve made the feature, we want to have actually validated that the feature is behaving as intended because people are using it in the way that we expect. That’s a really important discipline. It helps us know whether we can be one and done with the feature or whether we need to go back and iterate. Just shipping features and moving on is a huge problem for product teams. Then you leave this rubble of half-finished features behind you.
Brian: Absolutely. Product data is a real thing. You’ve got to validate. Did you solve the problem? Sometimes, you try to use outcome as our way to measure that as well as qualitative feedback there. Going back to a feature-driven approach, it feels like the anti-patterns or the riskiest ones are on the outside, which is one end from the business impact. If you have a project that’s on business impact, that’s high-risk. We’ve actually unwittingly asked teams to try to do this. They’re like: “We don’t even know where to start. We don’t even know how to think about this thing because we’ve just skipped to the end.” The way to approach this is to start with figuring out what customer outcomes you’re actually trying to orient this team around. You’ve got to do some work. You’ve got to do some research to actually identify those outcomes that map to that business impact. That’s an anti-pattern: to try to shoot from business impact straight into: “Okay, team. Kick off. Go move into execution mode here.”
Josh: That’s actually something I see a lot in my work with organizations. Organizations will be pretty good at defining the high-level impact that they want: say it’s to reduce costs or increase customer loyalty — whatever the focus is for that period of planning. Then they skip that middle layer, which is the outcome layer. They jump right to features and say, “We’re going to reduce costs by implementing System X.” No, you’re not. You’re not going to reduce costs with one feature. If you’re a team that’s been given a mandate — say you’re a team of five. Even in an organization of 20, you’re going to have a hard time reducing costs less than in an organization of 100 or 500. When you start to see those kinds of teams dealing with these very high-level business impacts, that’s a smell. You know there’s a problem there, and you need to decompose the problem in more detail.
Brian: Exactly. It’s the smell of, “This is unlikely to go well.” I think it comes from two sides. If it’s a feature-driven one, which is typically what happens, the risk is that you’re too far away from what the actual core problem is. Try to reorient that way before you progress down. If you’re coming from business impact, you can’t just jump straight down to the feature. You’ve got to orient around that customer income. It’s the safety net for how to get far more confident that you’re building something of value. That’s how we’re trying to put this into our way of thinking about building product without necessarily upending the whole approach we’re doing.
How to prioritize and execute
Brian: Another angle you mention in your book is execution and prioritization. Outcomes seem like they most naturally fit a prioritization, or is that a leap?
Josh: One of the things that happens when you start working with outcomes is you have to work to come up with your theory of how the business works. The non-profit world calls this the theory of change. What’s the big picture — where people are behaving this way, then these good things happen and that has these results for the business?
Starting to build those models leads very nicely into roadmapping and prioritization
The more you know about your business as your business goes on, the more you can start to build that. I don’t know if you want to call it a business model or a behavior model or an interaction model, but you understand the way your system works as a company. You have to do the work of saying, “We want to reduce costs, and so to reduce costs, here are the 25 things that people are doing that waste money.”
Starting to build those models leads very nicely into roadmapping and prioritization. If you’re a brand new startup, and you’re two months in, you don’t really know that stuff. Everything is uncertain. If you’re a few years in, you’re making money. You have some understanding of what your customers value about you. You know so much more, but you’re still looking across this model and saying: “Given what we’re trying to do, here are all the possible areas that we could work on. What are the things that we understand? What are the things that we really don’t understand and need to investigate? Where is the work that we need to do?” All of that happens as you start to build a model. Then, that model then gets reflected in roadmaps and prioritizations.
Brian: It’s a model. You really want all of R&D to understand, right? That’s something we’ve only done recently: actually bringing them through how our business works. It’s a complex business. There are a lot of pieces to it. We want everyone to understand them, so they can map what they’re working on or make those logical jumps and understand the work they’re doing and how that could conceivably impact the business model. It’s both having that model and sharing it and getting people fully immersed into it.
Josh: I think it’s more than just having everyone in R&D know it. I think everybody in the business needs to know it. Sales needs to know it. Marketing needs to know it. This is how you’re making the strategic decisions about what kind of accounts you approach, which market segments you’re chasing. All of it is tied back to this model. If you change one piece, suddenly enterprise sales looks great as opposed to direct-to-consumer. That’s a huge impact in terms of the prioritization that’s going to happen on your feature roadmap. This model needs to be an organization-wide picture of the causes and effects and the relationships.
Change is harder for leaders
Brian: Let’s switch tack here to the human side of it, the motivational side of it. When you move toward an outcome focus, is a team more motivated in their operations? Are you likely to actually get better, more creative ideas coming from this model because they can more directly see the impact of what they’re building?
Josh: I think so. At the end of the day, when you go from telling teams, “Make me a feature,” or “Make me an output,” and you switch to saying, “Create an outcome,” there’s a big difference. In one case, you’re telling them what to do. In the other case, you’re giving them a problem to solve. With that second way of working, you’re really granting the team more autonomy. Most teams find that autonomy appropriately scoped — and autonomy is very, very motivating. One of the really foundational books for me in managing software teams is a book called Peopleware. It’s a pretty old book by now, by a couple of guys named DeMarco and Lister, but they write that people who write software for a living and by extension, people who think for a living, tend to be intrinsically motivated. They tend to be people who come in to work and want to do a good job every day.
You have to say: “We’re not promising you any features. We’re promising you outcomes.”
As a manager, your job is to make it possible for those people to do a good job every day. One of the ways you can do that is by saying: “Here’s your problem. You own this problem. Go solve it.” From a leadership dynamic point of view, that’s the foundational shift between outputs to outcomes. It does have the second change, though, which is that it really changes leaders’ behaviors. It’s actually, I think it’s much easier for teams and much harder for leaders.
Brian: Explain that.
Josh: Leaders have to get out of the business of saying: “I have this solution in my head. Go do it.” Instead, they have to do the work of saying, “Actually, what problem am I asking people to solve?” And I have to be okay letting go of that control and giving the control to the team. If you’ve grown into that leadership role, presumably you’re pretty good at solving problems, which is why you got promoted. Now, suddenly you’re in the business of not solving problems any more, but delegating problems. That’s a new skill to learn. I think a lot of leaders hold on to the old skill because for all of us, it’s easier to use the skills we’ve mastered than to try and learn new ones. In my experience, just to come back to the question, it’s yes. It’s super motivating for teams. Teams love it, and the harder human change is in that leadership layer.
Brian: That’s interesting. That really resonates a lot. It’s a significant amount of effort to figure out that middle layer of the customer outcomes to go after, because you’re going to want some confidence in their value. You’re not going to just want to make it up. I think it’s not sufficiently recognizing: “Hold on. If we’re wanting to switch to this, we’ve got a decent chunk of work in front of us so that we can really frame the outcomes we think are worth betting on.” I’m sure, for some companies, some products, that’s more straightforward. At least certainly from where I’m coming from, there are a lot of possibilities out there. Underestimating the amount of work to get there is a real risk, so that resonates a lot.
Change starts small
Brian: Any other final advice that jumps out at you for thinking, for teams looking to change their perspective here, change their way of working?
Josh: In general, the way to do it is to start small. If this resonates with you, if you’ve read the book, and you’re excited about the ideas, start small. Pick one project or one set of stories where you’ll use the language of agile. Use one epoch that’s going to be outcome-focused. Try it. Expect it to go okay, not great. Expect to use retrospectives and to continuously improve your process as you try to implement this, because you’ll learn a lot.
You need to go into it with realistic expectations across the team that you’re going to have to take a couple of cracks at this
It’s very easy for me to sit here and write a little book that generalizes all of this stuff, but to actually apply it to the complexity is much more difficult. You live this stuff every day. You live the complexity of this product. To actually put it to work in the real world is challenging. You need to go into it with realistic expectations across the team that you’re going to have to take a couple of cracks at this before we start to feel good about it.
You have to push beyond the boundaries of your team
Brian: What resonates about what you’re saying is that we’ve had some teams internally who have said, “We’re going to push ahead here and really experiment with different ways of working.” But they’ve also realized that they have to hit a bunch of different levels. This is at the cultural level of how you build and think and talk about stuff, how you demo stuff, how you celebrate stuff as well as moving down to the actual process of how you execute it. It’s hitting on so many levels of how you function as an org that you’ve got to chip away at a bunch of different levels at the same time and see where you can get more confident — rather than having one holistic overview grand plan of: “We figured it all out. Now it’s time to put it in place.” God knows that would take a while.
Josh: You’re really right, and thinking about it at the cultural level is really important, because that also is going to push. To do this well means you have to push beyond the boundaries of your team. It’s not something that’s just self-contained. You need to be able to talk to the manager, talk to the stakeholder, talk to the salespeople, talk to the people who have visibility into the roadmap, who are asking, “When is this feature coming?” You have to say: “We’re not promising you any features. We’re promising you outcomes.” Then you have to fight with them for a couple of hours about that. There’s a lot of communication and change management that happens with this at the cultural level, and it is broad in the organization.
Brian: Why don’t we wrap it up there? Really appreciate you taking the time. I enjoyed our conversation today, Josh. Where can people get your book, keep up with your thinking?
Josh: The book is available on Amazon. It’s available in print, in ebook and in audiobook on Audible, and you can follow me on Twitter, @JSeiden. If you read the book and liked it, I’d love to hear from you.
