Square’s Rohini Pandhi on spotting and solving customer problems
Product management is about solving problems. How can we help our users complete their job-to-be-done cheaper, faster or easier?
Finding those problems to solve, however, becomes increasingly difficult as you scale – where the volume of feedback, and noise level of the vocal minority, compounds by the day. Not to mention changes to workflows affecting an increasingly large amount of users.
Rohini Pandhi, product management lead for Square’s Invoices product, knows this challenge well. She’s worked in product from early-stage startups all the way up to publicly traded companies like Rackspace and Square, a company with the sensitive task of handling customers’ money.
I hosted Rohini on our podcast to talk about everything from prioritization and product roadmaps to the nitty-gritty techniques for parsing customer feedback. If you enjoy the conversation check out more episodes. You can subscribe on iTunes, stream on Spotify or grab the RSS feed in your player of choice.
What follows is an edited transcript of our conversation. Short on time? Here are five quick takeaways:
- Instead of relying on word of mouth to permeate the team, write things down. It’s like rewriting your notes in school: the information sticks, and you can communicate it effectively to others.
- Iterate on process the same way you iterate on product. If a process doesn’t stick, don’t hesitate to move from it.
- If Rohini’s team is making changes to the UI or the front end, they can test quickly and easily, but because their core product is payments-based, much more rigor is required compared to your typical software product.
- Survey responses and numerical data are great, but they don’t replace the context of a 1:1 customer conversation. To this day, Rohini continues to cold-call customers.
- Rohini uses a “T-shirt” method for prioritization: categorizing problems into extra small, small, medium, large, and extra large – then assigning scores of one through five to help determine your focus.
Michelle Fitzpatrick: Rohini, welcome to Inside Intercom. Can you give us a quick feel for your career to date, as well as what you’re doing today at Square?
Rohini Pandhi: As I look back on my career, a lot of it was learning what I didn’t want to do. I studied computer engineering in school, and I love math, but I realized I didn’t want to be a computer engineer when graduating. I went into consulting for a few years, then realized I didn’t want to do it forever, so I got my MBA. Through having the technical and business background, I realized product management kind of fit, but I didn’t really know what it was – I don’t know if any of us really know what it is.
Michelle: I discovered product management through a tweet from Paul Adams. I looked it up and thought, “That’s what I want to do.” A number of years ago everyone was just discovering it through different channels.
Rohini: Now it feels like everyone knows what it is – or it feels more known. I went to the Grace Hopper Celebration last year, and everyone was asking me about how to get into product management. I was like, “You’re all in undergrad.”
Michelle: I know. When I was in school, there wasn’t a career path, or it wasn’t spoken about as an option.
Rohini: Exactly. I went into the startup world, figured out product management, and then came to Square about two years ago. At Square, I manage a product called Square Invoices. A lot of people know Square through our hardware: credit card readers that make that swiping action from back in the day. But a lot of our merchants aren’t face-to-face with their buyers, or they have a sale that happens 30-90 days after they deliver a good or finish a service. For those types of sellers, we offer Square Invoices where they can send a digital invoice to their buyer, and the client can pay online. All the magic happens through Square.
A reliance on writing
Michelle: As someone who’s worked in product at early-stage startups all the way up to a publicly traded company like Square, how do you think the role of the product manager changes as a company scales?
Rohini: As a product manager, you have to fit in and solve problems – or fix the gaps you see and then move out of the way for everything else to happen naturally. As you scale, I don’t know if your role changes as much as your flexibility shifts. You’re filling in different gaps, or you’re trying to find out where the biggest need is to go solve that problem and, again, stay out of the way of everything else that’s actually working. As you scale there are more people and more processes, so communication becomes really important. It’s an n-squared problem. It’s always like that, and at scale it becomes even more so.
We have documentation for everything we’re doing. It provides transparency.
Coming from the startup world to Square, I’ve learned how much written communication is critical to our process. We’re very much in the Amazon-esque style of working – we have documentation upon documentation for everything we’re doing. It provides transparency. As a product manager, one thing I struggled with, but I love now, is being articulate and comprehensive in writing, so the narrative is clear to anyone who is reading it. It helps you realize, “Oh, this is why we’re doing it.” There’s no option to hand-wave over the details, and then it provides that direction and articulation of the story to the team as well.
Michelle: I find that, too. We write a brief at the start of every project; it’s pretty short, maybe 250 words. To write something that explains the problem and is really concise is such a good forcing function to make you think about it and articulate it. Then you can talk to people clearly.
Rohini: It’s almost like when I used to do my homework in elementary school: if you have to rewrite your notes, it sticks with you a little bit more. You know the story in your brain. If you write it down, you can communicate the information in any other medium very easily.
Michelle: When I joined Intercom, it was much smaller: under 50 people. As we’ve grown, roles have been created, and those roles have turned into teams of people. The challenge has been keeping everybody in sync and communicating. Relying on face-to-face communication just doesn’t scale. That’s where using written communication and having some canonical documents in a project has become super important.
Rohini: Absolutely, especially if you’re the new person and trying to get up to speed. If it’s just tribal knowledge or hallway conversations, it takes so much longer to understand what the rhyme and reason was for any of the decisions made in the past.
Iterating on process
Michelle: As we’ve scaled we’ve introduced new processes. Sometimes they’re designed to help us work faster and sometimes to help us work better, but they don’t always work as intended. Do you have any examples of times when the process you implemented didn’t work as planned?
Rohini: Sometimes the word “process” gets a bad rap. When you look at it through a product lens, if the process is not working – if it isn’t serving the people it’s supposed to be or it doesn’t help you get your job done any better, faster, quicker – it becomes this ubiquitous thing everyone has to do but no one actually believes in. What we do on the Invoices team is iterate on process just like we iterate on product. Our team has tried a lot of different things; some work better than others, and we’re constantly iterating. For example, we recently tried a squad model for our engineering team, splitting everyone up into smaller squads. That didn’t work for us at this time, but it doesn’t mean we won’t try it again in the future.
Michelle: What were some of the challenges that you found?
We iterate on process just like we iterate on product.
Rohini: A lot of it was the communication piece. With so many projects going on, the core tech stack had to be managed by the entire team. When certain projects were releasing, half the team knew about it, but half didn’t. If there were bugs or tickets filed, whoever was on call had to deal with it regardless of which squad they were on, and so not knowing what was happening was the bigger issue. That was a core root problem, and the squad model didn’t fix that for us. It actually made it worse. But there’s going to be a tipping point where the team just becomes too big to be able to keep going in this unified route, so we’ll have to cross that bridge when we come to it.
Michelle: How do the processes then work at Square? Is each team free to work in different ways than other teams? How much is left for each team to figure out for itself?
Rohini: We have a good deal of autonomy within the teams. The thing I really like, and the reason I think the process works so well, is that you create a custom culture for your own team. It doesn’t have to happen at a big, grand level across the entire company. There’s almost a unique fingerprint that each team has. For example, our roadmapping process is customized in ways that work for our team. Then, we communicate it out in a singular fashion across the rest of the organization.
Bringing analytics into the fold
Michelle: Square is widely known as a data-driven company. What is your relationship like with the product analytics team? Do you have an analyst that works embedded in your team?
Rohini: We have an analyst on our team, so product analytics really does have a very close relationship with the product group. My job as a product manager is to fill in gaps where needed and then get out of the way for the smarter people in the room to do what they need to do. Product analytics and our data science teams are two examples: we work very closely together, but I try to stay out of their way so that they can do the best job they do.
Michelle: How do they influence the projects? Are they heavier at the start to help define the problem, or is it more at the end to measure success after you’ve released something?
Rohini: It’s both. Our product analyst is able to define the problem and our success metrics, and give us recommendations before we start the project. We evaluate the real problem from the quant side and create the metrics we should be driving towards. Then, we look at things as we’re releasing. If we’re doing an A/B test, our analyst is heavily involved in looking at that data. Then at the end of any project, she wraps it up: “Okay, here’s what we expected. Here’s what we actually saw, and this is good or bad because…”
Michelle: I’m curious about A/B testing because that’s something we don’t do a lot at Intercom. Can you give me an example of how you test at Square?
Rohini: We try to do as much A/B testing as we can. It’s probably not as quick and easy as many startups find, but we try to experiment with a lot of our new features as much as possible so that we’re not creating bad experiences for people, and we’re driving the right types of actions. Why doesn’t Intercom A/B test?
Michelle: We run betas of things. We’ll develop something, release it to a small group of users and compare it to a control group to see how it’s performing. We don’t necessarily build out two versions of a solution and compare one against the other against a control group. That’s just the way we’ve been working, but it will be interesting to see, as we grow and look at different challenges, if it will make more sense to try a couple different options.
Rohini: Definitely. We recently did an A/B test of a new form redesign and blogged about it because analytics, engineering, product and design were all part of this. We looked at how our form felt to users from an experience perspective. We saw in the data that the users’ eyes had to zigzag across the form in order to fill it out. If we use an “F” shape of how you should fill out a form – where it goes top to bottom, left to right, and then a little bit less data on the second part of that form – it was easier to fill out.
The redesigned “F shape” Square invoice. Read about Rohini’s team’s process here.
You could see that not only from the design, but the quantitative data showed people actually filling out the form more than before. You just couldn’t grok all of the information that was on two columns of a screen, and so we A/B tested that just to make sure. It’s closer to your beta process.
Michelle: That’s similar to beta. We don’t try two complete versions of something, but we do a lot of what you’re describing where we test with a small group, get feedback, make some iterations and try again. Some customers have seen it in very different shapes the whole way through.
Rohini: Do you have a process on the usability tests you do along the way?
Michelle: It varies between products. For some, we will do interviews with customers and watch them using it. Other times, we’ll just talk to them through Intercom, and we’ll look at data. It really depends on the kind of change we’re looking for. We’ll set our hypothesis for what we’re expecting and what we’re looking to learn during this beta. They’re either things we’re uncertain about in the solution or things that we’re trying to verify before we release it to a larger scale.
Rohini: That’s exactly how we do it, too. It’s either the uncertainty or the biggest risk you’re making an assumption about. You want to validate that so you can feel at least halfway decent about releasing something.
Michelle: When you’re working with financial payments, the margin of error is a lot smaller than if sending a message goes wrong with a customer. How do you assess those kinds of risk? Is there a lot of rigor that goes into testing and validating before any change goes to customers?
Rohini: Definitely, because we’re a payments company. If there are changes happening on the UI or the front end, we can test and move quickly. Those aren’t as much of a problem. But as soon as any change touches the way we send and receive money, that’s when we want to be so sure about everything we’re doing, because cash flow is really important for our sellers. These are a lot of small and medium businesses that need the cash at the time we said we’re going to deliver it. That is the core piece of our mission. We take it very personally to make sure that we don’t make any dumb mistakes.
Above all else, talk to your customers
Michelle: How do you look at the more qualitative side of feedback? How do you build a sense of the way people feel about your product?
Rohini: I love talking to customers. That is the one job of product that I think is most important. If you do nothing else, at least talk to customers or the folks that are using your product. I do interviews at least a few times a week: cold-calling customers and chatting with them. People are very responsive to that. They love sharing what’s working and definitely what’s not working. That input is really helpful to take a pulse on where we are. We also do things like qualitative research and usability testing. We try to get people through the door into Square at least once a week. We’re just starting out with that, and it’s been amazing. We love that process. We’re also doing NPS surveys. We’re doing other quick surveys on our dashboard that pop up every once in a while and ask people, “Is this working for you? How satisfied are you?”
Michelle: How do you know if what you’re hearing from customers feels representative of the large and varied user base that you have?
Rohini: You have a pretty good sense of it once you start hearing the same story repeated back to you a few times. You can pull that thread and find there’s a big nest of problems. There isn’t a number that we look for, like after 10 responses we know for sure it’s a 10x problem. It’s more that you hear the same pattern, and then you hear the pain that comes with that pattern. That’s why I love getting on the phone with people: you can hear an emotional chord, and you know there’s something very painful about this problem that you haven’t solved for them. Survey responses and numerical data don’t really capture everything that is actually happening or the color and context of actually talking to somebody.
Survey responses and numerical data don’t really capture everything.
Michelle: In those calls every week, are you asking general questions about Square, or do you use it as an opportunity to dig into whatever area you’re focusing on at the moment?
Rohini: In the weeks when we definitely have something we’re working on, we want to share that with as many people as possible. During slower weeks, I just ask general questions. We have a community forum with Square, and we did an ask-me-anything, Reddit-style conversation. That was very cool. It was like we did 50 customer interviews in one hour because people were just pinging us with random questions. As part of that, I offered out my personal calendar to everybody, and folks signed up and said, “Here’s what I want to talk about.” Those conversations were definitely more direct. Otherwise, it becomes, “Hey, can I talk to you? You’re a customer. I want to hear your feedback. Would you have 15 or 20 minutes to chat with me?”
Michelle: You mention how being on the phone can give you a sense of the problem’s magnitude that is hard to express in words. That can inform your gut and give you a conviction that this is the problem you need to solve. How do you balance that against the data? Do you ever find that they conflict with each other?
Rohini: Again, it’s about finding patterns. There might be a one-off, where someone is very frustrated or enthusiastic about something you’ve built. But when you hear it as a pattern, the data should support that. We don’t run off as soon as we hear one input to go build something that’s very niche.
When you’re on the phone with someone, and they actually choke up because they got a Square Capital loan at the moment that they needed it most, and they were able to sustain their business for another week or through the holidays, it just gets you.
Michelle: Do you bring that insight back to the team and share it? How do you get them to understand things the same way you do when you’re spending so much time talking with customers and looking at data much deeper than they could?
Rohini: It’s an interesting balancing act, because you don’t want to force the team into all of the meetings you have, but you do want to share these real conversations with people. We have engineering, design and marketing join these conversations whenever I have them, but it’s very much optional. Then, in our new weekly usability tests, we actually create a secondary conference room where people can dial into the meeting. That way, the participant who is actually in the office doesn’t feel overwhelmed by 20 Square employees in their face. We have a watching room where they can actually listen and see: where is this user tapping into the product? What’s working? What are they confused by? All without “leading the witness” too much.
The T-shirt method
Michelle: Given the scale of feedback that you’re getting from customers, prioritization must be a constant challenge for you. What kind of tools do you use to prioritize the work you think is valuable or worth doing?
Rohini: I use a framework I stole from Adam Nash, the previous CEO at Wealthfront, now at Greylock. He wrote a great post about metrics movers, customer feedback and the delightful and strategic parts of bucketing prioritization models. I use the same three general areas. In any given quarter, we ask: what are the biggest metrics we want to shift? It might be increasing revenue. It might be decreasing our loss rates. It might be something around acquisition or engagement metrics.
We then look at what our customers saying. There might be something the sales team is passing along from an enterprise merchant, whereas there might be other feedback I’m getting across all of our customers.
The third bucket is a catch-all for the delightful or strategic pieces. It might not be something that gets prioritized because it will affect a metric or a customer has been asking for it, but it will be a foundational piece for where we want to go in the next few years.
Sometimes I look at it like T-shirt sizing. Let’s say you have a project in your backlog: you can look at it at a small, medium or large scale within those three buckets. At Square we now quantify those on a scale of one through five. It’s still a T-shirt size, but one is an extra-small, and five is an extra-large. There’s so much we could build, and this helps create a discernible difference between all of these potential projects.
Let’s talk roadmaps
Michelle: I really love understanding how other teams and other product managers build their roadmaps. I always find – at conferences in particular – that if any PM talks about their roadmap and puts up a slide that has any visualization, you see everyone taking out their phones. No one knows what a roadmap actually looks like. It’s kind of like this dark art. Can you tell me a little bit about how you shape the roadmap for your team at Square?
Rohini: Two years ago, I was speaking at Industry, and I did show a visualization of a roadmap – there were so many phones out. It was like at a rock concert.
Each team at Square has the autonomy to build their roadmap however they want. There is a singular overarching roadmap, so if you do deviate from that, you have to go back and manually put in what’s coming out from your team that is significant or that marketing needs to know about. What we do on Invoices is create a Google spreadsheet that we’ve made into a Gantt chart. It’s super easy. It’s a tool that is flexible enough for us to use in whatever way we want. I link all product requirement docs and Jira tickets from there. Anyone who needs more details on a specific project that’s in flight can go into that doc.
We use that for roadmapping and then our backlog has that prioritization matrix. Once things bubble up to the top and we are planning for the next sprint or the next cycle, things get populated into the roadmap.
Michelle: How do you communicate the roadmap? Rich Mironov spoke here recently about how you need to communicate differently to different stakeholders.
Rohini: At Square there’s a cross-functional sync every so often where we say, “Here’s what’s coming up on the roadmap, here’s what’s been released recently, and here’s what you need to know about it.” We usually have some sort of documentation that lays out the top features or talking points that the salesperson needs to know, that the support person needs to know, etc.
Then, we’re usually working so closely together, even during the launch, that nothing should be a surprise to most of these teams in any way. I do not like surprising our cross-functional team members with anything, so it’s a process where we go over what’s happened and what’s going to happen. Then, for any details they need in between, there’s always Slack, finding me in the hallways, or just looking at the documents that we have linked.
Michelle: What kind of timeline is do you use? Are you looking at the next month, the next quarter, the next six months?
Product confidence is like a hurricane map.
Rohini: I look at the next quarter. I always give the analogy that product confidence is like a hurricane map. You’re really certain about the next month. From there, it becomes a broader and broader radius of where it could go. Of course, no timeline is promised, but I just want to give them a heads-up: “We do think this can be accomplished in the quarter. Things will come up, and we will not be able to foresee every challenge that arises, but if everything hits to plan, we’ll be able to deliver X, Y and Z.” Then there’s always the stipulation: “Do not tell any customers that this is the exact date. We do not want to share that information until we’re 100% confident we’re going to hit it, which usually doesn’t even happen until a week before launch.”
Michelle: That sounds very familiar. To close out, where can our listeners go to keep up to date with your insights or, more generally, what’s coming next at Square?
Rohini: The channel we use the most is Twitter. Our social media folks are hilarious, so I definitely recommend following @Square. If you need to reach me, I’m also there, @RohiniP.
