Data points: what should your startup measure?

When you’re building a startup, data can be like a warm blanket on a cold night. Data is comforting. It can be trusted. It’s black and white with no apparent shades of gray.

As Harvard Business School professor Clay Christensen has said, “Many people view numerical data as more trustworthy than qualitative data.” The reality, however, is that data can introduce even more uncertainty into your business if not used correctly. Because humans are deciding what to measure, how to measure and why to measure, almost all data is built on biases and judgment.

That doesn’t mean you should ignore data and act on intuition alone. The point is that data is only one lens on your business. What your users are saying is another perspective. What you want to do internally is another. What makes financial sense is another.

Get on the same page early on

I came to Intercom right after we closed our Series A funding round. At that point it was just product people at Intercom. Our series A investor, Mamoon Hamid from Social Capital, looked around and said it was time to get the house in order from a finance and measurement perspective.

Joining at that early stage, right before we hit one million dollars in annual recurring revenue (ARR), was really insightful. Eoghan and the founders had done a great job setting up the fundamentals, but by bringing in finance and analytics early, we could start to understand deeper questions about the business. We could figure out where that ARR was coming from. How could we think about it moving forward? Why might we need future capital? When might we need it? Who might we want to start hiring?

Getting these questions answered made our later rounds of fundraising easier. Your job while fundraising, whether you’re the CEO or the CFO, is to use data to build a case for why your company is going to be X times more valuable than it is today. Then it’s up to you to put that data together in a compelling story.

Intercom’s first pitch deck

Bringing finance and analytics in early also meant we could all get on the same page regarding metrics. One mistake I see at a lot of other early stage startups is that they don’t have well-defined sources of truth. People start talking about churn, but what are they really talking about? Are they talking about gross MRR (Monthly Recurring Revenue) churn? Are they talking about net MRR churn? Are they talking about customer churn? Too often, metrics get thrown out, and it’s unclear exactly what they mean. It’s easy to spend a lot of time arguing about the actual data and numbers instead of figuring out what that data and numbers are telling you.

The real challenge is working out how to turn signups into long-term customers.

For example, if Apple were driven by data, they would have shut down their Genius Bar after years of no activity. If Ryanair were customer driven they’d remove all their sneaky fees and charges. If Zappos were driven by margins, they’d abandon their generous returns policy. They’re all just perspectives. Just because data is objective, it doesn’t mean that it guides you to the right decision. Just because it’s precise, it’s not necessarily valuable.

Early stage companies have limited resources in terms of attention, so it’s critical to focus on data that’s important to them, not necessarily what conventional wisdom tells them to measure. For example, If you’re looking to disrupt a well defined market, then you should know that the success of your company won’t be measured by the same metrics used by the big players in the market. Airbnb and the Hilton group have very different definitions of success, but each are valid for their business

Alternatively if you’re establishing a new space, or new type of product, it’s nearly impossible to know what metrics you should be optimizing for. Sure you study your signup funnel, optimize your landing page and simplify the onboarding. That’s the easy bit. The real challenge is working out how to turn signups into serious long-term customers. The relevant metrics for this only emerge over time. Until then you can pore aimlessly over your dashboards, or as we’ve always found to be more productive, start talking to customers.

Regardless of your business, you’re likely going to want data to help you do two things:

1. Grow your product
2. Grow your business

Data to help your product grow

The world of startups is full of “golden rules” about dated metrics like bounce rate, time on site, pages per visit and more. For modern software products, these rules are bullshit. It’s not that they’re incorrect, it’s just that these rules offer information that isn’t actionable and doesn’t matter to your business. Google’s Avinash Kaushik puts it best, “The average conversion rate in the United States of America, whether you’re selling elephants or iPods, will be 2%”.

We’ve found three metrics to be fundamental to understanding the overall health of a product:

1. Intent to use: The actions customers are taking that tell you definitively they intend to use your product e.g. customer has imported custom data.
2. Activation: The point when a customer gets real value from your product e.g. customer has invited five teammates in seven days.
3. Engagement: How much, how often and how long customers continue to gain value from the product e.g. the number of photos uploaded per user per day.

These metrics mean very different things to different businesses. A $99 B2B SaaS app defines engagement very differently from an e-commerce website. So rather than fixating on what others are measuring, take time to understand which metrics are most important for your business.

Data to help your business grow

When a VC comes knocking with questions about your business, you need to have your house in order. Know the difference between your gross churn and your net churn.

Know the difference between your gross churn and your net churn.

It takes time to identify and understand each of these metrics in the context of your business. More importantly, it takes deep analysis and constant experimentation to understand how you can impact them. You don’t want to be considering these metrics for the first time when you get a query about them from a potential investor. Plan accordingly. If you don’t already have an analyst on your team, seriously consider hiring one that can own this.

With today’s analytics tools, it’s easy to measure hundreds, if not thousands of different metrics for your business. Cutting through all the noise to determine the most important or insightful metrics can be quite a challenge. If you’re struggling to identify the most important ones for your business, we’ve written before about the most important financial metrics startups should measure.

Beware false gods

Every business is so different. Every business has a different story, is selling to different segments, has different market positioning and has different buyers. For us, and it’s something I’d advise any other startup to do, it was all about finding the metrics that proved Intercom was on the road to being successful.

Data can be a powerful tool for any startup, but a lot of times it tells a story of what happened in the past, not necessarily what will happen in the future. If you simply rely on data, it can end up bringing you to a local maximum rather than allowing you hit your full potential. So you iterate and optimize instead of taking big bets, and never take the steps required to get you out of that local maximum.

The highest rewards come from the biggest risks. Sometimes you need to make a leap of faith, regardless of what the data says. That’s when things gets exciting.

---

Adapted from our recently published book Intercom on Starting Up.