Build A Better Business By Choosing The Right Metrics

For the last 8 years I’ve been helping businesses get value from their data. This has ranged from setting up a company’s first dashboard to building machine learning products but most recently I have been helping startups set up metrics to track. This is something that sounds simple but is actually quite easy to get wrong. After reading this blog you should know what kick-ass metrics look like and how to avoid the most common pitfalls.

Do metrics mislead or are we using the wrong ones?

One observation I have made in my career so far is that everyone talks about being data-driven but not many have an idea what it really means. Does it mean being like Spock from Star Trek? Is it possible to be too data driven? Before we tackle these questions let’s set up some fundamentals. It’s impossible to unlock the potential of your data if you aren’t tracking the right metrics. To make this clear, I want to start off with a real-world example.

The McNamara Fallacy

Last week I read an interesting piece about the McNamara fallacy and the dangers of being too data-driven. The jist of the fallacy is that being solely data driven leads to bad outcomes with the core example being Robert McNamara’s direction of the United States army during the Vietnam war in the 1960s.

McNamara, business executive turned Secretary of Defense, served from 1961 to 1968 and led the American strategy in Vietnam. McNamara tracked the progress of the war by focusing on the ratio of enemy fatalities to American fatalities. As long as there were more enemy deaths than American deaths, McNamara concluded that the military was on the path to victory. History tells us this was not the case as ultimately the war ended in failure for the USA.

Hence, McNamara’s story is used today as a lesson in how to not be data-driven and that context and qualitative data are just as important as metrics. Whilst I agree with this as a concept, I think it’s important to consider how a common pitfall is actually choosing the wrong metrics rather than being too data-driven. Had McNamara had numbers on how the size of Vietcong was increasing through the course of the war despite its losses, perhaps he would have considered an alternative strategy.

What are metrics?

First of all let’s be clear what we mean by metrics in the context of business. Metrics are quantifiable measures used to track processes to judge the performance level of your business. There a couple of key components of that statement:

• “quantifiable” - If it’s something you cannot convert to a number of quantify, it’s not a metric and needs to be tracked some other way
• “judge the performance level”- The metric is only useful if it can be used to benchmark in some way to understand performance. This means it can be compared to competitors or yourself at another point in time (Eg 1 year ago). It’s useless therefore to have a metric that no other company has or to change your metrics often because you will never be able to draw comparisons‍

Metrics checklist

Now we know what metrics are, let’s consider what good metrics look like:

• Drivers = Make sure your metrics represent the performance of your business. If you’re a telemarketing business, the “number of calls made” alone would be a bad metric. If no calls ever convert you wouldn’t make any money. Call conversion is a better metric
• Distinct = Don’t use metrics that are too inter-related so that each one gives you new information. Don’t track “Number of active users” and “total user sessions” as two core metrics. Replace the second with “sessions per user” to make it more informative
• Ratios = Using ratios rather than absolutes makes your metrics more comparable over time and in general (more on this in the section below). “Sessions per user” can be benchmarked over time better than “Total sessions” for a growing business
• Comparable = If you want to compare your performance against competitors make your metrics industry-standard. Calculate your retention using revenue vs users if that’s what others are doing.
• Well-defined = Define your metrics explicitly and clearly. They cannot be valuable if no-one understands them
• Measurable = Make sure the metric can be measured accurately. Don’t track the source of your customers (where they came from) if you can only capture it correctly 50% of the time
• Trustworthy = Links very much to measurable but make sure your data can be trusted. Don’t rely solely on survey data for metrics where bias can often play a big role.
• Segmentable = Choose metrics that can be broken down into component pieces. Tracking number of active users is not so useful if it cannot be split by New vs Existing users or geography for example

Too many metrics creates noise

The first common mistake I see in businesses is that they try to track too much and it becomes overwelming. With the explosion of Big data tech and the ability to track everything, many companies are doing just that. In reality however if you measure every data point in your business and create 100 dashboards with 20 charts on each, all you are creating is noise that only a computer or a rocket scientist can read. A study in 2018 showed that the most viewed dashboards have between 15–20 charts per dashboard. This might be what dashboard lovers like to build but it just not practical for the average decision-maker to have to decipher 20 dynamic charts in real-time to run their business.

Instead, I would encourage businesses to identify the handful (5–10) metrics that truly drive their performance and align the whole team in understanding and tracking those. Less really is more when it comes to analytics and when you get more people engaged in data you naturally become more data-driven.

Why ratios matter

One thing that McNamara did right is understand the power of ratios. He understood that doubling the number of enemy casualties was not a victory if it meant a tripling of your own casualties. Ratios are a powerful tool that can simplify the number of things you need to track whilst simultaneously improving insights from your metrics.

My go-to example here is one from a successful SaaS business I worked with a couple of years ago that had a solid analytics capability. They were trialling a new feature on one of their products and found that it increased user growth by 30% as well as bumped up revenue by 10%. This satisfied their two core Key Performance Indicators (KPIs) and so they decided to roll-out the features to all users. However, what they didn’t realise was that the revenue per new user coming from the new feature was half that of the average user. This killed their Customer lifetime value (LTV) and in-turn profit margin took a 5% hit. If they had focussed on the single ratio of revenue per new user that really mattered they would have seen the clear picture from the beginning.

There are many opportunities to implement ratio KPIs across businesses that are often not considered today and I would encourage decision-makers to think outside the box to implement ratios as often as possible. Cross-functional ratios are especially powerful and are something we love at Calliper. Cross-functional ratios are ratios that use metrics from 2 different functions / departments such as Finance and Product or Sales and marketing. Having north-star metrics that are cross-functional aligns different teams to the same goals and drives collaboration.

Here a few examples of cross-functional ratios you can use:

• Sales efficiency (Sales ARR as a ratio to Sales & Marketing spend) is a cracking example that incentivises both Sales and Marketing to focus on the channels that have the best payback per dollar spent. It’s calculated by adding up new revenue generated during a certain period (let’s say a quarter) and dividing it by the cost of the Sales & Marketing teams’ efforts. These expenses would include salaries, benefits, commissions, office space, software, and ad spend
• Revenue per active user combines revenue and product usage data and is great for getting a good view on pricing dynamics as well as understanding whether you are increasing the value proposition to customers over time. If you are this metric should be going up
• Support requests per new user combines Operations and Product and is essential for understanding the onboarding effectiveness for a SaaS product. Over time as onboarding becomes a more refined, well-oiled machine you would expect this ratio to decrease as customers need to submit fewer support requests to get onboarded and get value from the product

An easier solution to choosing metrics

If all of this sounds quite complex and you want the right metrics to track straight out of the box it’s worth me mentioning Calliper. Calliper is a tool that sets up best-in-class metrics and alerts at the click of a button for SaaS companies and makes analytics accessible and actionable for the whole team.

Below is a recommended metrics template for a typical high-growth B2B SaaS business straight from Calliper. This “Mission Control” provides the health of the entire business on a single page highlighting the key ratios that matter: User growth rate, MRR growth rate, Revenue per active user and more. It opens up the power to quickly build an accurate and useful analytics capability to any operator instead of limiting it to data whizzes.

Do you know anyone that is currently going through the process of defining north-star metrics in their company? Please go ahead and share this post if you think they will find it useful.
‍

Also, please comment below the metrics you have chosen for your business which are interesting and that allow you to ask the right questions as a company. We’d love to hear!