3 Ways to Develop a Metrics Oriented Mindset
The unexpected benefits of thinking with metrics
1. Ask ‘how will we measure this’?
This sound simple, doesn’t it? But it’s surprising how many folks still fail to ask this basic question when developing new product ideas, strategies or features.
New checkout flow. How will we measure it?
New reporting tool for B2B customers. How will we measure it?
Get yourself into the habit of continuously asking yourself, the product team and the wider business how you will measure feature requests and upcoming product developments.
The beauty of asking this question out loud and regularly is that not only will it help you to build a solid habit of measuring results but it will also encourage your stakeholders to think about how they might measure a problem or a feature request before they speak to you about it.
Fostering a metrics-oriented culture gently nudges your stakeholders to move away from vague notions of discontent such as ‘the product team haven’t delivered what we need’ to ‘our NPS has plummeted to 12 and the reason is the new mobile app has a 1* rating and is being uninstalled after 1 day because users aren’t happy with it. How can we address this?’
Practical ways to ask the metrics question
Asking the question of ‘How will we measure this?’ out loud is one tactic to developing a metrics-oriented mindset but there are a few other ways which are just as practical.
User stories – try asking the question when you’re building features by embedding metrics into user stories. If you want to make it clear to the engineering team that you’ll be monitoring the performance of the story after it’s been released, include the relevant metrics directly into the stories. This will motivate the team since they’ll be aware of the impact a particular story may have on the wider business.
The metrics can be embedded either in specific user stories, or, depending on your team setup, at the epic level instead. Either way, use your planning sessions to be explicit about how a new feature links to a specific metric and why.
This will not only motivate the team by providing additional context but it will also uncover any potential technical issues which might prevent you from getting the metrics you need. Occasionally, there can be legitimate technical problems which prevent or complicate retrieving the data you need to measure the metrics you want. Addressing the metrics you intend to measure in planning sessions can help to uncover this sooner rather than later.
Strategy – if you’re operating at the strategic level, be clear about how you’ll measure your strategy by asking the metrics question to yourself and the wider business.
Product strategy is the process of making clear strategic decisions and trade-offs which act as hypotheses that need to be proven or disproven over a set period of time. Without metrics, you’ll have no idea whether you’ve succeeded or not.
Netflix, strategy and the metrics-oriented mindset
In his excellent piece on Product Strategy the former VP of Product at Netflix, Gibson Biddle, articulates how his team measured strategy during his time at Netflix.
Netflix’s product strategies are characterised by 3 distinct properties:
- Strategies – what is the strategic decision?
- Tactics – how do we plan to execute this decision?
- Metrics – how will we measure the strategy?
In this example, you can clearly see that a number of clear strategic decisions were made by Netflix under Biddle’s tenure as VP and that each strategy has accompanying tactics and metrics.
If you’re operating at a strategic level, make it clear to the rest of the business how you intend to measure the success of your product strategy by adopting frameworks which clearly link your strategic decisions back to relevant performance metrics.
Linking product decisions and strategies back to relevant metrics sounds pretty straightforward – and it is. There’s only 1 problem: metrics are boring. Thankfully, visualising them makes them less so.
Podcast
Product Stacks Podcast by the Department of Product. A deep dive into the tools, frameworks and processes used by product managers from top companies.
2. Visualise your metrics
A powerful way to develop your metrics-oriented mindset is to create a metrics diagram which visualises the most important metrics for your product. Yes, that might sound a bit ridiculous since we’re not 7 year olds anymore and we all know how to put together a spreadsheet instead, but bear with me for a second.
There’s a few genuine reasons why visualising your metrics in this way is beneficial:
-
- Memory – physically doing something with your metrics i.e. turning them into a tangible object, will increase your likelihood of remembering the most important metrics for your product. Sure, life isn’t an exam, but remembering the metrics will prompt you to think about the impact of new features and strategic decisions against those metrics without digging through Gdrive to find the spreadsheet you originally stored your metrics in.
- Communication – ever tried chairing a meeting to discuss the importance of product metrics by using an Excel spreadsheet? If you have, you’ll know (and so will the people who were in the meeting) that it’s reasonably soul destroying. Humans prefer visual cues and a metrics diagram works much better at conveying the important metrics in the wider context of your product than a dry spreadsheet ever will.
- Motivation – involving team members in the creation of your metrics diagrams acts to motivate the team; if the wider team has been involved in deciding which metrics should be demonstrated in this way they’re likely to feel a greater sense of motivation and ownership of the metrics.
- Focus – focusing on too many metrics is easy. Focusing on a few metrics isn’t. Your diagram should fit onto 1 page which means it acts as an exercise in focusing your attention on the most essential metrics for your product.
Metric diagrams should aim to capture the most essential metrics for your product in 1 place. Simplicity is critical. You’ll need to be disciplined in your approach to building your diagram by ruthlessly prioritising which metrics are important to your product.
Of course, as your product evolves over time, so too will your metrics. During this evolutionar process you should be able to retire some of the older metrics which are no longer important. If, for example, you shift business model for a particular model from B2B subscription to a B2C marketplace, the metrics may be far different after such a fundamental shift. Have the confidence to know when to modify your most important metrics.
Examples of metrics diagrams
Here’s a few examples of metrics diagrams for different types of products. Many of these metrics are fairly standard for products operating within these verticals, but your product will no doubt have quirks – and unique metrics – of its own.
SAAS
In this example, we’ve highlighted some of the key metrics which you might include in your metrics diagram for a SAAS product:
- CPA – the cost of acquiring a customer
- LTV – the average lifetime value of a customer, or lifetime value of a specific customer segment
- Churn – the % of customers who stop using your product over a given time period
- Free to trial % – the percentage of customers who upgrade from a free version of your product to trial to eventually subscribing
Ecommerce
Your ecommerce product metrics diagram is likely to include some of the following metrics:
- AOV – average order value
- CPA – the cost of acquiring a customer
- LTV – the average lifetime value of a customer, or lifetime value of a specific customer segment
- Checkout conversion / abandonment % – what percentage of your customers are converting at the checkout vs. abandoning?
API
API product management is perhaps the unsung hero of the product management world. Managing a product which comprises APIs only and has no direct user interface with customers can seem a little unsexy but an API is simply an alternative mechanism for delivering value to your end customer. It’s essentially a product with the UI removed but it’s valuable nonetheless.
Here’s a few metrics you might include on your API metrics diagram:
- Calls per month – the number of API calls sent / received over a given period
- Failure rate – the number of failed API calls over a given period
- Customers – the number of external customers using your API in a given time period
- Revenue – the revenue your API has generated
These examples are for demonstrative purposes only and are by no means an exhaustive list of all the potential metrics you could measure within a specific product vertical. The critical task for you is to identify the metrics that might be most useful to demonstrate and share using a diagram.
How to build your metrics diagram
To build your diagram, list the metrics you’re measuring today and sketch out some of the most important routes your users take to trigger these metrics. For example, if you work on an ecommerce product, you’ll know that AOV and checkout abandonment are key. The next step is to ask yourself what steps do users take to trigger an instance of these metrics?
| Metric | Trigger steps |
| CPA | User sees ad and lands on a landing page or specific item |
| AOV | User adds items to the basket and buys |
| Referral % | Customer has great experience and is sent comms prompting them to refer for % off next purchase |
Link your trigger steps together so that when combined they create the overall narrative of the most important journeys your users take when using your product at a high level. By linking the steps and building a narrative you’ll be able to see the most important journeys from a bird’s eye view. In some cases, this visualisation may in prompt you to discover new metrics you hadn’t previously considered..
Try to get your team involved in the process of creating your diagram. Since the process allows you to see your product from a birds eye view, the process can be stimulating for the whole team which means it may be a suitable activity to try at an upcoming off-site or workshop.
3. Dissect other business’ metrics
The third method for developing a strong metrics-oriented mindset is to look away from your own product and instead examine other businesses and products and speculating what you think their key metrics might – or should – be.
If you’ve been working on a particular product or vertical for a long period of time, there’s a risk you’ll eventually feel a little stale as your key metrics are established. Yes, there’s likely to be some evolution over time but if you’re working in ecommerce, for example, the variety of metrics you prioritise is unlikely to shift too dramatically.
In order to keep your metrics-oriented mindset fresh, examining other businesses is a useful exercise.
Example
OK, so let’s imagine you work on an ecommerce product and so you’re used to measuring your AOVs, CPAs and so on. On your walk to work you grab yourself a coffee. Think to yourself what metrics this coffee business might need to measure. Customers per hour, average order value, repeat customer %, top product sold by day. The list of potential metrics is long.
You don’t need to make this super complicated.The important outcome is that you’ll train your brain to think in terms of metrics. The more diverse the business types you examine, the more adept you’ll become at listing metrics that might be relevant for the challenges or situations you face when managing your product.
The unexpected benefits of developing a metrics-oriented mindset
You’re probably aware that metrics matter; most people are in modern product teams, thanks to OKRs. What’s a little less known though, is that in becoming more metrics-oriented in your thoughts and actions, you’re likely to experience some rather unexpected benefits which may help you day to day.
Here’s a few ways unexpected ways developing a stronger metrics-oriented mindset might help you.
1. Improved stakeholder relationships
The Marketing Manager wants a new CRM with custom API integrations to 45 different external tools to create a ‘single unified view’ of the customer.
The Sales Manager wants to know why their team are unable to generate a report using a specific filter type that’s been promised for 6 months.
Your Head of Department is causing you what appears to be a mild headache but on closer inspection might possibly be the onset of stress-induced angina because ‘all users’ hate ‘everything’ about the new app which launched last week.
If you’ve ever found yourself in a situation where stakeholders are causing you anguish or if you’re required to negotiate with internal stakeholders, developing a metrics-oriented mindset can help.
How?
Metrics provide an objective framework to settle and frame discussions. If you’re negotiating with multiple stakeholders about problems and requests which need prioritising, having the ability to shift away from the personal aspects of a request or problem and instead focus the attention onto some objective criteria will help you to reach an amicable decision.
Positions vs. principles
In a publication written by Harvard University, two distinctly different forms of negotiation were examined: positional-based negotiation and principle-based negotiation.
In positional negotiations, you take one position and the other person takes the other position. In a negotiation context, these positions are typically opposite to each other; hard negotiators will often dig deeper into their positions as the negotiation plays out whilst softer negotiators will be more open to giving ground to the other side. The outcome of positional negotiation is some form of compromise; you give a little and so does the other side and you’ll eventually meet in the middle.
By contrast, principle-based negotiation challenges you to negotiate by following a set of 4 guidelines:
The 4 facets of principle-based negotiation
- People – separate the people from the problem
- Interests – focus on interests, not positions
- Options – invest multiple options looking for mutual gains before deciding what to do
- Criteria – insist that the result be based on some objective standard
We won’t delve too much into each of these facets since we’re not here to talk negotiation per se but you’ll notice that one characteristic of principle-based negotiation is the insistence that your negotiation be based upon some objective criteria.
This means de-personalising the narrative and the context of your discussion and instead looking outside your relationship to measure your negotiation or discussion.
Examples of objective criteria
- Fair market value
- Professional standards
- Scientific judgement
- Raw Data
For example, if you’re negotiating a commercial agreement with an external stakeholder about using their API to process a specific function, you might consider fair market value as a benchmark to influence how much you should pay. Determining fair market value might mean examining competitor products that perform a similar function and negotiating from there.
2. Improved personal performance
“Discipline equals freedom” – ex Navy Seal Jocko Wilinck
A focus on metrics instills discipline and accountability. A clear framework for measuring your performance will help you to become more disciplined, both in your professional and personal life.
One major aspect of metrics-oriented thinking that people sometimes struggle with is both the discipline and patience it requires from those who use metrics to drive results.
How exactly is discipline linked to metrics?
Let’s take OKRs. OKRs (objectives and key results) are a widely used framework by product teams around the world but they are only useful when combined with discipline and patience. All too often, teams set OKR for a quarter or a biannual time period and the teams working on them feel motivated and fired up. Fast forward 7 weeks later and the motivation has gone for some teams and the OKRs are nowhere to be seen.
The worst way to use OKRs is to set them and gradually forget about them because something new and shiny has come along. This defeats the entire purpose of OKRs and you’re better off not bothering in the first place. The ill-discipline demonstrated when you never refer back to OKRs sets a bad precedent for the team and shows you lack the discipline to commit to measuring your performance.
But it’s not just discipline – patience is the second element of improved personal performance. To see results, you need patience. Anything particularly meaningful will often take at least 6 months to achieve – and you’ll need the patience to see how your work pays off.
Returning to our Netflix example, the teams waited a whopping 2 years to conclude whether some strategic hypotheses such as open APIs and social viewing paid off. Now that’s patience.
3. Improved problem diagnosis and solving
Problem diagnosis and problem solving is made more effective with a metrics-oriented mindset.
A problem may start with some feedback from a customer or a stakeholder which takes the form of a vague comment or generalisation. Your job is to take this vague notion and turn it into a measurable problem which can be improved with measurable outcomes.
‘B2B customers don’t like the new reporting tool’. OK, how exactly many complaints have we had? What’s the engagement level now vs. before the release? Has the referral rate dropped?
Problem diagnosis involves investigative work and results in a set of metrics, combined with qualitative information so that you can build a deep picture of the problem you’re trying to solve – and to discover if the problem even exists in the first place.
Problem solving means generating ideas to tackle the problem and putting together the appropriate metrics to measure whether your solutions are working.
Developing a healthy gut instinct
A metrics-oriented mindset is a powerful tool to have but you should always be wary of focusing too much on metrics. An unhealthy focus on metrics risks turning you into a robot. You are not Alexa.
A healthier approach is to develop a healthy, balanced gut instinct. Just like the microbiome in your actual gut, your gut instinct needs inputs to feed upon. Metrics is one input, qualitative information and prior experiences are others.
Seek out the opinions of those who have developed a healthy gut instinct and you’ll be one step closer to finding the answers you need. Or, to put it more succinctly than I ever could:
Remember that believable opinions are most likely to come from people 1) who have successfully accomplished the thing in question at least three times, and 2) who have great explanations of the cause-effect relationships that lead them to their conclusions. Treat those who have neither as not believable, those who have one as somewhat believable, and those who have both as the most believable.
– Principles, billionaire investor Ray Dalio
Product launches, news and analysis delivered to your inbox. Weekly.
