How to Conduct a Product Health Check

I was chatting to a friend the other day who had just taken on a senior product director level role at a venture backed startup. The company wasn’t in top shape and one reason they’d been brought on board was to turn the ship around. They remarked that one of their biggest challenges was figuring out where to start.

And it’s a common problem for product people.

Given the vast, expansive nature of ‘product’, it can mean anything from understanding the business model, the new domain you’re operating in, user engagement data or the overall strategic direction of the business.

Sure, this is similar to other C-level tech leadership roles, but COOs, CTOs and Chief Revenue Officers are, I’d argue, a little less nebulous in their scope than ‘Product’.

With that, we had a little discussion about what the best places for a product leader to focus on might be in the early days after joining a company – and how a leader in product might be able to conduct a ‘Product Health Check’.

A health check as a diagnostic tool

Product strategy isn’t simply about clear decision making; before you can even begin to make any decisions about the direction of your product, you need to spend some time in diagnostic mode where you try to build a big picture snapshot of the health of your product. You can even present your health check in a digestible format with stakeholders to help them understand the health of your product, too.

Only when you’ve created a clear diagnostic overview of your product can you then start to consider a plan of action.

Here’s some of the areas we felt could be the most impactful as part of your product health check, spanning data, devices, value proposition, business and strategy.

User data

Metrics matter in product development, but as with most things, it’s very easy to get overwhelmed by the sheer number of metrics you could be measuring at any given time.

Sure, you can dig deeper later on, but these are fundamental metrics that are important to understand when you’re assessing the overall health of your product.

Overall user numbers

• How many visitors do you have?
• What % of those are active users of your product?
• What % of those are paying users of your product?

These are all important questions to start to build a picture of your product and its health.

Geolocation data

If you’re a company based in New York but 75% of your traffic is coming from Asia, this tells you something about your product.

Maybe your proposition is resonating with that audience, maybe your SEO content is highly engaging users from that region or maybe someone ran an ad campaign in Asia and forgot to turn it off!

Often, folks who have been at a company for a long time, forget to routinely analyse where visitors are located. And it’s important to understand the geographical make up of your audience because it directly links back to your product’s strategy.

Geolocation data and monetization strategies

If the majority of your customers are in developing economies, this has implications for your monetization strategy – especially if your strategy is based on X% of your visitors converting into paying customers.

If the reality is that the true number of visitors with purchasing power is far less than you originally imagined, you may need to rethink your assumptions.

Similarly, once you know the geographical makeup of your audience, you could potentially decide to monetize based upon the audience profile, too.

Offering purchasing power parity to audiences in countries with lower purchasing power could be one option, for example. But this is typically only doable for companies with the economic resources to do so.

Devices

• Are most of your visitors or users on desktop, tablet or mobile?
• How does this compare with the industry averages?
• Are we optimised for these devices or should we be?

Getting an understanding of the hardware existing customers are using to access your product is important since it understandably can have implications for your UX.

A SaaS product targeting veterinary practices is more likely to attract more desktop users than mobile users.

A consumer social media app is more likely to have more mobile users.

Figuring out what devices your consumers are using today and how this impacts your product strategy is critical.

But, a snapshot from today will still only tell you what users are doing today – and there may be various reasons for that. If 85% of your traffic is on desktop but 65% of your landing page traffic is on mobile devices, this could indicate that customers are discovering you through mobile devices and would prefer to use a mobile device to access your product but are unable to do so because the product isn’t optimised for mobile.

A health check would focus on understanding which devices users are using to access your product today, whether or not this is in line with company expectations and what opportunities arise as a result of this.

Value proposition

Product management is, at its heart, about creating and delivering value to users. And no product health check would be worth the effort without bothering to address questions about your value proposition. Here’s some questions to consider.

Do users understand our value proposition?

This sounds pretty straight forward, doesn’t it? But, sometimes one of the challenges that a failing product company faces is that users no longer understand why your product is valuable to them.

Dealing with ‘decision debt’

As companies get older, they accrue the debt of all of the previous decisions that were made.

‘Decision debt’ could mean specific strategic bets were placed and failed but the remnants of these decisions are still felt today.

It could mean 3 different CPOs had drastically different views of what the value proposition should be and as a result, the team ended up building half-baked versions of each of these 3 standalone visions.

It could mean a B2B enterprise strategy was pursued, didn’t generate any traction but is still reflected in the product’s language and marketing.

Decision debt accrues over time and it can lead you to a point where your value proposition is simply not clear or understandable any more. Instead, it’s a reflection of prior decisions made by people who have long since departed. It’s a bloated mess and your users are confused.

One of the most powerful things a new product exec can do as part of a health check is to get a sense of how easy the value proposition is to understand for both staff internally and customers.

If your team members can’t even articulate the value proposition, how can you expect customers to understand it?

Do they want what we’re building?

The second consideration is: do people actually want what it is we’re building?

Comprehension and understanding is one thing, but if nobody actually wants it, then what’s the point?

Figuring out the desirability of your product can be measured using product market fit metrics. These include: Very Disappointed score, LTV, NPS, CAC.

UX health

We touched upon critical journeys earlier when we covered data and those journeys could also be considered UX health.

The importance of UX and beginner’s mind

More specifically, when you’re conducting a product health check it’s helpful to spend some time actually using your own product. Yes, you’ll probably be bombarded with a bunch of stuff when joining a new company, but first impressions only ever happen once.

Beginner’s mind is a concept in Buddhism which describes the innocence of our mind before we fill it with all the baggage and preconceived ideas we accumulated over time. As children, we’re naturally curious creatures with an appetite for figuring out how and why things work the way they do. We approach the world with a ‘beginner’s mind’.

As we get older, we fall on our preconceived notions of how things operate based on our previous experiences or we take things for granted and fail to approach things with a clean, fresh slate.

In a product context, once you’ve gone through user journeys multiple times and you understand the technical challenges of why certain things work in the way they do, you forget what your first impressions of your product were.

You forget how frustrated you were at the quirks of the password suggestion tool, you ‘learn’ ways to overcome the slowness of specific parts of your product or you use shortcuts that other members of the team recommend you try as workarounds.

Once you’ve ‘learned’ these things, it’s almost impossible to approach them again with a beginner’s mind – and that’s one of the reasons why user testing is so important since each user brings no baggage to the product and approaches the critical user journeys with a fresh pair of eyes.

When you’re conducting a UX health check, here’s some aspects of your UX to consider.

What are our top tasks?

Top Tasks represent the most important or popular tasks users need to get done in order to realise value from using your product.

For a SaaS product helping ecommerce sellers print shipping labels and send parcels, top tasks could include generating a label and tracking parcels, for example.

When conducting a UX health check, figuring out what your top tasks are for users is essential. But understanding what your top tasks are is only part of the picture.

How easily can users complete these tasks?

Ease of completion doesn’t mean number of clicks. I’ve seen some UX reports focus heavily on the number of clicks required to complete a specific task, but this misses the point.

A user could be quite happy clicking multiple times and still get their job done efficiently. Rage clicking is an exception and if clicks are way above what you’d expect this could also be an indicator of a poor UX, but fixating too much on clicks can lead you to make unhelpful conclusions.

Task completion rates are a better indicator of UX health. Asking what percentage of users are able to complete their tasks without unnecessary friction can guide your decision making?

Conversion rates for critical journeys

Critical journeys are journeys that directly impact the overall health of your product. Sure, every user journey is important to some extent, but these are the journeys that, if either broken – or if optimised to their full potential – could fundamentally transform your business.

Examples of critical journeys could include:

• Sign up journeys
• Login journeys
• Payment / upgrade journeys
• Value discovery journeys – how quickly can a user understand what it is your product does and its value proposition?

How to measure UX friction

Figuring out these points of friction – especially in critical journeys – can have a significant impact on revenues. Which segways nicely onto our next area of focus when conducting a product health check.

Business / commercial health

It’s easy to get caught up in the fun of building products that you neglect or forget that whilst crafting a special UX is important, ultimately products are businesses.

Product people should equip themselves with essential business skills to succeed and it goes without saying that your health check should include a solid focus on the overall performance of your business. Here’s a few things to consider:

Is our monetization strategy clear?

Hitting revenue targets is only possible if your monetization strategy is clear. Products can adopt various different monetization strategies and the mechanism you choose to adopt will often be a collaborative effort between product and commercial / sales teams.

Monetization models can include a mix of any of these:

• Commercialising access to your technology and APIs (FourSquare, Plaid, AWS)
• Subscription models (SaaS)
• Advertising / commercial partnerships
• Bundling and packaging
• Monetizable services
• % transaction revenue

When you’re conducting your health check, ask yourself whether the pricing model is clear. It may be clear to you internally, but try user testing it with actual humans to see what their feedback is, too.

Usability testing is often reserved for testing new features, but proposition and pricing testing with users is extremely valuable, too.

Are we maximising revenues?

Chat to your sales teams to figure out whether they are closing deals and capturing enough of the market as they see it.

Speaking to your sales team will help you to understand how your product is performing and whether or not there are untapped elements. A strong relationship between product and sales is in itself a health indicator.

Are revenues growing or shrinking?

If you’ve started a new product role and you want to get a sense of how healthy the business is, chat to the financial and commercial members of the team to figure out what financial shape the business is in.

Revenue growth is clearly an indication of good product health. Declining revenues are often a symptom of poor health. Profitability, too, can be a symptom of strong product health but just because a product isn’t profitable doesn’t necessarily mean it’s not healthy; many businesses will sacrifice profitability for growth, especially in the early days.

What are our business opportunities?

Does the business have a sense of emerging trends and opportunities that could be capitalised to grow the company?

Often, strong, healthy businesses will be acutely aware of the changes in the wider macro economic climate – and how those changes might directly or indirectly impact the business. If your business seems savvy on this front, count this as a point for health but on the flip side, if the company seems blissfully unaware of emerging market trends or threats, this is an indicator of naive strategic thinking. Which leads us nicely onto our final product health check marker: strategic health.

Strategic health

Product strategy is about decision making and trade offs. You can almost ‘sense’ whether or not a company is strategic in its outlook after just a few weeks into a new role.

The vision is clear, decisions are made which align with the vision and the company isn’t chasing every single opportunity. A company that is clearly able to say yes to one opportunity and no to another is a sign of a company in good strategic health.

A company which seemingly pursues every potential course of action with little regard to an overall vision or clear decision making process is likely to be in poor strategic health. Other strategic health considerations include differentiation and sustainable unfair advantages.

What are our differentiators?

The 5 common types of differentiation include:

1. Features
2. Policies
3. UX
4. Pricing
5. Performance

After joining a new company, ask team members and execs what the company’s core differentiators are. If they’re all able to answer easily, this is a strong sign of strategic health.

If they struggle, this could be an area where product can lean in heavily to influence the company / product strategy – and the company’s overall health.

If the company does have a clear sense of its differentiators, this could be an area worth probing further to understand how sustainable or defensible those differentiators are.

Bringing it all together

The whole purpose of a product health check is to take a breath and build a picture of the overall health of your product.

Whilst we’ve focused on the idea of a health check as a tool when you’re starting a new role, it’s actually important to conduct a health check on a regular basis too. These little nuggets of information help you to collect relevant data points whether you’re starting a new role or whether you’ve been working on the same product for 10 years.

Just as there’s no single metric to measure the health of our bodies, in product, each of these indicators help you to build a wider picture of the overall health of your product.

And it’s only once you’ve built a picture that’s clear enough can you then decide what it is you’d like to do about it.