The best metric for determining quantitative product market fit
Intro
There are many definitions of product market fit:
Product market fit is your NPS score
Product market fit is when 40% of your users would be “very disappointed” if they no longer have access to your product
Product market fit is a feeling
You need a good distribution channel to have product market fit
… and the list goes on
Most of these definitions are pretty good - you can be successful using them and none of them are “wrong”. However, I think cohort retention rate is the most important product market fit metric, so I would recommend using it along with any other frameworks you use. This blog post describes what makes a good product market fit metric, evaluates a few definitions, and explains why cohort retention rate is a great metric to use.
Caveat: This only applies to multiple-use or subscription products, which is the vast majority of products. Examples of single-use products are mortgages and other types of loans, where a user is likely only going to purchase one over a long period of time.
What makes a good product market fit metric?
Prior to product market fit, you should be focused on making the product better for your existing users. When a product has “product market fit”, it means that the product is good enough to start shifting focus from improving the product to growing distribution channels. If you distribute a product that isn’t great, even if you are able to get a lot of initial users, a few months down the road your user base won’t grow significantly or may even decline. So, you can also think of product market fit as “time to start building scalable acquisition channels”.
After you reach product market fit, you can spend more effort building scalable acquisition channels
A good product market fit metric will have a low rate of false positives and false negatives. It won’t tell you that you should start distributing when your product isn’t good enough, and it won’t tell you that you shouldn’t start distributing when it is good enough.
The risk with false positives is that you work on distribution when you should be working on making the product better, which results in wasted distribution efforts due to churned users. The risk of false negatives is that you don’t work on distribution early enough to compete with competitors or meet goals needed to raise your next round before you run out of runway.
To summarize, a good product market fit metric:
Will tell you when the product is good enough to work acquisition channels
Minimizes false positives - won’t tell you to work on acquisition channels when the product needs more improvement
Minimizes false negatives - will tell you to on acquisition channels as soon as the product is good enough, to maximize growth
Let’s evaluate some product market fit definitions based these points.
Evaluating other product market fit metrics
Top tech companies in the world are able to grow with bad NPS
NPS is sometimes used as a product market fit metric, but some of the biggest tech companies in the world have terrible NPS, but are still able to grow to over a billion users:
NPS as a product market fit metric has both false positives and false negatives.
Other survey based metrics are better, but have a few flaws
“40% of your users would be ‘very disappointed’ if they could no longer use your product” is a commonly used metric that most recently was used by Superhuman to find product market fit, with great success. This is a much better metric than NPS and if you end up using this and reach it, you probably do have product market fit. However I think there are some flaws with survey based metrics:
Response bias - it’s really hard to get everyone to fill out a survey. As a result, you might find that that the distribution of survey results don’t actually match the true distribution of results if everyone answered due to the missing responses.
It’s a single snapshot in a long user journey - when is the best time to ask this question? If you ask it in the beginning, maybe their opinion will change over time. When a user signs up for a product, generally their intent is high. So, you should ask it later in the flow. However, if you ask it later, some users already dropped off so you are biasing results to higher intent users, which are the ones that stay.
Responses don’t match behavior - If you’ve worked on product or growth for a while, you should be familiar that what users aren’t always accurate at predicting their future behavior. For example, you could build a feature that a lot of users say they would use if available, but then after you launch, they might not actually use it.
While this metric is one of the best out there, I think that it can produce some false negatives, meaning you might have product market fit even if this number is not 40%.
Distribution shouldn’t be a requirement for product market fit
Some definitions of product market fit include a distribution aspect. I actually don’t think distribution is part of product market fit, because there is enough data on what distribution channels generally work, SEO and referrals for example. If you’re a consumer company, at least one of these channels should work for you. These two channels also have some sort of a moat - it takes a lot of effort and time to outrank current SEO “winners”, and referrals become a stronger channel the bigger userbase you have. Paid will also work to some degree, but shouldn’t be relied on as a scalable channel long term for most companies (unless your LTV is high and payback period is short). I’ve never met with a startup where after studying the product, my conclusion was “there are no distribution channels that will work for this startup”. In my opinion, any product with great cohort retention rate can be distributed with good growth strategy and execution.
Cohort retention rate
What is cohort retention rate?
There are many methods of measuring retention rate but I think cohort retention rate is the one to use. Any sort of retention rate metric that mixes differently tenured users doesn’t make sense because the retention rates vary drastically between a new user and long term user. Here’s my definition:
Cohort retention rate - given a group of users who joined around the same time, the % of those users that stay long term
This is an example of a cohort retention graph for a single cohort. The cohort retention rate is around 30% because that’s where the graph flattens out.
If the graph never flattens out, then the cohort retention rate is 0%
For more information on how to measure this, check out my other blog post: The most important growth metric for early startups
As you improve your product, newer cohorts will have high cohort retention rates. So, it’s important to have a cohort retention “triangle” chart to track progress:
Sample cohort retention “triangle” chart. This product has healthy retention for certain consumer verticals, bottoming out at around 30%. From the chart, you can see that product improvements were shipped around the week of 4/5 that improved retention of newer cohorts.
Once you have a few cohorts that level off at a vertical-specific number, then you’ve achieved product market fit!
Different types of products have different “points” of product market fit, so it’s important to find the retention rate of some comparable products that have been able to significantly grow to find the right benchmark for you.
A good rule of thumb is for consumer products, 25% is a good floor and and for B2B SaaS products, 70% is a good floor. Floor meaning if your cohort retention is below these numbers, you probably do not have product market fit. You’ll likely want a lot better though, with a lot of great consumer products having over 40% long term retention and B2B over 80%
Why cohort retention rate is a great product market fit metric
There are several advantages that cohort retention rate has over survey based product market fit metrics:
No response bias - you get information from all users, not just the ones with high enough intent to fill out a survey
Full user lifecycle data - if you run a survey to collect a metric at a specific point in the user lifecycle, that metric may be different later on, but unknown
Measuring actual user behavior - don’t run into the problem where survey responses don’t match behavior
All of this being said, it would be great to use both cohort retention and survey based metrics, but at the very least you should know your cohort retention.
You will feel product market fit, but maybe not at first
Emmett Shear (CEO Twitch) had a great tweetstorm recently on how product market fit is hard to define, but you’ll feel when you have it and when you don’t - “pushing a boulder: don’t have product/market fit. Chasing a boulder: have product/market fit.” This is a pretty good indicator of product market fit, since if you have great cohort retention, it will be a lot easier to grow. Good retention amplifies your acquisition efforts, since the users you acquire actually stay. Good retention is also usually correlated with how much users love your product, which amplifies your word of mouth. Users are more likely to share your product with friends if they are still using it after a long period of time.
The main thing to watch out for is if your product goes viral but doesn’t have good retention rates. You will have the “chasing a boulder” feeling since you might not be able to keep up with demand, but you don’t actually have product market fit, meaning you should still work on improving the product, not shift focus to acquisition.
If your retention rates are great, once you start some basic acquisition efforts, you will definitely “feel” product market fit though.
Key points
Actively measure your cohort retention rate using a “triangle” cohort retention chart
Find the retention rate of comparable successful products and set a product market fit goal
Improve your product and watch retention improve with new cohorts!
Thanks Felix Feng, Kevin Tu, and Vincent Tian for reading giving feedback on earlier versions of this post!
I only post every few weeks, so follow me on Twitter to receive daily growth advice!