The 3 major sources of great experiment ideas
About the author
Jeff Chang (@JeffChang30) is a growth technical leader at Pinterest and angel investor. If your startup is looking for an angel investor who can help with all things growth, please send over an email!
Intro
This is a part two to my previous post: The 4 most important factors when evaluating growth opportunities. Now that we know how to evaluate if an idea is good or not, how can we come up with the ideas in the first place? In this post I will talk about the three main sources of finding growth opportunities that I use.
Industry Research
Every experienced growth hacker has done this before, where they look at other products to gain inspiration. Later stage companies run a lot of AB tests so the current iteration of their product is likely optimal compared to their old versions, By observing the different iterations over time, you can learn what works for them and what doesn’t. For example, app interstitials and signup upsells are very common growth tactics for growth products. In these two areas, companies have used similar tactics to achieve metrics gains.
Old Pinterest mobile web app interstitial
Old 9gag mobile web interstitial
Old goodreads mobile web interstitial
Old Pinterest signup upsell
Old Wanelo signup upsell
Old Linkedin signup upsell
As you can see, trying similar tactics as other companies is a common growth strategy. It’s good to be up to date with the latest growth strategies top growth companies are trying.
Here are some growth engineered products (hundreds of AB tests ran) you should check out:
Facebook / Instagram
Linkedin
Airbnb
Pinterest
Twitter
Metrics deep dive
Expertise in company metrics is key to being successful in growth. Diving into your metrics can be a great place to find idea inspiration mainly because it tells you where the opportunity is. The first thing you should dive into is your user segments. For example, if you are working on SEO, you should know the distribution of your page traffic by page type. If you want large immediate gains, you likely should work on the highest traffic pages. Next, you should know general conversion rates such as signup conversion, login conversion, and activation rate. Note that these rates are usually different for different segments, and the aggregate rate is not nearly as useful as segmented rates.
For example, let’s say your desktop activation rate is 10%, mobile activation rate is 30%, but mostly get desktop signups so aggregated activation rate is 15%. The 15% number is not that useful to know because no major user segment activates at that rate, they activate at either near 10% or 30%.
With the combination of knowing user segments and conversion rates, you can get a bigger picture of what to work on. For example, for a few late-stage products, the home page is low traffic compared to product pages but already high converting, making it not a great opportunity. Prioritize the biggest segments first.
Another way that a metrics deep dive can help you find opportunities is spotting anomalous metrics. For example, let’s say one of your pages converts significantly worse than the rest. Perhaps there is a bug on the page or some optimization applied to the other pages wasn’t applied to that page.
Metrics deep dive summary:
Know your user segment distributions across apps, referrers, etc
Know the different conversion rates across major segments
Look for anomalous metrics
Experience deep dive
An experience deep dive is simply going through the user flows in your product. Use this with your metrics deep dive in order to test the most common flows. Similar to the metrics deep dive, this is a good way to spot issues such as bugs or missing optimization features. Also, going through the flow, you can experience the kinds of friction that users experience and probably think of optimizations. Here are some flows you should definitely try in the acquisition area:
Search different search terms your page rank for. Are the title and description enticing?
Sign up with the different methods
Go through the new user experience
Everything above, but on mobile web and apps
Download the app and sign up / log in
Conclusion
Although these methods can be used independently, you will find them effective in conjunction too. For example, starting with a metrics deep dive, you may find that email open rates for a specific email type are low. You find that the volume of users receiving that email is high, making it a big opportunity. Next, you do an experience deep dive by sending yourself a test email and notice that there is a typo in the email title, making it a bit confusing. Finally, you look at what other products are titling a similar email type to find inspiration for a high converting email title.
Using these three methods, hopefully, you will be able to come up with some great experiment ideas. Quality experiment ideas are hard to come by and can take time to come up with. In fact, I encourage my team members to spend significant time to think and evaluate ideas, instead of waiting for the next brainstorm to happen. Additionally, growing your knowledge base of what the industry is doing, what your key metrics are, and what your current experience is are all necessities to make great decisions in growth.
Want some advice about finding your biggest growth opportunities? Email me at jeff@growthengblog.com