How to Run a Growth Experiment - Learn the 4 Steps

Product, marketing and sales teams can learn to run growth experiments to validate new strategies in the real world before incurring significant resource investment.

Confirmation bias is a psychological tendency where we seek information that validates our existing beliefs as well as avoiding contradictory information. This bias often causes teams to waste resources on initiatives that validate existing views but are ineffective in the real world.

This article explains the 4 key steps of a growth experiment and a practical example of how you can run your own growth experiment quickly and cost effectively:

1. Hypothesis
2. Trigger
3. Action
4. Measure

Imagine we are in the product marketing team of a B2B software startup that provides a tool for capturing customer complaints called Capturo.

1. Hypothesis

Set a hypothesis early. Start by identifying a customer struggle and business outcome of what you are trying to achieve. Form your own explanation as to what is causing the customer struggle and come up with a solution as to what could ease that struggle. User interviews are an effective tool at this stage if you have the time.

It is important at this stage to avoid the pitfall of falling in love with your explanation and solution. Focusing on the struggle will allow you to run a new growth experiment if the current one fails.

Example Hypothesis

Struggle: Our new users are observed to create a lower number of customer complaint forms on the platform than our existing users which is leading to lower product engagement and complaints captured.

Explanation: We suspect two core reasons: 1) New users are unfamiliar with the types of questions to ask to gather complaints; 2) New users lack the time or know-how to digitise their existing questions into our platform.

Solution: Our proposed solution will address the first explanation of new users not knowing what questions to ask. We will research and propose the most frequently asked questions to address customer complaints by service type (e.g. contact centre, restaurant, consultation).

Hypothesis:

“Teaching new users the most commonly asked customer complaint questions relevant to their business will help them create more forms on the platform and gather more customer complaints”

2. Trigger

Establish a trigger point to communicate with your customer around their struggle and your proposed solution. The most effective mediums are email and calling customers directly. In-app banners, notifications and user flows are expensive if you do not already have the infrastructure in place. As this is a growth experiment we do not need to worry about solutions that don’t scale yet.

The three most important components of a trigger:

1. Audience: identify your target audience. Start with bigger groups, segmenting at this stage will be time consuming and take longer to gather results
2. Message: keep consistent with your original hypothesis. There is no need to A/B test on a small scale but avoid message variation overkill
3. Timing: identify the best time to contact your users to get their attention. Be sparing for when you trigger again due to lack of response

Example Trigger

Audience: our growth experiment will be to email new users who have created two forms or less in the platform from their first three months of joining

Message: “Real examples of customer survey questions [Dental Practices] are using in Capturo” or “Top 6 customer survey questions [Dental Practices] ask in Capturo”

Timing: Send when admin users are most active on the platform (for example mid-week mornings)

3. Action

Establish an action that you can measure based on the trigger. This can include opening a link to an article, registering interest via a form, logging in to the platform to perform an activity, emailing or calling back to respond.

Without creating an action it will be difficult to measure the success of your growth experiment.

Example Action

Create a webpage link in the email which takes a user to the top customer survey questions. E.g email link. “View top 6 customer survey question examples”

4. Measure

Measure and track the performance of the trigger and action stages. Start simple and use a spreadsheet to track key results. Free or cheap email tracking plug-ins are an effective way of tracking email open and click-through rates.

Once you feel you have gathered enough insight (can take a few weeks) assess the results and determine if your hypothesis was correct. Within the same experiment you can go back and modify the trigger and action to gather more data if needed.

Example Measure

Create a spreadsheet with columns to track the following: User contact email and id

• Email message sent
• Date and time sent
• Date and time opened email (use email tracking tools)
• Date and time opened link (use email tracking tools)
• Already sent (Y/N)
• Created a new template (Y/N) (check manually each week)

After measuring the performance assess engagement metrics for example:

• Total users contacted
• Average contact count per user
• % email open rate
• % click through rate
• % users creating new template

Happy case If the % click through rate and % users creating a new template is good we have quickly validated that knowing the right questions to ask can help boost form creation. We now have an improved business case for investing more resources in-app to build onboarding tours in the form creation stage.

Sad case If the % open rate is poor after adjusting for message and timing then the solution of our hypothesis is likely incorrect. We can go back to our original hypothesis and form a solution for our second explanation (e.g. already know what questions to ask but don’t have the time to create a new form).