Not everyone who tries your product will purchase and stay engaged. If you offer a free version, you’ll often see a significant drop in engagement shortly after sign up. The question then becomes, “How do I improve the onboarding experience to increase engagement and revenue?”
I’ll break this down into 5 steps, and show how product onboarding contributed to 500% revenue growth at my company Loggly. Loggly offers cloud-based application and system monitoring, and thousands of software developers and system operations people sign up for a free trial each month.
1. Understand Your Users
Even if a good number of your users are successful after signing up, what about the ones who are bouncing soon after or are struggling to make progress? Do you really understand what problem are they trying to solve at that point in time? What is motivating them to take action and what is their ideal outcome? Clayton Christensen described this in more depth as the job to be done. Remember that a user’s first job when trying a product is only part of their full set of jobs. It’s likely they first want to satisfy some immediate need, or learn more about your product so they can plan for the future.
At Loggly we created a survey on SurveyMonkey that asks about each user’s first use case. We sent invitations in our Marketo email program several days after the customer signed up. I listed out the top 10 use cases that I heard on sales calls, and asked them to pick up to 3. Over 150 users responded. Below you can see that the top use case was finding the root cause of errors, which was selected by 60% of our users. We also created separate surveys based on who successfully activated the product, and those who did not. Next, I’ll show you what to do with this information.
2. Analyze the Customer Journey
Once you understand your user’s current situation and their ideal outcome, you can create a map of all the steps they go through from beginning to end. This is also known a customer journey map. You can start building your map by adding what you already know. You can start as early as lead gen by including how they recognize the problem and how they find your company. For onboarding, include all the steps they go through in your app, and even common friction points.
Kick it up a notch by using behavioral data to enhance your journey map. A good map should be highly predictive of who purchases your product. Analyzing our data using regression and decision tree analysis, we identified 3 key behaviors that our paid users were highly likely to demonstrate, and our free users were unlikely to. Putting all the stages together in a funnel allowed us to build a model that predicted about 70% of our paid users. You can read more about the analysis we did in my blog Predicting Customer Conversions in R.
The “success stages” we identified were:
- Signed Up – They signed up for a trial account
- Activated – They sent some log data to Loggly
- Operationally Activated – They sent a significant amount of data for several days
- Engaged – They logged in to our web portal several days
- Invested – They added a team member, and personalized their settings
The invested stage is the most predictive of who is going to purchase a paid plan. Generally, user investment is “any activities in which users spend time or effort interacting with a product in a way that ultimately makes that product more valuable to them”. In the chart below, you can see that a user meeting our definition of invested is 10 times more likely to purchase than one who is only engaged. I removed the scale on the Y axis for privacy. The high prediction rate validates that our journey map is correct. Also, it gives us guidance to encourage as many users to become invested as we can.
3. Measure User Success
Next turn your customer journey map into a conversion funnel, and measure the user success rates at each step. This helps you identify where the biggest drop-offs are, indicating either lack of motivation or too much friction.
Common tools to measure user behavior include custom events in Google Analytics or KissMetrics, but they are limited to the included reports. We track behavioral data using Loggly, which is a low cost way to stores lots of data points. It also allows us to easily export data to an S3 bucket, where we can do custom analysis in map reduce or a relational database. We hook these data stores up to Tableau for visualization and reporting, and R for predictive analytics.
Below, I’ve plotted an example conversion funnel showing the percentage of trials in each stage. These numbers have been altered for privacy, but this shows a typical funnel. We can see that about 75% of accounts that sign up become activated. Also, only a small percentage of trials have reached the invested stage, and our goal is to increase this percentage.
It may require some creativity to track things done by users outside of your software. For example, at Loggly we previously asked users to manually configure their operating system or server environment to send logs to us. We knew they were having trouble, but we didn’t know why. As a result, we created tools that automatically verify the user’s setup. In addition, they report the success rate so we can track it.
If you’re trying to figure out what contributes to a given success rate, break it down into components. For example, there are many ways to configure a system for logging, including Linux, Windows, Java and more. We tracked success rates on each of them separately, so we could see which was the most challenging for our users.
4. Test New Product Improvements
Next look at your conversion funnel and pick one metric you want to drive first. You can get ideas on how to increase user success by reducing friction or adding more value. Look at steps in your journey map to simplify, or enhance existing steps with a bigger “wow” factor. Take a look at the design of the user experience, have you planned out a great first run experience? Samuel Hulick’s book The Elements of User Onboarding has tons of great design ideas.
Rank by the impact an improvement would offer, and how quickly you can offer a solution. The low hanging fruit is often in the metrics earlier in the funnel or newly discovered stages. You can do a before/after comparison if the improvement is obvious. A/B testing might be better if you are unsure which performs better.
We decided to focus on our activation rate first since too many people had trouble setting up their computer to use our service. We defined our activation rate as the percentage of signed up users who successfully sent logs to Loggly. Most of the improvement came from reducing friction in the setup process. Below, you can see an example of our setup instructions for Linux, which previously had 3 complicated steps to install and verify it was working. We made this into one easy step with a script that does all the steps automatically. When we saw it working in Linux, we created more scripts to automatically configure Mac OS X, Apache Web Server, and more.
We also shared our insights with marketing and sales so all our company’s touch points are improved. For example, we made our customer success stages available in Salesforce, so our sales team can offer tips when doing outreach. We also integrated them into Marketo to personalize our introductory email program. We got some great results with our personalized email, but I’ll save that for another post.
All together, we were able to lift our activation rate by 35% and now the large majority of our customers are successful sending data. Plotting by month or quarter cuts some variance and allows us to see long-term trends. The activation rate grew quite fast at first and then went into a steady climb as we ironed out the biggest friction points.
5. Continuous Improvement Leads to Big Growth
Everyone loves quick wins and breakthroughs in high level metrics like revenue. Most often we see the biggest improvements in low level metrics. For example, a bug fix we made in our Java instructions instantly lifted our success rate for Java. All these small improvements add up over time to make a big impact on higher level metrics like activation rate and revenue. You can read more about continuous improvement in the Kaizen management philosophy that lead to Toyota’s success.
Improving our activation rate each month leads to exponential growth over time. You can see proof of this in our aggregate activations. We launched our product with strong growth, but since then 35% of our activated accounts are due to optimizations that have accelerated our growth. In total, we saw a 500% increase in revenue last year.
What about improving the rest of our user’s job to be done? For people who want to learn more about log management, we’re launching some great new guides in a few weeks. For people who are engaged and debugging problems, we have a lot of work to do in offering better insight into errors and their root causes. Delivering this insight when and where people want it is part of our product roadmap for this year.
I love sharing and learning things from the community. Does your company look at customer success in the same way? What are some things that you tried that you tried to improve user onboarding? Please add comments below so we can all learn from you.