Thanks for signing up! Here's your first lesson. You'll also be receiving it via email shortly.

How to set up success metrics

Lesson #5

From our previous lessons, we talked about what is means to be results driven and how not to become overwhelmed by data. We also talked about how to develop reliable testing hypotheses so you can be sure the changes you make are helping you grow.

In the previous two posts, we talked about ensuring you have the right metrics and only track the metrics that are truly important. Instead of trying to track every little detail you should only track the most critical things to tell you that you are heading in the right direction and that you have enough fuel to get there (yep, we're going to keep on trucking with the car analogy).

Pre-Reading

In this post, we are going to dive deeper into success metrics for mobile applications. We will also help you to set up your metrics, so you can reliably follow through on our earlier advice to only track what's most important. From here on, everything you do should be solely focused on growing your success metrics. There is a bit of pre-reading you can do in this area. There are a ton of great articles out there that can help you to think about how “you make what you measure”.

These articles are great for a bit of inspiration.

If you are new to the concepts of analytics and tracking, it will be important to cover off some of the basics on how to set up tracking in your app. The best resources for this are probably over at Google Analytics' mobile resources. Here are a few good posts you can check out to start.

General Metrics

For mobile specifically, the most important success metrics can be categorized into three overarching areas:

  • Acquisition
  • Engagement
  • Retention

When you're going through this lesson, take some time to think about your app. As you're coming up with your most important metrics, take a stab at categorizing each of them as acquisition, engagement or retention. Categorizing things in this way will help you to refine the metrics you choose to track, based on how mature your app is.

Important Note

One thing you should remember when going through the rest of this lesson. It is ok for your metrics to change over time. You should only track what's important for you right now. You'll inherently have a desire to set yourself up for the future by tracking what you think are enterprise metrics, or VC suggested metrics for Series A companies. Don't be fooled. Analysis paralysis is real, so forcing yourself to think critically here is extremely important.

Acquisition

Your acquisition metrics will show you if you are successfully growing your user base. The three big metrics you want to track here are; what is your total install #, how many of those installs become active users, and how much are you spending to gain a new active user. You can't just track your total downloads because that won't give you the full picture. You also need to ensure that you are not overspending to attain your new users. If you are spending too much you will never see a return, and you will never have enough fuel to reach your destination.

The three acquisition-focused metrics we suggest you start off tracking are:

  • Total Installs: total number of users who installed your application.
  • Lifetime Value(LTV): How much revenue on average will an active user generate over their entire lifetime as a user of your app.
  • Customer Acquisition Cost(CAC): Cost of all marketing efforts divided by the total number of active users.

Your total installs, particularly install growth week-over-week, will show you whether you are improving your installs at an acceptable rate (pointing in the right direction). These can be measured easily by looking at your dashboards in iTunes Connect or Google Play. You can also make your life easier by pulling these together in one interface with a tool like AppFigures.

Your Lifetime Value and Customer Acquisition Cost are used to ensure you are not spending too much money to acquire a new user (your fuel gauge). If your Customer Acquisition Cost is higher than the Lifetime Value, you won't even break even. Therefore, you will never have enough fuel to reach your destination. To measure your Customer Acquisition cost, you can use tools like Kochava or Appsalar which can tell you exactly how much you're spending on mobile install ads.

Lifetime Value is harder to track in practical terms. It requires a deep understanding of what an active user is, some estimation of what their lifespan will be and how much value they deliver on average through that lifespan. This is easier with a shopping app because all you need to know is the average purchase value and the average number of purchases per user. In an ad-supported model, you can run calculations on how many ads on average are delivered to a user over the course of their life, but this tends to be less accurate. Whether exactly accurate or not, it is still important to go through the exercise of understanding LTV so you can properly assess your growth activities and spending.

Engagement

Once you acquire a new user, you want to make sure they have that magic moment that will get them hooked. This metric is going to be completely dependent on what your application is trying to achieve. For a commerce application, this will most likely be centered around shopping cart adds, product favoriting, sales and first time purchases (activations). For a social app, you may want to focus on the total amount of content being shared and the number of users sharing for the first time. For a dating app, you will want to focus on likes, messaging and other user-to-user interactions.

What this comes down to is asking yourself what is the one thing that you want a user to do in your application. From there, set your metrics to track how many new users achieve this for the first time and what is the percentage of all users achieving this action. By tracking new users achieving this action, you will be able to get a better sense of your fuel level, and the overall total will help to keep you pointed in the right direction. As with everything else, track this week-over-week to give you a clear understanding of how the changes you make are affecting your growth and performance in the short-term.

Practically speaking, your engagement metric should, in most cases, relate directly back to your LTV. The best way to know if it's a metric you should be tracking is if an increase in that metric directly increases your LTV.

Retention

One of the most important areas of focus for success metrics, that is often overlooked, is retention. These days most people working on mobile growth are focusing on virality and the viral coefficient (Read about the Viral Engine of Growth in Eric Ries' “The Lean Startup”) of their app. But they often overlook retention. The big mistake here is that if you aren't able to keep users, it doesn't matter how viral your app is; you're not going to grow.

Think of it like this, if you have a leaky gas tank, it doesn't matter how much fuel you put in your car, you're going to run out of gas earlier than expected.

Jamie Quint has a great post on the topic of how retention directly affects virality .

For most applications, a great place to start with retention is looking at when your users first showed up and when they returned for their next session. You can see this in a typical cohort graph focused on user retention. This graph will tell you how often each of your user cohorts is coming back to your app. Retention graphs give you a quick visual to understand when and how many of your users are returning.

User Retention and Cohort Graph

Most analytics tools give you the ability to set up this type of graph, and some even create it for you automatically.

Determining an acceptable number of days before a user returns is very dependent on your application. Here is a great article if you want to look at what good retention levels are for different apps by category . You should also look at your business model and decide if your application fits within the norm for your industry.

You need to determine at which intervals your active users should be returning and then look at your data to ensure your new users are returning within an acceptable range of that interval.

The bottom line is when it comes to retention metrics you want to be able to see when a user first returns and if they continue to come back.

Key Takeaways

Now that you have all three areas covered take some time to think about your application and start collecting only the data that matters. Think about your business and app in terms of Acquisition, Engagement and Retention.

As you grow your business, these metrics will change and grow with you. There can never be a once size fits all solution in this area. The main thing is to figure out what the handful of key metrics are that are most critical and then start driving your business and make changes to grow those metrics.

Always make sure you know how the actions you take change your metrics and make your decisions based on what your results are telling you. Everything you do should sound something like this:

"We are changing X because our week-over-week growth on acquisition has slowed to 1%."

or

"When we changed Y, customer activations increased by 3%, so I think we should spend extra time on Y."

If everyone has a common goal of growing your metrics, you should be well on your way of reaching your destination.

Extra-Curricular Reading

If you want to read more on this subject here is a compiled list of all the readings we suggested in the lesson: