Live Ops and Cohort based KPIs
There are 2 main perspectives when looking at mobile game KPIs: a cohort-based approach and a live ops one. Both are not ultimately opposed – but they do point to a different way of thinking about overall game performance. They also both have different advantages and are better suited for different moments of a game’s lifecycle.
For the cohort-based approach, the object of study is a cohort – a (supposedly) homogenous group of users defined by their install period, country, platform, device, UA source (paid/organic), etc. It will consider the performance of that group over a specific duration. More often than not that duration is considered in days since install (the initial input moment) – but that duration can also be defined as a feature reflecting game progression such as game level, team power, arena, etc. Either way, every aspect of the game is evaluated from the perspective of the total installs. The cohort-based approach will try to find regularities in patterns of behavior for users interacting with the game – the goal is to base a business/product decisions on those regularities.
The cohort-based approach usually thinks about performance in terms of initial investment: if you put x users or UA dollars into the system (i.e. the game), what is the output x days later? In theory, that output can be anything (matches played, consummables used, etc.) – but usually it’s revenue, users or payers. The cohort-based approach usually thinks about things in terms of $ in / $ out. More importantly, the game’s performance is always considered from the perspective of the initial input – i.e. the initial volume of installs. This is why this is the main perspective adopted when considering the viability of a title during soft launch or considering marketing strategies. If I were to simplify, I’d say that for a cohort-based way of assessing a title, what happens between install and day x is a bit of a black box. Of course, the goal still remains to analyze what goes on in the black box and make improvements to the game to optimize the output. But ultimately what matters is the output always considered relatively to the input – much more than how it’s produced.
The 2 main things about cohort based KPIs are 1) they are great to describe a title’s performance, but much less to prescribe a specific course of action, and 2) the longer the time-frame considered, the less actionable they become.
First, a cohort-based approach is great to assess the performance of a title and help define high-level strategic orientations. Namely, what are the game’s main strengths performance-wise, and what should we aim to improve: D1 retention, D7 conversion, D14 redeposit rate, etc? It will be much harder to use cohort-based metrics to prescribe a course of action – unless your problem lies in the cohort itself and you need to change UA geo/demographic composition. Knowing what your retention or conversion numbers are won’t directly help you improve them – but you’ll know to focus on those areas.
Second, cohort-based metrics are most relevant for short-term / early game behavior. You can look at D7/14/30 KPIs – and even a bit later. But long-term cohort considerations usually feel like an a posteriori move. You can retrospectively go back and determine what your D360 or D720 LTV is. It’s unlikely that at any point in the title’s lifecycle a business/product decision will be made on that metric. The longer the timeframe, the more diluted the impact of user behavior. What % of installs remain 360+ days after install? And how much can those isolated individuals significantly impact the cohort-level value?
This is precisely why a cohort-based approach should be combined with a Live Ops based approach. If cohort based KPIs bring everything back to the initial cohort, then a live ops approach takes the actual game as it’s object of study. Where the time frame of cohort-based will put day 0 on install day, a Live Ops approach will look at things from the perspective of the calendar date.
A Live-Ops perspective will consider the live game and its active users – regardless of where they came from or how much it costed to get them in the game. If you think of the lifecycle of a game as spanning over multiple years – which is the case for successful games – then you need to have the appropriate tools to assess your title’s performance as the userbase ages. It’s not uncommon for a game that has been around for a few years to have 50%+ of its revenue come from users who have a tenure (the “age” of the user – the time since s/he first installed the game) of over 1 year. No D360 metric will help you determine the course of action for your live game.
It’s also important to keep in mind that a non-negligible portion of your revenue will come from contingent inputs – seasonal events or offers, the introduction of new features (there always is some kind of novelty effect surrounding the introduction of any feature), etc. So, to a certain extent the regularity presupposed by cohort based KPIs flies out the window. A Live Ops approach is much better suited to help you navigate the contingencies of game inputs on a daily basis (events, offers, seasonal features, etc.), and the specificities of a userbase composed of various types of users – geographical/demographic, but also different tenures
What is a customer and what is customer concentration (and why it matters)?
There is a variety of relevant Live Ops based KPIs. Daily conversion (what % of active users have done an IAP that day) or ARPDAU (what is the average revenue per daily active user) are common ones. Following day/week/month retention is also a very valuable metric than can help you assess the engagement of your mature userbase or the impact of a newly introduced feature – something retention will not help you identify. Customer is a key live ops qualifier you should always consider when analyzing your game and your userbase. A customer is an active user who at any point in the past has spent money (IAP) in the game. So for starters, “customer” is a way to focus only on the part of your userbase that drives your IAP revenue. Also, even if your customer is not spending on a given day, it’s important to distinguish him/her from a nonpayer. A customer has a much higher spending potential than a nonpayer. Even if your customer has not spent in the last 30 days, it’s most certain there is a higher probability to see him/her spend than a nonpayer with the same tenure – or even a 30-day-old non-payer. You can expect that the longer a non-payer stays in the game, the harder it will be to get that user to convert (i.e. do their first IAP). So catering to your customers is a way to cater to your users who have auto-identified themselves as willing to pay – and therefore most likely to spend on a given day. Catering to customers will always be the path of least resistance (and therefore probably your highest chance at success). Customer/nonpayer is the most important way to segment your active userbase. All the live ops KPIs should be segmented for customers: ARPDAU, conversion, following day/week/month retention, currency spending, etc.
Retention and conversion are 2 crucial cohort-based KPIs. But if you only look at those KPIs, you might miss out on some important insights concerning your game and your userbase. For example, you can have a 15% retention and a 2% conversion day 7. But those 2 numbers together won’t tell you what % of users active 7 days after install are payers. It’s unlikely that 100% of your users who converted by day 7 are showing up day 7. In other words, looking at retention and conversion won’t tell you what an active userbase actually looks like 7 days after install. And that’s key information that can help you optimize your game for a number of reasons.
Customer concentration is a great metric because it’s a way to conflate both retention and conversion. Retention by itself won’t help you distinguish between customers and non-payers. Conversely, conversion won’t provide any indications concerning the activity levels of those customers – you can have good early conversion but a problem when it comes to retaining your customers. Customer concentration is the % of active users on a given period who are customers. This is not a cohort-based way of looking at things, but purely a live-ops one. In this case we are concerned with users actually playing the game – and how many of those are customers.
a) Customer concentration in days since install
Looking at the way customer concentration evolves as users age – in days since install – is one of the most interesting way to look at customer concentration. It’s a great indicator of your tuning and how “non-payer” friendly you are. For example, 30 days after install, what do you think is the % of active users who are customers in Game of War, in Candy Crush? I have not been exposed to metrics for either of those games, but my intuition is that the customer concentration in Game of War would be higher. It’s much harder to stay engaged and keep playing a game that has many frictions or payer-exclusive content as a non-payer. Also, because you can expect customers to be more engaged and retain better than non-payers, as time goes on your remaining userbase will have a higher concentration of payers. So the cohorted conversion value can be misleading when trying to assess the weight of your customer-base. Even if your D7 conversion is in the low single-digit percentage, it’s very possible you’re looking at double digit customer concentration when looking at users who are returning and active in your game. Even if it’s only a matter of changing your perception, it might be worth you try the exercise for your game. You might realize you have more customers in your active userbase than you think.
The harder it is to progress in a game without paying, the higher you expect the customer concentration at a given point in time – that should also mirror retention. The harder it is to progress in a game without paying, the lower your retention. If you have access to data of a portfolio of different genres of titles, you should be able to confirm that customer concentration for more casual games is lower (if not, would love to hear about it). In other words, customer concentration can be a great metric to see if your economy is a tight as it should be. Considering the proportion of active users who are customers – from the perspective of active users, not of initial installs – is a great metric to assess your game balancing. You can expect a game which is very generous or which has little frictions (or which is so engaging users are willing to put up with the high frictions to keep playing it) to have a lower % of active users who are customers than a more hardcore one. The lower the customer concentration, the more non-payer friendly it is. That might be the strategy you want to go with if you have a high volume/low monetization strategy. Conversely, if you are going after deep monetization, and you want to achieve it, having high customer concentration can be an indication you’re on the right track. If you have different games to compare, it’s a great way to compare yourself to another title – and determine if you should aim to be tighter or not.
You can use the sample query below to get the same results for your game
b) customer concentration in actual date – and day of the week
As your title ages, you’ll (hopefully) see your customer concentration go up over time. If you get a feature (or any large influx of new installs) that percentage will go down – and then it should go back up. If you see your customer concentration stagnates or goes down over an extended period of time, then that can provide good insights into problems of your customer base (or perhaps more unlikely the virality of your game which somehow skyrocketed and is bringing you in much more installs every day).
Just like tenure, customer concentration is important to keep in mind when you are planning on how to monetize your game. You don’t monetize a puzzle the same way you would monetize an RPG. In the same vein, you probably won’t be successful if you try to monetize your RPG the same way on launch day and 3 years later. If your game has a higher concentration of payers, then you should probably take that information into account when tuning your events, making your live ops, or releasing new content and characters. Your n.1 focus when designing features and offers you expect will drive the bulk of your revenue shouldn’t be non-payers [you should always have “conversion offers” or offers that cater to your lower spending customers. But those offers will probably not be driving most of your revenue]. You should try to get your larger and larger bulk of active customers to make a redeposit (maybe at a higher price point)
The query can further help illustrate the differences when looking at customer concentration in days since install and in calendar days
Looking at customer concentration from the perspective of activity date will help you identify problems and big shifts – but most of the time there really isn’t anything immediately actionable you can take away from this. One thing seems to be happening very consistently is that there are weekly cycles in customer concentration – just like there were dips in customer concentration every 7 days (the cause is similar but not identical). Namely, customer concentration tends to be lower on the weekends (Saturday and Sunday). These weekend dips occur despite the fact that you are probably looking at more active users – both customers and nonpayers – on weekends. Both customers and nonpayers play more on weekends; but relatively speaking more nonpayer play on weekends than customers. Of course you’re probably not contemplating a 2x variation. But even if you’re looking at a 5-10% variation, there might be some easy actionable points for you to organize your live Ops – and capitalizing on these slight variations can help you move the needle in the right direction. Looking at customer concentration from the perspective of the day of the week can help you determine or optimize your Live Ops calendar. When is the best moment to start a weekly event, when is the best moment for this offer to be successful, etc?
Leveraging this metric for your game can have potential for the way you plan and organize your Live Ops. For example, you might have a higher % of customers Tuesdays-Thursdays, and a lower one on weekends (check it out). With that information, it would be make sense to have a lower price-point offer aimed at conversion or redeposit for low customers on weekends, and higher priced offers in the middle of the week. Conversely, if your goal is to foster engagement with an event, would starting them on weekends be the best course of action? Would you reach more users (customers and nonpayer) at the beginning of the event, and therefore have a higher % of users participating in the event? Ultimately there is no one size fits all solution. But looking at the composition of your userbase, the way it varies depending on the day of the week and basing your Live Ops calendar on that information can be a step in the right direction when it comes to optimizing your game performance (whether you measure it in revenue, active users completing an event, or engagement and following day retention).