Engagement, retention and commitment: how demanding is your game (or how committed are your users)?

Engagement, time played and login

In a previous post I played around with the notion of lifetime engagement. Now, engagement is an elusive concept. What does it mean to be engaged – and what is a good indicator of engagement? In that post – building on (my understanding of) EA’s stated vision for the future – I was focusing on “time played”. But engagement is not reducible to time played. If user A plays 140 minutes on install day and never plays the game again, and user B plays the game 10 minutes each day for the first 14 days, who is most engaged? Both have the same day 14 lifetime minutes. Now imagine user C plays 5 minutes each day for the first 21 days. Who is the most engaged of the 3?

These are purely abstract questions that don’t help inform any product decision (so they can be fun but are not really useful or productive). Fundamentally, there is no objective description of what “engagement” is – and that also means there are no correct definitions. “Engagement” (like any other metric) will become useful if it’s defined in a way that helps address specific questions.

Now, if you’re approaching this from a streaming or pay-per-play point of view like EA would be, then tracking specifically minutes played can make sense. But I think mobile gaming remains a mode of entertainment that accompanies users in their day-to-day life. It’s what Colopl (I’m a huge fan of White Cat Project – which incidentally I feel hasn’t been discussed enough in the West) refers to as “Entertainment in Real Life”.

In the grand scheme of things mobile games so far have been a very mundane and modest part of users’ life – even for the “very engaged” users. Mobile games are all about becoming a part of our users’ day-to-day routines. About becoming an ordinary part of our users’ life. The success of a mobile game does not lie in its ability to provoke dramatic events and life-defining moments. It’s about blending in. And about staying part of the user’s day-to-day for weeks, months and years (we’ll see to what extent hyper-casual games are successful at contesting that paradigm).

From that point of view, logging into a game feels like a better indicator of this “becoming part of users’ life” than time played. And specifically, when looking at this discrete action – logging in or not – you want to be looking at when it occurs, and how frequently. Becoming a part of a user’s day-to-day means the user should be logging in repeatedly over a long period of time.

When retention of not enough: cumulative logins and login velocity

Retention is the metric that tracks users returning to the game. It measures users’ propensity to return to the game. At the same time, it provides a common standard to compare different games that otherwise might not have much in common.

retention

 

The main limitation of the retention metric lies in the fact it’s only a measure of something punctual: “did the user return 15 days after install or not?”. It’s a static metric that cannot account for the flow, speed and dynamism of engagement. Retention will indicate what % of install returns to the game 30 days after install, it won’t indicate how often users have logged in up to that point. And if engagement has to do with showing up over time – not just showing up at one given point – then there’s an important part of the equation retention is not reflecting.

The first thing to look at is how many days has the user logged in lifetime 3 days after install, 14, 30, etc. The graph should very much look like any other lifetime graph.

days_logged

In the example above, installs in game 1 have logged in 5.3 days within 28 days of installs. A user in game 2 has logged in 7.0 days in that same period and a user in game 3 has logged in 7.3 days. You don’t see differences of that magnitude when looking at retention alone. When you think about monetization from the perspective of engagement, looking at the lifetime days logged in can provide a good way to estimate the potential revenue of a game. This would be especially the case for games that focus on monetizing off user activity via ads. Say you designed the game to ensure you get $0.01 in ad revenue for every day the user logs in. The cumulative days logged provides you with tools to estimate your potential revenue through time – and a way to put a dollar number on an increase in engagement.

If you look only at total number of logins you are missing part of the picture. A user who logged in 27 days within 30 days of install isn’t as engaged as a user who logged in 27 days within 90 days of install – even though they both logged in the same number of days. What’s particularly useful about looking at logins in days since installs is that you can think about it from the perspective of login ratio. Another way to say it could be login velocity. A user who logs in 27 out of 30 possible days will have a login ratio of 0.9. A user who logs in 27 out of 90 possible days will have a login ratio of 0.3. Having a ratio provides a constant standard to evaluate engagement throughout time: you can look at what’s happening day 30 and day 90 with a same metric. If you were to look at the login ratio of all users, then the graph is likely to tell you very little.

lifetime_login_ratio_less_useful_relfects_the_decay_of_your_userbase_over_time_since_install

This graph really only reflects the decay of your install base over time. The longer since install, the fewer users are logging in, the more the curve decreases. But, if you were to look at the login ratio of active users (thereby combining retention and login ratio) then you can get a strong indication of how engaged your users are (or how demanding your game is). In other words, for each day, look at the login ratio of users logging into the game.

lifetime_login_ratio_retained

This graph is a great way to assess the engagement of your userbase. In the example above the login ratio of a user active 70 days after install is 0.74 in game 1 and 0.63 in game 2. So that’s a clear indication active users in game 1 are much more engaged than active users in game 2. This visualization helps you assess the level of engagement of your active users, see how it evolves for a same game over time as you introduce new features, and compare different games.

You can see a couple of important things from the visualization above. First, there are weekly cycles of login ratios. Consistently there are dips in login velocity at 7 days interval. That means users logging in on the same weekday they installed are less engaged – and this further strengthens some observations made in one of my previous post. Users are more likely to return to the game on a 7 day interval. Presumably because if a user was available on a Sunday to install your game, chances are his/her day to day revolves around a weekly schedule (and days off are probably the same days of the weeks). S/he will have some free time the following Sunday and a higher chance to return to your game. That tendency to return can be observed in spikes in retention every 7 days. Conversely, because there are fewer “real life” obstacles to log in at 7 day intervals, there are more casual (read: less engaged) users logging in then. Therefore, the dips.

How is this actionable? Knowing when your least engaged users are most likely to be in your game can provide you with some opportunities for slight optimizations. You can trigger a custom login bonus on the same weekday a user installed that would be specifically valued by your casual users. If you monetize mostly through offers, you can also have custom offers geared towards casual users – in an RPG for example, the offers you design for a super engaged user and a more casual user won’t be the same. Those 2 users don’t have the same play patterns, the same preferences and the same objectives.

Second, this provides you with some insights into the level of engagement and commitment of your active users. Having a game where the login ratio is high isn’t necessarily a problem in itself – although I would be concerned if only very engaged users were active in my game. What can be more concerning is if you combine low retention and high engagement. Active users might be very committed because your game is very great – or the exact opposite… Active users might be very committed because your game struggles to accommodate only moderately committed users – and turns them away. A game in which only the most committed return might not be a good thing – you might want to entertain the less committed users to come back and play your game. Maybe the bar to engage with your game in the long run is too high. If your retention is lower and your login velocity is higher, this might be an indication only the most committed can “survive”.

In the example above, game 1 had the lowest retention but the highest login ratio. That would strongly suggest the game is struggling to find a way to engage its userbase long term (there are numerous courses of action that can be taken once that diagnostic has been made). On the other hand, the retention of games 2 and 3 was within a same range, but the level of engagement starts to shift 3 weeks after install. Looking at the login velocity of a game can help identify general trend, and the inflexion points in your users’ lifecycle. It can provide a more in-depth and relevant insights into engagement than retention alone – while at the same time providing a standard to compare different games.

4 comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s