I’ve previously written a post on John Doerr’s book “Measure what Matters”. In his book Doerr repeatedly refers to Andy Grove as the father of OKRs. Grove’s book “High Output Management” was initially published in 1983 but is still extremely relevant (for me at least) when considering free-to-play games and digital entertainment. This week I’m posting a few random thoughts that have come to me as I read it.

 

Principles of production – output, not activities

The main idea of the book is Grove’s output-oriented approach to management. That pretty much means applying the principles of production to everything: focus on delivering a product in response to customers’ demands within a scheduled delivery time, with acceptable quality levels and at the lowest cost possible. The key point of the principle of production is focus on the output, not the activity.

Every activity must be done with a specific, clear and unambiguous operational goal in mind. “What do I want to achieve?” (the why) must be the operating guide that presides over every action undertaken. And the criterion to evaluate performance is always the output – not the actions taken to achieve that outcome.

In this context, what Grove goes over in more detail is what he calls Management by Objectives (MBO). There are 2 main questions that preside over all activities in this context:

• “Where do I want to go?” (the Objective)
• “How will I pace myself to see if I am getting there?” (the Key Results)

The Management by Objectives (or OKRs) help provide focus by just concentrating on a few clear and unambiguous objectives. There are many different key results that can guide you to your desired objective. The art becomes choosing the most relevant ones – and choosing a small enough number of key results to ensure you don’t lose focus. And always with the ultimate aim of achieving the objective you’ve set out.

 

What is the output when building a free-to-play game?

OKRs are specifically relevant for free-to-play games. More than traditional boxed product games, success in free-to-play games is about output rather than activity. It’s about the player behavior your game/feature induces – not what the feature is or the way it works.

Take the case of a traditional “boxed-product” game for console (or a movie if you want an example from another part of the entertainment industry). Players pay to acquire the game (or enter the movie theater). And that payment occurred before your customers actually experienced the product first hand. Once that purchase occurred, customers have access to the product – but then there is nothing left to purchase (again, this is an example of a traditional boxed-product model, not a hybrid model).

The difference between free-to-play and a boxed product model is that in the free-to-play model payment follows the player’s direct experience of the product. What players experience – and their behavior as they experience it – it the crucial aspect here. So the output for a free-to-play game is the way in which the game modifies the way players experience the game.

Engagement is a key factor for the business success of a free-to-play product. But not in the simple sense that more engagement equates to more monetization. There are different types of engagement, and different profiles of players engaging with your game. Just having players engage with your game and coming back regularly isn’t enough to ensure you can monetize your game.

In free-to-play games, monetization is tied to engagement in the sense that players will spend throughout the course of their engagement with the title. That means that in free-to-play games engagement results from the specific way players will interact with the game. Retention is one of those factors, but when it comes to monetization there is a lot of evidence that suggests it’s not the most important factor. Having your players in the game is a necessary but not sufficient condition to monetize well. And more often than not you’ll be better off if you prioritize monetization over retention. You can try to keep a player who’s on the fence and about to churn from your game. Chances are if s/he was about to churn from your game, s/he’snot very passionate about your game. That means those players are not very likely to spend – even if they don’t churn and retain a bit longer. Stated differently, for some players there is little ROI in trying to get them to stay in the game (the key is identifying high value/high potential players who already like your game and focus on monetizing them as well as possible) .

The fact that engagement is a prerequisite for monetization in free-to-play games has a direct impact when considering OKRs. In a boxed product model, what’s important is what the game is. Players spend in order to experience that game – players are purchasing the ticket of entry. So striving for a coherence and an overall unity of the product is key in the boxed product model. The desired outcome is at the level of the game – in order to ensure it’s good, inspirational and compelling enough to get potential customers to make that “leap of faith” purchase to experience it. In a free-to-play model, the desired outcome is not at the game level – the desired outcome is always a specific user behavior. It’s not just about having players in the game. It’s about having players in the game and acting a specific way.

In a boxed product, the objective has to do with the game (how does this feature impact the coherence or consistence of the game taken as a whole?). In a free-to-play game, the objective is a specific player behavior. The game is a means to an end.

How to leverage metrics in that context?

Management by Objectives – or what we would now call OKRs – is meant to pace and help ensure the planning process is successful. Its purpose is to help ensure the planning process leads to the desired objective. In other words, the Key Results are forward looking – they are not there to evaluate. OKRs are part of the planning process. They are always forward looking – with the final output in mind. In mobile games, it has become easy to focus uncritically on the metric itself, and lose sight of how it can help us achieve our objective. Retention, conversion, matches played per day are indicators that must help determine our future course of action, define the roadmap and revisit the game economy. What Grove’s book helps reminds us is that simply looking at a metric without having a clear objective in mind is not useful (actually it can often be counter-productive).

You can miss every single Key Result, but still achieve your objective (but in that case, the key results chosen were not the best ones). And that would be a success. Grove provides the example of Christopher Columbus. He didn’t meet his Key Result (which was to discover an alternative route to the Indies). But the overall objective which was to provide more resources for Spain to wage war was met. In that respect, his endeavor was successful.

The purpose of key results is to tell us how we are doing so we can make some adjustments as needed. You’re not un/successful if you meet your key results or not. Say you’re building a new game mode, and your objective is to increase your game’s overall redeposit rate by 20%. You might associate a few key results to your development process to help guide your design and implementation: say you want 50% of your active customers to interact with that feature on a daily basis, and you want your tutorial completion rate to increase by 15%. You could miss your 2 Key Results but still achieve your objective. And that would be a success (but you would have to work on your Key Result definition process).

The key indicators are not the desired outcome, they are guidelines. The better the guidelines, the more likely you are to assess whether or not you’re on track. There is a duality about metrics and indicators that is important to keep in mind. The metrics/indicators are not an end in themselves. They are means to an end. They are the tools that will help us achieve our goals. But as Grove mentions indicators tend to direct your attention towards them. They take on a life (and a purpose) of their own. That means – at least implicitly – the indicators you chose will guide your course of action. Also, the number of potential indicators you can track is virtually infinite. So you have to make sure that you are intentional in picking your KPIs, and that they serve a purpose to you – if not you end up blindly serving your KPIs. One of the main things about indicators is appropriately discriminating between the indicators that help you maintain the course to attain you main goal from those that don’t add value. You need to be very selective when choosing indicators. That will provide you with the guidance (and focus) to achieve your operational goal.

You need indicators to operate well. That’s why a good indicator must focus on the output of the work – not just the activity involved. These indicators must clearly spell out the objectives, be measurable (to provide a degree of objectivity) and provide a standard to compare the evolution of a product over time, or between different products. To be useful, key results need to be specific and be unambiguous

With OKRs you need to focus on the output, not on the activities. When managing a team and a product, you need to focus on the activities that generate the greatest output – that means you need to have the few indicators that are going to help you assess whether or not you are on track to achieving your goal (read: generating the user behavior you want).

It’s crucial to keep in mind the fact that Indicators are functional tools to perform and achieve your goals. Indicators are not the objective. They are the instrumental piece of information that will tell you if you are on track with your initial plan – and if not, the indicators should let you know there is a problem that needs addressing, and where you should be orienting your efforts to correct course and get back on track. That means you need to believe in the validity of any indicator you’re going to use. If you’re not confident in the indicator you are tracking, if you are not willing to act on it, then the indicator is not operational – it’s just a source of anxiety.

Stated differently, you must use and adapt metrics to your specific goal and purpose. You can’t just blindly aim for KPI targets if they don’t directly relate to a general goal.