John Doerr’s account of Objectives and Key Results (OKR) in his new book Measure what Matters – combined with the numerous examples and case studies in the book – makes for a compelling read (I strongly recommend). There are a lot of resources on OKRs out there (for example here) and Doerr’s book provides a comprehensive and in-depth account of the origins of OKRs and their application in various organizations.

OKRs are both a management methodology and a cultural model. The management methodology part provides guidelines to assess how you can work. The cultural aspect redefines what work is – how you define and evaluate activities within an organization. The main value of OKR is that it provides a framework to formalize the goal setting and evaluation process. By making explicit several things that are too often left implicit, OKRs help provide a framework for focus and structure. And this focus and structure can provide the tools for sustained performance over time in large organizations (and also smaller ones).

If OKR were reduced to one sentence, it would be:

Ideas are easy. Execution is everything

The goal of OKRs is to provide a framework to produce high-quality, high-impact outputs. Ultimately, it’s never about what the product is. It’s about what the product does and the ways it helps further your organization’s agenda.

What are OKRs?

OKRs provide clear and communicable ways to define, coordinate and align work within an organization in order to maintain high levels of performance through time. Objectives correspond to the high-level goals you plan to achieve. They involve a vision and provide inspiration for a faraway horizon. Objectives are what work must be oriented towards – they provide the basis to discriminate between what matters and what doesn’t. Does what you’re working on moving the needle? Is it helping you achieve your objective? If not, you should be doing something else.

Key Results on the other hand are the tangible, measurable and verifiable tasks that will ensure you reach your goal. Completing your key results should mean you have attained your objective. The main challenge in the OKR process consists in clearly identifying what matters and what doesn’t. And translating that objective into a series of key results that will help achieve it. If an objective is well defined and framed, there should be few key results.

Furthermore, key results are metrics driven, and should be defined in a way that removes any subjective judgment call: you have to be able to say without a doubt if the key results were achieved or not. For example, defining a key result as “users play more” doesn’t provide what you need to clearly say if it was met. What does a “user” refer to here (daily unique, weekly unique, users over 7 days, etc.)? How do you define “play” (is it time, missions, logins, etc.)? How much is “more”? On the other hand, having a key result defined as “daily active users who have a tenure of 7+ days spend on average 15 minutes in the app a day” is a clear and unambiguous key result (and therefore actionable within the framework of OKRs).

Doerr defines OKRs as a “collaborative goal setting protocol”. As such they are inseparable from an organizational model – from a way to organize, plan and coordinate work. But aside from this organizational model, there is a crucial cultural component to OKRs. Because of the way objective and key results are defined and evaluated, they also involve a different understanding of what defines work. OKRs force you to think about work inasmuch as it produces a measurable output. That’s what I find the most interesting when I think about the process required to make mobile games. Ultimately OKRs can help us formalize the way we think about defining the game’s development strategy (what are the aspects we should be focusing on, what should our goals be?) as well as the way we should approach and understand feature development (how do you define and evaluate a feature?). OKRs are always about execution: what are the outcomes? And the outcomes are defined by their measurable impact.

Thinking in terms of OKRs helps formalize 2 key things in mobile gaming that usually remain implicit. First, they help formalize the role of data in the feature development process. Second, the OKR model helps define the characteristic aspects of a feature – both the why and the what.

Defining OKRs – the role of data

How does data fit into all this? The OKR model is an intrinsically data-driven approach to organization and management. Targets are defined in a measurable and unambiguous way. Data will determine if we met the targets or not. But key results are defined in such a way that there is no judgment call required in their evaluation. Key results should be clear, measurable and unambiguous. Anybody should be able to answer in the same way to the question: “was the key result met or not?” If data were limited to measuring the output, it would be reduced to a very passive role – which is clearly not the case in an OKR framework.

Ultimately data permeates the entire process. Data is not only about measuring what has been done. A data frame of mind will inform the judgment calls involved in defining both objectives and key results. Data provides a framework to read and understand the business environment – and act accordingly. Data is involved in identifying the relevant metrics – and even more fundamentally thinking about your objective from a metrics point of view (even if an objective doesn’t necessarily have to be quantifiable). OKRs force you to think about products in terms of functions – in terms of outputs and of their impact.

Data plays a key role is in the definition of the objective. Doerr insists on this. If you can come up with your objective in 5 minutes, then you haven’t put enough thought into it (that’s also why OKRs are an exercise at which you get better over time). Data should be instrumental in defining your objective – instrumental in defining what matters. The YouTube case study in the book provides a great illustration of this. Initially success at YouTube was considered in terms of views or clicks (at the time YouTube ads were shown at the beginning of a video). By thinking about the product and its customers in an analytical way the team was able to redefine YouTube’s purpose differently. To make sense of its mission in a different way. In turn this meant redefining what really mattered – and setting an objective in accordance with that. Once “watch time” became the central metric, OKRs provided a framework to organize and orient work for the following 4 years.

Data also plays a key role in identifying the key results that will help accomplish the objective. And for mobile games this is the key part. In an organization objectives and key results cascade down. A manager’s key results cascade down and become the objective down to the next level of management. So, a GM’s objective might be to make a top grossing game, and one of her key result can be to generate $250M in FY2020. That’s a well-formed key result: both measurable and tangible. However, this can hardly be considered as a guideline to make a game or a feature.

 

In this example one of the GM’s key result becomes the product owner’s objective

 

And one of the product owner’s key result becomes the design team’s objective – which of course has its own key results

 

If you are to design a feature, you first need to know what levers in your game mechanics will move the lever – or be able to identify the levers that are missing and that you need to implement. Anybody can come up with key results. The point is precisely to identify the key results that will best help you achieve your objective. In mobile games this is the level at which data should be particularly central: at the feature definition level. Specifically, what are the levers that move a game’s performance. Data should be able to inform the design process by identifying the correlations that exist, the actions most conducive to the desired outcome. Data has a key role in informing the design of what should be done – it’s not just the a posteriori analysis of what was done.

So, data should inform the game’s strategic direction by highlighting the relevance of key behaviors, and the ways to move the needle in the good direction. Incidentally this has been my guiding principle when considering day 0 return rate, why early conversions are important and what can be the best ways to encourage early conversions, how to get users to redeposit shortly after a purchase, or of designing for your most engaged users – both those who pay the most and those who play the most. That and the strong belief that most of the levers that matter the most are pretty much game-agnostic. Some of the key factors in user behavior have to do with the way users interact with mobile games in general – more than the unique specificities of each game.

Executing OKRs – what is a feature?

There is a fundamental principle at the core of OKRs: what is valued and emphasized is the final output. That means it’s not the idea behind the feature that matters. It’s not the ideal image of what the product is. Applied to games, another way to say this is that what matters is the impact a feature has. It’s not about the idea or intention behind a feature. It’s about what is ultimately implemented – and what the implemented feature actually does. Game features are defined by their functions, by the type of impact they have.

OKRs help bring focus to the development process – only work on things that help complete the objective and that are in line with the key results. And by defining key results in an unambiguous manner, it’s clear for everybody on the team what the expected result of the feature is. Having clear and unambiguous key results is a way to ensure everyone can be aligned. Everybody should understand without interpretation what the goal is – and be clear on how everyone’s contribution helps achieve the main objective.

Having tangible and measurable key results is the glue that ensures everybody is aligned. But more importantly having tangible and measurable key results requires us to redefine the way we understand a game and its features. OKRs force you to think about features in a functional way. You need to think about features as having a tangible and measurable impact. What will define and characterize a feature is the effect is has on the userbase. You can no longer think about features in an abstract and ideal way. It’s not about the intention or idea behind the feature. It’s not about the “essence” of the feature. These are fuzzy ways to think about features that cause loss of focus – and also a costly and unproductive loss of alignment among members of the team. Everybody will have a different interpretation when you want to “design a compelling feature”. On the other hand, there is no ambiguity in “design a feature 80% of daily active users start at least once a day”.

When you focus on what a feature should be – and not on what a feature does – there are a series of problems that can arise. Using an OKR framework helps avoid the pitfalls that can happen when you fall in love with the idea of a feature – as opposed to an implemented feature. When you focus on what the feature is (or ought to be) you run the risk of doing things to design your ideal feature – even though it might not contribute towards achieving what you want. The feature can never become an end in itself. The goal is always the impact a feature has. Once you start sticking to an OKR framework, then you must be thinking about the impact a feature has. And that means you need to be selective when choosing what to do – and raise the bar for why something is worth doing.

Another way to frame it is this: it doesn’t matter what your feature is – features should never be considered in of themselves. Features should be considered as far as they accomplish something. As far as they have a measurable and tangible impact. Taking an OKR framework doesn’t mean everything you’ll do will be a slam dunk and that you’ll be releasing 20 top 10 grossing games a year. But it does ensure you’ll have to formalize what matters – and justify why. That means you’ll be raising the bar from both a process and content perspective. You’ll be taking a data and performance-based approach to product definition and strategy.