I first had the concept of lagging and leading indicators explained to me via the great book Continuous Discovery Habits by Teresa Torres. It turns out, the notion of leading indicators is intuitively understandable but a bit tricky to implement in practice.
I just experienced that myself (again). We recently defined OKRs for a product. After we were all set and done, we realized that all our key results were lagging indicators. We now need to revise those, since the product team needs leading indicators to assess the impact of product developments.
This is the reason I chose to dive into the topic of leading indicators. I will explain what they are, why they are so powerful, and share some best practices on using them. First, let me explain what we are talking about.
The Backlog is a bi-weekly newsletter about the undervalued and overlooked in modern product development. It covers product development, self organization, and productivity. I include methods, books, and write about my own experience. The target audience are Product Owners, Scrum Masters, Developers, and project leaders. The Backlog is about getting the most out of product development.
Subscribe to get new posts straight to your inbox.
The difference between lagging and leading indicators
As their names suggest, lagging and leading indicators reference how immediate they respond to change.
Lagging indicators are retrospective or historical metrics. They change as a result of past actions or decisions after a certain time period. Typically, lagging indicators are based on data or feedback that needs time to be collected. Their movements are by definition, well…, lagging the implementation of developments of the team.
Examples of lagging indicators are:
- renewal rates of subscriptions
- customer churn
- market share
- customer satisfaction
On the contrary, leading indicators are forward-looking and proactive. They are often based on human behavior and the interactions with our product. As such, they are much more sensitive to changes in our product and respond very quickly to those.
Leading indicators are a great tool to use in testing a product or feature before it is fully launched or implemented. They are great to evaluate experiments as you see the impact almost right away.
Examples of leading indicators are:
- conversion rates
- number of daily active users
- the number of shared social media posts
- anything related to product engagement, e.g. time spent with product, number of clicks, engagement with specific features, scrolling etc..
Seems easy enough. Why is the concept so important?
Why is it so crucial to use leading indicators in product development?
I already mentioned it above. Product teams need to use leading indicators so that they can immediately measure the effectiveness of their product development.
The core of agile development – it should be at the heart of all product development efforts for that matter – is iteratively moving forward. This means building something, evaluating its impact, and adjusting the plan to proceed according to the feedback.
For this to work, we need to understand the impact of what we are doing without having to wait a long time. Based on this impact we can decide our next steps.
I attempted to illustrate this in the picture above. The top half symbolizes Team 1 using a lagging indicator (e.g. revenue) and the bottom half Team 2 using a leading indicator (e.g. number of daily users). Both teams are equally fast in the actual “building” of their product, they are equally efficient in implementing features.
Team 2 realizes very quickly that whatever the first iteration was doesn’t work (-5% users) and is able to react and test two more ideas in the given timespan. Team 1 on the other hand, needs to wait significant time until it sees the impact on revenue. In that time, they are either working on something else or essentially flying blind. Only after that time period do they realize that their first iteration (or something else in between) may have caused revenue to go down.
Team 2 is able to test three ideas in the same time as Team 1 is able to test two. If we build something and then need to wait 90 days before we see an impact on revenue, that’s too late. The use of leading indicators is thus crucial in iteratively developing a product.
The relationship between business and product outcomes
Now, increasing revenue is still a worthy goal for a company. In fact, many lagging indicators like increasing market share or reducing churn are desirable business outcomes. They are also influenced by other functions like marketing, customer service, or sales.
For these reasons business outcomes are – generally speaking – not suitable as goals for the product team. Thus, the team needs to define product outcomes that it has full control over, measured by leading indicators. Crucially, these product outcomes need to correlate with the (lagging) business outcomes the company wants to achieve.
Setting the product outcomes is often a negotiation between the team and leadership. Oftentimes, this is not straightforward since we don’t always have proof that the leading and lagging indicators correlate. In the end, we need to jointly agree on leading indicators that – according to our best estimate – will eventually drive the lagging business outcomes.
It is important to remember that this is essentially a hypothesis we are making. During our day to day work we then need to monitor if the hypothesis was correct and the leading and lagging indicators do actually correlate (and adapt them if they don’t).
Best practices in defining leading indicators
Before getting to the best practices in defining leading indicators, a short preface.
When setting goals or OKRs, by far the biggest lever in improving the product and creating an empowered, happy team is moving this focus from features to outcomes that the team should achieve. So, before worrying about leading and lagging indicators, we need to do that.
Having said that, and assuming we are in an environment where outcomes are at the heart of product improvement, I want to share three of my best practices in setting leading indicators.
- The best source of leading indicators is user behavior with the product
- Spend time to investigate
- Negotiating leading indicators
The best source of leading indicators is user behavior with the product
Defining what leading indicators actually are can be a bit tricky. Depending on product maturity the definition of leading and lagging might also change slightly. Some teams may run a lot of experiments and are in need of feedback after a few hours already. For others it may be ok to get feedback after a few days or a week.
Regardless, the thing that changes the fastest with modifications in the product is user behavior. For this reason, the best source of leading indicators is anything that has to do with user interaction with the product (time spent on a website, number of clicks, conversions, etc…).
If you can’t come up with a good quantitative leading indicator it can also be a valid option to watch users interact and qualitatively assess changes in user behavior. This sometimes is perfectly fine to see the impact of changes – although it is hard to scale. It can, however, also be a good inspiration for setting quantitative leading indicators.
Spend time to investigate
I already mentioned it above but I do want to emphasize the importance here. Very rarely do we have mathematical proof that our leading indicator for the product outcome really correlates with our lagging business outcome. We are hypothesizing that this is the case.
It can be really frustrating if the team optimizes for one measure just to find out that this was the wrong one. There are two ways to deal with this situation. One is obvious. The team needs to monitor if the different indicators are moving in the same direction.
The second way is not as obvious, give the team time to learn about the indicators. Before defining the leading indicator and setting a performance goal (e.g. increase time spent on a website by 10%) it generally makes sense to do discovery work and first set a learning goal (e.g. understand what opportunities impact time spent on a website).
This way, the team has the chance to truly understand the measure they are attempting to impact, what a realistic goal is, and research if it correlates to the lagging indicator.
Negotiating leading indicators
Finally, the team will very likely not be able to set the goals themselves. At the very least someone will need to sign off on them. More likely, someone from leadership would like to have a more profound impact than simply signing off.
This is a negotiation. One that is often difficult, particularly if the team is part of an organization that is used to prescribing outputs the product team should deliver.
My suggestion is to always try and get the leader(s) to explain the business context and business outcome we are trying to achieve, even if this can be a very delicate discussion. Try to create a joint understanding of what product outcomes (instead of features) likely impact the desired business outcome.
It really depends on the leader. With some, this discussion is very easy and they get it right away. Others may be less open and start being defensive when asking how their proposed feature is supposed to impact the business outcome. There is unfortunately no one size fits all approach here.
The goal should always be to try and connect the dots from lagging business outcome to leading product outcome to features.
To summarize, many organizations use lagging indicators to set business goals. These are not suitable as goals for a product team. It needs leading indicators that are immediately impacted by changes in the product in order to assess the impact of product increments almost in real time. This is the only way for truly effective product development.
While this seems pretty intuitive. Actually defining the leading indicators can be a bit tricky. The best source for them is anything related to user behavior. If it’s difficult to determine which indicators to use, give the team some time to discover the best ones.
Then all that is left is to (delicately) push the leaders towards an outcome based approach and jointly link the leading product outcomes to the lagging business goals. Once you can confirm that they actually correlate, you are good to go. You can focus on the really important stuff, building!