The Backlog Prioritisation Dilemma

I worked as Product Owner across different industries (Banking, Finance, Real Estate etc) and in different countries. When it comes to prioritising features, I’ve noticed the same exact pattern over the years.

Our Prioritisation Process is Biased

Most product teams have a sketchy prioritisation process that goes like this:

We Put Together a List Of Features

First, we put together a list of features based on different stakeholders’ needs/wants:

We Prioritise Based On Biased Scores

The 15 features or so in the example above are all valid requests with valid reasons and need to be prioritised accordingly. We’ve only considered three features per team, but in reality each team has dozens of features to prioritise. As a result, Product Owners can easily find themselves with hundreds of items to prioritise and different stakeholders to satisfy.

To which extent are these features important? Which of these are game changers and which aren’t? To answer these questions, Product teams came up with a combination of two things:

• We arrange sessions with different teams to agree on a priority list. We need to walk them through different topics when each team member is only interested in his/her features.
• Very quickly we realised we needed some sort of well grounded arguments. So we devised an elaborated scoring framework to assign a score to each feature and prioritise based on metrics: Potential generated revenues, Savings, improved user experience and effort.

Again, there are all valid metrics. However, do we know how much revenue a reset password feature would generate? Or, how much we are saving or by how much we are improving the user experience? We don’t, because those metrics are almost impossible to quantify and take time to set. And if set, are all the teams familiar with these metrics?

So we end up with a very long list of features (or wishes) with biased scores that sometimes we cheat to meet our preferences. And what was intended to be a rational approach aimed at innovating and tackling real product issues, quickly becomes a rule of thumb approach.

What Can We Do About It?

You have a functioning product with existing customers you want to retain and potential new customers to whom you need to sell.

Existing customers want a performant and reliable product fulfilling its core tasks and constantly improving to reflect their changing needs. On the other hand, new customers may want the existing product to adapt to their use cases and processes.

We need to define quantifiable prioritisation metrics and track them over time. To that end, I devised an easy to quantify and track prioritisation framework called MIKS:

1. Maintain Your Platform

How?
Deployments must happen swiftly with automated testing running and without down time between deployments or compatibility issues. If not, our Product will fail to do anything it’s intended to achieve, engineering will lose time maintaining the platform and Customer will not be satisfied.

Metrics to collect and monitor:
A maintenance issue will be qualified as such and appear in the priority list if and only if it impacts the following metrics with defined thresholds:

• $Successful\mathbf{deployments}ratio\; =\frac{Nb\; of\; successful\; deployments\; in\; a\; given\; month}{Total\; Nb\; of\; deployments\; in\; the\; same\; given\; month}$
• $Service\mathbf{downtime}ratio\; =\frac{Target\; avg\; deployment\; downtime}{avg\; downtime\; in\; a\; given\; month}$
• Maintenance Ratios Target = 95% (for example)

Maintenance issues are encountered at early stages of our Product development BUT also as we develop new features and use new technologies. So it’s important that we always keep them in check and make sure our defined thresholds are met.

2. Improve Your Performances

How?
Performance relates to speed of execution, availability of service and robustness. All products have some performance thresholds they need to abide by. Optimal performance must be defined by and with customers and not by Engineering. Anything above Customer expectation is a nice to have. If for example most customers are using a daily report, why bother having a report loading every 10 seconds?

Metrics to collect and monitor (example):

In order to be able to monitor performance, we need to define some key metrics to monitor as well as their thresholds based on customers' needs.

• Speed of execution. Target = 5min max per job
• Availability of service. Target = 99.99%
• Throughput (robustness). Target = 10GB ingested per upload
• and so on

When those thresholds are not met, a new item needs to be added to the backlog.

3. Keep Your Product Metrics In Check

How?
Most of the time our products fulfil many tasks and have multiple components. The idea behind Key Product Metrics is to define and track metrics that, alone, encapsulate all the rest of the metrics and indicate how the product is performing its core tasks over time. All our metrics need to filter through to a few measures (usually ratios) that you can track daily and easily visualise and share.

Also, as the product is evolving, these metrics and their thresholds need to be updated to reflect the product change.

Metrics to collect and monitor (example):

• Visits VS Accounts created Ratio
• Processed files VS Uploaded files Ratio

4. Shorten the Product Life Cycle

Our IT products are becoming more and more complex. Multiple actors and doers work in harmony to complete a task. This is illustrated in the following diagram:

Customers complete a series of tasks, followed by back-end processing or automation, then customer support takes over and performs some other manual tasks before handing over back to the customer etc.

We retrieve this behaviour in most IT products today and this is what I call the Product Life Cycle: the series of tasks required to achieve a product use case.

For our product to scale, we want to reduce friction in the Product lifecycle, simplify the steps and add automation. This in return, will allow our teams and customers to perform more tasks, in other words to become more productive.

How?
We need to measure the complexity of completing a key product use case from end to end. This can be defined as a combination of time to completion metric AND number of human interventions (i.e by Customers OR Customer Support team).

Metrics to collect and monitor (example)

• Number of manual interventions of customer and customer support
• Time to completion from end to end

Limit The Number Of Features Prioritised In One Go
Some companies maintain a priority list of hundreds of items. This creates confusion for all stakeholders and is just impossible to prioritise with relevant metrics.

We all know we will never get to item 50 of the backlog this year, and by the end of the year the requirement may well change. So the Epic is sitting in the backlog consuming time to those who write it, those who prioritise it and to those who deal with its confusing appearance.

Instead, we should aim to keep the priority list as short as possible, taking into account only items that are effectively going to be worked on in the next quarter or so. My rule of thumb is 15 EPICs at maximum. Anything beyond that point is irrelevant to talk about, prioritise or even share with customers.

To summarise, your Jira list will look as simple as this:

The Benefits Of The MIKS Approach Are Multiple

• Remove the burden of dealing with unquantifiable prioritisation metrics
• Easy to track and showcase metrics that allow for easy prioritisation. In fact, once in place, it’s a matter of looking at the dashboard
• Literally metrics based and very transparent to everyone
• All stakeholders concerns and goals are taken into account and reflected in measures

As a result, our defined measures will ensure the product is scaling, is performant to the customers’ standards and fulfilling its intended core tasks.