Backlog refinement, frequently overlooked but so crucial to success in product development

Besides the retrospective, the backlog refinement (sometimes called grooming) is in my view the most important ceremony within the scrum framework (even though it’s not officially part of the Scrum Guide). One could even argue that it is the most important one. It certainly has an outsized positive impact on team productivity, created value, and team chemistry – if done correctly.

This also means that bad refinements can have a massively negative effect resulting in frequently changing scope and requirements, incomplete sprints, and in the end an unhappy, unproductive team. Refinement is pertinent to Scrum but most of what is written below can be adapted for other frameworks when defining requirements or work packages.

Email address:

Leave this field empty if you're human:

In the end, refinement is the process of getting stories or work packages in a ready state, ready meaning that all involved understand what is expected and any team member could start working on it. In Scrum terminology this means the definition of ready is met and the story can be pulled into the sprint.

The important word in the paragraph above is process. Refinement is usually understood to be a standing meeting within every sprint – and it’s usually a good idea to have that – but it actually is a constant and never-ending quest to eliminate misunderstanding between Product Owner (PO), Scrum Master, and development team and thus the creation of crystal-clear requirements.

It is the translation of requirements from business language to technical language. This translation is hard! That’s why it doesn’t work without the constant back and forth between team, PO, and Scrum Master.

Generally, what happens is that the PO specifies what features or requirements for the product should be implemented within a user story or task. In the discussions that follow, it is then adapted by rewriting aspects of it. The requirements can be split into multiple work packages (see also Slicing of Requirements). In the end the feature is ideally defined in such a way that everybody understands what the goal is, business requirements are kept, and that it is technically feasible for implementation within one sprint (usually within a couple of days).

There are a multitude of issues that arise in the refinement process. Three are in my experience the most common.

• Implicit requirements and misunderstanding
• Emphasizing the how instead of the what and why
• Efficiency of refinement

Luckily, there are ways to deal with these.

Since the refinement process is all about removing any and all misunderstanding as to what should be implemented, it is extremely important for the PO to be explicit about what is required. In my experience, the small things that are very obvious to her or him due to prior discussions with stakeholders are usually what cause the most problems, since they are not clear to the rest of the team.

The more experience and familiarity PO, Scrum Master, and development team have with one another, the more likely it is that the team already knows some of the implicit requirements. However, I believe it’s better to be on the safe side and actually make all requirements explicit by formulating them.

A good way to do this, is to write it out in such a way that a person with no prior knowledge of the product understands what is needed. One of my colleagues once suggested to formulate it in such a way that small children, your grandmother, and even your boss will understand it. That’s generally a good rule of thumb to use.

Another tactic to do to avoid misunderstanding is to have the team explain what needs to be done in their own words. Often times, this can lead to some very interesting insights.

Often during refinement discussions, the team will start talking amongst themselves about ways to implement the requirements. In case the Scrum Master or PO have some technical knowledge, it can be very beneficial for them to closely listen in order to understand if the proposed implementation satisfies the requirements. If there is some sort of misalignment they can then intervene, of course without prescribing the manner of implementation.

This is actually one of the most frequent mistakes POs make in the refinement process, focusing on the how of the implementation instead of why and what needs to be delivered.

The needed features, the requirements, or the contents of the user story are what needs to be implemented. This should be what is formulated.

In order to give the team some background information it also helps to explain why it is being done, what the customer benefit will be. There are only very few extremely specific cases, when the manner of implementation, the how of the work package should be defined upfront.

In the course of discussing the requirements the team might discuss how to implement it and it’s perfectly fine for the PO to comment. He or she should, however, not define it. The only way to influence the how for the PO is to change the requirements, the what.

For me as a PO with a technical background – and I have witnessed the same with colleagues – this can sometimes be very hard. Of course, it helps to have an idea of how the feature could be implemented to understand e.g., if the feature is even feasible. However, focusing on the matter of implementation by the PO severely limits the scope of possible solutions. In the end, quality will suffer.

It can be very surprising to see the type of results you get just by stating what you want without how it should be done. I would even go so far as to argue that the real magic happens when neither PO nor the team have a detailed idea about how to do it. In this case you have no choice but to state the goal and somehow help the team to work through the problem. I have witnessed some very creative and exciting solutions for these types of problems.

Refinement meetings often take longer than anticipated. Although this is usually time well spent, there are some ways to improve efficiency and maximize time spent on the difficult problems while minimizing time spent on trivial topics. First and foremost, a great Scrum Master obviously can have tremendous impact by guiding the discussion, asking the right questions, and also reminding the dev team that fine tuning the requirements is a collaborative effort where everybody has the responsibility to contribute.

In my current role, it has been really helpful to set-up a pre-refinement time block two days before the actual refinement. In the pre-refinement the team goes through the fully prepared stories and already checks for obvious flaws or unclear requirements upfront.

This has the benefit that I can then already modify the story based on that feedback before we have the discussion. Comments range from “I have no idea what you mean with this” through “we propose to split this story into two” all the way to “all clear, we estimate this story to X points”.

I then have two days to take those comments into account and adapt where necessary. This then leads to a much more efficient discussion. We only shortly talk about the tasks or stories that were designated as being clear in the pre-refinement. Thus, we have more time to spend on clarifying the really tricky or unclear things.

I am not sure if there is any official Scrum guideline for this. Nevertheless, I have had the most success when refining the stories at most two sprints (four weeks in our case) before they are actually worked on. I highly recommend not to refine stories for which implementation is too far in the future because you run the risk of having to re-refine them.

In discussing the requirements, the development team tends to talk about how to implement the solution. Although, this is often documented I feel the development runs much smoother if the actual implementation is done fairly soon after, since everybody still has the topic in the back of their mind. Instead of having to remind themselves what the feature was all about the developers tend to be productive much sooner.

The difficulty with this is, that POs often have to deliver some sort of mid-term plan or roadmap which is hard to do if you have not refined and estimated the stories.

What has been working well in my current role is performing high level estimates upfront. Every quarter I am creating (actually all POs in our company do the same) epics with very high-level requirements that will be included in the next months of work.

We then have a short clarification of the requirements, options for implementation, and identification of major dependencies or impediments. The result is then a very rough estimate of the necessary effort. This is generally enough to plan the next months. Once we get close to the point of starting implementing, we then break it down to individual stories and go through the regular refinement process.

I am aware that refinement is a very Scrum specific topic, albeit one of great importance. There are ways to use the principles also for other product development work. I think everybody will agree that misunderstanding requirements is bad.

Thus, regardless of what framework you are using, I recommend to make requirements as explicit as possible, leave the manner of implementation to the people doing the work, and try to streamline the process of defining work packages as much as possible. I am convinced it will have great effects regardless if you are developing a B2B service with Scrum or a plane engine using waterfall.