We have grown a lot in recent years and today we are more than 100 employees in Product distributed in 20 teams, which work independently, that is, everyone has the autonomy to deal with their problems in the way they deem most appropriate.
The delivery capacity of each team is finite and, in order to choose what to do first, it is necessary to create a problem management with well-defined classification and prioritization criteria. There are many different data sources and many cases that we need to understand, categorize and prioritize.
That’s why we’ve studied prioritization methods used in the market, we’ve done pilot projects, and we’d like to share our experiences to help you prioritize issues in a complex scenario!
We identified the categories as the starting point to better understand the problem backlog . Whether these are internal – issues related to code quality and maintainability – or external – issues that are already affecting our customers.
For this, we classify our product defects into 2 major categories:
- Technical debt – referring to internal quality;
- Bug – referring to external quality.
We consider a technical debt to be an architecture or infrastructure solution that does not meet current demands or that carries a risk of not meeting the increased usage of the software in the near future. These demands can be either of scale or difficulty in maintaining and expanding functionality.
It is common and expected for code to come up with a good design. However, after numerous changes by different people, this code starts to acquire complexity and becomes a debt for future changes.
We classify as a bug anything that directly affects the end user. It consists of an unexpected behavior of the application, something that shouldn’t be happening. It occurs when the application has been programmed to work in one way and, for some reason, it behaves differently, whether it is the interruption of a flow, an unexpected result or an inappropriate use.
Now that we are aligned on how we categorize issues here at Digital Results, we will move on to prioritization.
Prioritizing means doing the things that generate the most value for our customers first. But how do we know what’s most important to them?
For some time we used a methodology developed in the RD itself, based on criteria of severity and frequency to define the priority of problems. However, we were aware that this “homemade” methodology no longer reflected the reality of the teams.
In order to find a problem prioritization method that better suits our current context, we studied two of the most common on the market: RICE and GUT.
RICE is an acronym in English for the following terms that will be evaluated:
- Reach ( Reach ): number of customers that task will reach.
- Impact ( Impact ): number of customers that will be impacted.
- 3 – massive impact
- 2 – big impact
- 1 – medium impact
- 0.5 – small impact
- 0.25 – minimal impact
- Confidence ( Confidence ) : confidence level of the estimates.
- 100% – high estimate
- 80% – average estimate
- 50% – low estimate
- Effort ( Effort ): amount of time required for the task to be completed.
To calculate RICE just multiply the first 3 items and divide by the last:
RICE = (Reach x Impact x Confidence) / Effort
The idea is to classify our bugs and technical debts taking into account the severity, urgency and trend of the problem in order to define which we should attack first.
To facilitate prioritization we created a template with automated GUT calculation to help product teams prioritize our issues.
- Initially we should list our backlog of bugs and technical debts in a basic table;
- Score each bug and technical debt according to the Severity, Urgency and Trend criteria, following the information below:
- Severity: consequences that problems can cause if we don’t act on them.
- No seriousness – 1: it causes discomfort in the operation, generates frustration in the experience.
- Mild – 2: prevents completion of an operation stream. It is only possible to complete by retry or workaround.
- Severe – 3: interrupts a use case. It is not possible to work around or remedy the situation without company intervention.
- Very serious – 4: it causes reversible damage to the client’s operation, such as unavailability of data/operations.
- Extremely serious – 5: it causes irreversible damage to the customer’s operation, such as loss of information or opportunity.
- Urgency: estimated time in which we can resolve problems.
- Can wait – 1: more than 6 months – can be done next quarter.
- Not very urgent – 2: more than 3 months – we have to solve this quarter.
- Urgent, it deserves attention in the short term – 3: more than 1 month – we have to solve this month.
- Very urgent – 4: up to 1 month – we have to solve this week.
- Need for immediate action – 5: this week – we have to settle for yesterday.
- Trend: refers to what will happen to problems if they are not resolved.
- Alleviates – 0.5: Actions that have already been or are being performed tend to alleviate the problem over time.
- Will not change – 1: Remains stable.
- Worse in the medium term – 2: Increases linearly with an increasing number of customers.
- Worse in the short term – 3: Increases exponentially as the number of customers or engagement promoted in a feature increases.
- Get worse quickly – 5: Increases exponentially with projected usage and the inclusion of new conditions in product usage.
- Severity: consequences that problems can cause if we don’t act on them.
- The template spreadsheet is already automated with the GUT calculation, which occurs by multiplying the criteria:
GUT = Severity x Urgency x Trend
The result with the highest score is 125 points and the lowest is 0.5. After prioritizing all problems, sort the list from the largest GUT to the smallest.
Looking for the best solution
Initially, we conducted a survey with all Product Managers and Tech-Leaders of the product teams through a form containing questions such as:
Does your team use the criticality of bugs and technical debts in task prioritization?
What form of prioritization is used?
Is this form serving your team well?
The 16 teams responded to the survey, all of which used our “home-made” prioritization method and 5 of them also started using other methodologies such as GUT and RICE.
Of the 16 teams, only 4 of them were satisfied. Soon, 75% of the teams were dissatisfied. However, teams that used other methodologies such as GUT and RICE were 100% satisfied.
After the survey, the teams’ dissatisfaction with the current prioritization method became even more evident. What all teams want is: a way to verify the impact of bugs and technical debts on the product, measure how many customers will be affected, and measure the effort. Therefore, we came to the conclusion that the best alternative at the moment would be to carry out a pilot with RICE.
Pilot with RICE
A pilot was carried out in 5 product teams, whose objective was to prioritize all bugs and technical debts with RICE. When performing the prioritization, we are faced with the following problems:
- The RICE values were left with an extremely high variation, being able to exceed 500,000 (or more);
- A high effort is required to achieve the Reach of each feature;
- RICE will be invalid in a short time, as the number of clients/accounts grows very fast, making it impossible to survey the reach for all clients/functionalities.
Pilot with GUT
We carried out another pilot in 5 product teams, whose objective was to prioritize all bugs and technical debts with the GUT. After running the pilot, we were able to obtain the following benefits:
- The maximum value of the GUT is 125 and this value will not change as the customer base grows;
- By contemplating bugs and technical debts, we were able to concentrate all problems in the software in a single prioritized list;
- It helps to ensure that efforts are first invested in the most critical issues that, consequently, generate the most value for customers;
- It is scalable, as it is a simple methodology to apply.
- After the execution of the pilots, we came to the conclusion that the most viable methodology for classifying bugs and technical debts is GUT and we started to apply it to all product teams.
- In the universe of software development, we live in constant change and we need to be able to meet customer demands in an agile way. With the sorted and prioritized backlog, you can resolve issues faster and continuously improve the product. This is a costly process, but in the long run it guarantees the stability and reliability of the project.
- Regarding the choice of the best prioritization method, the pilot projects also helped to streamline the process and choose the methodology more assertively. Conducting tests with a reduced sample, we had less impact on the teams’ work, as not everyone needed to change their daily procedures while we were experimenting.
- To take these lessons to your day-to-day, our tip is: get to know your problems better and start pulling corrections to the team’s development cycle. People’s time is a very valuable resource for the company, so help ensure it is being spent optimally and in the right activities!