Story Points. A Friend Or Foe?

Product backlog items' sizing became a routine for numerous agile teams. It became a new normal way-way back, and these days, it’s hard to find a developer, Product Owner, or Scrum Master who never heard about the Story Point concept.

Internet is full of explanations, guidelines, step-by-step instructions, videos on how to do proper estimation using Story Points. This topic shows up in many SM/PO certifications. So, for me, it sounded like a done deal, an axiom.

Recently, several occasions triggered me to rethink the position of the Story Point concept.

First of all, numerous customers start using Agile Poker because of the Relative session. According to the app users, it’s ideal for teams that struggle with understanding Story Points because, in Relative session, users estimate without thinking of the estimation value. Some teams decide to switch to Poker sizing after several sprints. Some others like Relative so much that they choose to stay with Relative as the main estimation technique.

Secondly, I always wondered why every Story Points topic gets so much attention and so many comments, whether it’s Mike Cohn’s blog post, a youtube video, or a conference speech.

Lastly, an eye-opening one for me: it’s my team. Imagine my surprise when developers from the Agile Poker team, the ones who create this estimation software for thousands of people, showing them how it’s done correctly, having stable velocity for a long time, think of man-days while estimating in story points. Period.

All the above factors inspired me to write up this blog post to structure my knowledge and, hopefully, help others and their teams with this not-so-unambiguous topic.


The Story of Story Points

I believe to understand the whole concept of Story Points, it’s crucial to know why it appeared in the first place.

Prediction is very difficult, especially if it’s about the future! – Niels Bohr, the Nobel laureate in Physics and father of the atomic model.

In other words, we suck at estimating.

But on the other hand, what should you do as a horrible estimator when the Scrum guide (till 2020 release) says that estimates should be provided by people that will be doing the work, but at the same time, it doesn’t tell us how we should provide estimates? And what did Agile enthusiasts do after reading the scrum guide back in 2008 when they could not relate to any step-by-step practical instructions?

Yes, they started estimating Product Backlog Items (PBIs) in time, and yes, they did wrong. Mostly because of:

  1. Again, we suck at estimating.
  2. Estimates are treated as a commitment by the stakeholders.
  3. The outcome from the first two: estimations took a while or were overestimated.

The story point is an abstraction that solved these problems. It had no units, so it can’t be related to any commitment. If you asked the right questions (for example ‘How is this compared to that thing we did last time?') it could save a significant amount of time on the estimation process since you just compare things one to another. And if you fail with the estimation, it’s fine as long as it’s consistent. Here is why. If you say that a particular user story is three times more complicated than the reference one, but in reality, it’s only two times ‘bigger’, it’s still OK if you estimate all similar stories that way. Just keep it consistent.

Where did story points come from?

Story Points concept was the Extreme Programming (XP) idea, which was adopted by Agile practitioners. As Ron Jeffries describes, the original “Points” for estimating ‘Story’ were based on ‘Ideal Days’ to finish a specific PBI that might have been translated to the real days by multiplying by the "load factor" to convert it to the actual implementation time.

For example, a story estimated as three ‘Ideal Days’ multiplied by the load factor of three gave nine days estimation.

But the stakeholders were confused with three ideal days estimation versus nine non-ideal days, so Ron’s team started calling ideal days ‘Points’ or 'Story Points.

And that’s how the Story Point concept appeared - time estimation, that evolved into something much more prominent these days.


What is a Story Point?

A story point is an abstract measure of effort required to implement a user story. In simple terms, it is a number that tells the team about the difficulty level of the story. The difficulty could be related to complexity, risks, and efforts involved. Let’s briefly analyse each measure.

Effort or volume. How much is there to do?

  • What amount of work is there to do?
  • Is it big? Is it small? Is it average?
  • How much effort will it take to complete the job?

Complexity. How difficult is the work?

  • How complex is the work compared to other stories of the same volume?
  • Is it easy or hard to do?
  • What are the dependencies and circumstances?
  • Are there any uncommon practices involved?

Risks. What are the unknowns?

  • What do we know that we don’t know?
  • What is our experience with similar stories?
  • What are we not certain of?
  • Does it challenge our skills?

Story point estimation is typically performed at the Product Backlog Refinement Sessions. Product Backlog is evaluated by the team responsible for the actual development and testing work.


Benefits of using Story Points

Jeff Sutherland, in his article, mentioned that ‘Story Points are therefore faster, better, and cheaper than hours and the highest performing teams completely abandon any hourly estimation as they view it as waste that just slows them down.’

Let’s review all the benefits Story Points bring.

Estimating in Story Points is fast. Teams who estimate in days tend to take the discussion a few levels deep. If you estimate in days, the fear of commitment plays a role here.
As said by Jeff Sutherland, one of the CMMI level 5 company determined that story point estimation cuts estimation time by 80%, allowing teams to do other productive project activities.

It’s Relative. Mike Cohn best described it in Agile Estimating and Planning ‘Because the estimate for each feature is made relative to the estimates for other features, it does not matter if our estimates are correct, a little incorrect, or a lot incorrect. What matters is that they are consistent.' And Indeed, 'there is credible evidence that humans are good in relative estimation compared to absolute’. (Lederer and Prasad 1998).

Story Points remain the same for everyone. It doesn’t matter if a Senior or Junior developer will deliver the story. What matters is that the story is X times bigger than the reference one.

Story Points allays fear of commitment. When estimating using hours, you make a precise time commitment. Estimating in Story Points prevents giving an exact commitment. Nobody knows precisely how many hours you are appointing to a specific Item.

Eliminates the need for frequent re-estimation. When planning with Story Points and velocity, we essentially take those points as size-based estimates. As mentioned before, we don't expect the size of the story to tell us the time needed to complete it. If estimates are in days, the team finds itself in the position of needing to frequently re-estimate future stories based on current knowledge. When teams talk about “days of work” implies a level of specificity that isn’t real. We avoid this problem when using Story Points. The fact that a team learned to work twice as fast gets reflected in their velocity. As long as the story sizes are consistent (read the previous points) between stories, no re-estimation is necessary for planning to occur.

Story Points help drive cross-functional behaviour. Mike Cohn explains this very well in his book ‘Agile Estimation and Planning.'. Agile teams include people from different disciplines like programmers, analysts, testers, designers, or product owners. Agile Projects will benefit when each team member thinks about a built feature first and then being a specialist contributor. As such estimating in story points will help teams learn to work cross-functionally.

Since Story Point is a single number, all the teammates need to think about what they estimate from a big picture and as a complete feature. They need to shun their departmental mindset of how much programmer or tester or UI person would take and try to add the estimates. But at the end of the day, the scores are made up, and they’re numbers. It’s the conversation and collaboration during the session that really matters, brings value and helps to gain a common understanding of stories. The team behaviour becomes more prominent over individuals as conversations help team building, team members constructively criticize each other, and debate.


Weaknesses


I like to say that I may have invented story points, and if I did, I’m sorry now. – Ron Jeffries, one of the Extreme Programming (XP) founders and original Agile Manifesto signatories.

Story Points are not intuitive. My job provides me with the possibility to communicate with lots of Agile Poker users, Scrum Masters, Product Owners, and Tech leads. I regularly hear that they very often don’t understand the Story Points concept. Not only the person representing the concept has to repeat basic principles several times, but also the practice shows that the starting 3-5-10-… Story Point sizing will be full of questions and arguments:

Should we go for another round?

  1. Why don’t we just take the bigger number?
  2. I will do this in 3 SP, why are we putting 5 SP?
  3. Why can’t we use the average of 3 and 5 SP?
  4. What is the difference between 1, 2, and 3 SP?
  5. What is the lowest SP value, 0.5 or 1?

Moreover, have you ever seen somebody saying ‘it feels more like a 3 to me’ while all other members estimate as 2? I experienced such deadends in reaching a consensus. Eventually, it slows down the process and neglects the ‘Fast' benefit. While, for example, the T-Shirt size scale is much easier in adoption. As Linda Cook mentioned: 'Scrum Masters often guide the team to the desired answer rather than let the team arrive at their answer based on conversation and understanding of the work. Even worse, I have witnessed several Scrum Masters declaring the story point value once the story is read.’

Not truly relative / Translating Story Points to hours. Do you remember the story of Story Points? It evolved from time estimates, and even today, it is still mixed with time estimates. Teams keep converting time to Story Points and back since it’s natural and easy to understand. But why is that? Imagine a team estimating numerous PBIs, one by one. The group is supposed to compare each item to a baseline item (typically a ‘1’). It’s exactly where the main problem lies. The participants have to figure out how many times the item is bigger. The human brain tries to simplify the task by creating an image of each size on the scale and unconsciously translates it to time intervals (e.g. 2–3 days for size ‘2’).

The whole Story Point concept turns into absolute estimations without you noticing. It’s funny how time estimates evolved in relative Story Points to come back to the absolute estimates. However, in this case, you stop benefiting from the speed of relative estimation, so most of the benefits from the previous chapter don’t matter anymore.

Adjusting Story Point estimate with a specific developer to work on it in mind. Although this is one of the SP benefits, we see that our clients keep doing the following: a PBI might be 3 Story Points for a senior developer, but 8 Story Points for a junior developer, so the developer is selected first and then the SP value is set. It’s so easy to go with 3 SP in this case (and numerous Agile Poker clients do so!), but it’s wrong since SP estimate is a team commitment. What will happen if the junior dev will have to pick up the story?

Sizing unfinished issues again. The Story Points concept doesn’t provide a solution of what to do with stories that are moved to the next Sprint. Many teams re-estimate the story and put remaining estimates, while it is unnecessary to re-estimate at all. Think of the reference issue that has been re-estimated. This will endanger future Plannings. As a result of Sprint Planning, the team will know all necessary tasks to complete the issue. The estimation of these tasks is in hours. So in the next Sprint, the team will know how much time is still necessary to complete the PBI. The fact that the PBI was not completed will reflect the velocity.

Avoid adjusting reference PBIs. As an outcome of the previous point, revised reference PBIs lead to re-evaluating the team’s Story Point values and could seriously damage planning.

Summing up Story Points is artificial. Summing up the relative value of Story Points is an arguable process. Eight 1-point tasks take much less time than one 8-point task! Point totals across sprints are not comparable because of this math problem. While most of us try to forget this problem silently, some frameworks went even further. The exceptionally popular Weighted Shortest Job First (WSJF) prioritization framework suggests calculating the score based on 4 Story Points criteria. You sum and divide relative values to get another relative value for comparison purposes. It frightens me a lot. But still, it works.

Story Points vs Bugs and Tasks

From guides, you know that only User Stories should be sized. From real life, you know that teams estimate all possible issue types. Full stop.

My life is really complex since I've started using Story Points.




Should I use Story Points?

My short answer is YES! 2 main points to support this:

  1. Story Points concept has shown durability. Despite the tons of critics and numerous haters from the moment of introduction, the concept lives and prospers.
  2. There is no better alternative for Backlog sizing.

The right question to ask is ‘How to use Story Points’?

With the latest 2020 updates of the Scrum Guide, my understanding is that the industry becomes adjustable, and the community is fine with non-strict rules. So even if you make every possible mistake pointed out in the previous section, but in the end, it works for you, that’s fine. However, as a professional, you must understand all the consequences of not using the framework by the book and make a conscious decision to go this way.


How to make it better?

I think that at some point in time, every Agile team faces the need for estimation, and the vast majority is or will be using Story Points, so I gathered some recommendations from Agile Poker users on how they address estimation challenges:

Start estimating wisely, with Relative estimation types. As mentioned before, Story Points are often translated into time. Usually, it starts from the very first estimation, where estimators have to somehow think of some magic items to estimate effort. My suggestion: avoid Story Points completely at the beginning. Use the Relative estimation method of Agile Poker instead, where your team can group user stories into columns according to their relative weight and only then assign the story points values.
This approach has been confirmed by our numerous clients, who, after several Relative sessions, switched to Planning Poker, Bucket Sizing, or stayed with Relative.

Know the basics and adjust processes accordingly. There must be a person who knows the basics of Story Points estimation and can present it to the team. If you have read this blog post until this point, you might be the one. But more importantly, this person must understand all the constraints of modifying the by-the-book process, which happens to most teams.

Prevent estimate inflation. In Agile Poker, we implemented two features to help with keeping estimations consistent:

  • Reference issues are constant stories with different Story Points values used as a baseline for estimates.
  • Triangulation issues constantly change and display lately finished stories from the same project with the same labels and components that give users more context of a selected Story Point value.

Triangulation Issues

More theory on this topic might be found here.

Try different estimation methods. There are many different estimation methods for Story Points estimates, but Planning Poker is considered the default one. It might be limiting for some teams. For example, many of our clients use Bucket Sizing with a voting option as a faster alternative. My team and I use Async Poker, which enables estimators to go through the list of stories on their own, get into details if needed, and have time to explain the choice in the comment section.

Discuss incorrectly Story-Pointed issues in retrospective. Every now and then, the team Story Points an issue where it is clear that the estimate was entirely off. It is important to discuss these issues and try to learn, so future estimations are more accurate. Once my team stopped creating test tickets, considering it as a part of stories, the reference issues were still the same. Retrospective helped to figure this out and improve our planning.

Adopt. Cliche, but still, no one knows your team better than you. If you see problems or difficulties with Story Points estimation that is not answered in the books and theory, keep adjusting them until you solve those problems.


Conclusions

The Story Points concept is simple but difficult to apply. It’s well described but used differently. It’s coming out of time estimates but isn’t connected with the time estimates.

I hope this article will help you with sorting out the concept and help with a quick, reliable and pleasant planning process.