Stuff I'M Thinking About

Many development teams use story points as a way to indicate the relative size of pieces of work, and to get a sense of how much work (in points) can be achieved in a given timeframe. Numbers from the Fibonacci series are attached to each user story, with the total number of points completed being added up at the end of each sprint.

The purpose of this process is twofold.

The first and primary reason is to encourage a conversation about the work; is the work independent, valuable, and could (or should) it be broken into smaller stories? The acronym INVEST is often used to evaluate stories during this process.

The second is as a predictor of team’s ability to do a similar amount of work in future sprints. The average number of points completed in past sprints is used as a guide to ensure that a team does not commit to too much work in future sprints.

This measurement is probabilistic, and is intended only for use within the team. The numbers assigned by one team, and the total they complete in a sprint cannot be compared with any other team, even in the same company. The numbers have no meaning at all outside the team’s context.

This approach has some problems which can be summarised by our first axiom:

For a given outcome, the time taken for a team to complete the required work to deliver this will always vary, both by itself, and as a function of the knowledge needed for the work. This includes any knowledge yet to be acquired. As the amount of undetermined knowledge increases, as indicated by increases in the story point value, the uncertainty associated with that estimate increases at the approximate rate of the square of the Fibonacci number.

Fibonacci	1	2	3	5	8	13	21	34	55
Uncertainty	1	4	9	25	64	169	441	1156	3025

Stated simply, the bigger the Fibonacci number, the more uncertainty there is, the greater the expected variation in the time taken to complete the task, relative to smaller numbers. This variation increases exponentially rather than being grounded in simple addition. Also, as the numbers get bigger the amount of uncertainty will quickly exceed what is known, both for individual stories, and collectively for the sprint. When the total amount of uncertainty in a given sprint is high, this will have an impact on all work in that sprint.

In a nutshell: higher number estimates are exponentially less reliable.

This effect is completely independent of the amount of existing documentation, or the strength of the team’s belief in what still needs to be discovered to complete tasks in the sprint.

This means that a sprint containing four 5s is far less likely to completed as expected, than one with ten 2s, even though the number of points is the same. This is well and good if these numbers are used only by the team, and they understand the limitations of the approach.

But the use of numbers tends to attract the attention of others who may think that the use of a number suggests that something real and absolute is being measured, and that it indicates certainty. They postulate, wrongly, that use of this number should be extended into organisational-level planning, because we need more certainty.

If these numbers are used outside the team for the purposes of planning and providing certainty, the team will adjust the assignment process to serve this new purpose—meeting the expectations of people outside the team.

One of these expectations is that work will be completed in the time agreed. As soon as that happens, the team will allocate story points to work items such that they believe that the work can be done in the time expected, allowing for the uncertainty in axiom one, plus additional time ‘just in case’. And in no time at all we are linking an arbitrary numbering system designed as an internal team aid, to time.

This new state does not serve the team’s needs (or the company’s) in any way, and provides us with our second axiom:

When story points are used outside a team’s context, the values ascribed to work by the team will increase to ensure that there is no uncertainty, and that the expectations of those outside the team are always met.

The team is not gaming the system here, as this might suggest. The system is gaming the team.

Anyone who has been in a team that has worked under these conditions will tell you the result: delivery of outcomes will slow down, software quality will suffer, and the number of bugs will go up.

When an organisation has been doing this long enough, these symptoms quickly become part of the wider system, and it becomes impossible for the team to eliminate them. It is assumed to be the way things are, and that the only way to improve outcomes being better adherence to planning (and other) processes. This is the antithesis of agile ways of working.

All tools have an optimal use, but when that usage is expanded beyond their design parameters they quickly become useless, or negative in value. Which leads us to the Story Point Uncertainty Principle:

When story points are used in an attempt to eliminate uncertainty at organisational scale, a non-agile system with considerable built-in waste quickly arises.

When used in a team, story points can help them manage their work more effectively, and to deliver value more consistently. That is where they should stay.