What does velocity measure?

First, let’s be clear: by velocity, I mean story points delivered per iteration over time. Of course, that means we need more definitions, so let me get those out of the way now.

“A story point” is a unit that programmers invent solely to make it easier to estimate the work it takes to deliver a story. A story point is however big the programmers decide it is, no more, and no less. Story points are useful to the extent that they help programmers compare the effort required to deliver different stories, so that the team can decide how many stories to commit to delivering in a given iteration. Although story points necessarily fluctuate in value, one hopes that over a sufficiently short span of time – say the length of a release – the fluctuation is small enough that a constant approximation of velocity is helpful enough to plan the release.

“Delivered” means realizing (in the accounting sense) the value the customer intended to realize by asking the team to work on the story. A story is not delivered until it either reduces cost or generates revenue.

“Iteration” means equal-length time boxes the goal of which is to provide natural breaks in daily work to reconsider the plan and stop work from expanding indefinitely. On teams that have less trouble stopping when they’ve done enough, iterations are less important.

“Over time” refers to the trend of spot measurements over at least the length of a release.

I hope that makes it crystal clear what I mean by velocity. Now that I’ve clarified the meaning of velocity, what does it measure? It simply aggregates past estimates of effort of a set of stories the team selected to complete in the last several iterations. That doesn’t sound like much, does it? There’s more. Here are some things that velocity does not measure:

• productivity
• efficiency
• value
• suitability to re-hire or retain

...and, of course, past performance does not guarantee future results. Read your prospectus before investing. If velocity is so meaningless, then why measure it? Well, it turns out that:

• It is a good approximation (accurate about 2 times in 3, we think) of what the team can commit to in the next iteration
• It is a good approximation (we think) of what the team can commit to over the next handful of iterations (say 4-6)
• It is a good approximation of when the current stack of work we’ve identified will be done, as long as the stack of work isn’t too large (up to about 4-6 iterations’ worth)

I have just witnessed a conversation on the extremeprogramming Yahoo! group that illustrates the knots in which we tie ourselves when we allow ourselves to be too imprecise about what a measurement measures. The whole thing puts me in mind of personal financing and when budgets don’t work.

Think about budgeting your expenses at home: you know how much you actually pay for some expenses, like car payments, mortgage or rent, insurance… the amounts you owe are usually spread out over such a long period that you can anticipate what you’ll owe next month with certainty. Moreover, you know that it takes concerted effort on someone’s part, or human error, to change the amount you owe on a mortgage, car loan or your home insurance. Changes in those amounts, at least correct ones, generally don’t sneak up on you.

There are other expenses that vary from month to month: food, clothing, entertainment, communications… the amounts you owe usually depend on how much of a given resource you consume, such as how much you eat, how much you talk on the phone and how stressful your work is that month. You can budget these amounts fairly accurately, because the variation is low enough and mostly under your control: you can choose to eat less, take better care of your clothes, read more library books and rent fewer movies. Not only is it easier to guess what you’ll spend in a given month, but if you start spending too much, you can easily correct course to spend less.

There are still other expenses that you just can’t foresee. You are injured in an accident and rack up $20k in hospital bills; this is the year out of the last ten that you finally need new shingles on the roof; you find out your son needs braces. You might buy insurance as a way of smoothing out the costs over time, but they are generally big expenses that you know are coming in the larger sense, but which arrives without warning and needs to be paid immediately.

These are the expenses that make budgeting not work, because they prove that there’s no such thing as a typical month. This is also why you cannot rely on velocity to plan far ahead: there is no such thing as a typical project or even a typical iteration. If velocity doesn’t bring a project under control, then what might? For an answer, I invite you to consider personal finance again.

I read Your Money or Your Life to help me bring my personal finances under control and become financially free. One of the key exercises the book asked me to perform was to track all money coming in and out of our household for several months. This exercise is designed to call attention to our spending habits, so we can decide what to do about them. In particular, we identified those expenses we valued and those we didn’t value, so we could stop spending money on things we don’t value. This tactic, spending money only on things we value, doesn’t necessarily make one rich, but it ensures that one is not wasting money on things one doesn’t actually value. You’d be surprised how much money we were throwing away on things that didn’t ultimately make our lives any better, and you’d be amazed at the results when we stopped: we used that money as capital to generate passive income, and within five years, we’ve become financially free. But I digress. The point is what we measured, then how we analyzed it.

We measured actual spending, rather than estimated future spending or even estimated past spending. When we had the numbers, we didn’t start slashing expenses we cared about, and we didn’t start looking for cheaper alternatives to all our expenditures. Instead, we simply looked for those expenses we did not value, then cut them off. In some cases, this required investment of money: we bought a coffee grinder, an AeroPress, then started brewing coffee at home, rather than spending $2 per on coffee at our local coffee shop. In some cases, this required investment of time: we reviewed our work commitments and changed the way we work so that we wouldn’t be so tired so often, which meant we no longer “needed” to order dinner in as often as we did. We measured our actual spending, we looked at those numbers to identify waste, then we eliminated it. We did not waste effort staring at our budget, wondering how accurate it was, wondering how realistic it was, desperately trying to spread not enough money over too many expenses. We didn’t have to: we had actual expense numbers and could decide which items we valued and which we didn’t. More to the point, this exercise was far more instructive and effective than budgeting ever was.

Does your company value the areas where your software team spends its money? Do you know how to measure the value of what your team produces? How much money (time, energy, or actual cash) does your team spend on things no-one values? Could you pick three such wastes and eliminate them? What would you do with the recovered time, energy, or cash? How would you know that things got any better?

Forget velocity for the time being. Measure it and report it to whoever still wants it, but just between you and me, forget velocity and focus on these key questions. Try it for a few months. Tell me what happens, at ForgetVelocity atSign jbrains theDot ca. I’d love to hear from you.

Discuss

Jeff Patton is one seriously smart guy. I really need to pay much more attention to what he writes. While reading his older blog entries, I came across this nugget in an article attacking the myth of bad estimates sinking agile projects. Even entirely out of context, it’s great:

Treat estimates as solution budgets and focus on solving the original problem

Suppose the programmers estimate a story at 1 point, which happens to be around 3 days’ work on this project. Now suppose the team starts writing customer tests for this story and the story balloons into 2 weeks’ work. What should we do? Here are some options.

1. Let it be 2 weeks’ work, then re-plan and re-prioritize.
2. Work overtime in an attempt to squeeze 2 weeks’ work into 3 days.
3. Yell at each other over it.

Sadly, I see the teams take the last option more often than the others. They might yell subtly at each other, but they yell all the same. If the programmers have the upper hand in the relationship, they insist it’s 2 weeks’ work and they force the customer to re-prioritize and re-plan. If the customers have the upper hand in the relationship, they insist it be done in 3 days, and the programmers “knuckle down” to try to do it. In each of these cases, the result is not good.

Now there is another option I didn’t give earlier. Split the story into smaller pieces, the most urgent of which we hope is only 1 point. This is usually more effective, but I have already written about how difficult this is to do well. I wish I had read Jeff’s words before I wrote what I did, because Jeff’s advice provides a concrete goal for splitting this exploding story beyond “it’s a good thing to do¹”. Combining Jeff’s advice with mine, we arrive at something better.

Find the most urgent part of the story that costs only 1 point, then split the rest into less urgent parts that the customer can put back in the backlog.

This way, the customer gets the essence of what she wanted when she chose this story as the next one to build; but the programmers aren’t asked to do the impossible. It seems like a good outcome to me. Thanks, Jeff, for helping me justify my advice.

¹ I haven’t found much success justifying my advice to software people with “trust me; it’s good for you”. I’m constantly on the lookout for a better answer.

Discuss

Most user stories are too big—at least, this is how it tends to be for teams making the transition to incremental development and delivery. After discussing the benefits of splitting stories into manageable increments, a team generally goes through (at least) these stages of development:

1. splitting stories along process lines, then
2. splitting stories along architectural lines, then
3. splitting stories along procedural lines, then finally
4. splitting stories into smaller stories

What’s the difference? Allow me to explain with an example from my own work.

I am currently working on the xpday.info site, and one of the key themes is collecting registrations for XP Day North America. In particular, we are preparing to host XP Day Montreal 2006 on September 23, 2006. It’s obvious that “collect registrations” is more of a theme than a single story, so I split the theme into a coarse-grained stories like these—

• anyone can register by paying immediately with paypal.com
• anyone can register, then request an invoice
• anyone can register (and pay for) an entire group, rather than just themselves
• anyone can register and pay later
• automatically generate the invoice
• remind all unpaid registrants on demand
• automatically remind unpaid registrants on a regular basis
• a registrant can cancel his registration
• a registrant can add or remove people from the group

Now these might not seem coarse-grained, but when your iterations are on the order of a half-day each, it’s nice to have stories that I can complete in about an hour. (Yes, an hour.) Since I am also the Product Director on this project, and my primary goal is to be able to collect money from registrants, I identified the first story as the minimum for the first release, and estimated it at about 2 hours. That’s a little big for a 3-hour iteration, so I want to split it. The first thing I did was a little modeling, to make sure I knew what I meant by “anyone can register by paying immediately with paypal.com”.

Notice that, even here, I jumped to handling the case where the registrant did not pay. I didn’t realize that until I started writing this entry, so I suppose this is a kind of substitute for pairing. (Thank you.)

Stepping into the shoes of the teams that allow me to work with them a little, here is what I often see when we try to split the story anyone can register and pay immediately with paypal.com. First, the split along process lines:

1. design
2. code
3. write programmer tests
4. write acceptance tests
5. document

This is a fine checklist, but whether splitting a story into smaller ones or into tasks, this doesn’t work very well. None of these items produces value on their own. The closest, perhaps, is “write acceptance tests”, but even those don’t represent value until someone has tried to test-drive with them, because only then do we learn what we missed and what we mis-wrote. A team generally does this in their first planning session after deciding to “go agile”. It’s a good effort, but it does nothing to encourage them to change the way they work. It is certainly not incremental. When we apply the INVEST test, this split of stories mostly fails. While these “stories” are perhaps negotiable and somewhat estimatable (estimable? neither seems correct), they depend on one another, have little inherent value on their own, are likely not all small and are definitely not all testable. We need to try again.

Once the team gets past this troubling first phase, they tend to split across architectural lines:

1. test-drive the UI
2. test-drive the business logic
3. test-drive the database client
4. write acceptance tests
5. document

Now this is an interesting hybrid approach between architectural and process-based splitting. This looks more like a traditional work breakdown structure, which appears to allow for concurrent work, but doesn’t. (Can you see why?) It is tempting to say that this split satisfies the independent property of INVEST, but it really doesn’t, as you would find out as soon as you tried to integrate the work of the individuals who performed each task independently. There is still no value in just the UI or just the database client, the items are smaller, but probably not small enough, and as a result, I would not trust any estimates on them. We need to try again.

Once the team gets past this second phase, they tend to split across procedural lines:

1. collect registrant information
2. integrate paypal.com
3. e-mail registrant after payment
4. e-mail organizer after payment

This is much better, because at least we no longer see “write acceptance tests” and “document” as separate tasks. These are almost stories, as they are self-contained bundles of work that include “code, design, programmer tests, customer tests, document, deploy, install” and all those wonderful things. There is still one big problem: this split of “stories” fails the “V” property: just how valuable is collecting registrant information? We certainly can’t deliver that on its own. We need to try again.

After the team gets past this third phase, which tends to last a while, they finally get it, and split the stories into self-contained increments of value:

1. register with just e-mail, then pay with paypal.com
2. collect more information from registrant (name, address, phone)
3. notify both registrant and organizer after registration

That’s it! This story split satisfies all the INVEST properties—

Independent. It is relatively easy to see how to build each of these stories on its own. #1 is a one-field form whose submit action includes going to paypal.com; #2 is a multiple-field form with a simple “create” submit action; #3 is a one-field form whose submit action sends out an e-mail to two recipients. We can build these in any order.

Negotiable. We can negotiate the scope of each of these stories. For #1, is it just a link to paypal.com, or do we have to bring the user back to xpday.info? Do we have to handle an incomplete payment, or will humans verify that by looking at the list of registrants and the payers in their paypal.com account? For #2, how much information do we have to collect? For #3, do we send out a standard e-mail? Is it a form e-mail with some user-based personalization? Do we need a tool for writing or changing the e-mail template? For each story, we can easily negotiate economy or luxury versions.

Valuable. If we do #1 on its own, then at least we can take their money and we have enough information to contact them after they register. We can easily choose to do #2 or #3 in either order after that. Even better, we can do #2 on its own, if “the business” decides that a human requesting payment is good enough, but we really need an automated way to capture registrants’ information. We could even do #3 first: a person could take all the registration information (by phone or e-mail), then enter the registrant’s e-mail address in a form that sends out a form letter to her as well as the organizer. Any of these could have value, depending on the needs of the customer.

Estim(at)able. The first story is the biggest, because I don’t remember how to do the whole paypal.com thing, but I’m pretty sure I can put the bare bones together in 30 minutes. It might take another 30 minutes to handle a return link, and another 30 minutes for the “receiving page” to know whether payment was complete or not. I did this before, but about a year ago. For story #2, we can have something in 15 minutes, then it’s up to the product director to critique the UI layout and suggest changes. For story #3, we can have a standard e-mail in 15 minutes, it’ll take another 30 minutes for a form e-mail, and another 30 minutes to be able to change the e-mail template through the web UI. Make that 90 minutes, because we would have to add role-based security for the first time.

Small. Only one story is 90 minutes; the others are 30 minutes or less. For a 3-hour iteration, we can fit about 4-8 stories into an iteration. According to my good friend Jim Shore, that’s a good number of stories for an iteration, and I agree.

Testable. I would hate to see what someone like Michael Bolton would do with these little stories. I should ask him to test the site, just for laughs. (Well, I think I’d be crying by the end, but he would be laughing.) As a product director, though, I can easily tell when these stories are complete, so I can easily write acceptance tests for them.

So there you have it, I hope: a thorough example of splitting a story into smaller increments, each able to be built and delivered independently. If our plan starts to show that we’re running long, we can cut 2 of the 3 stories, but still meet our primary business objective: collect payments from registrants. That would be good enough for our first release. Now would be a good time to freshen my coffee (during which I have my stand-up meeting with myself), then get to work. We absolutely must have a working first release in the next 3 hours… or at least that’s what the product director claims.

Discuss