Wednesday, July 2, 2008

Spillner - Big Projects, Small Increments

by Kent Spillner - 2 July 2008

Businesses change too rapidly for monolithic IT deliveries to succeed, but large problems requiring large solutions still exist. Breaking solutions down into a series of small, frequent releases is the only way projects can adapt to succeed.

Increment or Die

There isn't time to deliver any other way. Delivery risks compound over time: business changes, technology changes, people come and go. Every day that passes between deliveries doesn't bring the team closer to the next release, it increases their odds of failure!

There isn't time to be cautious, either! How many projects survive the evolution of the business? No matter how hard team members work or how detailed the release plan, time works against us. The only survival option is to deliver early, deliver often.

Even when a project seems fixed and well-specified, teams can't expect to get things right. Consider NASA's recent Phoenix Mars Lander spacecraft. NASA is delivering new releases to the spacecraft on Mars daily! When the craft scoops up some clumpy dirt that's too big to filter through a screen, they beam up new code to wiggle the lander and shake the dirt loose. Frequent, regular delivery allows NASA to adapt without wasting time or missing opportunities.

Characteristics of Good Increments

Successful incremental delivery is the same on every project. Good increments are potentially releasable software delivered into a production environment (or replica) as small changes, at high frequency, and on a regular schedule. This definition is atomic. Failure to do any of these things is not just incrementing badly, it's putting the project at risk.

Increments must be potentially releasable. Technical debt accumulates over time and hobbles productivity. Teams can't afford to trade quality for convenience. All those quick fixes crammed into the codebase today will come back and take a bite out of productivity tomorrow. It's important to set the quality bar as high as possible as early as possible, and keep the team focused on delivering professional-grade software. Delivering potentially releasable increments also enables ultimate business agility: shipping at a moment's notice in response to changing business conditions!

Increments must be delivered into a production environment (or exact replica of one). Until changes are running in production, teams can't know what they've built. The release process also needs to be tested as much as the released software. Do whatever's necessary to get your team delivering into a production environment, and do it now!

Increments must be small. Really small. As small as one change at a time. This means that incremental delivery and continuous integration go hand-in-hand. Continuous integration is the process of automatically running the project's full end-to-end build cycle -- including all automated unit, integration, and functional tests -- while the team is working. Continuous integration gives the team frequent feedback on the health of the project as a whole, and reduces maintenance costs by finding bugs immediately when they are introduced. Running the full end-to-end quality assurance process for every change made to the product is a critical component of success. There's no such thing as too small!

Increments must be delivered with high frequency, daily at a minimum, more frequently if possible. There's no such thing as too short! Every feature that isn’t delivered to QA or production just piles onto the next. It isn’t helpful to have developers move on to new tasks before getting feedback on the previous one, or to load more work onto quality analysts than they can handle.

Increments must be delivered regularly. It's crucial to establish and sustain a rhythm for a project. That helps people in every role get into their flow, and makes it easier to spot when things go bad. Slipping schedules can be a great early warning system, but only after establishing a history of making very frequent deliveries.

Better than monoliths

Monolithic delivery is just too risky. The time between monolithic releases breeds opportunity for the competition, and creates space for business interests to diverge from the goals to which a project team is working.

Incremental delivery is an antidote to these problems because it reduces the feedback cycle to the point that teams are able to adapt instantly to changing business realities.

At RailsConf 2008, a speaker announced an open source cloud computing platform called Vertebra, which effectively killed a commercial product being developed by a company in attendance. They were unable to respond with their own product announcement since it wasn't the right time in their release cycle.

Or consider another example of a project that was written off after it was development complete. During development, the infrastructure group migrated their production environment to a different technology platform. The development team wasn't aware of the change until the final QA phase: that is, after thinking they were "done," but before going into production. If the issue was detected earlier, it would have been affordable to migrate platforms, instead of writing the whole thing off as an embarrassing loss.


Unfortunately, incremental delivery is easy to get wrong. Avoid the following anti-patterns:

  • Partial delivery: not delivering into production. Unless delivering into production, "done" is more an interpretation, less a fact. If every increment can't be delivered into production, they should be delivered into a cloned replica of production. If increments aren't even delivered into a cloned replica of production, then the team isn't delivering!

  • Infrequent delivery: more than one day between deliveries. There's no technical reason teams can't deliver on every commit, but at the very least they should be delivering daily. Otherwise, the team is asserting their work is done based on assumption. Finish the job!

  • Chunked delivery: too many changes since last release. Chunking development and QA work is a false-economy of scale. The cost of rework is amplified by latency in feedback; this completely destroys any economies of scale of chunking. Deltas need to be small enough so QA can test the entire delta before the next delivery. Feedback cycles need to be short enough so developers don't move on to the next task before their previous one is truly finished.

Embrace Agility

There is zero return to the business until software is delivered into production. Every decision on a project is subordinate to this fact, and no amount of hard work, experience or planning can change that. Incremental delivery makes teams more successful by enabling them to deliver a higher quality product sooner, roll-out updates more frequently, and respond immediately to changing business conditions. Stop hitting yourself: embrace agility, deliver incrementally.

About Kent Spillner: Kent loves writing code and delivering working software. He hates cubicles, meetings and Monday mornings.

No comments:

Post a Comment