by Kevin E. Breidenbach - 3 September 2008
What does the story of David and Goliath have to do with a technology journal? That will become obvious in short order, so bear with me.
I work in capital markets technology, specifically trading. Many years ago, a team I was on moved from a “waterfall” methodology to Agile. Waterfall is in quotes because, like any team that claims to follow waterfall, they really had no methodology. People will try to pass off a requirements document and a few design documents as “waterfall” despite the fact that the finished software (if there is finished software) rarely looks like the documents that were used to define it.
Anyway, without getting into the history of why that the team chose Agile, suffice it to say it was a success. So much so that it led to the adoption of Agile company-wide, with this team the reference implementation for how it should be done.
Large organization find it hard to change the way they behave. They employ a lot of people, and making sure those people are all moving in the same direction is very difficult. In other words, large organizations are not very agile. Case-in-point: consider that despite demonstrable benefits, a successful and high-profile implementation and a mass roll-out effort, Agile has still not made it to every development team after all these years. That is decidedly un-agile.
Which leads us to David and Goliath. You would have thought that by now big business would have learned at least something from that story. It’s not just a story from the Christian Bible, it’s also in the Torah; so it’s not really clear why it’s taking business so long to get it. The military did. They started using small agile teams with light but technically advanced weaponry to great success. So if they get it, why can’t big business?
Now, not every big business needs to have agile software development teams. Many firms' computer systems do not need to change for many years, so waterfall works well for them. But financial trading changes daily. Whether it’s new regulation, new products, new algorithms, the trading systems are constantly in need of update.
So while it’s good that trading teams are starting to become agile, there’s more to the problem than just using Agile as a development methodology: the organization needs to think agile. Development teams only produce software, they don’t put it in production. It needs to be deployed onto the infrastructure and administered. The larger the organization, the larger the infrastructure that needs to be maintained. Making changes to infrastructure often becomes an obstacle for development teams to get things done. There are a number of reasons for this.
Infrastructure teams are rarely exposed to Agile as a methodology. This is probably due to the fact that it has, until now, been directed mainly towards development teams. Infrastructure is often subject to rigid methodologies like CMM. Mixing CMM (or any document driven methodology) with Agile reminds me of the movie Die Hard 3, where Bruce Willis and Samuel L. Jackson watched a green and red liquid mix together just before the concoction blew half a New York block away. It isn’t going to work.
The Agile development team has consistent, time-bound iterations of, say, one or two weeks, while the infrastructure team has service level agreements that invariably are longer than one iteration length. This puts a spanner in the works. It means that the development team has to start thinking long in advance. And what if new requirements come along that change the infrastructure requirements of the application? This could mean the infrastructure team will need to go back to the requirements phase of their process, resetting the SLA clock (don’t think this doesn’t happen!) and delaying development.
This isn’t all bad for the development team, though. Because Agile is so transparent, the business will have been involved all along and will see how infrastructure impacts delivery. The result could be (and I pray it is) that the business may actually convince the infrastructure teams that they aren’t really deities, obliging them to start acting like the service they were intended to be. This doesn’t diminish the value of good infrastructure people. It simply means that infrastructure teams need to realize that some businesses, and thereby development teams, need to operate in a different fashion to others and that they should be able to accommodate this without impacting timelines or quality!
The infrastructure team is also afraid of another word: complexity. The more complex the infrastructure the more difficult and expensive it is to maintain, fear of the “c” word is understandable. Unfortunately, senior management becomes afraid of complexity and produces draconian rules that restrict development teams in their work. Ironically, the pursuit of controlling complexity can render your organization incapable of being agile.
For example, it’s fair to say that it’s good policy to keep the number of enterprise-wide tools to a minimum. For instance why support both Microsoft Word and Corel WordPerfect? The same can also be said for operating systems (why have Linux, AIX, Solaris, HP-UX and Windows) and even server hardware (one manufacturer).
But – and it's a big but – once the task of reducing complexity starts it takes on a life of its own and infiltrates every area of the technology organization. There are edicts that developers shall only use one type of IDE, or that certain open-source libraries or software languages are no longer allowed because we have other libraries or languages that do something almost similar and with some workarounds can do what you need.
This actually increases complexity of the software being produced, and reduces agility of the development team. This is because the developers now have to build those workarounds – they have to fit a square peg into a round hole in order to force some functionality out – when all they really needed was to use a different development language or different library.
Organizations justify this over-zealous complexity policy by saying that it adds cost to support these new libraries, when the fact is that no other support is needed as the libraries are embedded in an archive, or the language produces a simple executable, or the developer supports their own IDEs and tools. Indeed, the cost of the development teams building the workarounds often far exceeds any costs of supporting the tools that they intended to use. Not only that, but the workarounds may actually impact application performance costing the business money during trading!
Which brings us back to the title. David was smaller and more agile than the large and bulky Goliath, who relied on his size and strength. Not only that, but David realized, even if subconsciously, that standardizing on equipment doesn’t work when faced with an enemy like Goliath. Had he fought with just a spear or sword against Goliath then he may not have emerged victoriously and the Bible and Torah would be short of a good story! By picking up some new tools in the form of rocks and a slingshot he outsmarted the clear favorite and started the trend of people cheering for the underdog. So increasing the complexity actually had a positive affect on the task at hand!
The military’s small Special Forces teams recognize this too. They have different equipment to the regular large force, their own logistics and even command structure. It works.
Another group that has recognized this are companies that compete against large financial trading firms. Despite the economic climate, small to medium trading firms have been hiring furiously over the last 18 months while the large banks have hiring freeze after hiring freeze and lay offs on top of layoffs.
Bigger is better? I’ll let you decide. All I know is that I’d rather be is David’s shoes than Goliath’s!
About Kevin Breidenbach: Kevin has a BSc in computer science and over 15 years of development experience. He has worked primarily in finance but has taken a few brief expeditions into .com and product development. Having professionally worked with assembly languages, C++, Java and .Net he's now concentrating on dynamic languages such as Ruby and functional languages like Erlang and F#. His agile experience began about 4 years ago. Since that time, he has a serious allergic reaction to waterfall and CMM.
No comments:
Post a Comment