Saturday, November 24, 2007

The Seduction of Standardization

One of the driving tenents of Lean is that standardization significantly drives down variation. A reduction in variation means less wasted movement and a reduction in cost.

From a Lean perspective, standardization of process is more important then standardization of tools. Lean practitioners know that you must use the right tool for the job and not necessarily the same tool for the job. What you want is consistency in the process of how a job is executed.

One of the most common mistakes I have seen in IT organizations is the blind insistence on the adoption of a single technology platform. The idea is that by forcing everyone to use the same development language, framework, application server and technology platform an organization can significantly reduce their overall support and training costs.

This is a naive outlook to have because it basically says that every system you develop can be solved in the exact same manner. Every technology has specific constraints and costs associated with them. How you solve a problem is in many ways going to be dictated by the constrains of the technology being used.

Let me give you an example of this phenomena. J2EE is an enterprise stack that allows you to build high-volume applications that can manage transactionality across multiple data sources. Many organizations adopt J2EE as their standard and then in turn force everyone to build their applications on this stack. (Does the refrain: "Its the corporate standard" sound familiar?) Most applications in an organization do not have the complexity and overhead required to mandate the use of a J2EE stack. Over the last several years I have seen many organizations where even small applications that are read-only, with a low volume of activity and a single database end-up getting built using EJBs and deployed to a full blown J2EE application server.

Most of these applications could have been built using a dynamic scripting language (e.g. JRuby and Groovy) and deployed on Tomcat.

Here in lies the problem. Building applications on a complex, but "standard" stack is waste. It is waste in a number of different ways including:

  1. Defects - Using a complex platform significantly increases the chance that something will go wrong. This usually results in more defects and increased support costs as development resources have to deal with these defects instead of building new functionality.

  2. Unnecessary Lead Time - Because complex platforms require specialized skill sets you are naturally going to have fewer people with the ability to manage these platforms. This makes these individuals the bottleneck in your IT organization as they are the only ones who can identify and resolve issues.

  3. Financial Cost - Many organizations end up buying their application servers. These licensing costs can be extremely expensive. Making development teams use these products when they do not need them is an overallocation of overhead. Projects who should have never used a J2EE application server basically end up subsidizing the projects who do need this complexity.
If organizations truly want to see the benefits of standardization they need to let development teams work with the "right" tools to do the job. For instance, I personally no longer consider J2EE a platform. The Java Virtual Machine (JVM) is the platform and J2EE is an implementation choice. That means that depending on the problem being solved my development teams might use Groovy/Grails for web-based applications, Java/J2EE for components with multi-transactional requirements and a functional programming language like Scala to handle applications that require a high degree of scalability and throughput.

The benefits of standardization come into play when development teams standardize their development processes. Development teams in an organization should agree on a standard mechanism for documentation, how they manage their source control, how they build their code and how they unit test. Most developers can pick up a new language quickly. Standardizing on process allows developers to be moved between teams and be able to beging contributing quickly.

Organizations should be looking at supporting not a single one-size fits all language, but rather a toolbox with different options right-sized to the problem at hand. This is counter-intuitive as most organizations believe that learning new languages and techniques are incredibly expensive. However, software development is in many ways like a machining shop that produces a wide variety of different types of parts. Every part produced can be radically different. Machinists have to set their machine up to produce the right part.

Companies need to give developers flexibility and let them pick the right technology set for the problem at hand. Conversely, developers need to realize that they must constantly re-invest in themselves and learn new technologies and techniques. Training and new skills are not sole responsibility of your employer.

Ultimately it comes down to this: A machine shop would not lay down a mandate that all of their operators must use only a hacksaw and hammer because using these standardized tools will drive down operation costs. The corresponding time required to manufacture parts and the amount of re-work would go through the roof and quickly drive that shop out of business. Why require the development teams in your organization to do the same?

2 comments:

Tammy Adler said...

Great article, John. It amazes me that in the 21st century, with technology evolution, we continue to stay with the "one size fits all" philosophy. Instead, how about refining our processes and using the "right tools/stacks" for any given unique situation. It's a novel approach that actually works.

I believe organizations standardize on one platform/set of tools because of fearing the unknown, and concern for managing multiple environments. In our haste to "get things done quickly", we actually end up slowing the process because of old paradigms.

telecomtom said...

john, are you going to post the slideshow you used in your preso to newlug last week?