What do you do when you find yourself in quicksand?

What do you do when you find yourself in quicksand?

Climb out? Dive in deeper? Or flail around and hope it gets better?

More and more, corporate IT finds itself on the horns of a dilemma. The business wants more, delivered faster and cheaper, but IT’s budget for new development is constrained by support costs (as much as 75-80 percent). What is delivered often comes slower and at a greater cost than what would be available from an external provider. Ironically, this fact is often pointed out by the very chargeback systems that were designed to help control IT costs.

According to a study by Gartner and Financial Executives Research Foundation, Chief Financial Officers are already playing a greater role in technology decision making. A recent article on CIO.com highlighted a report written by The Economist Intelligence Unit for Dell Services in which two-thirds of CIOs consider their operations aligned with the business. Less than half of their CxO peers agree.

Before discussing how to resolve the dilemma, it’s useful to first look at how not to resolve it. Gojko Adzic, in his May 31 post How To Solve “Not Enough Time”, took on the issue of barriers to process improvement:

Many problems mask themselves as “not enough time”. Not enough time to test properly, not enough time to automate testing, not enough time to clean up code, not enough time to investigate issues or coordinate on what we’ll do… “Not enough time” is a pessimistic statement – we can’t do anything about it. Time flows at the same speed regardless of our intentions or actions. We can restate the problem into something more useful – “too much work”. Now that’s something that can be solved nicely.

It comes down to three steps:

• kill software already made that isn’t needed.
• kill software in making that won’t be needed
• kill software that was not successful

The very first comment in response:

Doesn’t spending time to kill software that wasn’t useful require more time? If we have too much work now, do you really expect me to take on more work to kill unnecessary stuff?

In other words, “I can’t save myself because I’m too busy drowning”.

Addressing the issues facing IT today requires positive action. The cavalry will not come riding over the hill and save us, we must do it ourselves. Doing so will require a fundamental shift in the way IT views itself in relation to the business.

Three concepts are key to transforming IT’s role in the enterprise:

• Customer Focus: The job of IT is not to provide technology, but to use technology to provide value. Doing so will require understanding the needs of business, both as individual units and as a whole. Fostering the partnership between business and IT must be made a top priority.
• Alignment and Execution on Strategy: A coherent strategy that is clearly communicated and gains definition as it traverses the levels of the organization is most likely to ensure that efforts are neither wasted nor contradictory.
• Active, Collaborative, and Innovative Governance: Providing greater value for less cost requires actively managing the enterprises’ technology portfolio. IT is uniquely positioned to provide both data and expertise to insure that optimal value is generated from technology investments and expenses, whether from internal or external providers.

Aspects of customer focus have been the topic of a number of posts. In “Holding Back the Tide”, I outlined the danger of being seen as the “Department of No”. This same theme appeared in “Deja Vu All Over Again”, detailing the prevalence of business users bypassing IT via cloud services. This is not to say that passively granting all wishes is the way to proceed; doing so can mean failing the enterprise as a whole which is equally your customer. Actively working with customers to try to find solutions that work for all concerned is far more likely to be successful than a curt denial.

TechRepublic’s recent “Good governance means reconsidering personal agendas” perfectly captured the essence of strategic alignment with this statement: “It means that the entire organization creates a partnership toward the common goal of laser focus on business outcomes”. Effective leadership develops a unified strategy and then ensures that the strategy filters down to the various components of the enterprise so that all are pulling in the same direction. This is not the same as micro-managing. Each unit sets the outcome to be attained by its constituent parts, each of which determine the best way to execute within the constraints they’re given. This provides both flexibility and coordination.

As I noted above, IT uniquely possesses the expertise needed to help insure that IT operations are aligned enterprise strategy and are both efficient and economical. This puts corporate IT operations in a dual role: coordinator of services and provider of services. It is a role that should be familiar: unless your enterprise builds its own servers, runs its own communications backbone, etc., some part of the IT role is already outsourced and managed. It is in the interests of both IT and the enterprise to have IT expand this across all areas. Rather than attempting to prevent use of mobile, cloud, etc., IT needs to be facilitating and managing so as to provide the best service to both individual units and the enterprise as a whole. This requires understanding the strengths and weaknesses of different options (in-house development vs. outsourced, owned infrastructure vs. cloud-based capacity, internally hosted applications vs. software as a service, etc.) and being able to provide guidance as to which option best fits the situation at hand. It requires being prepared to support the options chosen, regardless of whether those options are provided by IT or merely facilitated by IT.

Promoting the facilitator role is the missing ingredient for many enterprises. Chargebacks can be a very effective tool in understanding where the money is going, but are insufficient on their own. If the business has no official option to use another provider, then the chargeback is merely an insult. Giving them control is key to turning the business into a partner in the process instead of an adversary.

Effective Application Lifecycle Management and architecture need to be at the heart of the governance effort. Funds and staff can be freed up by culling redundant systems. Good architectural practices will help ensure that new development meets the needs of the business, and that existing applications avoid deteriorating. Promoting configurable applications that can be shared across the enterprise while still meeting the unique needs of disparate business units can also help in getting the most value out of both development and infrastructure.

Another extremely important consideration is tailoring the governance to the circumstances. One-size-fits-all schemes that attempt to impose the same process on all systems lead to unnecessary expense. Obviously the enterprise accounting system will be tightly controlled. If, however, every initiative is subject to the same restrictions, then innovation will be stifled. Taken to the extreme, this can lead to real cost issues: would you want to tie up a developer, tester, and project manager for every content change to the corporate web site? Low-maintenance, relatively self-service tools and services (collaboration, content management, reporting and business intelligence to name a few) can both increase customer satisfaction and lower costs.

There is no magic solution to the issues corporate IT is currently facing. The practices above can provide real value, but implementation cannot be a cookie-cutter process. Each organization’s circumstances and history impact on how and whether a given practice can be applied. What is universal is that inaction will not work. Flailing about and hoping that things will improve means that you sink deeper into the quicksand. At some point, your problem will be over, but not in the way you want.

Welcome to a Dogma-Free Zone

I was thinking of the immortal words of Socrates, who said, “… I drank what?”
(Val Kilmer as Chris Knight in “Real Genius”)

More than 2400 years ago, Socrates was convicted of corrupting the youth and impiety, for which he was sentenced to death. It was a high price to pay for asking embarrassing questions. And yet, Athens gained little by it. It’s prime was long past, and no matter how many critics it silenced, it could not regain its former glory.

So why the history lesson? Earlier this month, Dan North posted a brief notice that he had just returned from the Norwegian Developers Conference in Oslo and that they had published his article about opportunity cost in development leading up to the conference. The premise of the article was summed in the penultimate sentence: “So take nothing at face value, and instead look for the trade-offs in every decision you make, because those trade-offs are there whether or not you see them”. Encouraging people to evaluate their practices in light of the trade-offs involved did not strike me as a radical position, but it certainly attracted some heated comments. One in particular stated that Dan and all who agreed with him were “disingenuously misleading the ranks of up-and-coming programmers into wasting their time looking for better design methodologies than TDD when no such beast exists”.

That’s a bold statement. It assumes that x is universally applicable. It assumes that a x represents perfection and no further refinement is possible. Lastly, it assumes that questioning x is wrong. History has never been very kind to those holding these opinions, regardless of what we substitute for x.

There’s always an exception. There’s always something better down the road. Informed choice is superior to blind acceptance.

Those who question either prove the soundness of the current way or point the way to a better solution. Teaching the young to critically examine their methods lays the foundation for a stronger future. It certainly doesn’t corrupt. I know that I put more faith in anything strong enough to tolerate scrutiny than not.

Ironically, while all this was playing out, I stumbled across a post by Alistair Cockburn promoting “a discussion about whether idea (agile or plan-driven or impure or whatever) works well in the conditions of the moment”:

I’m tired of people from one school of thought dissing ideas from some other school of thought. I hunger for people who don’t care where the ideas come from, just what they mean and what they produce. So I came up with this “Oath of Non-Allegiance”.

I promise not to exclude from consideration any idea based on its source, but to consider ideas across schools and heritages in order to find the ones that best suit the current situation.

I think that covers it nicely.

Emergent Architectures and Growing Pains

Occasionally I hear people still raising the agile mantra against Big Design/Requirements Up Front. The thing is that Agile Manifesto never said to intentionally bury your head in the sand with regards to the purpose of the system. It was a push-back against spending months in analysis without anything but documents coming out, but the goal was to reach a middle ground. Nobody ever said “no design up-front” or “no requirements up front”.
(Udi Dahan from “A CQRS Journey – with and without Microsoft”)

In my premier post, I touched on the concept of emergent architecture in passing, noting that Wikipedia’s definition of architecture should be amended to read “Architecture should be both the process and product of planning, designing and construction, although it may just be the product of construction.” (my additions in bold). In essence, an architecture will develop regardless of whether that development is managed or not. The key question is whether the architecture provides value or headaches.

Lately the topic has been a popular one. A discussion on The Enterprise Architecture Network group on LinkedIn was active all last week. This weekend Igor Lobanov posted an excellent take on the subject, titled “Can an architecture emerge?”. Raja Bavani just posted “Can Software Projects Mature by Accident?”. It even made its way into the comments of the post where Dan North announced that he had reprinted “The Art of Misdirection” on his site. The person commenting took issue with a limitation of Test Driven Development (TDD) identified by Dan North:

“You need an overarching vision, a “big picture” design or architecture. TDD won’t give you that.” Wrong. TDD will give you precisely that: when you’re working on a large project, TDD allows you to build the code in small steps, where each step is the simplest thing that can possibly work. The architecture follows immediately from that: the architecture is just the accumulation of these small steps. The architecture is a product of TDD, not a pre-designed constraint.

It should come as little surprise that I find the concept of emergent architecture highly suspect, and the attitude in the quote above illustrates why. The “simplest thing that can possibly work” may or may not be sufficiently flexible. If that need for flexibility can be foreseen, then the “simplest thing that can possibly work” represents wasted time and effort. Likewise, an architecture that is “just the accumulation of these small steps” is unlikely to be robust or cohesive. It is extremely naive to expect a system’s requirements to exist without conflict. Working piecemeal, I find it unlikely that these natural tensions (such as security versus usability and performance versus simplicity) will be adequately addressed. Cross-cutting concerns ensure that a system must be more than the sum of its individual parts.

Igor Lobanov’s post compares the organic evolution of a system with Darwinian evolution, which is a compelling analogy. Left to its own devices, a solution will accumulate technical debt. Techniques such as TDD may allow the system to remain fit for purpose on a feature level, but the overall flexibility and robustness of the system will suffer. The more complex the system, the less amenable it will be to emergent architecture.