Who Needs Architects? Because Complexity Emerges

Why would you want to constrain creativity by controlling (Note: controlling does not necessarily imply dictating) the architecture of a system?

As Roger Sessions recently tweeted:

Another reason came out in an exchange between Roger and me:

Simplicity (“…as simple as possible, but not simpler”) is certainly a desirable system quality. Unnecessary complexity directly impairs maintainability and can indirectly affect qualities such as testability, security, extensibility, availability, and ease of deployment just to name a few. The coherent structure necessary to create and maintain simplicity when multiple people are involved is unlikely to happen accidentally. When it’s lacking, the result is often what Brian Foote and Joseph Yoder described in their 1999 classic “Big Ball of Mud”:

A BIG BALL OF MUD is haphazardly structured, sprawling, sloppy, duct-tape and bailing wire, spaghetti code jungle. We’ve all seen them. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated. The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition.

Sixteen years later, this tendency to entropy via unnecessary complexity remains an issue. As Roger Sessions recently noted on his blog:

As IT systems increase in size, three things happen. Systems get more vulnerable to security breaches. Systems suffer more from reliability issues. And it becomes more expensive and time consuming to try to make modifications to those systems. This should not be a surprise. If you are like most IT professionals, you have seen this many times.

What you have probably not noticed is an underlying pattern. These three undesirable features invariably come in threes. Insecure systems are invariably unreliable and difficult to modify. Secure systems, on the other hand, are also reliable and easy to modify.

This tells us something important. Vulnerability, unreliability, and inflexibility are not independent issues; they are all symptoms of one common disease. It is the disease that is the problem, not the symptoms.

Kent Beck, in a post on FaceBook, “Taming Complexity with Reversibility”, recently noted the same:

As a system scales, whether it is a manufacturing plant or a service like ours, the enemy is complexity. If you don’t confront complexity in some way, it will eat you. However, complexity isn’t a blob monster, it has four distinct heads.

  • States. When there are many elements in the system and each can be in one of a large number of states, then figuring out what is going on and what you should do about it grows impossible.
  • Interdependencies. When each element in the system can affect each other element in unpredictable ways, it’s easy to induce harmonics and other non-linear responses, driving the system out of control.
  • Uncertainty. When outside stresses on the system are unpredictable, the system never settles down to an equilibrium.
  • Irreversibility. When the effects of decisions can’t be predicted and they can’t be easily undone, decisions grow prohibitively expensive.

If you have big ambitions but don’t address any of these factors, scale will wreck your system at some point.

Kent’s conclusion, “What changes–technical, organizational, or business–would you have to make to identify such decisions earlier and make reversing them routine?”, implies that addressing these factors requires systemic rather than localized response.

The counter-argument that’s frequently made is that the architecture can “emerge” from the implementation by doing the “simplest thing that can possibly work” and building on that. I touched briefly on this in my last post. To that, I’d add that not all changes are the same. Trying to bolt on fundamental, cross-cutting concerns (e.g. security, scalability, etc.) after the fact risks major (i.e. expensive) architectural refactoring. This type of refactoring is generally a hard sell and understandably so. That difficulty makes the Big Ball of Mud that much more likely.

This does not mean that the concept of emergence has no place in software architecture. As the various contexts making up the architecture of the problem are identified, challenges will emerge and need to be reconciled. With this naturally occurring emergence, it makes little sense to artificially generate challenges by refusing to “peek” at what’s ahead. As Ruth Malan has observed, architecture should be both “intentional and emergent”:

This need to contend with emergent issues continues for the entire lifetime of the system:

I’ve noted in the past that both planning and design share similarities. Regardless of the task, an appropriate design/plan provides a coherent direction that enhances the likelihood of success. Without it, Joe Dager’s question below is directly on point:

Advertisements

4 thoughts on “Who Needs Architects? Because Complexity Emerges

  1. By controlling the architecture of a system, I want to channel creativity. With a proper architecture in place, different people in similar situations come up with similar solutions or bring innovations.

    In such a way, the number of duplications and re-works will be reduced and more time/resources/energy will be spent for addressing unique business challenges via innovations. In the other words, by reducing undesired complexity of a system it is possible to increase natural complexity of the system.

    Thanks,
    AS

    Like

    • Absolutely…excellent point re: channeling creativity. It’s often forgotten that every decision (even decisions not to decide) impose constraints – it’s a matter of whether the constraints were intelligently evaluated or not.

      Like

  2. Pingback: Organizations and Innovation – Swim or Die! | Form Follows Function

  3. Pingback: Babies, Bathwater, and Software Architects | Form Follows Function

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s