Designing Communication, Communicating Design

The Simplest metamodel in the world ever!

We work in a communications industry.

We create and maintain systems to move information around in order to get things done. That information moves between people and systems in combinations and configurations too numerous to count. In spite of that, we don’t do that great a job of communicating what should be, for us, extremely important information. We tend to be really bad at communicating the architecture of our systems – structure, behavior, and most importantly, the reasons for the decisions made. It’s bad enough when we fail to adequately communicate that information to others, it’s really bad when we fail to communicate it to ourselves. I know I’ve let myself down more than once (“What was I thinking here?!”).

Over the past few days, I’ve been privileged to follow (and even contribute a bit to) a set of conversations on Twitter. Grady Booch, Ivar Jacobson, Ruth Malan, Simon Brown, and others have been discussing the need for architectural awareness and the state of communicating architecture.

This exchange between Simon, Chris Carroll and Eoin Woods sums it up well:

First and foremost, an understanding of what the role of a software architect is and why it’s important is needed. Any organization where the role is seen as either just a senior developer or (heaven help us!) some sort of Taylorist “thinker” who designs everything for the “worker bee” coders to implement, is almost guaranteed to be challenged in terms of application architecture. Resting on that foundation of shifting sand, the organization’s enterprise IT architecture (EITA) is likewise almost guaranteed to be challenged barring a remarkable series of “happy accidents”. The role (not necessarily position) of software architect is required, because software architecture is a distinct set of concerns that can either be addressed intentionally or left to emerge haphazardly out of the construction of the system.

Before we can communicate the architecture of a system, it’s necessary to understand what that is. In “Software Architecture: Central Concerns, Key Decisions”, Ruth Malan and Dana Bredemeyer defined it as high impact, systemic decisions involving (at a minimum):

  • system priority setting
  • system decomposition and composition
  • system properties, especially cross-cutting concerns
  • system fit to context
  • system integrity

I don’t think it’s possible to over-emphasize the use of “system” and “systemic” in the preceding paragraphs. That being said, it’s important to understand that architectural concerns do not exist in a void. There is a cyclic relationship between the architectural concerns of a system and the system’s code. The architectural concerns guide the implementation, while the implementation defines the current state of the architecture and constrains the evolution of future state of the architecture. Code is a necessary, but insufficient source of architectural knowledge – it’s not enough. As Ruth Malan noted in the Visual Design portion (part II) of her presentation at the Software Architect Conference in London a year ago:

Slide from Ruth Malan's presentation on Visual Design

While the code serves as a foundation of the system, it’s also important to realize that the system exists within a larger context. There is a fractal set of systems within systems within ecosystems. Ruth illustrated this in the Intention and Reflection portion (part III) of the presentation reference above:

Slide from Ruth Malan's presentation on Intention and Reflection

[Note: Take the time to view the entirety of the Intention and Reflection presentation. It’s an excellent overview of how to design the architecture of a system.]

The fractal nature of systems within systems within ecosystems is illustrated by the image at the top of the post (h/t to Ric Phillips for the reblog of it). Richard Sage‘s humorous (though only partly, I’m sure) suggestion of it as a meta-model goes a long way towards portraying the problem of a language to communicate architecture.

Not only are we dealing with a nested set of “things”, but the understanding of those things differ according to the stakeholder. For example, while the business owner might see a “web site” as one monolithic thing, the architect might see an application made up of code components depending on other applications and services running on a collection of servers. Maintaining a coherent, normalized object model of the system yet being able to present it in multiple ways (some of which might be difficult to relate) is not a trivial exercise.

Lower-level aspects of design lend themselves to automated solutions, which can increase reliability of the model by avoiding “documentation rot”. An interesting (in my opinion) aspect that can also be automated is the evolution of code over time. What can’t be parsed from the code, however, is intention and reasoning.

Another barrier to communication is the need to be both expressive and flexible (also well illustrated by Richard’s meta-model) while also being simple enough to use. UML works well on the former, but (rightly or wrongly) is perceived to fail on the latter. Simon Brown’s C4 model aims to achieve a better balance in that aspect.

At present, I don’t think we have one tool that does it all. I suspect that even with a suite of tools, that narrative documents will still be way some aspects are captured and communicated. Having a centralized store for the non-code bits (with a way to relate them back to the code) would be a great thing.

All in all, it is encouraging to see people talking about the need for architectural design and the need to communicate the aspects of that design.

Modeling the Evolution of Software Architecture

Herve Lourdin‘s tweet wasn’t aimed at modeling, but the image nicely illustrates a critical deficiency in modeling languages – showing evolution of a system over time. Structure and behavior are captured, but only for a given point in time. Systems and their ecosystems, however, are not static. A map of the destination without reference to the point of origin or rationale for choices is of limited use in communicating the what, how, and why behind architectural decisions.

“The Road Ahead for Architectural Languages” on InfoQ (re-published from IEEE Software) recently noted the following reasons for not using an architectural language (emphasis added):

  • formal ALs’ need for specialized competencies with insufficient perceived return on investment,
  • overspecification as well as the inability to model design decisions explicitly in the AL, and
  • lack of integration in the software life cycle, lack of mature tools, and usability issues.

All of the items in bold above represent usability and value issues; a failure to communicate. As Simon Brown observed in “Simple Sketches for Diagramming Your Software Architecture”:

In today’s world of agile delivery and lean startups, some software teams have lost the ability to communicate what it is they are building and it’s no surprise that these teams often seem to lack technical leadership, direction and consistency. If you want to ensure that everybody is contributing to the same end-goal, you need to be able to effectively communicate the vision of what it is you’re building. And if you want agility and the ability to move fast, you need to be able to communicate that vision efficiently too.

Simon is a proponent of a sketching technique that answers many of these communication failures:

The goal with these sketches is to help teams communicate their software designs in an effective and efficient way rather than creating another comprehensive modelling notation. UML provides both a common set of abstractions and a common notation to describe them, but I rarely find teams that are using either effectively. I’d rather see teams able to discuss their software systems with a common set of abstractions in mind rather than struggling to understand what the various notational elements are trying to show.

Simon’s colleague, Robert Annett, recently posted “Diagrams for System Evolution”, which proposes using the color-coding scheme from diff tools to indicate change: red = remove, blue = change, green = new. Simon followed this up with two posts of his own, “Diff’ing software architecture diagrams” and “Diff’ing software architecture diagrams again”, which dealt with applying Robert’s ideas to Simon’s structurizr.com tool.

Simon’s work, coupled with Robert’s ideas, addresses many of the highlighted deficiencies listed above (it even touches on the third bullet that I didn’t emphasize). Ruth Malan’s work also contains some ideas that are vital (in my opinion) to being able to visualize and communicate important design considerations – explicit quality of service and rationale elements along with organizational context elements. A further enhancement might be incorporating these into a platform that can tie elements of software architecture together with elements of solution and enterprise architecture, such as the one proposed by Tom Graves.

Given the need for agility, it might seem strange to be talking about modeling, design documentation, and architectural languages. The fact is, however, that many of us deal with inherently complex systems in inherently complex ecosystems. Without the ability to visualize a design in its context, we run the risk of either slowing down or going down. Not everyone can afford to “move fast and break things”.

Who Needs Architects? Who’s Minding the Architecture?

shearing layers diagram

Shearing layers are an important concept in building architecture. Essentially, the idea is that a building is not a unary thing with a single lifecycle, but a composition of several layers comprising elements with different concerns (site, structure, skin, services, space plan, and contents stuff) and with varying lifecycles based on their amenability to change. This same concept can be applied to software systems. Between platform components and the fractal nature of systems and solutions, groupings of concerns with diverse lifecycles are readily apparent.

Richard Veryard, in his post “Agile and Wilful Blindness”, used the concept of shearing layers to help illustrate a weakness of development styles that stress emergence over deliberate design:

Some things are easier to change than others. The architect Frank Duffy proposed a theory of Shearing Layers, which was further developed and popularized by Stuart Brand. In this theory, the site and structure of a building are the most difficult to change, while skin and services are easier.

Let’s suppose Agile developers know how to optimize some of the aspects of a system, perhaps including skin and services. So it doesn’t matter if they get the skin and services wrong, because these can be changed later. This is the basic for @swardley’s point that you don’t need to know beforehand exactly what you are building.

But if they get the fundamentals wrong, such as site and structure, these are more difficult to change later. This is the basis for John Seddon’s point that Agile may simply build the wrong things faster.

And this is where @ruthmalan takes the argument to the next level. Because Agile developers are paying attention to the things they know how to change (skin, services), they may fail to pay attention to the things they don’t know how to change (site, structure). So they can throw themselves into refining and improving a system until it looks satisfactory (in their eyes), without ever seeing that it’s the wrong system in the wrong place.

Although I disagree that all agile developers fall into this category, Richard’s and Ruth Malan’s point is an important one. We must be aware of a need in order to attend to it. As Seth Godin noted in “I didn’t see it because I wasn’t looking”: “This is one reason we feel the need to yell ‘surprise’ at a surprise party.”

Even when we’re aware of something, failing to understand the true nature of it can hamper our ability to deal with it effectively. Too many consider the practice of architectural design to be just a matter of drawing pictures and creating documents. The important aspect, however, is the decisions and the rationales for those decisions that these artifacts should capture. As Ipek Ozkaya tweeted from Gregor Hohpe‘s SATURN 2015 keynote:

Architectural design is essentially about making (albeit collaboratively) and communicating decisions. The decisions that are architecturally significant are the decisions that affect the longer-lived, harder to change aspects of a system. Intentionally addressing these concerns is a better strategy than hoping that something coherent just “emerges”.

Your Code is not Enough

That tells me what, but why?

Although the Agile Manifesto proclaims a preference for “Working software over comprehensive documentation”, that’s a far cry from suggesting no documentation. In fact, the manifesto specifically notes “…there is value in the items on the right”. Some can be tempted to equate the code with the design, but as Ruth Malan has pointed out, this is erroneous:

… but to indicate that the code is the design language and it is the full (and most accurate) expression of the design misses key points. For example, it misses the “negative space” (things we don’t do) directed by the design. It misses the notion that design is an abstraction or conception, and not just any conception — the design is conceived just-so*, and there is a premise (or a conjoint set) that links intent (or aspiration or purpose) with the particular form the design takes, the organization, the elements, their relationships, their articulation or interaction points, their collaborative interactions, and more. The code contains neither the abstraction** nor the premise. Sure, we want the code to speak to the design, to realize the design and to imply and signify and convey the design as best the code can. And if we create the design in the process of writing the code, simultaneously thinking about design issues and code detail, the point still holds — we want the code to be as expressive of the design as we can reasonably make it. But both the code and informal tribal memory are going to be missing bits, so it is good to write and draw the design as design out. Or at least the architecture — the strategically and structurally significant bits. To draw out the relationship of the system to its various contexts, the organization of architectural abstractions (or elements) and their interrelationships and the key mechanisms — with diagrams and descriptions — and explicating the reasoning that drove those design choices (and eliminated others).

* When I say “just so” I don’t by any means mean all at once, but rather that the conception is particular. It is a set of choices (sometimes explicitly reasoned, sometimes more intuitively and implicitly or subconsciously arrived at) that we either can defend or need to try out. We apply insights from experience and knowledge that has been distilled over many experiences, and reason our way to the design approach. And then we test it. Of course.

** Abstractions, yes. Abstractions indicated by the design. But not the design abstraction that contains little of what is in code statements but much of what is in code relationships and form, and in what is not in the code, what is specifically, designedly, not done.

Simon Brown, in his July 2012 Skills Matter presentation, “The code doesn’t tell the whole story”, listed several attributes of a system that code is generally insufficient to document”

  • The “big picture” context of the system
  • Quality of service requirements significant to the architecture
  • Architecturally significant constraints on the design
  • Guiding principles of the design
  • The physical environment (infrastructure) the system will operate in
  • Deployment locations of the code components
  • Operational aspects (monitoring, management, security, disaster recovery)
  • Security

In short, the code the conveys the ‘what’ but not the ‘why’ (arguably the more important aspect). It defines ‘what is’ but is silent regarding ‘what should be’, ‘how fast’, etc. We cannot readily infer from the code the physical layout of the system nor its operating environment.

Although there is a need for documentation beyond the code, the traditional dangers (divergence between the documentation and reality, accessibility of the artifact, audiences ignoring artifacts, etc.) still apply. Documentation needs to be accurate and tailored to the task at hand in order to be effective. Quantity is far from the same thing as quality in this case.

Documentation that can be automatically generated from code, database schema, and other artifacts rather than maintained manually will help prevent divergence. Manual maintenance should be reserved for those things that cannot be generated and only when there is a need for continuity. Documenting architecturally significant design decisions and the rationale behind them makes sense in that the information will apply across a large portion, if not all, of the product’s lifecycle. Lower-level, tactical design documents (where used) will only be useful during the release for which it was created; time spent creating and maintaining these artifacts should be minimized.

Clear communication should be a top priority for all documentation. Diagrams can prove the old saying that a picture is worth a thousand words when the focus is on clarity. Semantics will tend to be more important than style or syntactic correctness. The audience and intended use of an artifact will also affect its usefulness as documentation. Automated tests may be useful in conveying some types of information to technically savvy audiences, but may be less useful to others. As with code, you should determine whether the artifact can convey the information you want understood.

A beneficial side-effect of creating lightweight design documentation is that it assists in the collaboration process. Describing the design helps get it out of your head (as well as those of your collaborators) and into a format more conducive to inspection and evaluation. Sometimes the mere organization of your thoughts in order to communicate them allows you to see potential problems.