Form Follows Function on SPaMCast 471

It’s time for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

Last week’s episode, number 471, features Tom’s essay on the top 20 transformation killers. Jeremy Berriault‘s QA corner is about involving testers in the requirements process. My Form Follows Function segment rounds out the podcast, covering my post “Systems Thinking Complicates Things”.

In this installment, Tom and I talk about how simplistic analysis is unlikely to fit a complex problem. We illustrate this by talking about the game of “rock, paper, scissors” (then graduate to “rock, paper, scissors, lizard, Spock”!). I don’t really think that’s what’s meant by game theory, but hey, it’s fun (and illustrative) nonetheless.

You can find all the SPaMCast episodes I’m in under the SPaMCast Appearances category on this blog. Enjoy!

Advertisement

Systems Thinking Complicates Things

 

I’ve had the honor and pleasure of appearing as a regular on Tom Cagley‘s SPaMCast podcast for almost three years now. Before I write one of my “Form Follows Function on SPaMCast x” posts, I always listen to the podcast to make sure that the summary is right (the implication being, relying purely on my memory won’t be right). I got a bonus while writing up last week’s appearance, because Tom asked an excellent question that deserved its own post: does thinking about a problem (legacy systems, in the instance of last week’s discussion) holistically/systematically complicate things?

Abso-freakin’-lutely.

It is much easier to avoid all the twists and turns and possibilities inherent in systems thinking. A simpler approach, picking one lever to pull/one button to push, makes it much easier to come up with a solution.

It just doesn’t work very well at coming up with solutions that actually work.

When there is a mismatch in complexity between problem and solution architectures, this mismatch will be an additional problem to deal with. This will apply when the solution is more complex than the problem space warrants and when the opposite is the case. Solutions that fail to account for the context they will encounter are vulnerable. This is the idea behind the quote attributed to Albert Einstein: “Everything should be made as simple as possible, but not simpler.”

Human nature can push us to fix problems quickly, and quick will generally equate to simple. It takes time to analyse the angles and consider the alternatives. How often have you seen people ask for “the best” way to do something absent any context? How often have you seen people ask “why would someone ever do that?”

I’ll answer that by asking 3 questions:

• since Rock beats Scissors, why would anyone ever choose Scissors?
• since Paper beats Rock, why would anyone ever choose Rock?
• since Scissors beats Paper, why would anyone ever choose Paper?

Reality isn’t binary. It’s not what’s “best”, it’s what’s fit for purpose in a given context and there are lots and lots of contexts out there.

This isn’t to say that all quick, simple interventions are wrong. If you find yourself in a house fire, more action and less comprehensive deliberation may well be in order. The key is matching the cost (largely in terms of time) of defining the problem space with cost (in terms of both effort and risk that the intervention adds to the problem) of crafting the solution.

It’s almost guaranteed that the system contexts we deal with (both technical and social) will evolve toward more and more complexity. Surprises will emerge as a matter of course. We don’t need to make more by failing to take a more holistic view when we have the time to do so.

This is not a project

My apologies to René Magritte, as I appropriate his point, if not his iconic painting.

After I posted “Storming on Design”, it sparked a discussion with theslowdiyer around context and change. In that discussion, theslowdiyer commented:

‘you don’t adhere to a plan for any longer than it makes sense to.’
Heh, agree. I wonder if the “plan as a tool” vs. “plan as a goal in itself” discussion isn’t deserving of a post of its own 🙂

Indeed it is, even if it did take me nearly four months to get to it.

The key concept to understand, is that the plan is not the goal, merely a stated intention of how to achieve the goal (if this causes you to suspect that the words “plan” and “design” could be substituted for each other without changing the point, move to the head of the class). Magritte’s painting stated that the picture is not the thing. The map is not the territory (and if that concept seems a bit self-evident, consider the fact that Wikipedia considers it significant enough to devote over 1700 words, not counting footnotes and links, to the topic).

Conflating plan and goal is a common problem. To illustrate the difference, consider undergoing an operation. Is it your desire that the surgeon perform the procedure as planned or that your problem gets fixed? In the former scenario, your survival is optional.

This is not, however, to say that planning (or design) is useless. The output of an effective planning/design process is critical. As Joanna Young noted in her “Four Signs of Readiness – Or Not”:

I’m all for consigning the traditional 50+ pages of adminis-trivia on scope, schedule, budget, risks that requires signing in blood to the dustbin. However no organization should forego the thoughtful and hard work on determining what needs to be done, why, how, by whom, for how much – and how this will all be governed and measured as it is proceeds through sprints and/or waterfalls to delivery.

The information derived from the process (not the form, not the presentation, but the information) is critical tool for moving forward intelligently. If you have no idea of what to do, how to do it, who can do it when and for how much, you are adrift. You’re starting a trip with no idea of whether the gas tank has anything in it. Conversely, attempting to achieve 100% certainty from the outset is a fool’s errand. For any endeavor, more will be known nearer the destination. Plans without “wiggle room” are of limited usefulness as you will drift outside the cone of uncertainty from the start and never get back inside.

Having a reasonable idea of what’s acceptable variance helps determine when it’s time to abandon the current plan and go with a revised one. Planning and design are processes, not events or even phases. It’s a matter of continually monitoring context and whether our intentions are still in accordance with reality. Where the differ, reality wins. Always.

Execution isn’t blindly marching forward according to plan. It’s surfing the wave of context.

Square Pegs, Round Holes, and Silver Bullets

People like easy answers.

Why spend time analyzing and evaluating when you can just take some thing or some technique that someone else has already put to use and be done with it?

Why indeed?

I mean, “me too” is a valid strategy, right?

Oh @abscond, I've seen oodles of blockchain / AI / big data "me too" projects in large orgs with no situational awareness (even user needs)

—
  (@swardley) March 21, 2017

And we don’t want people to get off message, right?

"What makes an experience compelling?" says a software company's sales person whilst presenting a pre-canned set of slides...

—
Matt Ballantine (@ballantine70) March 21, 2017

And we can always find a low cost, minimal disruption way of dealing with issues, right?

Why didn't we think of that? https://t.co/gTOB47YdBH

—
You Had One Job (@_youhadonejob1) March 24, 2017

I mean, after all, we’ve got data and algorithms, and stuff:

The thing is, actions need to make sense in context. Striking a match is probably a good idea in the dark, but it’s probably less so in daylight. In the presence of gasoline fumes, it’s a bad idea regardless of ambient light.

A recent post on Medium, “Design Sprints Are Snake Oil” is a good example. Erika Hall’s title was a bit click-baitish, but as she responded to one commenter:

The point is that the original snake oil was legitimate and effective. It ended up with a bad reputation from copycats who over-promised results under the same name while missing the essential ingredients.

Sprints are legitimate and effective. And now there is a lot of follow-up hype treating them as a panacea and a replacement for other types of work.

Good things (techniques, technologies, strategies, etc.) are “good”, not because they are innately right, but because they fit the context of the situation at hand. Those that don’t fit, cease being “good” for that very reason. Form absent function is just a facade. Whether it’s business strategy, management technique, innovation efforts, or process, there is no recipe. The hard work to match the action with the context has to be done.

Imitation might be the sincerest form of flattery, but it’s a really poor substitute for strategy.

Form Follows Function on SPaMCast 426

One of the benefits of being a regular on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast is getting to take part in the year-end round table (episode 426). Jeremy Berriault, Steve Tendon, Jon M. Quigley and I joined Tom for a discussion of:

1. Whether software quality would be a focus of IT in 2017
2. Whether Agile is over, at least as far as Agile as a principle-driven movement
3. Whether security will be more important than quality and productivity in the year ahead

It was a great discussion and, as Tom noted, a great way to finish off the tenth year of the SPAMCast and kickoff year eleven.

You can find all my SPaMCast episodes using under the SPAMCast Appearances category on this blog. Enjoy!

Situational Awareness – Where does it begin? Where does it end?

Situational awareness, according to Wikipedia, is defined as “…the perception of environmental elements and events with respect to time or space, the comprehension of their meaning, and the projection of their status after some variable has changed, such as time, or some other variable, such as a predetermined event”. In other words, it’s having a handle on what currently is and what is about to happen. It’s a concept that is invaluable to a wide range of interests, particularly management/leadership, architectural design, and innovation. It’s a concept that crosses levels, from tactical to strategic. Just as socio-technical systems architectures exist in a fractal space (application to solution to enterprise), so too does the concept of situational awareness. As such, it’s a common theme for this site, particularly over the last year or so.

The OODA (Observe-Orient-Decide-Act) Loop, developed by Air Force Colonel John Boyd, is a framework for decision-making that explicitly incorporates situational awareness:

Coupling sense-making with decision-making is critical to achieve a balance of both speed and effectiveness. In my opinion, acting without taking the state of the environment into account is a recipe for disaster. Equally important (likewise, in my opinion), is understanding the dynamic nature of situational awareness. As Boyd’s diagram above shows, it’s not a linear process. Additionally, the very nature of a loop should convey the fact that there’s neither beginning nor end. This is a key concept.

One of the sites that I follow is Slightly East of New, which is run by an associate of Boyd’s and dedicated to his theories. A recent post on that site, “The magic of the OODA loop”, related a paragraph from a sci-fi novel, The Apocalypse Codex, that referred to OODA:

Observe, orient, decide, act: words to live or die by. Right now, Persephone is disoriented — on the run, cut off. It’s time to go on the offensive, work out where she is and what’s going on, then get the hell out of this trap.

It was an interesting post, but nothing noteworthy, until I got to this comment:

I find the phrase, “…on the run, cut off.” very interesting, within the context of “disoriented”. To me, “on the run” mean a decision has been made and acted on, whereas “disorientation” usually means that one can’t make a decision.
Likewise, “cut off” is the position you find yourself in, after all the decisions have been made and, after thinking about it, it is the posture you observe yourself to be in.
In other words, on the run and cut off is not really a disorientation, but a reality.
So, while you may not survive, you have made a decision to run or you are about to make a decision and join the otherside.
I suppose it just depends on where those words show up in the narrative, as to if you made the decision or your competitor made the decision for you.

I may be over-sensitive to the phrasing, but “…decision has been made and acted on…” and “…after all the decisions have been made…” strike me as being too static and too linear. Every action/inaction follows on decision/indecision. The point “…after all the decisions have been made…” is terminal (for the person who has made all the decisions they will make). In my opinion, it is key to bear in mind that the clock is always running and that the reality being processed is already past. Too much attention to the state of what is (or rather, was) takes away from the more important task of getting to a better “to be” state. Additionally, decisions and contexts should be thought of as not just linear, but fractal (e.g. having multiple levels from tactical through strategic) as well.

Loops that have an end are no longer loops. Likewise, we have to be able to strike a balance between just focusing on what’s relevant (too much context/backstory can cause information overload) and the point where we’ve trimmed away necessary context.

Actively thinking about sense-making and decision-making can seem overly academic. The activities are so foundational to nearly everything that they can feel instinctual rather than learned. I suspect that’s a case of “familiarity breeds contempt”. Depending on the application, contempt for developing the best possible situational awareness could be fatal.

[OODA Loop diagram by Patrick Edwin Moran via Wikimedia Commons]

Pragmatic Application Architecture

I saw a tweet on Friday about a SlideShare deck that looked interesting, so I bookmarked it to read later. As I was reading it this morning, I found myself agreeing with the points being made. When I got to the next to the last slide, I found myself (or at least, this blog) listed alongside some very distinguished company under “Reading Material”.

Thanks, Bart Blommaerts and nice job!

Designing Communication, Communicating Design

We work in a communications industry.

We create and maintain systems to move information around in order to get things done. That information moves between people and systems in combinations and configurations too numerous to count. In spite of that, we don’t do that great a job of communicating what should be, for us, extremely important information. We tend to be really bad at communicating the architecture of our systems – structure, behavior, and most importantly, the reasons for the decisions made. It’s bad enough when we fail to adequately communicate that information to others, it’s really bad when we fail to communicate it to ourselves. I know I’ve let myself down more than once (“What was I thinking here?!”).

Over the past few days, I’ve been privileged to follow (and even contribute a bit to) a set of conversations on Twitter. Grady Booch, Ivar Jacobson, Ruth Malan, Simon Brown, and others have been discussing the need for architectural awareness and the state of communicating architecture.

This exchange between Simon, Chris Carroll and Eoin Woods sums it up well:

@KuhlWaterloo @ivarjacobson @Grady_Booch I've run architecture katas for 10K+ people and they do struggle to communicate architecture 😞

—
Simon Brown (@simonbrown) October 23, 2016

@simonbrown @KuhlWaterloo @ivarjacobson @Grady_Booch For want of a language or for want of understanding what needs to be communicated?

—
Chris F Carroll (@chrisfcarroll) October 23, 2016

@chrisfcarroll @simonbrown @KuhlWaterloo @ivarjacobson @Grady_Booch both in my experience

—
Eoin Woods (@eoinwoodz) October 24, 2016

First and foremost, an understanding of what the role of a software architect is and why it’s important is needed. Any organization where the role is seen as either just a senior developer or (heaven help us!) some sort of Taylorist “thinker” who designs everything for the “worker bee” coders to implement, is almost guaranteed to be challenged in terms of application architecture. Resting on that foundation of shifting sand, the organization’s enterprise IT architecture (EITA) is likewise almost guaranteed to be challenged barring a remarkable series of “happy accidents”. The role (not necessarily position) of software architect is required, because software architecture is a distinct set of concerns that can either be addressed intentionally or left to emerge haphazardly out of the construction of the system.

Before we can communicate the architecture of a system, it’s necessary to understand what that is. In “Software Architecture: Central Concerns, Key Decisions”, Ruth Malan and Dana Bredemeyer defined it as high impact, systemic decisions involving (at a minimum):

• system priority setting
• system decomposition and composition
• system properties, especially cross-cutting concerns
• system fit to context
• system integrity

I don’t think it’s possible to over-emphasize the use of “system” and “systemic” in the preceding paragraphs. That being said, it’s important to understand that architectural concerns do not exist in a void. There is a cyclic relationship between the architectural concerns of a system and the system’s code. The architectural concerns guide the implementation, while the implementation defines the current state of the architecture and constrains the evolution of future state of the architecture. Code is a necessary, but insufficient source of architectural knowledge – it’s not enough. As Ruth Malan noted in the Visual Design portion (part II) of her presentation at the Software Architect Conference in London a year ago:

While the code serves as a foundation of the system, it’s also important to realize that the system exists within a larger context. There is a fractal set of systems within systems within ecosystems. Ruth illustrated this in the Intention and Reflection portion (part III) of the presentation reference above:

[Note: Take the time to view the entirety of the Intention and Reflection presentation. It’s an excellent overview of how to design the architecture of a system.]

The fractal nature of systems within systems within ecosystems is illustrated by the image at the top of the post (h/t to Ric Phillips for the reblog of it). Richard Sage‘s humorous (though only partly, I’m sure) suggestion of it as a meta-model goes a long way towards portraying the problem of a language to communicate architecture.

Not only are we dealing with a nested set of “things”, but the understanding of those things differ according to the stakeholder. For example, while the business owner might see a “web site” as one monolithic thing, the architect might see an application made up of code components depending on other applications and services running on a collection of servers. Maintaining a coherent, normalized object model of the system yet being able to present it in multiple ways (some of which might be difficult to relate) is not a trivial exercise.

Lower-level aspects of design lend themselves to automated solutions, which can increase reliability of the model by avoiding “documentation rot”. An interesting (in my opinion) aspect that can also be automated is the evolution of code over time. What can’t be parsed from the code, however, is intention and reasoning.

Another barrier to communication is the need to be both expressive and flexible (also well illustrated by Richard’s meta-model) while also being simple enough to use. UML works well on the former, but (rightly or wrongly) is perceived to fail on the latter. Simon Brown’s C4 model aims to achieve a better balance in that aspect.

At present, I don’t think we have one tool that does it all. I suspect that even with a suite of tools, that narrative documents will still be way some aspects are captured and communicated. Having a centralized store for the non-code bits (with a way to relate them back to the code) would be a great thing.

All in all, it is encouraging to see people talking about the need for architectural design and the need to communicate the aspects of that design.

Form Follows Function on SPaMCast 415

This week’s episode of Tom Cagley’s Software Process and Measurement (SPaMCast) podcast, number 415, features Tom’s essay on recognizing risk and risk tolerance, Kim Pries on change models, a Form Follows Function installment based on my post “All Aboard the Innovation Band Wagon?”, and Jon Quigley on requirements management.

Innovation is the topic of our discussion this time – what it is, whether it’s something that every organization can achieve, and is there a recipe? Everyone wants to be an innovator, but there’s a lot of question around just what that entails.

You can find all my SPaMCast episodes using under the SPAMCast Appearances category on this blog. Enjoy!

Design for Life

 

The underlying theme of my last post, “Babies, Bathwater, and Software Architects”, was that it’s necessary to understand the role of a software architect in order to understand the need for that role. If our understanding of the role is flawed, not just missing aspects of what the role should be focusing on, but also largely consisting of things the role should not be concerned with, then we can’t really effectively determine whether the role is needed or not. If a person drowning is told that an anvil is a life-preserver, that doesn’t mean that they don’t need a life-preserver. It does mean they need a better definition of “life-preserver”.

Ruth Malan, answering the question “Do we still need architects?”, captures the essence of what is unique about the concerns of the software architect role:

There’s paying attention to the structural integrity of the code when that competes with the urge to deliver value and all we’ve been able to accomplish in terms of the responsiveness of continuous integration and deployment. We don’t intend to let the code devolve as we respond to unfolding demands, but we have to intend not to, and that takes time — possibly time away from the next increment of user-perceived value. There’s watching for architecturally impactful, structurally and strategically significant decisions, and making sure they get the attention, reflection, expertise they require. Even when the team periodically takes stock, and spends time reflecting learning back into the design/code, the architect’s role is to facilitate, to nurture, the design integrity of the system as a system – as something coherent. Where coherence is about not just fit and function, but system properties. Non-trivial, mutually interacting properties that entail tradeoffs. Moreover, this coherence must be achieved across (microservice, or whatever, focused) teams. This takes technical know-how and know-when, and know-who to work with, to bring needed expertise to bear.

These system properties are crucial, because without a cohesive set of system properties (aka quality of service requirements), the quality of the system suffers:

"Spare parts flying in close formation." Yes, a lot of production software is like that.

—
Grady Booch (@Grady_Booch) September 14, 2016

@Grady_Booch @GeneHughson slide 53 🙂 slideshare.net/RufM/design-vi… https://t.co/t9viY75ocb

—
VisArch (@ruthmalan) September 15, 2016

Even if the parts are perfectly implemented and fly in perfect formation, the quality is still lacking.

A tweet from Charles T. Betz points out the missing ingredient:

First use of the word "architecture" w/r/t computing, I think. Unsurprised it was Brooks. amturing.acm.org/Buchholz_10263… https://t.co/Pafm8NJw7a

—
Charles T. Betz (@CharlesTBetz) October 03, 2016

Now, even though Frederick Brooks was writing about the architecture of hardware, and the nature of users has drastically evolved over the last fifty-four years, his point remains: “Architecture must include engineering considerations, so that the design will be economica1 and feasible; but the emphasis in architecture is upon the needs of the user, whereas in engineering the emphasis is upon the needs of the fabricator.”

In other words, habitability, the quality of being “livable”, is a critical condition for an architecture. Two systems providing the exact same functionality may not be equivalent. The system providing the “better” (from the user’s perspective) quality of service will most likely be seen as the superior system. In some cases, quality of service concerns can even outweigh functional concerns.

Who, if not the software architect, is looking out for the livability of your system as a whole?