Stopping Accidental Technical Debt

Buster Keaton looking at a poorly constructed house

In one of my earlier posts about technical debt, I differentiated between intentional debt (that taken on deliberately and purposefully) and accidental debt (that which just accrues over time without rhyme or reason or record). Dealing with (in the sense of evaluating, tracking, and resolving it) technical debt is obviously a consideration for someone in an application architect role. While someone in that role absolutely should be aware of the intentional debt, is there a way to be more attuned to the accidental debt as well?

Last summer, I published a post titled “Distance…is the one true enemy…”. The post started with a group of tweets from Gregory Brown talking about the corrosive effects of distance on software development (distance between compile and run, between failure and correction, between development and feedback, etc.). I then extended the concept to management, talking about how distance between sense-maker and decision-maker could negatively affect the quality of the decisions being made.

There’s also a distance that neither Greg nor I covered at the time, design distance. Design distance is the distance between the design and the outcome. Reducing design distance makes it easier to keep a handle on the accidental debt as well as the intentional.

Distance between the architectural decisions and the implementation can introduce technical debt. This distance can come from remote decision-makers, architecture pigeons who swoop in, deposit their “wisdom”, and then fly away home. It can come from failing to communicate the design considerations effectively across the entire team. It can also come from failing to monitor the system as it evolves. The design and the implementation need to be in alignment. Even more so, the design and the implementation need to align with particular problems to be solved/jobs to be done. Otherwise, the result may look like this:

Distance between development of the system and keeping the system running can introduce technical debt as well. The platform a system runs on is a vital part of the system, as critical as the code it supports. As with the code, the design, implementation, and context all need to be kept in alignment.

Alignment of design, implementation, and context can only be maintained by on-going architectural assessment. Stefan Dreverman’s “Using Philosophy in IT architecture” identified four questions to be asked as part of an assessment:

  1. “What is my purpose?”
  2. “What am I composed of?”
  3. “What’s in my environment?”
  4. “What do I communicate?”

These questions are applicable not only to the beginning of a system, but throughout its life-cycle. Failing to re-evaluate the architecture as a whole as the system evolves can lead to inconsistencies as design distance grows. We can get so busy dealing with the present that we create a future of pain:

At first glance, this approach might seem to be expensive, but rewriting legacy systems is expensive as well (assuming the rewrite would be successful, which is a tenuous assumption). Building applications with a one-and-done mindset is effectively building a legacy system.

Square Pegs, Round Holes, and Silver Bullets

Werewolf

People like easy answers.

Why spend time analyzing and evaluating when you can just take some thing or some technique that someone else has already put to use and be done with it?

Why indeed?

I mean, “me too” is a valid strategy, right?

And we don’t want people to get off message, right?

And we can always find a low cost, minimal disruption way of dealing with issues, right?

I mean, after all, we’ve got data and algorithms, and stuff:

The thing is, actions need to make sense in context. Striking a match is probably a good idea in the dark, but it’s probably less so in daylight. In the presence of gasoline fumes, it’s a bad idea regardless of ambient light.

A recent post on Medium, “Design Sprints Are Snake Oil” is a good example. Erika Hall’s title was a bit click-baitish, but as she responded to one commenter:

The point is that the original snake oil was legitimate and effective. It ended up with a bad reputation from copycats who over-promised results under the same name while missing the essential ingredients.

Sprints are legitimate and effective. And now there is a lot of follow-up hype treating them as a panacea and a replacement for other types of work.

Good things (techniques, technologies, strategies, etc.) are “good”, not because they are innately right, but because they fit the context of the situation at hand. Those that don’t fit, cease being “good” for that very reason. Form absent function is just a facade. Whether it’s business strategy, management technique, innovation efforts, or process, there is no recipe. The hard work to match the action with the context has to be done.

Imitation might be the sincerest form of flattery, but it’s a really poor substitute for strategy.

Managing Fast and Slow

Tortoise and Hare Illustration

People have a complicated relationship with the concept of cause and effect. In spite of the old saying about the insanity of doing the same old thing looking for a different result, we hope against hope that this time it will work. Sometimes we inject unnecessary complexity into what should be very simple tasks, other times we over-simplify looking for shortcuts to success. Greger Wikstrand recently spoke to one aspect of this in his post “Cargo cult innovation, play buzzword bingo to spot it” (part of our ongoing conversion on innovation):

I am not saying that there is no basis of truth in what they say. The problem is that innovation is much more complex than they would have you believe. If you fall for the siren song of cargo cult innovationism, you will have all the effort and all the trouble of real innovation work but you will have none of the benefits.

I ran across an interesting example of this kind of simplistic thought not long ago on Forbes, titled “The Death of Strategy”, by Bill Fischer:

Strategy is dead!

Or, is it tactics?

In a world of never-ending change, it’s either one or the other; we can no longer count on having both. As innovation accelerates its assault on what we formerly referred to as “our planning process,” and as S-curves accordingly collapse, each one on top another, time is compressed. In the rubble of what is left of our strategy structure, we find that what we’ve lost is the orderly and measured progression of time. Tim Brown, of IDEO, recently put it this way at the Global Peter Drucker Forum 2016, in Vienna: “So many things that used to have a beginning, a middle and an end, no longer have a middle or an end.” Which is gone: strategy or tactics? And, does it matter?

Without a proper middle, or end, for any initiative, the distinction between strategy and tactics blurs: tactics become strategy, especially if they are performed in a coherent and consistent fashion. Strategy, in turn, now takes place in the moment, in the form of an agglomeration of a series (or not) of tactics.

The pace of change certainly feels faster than ever before (I’m curious, though, as to when the world has not been one of “never-ending change”). However, that nugget of truth is wrapped in layers of fallacy and a huge misunderstanding of the definitions of “tactics” and “strategy”. “Tactical and Strategic Interdependence”, a commentary from the Clausewitzian viewpoint, contrasts the terms in this manner:

Both strategy and tactics depend on combat, but, and this is their essential difference, they differ in their specific connection to it. Tactics are considered “the formation and conduct of these single combats in themselves” while strategy is “the combination of them with one another, with a view to the ultimate object of the War.”[8] Through the notion of combat we begin to see the differentiation forming between tactics and strategy. Tactics deals with the discrete employment of a single combat, while strategy handles their multiplicity and interdependence. Still we need a rigorous conception. Clausewitz strictly defines “tactics [as] the theory of the use of military forces in combat,” while “Strategy is the theory of the use of combats for the object of the War.”[9] These definitions highlight the difference between the means and ends of tactics and strategy. Tactics considers the permutations of military forces, strategy the combinations of combats, actual and possible.

In other words, tactics are the day to day methods you use to do things. Strategy is how you achieve your long term goals by doing the things you do. Tactics without strategy is a pile of bricks without an idea of what you’re going to build. Strategy without tactics is an idea of what to build without a clue as to how you’d build it.

Fischer is correct that strategy executed is the “…agglomeration of a series (or not) of tactics”, but his contention that it “…now takes place in the moment…” is suspect, predicated as it is on the idea that things suddenly lack “…a proper middle or end…”. I would argue that any notion of a middle or end that was determined in advance rather than retroactively, is an artificial one. Furthermore, the idea that there are no more endings due to the pace of change is more than a little ludicrous. If anything, the faster the pace, the more likely endings become as those who can’t keep up drop out. Best of all is the line “…tactics become strategy, especially if they are performed in a coherent and consistent fashion”. Tactics performed in “…a coherent and consistent fashion” is pretty much the definition of executing a strategy (negating the premise of the article).

Flailing around without direction will not result in innovation, no matter how fast you flail. While change is inevitable, innovation is not. Innovating, making “significant positive change”, is not a matter of doing a lot of things fast and hoping for the best. Breakthroughs may occasionally be “happy accidents”, but even then are generally ones where intentional effort has been expended towards making them likely.

In today’s business environment, organizations must be moving forward just to maintain the status quo, much less innovate. This requires knowing where you are, where you’re headed, and what obstacles you’re likely to face. This assessment of your operating context is known as situational awareness. It’s not simple, because your context isn’t simple. It’s not a recipe, because your context is ever-changing. It’s not a product you can buy nor a project you can finish and be done with. It’s an ongoing, deliberate process of making sense of your context and reacting accordingly.

Situational awareness exists on multiple levels, tactical through strategic. While the pace of change is high, the relative pace between the tactical and strategic is still one of faster and slower. Adjustments to strategic goals may come more frequently, but daily changes in long-term goals would be a red flag. Not having any long-term goals would be another. Very specific, very static long-range plans are probably wasted effort, but having some idea of what you’ll be doing twelve months down the road is a healthy sign.

Situational Awareness – Where does it begin? Where does it end?

Infinity symbol

Situational awareness, according to Wikipedia, is defined as “…the perception of environmental elements and events with respect to time or space, the comprehension of their meaning, and the projection of their status after some variable has changed, such as time, or some other variable, such as a predetermined event”. In other words, it’s having a handle on what currently is and what is about to happen. It’s a concept that is invaluable to a wide range of interests, particularly management/leadership, architectural design, and innovation. It’s a concept that crosses levels, from tactical to strategic. Just as socio-technical systems architectures exist in a fractal space (application to solution to enterprise), so too does the concept of situational awareness. As such, it’s a common theme for this site, particularly over the last year or so.

The OODA (Observe-Orient-Decide-Act) Loop, developed by Air Force Colonel John Boyd, is a framework for decision-making that explicitly incorporates situational awareness:

OODA Loop Diagram

Coupling sense-making with decision-making is critical to achieve a balance of both speed and effectiveness. In my opinion, acting without taking the state of the environment into account is a recipe for disaster. Equally important (likewise, in my opinion), is understanding the dynamic nature of situational awareness. As Boyd’s diagram above shows, it’s not a linear process. Additionally, the very nature of a loop should convey the fact that there’s neither beginning nor end. This is a key concept.

One of the sites that I follow is Slightly East of New, which is run by an associate of Boyd’s and dedicated to his theories. A recent post on that site, “The magic of the OODA loop”, related a paragraph from a sci-fi novel, The Apocalypse Codex, that referred to OODA:

Observe, orient, decide, act: words to live or die by. Right now, Persephone is disoriented — on the run, cut off. It’s time to go on the offensive, work out where she is and what’s going on, then get the hell out of this trap.

It was an interesting post, but nothing noteworthy, until I got to this comment:

I find the phrase, “…on the run, cut off.” very interesting, within the context of “disoriented”. To me, “on the run” mean a decision has been made and acted on, whereas “disorientation” usually means that one can’t make a decision.
Likewise, “cut off” is the position you find yourself in, after all the decisions have been made and, after thinking about it, it is the posture you observe yourself to be in.
In other words, on the run and cut off is not really a disorientation, but a reality.
So, while you may not survive, you have made a decision to run or you are about to make a decision and join the otherside.
I suppose it just depends on where those words show up in the narrative, as to if you made the decision or your competitor made the decision for you.

I may be over-sensitive to the phrasing, but “…decision has been made and acted on…” and “…after all the decisions have been made…” strike me as being too static and too linear. Every action/inaction follows on decision/indecision. The point “…after all the decisions have been made…” is terminal (for the person who has made all the decisions they will make). In my opinion, it is key to bear in mind that the clock is always running and that the reality being processed is already past. Too much attention to the state of what is (or rather, was) takes away from the more important task of getting to a better “to be” state. Additionally, decisions and contexts should be thought of as not just linear, but fractal (e.g. having multiple levels from tactical through strategic) as well.

Loops that have an end are no longer loops. Likewise, we have to be able to strike a balance between just focusing on what’s relevant (too much context/backstory can cause information overload) and the point where we’ve trimmed away necessary context.

Actively thinking about sense-making and decision-making can seem overly academic. The activities are so foundational to nearly everything that they can feel instinctual rather than learned. I suspect that’s a case of “familiarity breeds contempt”. Depending on the application, contempt for developing the best possible situational awareness could be fatal.

[OODA Loop diagram by Patrick Edwin Moran via Wikimedia Commons]

Capability Now, Capability Later

Mock tank, British Army in Italy, WWII

In my post “Strategic Tunnel Vision”, I touched on the concept of capability. I discussed how focusing on new capabilities can crowd out existing capabilities and the detrimental effects of that when those existing capabilities are still necessary. I also spoke to how choices about strategic capabilities can trickle down to effect tactical capabilities.

What I failed to do, however, was define what was meant by the term “capability”. That’s a pretty big oversight on my part, because, in my opinion, understanding the concept is critical across all levels of architectural concerns.

Tom Graves, in his “Definitions on capability”, defines the term (along with some related concepts):

— Capability: the ability to do something.

— Capability-based planning: planning to do something, based on capabilities that already exist, and/or that will be added to the existing suite of capabilities; also, identifying the capabilities that would be needed to implement and execute a plan.

— Capability increment: an extension to an existing capability; also, a plan to extend a capability.

— Capability map: a visual and/or textual description of (usually) an organisation’s capabilities.

Yes, I do know that those definitions are terribly bland and generic – and they need to be that way. That’s the whole point: they need to be generic enough to be valid and usable at every possible level and in every possible context – otherwise we’ll introduce yet more confusion to something that’s often way too confused already.

That last paragraph is critical. The concept of “capability” is a high-level one that is useful across multiple levels of architectural concern (ie. application, solution, enterprise IT, and the enterprise itself). Quoting Tom again:

Note what else is intentionally not in that definition of ‘capability‘:

  • there’s no actual doing – it’s just an ability to do something, not the usage of that ability
  • there’s no ‘how’ – we don’t assume anything about how that capability works, or what it’s made up of
  • there’s no ‘why‘ – we don’t assume any particular purpose
  • there’s no ‘who‘ – we don’t assume anything about who’s responsible for this capability, or where it sits in an organisational hierarchy or suchlike

We do need all of those items, of course, as we start to flesh out the details of how the capabilities would be implemented and used in real-world practice. But in the core-definition itself, we very carefully don’t – they must not be included in the definition itself.

The reason why we have to be so careful and pedantic about this is because the relationship between service, capability, function and the rest is inherently recursive and fractal: each of them contains all of the others, which in turn each contain all of the others, and so on almost to infinity. If we don’t use deliberately-generic definitions for all of those items, we get ourselves into a tangle very quickly indeed – as can be seen all too easily in the endless definitional-battles about the relationships between ‘business-function’ versus ‘business-process’ versus ‘business-service’ versus ‘business-capability’ and so on.

In short, it’s a crucial building block in our designs and plans (which is redundant, since design is a form of planning). If we don’t have and can’t get the ability to do something, it’s game over. However, as Tom noted, we need to move beyond the raw ability in order to make effective use of capabilities. We need to think timing and personnel (which will probably largely drive timing anyway). A capability later may well not be as valuable as the same capability right now.

This was brought to mind while skimming a book review on a military strategy site (emphasis added by me):

In March 2015, then-Chief of Staff of the U.S. Army General Raymond T. Odierno admitted to the British newspaper The Telegraph that the so-called special relationship between the United States and Great Britain isn’t what it used to be. “In the past we would have a British Army division working alongside an American army division,” he said, but he feared that in the future British battalions and brigades would have to operate “inside” American units. “What has changed,” Odierno declared, “is the level of capability.”

Later that week, I asked a senior British general about Odierno’s remarks. He replied, deadpan, that although Odierno’s candor was appreciated, his statement was factually incorrect. “We can still field a division,” the general insisted. “It is just a question of how long it takes us to field one.” Potential tanks, he seemed to think, were just as relevant as an actual ones.

The highlighted portion of the quote illustrates my point. Having the capability to do something immediately and the capability to do that same thing at some point in the future are not equivalent (just to be fair to the British Army, the US Army was in this same position during Operation Desert Shield – the initial ground forces that could be deployed were extremely thin). Treating them as equivalent potentially risks disaster.

It should be noted, however, that level of concern will color the perception of the value of a future capability versus a current one. At the tactical level, in business as well as in war, “…first with the most…” is likely a winning move. At the strategic level, however, where resources must be budgeted across multiple initiatives, priorities should dictate which capabilities get preference. Tactical leaders may have to be satisfied with “on time with just enough”.

Regardless of level, a clear assessment of capabilities, what’s available when, is key to making effective decisions.

Strategic Tunnel Vision

Mouth of a Tunnel

 

Change and innovation are topics that have been prominent on this blog over the last year. In fact, Greger Wikstrand and I have traded a total of twenty-six posts (twenty-seven counting this one) on the subject.

Greger’s last post, “Successful digitization requires focus on the entire customer experience – not just a neat app” (it’s in Swedish, but it translates well to English), discussed the critical nature of customer experience to digital innovation. According to Greger, without taking customer experience into account:

One can make the world’s best app without getting more, more satisfied and profitable customers. It’s like trying to make a boring games more exciting by spraying gold paint on the playing pieces.

Change and innovation are not the same thing. Change is inevitable, innovation is not (with a h/t to Tom Cagley for that quote). As Greger pointed out in his latest article, to get improved customer experience, you need depth. Sprinkling digital fairy dust over something is not likely result in innovation. New and different can be really great, but new and different solely for the sake of new and different doesn’t win the prize. Context is critical.

If you’ve read more than a couple of my posts, you’ve probably realized that among my rather varied interests, history is a major one. I lean heavily on military history in particular when discussing innovation. This post won’t break with that tradition.

The blog Defense in Depth, operated by the Defence Studies Department, King’s College London, has published two posts this week dealing with the Suez Crisis of 1956, primarily in terms of the Anglo-French forces. One deals with the land operations and the other with naval operations. They struck a chord because they both illustrated how an overreaction to change can have drastic consequences from the strategic level down to the tactical.

Buying into a fad can be extremely expensive.

The advent of the nuclear age at the end of World War II dramatically transformed military and political thought. The atomic bomb was the ultimate game-changer in that respect. In the time-honored tradition, the response was over-reaction. “Atomic” was the “digital” of the late 40s into the 60s. They even developed a recoilless gun that could launch a 50 pound nuclear warhead 1.25-2.5 miles. “Move fast and break things” was serious business back in the day.

This extreme focus on what had changed, however, led to a rather common problem, tunnel vision. Nuclear capability became such an overarching consideration that other capabilities were neglected. Due to this neglect of more conventional capabilities, the UK’s forces were seriously hampered in their ability to perform their mission effectively. Misguided thinking at the strategic level affected operations all the way down to the lowest tactical formations.

It’s easy to imagine present-day IT scenarios that fall prey to the same issues. A cloud or digital initiative given top priority without regard to maintaining necessary capabilities could easily wind up failing in a costly manner and impairing the existing capability. It’s important to understand that time, money, and attention are finite resources. Adding capability requires increasing the resources available for it, either through adding new resources or freeing up existing ones by reducing the commitment to less important capabilities. If there is no real appreciation of what capabilities exist and what the relative value of each is, making this decision becomes a shot in the dark.

Situational awareness across all levels is required. To be effective, that awareness must integrate changes to the context while not losing sight of what already was. Otherwise, to use a metaphor from my high school football days, you risk acting like a “blind dog in a meat-packing plant”.

Leadership Patterns and Anti-Patterns – The Growler

Grizzly Bear Attack Illustration

Prior to starting my career in IT (twenty years ago this month…seems like yesterday), I spent a little over eleven years in law enforcement as a Deputy Sheriff. Over those eleven years my assignments ranged from working a shift in the jail (interesting stories), to Assistant Director of the Training Academy, then Personnel Officer (even more interesting stories), and finally, supervisory and management positions (as many headaches as stories). To say that it was as much an education as a job is to put it mildly. I learned useful lessons about human nature and particularly about leadership.

One of the things that I learned is that leadership and management (they are related, but separate things) have patterns and anti-patterns associated with them. Just like in the realm of software development, it can be difficult to distinguish between what’s a pattern and what’s an anti-pattern (there’s an interesting discussing to be found on this topic in the classic “Big Ball of Mud”). Hammering a square peg into a round hole “works”, albeit sub-optimally. Pattern or anti-pattern?

One pattern/anti-pattern from my time with the Sheriff’s Office is what I call “The Growler”. A high-ranking member of the department was a master of this technique. When approached for something, particularly when the something in question was a signature on a purchase requisition, the default response was a profanity-laced growl (the person in question had retired from the Navy as a senior NCO) demanding to know why he should grant the request. This was extremely daunting, but I learned that the correct response was to growl back. When he growled, “%$@$ a !#&^ing $@!#*. More $%&^ing computer stuff, why the @#*& do you need this?”, I would answer, “You know when you ask me a question and I respond in five minutes instead of three hours”. This would result in a shake of his head, a “Yeah, yeah”, and most importantly, a signature.

More than just an endearing quirk of his character, it was a triage technique. If the person who wanted something tucked tail and ran, it wasn’t important. If, however, the person stood their ground, then he would put forth the effort to make a decision.

Right up front, I should make it clear that I don’t recommend this technique. First and foremost, Human Resources finds “salty” language even less endearing today than they did twenty-five plus years ago, and they weren’t crazy about it then. There’s also a big problem in terms of false negatives.

Most of my coworkers back in my badge and gun days were not shy, retiring types. Consequently, I never saw it backfire for that person. Later on, though, I did see it fail for an IT manager (and yes, while gruff, he was significantly less “salty” than the one at the Sheriff’s Office). This manager had a subordinate who would retreat no matter how valid the need. Consequently, that subordinate’s unit, one that several of us were dependent on, was always under-staffed and under-equipped. When his people attended training, it was because someone else had growled back for him. It was far from the optimal situation.

While not quite as bad as the “shoot the messenger” anti-pattern I touched on recently, “The Growler” comes close. By operating on a principle of fear, you can introduce a gap in your communications and intelligence network that you rely on (whether you know it or not) to get the information you need in a timely manner.

Fear encourages avoidance and no news now can be very bad news later.