Form Follows Function on SPaMCast 471

SPaMCAST logo

It’s time for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

Last week’s episode, number 471, features Tom’s essay on the top 20 transformation killers. Jeremy Berriault‘s QA corner is about involving testers in the requirements process. My Form Follows Function segment rounds out the podcast, covering my post “Systems Thinking Complicates Things”.

In this installment, Tom and I talk about how simplistic analysis is unlikely to fit a complex problem. We illustrate this by talking about the game of “rock, paper, scissors” (then graduate to “rock, paper, scissors, lizard, Spock”!). I don’t really think that’s what’s meant by game theory, but hey, it’s fun (and illustrative) nonetheless.

You can find all the SPaMCast episodes I’m in under the SPaMCast Appearances category on this blog. Enjoy!

Form Follows Function on SPaMCast 467

SPaMCAST logo

It’s time for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

This week’s episode, number 467, features Tom’s excellent essay on value (value is one of those simple-seeming, but complex concepts). Jeremy Berriault‘s QA corner covers testing in difficult circumstances. I bat cleanup with a Form Follows Function segment discussing my post “Management, Simple and Wrong – Semantics, Systems, and Self-Correction”.

In this installment, Tom and I talk about a post that came about, not intentionally, but because I got into a conversation on Twitter that played out along the lines of the post I really intended to write (dealing with organizations as complex, multi-level systems). When life hands you your topic, it’s generally a good idea to chase it to the conclusion!

You can find all the SPaMCast episodes I’m in under the SPaMCast Appearances category on this blog. Enjoy!

Form Follows Function on SPaMCast 463

SPaMCAST logo

I’m back for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

This week’s episode, number 463, features Tom’s essay on big picture stories. This is followed by our Form Follows Function segment discussing my post “Management, Simple and Wrong – Semantics, Systems, and Self-Correction”. Jeremy Berriault‘s QA corner finishes the cast with a segment on motivating testers.

In this installment, Tom and I talk about the way people approach complex (and often emotionally charged) subjects like management. Semantics, defining what we mean, is critical to keep discussions on track. Likewise, we need to be able to differentiate between a concept, differing theories around that concept, and just plain poor practice. Simplistic mental models are likely to generate much more heat than light. It’s not often that something good starts off “I got into a discussion on Twitter”, but this episode is the exception to the rule!

You can find all the SPaMCast episodes I’m in under the SPaMCast Appearances category on this blog. Enjoy!

Form Follows Function on SPaMCast 459

SPaMCAST logo

I’m back for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

This week’s episode, number 459, features Tom’s essay on resistance to change. This is followed by our Form Follows Function segment discussing my post “Innovation, Intention, Planning and Execution”. Jeremy Berriault‘s QA corner finishes the cast with a segment on testing packaged software.

In this installment, Tom and I talk about effectiveness, particularly the relationship between effectiveness and reasoned, intentional action. In short, organizations are (social) systems, and “things work better when they work together on purpose”. You can’t create serendipity, but if you want to be able to exploit what serendipity drops in your lap, you need to prepare the ground ahead of time.

You can find all the SPaMCast episodes I’m in under the SPaMCast Appearances category on this blog. Enjoy!

Systems Thinking Complicates Things

4th UK Rock Paper Scissors Championships by James Bamber via Wikimedia

 

I’ve had the honor and pleasure of appearing as a regular on Tom Cagley‘s SPaMCast podcast for almost three years now. Before I write one of my “Form Follows Function on SPaMCast x” posts, I always listen to the podcast to make sure that the summary is right (the implication being, relying purely on my memory won’t be right). I got a bonus while writing up last week’s appearance, because Tom asked an excellent question that deserved its own post: does thinking about a problem (legacy systems, in the instance of last week’s discussion) holistically/systematically complicate things?

Abso-freakin’-lutely.

It is much easier to avoid all the twists and turns and possibilities inherent in systems thinking. A simpler approach, picking one lever to pull/one button to push, makes it much easier to come up with a solution.

It just doesn’t work very well at coming up with solutions that actually work.

When there is a mismatch in complexity between problem and solution architectures, this mismatch will be an additional problem to deal with. This will apply when the solution is more complex than the problem space warrants and when the opposite is the case. Solutions that fail to account for the context they will encounter are vulnerable. This is the idea behind the quote attributed to Albert Einstein: “Everything should be made as simple as possible, but not simpler.”

Human nature can push us to fix problems quickly, and quick will generally equate to simple. It takes time to analyse the angles and consider the alternatives. How often have you seen people ask for “the best” way to do something absent any context? How often have you seen people ask “why would someone ever do that?”

I’ll answer that by asking 3 questions:

  • since Rock beats Scissors, why would anyone ever choose Scissors?
  • since Paper beats Rock, why would anyone ever choose Rock?
  • since Scissors beats Paper, why would anyone ever choose Paper?

Reality isn’t binary. It’s not what’s “best”, it’s what’s fit for purpose in a given context and there are lots and lots of contexts out there.

This isn’t to say that all quick, simple interventions are wrong. If you find yourself in a house fire, more action and less comprehensive deliberation may well be in order. The key is matching the cost (largely in terms of time) of defining the problem space with cost (in terms of both effort and risk that the intervention adds to the problem) of crafting the solution.

Rock, Paper, Scissor, Lizard, Spock rules diagram

It’s almost guaranteed that the system contexts we deal with (both technical and social) will evolve toward more and more complexity. Surprises will emerge as a matter of course. We don’t need to make more by failing to take a more holistic view when we have the time to do so.

Form Follows Function on SPaMCast 454

SPaMCAST logo

I’m back for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

This week’s episode, number 454, begins with Tom talking about iteration planning. Jeremy Berriault comes next with a segment on QA team leads and I bat cleanup with a Form Follows Function installment based on my post “Trash or Treasure – What’s Your Legacy?”.

Tom and I discuss the concept of legacy systems: what they are, whether they’re trash or treasure (sometimes both), and how they impact an organization.

You can find all the SPaMCast episodes I’m in under the SPaMCast Appearances category on this blog. Enjoy!

Microservices or Monoliths – Fences and Neighbors

Photo of fence separating fields from a road

 

At the end of my last post, “What Makes a Monolith Monolithic?”, I stated that I didn’t consider the term “monolithic” to be inherently derogatory. It is, rather, a descriptive term relating to the style of organizing an application’s architecture. Depending on the context the system operates within, a monolithic architectural style could lie anywhere on the continuum between perfectly suited and perfectly disastrous. Placing it on that continuum requires a sense of what qualities are most needed or desired and which can be traded off in their stead. Everything comes with a cost, and attempting to ignore that fact merely sets us up for unpleasant future surprises.

After an initial period of unbridled enthusiasm, opinion seemed to gel around the idea that highly distributed application architectures (aka microservice architectures) were not suitable to all contexts. There are prerequisites for jumping into the microservices pool in terms of problem architecture, infrastructure, and organization. Attempting to shoehorn a microservice architecture into an environment that cannot support it will be overly expensive at best and a failure of apocalyptic proportions at worst.

There are many aspects of application design that are commonly recognized as beneficial: modularity, loose-coupling, high cohesion, and separation of concerns. It is critical to realize that these aspects can be found in systems with microservice architectures, monolithic systems, and everything in between. Distributed architectures are not necessary for modularity, nor any of those other aspects. In fact, one could easily create an application with a microservice architecture whose qualities are opposite to these desirable ones.

There are, however, situations where the benefit of a microservice architecture outweighs the costs and complexity. The ability to independently deploy and scale the various parts of an application is a major benefit, in my opinion. A well designed microservice architecture can even allow for the components of an application to be replaced on the fly. These features are not unique to microservice architectures, but are arguably easier to achieve than in other application architectures.

Real design, balancing both costs and benefits, is required. Sticking a bit of network in between the components is insufficient to ensure success. Deliberate design, especially as the boundaries multiply, is critical for an effective system. Identifying and providing for those boundaries at the conceptual level (i.e. before they become physical) is key. Good fences can either make for good neighbors, or they can create a maze of barriers.

What Makes a Monolith Monolithic?

Photo of Stonehenge, 1877

 

It seems like everybody throws around the term “monolith”, but what do we mean by that?

Sam Newman started the ball rolling yesterday with this tweet:

My first response was a (semi) joke:

I say semi joke because, in truth, semantics (i.e. meaning) is critical. The English language has a horrible tendency to overload terms as it is, and in our line of work we tend to make it even worse. Lack of specificity obscures, rather than enlightens. The problem with the term “monolith” is that, while it’s a powerfully evocative term, it isn’t a simple one to define. My second response was closer to an actual definition:

The purpose of this post is to expand on that a bit.

The “mono” portion of the term is, in my opinion, the crucial part. I believe that quality of oneness is what defines a monolithic system. As I noted in the second tweet, it’s a matter of meta-coupling, whether that coupling exists in the form of deployment, data architecture, or execution style (Jeppe Cramon‘s post “Microservices: It’s not (only) the size that matters, it’s (also) how you use them – part 3” shows how temporal coupling can turn a distributed system into a runtime monolith). The following tweets between Anne Currie and Sam illustrate the amorphous nature of what is and isn’t a monolith:

Modules that can be deployed to run in a single process need not be considered monolithic, if they’re not tightly coupled. Likewise, running distributed isn’t a guarantee against being monolithic if the components are tightly coupled in any way. The emphasis on “in any way” is due to the fact that any of the types of coupling I mentioned above can be a deal killer. If all the “microservices” must be deployed simultaneously for the system to work, it’s a distributed monolith. If the communication is both synchronous and fault intolerant, it’s a distributed monolith. If there’s a single data store backing the entire system, it’s a distributed monolith. It’s not the modularity that defines it (you can have a modular monolith), but the inability to separate the parts without damaging the whole system.

I would also point out that I don’t consider “monolithic” to be derogatory, in and of itself. There is a trade-off involved in terms of coupling and complexity (and cost). While I generally prefer more flexibility, there is always the danger of over-engineering. If we’re hand-carving marble gargoyles to stick on a tool shed, chances are the customer won’t be pleased. The solution should bear at least a passing resemblance to the problem context it’s supposed to address.

Organizations as Systems – Kurosawa, Clausewitz, and Chess

16th Century Market Scene

In order to respond appropriately to the context we find ourselves in, it’s helpful that we be able to correctly define that context. It’s something humans aren’t always good at.

Not too long ago, Sun Tzu’s The Art of War was all the rage as among executives. While the book contains some excellent lessons that have applications beyond the purely military, as someone in my Twitter feed noted recently, “Business is not war”.

[Had I realized that the tweet, in combination with another article, would trigger something in my byzantine thought processes, I would have bookmarked it to give them credit – sorry!]

Business is, indeed, not war. In fact, one of the nuggets of wisdom to be found in Clausewitz’s treatise, On War, is that war is often not war. Specifically, what he is saying is that the reality of a concept often diverges from our (mis)understanding of that concept. Our perception is colored by factors such as our experience, beliefs, and interests. Additionally, our tendency to employ abstraction can be both tool and trap. Ignoring irrelevant detail can simplify reasoning about something, assuming that the detail ignored is actually irrelevant. Ignoring relevant detail can quickly lead to problems.

The game of chess illustrates this. Chess involves strategy and has its origins as an abstract simulation of war. Beyond promoting a very rudimentary type of strategic thought, chess is far from capable of simulating the complex social system of warfare. Perhaps if all the pieces were sentient and had both agency and agenda (bonus points for contradictory ones potentially conflicting with the player’s agenda), it might come closer. Perhaps if the boundaries of the arena were indeterminate, it might come closer. Perhaps if the state of the terrain, the composition and disposition of forces (friend, as well as foe), and the goals of the opponent were less transparent, it might come closer.

In short, the more certainty there is, the less accuracy there is. Where the human aspect is ignored or minimized, you may gain certainty, but it comes at the cost of losing contact with reality. Social systems are highly complex and treating them otherwise is like looking for a gas leak with a lighter – you may be able to do so, but your chances of liking the results are pretty small.

This post was originally planned to be for last week, but I stumbled into a Twitter conversation that illustrates my point (specifically re: leadership and management), so I wrote that first as a preamble. Systems of practice designed for a context where value equals effort expended are unlikely to work well in a knowledge work context where the relationship between effort and value is less direct (where, in fact, the value curve may invert past a certain point). Putting an updated veneer on the technique with data and algorithms won’t improve the results if the technique is fundamentally mismatched to the context (or if there is a disconnect between what you can measure and what you actually want). Sometimes, the most important thing to learn about management is when not to manage.

Disconnects between complex contexts and simplistic practices transcend the management of an organization, reaching into the very architecture of the enterprise itself (both in the organization and its relationship to its ecosystem). Poorly designed organizations (which includes those with no intentional design) can wind up with their employees faced with perverse incentives to act in a manner that conflicts with the best interests of the organization. When the employee is actually under pressure from the organization to sabotage the organization, the problem is not with the employee.

Just as with a software system, social systems have both problem and solution architectures. Likewise, in both cases the quality of the solution architecture is dependent on how well (or not) it addresses the architecture of the problem. Recognizing the various contexts in play and then resolving the conflicts between them (to include resolving challenges arising from the resolution of the original conflicts) is the essence of architectural design, regardless of the type of system (software or social). Rather than a static, one time activity, it is an ongoing need for sensing system health and responding appropriately throughout the lifecycle of the system (in fact, stopping the process will likely hasten the end of the lifecyle by way of achieving a state where the system cannot be corrected).

Management, Simple and Wrong – Semantics, Systems, and Self-Correction

Villain Caricature

Simple responses to complex situations are both seductive and dangerous. The difficulty in juggling lots of variables tempts us to employ abstraction so as to avoid being overwhelmed. Abraham Maslow’s observation, “I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail”, applies. Some things (e.g. landmines) react badly to being treated as if they were nails. Having more tools in the box may help avoid problems.

This isn’t the post I had in mind to write next, but it’s one that came about by accident (via a multi-day mass participant Tweet-storm, with my participation beginning here). I had planned an Organizations as Systems post re: multiple players in multiple contexts (competing, and possibly conflicting goals and motivations) and I stumbled into a conversation that should provide a nice preamble to that post which should follow this one.

Before I dive in, two quick notes:

  • Rather than try to summarize the entire conversation, I’m going to lay out what I brought to and took from it. There are far too many tweets and, as of this writing, I can’t be sure the conversation has concluded.
  • My thanks to everyone involved, whether named or not. This kind of civil, if contentious, dialog is much appreciated. When ideas rub together, it can produce irritation, but sometimes they also get polished.

Management is one of those things that, like landmines, tends to react badly to the hammer of simplistic thought. We can see this in managers who apply (or misapply) theories of management, particularly ones like scientific management (AKA Taylorism) to contexts where it is extremely inappropriate and counterproductive. Whether there really exists a context where Taylorism is actually appropriate or productive is a question for another day. We also see the hammer in reactions to abuses that dismiss all value of management out of hand. While the reaction is understandable, that doesn’t make it credible. The vicious circle just becomes more vicious; heat is generated but without corresponding light.

One thing that’s necessary to pin down is what we mean by the term “management”. Are we talking about a concept (“…the administration of an organization…”)? Are we talking about the job/profession? Perhaps the discipline (branch of knowledge) or academic discipline (field of study) is what we’re talking about. We could be talking about a theory management, or we could be talking about management practices, either individually or grouped into systems of management. Knowing specifically what’s being referred to is critical for evaluating statements. A very valid criticism of a specific theory or system (e.g. Taylorism) will likely fall apart when applied to the concept as a whole due to the fact that the concept is far broader and contrary examples are easily found.

Another issue relates to intent. Few would argue the universal detriment of poor management practices. Extracting the maximum possible effort from your employees is unlikely to result in the generation of the most value in the context of knowledge work. These practices are the very antithesis of fitness for purpose in that they do not materially benefit the organization and they alienate employees (which is yet another hit to the organization where the product is knowledge work). And yet, there are still managers that manage in that very manner. Are they, each and every one, evil? A simplistic answer, hard against either end of the spectrum, is almost surely going to be wrong. That being said, in my experience the distribution is skewed more towards the “no” side than not (just as I’ve found people who only perform when driven to it to be a very small minority).

Why would someone who wants to do their job well and in an ethical manner resort to practices that are harmful to all parties? With sadism eliminated as a motivation (there just aren’t enough in the population to account for all the positions to be filled), the far more plausible answer would be culture, tradition, and/or lack of knowledge regarding alternatives. In short, when the outcome of a system doesn’t match the intent, there’s a bug in the system.

The disconnect between leadership and management is also a problem. Leadership, admittedly, is a concept distinct from management. It makes sense that not every leader needs to be a manager. The extent to which we as a society tolerate management absent leadership, however, is shocking and part of the problem. Consider a tweet from Esther Derby:

I would argue that steering and enabling can be considered leadership qualities as much as management activities. There’s a place for supervision and compliance, however knowing how to achieve results without forcing the issue is, in my opinion, an extremely useful skill. This is not manipulation, rather a matter of understanding goals and how to achieve them intelligently. It’s a matter of understanding how to resolve potential conflicts between the goals and motivations of an organization, groups, and individuals and adapting the system so that the outcomes more closely track the intent. The alternative is allowing the system to degenerate into a web of perverse incentives that increase the gap between intent and outcome. This gap may benefit some individuals, but at the cost to other individuals and the organization as a whole.

Medicine is something that has been through a number of changes, large and small, by finding a way to adapt. While the concept of medicine (diagnosis, treatment, and prevention of disease and injury) has remained constant over time, the practices and theories have evolved greatly. The discipline itself has evolved so that not only does it adapt to change, but that it adapts in as optimal a manner as possible. In short, it has developed a culture of learning.

Understanding organizations from a systems standpoint means recognizing the need for sensing the fit between the system and its contexts (learning) and then steering to correct any mismatches (management). Simplistic approaches to management (particularly relatively static ones that have little save tradition to recommend them) can only lead to a widening gap between the intended outcome and actual results. At some point, this gap becomes wide enough to swallow the organization.

[Villain Carricature by J.J. via Wikimedia Commons.]