Abuse Cases – What Could Go Wrong?

Last week, in a post titled “The Flaw in All Things”, John Vincent discussed the problem of seeing “the flaw in all things”:

It’s overwhelming. It’s paralyzing.

I can’t finish a project because I keep finding things that could cause problems. I even mentioned this to our CTO and CEO at one point when we were trying to size some private deploys of our stack.

I couldn’t see anything but the largest configuration because all I could see was places where there was a risk. There were corners I wasn’t willing to cut (not bad corners like risking availability but more like “use a smaller instance here”) because I could see and feel and taste the pain that would come from having to grow the environment under duress.

I’m frustrated with putting everything in Docker containers because all I see is having to take down EVERYTHING running on one node because there’s may be a critical Docker upgrade. I see Elasticsearch rebalancing because of it. I see Kafka elections. Mind you the system is designed for this to happen but why add something that makes it a regular occurance?

I can certainly sympathize. For what it’s worth, it sounds like those making the trade-offs he’s worried about could stand to be a bit more inclusive. That doesn’t necessarily mean the decisions would change, but at least being heard and knowing the answers to his questions might reduce some of the stress (not to mention perhaps helping out those responsible for the decisions).

When making design decisions, having (or, at least, having access to) this level of knowledge and experience has a great deal of value. As I noted in “NPM, Tay, and the Need for Design”, you have to consider both the use cases and the abuse cases for a given system (whether software or social).

It’s not possible to foresee every potential flaw and it probably won’t be feasible to eliminate every risk that’s discovered. That being said, it doesn’t mean time spent in risk evaluation is wasted. Dealing with foreseeable issues before they become problems (where “dealing” is defined as either mitigating it outright or at least planning for a response should it occur) will work better than figuring it out on the fly when the problem occurs.

Understanding why “Should I?” is a more important question than “Can I?” is something I’ve touched on before. Snapchat is finding this out by way of a lawsuit over their filter that allows users to record their speed. Who would have thought someone might cause an accident using that?

Trial and error/experimenting is one method of learning, but it’s not the only method and is frequently not the best method. Fear of failure can hold back learning, but a cavalier attitude toward risk can make experiments just as, if not more costly. It’s the difference between testing a 9 volt battery by touching it to your tongue and using the same technique on a 240 volt circuit.

Advertisement

Abstract Dangers – When ‘And’ Meets ‘Or’

There’s an old saying that if you put one foot in a bucket of ice and the other in a bucket of boiling water, on average you’re comfortable. Sometimes analyzing information in the aggregate obscures rather than enlightens.

A statistician named Francis Anscombe pointed out this same principle in a more visual (though less colorful) manner more than forty years ago:

Just your daily reminder that #averagesarebad en.wikipedia.org/wiki/Anscombe%… all graphs have same mean, median and variance https://t.co/WcWHzMvf3C

—
Matt Warren (@matthewwarren) April 19, 2016

It’s an idea that I’ve been meaning to write about for a while, but was brought back to mind last week while reading an article on the Austrian school of economic theory posted on a site about medical practice and health care in the U.S. (diversity of interests and a very broad reading list is something I find useful, but that’s a topic for another day). The relevant passage:

When Ludwig von Mises began to establish a systematic theory of economics, he insisted on what he called the principle of methodological dualism: the scientific methods of the hard sciences are great to study rocks, stars, atoms, and molecules, but they should not be applied to the study of human beings. In stating this principle, he was voicing opposition to the introduction into economics of concepts such as “market equilibrium,” which were largely inspired by the physical sciences, and were perhaps motivated by a desire on the part of some economists to establish their field as a science on par with physics.

Mises remarked that human beings distinguish themselves from other natural things by making intentional (and usually rational) choices when they act, which is not the case for stones falling to the ground or animals acting on instinct. The sciences of human affairs therefore deserve their own methods and should not be tempted to apply the tools of the physical sciences willy-nilly. In that respect, Mises agreed with Aristotle’s famous dictum that ” It is the mark of an educated man to look for precision in each class of things just so far as the nature of the subject admits.”

I find myself agreeing and disagreeing with this at the same time. Human behavior is far from being as predictable as gravity and I agree with this for exactly the reason I disagree (at least in part) with the second paragraph. It is a mistake to characterize human action as intentional and rational. That’s not to say that all our choices are irrational and reactionary, but that there is a blend. Not only will different people respond in different ways to the same circumstance for different reasons, but the same person may react differently with different motivations on another occasion. Human nature isn’t rigidly deterministic and we consider it so at our peril.

Tom Graves post “Control, complex, chaotic” makes the same observation:

Attempting to ‘control’ complexity just doesn’t work: we need to treat the complex as complex, not as a ‘controllable problem for which we don’t quite know all the rules (but will know them all Real Soon Now, honest…)’.

Yet I’m also noticing another deeper problem: misguided attempts to apply complexity-theory to things that are neither rule-bound control nor pattern-based complexity, but are inherently ‘chaotic’ – a ‘market-of-one‘. Although we can identify definite patterns in health and health-care – that’s the whole basis of epidemiology, for example – neither rules nor statistics can help us deal with the blunt fact the everyone is different. The kind of patterns that we’d use in a complexity-model – probabilities, Bell-curve distributions, outliers, all that kind of thing – can all too easily mask the real underlying fact of uniqueness, from which that supposed ‘pattern’ will actually arise: somewhat like the barely-visible deep-randomness that underlies the visible patterns of Brownian-motion.

Trying to force something into a mold which it doesn’t fit is unlikely to work well.

Abstraction can be useful in understanding the contexts that influence the architecture of the problem. Designing an effective solution, however, will involve not just integrating the concerns of those contexts, but also dealing with any emergent challenges. The variability of human nature (in other words, sometimes the members of those contexts will not all think and act alike) can be one such emergent challenge.

Tom Grave’s again, this time from his “On mass-uniqueness”:

In practice, the scope of every system will comprise a mix of sameness and uniqueness – of predictable and unpredictable, certain and uncertain. If we design only on an assumption of sameness – as IT-systems often are – we set ourselves up for guaranteed failure. The same applies if – as is all too common – we say that our IT-system will handle all of the ‘sameness’ part of the context, and that the ‘not-sameness’ will Somebody Else’s Problem – without giving any means for that supposed ‘somebody else’ to be able to address the rest of the problem, or to link it up with the parts of the context that our system does handle.

The first requirement to make something that works in the real-world is to design for uniqueness, not against it.

In other words, a solution based on a poorly understood problem is unlikely to be a good fit. Abstraction is one tool to understand the problem, but doesn’t provide the whole picture. Shades of gray (black and white) is more likely than black or white.

Ignorance Isn’t Bliss, Just Good Tactics

There’s an old saying about what happens when you assume.

The fast lane to asininity seems to run through the land of hubris. Anshu Sharma’s Tech Crunch article, “Why Big Companies Keep Failing: The Stack Fallacy”, illustrates this:

Stack fallacy has caused many companies to attempt to capture new markets and fail spectacularly. When you see a database company thinking apps are easy, or a VM company thinking big data is easy  — they are suffering from stack fallacy.

Stack fallacy is the mistaken belief that it is trivial to build the layer above yours.

Why do people fall prey to this fallacy?

The stack fallacy is a result of human nature  — we (over) value what we know. In real terms, imagine you work for a large database company  and the CEO asks , “Can we compete with Intel or SAP?” Very few people will imagine they can build a computer chip just because they can build relational database software, but because of our familiarity with building blocks of the layer up,  it is easy to believe you can build the ERP app. After all, we know tables and workflows.

The bottleneck for success often is not knowledge of the tools, but lack of understanding of the customer needs. Database engineers know almost nothing about what supply chain software customers want or need.

This kind of assumption can cost an ISV a significant amount of money and a lot of good will on the part of the customer(s) they attempt to disrupt. Assumptions about the needs of the customer (rather than the customer’s customer) can be even more expensive. The smaller your pool of customers, the more damage that’s likely to result. Absent a captive customer circumstance, incorrect assumptions in the world of bespoke software can be particularly costly (even if only in terms of good will). Even comprehensive requirements are of little benefit without the knowledge necessary to interpret them:

"The road to hell is paved with clear requirements and poor domain knowledge." - @ibondare

—
Brian Marick (@marick) April 07, 2015

But, that being said:

increasingly of the view that domain expertise is a hinderance not an advantage when you’re doing service design & user research.

—
Leisa Reichelt (@leisa) February 13, 2015

This would seem to pose a dichotomy: domain knowledge as both something vital and an impediment. In reality, there’s no contradiction. As the old saying goes, “a little knowledge is a dangerous thing”. When we couple that with another cliche, “familiarity breeds contempt”, we wind up with Sharma’s stack fallacy, or as xkcd expressed it:

In order to create and evolve effective systems, we obviously have a need for domain knowledge. We also have a need to understand that what we possess is not domain knowledge per se, but domain knowledge filtered through (and likely adulterated by) our own experiences and biases. Without that understanding, we risk what Richard Martin described in “The myopia of expertise”:

In the world of hyperspecialism, there is always a danger that we get stuck in the furrows we have ploughed. Digging ever deeper, we fail to pause to scan the skies or peer over the ridge of the trench. We lose context, forgetting the overall geography of the field in which we stand. Our connection to the surrounding region therefore breaks down. We construct our own localised, closed system. Until entropy inevitably has its way. Our system then fails, our specialism suddenly rendered redundant. The expertise we valued so highly has served to narrow and shorten our vision. It has blinded us to potential and opportunity.

The Clean Room pattern on CivicPatterns.org puts it this way:

Most people hate dealing with bureaucracies. You have to jump through lots of seemingly pointless hoops, just for the sake of the system. But the more you’re exposed to it, the more sense it starts to make, and the harder it is to see things through a beginner’s eyes.

So, how do we get those beginner’s eyes? Or, at least, how do we get closer to having a beginner’s eyes?

The first step is to reject the notion of our own understanding of the problem space. Lacking innate understanding, we must then do the hard work of determining what the architecture of the problem, our context, is. As Paul Preiss noted, this doesn’t happen at a desk:

Architecture happens in the field, the operating room, the sales floor. Architecture is business technology innovation turned to strategy and then executed in reality. Architecture is reducing the time it takes to produce a barrel of oil, decreasing mortality rates in the hospital, increasing product margin.

Being willing to ask “dumb” questions is just as important. Perception without validation may be just an assumption. Seeing isn’t believing. Seeing and validating what you’ve seen, is believing.

It’s equally important to understand that validating our assumptions goes beyond just asking for requirements. Stakeholders can be subject to biases and myopic viewpoints as well. It’s true that Henry Ford’s customers would probably have asked for faster horses, it’s also true that, in a way, that’s exactly what he delivered.

We earn our money best when we learn what’s needed and synthesize those needs into an effective solution. That learning is dependent on communication unimpeded by our pride or prejudice:

Cross-functional comms is challenging: be a beginner when discussing the area of the other. Ask questions, learn their language & world.

—
Dan Creswell (@dancres) January 26, 2016

Architecture in Context – Part 2

By Charlie Alfred and Gene Hughson

Up until this point, we’ve discussed what it means for Architecture to have context.  Contexts enable us to reason about the behavior of a group of stakeholders, whether they be buyers, end users, support staff, distributors, manufacturers or suppliers.

As we’ve pointed out, virtually all products or services are multi-context.  This means that the architecture of these products or services, if they expect to be effective, must also be multi-context.  A multi-context architecture is a lot like juggling.  You must balance your attention across an array of concerns and try to satisfy an array of stakeholders.  The following sidebar illustrates this point:

In the early 1960’s, the United States found itself engaged in a space race with the USSR.  Government officials in both countries  believed strongly that the first country to travel successfully in space and land men on the moon would have a significant military advantage.

President Kennedy collaborated with NASA to initiate a space program designed to put men on the moon’s surface and return them safely to earth, and achieve this by the end of the 1960’s.  On July 20, 1969, the Apollo 11 mission was successful at landing Neil Armstrong and Buzz Aldrin on the moon, and four days later, on July 24th, 1969, the crew safely splashed down in  the Pacific Ocean, where they were brought to safety by the USS Hornet.

The Apollo 11 mission is a high point of 20th Century US history for many.  It also does an excellent job of highlighting several concepts related to Architecture in Context.

• Contexts:
• U.S. Executive Branch,
• U.S. Legislative Branch,
• NASA astronauts
• NASA scientists and engineers
• U.S. Taxpayers, etc.
• Value Expectations:
• Land astronauts safely on the moon during 1960’s
• Return the astronauts to earth safely
• Pain Points:
• Ceding space to the USSR would put the US Military at a strategic disadvantage
• Priorities:
• US President had more clout in this mission than Legislative Branch and NASA
• Return the astronauts safely to earth is higher priority than landing on the moon
• Landing on the moon is more important for NASA and US Executive Branch than for US Taxpayers as a whole
• Challenges:
• Difficulty of lunar landing complicated by moon’s craters and hills
• Difficulty of space capsule reentry is driven by speed, atmospheric friction, and earth’s orbit (i.e.land in water, close enough to battleship)
• Size and design of the space capsule depends on the power of the booster rockets that are needed to launch the capsule beyond the earth’s gravitational pull

[http://en.wikipedia.org/wiki/List_of_Apollo_missions]

Due to space constraints (pun intended), this sidebar merely scratches the surface of the interesting aspects of the Apollo 11 mission.  Another critical point is that the Apollo and Gemini missions that preceded Apollo 11 demonstrated feasibility of technical approaches to address important challenges.  For example, Apollo 7 was the first manned mission to orbit the earth and splashdown safely, and Apollo 8 was the first manned mission to orbit the moon and return safely.  It is worth noting how NASA recognized how difficult and important the reentry challenge was, and how early mission decisions were made to overcome it.

Balancing importance, difficulty and centrality (cross-dependency among challenges) can be a daunting problem.  What best practices exist to help you solve it?

Best Practice 1:  Identify and understand your key contexts.

Understanding your contexts means identifying them on the basis of similar goals, priorities, external forces, and pain points.  When doing this, be sure to focus on  the “why” questions:

• Why do stakeholder groups A and B have the same pain point?
• Do they have the same priority for relieving this pain point, or is this a much higher priority for one group vs. the other?
• Is the pain point caused by the same external forces?
• Is the pain point obstructing the same goals, or different goals?

Questions like these will enable you to cleanse your context notions so that the behaviors and external forces in each context are as consistent as possible.

Best Practice 2:  Understand your key contexts as soon as possible

One tendency in multi-context system development is to leave future contexts as some other day’s problem.  The tendency is to focus on the stakeholders for the first release as the only ones who matter:

• We won’t be marketing to those groups (countries) for years, why waste time thinking about them now?
• Those stakeholders are so new, they won’t know what they need.
• Why spend all the money and  effort interviewing or doing other forms of research when their needs are likely to change?

These objections are based on the perception that identifying and understanding a context requires a lot of effort.  In reality, it requires much less than it appears.  A context is a generalization of behavior and the quality of this generalization can vary with how imminently it is needed.

I realize this sounds like it violates the YAGNI (“you ain’t going to need it”) principle, but it doesn’t.  There is a big difference between anticipating a future context and doing a little defensive design to accommodate its variations than there is of building in full support for it.  The first case is one of defining good interfaces to encapsulate variation.  The other is an implementation and testing burden.   Additionally, it is better to be aware of potential conflicts between contexts before resolving them involves significant design and code changes.

Best Practice 3:  Understand the challenges in satisfying each context’s pain points

Pain points and challenges are similar, and may be confused.  Challenges are concerns of the solution provider.  A challenge is one or more forces that must be overcome to provide value.
A pain point is similar, however it is linked to stakeholders within a context.  This difference shows up in goals, priorities and external factors are considered important.  For example, a pain point might be the need to keep trade secrets confidential from hackers, while the challenge might deal with specific types of cyber threats.

As the solution architect, we derive challenges from pain points in contexts.   Contexts are important here, as similar pain points may exist in related contexts, but with different goals, priorities, or even external challenges.  For example, within an investment management firm, response time needs differ for portfolio managers, traders and compliance officers.

Challenges are framed as problem statements — specific issues that the solution provider must overcome in order to provide value.  It is important to keep the relationships between contexts, pain points and challenges.  In general, contexts have many-to-many relationships to contexts and to pain points.  In other words, there could be several challenges for certain pain points.  Some challenges may be quite similar across several pain points, potentially spanning many contexts.

Best Practice 4:  Carefully consider the nature of challenges when combining over contexts

Challenges are like chemical elements.  Each has its own structure and properties.  However, in a system (especially a multi-context system), challenges combine and form molecules.  Molecules have their own chemical properties, aside from their constituent elements.  For example, Carbon and Oxygen independently are staples of life, while their combination, Carbon Monoxide, is an odorless, tasteless, and most notably, deadly gas.

As mentioned above, challenges take a common form:

• How does the challenge create or detract from value expectations in a context?
• Which external forces cause the challenge’s impact to be magnified or compounded

These two properties make it relatively easy to look at a challenge as a chemical element, or combine it with other challenges to view it as a  molecule.  Here are a few ways to characterize challenges that can be useful in examining their impact:

Some useful dimensions for categorizing challenges are:

Compatibility (of two challenges)

• Compatible – independent problems, no issues solving simultaneously
• Friction – partially dependent problems, some tradeoffs and/or risks
• Antagonistic – highly dependent, serious tradeoffs and risks

Breadth (of a challenge)

• Pervasive – scope of challenges reaches throughout solution
• Regional – scope of challenges pervades an area but encapsulated
• Local – scope of challenges limited to a narrow area

Occurrence (of a challenge)

• Persistent – challenge occurs all or most of the time
• Intermittent – challenge occurs with a predictable frequency
• Conditional – challenge occurs in certain situations or conditions

These dimensions and categories can be useful for determining how to manage or aggregate challenges:

• Within a context, as in how the challenge interacts and combines with others
• Across context, as in whether similar challenges in two or more contexts are sufficiently alike to merge, or whether they combine into a more significant one

Best Practice 5: Prioritize challenges to identify the most important ones to address
 
Software architecture, just as software engineering, is a discipline of deciding among alternative approaches to reach a decision.  While some decisions get made in small sets (e.g. this system will be 3-Tier, .NET, IIS with a SqlServer database), many, if not most, decisions are made independently.

One important thing to remember is that virtually every decision made reduces the degrees of freedom for the rest.  Sometimes this reduction is desirable, but many times it paints the subsequent decisions into a corner.

If the goal of a software architect is to make good decisions for the system, it makes sense that s/he should address the highest priority challenges first (while the degrees of freedom are more available).  But how should challenges be prioritized?  Three criteria need to be balanced:

1. Importance:
   • How many contexts does the challenge affect?
   • How much of an  impact does the challenge make on each one?
   • What is the weighted average of this impact, given the relative importance of each context?
2. Difficulty:
   • The more difficult a challenge, the more degrees of freedom it is likely to need
   • The more difficult a challenge, the more degrees of freedom it will consume
   • The more degrees a challenge will consume, the earlier it should be tackled.
3. Centrality (also called Core):
   • Challenges are also dependent on the solutions to other challenges
   • The dependent challenge should be addressed before or concurrently with the challenges that depend on them

Architectural design takes place in an environment of constraints.  Constraint should be understood as a neutral concept, because it both prevents and enables.  The same cup that constrains the flow of water also enables you to bring that water to your lips.  Managed appropriately, constraints provide the structure and form that yield the desired function(s).  Part of that management is understanding that design decisions are constraints.  Decisions made in isolation risk inappropriately constraining a design in terms of the whole.

Remediation of architectural constraints is, by its very nature, expensive.  Rewiring is more involved than repainting; replacing a foundation is far more extensive still.  Understanding and accounting for all of the contexts involved allows you to see the architecture of the problem as a whole.  The architecture of the problem then becomes the skeleton upon which the architecture of the solution can be built, incrementally, iteratively, and most important, effectively.

Architecture in Context – Part 1

By Charlie Alfred and Gene Hughson

It’s a common occurrence on online forums to see someone ask what architectural style is the right one. Likewise, it’s common to see a reply of “it depends” because “context is king”. Many will nod sagely at this; of course it is, context is king. But, what is context? Specifically, what do we mean by the term “context” in terms of software and solution architecture?

At a high level, context defines the problem environment for which a solution is intended to provide value. That definition, however, is far too superficial to suffice, because only the most trivial of systems will have a single context. Refining our definition, we can state that a context is a grouping of stakeholders sharing similar goals, priorities, and perceptions. In practice, unless you’re dealing with a system that you are developing yourself for your sole use, you will be dealing with multiple contexts, and they are likely to overlap like the Olympic logo. If you are puzzled by this statement, consider silverware. We need knives, forks and spoons and we have different types of each of them.

Identifying contexts and understanding how they interrelate gives definition to the architecture of the problem, which is a necessary prerequisite to designing the architecture of a solution (note the use of “a solution” instead of “the solution” – it’s intentional). This involves considerable work to identify the goals, priorities, and perceptions in that the natural tendency is for a stakeholder to ask for what they think will solve their problem rather than enumerate their problems (which is the point of the apocryphal “faster horses” quote frequently attributed to Henry Ford). Nonetheless, breaking out those raw needs is crucial to success.

Where you have multiple contexts, chances are the goals, priorities, and perceptions diverge rather than complement. Take, for example, a pickup truck. One context may be those who haul bulky and/or heavy goods daily. Another context may be made up of those who frequently tow trailers. A third context might be those who haul or tow from time to time, but mainly use the truck for transportation. The value of any given feature of the truck: miles per gallon of fuel, size of the bed, whether or not the bed is covered, quality of the sound system, torque, etc. will vary based on the context we’re considering. Additionally, external forces (such as the laws of physics) will come into play and complicate making design decisions. For example, long, open truck beds maximize hauling capacity, but will also harm fuel efficiency.

Another complicating factor is the presence of peripheral stakeholders, those whose contexts will affect the direct stakeholders. In the sense of our pickup truck example, we can consider mechanics as an example of this. Trucks that have more room in the engine compartment are easier (therefore cheaper) to service but will suffer in terms of fuel efficiency due to increased size and weight. In the IT realm, development and support teams would fall into this category. Although their contexts would not be primary drivers, ignoring them could well impose a cost on the direct stakeholders in terms of turnaround time for enhancements and fixes.

Examples of multi-context software systems can be found many places. The more generic a system, the more contexts it will have. Commodity office software (word processors, spreadsheets, etc.) are a prime example. Consider Microsoft Excel, which is a simple row/column tabulator, a statistical analysis tool, a database client, and, with its macro language, is an application development platform. Stretching to cover this many needs turns the Excel User Experience into a game of Twister. Platforms, such as Android are another example. Highly configurable applications like Salesforce.com (which has arguably crossed the line between product and platform) span multiple contexts as well. In the enterprise IT space, first SOA and now microservice architectures are nothing if not an attempt to address the plethora of contexts in play via separation of concerns.

A holistic consideration of the contexts at hand is an important success factor when evolving a system iteratively and incrementally. Without that consideration, decisions can be made that are conducive to one context but antagonistic to another. Deferring the reconciliation of the two contextual conflicts until later may result in considerable architectural refactoring. This refactoring will involve time and expense at a minimum, and if time is constrained, the likelihood of incurring technical debt is high. Each additional conflict that is blundered into due to the lack of up front consideration increases the risk to the system. This is not to endorse a detailed Big Design Up Front (BDUF), but a recognition that problem awareness and problem avoidance is cheaper than rework. If the context has been identified, then it’s YAGNI – not “You Ain’t Gonna Need It” but “You Are Gonna Need It”.

The next post will in this series will address concrete practices to integrate multiple contexts into a unified architecture of the problem that can serve as the foundation for a coherent architecture of a solution.

Faster Horses – Henry Ford and Customer Development on Iasa Global Blog

Henry (“Any customer can have a car painted any color that he wants so long as it is black”) Ford was not an advocate of customer development. Although there’s no evidence that he actually said “If I had asked people what they wanted, they would have said faster horses”, it’s definitely congruent with other statements he did make.

See the full post on the Iasa Global Blog (a re-post, originally published on CitizenTekk and mirrored here).