I see you have a poorly structured monolith. Would you like me to convert it into a poorly structured set of microservices?—
Architect Clippy (@architectclippy) February 24, 2015
Is there a circumstance where the answer to Architect Clippy‘s question is “yes”? In “Microservice Architectures aren’t for Everyone” I used this tweet to underscore the observation that a team that can’t produce a well-modularized monolith is unlikely to be helped by trying to distribute the problem. On the other hand, a team (or teams) tasked with rehabilitating a “Big Ball of Mud” might well find some value in the principles behind microservice architectures.
Some of the relevant principles are cohesion and replaceability. As Dan North noted in “Microservices: software that fits in your head”:
One way to manage the mess is to maximise the likelihood that everyone knows what’s going on in the codebase. This requires two things: consistency and replaceability. Consistency implies you can make reasonable assumptions about unfamiliar parts of the application. Replaceability means you can kill code easily and replace it with something better.
Without achieving separation of concerns, any architectural refactoring effort will be an exercise in chasing fires across the codebase. A divide and conquer strategy that applies the single responsibility principle at a macro level will be more likely to facilitate identification and remediation of lower-level technical debt. Monoliths can benefit from being carved up, not because small is inherently better, but because they reach a point where independence of their components becomes beneficial, even crucial. Components that share fewer dependencies (such as a shared data store) and have independent release cycles offer a great deal of flexibility in structuring an application and the team(s) that develop it.
In “Microservices allow for localized tech debt”, Jim Plush stated: “It’s much easier mentally to tackle $10,000 of debt across 4 credit cards at $2500 each than 1 card at the full $10,000.” Even more to the point, it’s much easier to tackle that debt when you split it with three other people (teams) each working independently.
Re-writes have a well-deserved bad reputation. Shared platforms and shared data stores will often mean that the transition from the legacy system to the re-written one will be a high-risk “big bang” affair. As Edmond Lau observed in “How to Avoid One of the Costliest Mistakes in Software Engineering”, you want to “…get as quickly as possible to a state where you’re again making incremental improvements”. Getting to this state may well happen quicker when the parts are separated.
While it’s not as elusive as the unicorn, the concept of reuse tends to be talked about more often talked about than seen. Over the years, object-orientation, design patterns, and services have all held out the promise of reuse of either code or at least, design. Similar claims have been made regarding microservices.
Reuse is a creature of extremes. Very fine grained components (e.g. the classes that make up the standard libraries of Java and .Net) are highly reusable but require glue code to coordinate their interaction in order to yield something useful. This will often be the case with microservices, although not always; it is possible to have very small services with few or no dependencies on other services (it’s important to remember, unlike libraries, services generally share both behavior and data.). Coarse grained components, such as traditional SOA services, can be reused across an enterprise’s IT architecture to provide standard high-level interfaces into centralized systems for other applications.
The important thing to bear in mind, though, is that reuse is not an end in itself. It can be a means of achieving consistency and/or efficiency, but its benefits come from avoiding cost and duplication rather than from the extra usage. Just as other forms of reuse have had costs in addition to benefits, so it is with microservices as well.
Anything that is reused rather than duplicated becomes a dependency of its client application. This dependency relationship is a form of coupling, tying the two codebases together and constraining the ability of the dependency to change. Within the confines of an application, it is generally better for reuse to emerge. Inter-application reuse will require more coordination and tend to be more deliberately designed. As with most things, there is no free lunch. Context is required to determine whether the trade is a good one or not.
Replaceability is, in my opinion, just as important, if not more so, than reuse. Being able to switch from one dependency to another (or from one version of a dependency to another) because that dependency has its own independent lifecycle and is independently deployed enables a great deal of flexibility. That flexibility enables easier upgrades (rolling migration rather than a big bang). Reducing the friction inherent in migrations reduces the likelihood of technical debt due to inertia.
While a shared service may well find more constraints with each additional client, each client can determine how much replaceability is appropriate for itself.
There’s an old poem about six blind men and an elephant, where each in turn declare that an elephant is like a wall, a spear, a snake, a tree, a fan, and a rope. Each accurately described what he was able to discern from his own limited point of view, yet all were wrong about the subject as a whole. As the poet noted:
So oft in theologic wars,
The disputants, I ween,
Rail on in utter ignorance
Of what each other mean,
And prate about an Elephant
Not one of them has seen!
Sometimes our attitudes color our perception of others:
Jörgen Dahlberg (@greblhad) January 10, 2015
Management is often the butt of our disdain, expressed in cartoon form:
Woody Zuill (@WoodyZuill) December 31, 2014
However, as Sandro Mancuso related in “Not all managers are stupid”:
I still remember the day when our managers in a large organisation told us we should still go live after we reported a major problem a couple of months before the deadline…There was a problem in a couple of unfinished flows, which would cause hundreds of thousands of trades to be misreported to the regulators. After we explained the situation, managers told us to work harder go ahead with the release anyway.
How could they tell us to go live in a situation like that? They should all be fired. Arrested. How could they ask us to drop the quality and go live with a known problem of that size?…
More than once we made it clear that focusing our time on getting the system ready to production would not gives us any time to finish the automation for the problematic flows and thousands of trades would be misreported. But they did not listen. Or so we thought.
After a few meetings with the business, we discovered a few things. They were not being irresponsible or stupid, as we developers thought. The deadline was set by the regulators and could not be moved. The cost of not reporting the trades was far higher than misreporting them. Not reporting the trades would not only be followed by heavy fines, but also by possible reputation damage. Companies would have extra time to correct any misreported trades before being fined.
For us, in the development team, it was the first time we realised that going live with a few known issues would be better than not going live at all.
Designing the architecture of a solution, at its core, is an exercise in decision-making. Whether the system in question is a software system or a human system, effective decision-making must be preceded by sense-making to identify the architecture of the problem. Contexts need to be identified in order to be synthesized into the architecture of the problem.
Bias, being too certain of our understanding to make the effort to validate it, is a good way to miss out on what’s in front of us. Failing to recognize our potential for bias makes it harder to overcome that bias. That failure restricts our ability to appreciate the full range of contexts to be synthesized and puts us in the same position as the blind men with the elephant.
It’s extremely difficult to solve a problem you don’t understand.
Simon Brown says it nicely:
Simon Brown (@simonbrown) March 04, 2015
Architect Clippy is a bit more snarky:
I see you have a poorly structured monolith. Would you like me to convert it into a poorly structured set of microservices?—
Architect Clippy (@architectclippy) February 24, 2015
In both cases, the message is the same: microservice architectures (MSAs), in and of themselves, are not necessarily “better”. There are aspects where moving to a distributed architecture from a monolithic one can improve a system (e.g. enabling selective load balancing and incremental deployment). However, if someone isn’t capable of building a modular monolith, distributing the pieces is unlikely to help matters.
Even where the requisite architectural design capability exists, however, it isn’t a given that distributing the components of the application is the right choice. Distributed applications will incur additional complexity from both a development and an operations standpoint. The more granularly the application is distributed, the more complexity will result. It makes little sense to incur complexity costs that outweigh any benefits received. The decision, however is not a binary one; system architectures that fall between MSA and monolith are possible. One can selectively apply MSA and/or SOA principles from the application level all the way to the enterprise’s IT architecture.
Daniel Bryant’s “Drafting a Proposal for a Microservice Maturity/Classification Model” contains an interesting taxonomy of application architectural styles:
- Megalith Platform
- Humongous single codebase resulting in a single application
- Monolith Platform
- Large single codebase resulting in a single application
- Macro SOA Platform
- Classical SOA applications, and platforms consisting of loosely-coupled large services (potentially a series of interconnected monoliths)
- Meso Application Platform
- ‘Meso’ or middle-sized services interconnected to form a single application or platform. Essentially a monolith and microservice hybrid
- Microservice Platform
- ‘Cloud native’ loosely-coupled small services focused around DDD-inspired ‘bounded contexts’
- Nanoservice Platform
- Extremely small single-purpose (primarily reactive) services
As a taxonomy of application types, Bryant’s work is interesting. Some of the details he’s listed for each type are highly generalized (e.g. Megaliths and Monoliths are assumed to be highly coupled, low cohesion big balls of mud that are difficult to understand), but it could serve as a basis for categorizing applications. As a maturity model, however, it has serious issues. As a maturity model, it would assume that nanoservices(!) represent the zenith of application design. I would argue that this is inherently flawed. Not all enterprises will necessarily require applications fitting the full MSA model. The idea that all applications would benefit from this style of architectural design is extremely hard to believe. I would argue that forcing this style onto smaller applications with limited user bases would be harmful – incurring unnecessary expense for these applications would risk discrediting the technique in circumstances where it would be appropriate.
Where a technique is chosen because of evidence that it delivers specific benefits that meet specific needs and the costs involved don’t outweigh those benefits, there is a higher probability of success. Where the decision is made on the basis of using what’s “new” or “cool”, there’s no real way to give a probability of success because the selection criteria is divorced from fitness for purpose. That being said, my bias would be towards a much lower chance of success.
Innovations can be alternatives without necessarily being “better” in all cases.
I’m back for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.
SPaMCast 331 features Tom on Agile Coaching, a discussion of my “Microservices vs SOA – Is there any real difference?” post and an installment of Jo Ann Sweeny’s column, “Explaining Communication”, dealing with communication channels.
Eberhard Wolff‘s question set the stage.
If Conway's Law is so important - are #Microservices more an organizational approach than an architecture?—
Eberhard Wolff (@ewolff) February 18, 2015
Adrian CockCroft‘s reply tied everything together.
.@ewolff Agile, DevOps, Conways Law and Microservices all reinforce each other to speed up product development.—
adrian cockcroft (@adrianco) February 20, 2015
Conway’s Law is the common thread tying microservice architectures (MSAs) and DevOps together. Significantly, this common thread runs through the entire organization, not just the IT parts. As I noted in my previous post, paying attention to this principle allows you to work with, rather than against, the grain of an organization. Working with the grain of the organization is key, because DevOps, lovely as it is, is not an end, but a means. The desired end, as identified by Mike Kavis in “Is DevOps What Organizations Really Seek?” is to “become high-performing organizations”.
Of all the attributes of such an organization (as identified by Kavis): “Strong Leadership”, “Strong Culture”, “Sound Architecture”, etc., the most important is “High Customer Satisfaction”. For many organizations, high customer satisfaction is a problem area for IT. In a recent article on Business Insider, Red Hat’s CEO Jim Whitehurst noted an increased interest in DevOps on the part of CIOs to deal with what he terms IT’s “fight for its life”:
That’s because IT departments say they had better figure out how to be faster, cheaper, and better. If they don’t, the company’s employees will no longer depend on them. They bring their own PCs, tablets and phones to work and they buy whatever cloud services they want to do their jobs. And the CIO will find his budget increasingly shifted to other manager’s pockets.
IT has has a history of cycles of neglect or rejection of new/disruptive technologies followed by catch-up crises: PCs in the 80s, Web in the 90s, BYOD/Cloud/IOT/etc. now. What’s different this time is the increasing level of technical knowledge and access to solutions outside of IT. As Krishnan Subramanian has noted, playing catch-up may become less and less viable:
@GeneHughson This time impact will be orders of magnitude greater. Catching up in next refresh cycle might be too late—
Krish (@krishnan) February 20, 2015
MSAs enable flexible applications via composing vertical slices of business functionality rather than horizontal layers of technical concerns. These same principles can apply to higher levels of abstraction up to the enterprise’s IT architecture. Likewise, DevOps incorporates the same viewpoint shift in terms of the IT organization. This architectural change (both technical and business) can allow for integration between IT and the business it enables. As Twila Day recently noted, this integration goes far beyond mere alignment into “active partnership between IT and your business units”:
Partnership means working together, side by side. It means that technology leaders are actively involved in strategic development at the highest levels of your organization. It means that all the way up and down your organization, any talented person can propose an innovative idea, tactic or strategy, regardless of where s/he works. The business might have an idea first, or IT might have it first. No matter. What’s important is that the two groups work side by side to accomplish the most important business objectives.
Transforming IT from an adversary into a partner is primarily a cultural shift that involves both parties if it’s to be successful. Organizational and technical architecture cannot be neglected, however, in that they can either help or hinder that transformation. DevOps can facilitate this via its focus on the product rather than any one project (which is a concern shared by the customers) and by having the flexibility to tailor its pace to that of the customer rather than forcing a one size fits all (aka one size fits none).
In my part of the world, it’s not uncommon for people to say that someone wouldn’t recognize something if it “bit them in the [rude rump reference]”. For many organizations, that seems to be the explanation for the state of their enterprise IT architecture. For while we might claim to understand terms like “design”, “encapsulation”, and “separation of concerns”, the facts on the ground fail to show it. Just as software systems can degenerate into a “Big Ball of Mud”, so too can the systems of systems comprising our enterprise IT architectures. If we look at organizations as the systems they are, it should be obvious that this same entropy can infect the organization as well.
One symptom of this entropy is when the dividing lines blur, weakening or even removing constraints. While the term “constraint” commonly has a negative connotation, constraints provide the structure and definition of a system. Separation of concerns, encapsulation, and DRY are all constraints that are intended to provide benefit. We accept limits on concerns addressed, accessibility of internals and/or instances of code or data in order to reduce complexity, not just check off a philosophical box. If we remove or even just relax these types of constraints too much, we incur risk.
This blurring of lines can occur at any level and on multiple levels. Additionally, architectural weakness at a higher level of abstraction can negate strengths at lower levels. A collection of well-designed systems will not ensure a coherent enterprise IT architecture if there is overlap and redundancy without a clear understanding of which ones are authoritative. Accidental architecture is no more likely to work at higher levels of abstraction than lower ones.
Architectural design, at each level of granularity, should be intentional and appropriate to that level. The ideal, is not to over-regulate, but to strike a balance. Micromanaging internals wastes effort better spent on something beneficial; abdicating design responsibility practically guarantees chaos. An additional consideration is the fit between the human and technological aspects. Conway’s law is more than just an observation, it can be used as a tool to align applications to a specific business concern as well as aligning development teams to specific applications/application components.
Just as a carver takes note of the grain of the wood being shaped, so should an architect work with rather than against the grain of the organization.
Jessica Kerr’s post, “Microservices, Microbusinesses”, captures these concepts from the viewpoint of microservice architectures. Partitioning application concerns into microservices allows for internal flexibility at the cost of external conformance to necessary governance. As Kerr puts it, “…everybody is a responsible adult”:
That’s a lot of overhead and glue code. Every service has to do translation from input to its internal format, and then to whatever output format someone else requires. Error handling, caching or throttling, failover and load balancing and monitoring, contract testing, maintaining multiple interface versions, database interaction details. Most of the code is glue, layers of glue protecting a small core of business logic. These strong boundaries allow healthy relationships with other services, including new interactions that weren’t designed into the architecture from the beginning. For all this work, we get freedom on the inside.
Kerr also recognizes the applicability of this trade-off to the architecture of the organization:
Still, a team exists as a citizen inside a larger organization. There are interfaces to fulfill. Management really does need to know about progress. Outward collaboration is essential. We can do this the same way our code does: with glue. Glue made of people. One team member, taking the responsibility of translating cards on the wall into JIRA, can free the team to optimize communication while filling management’s strongest needs.
Management defines an API. Encapsulate the inner workings of the team, and expose an interface that makes sense outside. By all means, provide a reference implementation: “Other teams use Target Process to track work.” Have default recommendations: “We use Clojure here unless there’s some better solution. SQL Server is our first choice database for these reasons.” Give teams a strong process and technology stack to start from, and let them innovate from there.
“Good fences make good neighbors” not by keeping out, but by channeling traffic into commonly understood and commonly accepted directions. We recognize lines so as to influence those aspects we truly need to influence. More importantly, we recognize lines to prevent needless conflict and waste. The key is to draw the lines so that they work for us, not against us.