Monthly Archives: March 2015

Form Follows Function on SPaMCast 335

Posted on by

SPaMCAST logo

It’s time for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast. This time I’m taking on Knuth’s quote: “Premature optimization is the root of all evil (or at least most of it) in programming.”

SPaMCast 335 features Tom on the meaning of effectiveness, efficiency, frameworks and methodologies; a discussion of my “Wait, did I just say Knuth was wrong?” post and an installment of Jo Ann Sweeny’s column, “Explaining Communication”, talking about content and a framework to guide the development of content.

Advertisement

Microservice Principles, Technical Debt, and Legacy Systems

Posted on by

Is there a circumstance where the answer to Architect Clippy‘s question is “yes”? In “Microservice Architectures aren’t for Everyone” I used this tweet to underscore the observation that a team that can’t produce a well-modularized monolith is unlikely to be helped by trying to distribute the problem. On the other hand, a team (or teams) tasked with rehabilitating a “Big Ball of Mud” might well find some value in the principles behind microservice architectures.

Some of the relevant principles are cohesion and replaceability. As Dan North noted in “Microservices: software that fits in your head”:

One way to manage the mess is to maximise the likelihood that everyone knows what’s going on in the codebase. This requires two things: consistency and replaceability. Consistency implies you can make reasonable assumptions about unfamiliar parts of the application. Replaceability means you can kill code easily and replace it with something better.

Without achieving separation of concerns, any architectural refactoring effort will be an exercise in chasing fires across the codebase. A divide and conquer strategy that applies the single responsibility principle at a macro level will be more likely to facilitate identification and remediation of lower-level technical debt. Monoliths can benefit from being carved up, not because small is inherently better, but because they reach a point where independence of their components becomes beneficial, even crucial. Components that share fewer dependencies (such as a shared data store) and have independent release cycles offer a great deal of flexibility in structuring an application and the team(s) that develop it.

In “Microservices allow for localized tech debt”, Jim Plush stated: “It’s much easier mentally to tackle $10,000 of debt across 4 credit cards at $2500 each than 1 card at the full $10,000.” Even more to the point, it’s much easier to tackle that debt when you split it with three other people (teams) each working independently.

Re-writes have a well-deserved bad reputation. Shared platforms and shared data stores will often mean that the transition from the legacy system to the re-written one will be a high-risk “big bang” affair. As Edmond Lau observed in “How to Avoid One of the Costliest Mistakes in Software Engineering”, you want to “…get as quickly as possible to a state where you’re again making incremental improvements”. Getting to this state may well happen quicker when the parts are separated.

Microservices, SOA, Reuse and Replaceability

Posted on by

Unicorn

While it’s not as elusive as the unicorn, the concept of reuse tends to be talked about more often talked about than seen. Over the years, object-orientation, design patterns, and services have all held out the promise of reuse of either code or at least, design. Similar claims have been made regarding microservices.

Reuse is a creature of extremes. Very fine grained components (e.g. the classes that make up the standard libraries of Java and .Net) are highly reusable but require glue code to coordinate their interaction in order to yield something useful. This will often be the case with microservices, although not always; it is possible to have very small services with few or no dependencies on other services (it’s important to remember, unlike libraries, services generally share both behavior and data.). Coarse grained components, such as traditional SOA services, can be reused across an enterprise’s IT architecture to provide standard high-level interfaces into centralized systems for other applications.

The important thing to bear in mind, though, is that reuse is not an end in itself. It can be a means of achieving consistency and/or efficiency, but its benefits come from avoiding cost and duplication rather than from the extra usage. Just as other forms of reuse have had costs in addition to benefits, so it is with microservices as well.

Anything that is reused rather than duplicated becomes a dependency of its client application. This dependency relationship is a form of coupling, tying the two codebases together and constraining the ability of the dependency to change. Within the confines of an application, it is generally better for reuse to emerge. Inter-application reuse will require more coordination and tend to be more deliberately designed. As with most things, there is no free lunch. Context is required to determine whether the trade is a good one or not.

Replaceability is, in my opinion, just as important, if not more so, than reuse. Being able to switch from one dependency to another (or from one version of a dependency to another) because that dependency has its own independent lifecycle and is independently deployed enables a great deal of flexibility. That flexibility enables easier upgrades (rolling migration rather than a big bang). Reducing the friction inherent in migrations reduces the likelihood of technical debt due to inertia.

While a shared service may well find more constraints with each additional client, each client can determine how much replaceability is appropriate for itself.

Of Blind Men and Elephants and Excessive Certainty

Posted on by

Blind men and the elephant

There’s an old poem about six blind men and an elephant, where each in turn declare that an elephant is like a wall, a spear, a snake, a tree, a fan, and a rope. Each accurately described what he was able to discern from his own limited point of view, yet all were wrong about the subject as a whole. As the poet noted:

Moral:

So oft in theologic wars,
The disputants, I ween,
Rail on in utter ignorance
Of what each other mean,
And prate about an Elephant
Not one of them has seen!

Sometimes our attitudes color our perception of others:

Management is often the butt of our disdain, expressed in cartoon form:

However, as Sandro Mancuso related in “Not all managers are stupid”:

I still remember the day when our managers in a large organisation told us we should still go live after we reported a major problem a couple of months before the deadline…There was a problem in a couple of unfinished flows, which would cause hundreds of thousands of trades to be misreported to the regulators. After we explained the situation, managers told us to work harder go ahead with the release anyway.

How could they tell us to go live in a situation like that? They should all be fired. Arrested. How could they ask us to drop the quality and go live with a known problem of that size?…

More than once we made it clear that focusing our time on getting the system ready to production would not gives us any time to finish the automation for the problematic flows and thousands of trades would be misreported. But they did not listen. Or so we thought.

After a few meetings with the business, we discovered a few things. They were not being irresponsible or stupid, as we developers thought. The deadline was set by the regulators and could not be moved. The cost of not reporting the trades was far higher than misreporting them. Not reporting the trades would not only be followed by heavy fines, but also by possible reputation damage. Companies would have extra time to correct any misreported trades before being fined.

For us, in the development team, it was the first time we realised that going live with a few known issues would be better than not going live at all.

Designing the architecture of a solution, at its core, is an exercise in decision-making. Whether the system in question is a software system or a human system, effective decision-making must be preceded by sense-making to identify the architecture of the problem. Contexts need to be identified in order to be synthesized into the architecture of the problem.

Bias, being too certain of our understanding to make the effort to validate it, is a good way to miss out on what’s in front of us. Failing to recognize our potential for bias makes it harder to overcome that bias. That failure restricts our ability to appreciate the full range of contexts to be synthesized and puts us in the same position as the blind men with the elephant.

It’s extremely difficult to solve a problem you don’t understand.

Microservice Architectures aren’t for Everyone

Posted on by

Simon Brown says it nicely:

Architect Clippy is a bit more snarky:

In both cases, the message is the same: microservice architectures (MSAs), in and of themselves, are not necessarily “better”. There are aspects where moving to a distributed architecture from a monolithic one can improve a system (e.g. enabling selective load balancing and incremental deployment). However, if someone isn’t capable of building a modular monolith, distributing the pieces is unlikely to help matters.

Even where the requisite architectural design capability exists, however, it isn’t a given that distributing the components of the application is the right choice. Distributed applications will incur additional complexity from both a development and an operations standpoint. The more granularly the application is distributed, the more complexity will result. It makes little sense to incur complexity costs that outweigh any benefits received. The decision, however is not a binary one; system architectures that fall between MSA and monolith are possible. One can selectively apply MSA and/or SOA principles from the application level all the way to the enterprise’s IT architecture.

Daniel Bryant’s “Drafting a Proposal for a Microservice Maturity/Classification Model” contains an interesting taxonomy of application architectural styles:

  • Megalith Platform
    • Humongous single codebase resulting in a single application
  • Monolith Platform
    • Large single codebase resulting in a single application
  • Macro SOA Platform
    • Classical SOA applications, and platforms consisting of loosely-coupled large services (potentially a series of interconnected monoliths)
  • Meso Application Platform
    • ‘Meso’ or middle-sized services interconnected to form a single application or platform. Essentially a monolith and microservice hybrid
  • Microservice Platform
    • ‘Cloud native’ loosely-coupled small services focused around DDD-inspired ‘bounded contexts’
  • Nanoservice Platform
    • Extremely small single-purpose (primarily reactive) services

As a taxonomy of application types, Bryant’s work is interesting. Some of the details he’s listed for each type are highly generalized (e.g. Megaliths and Monoliths are assumed to be highly coupled, low cohesion big balls of mud that are difficult to understand), but it could serve as a basis for categorizing applications. As a maturity model, however, it has serious issues. As a maturity model, it would assume that nanoservices(!) represent the zenith of application design. I would argue that this is inherently flawed. Not all enterprises will necessarily require applications fitting the full MSA model. The idea that all applications would benefit from this style of architectural design is extremely hard to believe. I would argue that forcing this style onto smaller applications with limited user bases would be harmful – incurring unnecessary expense for these applications would risk discrediting the technique in circumstances where it would be appropriate.

Where a technique is chosen because of evidence that it delivers specific benefits that meet specific needs and the costs involved don’t outweigh those benefits, there is a higher probability of success. Where the decision is made on the basis of using what’s “new” or “cool”, there’s no real way to give a probability of success because the selection criteria is divorced from fitness for purpose. That being said, my bias would be towards a much lower chance of success.

Innovations can be alternatives without necessarily being “better” in all cases.

Form Follows Function on SPaMCast 331

Posted on by

SPaMCAST logo

I’m back for another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

SPaMCast 331 features Tom on Agile Coaching, a discussion of my “Microservices vs SOA – Is there any real difference?” post and an installment of Jo Ann Sweeny’s column, “Explaining Communication”, dealing with communication channels.