‘Customer collaboration over contract negotiation’ and #NoEstimates

Faust and Mephistopheles

Since my last post on #NoEstimates, the conversation has taken an interesting turn. Steve McConnell weighed in on July 30th, with a response from Ron Jeffries on the following day. There have been several posts from each since, with the latest from Jeffries titled “Summing up the discussion”. In that post, he states the following (emphasis added):

The core of our disagreement here is that Steve says repeatedly that if the customer wants estimates, then we should give them estimates. He offers the Manifesto’s “Customer collaboration over contract negotiation”1 as showing that we should do that. (It’s interesting that he makes that point in a kitchen where surely he expects that the couple and the contractor(!) are going to negotiate a contract.)

To anyone who has been even minimally involved with business, the idea that a contract will be negotiated should not a surprise, no matter how agile the provider. In spite of his footnote warning “I confess that I am singularly unamused when people bludgeon me with quotes from the Manifesto”, I have to note that the Agile Manifesto does say “…while there is value in the items on the right, we value the items on the left more” rather than “only the items on the left have value”.

In the post, Jeffries says that in an agile transformation, the “…contractor-customer relationship gets inverted”. I have to confess that the statement is a bit baffling. A situation where “the customer is always right” can certainly degrade into a no-win scenario. Inverting that relationship, however, is equally bad. Good, sustainable business relationships should be relatively (even if not perfectly) balanced. Moving away from minutely detailed contracts that prevent any change without days of negotiation just makes sense for both parties. Eliminating all obligation on the part of the provider, however, is a much harder sell.

Jeffries asserts that “Agile, if you’ll actually do it, changes the fundamental relationship from an estimate-commitment model to a truly collaborative stream of building up the product incrementally.” If, however, you fail to meet your customer’s needs (and, as I pointed out in “#NoEstimates – Questions, Answers, and Credibility”, one of the best lists of why estimates are needed for decision-making comes from Woody Zuill himself), how collaborative is that? Providing estimates with the caveat that they are subject to change as the knowledge about the work changes is more realistic and balanced for both parties. That balance, in my opinion (and experience) is far more likely to result in a collaboration between customer and provider than any inversion of the relationship.

Trust, is essential to fostering collaboration. If we fail to understand our customers’ needs (which impression is given when we make statements like that in the tweet below), we risk fatally damaging the trust we need to create an effective collaboration.

Microservices, Monoliths, and Conflicts to Resolve

Two tweets, opposites in position, and both have merit. Welcome to the wonderful world of architecture, where the only rule is that there is no rule that survives contact with reality.

Enhancing resilience via redundancy is a technique with a long pedigree. While microservices are a relatively recent and extreme example of this, they’re hardly groundbreaking in that respect. Caching, mirroring, load-balancing, etc. has been with us a long, long time. Redundancy is a path to high availability.

Centralization (as exemplified by monolithic systems) can be a useful technique for simplification, increasing data and behavioral integrity, and promoting cohesion. Like redundancy, it’s a system design technique older than automation. There was a reason that “all roads lead to Rome”. Centralization provides authoritativeness and, frequently, economies of scale.

The problem with both techniques is that neither comes without costs. Redundancy introduces complexity in order to support distributing changes between multiple points and reconciling conflicts. Centralization constrains access and can introduce a single point of failure. Getting the benefits without the incurring the costs remains a known issue.

The essence of architectural design is decision-making. Given that those decisions will involve costs as well as benefits, both must be taken into account to ensure that, on balance, the decision achieves its aims. Additionally, decisions must be evaluated in the greater context rather than in isolation. As Tom Graves is fond of saying “things work better when they work together, on purpose”.

This need for designs to not only be internally optimal, but also optimized for their ecosystem means that these, as well as other principles, transcend the boundaries between application architecture, enterprise IT architecture, and even enterprise architecture. The effectiveness of this fractal architecture of systems of systems (both automated and human) is a direct result of the appropriateness of the decisions made across the full range of the organization to the contexts in play.

Since there is no one context, no rule can suffice. The answer we’re looking for is neither “microservice” nor “monolith” (or any other one tactic or technique), but fit to purpose for our context.

“Microservices, SOA, and EITA: Where To Draw the Line? Why to Draw the Line?” on Iasa Global

In my part of the world, it’s not uncommon for people to say that someone wouldn’t recognize something if it “bit them in the [rude rump reference]”. For many organizations, that seems to be the explanation for the state of their enterprise IT architecture. For while we might claim to understand terms like “design”, “encapsulation”, and “separation of concerns”, the facts on the ground fail to show it. Just as software systems can degenerate into a “Big Ball of Mud”, so too can the systems of systems comprising our enterprise IT architectures. If we look at organizations as the systems they are, it should be obvious that this same entropy can infect the organization as well.

See the full post on the Iasa Global site (a re-post, originally published here).

Form Follows Function on SPaMCast 339


This week’s episode of Tom Cagley’s Software Process and Measurement (SPaMCast) podcast features Tom’s essay on demonstrations and a Form Follows Function installment on microservices, SOA, and Enterprise IT Architecture.

For SPaMCast 339, Tom and I discuss my “Microservices, SOA, and EITA: Where To Draw the Line? Why to Draw the Line?” post.

Are Microservices the Next Big Thing?


The question popped up in a LinkedIn discussion of my last post:

“A question for you, are microservices the next big thing?”

It was a refreshing change of pace to actually be asked about my opinion rather than be told what it is. The backlash against microservices is such that, anything less that outright condemnation is seen as “touting” the latest fad (this in spite of having authored posts such as “Microservice Architectures aren’t for Everyone”, “Microservice Mistakes – Complexity as a Service”, and “Microservices – The Too Good to be True Parts”). Reflex reactions like this will most likely be so simplistic as to be useless, regardless of whether pro or con.

Architectural design is about making decisions. Good design, in my opinion, is about making decisions justified by the context at hand rather than following a recipe. It’s about understanding the situation and applying principles rather than blindly replicating a pattern. No one cares about how cutting edge the technology is, they care that the solution solves the problem without bankrupting them.

So, to return to the question of whether I think microservices are “the next big thing”:

They shouldn’t be, but they might be. Shouldn’t, because a full-blown microservice architecture app, while perfectly appropriate for Netflix, isn’t likely to be appropriate for most in-house corporate applications. They won’t get a benefit because of the additional operational and development complexity. That being said, lots of inappropriate SOA initiatives sucked up a lot of money not that long ago – if software development had a mascot it would be an immortal lemming with amnesia.

The principles behind MSA, however, have some value IMHO. When a monolithic architecture begins to get in the way, those principles can provide some guidance on how to carve it up. The wonderful thing about SRP is its fractal nature, so you split out responsibilities at a level of granularity that’s appropriate for your application’s situation when it makes sense. There’s no rule that you have to carve up everything at once or that you have to slice it as thin as they do at Netflix.

That’s why my posts on the subject tend to sway back and forth. It’s not a recipe for the “right way” (as if there could be one right way) to design applications, it’s merely another set of ideas that, depending on the context, could help or hurt.

It’s not the technique itself that makes or breaks a design, it’s how applicable the technique is to problem at hand.

Microservices, SOA, and EITA: Where To Draw the Line? Why to Draw the Line?

Surveying and drafting instruments and examples

In my part of the world, it’s not uncommon for people to say that someone wouldn’t recognize something if it “bit them in the [rude rump reference]”. For many organizations, that seems to be the explanation for the state of their enterprise IT architecture. For while we might claim to understand terms like “design”, “encapsulation”, and “separation of concerns”, the facts on the ground fail to show it. Just as software systems can degenerate into a “Big Ball of Mud”, so too can the systems of systems comprising our enterprise IT architectures. If we look at organizations as the systems they are, it should be obvious that this same entropy can infect the organization as well.

One symptom of this entropy is when the dividing lines blur, weakening or even removing constraints. While the term “constraint” commonly has a negative connotation, constraints provide the structure and definition of a system. Separation of concerns, encapsulation, and DRY are all constraints that are intended to provide benefit. We accept limits on concerns addressed, accessibility of internals and/or instances of code or data in order to reduce complexity, not just check off a philosophical box. If we remove or even just relax these types of constraints too much, we incur risk.

This blurring of lines can occur at any level and on multiple levels. Additionally, architectural weakness at a higher level of abstraction can negate strengths at lower levels. A collection of well-designed systems will not ensure a coherent enterprise IT architecture if there is overlap and redundancy without a clear understanding of which ones are authoritative. Accidental architecture is no more likely to work at higher levels of abstraction than lower ones.

Architectural design, at each level of granularity, should be intentional and appropriate to that level. The ideal, is not to over-regulate, but to strike a balance. Micromanaging internals wastes effort better spent on something beneficial; abdicating design responsibility practically guarantees chaos. An additional consideration is the fit between the human and technological aspects. Conway’s law is more than just an observation, it can be used as a tool to align applications to a specific business concern as well as aligning development teams to specific applications/application components.

Just as a carver takes note of the grain of the wood being shaped, so should an architect work with rather than against the grain of the organization.

Jessica Kerr’s post, “Microservices, Microbusinesses”, captures these concepts from the viewpoint of microservice architectures. Partitioning application concerns into microservices allows for internal flexibility at the cost of external conformance to necessary governance. As Kerr puts it, “…everybody is a responsible adult”:

That’s a lot of overhead and glue code. Every service has to do translation from input to its internal format, and then to whatever output format someone else requires. Error handling, caching or throttling, failover and load balancing and monitoring, contract testing, maintaining multiple interface versions, database interaction details. Most of the code is glue, layers of glue protecting a small core of business logic. These strong boundaries allow healthy relationships with other services, including new interactions that weren’t designed into the architecture from the beginning. For all this work, we get freedom on the inside.

Kerr also recognizes the applicability of this trade-off to the architecture of the organization:

Still, a team exists as a citizen inside a larger organization. There are interfaces to fulfill. Management really does need to know about progress. Outward collaboration is essential. We can do this the same way our code does: with glue. Glue made of people. One team member, taking the responsibility of translating cards on the wall into JIRA, can free the team to optimize communication while filling management’s strongest needs.

Management defines an API. Encapsulate the inner workings of the team, and expose an interface that makes sense outside. By all means, provide a reference implementation: “Other teams use Target Process to track work.” Have default recommendations: “We use Clojure here unless there’s some better solution. SQL Server is our first choice database for these reasons.” Give teams a strong process and technology stack to start from, and let them innovate from there.

“Good fences make good neighbors” not by keeping out, but by channeling traffic into commonly understood and commonly accepted directions. We recognize lines so as to influence those aspects we truly need to influence. More importantly, we recognize lines to prevent needless conflict and waste. The key is to draw the lines so that they work for us, not against us.

Problem Space, Solution Space, and Tunnel Vision

Peter Kretzman‘s tweet about Sir Roland and his lightweight mini-shield brought both a smile to my face and the idea for this post. That idea actually has little to do with #NoEstimates (which I’ve touched on previously) and everything to do with architectural design. The cartoon highlights a design dysfunction that frequently manifests in systems:

Sir Roland has a point in that his shield is far easier to carry and wield than the traditional non-agile shield. Unfortunately, due to his tunnel vision, he probably won’t discover that the aspects that he focused on were peripheral to overall solution (i.e. keeping sharp implements out of Sir Roland’s innards) until it’s too late. Learning via experimentation is a powerful technique, but analysis has its place too, particularly when the value at risk is high.

Just as software is a system, so too are organizations (admittedly, systems that run on far less deterministic “hardware”, but systems nonetheless). Designing systems, particularly those that involve multiple stakeholders (i.e. nearly all that have more than one person involved with them), involves designing the solution space to best match the problem space. Note that I didn’t say “perfectly match the problem space”, as this conjunction is, in my opinion, unlikely to occur and should it occur, highly unlikely to persist. That being said, getting and keeping that match as close as possible to the theoretical perfect one is important. When a stakeholder’s influence on the solution is out of balance with their centrality to the problem, expect conflict.

IT’s traditional customer service is a notorious example of this type of imbalance at the organizational level. That imbalance also manifests in the technology realm in the forms of choosing solutions on the basis of justifying sunk costs, being apathetic toward user experience, chasing the tool/technique of the day, experimenting at the expense of the customer, and indulging in egotism.

Value for the owner of the system is a better tool to keep the proper balance. As the owner(s) should be the stakeholder(s) central to the problem space, where the solution is geared toward their needs it will most likely be well aligned to the problem. Where the concerns of peripheral stakeholders provide benefit to the central stakeholder(s) is where those concerns become important to the solution.

Unlike Sir Roland, failure to choose wisely may not be literally fatal, but it could be figuratively so.