Technical Debt and Rolling Re-writes (Who Needs Architects?)

If you think building a system is challenging, try maintaining one.

Tom Cagley‘s recent post “Plan to Throw One Away Re-Read Saturday: The Mythical Man-Month, Part 11”, was a good reminder that while “technical debt” may be something currently on the radar for many, it’s far from a new phenomenon. The concept of instant legacy applications was in place when forty years ago when Frederick Brooks wrote his masterpiece, even if they weren’t called that. As Tom observed in the post:

Rarely is the first attempt useful to the end consumer, and the usefulness of that first attempt is less in the code than in the feedback it generates. Software development is no different. The initial conceptual design and anticipated technical architecture of a large project rarely stands up to the rigors of the discovery process, and those designs should be learned from and then thrown away.

The faulty assumptions and design flaws accumulate not only from sprint to sprint leading up to the initial release, but also from release to release. In spite of the fact that a product can be so seriously flawed, throwing it away and starting over is easier said than done. While sunk costs cannot be recovered, too sanguine an attitude towards them may not enhance your credibility with the customer. Having to pay for the same thing over and over can make them grumpy.

This sets up a dilemma, one that frequently leads to living with technical debt and attempting to incrementally patch it up. There are limits, however, to the number of band-aids that can be applied. This might make it tempting to propose a rewrite, but as Erik Dietrich stated in “The Myth of the Software Rewrite”:

Sure, they know things now that they didn’t know when they started on this code 3 years ago. But won’t the same thing be true in 3 years? Won’t the developers then be looking at the code and saying, “this is a mess — if only we knew in 2015 what we now know in 2018!” And, beyond that, what makes you think that giving the same group of people the same marching orders won’t result in the same kind of code?

The “big rewrite from scratch because this is a mess” is a losing strategy.

Fortunately, there is an alternative. Quoting Tom Cagley again from the same post as above:

If change is both inevitable and good (within limits), then both systems and organizations (a type of system) need to be engineered to support and facilitate change. Architecturally, techniques such as modularization, object-oriented design and other processes that foster simplification and incremental change create an environment in which change isn’t avoided, but rather encouraged.

While we may laugh at the image of changing a tire while the vehicle is in motion, it is an accurate metaphor. Customers expect flexibility and change on the go; waiting equals lost business. The keys to evolving in place are having an intentionally designed, modular architecture and an understanding of where the weaknesses lie. Both of these are concerns that reside squarely on the architect’s plate.

Modularity not only makes an application more easily maintainable via separation of concerns, but it also embraces change by making components replaceable. This is one of the qualities that has made microservices such a hot topic, although it would be a mistake to think that microservices are the only way (or best way in all cases) to achieve modularity.

Modularity brings benefits beyond the purely technical as well. Rewrites of a fraction of an application are more easily sold than big-bang efforts. Demonstrating forethought (while you can’t predict what the change will be, predicting the need for change is more of a sure thing) demonstrates concern for the customer’s welfare, which should make for a better relationship.

Being able to throw a system away a little at a time allows us to keep the car on the road while it changes and adapts to changing conditions.

, , , , , , , , , ,


Form Follows Function on SPaMCast 361


This week’s episode of Tom Cagley’s Software Process and Measurement (SPaMCast) podcast features Tom’s essay on software measurement and a Form Follows Function installment on the need for thought and intentionality in architectural design.

In SPaMCast 361, Tom and I discuss my post “Who Needs Architects? Because YAGNI Doesn’t Scale”.

Leave a comment

“Laziness as a Virtue in Software Architecture” on Iasa Global

Laziness may be one of the Seven Deadly Sins, but it can be a virtue in software development. As Matt Osbun observed:

Robert Heinlein noted the benefits of laziness:

See the full post on the Iasa Global Site (a re-post, originally published here).

, , , , , ,

Leave a comment

Don’t Teach Coding

Before the pitchforks and torches come out, allow me to clarify – don’t just teach coding.

Jeff Susna and Mark M captured the essence of what I’m talking about:

The article Mark M referenced, “Is coding the new literacy?”, had this to say:

So you might be forgiven for thinking that learning code is a short, breezy ride to a lush startup job with a foosball table and free kombucha, especially given all the hype about billion-dollar companies launched by self-taught wunderkinds (with nary a mention of the private tutors and coding camps that helped some of them get there). The truth is, code—if what we’re talking about is the chops you’d need to qualify for a programmer job—is hard, and lots of people would find those jobs tedious and boring.

But let’s back up a step: What if learning to code weren’t actually the most important thing? It turns out that rather than increasing the number of kids who can crank out thousands of lines of JavaScript, we first need to boost the number who understand what code can do. As the cities that have hosted Code for America teams will tell you, the greatest contribution the young programmers bring isn’t the software they write. It’s the way they think. It’s a principle called “computational thinking,” and knowing all of the Java syntax in the world won’t help if you can’t think of good ways to apply it.

Whether you call it computational thinking or computer literacy, understanding the high-level basics of the technology is what is useful. Conflating the ability to create Hello World in JavaScript with that understanding is both simplistic and counter-productive. Chris Granger, in “Coding is not the new literacy”, observed:

Being literate isn’t simply a matter of being able to put words on the page, it’s solidifying our thoughts such that they can be written. Interpreting and applying someone else’s thoughts is the equivalent for reading. We call these composition and comprehension. And they are what literacy really is.

Languages come and go. Technologies evolve. Concentrating exclusively on specific languages and technologies risks creating future obsolescence. The underlying concepts that must be understood to effectively use them, however, have longer lives. It’s the 21st century equivalent of the difference between giving someone a fish and teaching them to fish. Erik Dietrich in “Don’t Learn to Code — Learn to Automate” points out the broader utility of computer literacy:

It’s obtuse to suppose that a prerequisite for every job in the future will be the ability to implement sophisticated, specialized computer applications. But it’s not at all obtuse to suppose that, given the ubiquity of computing, a prerequisite for every job in the future will be the ability to recognize which tasks are better suited for humans and which for computers. Learn at least to recognize which parts of your job are a poor use of your time. After that, perhaps learn to use your ingenuity and creativity to automate using the tools that you know (such as googling for solutions, leveraging apps, etc). And, if you’ve come that far, maybe it’s time to roll up your sleeves and take the plunge into learning to code a little bit to help you along.

The need for greater numbers of people with computer literacy is real, as is the need for greater diversity within the ranks of those who work with technology. It serves no one to misrepresent the skills needed or the nature of those skills. People who know how to google for code but lack the ability to understand and evaluate what they get back are no better off than if they had never seen a computer. We owe them, and ourselves, better.

, ,

Leave a comment

Who Needs Architects? Well, Nobody Needs this Kind

The question above came up while recording SPaMCast 357 with Tom Cagley, and it’s an extremely important one. The post we were discussing, “Who Needs Architects? Because YAGNI Doesn’t Scale”, is one of many discussing the need for architectural design in software development. While I’m firmly convinced that the need is real, it should also be realized that there is a real danger in unilaterally imposing the design on a team.

Tom’s question about an “aristocracy of architects” was taken from his post “Re-Read Saturday: The Mythical Man-Month, Part 4 Aristocracy, Democracy and System Design”, part of a series in which he is reviewing Frederick Brooks’ The Mythical Man-Month. In the essay reviewed in this post, “Aristocracy, Democracy and System Design”, Brooks discussed the importance and value of conceptual integrity (i.e. a cohesive, unified design) to software systems. While I agree wholeheartedly that both architectural design and someone (or more than one someone) responsible for that design is necessary, I disagree that establishing an aristocracy is beneficial or even necessary. In fact, the portion labeled “Reality” on the graphic in Kelly Abuelsaad‘s tweet below, although talking about imposter syndrome, also illustrates why dictating design can be a bad idea.

One can certainly influence, even control the architecture of a system via a mandate. The problem with being given control is that no one can give effectiveness to go with it. As such, it’s brittle, subject to the limitations of the person given the authority and the compliance of those implementing the system. This brittleness exists even when the architect stays within their level of detail. Combining a dictatorial style with Big Design Up Front all but ensures failure.

In my experience, a participative, collaborative style of design yields better designs. In addition to benefiting from a variety of skills and experience, it also engenders greater understanding and ownership across the team. Arrogance, on the other hand, can be costly.

I firmly believe that a product will benefit from having someone whose focus is the cross-cutting, architecturally significant concerns. I also believe that part of that job is teaching and mentoring as well as listening to the rest of the team so that architectural awareness permeates the entire team. There are many aspects to being an architect, but being a dictator should not be one of them.

, , , ,

Leave a comment

Conflicts, Collaboration, and Customers

With Ron Jeffries comment on my “Negotiating Estimates” post, I think we’ve reached agreement on a common problem, even if we disagree (perhaps?) over how to fix it. As I noted in my short answer, my position is that abuse of estimates stems from a deeper issue. Changing a practice won’t eliminate the abuse. Changing culture will. Changing that culture (where possible) involves convincing the players that it’s in their best interests to change, which it is.

This is my biggest objection to #NoEstimates. It makes an illogical leap that the practice is responsible for problems rather than some underlying dysfunction without showing causality. This putting the cart before the horse risks encouraging people to espouse action that is probably both ineffective and detrimental to their relationship with their customer. The likeliest outcome I see from that is more problems, not fewer.

The word “customer” was a point of contention:

It’s my position that, for better or for worse, there is a customer/provider relationship. Rather than attempt to deny that, it makes more sense, in my opinion, to concentrate on having the best possible customer/provider relationship. While poor customer service is extremely common, I’d hope everyone can think of at least one person they enjoy doing business with, where the relationship is mutually beneficial. I know I certainly can, which stands as my (admittedly unscientific) evidence that the relationship need not be adversarial. A collaborative, mutually beneficial relationship works out best for both parties.

“Collaborative”, however, does not mean conflict free. In his post “Against Estimate-Commitment”, Jeffries states:

The Estimate-Commitment relationship stands in opposition to collaboration. It works against collaboration. It supports conflict, not teamwork.

I both agree and disagree.

There will be conflict. Different team members will have different wants. Different stakeholders will have different wants. The key is not in preventing conflict, but resolving it in a way agreeable to all parties:

Recognizing, rather than repressing conflicts might even be beneficial:

What is critical, is that the relationship between the parties not be one-sided (or even be perceived as one-sided). In-house IT often has the peculiar characteristic of seeming one-sided to both customers and providers. The classic model of IT provision is, in my opinion, largely to blame. It seems almost designed to put both parties at odds.

The problem can be fixed, however, with cultural changes that encompass both IT and the business units it serves. One such change is moving from a project-centric to a product-centric model that aligns the interests of both groups. This alignment cements the relationship, making IT a valuable partner in the process rather than an obstacle to be overcome to get what’s wanted/needed. Relationships are key to success. By structuring the system so that each group’s incentives, conflicts and all, are aligned to a common goal, the system can be made to work. We’ve certainly had plenty of examples of what we get from the opposite situation.

, , , , , , ,


Form Follows Function on SPaMCast 357


This week’s episode of Tom Cagley’s Software Process and Measurement (SPaMCast) podcast features Tom’s essay on mind mapping, Steve Tendon discussing the TameFlow methodology, and a Form Follows Function installment on the need for architectural design.

In SPaMCast 357, Tom and I discuss my post “Who Needs Architects? Because YAGNI Doesn’t Scale”.

Leave a comment


Get every new post delivered to your Inbox.

Join 164 other followers

%d bloggers like this: