Babies, Bathwater, and Software Architects

'Fixing Problems' - XKCD 1739

I try to be disciplined about my writing (picking themes, creating a backlog, collecting notes and links on those topics, etc.), but it seems like serendipity won’t be denied, no matter what I do. On the same day that XKCD published this cartoon, Erik Dietrich published “Software Architect as a Developer Pension Plan”. While I agree with a lot that Erik says in the post, I think he also gets into a “throwing out the baby with the bathwater” situation by dismissing the need for the software architect role, part of which the cartoon illustrates.

Before I go any farther, I do need to point out that I’ve been following and interacting with Erik for about four years now. It’s a friendly disagreement; meant to be more debate than argument.

In the post, Erik makes the point (which I largely agree with) that companies see the role of software architect through a Taylorist lens. They see the software architect as a “thinker”, meant to drive a herd of “doers”:

The modern corporation, like Taylor before it, loosely divides into three categories of humans: owners/executives, managers, and grunts. Owners own and charge executives with executing their will. Executives delegate to managers. Managers think and assemble the workers into org charts designed to optimize efficiency. Workers keep their simple heads down and do as they’re told.

Theoretically, proponents would say, this applies to any domain and type of work. Corporations form themselves into these pyramids without considering any other options, whether they’re private military outfits, gigantic manufacturing facilities, or companies designing and selling software. Of course, the reality turns out to be that not all operations do well splitting thinking and labor. Let’s pick one at random. Oh, I dunno, say, writing software.

We try it. We hire junior developers to be directed by senior ones. And all of them fall in line under the watchful guise of a “tech lead,” who defers to a council of architects. And somewhere at the center of the organizational maelstrom stands The Lead Architect, directing elaborate bucket marches, water flows, and all sorts of magic, like Mickey Mouse in Fantasia. It’s a beautiful, cascading symphony of work flowing from the cleverest down to the simplest. Or… at least, that’s how it goes on paper.

Erik rightly argues that this model is fatally flawed. This is doubly so in the case of the example he gives, an individual offered a position as a software architect doing the thinking for a set of offshore “commodity” developers. Offshore development can be done well (that’s a post I’ll save for another day), but not using that model. Where Erik goes wrong, in my opinion, is in adopting this flawed definition of the software architect role.

Developers should be perfectly competent to do their own thinking. If they’re not, no software architect will be able to help. Even if that person were capable of handling the load of providing detailed directions for several developers, doing so would prevent them from attending to their own areas of concern. It would require being able to anticipate and make all the functional design decisions up front, as well as communicating them to multiple “typists” quicker than they could mindlessly hammer out the code, while still having time to consider all the requisite cross-cutting, quality of service considerations for an application. Do we really need to bother with experiments to determine that this is beyond improbable?

There is a separation between the developer role and software architect role, but it needs to be clearly understood as a difference in focus. A developer’s focus is more vertical, concentrating on implementing functionality while maintaining/improving/not harming the quality of service (aka the horrendously misnamed “non-functional”) aspects of the system. A software architect’s focus should be more horizontal, concentrating on the architecturally significant quality of service aspects that will enable the system to meet the needs of its stakeholders (or, at least, not unreasonably prevent the system from doing so).

The difference between the roles is a matter of thinking at different levels of abstraction, not of one being superior to the other. The two mindsets are both necessary and complementary. A high-quality architectural design without quality implementation cannot produce a high-quality system. Likewise, where there is implementation without architectural coordination, the quality of the resulting system is a matter of chance. For very small systems maintained by small, permanent teams, it may not be necessary for these roles to be distinct. In my experience, however, dealing with both the broadly general and the very specific at the same time scales poorly. Non-trivial systems quickly come to require architectural leadership, otherwise people are left fixing problems caused by problems fixed previously.

Leadership has been the main theme of almost all of my posts this month and has figured prominently in several of what’s been written this year. Many of these posts are assigned to the “Architectural Practice” category. This is because I absolutely believe that the software architect role is a leadership role. That being said, I don’t consider the Taylorist model to be effective leadership practice. In fact, I consider it an anti-pattern.

I’ve long been an advocate of a more collaborative model of practice. Rather than dictating, the software architect’s role should be one of consensus building and communication. Significant architectural disagreement indicates an issue with one or more members of the team. Significant uncertainty about the architecture indicates an issue with the software architect role.

Effective systems require both breadth and depth of effectiveness. This holds true for software systems as much as social systems. For the social systems producing software systems for use by other social systems (i.e. software development teams), it is imperative.

Leadership Anti-Patterns – The Thinker

'The Thinker' by Rodin

My interest in leadership, how it works and how it fails, goes back a long way. Almost as soon as I learned how to read, history, particularly military history, has been a favorite of mine. Captains and kings, their triumphs and their downfalls, fascinated me. The eleven years I served with the Henrico Sheriff’s Office honed that interest. It allowed me to go beyond theory and the hearsay attendant in just reading about leaders. It allowed me to both observe firsthand and to get practical experience of my own.

For most of the latter part of my career with the Sheriff’s Office, I led a unit responsible for providing support services for the jail (canteen, recreation, laundry, hair-care, maintenance, and coordination of volunteer programs). In addition to their primary function, members of my unit, as sworn staff, also served as a ready reserve to the security staff during the busiest parts of the day. It was a challenge for me, because I was responsible for managing a group of experts with diverse duties. It was a challenge for them, because the way I led was something of a change for them.

Like everyone, I started my career working a security platoon in the jail. My second posting, however, was in our training academy. Roughly half of my responsibilities involved record keeping and administration of the training program. The other half of my time was spent in class, delivering training. Something about that resonated with me; I loved it and it strongly influenced my leadership style.

When necessary, I was fully capable of issuing orders to be complied with immediately. The operative word here being “necessary”. What’s necessary under emergency conditions and what’s necessary the other ninety-nine percent of the two are two different things. The majority of the time, my leadership style involved more coaching and little or no dictating.

I still laugh when I think of the first time a subordinate came to me for a decision. After her laying out the problem, rather than serving up a solution, I asked a simple question, “What do you want to do about it?” The speechless expressions I received in response were priceless. Having someone in authority ask for her opinion was so utterly foreign as to be beyond belief.

[Note – this is not the same thing as “don’t bring me problems, bring me solutions”. That type of bumper sticker philosophy is a good way to have people avoid you when they don’t know what to do, which is when they actually need you the most (and likewise, when you most need to know what they’re dealing with).]

By making sure that those who worked under me knew the boundaries they had to work with, we were able to divide things into three groups: the routine that was unremarkable, that which they could deal with on their own and tell me later, and that which they needed to get me involved in up front. Even with the last category, I asked for opinions. If it was a decision that I could make, my bias was to go with their recommendation unless I had a really good reason not to (as I noted above, most were specialists and knew the details of their position far better than I, so overruling them solely because I was “in charge” would be a really bad idea). When the decisions were above my pay grade, I would still give them credit and endorse their recommendation. My goal was to have a unit that could operate as well when I was not there as when I was.

At this point, you may be wondering where’s the information about this “Thinker” anti-pattern. All the background above, is to serve as contrast to one of my colleagues, a lieutenant in charge of a jail security platoon. This person informed their platoon that they were paid to do the thinking and their subordinates were paid to do the doing. No one other than this lieutenant was allowed to make any decisions, period. While I doubt this person had ever heard of Taylorism, they had a firm grasp of the principles of it.

Under the most normal of circumstances, there’s much to criticize about The Thinker’s way of doing things. The method doesn’t scale. It places a lot of burden on the Thinker. Getting a bathroom break in between decisions to be made is hard enough, but vacation? Forget about it.

The thing about a jail is that it’s not the most normal of circumstances, so there’s an added disadvantage. In the event of a hostage situation (something that’s more of a risk in a correctional environment), if the Thinker was the hostage, then they would be in for a very long ordeal. Unless one of their subordinates decided to ignore the Thinker’s orders and call for help, they would be stuck until someone “authorized” to think reported for duty.

Ignorance of the law (of unintended consequences) is no excuse.