Form Follows Function on SPaMCast 442

SPaMCAST logo

A new month brings a new appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

This week’s episode, number 442, features Tom’s excellent essay on capability teams (highly recommended!), followed by a Form Follows Function installment based on my post “Systems of Social Systems and the Software Systems They Create”. Kim Pries bats cleanup with a Software Sensei column, “Software Quality and the Art of Skateboard Maintenance”.

In this episode, Tom and I continue our discussion on the organizations as system concept and how systems must fit into their context and ecosystem. In my previous posts on the subject, I took more of a top down approach. With this post, I flipped things around to a bottom up view. Understanding how the social and software systems interact (including the social system involved in creating/maintaining the software system) is critical to avoid throwing sand in the gears.

You can find all my SPaMCast episodes using under the SPaMCast Appearances category on this blog. Enjoy!

Form Follows Function on SPaMCast 438

SPaMCAST logo

Once again, I’m making an appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

This week’s episode, number 438, features Tom’s essay on using sizing for software testing, Kim Pries with a Software Sensei column (canned solutions), and a Form Follows Function installment based on my post “Organizations as Systems and Innovation”.

In this episode, Tom and I discuss how systems must fit into their context and ecosystem, otherwise it can be like dropping a high-performance sports car engine into a VW Beetle. Disney-physics may work in the movies, but it’s unlikely to be successful in the real world. If all the parts don’t fit together, friction ensues.

You can find all my SPaMCast episodes using under the SPaMCast Appearances category on this blog. Enjoy!

You can’t always get what you want…

Lucifer

You can’t always get what you want
But if you try sometimes well you just might find
You get what you need

When it comes to systems, you can’t always get what you want, but you do get what you design (intentionally or not), whether it’s what you need or not.

In other words, the architecture of systems, both social and software, evolve through some combination of intentional design and accidental emergence. Regardless of which end of the continuum the system leans toward, the end result will reflect the decisions made (or not made) in relation to various stimuli. Regarding businesses (a social system), Ruth Malan, in her February 2012 “Trace in Sand” post, put it this way:

I have been talking about agility in terms of evolutionary ecology, but with the explicit recognition that companies, comprised after all of individuals, attempt to speed and alter and intervene and interject and intercede and (I’m looking for the right word here) shape evolutionary processes with intentional actions — concerted, but also emergent from more and less choreographed, actions and intentions. Being bumped along by the unpredictable interactions of others, some from within, but also from “invasive species” from other ecosystems looking for new applications for their adaptable, mutable capability set.

Organizations create and participate in business ecologies. They build up the relationships that stabilize parts of the broader ecosystem, and create conditions for organizational forms to thrive there. They create products, they create the seeds of the next generation of harvest. They produce variants on their family tree, to target and develop niches.

Ruth further notes that while business adapt and improve in some cases, in other case they have “…become too closely adapted to and integrated within an ecosystem that has been replaced or significantly restructured by some landscape reshaping change…”. People generally refer to this phenomenon as disruption and the way they refer to it would seem to imply that it’s something that happens to or is done to a company. The role of the organization in its own difficulties (or demise) isn’t, in my opinion, well understood.

Last Friday, I saw a tweet from Noah Sussman that provides a useful heuristic for predicting the behavior of any large social system:

the actual reason behind the behavior:

It’s not that people are actively working to harm the organization, but that when there is no leadership, where there is no design, where there is no learning, the system ossifies and breaks down. Being perfectly adapted to an ecosystem that no longer exists is indistinguishable from being poorly adapted to the present context. I’m reminded of what Tom Graves stated in “The game of enterprise-architecture”: “things work better when they work together, on purpose”.

Without direction, entropy emerges where coherence is needed.

This is not to say, however, that micro-management is the answer. Too much design/control is as toxic as too little. This is particularly the case when the management system isn’t intentionally designed. Management that is both ad hoc and rigid can cause new problems while trying to solve existing ones. This is illustrated by another tweet from Friday:

The desire to avoid the “…embarrassment of cancellation” led to the decision to risk the lives of a plane load of “…foreign TV and radio journalists and also other foreign notables…”. I suspect that the fiery deaths of those individuals would have been an even bigger embarrassment. The system, however, led to the person who had the decision-making power to take that gamble.

The system works the way you built it, even when you didn’t intend to build it to work that way.

Microservices, Monoliths, and Modularity

Iceberg

 

There are very valid reasons for considering a microservice architecture (MSA) when building/evolving an application. In my opinion, however, forcing modularity isn’t one of those very valid reasons.

Just the other day, I saw tweet from Simon Brown saying this same thing:

I still like his comment from two years back: “I’ll keep saying this … if people can’t build monoliths properly, microservices won’t help”. I believe that if you’re having problems building a monolith properly, trying to use a distributed architecture to force modularity may actually cause harm.

MSAs, like any distributed application architecture, involve increased complexity and costs; table stakes, if you will. Like an iceberg, there’s both a lot more to it than just what’s showing above the waterline and a fair amount of hazard for the unwary. If a development team cannot or will not comply with design guidelines (e.g. modularity requirements), injecting additional complexity is probably not the solution you need.

Distributing an application makes it harder to accidentally entangle different concerns, but it doesn’t make it impossible:

I’d argue that making it harder to accidentally break modularity addresses neither of the groups I mentioned earlier: those that cannot or will not comply. It’s ironic, but those who fail to understand the need for modularity can be very creative in their “solutions”, regardless of the obstacles. Likewise, those who refuse to comply.

In short, distribution as a means of “ensuring” modularity fails the fitness for purpose test.

The situation becomes worse when you factor in the additional complexity inherent in a distributed system. Likewise, there’s the cost of the table stakes (infrastructure, process, staffing, etc.) mentioned above. Of course, having abandoned the principle of cause and effect, one could attempt some “creative” workarounds to avoid having to pay the price (in other words, adding more and more complexity).

When you introduce significant additional complexity (with all its attendant risk) with little chance of the technique actually achieving its goal, you’ve caused harm.

These concerns are not solely limited to the application architecture. Distributing the data architecture has the same limitations in terms of ensuring modularity and introduces additional complexity. Adding boundaries adds the need for governance. A disciplined, monolithic team can maintain modularity in a monolithic data architecture. Multiple separate teams trying to share a monolithic data architecture will either experience a crippling level of governance overhead or a complete breakdown in modularity.

MSAs can be useful when you need independently scalable and replaceable components. When you have multiple teams working on one logical application, they can also be appropriate as well. Using the technique when the cost outweighs the potential payoff, however, is a losing bet.

When One System Fails Another

Robinson Crusoe Shipwrecked

Ten days ago, when I wrote the post “Uber and the Cost of a Culture of Corruption”, I said that assuming there will be negative consequences (both legal and financial) from the incidents in the news, then it is in Uber’s best interests to fix the problem that led to them in the first place. The negative consequences are now becoming visible in the form of people abandoning ship.

Over the weekend, Uber’s president, Jeff Jones, resigned with the following statement:

I joined Uber because of its Mission, and the challenge to build global capabilities that would help the company mature and thrive long-term.

It is now clear, however, that the beliefs and approach to leadership that have guided my career are inconsistent with what I saw and experienced at Uber, and I can no longer continue as president of the ride sharing business.

There are thousands of amazing people at the company, and I truly wish everyone well.

Travis Kalanick’s announcement to Uber’s employees, while factually accurate (the decision did come after the announcement of the search for a COO), doesn’t quite convey Jones’ reasons for leaving:

Team,

I wanted to let you know that Jeff Jones has decided to resign from Uber.

Jeff joined Uber in October 2016 from being CMO at retailer Target. In 6 months, he made an important impact on the company—from his focus on being driver obsessed to delivering our first brand reputation study, which will help set our course in the coming months and year.

After we announced our intention to hire a COO, Jeff came to the tough decision that he doesn’t see his future at Uber. It is unfortunate that this was announced through the press but I thought it was important to send all of you an email before providing comment publicly.

Rachel, Pierre and Mac will continue to lead the Global Ops teams, reporting to me until we have signed a COO. Troy Stevenson, who leads CommOps, and Shalin Amin who leads brand design will report to Rachel Holt. Ab Gupta will report to Andrew MacDonald.

Thanks,

Travis

Jones is not the only Uber executive to resign this weekend. Brian McClendon, vice president in charge of Uber’s mapping, is “…leaving to return to his hometown in Kansas”.

These resignations are also not the only recent executive casualties. Ed Baker, vice president of product and growth, had also announced his departure earlier this month amid questions regarding his conduct with other Uber employees. This came after senior vice president of engineering Amit Singhal was asked to resign when it was discovered that he failed to disclose that he was investigated for sexual harassment while at Google.

It’s cliché to talk about people as assets, but for companies like Uber, their talent really does comprise the majority of their value. While the media will take note of high-profile departures like these, it would be a mistake to consider them the entirety of the damage. How many lesser known employees have left or will be leaving as a result of the recent scandals? How much potential talent will pass by opportunities at Uber due to what’s happened? In particular, how much harder will this make Kalanick’s search for a Chief Operating Officer who can turn things around?

If the talent drain is not quickly plugged, what happens to the quality of Uber’s service?

This situation perfectly illustrates the theme of organizations as systems. Uber’s software and business model have done well for it, but the culture created by the lack of leadership and lack of ethics of its management may well sink it. One bad component can bring down a system, whether software or social. The tragedy is that the innocent would be harmed along with the guilty.

Uber and the Cost of a Culture of Corruption

'Personification of the Faculty of Law' from the pedestal of the statue of Emperor Charles IV, Prague, Czech Republic - via Wikimedia Commons

Even before I hit the “Publish” button on Monday’s post, “Regulating Software Development”, I had already started composing this post in my head. In that post I had used the words “corrupt culture” in passing. I needed to expand on that, because I believe that’s what lies at the heart of Uber’s cascading collection of scandals.

Uber’s business model has always displayed a certain flexible attitude towards government regulation. Greyball represented a departure from dancing on the line to barging over it. Caught with their hand in the cookie jar, Uber has now announced “We are expressly prohibiting its use to target action by local regulators going forward”. I doubt this act of contrition on their part will be deemed sufficient.

In this environment, Susan Fowler’s account of her time at Uber becomes less of a “how could they be so stupid” story and more of a foregone conclusion. When, to all appearances, violating the law is part of your business model and you’re building software intended to thwart enforcement of the laws you’re violating, your moral authority is rather thin. In this type of culture, I’d imagine things like unwanted sexual advances and retaliation aren’t seen as a big deal, rather business as usual. Crossing lines, like other activities, becomes easier with practice.

Assuming that there will be negative consequences (both legal and financial), from these incidents, then it is in Uber’s best interests to fix the problem that led to them in the first place. This means radically changing Uber’s culture, otherwise new problems will continue to arise. Uber’s CEO has announced that the company is looking for leadership assistance:

“This morning I told the Uber team that we’re actively looking for a Chief Operating Officer: a peer who can partner with me to write the next chapter in our journey,” Kalanick said in a statement on Tuesday.

It remains to be seen whether this will represent a radical change in leadership or not. Anything less than a radical change will be unlikely to affect the current culture which appears to be a deeply entrenched. Debbie Madden, CEO of Stride, published an open letter to Uber’s Travis Kalanick on Wednesday. In “Dear Travis Kalanick: Here’s What You Must Demand From Uber’s New COO”, she noted that “Uber’s culture is broken and you need help to fix it”. She outlined seven steps to do just that:

Step 1: Change Uber’s core values

Step 2: Kill Greyball

Step 3: Adopt a zero-tolerance harassment policy and fire offenders

Step 4: Hire a strong head of HR and an employment lawyer and educate employees

Step 5: Fire or PIP each manager and HR employee who turned a blind eye

Step 6: Shift focus from individual productivity to team productivity

Step 7: Change your recruiting process

Uber isn’t the only organization in trouble due to a troubled culture. Volkswagen is in the same boat and it isn’t over yet. It’s reported that the pollution hidden by their cheating on emissions testing could contribute to the early death of 1,200 Europeans. It’s a solid bet that there will be more litigation to come.

Uber hasn’t killed anyone as a result of their corporate culture, but with their interest in self-driving vehicles, we should all be rooting for a turn-around in the ethics department. Uber can only exist in the future by eliminating the Uber of the past.

Fear of Failure, Fear and Failure

Capricho 43, Goya's 'The Sleep of Reason Produces Monsters'

Some things seem so logically inconsistent that you just have to check them out.

Such was the title of a post on LinkedIn that I saw the other day: “Innovation In Fear-Based Cultures? Or, why hire lions to be dogs?”. In it, Michael Graber noted that “…top-down organizations have the most trouble innovating.”:

In particular, the fearful mindsets that review, align, and sign off on “decks” to be presented to Vice President-level colleagues often edit out the insights and recommendations that have the power to grow the business in new ways.

These well-trained, obedient keepers of the status quo are rewarded for not taking risks and for not thinking outside of the existing paradigm of the business.

None of this is particularly shocking, a culture of fear is pretty much the antithesis of a learning culture and innovation in the absence of a learning culture is a bit like snow in the desert – not impossible, but certainly remarkable.

Learning involves risk. Whether the method is “move fast and break things” or something more deliberate and considered (such as that outlined in Greger Wikstrand‘s post “Jobs to be done innovation”), there is a risk of failure. Where there is a culture of fear, people will avoid all failure. Even limited risk failure in the context of an acknowledged experiment will be avoided because people won’t trust in the powers that be not to punish the failure. In avoiding this type of failure, learning that leads to innovation is avoided as well. You can still learn from what others have done (or failed to do), but even then there’s the problem of finding someone foolhardy enough to propose an action that’s out of the norm for the organization.

Why would an organization foster this kind of culture?

Seth Godin’s post, “What bureaucracy can’t do for you”, holds the key:

It lets us off the hook in many ways. It creates systems and momentum and eliminates many decisions for its members.

“I’m just doing my job.”

“That’s the way the system works.”

Decisions involve risk, someone could make the wrong one. For that reason, the number of people making decisions should be minimized (not a position I endorse, mind you).

That’s the irony of top-down, bureaucratic organizations – often the culture is by design, intended to eliminate risk. By succeeding in doing so on the mundane level, the organization actually introduces an existential risk, the risk of stagnation. The law of unintended consequences has a very long arm.

This type of culture actually introduces perverse incentives that further threaten the organization’s long-term health. Creativity is a huge risk, you could be wrong. Even if you’re right, you’ve become noticeable. Visibility becomes the same as risk. Likewise, responsibility means appearing on the radar. This not only discourages positive actions, but can easily be a corrupting influence.

Fear isn’t the only thing we have to fear, but sometimes it’s something we really need to be concerned about.


This post is another installment of an ongoing conversation about innovation with Greger Wikstrand.