When One System Fails Another

Robinson Crusoe Shipwrecked

Ten days ago, when I wrote the post “Uber and the Cost of a Culture of Corruption”, I said that assuming there will be negative consequences (both legal and financial) from the incidents in the news, then it is in Uber’s best interests to fix the problem that led to them in the first place. The negative consequences are now becoming visible in the form of people abandoning ship.

Over the weekend, Uber’s president, Jeff Jones, resigned with the following statement:

I joined Uber because of its Mission, and the challenge to build global capabilities that would help the company mature and thrive long-term.

It is now clear, however, that the beliefs and approach to leadership that have guided my career are inconsistent with what I saw and experienced at Uber, and I can no longer continue as president of the ride sharing business.

There are thousands of amazing people at the company, and I truly wish everyone well.

Travis Kalanick’s announcement to Uber’s employees, while factually accurate (the decision did come after the announcement of the search for a COO), doesn’t quite convey Jones’ reasons for leaving:


I wanted to let you know that Jeff Jones has decided to resign from Uber.

Jeff joined Uber in October 2016 from being CMO at retailer Target. In 6 months, he made an important impact on the company—from his focus on being driver obsessed to delivering our first brand reputation study, which will help set our course in the coming months and year.

After we announced our intention to hire a COO, Jeff came to the tough decision that he doesn’t see his future at Uber. It is unfortunate that this was announced through the press but I thought it was important to send all of you an email before providing comment publicly.

Rachel, Pierre and Mac will continue to lead the Global Ops teams, reporting to me until we have signed a COO. Troy Stevenson, who leads CommOps, and Shalin Amin who leads brand design will report to Rachel Holt. Ab Gupta will report to Andrew MacDonald.



Jones is not the only Uber executive to resign this weekend. Brian McClendon, vice president in charge of Uber’s mapping, is “…leaving to return to his hometown in Kansas”.

These resignations are also not the only recent executive casualties. Ed Baker, vice president of product and growth, had also announced his departure earlier this month amid questions regarding his conduct with other Uber employees. This came after senior vice president of engineering Amit Singhal was asked to resign when it was discovered that he failed to disclose that he was investigated for sexual harassment while at Google.

It’s cliché to talk about people as assets, but for companies like Uber, their talent really does comprise the majority of their value. While the media will take note of high-profile departures like these, it would be a mistake to consider them the entirety of the damage. How many lesser known employees have left or will be leaving as a result of the recent scandals? How much potential talent will pass by opportunities at Uber due to what’s happened? In particular, how much harder will this make Kalanick’s search for a Chief Operating Officer who can turn things around?

If the talent drain is not quickly plugged, what happens to the quality of Uber’s service?

This situation perfectly illustrates the theme of organizations as systems. Uber’s software and business model have done well for it, but the culture created by the lack of leadership and lack of ethics of its management may well sink it. One bad component can bring down a system, whether software or social. The tragedy is that the innocent would be harmed along with the guilty.

Uber and the Cost of a Culture of Corruption

'Personification of the Faculty of Law' from the pedestal of the statue of Emperor Charles IV, Prague, Czech Republic - via Wikimedia Commons

Even before I hit the “Publish” button on Monday’s post, “Regulating Software Development”, I had already started composing this post in my head. In that post I had used the words “corrupt culture” in passing. I needed to expand on that, because I believe that’s what lies at the heart of Uber’s cascading collection of scandals.

Uber’s business model has always displayed a certain flexible attitude towards government regulation. Greyball represented a departure from dancing on the line to barging over it. Caught with their hand in the cookie jar, Uber has now announced “We are expressly prohibiting its use to target action by local regulators going forward”. I doubt this act of contrition on their part will be deemed sufficient.

In this environment, Susan Fowler’s account of her time at Uber becomes less of a “how could they be so stupid” story and more of a foregone conclusion. When, to all appearances, violating the law is part of your business model and you’re building software intended to thwart enforcement of the laws you’re violating, your moral authority is rather thin. In this type of culture, I’d imagine things like unwanted sexual advances and retaliation aren’t seen as a big deal, rather business as usual. Crossing lines, like other activities, becomes easier with practice.

Assuming that there will be negative consequences (both legal and financial), from these incidents, then it is in Uber’s best interests to fix the problem that led to them in the first place. This means radically changing Uber’s culture, otherwise new problems will continue to arise. Uber’s CEO has announced that the company is looking for leadership assistance:

“This morning I told the Uber team that we’re actively looking for a Chief Operating Officer: a peer who can partner with me to write the next chapter in our journey,” Kalanick said in a statement on Tuesday.

It remains to be seen whether this will represent a radical change in leadership or not. Anything less than a radical change will be unlikely to affect the current culture which appears to be a deeply entrenched. Debbie Madden, CEO of Stride, published an open letter to Uber’s Travis Kalanick on Wednesday. In “Dear Travis Kalanick: Here’s What You Must Demand From Uber’s New COO”, she noted that “Uber’s culture is broken and you need help to fix it”. She outlined seven steps to do just that:

Step 1: Change Uber’s core values

Step 2: Kill Greyball

Step 3: Adopt a zero-tolerance harassment policy and fire offenders

Step 4: Hire a strong head of HR and an employment lawyer and educate employees

Step 5: Fire or PIP each manager and HR employee who turned a blind eye

Step 6: Shift focus from individual productivity to team productivity

Step 7: Change your recruiting process

Uber isn’t the only organization in trouble due to a troubled culture. Volkswagen is in the same boat and it isn’t over yet. It’s reported that the pollution hidden by their cheating on emissions testing could contribute to the early death of 1,200 Europeans. It’s a solid bet that there will be more litigation to come.

Uber hasn’t killed anyone as a result of their corporate culture, but with their interest in self-driving vehicles, we should all be rooting for a turn-around in the ethics department. Uber can only exist in the future by eliminating the Uber of the past.

Regulating Software Development

'Belvidere Street construction, pouring concrete', Library of Virginia


Another weekend, another too good to pass up Twitter conversation during my “unplugged” time. This weekend, Grady Booch hooked me by retweeting Mike Potts tweet:

Mike’s tweet was a reply to Grady’s comment on the latest news out of Uber:

It’s an understandable question. It’s a reasonable question. It’s one that came up back during the healthcare.gov fiasco and it’s one raised by Volkswagen’s recent criminal misconduct.

However, when contemplating fixing a problem, we need to be extremely mindful of the potential for creating harm as a result of the “fix”. Particularly we should be wary of creating harm out of proportion to any good we do (i.e. we don’t want to kill roaches by burning down the house). I chose the image at the top to illustrate something key to this discussion – changing laws (the software of our meta-enterprise) is only slightly harder than moving a roadway once laid down.

Now for the caveats:

  • I do my utmost to avoid politics on this site – I really doubt you’re looking to me for guidance or even just my opinion. I’m not intending this post as a political statement. I’m not asserting that government is never the answer, merely that it’s a rather blunt instrument that we need to use with care.
  • I agree with Grady and Mike that those who took part in this are a disgrace. Moreover, I believe everyone involved, top to bottom, needs to be prosecuted and, if convicted, punished to the fullest extent of the law.
  • My tl;dr position is this: if we have regulation, it should be effective and without avoidable harmful side effects.

As I noted above, it’s human nature to respond to problems with some proposal to fix the problem. It also seems to be human nature to respond in a manner that doesn’t necessarily deal with an issue from a systemic perspective. We tend to allow ourselves to concentrate on the need to “do something” and ignore the hard work of making sure what we do is effective (and doesn’t cause further problems). In other words, we put band aids on bullet wounds.

In both the case of VW and Uber, the conduct alleged is criminal. We could pass new laws making it a crime to commit a crime, but that seems to be an exercise in recursive futility. If the potential penalty in the first case was insufficient to induce compliance, should we really believe adding another layer will make it better?

An element that’s present in both cases is that the illegal conduct involves creating software to help avoid detection of the fact that the company was breaking another law. Regulatory pressures coupled with a corrupt culture can create perverse incentives to cheat. This does not in any way excuse the conduct, particularly in the case of VW. It is, however, one of the systemic factors that should be taken into account.

In my experience, the most effective compliance program is one where compliance is the path of least resistance. Self-imposed compliance cannot fail to be more effective than compliance enforced externally. Corrupt agents will still violate the rules, but ideally you want to make it so that the lazy way out is the desired behavior.

Another aspect of regulation that comes up is something along the lines of professional standard similar to those of attorneys, accountants, and doctors. Increasing the level of professionalism is laudable, but would it be an effective response to the issue of criminal misconduct? Additionally, assuming it was legally enforced, what would the cost be? Everything from administration of the program to salary increases would introduce new costs and would likely affect the pace of innovation (due to the impact on both supply and demand). Again, without justifying the conduct, what was Uber’s motivation to develop its code to defeat detection by regulators?

I can well imagine other potential issues with a regulatory regime that requires a license to code. Not only commercial innovation would suffer, but the effects on the Open Source community could be disastrous if the licensing regime was expensive.

Doing “something” is easy. Doing something effective is a bit harder. I’m all aboard for punishing the guilty (each and every one), but we should move carefully when considering actions that might be more difficult to undo.

Volkswagen and the Cost of Culture

Hand holding a wad of cash

Thanks to Volkswagen, we now have an idea of the cost of failing to maintain an ethical culture, roughly $18 billion US (emphasis added in the quoted text below by me):

Volkswagen’s financial disclosure on Friday, in a preliminary earnings report, came a day after the company agreed on the outlines of a plan to settle some legal claims in the United States, which would include giving owners of about 500,000 affected vehicles the option of selling the cars back to the company or having them repaired.

Volkswagen is still negotiating the size of the fines it will pay to the United States government for violations of clean-air laws, as well as how much additional compensation it will provide to owners. The money set aside by the company on Friday provides an indication of what Volkswagen expects the total global costs of the scandal to be, although the figure could rise further.

Since the scandal broke in September, 2015, the news has steadily worsened. Last December, Volkswagen’s chairman admitted that the cheating found was not an isolated lapse:

…the decision by employees to cheat on emissions tests was made more than a decade ago, after they realized they could not meet United States clean air standards legally.

Hans-Dieter Pötsch, the chairman of Volkswagen’s supervisory board, said the cheating took place in a climate of lax ethical standards.

“There was a tolerance for breaking the rules,” Mr. Pötsch said here on Thursday during his first lengthy news conference since the company admitted in September that 11 million cars with diesel engines were rigged to fool emissions tests.

Volkswagen’s executive leadership explanation at the time:

Mr. Müller and Mr. Pötsch conceded that the deception reflected organizational shortcomings.

For example, the people who developed the software were the same ones who approved it for use in vehicles. At other companies, it is standard practice for one team to develop components and another to check them for quality. Volkswagen said it would correct those procedures.

Mr. Müller also said he wanted to change the company’s culture so that there was better communication among employees and more willingness to discuss problems. His predecessor, Martin Winterkorn, who resigned after the scandal, was criticized for creating a climate of fear that made managers afraid to admit mistakes.

“We don’t need yes men,” Mr. Müller said, “but managers and engineers who make good arguments.”

I would argue that what’s needed more than “good arguments” is a corporate culture where it’s understood that refusing to break the law is not only allowed, but expected. Given that the size of the loss reserve has more than doubled since then, perhaps they’ve realized that now as well.

What is not needed, however, is the traditional response to high-profile issues, layering on additional ad hoc rules and regulations with an eye toward making sure this “never again happens”. For one thing, there’s no indication that anyone was not aware of the fact that this behavior was wrong. Additional compliance theater is unlikely to improve anything in that respect, and may actually cause new problems in addition to exacerbating the root problem, VW’s culture.

A recent study (reported on in The Atlantic) by Simon Gächter and Jonathan Schulz, University of Nottingham, reports that corrupt cultures breeds corruption. In this study, they:

…asked volunteers from 23 countries to play the same simple game. The duo found that participants were more likely to bend the game’s rules for personal gain if they lived in more corrupt societies. “Corruption and fraud are things going on in the social environment all the time, and it’s plausible that it shapes people’s psychology, what they can get away with,” says Gächter. “It’s okay! Everybody does it around here.”

This study also has implications for Volkswagen’s ability to fix the problem:

Causality could eventually flow in the other direction. “If people are dishonest or think it’s okay to violate rules, it would also be harder to fight corruption and install institutions that work,” says Gächter. “In the long run, these things move together. But to show that, you’d need a 20 year project measuring this on an annual basis.”

Volkswagen, however, probably does not have twenty years to fix their problem. In the US alone, VW will have to fix or buy back (at the owner’s option) over 500,000 vehicles. I suspect VW’s reputation is severely impaired with those that opt to have their vehicles repaired and I would be willing to bet that the majority of those who sell their cars back won’t be returning as customers. This loss for Volkwagen is only the beginning of their financial problems, and it could all have been avoided.

Back in November, Matt Balantine floated an interesting (and very plausible) theory:

That may well turn out to be the case, but I also have to agree with what Grady Booch tweeted when the scandal first broke:

There’s plenty of blame to go around, but ultimately I believe only a systemic fix, top to bottom, will have any chance of correcting the problem (not that I’d be willing to give VW very good odds on remaining in business long enough for that to take effect). Their value going forward may be to serve as empiric confirmation of Gächter and Schulz’s work. Their bad example may serve as a wake up call for others to pay attention to the culture they’ve fostered (and are fostering) before their employees, innocent and guilty alike, pay the price.

“Want Fries with That?”

Hamburger and French Fries

Greger Wikstrand and I have been trading posts about architecture, innovation, and organizations as systems (a list of previous posts can be found at the bottom of the page) for quite a while now. His latest, “Technology permeats innovation”, touches on an important point – the need for IT to add value and not just act as an order taker.

It’s funny how this series of innovation posts keeps taking me back to posts from the early days of this blog. In my last post, “Accidental Innovation?”, I referred to my very first post, “Like it or not, you have an architecture (in fact, you may have several)”. Less than a month after that first post, I published “Adding Value”, which had the exact same theme as Greger’s post: blindly following orders without adding value (in the form of technical expertise) is not serving your customer. In fact, failing to bring up concerns is both unprofessional and unethical. Acceding to a request that you know will harm your customer without pushing back is tantamount to sabotage.

Innovation involves multiple disciplines. In a recent tweet, Brenda Michelson illustrated this important truth in the context of digital technology:

Both Brenda and Greger make the same point – successful innovation is a team effort. In fact, using Scott Berkun’s definition of the word, it’s redundant to say “successful innovation”:

If you must use the word, here is the best definition: Innovation is significant positive change. It’s a result. It’s an outcome. It’s something you work towards achieving on a project. If you are successful at solving important problems, peers you respect will call your work innovative and you an innovator. Let them choose the word.

In a recent series of post, Casimir Artmann noted that innovation comes in many forms: improving existing products, developing new products, and finding better ways to work. Often, as shown in examples of innovation in music, photography, and telephony, innovation comes from a combination of these forms. He sums it up this way:

Regardless if we talk about innovation for existing products, new products or new ways of working, inventions in technology is one of the drivers.

Internet of Things, Cloud, Autonomous devices, Wearables, Big Data etc, are all enablers for innovation in the organisations. The challenge is to find out the benefit our clients customers will have from these technology enablers.

Meeting that challenge requires integrating the expertise of both business and IT. Innovation and value aren’t picked from a menu and served up at a drive-through.

Previous posts in this series:

  1. “We Deliver Decisions (Who Needs Architects?)” – I discussed how the practice of software architecture involved decision-making. It combines analysis with the need for situational awareness to deal with the emergent factors and avoiding cognitive biases.
  2. “Serendipity with Woody Zuill” – Greger pointed me to a short video of him and Woody Zuill discussing serendipity in software development.
  3. “Fixing IT – Too Big to Succeed?” – Woody’s comments in the video re: the stifling effects of bureaucracy in IT inspired me to discuss the need for embedded IT to address those effects and to promote better customer-centricity than what’s normal for project-oriented IT shops.
  4. “Serendipity and successful innovation” – Greger’s post pointed out that structure is insufficient to promote innovation, organizations must be prepared to recognize and respond to opportunities and that innovation must be able to scale.
  5. “Inflection Points and the Ingredients of Innovation” – I expanded on Greger’s post, using WWI as an example of a time where innovation yielded uneven results because effective innovation requires technology, understanding of how to employ it, and an organizational structure that allows it to be used well.
  6. “Social innovation and tech go hand-in-hand” – Greger continued with the same theme, the social and technological aspects of innovation.
  7. “Organizations and Innovation – Swim or Die!” – I discussed the ongoing need of organizations to adapt to their changing contexts or risk “death”.
  8. “Innovation – Resistance is Futile” – Continuing on in the same vein, Greger points out that resistance to change is futile (though probably inevitable). He quotes a professor of his that asserted that you can’t change people or groups, thus you have to change the organization.
  9. “Changing Organizations Without Changing People” – I followed up on Greger’s post, agreeing that enterprise architectures must work “with the grain” of human nature and that culture is “walking the walk”, not just “talking the talk”.
  10. “Developing the ‘innovation habit’” – Greger talks about creating an intentional, collaborative innovation program.
  11. “Innovation on Tap” – I responded to Greger’s post by discussing the need for collaboration across an organization as a structural enabler of innovation. Without open lines of communication, decisions can be made without a feel for customer wants and needs.
  12. “Worthless ideas and valuable innovation” – Greger makes the point that ideas, by themselves, have little or no worth. It’s one thing to have an idea, quite another to be able to turn it into a valuable innovation.
  13. “Accidental Innovation?” – I point out that people are key to innovation. “Without the people who provide the intuition, experience and judgement, we are lacking a critical component in the system.”
  14. “Technology permeats innovation” – Greger talks about how tightly coupled innovation and technology are and the need for IT to actively add value to the process.

First Do No Harm – the Practice of Software Development

Medieval Anatomy Illustration

Analogies are never perfect, but reading Erik Dietrich’s “Do Programmers Practice Computer Science?” brought one to mind. Software development has much in common with the practice of medicine. Software development, like medicine, involves the application of knowledge. Also like medicine, this application is made complex by considerations of context. Yet another commonality is that in both disciplines, there are (or, at least, should be, limits regarding experimentation).

Erik’s post used the following comparison of developers to electricians:

Let’s consider three actors in the realm of physics, as a science.

  1. A physicist, who runs electricity through things to see if they explode.
  2. An electrical engineer, who takes the knowledge of what explodes from the physicist and designs circuitry for houses.
  3. An electrician, who builds houses using the circuits designed by the electrical engineer.

I list these out to illustrate that there are layers of abstraction on top of actual science. Is an electrician a scientist, and does the electrician use science? Well, no, not really. His work isn’t advancing the cause of physics, even if he is indirectly using its principles.

Let’s do a quick exercise that might be a bit sobering when we think of “computer science.” We’ll consider another three actors.

  1. Discrete mathematician, looking to win herself a Fields medal for a polynomial time factoring algorithm.
  2. R&D programmer, taking the best factoring algorithms and turning them into RSA libraries.
  3. Line of business programmer, securing company’s Sharepoint against script kiddies uploading porn.

Programming is knowledge work and non-repetitive, so the comparison is unfair in some ways. But, nevertheless, what we do is a lot more like what an electrician does than what a scientist does. We’re not getting paid to run experiments — we’re getting paid to build things.

There is definitely some validity in this. The three roles in each example have many similarities. His observation that development work is “non-repetitive”, however, is key. Electricians work in a more certain context than doctors who may need to account for body chemistry or metabolism. Likewise, developers may find environmental factors (e.g. memory usage profile, network load, etc.) produce uncertainty in the course of their work. Whereas the plumbing and electrical systems in a house are mostly separate, biological systems and information systems tend to be more intertwined.

Another similarity between software development and the practice of medicine is the feedback loop. The physicist will never hear back from the electrician, but physicians doing research are not similarly removed from practitioners. Practice and theory in medicine have a chicken and egg relationship where neither is clearly dominant, but each influences the other. Likewise with software development. Ethics and practicality in both cases constrain pure research.

As Erik noted, developers are “…not getting paid to run experiments — we’re getting paid to build things”. That being said, the uncertainties mean that, like physicians, we can’t be positive about the exact outcome without trying a particular course of action (which isn’t really an experiment):

Like doctors, those involved in software development have an ethical obligation to let our “patients” know when we’re learning on the job and what the risks are (not to mention the obligation to try things that are in their best interests and not just something we want to test drive). In addition to considerations of professionalism, more open communication has its benefits. We can solve problems and advance the practice at the same time.

Design Follies – Architect Knows Best

Carmen Miranda

Last (for now), but most definitely not least of the design follies is putting your own “vision” above the needs of the customer. Worse than falling for the latest technology fad or failing to adequate think things through, from an ethical standpoint, putting your ego ahead of your duty to your customer is as bad as making current design decisions with an eye to trying to justify prior mistakes. None of these reflect favorably on the perpetrator.

Whenever I consider this particular anti-pattern, I tend to remember the reality TV series Trading Spaces. In contrast to most of the designers on the show, one designer was renowned for ignoring the wishes of the owners, at some times deliberately doing things he was asked not to. Acting the diva might make for good television, but is abhorrent in terms of professionalism.

“Learn by shipping” can be a valid product development technique when dealing with the truly innovative. As the past two years in the operating system space have shown, that technique may not work as well in mature markets (note: phones/devices that can take on duties previously in the realm of personal computers = absolutely brilliant; computers downgraded to phone/tablet capabilities = not even close). Learning by listening can be much cheaper and just as effective. Giff Constable recently asserted that “…companies, whether startup or enterprise, that do not aggressively build learning into their processes will spend 3x to 5x more time and money…”. Failing to listen to pre-release criticism is, in my opinion, failure to learn at an opportune time.

Change represents both opportunity and danger, more so when we add in people’s reaction to change. The opportunity to innovate can disappear if we are insensitive to the customer’s potential reaction and the reasons for that reaction:

Imagine living in the same house for 10 years. Over that period, you’ve accumulated a lot of stuff.

To keep your house organized, you found places to put everything. Every place made sense to you. Most of the time, you have no trouble finding anything you want. Occasionally, there’s something you can’t find, like a tape measure, because you can’t remember where you last put it, but with a little poking around (and asking your housemates,) you come upon it and all is well.

One morning, you wake up and the house is completely different. Not a little different–completely different.

Nothing is where it used to be. The glasses in the kitchen, the clothes in your closets, and the furniture are reorganized. Even the walls and windows are all completely rearranged.

Whoever rearranged everything didn’t consult you. They didn’t warn you it was coming. They just took it upon themselves to make it happen.

In this “new” house, nothing seems to be where you’d expect it. The coffee cups are stored under your bed. You find your pants on the bottom shelf of the freezer. Logic doesn’t seem to be part of the organization scheme.

The worst part is that you still need to get to work on time. Usually, it only takes you about 45 minutes to get ready, so that’s all you allotted yourself. After all, you didn’t know this was coming, so why would you set your alarm differently? Nothing is where it’s supposed to be, you’re spending a lot of time trying to find everything, and the clock is running out–you’re going to be late and it isn’t your fault!

Jared M. Spool, “Designing Embraceable Change”

Reading that particular passage, I find my “inner voice” rising in pitch and cadence. It evokes a sense of hysteria, and understandably so. Later in the post, Jared points out a key concept when dealing with change:

It’s not that people resist change whole-scale. They just hate losing control and feeling stupid.

It’s important to remember that your intention is in most cases less important than the impact of change on the customer. Things like continuous deployment, although they may be adopted to improve customer satisfaction, can backfire if the intent and the effect do not align. As I’ve noted previously, user experience is extremely important. Unintentionally degrading that experience is bad enough. Purposely making people feel out of control and “stupid” is probably the one case where your intention is more important to the customer and not in a good way.