Conflicts, Collaboration, and Customers

With Ron Jeffries comment on my “Negotiating Estimates” post, I think we’ve reached agreement on a common problem, even if we disagree (perhaps?) over how to fix it. As I noted in my short answer, my position is that abuse of estimates stems from a deeper issue. Changing a practice won’t eliminate the abuse. Changing culture will. Changing that culture (where possible) involves convincing the players that it’s in their best interests to change, which it is.

This is my biggest objection to #NoEstimates. It makes an illogical leap that the practice is responsible for problems rather than some underlying dysfunction without showing causality. This putting the cart before the horse risks encouraging people to espouse action that is probably both ineffective and detrimental to their relationship with their customer. The likeliest outcome I see from that is more problems, not fewer.

The word “customer” was a point of contention:

It’s my position that, for better or for worse, there is a customer/provider relationship. Rather than attempt to deny that, it makes more sense, in my opinion, to concentrate on having the best possible customer/provider relationship. While poor customer service is extremely common, I’d hope everyone can think of at least one person they enjoy doing business with, where the relationship is mutually beneficial. I know I certainly can, which stands as my (admittedly unscientific) evidence that the relationship need not be adversarial. A collaborative, mutually beneficial relationship works out best for both parties.

“Collaborative”, however, does not mean conflict free. In his post “Against Estimate-Commitment”, Jeffries states:

The Estimate-Commitment relationship stands in opposition to collaboration. It works against collaboration. It supports conflict, not teamwork.

I both agree and disagree.

There will be conflict. Different team members will have different wants. Different stakeholders will have different wants. The key is not in preventing conflict, but resolving it in a way agreeable to all parties:

Recognizing, rather than repressing conflicts might even be beneficial:

What is critical, is that the relationship between the parties not be one-sided (or even be perceived as one-sided). In-house IT often has the peculiar characteristic of seeming one-sided to both customers and providers. The classic model of IT provision is, in my opinion, largely to blame. It seems almost designed to put both parties at odds.

The problem can be fixed, however, with cultural changes that encompass both IT and the business units it serves. One such change is moving from a project-centric to a product-centric model that aligns the interests of both groups. This alignment cements the relationship, making IT a valuable partner in the process rather than an obstacle to be overcome to get what’s wanted/needed. Relationships are key to success. By structuring the system so that each group’s incentives, conflicts and all, are aligned to a common goal, the system can be made to work. We’ve certainly had plenty of examples of what we get from the opposite situation.

Form Follows Function on SPaMCast 347

SPaMCAST logo

This week’s episode of Tom Cagley’s Software Process and Measurement (SPaMCast) podcast features Tom’s essay on project management in an Agile environment (aka “Project Management is Dead”) and a Software Sensei column on testing from Kim Pries in addition to a Form Follows Function installment on microservices, Devops and Conway’s Law.

In SPaMCast 347, Tom and I discuss my “Fixing IT – Microservices and DevOps to the Rescue?” post, specifically on how microservice architectures are not just a technical approach but an organizational one as well.

Fixing IT – Remembering What They’re Paying For

Dan Creswell‘s tweet said it well:

Max Roser‘s tweet, however, illustrated it in unmistakable fashion:

Success looks like that.

Not everyone gets to create the kind of magic that lets a blind woman “see” her unborn child. Nearly everyone in IT, however, has the ability to influence customer satisfaction. Development, infrastructure, support, all play a part in making someone’s life better or worse. It’s not just what we produce, but also the process, management and governance that determines how we produce. The product is irrelevant, it’s the service that counts. The woman in the picture above isn’t happy about 3D printing, she’s overjoyed at the experience it enabled.

Customer service isn’t just a concern for software vendors. It’s remarkably easy to destroy a relationship and remarkably hard to repair one. The most important alignment is the alignment of concerns between those providing the service and those consuming it. Mistrust between the two is a common source of “Shadow IT” problems, and far from helping, cracking down may well make things worse.

Building trust by meeting needs creates a virtuous circle. Satisfaction breeds appreciation. Appreciation breeds motivation. Motivation, in turns, yields more satisfaction.

As the saying goes, you catch more flies with honey.

Fixing IT – Microservices and DevOps to the Rescue?

Eberhard Wolff‘s question set the stage.

Adrian CockCroft‘s reply tied everything together.

Conway’s Law is the common thread tying microservice architectures (MSAs) and DevOps together. Significantly, this common thread runs through the entire organization, not just the IT parts. As I noted in my previous post, paying attention to this principle allows you to work with, rather than against, the grain of an organization. Working with the grain of the organization is key, because DevOps, lovely as it is, is not an end, but a means. The desired end, as identified by Mike Kavis in “Is DevOps What Organizations Really Seek?” is to “become high-performing organizations”.

Of all the attributes of such an organization (as identified by Kavis): “Strong Leadership”, “Strong Culture”, “Sound Architecture”, etc., the most important is “High Customer Satisfaction”. For many organizations, high customer satisfaction is a problem area for IT. In a recent article on Business Insider, Red Hat’s CEO Jim Whitehurst noted an increased interest in DevOps on the part of CIOs to deal with what he terms IT’s “fight for its life”:

That’s because IT departments say they had better figure out how to be faster, cheaper, and better. If they don’t, the company’s employees will no longer depend on them. They bring their own PCs, tablets and phones to work and they buy whatever cloud services they want to do their jobs. And the CIO will find his budget increasingly shifted to other manager’s pockets.

IT has has a history of cycles of neglect or rejection of new/disruptive technologies followed by catch-up crises: PCs in the 80s, Web in the 90s, BYOD/Cloud/IOT/etc. now. What’s different this time is the increasing level of technical knowledge and access to solutions outside of IT. As Krishnan Subramanian has noted, playing catch-up may become less and less viable:

MSAs enable flexible applications via composing vertical slices of business functionality rather than horizontal layers of technical concerns. These same principles can apply to higher levels of abstraction up to the enterprise’s IT architecture. Likewise, DevOps incorporates the same viewpoint shift in terms of the IT organization. This architectural change (both technical and business) can allow for integration between IT and the business it enables. As Twila Day recently noted, this integration goes far beyond mere alignment into “active partnership between IT and your business units”:

Partnership means working together, side by side. It means that technology leaders are actively involved in strategic development at the highest levels of your organization. It means that all the way up and down your organization, any talented person can propose an innovative idea, tactic or strategy, regardless of where s/he works. The business might have an idea first, or IT might have it first. No matter. What’s important is that the two groups work side by side to accomplish the most important business objectives.

Transforming IT from an adversary into a partner is primarily a cultural shift that involves both parties if it’s to be successful. Organizational and technical architecture cannot be neglected, however, in that they can either help or hinder that transformation. DevOps can facilitate this via its focus on the product rather than any one project (which is a concern shared by the customers) and by having the flexibility to tailor its pace to that of the customer rather than forcing a one size fits all (aka one size fits none).

Form Follows Function on SPaMCAST 319

SPaMCAST logo

I’m back with another appearance on Tom Cagley’s Software Process and Measurement (SPaMCast) podcast.

SPaMCast 319 features Tom’s “Why Are Requirements So Hard To Get Right?” segment, followed by Jo Ann Sweeny’s new column, “Explaining Change”. Tom and I close it out with a discussion of one of my previous posts, “Fixing IT – Credible or Cassandra?”.

Fixing IT – Credible or Cassandra?

Portrait of Cassandra in front of Troy

In Greek mythology, Cassandra was a princess of Troy who possessed the gift of prophecy, but was cursed never to be believed. I was reminded of the story by a tweet from Peter Kretzman:

Credibility is a precious commodity in the business world. It’s hard to earn and yet can be lost astonishingly easily. Poor customer service, regardless of whether it’s due to malice, apathy, or just plain ignorance, damages trust.

Likewise, commitment mismatches can leave customers (internal and/or external) disaffected. The project that will be “out of sight, out of mind” for IT is the product that they will be saddled with for years to come. IT’s perceived lack of commitment (justified or not) is a source of conflict and mistrust.

Lack of a business focus is a credibility killer as well. Things like indulging faddish practices, essentially engaging in one-sided experiments with the enterprise’s money, are seen as evidence of immaturity. A dictatorial attitude toward technology issues is typically resented (regardless of whether the opinion is correct). Failure to communicate business value, whether out of arrogance or ignorance, can lead to ill-advised decisions on the part of the business. When you’re asking for seven figures and “trust me” is your sole justification, you cannot complain when you get turned down.

Things that might seem purely technical can damage the relationship with the customer. Technology for technology’s sake, putting your vision ahead of the customer’s needs, ignoring user experience, and inadequate attention to quality can all lead to a loss of trust.

Quality and reliability can be particularly problematic. Stepping into the breach and heroically fixing issues can be perceived as admirable in some organizations. All a customer sees is an outage. When dial tone services go down, so too does credibility. As Matt Ballantine observed in his post “Firefighting”:

But if your world is one where you can only justify your own existence through the solving of problems that are of your own creation, you’re in trouble long term. That’s where IT has been – and why commodity services have become so pervasive so quickly. The IT team wins no points for fixing stuff that’s gone wrong when someone else can be providing that stuff without it failing all the time.

Working with the business, IT can serve as a powerful force multiplier. Opportunities can be seized and risks averted. For that to happen, however, IT has to be heard. The less we shoot ourselves in the foot, the better chance we have.

#NoEstimates? #NoProjects? #NoManagers? #NoJustNo

Drawing of Oliver Cromwell

#NoX seems to be the current pattern for hashtags designed to attract attention. While they certainly seem to serve that purpose, they also seem to attract more than their fair share of polarized viewpoints. Crowding out the middle ground makes for better propaganda than discussion.

So why does a picture of a long dead English political figure grace a post about hashtags? Oliver Cromwell was certainly a polarizing figure, so much so that when the English monarchy was restored after his death, he was disinterred, posthumously hung in chains, and his head was displayed on a spike. Royalists hated him for overthrowing the monarchy and executing King Charles I. The more democratically inclined hated him for overthrowing Parliament to reign as Lord Protector (which had a nicer sound to it than “dictator”) for life. To his credit, however, he did have a way with words. One of my favorites of his quotes:

I beseech you, in the bowels of Christ, think it possible you may be mistaken.

That particular exhortation would well serve those who have latched onto the spirit of those hashtags without much reflection on the details. Latching onto the negation of something you dislike without any notion of a replacement doesn’t convey depth of thought. Deb Hartmann Preuss put it well:

For many of the #NoX movements, abuse of the X concept seems to be the rationale for doing away with it. Someone has done X badly, therefore X is flawed, irrational, or even evil. Another Cromwell quote addresses this:

Your pretended fear lest error should step in, is like the man that would keep all the wine out of the country lest men should be drunk. It will be found an unjust and unwise jealousy, to deny a man the liberty he hath by nature upon a supposition that he may abuse it.

That some who do things will do those things badly should not come as a surprise. My singing voice is wretched, but to universally condemn singing as a practice because of that does not follow. Blatant fallacious thinking reflects poorly on advocates of a position.

In many cases, there seems to be a disconnect between those advocating these positions and the realities of business:

Owly Images

In response to his posting a link to “The CIO Golden Rule – Talking in the Language of the Business”, Peter Kretzman reported “…I saw people interpret “you need to talk in language of the biz” as being an “us vs. them” statement!” Another objected to the idea that the client’s wishes should be paramount: “the idea that the person with the purse has more of a voting right is one I don’t live under. i can vote to leave the table.” Now, I will give that one credit in that they recognize that “leaving the table” is the price for insisting on their own way (I’m assuming that they know that means quitting), but it still betrays a lack of maturity. In most case, we work for a client, not ourselves. How many times can one “leave the table” before they’re no longer invited to it in the first place.

The business is not a life support system for developers following their passion. Rather than it being their job to fund us, it is our job to meet their needs. Putting our interests ahead of the clients’ is wrong and arrogant. It is the same variety of arrogance that attempts to keep BYOD at bay. It is the same arrogance that tried to prevent the introduction of the PC into the enterprise thirty years ago. It failed then and given the rising technological savvy of our customers, has even less of a chance of succeeding now. Should we, as an profession, continue to attempt to dictate to our customers, we risk hearing yet another of Cromwell’s orations:

You have sat too long for any good you have been doing lately… Depart, I say; and let us have done with you. In the name of God, go!

This is not to say that the various #NoX movements are completely wrong. Some treat estimates as a permanent commitment rather than a projection based on current knowledge. That’s unrealistic and working to change that misconception has value. Management does not always equate to leadership and improving agility is a worthy goal provided focus is not lost. I’ve even written on the danger of focusing on projects to the detriment of the product. What is needed, however, is a balanced assessment that doesn’t stray into shouting slogans.

I’ve seen some defend their #NoX hashtag as a springboard to dialog (see the parts re: slogans, hyperbole, and propaganda above). They contend that “of course, I don’t mean never do X, that’s just semantics”. John Allspaw, however, has an excellent response to that:

Fixing IT – IT as Concierge or General Contractor?

At your service

Last fall I had the pleasure of engaging Jeff Sussna, Christian Verstraete and Christian Reilly in a multi-day discussion on Twitter regarding IT’s relationship with cloud providers. As it developed, the conversation morphed into a discussion of potential metaphors for IT.

At the mention of “metaphor”, I’d imagine that eyes are glazing over, but bear with me while I establish its relevance to IT. Now Merriam-Webster tells us that “metaphor” is “a figure of speech in which a word or phrase literally denoting one kind of object or idea is used in place of another to suggest a likeness or analogy between them”. We find this same type of analogy used with patterns, the concept of which came out of Christopher Alexander’s A Pattern Language. In essence, these metaphors are patterns, communicating a potential solution to problems of enterprise IT architecture in the same way that “bridge” and “factory” communicate potential solutions to software design problems. As such, they provide a frame of reference to those of us within IT as well as those we serve. Their importance lies in the message they convey to both sides of the table.

Two candidate metaphors emerged from the discussion. Christian Reilly took the view that an IT department’s role should become like that of a concierge, a service aggregator. While agreeing in principle, I took the position that a better analogy was that of a general contractor. A general contractor works to meet the client’s needs, performing some services while subcontracting others out, in a manner similar to the concierge. The difference is that the contractor possesses a set of domain expertise useful for advising the client. The contractor is also bound by building codes that constrain him/her in delivering the customer’s requests. This describes what, in my opinion, the role of IT should be.

The dysfunctional situation described in the first post of this series, “Fixing IT – How to Make a Monster (Customer)”, defines the antithesis of an effective IT operation. “Fixing IT – Taming the (Customer) Beast”, the second post in the series, discusses some of the mindset changes needed to repair the relationship between IT and the business and why that relationship is so important. The third post, “Fixing IT – Products, Not Projects Revisited”, outlines the way delivery processes can affect IT’s relationship with the business. “Fixing IT – The Importance of Ownership”, the fourth post in the series, takes on how the issues of budget and ownership can affect the relationship. This post, will wrap up the series by bringing those issues together to outline my opinion on how IT can move forward in spite of the challenges it faces today.

The ZDNet post that inspired the discussion I mentioned above, “4 ways IT can rise above outside cloud competitors”, is an excellent example of how IT can turn things around by providing additional value to cloud services. This can be done by acting as a pseudo-VAR in place of acting as a competitor as the ZDNet article proposed. In either case, IT provides real value to the business via its specific knowledge of both the business domain and the in-house technical environment. This approach, advocated by Christian Verstraete in his HP Blogs post “Respond to Shadow-IT, source services from external providers”, is becoming more and more necessary because 7 out of 10 organizations are using unapproved cloud applications (according to a study from January, 2013).

IT has been in this position before. Thirty years ago the disruptive, security challenging technology that IT had no time and budget for was the PC. History teaches that that genie is not going back in the bottle. Instead, the winning approach is to enable the use of technology that furthers the goals of the business (which is all the business wants, anyway) in a safe manner.

The words “in a safe manner” above is what makes the general contractor metaphor appropriate. IT serves two sets of masters, the individual business units and the enterprise as a whole. Sometimes, the interests of a business unit can conflict with those of the organization as a whole. In these situations, IT has a valid reason to refuse to break the rules (although finding a way to achieve the desired outcome and stay within the rules is better than a flat refusal). What is important is that the rules, as such, are not owned by IT, but by the organization. IT is well placed to advise the enterprise about the technical capabilities and trade-offs around decisions regarding security, business processes, etc. It is inappropriate, however, for IT to attempt to dictate these decisions (this is actually an example of a business unit’s interests potentially conflicting with those of the enterprise). Steve Jones illustrated how counter-productive this usurpation can be in his post “Why your IT strategy failed or why the business hates IT”:

IT Guy: “The new systems have new processes and we need to tell you what they are so you can change.”

Business Guy: “Have you done an impact analysis against our current processes?”

IT Guy: “No we’ve just defined the Best Practice To-Be processes, you need to do the impact and change management. We need the meeting so we can tell you what the To-Be processes are”

Business Guy in a voice so dripping with sarcasm I thought we’d have a flood: “I look forward to our IT department telling the business what Best Practice is for our industry.”

IT Guy, completely failing to read the sarcasm: “Great we’ll get it organised”

Some take exception to referring to “the business”, observing that IT is part of the business. This, of course, is true. However, the best way to demonstrate that is not via the terms used in conversations, but via the support given to executing the enterprise’s strategy. Making things happen is the way show that support.

Fixing IT – Taming the (Customer) Beast

Detente

In my previous post, I outlined the source of the dysfunctional nature of the relationship many IT departments have with their customers. This post will be about fixing that relationship. Having taken part in just such a venture, I can state that it is possible.

When I first became involved with corporate IT, I worked for a group that was developing a core line of business application for the company. Our process worked (well, “worked”) like this:

  1. The users would produce voluminous “specs” minutely detailing the new features they wanted in terms of behavior and presentation. These were delivered to product managers.
  2. The product managers would take the users’ “specs” and file them away for reference in their trash cans. The product managers would then generate their own “specs” minutely detailing the behavior and presentation of the features in accordance with the way they knew it really should have be.
  3. The new “specs” were presented to the remainder of the team for estimation, and the final project plan was created.
  4. Product managers, developers, testers, technical writers, trainers, etc. would all gather with their management and everyone would sign a “contract” to deliver what was specified by the due date.
  5. Developers would then proceed to implement the features as best they could, substituting their own ideas for those parts that were ambiguous or unrealistic. When everything was completed (or when the “code freeze” date was reached, whichever came first) the features were delivered to the testers.
  6. Testers would then generate bug tickets for anything that did not work the way they thought it should (regardless of whether it matched the documentation or not). Bug tickets were sent to the developers.
  7. Developers would remedy the defects according to the instructions in the ticket and batches of bug fixes would be released to the testers.
  8. Testers would pass the the tickets and forward them to the product managers.
  9. The product managers would re-open the tickets and direct that the features be coded according to the original specification (or not, if they had had a new idea in the interim). These tickets would then be returned to the developers.
  10. Developers would remedy the defects according to the new instructions in the ticket and batches of bug fixes would be released to the testers.
  11. Steps 6 through 10 would repeat until either the tester or product manager relented, typically due to exhaustion. When this point was reached for all features, user acceptance testing was began by the user representatives who had composed the original “specs”. The original go-live date was, of course, long since passed by.
  12. User acceptance testing consisted of repeated rounds of find-fix-push. The goal of the user representatives was to bend the meaning of “bug” such that something resembling the original requests would emerge. Really good ones could manage to get entirely new enhancements implemented as “fixes”. At the end of this process, a release to production was scheduled.
  13. After the deployment (and all the downtime due to remediation of the release management issues), the end users could then contemplate how they were going to work around the software to earn their paycheck.

Needless to say, this process led to our having a warm relationship with our customers. They frequently assured us that should we burst into flames, they wouldn’t hesitate to stomp the fire out. Several even routinely produced books of matches and offered to demonstrate.

In all seriousness, the process we worked under at that time seriously impaired our relationship with our customers, both on an organizational and an individual basis. It destroyed our customer’s trust in us as it all but guaranteed that we would be lying to them about what they would be getting and when they would get it. The fact that they would not find this out until the very last second only added insult to injury. When the product was declared dead and the majority of the group laid off, it came as no surprise to anyone.

What did come as a surprise was that a small cross-functional group from the team was retained and tasked with teaching a new process to the remainder of the IT group. The nine months prior to the demise of the product had been spent re-engineering the process by which it was developed. While the effort wasn’t sufficient to save the product, it did get the attention of those looking to transform the IT function.

Reduced to its essence, that shift in process was nothing more than a recognition that our job was to make it easier for others to do their job. It wasn’t about technology or software development, it was about providing value to our customers and meeting their expectations. Value was understood to be more than just taking orders, but actually working with customers to define their needs and jointly determine a solution that met those needs. Meeting expectations was understood to be involving customers in the planning and keeping them in the loop as it evolved, not committing to a date with insufficient information and then hiding the changing circumstances until it could be denied no longer.

Service Cycle

The result of this communication and collaboration was, slowly, step by step, the building of trust. As Tom Graves noted in “Product, service and trust”, “Trust is at the centre of everything in any enterprise – every product, every service”. In that same post, Tom used the image on the right to illustrate the cyclical nature of trust built via mutual respect, attention to needs, and delivery of results. Each success enhances the trust of the customer and the reputation of the provider. These successes can also form a virtuous circle where customer satisfaction increases the motivation of the team serving them.

It is important to understand that this is a process, rather than an event. As Seth Godin stated in “Gradually and then suddenly”:

It didn’t happen suddenly, you just noticed it suddenly.

The flipside works the same way. Trust is earned, value is delivered, concepts are learned. Day by day we improve and build an asset, but none of it seems to be paying off. Until one day, quite suddenly, we become the ten-year overnight success.

The reason this is so important should be clear. Business is dependent on technology more and more with each passing day. However, being dependent on technology and being dependent on a particular provider are two different things. When it is responsive and trustworthy, in-house IT can provide tremendous value to an enterprise. When it is not, the value proposition for outside providers becomes clearer.

Fixing IT – How to Make a Monster (Customer)

he may be ugly, but he is mine

Software development in general and IT in particular seems to have a love-hate relationship with our customers – as in, we really love to hate on our customers. We have Stupid User Tricks, ID10T issues, PEBCAK, and of course, Clients From Hell. Every once in a while, even Dilbert takes a break from bashing managers to take a swing or two at customers.

There’s even some evidence that the feelings are mutual.

Why?

How is it that we’ve managed to come to the point where distrust, even open hostility, is the norm? How is it that this situation is allowed to continue? What if we don’t change the dynamic?

To get an idea of how we got here, imagine the following scenario:

  • There’s a restaurant where you’re required to eat.
  • You don’t get to decide when you can eat; you have to ask (and ask) and eventually you’re allowed to sit at the table without any idea of when you’ll get another chance.
  • You don’t pay for what you eat, but you will have to justify each menu item you order.
  • The kitchen staff will be required to say how exactly long it will take to prepare the order, even if the item is not on the menu and no one has ever made it before.
  • The waiter, the chef, and the maitre d’ may not understand or be able to prepare your order, so they reserve the right to alter it without any notice – you’ll find out when it arrives.
  • Waiters interacting with diners after the initial order is considered poor practice; kitchen staff doing so is completely out of the question.
  • If the order doesn’t meet your approval, you can send it back to be fixed as much as you like.
    • Under those circumstances, one might expect the restaurant patrons to be a tad distrustful of the staff, who will probably respond in kind. The experienced patrons will have learned to order as much as possible, regardless of need, subject only to the restraint of getting approval. They will have learned to be vague enough to allow them to keep sending dishes back for fixes that are enhancements in disguise. The bolder patrons will either learn to cook for themselves or find another restaurant, perhaps both.

      Is this starting to sound familiar?

      The second question, how is it that this has been allowed to continue, is something of a mystery. While there has been a growing significant incidence of shadow IT, things still haven’t broken out into open rebellion. How much of this is inertia and how much of this is the current economy holding back expenditures? More ominously, how close to the edge are we?

      This brings us to the third question, the answer to which should be obvious. Trying to maintain the status quo will not work. In fact, doing so will be more likely to hasten the demise of IT as it becomes more of a commodity. Without major changes, IT risks becoming irrelevant and marginalized. Rather than worrying about blame (it should be obvious that this is a systemic problem rather than one or two bad actors), both business and IT need to find a way forward that maximizes value and minimizes friction. The risk to those organizations that cannot make this transition increases with each passing day.