“Avoiding Platform Rot” on Iasa Global Blog

just never had the time to keep up with the maintenance

Is your OS the latest version? How about your web server? Database server? If not now, when?

A “no” answer to the first three questions is likely not that big a deal. There can be advantages to staying off the bleeding edge. That being said, the last question is the key one. If the answer to that is “I haven’t thought about it”, then there’s potential for problems.

See the full post on the Iasa Global Blog (a re-post, originally written for the old Iasa blog).

Advertisement

Fixing IT – Products, Not Projects Revisited

Always under construction

Google “disconnect between IT and business” and you get 16.9 million hits in just under half a second. In the first page of results you’ll find names like Cutter, Forbes, Huffington Post, and Business Insider. These facts should make it clear that the problem is real and pervasive. Where did this disconnect come from and how do you remedy it?

In the first two posts of this series (“Fixing IT – How to Make a Monster (Customer)” and “Fixing IT – Taming the (Customer) Beast”), I discussed the systemic problem of customer service and how it might potentially be overcome. Another disconnect, one that I’ve touched on before, is IT’s focus on projects instead of products. Although this issue relates to customer service and customer-centricity, it has other impacts that cause it to warrant its own post.

Projects, a concept which seems to be the center of the universe for so many IT departments, are not priority for most stakeholders; the resulting product is. For IT, unless you’re running a pure flow process where the product’s lifetime is a continuous stream of changes and additions, projects are the events/activities that result in the creation or enhancement of the product. They are what happens to the thing, not the thing itself. That being said, I’m not a fan of the #NoProjects viewpoint. Just because they’re not the most important concept doesn’t mean they’re not an important concept; it’s just that perspective and priorities are needed.

How often do you think of the construction of your home? While it was a project that took time, effort and money, in the end, you probably don’t care. I’d be willing to bet that you derive more value and satisfaction from the use of it than its construction (which generally only comes to mind if a problem surfaces). Likewise, with IT’s customers, they care about the result. In fact, they start to care about that result at the same time that IT is traditionally considering it “done”.

IT Projects which are “done” represent a dangerous situation. In many cases, the project team moves on and the system is consigned to “support”, meaning that those best qualified to fix issues are most likely unavailable. This degraded level of support erodes trust, acting like the grenade in Tom Graves’ “Playing ‘pass the grenade’?”:

What’s the ‘grenade’? Well, it’s kinda like a hot-potato – except that you’ll know when you have a hot-potato, because it burns everyone’s fingers somewhat in passing, whereas a grenade feels perfectly safe and ordinary and unimportant until it blows up in your face…

One of the “grenades” Tom lists is “so-called ‘customer-service’ whose design, either by default or by intent, denies those who need that service from receiving it”. In other words, poor service due to ignorance of our customers needs and concerns can cause as big a rift as that due to outright malice. This is important because, as Seth Godin notes in “Thinking lifetime (don’t break the chain)”, only “The traveling salesman, the carnival barker and the old-time businessman can hit and run”. For all others, their benefit lies in the lifetime of their relationship with the customer. Think of this lifetime as a chain, each link representing an exchange of value between you and your customer:

Think of the interaction at the deli counter or the pump or the bursar’s office or the alumni office or on the website from the point of view of the customer and the chain. Where are the moments where you might lose her forever? What are the key places where you need to intervene and invest in the relationship instead of milk it, or drag it through the mud? Assuming that your competitors are just as selfish and metric-driven as you are isn’t a great strategy, because you’re still losing when you break the chain.

Support is not a cost center, it’s a profit center. Treating customers with urgency and clarity and respect (maintaining the chain) is more urgent than ever…

Think lifetime, all the time.

Having a product focus, rather than a project focus, affects the architectural design and the development of the system as well. A tweet from Rebecca Wirfs-Brock sums up the issue: “One tension that always exists (especially agile design) is when&how to support variability in the prob with a flexible design solution”. With a product focus, the product is not done until the day it’s taken out of service. As Brenda Michelson tweeted, “if software is never done, architecture must allow equally for structure and next transition; flex don’t break”. With a project focus, the team can “hit and run”, making it a tempting proposition to over-rely on YAGNI.

Evan Bottcher, in “Projects are evil and must be destroyed”, observed;

Projects deliver exactly what they promise. Project teams have little incentive to invest in the long term operation and maintenance of the systems that they create. I’m not saying that the team doesn’t care or are intentionally acting irresponsibly, but when delivery pressure is applied the first things to be dropped from the project schedule will be the cross-functional concerns that make the system reliable, monitorable, deployable, and maintainable ongoing.

The DevOps tenet of a team supporting what they build is an excellent example of product focus in action. Working under those conditions, the team has a powerful incentive to grow and evolve the system over its entire lifetime. In essence, the team develops an ownership interest in the product that helps to cement their relationship with the system’s stakeholders. Ownership, particularly in relation to those stakeholders, is a word that you will hear more about in the next installment of this series of posts.

[I have to give a shout-out to Tony DaSilva, his post “Thanks, But No Thanks” came out while I was still writing this one and in commenting on it re: #NoProjects, I realized that there was something more I needed to include in this post. Tony’s posts are always thought-provoking, literally in this case!]

[More serendipity – this exchange on Twitter between Lorri MacVittie, Dave Roberst, James Urquhart, and Brenda Michelson occurred after I finished the post but before I published – the money quote from Brenda: “you say “DOOM”, PMO says “Next”. Project centricity (funding, attention) dooms corporate codebase”]

Fixing IT – Taming the (Customer) Beast

Detente

In my previous post, I outlined the source of the dysfunctional nature of the relationship many IT departments have with their customers. This post will be about fixing that relationship. Having taken part in just such a venture, I can state that it is possible.

When I first became involved with corporate IT, I worked for a group that was developing a core line of business application for the company. Our process worked (well, “worked”) like this:

  1. The users would produce voluminous “specs” minutely detailing the new features they wanted in terms of behavior and presentation. These were delivered to product managers.
  2. The product managers would take the users’ “specs” and file them away for reference in their trash cans. The product managers would then generate their own “specs” minutely detailing the behavior and presentation of the features in accordance with the way they knew it really should have be.
  3. The new “specs” were presented to the remainder of the team for estimation, and the final project plan was created.
  4. Product managers, developers, testers, technical writers, trainers, etc. would all gather with their management and everyone would sign a “contract” to deliver what was specified by the due date.
  5. Developers would then proceed to implement the features as best they could, substituting their own ideas for those parts that were ambiguous or unrealistic. When everything was completed (or when the “code freeze” date was reached, whichever came first) the features were delivered to the testers.
  6. Testers would then generate bug tickets for anything that did not work the way they thought it should (regardless of whether it matched the documentation or not). Bug tickets were sent to the developers.
  7. Developers would remedy the defects according to the instructions in the ticket and batches of bug fixes would be released to the testers.
  8. Testers would pass the the tickets and forward them to the product managers.
  9. The product managers would re-open the tickets and direct that the features be coded according to the original specification (or not, if they had had a new idea in the interim). These tickets would then be returned to the developers.
  10. Developers would remedy the defects according to the new instructions in the ticket and batches of bug fixes would be released to the testers.
  11. Steps 6 through 10 would repeat until either the tester or product manager relented, typically due to exhaustion. When this point was reached for all features, user acceptance testing was began by the user representatives who had composed the original “specs”. The original go-live date was, of course, long since passed by.
  12. User acceptance testing consisted of repeated rounds of find-fix-push. The goal of the user representatives was to bend the meaning of “bug” such that something resembling the original requests would emerge. Really good ones could manage to get entirely new enhancements implemented as “fixes”. At the end of this process, a release to production was scheduled.
  13. After the deployment (and all the downtime due to remediation of the release management issues), the end users could then contemplate how they were going to work around the software to earn their paycheck.

Needless to say, this process led to our having a warm relationship with our customers. They frequently assured us that should we burst into flames, they wouldn’t hesitate to stomp the fire out. Several even routinely produced books of matches and offered to demonstrate.

In all seriousness, the process we worked under at that time seriously impaired our relationship with our customers, both on an organizational and an individual basis. It destroyed our customer’s trust in us as it all but guaranteed that we would be lying to them about what they would be getting and when they would get it. The fact that they would not find this out until the very last second only added insult to injury. When the product was declared dead and the majority of the group laid off, it came as no surprise to anyone.

What did come as a surprise was that a small cross-functional group from the team was retained and tasked with teaching a new process to the remainder of the IT group. The nine months prior to the demise of the product had been spent re-engineering the process by which it was developed. While the effort wasn’t sufficient to save the product, it did get the attention of those looking to transform the IT function.

Reduced to its essence, that shift in process was nothing more than a recognition that our job was to make it easier for others to do their job. It wasn’t about technology or software development, it was about providing value to our customers and meeting their expectations. Value was understood to be more than just taking orders, but actually working with customers to define their needs and jointly determine a solution that met those needs. Meeting expectations was understood to be involving customers in the planning and keeping them in the loop as it evolved, not committing to a date with insufficient information and then hiding the changing circumstances until it could be denied no longer.

Service Cycle

The result of this communication and collaboration was, slowly, step by step, the building of trust. As Tom Graves noted in “Product, service and trust”, “Trust is at the centre of everything in any enterprise – every product, every service”. In that same post, Tom used the image on the right to illustrate the cyclical nature of trust built via mutual respect, attention to needs, and delivery of results. Each success enhances the trust of the customer and the reputation of the provider. These successes can also form a virtuous circle where customer satisfaction increases the motivation of the team serving them.

It is important to understand that this is a process, rather than an event. As Seth Godin stated in “Gradually and then suddenly”:

It didn’t happen suddenly, you just noticed it suddenly.

The flipside works the same way. Trust is earned, value is delivered, concepts are learned. Day by day we improve and build an asset, but none of it seems to be paying off. Until one day, quite suddenly, we become the ten-year overnight success.

The reason this is so important should be clear. Business is dependent on technology more and more with each passing day. However, being dependent on technology and being dependent on a particular provider are two different things. When it is responsive and trustworthy, in-house IT can provide tremendous value to an enterprise. When it is not, the value proposition for outside providers becomes clearer.

Fixing IT – How to Make a Monster (Customer)

he may be ugly, but he is mine

Software development in general and IT in particular seems to have a love-hate relationship with our customers – as in, we really love to hate on our customers. We have Stupid User Tricks, ID10T issues, PEBCAK, and of course, Clients From Hell. Every once in a while, even Dilbert takes a break from bashing managers to take a swing or two at customers.

There’s even some evidence that the feelings are mutual.

Why?

How is it that we’ve managed to come to the point where distrust, even open hostility, is the norm? How is it that this situation is allowed to continue? What if we don’t change the dynamic?

To get an idea of how we got here, imagine the following scenario:

  • There’s a restaurant where you’re required to eat.
  • You don’t get to decide when you can eat; you have to ask (and ask) and eventually you’re allowed to sit at the table without any idea of when you’ll get another chance.
  • You don’t pay for what you eat, but you will have to justify each menu item you order.
  • The kitchen staff will be required to say how exactly long it will take to prepare the order, even if the item is not on the menu and no one has ever made it before.
  • The waiter, the chef, and the maitre d’ may not understand or be able to prepare your order, so they reserve the right to alter it without any notice – you’ll find out when it arrives.
  • Waiters interacting with diners after the initial order is considered poor practice; kitchen staff doing so is completely out of the question.
  • If the order doesn’t meet your approval, you can send it back to be fixed as much as you like.
    • Under those circumstances, one might expect the restaurant patrons to be a tad distrustful of the staff, who will probably respond in kind. The experienced patrons will have learned to order as much as possible, regardless of need, subject only to the restraint of getting approval. They will have learned to be vague enough to allow them to keep sending dishes back for fixes that are enhancements in disguise. The bolder patrons will either learn to cook for themselves or find another restaurant, perhaps both.

      Is this starting to sound familiar?

      The second question, how is it that this has been allowed to continue, is something of a mystery. While there has been a growing significant incidence of shadow IT, things still haven’t broken out into open rebellion. How much of this is inertia and how much of this is the current economy holding back expenditures? More ominously, how close to the edge are we?

      This brings us to the third question, the answer to which should be obvious. Trying to maintain the status quo will not work. In fact, doing so will be more likely to hasten the demise of IT as it becomes more of a commodity. Without major changes, IT risks becoming irrelevant and marginalized. Rather than worrying about blame (it should be obvious that this is a systemic problem rather than one or two bad actors), both business and IT need to find a way forward that maximizes value and minimizes friction. The risk to those organizations that cannot make this transition increases with each passing day.

Professional Software Development – Can We Mandate What We Can’t Define?

The law is a what?!?

The only true wisdom is in knowing you know nothing.
Socrates

What types of software products have you worked on: desktop applications, traditional web, single-page applications, embedded, mobile, mainframe?

How about organizations: private for-profit, government, non-profit?

How about domains: finance, retail, defense, health care, entertainment, banking, law enforcement, intelligence, real estate, etc. etc. etc.?

Given that the realm of “software development” is currently huge (and probably expanding as you read this), how logical is it that someone (or even a group) could regulate what is acceptable process and practice? I won’t say that it would be impossible to come up with one unified set of regulations that would fit all circumstances, but I’m very comfortable estimating the likelihood as a minute fraction of a percent. If the entire realm were broken down into smaller groupings, the chance might increase, but the resulting glut of regulations would become an administrative nightmare and still wouldn’t address those circumstances that aren’t in the list above but are on the horizon.

Nonetheless, people continue to float the idea of regulation.

Last fall, Bob Martin floated the idea of government regulation as a reaction to the healthcare.gov fiasco. That would be the same government whose contracting regulations contributed to the fiasco in the first place, correct? That would be the same government that has legally mandated Agile for Department of Defense contracts? Legally mandated agility just seems to sound a bit suspicious. As Jeff Sutherland noted “Many in Washington are still trying to figure out what exactly that means but it is a start”. A start, for sure, but the start of what?

Ken Schwaber’s blog post “Can Software Developers Meet the Need?” takes a different approach. Schwaber proposes that:

A software profession governing body is needed. We need to formalize and regulate the skills, techniques, and practices needed to build different types of software capabilities. On one side, there is the danger of squeezing the creativity out of software development by unknowledgeable bureaucrats. On the other side is the danger of the increasingly vital software our society relies on failing critically.

We can either create such a governance capability, or the governments will legislate it after a particularly disastrous failure.

Call me a cynic, but I’m betting that the amount of bureaucratic squeezing that would result from this would far outweigh any gain in quality.

Most of the organization types listed above are already on the hook for harm caused by their IT operations; just ask Target and Knight Capital (don’t ask the Centers for Medicare & Medicaid Services). Is it more likely that a committee, whether private or public, can better manage the quality of software across all the various categories listed above? Could they be more likely to keep up with change in the industry? Color me doubtful.

Surfing the Plan

Hang loose

In a previous post, I used the Eisenhower quote “…plans are useless but planning is indispensable”. The Agile Manifesto expresses a preference for “Responding to change over following a plan”. A tweet I saw recently illustrates both of those points and touches on why so many seem to have problems with estimates:

Programming IRL:
“ETA for an apple pie?”
“2h”
8h later:
“Where is it?”
“You didn’t tell me the dishes were dirty and you lacked an oven.”

At first glance, it’s the age-old story of being given inadequate requirements and then being held to an estimate long after it’s proven unreasonable. However, it should also be clear that the estimate was given without adequate initial planning, no “plan B” and when the issues were discovered, there was no communication of the need to revise the estimate by an additional 300%.

Before the torches and pitchforks come out, I’m not assigning blame. There are no villains in the scenario, just two victims. While I’ve seen my share of dysfunctional situations where the mutual distrust between IT and the business was the result of bad actors, I’ve also seen plenty that were the result of good people trapped inside bad processes. If the situation can be salvaged, communication and collaboration are going to be critical to doing so.

People deal with uncertainty every day. Construction projects face delays due to weather. Watch any home improvement show and chances are you’ll see a renovation project that has to change scope or cost due to an unforeseen situation. Even surgeons find themselves changing course due to circumstances they weren’t aware of until the patient was on the table. What the parties need to be aware of is that the critical matter is not whether or not an issue appears, but how it’s handled.

The first aspect of handling issues is not to stick to a plan that is past its “sell by” date. A plan is only valid within its context and when the context changes, sticking to the plan is delusional. If your GPS tells you to go straight and your eyes tell you the bridge is out, which should you believe?

Sometimes the expiration of a plan is strategic; the goal is not feasible and continuing will only waste time, money, and effort. Other times, the goal remains, but the original tactical approach is no longer valid. There are multiple methods appropriate to tactical decision-making. Two prominent ones are Deming’s Plan-Do-Check-Act and Boyd’s Observe-Orient-Decide-Act. Each has its place, but have a looping nature in common. Static plans work for neither business leaders nor fighter pilots.

The second aspect of handling issues is communication. It can be easy for IT to lose sight of the fact that the plan they’re executing is a facet of the overarching plan that their customer is executing. Whether in-house IT or contractor, the relationship with the business is a symbiotic one. In my experience, success follows those who recognize that and breakdowns occur when it is ignored. Constant communication and involvement with that customer avoids the trust-killing green-green-green-RED!!! project management theater.

In his post “Setting Expectations”, George Dinwiddie nailed the whole issue with plans and estimates:

What if we were able to set expectations beyond a simple number? What if we could say what we know and what we don’t know? What if we could give our best estimate now, and give a better one next week when we know more? Would that help?

The thing is, these questions are not about the estimates. These questions are about the relationship between the person estimating and the person using the estimate. How can we improve that relationship?