Getting there from here

About three weeks ago, Simon Brown issued a challenge: “Can you explain how you design software within the time it takes to finish your coffee break?”. I left a comment to the effect that I could give an overview in that time period, but an actual explanation would take far longer. He followed it up with another post last week, reiterating the need to be able to answer the question “How do you design software?”. As he noted, “But if we can’t articulate how we design software, how are we going to teach this skill to others?”. The next day we had the following exchange on Twitter:

If you design software by drawing pictures on a whiteboard, how do you know what to draw? bit.ly/K4Digm

— Simon Brown (@simonbrown) May 11, 2012

@simonbrown Here’s what influences the decisions that determine what’s drawn: genehughson.wordpress.com/2012/03/10/wha…

— Gene Hughson (@GeneHughson) May 11, 2012

.@GeneHughson great list of things that should be taken into account; the next question is how do you get from the drivers to a design?

— Simon Brown (@simonbrown) May 11, 2012

Well thanks, Simon…won’t let me weasel my way out of that one, will you?

The truth is, explaining the design process is hard. It’s not a linear process; there is no defined set of ordered steps that yield a design. Additionally, design is about trade-offs. Any given decision has the ability to negatively impact one or more other concerns (security versus functionality, performance versus maintainability, etc.). Worst of all, roles get muddled. In spite of all of that, it’s something I truly love. Whether building a system from the ground up or evolving an existing system, it’s something I find truly rewarding.

Before proceeding, I should deal with a couple of caveats. First, is that the more detail I provide, the more the post will have an unavoidable bias towards the .Net platform, as that is what I work with. Second is that my practices are also influenced by my work environment: a corporate development team working on line of business applications, some for internal use, some customer facing. If you work on another platform or work for an ISV, I can’t guarantee that all of the post will apply to you. Much of what I say should be universal. When it’s not, see the preceding.

One last word before diving into the nuts and bolts: my ideal way to design is collaboratively. For reasons both noble and selfish, it’s the route I recommend. Communicate and collaborate as many of your stakeholders as possible: clients, developers, testers, support, and operations. Their input and feedback are priceless.

If you’re lucky, the process begins at the vision stage, whether that vision is of a new system or of changes to an existing one. You want to be in as early as possible for a number of reasons, but the primary one is that you want to partner with the customer in creating their system rather than be perceived as a roadblock. Early involvement means that you can explore options and suggest ways to accomplish what the users wants and need. This benefits both parties; I’ve seen customers ecstatic about what I could provide for them as often as I’ve headed off train wrecks from unrealistic expectations. The later in the process that you’re brought in, the more likely you’ll be faced with something that, no matter how untenable, someone is wedded to because of the effort they’ve put in. Then you either have to be the one to tell them that their “baby” is ugly or take it to raise yourself.

As the concept develops, I keep the following architectural drivers in mind:

• Functionality
• Data Profile
• Audience
• Usage Characteristics
• Business Priority
• Regulatory and Legal Obligations
• Architectural Standards
• Audit Requirements
• Reporting Considerations
• Dependencies and Integrations
• Cost Constraints
• Initial State

Each of these drivers influence multiple quality of service aspects. Understanding those influences allow you to present options to the customer so that their priorities shape your design. When the customer is an informed participant in making these trade-offs, their ownership of the results is increased and their level of frustration is decreased.

At this point in the process, things should still be high-level, concentrating on who (role, not individual) does what in an effort to determine scope. UML use case diagrams can be useful for capturing this rough outline of the system. Your focus should not be on the tool (white boards or paper can be put to as good a use as high-dollar packages) or on whether the diagram is syntactically correct. Your objective is capturing and communicating. If a drawing/diagram increases understanding, then that is the most important consideration.

Along with the use case diagram(s) noted above, I will generally use class diagrams to capture the major “things” that the system deals with (analysis classes) and their relationships. The idea is not to capture fine detail (attributes and minor associations), but to make sure the “big picture” is understood. Likewise, package diagrams can be used to capture the conceptual structure of the system. It’s not standard UML, but they’re easier to sketch on a white board and people (i.e. customers, developers, testers, support, and operations) can quickly visualize what you’re talking about.

One of the hardest things to master is learning not to jump in too early with a solution. You need to get the big picture (including features that will be very likely wanted in future releases) and address the drivers I listed above in order to determine the architecture. If at all possible, you want input and validation from customers, developers, testers, support, and operations. Throughout the design process, you should be defaulting to as simple a design as possible, only adding the complexity the drivers require.

Technologies, tools, or techniques that are new to the architect and/or the developers must be validated as well before being incorporated into a design. A proof of concept is vital. The one unforgivable sin for an architect is a design that is expensive to change. Insufficient analysis and failing to validate your assumptions are the quickest ways to an inadequate and inflexible design.

In my opinion, the most important skill to master is self-auditing. If you require an honest reason for all of your decisions, you will be far ahead of the game. Self-awareness is critical to avoid making thoughtless choices or going with a technology, not because it fits, but because it’s the current thing. If you wouldn’t accept “um…because” as a reason from someone else, you definitely shouldn’t accept it from yourself. Again, this is where your collaborators can be an extremely valuable sounding board.

For me, a layered design provides a flexible base architecture. Layered architectures are flexible enough to support a varying number of physical tiers depending on your needs: from all code layers on one tier for web applications, to separate front and back-end tiers for Click Once applications and SharePoint pages and web parts. Coupled with a message-oriented communication pattern, this style of architecture also allows you to expose services to other applications without duplicating functionality. For most applications, this either becomes my starting point for the software architecture or the desired to-be architecture.

My preferred structure is a base of three layers (User Interface, Business Process, and Data Access) plus a cross-cutting layer for messages and payloads (data transfer objects), exceptions, and common interfaces. By default, each layer would map to an assembly, but additional partitioning could make sense in some cases (for example, in order to isolate dependencies). The main considerations here are that responsibilities are uniquely assigned to a given layer and that a given layer only communicates with the layer below it.

Use cases map well to business process methods. I group related business process methods into business process classes. The analysis classes become the basis of the request and response message payloads for those methods. The use cases also provide the basis of a granular, task-based permission system. These permissions I then group into roles corresponding to the actors from the use case diagram(s).

My approach to communicating the architecture is to use a variety of techniques. The beauty of new development is that the diagrams noted above can be supplemented with the “skeleton” of the application: a Visual Studio Solution file with the various project files and key classes roughed out. Part of the “roughing out” process should be setting up the dependencies, both between the projects and for external components. Any proofs of concept should be considered as an architectural artifact. Sometimes text is the best mode of communication for a particular concept; it would be hard to diagram “favor chunky over chatty communication patterns”. The thing to remember is that “right” is defined by what clearly communicates the architectural design, not by what conforms to some arbitrary style.

The level of detail I’ve noted to this point is what I consider to be architectural in nature. This does not, however, mean that an architect’s job is finished upon communicating the structure of the application. It is an ongoing responsibility to ensure that the architecture is neither impeding the design and implementation of the application, nor become irrelevant due to poor alignment with the drivers listed above. Maintaining this connection is easiest if the architect has a design role as well. It is possible to federate the design role to the remainder of the development team provided that the architect maintains close contact with the project. What is not possible is showing up, dropping off a “fully defined” fill in the blanks design, and walking away, never to return.

Detailing how I do low-level design would probably triple the size of this post. For me, it entails mapping the details of the use cases to elements of the application working within the bounds of the architecture. The public structure of classes, forms, tables, stored procedures, etc. are all fleshed out according to the requirements and the details of the interactions decided. As with the architectural designs, the watch words should be “collaborative” and “iterative”.

Whether done for the designer or by the designer, activity diagrams using swim lanes for user actions and system reactions are a good tool for capturing the various paths through a given use case. Class diagrams can be used at this stage as well, now with more detail in terms of methods, properties, and associations. Likewise, code “skeletons” coupled with “ToDo” comments can be a powerful way to convey design decisions.

Regardless of the techniques used, it is important to remember that design documentation has a short shelf-life. Requirements should be living documents and an architectural overview may be well worth maintaining. However, design documentation will generally become stale very quickly. Tools that allow you to generate diagrams from the code typically make the effort to keep a static artifact synchronized unnecessary. Accordingly, substance should be preferred over style.

Having written over 1800 words to this point, I have little doubt that this is far from being the perfect explanation of how to design software. That’s appropriate. Software architects do not do “perfect”. Instead, the goal is an optimal solution given the constraints we have to work under that’s capable of changing with the needs of its users. Perfect does not get delivered. What does not get delivered never gets the opportunity to solve problems. Solving problems is what we should be doing.