Canonical Data Models, ESBs, and a Reuse Trap

The original uphill battle

In “Reduce, Reuse, Recycle”, I discussed how reuse introduces additional complexity and cost as the scope expands (application to family of applications to extra-organization) and in “Coping with change using the Canonical Data Model”, I illustrated how that technique could be used to manage API changes without breaking existing clients. Last week on LinkedIn, an interesting question was posted that combined both reuse and the Canonical Data Model pattern – how do you create a service-oriented architecture around a canonical data model when the teams for the component applications are unaware of the benefits of reuse?

Initially, the benefit seems to be self-evident. If you have ten applications publishing messages to each other, this can involve up to 90 mappings per service method [n(n – 1)], 180 if the services are request-response. If, however, all ten applications use the same set of messages, you have one format per call (two for request-response). Things get sticky when you realize that you must now coordinate ten development teams to code an interface to your service bus (we’re not even considering the case of a third-party product). Assuming you manage to achieve that feat, things get stickier still when the inevitable change is needed to the canonical format. Do you violate the one true message layout rule or do you try to get ten application teams to deploy an update simultaneously?

Using the canonical data model pattern inside the service bus (the incoming native message is transformed to the canonical format on the way in and then to the appropriate outgoing native format on the way out) allows you to vary the external formats while reducing the number of mappings to a maximum 20 [2n] (40 if using request-response). Any logic inside the bus (e.g. orchestrations) retains the same benefit of operating on the canonical format without the constraint of having all clients constantly synchronized. Obviously, nothing prevents multiple applications from using the same format, so this method can reap the benefits of reuse without constraining change.

A successful service-oriented architecture will need to balance two potentially contradictory forces: stable service contracts for existing consumers and the ability to quickly accommodate new consumers. Using a canonical data model internally while allowing for a variety of external formats can allow you to meet both objectives with minimal complexity. Attempting to enforce a canonical model externally pushes changes onto all consumer applications regardless of need and will slow the pace of change.

“Selling SOA” on the Iasa Blog

Just what the doctor ordered?  More snake oil?

Nobody gets excited about plumbing (until it fails). Communicating the business value of architectural infrastructure is as important as the design.

My latest post on the Iasa blog discusses how to make the linkage between technical value and business value and how to present it.

Update 4/12/2013: Mirrored here.