I must admit that after a dozen years at ZapThink, the first six helping vendors tell the SOA story and the rest helping architects get SOA right, that a deep sense of frustration was never far away. The vendors talked a good game, of course, but for them architecture was never more than an excuse to sell more software. Couldn’t really blame them, after all – that’s what they did for a living. But the deeper frustration came from conversations with the enterprise practitioners: the architects, systems engineers, developers, and others who would come to my SOA and Cloud classes. For them, I would lay out a picture of how SOA was supposed to work, both in the enterprise and in the Cloud, knowing full well that most of them wouldn’t be able to put my advice into practice.
The “next generation” SOA story did move this ball forward somewhat. I had the opportunity to contrast first generation, middleware-centric, Web Services-based SOA with the next generation approach that was RESTful, lightweight, decentralized, and Cloud-friendly. Yes, people understood the difference. Yes, everybody was getting wise that buying traditional, heavyweight middleware wasn’t the key to SOA, and in fact, frequently impeded success. But no, people generally weren’t able to take the next-generation SOA story and actually put it into practice.
In fact, my SOA course had only one example of an organization who was able to get the next-generation SOA story to work: the US Coast Guard, with their SPEAR architecture. But SPEAR’s greatest strength was also its weakness: their deeply iconoclastic approach to implementing REST-based SOA. Basically, they had to invent it as they went along, putting into production an infrastructure that they had carefully crafted to meet their unique requirements. And while SPEAR made a wonderfully insightful case study, it didn’t provide an explanation of next-generation best practice that would be broadly applicable, especially across the private sector.
The conclusion, therefore, of my final version of the SOA course was a choice between two evils: first-generation SOA, with its expensive and inflexible dependence on middleware and fixed interfaces. And next-generation SOA, with its huge gaps, where no one had yet to figure out how all the pieces might fit together in order to implement a truly Agile Architecture as I define it in my book, The Agile Architecture Revolution.
Shifting the Perspective on SOA
At its most fundamental level, SOA is an approach to abstracting legacy application and data assets as Business Services to provide greater agility to the organization by supporting flexible business processes. Furthermore, the way SOA is supposed to support such processes is via Service composition.
There are two essential challenges with this perspective on SOA. First, if you start with inflexible legacy assets, putting Service interfaces on them doesn’t magically make them more flexible. Second, stringing such Services together into some kind of flowchart-style orchestration simply compounds the inflexibility problem. A chain is only as strong as its weakest link, after all.
Getting SOA to work properly, therefore, becomes an exercise in getting around these challenges. Instead of abstracting individual, static Service interfaces, Business Services must abstract sets of such interfaces via content-based routing and transformation operations on an intermediary like an ESB. Implement those abstraction operations as a matter of policy, and you shift control of the behavior of your SOA deployment to the metadata-driven policy layer. Get all this right and you have enormous control and flexibility over your legacy environment. The problem is, of course, that it’s extraordinarily difficult to get all these moving parts right.
For the sake of argument, however, let’s say that you are able to build a proper Business Service abstraction, and furthermore, you’ve managed to implement a functional SOA governance infrastructure to manage, enforce, and update the policies that drive the whole shebang. To be fair, many organizations have been able to achieve some measure of success with this level of SOA maturity.
OK, now move everything to the Cloud.
Oops.
And there’s the rub: first-generation SOA came along well before the Cloud was on the horizon, and the mechanisms that you must put into place are simply not Cloud-friendly. The underlying legacy apps are still legacy apps, as ornery as ever. The intermediary is a choke point and a single point of failure. And if you’re using an ESB, it suffers all the ills of traditional middleware, including thread-managed state, limited horizontal scalability, centralized brokering, and fundamentally, a deep-seated bias toward the “connecting things” view of distributed computing, rather than the “governing Services” view that successful SOA requires.
Cloud-Centric Agile Architecture
The Cloud does far more than add additional options for your development and execution environments. In reality, the Cloud offers a true paradigm shift in how people deploy, manage, and consume IT assets. While today, enterprises are fleshing out some kind of hybrid story – connect on-premise to Cloud, extend on-premise with Cloud, or migrate some on-premise apps to the Cloud – the true Cloud story for the enterprise is getting to the point that all IT assets live and run in the Cloud, because they are created for the Cloud in the first place. After all, it’s a fool’s errand to assume that you can take some ancient legacy spaghetti-code app, shoehorn it into the Cloud somewhere and expect it to gain all the advantages of the Cloud.
The effect this cloud-centric perspective has on Agile Architecture is dramatic. We must completely rethink the SOA story, because Services as we’ve come to know them no longer serve a central role in the architecture. Instead, the declarative, model-driven, metadata-based policy layer takes the central spot in our conceptual Agile Architecture, finally freed from the middleware as I’ve been championing for the better part of a decade now. After all, serving as a policy execution layer has been the most important role of the SOA intermediary, in spite of all the middleware vendors’ exhortations to the contrary.
Today, the Cloud is forcing enterprises to rethink their architectures. What is this new Agile Architecture? Is it SOA? Not really – yes, it retains the intermediary pattern but reinvents it for the Cloud, and Services aren’t the central abstraction any more. Is it Event-Driven Architecture? Event-driven patterns are essential to the Cloud, but traditional EDA is still middleware-centric. Is it REST? In part, yes, but REST is an abstraction of client/server architecture patterns, while Agile Architecture mixes in the role of the intermediary. Welcome to the new paradigm: similar in ways to what has gone before, but different and novel in even more fundamental ways.
The EnterpriseWeb Take
As EnterpriseWeb’s new Chief Evangelist, my role has two facets: a traditional marketing role combined with a thought leadership role. This new newsletter is a venue for the latter. Yes, I will be communicating the EnterpriseWeb value proposition, and I hope you visit our Web site for more information (and we’d love to give you a demo, of course – email me for more information). But this newsletter focuses on architecture: how organizations like yours can leverage technology (ours or someone else’s) to achieve the agility goals of the business.
Of course, I would be disingenuous if I didn’t feel that these two missions weren’t fully aligned. After all, I chose to join EnterpriseWeb for a reason: I felt that among all the vendors in this space, they stood alone in their ability not only to understand the Agile Architecture vision, but to implement it as well. I spent a dozen years hitting my head against the wall of SOA, trying to explain an architectural approach that the technology on the market was ill suited to support. I’m looking forward to telling a different story now. After all, the best thing about hitting your head against a wall is how good it feels when you stop!
