"SOAP is Dead" -- if you believe the echo chamber
This is something of a followup to my post on the XML Team weblog last week, taking into consideration some of the feedback. I'm getting just a wee bit annoyed about the echo chamber repeating "SOAP is dead" to itself around the blogosphere, with nobody adding any actual evidence other than the success of various Web applications built mainly with HTTP. There are plenty of people I know and respect, and plenty I don't know but enjoy reading their blogs, making this point. I'll use Carlos Perez's post [someone I don't know personally] as the whipping boy, just to get back at him for "Mike Champion of course is prepared to defend the faith" :-)
To jump to the bottom line, I'm not defending the faith, I'm defending agnosticism. There are a range of technologies available to developers: some simple, some complex; some MS-specific, some Java-specific, some platform-neutral; some Web-based, some protocol-neutral; some tightly coupled, some loosely coupled; some favor up-front design, some are more evolutionary. The choice of one over another in a specific application is a business and engineering decision, not something to be determined by "faith-based programming" in accordance to either the SOAP or REST religion.
Carlos Perez writes:
Isn't SOAP the underpinnings of Web Services, the same technology that was billed as the silver bullet to extinguish our collective integration nightmare?
Uhh, no it's not billed that way. I suppose someone can find Steve Ballmer speeches from 2001 that can be construed that way, but let he whose marketing people are without hyperbole light the first flame :-) If you really want to judge the web services technologies by checking how well they realize the coolest visions of their early advocates, then I suppose SOAP has failed... but so has Java, the Web, and just about everything else I can think of, if judged by those criteria.
The RESTful approach has now born fruit. Applications like BlogLines, Flickr, Mappr, Del.icio.us and 43Things are revealing that proof is in the pudding
I'm not sure these are really RESTful in any meaningful way beyond using HTTP to move information around. [see Steve Maine's post and comments thread] More importantly, it's not at all clear to me how SOAP, WSDL, or WS-* could plausibly be central to these applications even if all the hype had already become reality. There's no client-side software to integrate with other than the browser, and the overwhelmingly obvious way to integrate browser-based applications with services is HTTP. Furthermore, there's no underlying reliable messaging, transaction processing, end-to-end encryption, etc. etc. technology in a typical user's internet environment that the SOAP headers can trigger. (Remember that all the WS-* specs do is describe a standard way to request such services). Maybe in some future world where Indigo and Avalon are pervasive in the Windows world and XUL and some open source SOAP, etc. toolkit are pervasive in the Mac / Linux / Java worlds, one could imagine building this kind of consumer oriented service using WS-*. Until then, the web services technologies don't have much potential to address the challenges faced by a designer of such a service, so seems a bit pointless to note that they aren't used for these applications in practice.
The web services technologies have a place, or at least a great potential, that should not be minimized. People who think that the Web is where almost all the action is might want to spend a little time in the enterprise software world. The dominant vendors there, such as IBM and SAP on the back end, and BEA in the middle, have joined with MS in the web services initiatives because they have determined that the raw XML and HTTP technologies do not do the job by themselves. I honestly don't have a lot of first hand knowledge of whether SOAP is really thriving today in that world, but that's where you want to take its pulse, not in the world of consumer-oriented, browser-based Web applications.
Carols Perez takes me to task:
What Mike seems to be saying is that REST is the Yugo and if you want the features then you have to buy a Rolls Royce. It's a sales pitch used all too commonly. Unfortunately, if I wanted a Rolls Royce I would have purchased a JMS implementation, not a Web Services implementation with a yet to be defined solution for security.
That is exactly the problem: You may have a JMS implementation, but it won't interoperate at the message level with other vendors' JMS implementations, and it won't even be consistent at the API level with all sorts of proprietary message queuing and transaction processing systems that are deeply rooted in enterprises. WS-* is about getting these things to interoperate, leveraging XML's inherent platform and application neutrality, and adding a couple layers to request services from the underlying messaging, security, and transaction processing systems.
Furthermore, the employees of those enterprises do a lot of their actual work with desktop applications, and these currently exchange information with the enterprise systems only with considerable expense and inefficiency. At my previous job , a common use case would be to get data back and forth between a legacy Natural mainframe application and an Excel spreadsheet or other MS Office application. Web services technologies let developers do this kind of thing by running a Natural -> WSDL translation wizard and leveraging the SOAP support built into both Office and various vendors' enterprise middleware. Is this RPC-oriented rather than document-oriented? Tightly coupled? Doing what CORBA tried to do, only with XML underneath? Perhaps, but you'll have to explain to the people whose actual jobs today are made easier by web services why this is a Bad Thing.
With that little rant out of my system, I'd like to point to reponse by Bill de Hora
that I found very thought provoking. I didn't see much that I disagreed with. He confirms my understanding that REST developers don't really want those VisualStudio wizards that MS doesn't offer :-)
My impression of those who are interested in using REST is that they want to be close to wire and to the data and if anything, want to reduce the number of tools in use. REST is a minimalist approach to systems building - when you're using it you're making a decision to some of the heavy lifting on your own
Similarly, I agree (and AFAIK the Indigo people do as well)
there's not much question that WS-* is in need of rationalizing. Entirely natural in the tech sector, competitive pressures have resulted in duplicated effort and excess choice for technology decision makers, which lessens the value of having standards to begin with.
I'm not so sure about
WS can be positioned primarily for those companies whose requirements are effectively off the bell curve
He may be seeing a different bell curve than I do. If we are talking about consumer applications over the web, sure. If we are talking about enterprise scale integration efforts in a world where technologies such as SAP and other ERP systems are pervasive, proprietary message queuing systems are the transport mechanism of choice, COBOL and other mainframe-rooted development languages are remain very important, and millions of terabytes of operational data remain in IMS (a dinosaur-era hierarchical DBMS), then I simply have to disagree. One lesson I learned in my previous job is that while this stuff has little geek mindshare anymore, it is pervasive and persistent in big business. I don't expect a RESTifarian story for mainframe modernization anytime soon!
The bulk of the post is an intriguing discussion of simple technologies and disruptive innovations, drawing heavily on THE INNOVATORS DILEMMA. I agree with Bill's point that while WS is generally marketed as if it were a disruptive technology, it is not really. (Well, OK, maybe it disrupts the moribund EAI industry ....)
Bill's analysis helped me clarify my pushback against all the "SOAP is comatose, put it out of its misery" posts: WS-* is a collection of sustaining innovations intended to incrementally extend and improve existing enterprise infrastructures and PC desktop applications. It's hard to disagree with the proposition that these will eventually be replaced by a disruptive technology that hits a very different price-performance point, and it's quite possible that XML and HTTP will be at the center of that disruption. I'm not at all confident that "eventually" will come anytime soon, however. Great source of ideas for entrepeneurs and experimenters, sure; proven technologies for enterprises, no.
Once again, we're talking about a world where SOAP-based specs are being proposed as standard interfaces to existing enterprise-class technologies that have evolved slowly over decades while the internet has gone through a series of disruptive changes. Many of us jumped on the chance to play with Bloglines, Flickr, and other disruptive applications that are presented as poster children for REST. I doubt if any of us wants our bank or insurance company to risk disruption (in the usual sense of the word) of its business in the name of disruptive innovation. As Bill mentions, CIOs in these industries have been burned so often by unsuccessful projects -- often planned on overly optimistic predictions about the technology fads of yesteryear -- that they are not likely to jump on XML, REST, or agile methods just because they are hot this year (or this decade). IT departments in big organizations which spend a lot of energy fighting spammers and phishing attacks can be forgiven for not wanting to jump on the Next Big Thing that internet technologies such as HTTP and XML offer. They'll migrate when they see their competition getting ahead of them by adopting mature and proven versions of this stuff. Until then, WS-* looks like a promising way to integrate business systems and improve business processes without a lot of "disruption".
I don't know Bill personally, but I do know / admire his colleague Sean McGrath, and I get the impression that their company Propylon is one of the relative few who really know how to apply REST principles to the industrial-strength integration challenges that WS-* was designed for. They are as likely as anyone to create a situation where
...WS specs and tools are being commoditized by disruptive technologies. If so the natural progression for them and the software based on them is to move up the value chain. When the mass market for enterprise technology is either over-served by technology, quality or the price is too high that leaves an opportunity for disruptive technology to take hold at the bottom end of the market.
It will be very interesting to see if they or anyone else pulls this off. I wouldn't be broken hearted, personally, if plain ol' XML disrupts WS-* -- As I said in the team blog post, if the RESTifarians are proven right, it just creates more direct demand for the WebData XML technologies, whereas if the WS people are right, more people will use our stuff via Indigo. I am personally, and I think MS is collectively, hedging the bet by covering both side, and working to better understand the circumstances in which one or the other will really work best. It's an extremely good bet that there are plenty of niches out there for both. MS can and should supply both SOAP-based and Plain O' XML tools; if others want to focus on "REST", go for it. I for one won't suggest that you drink any SOAPy Kool-Aid, although I would appreciate a bit less rhetoric about how tasty the RESTful Kool-Aid is until there are "reference customers" that use that approach instead of WS-* in a scenario that WS-* is intended to handle.
Finally, I was very glad to see James Governor's response to the various commentaries on his article, including mine.
a. niche - my intention was never to say SOAP is dead, merely that is not a necessary condition for web service development. There is a class of applications outside the traditional requirements of high end transactional enterprise development and workflow that just don't require the reinvention of Corba-class integration.
b. the comment there isnt any demand for SOAP refers to comments from the Creative Commons.
It's clear that we're on the same page here: "its all about niches or not."