Securing Cross Site XMLHttpRequest


As I mentioned in my post on Cross Document Messaging, client side cross domain request is an important area of interest for AJAX developers looking for ways to avoid expensive server side proxying calls. While Cross Document Messaging is useful for allowing third party components or gadgets embedded in a page to communicate/converse using script on both sides, other cross domain scenarios like web services require access to cross domain content using network requests from a client side web application. For example, you may want to use your client side map based mashup to pinpoint Chinese restaurants for your current neighborhood. This could require the mashup to request a text file from Zagat.com with the locations of Zagat rated restaurants in the area which can then be superimposed on the map.

Along those lines, a few proposals and implementations exist like XDomainRequest in IE8, JSONRequest and the W3C’s Web Applications Working Group’s Cross Site XMLHttpRequest (CS-XHR) draft specification, which combines an Access control framework with XMLHttpRequest or other features. While XDomainRequest is focused on enabling anonymous access of third party public data, Cross Site XMLHttpRequest has added functionality and consequently enables a broader set of scenarios that may appeal to the developer who may choose to use cross domain authentication and access control among other features.  As can be expected with securing a large cross section of cross domain scenarios, a number of concerns have been identified with the CS-XHR draft by the web development community, the IE team members and members of the Web Apps Working Group. For a list of our recent feedback on security on CS-XHR and our take on important security principles in cross domain, please read our Security Whitepaper on Cross Domain. The paper also covers best practices and guidance for developers who will choose to build on the current draft if it’s supported by a future browser. Note that issues here are currently being discussed and some concerns may be mitigated as the draft evolves.

Meanwhile, your participation in the Web Apps Working Group can add a broader perspective and help raise further issues in the draft so that browser vendors like us can implement it in the future, so if you want to help, sign up with the Web Applications Working Group public alias!

For all those of you who would like cross domain public data and want it soon, there’s XDomainRequest in IE8. We’d love to hear feedback on XDR, and from projects that have been built using it. Hit the comments section with links or just email them to me. I’ll be blogging more about this feature in a few weeks!

Sunava Dutta
Program Manager

Comments (62)

  1. Anonymous says:

    What is most important, is following specs.  We’ve been horribly bitten when browser developers have strayed from this in the past, so rightly so, we are a bit concerned.

    If everything boils down to: "MSFT has built ‘x’, based on what the W3C drafts currently contain, with minor deviations", then that is A-OK.

    If however MSFT has built ‘x’, which has NOTHING to do with the W3C draft, or deviates from it wildly, then WE WANT NOTHING TO DO WITH IT.

    Now I’ll admit, I haven’t read any of the specs for this, (but will now), but I’m sure that my perspective on this is shared by all web developers.

    Jerry

  2. Anonymous says:

    Microsoft won’t implement insecure features. If the "community" wants to standardize on insecure features, IE users will be safe and non-standard.  Sounds good to me.

  3. Anonymous says:

    Matt that is correct. We’ve also proposed XDR to the Web Apps Working Group (Formerly Web API) as a simple focused object for cross domain that addresses a subsection of scenarios of CS-XHR but does so safely.

    XDR is designed from the grounds up to be secure for the cross domain scenarios it addresses and caters to developers not looking for authentication scenarios or cross domain SOAPAction headers. (As my paper mentions, we hope these are addressed as well, however the way CS-XHR does them is scary) Unfortunately there was strong resistance from the members supporting CS-XHR and was not accepted as a draft. We’re still continuing our efforts here as we believe developers and IE users can still benefit from client side cross domain for public data using XDR as early as IE8. Recently, the Open AJAX Alliance reached consensus that XDR had the promise of being more simple and secure than CS-XHR.

    http://www.theregister.co.uk/2008/03/28/openajax_alliance_internet_explorer_eight/

  4. Anonymous says:

    it’s insane how much better Firefox is, it’s like inexcusable that IE is soo terrible for developers…

    have you seen tools like firebug?

    it actually makes web development possible.

    please if there is any love in your heart model IE after firefox. or at least make some tols that rock like firebug.

  5. Anonymous says:

    Why didnt I hear about this before? XDR is pretty much what I need. Any chance this will be changed between now and the next beta?

    My enterprise for one could do without cross site XHR. That just cries ‘security nightmare’.

  6. Anonymous says:

    have you seen Firefox?:  have you seen ie8?  it’s got built in developer tools.  eh, your trolling is lame.

  7. Anonymous says:

    If this is a Microsoft only thing, I’m not interested in the least.  Please stick to common web standards, rather than making up your own half-assed proposals.

  8. Anonymous says:

    Mogden– The proposed CSXHR proposal is the one that is scary-dangerous.  Microsoft submitted XDR as a proposed standard; let’s see if the other guys can keep their egos in check and put security first!

  9. Anonymous says:

    There is one thing that UNIX did well, and one thing Microsoft botched up badly and which is still biting all of our collective arses:

    You DON’T default authorizations to "allow all"!

    Reading both specs, it appears that the W3C wants to enable cross-domain XmlHttpRequest usage when some HTTP rules are defined; so, XHR remains the only required object, its methods and properties remain the same, but in cases where both the agent and the requested domain have matching rules, cross domain scripts can be used. If you know XHR, Xdomain XHR becomes logical and fail-safe: a rule doesn’t match, request denied.

    Microsoft’s proposal here is to create a completely new object, with its own methods, to do what XHR does but on cross domain scripts without any need for validation and absolutely no setup.

    First, it is obviously redundant to have 2 objects doing the same thing. Extending XHR seems logical.

    Second, guess who is going to make use of an object that requires no security settings, that can be used on any domain you want, to run whatever script you want? Yes, the guy next door wiping up sites for mom’n’pop websites, but also the guy paid a few thousands bucks to create that catalog website for yesterday – which will then add a payment option. Do you want to buy stuff from a website where any third party ad could have rewritten the payment URL?

    In short: everybody is going to make use of the low-security object, even in cases where the high-security one should be used, because "it’s not scary".

    I read all I could about this new MS object (which isn’t much): the only thing relevant I found was, "The XDomainRequest object is a safe, reliable, and lightweight data service, that allows script on any page to anonymously connect to any server and exchange data. Developers can use the XDomainRequest object when cross-site security is not an issue."

    What happens if the addressed domain has been taken over? Well, with XHR you can check against its credentials (SSL certs, etc.); with XDR, you can’t. I can’t find anything describing how it is "safe and reliable". Should we take your word for it?

    I think I see why this proposal was point-blank refused by the W3C WG: it falls into the same category as system-wide-ActiveX and Administrator-as-default-user.

    I’ll now try to locate the actual mailing list archived thread where the object was proposed then rejected, because it was conveniently dropped from the MSFT article.

    Mitch

  10. Anonymous says:

    After reading said mailing list, it appears that:

    – XHR "v1.0" as we know it currently is broken, as it contains almost no security settings;

    – XHR "v1.1" is the one implemented in most browsers currently (IE7, Firefox, Opera, Safari): it works like v1.0, but doesn’t allow cross-domain access;

    – XHR v.2, as planned by the W3C, takes v1.1 and expands access rights by making it modular: you can add explicit exceptions (the "scary looking rules");

    – XDR would work like XHR v1.0 does, and XHR would remain v1.1.

    I should add that you, Mr. Dutta, got a small reprimand from Chris Wilson for putting a bit too much editing on the MS XDR proposal and smackdown on XHR v2.

    I’ll add an URL to "the other side"’s vision:

    http://lists.w3.org/Archives/Public/public-webapps/2008AprJun/0217.html

    In short: MS took the W3C’s XHR ACLs as they were in 2007, ran away with them, and created an independent and incompatible solution. They remained silent for months on the WG, then criticized the ACLs all in one go, and put forward their XDR solution (and disregarded all comments made on it).

    Funny thing is, the W3C’s XHR v2 proposal tries to be compatible with XHR v1.0 browsers (all access rules ignored), XHR v1.1 browsers and sites (cross-domain scripts refused) while the MS proposal requires browser specific code.

    By the way, what happens to XDR when IE8 "works like IE7"? Or will XDR be backported to IE7 and IE6?

    Mitch

  11. Anonymous says:

    The Web Apps working group has been developing CS-XHR for THREE YEARS. Why have you waited until now to raise all these concerns? Why did you not engage with the working group, instead of unilaterally developing a separate spec without even telling the working group you were doing so?

    Do you think we are all stupid? It doesn’t matter how much positive spin you try to put on this behaviour, or how much you huff and puff about standards and security, it is transparently disgraceful. Whereas other people have invested their time and effort in co-operatively producing a powerful and useful standard, you have treated their efforts with customary contempt and arrogance, and are now attempting to foist your own solution on the world by leveraging your dominant market position to railroad it through.

    But hey, same old, same old. It’s not like Microsoft has ever behaved any differently. But please, at least have the decency not to treat us like idiots, and stop these lame attempts to convince us that you’ve changed or that your care about open standards.

  12. Anonymous says:

    Embrace, extend and extinguish

    Bill might have left but it seems like the same s**t to me

  13. Anonymous says:

    Reading through the whole thread and adjacent ones, it appears that:

    – MS was originally taking an active part in setting up the ACLs; there were 3 MS employees ‘dedicated’ to working on that issue with the WG. Only one actually did, though, but was responsive and actively participated

    – there was a job change: the only MS employee working with the WG got a new job, and forgot to mention what exactly he was doing with the WG

    – whoever took up the slack decided that IE shouldn’t bother with participating, and started on a wild tangent based on whatever already existed: enter XDR

    – after developing XDR, MS integrated it into IE8 and, forgetting where they came from and what they forgot to do, decided that their solution was right, everybody else was wrong, and voilà! Instant W3C WG uproar, just add markup.

    – to MS credit, those who made mistakes apologized for their attitude cavalière – but XDR remains.

    The sensible solution would thus be, scrap XDR (valuate it as an interesting experiment) and work with the WG to get a ‘perfect’ XHR v2 out; after all, wastin 6 months of development at MS doesn’t compare with wasting 3 years of work at Mozilla, Apple, Opera, KDE and 2 years at MS.

    Mitch

  14. Anonymous says:

    XDR by any other name would be Botched?

    From Mitch’s investigation and the other comments on this thread and the subsequent WG seem to point to the N.I.H. syndrome.  MS didn’t come up with this, therefore they’ll go build their own system.

    If there were serious concerns about the CS-XHR, then get them on the table, sort them out, fix them, and move on.

    I have no intention of writing another wrapper for CS-XHR’s just to account for IE differences,… AGAIN!

    Please tell me that the IE8 Web Slices, and Activities do not "internally" depend on this botched XDR thing… if so, it is time for a quick re-write to get back on track.

    stan

  15. Anonymous says:

    Does all this negative feedback mean that you guys will reconsider your implementation? Or will you just pretend that you did not get any feedback?

  16. Anonymous says:

    @ted – I have read the white paper and W3C mailing list threads in detail.  I don’t find the Microsoft objections very compelling.  It reads more like an after the fact justification for an earlier decision to go their own way.

  17. Anonymous says:

    "The sensible solution would thus be, scrap XDR (valuate it as an interesting experiment) and work with the WG to get a ‘perfect’ XHR v2 out; after all, wastin 6 months of development at MS doesn’t compare with wasting 3 years of work at Mozilla, Apple, Opera, KDE and 2 years at MS.

    Mitch"

    Mitch, as I’ve mentioned in my draft, Mozilla hasnt even been able to secure XHR in the past. Slapping on cross domain there doesnt really sound wise.

    Other than that, I really wont attempt to answer your flaming rhetoric. From what I’ve observed this is classic Mitch and not worth my time.

    "I should add that you, Mr. Dutta, got a small reprimand from Chris Wilson for putting a bit too much editing on the MS XDR proposal and smackdown on XHR v2."

    This is hilarious. What’s your point?

  18. Anonymous says:

    Mogden, here’s what Jonas from Mozilla said about our concerns in my whitepaper btw and I do think there reasons are very valid.

    "Looking forward to continued discussion on these topics. There is definitely some interesting stuff in here so I’m glad we got this feedback!"

  19. Anonymous says:

    Looking through the paper it seems there are many people from the development community who share the same concerns on CS-XHR. MSFT has outlined that in a section on ‘community comments’.

    I’ve also read a few of the threads and cant help but feel that the members pushing CS-XHR are being emotional in their decision given that is been a time sink and cant move forward, but insist on fixing this feature that seems broken to me.

  20. Anonymous says:

    Except Jonas from Mozilla had much more to say than empty fluff, Sunava. Specifically:

    – You have fundamentally misunderstood the spec in many places.

    – Several of the concerns have already been addressed since you disappeared with an old version of the spec.

    – Many of your concerns can simply be labeled FUD caused a misunderstanding of how the web works (and linking to random bugzilla comments and IRC discussions without context does not make invalid speculation any more valid).

    This is just further proof at how rapidly irrelevant Microsoft is becoming to the future of the web. Saying you’re going to work more closely with standards organizations simply for PR reasons is NOT going to cut it anymore – *active* participation is required and remaining silent for months and then dropping a large, poorly researched and understood, out of date chunk of feedback in a proprietary document format initially locked behind a shady license on a spec that has moved significantly beyond the point you last bothered to look at it is so far from active participation it’s simply ridiculous.

    It also seems your aversion to being a useful contributor to the standardization community goes both ways given the first anyone heard of your XDR implementation was when it appeared in IE8 and the fact you have simply ignored almost (if not) all of the feedback on that implementation.

    Please do all of us web developers a favor and accelerate your decline into irrelevancy, we’re tired of dragging you and your half-baked product around.

  21. Anonymous says:

    @fudnation

    Sounds like lots of MSFT trashing to me. You dont speak on behalf on web devs so dont try. I’m a web dev and I think XDR is something that should be a standard.

  22. World says:

    As always a very informative article.

  23. Anonymous says:

    Anyone else having problems posting comments on here?

  24. Anonymous says:

    Why isn’t there more granular control over timeouts, like with ServerXMLHTTP (has four different timeouts controllable VIA setTimeouts method)?

    http://msdn.microsohttp://msdn.microsoft.com/en-us/library/ms760403(VS.85).aspxft.com/en-us/library/ms760403(VS.85).aspx

    And hopefully third time is a charm for posting this question… The first two tries from Firefox didn’t appear to work.

  25. Anonymous says:

    @Glen: Why would you want that level of control?  

    @Mitch: XDR default is "allow none"– the only safe setting.  If you didn’t recognize that, you need to re-read the paper.  

    XDR looks a lot like JSONRequest from Doug Crockford at Yahoo.  It’s a mistake to assume that Microsoft came up with this model in isolation.

  26. Anonymous says:

    @Glen: Why would you want that level of control?  

    @Mitch: XDR default is "allow none"– the only safe setting.  If you didn’t recognize that, you need to re-read the paper.  

    XDR looks a lot like JSONRequest from Doug Crockford at Yahoo.  It’s a mistake to assume that Microsoft came up with this model in isolation.

  27. Anonymous says:

    Mitch– A few corrections:

    "XHR remains the only required object, its methods and properties remain the same"

    Wrong.  Some methods and properties are disabled in "cross-domain mode".  How exactly the developer is supposed to know this is not yet defined.  What errors or exceptions are thrown in invalid states are yet to be defined.

    "completely new object, with its own methods, to do what XHR does"

    Wrong.  It aims to do less than XHR does (e.g. no setting/viewing of headers, no non-GET/POST, etc).

    "but on cross domain scripts without any need for validation and absolutely no setup."

    Wrong.  The server must explicitly opt-in to exposing data x-domain.

    "Extending XHR seems logical."

    As noted previously, doing so is problematic because the object behaves much differently in cross-domain mode. It must, for security reasons.

    "requires no security settings, that can be used on any domain you want, to run whatever script you want?"

    I don’t think you understand what you’re talking about.  Neither XHR nor XDR "runs" script.  They’re simply used to retrieve textual data.

    "Do you want to buy stuff from a website where any third party ad could have rewritten the payment URL?"

    You do not understand how cross-domain security works.  Neither CS-XHR nor XDR enables what you’re talking about.

    "I read all I could about this new MS object (which isn’t much)"

    Uh, did you try the whitepaper, MSDN documentation, etc?

    "Well, with XHR you can check against its credentials (SSL certs, etc.);"

    Wrong. XHR offers no functionality that would enable the developer to examine the site’s "credentials".  Both XHR and XDR will automatically fail any HTTPS request where the server presents an invalid certificate.

    I know it’s hard, but please try to do a little bit more research before you post.  It’s clear you are not a web developer and thus you do not understand the issues you’re trying to write about.

    Thanks!

  28. Anonymous says:

    mike sez <<Embrace, extend and extinguish>>

    Errr… except they didn’t embrace or extend anything, there’s no currently implemented competitive feature to "extinguish" and they proposed their work for a new standard that anyone can use.  So its hard to say this is a case of EEE.

  29. Anonymous says:

    fudnation, it’s hard to say that IE is irrelevant with >80% marketshare.  Or do you not actually care about reality?  It sorta cracks me up that you’re trying to imply that the IE team doesn’t know how the web works.  

  30. Anonymous says:

    Thanks God I don’t need cross-site AJAX calls – saves me from a lot of troubles as I see.

  31. Anonymous says:

    fudnation, it’s hard to say that IE is irrelevant with >80% marketshare.  Or do you not actually care about reality?  It sorta cracks me up that you’re trying to imply that the IE team doesn’t know how the web works.  

  32. Anonymous says:

    Around 20-25% of my customers do NOT use IE of any version.  I think I speak for all web developers when I say we are tired of having one standard for IE, but then another for everyone else.  We are not prepared to use proprietary hacks only to have them removed when you decide that IE10 will actually adhere to the W3C HTML5 standard.

    Please just work out the CS-XHR spec so that we can all get on with writing better sites rather than constantly hacking our code for your browser.  As it stands we still have the legacy IE6 problem and you are just creating more future problems by not adhering to the spec.  Even in so-called standards compliance mode IE8 still deviates from the spec because you have unilaterally decided to alter the spec to your whims.

    Working with the ‘community’ includes listening to developers AND working with people on W3C on creating standards.  Try to listen more and work better with the community.

    It is only IE-only developers that you hurt, like the guy complaining on another blog that you had removed scriptlet support and how it was costing him lots of money.  The developers who use the standards will not be affected because we will not use this sort of extension.  The web will be frozen in IE6 era if you continue to pursue your own path.

  33. Anonymous says:

    @Dan: The point is, XHR (and XDR) retrieve data from the server asynchronously.

    Some methods and properties are disabled in "cross-domain mode": yes, and that’s the point: trying to use them returns an error, allowing you to write the same query for same-domain and cross-domain functions, and you deal with the differences in the function – after all, you’re supposed to use error management somewhere. The very fact that XHR and XDR are similar in use but use wildly different syntax forces you to write two different functions, and two different error management routines.

    "Wrong.  It [XHR] aims to do less than XHR does" or as much as XHR does (or would ideally do in v2) in cross-domain mode; redundancy.

    "Wrong.  The server must explicitly opt-in to exposing data x-domain." but if that’s the case, what’s so troublesome about setting up x-domain XHR v2? If the server doesn’t opt in, the UA throws an error on setting up the connection – be it with XHRv2 or XDR.

    "Uh, did you try the whitepaper, MSDN documentation, etc?" Yup, that, and a bunch of e-mails sent back and forth b/w MS and W3C. I also read comments from guys that conceived the objects.

    Comments by smart people can be interesting; too bad they were omitted in this post.

    "Wrong. XHR offers no functionality that would enable the developer to examine the site’s "credentials": whoops, true – my bad. That should be done in error management routines. It has indeed no direct link with XHR or XDR – except that you’d attempt to load either object, and check their return before starting your work on the external resource. If you have one object, you have 3 cases:

    1- resource available on same domain: all methods and properties available

    2- resource available on outer domain: only some methods and properties available

    3- resource not available

    With 2 different objects, you have 2 cases per object to cover, and 2 objects to detect.

    XHRv2 tends to allow more flexibility with the use of access lists – at the costs of more rule setting (with everything blocked by default).

    "It’s clear you are not a web developer": I’m just not fluent in AJAX – I content myself with dealing with DOM manipulation and event models on gracefully degrading webpages that don’t load much external content.

    [sarcasm]

    The fact that that I did pass the FizzBuzz test means I’m no programmer at all, since currently a programmer mustn’t pass it, and an experienced lead programmer needs 15 minutes to do so.

    [/sarcasm]

    Back to serious stuff: I didn’t say that XDR is crap, I said that in this post, XDR was shown off as the best thing since sliced bread while XHR done by W3C was scary.

    Actually, XDR is an interesting experiment on a different take on x-domain queries, which can be balanced by the draft XHR v2 experiment done by Mozilla in alpha builds of Firefox 3; meaning, there’s one idea, two different test implementations, both have nice stuff in them, both have problems, nobody has agreed that any is better than the other – a good solution would thus be take those two attempts, mix them, and if a separate object really is needed, allow everybody to participate; currently, XDR isn’t it, and XHR is still a draft with problems. Still, since XHR and XDR are still extremely similar, WHY should we have a new object?

    Reinventing the wheel and all that. As said before, it really looks like a NIH case – like OOXML vs. ODF.

    XHR was a really nice MS invention; it supported review, got better with time and external review, and begs to be improved again. Why throw out the baby with the bathwater?

    I guess XDR looks like a good idea to people who don’t care if they load a 150k JS library to use a single 10-line function.

    I care.

    Mitch

  34. Anonymous says:

    billyboob– Anyone who claims to speak for all developers is clearly a delusional egomaniac.  Let’s be clear, there’s no "spec" for CS-XHR– there’s a "proposed" spec, and given the concerns raised, it seems unlikely to ever pass that stage.

    ————-

    Mitch– Asyncronicity has exactly nothing to do with this.  

    Oh, so you’ve decided that CS-XHR returns an error, eh?  What error?  And how does the design of the object let the user anticipate the behavior in XDomain mode?  Or should the developer wrap every call to every method of the object with an exception handler?  Sounds like a brutally bad programming model.

    Frankly, XHR and XDR aren’t all that similar in use, as noted previously, XDR looks intentionally designed to do less (modulo its xdomain capability).  CS-XHR, by design, aims to do more than XDR, and thus was considered less secure by those in the community as shown by Sunva’s paper.

    Neither CS-XHR nor XDR returns an error "upon setting up the connection."

    Microsoft already submitted XDR as a candidate replacement for the insecure CS-XHR proposal.  The CS-XHR proposal was so unsafe that it was pulled out of FF3.  What does it tell you when neither of the major browsers is implementing the CS-XHR proposal?

    Microsoft released XHR from whole cloth in the IE5 timeframe.  As described here and elsewhere, it’s actually a pretty bad design from both a programmability and security point of view.  But somehow XHR was okay but XDR was not?  Hrm….

    <<I guess XDR looks like a good idea to people who don’t care if they load a 150k JS library to use a single 10-line function.>>

    Uh, XDR is a simpler & more-concise programming model, so this remark just reiterates that you don’t know what you’re talking about.

  35. Anonymous says:

    Mitch, why put a bunch of branching in one object to make it function differently, when two objects would be a cleaner, more-efficient way of doing it?

  36. Anonymous says:

    @Glen: ‘to make it function differently’? It’s not different, it’s reduced functionality with a slightly different syntax.

    What XHR and XDR do (thanks Dan for re-centering the subject to its principles) is transmit data from an agent displaying a webpage to a server. The only real difference between the two is that XHR is more capable than XDR (I’m talking functionalities). After all, nothing prevents you from using XDR on a local domain too, if you only need its restricted capabilities.

    Writing two complete similar algorithms to take into account two objects WILL entail code duplication – and code duplication is NEVER clean. After all, that’s what objects are for!

    Say, you start writing a connection routine for your local domain, add error management to it (let’s face it, nothing is stable on the web), test it, and it works. You use it as a template, and write a similar one for x-domain connection (that you castrate where required) with the same error handling code, changing only the object’s name and some properties syntax.

    Chances are, you’ve done a copy-paste, then tailored the second one to use the second object.

    Now, you discover a bug in your XHR code, or an error case you hadn’t planned for; you correct it.

    Let’s imagine you’ve programmed said routine months (if not years) ago; you didn’t comment the fact that you used it as a template for another (it wastes bandwidth and time) and you don’t remember how you did it. Or, you’re not the one in charge of that piece of code anymore (even more likely in a web environment).

    Even if the bug is fixed on the XHR side, it lives on the XDR side. Had you programmed a slightly beefier routine for both local and cross domain accesses, you correct once, it’s done. And Murphy’s principle dictates that this second instance of that first bug will come and bite you in the @r$e.

    ‘cleaner, more efficient’ coding means using one object for one task; the task at hand is connecting asynchronously (after all, we’re talking AJAX here, right?) to a data source external to the webpage we’re using. That the external data source is internal or external to the domain is a security matter that has nothing to do with basic functionality – it has everything to do with implementation details.

    Creating a new object to take into account implementation details means the original one was flawed and should be deprecated – so you shouldn’t use XHR while you use XDR. But then, with XDR you can’t do stuff that you did with XHR.

    There is also one case that isn’t covered by XDR that CS+XHR covers; let’s take an example.

    Imagine you own domain mystuff.exa; you program an AJAX web app running on http://www.mystuff.exa, and that calls upon a data source hosted on sql.mystuff.exa; it’s the same top domain, so XHR should allow you to access it. You were careful, so connection takes place over https with certs duly up to date.

    Now, imagine the extremely rare case where your company changes its name, expands or merges with another and gets a few new domains; data servers are merged together, and your data source moves to sql.otherstuff.exa; suddenly, you AJAX app doesn’t work anymore.

    Solution 1: you create a bunch of aliases, and need to generate two sets of certificates. It’s workable, but you end up maintaining a dummy structure just to keep your app working

    Solution 1 bis: …or you scrap it and start over, tinkering with XDR and abusing the server’s content sniffing system (while XHR allows content negotiation, XDR relies upon server sniffing the query’s content; we all know how efficient content sniffing is, see character encoding sniffing!) to maintain the same functionality XHR gave you. And pray that said content sniffing doesn’t change (if it does, re-tinker, pray again).

    Solution 2: you use CL+XHR (XHRv2), which retains the exact same properties and methods ‘classic’ XHR had, but you merely have to add a few rules on both the server and your app to consider the external domain as having the same rights as local. You don’t need to modify the actual code.

    If you were a good little coder (meaning, a lazy coder), you created a toolkit you reused all over the place. You wouldn’t have to update the toolkit in case 2; in case 1bis though, you would have to track down every single copy of it.

    I don’t mean to say that you won’t ever have trouble programming a single routine; but when you can remove a few sources of major hassle with some careful initial code factoring, you should go for it.

    I’ll also take a bunch of branching on one object – if only because you’ll need branching anyway (timeout management, resource availability, browser capabilities, etc.), so one branch (if even, many cases can be merged with resource availability tests) more or less…

    Mitch

  37. Anonymous says:

    FWIW, there are more sides to this story: http://annevankesteren.nl/2008/06/microsoft-feedback

  38. Anonymous says:

    Mitch– Your sniffing claim is entirely bogus.  

    [1] HTTP Content-Negotiation seemed like a good idea at the time, but it’s not really used.  In practice virtually no one builds web services that accept arbitrary data types on a single URL; people create different service URLs to accept different formats.  So the Content-Type header is redundant anyway.

    [2] There’s exactly nothing from stopping you from uploading the content type in the body of the upload if you wanted to.  It’s trivial.

  39. Anonymous says:

    There are ways to solve the problems you mentioned, without the copy-and-paste anti-pattern. Your implementation isn’t limited to just two objects (although your public implementation should be to match your standard, unless the standard explicitly says how to implement the thing). Think outside the box a little more and you might figure out there are other ways to do it besides the scenarios you gave above.

    "If you were a good little coder (meaning, a lazy coder)"

    I’m tired of the "lazy coder" argument. Lazy coders don’t take the time to do things properly because they’re lazy. They don’t update documentation (or comments for that matter – they just say they write "self-documenting" code), they don’t take the time/effort needed to create good APIs (too much effort for a lazy person), and they don’t follow coding standards because they’re that would take too much effort. Lazy programmers don’t write good unit tests because they’re LAZY. You know what else is lazy? Copy and paste solutions. You know what else is lazy? Not taking the time to go back and improve performance of heavily-used code.

  40. Anonymous says:

    @Dan: why nobody use HTTP Content negotiation: as far as I know, it’s mostly because IIS and IE overrode or ignored it in wildly used cases: ‘accept */*’ in IE brought so many problems developers were forced to forget about it (Netscape caused other problems with ‘accept-encoding’, but at least they solved it in time), and IIS requires an extension to make use of it. On the other hand, making use of it on an Opera/Mozilla/Safari+Apache structure is incredibly easy and allows very fast negotiation.

    Instead (your [2]), everybody is forced to establish a connection, download the resource, sniff the result, and deal with it. With content negotiation, you can stop at the connection step – and you don’t need to download and parse 4 Mb of data only to notice your XML resource is binary noise.

    We could add the benefit of being able to use compression methods with ‘accept-encoding’, too.

    Using URIs for that would mean resources formatted like this:

    domain/myresource.xml.gz

    domain/myresource.xml.bz2

    instead of

    domain/myresource

    with header:

    resource-type:application/xml

    accept-encoding:gzip,bz2

    You may tell me that both Apache and IIS can do URL rewriting; said rewriting is based on content-type anyway, so we might as well use it directly, no?

    Of course, it wouldn’t hurt to use a dual approach:

    – use content negotiation

    – also get an hint as to the type of the resource through the URI

    And in all case check the content against its surmised type.

    But arbitrarily removing one isn’t a good idea – if only because content-type can also indicate a version that an URI can’t contain without making use of quite constraining and risky string manipulations.

    For example, ‘text/xml’ and ‘application/xml’ contain an indication that the .xml suffix alone doesn’t contain.

    The thing is, content negotiation allows faster operation when it works; removing it because many developers have gotten use to it not working is a very bad reason. It should at least be kept for developers who know how to use it. This could be added to XDR if a second object really is needed, but then that still leaves the question for its need since the existing one could be extended to do the same and more.

    @Glen: I should have been more precise, responsible and lazy (but then, a pro, or any adult, is supposed to be responsible). However you are right, precision is required: too many programmers are, indeed, irresponsible – and back to my previous comment.

    As for your first objection, it could indeed be done to write a procedure that calls upon one object or the other depending on context, with little to no code duplication; but, some attempts in similar matters (think event models) tell me it’s quite messy – the kind of stuff that isn’t appealing, and often results in creative cut’n’pasting.

    Mitch

  41. Anonymous says:

    This may seem out of left field, but Adobe implemented the "crossdomain.xml" server published policy authorization mechanism (fully implemented with their latest player version 9.0.124) to give flash the ability to have a secure cross-domain communication line. And, I just read that silverlight (from MS) is going to accept crossdomain.xml files. So, why couldn’t XDR or XHRv2 use that same mechanism? I fail to see why that wouldn’t address a good deal of the concerns.

    Yes, I do understand the point that those who’ve been working so diligently on the CD-XHR proposal are mad at MS for staying out of the game so long and then springing a competing solution on everyone with no notice. So yeah, feelings are hurt, time has been wasted, blah blah blah.

    But meanwhile, developers have to have real, and secure, solutions, and as far as I can tell, it appears that Adobe (and now Silverlight) have slipped in under the radar and done something at least worth considering.

    I think we all can agree that more work needs to be done. But, shouldn’t we move toward leveraging not just what’s already been done, but more specifically, what’s already been implemented and put out into the wild and tested?

    (and no, I’m not saying the Adobe model is perfect, or invulnerable, I’m just saying it seems like a good place to think about starting a compromise approach.)

  42. Anonymous says:

    @Kyle: The Flash cross-domain.xml file is the spirit behind the W3C proposal.  The W3C is just trying to patch all of the security holes in it, which is the only reason why their proposal different than the Adobe model.

    If you look at Adobe’s update history, they’ve had to patch the security holes in their cross-domain feature every few months… bad for security.

  43. Anonymous says:

    @Ted-yes, I understand that Adobe’s had to evolve their model as they rolled it out, and it’s not ideal to have to patch holes. but no amount of hem’ing and haw’ing about the W3C proposal is going to prevent the rollouts of it (partial or complete) from suffering the same fate… it’s the nature of the beast.

    by my real point was not just that the W3C could use a "similar" or the "spirit" of what Adobe did (and what Silvelight is now hopefully going to adopt or at least adapt), but that if this model is already rolled out, and it’s already under gone a fair amount of battle (and is nonetheless worse for the wear, with some patches for the wounds it received), then perhaps it’s a good place to start and see if it represents enough (in its current state) for a compromise between the two models.

    It would be worse for the W3C and Microsoft and Adobe to all have completely or partially incompatible approaches to the same problem, than it would be to have them all start hammering on the same solution.

    For instance, if the W3C and Microsoft were to say, ok, lots of sites already have out there the cross-domain policy files, we’ll just make the native XHR object look for those, wouldn’t it simplify this whole argument quite a bit?

    Even if as we went on, everyone (all 3 players, and all their constituents) pushed and agreed that the approach that the current generation of policy files has needs to be beefed up, it’s at least a decent starting point, rather than a wholly different approach with custom headers or content-types or other more exotic handshake negotiations.

  44. Anonymous says:

    btw, after reading the W3C proposal draft for this XHRv2, I think there are some important differences to point out compared to the "spirit" of what Adobe has already done:

    1. Adobe has support for both file based and header based transport of authorization/ACL info. This makes it more flexible and accessible to developers who author in shared environments where they cannot control the web server response environment. To mitigate the risks that unintended policy injection cause, they have a "policy on policies" which gives final say-so to a root level server policy (file or header) of whether and which custom policy locations are allowed.

    2. By using file based authorization, combined with caching, the need for the "max age" as part of the policy is removed, and is instead put on the onus of the delivery/response environment (ie, with custom cache headers), which makes this part of the system more secure and less susceptible to either developer ignorance or malicious intent.

    3. Adobe has an on-demand system for loading policies, rather than the hybrid approach of sometimes inline with the response and sometimes in pre-flight. Basically, with Adobe, it’s pretty much always pre-flight, although they do allow the policy to come in via headers PRIOR TO the request in question, as part of the payload of some other response, including for instance the actual SWF about to make the call.  So, the W3C rec could instead say that the policy could (or must) come in as part of the payload of the loading HTML file rather than requiring all the extra (but only sometimes) pre-flight requests and such.  The W3C approach seems needlessly complicated to me — just my opinion.

    —————-

    rather than just dismissing my point and lamely saying that Adobe’s just had a bunch of holes to patch, I’d love to see someone look at their current (patched) model and show how it doesn’t represent a pretty good starting point (and simpler) as compared to the W3C or MS approaches.

    In other words, what attack vectors exist for proving why Adobe’s model is insufficient?

  45. Anonymous says:

    Kyle, you’re pretty much missing the point.  Everyone started from the crossdomain.xml standpoint, for the same reason you’ve described… there’s already a deployed base of servers that already have policy files.  

    But the W3C and Mozilla and others recognized the significant number of bugs in the model made the Adobe model insecure and inappropriate for putting into the browser.  So they’ve tried to "pave over" the holes in the proposal, which is where all of this exotic handshaking junk comes from.

  46. Anonymous says:

    And, as for the notion that the patched up Adobe model is now perfect, keep in mind that there’s already an unpatched bug in the Adobe policy file model, due to their liberal behavior in allowing policy to load from anywhere on the server, and their improper canonicalization of url paths.

  47. Anonymous says:

    "The CS-XHR proposal was so unsafe that it was pulled out of FF3."

    It was turned off because the spec was changing and Firefox wasn’t conforming anymore.

  48. Anonymous says:

    Dao– exactly.  And guess why the spec kept changing?  You got it: security problems!

  49. Anonymous says:

    @Ted and Dao: yes, it was pulled from Firefox 3 because the spec wasn’t final, because it wasn’t ready, and wasn’t safe enough.

    However, if Microsoft keeps championing XDR, which isn’t perfect either, then we’ll have a known insecure object in a browser that everybody will have to work around for a dozen years (see ActiveX for a precedent) – remember, "dn’t break the web" also means "keep all the outdated cruft and bugs forever".

    The biggest part of the outrage at W3C came from the fact that MS stopped communicating and now pushes a solution with visibly no will to amend it, while if they had worked with the rest of the community, we’d currently have a cross domain object (be it CL+XHR or XDR, probably the former) which would be safe (and implemented in Firefox 3.1, Opera 10 and Safari 4).

    Mitch

  50. Anonymous says:

    Ted: I don’t think anyone claimed that the spec was final at some point in the past, so this seems totally natural.

  51. Anonymous says:

    @@Mitch: the IE team’s threads with the group and the whitepaper demonstrate how XDR is ~provably~ secure.  xdr cannot send anything that cannot already be sent using html4, so if you’re trying to say that xdr is insecure, then you’re saying html is insecure, and frankly, i doubt browsers are going to stop supporting html anytime soon.  

    As for this "outrage" nonesense– they access control spec, until very recently, wasn’t positioned as being something that would be layered on top of XHR (an object invented by microsoft) so it seems kinda silly to complain that they somehow didn’t give feedback.  

    as for the idea that "if everyone implements a bad idea (CS-xhr) then it’s somehow magically safe"— well, that’s just as ridiculous as it sounds.

    @@Dao: Okay, so you’ve got CS-XHR which no one can say is done because they cant fix the security problems, and then you’ve got xdr, which is provably secure.  wouldn’t it make sense for the WG to standardize on xdr, which microsoft offered to them for free?

    this whole argument is pretty silly IMO.

  52. Anonymous says:

    Ted: CS-XHR isn’t done because it’s in the works. Security concerns have been addressed, are being addressed and will be addressed. You think this is taking longer than it should? Well, Microsoft can help to speed this up. That’s how specs are made.

    XDR cannot replace Access Control, and applying AC to XMLHttpRequest is an inherently logical thing to do. Microsoft is free to submit XDR in parallel, but it would be wise for them to not hold it up as an alternative to CS-XHR or even AC, as they can only lose that fight. It obviously doesn’t meet the requirements that AC and CS-XHR have; it’s a crippled interim solution at the most.

  53. Anonymous says:

    @Ted: what Dao said.

    I’ll add that XDR relies on content sniffing. Every and all cases of content sniffing have been abused one way or another. The plain dumb system of control lists, much less so. The biggest problem with CL+XHR is setting up what must be checked, and how. This is where having several eyes and sample implementations (even on a different tack, like XDR) is interesting.

    But there’s a huge gap between a test type and a final version.

    Mitch

  54. Anonymous says:

    Oh, mitch… must we continue?

    <<<I’ll add that XDR relies on content sniffing>>>

    …and I’ll add that you don’t know what you’re talking about.  XDR doesn’t "rely" on content-sniffing in any way.  you shouldn’t try to talk about things you don’t understand.

    <<< The plain dumb system of control lists, much less so >>>

    I won’t even bother asking for your data, since I know you haven’t got any.

  55. Anonymous says:

    Dao… really now… check the facts…

    <<<CS-XHR isn’t done because it’s in the works.>>>

    … And it will stay in the works forever until they realize that they cannot actually make it safe.  So they’ll eventually just give up and call it done and hope that no one cares about the security problems.

    <<<Security concerns have been addressed, are being addressed and will be addressed.>>>

    yup.  and patch after patch after patch will come up.  we’ve seen this show before.  go look at the adobe flash security update change log for the past few years.

    i don’t like watching reruns.

    <<<applying AC to XMLHttpRequest is an inherently logical thing to do.>>>

    Sez who?  Microsoft INVENTED XHR and they say that this is a bad idea, so i’m gonna go with them.  that and they explained why and it makes sense to me.  trying to merge cs into xhr is a leaky abstraction, and leaky abstractions are what lead to bugs, security and otherwise.

    <<< Microsoft is free to submit XDR in parallel >>>

    they did…

    <<<but it would be wise for them to not hold it up as an alternative to CS-XHR or even AC, as they can only lose that fight.>>>

    IE’s 80% marketshare vs less than 20%.  i think xdr is gonna win, especially when people move away from any browser with cs-xhr because it’s not safe…

    <<<It obviously doesn’t meet the requirements that AC and CS-XHR have; it’s a crippled interim solution at the most.>>>

    Sounds like you’re describing html.  funny, it’s still with us today and it seems to work pretty well.

    simplicity is usually the best design.  

    xdr is simple.

  56. Anonymous says:

    @Ted-

    I don’t think I’m missing the point. And I certainly don’t think Adobe’s model is perfect. But the good thing is that it has been out there for awhile, and it’s been blasted at and tested, and many bugs have been found and fixed. Is that process done?  Certainly no. But is it farther along than even the FF3 implementation of CS-XHR or the MS XDR implementations? I think possibly.

    I was asking someone who’s maybe been a little more "in" on this process than I have been to comment on if the current implementation of the Flash Player security model (which includes the "policy on policies" which I believe addresses a good deal of the problems you talk about with policy files loading from anywhere) has been (or even could be) re-examined by both W3C and MS to see if it’s now any better at being a common ground?

    If it’s already out there, and has been used in a lot of places, and it’s been bug fixed and improved over time, then I think it deserves more than just a cursory over-glance by those in this discussion and dismissed as "been there, done that".

    I am trying to think more practically than theoretically here. I’m a web developer and have been for years, and I’d rather do my job and roll something out that I can do so in an intelligent way to minimize risks, than simply not have a functionality for years because the browser’s can’t get their common ground in order.

    Arguing about who’s right or wrong isn’t helping any of us in the trenches get a workable (if not perfect yet) solution that we can build on.

    (BTW, check out the link on my name on my posts if you want to see where my perspective on Adobe and this whole cross-domain XHR thing is coming from.)

  57. Anonymous says:

    @Ted: sources, XDR submission to W3C, and W3C analysis of said submission.

    As for browsers ignoring mimetypes, look again: even IE6 looks mimetypes up in Strict mode (it won’t load an external Javascript file with a mimetype it doesn’t know, or try to parse a file sent with application/xhtml+xml – it doesn’t know about application/ecmascript when specified in html, either).

    The same, with CSS: if the browser works in Strict mode and a CSS file is served with the wrong mimetype, all major browsers will refuse to load it (however, here, mimetype specified in HTML will be ignored – but said HTML won’t validate).

    I will admit that there are very lax interpretations of resource type matching, but to claim that it is irrelevant is misinformed.

    @Kyle: taking flXHR into account is a good idea, but not for a ‘final’ piece of code: using Flash as a security layer; while it has gotten better security-wise, I wouldn’t rely on it. It is however extremely interesting due to its cross-browser implementation, and the way it interacts with the browser itself.

  58. Anonymous says:

    @Mitch-

    Yeah, I know Flash Player’s security model isn’t perfect, but do we ever actually get to that utopian spot? It’ll always be a game where the hackers are trying to find a way to exploit things.

    My money has to go on things which seem pretty close, and importantly are trending toward improvement, which I believe the Flash Player model is. Moreover, it’s (much) closer to widespread acceptance than either of the two "mainstream" ideas we’re punting about here. Given that (at least some of) flash’s model, however imperfect, is already in almost 99% of browsers, 67% of which are already upgraded to the latest and best, IMHO, that’s a pretty good place to start.

    flXHR is certainly not meant to be "final", and of course if an author is really paranoid or dealing with really sensitive things, they probably wouldn’t yet be comfortable with it (not because of the code itself, but because of using Flash as a transport layer and its associated security holes, known and unknown).

    I’m really not here not to advertise flXHR, but to be an advocate for a workable solution for the "right now". This is how and why I built flXHR, to be a candidate for how web authors could standardize on something (an API, for instance) and then the actual internals could be different as time goes on.

    Who knows? Maybe flXHR can gain some acceptance, and then a few years from now we’ll have a better browser consensus and hopefully its internals can change (or go away) without authors having to start from scratch on how they write cross-domain communication code.

  59. Anonymous says:

    An an earlier comment, Sunava referenced an OpenAjax Alliance discussion from March 2008 where we talked about W3C Access Control versus XDR, so I want to clarify. (I manage things at OpenAjax Alliance.)

    At the time, the consensus among the OpenAjax folks was that XDR provided a more secure basis than Access Control. Most of us liked XDR’s conservatism, such as not transmitting cookies with cross-site requests. However, some months have passed now, and the Access Control effort has been evolving and all of the email discussion have helped to educate everyone involved. At this point, if asked again, I’m not sure what the OpenAjax members would say on the subject. Probably everyone would like to see what Access Control looks like after the W3C emerges from this week’s meetings and then take time to study the security implications.

    Clearly the best case is if there is a single standard for cross-site requests from the W3C, and that standard addresses security concerns appropriately.

  60. Anonymous says:

    a {color : #0033CC;} a:link {color: #0033CC;} a:visited.local {color: #0033CC;} a:visited {color : #800080;}