Preparing for Indigo ... the complete essay
Preparing for Indigo
Indigo is the code-name for a framework for communication among software entities that will be added to the second version of the .NET framework class library. While a second beta version is scheduled to be released in late 2005, version one of Indigo will debut in the second half of 2006. This document provides guidance to those who wish to use current technologies now, but also desire a smooth transition to Indigo in future.
Two articles intending to provide such guidance have already been published. Preparing for Indigo—Choosing the Right Technology Today, by Juval Lowy is available at https://www.code-magazine.com/Article.aspx?quickid=0503031. Developing Distributed Systems Today, by Richard Turner, is available at https://msdn.microsoft.com/longhorn/default.aspx?pull=/library/en-us/dnwebsrv/html/dsgprescriptiveguidance.asp. Those articles differ in some of their recommendations, but their method is the same. They both map hypothetical, isolated technical requirements to the capabilities of existing technologies, and identify the most suitable technology to satisfy each individual requirement. Thus, Lowy writes,
If you need to interoperate across boundaries, then Web services are pretty much your only option. If you do not need to cross technology boundaries, and you can assume that both the client and the server are developed in .NET, you need to ask yourself whether you need extensibility. If the answer is yes, then .NET [R]emoting is for you. […] If you do not need extensibility, then […] use Enterprise Services.
The approach that will be adopted in this document is somewhat different. It begins not with isolated technical requirements, but with the larger context of a software development enterprise, and then goes on to identify some common application architectures, and to provide suggestions on how to implement those with technologies that are available now.
A most important thing to realize is that the infrastructure provided by one’s operating system and development tools vendors will always be evolving. Successfully adopting Java Enterprise Edition, or Microsoft .NET, or Indigo, will barely register as a milestone within the endless process of adapting one’s abilities and solutions to take advantage of new features within operating systems, development tools and software products. Consequently, what is most important is to be prepared to continuously absorb new capabilities provided by one’s software infrastructure.
That objective is best achieved by adopting a suitable organizational structure. One’s development staff should be divided into at least two groups, with one group being responsible for producing finished products, and the other group being responsible for developing and maintaining one’s own framework.
Having one’s own framework is essential to maintain the productivity and agility of one’s development staff. Development tool vendors must satisfy a broad variety of customers, so the frameworks that they provide invariably offer far more options and features than any enterprise actually requires. Therefore, having one’s developers work directly with the .NET Framework Class Library or the Java Enterprise Edition application programming interface would invariably result in much wasted effort as each developer independently figured out the right way to wield those generic tools to suit their specific problems. Worse, each developer might well use the same generic tools in a different way, resulting in code that is not uniform, and, consequently, more difficult to maintain. So, rather than having one’s developers work directly with any tool vendor’s framework, one should devise a framework of one’s own, on top of the tool vendor’s generic framework, that is specifically suited to one’s purposes, and have one’s developers use that unique framework instead.
More precisely, one should have a group of developers whose task it is to develop and maintain one’s own framework. Their work will involve three steps.
First, they should figure out what the ideal architecture for their suite of applications is to be. That ideal architecture certainly need not be one that they aim to achieve very quickly, but, instead, should be one that they will strive to implement over several years.
The group’s next step should be to decide how to implement each element of their architecture. More precisely, they should figure out how their implementation of each element of their architecture should evolve from how it is implemented at present to how they would like it to be implemented in future. The architecture of any application typically incorporates five major elements: data storage and access, communication, processing, identity and access management, and user interaction.
At this point, the framework development group will not only have determined the architecture they want to achieve a few years in the future, but, also, how they will begin working towards that objective. Consequently, they can begin constructing the framework, the class libraries, that will encapsulate their decisions about how the very next version of their architecture.
Once that construction process is complete, their framework should be validated through the development of an exemplar. An exemplar is an application that satisfies a very few real functional requirements of the organization, but most of the non-functional requirements for its applications. The exemplar serves to confirm the utility of the framework, and, once that goal has been accomplished, it is turned over to the rest of the developers, those focused on building finished products, for whom it should serve as an example of how they should do their work.
It should be the task of the framework development team to absorb new capabilities of the operating system, the development tools, and other software that the organization’s software is built upon. In so doing, they will be determining how best to leverage those capabilities to achieve the more complete realization of the ideal architecture they are striving to achieve. So, if one seeks guidance on how to develop one’s software today so that it can transition smoothly to using Indigo tomorrow, then one should seek that guidance from one’s framework development team. If they have been doing their work properly, they will have studied the material that has been made available concerning that technology, and figured out how and when one’s framework will be adapted to incorporate it. Their detailed knowledge of one’s architecture and one’s own framework will certainly ensure that their recommendations are more useful than anything that Lowy or Turner could offer without the benefit of that knowledge.
However, there may be some that need guidance and cannot repair their organization first in order to get it. For them, one can imagine how a framework team would think about Indigo.
They would begin by properly understanding what Indigo is. It is often misrepresented as Microsoft’s next-generation Web services technology. If that is all that it is, then there would be no urgency to use it except in organizations that require more-advanced Web services. Actually, Indigo is the infrastructure to be added to Windows for all communication among software entities, even with entities that are not deployed on Windows. Because, as mentioned earlier, the design of almost any software application will incorporate some element of communication among software entities, there is almost no Windows software that one could develop today that would not benefit from Indigo.
The benefits of Indigo are myriad. Most importantly, it separates everything to do with how a software entity is to communicate from the code of the entity itself. One specifies all of the ways in which the entity is to communicate in a configuration file, and Indigo, as a software factory for communication, generates the runtime components needed to facilitate the specified ways of communicating. As a result, if the entity’s modes of communication need to be modified or enhanced, that can be done simply by editing the configuration file, rather than the code for the entity. The same cannot be accomplished with any other technology that exists today, or with anything else on the horizon. Today, in planning how a software entity is to communicate, one is forced to follow the kind of reasoning exemplified in the quote from Lowy’s article above: if one wishes to have the entity communicate in this way, then one must use that technology, but if one wishes to have it communicate in another way, then one must use a different technology. Thus, far from an entity’s modes of communication being separated from the entity itself, as it is in Indigo, with today’s technologies, changes to how the entity is to communicate entail the entity having to be re-architected to incorporate the suitable technology. Furthermore, because the Windows Communications Framework can facilitate any mode of communication, it is the only technology one must assimilate to accomplish any type of communication. With today’s technologies one must adopt a different one for each kind of communication one wishes to accomplish, and adopting each technology bears the cost of learning it, and getting accustomed to its strengths and weaknesses as well as those of its associated tools.
To properly appreciate the choice that Indigo presents to one, consider this analogy with modes of human transportation. Today, each standard transportation technology is restricted to facilitating a particular type of transportation. If one requires inexpensive transportation with the lowest drain on natural resources, then one should use a commuter train, but if one needs fast, private transportation over relatively short distances, then one should use an automobile, and if one must have fast transportation to anywhere on the planet, then a jet aircraft is the only option. Now, imagine if one received notice from one’s government that it would be providing everyone with a vehicle, free of charge, that one could keep forever to fly quickly to anywhere on Earth, traveling more safely than in any vehicle existing today, and as easily to a different continent as to the local grocery store, all without burning fossil fuels and costing no more to use than a monthly commuter train pass. The only bad news in the government’s notice is that one would have to learn to pilot the new vehicles, although that was demonstrably much easier than learning to drive a car, and that the vehicles themselves would only be available at some unspecified date next year. However, the government was also encouraging everyone to try out prototypes that were available now, and providing large teams dedicated to assisting in the maintenance of the vehicles, and a second generation prototype at the end of the year that would have a government-issued safety rating. In that scenario, one might or might not leap into one of the current prototypes for some low-speed and low-altitude hops, although most people probably would. However, almost no sensible person would ask, “given that these new vehicles will be given to everyone next year, which type of car should I invest in today?” Asking which technology for communication among software entities one should adopt today anticipating the forthcoming release of Indigo next year, is about as absurd. One should naturally begin using the prototypes of Indigo right now, and certainly forego any investment in existing technologies for the time being.
For those extremely rare cases where investing in an existing technology is unavoidable, here are the bare facts that the framework team would consider:
1. COM+ and Enterprise Services software entities, without the use of third-party tools, are restricted to communicating with COM-compliant Windows software entities, and the communication cannot traverse firewalls without non-HTTP ports being opened. However, the communication is very fast, and can be configured to incorporate authentication data and information about the state of a transaction. The format of the communication will not be compatible with that of Indigo, but Indigo will incorporate a tool that will provide for transparent translation between the two formats, and thereby make COM+ and Enterprise Services software entities appear to be Indigo applications to native Indigo clients.
2. .NET Web Services can be used for communication between software entities developed using Microsoft tools or compliant alternatives from different vendors, and other species of software entities that may or may not be deployed on Windows. The communication is not very fast, and does not incorporate authentication data or information about the state of a transaction without considerable effort at customization. Extraordinary effort would be required to make the communication reliable. However, the communication is via HTTP, so it traverses firewalls via the HTTP port. Also, the format of the communication will be fully compatible with one of those of Indigo.
3. The Web Services Enhancements Toolkits for .NET are primarily for the purpose of easily incorporating authentication data into .NET Web Services communications. With version two and later versions, it is also possible to have the communication work via the TCP protocol, which provides for reliability, albeit at the expense of being able to able to traverse firewalls via the HTTP port, and, due to quirks in the implementation, also at the expense of performance. The format of communication will be fully compatible with that of Indigo only for version three and higher, and only via HTTP.
4. .NET Remoting is intended for facilitating communication among .NET software entities running in different processes on the same machine, or running on different machines. It provides for various modes of communication, including HTTP for easy traversal of firewalls, and TCP for reliability. Considerable effort at customization is required for authentication data and information about the state of a transaction to be incorporated. None of its communication formats will be compatible with those of Indigo, and no tools will be provided for translation between the formats of the two technologies.
5. MSMQ and System.Messaging are for durable, reliable communication among software entities developed using Microsoft tools or compliant alternatives from different vendors, although products exist for extending that communication to other species of software entities that may or may not be deployed on Windows. Authentication data and information about the state of transactions can be readily incorporated. The messaging pattern is natively asynchronous and not intended to be speedy. From version three, the communication can traverse firewalls via the HTTP port, and the format of communication will be fully compatible with one of those provided by Indigo.
One’s framework development team would seek to apply these facts in weighing their options for implementing the communication elements of their architecture. To surmise the conclusions they might draw, one must begin by envisaging the architecture they might be contemplating. Here are three common architectures that might cover many of the actual possibilities:
1. a typical Web application, with browser-based user interaction implemented using ASP.NET, and data storage and access provided by Microsoft SQL Server and ADO.NET
2. a smart client application, in which a Windows user interface accesses data in a remote, shared data store
3. a human machine interface, in which a Windows user interface displays data acquired from an external programmable logic controller by a software entity the data to subscribers
In implementing the first of these common architectures, that of the Web application, one should avoid cross-process communication as much as possible to maximize throughput. The reason for having to maximize throughput is that the resources of the few servers on which the application is deployed will need to provide for almost all of the application’s capabilities for every user, because the users’ browsers generally have very limited capabilities, and even fewer that every browser is certain to have. Besides the communication of the browser with the components deployed on the Web servers, the only cross-process communication should be between those components deployed on the Web servers and the relational database management system. That communication should be via ADO.NET and secured using IPSec. For the communication between the browser and the components deployed on the Web servers, HTML form submissions would be the most widely-supported by browsers. However, another increasingly common alternative is to have script embedded within the data initially transferred to the browsers communicate with components deployed on the Web servers in the form of XML over HTTP. .NET Web services would be a perfectly good way to implement the components on the Web servers.
For the smart client application, some of the data that will need to be transferred from the remote, shared data store to the user interface is data that would not be modified by the user. Such data might include product and price lists in an application used by a mobile sales force, for example. One good way of communicating that data from the remote, shared data store to the user interface would be to have a data store local to the user interface managed by a free relational database management system such as the Microsoft Database Engine or SQL Server Express, and configure replication between that data store and the remote, shared one. System.Messaging via MSMQ over HTTP would be the easiest and most robust way of implementing the communication of data that the user might modify, and would also be the option that would be most easily adapted for use with Indigo later on. For cases where that option might not be appropriate for some reason, using version three of the Web Services Enhancements Toolkit for .NET would be next best choice.
Another scenario might be one in which components of the Web application are also to be used by components of the smart client application. Since asynchronous communication is of limited applicability in Web applications where the browser’s mode of communication is natively synchronous, System.Messaging and MSMQ are less likely to appropriate, and using version three of the Web Services Enhancements Toolkit for .NET to make the components available would generally be a good and workable option.
In the human machine interface scenario, the human machine interface would register a subscription to updates with the component acquiring data from the programmable logic controller. In some cases, using System.Messaging to accomplish the subscription and the subsequent transmission of updates to the human machine interface would suffice, and that solution would transition smoothly to Indigo. However, there are other cases in which a speedier mode of communication may be necessary. Those would be ones in which .NET Remoting would seem to be the only practical option. However, the limitations of that technology and the non-existent support for integrating it with the Windows Communications Foundation also make it the very best example of a case one would be well advised to use the beta version of the Windows Communications Foundation instead.
So, the foregoing has taken a different approach to providing guidance on what to do in anticipation of Indigo than the approach taken by Lowy and Turner in their articles. Here, the primary recommendation has been to organize one’s software development process properly to assimilate ongoing change in one’s software infrastructure. The second important recommendation was to recognize the absurdity, in all but the most exceptional cases, of not adopting Indigo right away, even in its beta release forms, and continuing to invest in the current technologies, which are so severely limited by comparison. Finally, for those exceptional cases, some recommendations were offered for various commonplace architectural scenarios.