Multi-tenancy and Virtualization
Multi-tenancy is hard work.
Very often SaaS application will scale better if it is designed from the ground up to take care of multi-tenancy issues such as application customization and data security isolation, and adhering to the principle of minimizing hard-binding of runtime computing resources to specific tenants. However, re-implementing an application completely is not always a viable option for the SaaS ISV initially, perhaps because of time-to-market restrictions or because there is a need to quickly prove that the SaaS market demand exist without investing too much in re-engineering effort. In some of these situations, virtualization technology can help address multi-tenancy concerns to a certain degree.
In addition to SaaS ISVs, multi-tenancy issues also impact SaaS hosters who are hosting ISV applications. Resources in the hosting environment must be efficiently partitioned and shared among the hosted ISV tenants, while simultaneously respect the performance and security isolation boundaries set between different applications. Here again, virtualization technology can help address some of the multi-tenancy needs that SaaS hosters have.
To date, there are many types of products and solutions that claim to offer virtualization capabilities, with more than a few promising quick routes to the on-demand software delivery market. It is important to know that not all virtualization solutions are created equal. Although most virtualization products can play a part in delivering SaaS, it is critical that the right kind of virtualization technology is used to address the appropriate multi-tenancy issue.
The figure below is an attempt to classify the kinds of strategies and virtualization patterns for addressing multi-tenancy concerns. At a high level, there are three buckets of multi-tenancy strategies:
- Physical Separation. Physical separation relies on giving each tenant his own dedicated hardware resources. For example, assigning separate physical servers to different tenants, or in some extreme cases, a significant section of data center to a large client.
- Virtualization. Virtualization is about using software to create application hosting environments that provide logical boundaries between each tenant. We will take a further look at the various virtualization techniques shortly.
- Design the application to be multi-tenant aware. The third strategy is to design different aspects of the application to automatically adjust its behavior differently for different tenants at runtime. This has been the main focus of most of my work on SaaS architecture guidance, which is to provide ISVs with best practices so that multi-tenancy is a first class consideration in application design.
The techniques represented in the figure are stack ranked according to a tenant “density” index. You can think of tenant density as the number of tenants that can share a given amount of computing resources. For instance, comparing the techniques of dedicating a complete physical machine to a single tenant versus allocating a virtual machine slice of the physical machine to each tenant, the latter yields a higher tenant density index as the same amount of computing resource (physical machine) is transformed into a number of logical entities that can host more tenants. The density index can provide a rough sense of how cost effective a virtualization solution is relative to the number of expected tenants and the price each tenant is willing to pay for the service that is being virtualized. In general, for the purpose of enabling multi-tenancy, SaaS ISVs will require virtualization solutions with higher tenant density index than SaaS hosters.
On the various forms of virtualization:
Virtual machine technology provides a hardware emulation layer over the real hardware so that it is possible to run multiple copies of server operating systems within one physical machine. Physical hardware such as network card and disk storage is shared between the virtual operating system instances. The Microsoft Virtual Server product fits into this category of technology. Many hosters use virtual machine technology to consolidate and reduce the number of physical machines that they have to manage, which can also reduce the size of the needed data center floor space. In general, it is still quite ineffective for SaaS ISV to depend on virtual machine technology to meet their multi-tenancy needs, given the fact that it is difficult to get more than a few virtual instances of operating systems running on each physical machine. Imagine a SaaS ISV with tens of thousands of subscribing tenants - giving each tenant their own virtual machine instance is not a very cost effective multi-tenant solution. There are of course exceptions to this statement, especially in the case where the tenant is a large enterprise customer and may not mind paying significant extra dollars to cover the cost of having an exclusive solution deployed on separate OS instances. On the other hand, virtual machine solutions are very valuable to SaaS hosters providing hosting services to SaaS ISVs. Each virtual machine instance allows individual SaaS ISV to configure the operating system and install the right application platform that their solutions need. For example, an ISV can configure their virtual machine instance to run Windows 2003 Enterprise edition with Active Directory and the Kerberos Protocol transition feature turned on without causing security-related ill effects to any other ISV’s application.
Virtual service is a relatively new innovation in virtualization. SWSoft’s Virtuozzo product is an example of the virtual service technology (Also known as Virtual Private Servers in Virtuozzo’s product terminology). The primary motivation for virtual service is to reduce the amount of exclusive computing resource that has to be dedicated to a unit of isolation. So instead of having a virtualized environment that requires an entire separate OS stack, multiple virtual service instances can share and rely on the same instance of the base OS kernel stack, for e.g. the hardware and network device drivers. This architecture allows service partitions to be created and configured differently from one another. Virtual service is ideal for partitioning a machine into compartments to run different user-mode (as opposed to kernel-mode) application platforms and services. This setup allows different application platform environment to be configured without hogging excessive computing resources that may not be used effectively. For example, a virtual service partition can be configured to run IIS and SQL server 2005 database while another partition is configured to only run LDAP services. The two service partitions can be viewed as two separate server roles as application server and directory services respectively. Like virtual machine, virtual service partitions in general are not effective means of attaining the multi-tenancy goals of SaaS ISV, as the cost of pre-allocating service partition for individual tenant is still high relative to what the average tenant is willing to pay for the service subscription. However, this technology is tremendously valuable to SaaS hosters for optimizing their machine allocation, especially when there are many ISVs who have the same OS kernel requirements.
So far, what we have been describing are virtualization techniques that can be used for pre-allocating resources on a tenant basis. User session virtualization allocates computing resources dynamically to the authenticated user session within a multi-user operating system. The tenant-user relationship is also shown in the previous figure. Basically, a tenant can have multiple users. Microsoft Terminal Service is a good example of user session-based virtualization technology. To use user sessions in a SaaS deployment, a farm of terminal services is set up to accept connections from browser-based terminal service clients. In addition, the terminal services can be configured so that users can only get access to a particular application instead of the entire Windows console desktop. Unfortunately, user session based virtualization also has a rather restricted scalability constraint. This is because the original design intent of terminal service is to allow remote Windows desktop access, so lots of user shell and profile related information and services are loaded into memory every time a user session is created. As a result, user session virtualization, while it is a good tool for existing ISV to test out the SaaS business model without heavy reengineering, can put a limit on the business growth potential. The cost of scaling users directly relates to the number of additional terminal server licenses and hardware you have to purchase. However, in some cases where the ISV SaaS market is pretty unique and high end (for instance, the application is a financial tool sold to the top 100 global banks and the average bank only has 10 users), this delivery technology may suffice. You can see this URL for more info.
Application pools are server-side sandboxes for isolating applications processes. IIS application pool is an example of this technology. Application pools can help isolate security and performance related problems so that a badly-behaving process does not affect the behavior of another. Each application pool consists of a set of operating systems processes that have been created to accept application requests. By setting the limit on the number of processes that can be created in each application pool, it helps cap the system’s maximum limit on resource use and aids capacity planning. This approach is used by many existing website hosters for hosting web applications from different customers. SaaS hosters and their ISV tenants will likely use hybrid approaches that may include application pools to attain their multi-tenancy objectives. For instance, a SaaS hoster may create a new application pool for hosting each ISV tenant’s code separately but chooses to allocate a new virtual server machine instance for hosting each ISV’s SQL database. In most cases, SaaS ISV will use the same application pool to host the shared code that is access by all its tenants. In some cases, there may be large customers who demand that their application experiences not be adversely affected by other users, and dedicating an application pool to those customers may mitigate the performance concerns.
Client side application sandboxing can be an effective way of delivering standalone applications through the SaaS model. For example, an existing desktop application such as Microsoft Office can be packaged and streamed to the customer’s computers. The application is then loaded into a client side execution sandbox and executes within a virtual application runtime environment installed at the client desktop. By emulating configuration and software library facilities that the application needs to execute but without really modifying the underlying operating system settings and configurations, the client side sandbox increases the “portability” of applications and enables them to run in OS environments where the applications would otherwise have been unable to. For example, if certain system library files or registry settings are required by the application, the application sandbox is configured and packaged to be self-contained in that regard. The Microsoft SoftGrid product offers such application streaming and deployment functionalities. In addition, license control and management features are also integrated with the sandbox so that only legitimate subscribers are allowed to download and use the applications. Although there are exceptions, this kind of virtualization technology is most suited for delivering SaaS applications that exhibit the following characteristics:
- Applications that require richer interface than Web-based UI, so Windows desktop applications are good candidates.
- Application customization settings and dependencies can be effectively packaged up and redelivered to the client. The kinds of customizations usually concern the application host environment rather than the inner workings of the applications (like custom workflow or extensions to data model).
By pushing the execution of the application to the client side, this technology removes many multi-tenancy concerns that a centralized server-side application runtime host will have to address. Instead, the server-side infrastructure can focus on custom software packaging, licensing, distribution and usage control management. By the way, the SoftGrid product has many of these management facilities built-in. I highly recommend taking a look at the product if you haven’t ( https://www.softricity.com ).
Finally, in case there is any confusion, it’s important to understand that virtualization does not create more computing cycles in the machine to run parallel tenants, but rather, relies on smart time and space management of available computing resources to accommodate multi-tenancy. In fact, not only does virtualization not give you more cycles, it actually takes cycles away to do the resource management. The important point here is that virtualization is not a panacea for multi-tenancy.
Which bring us full circle back to the opening statement of this post – multi-tenancy is (still) hard work.