WebSphere Loves Windows…Who Knew?


For years I’ve heard people say that Microsoft technology is great for mid range solutions.  I can’t tell you how many CIOs have told me things like “we use WinTel for department level applications but the big iron apps run on Unix.”  There’s a general belief in the industry that the further you get into the datacenter, the less Microsoft technology you tend to find.  While license numbers, data from IDC and other major third parties paint a different story, we decided it was time to put ourselves to the test against one of the biggest players in the space – AIX running on optimized IBM Power6 with WebSphere. 


About a year ago, I blogged about some .NET / Windows Server benchmark testing results produced by Greg Leake. After taking some well deserved time off, Greg went back to the lab, expanded his testing and agreed to help get to the bottom of the “back office” debate. For the first time, the results include IBM hardware (POWER6) which allows us to evaluate price / performance using typical customer configurations. In an era of cost cutting and the need to squeeze as much optimization out of systems as possible, we think customers will find this information very interesting.  


Let’s start with costs… Greg’s findings demonstrate that customers save up to 81% in total system costs by running applications on Microsoft .NET and Windows Server 2008 vs. IBM WebSphere 7 on POWER6/AIX. The study also showed that customers who run their IBM WebSphere 7 applications on Windows Server 2008 and Hewlett Packard/Intel 64-bit blade servers can save up to 66% in total system costs when compared to running the same applications on IBM WebSphere 7 on an IBM Power6/AIX platform.  These results illuminate significant savings for businesses of all sizes, and particularly speak to the value of the Windows Server platform.  Our hope is that people are able to make use of these findings to get more for their money, either by making new investments or by maximizing assets they are already using.  Who knew WebSphere and Windows Server were such a match?  


Most folks tend to buy into the potential for cost savings but speculate that they give up performance to get it.  The study also found that for the hardware configurations tested the Microsoft .NET Framework on Windows Server 2008 handles 57% more load than WebSphere 7 running on IBM Power6/AIX.  We also found that Windows Server 2008 running WebSphere 7 handles 37% more load than IBM WebSphere 7 running on POWER6.


So, what do these findings really mean and why am I sharing them with you today? A few important things to know:


1) Windows Server and the .NET Framework continue to be a powerful, leading combination for application development, deployment and management.


2) Customers who have made a bet on WebSphere can improve performance and reduce costs by running WebSphere on Windows.


Please visit http://www.wholoveswindows.com/websphere/ to read more about these results and check out the .NET StockTrader downloads. We think the .NET StockTrader is a GREAT example of how small bits of innovation can have BIG impact.   Don’t take our word for it – these sample applications and guidelines are available to anyone. Instructions on how to replicate the testing we conducted are also available here, and I strongly encourage customers, and all third parties, to conduct tests for themselves.  Let us know what you think

Comments (9)

  1. Websphere on Windows Outperforms Websphere on AIX by 37%, Who Knew?

  2. Microsoft is taking it's "I'm a PC" campaign to the server and busting some myths in

  3. Yesterday I blogged about some recent findings regarding both system cost and performance when comparing

  4. VirtualGreg says:

    Why test an HP blade against an enterprise server like a 570?  The 570 is better suited to large scale server consolidation as a system that supports 32-cores and 3/4TB of memory.  Why pit blade against blade and test against the JS22?  Why not put your money where your mouth is against the new JS23/JS43?  If I read the pdf correctly you also test 8-core POWER6 single system with 32GB of memory against a scale out cluster of 4 blades each with 4-cores and 32GB of memory.  8-core 32GB vs. 16-cores 128GB?  Sorry, this benchmark seems a bit tainted to anyone who knows what a 570 is.

  5. Were consistently asked how high-end RISC-based systems running UNIX (in this case AIX and Power6) perform vs. the ntel/AMD-based servers running Windows Server 2008.  A common question we get from WebSphere customers is what is the performance advantage of running WebSphere on high-end RISC based systems such as the IBM Power 570 with a “scale-up” approach, and is it worth the investment?   In that sense, this test is simply one comparison – we will do others as well.  In this case we are testing a server that is a higher-end system than the Power6 blades you suggest testing, and showing a substantial performance *and* cost advantage of instead running WebSphere (or .NET) on a small cluster of industry-standard servers from HP running Windows Server 2008.  

    Another interesting comparison would be WebSphere on a mainframe vs. a similar Windows Server configuration.  Any comparison is valid, after all, as long as the system configurations and costs are fully disclosed.  The cost data is particularly noteworthy – such a huge cost difference in the systems being compared; yet the lower-cost platform is delivering better performance.  You are right in that testing other hardware configurations would be valuable as well.  

    We invested well over $215,000.00 in hardware for the Power 570 server precisely to do these types of comparisons—adding more cores would boost performance but also significantly raise the costs.  We would love to have IBM test a 16, 32 or 64-core Power 570 server in these WebSphere tests with us in an independent lab.  The BladeSystem from HP can handle up to 16 ½ height blades—we only used 4 blades—so at a cost of roughly $5000 per blade, we could also add significant scale to the HP BladeSystem/Windows Server 2008 setup.

    Similarly, a customer might invest in a Power 570 to run enterprise workloads and/or consolidate servers.  IBM positions the Power 570 as follows:

    • For mid-to-large transaction processing such as ERP and CRM applications

    • For mid-to-large database serving

    • For server consolidation across UNIX®, IBM i (formerly known as i5/OS®) and Linux® workloads

    • For a complete business system combining all aspects of a company’s IT infrastructure

    So we are comfortable in the system configurations as tested.  But we also provide a tool in the download such that customers can perform their own testing on different servers—Power6-based blades or even WebSphere on a mainframe.  Again, these comparisons are valid as well—as long as customers understand both the performance characteristics and the costs of the platforms being compared.  In this test, I think we show that running IBM WebSphere on a high-end Power system may not make a lot of sense, considering that system is three-times as expensive and performs less well than a simple cluster of 4 HP blades running WebSphere on Windows Server 2008.  Apparently, you agree, and even seem to suggest this is obvious.  We do not think it is necessarily obvious to many customers, hence the test and the “Who Knew” campaign. We encourage customers to use the kit to test other hardware/software platforms as well; such as the Power blades you suggest.

  6. CK@SH says:

    Though it is a waste of time to reply a "NO WARRANTIES, EXPRESS OR IMPLIED" document. Rumor stops at wise, I am willing to provide evidence for wise.

    It is shameful to see such a highly respected senior personel at Microsoft standing behind such a flawed methodology and benchmark results, and still trying to claim victory. May I ask a few questions about the scenario to clear my mind?

    Fairness comparison!

    ====================

    1. IBM p570 vs HP c7000 is like using a truck vs 1000 bikes for mail delivery. 1000 bikes will be more efficient and cheaper, but try using 1000 bikes to deliver a nuclear reactor instead of a truck?

    2. Please don’t ignore p570’s cost on Reliability Availability and Serviceability (RAS)! The idea of Price Performance Ratio comparison between P570 and HP c7000 misleads readers by ignoring the extra cost on Reliability Availability and Serviceability (or why you ordered AIX service?). It is obviously unfair and even ridiculous! If we find a middle school student who is better in java programming than your senior director, can I get a conclusion that this guy has high price performance ratio?Why order things that are not used in the paper, such as Advanced Power Virtualization, flat panel console, OS maintenance, 2 DVD-RAM drives, etc, while not making equivalent order in the blade system?

    What is fair then?

    ==================

    3. IBM p570 with AIX vs. HP Integrity with Windows 2008 DataCenter.

    4. IBM BladeCenter H vs. HP c7000.

    Flaws in cost comparison!

    =========================

    5. Why order things that are not used in the paper, such as Advanced Power Virtualization, flat panel console, OS maintenance, 2 DVD-RAM drives, etc, while not making equivalent order in the blade system?

    6. HP hardware price: why no Ethernet switch in the chassis, no console kit, etc?

    7. WebSphere AppServer price includes support, .NET solution has none.

    Misleading in Test bed architecture:

    ====================================

    8. Why using only 1 instance of WAS against 2 database servers? It’s not possible to make use of both of them.

    9. It would be fairer to run 4 instances of WAS on p570, distributing SQL workload to both database servers equally.

    10. The p570 has 8 cores, 32GB memory and 4 NICs, while the c7000 has 16 cores, 128GB memory and 8 NICs combined. They are not equivalent hardware or architecture, but the wide difference tells a lot.

    Misleading in Middleware optimization

    =====================================

    11. EJB is heavier for simple tasks, but better for high volume workload such as this one. Why is servlet mode being used for WebSphere Trade on p570?

    12. Setting WebSphere on p570 with 32GB memory to run at max heap size of 2GB, why waste the remaining 30GB? You could run 14 of such instances concurrently!

    13. Setting Trade to run at servlet mode with maximum web container threads at 50 on a 8-core p570? It means you’re using 8 cores to run only 50 threads max, while this system was designed to accommodate thousands of concurrent threads.

    14. On the p570 where only 1 WebSphere instance is run, setting maximum JDBC connection pool to 51 while each database server can accept 400 connections, what’s the point of this limitation? In the .NET scenario there are 4 servers, each one have 60 connections to SQL server, how is that an fair comparison?

    15. You could easily run 4 instances of WebSphere on p570 in this case, each one with 7GB heap size, ORB theads = 100, web container threads = 200, JDBC connection pool = 100, using EJB mode instead of servlet mode.

    Don’t conceal facts

    ===================

    The more concealed, the more unveiled. Since the database servers are outside of the SUT and assumed to be unlimited scalable, I have doubt that the .NET StockTrader application is using store procedures or XML processing capabilities of the SQL Server, while I know that the WebSphere Trade application is not doing that. To be fair you should put the database tier in the entire picture for more realistic end-to-end TPS cost.

    Summary

    =======

    IBM System p and commodity blade servers are not designed to run the same exact workload. Granted, there are things that the System p can do that blades cannot, while there are things that blades can do at a lower cost than System p. This is not a comparison of apple-to-apple and anyone can think of a counter-scenario where the blade system is in disadvantage.

    The p570 server was ordered with many advanced features that are totally wasted in the proposed setup. No reasonable customers would make this kind of hardware choice and software configuration.

    If software price is the only factor to consider, there are many software solutions at no cost, but they won’t have the scalability, value-added features and world class support provided by WebSphere and AIX. You could run a story comparing Microsoft Office vs Open Office where only basic features are tested, there’s no way MS Office could win in such scenario.

    Just a simple glance of the document, I can figure out those flaws, I don’t have interest to find more, but if you publish with "Microsoft is responsible for all the content in the document", IBM might have interest…

    Oh, by the way, could you please kindly do "splel chcek" for the doc again? If "Database Storeage Specification" on page 13 means "Database Storage".

  7. Well, if you have not noticed there are new benchmark results for the Trade application running as a

  8. Greg Leake says:

    Interesting feedback and objections.  Since I ran these tests over the past 6 months, I have re-posted your objections along with my responses on my blog at:

    http://blogs.msdn.com/gregleak/

    Greg Leake

    Microsoft Corporation

    PS:  excellent debate!