Jar Hell

First, pardon the interruption in your normally scheduled program. This isn't a post on interop, it is a rant. I'm not normally a raving lunatic, but this situation just blows me away.

I wanted to try Sun Java 1.5 for my Java app server install, in order to take advantage of some new ImageIO stuff: BMP Support for WordML generation. The version of JBoss I have running (v3.0.4, from 2002!) does not run on Java 1.5, so need to upgrade to 3.2.6 at least. I am afear'd of JBoss v4, with the implicit "major version upgrade", so I decide to go with v3.2.7. That sounds safe.

So I installed JBoss 3.2.7, then moved over all my WAR files - one for AXIS 1.2RC3 (axis1.2.war) and one for AXIS v1.1 (axis.war) and all the JSPs and classes under those dirs. I also moved over a couple other war dirs. I start JBoss using the Sun Java 1.5, and my WordML app and the Java ImageIO library stuff works. Molto Bene.

Then I try Axis1.2, and the initial JSPs, seem to work, but happyaxis.jsp, does not. Google tells me that to run AXIS with Tomcat (I have never done this, always Jboss+Jetty or Jetty alone), you should move saaj.jar and jaxrpc.jar from the AXIS distribution to the tomcat library directory. Why? Who knows. With the old version of JBoss (v3.0.4), I had jetty, and so didn't have this problem. But the new JBoss bundles Tomcat.

Crossing my fingers, I make the recommended change.

Start up again (60 seconds) and happyaxis.jsp is now truly happy. Ecstatic even. Other basic JSPs also work. But, now I get new errors when I try to invoke an AXIS webservice. This time it appears to point to log4j mismatches. I skulk through my directories and surmise that I don't need 7 copies of the log4j library. So I remove log4j from the AXIS war directories, (AXIS 1.1 ships 2 versions!!, AXIS 1.2 ships another version). I will hope that the log4j shipped by Jboss is the one everybody (Jboss AXIS Tomcat, and whoever else) can agree on.

Ok, no more log4j proliferation. That feels better.

Then restart. Wait 50 seconds.

Run the apps. Again the JSPs look ok. But when I try invoking a webservice, still no good. A new error:

22:04:34,158 ERROR [Engine] StandardWrapperValve[AxisServlet]: Servlet.service() for servlet AxisServlet threw exception java.lang.NoSuchMethodError: org.apache.axis.description.TypeDesc: method (Ljava/lang/Class;Z)V not found

WTF?

Not able to decipher this - it was late, a long day - I start guessing. Maybe it's a Java 1.5 thing. Maybe AXIS does not work on Java 1.5? I googled a bit but found nothing. So I tried running instead with BEA's jRockit JVM 1.4.1_02, but that also failed, but with a different exception. Actually, a cascade of about 6 nested exceptions, each with a 200-line stack trace. The important one, as far as I am able to tell, is

at COM.jrockit.reflect.NativeMethodInvoker.invoke0(ILjava.lang.Object;[Ljava.lang.Object;)Ljava.lang.Object;(Native Method)
at COM.jrockit.reflect.NativeMethodInvoker.invoke(Ljava.lang.Object;[Ljava.lang.Object;)Ljava.lang.Object;(Unknown Source)
Caused by: org.dom4j.DocumentException: Can't find bundle for base name org.apache.xerces.impl.msg.SAXMessages, locale en_US Nested exception: Can't find bundle for base name org.apache.xerces.impl.msg.SAXMessages, locale en_US
at org.dom4j.io.SAXReader.read(Lorg.xml.sax.InputSource;)Lorg.dom4j.Document;(SAXReader.java:353) at org.dom4j.io.SAXReader.read(Ljava.net.URL;)Lorg.dom4j.Document;(SAXReader.java:236)

"Can't find bundle"? Looks like a localization mixup. Complaining about a SAX library. Hmmm. SAX. I don't use SAX. Maybe AXIS uses SAX. I remember vaguely that BEA ships a SAX implementation in the jRockit distrib. Maybe it is different than the one in JBoss and they are tripping on each other?

So I try IBM JDK 1.4.1. IBM doesn’t ship SAX libraries, I know that. So no SAX-related "can't find bundle" crap. But now I am back to the same error as with the Sun JVM 1.5. a NoSuchMethodError.

Looking a little more closely, maybe it is because I have both Axis1.1 and Axis1.2 libs installed on the same Tomcat+Jboss. This worked fine with Jboss v3.0.4. It also works fine on my current Jetty install. But the error message I get here says, it is finding the class, but not the method. And it is an AXIS class. So maybe it has loaded an old version of the class it needs. Why would it do this? Why is this different than the other installations I have? Who knows.

On this hunch, I remove the Axis1.1 war dir.

Again startup on IBM JDK 1.4.1. This takes 80 seconds. Again invoke the AXIS 1.2 webservices, this time it actually works. Woo hoo! So it was a library problem. Again.

Back to the WordML sample, this does not work now, because we are running on IBM JDK 1.4.1. In Java 1.4, there is no support for reading BMP. The ImageIO libraries are there, but they don't do BMP. The operation SILENTLY fails. When I try to read a BMP stream, it doesn't throw, I just get null. No image.

So, stop JBoss again, start on Sun JVM 1.5. Now I am no longer getting the "NoMethodFoundError" because I have yanked AXIS 1.1. Good.

Again try the WordML example with the ImageIO and BMP thing. It works (again). Try AXIS 1.2 samples, they work also.

Whew.

For those counting at home, that is 5 distinct class mismatch problems I experienced when upgrading my JBoss app server, and the 2 tiny apps I run on it. Each of those 5 problems exhibited a totally different failure mode, a totally different error message, and a totally different avoidance or solution.

This is Jar Hell. What if I had real apps? Also, where is the frigging doc? Does everyone wander around in the wilderness like this, or is it just me? How does anybody get anything done?

If this isn't a clear illustration of the value of strongly-named (signed and versioned) libraries as with .NET, I don't know what is. Java's classpath and classloader mechanism is looking more and more broken by the day. Have you ever looked at the JBoss doc on classloading? Or WebSphere's? How does it interact with Tomcat's classloader, or Java's built-in classloader? Why is everybody inventing their own solutions? Why is everything different? Can anyone explain all this?

Every new product release that ships Jars (like JBoss, JVMs, Tomcat, or AXIS) just multiplies the problem. If I scanned my HD right now I'll bet I could find 12 versions of log4j. Easy. Not one is signed or versioned. Same thing with SAX, Xerces, Servlet.jar, and every other utility library. Jaxrpc.jar is next.

Compounded with that, there's the exploding matrix of classpaths from JVM JBoss Tomcat and AXIS . Each has their own set of lib directories: JBoss has at least 3 directories, the JVM has 2 or 3 (JRE + JDK + the ext directory), Tomcat has its own lib dir, and AXIS has its own. If you have jars that are duplicated in any of these, try guessing which library gets loaded. Try guessing which version your app will get.

How is this usable?

There, I feel better now.

Update 13:00 (GMT-5:00), April 1: note more on this