Jar Hell


First, pardon the interruption in your normally scheduled program. This isn’t a post on interop, it is a rant. I’m not normally a raving lunatic, but this situation just blows me away.

I wanted to try Sun Java 1.5 for my Java app server install, in order to take advantage of some new ImageIO stuff: BMP Support for WordML generation. The version of JBoss I have running (v3.0.4, from 2002!) does not run on Java 1.5, so need to upgrade to 3.2.6 at least. I am afear’d of JBoss v4, with the implicit “major version upgrade”, so I decide to go with v3.2.7. That sounds safe.

So I installed JBoss 3.2.7, then moved over all my WAR files – one for AXIS 1.2RC3 (axis1.2.war) and one for AXIS v1.1 (axis.war) and all the JSPs and classes under those dirs. I also moved over a couple other war dirs. I start JBoss using the Sun Java 1.5, and my WordML app and the Java ImageIO library stuff works. Molto Bene.

Then I try Axis1.2, and the initial JSPs, seem to work, but happyaxis.jsp, does not. Google tells me that to run AXIS with Tomcat (I have never done this, always Jboss+Jetty or Jetty alone), you should move saaj.jar and jaxrpc.jar from the AXIS distribution to the tomcat library directory. Why? Who knows. With the old version of JBoss (v3.0.4), I had jetty, and so didn’t have this problem. But the new JBoss bundles Tomcat.

Crossing my fingers, I make the recommended change.

Start up again (60 seconds) and happyaxis.jsp is now truly happy. Ecstatic even. Other basic JSPs also work. But, now I get new errors when I try to invoke an AXIS webservice. This time it appears to point to log4j mismatches. I skulk through my directories and surmise that I don’t need 7 copies of the log4j library. So I remove log4j from the AXIS war directories, (AXIS 1.1 ships 2 versions!!, AXIS 1.2 ships another version). I will hope that the log4j shipped by Jboss is the one everybody (Jboss AXIS Tomcat, and whoever else) can agree on.

Ok, no more log4j proliferation. That feels better.

Then restart. Wait 50 seconds.

Run the apps. Again the JSPs look ok. But when I try invoking a webservice, still no good. A new error:

22:04:34,158 ERROR [Engine] StandardWrapperValve[AxisServlet]: Servlet.service() for servlet AxisServlet threw exception java.lang.NoSuchMethodError: org.apache.axis.description.TypeDesc: method (Ljava/lang/Class;Z)V not found

WTF?

Not able to decipher this – it was late, a long day – I start guessing. Maybe it’s a Java 1.5 thing. Maybe AXIS does not work on Java 1.5? I googled a bit but found nothing. So I tried running instead with BEA’s jRockit JVM 1.4.1_02, but that also failed, but with a different exception. Actually, a cascade of about 6 nested exceptions, each with a 200-line stack trace. The important one, as far as I am able to tell, is

at COM.jrockit.reflect.NativeMethodInvoker.invoke0(ILjava.lang.Object;[Ljava.lang.Object;)Ljava.lang.Object;(Native Method)
at COM.jrockit.reflect.NativeMethodInvoker.invoke(Ljava.lang.Object;[Ljava.lang.Object;)Ljava.lang.Object;(Unknown Source)
Caused by: org.dom4j.DocumentException: Can’t find bundle for base name org.apache.xerces.impl.msg.SAXMessages, locale en_US Nested exception: Can’t find bundle for base name org.apache.xerces.impl.msg.SAXMessages, locale en_US
at org.dom4j.io.SAXReader.read(Lorg.xml.sax.InputSource;)Lorg.dom4j.Document;(SAXReader.java:353) at org.dom4j.io.SAXReader.read(Ljava.net.URL;)Lorg.dom4j.Document;(SAXReader.java:236)

“Can’t find bundle”? Looks like a localization mixup. Complaining about a SAX library. Hmmm. SAX. I don’t use SAX. Maybe AXIS uses SAX. I remember vaguely that BEA ships a SAX implementation in the jRockit distrib. Maybe it is different than the one in JBoss and they are tripping on each other?

 

So I try IBM JDK 1.4.1. IBM doesn’t ship SAX libraries, I know that. So no SAX-related “can’t find bundle” crap. But now I am back to the same error as with the Sun JVM 1.5. a NoSuchMethodError.

Looking a little more closely, maybe it is because I have both Axis1.1 and Axis1.2 libs installed on the same Tomcat+Jboss. This worked fine with Jboss v3.0.4. It also works fine on my current Jetty install. But the error message I get here says, it is finding the class, but not the method. And it is an AXIS class. So maybe it has loaded an old version of the class it needs. Why would it do this? Why is this different than the other installations I have? Who knows.

On this hunch, I remove the Axis1.1 war dir.

Again startup on IBM JDK 1.4.1. This takes 80 seconds. Again invoke the AXIS 1.2 webservices, this time it actually works. Woo hoo! So it was a library problem. Again.

Back to the WordML sample, this does not work now, because we are running on IBM JDK 1.4.1. In Java 1.4, there is no support for reading BMP. The ImageIO libraries are there, but they don’t do BMP. The operation SILENTLY fails. When I try to read a BMP stream, it doesn’t throw, I just get null. No image.

So, stop JBoss again, start on Sun JVM 1.5. Now I am no longer getting the “NoMethodFoundError” because I have yanked AXIS 1.1. Good.

Again try the WordML example with the ImageIO and BMP thing. It works (again). Try AXIS 1.2 samples, they work also.

Whew.

For those counting at home, that is 5 distinct class mismatch problems I experienced when upgrading my JBoss app server, and the 2 tiny apps I run on it. Each of those 5 problems exhibited a totally different failure mode, a totally different error message, and a totally different avoidance or solution.

This is Jar Hell. What if I had real apps? Also, where is the frigging doc? Does everyone wander around in the wilderness like this, or is it just me? How does anybody get anything done?

If this isn’t a clear illustration of the value of strongly-named (signed and versioned) libraries as with .NET, I don’t know what is. Java’s classpath and classloader mechanism is looking more and more broken by the day. Have you ever looked at the JBoss doc on classloading? Or WebSphere’s? How does it interact with Tomcat’s classloader, or Java’s built-in classloader? Why is everybody inventing their own solutions? Why is everything different? Can anyone explain all this?

Every new product release that ships Jars (like JBoss, JVMs, Tomcat, or AXIS) just multiplies the problem. If I scanned my HD right now I’ll bet I could find 12 versions of log4j. Easy. Not one is signed or versioned. Same thing with SAX, Xerces, Servlet.jar, and every other utility library. Jaxrpc.jar is next.

Compounded with that, there’s the exploding matrix of classpaths from JVM JBoss Tomcat and AXIS . Each has their own set of lib directories: JBoss has at least 3 directories, the JVM has 2 or 3 (JRE + JDK + the ext directory), Tomcat has its own lib dir, and AXIS has its own. If you have jars that are duplicated in any of these, try guessing which library gets loaded. Try guessing which version your app will get.

How is this usable?

There, I feel better now.

Update 13:00 (GMT-5:00), April 1: note more on this

Comments (12)

  1. matt says:

    I agree. That’s the biggest problem that I see with open source products. Nobody cares about compatibility. It doesn’t help that Java uses such a moronic methodology for determining jar file dependencies. Also, the mindset of the OS community is to borrow as much as you can from others. This just compounds the problem as none of the OS projects out there care or know about the compatibility issues they create when changing jar file dependencies.

    I went through similar issues with Interbase and Tomcat. I finally worked it all out after an awful lot of searching. The typical response you get to these issues is to RTFM or recompile everything yourself. That just isn’t a usable solution in my mind.

  2. Jim Arnold says:

    If you think that’s bad, you should try getting all the .Net 2.0 CTP builds working together…

    Jim

  3. Dino says:

    Hani, talking about PicoContainer 2 days ago, said, "It also has next to no dependencies, which makes the thought of depending on it (no pun intended) far less of a house of cards that is the usual case with opensores drivel."

    http://www.jroller.com/page/fate/20050328#picocontainer_tagnuts_from_a_thousand

    Apparently others have also explored this landscape …

  4. Dino says:

    Yeah, Jim, I’ll give you that, there is a cost to using beta software. But that is a "known" cost, isn’t it? When people see "beta", they should think "I’ll need to invest time to make this work." And the .NET 2.0 CTP isn’t even "beta". The CTP’s are really for bleeding edge adopters.

    But this Java stuff I am talking about – these are all released products or projects. Well, except for AXIS 1.2, which is at RC3 (near final). Generally it’s Mature Stuff. Or it says so on the label.

    The .NET final release will be much more solid and better integrated than the CTP. When will that happen with the projects I mentioned? Oy.

  5. Walt says:

    I just went through a similar process (ie pain). What I finally did was individually download and install the separate pieces. For example, Xerces can be downloaded and installed separately. Usually, I create a directory (eg C:JavaDev) and throw (unzip) all the stuff into this directory — notice the directory dosn’t have any spaces, Java still has a hard time dealing with directories with spaces (isn’t it 2005?). Anyway, I setup a batch file with its own CLASSPATH that points to all the necessary JAR files under JavaDev. Then the batch file launches Tomcat, etc…

    I feel your pain.

  6. Dino says:

    addendum, I later found this:

    UseJBossWebLoader: This flag indicates that Tomcat should use a JBoss unified class loader as the web application class loader. The default is true, which means that the classes inside of the WEB-INF/classes and WEB-INF/lib directories of the WAR file are incorporated into the default shared class loader repository described in Chapter 2, The JBoss JMX Microkernel. This may not be what you want as its contrary to the default servlet class loading model and can result in sharing of classes/resources between web applications. You can disable this by setting this attribute to false.

    from the JBoss doc. http://docs.jboss.org/jbossas/jboss4guide/r1/html/ch9.chapt.html

    Had I known this, I could have avoided the "NoMethodFoundError" .

  7. Very few of the programs I use handle installation and configuration management well. I don’t consider it a high priority in the programs I work on either, so it’s done just well enough to get by. That just follows from the fact that my customers don’t consider it important compared to other issues that we could be working on.

    I’ve used a small number of commercial programs that have really satisfied me in this area. I don’t think the top-end open source programs do as well. Installation and configuration issues just may not be sexy enough to attract enough developers to work on the problem.

    It is a problem that people are concocting their own solutions for things like loading and configuration. They almost never do as good a job as the system provided equivalent. Early on this seems like a good approach because your custom version is smaller/faster/simpler or whatever. But over time, it seems to always grow to have almost the same scope as the system version and you lose all the advantages.

    However, I don’t think strong naming is really the solution. Installation and configuration is more fundamentally broken than that in many cases. I like the copy-is-install approach taken by Apple and some groups using .NET and Java. Things like no registration COM are just great and make people’s lives better even if it doesn’t sound very exciting to talk about.

    This way of handling installation doesn’t totally solve the problem either, but I think when people build programs using this approach they more naturally make the system robust against these types of problems. Not having an installer cures a lot of 1) installer bugs and 2) program assumptions about how the installer will leave the system.

  8. SjH says:

    I work in a company which uses almost exclusively open source bits for development (i.e. Java development) and have come up against similar problems from time to time, usually at the start of a project. The thing is that once you’ve got over the worst of it (and it can, I agree, be eye-crossingly infuriating) and settled on versions of this and that (which we, ironically enough, distribute to developers via a years-old version of Visual Source Safe), you tend not to come across it again for a reasonable amount of time. Usually not until the start of the next project, by which time if the previous experience was bad enough the memory of it will have been nicely repressed..

    SjH

  9. Saw this in my referrers, so thought I might as well give some advice.

    The problem you have is with your choice of tools. Why the hell are you using jboss? Switch to Orion or Resin (or pretty much any decent appserver) and you’ll find that a lot of this pain goes away.

    The java world is, sadly, full of absolutely abysmal popular products, that people use simply because they’re popular and because they don’t know better. it’s not really fair to say that food sucks when all you eat is dung!

  10. @shish daga says:

    Boss — my  2 resources spent 4 hrs of time on this issue. Finally got u’r note and able to resolve it.

    I wish this solution comes as first result when u search in google or live.

Skip to main content