Experimental fix for sunbug 4670071

The Hotspot VM has a long-standing class loader deadlock, sunbug 4670071, which is top of Sun's bug parade. When the VM loads a class, it calls the loadClass method of java.lang.ClassLoader via a private, synchronized loadClassInternal method. The result is that multiple threads using multiple class loaders can deadlock.

This is a particular problem for networks of custom class loaders in JBoss, OSGi, JSR 277 (see section 8.3.4 of the EDR spec.), and elsewhere.

Sun have implemented an experimental fix, which I'll come to shortly. The point of describing the fix here is to make the caveats clear and to solicit feedback which can be taken into account in future releases of Hotspot. I, as a member of the JSR 277 Expert Group, am talking to the engineers in Sun who are looking at potential class loading improvements for Dolphin.

So what's the fix? Well, it turns out you need to specify some VM options:

-XX:+UnlockDiagnosticVMOptions -XX:+UnsyncloadClass

Please note that I am not suggesting that this is a robust fix to bug 4670071. You must treat it as unstable, experimental, temporary, unsupported, etc. In particular a custom class loader's loadClass method must be threadsafe and findClass should be synchronized to rule out various races in the VM.

However, it's not all bad news: the fix seems to work in some cases. For example, Equinox supports unsynchronized calling of loadClass. This is the default behaviour in the IBM JDK and can occur with Hotspot when loadClass is called directly from Java code (rather than implicitly by the VM while loading another class). Equinox's DefaultClassLoader synchronizes findClass.