Experimental fix for sunbug 4670071

The Hotspot VM has a long-standing class loader deadlock, sunbug 4670071, which is top of Sun's bug parade. When the VM loads a class, it calls the loadClass method of java.lang.ClassLoader via a private, synchronized loadClassInternal method. The result is that multiple threads using multiple class loaders can deadlock.

This is a particular problem for networks of custom class loaders in JBoss, OSGi, JSR 277 (see section 8.3.4 of the EDR spec.), and elsewhere.

Sun have implemented an experimental fix, which I'll come to shortly. The point of describing the fix here is to make the caveats clear and to solicit feedback which can be taken into account in future releases of Hotspot. I, as a member of the JSR 277 Expert Group, am talking to the engineers in Sun who are looking at potential class loading improvements for Dolphin.

So what's the fix? Well, it turns out you need to specify some VM options:

-XX:+UnlockDiagnosticVMOptions -XX:+UnsyncloadClass

Please note that I am not suggesting that this is a robust fix to bug 4670071. You must treat it as unstable, experimental, temporary, unsupported, etc. In particular a custom class loader's loadClass method must be threadsafe and findClass should be synchronized to rule out various races in the VM.

However, it's not all bad news: the fix seems to work in some cases. For example, Equinox supports unsynchronized calling of loadClass. This is the default behaviour in the IBM JDK and can occur with Hotspot when loadClass is called directly from Java code (rather than implicitly by the VM while loading another class). Equinox's DefaultClassLoader synchronizes findClass.

Early release of IBM's Java 6

The Early Release Programme for IBM's Java SE 6 on a number of Linux platforms just began.

Apart from the functional enhancements inherent in Java 6 and the additional performance and RAS features we put into this release, there were significant changes in the development process.

Of particular note is the way we handled the class library code licensed from Sun. In Java 5 and other recent releases, we carried forward a base from release to release and merged in Sun's changes -- fairly labour intensive. This time we introduced a much more efficient process.

We developed a system to capture the IBM changes to Sun code such as bug fixes, performance improvements, and support for IBM platforms. Each logical group of changes is captured as a "smart patch" based on a UNIX patch file. The system automatically applies all the smart patches to each new drop of Sun code. Manual intervention is only required where the Sun code has changed significantly in an area covered by a smart patch.

The net effect is that we have been able to track the development of Sun's Java 6 much more closely than in the past. Also, the changes we make have been separated out in to logical, manageable units.

I'm hoping to contribute the smart patch system into open source as it would be useful to any project that needs to apply its own changes to a moving base.

More on JSR 277

InfoQ joined the fray. Stanley Ho's latest post drew a few comments.

Murphee pipped Mike Milinvokich and Alex Blewitt in the prize for least understated. Compare Robilad.

Public discussion of JSR 277

I'm pleased to see some public feedback.

The net of the negative feedback, e.g. on JavaLobby, seems to be "why re-invent aspects of OSGi/Maven/Ivy?". Peter Kriens' review describes some technical issues previously encountered and solved by OSGi.

A relatively positive review asks why the Expert Groups of JSR 277 and JSR 291 didn't work more closely together. BEA, IBM, Intel, Oracle, Richard (Felix) Hall, and Sun are members of both Expert Groups, so lack of working closely together isn't the reason the results are turning out rather different.

JSR 277, with the help of JSR 294, is aiming to produce a static module system integrated into the Java SE 7 platform. JSR 291 on the other hand aims to support dynamic components on existing Java SE platforms in a way which is compatible with JSR 232 on Java ME.

Looking at it another way, JSR 277/294 are attempting to produce an evolution of java.lang.ClassLoader with a distribution format, repositories, language, and VM support. JSR 291 on the other hand can be implemented in pure Java on top of java.lang.ClassLoader.

OSGi introduction

Some of my colleagues need to develop OSGi bundles and run them on Equinox. They are new to OSGi, so they asked me for some introductory reading. I thought I'd post it here for posterity and more general consumption.

I recommend starting with the technology overview on the OSGi web site, but skipping the sections on specific services on the first reading.

After reading this, you should have a feel for the three major layers in the OSGi framework.

The modularity layer enables you to package Java code with some metadata into a JAR known as a bundle, which is OSGi's module concept. Each bundle is loaded in its own class loader. Bundles share classes in a couple of ways. A bundle may import a package that another bundle exports or a bundle may require another bundle, which is similar to importing all the packages that the other 'required' bundle exports. Bundles may also contain native libraries and arbitrary kinds of resource files.

The life cycle layer allows you to dynamically install, uninstall, start, stop, and update bundles. This is essential when the OSGi framework needs to run for a long time but applications need to be managed, e.g. taken offline or have a bug fix applied, without restarting the framework, which would impact all the applications running in the framework.

The service layer enables bundles to provide services to other bundles. A service is a Java object that implements an agreed interface. Services are registered in a service registry so that users can locate the service they need, cast it to the agreed service interface, and then call the service.

If you would like to re-enforce these concepts or get a bit more detail, take a look at the architecture description.

After that, you could read about an example application, you could dip into the core specification, or you may prefer to try developing and running a simple bundle using one of the open source OSGi framework implementations. For example, the Equinox OSGi framework comes with a quick start guide.

Finally see this guide to writing a trivial bundle in 10 minutes with no special tools. Once you get beyond a "hello world" bundle, you'll want to refer to the javadoc on the OSGi web site.