Java modularity: where do we go from here?

Things have moved on somewhat since March, so I thought an update would be helpful. Let's recap first.

JSR 277 is trying to create a static module system for Java. Its offshoot, JSR 294, is focused on the VM and language support for modularity. Both these JSRs are targeting Java 7.

JSR 291, which recently passed the Final Approval Ballot, references and extends the OSGi dynamic component system. JSR 291 is compatible with the Java ME JSR 232, which also references OSGi.

OSGi is a mature technology, having been developed in four releases over the last eight years. There are three independent open source implementations (Equinox, Felix, and Knopflerfish), a specification, reference implementation, and TCK.

I am an active member of the JSR 277 and JSR 294 Expert Groups and am the spec. lead of JSR 291.

Design goals

Crucially, the goals of these JSRs are rather different, although the underlying concepts are similar. JSRs 277 and 294 are building basic support into the Java SE platform whereas JSR 291 is pure Java built on top of the Java platform.

Look at it another way. JSRs 277 and 294 are attempting to produce an evolution of java.lang.ClassLoader with a new distribution and packaging format, a new repository, and new language features supported by the VM. JSR 291 on the other hand is implemented in pure Java on top of java.lang.ClassLoader and uses standard JAR files with standard manifests as a distribution and packaging format.

The design point of OSGi has always been to support a broad set of Java ME environments, such as profiles based on the Connected Device Configuration, and Java SE versions, from Java 1.2 onwards, in a compatible way. It has achieved this by building on commonly available Java concepts, notably class loaders.

I think of OSGi as being the pinnacle of custom class loader systems. The way it manages to install, uninstall, start, stop, and update modules without restarting the VM is quite an achievement - not my achievement, I should add, as the dynamic features were mostly complete before I got involved. Of course, other systems have provided some of this capability, but not in such a generally reusable way.

So you can package OSGi with your application, or list it as a prerequisite, and then run on a wide range of Java platforms.

On the other hand, JSRs 277 and 294 have the advantage that if you are happy to pre-req. Java 7 or later, then those JSRs will be built in to the platform.

Why doesn't JSR 277 reuse OSGi?

So why can't the JCP simply adopt JSR 291 as a standard module system for the Java platform? That's what many of the critics of JSR 277 are calling for.

A crucial issue is one of control. I hope the OSGi Alliance would allow OSGi to be included in the Java platform so long as they retained control of the technology. But I suspect Sun would be nervous about the core module system of the Java platform not being under their direct control. Such nervousness would be understandable if it was not in the interests of the OSGi Alliance to maintain the value of Java, but the reality is just opposite: the success of the OSGi Alliance depends to a large extent on the success of Java.

In my earlier blog, I made the point about the OSGi technology still evolving. But I think that problem is soluble. There would need to be a way of overriding the built-in module system with a later, backward compatible version.

Why not scrap JSR 277?

A radical option, favoured by some, of simply scrapping JSR 277 is less than ideal. It would temporarily remove the confusion that competing module systems inevitably create. But before long, the requirements for a module system built in to the Java platform would surface again.

The consumer JRE work has shown there is a real need to modularise the JRE itself and this requires a built-in module system. However, the consumer JRE doesn't seem to be proposing a properly architecture module system. It's a 'quick fix' which can be delivered in Java 6 update releases.

So if JSR 277 was scrapped, then whatever mechanisms evolve out of the consumer JRE work could become the defacto module system for the Java platform. The Java platform would still lack a properly architected built-in module system.

JSR 277 also has a stated objective of interoperating with JSR 291, which means JSR 277 modules will be able to make effective use of OSGi modules and vice versa with minimal restrictions. If this objective is achieved, it will protect the significant investment in OSGi modules in the Java community. Scrapping JSR 277 would jeopardise this investment if a non-interoperating module system grew up in place of JSR 277.

The road ahead

So, given the momentum behind JSR 277 and the emphasis placed on it by Sun at this year's JavaOne, what lies ahead?

Of course, JSR 291 should ideally benefit from the features introduced by JSR 277 and JSR 294. But the immediate requirement is that JSR 277 and JSR 291 interoperate well so that modules written in terms of one JSR are able to make effective use of modules written in terms of the other JSR.

To achieve this, the Expert Groups of JSR 277 and JSR 291 will need to work together. The JSR 277 Expert Group are responsible for producing an interoperation proposal which the JSR 291 Expert Group will review. At the time of writing, the interoperation proposal is still being worked on by the JSR 277 spec. lead.


The dream solution

The dream solution is clear: JSR 277 should adopt JSR 291, underpinned by language and VM support in JSR 294, and add the JSR 277 repository architecture. This would accelerate the progress of JSR 277 by several years and guarantee perfect interoperation between JSR 277 and JSR 291.

But it would take a complete about turn by Sun to turn this dream into reality. Maybe, just maybe, Sun are sufficiently committed to the success of Java that they will eventually decide to do just that. Maybe, just maybe, the Java community will be able to look back later and paraphrase Winston Churchill: "Sun could always be counted on to do the right thing, after having exhausted all the alternatives".

Further information

The JSR 291 Expert Group mailing list and the JSR 294 list have been openly readable since their Expert Groups started. JSR 277 ran in private for many months, but has now opened up its list.

Disclaimer

I'm biassed.

I have been involved in the OSGi Core Platform Expert Group for a couple of years during which time I co-authored some of the modularity improvements for OSGi R4 with Richard Hall (the leader of the Apache Felix OSGi project).

I work for IBM which belongs to the OSGi Alliance and which has based some of its products on OSGi. IBM and Sun co-operate on Java, but compete in related areas such as Java EE, not to mention UNIX-style operating systems and hardware. So IBM and Sun don't necessarily agree on everything that happens in Java. In fact, I think it would be pretty unhealthy if that were the case.

I should also re-iterate that my opinions are not necessarily shared by IBM.

That said, I hope this blog objectively lays out the situation.