JSR 291 but not as we know it

Google sometimes coughs up unexpected stuff, like this which could almost be talking about JSR 291 and JSR 277. :-)

Eclipse Europa

Coté's coverage of Eclipse Europa expresses a view on OSGi and JSR 277.

Representations of module dependencies

One of the key differences between JSR 291 and JSR 277 is the way in which module dependencies are represented, satisfied, and managed. Both module systems express dependencies using import declarations with version ranges and so, in general, there can be multiple ways of satisfying any particular dependency.

Using an electronics analogy, there can be multiple different ways of wiring a module's dependencies to other modules. Without further constraints, the choice of wiring is essentially non-deterministic, which would make the module system hard to understand and unpredictable in behaviour. The non-determinism needs to be removed.

JSR 291 removes the non-determinism by specifying rules for how dependencies should be satisfied in the presence of a given collection of modules, some of which are already resolved (or 'wired') and others which are unresolved (not yet 'wired').

JSR 277 also specifies some rules to remove the non-determinism, but it allows a module to implement an import policy if the module wants to override these rules. The hope is that, in many simple cases, the import policy will not be needed. But, in general, JSR 277 removes the non-determinism using a combination of rules in the specification and user-written import policies.

Why is this difference important? Well, an import policy is a piece of Java code running inside the module whose dependencies need to be satisfied. So the import policy is in the unusual position of not being able to use the modules upon which its module depends. This is a potential source of errors.

But a more important difference relates to the need to be able predict the behaviour of a collection of modules. This is crucial when managing the dependencies of modules. With JSR 291, an external management system is able to read the dependency declarations in each module and apply the rules in the specification to determine how these modules will be wired up, whether there are any missing dependencies, and, if there are, how such dependencies could be satisfied.

With JSR 277 the position is quite different when import policies are used. The only way to determine the behaviour of an import policy is to execute it. But even then, there is no guarantee that the import policy will give the same result each time it runs. Also, if there are missing dependencies, it is not feasible to examine the import policy to determine how the missing dependencies could be satisfied.

(For further reading, I recommend Peter Kriens' JSR 277 review from last October.)

Module-superpackage relationship

In the comparison of JSR 277 and JSR 291 features, I alluded to the fact that the module-superpackage relationship is a difference, but Andreas pointed out another way in which the relationship differs.

My original observation was that the cardinality of the relationship is different. JSR 277 specifies modules which build directly on superpackages: there is a one-one relationship between deployment modules and superpackages.

JSR 291, on the other hand, makes no explicit reference to superpackages as these are yet to find their way officially into Java. But, if JSR 294 continues in its current direction, it will be possible to include one or more superpackages inside a JSR 291 module. So the relationship between JSR 291 modules and superpackages will have 1:0..n cardinality.

Andreas' point was that JSR 277 is integrated with JSR 294 in that a JSR 277 module's metadata consists of a superpackage and its annotations.

JSR 291, in contrast, is currently agnostic about superpackages and their names, members, exports, and annotations. JSR 291 maintains its own metadata in a separate manifest file and so is not integrated with JSR 294.

Sun VP 'gets' OSGi

I just ran across a video where James Governor interviews Jeet Kaul, the VP of developer products and programmes for Sun Microsystems. If you don't have much time, fast forward to three minutes from the start and listen to the next minute and a half on OSGi.

Some soundbytes plucked slightly out of context for maximum effect:

JG: "I'm looking at OSGi ... and I think it's possibly the most important standard in the industry right now." (What a nice man.)

JK: "... so much has been done already that it would be insane to ignore that ..."

and later

JK: "... in Java SE 7 where we are trying to build the standard which will incorporate supporting OSGi bundles - it's the right approach."

Default export granularity

In the comparison of JSR 277 and JSR 291 features, I caused some confusion by saying the default export granularity was 'class' for JSR 277 and 'package' for JSR 291. I'll try to explain.

The primitive unit of export in JSR 277 is the class. You can use a wildcard to export all of a module's classes in a specified package, but that's just syntactic sugar. The wildcard will be expanded to a list of classes in the binary file representing the module's superpackage. There is no runtime notion of an exported package.

The primitive unit of export in JSR 291, on the other hand, is the package. You can filter the set of classes available to importers, but the runtime deals in exported packages rather than classes.

I hope that's a bit clearer. Also, try not to confuse default export granularity with import semantics. JSR 277 supports only module import whereas JSR 291 supports package and module imports.

Comparison of JSR 277 and JSR 291 features

A number of people have expressed an interest in the similaries and differences between these JSRs, so this post compares the features of the Early Draft Review specification of JSR 277 and the Final Specification of JSR 291, although you'll have to read the specifications to understand the differences in detail.

This a functional comparison and doesn't attempt to draw out advantages and disadvantages of various features or other aspects of the JSRs. For ease of comparison, I've referred to JSR 291 bundles as modules.

Equivalent features:

• Module name
• Module version
• Module attributes
  
• Module import
• Module re-export
  
• Version range
• Optional import
• Executable modules
• One class loader per module
• Platform dependencies
• Modules may contain classes, native libraries, resources, and extensible metadata
  
• Module signing
• JAR distribution format
• Reflective API
  

Differences (order: JSR 277 vs JSR 291):

• Programmatic import policy vs declarative import constraints (details)
  
• Class space consistency: validation vs constraint solving
  
• Default export granularity: class vs package (details)
  
• Module-superpackage relationship (details)
  

Unique to JSR 277:

• Standard repository:
• organised hierarchically with parental delegation
• multiple standard implementations for local or networked deployment
• support for plugging in 3rd party repository implementations
• JAM distribution format
• Import override policy
  
• Part of the Java SE platform (planned for Java 7)
  

Unique to JSR 291:

• Dynamic lifecycle support both for module lifecycle and service lifecycle
• Service support (above and beyond the basic service provider support in Java SE)
  
• Package import
• Dynamic package import
  
• Package attributes
• Mandatory attributes
• Split package support
• Module classpath
  
• Fragment modules
• Extension modules
• Pure Java implemented on top of Java SE
• Compatible with Java ME (JSR 232)
  

One last hurdle for JSR 291?

The PMO acknowledged receipt of the Final Release materials I sent them a couple of days ago, so at least their email system worked this time. However, they said the spec. license and the final business terms on the OSGi site would need another legal review. Nice to know I'm keeping lawyers as well as customs people in jobs!

It's a funny thing since the JCP process (section 3.5) doesn't mention a legal review, although the spec. lead guide does. There's no state in the process for having failed a legal review after the FAB, so I guess the JSR will just stay in limbo until the lawyers are happy.

I pointed out that nothing has changed since the Final Approval Ballot, so let's hope the legal review is just a formality...

JSR 291 Final Release on its way

"But I thought JSR 291 had already gone final?", I hear you say. Nope: it passed the Final Approval Ballot a few weeks ago, but is still to enter Final Release state.

Since then I've been getting hold of an Export Classification Control Number (ECCN), which is really the only additional information needed by the JCP PMO to publish the Final Release. Having obtained an ECCN¹, I emailed the PMO today and await their response.

I should mention that the JCP bureaucracy needn't hold up anyone who wants to use JSR 291, as everything they need is already available.

[1] The ECCN is EAR99 for anyone who's interested, although I can't imagine who would be. EAR99 seems to be code for "this doesn't really need an ECCN". Keeps the customs people in jobs though.

IcedTea fork of OpenJDK

This news has been around for a few days, but I heard it first from the JavaPosse.

Red Hat has forked OpenJDK and plans to fill the encumbered gaps with GPL code in the newly formed IcedTea project (details here).

Eventually they hope to contribute the changes to OpenJDK, but what kind of contribution/governance model will they want? Will Red Hat happily sign the Sun Contributor Agreement? Meanwhile, will IcedTea qualify as an OpenJDK project and gain access to the JCK?

Lot's of fun to be had...

JSR 277 and JSR 294 observer lists

You may like to know there is a public observer list for JSR 277 and one for JSR 294 too, both with publicly visible archives.

This is not obvious from the JSR homepages.

OSGi community event

The OSGi community event in Munich is approaching fast. I can't make it, but I know there will be good speakers including BJ Hargrave, Peter Kriens, and Neil Bartlett. Register here.

Adrian Colyer on Spring & OSGi

Just dipped into one or two segments of Adrian Colyer's InfoQ video interview on Spring & OSGi. There are bookmarks for anyone else who doesn't want to hear the full half hour.

More OSGi tutorials

Neil Bartlett has published another excellent OSGi tutorial in his EclipseZone series, this time introducing Declarative Services.

A must for anyone learning OSGi, and I include myself since I'm not an application programmer.

He'll also be giving a talk at the OSGi Community Event in Munich later this month.

Module system interoperation kernel

Interoperation is currently a hot topic on the JSR 277 mailing list -- see the "Relationship to JSR 291" thread in the archive.

One of the approaches to interoperation is a 'kernel'. What follows is an extract from the README of the prototype plus the related class diagrams.

Goal

The goal is to enable JSR 277 to interoperate in a first class way with JSR 291 (OSGi). JSR 277 modules must be able to use JSR 291 modules and vice versa without significant restrictions.

The framework can probably enable other types of module systems to interoperate, but that isn't the primary goal.

Strawman

The code [of the prototype] is only a strawman. It is not integrated with Felix or any other OSGi implementation. Nor is it integrated with JSR 277.

However, it outlines the interfaces that are likely to be needed to enable such module systems to interoperate well and provides several JUnit 4.1 testcases which drive the framework to illustrate the framework's features.

The code is licensed under the Apache License v2.0. See LICENSE.txt.

Module systems and wiring

The lack of a standard method of building delegation networks between class loaders requires each module system to interoperate specifically with each other module system. The framework provides a standard way for module systems to create delegation paths, or 'wires', to modules in other module systems.

We need a module system registry or 'kernel' to keep track of the available ModuleSystems. Something like the following.

public interface Kernel { 
    boolean registerModuleSystem(String moduleSystem, ModuleSystem ms);
    boolean deregisterModuleSystem(String moduleSystem);
    ModuleSystem getModuleSystem(String moduleSystem);
    ModuleSystem[] getModuleSystems();
    ...
}

Module systems need to co-operate to resolve references to modules in other module systems. So each module system has an associated wire factory.

public interface WireFactory extends ModuleSystemRelated {
    Wire resolve(Dependency dep);
    Wire create(ResolutionContext ctx, Dependency dep);
    Wire createLocal(ResolutionContext ctx, Dependency dep);
    void complete(ResolutionContext ctx);
}

The most basic kind of dependency consists of a module name and a module version range. More sophisticated dependencies, such as package dependencies, are conceivable but omitted since they introduce much extra complexity.

A wire is a delegation path to the classloader of another module.

That leaves resolution contexts...

Resolution context

A resolution context is needed so that module systems can co-operate to resolve cyclic dependencies without ending up in an infinite loop. A resolution context also provides somewhere to anchor the state for sharing partially resolve modules and for back-tracking.

Constraint checking and back-tracking

A module system can anchor state off a resolution context so that, when asked more than once to create a wire for a given dependency, it can return each possibility in turn. To avoid interference between similar dependencies, new Dependency instances must be created for each module. If two modules have what appear to be the same dependency, then there need to be two Dependency instances.

Diagrams

The kernel class hierarchy shows the kernel with which module systems are
registered. Module systems and resolution contexts are each related to a
particular kernel (of which there is usually one per VM).
The module system class hierarchy shows wire factories, dependencies, and
wires which are each related to a particular module system.