SpringSource

A couple of weeks in the new job and I work for a different company! Well almost - Interface21 has been renamed to SpringSource to strengthen the association with the Spring portfolio of products.
New web site, new email addresses, new logo, new business cards, etc. On the other hand the job is unchanged, as is the green corporate colour scheme...

Life in a small company

I'm gradually noticing what's different about working in a small company. For one thing, there are very few processes, other than for software development where there are one or two.

For instance, I just took this photo. of the room I work in. Damilola (on the left) and Chris (on the right) noticed the idiot lifting his macbook into mid-air, but Nancie (in the distance) apparently didn't. I wonder how many levels of sign-off would have been needed to publish such a photo. in my previous company?

A more telling example of the minimal processes was that it recently took about an hour to obtain licenses for a piece of commercial software we decided to use. This included getting financial sign-off from someone on a business trip on the West Coast, submitting the order, and getting the license keys ready to use.

This is a pleasant change for me. Processes are sometimes essential, but it's so easy to make them too heavy. The emphasis in a small company seems to be talking to other people and trusting them to do the right thing.

Joining Interface21

Today I start work for Interface21 in Southampton. After more than a month of gardening leave, or more accurately decorating leave, I can't wait!

OSGi, type-safety, and the 'uses' directive

Java preserves type-safety when loading classes, even if there are buggy or malicious custom class loaders present. This is important as without type-safety, Java would not be secure.

For example, it would be possible to access an object's private data using a spoofed version of a class with the private data declared public. Vijay Saraswat found just such a type-safety bug in an early version of Java which was fixed by the introduction of 'loading constraints'.

So now that Java is type-safe, why need OSGi be concerned? Well, although OSGi can't break Java's type-safety, its support for versioning increases the risk that exceptions, such as ClassCastException, will be thrown to preserve type-safety. This usually occurs where two loaded versions of a particular class come into 'contact'.

The most common case is where an attempt is made to cast an object of one version of a class to another version of the same class. There isn't necessarily anything malicious going on - it's simply that two legitimate versions of a class loaded in the same JVM have accidentally come into contact.

OSGi increases the risk of such exceptions because it supports versioning of bundles and their contents: two (or more) versions of a given bundle can run in a JVM at the same time. These bundles will typically export similar classes. Since each OSGi bundle has its own class loader and a class is uniquely identified at runtime by the combination of its fully qualified class name and the class loader instance that defined the class, there can be two versions of each exported class present in the JVM at the same time.

So, to minimise these problems, OSGi works hard to wire together bundles so that they will consistently use a particular version of each class. At a basic level, when OSGi is resolving the dependencies of a bundle, it uses resolved bundles to satisfy those dependencies in preference to unresolved bundles. This avoids gratuitously introducing two or more versions of a given bundle, together with its exported classes, into the system.

But sometimes multiple versions of a class can, and indeed must, legitimately coexist. This can occur when assembling a large system from pieces which have been developed independently. The separate pieces can pre-req. distinct versions of common bundles. If these common bundles don't 'leak out' of the pieces of the system that use them, there is no problem.

But if they do leak out, OSGi's 'uses' directive enables the bundles importing from the common bundles to specify the amount of 'leakage'. For example, the following method signature exposes a common type PType:

package org.bar.q;
...
public org.foo.common.p.PType someMethod() {...}

and the following manifest statements record the leakage:

import-package: org.foo.common.p
export-package: org.bar.q,uses:="org.foo.common.p"

OSGi always resolves bundle dependencies so as to respect the 'uses' constraints. There's even support in Peter Kriens' bundle manifest creation tool, bnd, which spots the most straightforward leakages, like the example above, and create the corresponding 'uses' clauses automatically.

More subtle leakages need to be handled by the bundle programmer, but usually it is fairly obvious that something tricky is going on. For example, the following method take a parameter and casts it to the common type PType:

package org.bar.q;
...
public void anotherMethod(Object o) {
... (org.foo.common.p.PType)o ...
}

To avoid class cast exceptions due to o being an instance of the wrong version of PType, the same manifest statements as in the first example above should be specified, although these could not be inferred automatically by bnd.

So with appropriate 'uses' clauses, OSGi is able to manage multiple versions of bundles and minimise type-safety exceptions.

Writing good code

James Governor drew my attention to Laura Thomson's summary of good coding practice - like a memory jogger for Code Complete. Don't worry about the title if, like me, you have little interest in PHP.

Upcoming 'webinar' on Equinox OSGi

Tom Watson and Simon Kaegi are giving an introduction to Eclipse Equinox and OSGi at noon EST on November 27th. Register if you're interested.

I hadn't come across Eclipse live before, but the archive has some OSGi podcasts which may be of interest. I guess Tom and Simon's talk will appear there in due course.

Free Burma!

An estimated 200 people have been killed by the ruling military junta during the recent peaceful demonstrations in Burma. Although the country currently has a high profile in the news, Burma has a history of human rights abuses. More background may be found in the BBC's profile of Burma.

Today, October 4th 2007, thousands of bloggers worldwide are making a single post to raise awareness of the situation and press for change. If you have a blog, why not join in?

Leaving IBM

Today I handed in my resignation and will be leaving IBM on November 5. I've had a very enjoyable 26 years here in Hursley working on a variety of projects from workstation graphics algorithms to high-end transaction servers. A year ago I expected to work in IBM until retirement, but I had no idea what an interesting opportunity was about to open up a few miles down the road.

Although I'll still be involved with OSGi, I'll be handing over the job of representing IBM on the JSR 277 and 294 Expert Groups to someone else. I'll be lucky to find anything else quite so contentious to blog about, although I'll still be very interested in the outcome and may find something to say.

Hope for interoperation between JSR 277 and JSR 291

Infoq and Alex Miller picked up on my recent comments about the somewhat closed approach being adopted by JSR 277. Since the whole concept of interoperation between JSR 277 and JSR 291 seems to be doubted, I thought I'd better provide some reassurance.

The saving grace of both these JSRs is that they ultimately boil down to delegation networks between module class loaders. So there is a lingua franca lurking not very far under the surface. Granted the views presented to the module developer are quite different in detail.

Quite a while ago, I wrote a prototype module system interoperation framework, described in more detail here, to start to flesh out how these JSRs could interoperate. I regard it more of an existence proof that a reasonable solution may exist. However, it requires 'cutting' into the module systems at a deeper level than I think is currently being considered by the JSR 277 spec. lead and colleagues.

We discussed some of the options in an informal Expert Group meeting at JavaOne, but all we really concluded was that there was a lot of work ahead to get interoperation working smoothly. I think there is also a consensus among the interested parties that reasonable interoperation is essential to the success of JSR 277. So the sooner we can get more eyes, and brains, on the current proposal, the better.

FeatherCast

FeatherCast occasionally has interesting short podcasts about goings on in Apache.

Today I noticed a Google advert on the site for humane poultry slaughter tools, presumably targetting open source, free range chicken farmers.

How to run a JSR Expert Group

The modus operandum of the JSR 277 Expert Group is fascinating.

Strawman proposals are developed in private by the spec. lead, colleagues from Sun, and occasionally one or two others from elsewhere. These are then presented to the Expert Group for comment. After more or less debate, the spec. lead asserts the consensus and the strawman goes forward for inclusion in the draft spec. Typically, there can be a change of position when something small is clearly broken, but significant design changes and smaller tweaks tend to be resisted.

On the whole, the real JSR 277 contributors are working in private and the so-called Expert Group acts as a sounding board without any significant influence on the outcome. Some would call this rubber-stamping.

I'm trying to change this on one crucial front - that of interoperation between JSR 277 and JSR 291. But progress is dreadfully slow.

Apparently, interoperation prototypes are being developed and thinking is going on, but none of this is visible to the Expert Group or the broader community. This is a bit odd as a modules project has been set up on OpenJDK, so I would have expected that kind of prototyping to go on in a branch or subdirectory for early feedback from others.

It's worrying because the OSGi experts, Richard Hall and myself, in the JSR 277 Expert Group and the others in the JSR 291 Expert Group are not currently able to help. By the time we see a strawman, it may be too late to make any significant changes.

Clearly my experience of Apache and Eclipse is no guide to the way things happen in OpenJDK. Similarly, my experience of other standards bodies is also no guide to how a JSR Expert Group is necessarily run.

Decimal syntax

On this topic, Danny Coward recently referred to a limited form of operator overloading specifically for BigDecimal. Good to know this is still in Sun's thinking.

I'm continuing to look for users and companies who are willing to back the requirement for decimal syntax in Java, regardless of whether this is via limited operator overloading or decimal-specific support, so if anyone is interested, I'm all ears.

Is enjoyable standards work a contradiction in terms?

Eric Newcomer says he's enjoyed working in the OSGi Enterprise Expert Group. From my experience of the OSGi Core Platform Expert Group, which has a significant overlap with the EEG, I'd certainly agree.

The defining characteristic of the OSGi Alliance seems to be that the members are there with a fairly clear common purpose. Other standards groups I've been involved in - and I'm not just talking about the JCP - have tended to suffer from divergent objectives among the members.

Maybe the key is that OSGi provides a dynamic component model which is broadly applicable across small devices, PCs, and large servers and that it has strong conceptual integrity making it easy to learn and apply.

I also found Eric's recent InfoQ interview on OSGi futures interesting - take a look.

JSR 291 goes Final Release

The last administrative hurdle for JSR 291 has been cleared and the spec., RI, and TCK are now available, either via the JCP site or directly and, somewhat more simply, from the OSGi site.

The Expert Group however lives on...

JCP Awards

The JCP awards brightened up a wet Monday morning.

The Apache Software Foundation got the "Member of the Year" (!) award for its contributions and active participation in 2007.

JSR 308 was hailed as the most innovative JSR for Java SE/EE, although the other two contenders haven't issued a draft spec. yet either. A sign of things to come?

Not sure which JSR beat 291 in the "most contentious" category, but it could have been the abortive JSR 313.

But seriously, congratulations to "Participant of the Year" Wayne Carr who was a welcome voice of reason in the early days of JSR 277.

Java modularity: where do we go from here?

Things have moved on somewhat since March, so I thought an update would be helpful. Let's recap first.

JSR 277 is trying to create a static module system for Java. Its offshoot, JSR 294, is focused on the VM and language support for modularity. Both these JSRs are targeting Java 7.

JSR 291, which recently passed the Final Approval Ballot, references and extends the OSGi dynamic component system. JSR 291 is compatible with the Java ME JSR 232, which also references OSGi.

OSGi is a mature technology, having been developed in four releases over the last eight years. There are three independent open source implementations (Equinox, Felix, and Knopflerfish), a specification, reference implementation, and TCK.

I am an active member of the JSR 277 and JSR 294 Expert Groups and am the spec. lead of JSR 291.

Design goals

Crucially, the goals of these JSRs are rather different, although the underlying concepts are similar. JSRs 277 and 294 are building basic support into the Java SE platform whereas JSR 291 is pure Java built on top of the Java platform.

Look at it another way. JSRs 277 and 294 are attempting to produce an evolution of java.lang.ClassLoader with a new distribution and packaging format, a new repository, and new language features supported by the VM. JSR 291 on the other hand is implemented in pure Java on top of java.lang.ClassLoader and uses standard JAR files with standard manifests as a distribution and packaging format.

The design point of OSGi has always been to support a broad set of Java ME environments, such as profiles based on the Connected Device Configuration, and Java SE versions, from Java 1.2 onwards, in a compatible way. It has achieved this by building on commonly available Java concepts, notably class loaders.

I think of OSGi as being the pinnacle of custom class loader systems. The way it manages to install, uninstall, start, stop, and update modules without restarting the VM is quite an achievement - not my achievement, I should add, as the dynamic features were mostly complete before I got involved. Of course, other systems have provided some of this capability, but not in such a generally reusable way.

So you can package OSGi with your application, or list it as a prerequisite, and then run on a wide range of Java platforms.

On the other hand, JSRs 277 and 294 have the advantage that if you are happy to pre-req. Java 7 or later, then those JSRs will be built in to the platform.

Why doesn't JSR 277 reuse OSGi?

So why can't the JCP simply adopt JSR 291 as a standard module system for the Java platform? That's what many of the critics of JSR 277 are calling for.

A crucial issue is one of control. I hope the OSGi Alliance would allow OSGi to be included in the Java platform so long as they retained control of the technology. But I suspect Sun would be nervous about the core module system of the Java platform not being under their direct control. Such nervousness would be understandable if it was not in the interests of the OSGi Alliance to maintain the value of Java, but the reality is just opposite: the success of the OSGi Alliance depends to a large extent on the success of Java.

In my earlier blog, I made the point about the OSGi technology still evolving. But I think that problem is soluble. There would need to be a way of overriding the built-in module system with a later, backward compatible version.

Why not scrap JSR 277?

A radical option, favoured by some, of simply scrapping JSR 277 is less than ideal. It would temporarily remove the confusion that competing module systems inevitably create. But before long, the requirements for a module system built in to the Java platform would surface again.

The consumer JRE work has shown there is a real need to modularise the JRE itself and this requires a built-in module system. However, the consumer JRE doesn't seem to be proposing a properly architecture module system. It's a 'quick fix' which can be delivered in Java 6 update releases.

So if JSR 277 was scrapped, then whatever mechanisms evolve out of the consumer JRE work could become the defacto module system for the Java platform. The Java platform would still lack a properly architected built-in module system.

JSR 277 also has a stated objective of interoperating with JSR 291, which means JSR 277 modules will be able to make effective use of OSGi modules and vice versa with minimal restrictions. If this objective is achieved, it will protect the significant investment in OSGi modules in the Java community. Scrapping JSR 277 would jeopardise this investment if a non-interoperating module system grew up in place of JSR 277.

The road ahead

So, given the momentum behind JSR 277 and the emphasis placed on it by Sun at this year's JavaOne, what lies ahead?

Of course, JSR 291 should ideally benefit from the features introduced by JSR 277 and JSR 294. But the immediate requirement is that JSR 277 and JSR 291 interoperate well so that modules written in terms of one JSR are able to make effective use of modules written in terms of the other JSR.

To achieve this, the Expert Groups of JSR 277 and JSR 291 will need to work together. The JSR 277 Expert Group are responsible for producing an interoperation proposal which the JSR 291 Expert Group will review. At the time of writing, the interoperation proposal is still being worked on by the JSR 277 spec. lead.


The dream solution

The dream solution is clear: JSR 277 should adopt JSR 291, underpinned by language and VM support in JSR 294, and add the JSR 277 repository architecture. This would accelerate the progress of JSR 277 by several years and guarantee perfect interoperation between JSR 277 and JSR 291.

But it would take a complete about turn by Sun to turn this dream into reality. Maybe, just maybe, Sun are sufficiently committed to the success of Java that they will eventually decide to do just that. Maybe, just maybe, the Java community will be able to look back later and paraphrase Winston Churchill: "Sun could always be counted on to do the right thing, after having exhausted all the alternatives".

Further information

The JSR 291 Expert Group mailing list and the JSR 294 list have been openly readable since their Expert Groups started. JSR 277 ran in private for many months, but has now opened up its list.

Disclaimer

I'm biassed.

I have been involved in the OSGi Core Platform Expert Group for a couple of years during which time I co-authored some of the modularity improvements for OSGi R4 with Richard Hall (the leader of the Apache Felix OSGi project).

I work for IBM which belongs to the OSGi Alliance and which has based some of its products on OSGi. IBM and Sun co-operate on Java, but compete in related areas such as Java EE, not to mention UNIX-style operating systems and hardware. So IBM and Sun don't necessarily agree on everything that happens in Java. In fact, I think it would be pretty unhealthy if that were the case.

I should also re-iterate that my opinions are not necessarily shared by IBM.

That said, I hope this blog objectively lays out the situation.

New development models for IBM

IBM launched Project Zero on Friday. The aim is to provide simpler ways to build web applications using an approach termed Community-Driven Commercial Development.

The whole idea is to engage with the community of potential users to provide something that they want to use: similar to an open source project, but with a commercial license. There is a big emphasis on 'consumability' these days in IBM and this is an example of IBM changing its ways to produce more usable software.

Another new initiative, Jazz, is in contrast only accessible to IBM Rational's customers and partners plus invited others. I heard about it on a DeveloperWorks interview with Erich Gamma, of design patterns and Eclipse fame, but haven't had time to watch the videos. Anything Erich is endorsing is bound to be pretty interesting. If anyone has a link to a white paper or some such, please let me know.

JSR 291 but not as we know it

Google sometimes coughs up unexpected stuff, like this which could almost be talking about JSR 291 and JSR 277. :-)

Eclipse Europa

Coté's coverage of Eclipse Europa expresses a view on OSGi and JSR 277.

Representations of module dependencies

One of the key differences between JSR 291 and JSR 277 is the way in which module dependencies are represented, satisfied, and managed. Both module systems express dependencies using import declarations with version ranges and so, in general, there can be multiple ways of satisfying any particular dependency.

Using an electronics analogy, there can be multiple different ways of wiring a module's dependencies to other modules. Without further constraints, the choice of wiring is essentially non-deterministic, which would make the module system hard to understand and unpredictable in behaviour. The non-determinism needs to be removed.

JSR 291 removes the non-determinism by specifying rules for how dependencies should be satisfied in the presence of a given collection of modules, some of which are already resolved (or 'wired') and others which are unresolved (not yet 'wired').

JSR 277 also specifies some rules to remove the non-determinism, but it allows a module to implement an import policy if the module wants to override these rules. The hope is that, in many simple cases, the import policy will not be needed. But, in general, JSR 277 removes the non-determinism using a combination of rules in the specification and user-written import policies.

Why is this difference important? Well, an import policy is a piece of Java code running inside the module whose dependencies need to be satisfied. So the import policy is in the unusual position of not being able to use the modules upon which its module depends. This is a potential source of errors.

But a more important difference relates to the need to be able predict the behaviour of a collection of modules. This is crucial when managing the dependencies of modules. With JSR 291, an external management system is able to read the dependency declarations in each module and apply the rules in the specification to determine how these modules will be wired up, whether there are any missing dependencies, and, if there are, how such dependencies could be satisfied.

With JSR 277 the position is quite different when import policies are used. The only way to determine the behaviour of an import policy is to execute it. But even then, there is no guarantee that the import policy will give the same result each time it runs. Also, if there are missing dependencies, it is not feasible to examine the import policy to determine how the missing dependencies could be satisfied.

(For further reading, I recommend Peter Kriens' JSR 277 review from last October.)

Module-superpackage relationship

In the comparison of JSR 277 and JSR 291 features, I alluded to the fact that the module-superpackage relationship is a difference, but Andreas pointed out another way in which the relationship differs.

My original observation was that the cardinality of the relationship is different. JSR 277 specifies modules which build directly on superpackages: there is a one-one relationship between deployment modules and superpackages.

JSR 291, on the other hand, makes no explicit reference to superpackages as these are yet to find their way officially into Java. But, if JSR 294 continues in its current direction, it will be possible to include one or more superpackages inside a JSR 291 module. So the relationship between JSR 291 modules and superpackages will have 1:0..n cardinality.

Andreas' point was that JSR 277 is integrated with JSR 294 in that a JSR 277 module's metadata consists of a superpackage and its annotations.

JSR 291, in contrast, is currently agnostic about superpackages and their names, members, exports, and annotations. JSR 291 maintains its own metadata in a separate manifest file and so is not integrated with JSR 294.

Sun VP 'gets' OSGi

I just ran across a video where James Governor interviews Jeet Kaul, the VP of developer products and programmes for Sun Microsystems. If you don't have much time, fast forward to three minutes from the start and listen to the next minute and a half on OSGi.

Some soundbytes plucked slightly out of context for maximum effect:

JG: "I'm looking at OSGi ... and I think it's possibly the most important standard in the industry right now." (What a nice man.)

JK: "... so much has been done already that it would be insane to ignore that ..."

and later

JK: "... in Java SE 7 where we are trying to build the standard which will incorporate supporting OSGi bundles - it's the right approach."

Default export granularity

In the comparison of JSR 277 and JSR 291 features, I caused some confusion by saying the default export granularity was 'class' for JSR 277 and 'package' for JSR 291. I'll try to explain.

The primitive unit of export in JSR 277 is the class. You can use a wildcard to export all of a module's classes in a specified package, but that's just syntactic sugar. The wildcard will be expanded to a list of classes in the binary file representing the module's superpackage. There is no runtime notion of an exported package.

The primitive unit of export in JSR 291, on the other hand, is the package. You can filter the set of classes available to importers, but the runtime deals in exported packages rather than classes.

I hope that's a bit clearer. Also, try not to confuse default export granularity with import semantics. JSR 277 supports only module import whereas JSR 291 supports package and module imports.

Comparison of JSR 277 and JSR 291 features

A number of people have expressed an interest in the similaries and differences between these JSRs, so this post compares the features of the Early Draft Review specification of JSR 277 and the Final Specification of JSR 291, although you'll have to read the specifications to understand the differences in detail.

This a functional comparison and doesn't attempt to draw out advantages and disadvantages of various features or other aspects of the JSRs. For ease of comparison, I've referred to JSR 291 bundles as modules.

Equivalent features:

• Module name
• Module version
• Module attributes
  
• Module import
• Module re-export
  
• Version range
• Optional import
• Executable modules
• One class loader per module
• Platform dependencies
• Modules may contain classes, native libraries, resources, and extensible metadata
  
• Module signing
• JAR distribution format
• Reflective API
  

Differences (order: JSR 277 vs JSR 291):

• Programmatic import policy vs declarative import constraints (details)
  
• Class space consistency: validation vs constraint solving
  
• Default export granularity: class vs package (details)
  
• Module-superpackage relationship (details)
  

Unique to JSR 277:

• Standard repository:
• organised hierarchically with parental delegation
• multiple standard implementations for local or networked deployment
• support for plugging in 3rd party repository implementations
• JAM distribution format
• Import override policy
  
• Part of the Java SE platform (planned for Java 7)
  

Unique to JSR 291:

• Dynamic lifecycle support both for module lifecycle and service lifecycle
• Service support (above and beyond the basic service provider support in Java SE)
  
• Package import
• Dynamic package import
  
• Package attributes
• Mandatory attributes
• Split package support
• Module classpath
  
• Fragment modules
• Extension modules
• Pure Java implemented on top of Java SE
• Compatible with Java ME (JSR 232)
  

One last hurdle for JSR 291?

The PMO acknowledged receipt of the Final Release materials I sent them a couple of days ago, so at least their email system worked this time. However, they said the spec. license and the final business terms on the OSGi site would need another legal review. Nice to know I'm keeping lawyers as well as customs people in jobs!

It's a funny thing since the JCP process (section 3.5) doesn't mention a legal review, although the spec. lead guide does. There's no state in the process for having failed a legal review after the FAB, so I guess the JSR will just stay in limbo until the lawyers are happy.

I pointed out that nothing has changed since the Final Approval Ballot, so let's hope the legal review is just a formality...

JSR 291 Final Release on its way

"But I thought JSR 291 had already gone final?", I hear you say. Nope: it passed the Final Approval Ballot a few weeks ago, but is still to enter Final Release state.

Since then I've been getting hold of an Export Classification Control Number (ECCN), which is really the only additional information needed by the JCP PMO to publish the Final Release. Having obtained an ECCN¹, I emailed the PMO today and await their response.

I should mention that the JCP bureaucracy needn't hold up anyone who wants to use JSR 291, as everything they need is already available.

[1] The ECCN is EAR99 for anyone who's interested, although I can't imagine who would be. EAR99 seems to be code for "this doesn't really need an ECCN". Keeps the customs people in jobs though.

IcedTea fork of OpenJDK

This news has been around for a few days, but I heard it first from the JavaPosse.

Red Hat has forked OpenJDK and plans to fill the encumbered gaps with GPL code in the newly formed IcedTea project (details here).

Eventually they hope to contribute the changes to OpenJDK, but what kind of contribution/governance model will they want? Will Red Hat happily sign the Sun Contributor Agreement? Meanwhile, will IcedTea qualify as an OpenJDK project and gain access to the JCK?

Lot's of fun to be had...

JSR 277 and JSR 294 observer lists

You may like to know there is a public observer list for JSR 277 and one for JSR 294 too, both with publicly visible archives.

This is not obvious from the JSR homepages.

OSGi community event

The OSGi community event in Munich is approaching fast. I can't make it, but I know there will be good speakers including BJ Hargrave, Peter Kriens, and Neil Bartlett. Register here.

Adrian Colyer on Spring & OSGi

Just dipped into one or two segments of Adrian Colyer's InfoQ video interview on Spring & OSGi. There are bookmarks for anyone else who doesn't want to hear the full half hour.

More OSGi tutorials

Neil Bartlett has published another excellent OSGi tutorial in his EclipseZone series, this time introducing Declarative Services.

A must for anyone learning OSGi, and I include myself since I'm not an application programmer.

He'll also be giving a talk at the OSGi Community Event in Munich later this month.

Module system interoperation kernel

Interoperation is currently a hot topic on the JSR 277 mailing list -- see the "Relationship to JSR 291" thread in the archive.

One of the approaches to interoperation is a 'kernel'. What follows is an extract from the README of the prototype plus the related class diagrams.

Goal

The goal is to enable JSR 277 to interoperate in a first class way with JSR 291 (OSGi). JSR 277 modules must be able to use JSR 291 modules and vice versa without significant restrictions.

The framework can probably enable other types of module systems to interoperate, but that isn't the primary goal.

Strawman

The code [of the prototype] is only a strawman. It is not integrated with Felix or any other OSGi implementation. Nor is it integrated with JSR 277.

However, it outlines the interfaces that are likely to be needed to enable such module systems to interoperate well and provides several JUnit 4.1 testcases which drive the framework to illustrate the framework's features.

The code is licensed under the Apache License v2.0. See LICENSE.txt.

Module systems and wiring

The lack of a standard method of building delegation networks between class loaders requires each module system to interoperate specifically with each other module system. The framework provides a standard way for module systems to create delegation paths, or 'wires', to modules in other module systems.

We need a module system registry or 'kernel' to keep track of the available ModuleSystems. Something like the following.

public interface Kernel { 
    boolean registerModuleSystem(String moduleSystem, ModuleSystem ms);
    boolean deregisterModuleSystem(String moduleSystem);
    ModuleSystem getModuleSystem(String moduleSystem);
    ModuleSystem[] getModuleSystems();
    ...
}

Module systems need to co-operate to resolve references to modules in other module systems. So each module system has an associated wire factory.

public interface WireFactory extends ModuleSystemRelated {
    Wire resolve(Dependency dep);
    Wire create(ResolutionContext ctx, Dependency dep);
    Wire createLocal(ResolutionContext ctx, Dependency dep);
    void complete(ResolutionContext ctx);
}

The most basic kind of dependency consists of a module name and a module version range. More sophisticated dependencies, such as package dependencies, are conceivable but omitted since they introduce much extra complexity.

A wire is a delegation path to the classloader of another module.

That leaves resolution contexts...

Resolution context

A resolution context is needed so that module systems can co-operate to resolve cyclic dependencies without ending up in an infinite loop. A resolution context also provides somewhere to anchor the state for sharing partially resolve modules and for back-tracking.

Constraint checking and back-tracking

A module system can anchor state off a resolution context so that, when asked more than once to create a wire for a given dependency, it can return each possibility in turn. To avoid interference between similar dependencies, new Dependency instances must be created for each module. If two modules have what appear to be the same dependency, then there need to be two Dependency instances.

Diagrams

The kernel class hierarchy shows the kernel with which module systems are
registered. Module systems and resolution contexts are each related to a
particular kernel (of which there is usually one per VM).
The module system class hierarchy shows wire factories, dependencies, and
wires which are each related to a particular module system.

Modular Swing Apps

Thanks to Colm Smyth for pointing out Jasper Potts' presentation from JavaOne on modularising Swing applications using a Spring-OSGi derivative known as SPAR. Nice to see OSGi being used by Sun.

OSGi the most important technology of the decade?

It may have been for me, but read the SD Times' view in OSGi Alliance Reaching Universal Middleware Goal.

JSR 291 Final Approval

JSR 291 just passed the Final Approval Ballot with twelve "yes" votes and two "no" votes.

Fourteen out of the sixteen Executive Committee members voted. The majority increased to ten compared to eight at the earlier ballots.

The positive comments praised the open approach of JSR 291, the suitability of the licenses, and the effectiveness of JSR 291 in influencing the other modularity JSRs. One comment urged JSR 277 and JSR 291 to become integrated, which will be achieved if JSR 277 delivers truly first class interoperation with JSR 291.

Voting comments from the detractors predictably concerned the work of the Expert Group rather than the underlying OSGi technology. They don't seem to like the JCP building on the work of other Java standards bodies. The concerns are discussed on the Expert Group mailing list and in an earlier blog.

I would like to express my thanks to many people who have contributed to the success of this JSR. They are too numerous to mention by name but include the JSR 291 Expert Group, the OSGi Alliance's Core Platform Expert Group, the Eclipse Equinox project, and numerous colleagues in IBM and elsewhere.

Designing module system interoperation

The various approaches to interoperation between JSR 277 and JSR 291 (OSGi) raise some interesting design considerations. The crucial distinction between these JSRs is that JSR 277 aims to be part of the Java 7 platform whereas JSR 291 is implemented as pure Java that will run on more or less any Java platform.

The JSR 277 Expert Group, or more accurately the spec. lead, is exploring the option of making some components such as the repository, module definition, and module instance pluggable and then having JSR 291 provide alternative implementations. This requires JSR 291 to surface its modules as JSR 277 modules, which will either require some generalisation of the notion of a JSR 277 module or a force fit. A force fit would probably result in unacceptable restrictions on interoperation and should be avoided.

A more ambitious approach, favoured by several Expert Group members including myself, is to rework JSR 277 to provide a 'custom module system' API analogous in concept to the way custom class loaders can be built on top of the existing class loader API. In this scheme, the JSR 277 module system would simply be the default module system packaged with the Java 7 platform and JSR 291 could implement its module system on top of the API. The challenge in this approach would be to design the API so that multiple module systems implemented to the API would naturally interoperate. A benefit of this design is that it would allow module system implementations to evolve faster than the underlying platform - something that JSR 277 doesn't yet enable its own module system to do.

A third approach, which I prototyped to prove that reasonable interoperation is achievable, involves a module system interoperation 'kernel' with which multiple module systems may register. Each module system provides a class, known as a 'wire factory' , for creating delegation paths ('wires') to its modules' class loaders.

One benefit of both the custom module system API and of the interoperation kernel is that they do not complicate the API that the module developer has to deal with. The 'plugin' approach requires generalisations of the concepts of repository, module, and module definition to be exposed on the module developer API. Although this kind of generalisation may actually improve the API, it could also end up making it more obscure. We'll have to wait and see.

Open Source Java at JavaOne

Much was made of OpenJDK at last week's JavaOne conference. On Tuesday morning, Rich Green delighted in telling the assembled masses "We are now, as regards the open-sourcing of Java, done" and then posted the following:

From: Rich Green  
To: announce@openjdk.java.net
Subject: Open JDK is here!

Today begins the next phase for Java. I am pleased to  announce that  
we have completed the open source release of Open JDK.
It's a great day for innovation - and remember, compatibility matters!
Rich

It turns out there is a bit more to do, like replacing some of the 'encumbered' code with a GPL version. Currently, the 6.5 MLOCs of OpenJDK source code must be built with some binaries which provide the encumbered code. Also OpenJDK doesn't yet build on Windows.

Sun have set up an Interim Governing Board to create a constituion for OpenJDK and to set up a proper Governing Board. The IGB has the following initial membership: Doug Lea, Fabiane Nardon, Dalibor Topić, and Sun's own Mark Reinhold and Simon Phipps.

The Apache Harmony technical session was well attended with more focus on the technology than on licensing issues and the JCK open letter.

Some anomalies still need working out. Sun said at JavaOne that it would release the JCK to the open source community, but later clarified that this meant the OpenJDK community. I guess that might even exclude forks of OpenJDK, but we'll have to wait and see.

Also, there is some strangeness in the way copyright assignment will be handled for eventual non-Sun committers to OpenJDK. The GNU definition of free software describes four freedoms including "The freedom to improve the program, and release your improvements to the public, so that the whole community benefits". So, in normal GPL projects, no-one has the right the make a fork without publishing their changes under the GPL.

Contrast this with the Apache license which allows anyone to create a fork and essentially do what they like with their changes, either publishing them or keeping them private.

OpenJDK is different from both these models. Sun, and only Sun, has the right to fork the code, make changes, and keep the changes private, although it seems unlikely that Sun would ever want to do that.

At this point when the source code is Sun's donation and when Sun is obliged to provide the source code to its licensees under the TLDA license, this is quite reasonable. But I wonder how many potential contributors will be happy to sign the Sun Contributor Agreement and give Sun these extended rights over their contributions.

Modularity at JavaOne

Modularity had a lot of focus at last week's JavaOne conference. There were multiple references to JSRs 277 and 294 by Sun speakers, specific technical sessions on these JSRs, and a combined JSR 277 and 294 BOF.

There were a number of questions and comments at the JSR 294 session on the strawman's structuring of superpackage membership information in a separate source file rather than distributed across the source files of the member classes. This is something we should discuss on the mailing list (which you can read via the observer list).

Apart from an OSGi Best Practices technical session and the JSR 291 BOF, there were several mentions of OSGi and JSR 291 in other sessions. The goal of interoperating well with JSR 291 was presented in the JSR 277 session and even mentioned in one of Sun's general sessions¹. Stanley Ho will float a proposal on the JSR 277 mailing list in due course, so subscribe to the observer list if you are interested (Expert Group discussions should begin to be mirrored there soon).

Ethan Nicholas gave an update on the browser edition work to minimise the download and initial install size of the JRE. This seems to be making good progress, although it is not based on any of the above JSRs, so there may be some 'shake-out' in due course.

In one of the "meet the team" sessions, I asked Sun's class library team what their plan was for exploiting JSR 277 and JSR 294 in the JRE. It seems they don't yet have one. I expect this to change, particularly after the JSR 277 and JSR 294 prototypes become available in a subproject of OpenJDK.

In contrast, the Apache Harmony session described how OSGi had been used from the start to modularise the class libraries. I gave a demo. of prototype JVM support for modular Harmony in the JSR 291 BOF.

GlassFish v3 is developing a module system known as HK2 ("Hundred Kilobytes Kernel") which is loosely based on JSR 277. It also has a dependency injection framework similar to Guice. (Unlike GlassFish which is mostly licensed under the CDDL, HK2 currently has no license defined, so you may want to avoid looking at the source code until there is a proper license.)

I had face to face meetings with Andreas Sterbenz to discuss aspects of JSR 294 and with Stanley Ho and Michał Cierniak to discuss interoperation of JSR 277 with JSR 291. There's a lot of work ahead and we won't know for quite a while how good the interoperation with JSR 291 will really be, but everyone concerned seems to be well aware of the importance of this capability.

1. See a couple of minutes from Intro. and Technical Deep Dive starting at 18 minutes and 45 seconds from the start when Danny Coward talks about modularity and refers to the goal of interoperation with OSGi.

JSR 291 Final Approval Ballot starts

The FAB for JSR 291 began today (Tuesday 8 May) - just in time for tomorrow's JSR 291 BOF session at JavaOne.

The Eve of JavaOne 2007

This week's JavaOne should be an interesting experience as it's my first.

After checking in this afternoon, I thought I'd sample James Gosling's session at the end of the Netbeans day. The content/time ratio was rather low, so I went for a walk in the daylight instead.

Perusing the pocket schedule in a Mexican restaurant later, I was pleased to see the JSR 291 BOF had made it into print, although it was too late for the IBM flyer. It seems like the BOF is competing with the tail end of the Eclipse party and Oracle's free screening of Spiderman 3. I'll be lucky if my co-presenter turns up (only kidding Richard).

The online session scheduler (JSP) seems to be struggling to stay up. How many thousands of people are hammering it? It finally let me switch a session.

Well, even blogging can't keep me awake. It's 9pm and 22.5 hours since I woke up. Better get to bed - tomorrow looks like a long day with sessions more or less continuously from 8.30am to 10.50pm. Boy do these American's take work seriously...

Harmony, Guice, and OSGi

I've recently been tinkering with Google's dependency injection framework, Guice, and getting a simple example to run under the Equinox OSGi framework. It was straightforward except that there are a couple of extra dependencies (i.e. that you couldn't infer from the source code of the example) that need to be expressed as imported/exported packages because of a code generation library, cglib, that Guice uses.

This morning, I thought I'd try running the example using the Apache Harmony JDK 5.0M1 and it ran cleanly first time. I'm very impressed since Guice makes extensive use of generics and annotations and cglib is hardly typical application code.

OSGi for application modularity

InfoQ published an article on why one company has decided to use OSGi, in conjunction with Spring, for application modularity.

They are not starting from scratch, so it will be interesting to see how pleased they are with the result. On the face of it, OSGi is a good match to their requirements, particularly for versioning and for deploying application updates without restarting the server.

OSGi Community Event

JSR 291 BOF at JavaOne 2007

After reconsideration, a JSR 291 BOF¹ has been accepted for JavaOne 2007. The plan is for Richard Hall and I to lead the session.

I hope to include some thoughts on JSR 277 interoperation as well as JSR 294 (superpackages) exploitation.

1. BOF-21240 entitled "JSR 291: Dynamic Component Support For The Java SE Platform", Wednesday 9 May, 9:55pm-10:45pm

More OSGi assistance for Maven users

A former colleague, Stuart McCulloch, just released some tools for building an OSGi application using Maven.

Apparently it uses bnd under the covers, although the example provided seems to need quite a bit spelling out by hand. Since I'm only a neophite Maven user, it's hard to tell...

Jeff McAffer's blog

Jeff is a colleague from Eclipse who often has an interesting perspective. You may be interested in his new blog or the first post on provisioning changes in Eclipse.

Welcome to the blogosphere, Jeff!

JavaOne, OSGi, and Emacs

According to the advance publicity, there seems to be one session¹ at this year's JavaOne conference devoted to OSGi and only a couple of others that refer to OSGi. Not a single mention of JSR 291, although I did try putting in a submission.

I noticed an interesting tidbit in the conference guide: "[James Gosling] has also built a WYSIWYG text editor; a constraint-based drawing editor; and a text editor called Emacs, for UNIX systems."

So did Gosling invent Emacs as well as Java? Well, no, he didn't. The truth is rather more interesting.

Richard Stallman wrote the first version of Emacs and Gosling produced a C version to run on UNIX. A twist is that a legal tussle over the Emacs source code provoked Stallman to write the first precursor of the GPL. Stallman has since endorsed Sun's use of the GPL for their OpenJDK project, so I guess he and Gosling are friends again.

1. TS-1419 "Best OSGi Practices" on Thursday May 10 1:30 PM - 2:30 PM

Spring-OSGi Milestone 1

Thanks to Hal for the link to Costin's blog on Spring-OSGi which recently shipped the first milestone build.

Now I'm on the look out for the first application to take advantage of Spring-OSGi. Any takers?

JSR 294 Expert Group

The JSR 294 Expert Group now has a publicly visible mailing list which is good timing as the Expert Group only kicked off a couple of weeks ago, so there'll be a complete archive of its deliberations.

It's a little disappointing that the newly revamped jcp.org site doesn't appear to be capable of hosting openly readable mailing lists as I would like that to be the default for all JSR Expert Groups.

SIP Communicator and OSGi

Nice to see another project successfully using OSGi in the form of SIP Communicator over on java.net. It was good to see they upgraded from Oscar to Felix painlessly.

They are even participating in the Google "Summer of Code™", so more developers will see how useful OSGi is. Thanks to Greg Wilson for the link.

JSR 291 Status

There is some confusion about the effects on JSR 291 of jcp.org being in a mess. In particular the Final Approval Ballot won't be happening for at least a week or two.

The Proposed Final Draft spec. has been posted, but is inaccessible as far as I can tell. Fortunately, this is unlikely to matter. It's a pretty mature spec. which has had a few careful modifications due to JSR 291. The spec. is already undergoing a 45 day review by OSGi members. Also JSR 291 received no comments during the Public Review draft of the specification and only non-technical comments during the PR ballot.

So what's happening with the Final Approval Ballot? I submitted the FAB materials, but the email bounced after a couple of days so I just resent it. The spec. license needs a review by Sun lawyers because we used an IBM license rather than a standard Sun license, which will cause an additional unknown delay. After that, the FAB will begin. Hopefully jcp.org will be stable in time...

GlassFish Modularisation

Let's keep an eye on Glassfish and see what we can learn as they select a modularisation technology.

JSR 291 Proposed Final Draft

The PFD of JSR 291 has appeared on jcp.org, but the whole JCP site is broken at the moment due to some substantial changes that went in recently.

Further interest in OSGi and SCA

Yesterday the Open SOA Collaboration announced they are planning to standardise the SCA and SDO specifications via OASIS.

A couple of the participants, IONA and Siemens AG, mentioned their interest in using OSGi with SCA. This is consistent with the OSGi Alliance's Enterprise Expert Group track on SCA and OSGi which IONA's Eric Newcomer mentioned back in January.

Background to Java modularity

Should Java have included modularity features from the start or is the kind of modularity really needed in Java rather different from that baked in to some other languages?

In 2003, IBM Research published a paper known internally as "modjava" but named "MJ" externally. The authors' approach was to create a static module system not dissimilar to the strawman for JSR 294 except that MJ was built from class loaders for ease of prototyping and didn't encompass versioning (although a couple of us refactored the internal MJ prototype to enable versioning).

Vernon Green and I then led a working group in IBM's Software Group Architecture Board on Java versioning and modularity which started from the MJ paper. Later we discovered OSGi and how that matched in many respects what the working group was looking for. I was given the task of taking our additional requirements into the OSGi R4 standards activity where I struck up a collaboration with Richard Hall who had a similar perspective based on his research work on the Oscar OSGi framework.

The real world requirements of versioning and dynamic modification of applications and supporting software without restarting the JVM have effectively forced Java to develop the new, and highly successful, OSGi technology.

It's interesting to speculate what would have happened if a static module system had been put into Java much earlier. My guess is that it wouldn't have bothered to address versioning or dynamicity requirements which only become crucial in the context of relatively large systems with continuous operation.

The JSR 294 Expert Group is just starting up, so this background suddenly becomes relevant...

JSR 291 Final Approval Ballot materials

As I mentioned to the JSR 291 Expert Group, I just submitted the specification, reference implementation, and technology compatibility kit for JSR 291 for the Final Approval Ballot by the Java SE Executive Committee.

Modularity and the Java platform

Neil questioned the point of putting modularity support into the Java platform.

Of course, OSGi does a pretty good job of modularity support based on java.lang.ClassLoader. So why isn't that sufficient?

At one level, OSGi is quite sufficient. There is already massive adoption of OSGi and this is continuing to spread into new and interesting areas.

At a more theoretical level, there are some things that custom class loader networks will never achieve.

Most importantly is VM enforcement of module boundaries. It is possible to obtain unexported Class instances from a class loader. OSGi bundles are implemented as class loaders. Similarly, JSR 277 modules will boil down to specialised class loaders. So this is where JSR 294 comes in with its objective of enforcing module boundaries in the VM.

This extra level of enforcement is irrelevant if the aim of using modules is to divide a piece of software into manageable units with well-defined interfaces.

However, VM enforcement starts to become interesting if modules need to protect some parts of an application or the system from inappropriate access. There is also some scope for additional, or at least simplified, performance optimisations if certain classes can be guaranteed never to be accessed outside their defining module.

I would dearly like JSR 294 to provide mechanisms that will be equally useful to OSGi as to JSR 277. I think OSGi would benefit from VM support, although it would be an exaggeration to say that OSGi really needs VM support.

JSR 277 and class loaders

Neil wondered what changes are likely to be made to java.lang.ClassLoader in support of JSR 277.

The Early Draft spec. lists some requirements for class loader behaviour in section 8.3.4. These boil down to class loaders being able to cope with more concurrency, especially in the presence of cyclic dependencies between JSR 277 modules.

Currently the deadlock exhibited by Sun's reference implementation has an experimental fix activated by a switch. There is some work in the pipeline to improve this situation for Java 7 which should also benefit OSGi at least when running on Sun's RI. Keep an eye on the Hotspot VM on OpenJDK if you are interested.

The state of Java modularity: JSRs 277, 291, and 294

People are pretty perceptive. They have picked up that there's a struggle to work out the right model for Java modularity. However, it's not a simple choice between JSR 277 and JSR 291 as many seem to think. Let's look at what's going on in a bit more depth.

Background

Firstly, a brief recap for the uninitiated.

JSR 277 is trying to create a static module system for Java 7. JSR 291 is referencing and extending the already mature OSGi dynamic component system while maintaining compatibility with JSR 232 which did a similar thing for Java ME. JSR 294, an offshoot of JSR 277, is focused on the VM and language support for modularity.

My personal involvement is that I belong to the JSR 277 Expert Group (but cannot disclose the discussions which are private to the group) and am the spec. lead of JSR 291 (which has an openly readable mailing list). I am also in the JSR 294 Expert Group, which hasn't started work yet.

Secondly, I want to be up front about my potential bias.

I have been involved in the OSGi Core Platform Expert Group for a couple of years during which time I co-authored some of the modularity improvements for OSGi R4 with Richard Hall (the leader of the Apache Felix OSGi project).

I work for IBM which belongs to the OSGi Alliance and which has based some of its products on OSGi. IBM and Sun co-operate on Java, but compete in related areas such as Java EE, not to mention UNIX-style operating systems and hardware. So IBM and Sun don't necessarily agree on everything that happens in Java. In fact, I think it would be pretty unhealthy if that were the case.

I should also re-iterate that my opinions are not necessarily shared by IBM.

That said, I'll try to lay out what's going on as objectively as I can.

Design points

The crucial thing to notice is that the goals of these JSRs are rather different even though the underlying technology is similar. JSRs 277 and 294 are building basic support into the Java SE platform whereas JSR 291 is pure Java built on top of the Java platform.

Looking at it another way, JSRs 277 and 294 are attempting to produce an evolution of java.lang.ClassLoader with a new distribution and packaging format, a new repository, plus new language features supported by the VM. JSR 291 on the other hand is implemented in pure Java on top of java.lang.ClassLoader and uses standard JAR files with standard manifests as a distribution and packaging format.

The design point of OSGi has always been to support a broad set of Java ME environments, such as profiles based on the Connected Device Configuration, and Java SE versions, from Java 1.2 onwards, in a compatible way. It has achieved this by building on commonly available Java concepts, notably class loaders. I think of OSGi as being the pinnacle of custom class loader systems. The way it manages to install, uninstall, start, stop, and update modules without restarting the VM is quite an achievement. Of course, some other systems have provided some of this capability, but I am not aware of any that have done it in such a generally reusable way.

So you can package OSGi with your application, or list it as a prerequisite, and then run on a wide range of Java platforms. JSR 277 on the other hand has the advantage that if you are happy to pre-req. Java 7 or later, then JSR 277 will be built in to the platform.

Not invented here?

So why can't the JCP simply adopt JSR 291 as a standard module system for Java? That's what many of the critics of JSR 277 suggest should happen.

One problem is that the OSGi specification, although mature, is still evolving. A number of new requirements were surfaced by the JSR 291 Expert Group. OSGi is rapidly spreading into new areas of application which are bound to result in additional requirements for OSGi R5 and beyond. To lock a particular Java SE version, such as Java 7, to a particular version of OSGi would be limiting as there wouldn't be an opportunity to upgrade the OSGi support significantly before Java 8. For example, an application that required some new features of OSGi wouldn't necessarily want to limit its deployment to Java 8 environments. So the ability to mix and match Java and OSGi versions as required is essential.

You may be wondering whether JSR 277 has the same problem, but since it is deliberately positioned to be the module system packaged with the Java 7 platform, its subsequent rate of evolution is, by definition, the same as that of Java. So the design point is that it has to be fairly basic in its support and not aim for the level of sophistication of OSGi, particularly in its support for legacy applications, which is one of OSGi's strong points.

The challenge for JSR 277 is to address its requirements in a sufficiently general way that it does not need to evolve quickly. The classic way of achieving this would be to divide the function into a lower level and relatively stable API and a user overridable piece implemented in terms of the lower level API. Like a module version of custom class loaders. JSR 277, as you can see from the Early Draft, is not adopting this approach but is attempting to produce one module system which can cope without being overridden. This will be quite an achievement if the Expert Group can pull it off.

The road ahead

So it seems there is a place for both JSR 277 and JSR 291. Of course, JSR 291 should ideally benefit from the features introduced by JSR 277 and JSR 294. But the immediate requirement is that JSR 277 and JSR 291 interoperate well so that modules written in terms of one JSR are able to make effective use of modules written in terms of the other JSR.

To achieve this, the Expert Groups of JSR 277 and JSR 291 will need to work together. I expect this to take the form of the JSR 277 Expert Group proposing a means of interoperating with other module systems and then the JSR 291 Expert Group reviewing this proposal and providing feedback.

Interoperation is certainly an interesting challenge to keep us occupied, but it's just part of the challenge of working out the right model for modularity in the Java platform.

BEA's experience of OSGi in building mSA

Another excellent talk BJ pointed out was BEA's "The Good, Bad, and Ugly of OSGi: What we learned building the mSA Backplane".

Not only is this a great OSGi adoption story, but it points out several requirements that could feed in to a future version of OSGi. :-)

See Alex Blewitt's summary of the talk if you don't have time to download the slides.

IBM, Cisco, and OSGi

Quite a week for OSGi! IBM and Cisco just announced a platform based on OSGi known as the "unified communications and collaboration Client Platform ", although UC2 (or should that be UC²?) is less of a mouthful.

There were some hints inside IBM earlier this week that some interesting OSGi news was about to break, but I had no idea what it was.

UC2 is based on a subset of the Lotus Sametime 7.5 product that is popular for instant messaging (at least inside IBM). It aims to handle IM, voice, and video.

BEA's use of OSGi

Greg Brail gave some insights into how BEA are using OSGi. Seems like they are decomposing their products into OSGi bundles which can then be composed into new products.

The WebLogic Server has provided some of these bundles, but whether it will be structured as a collection of OSGi bundles is not yet publicly known. But given BEA's commitment to OSGi, I'd be very surprised if WebLogic didn't eventually end up being componentised in terms of OSGi.

eWeek article on OSGi

I recommend reading the eWeek article mentioned by BJ. Apart from giving a useful survey of the history of OSGi, it lists a number of projects that are exploiting OSGi, including one I personally hadn't heard of before.

The article takes the popular, if rather simplistic, position on JSR 277's relationship to OSGi, which makes me a little uncomfortable since I'm trying to get the two to interoperate.

Part of the success of OSGi is due to the fact that it isn't bolted into a particular Java platform but provides compatible behaviour across a broad range of ME and SE platforms. I should expand on this point some time.

Oracle TopLink being donated to Eclipse

BJ posted a really interesting development from EclipseCon. Oracle is donating a sophisticated persistence framework to Eclipse.

I can't tell from the FAQ whether the framework consists of 'pure' OSGi bundles or whether they make use of any Eclipse-specific features, but clearly the former would be ideal.

OSGi universal middleware

The buzz around OSGi is increasing. Not only is there an exciting OSGi track at EclipseCon this week, but more Java developers are slowly realising that OSGi isn't just a buzz-word - it can actually help solve their versioning and deployment issues (and a whole lot more).

Take Marc Loggeman's thoughts on how he's thinking of using OSGi as one tangible example.

How many technologies have a mature spec. and three open source implementations, with friendly licensing, to choose from? There is no barrier to entry other than getting up to speed with the fairly basic principles of creating and managing OSGi bundles.

I like the EclipseCon term for OSGi: "universal middleware".

JSR 291 Proposed Final Draft submitted

I just sent the Proposed Final Draft specification for JSR 291 to the JCP Program Management Office. It should emerge for download/review in a couple of weeks, so watch this space.

The technical work is essentially done and now there is a wealth of administrivia to attend to before the JSR can go to the Final Approval Ballot and then, all being well, into Final Release.

Glad I've got the module system interoperation discussion going on in JSR 277 to keep me sane...

JSR 291 rubber-stamp?

My colleague Jim Colson was interviewed by SD Times for the article "OSGi, JCP Tussle Over Component Support in Java: Critics decry effort as a rubber-stamp move".

The piece notes that JSR 291 is an additional source of requirements for OSGi, but doesn't go into specifics. It ends by quoting Hani Suleiman as saying "I was told that this is in fact a rubber-stamping effort and that the OSGi spec will not change".

So is this just a matter of hearsay and opinion or are there some relevant facts? Did the OSGi specification pop out of JSR 291 without any input from the Expert Group?

Well a number of specific requirements were raised by the JSR 291 Expert Group and resulted in changes in the underlying OSGi specification.

Take a specific example of a non-rubber-stamp change: the addition of an API to enable declarative service support and other similar features to obtain the Bundle context of a specific bundle.

This requirement was initially raised by BEA on the JSR 291 Expert Group mailing list as part of a thread relating to SPIs. (I'm glad we decided to run the Expert Group mailing list in the open!)

Since then the new Bundle.getBundleContext method has appeared in the early draft and public review draft specifications and in the Equinox open source reference implementation.

Does that seem like rubber-stamping to you?

Visualising dependencies between OSGi bundles

You may be interested in a tool for graphing the dependencies between OSGi bundles.

I haven't tried it, but I'd like to see what it makes of the dependencies of the Apache Harmony bundles. In the past we've run similar tools against class library code, but the resultant dependency graphs are pretty messy.

Focussing on OSGi will raise the level to package and bundle dependencies.

I like to structure large systems into layered collections of subsystems and sometimes the only way to achieve that is to see what a system's dependencies look like and then get refactoring.

OSGi gathering at JavaOne

Robert Varttinen is planning an OSGi gathering at this year's JavaOne. My plans are currently up in the air, but it sounds like a great idea for those that can make it.

JSR 291 Reference Implementation candidate

Alex's mention of the release of Eclipse driver 3.3M5eh (Canadian for "3.3M5a") reminds me to mention that the 3.3M5 version of Equinox is the current candidate for a Reference Implementation for JSR 291.

Download it here if you want to try out any of the new features mentioned in the PR draft.

Note that Equinox hasn't changed between 3.3M5 and 3.3M5eh.

Thanks to Tom Watson for another interpretation of "eh".

JSR 291 nears Proposed Final Draft

As I posted on the JSR 291 Expert Group mailing list, I hope to send the Proposed Final Draft specification for JSR 291 to the JCP Program Management Office on 5 March in which case it should pop up for review around 19 March.

It'd be nice if JSR 291 was pretty much done by this year's JavaOne.

Decimal syntax and operator overloading

As mentioned previously, I'm keen for Java to provide nice syntax for BigDecimal types. Someone asked me whether I was trying to sneak full operator overloading into Java through the back door. It's a reasonable concern.

Much as I like some other features of C++, I was very comfortable that Java omitted general operator overloading. So I am definitely not suggesting that BigDecimal syntax be introduced along with general operator overloading.

Of course, if these syntax improvements go ahead, Sun are free to add them via a limited form of operator overloading mechanism, but that's an implementation choice from my perspective and certainly not part of the requirement.

Geronimo's use of OSGi

I've noticed a couple of references which have hinted at Geronimo being based on OSGi, so I wanted to clarify the current situation for the benefit of those not closely involved.

While we were extending the Equinox OSGi framework in parallel with the spec. work for the modularity improvements in OSGi R4, my colleague Simon Burns had a crack at modularising Geronimo to make sure there were no obvious gotchas. This was way before WebSphere Application Server had exploited OSGi, so I was keen for Simon to try this out to smooth the path for WAS.

Simon posted a brief summary on the Geronimo dev list, but we didn't follow up as the Geronimo team was undecided about whether they wanted to exploit OSGi or extend the Geronimo GBean technology.

Of course, I would love eventually to add Geronimo to the list of application servers based on OSGi...

JSIG material on OSGi and JSR 291

The presentation and demo's from last week's excellent JSIG meeting on OSGi and JSR 291 are now available from EclipseZone.

JSR 277 and JSR 291 Interoperation Strawman

For anyone who's interested, I contributed a strawman module system interoperation framework to Apache Felix for discussion there and in the Expert Groups of JSR 277 and JSR 291. I hope we can make these module systems work well together...

See a post to the JSR 291 EG mailing list and docs/README in the JAR attached to the JIRA issue for some technical background.

OSGi and JSR 291 at JSIG

Today's JSIG meeting on OSGi and JSR 291 in London was well worth the commute.

I met beforehand for coffee with three guys from Paremus: Richard Nicholson (CEO), Mike Francis (Sales and Marketing Director), and Robert Dunne. They described their 'Infiniflow' product built, you guessed it, on top of OSGi. It runs on Knopflerfish and Equinox, although they hope to support Felix in due course. They ship the product's base layer, Newton, as open source. Newton provides basic services and is responsible for propagating SCA components, known as composites, around a network according to defined declarative specification documents.

I wondered if they might have to overcome IBM's perennial problem of creating software "by PhD's for PhD's", but they assured me there were simple use cases which made it easy for application programmers to get started.

The JSIG meeting was well attended with Alex Blewitt, the EclipseZone editor (on the left below), presenting and Neil Bartlett (on the right), running live demo's to illustrate the theory.


Alex explained the problem that OSGi solves and gave details of the key framework features. (I'll forgive Alex for the stunning slide transitions courtesy of Neil's Mac. Is he on commission?)

Neil's demo's were coded and built using NetBeans and included a Swing application to emphasise the point that OSGi is far from just 'an Eclipse thing'. He used Knopflerfish to run the demo's as it has an excellent GUI for installing bundles and starting and stopping them, etc.

They promised to dump the presentation and code on the JSIG website.

The audience, of around 30-40, seemed fairly lively and there were lots of questions, some of which were quite searching. It was good to bump into a former colleague of mine from Hursley. Sunny Chan worked on the initial version of the shared classes support we put into IBM's Java 5. He now seems to be enjoying working for an investment bank maintaining Java environments.

Afterwards I had lunch with Neil and Benoît Xhenseval of ObjectLab. The conversation centred on OSGi, although the need for decent decimal syntax in Java financial software came up again without my prompting. Must follow that up.

There was some mention of the various modularity JSRs throughout the day. It's amazing to see how different things look to complete outsiders (to the JCP). There seems to be a general bewilderment over JSR 277. Also, since some of these guys have only recently stopped supporting Java 1.3, Java 7 seems eons away from being the default platform for deployment.

The lasting impression of the day was how incredibly useful OSGi is for underpinning various kinds of commercial software. It seems that the OSGi Alliance has created the start of a potentially enormous value network which provides business opportunities for framework and middleware vendors, such as Paremus and ObjectLab, as well as to 'real' customers.

OSGi Alliance expands

In spite of the recent debate on SOA and OSGi, the SOA digest just published news that the OSGi Alliance has some new members: BEA Systems, IONA Technologies, Jayway AB, Eclipse, and Interface21 (the Spring guys).

The more the merrier!

OSGi at EclipseCon

Next month's EclipseCon features an OSGi track for the first time.

One of my close colleagues, Tim Ellison, will be talking about Apache Harmony, including its use of OSGi.

Another close colleague, BJ Hargrave, is co-leading a seminar on Spring-OSGi integration.

BEA are presenting their experiences of using OSGi to underpin the microService Architecture, while JBoss are presenting on their thoughts for how they could use OSGi.

All in all, sounds fascinating...

Is OSGi a SOA?

There's a heated debate in Markus Voelter's blog on the relationship of OSGi to Service Oriented Architecture (SOA). I must confess to having thought of OSGi as an example of a SOA until now.

One of the big benefits of SOA is a standardised way of lashing together myriad legacy (and new) systems. You may have seen the 'circuit diagrams' some organisations draw to illustrate the problems they face.

Wikipedia discusses SOA and quotes an OASIS definition:

A paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains. It provides a uniform means to offer, discover, interact with and use capabilities to produce desired effects consistent with measurable preconditions and expectations.

While OSGi can provide a very useful single process component model, it doesn't directly address distributed systems. Perhaps Markus should add 'distributed' to his list of SOA attributes.

Subsystems in OSGi

Neil Bartlett's excellent article on Eclipse Extensions versus OSGi Services touches on an interesting issue. OSGi services are always potentially dynamic - they can come and go when you least expect it.

Some developers may prefer always to use services to interface between bundles, but that is probably not a winning strategy for larger systems.

An ideal structure for such systems, IMO, is to group bundles into subsystems and have subsystems interact with other subsystems via services. Subsystems are not a formal OSGi concept, but they are an important concept in the mind of the designer.

Within a subsystem, bundles can depend on each other more tightly, e.g. by sharing classes. They could even use require-bundle for really tight coupling, although importing and exporting packages with a suitable attribute is sufficient to prevent subsystems from interfering with each other while maintaining the benefits of a somewhat looser coupling.
The benefit of tighter coupling in a subsystem is that there are fewer failure modes (types of error that can arise) due to bits of a subsystem coming and going when other bits of the subsystem least expect it.

In the example above, bundle a1 will not resolve unless its package import can be resolved. So by the time any code in bundle a1 runs, it can be sure that the dependency on a2 has been met and will not break for the lifetime of a1.