Nice to see another project successfully using OSGi in the form of SIP Communicator over on java.net. It was good to see they upgraded from Oscar to Felix painlessly.
They are even participating in the Google "Summer of Code™", so more developers will see how useful OSGi is. Thanks to Greg Wilson for the link.
There is some confusion about the effects on JSR 291 of jcp.org being in a mess. In particular the Final Approval Ballot won't be happening for at least a week or two.
The Proposed Final Draft spec. has been posted, but is inaccessible as far as I can tell. Fortunately, this is unlikely to matter. It's a pretty mature spec. which has had a few careful modifications due to JSR 291. The spec. is already undergoing a 45 day review by OSGi members. Also JSR 291 received no comments during the Public Review draft of the specification and only non-technical comments during the PR ballot.
So what's happening with the Final Approval Ballot? I submitted the FAB materials, but the email bounced after a couple of days so I just resent it. The spec. license needs a review by Sun lawyers because we used an IBM license rather than a standard Sun license, which will cause an additional unknown delay. After that, the FAB will begin. Hopefully jcp.org will be stable in time...
Let's keep an eye on Glassfish and see what we can learn as they select a modularisation technology.
Yesterday the Open SOA Collaboration announced they are planning to standardise the SCA and SDO specifications via OASIS.
A couple of the participants, IONA and Siemens AG, mentioned their interest in using OSGi with SCA. This is consistent with the OSGi Alliance's Enterprise Expert Group track on SCA and OSGi which IONA's Eric Newcomer mentioned back in January.
Should Java have included modularity features from the start or is the kind of modularity really needed in Java rather different from that baked in to some other languages?
In 2003, IBM Research published a paper known internally as "modjava" but named "MJ" externally. The authors' approach was to create a static module system not dissimilar to the strawman for JSR 294 except that MJ was built from class loaders for ease of prototyping and didn't encompass versioning (although a couple of us refactored the internal MJ prototype to enable versioning).
Vernon Green and I then led a working group in IBM's Software Group Architecture Board on Java versioning and modularity which started from the MJ paper. Later we discovered OSGi and how that matched in many respects what the working group was looking for. I was given the task of taking our additional requirements into the OSGi R4 standards activity where I struck up a collaboration with Richard Hall who had a similar perspective based on his research work on the Oscar OSGi framework.
The real world requirements of versioning and dynamic modification of applications and supporting software without restarting the JVM have effectively forced Java to develop the new, and highly successful, OSGi technology.
It's interesting to speculate what would have happened if a static module system had been put into Java much earlier. My guess is that it wouldn't have bothered to address versioning or dynamicity requirements which only become crucial in the context of relatively large systems with continuous operation.
The JSR 294 Expert Group is just starting up, so this background suddenly becomes relevant...
As I mentioned to the JSR 291 Expert Group, I just submitted the specification, reference implementation, and technology compatibility kit for JSR 291 for the Final Approval Ballot by the Java SE Executive Committee.
Neil questioned the point of putting modularity support into the Java platform.
Of course, OSGi does a pretty good job of modularity support based on java.lang.ClassLoader. So why isn't that sufficient?
At one level, OSGi is quite sufficient. There is already massive adoption of OSGi and this is continuing to spread into new and interesting areas.
At a more theoretical level, there are some things that custom class loader networks will never achieve.
Most importantly is VM enforcement of module boundaries. It is possible to obtain unexported Class instances from a class loader. OSGi bundles are implemented as class loaders. Similarly, JSR 277 modules will boil down to specialised class loaders. So this is where JSR 294 comes in with its objective of enforcing module boundaries in the VM.
This extra level of enforcement is irrelevant if the aim of using modules is to divide a piece of software into manageable units with well-defined interfaces.
However, VM enforcement starts to become interesting if modules need to protect some parts of an application or the system from inappropriate access. There is also some scope for additional, or at least simplified, performance optimisations if certain classes can be guaranteed never to be accessed outside their defining module.
I would dearly like JSR 294 to provide mechanisms that will be equally useful to OSGi as to JSR 277. I think OSGi would benefit from VM support, although it would be an exaggeration to say that OSGi really needs VM support.
Neil wondered what changes are likely to be made to java.lang.ClassLoader in support of JSR 277.
The Early Draft spec. lists some requirements for class loader behaviour in section 8.3.4. These boil down to class loaders being able to cope with more concurrency, especially in the presence of cyclic dependencies between JSR 277 modules.
Currently the deadlock exhibited by Sun's reference implementation has an experimental fix activated by a switch. There is some work in the pipeline to improve this situation for Java 7 which should also benefit OSGi at least when running on Sun's RI. Keep an eye on the Hotspot VM on OpenJDK if you are interested.
People are pretty perceptive. They have picked up that there's a struggle to work out the right model for Java modularity. However, it's not a simple choice between JSR 277 and JSR 291 as many seem to think. Let's look at what's going on in a bit more depth.
Background
Firstly, a brief recap for the uninitiated.
JSR 277 is trying to create a static module system for Java 7. JSR 291 is referencing and extending the already mature OSGi dynamic component system while maintaining compatibility with JSR 232 which did a similar thing for Java ME. JSR 294, an offshoot of JSR 277, is focused on the VM and language support for modularity.
My personal involvement is that I belong to the JSR 277 Expert Group (but cannot disclose the discussions which are private to the group) and am the spec. lead of JSR 291 (which has an openly readable mailing list). I am also in the JSR 294 Expert Group, which hasn't started work yet.
Secondly, I want to be up front about my potential bias.
I have been involved in the OSGi Core Platform Expert Group for a couple of years during which time I co-authored some of the modularity improvements for OSGi R4 with Richard Hall (the leader of the Apache Felix OSGi project).
I work for IBM which belongs to the OSGi Alliance and which has based some of its products on OSGi. IBM and Sun co-operate on Java, but compete in related areas such as Java EE, not to mention UNIX-style operating systems and hardware. So IBM and Sun don't necessarily agree on everything that happens in Java. In fact, I think it would be pretty unhealthy if that were the case.
I should also re-iterate that my opinions are not necessarily shared by IBM.
That said, I'll try to lay out what's going on as objectively as I can.
Design points
The crucial thing to notice is that the goals of these JSRs are rather different even though the underlying technology is similar. JSRs 277 and 294 are building basic support into the Java SE platform whereas JSR 291 is pure Java built on top of the Java platform.
Looking at it another way, JSRs 277 and 294 are attempting to produce an evolution of java.lang.ClassLoader with a new distribution and packaging format, a new repository, plus new language features supported by the VM. JSR 291 on the other hand is implemented in pure Java on top of java.lang.ClassLoader and uses standard JAR files with standard manifests as a distribution and packaging format.
The design point of OSGi has always been to support a broad set of Java ME environments, such as profiles based on the Connected Device Configuration, and Java SE versions, from Java 1.2 onwards, in a compatible way. It has achieved this by building on commonly available Java concepts, notably class loaders. I think of OSGi as being the pinnacle of custom class loader systems. The way it manages to install, uninstall, start, stop, and update modules without restarting the VM is quite an achievement. Of course, some other systems have provided some of this capability, but I am not aware of any that have done it in such a generally reusable way.
So you can package OSGi with your application, or list it as a prerequisite, and then run on a wide range of Java platforms. JSR 277 on the other hand has the advantage that if you are happy to pre-req. Java 7 or later, then JSR 277 will be built in to the platform.
Not invented here?
So why can't the JCP simply adopt JSR 291 as a standard module system for Java? That's what many of the critics of JSR 277 suggest should happen.
One problem is that the OSGi specification, although mature, is still evolving. A number of new requirements were surfaced by the JSR 291 Expert Group. OSGi is rapidly spreading into new areas of application which are bound to result in additional requirements for OSGi R5 and beyond. To lock a particular Java SE version, such as Java 7, to a particular version of OSGi would be limiting as there wouldn't be an opportunity to upgrade the OSGi support significantly before Java 8. For example, an application that required some new features of OSGi wouldn't necessarily want to limit its deployment to Java 8 environments. So the ability to mix and match Java and OSGi versions as required is essential.
You may be wondering whether JSR 277 has the same problem, but since it is deliberately positioned to be the module system packaged with the Java 7 platform, its subsequent rate of evolution is, by definition, the same as that of Java. So the design point is that it has to be fairly basic in its support and not aim for the level of sophistication of OSGi, particularly in its support for legacy applications, which is one of OSGi's strong points.
The challenge for JSR 277 is to address its requirements in a sufficiently general way that it does not need to evolve quickly. The classic way of achieving this would be to divide the function into a lower level and relatively stable API and a user overridable piece implemented in terms of the lower level API. Like a module version of custom class loaders. JSR 277, as you can see from the Early Draft, is not adopting this approach but is attempting to produce one module system which can cope without being overridden. This will be quite an achievement if the Expert Group can pull it off.
The road ahead
So it seems there is a place for both JSR 277 and JSR 291. Of course, JSR 291 should ideally benefit from the features introduced by JSR 277 and JSR 294. But the immediate requirement is that JSR 277 and JSR 291 interoperate well so that modules written in terms of one JSR are able to make effective use of modules written in terms of the other JSR.
To achieve this, the Expert Groups of JSR 277 and JSR 291 will need to work together. I expect this to take the form of the JSR 277 Expert Group proposing a means of interoperating with other module systems and then the JSR 291 Expert Group reviewing this proposal and providing feedback.
Interoperation is certainly an interesting challenge to keep us occupied, but it's just part of the challenge of working out the right model for modularity in the Java platform.
Another excellent talk BJ pointed out was BEA's "The Good, Bad, and Ugly of OSGi: What we learned building the mSA Backplane".
Not only is this a great OSGi adoption story, but it points out several requirements that could feed in to a future version of OSGi. :-)
See Alex Blewitt's summary of the talk if you don't have time to download the slides.
Quite a week for OSGi! IBM and Cisco just announced a platform based on OSGi known as the "unified communications and collaboration Client Platform ", although UC2 (or should that be UC²?) is less of a mouthful.
There were some hints inside IBM earlier this week that some interesting OSGi news was about to break, but I had no idea what it was.
UC2 is based on a subset of the Lotus Sametime 7.5 product that is popular for instant messaging (at least inside IBM). It aims to handle IM, voice, and video.
Greg Brail gave some insights into how BEA are using OSGi. Seems like they are decomposing their products into OSGi bundles which can then be composed into new products.
The WebLogic Server has provided some of these bundles, but whether it will be structured as a collection of OSGi bundles is not yet publicly known. But given BEA's commitment to OSGi, I'd be very surprised if WebLogic didn't eventually end up being componentised in terms of OSGi.
I recommend reading the eWeek article mentioned by BJ. Apart from giving a useful survey of the history of OSGi, it lists a number of projects that are exploiting OSGi, including one I personally hadn't heard of before.
The article takes the popular, if rather simplistic, position on JSR 277's relationship to OSGi, which makes me a little uncomfortable since I'm trying to get the two to interoperate.
Part of the success of OSGi is due to the fact that it isn't bolted into a particular Java platform but provides compatible behaviour across a broad range of ME and SE platforms. I should expand on this point some time.
BJ posted a really interesting development from EclipseCon. Oracle is donating a sophisticated persistence framework to Eclipse.
I can't tell from the FAQ whether the framework consists of 'pure' OSGi bundles or whether they make use of any Eclipse-specific features, but clearly the former would be ideal.
The buzz around OSGi is increasing. Not only is there an exciting OSGi track at EclipseCon this week, but more Java developers are slowly realising that OSGi isn't just a buzz-word - it can actually help solve their versioning and deployment issues (and a whole lot more).
Take Marc Loggeman's thoughts on how he's thinking of using OSGi as one tangible example.
How many technologies have a mature spec. and three open source implementations, with friendly licensing, to choose from? There is no barrier to entry other than getting up to speed with the fairly basic principles of creating and managing OSGi bundles.
I like the EclipseCon term for OSGi: "universal middleware".
I just sent the Proposed Final Draft specification for JSR 291 to the JCP Program Management Office. It should emerge for download/review in a couple of weeks, so watch this space.
The technical work is essentially done and now there is a wealth of administrivia to attend to before the JSR can go to the Final Approval Ballot and then, all being well, into Final Release.
Glad I've got the module system interoperation discussion going on in JSR 277 to keep me sane...
My colleague Jim Colson was interviewed by SD Times for the article "OSGi, JCP Tussle Over Component Support in Java: Critics decry effort as a rubber-stamp move".
The piece notes that JSR 291 is an additional source of requirements for OSGi, but doesn't go into specifics. It ends by quoting Hani Suleiman as saying "I was told that this is in fact a rubber-stamping effort and that the OSGi spec will not change".
So is this just a matter of hearsay and opinion or are there some relevant facts? Did the OSGi specification pop out of JSR 291 without any input from the Expert Group?
Well a number of specific requirements were raised by the JSR 291 Expert Group and resulted in changes in the underlying OSGi specification.
Take a specific example of a non-rubber-stamp change: the addition of an API to enable declarative service support and other similar features to obtain the Bundle context of a specific bundle.
This requirement was initially raised by BEA on the JSR 291 Expert Group mailing list as part of a thread relating to SPIs. (I'm glad we decided to run the Expert Group mailing list in the open!)
Since then the new Bundle.getBundleContext method has appeared in the early draft and public review draft specifications and in the Equinox open source reference implementation.
Does that seem like rubber-stamping to you?
You may be interested in a tool for graphing the dependencies between OSGi bundles.
I haven't tried it, but I'd like to see what it makes of the dependencies of the Apache Harmony bundles. In the past we've run similar tools against class library code, but the resultant dependency graphs are pretty messy.
Focussing on OSGi will raise the level to package and bundle dependencies.
I like to structure large systems into layered collections of subsystems and sometimes the only way to achieve that is to see what a system's dependencies look like and then get refactoring.
Robert Varttinen is planning an OSGi gathering at this year's JavaOne. My plans are currently up in the air, but it sounds like a great idea for those that can make it.
