Java update breaks jps, jconsole, etc

The latest update of Java broke jps and jconsole when Java is launched specifying "-Djava.io.tmpdir" with a value of anything other than $TMPDIR. This affects Virgo and Tomcat.

The problem was introduced in Java 6 update 21 and is fixed by bug sunbug 7009828 in Java 6 update 25.

A workaround for Virgo is as follows. On Mac and Linux to delete the following line in dmk.sh:

-Djava.io.tmpdir=$TMP_DIR \

and on Windows to delete the following line in dmk.bat:

set KERNEL_JAVA_PARMS=%KERNEL_JAVA_PARMS% -Djava.io.tmpdir="%TMP_DIR%" 

Eclipse Virgo diagnostics for "uses" conflicts

Congratulations to Felix on producing some really nice diagnostics for uses conflicts, and thanks to Neil Bartlett for his blog on the subject.

I thought I'd try a similar example in Virgo for comparison. Here are Virgo's diagnostics:

Unable to satisfy dependencies of bundle 'org.example.foo.xydata' at version '0.0.0': Cannot resolve: org.example.foo.xydata
    Resolver report:
        Uses violation: ⟨Import-Package: org.example.foo.bar.datastore; version="0.0.0"⟩ in bundle ⟨org.example.foo.xydata_0.0.0[1301563814803]⟩
            Found conflicts:
                package        'org.example.foo.util.verify_1.0.0' in bundle 'org.example.foo.platform.util_0.0.0[1301563814806]' used by 'org.example.foo.bar.datastore_0.0.0' in bundle 'org.example.foo.bar.datastore_0.0.0[1301563814805]'
                conflicts with 'org.example.foo.util.verify_2.0.0' in bundle 'org.example.foo.framework_0.0.0[1301563814804]' imported by bundle 'org.example.foo.xydata_0.0.0[1301563814803]'

(The numbers in square brackets are bundle ids - they are enormous because Virgo is attempting resolution in a "side state". Bundles with enormous ids have either been deployed directly or auto-installed from the Virgo repository and are in the process of being resolved. They are committed to the OSGi framework only if the directly deployed bundle(s) can be resolved successfully.)

The Virgo formatting isn't quite as pretty as Felix's, but it looks reasonable on a wide terminal window. However, there are some subtle differences:

• Felix dumps out the dependency chains whereas Virgo does not attempt that. We had in mind examples where the dependency chains are unmanageably long, i.e. those where good diagnostics really are essential. But having seen Felix's diagnostics, we would like to dump out the chains in simple cases. On the other hand, that area of code is pretty complex, so this is not something we'll be rushing to do.
• Felix depicts wirings from import to export which could be especially useful for newcomers to OSGi.
• The Felix diagnostics are built in to the framework whereas non-Virgo users of Equinox do not get the Virgo diagnostics.
• The Virgo diagnostics highlight the fact that there is a uses violation whereas Felix leaves the user to infer that from the rest of the message. It would be nice if Felix was more explicit because there are other reasons a bundle cannot be resolved.
• Felix highlights the duplicated package whereas Virgo highlights the import that encountered the uses constraint. Virgo is designed to cope well when there are multiple duplicated packages. It's not clear what the Felix diagnostics would look like in such cases.

Note: on a resolution failure Virgo dumps the "side state" to disk for offline analysis. With this example, the offline analysis produced poorer diagnostics than were printed to the console, so I raised bug 341460 to investigate. The beauty of offline analysis is it lets you explore the details of the bundles. For example, the following screenshots shows the details of the duplicated package, including the import version ranges which result in the attempt to use the distinct versions of the duplicated package:

Anyway, I'm really pleased that we are starting to see some stiff competition on diagnostics. This will help OSGi users enormously.

Postscript
Bug 341460 is fixed in the Virgo 3.0 line. The admin console now displays the same diagnostics that were printed to the console.

Enterprise OSGi Runtimes and Tooling

It was great to see several enterprise OSGi runtimes in action at EclipseCon.

Readers of this blog will be familiar with Eclipse Virgo, so suffice to say that tutorials covered developing  applications using Eclipse Gemini components (reference implementations of several OSGi Enterprise standards) in Virgo and configuring and managing web applications in Virgo.

Several of the tutorials developed Web Application Bundles, or WABs, which are part of the OSGi Enterprise standard.  A Web Application Bundle is like a WAR file which specifies the web context path in its manifest and which obtains its dependencies using OSGi instead of having to include them all in WEB-INF/lib. The tutorials also implemented persistence using the OSGi Enterprise standard mapping of Java Persistence API (JPA).

Apache Aries provides a collection of components including implementations of several Enterprise OSGi standards and an application model similar to Virgo's. The EclipseCon tutorial had us creating a blogging application using the free IBM® Rational® tooling. The application consisted of an Aries application containing a plain OSGi bundle, a bundle utilising JPA persistence, and a WAB. The JPA bundle used Aries extensions to the standard blueprint service. The WAB used JNDI to look up the service published by the persistence bundle - the same approach that was taken in the Gemini and Virgo tutorial. We then deployed the application to a target platform which included Aries bundles.

Apache Karaf is similar in concept to the Virgo kernel. Karaf has several functions that Virgo lacks, some of which are good candidates to add to Virgo. Apache Geronimo is based on Karaf.

GlassFish now has good support for OSGi. The EclipseCon tutorial started with us developing a basic client bundle calling a service published by a back-end bundle. We then replaced the client bundle with a WAB and the back-end bundle by a JPA persistence bundle which published a service in the form of an EJB. The WAB used custom annotations to inject the service into the WAB's servlet. The EJB was used to demarcate transactions and security. The M2Eclipse "core" and "extras" plugins provided a basic Eclipse development environment, although the bundles were deployed via the command line. Kudos to the GlassFish team, including Arun Gupta and Sanjeeb Sahoo who led the tutorial, for possibly the most polished hands-on tutorial I've experienced.

I was also very pleased to see the Eclipse Libra project making progress. The version destined for inclusion in this year's Eclipse release train can be used to develop WABs. Runtime launching support is in the pipeline. We also hope to migrate standards based features from the Virgo tooling into Libra.

git tip: listing or editing the commits on a branch

Sometimes I need to list all the commits on a branch. I found the following command does the trick, even after I have merged from master into the branch one or more times:

git rev-list my-branch ^master

This lists all the commits reachable from my-branch but not reachable from master.

You can also change the committer or author info of commits on a branch by specifying  my-branch ^master as the on git filter-branch (see git help filter-branch). 

Quick survey of Virgo and SpringSource dm Server usage

I would be grateful if users of Virgo and SpringSource dm Server would take a short, anonymous survey to help us in our planning. Should take no more than 2 minutes to complete.

Correction: Oracle do not contribute to Virgo

A recent Oracle blog stated:

Also presented were Eclipse projects, that Oracle engineers are contributing to: Gemini, Virgo, Sapphire and Web Tool Platform.

I submitted a comment on the blog pointing out that Oracle engineers contribute to Gemini but not Virgo. Unfortunately, the comment was not published and the blog was not corrected so perhaps this blog will help to set the record straight.

That said, Virgo would welcome contributions from Oracle (and others). We've enjoyed collaborating with Oracle in the Gemini project and we really appreciate having Virgo committers from three different companies and a few more would be even better.

Incremental OSGi with Virgo, Gemini, and Libra

As I anticipate the start of EclipseCon 2011 later today, I'm reflecting on the current state of the Eclipse Virgo, Gemini, and Libra projects.

As I'll be discussing on Tuesday, Virgo evolved out of a relatively mature project whose aim was to make it easy for enterprise applications and enterprise application developers to adopt OSGi. The goal was to make the transition to OSGi incremental. Essentially you can start with a standard WAR file and deploy it to Virgo as-is. Then you can incrementally refactor the WAR file into an application comprising multiple OSGi bundles while continuing to use familiar technologies such as Spring, Hibernate, JPA, etc. We based Virgo on Tomcat suitably embedded into OSGi, again to provide familiarity for enterprise developers as well as systems administrators.

In parallel with the evolution of Virgo, the OSGi Alliance Enterprise Expert Group, being discussed on Tuesday, produced a series of enterprise specifications again with the goal of enabling enterprise applications to be migrate straightforwardly to OSGi. The reference implementations of these specifications became today's Eclipse Gemini project:

• Gemini Blueprint - an OSGi standard dependency injection container which also supports the popular Spring and Spring DM namespaces
• Gemini Web - support for servlets in the form of OSGi standard Web Application Bundles (WABs) or standard WAR files
• Gemini JPA - support for JPA persistence in OSGi
• Gemini DBAccess - modularised JDBC drivers for use in OSGi
• Gemini Management - JMX management for OSGi
• Gemini Naming - support for JNDI naming in OSGi

Until recently, integrating the Gemini projects into Virgo has been a fairly specialised skill. As we'll see in a workshop on Thursday, things are now really starting to come together and Gemini Blueprint, Gemini Web, and various other Gemini components can now be deployed and used in Virgo. We'll also be touching on how to manage and configure WABs in Virgo as part of a workshop later this morning.

So much for the past. What about the future? The primary goal of Virgo 3.0 is better integration with other EclipseRT technologies such as Equinox, Jetty, and p2. This will provide richer function and greater choice for enterprise applications as they migrate to OSGi. We'll be hearing about how Virgo and EclipseRT technologies work together on Wednesday. The recently released third milestone of this release shows the progress towards the goal of better integration with EclipseRT:

• Virgo now uses Equinox Configuration Admin and Event Admin services instead of their Apache equivalents.
• Virgo has moved to the new framework hooks model for isolation in Equinox which is part of the OSGi 4.3 standard soon to be issued for public review.
• Virgo now provides two web servers: Virgo Tomcat Server, which continues to be based on Gemini Web, and Virgo Jetty Server. Some exciting technology for modularising the view portion of web applications is also emerging into the Virgo mainstream as we'll hear on Wednesday.
• p2 support in Virgo is also making good progress although this is currently in prototype form.

Development tooling is also entering an exciting new phase which will greatly help applications migrate to enterprise OSGi:

• The Libra project, which will be presented on Wednesday, is providing development tooling for building WABs and will ship in the Eclipse Indigo release train later this year.
• The Virgo development tooling has entered the Eclipse IP process and will soon emerge in the Virgo project. Over time, the standards-based features of the Virgo tooling will be factored out and migrated into Libra.
• OSGi runtime launchers are also being contributed to Libra. I would hope to see a Virgo launcher for Libra emerge so that Libra can start to take over from the Virgo tooling for many aspects of enterprise OSGi application development.

If you're attending EclipesCon, come and hear the talks, join in the workshops and BoFs and help to shape incremental OSGi in the Virgo, Gemini, and Libra projects.

Importing packages into a 3rd party bundle

I regularly come across the situation where someone needs to import additional packages into a 3rd party OSGi bundle. Sometimes this is because the bundle manifest is incorrect, but more often it's because the bundle uses Class.forName or similar to load classes that are somehow configured at runtime.

Unless the bundle is signed, it's possible to edit the manifest to add the import, but this is a really bad idea. If the original bundle needs upgrading, you have to remember to re-edit the manifest. Also, it's easy to mess something up in the process. Finally, it's bad practice to change the contents of a bundle without modifying the bundle version.

Thankfully, there is a simple way to add imports to a bundle without resorting to such invasive surgery: use a fragment bundle. Fragment bundles can extend the manifest of their host in various ways. In particular, they can add package imports. The fragment bundle need only contain a manifest, for example:

Manifest-Version: 1.0

Bundle-ManifestVersion: 2

Bundle-SymbolicName: my.fragment

Bundle-Version: 1

Bundle-Name: My Fragment

Fragment-Host: host.symbolic.name;bundle-version="[2.4,2.5)"

Import-Package: extra.package;version="[1.1,1.2)"

More than one fragment bundle can attach to a given host bundle, so this technique can be used when unrelated groups of packages need to be imported: a separate fragment can be used for each group.

Each fragment bundle needs to be installed before its host is resolved at which point the fragment's manifest is merged into the host's and the resultant Bundle instance imports the packages added by the fragment. On Virgo, this can be achieved by placing the fragment bundles in repository/usr. When the host is deployed, the fragments are automatically installed along with the host.

Note: make sure you use the "bundle-version" attribute of Fragment-Host. "version" is not a standard attribute of Fragment-Host and if you use it accidentally in place of "bundle-version", chances are your fragment will not attach to its host.

Virgo 3.0.0.M03 and EclipseCon

Milestone 3 of Virgo 3.0 is available for download. Apart from several bug fixes, this milestone includes a contribution from Tom Watson who is collaborating with the Virgo team to prepare the region digraph for its move to Equinox so that it is available to all Equinox users rather than just Virgo users.

We will be using milestone3 in tutorials at next week's EclipseCon. If you're attending, watch out for Virgo committers delivering the following sessions:

• Building web applications with EclipseRT
• Donating a mature project to Eclipse
  
• Snaps, modular & dynamic web applications with Virgo
  
• Virgo and RT playing together
  
• Hands on with Gemini and Virgo--The new Stars of the Eclipse RT project!
  
• Virgo BoF -- to be arranged, keep a look out!

Hope to see you there!