Virgo 3.5.0 with Nano Technology

Virgo 3.5.0.RELEASE is available for download.

Although the deliverables are mostly backward compatible with Virgo 3.0.x, we decided to indicate the importance of the release by bumping the version from 3.0 to 3.5.

All Virgo deliverables are now launched using the Equinox launcher and have a corresponding directory structure which is slightly different from that of 3.0.x. All the deliverables can all be initially provisioned using p2, which is useful, for instance, when setting up remote nodes or automating installation.

There are two new deliverables which the team is pretty excited about. Virgo nano is a cut-down subset of the kernel. It supports hot deployment of bundles via the pickup directory, the Gogo shell, a single (kernel) region, and the medic component which provides Virgo's basic diagnostic capabilities.

The second new deliverable is Nano Web which is essentially Gemini Web -- the Apache Tomcat based reference implementation of the OSGi Web Applications specification -- running on Nano. In addition to Nano's features, Nano Web supports hot deployment of Web Application Bundles and WAR files.

Compared to the kernel-based deliverables, which are already pretty small and fast, Nano starts much faster and has a much smaller footprint as the following charts indicate.¹

Both Nano and Nano Web can be provisioned with application bundles at runtime using p2. The kernel based deliverables can only be initially provisioned using p2. Applications are then deployed into the user region in the usual way. This provisioning limitation in p2 triggered the initial development of Nano, although Nano soon appeared to have other benefits such as smaller runtime footprint (particularly important in the cloud) and the possibility of using PDE for bundle development. Also, Nano Web is likely to integrate additional enterprise Java components in the future to support applications written for the Java EE web profile.

There are a number of other interesting features in 3.5.0 including integrated support for the OSGi Blueprint Service, several significant kernel enhancements, and various dependency upgrades - all described in the release notes.

Additionally, Virgo Bundlor 1.1.1 and the Greenpages 2.5.0 sample application are released in association with Virgo 3.5.0.

In parallel with the development of Virgo 3.5.0, the Virgo IDE tooling has been overhauled and will be released in the next few weeks. Meanwhile you can use milestones and snapshots of the new tooling as described on the tooling wiki page.

Please see the release notes for further details of Virgo 3.5.0 and the migration notes for information about upgrading from Virgo 3.0.x.

Footnote:

1. Startup times are approximate. Measurements were taken using Mac OS X 10.7.4, and Java 1.6.0_33 on a Mac Pro with 12 GB RAM and 2 x 2.66 GHz dual core Intel Xeon processors.

OSGi case study: a modular vert.x

OSGi enables Java code to be divided cleanly into modules known as bundles with access to code and resources controlled by a class loader for each bundle. OSGi services provide an additional separation mechanism: the users of an interface need have no dependency on implementation classes, factories, and so forth.

The following case study aims to make the above advantages of OSGi bundles and services concrete. It takes an interesting Java project, vert.x, and shows how it can be embedded in OSGi and take advantage of OSGi's facilities.

Disclaimer: I am not proposing to replace the vert.x container or its module system. This is primarily a case study in the use of OSGi although some of the findings should motivate improvements to vert.x, especially when it is embedded in applications with custom class loaders.

vert.x

The vert.x open source project provides a JVM alternative to node.js: an asynchronous, event-driven programming model for writing web applications in a number of languages including Java, Groovy, JavaScript, and Ruby.

vert.x supports HTTP as well as modern protocols such as WebSockets and sockjs (which works in more browsers than WebSockets and can traverse firewalls more easily).

vert.x has a distributed event bus which allows JSON messages to be propagated between vert.x applications known as verticles and shared code libraries known as busmods. A busmod is a special kind of verticle which handles events from the event bus. vert.x ships some busmods, such as a MongoDB 'persistor', and users can write their own.

vert.x's threading model is interesting as each verticle (or busmod) is bound to a particular thread for its lifetime and so the code of a verticle needn't be concerned about thread safety. A pool of threads is used for dispatching work on verticles and each verticle must avoid blocking or long-running operations so as not to impact server throughput (vert.x provides separate mechanisms for implementing long-running operations efficiently). This is similar to the quasi-reentrant threading model in the CICS transaction processor.¹

Of particular interest here is the vert.x module system which has a class loader per verticle and code libraries, known as modules, which are loaded into the class loader of each verticle which uses them. So there is no way to share code between verticles except via the event bus.

vert.x has excellent documentation including a main manual, a java manual (as well as manuals for other language), tutorials, and runnable code examples.

OSGi

If you're not already familiar with OSGi, read my OSGi introduction post, but don't bother following the links in that post right now - you can always go back and do that later.

Embedding vert.x in OSGi

I did this in several small steps which are presented in turn below: converting vert.x JARs to OSGi bundles and then modularising verticles, busmods, and event bus clients.

Converting vert.x JARs to OSGi Bundles

The vert.x manual encourages users to embed vert.x in their own applications by using the vert.x core JAR, so the first step in embedding vert.x in OSGi was to convert the vert.x core JAR into an OSGi bundle so it could be loaded into an OSGi runtime.

I used the bundlor tool, although other tools such as bnd would work equally well. Bundlor takes a template and then analyses the bytecode of the JAR to produce a new JAR with appropriate OSGi manifest headers. Please refer to the SpringSource Bundlor documentation for further information about bundlor for now as the Eclipse Virgo Bundlor documentation is not published at the time of writing even though the bundlor project has transferred to Eclipse.org.

The template for the vert.x core JAR is as follows:

Bundle-ManifestVersion: 2
Bundle-SymbolicName: org.vertx.core
Bundle-Version: 1.0.0.final
Bundle-Name: vert.x Core
Import-Template:
 org.jboss.netty.*;version="[3.4.2.Final,4.0)",
 org.codehaus.jackson.*;version="[1.9.4,2.0)",
 com.hazelcast.*;version="[2.0.2,3.0)";resolution:=optional,
 groovy.*;resolution:=optional;version=0,
 org.codehaus.groovy.*;resolution:=optional;version=0,
 javax.net.ssl;resolution:=optional;version=0,
 org.apache.log4j;resolution:=optional;version=0,
 org.slf4j;resolution:=optional;version=0
Export-Template: *;version="1.0.0.final"

(The template and all the other parts of this case study are available on github.)

What this does is define the valid range of versions for packages that the JAR depends on (the range "0" represents the version range of 0 or greater), whether those packages are optional or mandatory, and what version the JARs own packages should be exported at. It also gives the bundle a symbolic name (used to identify the bundle), a version, and a (descriptive) name. Armed with this information, OSGi then wires together the dependencies of bundles by delegating class loads and resource lookups between bundle class loaders.

Thankfully the netty networking JAR and jackson JSON JARs which the vert.x core JAR depends on ship with valid OSGi manifests.

As a sniff test that the manifest was valid, I tried deploying the vert.x core bundle in the Virgo kernel. This was simply a matter of placing the vert.x core bundle in the pickup directory and its dependencies in the repository/usr directory and then starting the kernel. The following console messages showed the vert.x core bundle was installed and resolved successfully:

<hd0001i> Hot deployer processing 'INITIAL' event for file 'vert.x-core-1.0.0.final.jar'.
<de0000i> Installing bundle 'org.vertx.core' version '1.0.0.final'.
<de0001i> Installed bundle 'org.vertx.core' version '1.0.0.final'.
<de0004i> Starting bundle 'org.vertx.core' version '1.0.0.final'.
<de0005i> Started bundle 'org.vertx.core' version '1.0.0.final'.

Using the Virgo shell, I then checked the wiring of the bundles:

osgi> ss
"Framework is launched."

id      State       Bundle
0       ACTIVE      org.eclipse.osgi_3.7.1.R37x_v20110808-1106
...
89      ACTIVE      org.vertx.core_1.0.0.final
90      ACTIVE      jackson-core-asl_1.9.4
91      ACTIVE      jackson-mapper-asl_1.9.4
92      ACTIVE      org.jboss.netty_3.4.2.Final

osgi> bundle 89
org.vertx.core_1.0.0.final [89]
  ...
  Exported packages
    ...
    org.vertx.java.core; version="1.0.0.final"[exported]
    org.vertx.java.core.buffer; version="1.0.0.final"[exported]
    ...
  Imported packages
    org.jboss.netty.util; version="3.4.2.Final"<org.jboss.netty_3.4.2.final [92]>
    ...
    org.codehaus.jackson.map; version="1.9.4"<jackson-mapper-asl_1.9.4 [91]>
    ...

I also converted the vert.x platform JAR to an OSGi bundle in similar fashion as it was needed later.

Modularising Verticles

A typical verticle looks like this:

public class ServerExample extends Verticle {

  public void start() {
    vertx.createHttpServer().requestHandler(new Handler<httpserverrequest>() {
      public void handle(HttpServerRequest req) {
        ...
      }
    }).listen(8080);
  }
}

When the start method is called it creates a HTTP server, registers a handler with the server, and sets the server listening on a port. Apart from the body of the handler, the remainder of this code is boilerplate. So I decided to factor out the boilerplate into a common OSGi bundle (org.vertx.osgi) and replace the verticle with a modular verticle bundle containing the handler and some declarative metadata equivalent to the boilerplate. The common OSGi bundle uses the whiteboard pattern to listen for specific kinds of services in the OSGi service registry, create boilerplate based on the metadata, and register the handler with the resultant HTTP server.

Let's look at the modular verticle bundle. Its code consists of a single HttpServerRequestHandler class:²

public final class HttpServerRequestHandler implements Handler<httpserverrequest> {

    public void handle(HttpServerRequest req) {
        ...
    }

}

It also has declarative metadata in the form of service properties which are registered along with the handler in the OSGi service registry. I used the OSGi Blueprint service to do this, although I could have used OSGi Declarative Services or even registered the service programmatically using the OSGi API. The blueprint metadata is a file blueprint.xml in the bundle that looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">
        
    <service interface="org.vertx.java.core.Handler" ref="handler">
        <service-properties>
            <entry key="type" value="HttpServerRequestHandler">
            <entry key="port" value="8090">
        </service-properties>
    </service>
        
    <bean class="org.vertx.osgi.sample.basic.HttpServerRequestHandler"
          id="handler"/>

</blueprint>

This metadata declares that a HTTP server should be created (via the type service property), the handler registered with it, and the server set listening on port 8090 (via the port service property). This all happens courtesy of the whiteboard pattern when the org.vertx.osgi bundle is running as we'll see below.

Notice that the modular verticle depends only on the Handler and HttpServerRequest classes whereas the original verticle also depends on the Vertx, HttpServer, and Verticle classes. This also makes things quite a bit simpler for those of us who like unit testing (in addition to in-container testing) as fewer mocks or stubs are required.

So what do we now have? Two bundles to add to the bundles we installed earlier: an org.vertx.osgi bundle which encapsulates the boilerplate code and an application bundle representing a modular verticle. We also need a Blueprint service implementation -- as of Virgo 3.5, a Blueprint implementation is built in to the Virgo kernel. The following interaction diagram shows one possible sequence of events:

In OSGi, each bundle has its own lifecycle and in general bundles are designed so that they will function correctly regardless of the order in which they is started relative to other bundles. In the above example the assumed start order is: blueprint service, org.vertx.osgi bundle, modular verticle bundle. However, the org.vertx.osgi bundle could start after the modular verticle bundle and the end result will be the same: a server will be created and the modular verticle bundle's handler registered with the server and the server set listening. If the blueprint service is started after the org.vertx.osgi and modular verticle bundles, then the org.vertx.osgi bundle won't detect the modular verticle bundle's handler service appear in the service registry until the blueprint service has started, but then the end result will again be the same.

The github project contains the source for some sample modular verticles: a basic HTTP vertical (which runs on port 8090) and a sockjs verticle (which runs on port 8091). The org.vertx.osgi bundle needed more code to support sockjs and the modular sockjs verticle needed to provide a sockjs handler in addition to a HTTP handler.

Modularising BusMods

The MongoDB persistor is a typical example of a busmod which processes messages from the event bus:

public class MongoPersistor extends BusModBase implements Handler<message<jsonobject>> {

  private String address;
  private String host;
  private int port;
  private String dbName;

  private Mongo mongo;
  private DB db;

  public void start() {
    super.start();

    address = getOptionalStringConfig("address", "vertx.mongopersistor");
    host = getOptionalStringConfig("host", "localhost");
    port = getOptionalIntConfig("port", 27017);
    dbName = getOptionalStringConfig("db_name", "default_db");

    try {
      mongo = new Mongo(host, port);
      db = mongo.getDB(dbName);
      eb.registerHandler(address, this);
    } catch (UnknownHostException e) {
      logger.error("Failed to connect to mongo server", e);
    }
  }

  public void stop() {
    mongo.close();
  }

  public void handle(Message<jsonobject> message) {
    ...
  }

}

Again there is a mixture of boilerplate code (to register the event bus handler), start/stop logic, configuration handling, and the event bus handler itself. I applied a similar approach to the other verticles and separated out the boilerplate code into the org.vertx.osgi bundle leaving the handler and metadata (including configuration) in a modular busmod. The persistor's dependency on the MongoDB client JAR (mongo.jar) is convenient because this JAR ships with a valid OSGi manifest.

Here's the blueprint.xml:

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">
        
    <service ref="handler" interface="org.vertx.java.core.Handler">
        <service-properties>
            <entry key="type" value="EventBusHandler"/>
            <entry key="address" value="vertx.mongopersistor"/>
        </service-properties>
    </service>
        
    <bean id="handler" class="org.vertx.osgi.mod.mongo.MongoPersistor"
          destroy-method="stop">
        <argument type="java.lang.String"><value>localhost</value></argument>
        <argument type="int"><value>27017</value></argument>
        <argument type="java.lang.String"><value>default_db</value></argument>
    </bean>
      
</blueprint>

Notice that the boilerplate configuration consists of the handler type and event bus address. The other configuration (host, port, and database name) is specific to the MongoDB persistor.

Here's the modular MongoDB busmod code:

public class MongoPersistor extends BusModBase
                            implements Handler<Message<JsonObject>> {

    private final String host;

    private final int port;

    private final String dbName;

    private final Mongo mongo;

    private final DB db;

    public MongoPersistor(String host, int port, String dbName)
           throws UnknownHostException, MongoException {
        this.host = host;
        this.port = port;
        this.dbName = dbName;

        this.mongo = new Mongo(host, port);
        this.db = this.mongo.getDB(dbName);
    }

    public void stop() {
        mongo.close();
    }

    public void handle(Message<JsonObject> message) {
        ...
    }

}

The code still extends BusModBase simply because BusModBase provides several convenient helper methods. Again the resultant code is simpler and easier to unit test than the non-modular equivalent.

Modularising Event Bus Clients

Finally, I needed a modular verticle to test the modular MongoDB persistor. All this verticle needs to do is to post an appropriate message to the event bus. Normal vert.x verticles obtain the event bus using the Vertx class, but I used the Blueprint service again, this time to look up the event bus service in the service registry and inject it into the modular verticle. I also extended the org.vertx.osgi bundle to publish the event bus service in the service registry.

The blueprint.xml for the modular event bus client is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">

    <reference id="eventBus" interface="org.vertx.java.core.eventbus.EventBus"/>

    <bean class="org.vertx.osgi.sample.mongo.MongoClient">
        <argument ref="eventBus"/>
        <argument type="java.lang.String">
            <value>vertx.mongopersistor</value>
        </argument>
    </bean>
      
</blueprint>

Then the modular event bus client code is straightforward:

public final class MongoClient {

    public MongoClient(EventBus eventBus, String address) {
        JsonObject msg = ...
        eventBus.send(address, msg,
                      new Handler<Message<JsonObject>>(){...});
    }

}

Taking it for a Spin

1. I've made all the necessary OSGi bundles available in the bundles directory in git. You can grab them either by cloning the git repository:

git clone git://github.com/glyn/vert.x.osgi.git

or by downloading a zip of the git repo.

2. vert.x requires Java 7, so set up a terminal shell to use Java 7. Ensure the JAVA_HOME environment variable is set correctly. (If you can't get Java 7 right now, you'll see some errors when the bundles are deployed to OSGi and you won't be able to run the samples in steps 8 and 9.)

3. If you are an OSGi user, simply install and start the bundles in your favourite OSGi framework or container and skip to step 8. If not, then use the copy of the Virgo kernel in the git repository as follows.

4. Change directory to the virgo-kernel-... directory in your local copy of the git repo.

5. On UNIX, issue:

bin/startup.sh -clean

or on Windows, issue:

bin\startup.bat -clean

6. The Virgo kernel should start and deploy the various bundles in its pickup directory:

• org.vertx.osgi bundle (org.vertx.osgi-0.0.1.jar)
• HTTP sample modular verticle (org.vertx.osgi.sample.basic-1.0.0.jar)
• SockJS sample modular verticle (org.vertx.osgi.sample.sockjs-1.0.0.jar)
• MongoDB persistor sample modular busmod (org.vertx.osgi.mods.mongo-1.0.0.jar)

7. If you want to see which bundles are now running, start the Virgo shell from another terminal:

telnet localhost 2501

and use the ss or lb commands to summarise the installed bundles. The help command will list the other  commands available and disconnect will get you out of the Virgo shell. Here's typical output of the ss command:

...
89      ACTIVE      org.vertx.osgi_0.0.1
90      ACTIVE      jackson-core-asl_1.9.4
91      ACTIVE      jackson-mapper-asl_1.9.4
92      ACTIVE      org.jboss.netty_3.4.2.Final
93      ACTIVE      org.vertx.core_1.0.0.final
94      ACTIVE      org.vertx.osgi.mods.mongo_1.0.0
95      ACTIVE      com.mongodb_2.7.2
96      ACTIVE      org.vertx.platform_1.0.0.final
97      ACTIVE      org.vertx.osgi.sample.basic_1.0.0
98      ACTIVE      org.vertx.osgi.sample.sockjs_1.0.0

and of the lb command (which includes the more descriptive Bundle-Name headers):

   ...
   89|Active     |    4|vert.x OSGi Integration (0.0.1)
   90|Active     |    4|Jackson JSON processor (1.9.4)
   91|Active     |    4|Data mapper for Jackson JSON processor (1.9.4)
   92|Active     |    4|The Netty Project (3.4.2.Final)
   93|Active     |    4|vert.x Core (1.0.0.final)
   94|Active     |    4|MongoDB BusMod (1.0.0)
   95|Active     |    4|MongoDB (2.7.2)
   96|Active     |    4|vert.x Platform (1.0.0.final)
   97|Active     |    4|Sample Basic HTTP Verticle (1.0.0)
   98|Active     |    4|Sample SockJS Verticle (1.0.0)

8. You can now use a web browser to try out the basic HTTP sample at localhost:8090 which should respond "hello" or the SockJS sample at http://localhost:8091 which should display a box into which you can type some text and a button which, when clicked, produces a pop-up:

9. If you want to try the (headless) MongoDB event bus client, download MondoDB and start it locally on its default port and then copy org.vertx.osgi.sample.mongo-1.0.0.jar from the bundles directory to Virgo's pickup directory. As soon as this bundle starts, it will send a message to the event bus and drive the MongoDB persistor to update the database. If  you don't want to use MongoDB to check that an update was made, take a look in Virgo's logs (in serviceability/logs/log.log) to see some System.out lines like the following that confirmed something happened:

System.out Sending message: {action=save, document={x=y}, collection=vertx.osgi} 
... 
System.out Message sent 
...
System.out Message response {_id=95..., status=ok}

OSGi and vert.x Modularity

In this case study the various sample OSGi bundles all depend on, and share, the vert.x core bundle. Each bundle is loaded in its own class loader and OSGi controls the delegation of class loading and resource lookups according to how the OSGi bundles are wired together. In the same way, verticles written as OSGi bundles are free to depend on, and share, other OSGi bundles.

This is quite different from the vert.x module system in which any module (other than a busmod) which a verticle depends on is loaded into the same class loader as the verticle.

The advantages of the OSGi module system are that a single copy of each module is installed in the system and is visible to and may be managed by tools such as the Virgo shell. It also minimises footprint.

The advantages of the vert.x module system are that there is no sharing of modules between verticles so a badly-written module could not inadvertently or deliberately leak information between independent verticles. Also, there is a separate copy of each (non-busmod) module for each verticle that uses it and so the module can be written without worrying about thread safety as each copy will only be executed on its verticle's thread. OSGi users may, however, be happy to require reusable modules to be thread-safe and manage any mutable static data carefully to avoid leakage between threads.

Replacing the Container?

When I raised the topic of embedding vert.x in OSGi, the leader of vert.x, Tim Fox, asked me whether I was writing a replacement for the current container, to which I replied "not really". I said this because I liked vert.x's event driven programming model and its threading model, which seem to be part of "the container". But I was trying to replace a couple of aspects of the vert.x container: the module system and the way verticles register handlers.

Later it struck me that perhaps the notion of "the container" as a monolithic entity is a little odd in a modular system and it might be better to think of multiple, separate notions of containment which could then be combined in different ways to suit different users. However, the subtle interaction between the class loading and threading models seen above shows that the different notions of containment can depend on each other. I wonder what others think about the notion of "the container"?

Conclusions

vert.x's claim that it can be embedded in other applications is essentially validated since the OSGi framework is a fairly exacting application.

The vert.x module system, although not providing isolation between modules, does neatly provide isolation between applications (comprising verticles and their modules) and it enables modules to be written without paying attention to thread safety.

One vert.x issue was raised² which should make vert.x easier to embed in other environments with custom class loaders.

vert.x could follow the example of netty, jackson, and MongoDB JARs and include OSGi manifests in its core and platform JARs to avoid OSGi users having to convert these JARs to OSGi bundles. I will leave this to someone else to propose as I cannot gauge the demand for using vert.x inside OSGi.

Running vert.x in OSGi addresses some outstanding vert.x requirements such as how to automate in-container tests (OSGi has a number of solutions including Pax Exam while Virgo has a integration test framework) and how to develop verticles and deploy them to vert.x under control of the IDE (see the Virgo IDE tooling guide). Virgo also provides numerous ancillary benefits including the admin shell for inspecting and managing bundles and verticles, sophisticated diagnostics, and much more (see the Virgo white paper for details).

The exercise also had some nice spin-offs for Virgo. Bug 370253 was fixed which was the only known issue in running Virgo under Java 7. Virgo 3.5 depends on Gemini Blueprint which broke in this environment and so bug 379384 was raised and fixed. I used the new Eclipse-based Virgo tooling to develop the various bundles and run them in Virgo. As a consequence, I found a few small issues in the tooling which will be addressed in due course.

Finally, running vert.x on the Virgo kernel is a further validation that the kernel is suitable for building custom server runtimes since now we have vert.x in addition to Tomcat, Jetty, and one or two custom servers running on the kernel.

Footnotes:

1. I worked in the CICS development team in my IBM days. A colleague at SpringSource gave me a "CICS Does That!" T-shirt soon after we'd started working together. Old habits die hard.
2. The modular vertical currently needs to intercept vert.x's resource lookup logic so that files in the bundle can easily be served. It would be much better for this common code to move to the org.vertx.osgi bundle, but this requires vert.x issue 161 to be implemented first.

Temporary simplification

New software projects or products are often introduced which take an existing, successful piece of software and offer similar function, but in a much simpler way.

Sometimes there is a genuine improvement by capturing the gist of the requirements in a much more general way. Take git for example. If you've used certain older version control systems, git seems like a dream come true. This is because git's underlying design cracks the version control problem really neatly. Branching and merging is no longer a sweat. It just works.

But other times a new piece of software only appears simpler than an established alternative because it hasn't yet had time to meet all the requirements. I've noticed this mostly in applications. Newer versions of Apple mail have (useful) features deleted. Google appear to be systematically ruining their applications in the search for simplicity. This kind of temporary simplification where, as important features are added back over time, the result ends up being no simpler than the original, seems like a bit of a con to me.

I'd be interested in other people's views on this phenomenon. Is temporary simplification a good way of appealing to a new group of users or is it a failure to make a genuine design improvement?

Gemini Blueprint 1.0.1.M01

Gemini Blueprint 1.0.1.M01 is available for download. The main highlight is the support for Spring 3.1.x. A full list of bugs in the milestone is here.

Kudos to Aaron Whiteside for providing the vast majority of the changes for this milestone. He is in the process of becoming a committer for Gemini Blueprint.

Virgo 3.0.3.RELEASE available

The latest service refresh Virgo 3.0.3.RELEASE is available for download. Please see the release notes for details.

Virgo Experience at CME Group

At last week's EclipseCon, Jan Fetyko presented CME Group's experience of using Virgo. The slides make for an interesting read. The highlight for me was that they seemed pleased with snaps, although they found issues combining snaps and tiles. They haven't used the Virgo/Libra tooling so far, but I would hope that tooling is now sufficiently mature that they will be able to start using it soon.

The conclusions are interesting too:

• Did not hit any Virgo bugs (yet), stable, didn’t experience any crashes
• Problems only come from [application] code
• Virgo 3.x release improved memory consumption comparing to 2.1
• The learning curve is steep
• Design for modularity upfront is important
• Cannot go back to monolithic app

The last bullet really means they wouldn't want to go back to a monolithic app.

Thanks to Jan and CME Group for being so open about their activity.

Eclipse Virgo White Paper

I just published the first version of a white paper on Virgo. It summarises and pulls together much of the high-level technical information on the Virgo web site and should be a useful technical introduction to Virgo for newcomers.

Virgo and OSGi Subsystems

The OSGi Subsystems specification standardises the multi-bundle application constructs in various projects, including Virgo's plan and PAR files. The full spec will take quite a while to implement on Virgo, so as a tactical measure, we've implemented one of the main features of Subsystems which Virgo did not previously support: the ability for plans to share their contents and thus form a Directed Acyclic Graph (DAG) rather than a "forest" of trees.

Previously, if you deployed a plan which included artefacts that were already deployed, either on their own or as part of another plan, the plan would fail to deploy. With the DAG support, the plan will share the artefacts which were already deployed. The artefacts are undeployed when they are no longer needed.

Here's an example where two plans share some artefacts (B is a plan with a single child artefact C):

Artefacts Shared Between Plans

This is the key functional change in Virgo 3.5.0.M03 which is available for download today. See the release notes for more details.

Meanwhile the Subsystems spec is available as a draft and should be finalised in the next month or so.

CME Group and Virgo

Today CME Group released an official statement about their use of Virgo. This is posted under the "Virgo Powered" section of the Virgo home page, but the full text is also reproduced below.

I'm delighted both that CME Group are using Virgo and that such a large player in the finance area was happy to go public.

CME Group, formerly known as Chicago Mercantile Exchange Holdings Inc., was founded in 1898 and operates the CME, CBOT, NYMEX, and COMEX regulatory exchanges worldwide. It is the world’s leading derivatives marketplace with 2570 full time employees and a market capitalisation of over $15B.

CME Group kindly released the following information in January 2012.

At CME Group, the goals for our project were to use and create a platform that would be easy to maintain, expand and reuse. Our applications are of various sizes and complexities, but there are certain aspects that are common, which was the reason to find a very modular solution. In particular, our current efforts are in Surveillance tools which allow monitoring the state of the exchange and the market.

We started using Virgo 2.0.1 during a “proof of concept” in November of 2010 and spent 2011 working closely with them to build the application. Today we use version 3.0.1. with 48 service bundles and 20 web bundles. These services range from simple DAO type services, through authorization and authentication, web services, and even distributed caching. All web bundles use snaps and customized apache tiles views.

What Virgo and OSGi provides for us is complete modularity and plug-and-play capability. This enables us to work more intelligently with other teams who need to build bundles independently of us where the strong isolation of services and APIs allows for a clear separation of control. We have created a core set of services that will be a platform for future projects across CME Group and are already actively building two projects on top of that platform. Having Virgo bundled with Apache’s Tomcat also helped in deciding which container to use as our projects are mostly web applications.

Virgo Nano Technology

The first milestone of Virgo 3.5.0 is available. It introduces two significant new features: p2 support (covered previously) and a new Virgo distribution known as nano. (Apologies for the pun in the blog title - it was too hard to resist.)

Nano is essentially a cut-down version of Virgo which starts up really fast and has a single (kernel) region. It is intended for simpler scenarios where you don't need to isolate applications from each other or from the kernel.

The current nano distribution includes Gemini Web and so is a fully-functional OSGi web server which starts in about 3 seconds (compared to Virgo Tomcat/Jetty Server which take about 9-10 seconds). However, we plan to factor out Gemini Web so that nano becomes a small/fast subset of the Virgo kernel. We will then provide a separate distribution which adds Gemini Web to nano, as in the first milestone.

As well as introducing nano, the milestone re-bases the kernel and the Tomcat/Jetty servers on nano. Initial provisioning via p2, with full instructions in the user guide, is provided for all distributions in addition to the familiar ZIP file installs (the ZIP contents are now constructed using p2 at build time). All distributions use a regular Equinox directory layout and launcher which will simplify third party tooling integration (e.g. Pax Exam). A short-term downside is that we need to update the Virgo IDE tooling to work with Virgo 3.5.0.

The current nano distribution can also dynamically provision content via p2 which is particularly useful for automated or cloud deployments. (We tried to implement this for the multi-region kernel and Tomcat/Jetty servers, but that was beyond the scope of the current p2 design, so we've put it on hold.)

There is also a nice engineering side-effect of the introduction of nano: we are starting to refactor the kernel — by far the most complex Virgo component — and move the more general, region-agnostic features down into nano.

The following picture summarises the distributions we anticipate in Virgo 3.5.0:

As well as being the fruit of several months of effort, this first milestone kicks off Virgo development in 2012 in a very exciting way.

OSGi testing with Gradle and Pax Exam

I've recently been working on a Spring issue which involved writing a Gradle-built test to ensure that the Spring framework 3.1.x bundles successfully resolve in an OSGi container.

The Gradle side of this was a very simple and pleasant experience, as the following extract of the file build.gradle shows:

apply plugin: 'java'
apply plugin: 'eclipse'

repositories {
    mavenCentral()
}

dependencies {
    ...                
}

There were minor usability issues¹ with the Gradle Eclipse plugin, but I gather these will soon be resolved.

I decided to use Pax Exam to launch the testcase inside Equinox, and this require specifying some dependencies in build.gradle:

dependencies {
    paxExamVersion = '2.3.0.M1'
    equinoxVersion = '3.6.0.v20100517'
    
    testCompile "junit:junit:4.+",
                "org.ops4j.pax.exam:pax-exam-junit4:$paxExamVersion",
                "org.eclipse.osgi:org.eclipse.osgi:$equinoxVersion",
                "javax.inject:javax.inject:1"
                
    testRuntime "org.ops4j.pax.exam:pax-exam-container-native:$paxExamVersion",
                "org.ops4j.pax.exam:pax-exam-link-mvn:$paxExamVersion",
                "org.ops4j.pax.url:pax-url-aether:1.3.5"
}

Let's go through the dependencies in turn:

• JUnit 4 is the test framework.
• @RunWith(JUnit4TestRunner.class) marks the test for launching under Pax Exam.
• Equinox provides OSGi types at compile time and an OSGi container at runtime.
• A bundle context is injected into the test using @javax.inject.Inject.
• The test uses the native Pax Exam container - all the tests run in a single JVM.
• The test uses Pax Exam to install Spring bundles directly from Maven repositories.²
• The test caches bundles from the SpringSource Enterprise Bundle Repository using the Pax URL Mvn protocol.

The Pax Exam developers kindly helped me sort a basic setup issue which wasn't too obvious from the documentation. Thanks Harald and Toni! I needed to explicitly install a JUnit bundle, by calling junitBundles from the configuration method:

    @Configuration
    public static Option[] configuration() throws Exception {
        return options(//
            provisionSpringBundle("spring-core"), //
            provisionSpringBundle("spring-beans"), //
            // etc.
            junitBundles());
    }

The full source of the project is available on github (under the Apache 2.0 license) for anyone who wants to give it a spin or use it for their own purposes. If you'd prefer to write your tests in Groovy, take a look at Hamlet D'Arcy's blog.

Footnote:
1: The generated .classpath contains absolute paths rather than paths relative to the project and so is not suitable for committing and sharing with others. Also re-running gradle eclipse without first deleting .classpath can result in it containing duplicate paths. Luke Daley tells me that both these problems will soon be fixed, the latter fix appearing in gradle 1.0 milestone 7.
2: If the testcase was intended a a stand-alone test of published versions of Spring, it would cache Spring bundles in the local Maven repository for efficiency. But we plan to integrate the testcase into the Spring build in due course and so we'll want to grab the latest built version each time to cope with re-spins, so cacheing isn't helpful.

Virgo 3.0.2 released

Virgo 3.0.2 is available for download. Please see the release notes for details of this bug fix release.

Opening SourceTree from the command line

I've recently started using SourceTree to visualise and manage my git repos. It's very impressive.

The one thing I missed from gitx was the ability to launch SourceTree from the command line, in any directory of the repo. One solution is to set a git alias:

git config --global --add alias.sourcetree '!open -a SourceTree .'

and add an alias in the shell profile:

alias st='git sourcetree'

Now I can issue "st" from the command line and the relevant SourceTree window opens up.

(Thanks to Chris Frost for providing part of this solution.)

What did we learn in 100 sprints?

This week marks the 100th Sprint on Eclipse Virgo! That includes the precursor project, SpringSource dm Server. What have we learned about scrum? Well, obviously, it's pretty good otherwise we would have ditched it long ago (and believe me I've been tempted a couple of times). But specifically...


Firstly, that one week sprints are by far the easiest to manage, both in terms of planning effort, but also psychologically. Even in a two week sprint, it's easy to put off unpleasant tasks until it's too late.


Secondly, that the main value of scrum is to keep the team focussed on specific tasks during a sprint and enable progress to be shared easily at daily stand-up meetings.


Thirdly, that story points are more effort than they are worth, so it's more effective to go with gut feel.


Fourthly, that you have to take a relaxed attitude towards planning design work as it's almost impossible to predict the effort up front. Spikes are ok for small items, but sometimes you just need plenty of time to get your head round a large area.

Enterprise Bundle Repository Samples

After opening up the Enterprise Bundle Repository build system, I've added more sample bundlor templates along with Ant/Ivy build files to github.

So while we are trying to come up with a community-based successor to the EBR, if you need to get a new version of one of the bundles in the EBR, you may be able to find a bundlor template to use as a starting point.

Latest draft subsystems spec

The OSGi Alliance has made an early draft of some RFCs available. This includes the subsystems RFC 152 I blogged about earlier: see pages 72-144 of the PDF.

Eclipse Dance Steps

Sometime Eclipse fails to work as expected. The following 'dances', in increasing order of desperation, may help. Remember to maintain a fixed grin throughout and try not to tread on your own toes.

Refresh and clean all projects ("Quickstep")

Click on a project in the Package Explorer. Hit Apple-A (or Ctrl-A) to select all projects. Hit F5 to refresh. Select Project->Clean..., select "Clean all projects", and click Ok.

Open and close all projects ("Foxtrot")

Click on a project in the Package Explorer. Hit Apple-A (or Ctrl-A) to select all projects. Right click the projects and select "Close Project". Right click the projects again and select "Open Project".

Open and close all projects restarting Eclipse in between ("Tango")

Click on a project in the Package Explorer. Hit Apple-A (or Ctrl-A) to select all projects. Right click the projects and select "Close Project". Restart Eclipse. Click on a project in the Package Explorer. Hit Apple-A (or Ctrl-A) to select all projects. Right click the projects again and select "Open Project".

Delete and re-import all projects ("Paso Doble")

Click on a project in the Package Explorer. Hit Apple-A (or Ctrl-A) to select all projects. Press the delete key, select "Do not delete contents", and click Ok. Right click in the Package Explorer and select Import..., then choose General and "Existing Projects into Workspace" and click Next. Browse to select the platform checkout directory and click Choose. Select all the projects for import. Click Finish.

Rebuild the search indices ("Slow Waltz")

Exit the workbench and delete the Java search index:

• go into \plugins\.metadata\.plugins\org.eclipse.jdt.core
• delete 'savedIndexNames.txt'
• delete all *.index
  Restart

If all else fails

If you're still stuck, try the "Hokey Cokey" by repeating all of the above until you've had enough.

How Infor ION Uses Virgo

Infor have kindly made available a presentation on how their ION integration product is based on the Virgo kernel. They migrated from Equinox 3.5 to Virgo 2.1, added a lightweight web server (in preference to using the pre-built Virgo Tomcat or Jetty servers) and a service wrapper, added Spring integration, upgraded Spring framework, added a watched repository, and then added various enterprise service bundles. This is a perfect example of a Virgo kernel application, a so-called OSGi stackless stack.

The Infor team also helpfully pointed out some enhancements they'd like to see, some of which are already supported or planned for Virgo 3.5.

According to their website Infor is the world's third largest provider of enterprise applications and services, so it's nice to see another large company adopt Virgo.

Thanks to Ian Skerrett for bringing this user to my attention. See his blog for more.

Extenders: Pattern or Anti-Pattern?

The extender pattern has become popular in recent years and has even been utilised in OSGi standards such as the Blueprint service and the Web Applications specification. In Virgo, we've been working with extenders from the start, but in spite of their advantages, they have some significant downsides. Since the OSGi Alliance is considering using extenders in other specifications, I agreed to document some of the issues.

The first difficulty is knowing when an extender has finished processing a bundle. For example, a bundle containing a blueprint XML file will transition to ACTIVE state as soon as any bundle activator has been driven. But that's not the whole story. Administrators are interested in when the bundle is ready for use and so the management code in Virgo tracks the progress of the extender and presents an amalgamated state for the install artefact representing the bundle. The install artefact stays in STARTING state until the application context has been published, at which point it transitions to ACTIVE. Without such additional infrastructure, administrator cannot tell when a bundle processed by an extender really is ready for business.

That's the successful case, but there are complications in error cases too. The first complication is that since an extender runs in a separate thread to that which installed the bundle, if the extender throws an exception, this is not propagated to the code which installed the bundle. So the installer needs somehow to check for errors. Therefore Virgo has infrastructure to detect such errors and propagate them back to the thread which initiated deployment of the bundle: the deployment operation fails with a stack trace indicating what went wrong.

The other error complication is where there is a (possibly indefinite) delay in an extender processing a bundle. For this kind of error Virgo tracks the progress of extender processing and issues warnings to the event log (intended for the administrator's eyes) saying which pieces of processing have been delayed and in some common situations, for example when a blueprint is waiting for a dependency, what is causing the delay.

Extenders suffer from needing to be able to see bundle lifecycle events and so for systems that partition the framework, it is necessary to install each extender into multiple partitions. On the flip side it is crucial to prevent multiple instances of an extender from ever seeing the same bundle event otherwise they will both attempt to extend the bundle.

Another issue with extenders is the need to keep them running and healthy as there is little indication that an extender is down or sickly other than bundles not being processed by the extender. Virgo takes care to ensure its extenders are correctly started and its infrastructure for detecting delays helps to diagnose extender crashes or sickness (both of which are extremely rare situations).

There is also an issue in passing parameters to an extender to affect its behaviour. This is typically done by embedding extender configuration in the bundles being processed or by attaching a fragment containing configuration to the extender bundle. But since the extender is not driven by an API, the normal approach of passing parameters on a call is not available. Essentially, an extender model implies that the programming model for deployment is restricted to BundleContext.installBundle.

With considerable investment in additional infrastructure, Virgo has managed to support the Blueprint and Spring DM extenders reasonably well. But in the case of the Web Applications extender, Virgo couldn't make this sufficiently robust and so it drives the underlying web componentry directly from the Virgo deployment pipeline to avoid the above issues.

I understand at least one other server runtime project has encountered similar issues with extenders, so Virgo is not alone. There is a trade-off between loosely coupling the installer from the resource-specific processing, the main strength of the extender pattern (but far from unique to that pattern), and providing a robust programming model and usable management view -- crucial features of a server runtime -- which is far more straightforward without extenders.

OSGi Subsystems in Virgo

A public draft(*) of the OSGi subsystems RFC (152) should soon emerge from the OSGi Alliance. A subsystem is a multi-bundle application, not dissimilar to a PAR or plan in Virgo. IBM is leading the spec work and a number of other vendors, including SpringSource/VMware, are contributing. Quite a few projects have multi-bundle application constructs, so it makes sense to agree a standard form.

After going through the RFC again with my implementer's hat on, I listed the features necessary to support subsystems in Virgo. Most of the changes are in the Virgo kernel, although I hope to structure the support into non-subsystem specific generalisations of the kernel plus subsystem-specific code running in the user region.

Currently, it is not possible to deploy a plan which contain artefacts that are already deployed. This will need generalising so that we can support subsystems using a common data structure of deployed artefacts: a directed acyclic graph (DAG) rather than the current collection of trees.

The switch to a DAG has interesting implications for lifecycle management of shared subgraphs. With today's tree, when a node is stopped, started, or uninstalled, any subtrees are also stopped, started, or uninstalled, respectively. With a DAG, shared subgraphs need to be sensitive to all their parents. This boils down to keeping each shared subgraph at the maximum state required by any of its parents. States are ordered: ACTIVE > RESOLVED > UNINSTALLED.

Lifecycle management will also get interesting when a shared subgraph belongs to one or more atomic subgraphs as then lifecycle changes in the common subgraph will propagate to all the containing atomic subgraphs. I think that will just "work", but users might need to be careful in their use of atomic plans if they want to avoid management operations on one application affecting other applications.

I'm also considering using garbage collection as a means of uninstalling artefacts which are no longer needed. Given the number of types of dependency that are possible, this is likely to be more reliable than the alternative of maintaining reference counts.

I'm a little concerned about a possible race between garbage collection detecting an artefact as dead and a new dependency being created on the artefact just before garbage collection goes ahead and uninstalls the artefact, but there would be a similar concern for reference counting. The basic issue in this race is that a dead artefact may be found by a live bundle and a new dependency created before the dead artefact can be uninstalled. For instance, a dead bundle may be found by using the OSGi API to list all bundles. It may be possible to use some technique such as a special region in the region digraph to isolate dead bundles, although this issue is probably something to discuss among those working on the RFC.

Anyway, there's plenty of work to be getting on with. I haven't done detailed estimates of the features identified so far, but I guess there's a person year or so of effort needed, so I'm initially targeting Virgo 4.0. If you feel like lending a hand, please get in touch on virgo-dev.

* - RFC 152 has changed quite a bit since the version in the Enterprise spec early access draft dated 16 May 2011. A new draft is being prepared.