My holiday reading is a fascinating book by Peter Woit entitled "Not Even Wrong" which argues that string theory has failed. There was quite a backlash from certain outspoken string theorists, but no-one appears to have rebutted Woit's specific criticisms.
The gist of the book's argument is that string theory isn't really a theory at all as it makes no experimentally falsifiable predictions and it doesn't yet have an underlying mathematical foundation. It's really a highly fashionable piece of wishful thinking about where the next fundamental theory of physics might come from. I can think of other fashion industries closer to home, but that's a topic for the future.
I think the crucial question is how long a speculative search for a new physical theory should continue before it makes a prediction that can be checked experimentally. Perhaps string theory will ultimately deliver something useful, but, in the interim, other directions are effectively being starved of resources.
The book mentions in passing that the Massachusetts Institute of Technology (MIT) is now publishing its courses online under an initiative known as "Open Course Ware". If you want an easy example of a computing course, take a look at the lecture notes on module dependence diagrams, given by the Department of Electrical Engineering and Computer Science.
I've been investigating a mathematical model of software modules recently and some of the dependency relations look like they would benefit from a bit of graph theory, so I was delighted to find some mathematics course notes and even a link to a softcopy graduate textbook on graph theory via the MIT site.
Unfortunately, the graduate course notes are handwritten and then scanned in, so would need more effort to explore. However, since I'm a novice at graph theory, I was pleased to find some undergraduate material nicely captured in PDF files.
Saturday, December 30, 2006
Tuesday, December 12, 2006
AOP meets OSGi - phase II
Thursday, November 16, 2006
Experimental fix for sunbug 4670071
The Hotspot VM has a long-standing class loader deadlock, sunbug 4670071, which is top of Sun's bug parade. When the VM loads a class, it calls the
This is a particular problem for networks of custom class loaders in JBoss, OSGi, JSR 277 (see section 8.3.4 of the EDR spec.), and elsewhere.
Sun have implemented an experimental fix, which I'll come to shortly. The point of describing the fix here is to make the caveats clear and to solicit feedback which can be taken into account in future releases of Hotspot. I, as a member of the JSR 277 Expert Group, am talking to the engineers in Sun who are looking at potential class loading improvements for Dolphin.
So what's the fix? Well, it turns out you need to specify some VM options:
Please note that I am not suggesting that this is a robust fix to bug 4670071. You must treat it as unstable, experimental, temporary, unsupported, etc. In particular a custom class loader's
However, it's not all bad news: the fix seems to work in some cases. For example, Equinox supports unsynchronized calling of
loadClass method of java.lang.ClassLoader via a private, synchronized loadClassInternal method. The result is that multiple threads using multiple class loaders can deadlock.This is a particular problem for networks of custom class loaders in JBoss, OSGi, JSR 277 (see section 8.3.4 of the EDR spec.), and elsewhere.
Sun have implemented an experimental fix, which I'll come to shortly. The point of describing the fix here is to make the caveats clear and to solicit feedback which can be taken into account in future releases of Hotspot. I, as a member of the JSR 277 Expert Group, am talking to the engineers in Sun who are looking at potential class loading improvements for Dolphin.
So what's the fix? Well, it turns out you need to specify some VM options:
-XX:+UnlockDiagnosticVMOptions -XX:+UnsyncloadClassPlease note that I am not suggesting that this is a robust fix to bug 4670071. You must treat it as unstable, experimental, temporary, unsupported, etc. In particular a custom class loader's
loadClass method must be threadsafe and findClass should be synchronized to rule out various races in the VM.However, it's not all bad news: the fix seems to work in some cases. For example, Equinox supports unsynchronized calling of
loadClass. This is the default behaviour in the IBM JDK and can occur with Hotspot when loadClass is called directly from Java code (rather than implicitly by the VM while loading another class). Equinox's DefaultClassLoader synchronizes findClass.
Wednesday, November 15, 2006
Early release of IBM's Java 6
The Early Release Programme for IBM's Java SE 6 on a number of Linux platforms just began.
Apart from the functional enhancements inherent in Java 6 and the additional performance and RAS features we put into this release, there were significant changes in the development process.
Of particular note is the way we handled the class library code licensed from Sun. In Java 5 and other recent releases, we carried forward a base from release to release and merged in Sun's changes -- fairly labour intensive. This time we introduced a much more efficient process.
We developed a system to capture the IBM changes to Sun code such as bug fixes, performance improvements, and support for IBM platforms. Each logical group of changes is captured as a "smart patch" based on a UNIX patch file. The system automatically applies all the smart patches to each new drop of Sun code. Manual intervention is only required where the Sun code has changed significantly in an area covered by a smart patch.
The net effect is that we have been able to track the development of Sun's Java 6 much more closely than in the past. Also, the changes we make have been separated out in to logical, manageable units.
I'm hoping to contribute the smart patch system into open source as it would be useful to any project that needs to apply its own changes to a moving base.
Apart from the functional enhancements inherent in Java 6 and the additional performance and RAS features we put into this release, there were significant changes in the development process.
Of particular note is the way we handled the class library code licensed from Sun. In Java 5 and other recent releases, we carried forward a base from release to release and merged in Sun's changes -- fairly labour intensive. This time we introduced a much more efficient process.
We developed a system to capture the IBM changes to Sun code such as bug fixes, performance improvements, and support for IBM platforms. Each logical group of changes is captured as a "smart patch" based on a UNIX patch file. The system automatically applies all the smart patches to each new drop of Sun code. Manual intervention is only required where the Sun code has changed significantly in an area covered by a smart patch.
The net effect is that we have been able to track the development of Sun's Java 6 much more closely than in the past. Also, the changes we make have been separated out in to logical, manageable units.
I'm hoping to contribute the smart patch system into open source as it would be useful to any project that needs to apply its own changes to a moving base.
Monday, November 06, 2006
More on JSR 277
InfoQ joined the fray. Stanley Ho's latest post drew a few comments.
Murphee pipped Mike Milinvokich and Alex Blewitt in the prize for least understated. Compare Robilad.
Murphee pipped Mike Milinvokich and Alex Blewitt in the prize for least understated. Compare Robilad.
Friday, November 03, 2006
Public discussion of JSR 277
I'm pleased to see some public feedback.
The net of the negative feedback, e.g. on JavaLobby, seems to be "why re-invent aspects of OSGi/Maven/Ivy?". Peter Kriens' review describes some technical issues previously encountered and solved by OSGi.
A relatively positive review asks why the Expert Groups of JSR 277 and JSR 291 didn't work more closely together. BEA, IBM, Intel, Oracle, Richard (Felix) Hall, and Sun are members of both Expert Groups, so lack of working closely together isn't the reason the results are turning out rather different.
JSR 277, with the help of JSR 294, is aiming to produce a static module system integrated into the Java SE 7 platform. JSR 291 on the other hand aims to support dynamic components on existing Java SE platforms in a way which is compatible with JSR 232 on Java ME.
Looking at it another way, JSR 277/294 are attempting to produce an evolution of java.lang.ClassLoader with a distribution format, repositories, language, and VM support. JSR 291 on the other hand can be implemented in pure Java on top of java.lang.ClassLoader.
The net of the negative feedback, e.g. on JavaLobby, seems to be "why re-invent aspects of OSGi/Maven/Ivy?". Peter Kriens' review describes some technical issues previously encountered and solved by OSGi.
A relatively positive review asks why the Expert Groups of JSR 277 and JSR 291 didn't work more closely together. BEA, IBM, Intel, Oracle, Richard (Felix) Hall, and Sun are members of both Expert Groups, so lack of working closely together isn't the reason the results are turning out rather different.
JSR 277, with the help of JSR 294, is aiming to produce a static module system integrated into the Java SE 7 platform. JSR 291 on the other hand aims to support dynamic components on existing Java SE platforms in a way which is compatible with JSR 232 on Java ME.
Looking at it another way, JSR 277/294 are attempting to produce an evolution of java.lang.ClassLoader with a distribution format, repositories, language, and VM support. JSR 291 on the other hand can be implemented in pure Java on top of java.lang.ClassLoader.
Wednesday, November 01, 2006
OSGi introduction
Some of my colleagues need to develop OSGi bundles and run them on Equinox. They are new to OSGi, so they asked me for some introductory reading. I thought I'd post it here for posterity and more general consumption.
I recommend starting with the technology overview on the OSGi web site, but skipping the sections on specific services on the first reading.
After reading this, you should have a feel for the three major layers in the OSGi framework.
The modularity layer enables you to package Java code with some metadata into a JAR known as a bundle, which is OSGi's module concept. Each bundle is loaded in its own class loader. Bundles share classes in a couple of ways. A bundle may import a package that another bundle exports or a bundle may require another bundle, which is similar to importing all the packages that the other 'required' bundle exports. Bundles may also contain native libraries and arbitrary kinds of resource files.
The life cycle layer allows you to dynamically install, uninstall, start, stop, and update bundles. This is essential when the OSGi framework needs to run for a long time but applications need to be managed, e.g. taken offline or have a bug fix applied, without restarting the framework, which would impact all the applications running in the framework.
The service layer enables bundles to provide services to other bundles. A service is a Java object that implements an agreed interface. Services are registered in a service registry so that users can locate the service they need, cast it to the agreed service interface, and then call the service.
If you would like to re-enforce these concepts or get a bit more detail, take a look at the architecture description.
After that, you could read about an example application, you could dip into the core specification, or you may prefer to try developing and running a simple bundle using one of the open source OSGi framework implementations. For example, the Equinox OSGi framework comes with a quick start guide.
Finally see this guide to writing a trivial bundle in 10 minutes with no special tools. Once you get beyond a "hello world" bundle, you'll want to refer to the javadoc on the OSGi web site.
I recommend starting with the technology overview on the OSGi web site, but skipping the sections on specific services on the first reading.
After reading this, you should have a feel for the three major layers in the OSGi framework.
The modularity layer enables you to package Java code with some metadata into a JAR known as a bundle, which is OSGi's module concept. Each bundle is loaded in its own class loader. Bundles share classes in a couple of ways. A bundle may import a package that another bundle exports or a bundle may require another bundle, which is similar to importing all the packages that the other 'required' bundle exports. Bundles may also contain native libraries and arbitrary kinds of resource files.
The life cycle layer allows you to dynamically install, uninstall, start, stop, and update bundles. This is essential when the OSGi framework needs to run for a long time but applications need to be managed, e.g. taken offline or have a bug fix applied, without restarting the framework, which would impact all the applications running in the framework.
The service layer enables bundles to provide services to other bundles. A service is a Java object that implements an agreed interface. Services are registered in a service registry so that users can locate the service they need, cast it to the agreed service interface, and then call the service.
If you would like to re-enforce these concepts or get a bit more detail, take a look at the architecture description.
After that, you could read about an example application, you could dip into the core specification, or you may prefer to try developing and running a simple bundle using one of the open source OSGi framework implementations. For example, the Equinox OSGi framework comes with a quick start guide.
Finally see this guide to writing a trivial bundle in 10 minutes with no special tools. Once you get beyond a "hello world" bundle, you'll want to refer to the javadoc on the OSGi web site.
Saturday, October 28, 2006
Joel Spolsky podcast
Just listened to an interesting interview with Joel Spolsky of Joel on Software fame. He hires competent developers to work on some project. As the real needs of customers emerge, the project is adjusted or new projects are spawned. So his approach is to start with technology and "lead" customers, but then listen hard for their feedback and react accordingly. Very sensible.
Tuesday, October 17, 2006
AOP meets OSGi
The Aspects Equinox incubator project just announced its first release.
Now you can write AspectJ aspects, package them as OSGi bundles, and apply them to bundles which explicitly "opt in" to these aspects. The "co-opt" model is coming soon and will enable aspects to be applied to unmodified bundles (without them having to opt in).
Now you can write AspectJ aspects, package them as OSGi bundles, and apply them to bundles which explicitly "opt in" to these aspects. The "co-opt" model is coming soon and will enable aspects to be applied to unmodified bundles (without them having to opt in).
Friday, October 13, 2006
JSR 277 draft spec.
JSR 277 has produced an Early Draft Review specification which I hope the Java community will review carefully. As a member of the Expert Group that produced the spec., I don't think it would be appropriate to express my opinion here, much as I would like to.
Monday, October 09, 2006
Monday, October 02, 2006
OSGi for the enterprise?
The OSGi Alliance recently held a workshop on applying OSGi to the enterprise. You might find it surprising that a technology initially created for small devices is starting to be used on large enterprise systems such as application servers.
But businesses need to keep their IT costs down, especially price/performance. Salaries for application developers, operations staff, etc. can be reduced by using languages such as Java and standardised middleware. But if that impacts performance, then price/performance suffers.
OSGi provides good support for Java modularity, versioning, and lifecycle management but is also efficient because of its roots in small devices. Modularity helps keep development and maintenance costs down; versioning and lifecycle management reduce operational costs.
So, apart from the coolness of OSGi technology, I would expect its price/performance to appeal to middleware vendors and their enterprise customers. I would also expect customers to see value in using OSGi directly, e.g. in conjunction with Spring.
But businesses need to keep their IT costs down, especially price/performance. Salaries for application developers, operations staff, etc. can be reduced by using languages such as Java and standardised middleware. But if that impacts performance, then price/performance suffers.
OSGi provides good support for Java modularity, versioning, and lifecycle management but is also efficient because of its roots in small devices. Modularity helps keep development and maintenance costs down; versioning and lifecycle management reduce operational costs.
So, apart from the coolness of OSGi technology, I would expect its price/performance to appeal to middleware vendors and their enterprise customers. I would also expect customers to see value in using OSGi directly, e.g. in conjunction with Spring.
Thursday, September 21, 2006
JRE provisioning
Ethan Nicholas is experimenting with a browser edition of Java. I've also been thinking about JRE provisioning: installing the core of a Java runtime and having it download and install additional pieces as required.
To work well, this needs a modularised JRE - a JRE which has been broken into 'modules' with well-defined dependencies. This is happening in Apache Harmony1 where modules are defined using OSGi metadata, as in this example of the NIO module:
In addition to the basic mechanism, the Java standard may need tweaking to define what is supposed to happen when provisioning fails because of network errors.
Finally, for best results, the JRE modules would need carefully layering to ensure the necessary machinery, such as network access, is available before the first download is initiated and to avoid dragging in essentially unnecessary modules too early. This could take a lot of work since existing JREs aren't layered this way. However, it should be possible to demonstrate reasonable results without having to rework the JRE modules significantly.
It'll be interesting to see how Ethan gets on. Can he do without a proper module system or will he, for example, try to use the new Java module system, JSR 277, which is also planned for Java 7?
Footnotes:
To work well, this needs a modularised JRE - a JRE which has been broken into 'modules' with well-defined dependencies. This is happening in Apache Harmony1 where modules are defined using OSGi metadata, as in this example of the NIO module:
Manifest-Version: 1.0Then a provisioning mechanism is needed. In OSGi, dependencies are usually resolved statically. Before a module is executed, each of its static dependencies is resolved to some module which satisfies the dependency. For provisioning, dynamic dependencies are required and, fortunately, OSGi supports these. During class loading the relevant dynamic dependency, such as the need for a particular package, is resolved. This suits provisioning nicely: during class loading if the imported package is not installed, the corresponding module can be downloaded and installed and then the dependency can be resolved. Needs a bit of work, but not very much.
Bundle-ManifestVersion: 2
Bundle-Name: Harmony NIO
Bundle-SymbolicName:org.apache.harmony.nio
Bundle-Version: 1.0.0
Bundle-ClassPath: .
Import-Package:java.io, java.lang, java.lang.ref, java.net,
java.nio.charset, java.security, java.util,…
Export-Package:java.nio, java.nio.channels,
java.nio.channels.spi, org.apache.harmony.nio
In addition to the basic mechanism, the Java standard may need tweaking to define what is supposed to happen when provisioning fails because of network errors.
Finally, for best results, the JRE modules would need carefully layering to ensure the necessary machinery, such as network access, is available before the first download is initiated and to avoid dragging in essentially unnecessary modules too early. This could take a lot of work since existing JREs aren't layered this way. However, it should be possible to demonstrate reasonable results without having to rework the JRE modules significantly.
It'll be interesting to see how Ethan gets on. Can he do without a proper module system or will he, for example, try to use the new Java module system, JSR 277, which is also planned for Java 7?
Footnotes:
- For more detail, see this presentation, slide 23 and following.
Tuesday, September 19, 2006
Mind the gap
In software, things you haven't thought of cause the most trouble. 'Underlaps' are much worse than overlaps. "Mind the gap!" as they say on the London Underground.