OSGi: Lucee’s white whale or white knight?

Lucee 5 server platform is built on an OSGi architecture using the Apache Felix framework. The Lucee Core jar file is relatively small and all principal feature sets of the CFML language are isolated as individual extensions which in turn reference OSGi bundles; for example, the Hibernate powered ORM features.

Lucee Extensions & OSGI Bundles

The problem with monolithic JARs

Earlier versions of Lucee (and Railo code base it was forked from) were built on a more monolithic architecture amalgamating all features into a single JAR. Lucee 4.5 developers need to coordinate the use of any JAR based library file across all features; that is, if you want to update any particular java library you have to be mindful of every feature that leverages that library.

For example, log4j is a common JAR library that can not be updated on Lucee 4.5 without first addressing that upgrade across every feature that uses it — in this case almost everything.

When you work on any piece of functionality you have to worry about how the dependencies are going to ricochet around the rest of the code base; a simple update can lead to a chase down a never ending rabbit hole :rabbit2: :hole:

A case for hunting the white whale

The core benefit of the OSGi approach is the ability to leverage different versions of the same library within the same Java application. For example, if you wanted to upgrade ORM, the underlying Hibernate libraries and all their dependencies you can do so independent of any other feature. Or if you wanted to leverage two different extensions for say Word Doc and Excel manipulation you could do so using different versions of Apache POI .

The nightmare dependency cascade we used to have with respect to Java libraries in Lucee 4.5 is no longer an issue in Lucee 5.x.

As a consequence, development work on a feature like ORM can be externalised to a party that has no part in the core lucee.jar development. The ORM Hibernate-based extension can be developed independently and in parallel. Users can upgrade to the “latest Hibernate” and trial more advanced ORM features by updating the specific ORM extension. This allows for more robust testing and trialling of advanced features with specific focus groups ahead of making the extension part of the general Lucee distribution. The extension could even be distributed independent of the normal Lucee release cycle.

A significant part of the initial Lucee 5 development effort was breaking up the old monolithic architecture into a more modular, extensions based architecture. Now that the OSGi framework is in place, more features are spun out of the Lucee core library every release and externalised as extensions and bundles. And additional features can be added as extensions without fear of Java library conflicts.

The move to OSGi was motivated by the desire for architectural change in the underlying Lucee core; to make it more modular and manageable. This was not motivated by a desire to enable OSGi features in user applications built on top of Lucee server.

OSGi interoperability for apps is a happy consequence of these architectural changes.

Knock-on effects

Moving to OSGi was a Herculean effort, and the community is right to be curious about whether or not the approach was worth the investment of resources. There are several important points that should not be overlooked in this assessment.

A tremendous amount of technical debt was addressed in pulling apart and re-assembling the Lucee server code base. It’s unlikely such an effort would or could have been made without the impetus of the OSGi architectural changes.

Given the modular nature of extensions and bundles, LAS are better positioned to expand development resources independent of an intimate knowledge of the inner-workings of the Lucee core. One of the greatest bottlenecks to accepting Pull Requests and submissions has been the unintended consequences of changes to other areas of the platform.

Anyone who has worked on an extension can attest to the improved flexibility of the OSGi framework. We’re working hard to encourage contributors to adopt extensions in their area of expertise instead of having to wrangle with the entirety of the Lucee server offering.

The Lucee 5.x server artefacts are now stored and assembled through the popular Maven platform:

Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project’s build, reporting and documentation from a central piece of information.

Maven builds were made possible by the Lucee core refactor, made easier by the OSGi approach, and enable automated assembly and continuous integration for the Lucee server build process.

The automation workflow has meant we can manage SNAPSHOTS and a thorough history of all builds and releases:

The last outstanding issue on our build pipeline is the automation of the installers so we no longer have to impose upon our wonderful friends at Viviotech to periodically build installers by hand.

White knight to the rescue

Was the OSGi refactor “worth the effort” — that is a much harder question to quantify, even in hindsight. There are clear architectural advantages to where we are now compared with where we were with the initial Railo fork for Lucee 4.5. The OSGi journey has been a long and arduous one, but not without its considerable benefits.

As to who is using OSGi? The answer is EVERYONE!

Ongoing Lucee server development is not being hamstrung by Java library conflicts, so just by running Lucee 5.x you are taking advantage of OSGi awesomeness every day :tada:

4 Likes

An especially strong point that the OSGi architecture is for Lucee primarily, and apps secondarily. It’s somewhat analogous to servlet filters, I’d argue. ACF (JRun) supported servlet filters, which in theory could have been a boon to apps/developers (that’s how SeeFusion was implemented), and then, when you looked under the covers, you saw that tons of ACF functionality had been implemented as servlet filters, too.

That said, I think it will eventually be a boon to developers and their apps running on Lucee, too. It’s good to be aligned with good people and good tech, and products like JIRA and Eclipse, not to mention other Java app servers like JBoss and Glassfish who also leverage OSGi.

1 Like

In some situation it is impossible to find a library in a version that fits all!

What I miss is the benefit of hot deploy of an extension. With lucee 4.5 it was impossible that an extension could have jars. Because you first could not load jars at runtime (multiple interconnected) and more importantly not update. A jar was loaded by the servlet engine and then specially on Windows locked for any file access (replace). Now we can even hot update without restarting lucee or the servlet engine.

Big + on a system that should have no down time.

Perhaps, in the NEAR future we can even declare extension we wanna use in the Application.cfc

7 Likes

I love the OSGi approach. In my 18 years of CF development I have ran into the JAR hell issue more than once. I am glad someone finally fixed it.

2 Likes