Lucee is getting smaller / faster, but custom building it can get you the rest of the way to Modern CFML

featured

#1

I spent my holiday weekend hacking at Lucee core for fun. This post is like an blog article about just how open Lucee is and how easy it is to modify and participate as an outsider. They are really giving us an excellent way to get involved for several years now, and I continue to invest more into making Lucee better.

I’ve been using Lucee & Railo for many years, and while innovation is great, I mostly want the server to be really stable and fast because we depend on it. In most languages, there is a core set of features you rely on, and that’s all you need. To some extent, CFML has always had too many features, and it doesn’t give us enough insights into what it is doing under stress conditions. We still have to pay the cost of the Coldfusion legacy in bugs and complexity. New adobe releases may or may not be helpful to making Lucee better. Adding new language syntax seems especially bad considering the rough IDE support and the lag and lack of cooperation in the ACF product.

I’ve been working hard lately to solve all the issues I’ve had in Lucee over the years through Lucee source changes. As I invest more in Lucee, I want to make it more my own, because it is getting easier to do that.

For the last few days, I challenged myself with the task of making a version of Lucee that is truly without the Coldfusion legacy by trimming all the bloat out of the Lucee Java code. Thousands of changes, and removal of numerous libraries, and now the Lucee jar is like 33mb instead of the 60mb+ it was. 18mb of that is pdf, image and mysql extensions, so Lucee core and the loader are now only 15mb. It’s very hard to delete any more of it without breaking it a lot of how it works, but it could go further. I was able to keep it working by using git and carefully testing each round of changes. If even runs my big application still. Yet, this build only has the modern features that are considered the best way to do things in Lucee/CFML and nothing else.

Lucee dialect presented itself as a way to have cleaner CFML, yet I was surprised to find it doesn’t actually work right now, so I thought about trying to fix that, but there is already a really clean version of Lucee under the hood on the CFML side that better matches my real application.

I deleted or rewrote all the code related to the legacy modes.
I deleted like extensions, functions and tags I don’t use and that the admin doesn’t use.
I removed the cluster and client scopes.
I forced null support to work in one way that makes sense everywhere.
I modified how scope cascading works so it only does the least work possible, and the other scope searching code was deleted, even for the bytecode generated parts.
I removed some of the ORM features, since I don’t use it. Though some of this is required for the regular queries to work.
I’m also running the bug fixes I reported to Lucee
I upgraded to Java 8’s concurrenthashmap because it is faster.
I made the Lucee build process much faster by doing less and reusing more of what it generates.
I also removed things from Lucee admin when I removed features.

I removed the Lucee dialect because the CFML dialect without the extra mode logic is more complete & consistent then Lucee dialect currently is. Also having 2 languages slightly effects performance due to extra logic branches everywhere to support both.

Some features of the CFML language are dangerous. For example, createObject(“java”), cfexecute, direct java access, etc. While the Lucee admin can disable these features, it would be even better if the features didn’t even exist so I deleted them altogether.

The Lucee admin also relies on old features like custom tags, and cfm and cfform, cfinput, cfhtmlbody and other tags that I don’t agree with using in my application. I’d prefer all the code was only modern cfc code, so then I could remove the custom tags and .cfm code from Lucee, and simplify the bytecode and possibly delete all scope cascading. I’ll probably rewrite a few parts of the admin and delete the rest. Things like debugging logs make sense, but most of the other features open the server to having additional attack surface area even if you are protecting the /lucee/ with a web server like I am.

The cfadmin tag is a powerful and dangerous feature, which I could delete and replace with a few functions that expose only what I need.

I’ve always felt like you have to babysit a complex Java/CFML application a bit too much. Lucee still has some serious bugs in it. After weeding these out, it will still be possible to break it again. I want to do more to protect it with my future changes, and to make sure I understand all the error conditions better, which gets easier as you learn Java. I am reporting these fixes back on the ticket system.

I also wanted to reduce how much is Java JDK version dependent so I can upgrade to new versions of Java faster. Fewer dependencies will make it easier to upgrade on the new 6 month release cycle.

I’m doing this work on github in a branch of my public fork, but I don’t expect anyone to use my version directly really. I’d think that other people would instead get inspired to modify and extend Lucee themselves from reading this, perhaps cherry pick some of my code or think about their problems in a new way.

There is value in reading the Java that powers your application. It makes the CFML you write more informed about how it works internally. You can validate how good or bad the code is in Java in Lucee and see if you want to address it there instead. I’ve found that Lucee’s code is really good, and it has taught me a lot of advanced concepts that directly relate to my application. I’ve also learned that Java is really a better platform overall especially when you are using IntelliJ Idea. It moves more of the basic debugging to happening in the IDE instead of needing to run the application and gives you a ton of keyboard shortcuts for faster code navigation that I simplify can’t use in CFML.

I’d encourage anyone curious in what I’m doing to go fork Lucee and try to do something with it to make it yours too. You’ll probably have fun at the least, and be surprised when you figure something out on your own. I got myself super stuck and unstuck numerous times. It still gets awkward when OSGi, Tomcat or Java do something unexpected.


#2

Thanks for this wonderful topic – have re-classified as a blog post :slight_smile:


#3

#4

@brucekirkpatrick Very interesting stuff indeed. just wiping all the extensions will give you a jar that’s only around 20MB (and this is already available on the Lucee download page). I’m also interested in some of the additional removals you did. I don’t think I’d go for all of them since stuff like java interop is bread and butter for me and a major selling point of pretty much every JVM lang in existence, but I don’t use Lucee lang and don’t’ care about things like the Admin in the CLI.

Clearly, what you’ve done is for your custom purposes and isn’t backwards compat, but I’m curious if any of your work would be useful to the Lucee core to help weed out more optional functionality that can be extracted into Extensions. I’d love to get the core as small as humanly possible so I can just use straight CFML as easily as possible, but I still like having the option to drop back in functionality that I may want. The ability to generate PDFs, for instance, is something I’ve used several times from the CLI in CommandBox.


#5

Very interesting :wink:

Our goal was (and still is) to move as much as possible to extensions. The benefit of extensions are that even they are installed with Lucee their bundles not get loaded into memory unless they are used (we are making sure of that) because they run in different OSGi bundles. With Lucee 5.3 we moved again features to extensions.

You can see that in the bundle overview page of the admin, what is loaded and what not. Based on this overview page we also decided what the next candidate is to move to extensions. Biggest still loaded by the core always has to go :wink:

ATM we are also look into the possibility to use 7 zip (7z) instead of zip what gives a much better compression for jars/extensions. i would like Lucee/Felix to teach that it can handles 7z directly.

With Lucee 6 we also change how Lucee makes the decision how and which extension to check/install. ATM all extensions bundled with Lucee are “required” by nature, i would like to have optional extension, that are never touched unless needed. But i’m not sure atm how to do that without hardcoding references in Lucee.


#6

Side note:

Lucee has disabled the Lucee dialect by default (for performance reasons), so to get it working you need to enable it like this

we actually have a couple of test cases that are using/testing that dialect with every commit.