Lucee 6.0 - no Docker image yet

Given 6.x is required for the latest security fixes it’s a bit annoying I came to start migrating all our servers, and there only ever seem to be SNAPSHOT Docker images for 6.x

Assuming it’s a time/resource thing ? Is there some way this can be automated ?
I’m still flush with success over the installer script :slight_smile:

In fact it still says “Latest stable release (5.4)” on Docker Hub @ Docker and there’s been no 6.x images for 7 months.

@justincarter do you need a hand here ?

+1, hoping to find out what stable docker image might be available/recommended for Lucee v6.x given the new security issue that’s been uncovered.

I’ve tried to get the issues around Docker on the GitHub repo (https://github.com/lucee/lucee-dockerfiles/issues) sorted by commenting the ones that have been opened since 2015 (!) and suggesting we close them or Just Get It Done because

a) are we really going to make wide ranging changes to how they work at this point ?
b) if there’s so little time spent on them already that they’ve not been built this year yet, is there much appetite for spending more time ?

Anyone effected by a 5 year old issue will have either fixed or worked around it already. Or moved on.

Thanks for pitching in @markdrew :slight_smile:

1 Like

For instance, I’m not sure how to test changes made to https://github.com/lucee/lucee-dockerfiles/blob/master/.github/workflows/main.yml (LUCEE_MINOR: "5.3,5.4,6.0") before submitting a MR, because I’ve never actually used a GitHub Action.

Thanks @thefalken and @markdrew for the kick :sweat_smile:

I’ve added notes on the GitHub Issue but I’ll repost them here as well;
https://github.com/lucee/lucee-dockerfiles/issues/80

The current work in progress branch for Lucee 6.x support is here:
https://github.com/lucee/lucee-dockerfiles/commits/wip-next/

I think the things left to address are:

  • adding default JSON config files
  • supporting XML file upgrades without the default/existing JSON file blocking the migration of the XML config (I think this is an issue for anyone expecting their existing XML files to just work?)
  • single mode for 6.x by default (users can change back to multi-mode in their config files if they require it, though I would guess ~95% users would not)
  • check which Java versions are/should be supported

The “7 month old” Lucee 6.x images are indeed very old snapshots and don’t have any of the proper work done to them that was needed for the build. There’s also Lucee 6.x “Zero” which is smaller/different to “Light” – I think I have already added the support for that.

Technically, I can do a Lucee 6.x build with what we have, using the wip-next branch to produce a SNAPSHOT build.

If people are super keen for that I can trigger it any time, I just think that without some of the above work being done you might run into issues. If you’re able to easily migrate from the XML config to the JSON config yourself then perhaps it’ll be all good, and as a snapshot release that’s probably fine.

For the generally available “Release” release I’d want to make sure it works for everyone without issue (where possible) :slight_smile:

1 Like

default JSON config

I’m not sure where the complexity is here - start Lucee outside Docker, see what files it makes, copy them for use in Docker ?

supporting XML file upgrades without the default/existing JSON file

When I started porting our downstream images, I accepted that because Lucee 6.x has JSON config rather than XML, that I’d need to re-do the settings changes in the new style. I’m not sure downstream authors would expect their old Dockerfile/config files to Just Work ?

For instance, adding in new default DSN was fairly trivial - login to admin, add global DSN, see what the JSON looks like, make that happen in our Dockerfile.

As long as actual sites Just Work, because Lucee v6 has (one time ?) automated conversions for WEB-INF/lucee/lucee-web.xml.cfm that’d be OK ?

In the testing I’ve done of our downstream images based on 6.0.0.524-SNAPSHOT, it all seemed to work as expected.

supported Java versions

Now’s a good time to reduce the test matrix, and just pick (say) whatever upstream Tomcat bundles, at least as a starting point ?

can do a Lucee 6.x build with what we have

I think that’d help, at least to have a snapshot from wherever the most recent 6.0.x is, even if it’s not perfect.

We’ve got a mix of sites using the Lucee installer, and ones using our own downstream Docker, and right now only the non-Docker ones can get the security fixes that were not back ported to 5.x (Heads-up on Lucee vuln, publicized today, Feb 15 - #4 by sbleon).

1 Like

People definitely need a documented, easy to understand upgrade path. It’s my understanding that Lucee 6.x will only migration configuration from XML to JSON if no JSON file exists (I could be wrong, happy to be corrected). However, if I publish a build as-is, there will already be a JSON config inside the image (due to prewarm), so this may catch people out and the might not understand that they would have to delete the JSON file to get their old XML file to work/be migrated.

It seems like it could already be catching people out in non-Docker scenarios as well, I’m not sure if this is related?

So I’m just saying it’s something that needs to be tested and perhaps documented/worked around before we publish a “Release” release version :slight_smile:

Definitely, but I also need to understand which Java versions are supported (particularly LTS versions) for running Lucee. AFAIK Neither Java 17 LTS nor Java 21 LTS are working yet, so that leaves us with Java 11 I suppose. There’s some context in this old thread:

I’m happy to kick this off now :smiley:

I’m building 6.0.0.585-SNAPSHOT now so it should appear in Docker Hub in the next 30 mins or so I think. I’ll try to check back in a couple of hours (almost dinner time here).

1 Like

The 6.0.0.585-SNAPSHOT builds are up.

Fair warning, there may be some hoops you need to jump through with XML → JSON configs as I mentioned, and I haven’t tested these builds myself to know what other issues you might find. If you do hit any roadblocks please report them in this thread and I’ll see what I can do to rectify them as quickly as I can!

1 Like

Justin! You’re my hero!!! I renamed .CFConfig.json and reimported the config file and it worked!!! You saved me SO MUCH WORK!!!

1 Like

Lots to unpack here - “it’s complicated” :slight_smile:

Config:
Could the image delete the JSON config after prewarm ? Or for now disable prewarm and explain how to do it in downstream images ? Or just say “you can’t migrate your configs unless you do X” ? I don’t know how other users of the Docker images feel about this.

We build our own config as part of our own downstream, so rewriting all our xml-hacking to be json-hacking isn’t so bad. A one time conversion of thousands of datasources doesn’t seem unreasonable for a major version update, but I’m not having to write that and @ChipNCharge would have to…

JVM
Current LTS - version 11 - is surely a good baseline. Initial testing with the 6.0-SNAPSHOT seems fine, though I’ve not tackled our custom AWS Java SDK integrations - but those need updating to a new AWS SDK version anyway, in the event of issues we’d just do that.

I don’t know how many people used the different JVM variants.

Staying with whatever upstream Tomcat uses has more benefits than negatives, even if it moves from Debian stable to unstable, doesn’t use FreeBSD or whatever. It’s going to be supported and updated.

I’ll update our local day-to-day testing Docker to the new snapshot. Gotta crack on and convert all the QueryToStruct() calls, change every Application.cfc :confused:

I’m not convinced this is the job of a Docker image. it’s not an “upgrade path”, it’s an image of Lucee 6. If Lucee 6 now uses JSON config and ignores XML config then… do exactly that. Ship with default config for a Lucee 6 installation.

There might be a case to answer for LAS to provide an upgrade path for Lucee, or even the update functionality in Lucee Admin to deal with this migration when updating from 5.x to 6.x. But I don’t see how this is the job of a Docker image.

A spun-up Lucee 6 Docker image should behave the same as a fresh on-metal install of same. It would be completely unaware of any previous metal that might have run Lucee 5. It’s up to whoever is managing the instances to deal with the migration.

It’s a chicken and egg thing. When Lucee 6.x finds no JSON configuration it will migrate an existing XML configuration to JSON, and that’s great for anyone upgrading (via any method, not just Docker users). Agreed it definitely should ship a default Lucee 6 JSON config out of the box, but right now it’s technically upgrading an old XML file which is not what the final release image should do.

I probably just need to document that users can delete the JSON config in their Dockerfile so that this upgrade from XML will work for them the first time, and after that they can take a copy of the JSON config and must use that going forward. (I say this having not actually tested it myself, it’s just based on what I understand / have heard). This is not a difficult thing, it just needs to be done, along with a few other bits and bobs so that everyone can move forward without getting caught out.

Yep agreed. I was hoping that we could build images for newer Java versions but currently I don’t think Lucee runs on Java 17 or Java 21 so we’re stuck with Java 11 as the latest. Those are all supported by the official Tomcat 9.x image so it’s fine for now, but working with newer Java versions will become more important (for all installs, not just Docker). Java 8 can still be supported too while Tomcat still supports it.

Good luck :slight_smile: Like I said, if you hit any road blocks please post here and I’ll see what I can do.

2 Likes

Previous post deleted asking about how to upgrade lucee. My path’s were wrong. The correct way is what you would expect:

ENV LUCEE_SERVER /opt/lucee/server/lucee-server
ADD https://cdn.lucee.org/6.0.1.79-SNAPSHOT.lco “${LUCEE_SERVER}/deploy/6.0.1.79-SNAPSHOT.lco”

Excellent clarification @justincarter, thanks.

Hi @justincarter

We’ve been using our own local Docker image, based on the current 6.x snapshot from Lucee, and there have been no unexpected issues so far.

Any news on when we might get an official Docker for Lucee 6.0 ? Do you need a hand anywhere ?
We’re kinda blocked from moving forwards with several 5.x to 6.x migrations because a lot of our environments are Dockerised.

Thanks for the feedback @thefalken, much appreciated :+1:

Other than using the snapshot are there any particular things you needed to change in your Dockerfile or any particular challenges that you faced? e.g. were you able to use it to automatically generate the JSON file from your existing XML configuration? Any other issues that I have not anticipated previously in this thread?

I’m super short on spare time so I’m just trying to gauge what critical changes I might need to make, if any, and so if you think the state of the existing snapshot is fine then I can easily do some newer builds for you, no problem.

The only real challenge was redoing our settings in the server config, instead of string-bashing XML we needed to string-bash JSON, it really wasn’t too bad, because we knew the 3 or 4 settings we were changing. Biggest hurdle was the format isn’t documented, but it’s easy enough to login to the server admin, change the setting, see what it writes to the file.

As an example:

RUN sed -i 's!</mail>!<server port="587" smtp="email-smtp.eu-west-1.amazonaws.com" ssl="false" tls="true" username="u" password="p"/></mail>!' /opt/lucee/server/lucee-server/context/lucee-server.xml

became

COPY requires/mail_server.json /tmp/
RUN jq '.mailServers += [input]' /opt/lucee/server/lucee-server/context/.CFConfig.json /tmp/mail_server.json > foo
RUN mv foo /opt/lucee/server/lucee-server/context/.CFConfig.json

Just something people would need to be aware of; “if you were doing X to set a setting, you need to do Y now”. Same as “if you were depending on Java version X, it’s now Y”.

If expectations were set that it’s a new major version, with new everythings, not sure anyone would be too bothered.

I’d say release a Docker version of 6.x, massively trim down the text on Docker Hub, call out an “Upgrading from 5x” section, and call it done.

Then get it integrated into the main release as time allows.

2 Likes

The release build for 6.0.1.83 is now in Docker Hub.

It’s based on the same wip-next branch as the previous 6.0.0.x image build so no significant change to report, just a newer Lucee version.

1 Like