We have Lucee on a client server that is has firewall rules preventing traffic out by default (any traffic out of the server will require approval). The question is, can we prevent Lucee from attempting to download packages when it starts up and during execution? And, if it is prevented from doing so, what options do we have to ensure we have all the necessary packages?
This has come up a lot, and Micha’s answer has always been that there is no way to disable the automatic downloading. I thought for sure there was a ticket to be able to turn that off, but I can’t find one. I guess it’s just been mentioned in passing in other tickets. This causes a lot of issues, especially for clients of mine who have zero internet access on their servers. Heck, I had a govt shop that about flipped their lid when Lucee phoned out to the time server to sync the clock. I’d say let’s put in a ticket for have a flag to disable this.
As far as your second question, I’m not sure if Lucee will use the “next best” bundle it can find or just blow up if the OSGI bundles it has aren’t the latest version. I know the list of bundles is in an XML file that the Lucee build uses, but any time you update Lucee, I think there’s the chance that Lucee will then want a new version that it doesn’t have. I do know that if Lucee is unsuccessful in downloading, the deploy log or console log (I forget) will have a message that tells you what to download and where to put it. I hardly call that a comprehensive solution, but that’s everything I know
Yeah this needs to become a top priority, for a couple reasons:
Configuration Management: we are deploying via war distribution, and two deploys of the same application must use the same libs. Currently, the same app deployed in two locations could inadvertently end up running on different versions of the underlying libraries. That is a CM nightmare.
Client networks (and unfortunately even our corporate network) often disallow downloading from github for security purposes, meaning the app could start fine one day and fail to start the next day. In addition to not allowing to disable the update checks, there is no option to allow changing where these libraries are downloaded from - it is hardcoded. That causes security managers to lose their…cool.
This needs to be fixed.
I agree - this should be addressed and/or documented. Here is a related ticket which was tagged Need more details yet I thought OP provided pretty good detail: https://luceeserver.atlassian.net/browse/LDEV-2134
@dajester2013 hit the nail on the head with why this is important.
So it seems that all may be needed is an if statement around this block that prevents it from reaching out to the network to load the extension based on a environment / system property: https://github.com/lucee/Lucee/blob/29b153fc4e126e5edb97da937f2ee2e231b87593/core/src/main/java/lucee/runtime/config/DeployHandler.java#L201-L278
That appears to be the extensions, not the OSGi bundles.
Extensions already has the ability to disable update checks by removing all the extension provider urls from the web/server config.
The OSGi bundles are downloaded from a hard-coded url that redirects to github. I’ll try to find it again.
Ah, ok let me know if you find it.
I’m not sure if that is true, looking at the code it looks like it will always load the default extension providers, but you can add more providers if you want: https://github.com/lucee/Lucee/blob/29b153fc4e126e5edb97da937f2ee2e231b87593/core/src/main/java/lucee/runtime/config/XMLConfigWebFactory.java#L4273-L4275
It looks like it will download jars from the URL specified in the lucee-server.xml in the value which is set to http://release.lucee.org by default.
Relevant code: https://github.com/lucee/Lucee/blob/29b153fc4e126e5edb97da937f2ee2e231b87593/loader/src/main/java/lucee/loader/engine/CFMLEngineFactory.java#L680
So, most of the jars are bundled into the extensions or into the loader itself, with a few exceptions (S3 I’m looking at you). It is intended that they are simply extracted and loaded, and only downloaded if they cannot be found.
However, there is a problem with Java’s File.renameTo, where it will not always move a file, especially across partitions, which forces the bundles to be downloaded every time. I’ve opened up an issue https://luceeserver.atlassian.net/browse/LDEV-2475?atlOrigin=eyJpIjoiMWFjYWU2ZTJlYmJhNDk1MTk4ODRmNDYyYjExNTcyZTQiLCJwIjoiaiJ9 to address this.
This should address the majority of the concerns. Now, for anal-retentive individuals such as myself that want to specify and pre-package every dependency, you can use the Maven dependencies plugin such as: https://gist.github.com/dajester2013/7e0c6158919d2b5b126a42ede4acc574