Lucee 5.3.9.133 Stable Release

The Lucee Team is proud to announce 5.3.9.133 STABLE. Thanks you to all the community for their ongoing support testing and working with the Lucee Team to make Lucee as robust as possible!

This release took a lot longer than expected, due to all the log4j2 updates and eliminating all the interdependencies between all Extensions, thanks for your patience! The Adobe team are still battling with their own Update for Log4j2.

Highlights

  • Log4j 2.17.2 (Lucee was never vulnerable to Log4j problems)
  • Native M1 Mac JVM support (requires 5.3.9 loader)
  • Updated Extensions (PDF, ESAPI, S3, Compress, Ajax)
  • QoQ improvements, the native engine is much stricter and matches normal RDMBS behavior
  • Enhanced numeric precision support
  • Improved performance for arguments (Testbox runs 12% faster for Lucee)
  • ORM improvements

Available via your Lucee Admin, Forgebox/Commandbox, Lucee Installers, docker builds

Java 17 isn’t supported yet, the installers bundle

  • Java 11.0.15+10
  • Tomcat 9.0.62

Please support the ongoing development of Lucee

Regressions / bugs
We worked really hard to ensure that there are currently no know regressions since 5.3.8 as of this release.

If you find a bug, please always post here to to the mailing list first, don’t just file bugs

Bundled Extensions

You need to be running this matrix of extensions if you want to ensure there are no traces of Log4J v1 on your Lucee server.

Changelog

Logging

LDEV-1136 - update to Log4j 2.17.1
LDEV-2516 - OSGI logging bundles added from this.javaSettings on every request to application.log
LDEV-3289 - deploy log level INFO for bundle downloading from the update provider
LDEV-3775 - SMTPClient incorrectly putting stack trace into the message of a MailException it raises
LDEV-3853 - the application mail listener logs NPE in remoteClient.log
LDEV-3839 - Mail.log missing the mail server info which is used for sent mails
LDEV-3810 - add trace logging for cfhttp calls
LDEV-3922 - Server Admin Settings - Logging page throws NPE for datasource appender
LDEV-3891 - Lucee’s custom Log4j OSGI bundles are missing important metadata from the originals

Query of Queries

LDEV-3615 - QoQ mishandles null and boolean column aliases
LDEV-3522 - QoQ cast()/convert() functions not fully implemented
LDEV-3640 - QoQ needs to preserve nulls internally regardless of full null support
LDEV-3734 - QoQ treats nulls differently than real DB’s in arithmetic expressions
LDEV-3735 - QoQ allows divide by zero
LDEV-3736 - QoQ doesn’t convert empty strings to 0 in arithmetic operations
LDEV-3801 - ArrayIndexOutOfBoundsException in QoQ with using ORDER BY
LDEV-3822 - SELECT DISTINCT with ORDER BY in QoQ incompatibility - ACF
LDEV-3830 - QoQ UNION can still return duplicates
LDEV-3823 - QoQ doesn’t support ordinal position syntax for ORDER BY

Numeric Precision

LDEV-3661 - deserializeJSON() converts large decimals to string
LDEV-3662 - Large decimal number strings lose precision when converted to a number format

Extensions

LDEV-3686 - Axis Extension - Provider for class javax.xml.parsers.DocumentBuilderFactory cannot be created (webservice / axis)
LDEV-3695 - admin application - Uninstall the extension ESAPI/Compress Tags doesn’t available in not installed
LDEV-3688 - do not install extension that are already installed

PDF Extension

LDEV-1519 cfpdf addwatermark
LDEV-3240 cfdocument - margin doesn’t work properly with orientation
LDEV-1500 Cfdocument looses bookmarks if more than one section is used.
LDEV-3391 CFPDF action merge when source and destination are the same file.
LDEV-3781 cfdocument attribute saveAsName not implemented
LDEV-3928 Passing URL resource to CFPDF source attribute causes Null Pointer Exception
LDEV-3836 add Chinese support in cfdocument pdf output with the flying saucer engine
LDEV-3587 cfpdf action=addHeader, addFooter without destination or text attribute throws NPE

Ajax Extension

LDEV-3535 - update google maps api to v4 - Lucee
LDEV-3372 - update jquery-1.8.3.js in ajax extension
LDEV-3425 - ajax extension is slow to start

Compress Extension

LDEV-3866 - zip action=list filter UDF is a passed a completely invalid entryPath (1.0.0.5-SNAPSHOT)
LDEV-2660 - CFZIP action=“unzip” overwrite=“true” deletes existing directories. (1.0.0.6-SNAPSHOT)
LDEV-3882 - zip action = delete filter - udf filter passed invalid path
LDEV-3880 - cfzip-filterdelimiters doesn’t take pipe( | ) character as the default value (not worked as per docs).

ESAPI Extension

LDEV-3953- add function sanitizeHTML to esapi extension

JDBC

LDEV-3711 - Lucee discards exception cause from JDBC connection errors
LDEV-3712 - ojdbc7 bundle missing
LDEV-3793 - update postgres to 42.2.20
LDEV-3908 - <cfstoredproc /> is not returning correct exception when MSSQL when raiserror() is used
LDEV-3924 - lucee.runtime.exp.DatabaseException: No operations allowed after statement closed.

ORM

LDEV-3860 - Error occurred in the transaction block with ORM throws the different exception
LDEV-3659 - Transactions with mixture of ORM and vanilla SQL do not complete (3.55, 5.4 beta doesn’t work yet)

Whitespace / Output

LDEV-3760 - NPE with lucee.runtime.writer.DevNullBodyContent in flush
LDEV-3777 - cfsavecontent ignores whitespace management setting
LDEV-3784 - Lucee request fails when Accept-Encoding is not passed and gzip compression is enabled
LDEV-3338 - whitespace in component attribute “implements” cause incorrect return when using getMetaData/getComponentMetaData

Java

LDEV-3752 - duplicate() incompatible with java.util.List (return of List.subList)
LDEV-3804 - ClassUtil.loadInstance() has code path that returns exception instead of throwing it
LDEV-3846 - catch block cannot be serialized
LDEV-3687 - cfmail crashes on email addresses with trailing commas
LDEV-3658 - Cannot duplicate Environment map in Lucee
LDEV-3526 - Update Felix to 6.0.5 to support Java >= 16
LDEV-3536 - update jna library to support Apple M1 architecture

Performance

LDEV-3520 - Slow performance on arguments scope due to casting strings to Double
LDEV-3621 - Encrypting large data strings times out when using HEX encoding

Event Gateways

LDEV-3462 during shut down, stopped event gateways are restarted
LDEV-3923 start event gateway after startup

Bug fixes

LDEV-3842 - breadcrumbs missing styling for tags with local docs
LDEV-3851 - build process should use an older loader jar to expose problems
LDEV-3545 - Multipart http response doesn’t handle quoted boundary
LDEV-3716 - _internalRequest() losses the form scope with sameFormFieldsAsArray=true
LDEV-3685 - Scheduled Tasks (Daily) NOT running
LDEV-3732 - Incorrect argument count requirements in error message on method call
LDEV-3742 - cfcontent delivers wrong content-type
LDEV-3829 - typo in argon2 code with handling of memory argument
LDEV-2982 - cfexecute terminateontimeout isn’t supported
LDEV-3166 - CFFTP (secure) resets connections (ssh-dss)
LDEV-3222 - cfml2js don’t return the correct type for the values like SerializeJSON does
LDEV-3465 - Regression: inherited static variables no longer accessible in child components
LDEV-3911 5.3.9 regression, cookie encoding/decoding problems
LDEV-3910 - 5.3.9 regression, unknown content type causes null pointer exception
LDEV-3911 - 5.3.9 regression, cookie encoding/decoding problems
LDEV-3761 - _internalRequest() doesn’t work with cfcontent
LDEV-3927 - cookies set via cfheader are ignored
LDEV-3940 - REGRESSION - 5.3.9 is now spilling JSR-223 exceptions to the console

New features

LDEV-3778 - Web.cfc in webroot
LDEV-3790 - add function ConfigImport
LDEV-2331 - allow specifying a file extension for getTempFile

Admin

LDEV-2060 - Lucee Admin → Security Access → File Access must allow adding multiple directories in one request
LDEV-3363 - Admin debugging - Disable template option throws an error in debugging logs page
LDEV-3660 - Without Network admin application page shows error
LDEV-3355 - internal calls to the update provider need (shorter) timeouts
LDEV-3939 - Application list in server admin dumps stack trace out to console each page load

All the code changes

Roadmap

Next up is Lucee 6.0, here is the board for the sprint Log in with Atlassian account

With Lucee 6.0, as a major release, we are going to make some breaking changes, old insecure defaults or features which don’t work as expected are going to be addressed. Let us know if you have any suggestions in the thread below!

https://lucee.daemonite.io/t/lucee-6-changing-some-old-defaults-to-be-secure-by-default/8182

17 Likes

Can you maybe make it a config option in 6, that way if someone is running insanely old code, it doesn’t break?

Toggle it on by default, in the session area?

OHHH MY GOD!!! What an AWESOME release! Not just the number of important bug fixes, but the new features Web.cfc and configImport are great. :tada: Lucee 6.0 is knocking the door!

2 Likes

Make what an option ?
The addToken stuff ?
It’s already got a per-application setting, so I assume Lucee just changes the server default and you can turn it back on per-project

Does anyone know what version of the Redis Extension is good to go? I’ve tried the latest 2 versions but we’re having issues deploying the extension.

We’re running Lucee in Docker building against Tomcat. Dropping the Lucee-Lite jar in, then bringing the system up having dropped the extensions we use in the deploy folder.

The current extensions we’re using are:

compress-extension-1.0.0.7.lex
esapi-extension-2.2.4.5.lex
lucee.image.extension-1.0.0.42.lex
org.lucee.axis.extension-1.4.0.37-SNAPSHOT.lex
org.lucee.mssql-7.4.1.jre8.lex
pdf-extension-1.1.0.7.lex
redis.extension-3.0.0.39-BETA.lex
s3-extension-0.9.4.154.lex

When the image is finally running the Redis extension hasn’t deployed and the deploy log says:

"Severity","ThreadID","Date","Time","Application","Message"
"ERROR","main","04/29/2022","12:55:25","extract-extension","could not found [/extensions/.index] defined in the index in the lucee.jar"
"ERROR","main","04/29/2022","12:55:45","Extension","Unable to resolve redis.extension [59](R 59.0): missing requirement [redis.extension [59](R 59.0)] osgi.wiring.bundle; (&(osgi.wiring.bundle=org.lucee.aws-core)(bundle-version>=1.11.877.0001L)) [caused by: Unable to resolve org.lucee.aws-core [68](R 68.0): missing requirement [org.lucee.aws-core [68](R 68.0)] osgi.wiring.package; (osgi.wiring.package=com.amazonaws.services.s3.internal)] Unresolved requirements: [[redis.extension [59](R 59.0)] osgi.wiring.bundle; (&(osgi.wiring.bundle=org.lucee.aws-core)(bundle-version>=1.11.877.0001L))];lucee.runtime.exp.NativeException: Unable to resolve redis.extension [59](R 59.0): missing requirement [redis.extension [59](R 59.0)] osgi.wiring.bundle; (&(osgi.wiring.bundle=org.lucee.aws-core)(bundle-version>=1.11.877.0001L)) [caused by: Unable to resolve org.lucee.aws-core [68](R 68.0): missing requirement [org.lucee.aws-core [68](R 68.0)] osgi.wiring.package; (osgi.wiring.package=com.amazonaws.services.s3.internal)] Unresolved requirements: [[redis.extension [59](R 59.0)] osgi.wiring.bundle; (&(osgi.wiring.bundle=org.lucee.aws-core)(bundle-version>=1.11.877.0001L))]
	at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:4368)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2281)
	at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:998)
	at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:984)
	at lucee.loader.osgi.BundleUtil.start(BundleUtil.java:112)
	at lucee.loader.osgi.BundleUtil.start(BundleUtil.java:108)
	at lucee.runtime.osgi.OSGiUtil._start(OSGiUtil.java:1229)
	at lucee.runtime.osgi.OSGiUtil._startIfNecessary(OSGiUtil.java:1182)
	at lucee.runtime.osgi.OSGiUtil.startIfNecessary(OSGiUtil.java:1177)
	at lucee.runtime.osgi.OSGiUtil._loadBundle(OSGiUtil.java:553)
	at lucee.runtime.osgi.OSGiUtil.loadBundle(OSGiUtil.java:505)
	at lucee.commons.lang.ClassUtil.loadClassByBundle(ClassUtil.java:155)
	at lucee.transformer.library.ClassDefinitionImpl.getClazz(ClassDefinitionImpl.java:110)
	at lucee.runtime.config.XMLConfigAdmin.setClass(XMLConfigAdmin.java:6611)
	at lucee.runtime.config.XMLConfigAdmin._updateCache(XMLConfigAdmin.java:4244)
	at lucee.runtime.config.XMLConfigAdmin.updateRHExtension(XMLConfigAdmin.java:4873)
	at lucee.runtime.config.XMLConfigAdmin.updateRHExtension(XMLConfigAdmin.java:4692)
	at lucee.runtime.config.XMLConfigAdmin._updateRHExtension(XMLConfigAdmin.java:4674)
	at lucee.runtime.config.DeployHandler.deploy(DeployHandler.java:88)
	at lucee.runtime.engine.CFMLEngineImpl.<init>(CFMLEngineImpl.java:414)
	at lucee.runtime.engine.CFMLEngineImpl.getInstance(CFMLEngineImpl.java:746)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at lucee.loader.engine.CFMLEngineFactory.getEngine(CFMLEngineFactory.java:1451)
	at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:384)
	at lucee.loader.engine.CFMLEngineFactory.initEngineIfNecessary(CFMLEngineFactory.java:262)
	at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:168)
	at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:202)
	at lucee.loader.servlet.CFMLServlet.init(CFMLServlet.java:42)
	at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1134)
	at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1089)
	at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:983)
	at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4902)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5211)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909)
	at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:843)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909)
	at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:434)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:772)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:342)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:473)
Caused by: org.osgi.framework.BundleException: Unable to resolve redis.extension [59](R 59.0): missing requirement [redis.extension [59](R 59.0)] osgi.wiring.bundle; (&(osgi.wiring.bundle=org.lucee.aws-core)(bundle-version>=1.11.877.0001L)) [caused by: Unable to resolve org.lucee.aws-core [68](R 68.0): missing requirement [org.lucee.aws-core [68](R 68.0)] osgi.wiring.package; (osgi.wiring.package=com.amazonaws.services.s3.internal)] Unresolved requirements: [[redis.extension [59](R 59.0)] osgi.wiring.bundle; (&(osgi.wiring.bundle=org.lucee.aws-core)(bundle-version>=1.11.877.0001L))]
	... 64 more
"

We sometimes have this same issue installing via the UI… but sometimes it does work, then upgrading/downgrading the latest versions work…

My initial thoughts (which I’m about to test) is that we deploy the current extensions, then bring down, bring up and deploy just the redis hoping that S3 has deployed properly hoping it fixes what looks like the issue…

[org.lucee.aws-core [68](R 68.0)] osgi.wiring.package; (osgi.wiring.package=com.amazonaws.services.s3.internal)] Unresolved requirement

Update: Nope, the double deploy didn’t seem to work, so we’ve rolled back to redis.extension-2.9.0.4-BETA for the time being

Docker builds are up!

https://hub.docker.com/r/lucee/lucee/tags

3 Likes

I always tried to understand what the Lucee development team was working on.
I’m happy that this sprint is public.

Were the others in the past too?

Thanks for your work guys! :sparkling_heart:

1 Like

We’ve been improving our processes!

Sprints are the way we run these days

Feedback or suggestions on sprints always welcome

2 Likes

5 posts were split to a new topic: Errors upgrading to 5.3.9

Currently, we have one outstanding regression for 5.3.9

https://luceeserver.atlassian.net/issues/?jql=labels%20%3D%20"reg539"

Please try out the latest snapshot 5.3.9.140 and let us know if there are any others we’ve missed

1 Like

Now up to four, including logs going missing, again :frowning:

nah, there’s only one outstanding, the two are in QA, one is already marked deployed and the last one is currently being solved, aka in development

1 Like

RE: Logs - I believe that fix was applied to 5.3.9.137-SNAPSHOT … we’re deploying 5.3.9.140-SNAPSHOT to our dev/test environments today to shake any other issues loose. :crossed_fingers: nothing comes up.

1 Like

the 5.3.9.140-SNAPSHOT is up on docker hub for testing too

https://hub.docker.com/r/lucee/lucee/tags

I think I’ve solved the the last step with automating this whole triggering docker builds process, so hopefully we will have docker images for every SNAPSHOT for now on

5 Likes

If Lucee starts pushing it’s Docker images in automated fashion, that means we can too, at least for our Desktop day-to-day images.

Currently feeling like I just bought a $3K paper-weight (Apple M1 MBP). Feeling so out-of-my-depth since I know very little about architectures and platforms. When you say that this version is M1 compatible, does that mean that the nginx and JDK that come in some of the builds are also M1 compatible? I’m hoping that upgrading locally - even if I’m running an older version in prod - would at least make local-development possible.

you mean the docker builds? pretty sure they are all intel ATM

Ah, ok :confused: I saw an older post about running the build locally in order to create a ARM version for Apple M1. When I have some more time (and confidence), I’ll take a look at that.

Apparently it should Just Work to run Intel Docker images on an ARM Mac : Run x86 (Intel) and ARM based images on Apple Silicon (M1) Macs? - Docker Desktop for Mac - Docker Community Forums

We never got Tomcat to start correctly though. We don’t have any ARM Macs anymore. It’ll be an issue at some point in future though.