We are running four Lucee servers behind a load balancer. They were originally deployed with 18.104.22.168, but the upgrade to 22.214.171.124 went smooth. Now I wanted to upgrade to 126.96.36.199 and was very much loking forward to finally getting the fix for LDEV-1617.
After the upgrade of the first server, we were bombarded with Null Pointer Exceptions though - and those didn’t just hit the freshly upgraded server, but the other servers, too, via the Memcache layer we use.
We employ the Lucee Memcache extension and use CacheGet, CachePut etc. to cache objects in the database. The cache connections is configured in Lucee admin to use two memcache instances. After taking the freshly upgraded box out of the loadbalancing, we managed to get rid of the lingering NPE by clearing the cache for the affected keys where the content was generated on the upgraded box.
I have of course tested the upgrade on a separate VM which has an identical configuration to our live webservers. However, I couldn’t recreate this situation on that machine, it seems to happen only under some server load or some other special condition I haven’t reproduced in my simplyfied test case. Most of the NPEs don’t even provide any stack trace in the logs, but those which did mostly stumbled over a cfloop on a query object that was just fetched from cache via UDF.
We’re running Tomcat 8.5.14-1+deb9u3 on Oracle Java build 25.181-b13.
The line where the NPE happens is
and the NPE is thrown in lucee.runtime.type.QueryImpl.getCurrentrow:
Message string java.lang.NullPointerException StackTrace string lucee.runtime.exp.NativeException: java.lang.NullPointerException at lucee.runtime.type.QueryImpl.getCurrentrow(QueryImpl.java:1079) at my.path.to.mycomponent_cfc$cf.udfCall1(/my/path/to/mycomponent.cfc:1234)
As I still haven’t found out how exactly this can be reproduced without causing havoc to our live sites, I unfortunately do not have more info than this - I have rolled back the upgrade and our applications are running fine again. I suspect that this has something to do with the changes made for LDEV-1945.