Are Lucee's custom class loaders single-threaded or multithreaded?

A question to Team Lucee: Are Lucee’s custom class loaders single-threaded or multithreaded?

I ask because our logs show many situations where a lock is associated with Lucee’s custom class loaders. Beyond a certain request load, the application’s performance falls sharply. Requests that should normally take several seconds begin to last 3 minutes.

The result is that our application, good for thousands of users per day, practically comes to a standstill. Related ticket.

Here is an excerpt from one of the typical stacktraces we’re getting from FusionReactor’s profiler:


dvnt.stdrdr.vo.basicuservo_cfc$cf.udfCall(/nl/dvnt/stdrdr/vo/BasicUserVO.cfc)
lucee.runtime.functions.decision.IsInstanceOf.call(IsInstanceOf.java)
lucee.runtime.reflection.Reflector.isInstaneOf(Reflector.java) (hide)
lucee.commons.lang.ClassUtil.loadClass(ClassUtil.java)
lucee.commons.lang.ClassUtil.loadClass(ClassUtil.java)
lucee.commons.lang.ClassUtil._loadClass(ClassUtil.java)
lucee.commons.lang.ClassUtil.__loadClass(ClassUtil.java)
lucee.commons.lang.ClassUtil$ClassLoaderBasedClassLoading.loadClass(ClassUtil.java)
java.lang.ClassLoader.loadClass(Unknown Source)
java.lang.ClassLoader.loadClass(Unknown Source)
Waiting on lock <0x54d4dccc> (a java.lang.Object)

The line of code at /nl/dvnt/stdrdr/vo/BasicUserVO.cfc is:

/*** /nl/dvnt/stdrdr/vo/BasicUserVO.cfc ***/
if ( isInstanceOf(arguments.data, "nl.dvnt.stdmtr.orm.UsersRO") ) {

See attached for more on the FusionReactor Profiler stacktrace.

The lock makes me wonder whether every custom class loader used by Lucee follows the “Recommendations for Multithreaded Custom Class Loaders”


FusionReactor_Profiler_stacktrace_longlasting_request.txt (16.7 KB)

The interesting thing about locks, is they don’t consume any CPU. One of your first posts regarding performance stated you were seeing CPU saturation, but on servers experiencing deadlocks or just over-aggressive locking, the CPU is often quite low even though pages can take a long time to finish. Just to clarify, are you seeing high CPU on the server at the same time that you’re seeing requests “wait on lock” or are these unrelated incidents?

Are Lucee’s custom class loaders single-threaded or multithreaded?

The good question here is whether a given class is written to correctly its internal integrity when used by more than one thread at a time. I can’t speak entirely for Micha, but I am quite sure he would say “Yes, of course Lucee’s class loaders are designed to be used by more than one thread at a time”.

The Oracle article is interesting, however

  • I see no evidence of deadlocks at this point (which isn’t the same as a lock)
  • The actual locks I’m seeing in the profile are coming from java.lang.ClassLoader (the core class loader class), not from any custom Lucee class loader, so I’m not even sure any of that article applies here!

The profiler will tell you what lines of code are running the most or the longest, but it won’t tell you what is locking your thread. For that, you need to capture full JVM thread dumps that catches in-the-act

  • the thread holding the lock
  • all the threads waiting to acquire the same lock
  • the actual lines of code this stuff is happening on (The profiler does not track at the line level for performance reasons)

Can you post a full JVM thread dump while this is going on. The question is what other thread was holding the lock and why. But without knowing the lines of code to go review the source, it’s all just guessing right now. Also, if you haven’t mentioned it elsewhere, can you let us know what exact version of Java you’re using? That can change what the line numbers in the stack mean.

Also, to add some more info-- I was looking at what it takes to register a class loader as parallel capable-- as the base class loader class actually behaves differently based on that.

It’s interesting to note that there are 12 classes in Lucee’s core that appear to extend a class loader class of some kind, but only 3 of them appear to register themselves as parallel (noted in bold)

  • Lucee/core/src/main/java/lucee/commons/io/res/util/RCL.java
  • Lucee/core/src/main/java/lucee/commons/io/res/util/ResourceClassLoader.java (parallel)
  • Lucee/core/src/main/java/lucee/commons/lang/ArchiveClassLoader.java
  • Lucee/core/src/main/java/lucee/commons/lang/ExtendableClassLoader.java (parallel)
  • Lucee/core/src/main/java/lucee/commons/lang/LogClassLoader.java
  • Lucee/core/src/main/java/lucee/commons/lang/MemoryClassLoader.java
  • Lucee/core/src/main/java/lucee/commons/lang/PClassLoader.java
  • Lucee/core/src/main/java/lucee/commons/lang/PCLBlock.java
  • Lucee/core/src/main/java/lucee/commons/lang/PhysicalClassLoader.java (parallel)
  • Lucee/core/src/main/java/lucee/commons/lang/compiler/DynamicClassLoader.java
  • Lucee/core/src/main/java/lucee/runtime/osgi/EnvClassLoader.java

Now, I have NO CLUE if that is causing any issues whatsoever as @micstriit would need to come along and comment on why that was the case. And I’ll point out again your stack shows the locking waiting happening inside of a core JDK class loader-- or at least it appears to. Reading stacks is tricky since if a custom class loader extends java.lang.ClassLoader and calls its super loadClass() method, the stack will show the load class method from the core class loader is running, but the actual class loader instance may still be a sub classed instance that has NOT registered itself as parallel capable. So without further debugging, there’s no easy way to tell if the

java.lang.ClassLoader.loadClass(Unknown Source)

call that was waiting on the lock was from an non-parallel-capable subclass or not.

It’s also worth noting that classes such as Lucee’s ExtendableClassLoader register themselves as parallel capable…

public abstract class ExtendableClassLoader extends ClassLoader {

	static {
		boolean res = registerAsParallelCapable();
	}
...

HOWEVER, this note in the Javadocs for ClassLoader gives me pause:

 * Note that the {@code ClassLoader} class is registered as parallel
 * capable by default. However, its subclasses still need to register themselves
 * if they are parallel capable.

This seems to imply that simply registering a base class is not enough, but every sub class must also register themselves explicitly as parallel capable. Again, we need @micstriit to chime in here and explain.

@Zackster Also just pointed out this ticket to me which indicates the addition of the parallel code was rather recent (last month). See my comment at the bottom of the ticket based on our discoveries here:
https://luceeserver.atlassian.net/browse/LDEV-4169

1 Like

Hi @bdw429s ,
Thank you for looking into this.

Sorry about my late reply. I have only been able to come up for air just now. It’s been hectic in our Lucee kitchen as we try to get a workable application going.

To answer your question on “wait on lock”, we do in fact see that on the same instance on occasions when the CPU is low.

I am fixing our thread-dump code and shall later share what the dumps say.

Thank you for your time and for the insights.

good catch, i did improve that part.
not all classloader implementations are shared between threads and need to be “parallel capable”

1 Like

Hi,
There’s been a development. I wish to share what I’ve found after analyzing the thread dumps.

Our CPU problems are apparently caused by

Blockquote
Waiting on lock <0x754d0045> (a org.hibernate.boot.registry.classloading.internal.AggregatedClassLoader)”.

Thread dumps show dozens of request threads being blocked by the AggregatedClassLoader. See attached.

This happens invariably when the application is under load. The “waiting on lock” makes you wonder whether the AggregatedClassLoader is single-threaded. I would be grateful if you could suggest how to solve this.

Lucee 5.3.9.163-SNAPSHOT on CommandBox
Hibernate 5.4.29.15-BETA
Windows Server 2016

Thread Dump - sm1-3.txt (446.7 KB)

I’ve reached out to a hibernate expert

Thanks, @Zackster !

can you knock up a reduced test case?

That Hibernate classloader does not appear to be parallel

That said, locks don’t cause high CPU. So the locks may very well cause requests to back up, but sitting threads don’t consume any CPU.

ahh I see @BK_BK asked this on the hibernate forum (next time please drop a link here too)

@bdw429s , Thanks for confirming that the Hibernate classloader is not parallel.

@bdw429s: That said, locks don’t cause high CPU. So the locks may very well cause requests to back up, but sitting threads don’t consume any CPU.

Yes, of course. This ties in with Zackster’s question.

@Zackster : can you knock up a reduced test case?

I am looking for one. In the meantime, let me share something else I found. In the request handled by the RUNNING thread causing the hold-up, the following line seems to be the blocker:

var aResult=entityLoad("UserResults", {userId: arguments.userId, exerciseUUID: arguments.exerciseUUID, exerciseId: arguments.exerciseId});

In particular, the use of the arguments scope within entityLoad. Not only that. We also noticed elsewhere in the application that the following can also block:

var aUserTools=entityLoad("UserTools", {user: someObject.getUserById(1234)});
var aUserPages=entityLoad("UserPages", {userId: getUserId()});
var aUserChapters=entityLoad("UserChapters", {chapterId: chaptersStruct.id});

In short, Hibernate doesn’t seem to like it when you pass an object or complex variable in entityLoad.

I also wonder whether the Hibernate setting hibernate.query.conventional_java_constants=false could help. I got this idea from Vlad Mihalcea.

No, not any more.
Update: while searching for a test case, I have been able to eliminate entityLoad calls having a struct argument that has just one key-value pair.

The most probable culprits have 2 or more key-value pairs. Such as the first example I gave here:

var aResult=entityLoad("UserResults", {userId: arguments.userId, exerciseUUID: arguments.exerciseUUID, exerciseId: arguments.exerciseId});

The codebase originates from Adobe CFML. So this got me to look at the Lucee documentation with a closer lens. The doc says:

idOrFilter
The primary key value of the entity that must be loaded or Key-value pair (Struct) of property names and values.
If there are more than one key-value pair, then the AND operator is used.If specified, loads and returns an array of entities of the specified entity name that matches the filtercriteria.

Could it be that 2 or more key-value pairs require an AND operator between them? Even when the combination of key-value pairs represents a composite primary key?

There is this other matter. I found that you will get

“lucee.runtime.exp.CasterException: null can not be casted to a Struct”.

if you try to use named arguments in entityLoad. Why is that?

Lucee’s entityLoad documentation tells us that the function’s arguments are as follows:

  • name - string, required;
  • idOrFilter (aliases: id, filter) - any, optional;
  • uniqueOrOrder (aliases: unique, order) - any, optional;
  • options - any, optional.

Accordingly, the following lines of code do work:

oBirds=entityLoad("birds");
oBirds1=entityLoad("birds",{id:1,birdName:"ostrich"});
id_Filter={id:1,birdName:"ostrich"};
oBirds2=entityLoad("birds", id_Filter);

But “optional” implies that you should be able to use named arguments to make your code more efficient. However, the following lines of code - with named arguments - each fail:

oNamedBirds1=entityLoad(name="birds");
oNamedBirds2=entityLoad(name="birds",idOrFilter={id:1,birdName:"ostrich"});
id_Filter={id:1,birdName:"ostrich"};
oNamedBirds1=entityLoad(name="birds", idOrFilter=id_Filter);

The exception then produced is ““lucee.runtime.exp.CasterException: null can not be casted to a Struct”. I can imagine this being related to the issue, “Waiting on lock <0x754d0045> (a org.hibernate.boot.registry.classloading.internal.AggregatedClassLoader)”. Under the hood, Lucee might be passing parameters dynamically to Hibernate, encouraging it to load new classes at runtime. Just a guess.

Your thoughts on this will be appreciated. I have attached the details of a simple test case.

Test_case-EntityLoad_named_argument_results_in_null_cannot_be_casted_to_a_Struct.txt (1.7 KB)

the casting to struct happens here (including the top of a java stacktrace plus any caused by sections is really useful)

that code says unless uniqueOrOptions or idOrFilter is a simple value, both those arguments need to be structs

Thanks.

Now, on to the stacktrace. The lines

oBirds1=entityLoad(“birds”,{id:1,birdName:“ostrich”});
oBirds2=entityLoad(“birds”,{id:1,birdName:“ostrich”},true);

both work as expected, producing the entity.

However, either line

oBirds3=entityLoad(name=“birds”,idOrFilter={id:1,birdName:“ostrich”});
oBirds4=entityLoad(name=“birds”,idOrFilter={id:1,birdName:“ostrich”},unique=true);

results in:

Java Stacktrace lucee.runtime.exp.CasterException: null can not be casted to a Struct
at lucee.runtime.op.Caster.toStruct(Caster.java:2736)
at lucee.runtime.op.Caster.toStruct(Caster.java:2712)
at lucee.runtime.functions.orm.EntityLoad.call(EntityLoad.java:78)
at ormtestrequest_cfm$cf$d.call(/ormTestRequest.cfm:8)
at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:1054)
at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:946)
at lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:219)
at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:44)
at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2490)
at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2476)
at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2447)
at lucee.runtime.engine.Request.exe(Request.java:45)
at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1198)
at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1144)
at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:97)
at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:770)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:834)

I wish to share some more information on the problem we’ve been having with Hibernate on Lucee.

As I have reported, we see requests invoving entityLoad taking too long. A recent development is that the call has been timing out. So I can now share the stacktrace when timeout occurs:

java.lang.Class.forName0(Native Method)
java.lang.Class.forName(Unknown Source)
org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl.classForName(ClassLoaderServiceImpl.java:130)
org.hibernate.metamodel.internal.MetamodelImpl.getImplementors(MetamodelImpl.java:661)
org.hibernate.internal.SessionImpl.list(SessionImpl.java:1851)
org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:370)
org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:392)
org.lucee.extension.orm.hibernate.HibernateORMSession.load(HibernateORMSession.java:854)
org.lucee.extension.orm.hibernate.HibernateORMSession.load(HibernateORMSession.java:648)
lucee.runtime.functions.orm.EntityLoad.call(EntityLoad.java:66)
dvnt.stdmtr.service.leveltests_cfc$cf.udfCall2(/nl/dvnt/stdmtr/service/LevelTests.cfc:186)
dvnt.stdmtr.service.leveltests_cfc$cf.udfCall(/nl/dvnt/stdmtr/service/LevelTests.cfc)
lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
lucee.runtime.type.UDFImpl.callWithNamedValues(UDFImpl.java:213)
lucee.runtime.ComponentImpl._call(ComponentImpl.java:698)
lucee.runtime.ComponentImpl._call(ComponentImpl.java:585)
lucee.runtime.ComponentImpl.callWithNamedValues(ComponentImpl.java:1951)
lucee.runtime.util.VariableUtilImpl.callFunctionWithNamedValues(VariableUtilImpl.java:866)
lucee.runtime.PageContextImpl.getFunctionWithNamedValues(PageContextImpl.java:1784)
dvnt.stdmtr.service.userdata_cfc$cf.udfCall1(/nl/dvnt/stdmtr/service/UserData.cfc:318)
dvnt.stdmtr.service.userdata_cfc$cf.udfCall(/nl/dvnt/stdmtr/service/UserData.cfc)
lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
lucee.runtime.type.UDFImpl.call(UDFImpl.java:223)
lucee.runtime.ComponentImpl._call(ComponentImpl.java:697)
lucee.runtime.ComponentImpl._call(ComponentImpl.java:585)
lucee.runtime.ComponentImpl.call(ComponentImpl.java:1932)
lucee.runtime.util.VariableUtilImpl.callFunctionWithoutNamedValues(VariableUtilImpl.java:787)
lucee.runtime.PageContextImpl.getFunction(PageContextImpl.java:1765)
nl.dvnt.stdmtr.facades.userdatafacade_cfc$cf.udfCall(/nl/dvnt/stdmtr/facades/UserDataFacade.cfc:22)
lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
lucee.runtime.type.UDFImpl.callWithNamedValues(UDFImpl.java:213)
lucee.runtime.ComponentImpl._call(ComponentImpl.java:698)
lucee.runtime.ComponentImpl._call(ComponentImpl.java:585)
lucee.runtime.ComponentImpl.callWithNamedValues(ComponentImpl.java:1951)
lucee.runtime.util.VariableUtilImpl.callFunctionWithNamedValues(VariableUtilImpl.java:900)
lucee.runtime.functions.dynamicEvaluation.Invoke.call(Invoke.java:49)
application_cfc$cf.udfCall(/Application.cfc:87)
lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
lucee.runtime.type.UDFImpl.call(UDFImpl.java:223)
lucee.runtime.ComponentImpl._call(ComponentImpl.java:697)
lucee.runtime.ComponentImpl._call(ComponentImpl.java:585)
lucee.runtime.ComponentImpl.call(ComponentImpl.java:1932)
lucee.runtime.listener.ModernAppListener.call(ModernAppListener.java:444)
lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:200)
lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:44)
lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2482)
lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2468)
lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2439)
lucee.runtime.engine.Request.exe(Request.java:45)
lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1198)
lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1144)
lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:97)
lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)
javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
org.cfmlprojects.regexpathinfofilter.RegexPathInfoFilter.doFilter(RegexPathInfoFilter.java:47)
io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
jdk.internal.reflect.GeneratedMethodAccessor52.invoke(Unknown Source)
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:134)
com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doNext(FusionReactorRequestHandler.java:772)
com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doHttpServletRequest(FusionReactorRequestHandler.java:344)
com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doFusionRequest(FusionReactorRequestHandler.java:207)
com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.handle(FusionReactorRequestHandler.java:809)
com.intergral.fusionreactor.j2ee.filter.FusionReactorCoreFilter.doFilter(FusionReactorCoreFilter.java:36)
jdk.internal.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:71)
jdk.internal.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
com.intergral.fusionreactor.agent.filter.FusionReactorStaticFilter.doFilter(FusionReactorStaticFilter.java:54)
com.intergral.fusionreactor.agent.pointcuts.NewFilterChainPointCut$1.invoke(NewFilterChainPointCut.java:42)
io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java)
io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117)
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52)
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275)
io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79)
io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134)
io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131)
io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255)
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79)
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100)
io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852)
org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2019)
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1558)
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1449)
org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280)
java.lang.Thread.run(Unknown Source)

The relevant line of code is:

/* /nl/dvnt/stdmtr/service/LevelTests.cfc:186 */
oResult=entityLoad("LevelTestUserStatusForOverview", {methodId: arguments.methodId, userId: arguments.userId, licenseKey: arguments.licenseKey, licenseSession: arguments.licenseSession, subSkillId: arguments.subSkillId}, true);

Perhaps related to the issue in the vladmihalcea.com link. However, when we tried the JVM flag

-DHibernate.query.conventional_java_constants=false

it didn’t help.

I suppose the question is:
why is it that whenever Lucee runs entityLoad it gets Hibernate to run the classloader?