Anyone else having major hanging issues with sessionCluster/clientCluster?


#1

Hi all,

I recently upgraded a few clustered servers to Lucee 5.2.4.37 (latest stable a.f.a.i.k.), from 5.0.x. They are using a datasource for session and client scope storage, with “sessionCluster” and “clientCluster” enabled.
Since I did the Lucee upgrade, I am having major problems with requestTimeouts almost daily. If the situation occurs, all requests in a few minutes will timeout, with the stacktrace underneath. That means tens to hundreds of errors. 2 times, Tomcat needed a restart to overcome this (it kept going at 100% cpu).

Looking at the stacktrace, timeouts occur in


which is a “synchronized” method, meaning (if I recall correctly), only one request can execute it at any given time.
I also found the stacktrace of the first erroring request, which was one step further, at

That stacktrace is also underneath, and indicates the request is waiting for “DatasourceConnectionPool.getDatasourceConnection”.

Meaning, if the datasource is unavailable for a specific request, all other requests will hang untill a request timeout occurs.

Is anyone else having these issues? I already created a ticket for it, but no me-too’s yet :confused:

Kind regards, Paul Klinkenberg

Stacktrace of all error pages, except the first request:
lucee.runtime.exp.RequestTimeoutException: request /index.cfm (/var/www/clientname/www/index.cfm) has run into a timeout (50 seconds) and has been stopped. at lucee.runtime.type.scope.storage.IKStorageScopeSupport.hasInstance(IKStorageScopeSupport.java:213) at lucee.runtime.type.scope.ScopeContext.hasExistingCFSessionScope(ScopeContext.java:549) at lucee.runtime.type.scope.ScopeContext.hasExistingSessionScope(ScopeContext.java:487) at lucee.runtime.PageContextImpl.initApplicationContext(PageContextImpl.java:3030) at lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:119) at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:43) at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2405) at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2395) at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2363) at lucee.runtime.engine.Request.exe(Request.java:46) at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1084) at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1032) at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102) at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

** Stacktrace of the first request**
stop thread (66) because run into a timeout path: /var/www/myclientname/www/index.cfm (/var/www/myclientname/www/Application.cfc).;java.lang.Throwable;java.lang.Throwable at lucee.runtime.db.DatasourceConnectionPool.getDatasourceConnection(DatasourceConnectionPool.java:77) at lucee.runtime.type.scope.storage.IKHandlerDatasource.loadData(IKHandlerDatasource.java:36) at lucee.runtime.type.scope.storage.IKStorageScopeSupport.hasInstance(IKStorageScopeSupport.java:213) at lucee.runtime.type.scope.ScopeContext.hasExistingCFSessionScope(ScopeContext.java:549) at lucee.runtime.type.scope.ScopeContext.hasExistingSessionScope(ScopeContext.java:487) at lucee.runtime.PageContextImpl.initApplicationContext(PageContextImpl.java:3030) at lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:119) at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:43) at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2405) at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2395) at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2363) at lucee.runtime.engine.Request.exe(Request.java:46) at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1084) at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1032) at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102) at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)


#2

Although the scenario is different, the error looks the same as my issue surrounding DB connections not being closed in the DB pool:


#3

Yes, I have run into major issues with using database session storage since 5.2.x. I had some back-and-forth with the engineers about it, but could not pinpoint the exact issue in order to open a bug report. We had lockups and heap overflows. However, it all runs great on 5.1.x.

Apparently, something changed with the serialization of the session scope in one of the releases with 5.2.x. If the scope is large, it cannot handle it. Reducing the session timeout seemed to help a little, but was not a complete solution.

For now, we had to abandon using a database for shared sessions. It’s not ideal, but at least our websites stay up.


#4

Is there a ticket for that?

@IamSigmund Are you aware of such a regression?


#5

I have found 5.2.4.37 to be very unstable. My site locks up (requests just hang) within a day on that version. Version 5.2.1.9 is very stable by comparison.

I’m not using clustering. I’m using memory storage for sessions. So, I think there may be a larger problem than this thread described. I added my #metoo to https://luceeserver.atlassian.net/projects/LDEV/issues/LDEV-1587 anyway.


#6

Igal, we don’t have an official regression identified, but as we look into it I think the evidence it building. FYI to everyone watching this one, it will be a top priority in our end-of-year planning work. Stay tuned. @sbleon


#7

We had an issue with it and had to stop using it, but in our case the problem was there in Railo as well. I didn’t conclusively get to the bottom of it but I suspect in our case it came down to serialization failing on objects (i.e CFCs or Java objects) that were stored in the session scope.

We ended up “rolling our own”- our main concern was just auto-login rather than preserving an entire session, so storing a token in a cookie and matching that against a user account to recreate a login.


#8

Any news on this, @IamSigmund? We’re now frozen on 5.2.1.9 until this issue gets resolved.


#9

I am also seeing hanging threads that never timeout which appear to be related to jdbc connections never timing out and hanging forever. I have to restart my server daily. I’m running Lucee 5.2.4.37 on Windows x64 and sql server.

I created a bug report here: https://luceeserver.atlassian.net/browse/LDEV-1622


#10

Hey Leon. No official plans to report on this one, yet, but, we have gotten similar reports from other users, so it’s very high on the priority list as we get ready to start 2018 sprints. Ideally, we’ll have this fixed in the next RC, which should be out end-of-January. More soon!


#11

Sorry, I just realized this wasn’t very clear. I meant we hope to have the memory/session/etc. problems addressed by the next next RC. :wink: That is, the current RC is 5.2.5-RC, which will come out end-of-Dec./beginning-of-Jan. But that likely won’t contain any new fixes. The 5.2.6-RC (slated for end-of-January) would be the next RC where we could get this addressed.


#12

How can we help to make sure the instability gets fixed? I know that it may
be hard to reproduce the problem, and so far I haven’t been able to get any
good info on the state of Lucee when it has become unresponsive. Any
suggestions on gathering more useful data?

Leon


#13

Hi Leon. Thanks for the offer to help. I’ll ponder that. Creating valid test cases/environments is easily one of our biggest challenges, given the huge range of use cases that Lucee solves.


#14

We’re running into the same issue. Working on a new environment on AWS with clustered sessions and the behavior described in the original post happens within seconds of traffic hitting the backend on version 5.2.3.35.

I have saved the stacktraces in FR-CLOUD. There are about 100 threads waiting on the following request:


java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:242)

java.io.File.exists(File.java:819)

sun.misc.URLClassPath$FileLoader.getResource(URLClassPath.java:1282)

sun.misc.URLClassPath.getResource(URLClassPath.java:239)

java.net.URLClassLoader$1.run(URLClassLoader.java:365)

java.net.URLClassLoader$1.run(URLClassLoader.java:362)

java.security.AccessController.doPrivileged(Native method)

java.net.URLClassLoader.findClass(URLClassLoader.java:361)

java.lang.ClassLoader.loadClass(ClassLoader.java:424)
- locked <0x1C357766> (a java.lang.Object)

java.lang.ClassLoader.loadClass(ClassLoader.java:357)

org.apache.felix.framework.Felix.loadBundleClass(Felix.java:1922)

org.apache.felix.framework.BundleImpl.loadClass(BundleImpl.java:959)

org.apache.felix.framework.Felix.loadClass(Felix.java:106)

lucee.runtime.osgi.OSGiUtil.loadClass(OSGiUtil.java:304)

lucee.commons.lang.ClassUtil$OSGiBasedClassLoading.loadClass(ClassUtil.java:844)

lucee.commons.lang.ClassUtil$OSGiBasedClassLoading.loadClass(ClassUtil.java:849)

lucee.commons.lang.ClassUtil._loadClass(ClassUtil.java:284)

lucee.commons.lang.ClassUtil.loadClass(ClassUtil.java:172)

lucee.transformer.library.ClassDefinitionImpl.getClazz(ClassDefinitionImpl.java:79)

lucee.runtime.cache.CacheConnectionImpl.<init>(CacheConnectionImpl.java:53)

lucee.runtime.listener.ModernApplicationContext.toCacheConnection(ModernApplicationContext.java:898)

lucee.runtime.listener.ModernApplicationContext.toCacheConnection(ModernApplicationContext.java:876)

lucee.runtime.listener.ModernApplicationContext._initCache(ModernApplicationContext.java:829)

lucee.runtime.listener.ModernApplicationContext._initCache(ModernApplicationContext.java:821)

lucee.runtime.listener.ModernApplicationContext.initCache(ModernApplicationContext.java:792)

lucee.runtime.listener.ModernApplicationContext.getCacheConnection(ModernApplicationContext.java:740)

lucee.runtime.PageContextImpl.getCacheConnection(PageContextImpl.java:3426)

lucee.runtime.cache.CacheUtil.getCacheConnection(CacheUtil.java:136)

lucee.runtime.type.scope.storage.IKHandlerCache.getCache(IKHandlerCache.java:90)

lucee.runtime.type.scope.storage.IKHandlerCache.loadData(IKHandlerCache.java:23)

lucee.runtime.type.scope.storage.IKStorageScopeSupport.hasInstance(IKStorageScopeSupport.java:213)
- locked <0x4E91E337> (a java.lang.Class)

lucee.runtime.type.scope.ScopeContext.hasExistingCFSessionScope(ScopeContext.java:557)

lucee.runtime.type.scope.ScopeContext.hasExistingSessionScope(ScopeContext.java:487)

lucee.runtime.PageContextImpl.initApplicationContext(PageContextImpl.java:3030)

lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:119)

lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:43)

lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2405)

lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2395)

lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2363)

lucee.runtime.engine.Request.exe(Request.java:46)

lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1048)

lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:996)

lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102)

lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)

javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)

org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)

org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)

org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)

org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)

org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)

sun.reflect.GeneratedMethodAccessor140.invoke(Unknown source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

java.lang.reflect.Method.invoke(Method.java:498)

com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:134)

com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doNext(FusionReactorRequestHandler.java:763)

com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doHttpServletRequest(FusionReactorRequestHandler.java:351)

com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doFusionRequest(FusionReactorRequestHandler.java:214)

com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.handle(FusionReactorRequestHandler.java:800)

com.intergral.fusionreactor.j2ee.filter.FusionReactorCoreFilter.doFilter(FusionReactorCoreFilter.java:36)

sun.reflect.GeneratedMethodAccessor137.invoke(Unknown source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

java.lang.reflect.Method.invoke(Method.java:498)

com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:71)

sun.reflect.GeneratedMethodAccessor136.invoke(Unknown source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

java.lang.reflect.Method.invoke(Method.java:498)

com.intergral.fusionreactor.agent.filter.FusionReactorStaticFilter.doFilter(FusionReactorStaticFilter.java:54)

com.intergral.fusionreactor.agent.pointcuts.NewFilterChainPointCut$1.invoke(NewFilterChainPointCut.java:41)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java)

org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)

org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)

org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)

org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)

org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)

org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)

org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:676)

org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)

org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)

org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132)

org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)

org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2458)
- locked <0x1E0B1FD> (a org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper)

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

java.lang.Thread.run(Thread.java:748)

Locked ownable synchronizers: 
- java.util.concurrent.ThreadPoolExecutor$Worker@67852C19
- java.util.concurrent.locks.ReentrantLock$FairSync@1107D2CC

Here’s a stack trace from one of the hanging threads:

- waiting on <0x4E91E337> (a java.lang.Class)

lucee.runtime.type.scope.ScopeContext.hasExistingCFSessionScope(ScopeContext.java:557)

lucee.runtime.type.scope.ScopeContext.hasExistingSessionScope(ScopeContext.java:487)

lucee.runtime.PageContextImpl.initApplicationContext(PageContextImpl.java:3030)

lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:119)

lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:43)

lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2405)

lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2395)

lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2363)

lucee.runtime.engine.Request.exe(Request.java:46)

lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1048)

lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:996)

lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102)

lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)

javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)

org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)

org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)

org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)

org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)

org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)

sun.reflect.GeneratedMethodAccessor140.invoke(Unknown source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

java.lang.reflect.Method.invoke(Method.java:498)

com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:134)

com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doNext(FusionReactorRequestHandler.java:763)

com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doHttpServletRequest(FusionReactorRequestHandler.java:351)

com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doFusionRequest(FusionReactorRequestHandler.java:214)

com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.handle(FusionReactorRequestHandler.java:800)

com.intergral.fusionreactor.j2ee.filter.FusionReactorCoreFilter.doFilter(FusionReactorCoreFilter.java:36)

sun.reflect.GeneratedMethodAccessor137.invoke(Unknown source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

java.lang.reflect.Method.invoke(Method.java:498)

com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:71)

sun.reflect.GeneratedMethodAccessor136.invoke(Unknown source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

java.lang.reflect.Method.invoke(Method.java:498)

com.intergral.fusionreactor.agent.filter.FusionReactorStaticFilter.doFilter(FusionReactorStaticFilter.java:54)

com.intergral.fusionreactor.agent.pointcuts.NewFilterChainPointCut$1.invoke(NewFilterChainPointCut.java:41)

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java)

org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)

org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)

org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)

org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)

org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)

org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)

org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:676)

org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)

org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)

org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132)

org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)

org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2458)
- locked <0x5EEE6FB3> (a org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper)

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

java.lang.Thread.run(Thread.java:748)

Locked ownable synchronizers: 
- java.util.concurrent.ThreadPoolExecutor$Worker@2443C6C8
- java.util.concurrent.locks.ReentrantLock$FairSync@6B05B81F

Trying to downgrade now. If I can provide any other details let me know.

Thanks,
-Nick


#15

ATM it is not clear to me, if this is a single issue people experience or multiple that looks as they are related. there are 3 ways to solve this (#1 is the best way and #3 the worst).

  1. reproduce the issue on a test system and then figuring out exactly which version cause the issue.
  2. reproduce the issue and then figuring out what the problem causing this.
  3. based on existing data (stacktraces) try to figuring out what could possible cause this.

for #1 and #2 i need to know if and how to reproduce this. So i will start with #3, if you have any further input please add to the ticket from Paul.
https://luceeserver.atlassian.net/projects/LDEV/issues/LDEV-1587
i will post further information in this ticket and if this lead to more than one issue i will point to other tickets in that ticket.