Lucee 5.3.7.47: crash instance

Hello,

in my recent adventure to upgrade lucee from version 5.3.6.61 to the latest snapshot i had some problems so i downgraded from the latest snapshot 5.3.8.138-SNAPSHOT to version 5.3.7.47

today i found the lucee service not responding (the cpu was normal) and i had to downgrade again from version 5.3.7.47 to version 5.3.6.61

I attach a part of catalina.out: Dropbox - catalina.zip - Simplify your life

did you find anything strange in the logs?


UPDATE1

initially I thought it was not a problem related to the CPU, instead the problem also occurred with version 5.3.6.61 (the CPU arrives 90/100% and lucee does not respond)

I really don’t know what the problem could be…


UPDATE2

today I realized that after each crash some java dump files are generated in the root.


@andreas I also shared the log on dropbox


Lucee 5.3.7.47
Servlet Container Apache Tomcat/9.0.19
Java 11.0.9 (AdoptOpenJDK) 64bit
OS Linux (4.15.0-132-generic) 64bit
Architecture 64bit

I can’t access the file at google drive. The access seems to be restricted.

I would caution you against posting heap dumps publicly. A malicious user can look in them and find out things such as passwords and anything else in memory of your server.

2 Likes

thanks for the advice. I was not aware of this … -_-

I look a peek at one of the heap dumps, and it’s releated to unbound creation of threads. There is 75% of the heap ( 1.1 GB) taken up by an instance of java.lang.ThreadGroup. Looking at the javacore files, they show a HUGE amount of threads created in the JVM. I see 4,800 instance of java.lang.Thread and I also see 67,000 instance of com.intergral.apm.transit.txn.ThreadMemorySnapshot I’ve seen some issues with the FR memory snapshot feature in some apps with specific behaviors such as try/catching a lot of exceptions. I would recommend turning off the snapshot features of fusionreactor and looking into what is creating all the threads. Possibly uses of the parallel array or struct iterating functions.

hello Brad and thanks for the tip.
Fusion Reactor recently installed it to check for errors (I didn’t use it before). I don’t even know how to turn off snaphsots (I couldn’t find the setting).

I’m not 100% if this is the setting that creates the ThreadMemorySnapshot objects in FR, but in your FR admin go to “Debug” > “Event Snapshot” > and change “Enabled” to be set to “disabled”.

This will at least rule out that from interfering in our tests.

1 Like