Hi everyone.
I have a problem with an application that I deployed for testing a few days ago.
The machine that hosts it (on Layershift) has few resources (2.5 GB of RAM), but the software, after a day of activity, returns the error:
“unable to create new native thread”
This error seems completely senseless to me because only two people are using it to test it.
So I dipped into the shell, and looked up the number of Java processes running:
ps -elfT | grep java | wc -l
The number of Java processes, which starts at 60 just after boot, continuously increases (in one day) to around 4000, before collapsing into the error “unable to create new native thread”.
I have a large application where the number of processes reaches a maximum of 900 and remains constant over time.
I never have this problem even though my other applications are deployed with Commandbox and Undertown; this one where I have problems is instead deployed on Tomcat.
I don’t really know how Java works.
I imagine these processes need to be killed somehow, but it seems my application doesn’t do that.
In catalina.log ho found many lines like this:
08-Jul-2024 10:12:53.768 SEVERE [Thread-4] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@59c7d456]) and a value of type [lucee.commons.lang.types.RefBooleanImpl] (value [false]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
08-Jul-2024 10:12:53.769 SEVERE [Thread-4] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@59c7d456]) and a value of type [lucee.commons.lang.types.RefBooleanImpl] (value [false]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
My stack is:
- Tomcat 9
- OpenJdk 11
- Lucee 5.4.6.9
- Coldbox & Wirebox 6.9.0
- OS: AlmaLinux release 9.3
Do you have any ideas?
The only one I have in mind is to use FusionReactor.
Thanks for any advice.