Migration Performance Issue - memory leak

dinuk · July 19, 2022, 10:20am

We are migrating an application from Adobe CF to Lucee, and the application is of heavy traffic one. As part of the functional / load test its observed that the heap memory is growing up gradually and server not responds after 2-3hrs and need a service restart. Gone through various articles and couldn’t find the exact cause of the issue. Any thoughts on this will be greatly appreciated.

OS: Windows server 2019
Java Version: 1.8.0_301 (Oracle Corporation) 64bit
Tomcat Version: Apache Tomcat/9.0.62
Lucee Version: Lucee 5.3.9.141

JVM options

-XX:PermSize=512m
-XX:NewSize=1024m
-XX:MaxNewSize=2048m
-XX:SurvivorRatio=8
-XX:MetaspaceSize=256m
-XX:MaxMetaspaceSize=1024m
-XX:+UseCompressedOops
-XX:+DisableExplicitGC
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+CMSClassUnloadingEnabled
-XX:+CMSScavengeBeforeRemark
-XX:CMSInitiatingOccupancyFraction=70
-XX:+UseG1GC
-Xss128k
-Xms2048m
-Xmx10420m

Server Admin settings

session type: Application
session timeout: 30

Zackster · July 19, 2022, 11:03am

without any other details, I’m guessing that coz lucee passes by reference, that could be the culprit?

you can also try doing a heap dump to see what’s in the heap, or try fusion reactor

https://lucee.daemonite.io/t/lucee-java-heap-space-error-on-script-after-lucee-reinstallation/10649/8?u=zackster

dinuk · July 19, 2022, 2:36pm

Is it something memory leak?

Zackster · July 19, 2022, 7:50pm

looks like it, can you dig into it and see what’s in there?

dinuk · July 20, 2022, 9:00am

I think almost reached it.

We were using isSafeHTML in ColdFusion for the input validation. That got replaced with AntiSamy in our Lucee project. And heap dump shows it is causing a lot. When removed the validation, its running pretty well.

Any other recommendation for isSafeHTML

Zackster · July 20, 2022, 9:02am

you can try sanitizehtml() :: Lucee Documentation (needs the latest ESAPI ext)

how are you loading the antiSamy jars? they only need to be loaded once, loaded jars don’t get GCed, which is probably causing the problem

dinuk · July 20, 2022, 9:10am

In each request the input needs validation and was loading every request. Yes, moving them to application scope as needed to load only once.

Thanks a lot for the inputs.

Zackster · July 20, 2022, 9:36am

my pleasure, glad I could help!

dinuk · July 20, 2022, 11:52am

@Zackster, Moved them to application scope. Issue got resolved and its amazing performance. Really appreciate your help.

Phillyun · July 20, 2022, 12:31pm

GTK