EHCache works for several days and then chokes Tomcat with heavy use

I’m using EHCache to cache YouTube API requests with a ttl of 30 minutes. The JSON response of each API request is cached, and at peak times, I am getting 15K page views per 30 minutes, so at minimum, there will be 15K cache entries in the cache. However, many page views make multiple API requests, so it’s more likely there will be upwards of 40K-50K entries in the cache. I’m getting upwards of 350K page views per day, so needless to say, this cache collection is getting hit pretty hard.

What’s happening is that everything works great for several days and then Tomcat basically chokes and stops responding - using 100% of the CPU. If I disable caching altogether, I can run the server forever, so I know this issue is related to my EHCache settings.

I’m running Lucee 5.1.3.18 with EHCache 2.10.0.21.

The question is… what can I do to troubleshoot this?

Log files, Thread dumps, Heap dumps

What method(s) would you use in a situation where the server works great for many days and then all of a sudden it stops responding? At the time of failure, the server is not responding. How would I get a thread dump or a heap dump?

Assuming that you’re using the HotSpot JVM. See http://www.oracle.com/technetwork/articles/java/vmoptions-jsp-140102.html

Note in particular the options:

-XX:ErrorFile=./hs_err_pid<pid>.log

-XX:-HeapDumpOnOutOfMemoryError

Igal… thanks. This looks very useful. I am running Lucee right out of the box, so I have no idea what JVM is being used. How can I tell if I’m running the hotspot JVM?

If you don’t know then you’re running HotSpot :wink: That’s the default JVM from Oracle.

1 Like