Lucee delivering empty pages

dev support
1

Hi!

I’ve got a server that just startet to do deliver empty pages (1 line source/ carriage return) HTTP Status 200 just over night.
I cannot yet get this machine back up running, although I changed, the Java Version, updated Lucee, and changed some jvm_args
I suspect my problem might be related to one of these?
https://luceeserver.atlassian.net/browse/LDEV-2685
https://luceeserver.atlassian.net/browse/LDEV-2376
https://luceeserver.atlassian.net/browse/LDEV-2514

The server runs fine on the first “main” virtual host, that is serving files from
d:\wwwroot
I’ve got another host that is serving files from another directory: d:\wwwroot\folder\folder_www\ based on the host_header (Which is a setup to test files generated from our CMS)

According to [LDEV-2514] - Lucee Lucee might have a problem with application.cfc while searching up to the root, but I’m not sure if that is really the culprit
However, I have several application.cfc in that directory-structure as that’s the way the CMS works.

This has not been a problem for month, until I made a windows update last Saturday. No Java update or so, just a simple windows update.

Lucee logs are empty, jvm logs are empty, FusionReactor does not show issues (but also no database-activity, although I create a session cache database), Resin access log shows status 200.
I removed all generated class-files: No changes
I touched the files, that are delivering empty content: No changes.

So currently I’m stuck, maybe someone here has an idea where to start?

Thanks in advance
Michael

*Here is my Stack

OS: Windows Server 2016
Java Version: Was Oracle JDK => switched to AdoptJDK while searching for error
Tomcat Version: No Tomcat but Caucho Resin 4
Lucee Version: Upgraded to 5.3.4.80 while searching for error

2

Try the latest snapshot?

3

Hi did have a similar issue with some website and I end up poking the different website every 5 minutes non stop.

For me a website was running fine for few hours and suddenly if I was something in /de folder wasn’t accessible but in the /fr folder was accessible.

I cannot test with the latest snapshot due to policies with the client…

Lucee version Lucee 5.3.3.62

Servlet Container
Apache Tomcat/9.0.29
Java
11.0.5 (Oracle Corporation) 64bit
OS
Linux (4.15.0-1060-aws) 64bit
Architecture
64bit

4

Michael, you mention having changed “some jvm args”. Do you have a maxmetaspace set? To a relatively low number, like hundreds of megs? Try removing it (carefully). For more, see a post I’ve done on the problem especially for CF, as Adobe sets it by default–mistakenly, I would argue. More at Solving metaspace errors, once and for all - Charlie Arehart - Server Troubleshooting.

Yet when you say the logs are empty, do you mean lacking anything related to this, or truly empty (when normally they would have something)? The latter would be a bigger concern, of course.

5

I tried the latest snapshot, which did not help.

6

I added -Djdk.attach.allowAttachSelf=true in the jvm_args
According to one comment in [LDEV-2685] - Lucee

Metaspace settings are there as well: -XX:MaxMetaspaceSize=5g -XX:MinMetaspaceFreeRatio=50 -XX:MaxMetaspaceFreeRatio=80

In fact, JVM Settings are making no difference for me.

The logiles just look fine, there are no errors
Status 200, 0 KB size for an empty request.

7

Micha, ok on those confirmations. So back to square one: you say in your first note that things changed “only because you did a Windows update”. Well, did that involve a reboot of Windows? If so, then note that something could have been implemented/changed in days or weeks before that, which did not take place until that restart.

It would be interesting if you could confirm (from the logs) when you last did restart Windows (or more specifically, Lucee). Then you’d know how far back in time to consider any other possible changes, whether to code or config.

And since you say this affects only one site/virtual host, consider then what changes may have been made either to the Apache conf for that host, or the Lucee conf (for that context), or of course the code in that site. (There are code changes that won’t take effect until the application or the app server restarts.)

Finally, since you are wondering if the search for application.cfc is a factor, help yourself out with a quick, easy test:

  1. In the webroot of the site in question, introduce a new test folder

  2. In that folder, put in a test.cfm that does something simple, like <cfoutput>#now()#</cfoutput> (or of course the script equivalent if you prefer).

  3. Finally, most important, put a blank application.cfm (or cfc, it doesn’t matter) in that same folder, which will stop Lucee (or CF) from looking “up” to parent folders for any application.cfc/cfm they may have.

If a call to that test.cfm (through apache, via that virtual host) works, whereas a call to a page in the folder above it is “blank”, then you know that your problem IS indeed about some application.cfc/cfm processing in a folder above that.

In that case, time to start debugging that application.cfc/cfm, to find where things are failing. If you can’t do old-school cfdump/cfabort, you could use cflog (which writes to the application.log by default)–or script equivalents.

And yep, you could do this last step first if you want. Some would. I just wanted to highlight that other idea above, as folks often don’t consider it, and it can prove diagnostically useful (and does NOT require changes to the app in the folders above it, which may be important when the app IS live and perhaps working “sometimes”, so you don’t want to or can’t “touch” it).

Let us know if you learn anything more, from this or other analysis.

8

Hi Charlie,

thank you for your feedback. I removed all application.cfcs up to the root, which did not change anything. I still have no idea what’s going on there, but I decided not to spend more time in investigating.

So, to get the system back running, I installed a second instance of Resin for my other webroot and made some changes in Nginx so it does not change anything for the users.

At least, now I can run different versions of Lucee for testing. So the effort is not wasted at all.

Thanks again. See you next CFCamp.

Regards
Michael