Getting "502 Proxy Error" when trying to generate a Excel file with large dataset

dev support
1

I am migratin my application from CFML to Lucee. Here I am trying to generate Excel file with xml data. For less data where server takes less time to process it is working fine but for large data like 50000 rows server is processing for longer time and in between like after 3 - 5 minutes, I am getting “502 Proxy Error”. Attaching the image of the error here:

image (2)
image (2)3826×1863 96.2 KB

in my “apache2.config” file I have code like:

Timeout 3000
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 20

<IfModule mod_proxy.c>
        Timeout 3000
        ProxyTimeout 3000
        ProxyBadHeader Ignore
        ProxyPreserveHost On
        ProxyPassMatch ^/(.+\.cf[cm])(/.*)?$ http://127.0.0.1:8008/$1$2
        ProxyPassMatch ^/(.+\.cfml)(/.*)?$ http://127.0.0.1:8008/$1$2
        # optional mappings
        #ProxyPassMatch ^/flex2gateway/(.*)$ http://127.0.0.1:8008/flex2gateway/$1
        #ProxyPassMatch ^/messagebroker/(.*)$ http://127.0.0.1:8008/messagebroker/$1
        #ProxyPassMatch ^/flashservices/gateway(.*)$ http://127.0.0.1:8008/flashservices/gateway$1
        #ProxyPassMatch ^/openamf/gateway/(.*)$ http://127.0.0.1:8008/openamf/gateway/$1
        #ProxyPassMatch ^/rest/(.*)$ http://127.0.0.1:8008/rest/$1
        ProxyPassReverse / http://127.0.0.1:8008/
</IfModule>

and while I am looking into logs of apache I am getting this:

[Tue Dec 08 07:16:14.185909 2020] [proxy:error] [pid 10020:tid 140463935837952] [client 172.20.44.239:52760] AH00898: Error reading from remote server

OS: Linux
Java Version: 1.8.0_261
Tomcat Version: Apache Tomcat/9.0.35
Lucee Version: Lucee 5.3.6.61

2

Did you try this? In the backend they use Jetty, but I would say that should work also for you.

3

@andreas Yes I tried this. As I don’t have

ProxyPass / http://www.dom.fi:8080/ retry=1 acquire=3000 timeout=600 Keepalive=On

line in my apache2.conf code, I added it, restarted Apache, but issue was not resolved. Additionally after adding this line when I am redirecting to my home page it is looking for “index.cfm” file, but I have configured apache to look for some other file. That is why I removed it.

4

tomcat has a timeout setting too

5

Thanks @Zackster for replying, I checked tomcat timeout, it is set to 30 minutes.
tomcatTimeout

1 Like
6

How are you generating your Excel file exactly? You might want to take a look at this:

1 Like
7

Did you adapt the line above to work with your environment, or not? Because just adding it can’t work. And also: this line should only affect your index welcome page. And: The port is different, your configuration in apache2 uses 8008, and also is your proxy domain name. Let’s forget that for now… If @Julians post didn’t help, first thing to do is to see what Tomcats direct reponse looks alike. That means… Try accessing your big excel file through your Tomcat port directly, bypassing Apache2: That should work if you generate it by browsing from that machine locally to port 8008 (I can see that port configurated in your apache2.conf, so I’m assuming you are running Lucee on 8008). Can you load the excel file from Tomcat directly or is it generating an error?

8

Thanks @Julian_Halliwell, I am getting same error when I am generating report in HTML format, with lot of queries to fetch data. So I think this error is not because of Excel file size but this is because of late response from the server.

9

Thanks for the reply @andreas. I tried accessing through the Tomcat directly bypassing Apache and after a long processing I got different error which is ERR_EMPTY_RESPONSE, pasting the image of the error bellow.
page_isnt_working

10

Ok. Did you find any valuable information in tomcats catalina logs?

11

No andreas.
Here i would like to mention some things:

  1. In the apache.conf, the below config is added by mod_proxy while installing lucee.

    ProxyPreserveHost On
    ProxyPassMatch ^/(.+.cf[cm])(/.)?$ http://127.0.0.1:8008/$1$2
    ProxyPassMatch ^/(.+.cfml)(/.    )?$ http://127.0.0.1:8008/$1$2
    # optional mappings
    #ProxyPassMatch ^/flex2gateway/(.)$ http://127.0.0.1:8008/flex2gateway/$1
    #ProxyPassMatch ^/messagebroker/(.    )$ http://127.0.0.1:8008/messagebroker/$1
    #ProxyPassMatch ^/flashservices/gateway(.)$ http://127.0.0.1:8008/flashservices/gateway$1
    #ProxyPassMatch ^/openamf/gateway/(.    )$ http://127.0.0.1:8008/openamf/gateway/$1
    #ProxyPassMatch ^/rest/(.*)$ http://127.0.0.1:8008/rest/$1
    ProxyPassReverse / http://127.0.0.1:8008/
  2. We are hosting application with a hostname “foo.domain.com” and in a custom port “7444” via apache. where the url would be “https://foo.domain.com:7444/”. The lucee/tomcat is installed in 8008 port.

I see 2 things here,

  1. Poor performance, though the RAM is 32 GB (-Xms2g -Xmx8g). No cache configurations made in server admin console.
  2. When accessing report(s), like 20K records seeing the proxy error. In the apache log, i see the below:
    [Wed Dec 16 04:56:25.475340 2020] [proxy_http:error] [pid 25777:tid 140362232338176] (20014)Internal error (specific information not available): [client 153.64.42.163:59529] AH01102: error reading status line from remote server 127.0.0.1:8008
    [Wed Dec 16 04:56:25.475471 2020] [proxy:error] [pid 25777:tid 140362232338176] [client 153.64.42.163:59529] AH00898: Error reading from remote server returned by /page.cfm

We are go to production on 18-Dec-2020. Please help here to fix the problem.

Thanks for everyone who are helping here.
Regards,
Madhusudhana

12

firstly, I’d be using ajp rather than http

https://viviotech.github.io/mod_cfml/install-win-apache.html

turn on debugging, including implicit variable access, the main problem is your code is slow,
I suspect you have some scope problems, fixing that will help a lot

2 Likes