App Folders Sync and Lucee performance

dev support
1

Hi,

I trust everyone is doing well. We’re currently in the midst of transitioning our CF applications to Lucee. I have a few questions to ask. Thanks in advance.

OS : Windows Server 2019 Datacenter 64bit
Java Version : 11.0.20
Tomcat Version : 9
Lucee Version : 5.4.3.2
DB : MySQL
Sync: SyncThing - For syncing folders across all nodes

All of our servers are load-balanced and application folders are synced in real time. I wanted to confirm/clarify that all WEB-INF folders can’t be replicated across all nodes as they are machine specific. Are there any other folders I should be avoiding during sync? Thanks

We have the below folder structure. Each app folder may contain 2 WEB-INF folders.

Application Folder 1
WEB-INF
api
WEB-INF

Application Folder 2
WEB-INF
api
WEB-INF

Application Folder 3
WEB-INF
api
WEB-INF

My other question is regarding the 1-second delay, I have applied the address="::1" to the <Connector protocol="AJP/1.3"> element but still no luck. I have even tried a few additional ones.

<Connector protocol=“AJP/1.3” port=“8009” secret=“########” secretRequired=“true” redirectPort=“8443” keepAliveTimeout=“-1” connectionTimeout=“60000” maxThreads=“1000” packetSize=“65536” address=“::1” />

Last question, what would be the recommended JAVA setting for a busy environment?

Any help would be greatly appreciated.

Kind Regards

2

Have you verified a direct connections to your Tomcat through port 8888, to see if the 1s delay goes away? Can you confirm that?

3

Hi thanks for the reply, yes I have and the direct connection has no delays. It is related to IIS.

4

I have just recalled that there was an entry on github about that… That the ::1 solution only worked with IPv6 Interface activated. Can you check if this applies to you? Also, I remember that @carehart had some intetsting findings.

Please see this thread. Don’t know if have seen it already (read i it completely):

5

Hi,

I did try that before posting it here and added the below to the server.xml but no luck. I thought I might be missing the obvious.

Connector protocol=“AJP/1.3”
port=“8009”
secret=“xxx”
secretRequired=“true” redirectPort=“8443”

Connector protocol=“AJP/1.3”
port=“8009”
secret=“xxxxx”
secretRequired=“true”
redirectPort=“8443” maxThreads=“1000” packetSize=“65536” address=“::1”

Regards

6

Did you try 0.0.0.0?

7

I haven’t but on which one first or second? Thanks

8

I think there should only be one AJP connector definition in your server.xml. Could that be why ::1 isn’t working?

9

Julian, yes, there CAN be two. See the end of the thread that Andreas pointed to in his comment above.

Tanvir, why (in your example above showing two) do you have the address (and some other attributes) only in the second? Duplicate the second, and have one as ::1 and one as 127.0.0.1.

Or just have one that uses 127.0.0.1. You don’t show yet that you’ve used it at all yet.

Finally, as for why some need one, the others, or both, see that other thread where I’d shared more. As for what address’s one may need and why, or why it is that this delay happens when it’s not the “correct” value, I never got around to a blog post where I’d dig into and document more. If anyone’s motivated, the community would thank you–well, maybe. :slight_smile:

1 Like
10

Hi @carehart

When I pasted it here it wouldn’t change the second one so I deleted the tag but I do have 2 as below.

First one.

<Connector protocol=“AJP/1.3”
port=“8009”
secret=“XXXXX”
secretRequired=“true”
redirectPort=“8443” address=“127.0.0.1” />

Second one.

<Connector protocol=“AJP/1.3”
port=“8009”
secret=“XXXXX”
secretRequired=“true”
redirectPort=“8443” address=“::1” />

11

Ok, so:

  1. You’re saying you’ve tried each one alone also, and always have the “1-second delay” you refer to at the outset?
  2. And you’re restarting Lucee between changes and tests, right?
  3. And I assume you’ve used the Lucee installer, rather than installing a Lucee war on a Tomcat you installed, right?
  4. Can you confirm that the server.xml you’re changing is indeed the right one, by introducing an error (such as leaving off the opening < of that connector line), which should either prevent Lucee starting or at least prevent your test page running at all?
  5. Do you have anything at all confirming that the “1-second delay” is not in your app itself? Debugging output? Logging? FusionReactor?
  6. One of your later comments indicate you’re using IIS. You’ve not clarified, but I assume you’re using the boncode connector, right?
  7. Have you setup all this the same way (iis talking to Lucee, installed the same way) on some development machine or VM? I’m wondering if you’d see this delay there. And if not, we’d wonder what was different. (Even if you may prefer to use Commandbox on your local machine, you can certainly install Lucee separately from that–and there can certainly be benefit, as in this case, for more closely mimicking that other server.)

Finally, about your other two questions (about syncing files and about Java settings), they’re being lost in all this. You should consider breaking this into it’s own, same with your last one about Java.(The current title relates to the sync aspect, at least in part, and it’s not clear if that has any connection to any performance issue.)

1 Like
12

My comments are in BOLD.

  1. You’re saying you’ve tried each one alone also, and always have the “1-second delay” you refer to at the outset? - That’s correct
  2. And you’re restarting Lucee between changes and tests, right? - I am indeed, sometimes even twice
  3. And I assume you’ve used the Lucee installer, rather than installing a Lucee war on a Tomcat you installed, right? - I used the Lucee installer
  4. Can you confirm that the server.xml you’re changing is indeed the right one, by introducing an error (such as leaving off the opening < of that connector line), which should either prevent Lucee starting or at least prevent your test page running at all? - I wondered as well and did the same and it errors so I was changing the correct file
  5. Do you have anything at all confirming that the “1-second delay” is not in your app itself? Debugging output? Logging? FusionReactor? - For now I am just outputting #now()# there is nothing else in the file, in the network console the TTFB is most of the time 1.2 or 1.3 seconds, if I run the same file in CF it’s 60ms
  6. One of your later comments indicate you’re using IIS. You’ve not clarified, but I assume you’re using the boncode connector, right? - Sorry It’s IIS with boncode connector
  7. Have you setup all this the same way (iis talking to Lucee, installed the same way) on some development machine or VM? I’m wondering if you’d see this delay there. And if not, we’d wonder what was different. (Even if you may prefer to use Commandbox on your local machine, you can certainly install Lucee separately from that–and there can certainly be benefit, as in this case, for more closely mimicking that other server.) - Currently I have Lucee, IIS on the same VPS

I will start a separate thread for my other 2 questions. Thanks

Thanks

13

Thanks for all the clarifications, though on the last point I think you misunderstood my intention. Please read it again and ask if you need me to reword/clarify it.

  1. Also, in your boncode settings file (if you use one), is the server element set to localhost or to something else?
  2. And what do you get when you ping localhost (on the windows machine)?
  3. Have you tried setting that server element of the boncode settings file to be 127.0.0.1 or ::1? I’m not saying you should have to: it’s just a diagnostic. See the boncode docs for what I mean by that server element, including the section above the place where that link will take you in the large doc page.
    And you may need to restart iis (or recycle the app pool processing the request) for such settings file changes to take effect–I didn’t see that clarified in the docs, but they are substantial. Perhaps Bilal or someone else can confirm.
  4. Note that the docs DO propose one OTHER possible contributor to slowness can be the use of the secret in the connector (required by default in tomcat since the ghostcat fiasco of early 2021). Bilal proposes one can consider using requiredSecret=”false” on the server.xml connector line. See the brief docs discussion of this for more at the bottom of the troubleshooting table.
  5. I realize these are all shots in the dark. There should be more clarity by studying the logs, whether tomcat’s or boncode one’s you can enable, as also discussed in the docs.
1 Like
14

Hi @carehart,

I have 8 nodes on 2 different data centres all configured the same way and delay is on all of them, I also used Commandbox but had no luck.

  1. localhost
  2. ::1
  3. I have tried with 127.0.0.1 and ::1 in the “BonCodeAJP13.settings” under the windows folder, after every change I am restarting IIS and tomcat
  4. requiredSecret=”false” is the same
  5. I will check the logs in more detail.

I also wondered that I may have missed something in the IIS install but I checked all the ones explained in the video.

15

Some suggestions:

(1) Delete the attribute keepAliveTimeout. That is, use just

<Connector protocol=“AJP/1.3” 
	port=“8009” 
	secret=“abra########cadabra” 
	secretRequired=“true” 
	redirectPort=“8443” 
	connectionTimeout=“60000” 
	maxThreads=“1000” 
    packetSize=“65536” 
	address=“::1” />

(2) Ensure that the values of the connector attributes match the respective values in the workers.properties file of the worker concerned. Using the above data, for example,

worker.list=workerName
worker.workerName.type=ajp13
worker.workerName.host=localhost
worker.workerName.port=8009
worker.workerName.secret=abra########cadabra
worker.workerName.connection_pool_timeout=60
worker.workerName.connection_pool_size=1000
worker.workerName.max_reuse_connections=1000
worker.workerName.max_packet_size=65536

Note:

  • In this example, the connector is used by just the one worker, WorkerName. Hence, maxThreads represents the total connection pool size.
  • However, suppose three workers, worker1, worker2 and worker3 share the above connector. Then maxThreads represents the maximum value of the sum of their respective pool sizes. With maxThreads=1000, you would then have, for example,
    worker.worker1.connection_pool_size=300,
    worker.worker2.connection_pool_size=300,
    worker.worker3.connection_pool_size=400.
  • You should also ensure that, for each worker, worker.workerName.max_reuse_connections never exceeds worker.workerName.connection_pool_size.
  • For details, see Pete Freitag’s blog-post on Tuning Tomcat IIS Connector worker properties.
16

address=“0” is what worked for me to resolve that 1sec delay, FYI

1 Like