Very slow file uploads

dev support
1

Hi there,

We are currently facing the same issue as @Jay.MC in File Upload Slow (extreme), though the solution there unfortunately didn’t work for us.

A simple test upload of a ~50MB file takes < 2s to process in our dev environment while it takes almost 3m on a freshly set up live environment using the Lucee installer.

Our tech stack for that system:

Debian GNU/Linux 12 (bookworm)
Server version name Apache Tomcat/9.0.86
Server built Feb 14 2024 08:15:12 UTC
Server version number: 9.0.86.0
Name Linux
Version 6.1.0-20-amd64
Architecture amd64
Home /opt/lucee/jre
Version 11.0.22+7
Vendor Eclipse Adoptium

Lucee 5.4.5.23

Here’s our most recently tested server.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Server port="8005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>
  <Service name="Catalina">
    <Connector port="8888" protocol="HTTP/1.1"  connectionTimeout="20000"  redirectPort="8443" />
    <Connector protocol="AJP/1.3" port="8009" packetSize="65536" secretRequired="true" redirectPort="8443" secret="secret"/>
    <Engine name="Catalina" defaultHost="127.0.0.1">
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>
      </Realm>
      <Host name="127.0.0.1" appBase="webapps" unpackWARs="true" autoDeploy="true">
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"  prefix="localhost_access_log" suffix=".txt"  pattern="%h %l %u %t &quot;%r&quot; %s %b" />
        <Valve className="mod_cfml.core" loggingEnabled="false"  maxContexts="200"  timeBetweenContexts="2000"  scanClassPaths="false"  responseCode="307"  sharedKey="secret"/>
      </Host>
    </Engine>
  </Service>
</Server>

So, neither adding packetSize="65536" nor removing line breaks and comments from the XML fixed this issue.

Accessing Tomcat directly and circumventing the AJP connector didn’t make a difference.

Any suggestions? Am I missing something?

Sebastian

2

Yesterday, we could finally find culprit and fix the issue!
After trying many more different configuration changes, in Tomcat, Lucee, and Apache, and even setting up a completely separate server, it turned out the issue was related to where the files were uploaded to.

They were saved to a mounted volume, which had the following mount options:

_netdev,rw,sync,hard,intr 0 0

We changed the mount options now to

_netdev,rw,async,noatime,rsize=32768,wsize=32768,tcp,vers=4 0 0

This change reduced the upload time of a 50MB file from 3.2 minutes to less than 2 seconds!

I hope this helps anyone running into similar problems!

Sebastian

2 Likes
3

The leaving out that it was a remote file share would have helped.

:slight_smile: