Yesterday, we could finally find culprit and fix the issue!
After trying many more different configuration changes, in Tomcat, Lucee, and Apache, and even setting up a completely separate server, it turned out the issue was related to where the files were uploaded to.
They were saved to a mounted volume, which had the following mount options:
_netdev,rw,sync,hard,intr 0 0
We changed the mount options now to
_netdev,rw,async,noatime,rsize=32768,wsize=32768,tcp,vers=4 0 0
This change reduced the upload time of a 50MB file from 3.2 minutes to less than 2 seconds!
I hope this helps anyone running into similar problems!
Sebastian