File Upload Slow (extreme)

dev support
1

Hello,

New to Lucee coming over from ACF 2016.

File Uploads to Lucee/Tomcat are taking a Ridiculous amount of time.
I’ve tested using 2 different machines. 1 is my local machine and the 2nd is a remote server up on the web both running basically the same setup.

Machine 1 : Local Desktop : Win10 AMD

  • Win10/IIS : Adobe CF 2016
  • Virtual Machine WinServer 2016: Lucee : lucee-5.3.8.189

Machine 2 : Bare Metal Rack Server : Win 2016 Server Intel

  • WinServer 2016/IIS : Lucee/Tomcat : lucee-5.3.8.189

I have traced/isolated the problem to Lucee/Tomcat. Using a very basic form file upload to : Function : FileUpload

  • Uploading a 10-Megabyte file to ACF-2016 takes less than 1 second
  • The same 10-Megabyte file uploaded to Lucee/Tomcat takes about 30-40 seconds.

Larger files up to 80MB

  • ACF-2016 : about 3 seconds (basically the Transfer/Write Time)
  • Lucee/Tomcat : 2+ Minutes
  • I’m sure it’s Not a network issue
  • Disk Read/Write is also Not an issue (Tested copying a 110MB file on the VM)

I have been up and down google and the lucee support forums for 2 days straight (searching for at least 12 hours total before posting this - I Did Try). The only thing that seems remotely close to being the bottle neck is possible virus scanning of files uploaded to Lucee/Tomcat. However, when uploading an 80MB file there is no change in CPU Load (it’s flatline like 2% idle) and I would think that virus scanning would hit the CPU and be noticed.

So it feels like a bottle neck in the Lucee to Tomcat Connection. Or something with Tomcat. I searched google extensively on Tomcat upload issues Not related to Lucee and that is where people talked about virus scanning file uploads. But as mentioned I find that unlikely due the 0-Zero change in CPU Load when uploading a file to Lucee/Tomcat.

I started out using lucee-5.3.7.47 and moved up to lucee-5.3.8.193

  • No Change

Any help would be great… Very Frustrating
Many Thanks!

OS: Windows 2016
Java Version: ??
Tomcat Version: 9
Lucee Version: Tested from lucee-5.3.7.47 up to lucee-5.3.8.193

2

Here are a few things to consider:

  • you’ve not said so specifically, but for the cf page that is “receiving the upload”, are you doing a cf file upload (or the script equiv)? What if you skip that, and just push the upload to the template anyway? Cf or lucee will still “receive” the file, as a temp file in the temp folder. (Yes, that does mean an upload can be sent to ANY cfml page, even an empty one). Is it any faster then? Just curious.
  • you’ve not said how you are doing the upload. Pure html, input type=file? Or some js-based tool?
  • you may want to use a tool like fusionreactor (free trial) to see where in any request time is being spent, presuming the time IS in the cf request
  • you mention using iis with cf but not lucee. So your request is going through iis with cf, but not lucee?
  • if you ARE going through iis with lucee, using what? Boncode? Mod_cfml? What if you instead go straight to the lucee /Tomcat web server (same one serving the admin)?

Others may have different questions or things to consider, of course. But answers to these may rule some thing in or out.

3

does it also happen directly without IIS involved, i.e on port 8888?

4

Hey Zac… I was trying to find a way to message you directly as I have found numerous answers from you in the forums. I’m new to these forums is that even possible here?

To answer your question I have not attempted to cut IIS out of the process. I’m new to Lucee but old to ACF

Like most people here I have around 20 years exp with CF. I’ve run ACF enterprise using the WSCONFIG to connect websites in IIS. At this moment I am exploring doing a Vanilla installation of the JRE, Tomcat and boncode connector (opposed to the mostly prepackaged setup in the Windows Lucee installer).

I feel like the boncode connector will be similar to the wsconfig untility… but this is quickly turning into a Non-Straight forward way of dealing with things.

I’m pressed for time and if your available for consultation that would be great. Can I direct message you somehow?

5

Please only contact me like this, that way other’s can benefit. I do paid consulting, but you’ll need to talk to @Gert

However, I’m working on other projects at the moment

@andreas is quite good with IIS and boncode problems

6

Got it… thanks! I’ll try connecting with them. Ultimately I need to learn all this stuff. But at the moment I need to weigh out how practical it is to switch from ACF. Basically every DEV I’ve talked to recently has in some way indicated performance issues. I’m hoping that’s not true : But that’s just what I want to believe.

Thanks for the fast response…

7

i.e?

8

i.e : out of about 40 devs : around 10 had a hand in Lucee… and of those 10 … 5-7 mentioned performance issues. However this was recently and those opinions may be out of date.

I decided to get my own opinion and just Do it. Aside from this File Upload issue the only other performance issue I have seen is the 1 second delay in Tomcat due to iPV-4/6 situation in which the address needs to be changed from “localhost/127.0.0.1” - to - “::1”

This was an answer you provided… but still it took a good amount of digging to turn up and ultimately it was 1 link leading to another in a forum post that will age out. Through various different sources I was able to find the location of that setting.

Ultimately there was no consolidated answer. I just grinded through and I would prefer to avoid that this time…

9

Thanks for asking. Unfortunately I’ve no time at the moment as I’d like to. Had to deal with lots of privat stuff lately because of the flood disaster.

10

Sorry to hear that. I’m new here and unaware of peoples locations. Good Luck… Stay Safe.!

11

First Jay, a warm welcome to our community. Thanks for the wishes. We are all fine, safe and sound. Thanks

12

[UPDATE] : I’ve traced the issue to the AJP/Boncode Connector (as suspected)
Deployment:

  • JRE : Direct install (not from package)
  • Tomcat9 : Direct install (not from package)

Tomcat installation set to : C:\Tomcat9
Subsequent ROOT : C:\Tomcat9\webapps\ROOT

  • When the tomcat ROOT was deployed I moved it’s contents to another folder ( ROOT_HOLD )
  • I then created a new empty ROOT folder
  • In the new empty ROOT Folder (C:\Tomcat9\webapps\ROOT)
    ---- I Paced the : lucee-5.3.8.189.war (which auto extracted)

I then placed my test upload code into the ROOT directory

  • File : HTML : Basic Form Submission: ManualUpload.html … Which submits to : FileUpload.cfm
  • The upload execution was 1.6 seconds on an 80MB
  • This is the expected and same performance as ACF

After that I attempted the addition of the Boncode AJP Connector.

  • The result was much better than the prepackaged setup from the Lucee windows installer
  • The result was 2-3 seconds on a 10MB File VS the previous 30-40 Seconds on the same file

So, Taking the LONG Way around paid off…

However, the scenario is not complete. While I may be willing to bear the now Minor slowdown on uploads due to the AJP connector (or my lack of experience in using it) - it is still a bottleneck.
But regardless of my lack of experience using this connector: I feel like it is a problem that should not exist this many years into it. From the looks of it the AJP Bond Connector is basically and abandoned component.

Here’s a quick link to the AJP Connector Repository (the others are easy to find)

13

You mean the Boncode connector? It hasn’t been abandoned. @Bilal keeps it up to date and frequently helps people out in this forum. He also maintains extensive documentation.

http://www.boncode.net/connector/webdocs/Tomcat_Connector.htm

I’ve been doing file uploads with IIS/Boncode/Tomcat/Lucee for several years since switching from ACF and haven’t experienced any significant performance issues, although we deal with sub-10MB files for the most part.

Bilal may be able to offer advice if large file requests do prove a problem for you.

14

It is only Incidentally that I ran into this problem or I probably wouldn’t have noticed either. But there is certainly an issue with that connector. I’m now attempting to make the Jump to Tomcat and get off IIS as my ultimate goal is to switch to LINUX and leave windows behind…

On the other hand Windows does make many things easy and people don’t have to live and breath code to deal with it.

15

so we still need to figure the correct approach out for the lucee installer

We have since moved the installer to a Lucee repo

the 5.3.8 currently sets no config

16

@Jay.MC
Sorry to hear about your journey. Most devs seem to have an easier time transitioning to Lucee. There are differences for sure, but they can be overcome.

A few questions/suggestions:

One

The machine on which you installed Lucee first was it the machine on which you were previously running ACF?
Did you remove the ACF connectors from IIS when you switched over? Uninstall ACF?
The ACF connectors will not play well with other handlers and can cause large timeouts, especially if the main ACF service is not running. During uninstallation the wsconfig handler may not be removed either and have to be manually removed sometimes.

Two

The next area of interest to consider is the exact setting you use in IIS and Tomcat/Lucee.
For large files you should increase the packet size to its max of 65K on both BonCode and Tomcat side.
see here for details:
http://www.boncode.net/connector/webdocs/Tomcat_Connector.htm#_Toc38268637

Three

IIS does have its own transfer limits, so I assume you used the same machine and that would not be a factor here. But did you ensure that File transfer limits were adjusted?

Four

If the connector is a bottleneck you can run a test with connectors alone. You can use BonCode instead of ACF connectors on the same machine. BonCode is flexible and will work with ACF installs even with the proprietary changes that Adobe uses to AJP protocol.
Set the connector for performance (see above link) and try directly with ACF server as backend. Please make sure you used the right Adobe ports.
What are the changes you see?
http://www.boncode.net/boncode-connector/using-boncode-with-adobe-coldfusion

2 Likes
17

This is a Windows network problem. there are volumes of internet pages on how “localhost” on a Windows Server running IIS is not browsable. The issue has been around as long as IIS has been available.

Anyrate, the workaround is as follows.

C:\windows\system32\drives\etc\ and open hosts with notepad++ as administrator or edit via the command line as administrator.
You will see a line like below
#127.0.0.1 localhost

You need to add
127.0.0.1 localhost
::1 localhost

save the file.

18

Terry_Whitney

the workaround is as follows.
C:\windows\system32\drives\etc\ and open hosts with notepad++ as administrator or edit via the command line as administrator.
You will see a line like below
#127.0.0.1 localhost

You need to add
127.0.0.1 localhost
::1 localhost

Hey I’m new around here so I shouldn’t start by shooting my mouth off…
But For Real… How could you possibly have come to that conclusion…
I mean… did you even read the thread?

19

Bilal

Sorry to hear about your journey. Most devs seem to have an easier time transitioning to Lucee. There are differences for sure, but they can be overcome.

Hey Bilal… your questions seem to have pointed me in the right direction. And Yes my transition has been pretty easy for the most part ( people only post when things are going wrong).
To answer your question on machine: I’m running a VM with Win2016 Server for Lucee so they are to isolated setups plus the cloud server is also running A VM while I tinker around with this.

One thing I have notices is that uninstalling lucee… is Not complete. When I reloaded it from the WAR (without using the Lucee installer) there were fragments left over from the previous install (even after deleting the Lucee Directory)

So what you say about things not being fully removed is true for both ACF and Lucee. Your item Four is probably my best shot ( Tweaking the BonCode connector ). But I think I’m going to wipe the whole VM because of now the previous install is interfearing (can’t get the lucee admin online : getting the same message as the previous lucee install)

Perhaps things are in the registry…? Either way… fragments of the previous install are lurking around causing issues.

If anyone knows how to Totally Kill All Lucee Elements that would be great. and Yes : When uninstalling I removed all settings but that does not seem to have worked. I of course restarted a number of times.

20

As far as I’m aware, there aren’t really big registry setting made by Lucee. The only one I’m aware of is the Windows service registration for the Lucee/Tomcat service, which I’m sure is deleted when uninstalling Lucee. There maybe some additional registry settings involved by the installer software itself, but neither the lucee/tomcat windows service nor the installer should have this type of issues you’ve mentioned.

What I suspect is, at least what sounds to me as the most likely to what you’ve described is, that somehow your web-contexts directories named “web-inf” are not being updated. Thus they will hold outdated data and deliver old data (e.g. old lucee web admin fragments).

Here is what server-context and web-context is about (in a simplistic nutshell).
When you install Lucee (with installer or whatever) Lucee will create by default:

  1. A server-context, which is created when you first run Lucee. You’ll find it as a directory named “lucee-server” at C:\path-to-lucee-installation\tomcat\lucee-server This directory holds the global Lucee server engine.

  2. A web-context named “web-inf” in every single website/webroot. This will hold something like an inherited copy of parts of the global server-context (as mentioned in 1). It will also hold a compiled version of your web application with all your components and cfm templates and everything else Lucee/Tomcat needs to run your web application.

Now, there are situations ( e.g. when reinstalling or upgrading Lucee) that these contextes won’t get 100% correctly updated. That doesn’t happen very often and isn’t very usual, but it may happen from time to time.

So, if you want to wipe out Lucee, make also sure to wipe out these ‘web-inf’ directories as well. Otherwise you might see outdated data coming from that web-inf directory.

Because you are new to Lucee, this all might sound unfamiliar to you, but I’m pretty sure you’ll find your way through this quite soon, and as soon as you are getting more involved with Lucee, you’ll find everything quite straight forward. I’m telling you this because I’ve been there also a while back.

Another important information: Because server-context and web-context also holds settings of your lucee administrators and other important settings as scheduler settings, it’s always good to back them up before deleting.

To backup Lucee administrator settings, I warmly recommend using CommandBox “cfconfig”. This is an awesome tool to export/import Lucee administration settings to a json file. With cfconfig you can even export admin settings from ACF and import them to Lucee.

I hope your getting your issues resolved pretty soon. Please keep us informed about your progress!