Unable to download large files (600 MB)


#1

Hi,

I’m having problems when calling cfcontent with large files on Lucee.
The content somehow gets corrupted and I can’t download file.

<cfheader name=“Content-Length” value="#fileSize#">
<cfheader name=“content-disposition” value=“attachment; filename=myFile.zip”>
<cfcontent type=“application/octet-stream” file="#pathToZipFile#" deletefile=“no” reset=“no”>


#2

Try .
Had similar issues years ago.
Hope that helps.

Regards
Michael


#3
<cfsetting requesttimeout="60000">

#4

Hi,

nope, it’s not requestTimeout issue - I have that set to 20 minutes.

Thanks anyway.


#5

What does corrupted mean in that context? Did you have look at the file in an hex-editor or something like that?

How large is the downloaded file?
Did you try the download from different clients/browser? Is the result always the same?


#6

Corrupted means, that browser offers wrong file name to download - (it offers template name of the running code, because the file does not exist). And when I click save, it says that file doesn’t exist.

The same code runs ok on coldfusion but not on Lucee.
I tried changing max heap size to 2 GB and no changes.
I also tried Ben Nadels solution “Using ColdFusion To Stream Files To The Client Without Loading The Entire File Into Memory” but it also doesn’t work.


#7

According to <cfheader name=“content-disposition” value=“attachment; filename=myFile.zip”> your file-name is always the same: myFile.zip
The current-template-name should not be exposed to the client, based on that source-code.
Where do you store the file-name and file-size? Are you doing a <cfdirectory> or some kind of database-access?

Lucee does not load the file to memory before it is sent to the client.
What kind of infrastructure are you running? Lucee, Tomcat, Apache? Windows, Linux? Are you proxying the requests?

Keep in mind, that a 20 minutes requesttimeout means an average connection speed of 4MBit/s to download a 600MB file.
I don’t know your use-cases, but nevertheless I’d suggest to increase the value.


#8

I just gave you modified example. My fileName is stored in a variable and it’s never the same. The file is always generated, before cfcontent.
Lucee is running on Tomcat with IIS connectors in Windows - so yes, request are proxied from IIS to Tomcat.
And in IIS web.config I have enabled:

requestLimits maxAllowedContentLength=“4294967295”
httpRuntime executionTimeout=“2300” maxRequestLength=“2147483647” requestValidationMode=“2.0”

PS.
After my options have ran out, the last chance is posting the problem here. If I would have found other solution, I wouldn’t post the problem here.


#9

I don’t think you need to set the Content-Length header, did you try removing that?

For this type of task I would consider something like mod_xsendfile - with that you just pass a Response header to the web server containing the file path, and then web server will serve the file back to the client for you, freeing up Lucee to do the work it is good at, and letting the web server do the work it is highly optimized for (serving files).


#10

Just tried with a 800mb file with Apache 2.4 on Windows 10 with Lucee 5.2.5.20-RC and I had no problem

what do the http response headers look like when you get the wrong filename?

you might need to use fiddler or the like to see them as dev tools often hides downloads…


#11

Originaly, I had it without Lenght header, but then even the fileSize of downloaded file was wrong - only 400 instead 620 MB .


#12

Try escaping the filename correctly. That might be a problem as well, especially with spaces.

<cfheader name="Content-Disposition" value="attachment; filename=""#filename#""" > 

As I’m running Lucee without IIS, I can’t help in that scenario. But in general there is no problem with downloading files. I distributed files with more than 7 GB with Lucee.
Does the exception.log have any information?

The content-length header should work.

Why do you have reseet=“no” in the cfcontent-tag? That is not required.


#13

There was a bug with fileSize that I fixed earlier this month. I’m on my phone so can’t look it up ATM, but if you send a content length harder that is too small then the downloaded file is corrupt.


#14

See
https://luceeserver.atlassian.net/plugins/servlet/mobile#issue/LDEV-1533


#15

The fileSize was correct but anyway I updated Lucee to latest version although nothing helps :frowning:


#16

Can you explain this part a bit more in detail?
Why don’t you do some fileexists() before trying a cfcontent?
We are all trying to help you, but you still did not answer all my questions.


#17

Blockquote
Corrupted means, that browser offers wrong file name to download - (it offers template name of the running code, because the file does not exist).

That means your Content-Disposition header is wrong, or not present; not that the file doesn’t exist. I would hope any file you download doesn’t exist on the destination already. And that’s browser behavoir, not Lucee behavior.

If you got a Lucee error screen with “File does not exist” as an error - then that’s on the server side.

Blockquote
And when I click save, it says that file doesn’t exist.

It = browser? It = lucee error page? It = what?

Have you tried using developer tools to look at the network request? Verify all the headers are present and correct?


#18

I appreciate your help.
Yes, I did try fileExists and it does exists before cfcontent.


#19

Browser reports that file doesn’t exist. So on server side file is ok.

I use firefox’s debugger but I find nothing useful to be seen there in this case.
I used Content-Disposition header 1000 times and there is nothing special that is different into it.

<cfset f.pravaVelikost = application.functions.fileSize(f.filePath)>
<!— the file silze is correctly returned —>
<cfheader name=“Content-Length” value="#f.pravaVelikost#">
<cfheader name=“content-disposition” value=“attachment; filename=zadeva_#fileName#.zip”>
<cfcontent type=“application/octet-stream” file="#f.filePath#">

The same code works on ColdFusion + IIS (tested on same machine and also tested on more machines) and it also works on Lucee with smaller zip files so…
I believe it’s a problem with Lucee - IIS communication.


#20

You did not escape the fileName correctly. The way you do it, there are no spaces/blanks allowed in the filename.

I’m not experienced with IIS, as I removed IIS years ago. Depending on your setup, you should be able to access the same webroot on another port on Tomcat directly, bypassing IIS (maybe :8080). Does the error occur there as well?

Do you need IIS for any really good reason?