Trying to send a file to S3 and it ends in corruption

dev
1

I am bringing some more code from my old OpenBlueDragon (and the code does work when I tested it, just in case there was a change on Amazon I missed).

I found this old thread on the forum:

Which it sounds like the issue I am having, but not quite sure. When I 1st set this bit of code up, I remember that the file had to go up in a binary format. And it seems that the implementation of cfhttp I was using already did the proper encoding to send to S3, so all I had to do was send the file:

<cfhttpparam type="file" name="data-raw" file="#tFile#" />

File goes up find, not corrupted, but if I run the same upload thru Lucee, it does not go up correctly. I have tried both mimetypes:

<cfhttpparam type="file" name="data-raw" file="#tFile#" mimetype="text/html" />

and

<cfhttpparam type="file" name="data-raw" file="#tFile#" mimetype="text/plain" />

as well as image/jpeg and image/jpg, to no avail. I even setup the call to use the body instead:

<cfhttpparam type="body" name="data-raw" value="#imageRead(tFile)#" />

As well as FileRead and ven a cffile readbinary, and cannot find any method that works.

So I am hoping someone knows the secret to sending this file in a way S3 can read it. But I am also wondering if this is just an issue because I do not have it in a cfscript format.

2

I’m using cfscript in my example as it’s 2025, but this would be fine with tags

here’s a round trip example of creating an image, uploading as a file, processing the file upload and dumping

<cfscript>

	if (cgi.REQUEST_METHOD eq "post"){
		echo("<h1>Upload image test result</h1>")
		dump(form);
		upload = fileUpload(destination=getTempDirectory(),fileField="file",nameconflict="makeunique");

		dump(upload);
		dest = upload.serverdirectory & "/" & upload.serverfile;
		echo(isImageFile(dest));

		dump(imageRead(dest));
		abort;
	}

	imagefile = getTempFile("","testPng", "png");
	img = ImageNew("",50,50,"rgb", "red");

	dump(img);

	imageWrite(img, imageFile );
	dump(cgi.request_url);
	http method="post" url="#cgi.request_url#" result="result" {
		httpparam name="file" type="file" file="#imageFile#";
	}
	echo("<hr>");
	echo(result.filecontent);

	
</cfscript>

image
image728×992 33.5 KB

3

here’s a working cfml library which does s3 uploads using pure cfml

or you can just use

but I’m curious, how is the file corrupted?

4

I am familiar with that repo. It uses a cfscript version of cfhttp. And it is sending it in the body as FileReadBinary(), which I am doing. As for the corruption, there is something within the file that is added (or removed). When you click the jpg, it downloads instead of shows in the browser, and after downloading, Windows Explorer says the file is not readable. The file sizes match when I try either method (OpenBlueDragon vs Lucee) but one is openable, the other is not.

I will run a test and upload the 2 files here later so they can be seen.

5

I went back and redid some of my tests thinking I might have overlooked something in all the different combos I did. It turns out, that indeed, sending the file via the body using FileReadBinary() worked. I guess I overloaded the browser cache so much it just started downloading the file vs just displaying it (normal behavior was displaying the file).

HOWEVER testing identical coding between OpenBlueDragon (~CF8) vs Lucee 6, the

<cfhttpparam type="file" name="data-raw" file="#tFile#" />

Does something completely different to the files. I have attached 2 files here. mfd.jpg was sent via OpenBlueDragon, mfd1.jpg was sent via Lucee.

As can be seen, the file sizes are slightly different, so I do not know what Lucee is doing with the file during upload.

mfd.zip (1.4 MB)

Sorry, I had to add as a zip, the lucee uploaded file gave an error saying it could not identify the file.

6

I opened up the mfd1.jpg in a text editor and it seems Lucee added this to the beginning of the file:

--c-7eJFxDE19UMsXBDDNC0Zmk_fPJD9f0Jx
Content-Disposition: form-data; name="data-raw"; filename="mfd1.jpg"
Content-Type: image/jpeg
Content-Transfer-Encoding: binary

And this to the end:

--c-7eJFxDE19UMsXBDDNC0Zmk_fPJD9f0Jx--

So my question is, why is Lucee altering a file?

EDIT:
One last note, I converted the cfhttp request to the 2025 version and it still appends the above to the file.

7

that’s a mime multi type header, otherwise, only the file content is uploaded

if you don’t want that metadata, don’t use type="file"

8

I am not sure I am following what you are saying. I am trying to send a file via s3, like I used to vs using the body. If it is not possible with Lucee then it is not, I have it working with the Body now (which I could not do in OpenBlueDragon). And it is just another change to keep in mind when using Lucee.

Here is the exact call I am making:


http method="PUT" url="#r.data.url#" result="result"
				{
						httpparam type="header" name="Authorization" value="AWS4-HMAC-SHA256 Credential=#S3_AWS_ID#/#DateFormat(r.data.date,"yyyymmdd")#/#region#/#service#/#termString#,SignedHeaders=#SigningHeaders#,Signature=#r.data.signature#";
            httpparam type="header" name="host" value="#HostHeader#";
            httpparam type="header" name="x-amz-acl" value="public-read";
            httpparam type="header" name="x-amz-content-sha256" value="UNSIGNED-PAYLOAD";
            httpparam type="header" name="x-amz-date" value="#r.data.amzStamp#";
            httpparam type="header" name="x-amz-expected-bucket-owner" value="#canSt['x-amz-expected-bucket-owner']#";
            httpparam type="file" name="data-raw" file="#tFile#";
				}

adding

multipart="true"

ends with Amazon returning an unmatching signature, and false, puts the extra data in the file.

note type="file" as uploading the contents of the file, nothing about adding header information to the file it uploads.

9

After reading the fine print of PUT and POST, I think I see what my issue is. PUT uses the body tag for file contents and POST uses the file tag for file contents. It appears the my old cfml engine automagically put the file into the body.