Basic file upload and permission issues

[derrickpeavy]

I fear I am stepping in a big ol’ pile by asking this… but here goes.

I have used DropZone to create a drag and drop file upload. The files go to /tmp initially. But when they are moved to the appropriate end point (for example: /web/images/) the files are not viewable by the world unless I CHOWN the file with tomcat:www-data.

So, question - does anyone have a basic, bare bones, “proper” file upload CML code which assigns a file the proper permissions and ownership? Are you willing to share, or point me in the direction of such code or other resource?

As you can imagine, it’s easy to get bogged down in tens or hundreds of examples, many of which are out of date. 100 Thanks in advance!

Ubuntu 20.04 LTS
JDK 16.0.1
Tomcat 9.0.41
Lucee 5.3.8.189

[andreas]

Glad you are asking, because I’m about to create a similar thing. Never did an upload on Ubuntu, but that is about to come for an application soon, so I can’t help you with code.

@derrickpeavy

the files are not viewable by the world unless I CHOWN the file with tomcat:www-data

I suppose that’s because of the security by default. Here are my question:

• How are you copying the files to the endpoint? cffile or another tool?
• If you are using cffile, did you also set the attribute “mode” in such a manner that “others” have read permissions?

[derrickpeavy]

Rather than upload what I fear is not very strong code…

Two good links I found since I posted the question this morning:

CFDocs has a very, very basic example of checking mime type on the server:

https://cfdocs.org/fileupload

And Pete Freitag has an updated note from January of this year (2021) which is probably what I will use to rework my file upload to improve security. Then, I’ll see if I am still having the same permissions problem. I also need to look at using an off site storage source like S3 or something. But, one step at a time.

https://www.petefreitag.com/item/701.cfm

[andreas]

petes post is a good one. What I always did on windows

• upload the image files to a temp directory first
• check file extension and mime types (whitelist)
• try to read the image with image magick before any conversion is done, to check these are recognizable images.
• manipulate the image and save it with own settings and naming conventions.
• copy the converted files to a directory that is mapped by the webserver but NOT by lucee/tomcat. The files must not have executable permission (use mode attribute on Linux)
• Additionally you could add some URL rewirte to only allow image readings for that mapped image directory, and not allow any other file extension request there

[derrickpeavy]

@andreas

not have executable permissio

On Linux, I could previously use CFEXECUTE to change group and owner as well as mode. But that seems to no longer work. That’s part of the problem I am having now, which - granted - is a problem that is probably due to not following Pete’s process. But, still a problem cause you know, do you take a full day (in my case) to revise this, or can you find a way to get your current code to work correctly. I know the right answer, but time is an issue.

[andreas]

@derrickpeavy

I could previously use CFEXECUTE to change group and owner as well as mode

I think that cfexecute will only work on files where Tomcat has permission to do so. Since the files are being uploaded by cfml Tomcat will be the owner and file manipulation shouldn’t be a problem to those files in directories Tomcat/Lucee has write access to. I even think you won’t need to change ownership at all. You just need to copy the files to the web directory and leave/save them with read permissions for the “other” groups, so your webserver can read these files. Did you try to use cffile action="copy" mode=...? for copying instead of cfexecuting stuff?

@derrickpeavy

I know the right answer, but time is an issue.

That’s the biggest issue I think. Time is all you need, because this is a crucial and very critical part of an application because of security, and that should be all well tested. I’m also not the quickest, and if something like this would take me a month, then I’d take that time. I’ve seen devs giving root permission to Tomcat as a solution. That seems to be an easy fix for some people, but I wouldn’t sleep well at all with that

[Zackster]

Try FileSetAccessMode() :: Lucee Documentation

[derrickpeavy]

That sounds awesome! Unfortunately…

<cfscript>
	FileSetAccessMode(#filePath# & "logo-small.png", "600" );
</cfscript>

…gives…

"chmod [600] [/path/logo-small.png] failed"

[andreas]

@derrickpeavy

"chmod [600] [/path/logo-small.png] failed"

@derrickpeavy that doesn’t make much sense. If you upload the file “logo-small.png” with cfml to a temp directory that Tomcat has wirte access to, then Tomcat would be the owner and have full permissions on that file in that directory, thus FileSetAccessMode(#filePath# & "logo-small.png", "600" ); should be possible there. I’d say you are doing something else that is changing the permissions of that file (e.g. uploading it through ssh, editing it with another tool that is not invoked by cfml, or doing chown/chmod with a sudo commandline on a ssh session).

Another point: you are setting chmod 600. That means, you are giving read and write access only to the owner, that would be Tomcat only. That means that if your webserver is running with another user than Tomcat is running, it won’t be able to read it and show it to the world. Cause: in a typical setup with Apache2 fronting Tomcat, as soon as you have uploaded the png file Tomcat won’t need to read the file at all anymore. When that file gets requested by a browser apache2 will see that file as a static file and deliver it directly. It will try to read the file directly. But apache2 is very probably running with another user than Tomcat is, so it wonÄt be able to read it with the file permission chmod 600. Tomcat and cfml is only used for upload and file manipulation, but after that .png is treated as a static file like .css, .js, .jpg are.

[derrickpeavy]

I agree, it makes no sense, but I assure you I am not doing any of those other things. I’ll try to show the whole issue here…

Files are being uploaded to:

/web/user-receipts/

… which is owned by “tomcat” and the group is “www-data”

The CFM file/code/page uploads the file into a subdirectory it creates at the time of upload (if it does not already exist) called “user-0000000001” (or another user number) which is ALSO owned by “tomcat” and the group is “www-data”

Yet another subdirectory is also created which is today’s date: “2021-07-05” which again is also created at the point of upload if it does not exist, is also owned by “tomcat” and the group is “www-data”

The image file is then renamed to something else and uploaded into that directory.

Below, are screen shots of the directories and associated owners and permissions as seen through an FTP/SFTP client (again, CFM is doing the work, I am just using the FTP client to show you the results visually). I could just as easily screen shot the Terminal, but this is faster (visually) to look at.

After CFM uploads the file - which it does, no problem - then I call ImageMagick to do some things. But - and this is key - it doesn’t matter if I try to run (either) CFEXECUTE or FileSetAccessMode() to change file attribtues, before ImageMagick or after ImageMagick - the result is the same - an error.

But, wait, it gets even more weird.

I also have another directory called “app-style” this is where default style sheets go. These style sheets are actually generated from a database table so that I can edit, add or adjust styles on the fly without touching a style sheet (long story). But I have a CFM form page with each CSS attribute in a field in the DB. CFM then pulls all of that, parses and creates a style sheet. The resulting file is saved to disk in “app-style” which is… owned by Tomcat and the group is WWW-DATA. Same as with the receipts.

But unlike the “user-receipts” directory, I can run CFEXECUTE on the “app-style” directory and it works everytime, I can change MOD to 600, 644, whatever (it’s always 644).

This is the only line of code for updating the /app-style/ directory contents after generating the style sheet file, it works every time, no error, and I can change mod to anything I want as a way to test:

<cfexecute name="/bin/chmod" arguments="-R 0644 #fullPath#" timeout="30" />

So, bottom line, CFEXECUTE works in one directory, but not another and the permissions are the same on each.

And, in my prior example, changing the mod to 600 was an example. The file is/was 644, but I used 600 just to see if it would work. Directories are always 755 and files are 644.

Screen shots. I highlighted (clicked) on the directory or file which I want to show you in each of the four screen shots.

Permissions and owner for "/user-receipts/
(tomcat:www-data 755)

Permissions and owner for "/user-receipts/user-00000000001/
Created by CFM at time of upload but in this case, permissions have previously been corrected to 755

Permissions and owner for "/user-receipts/user-00000000001/2021-07-06/
Created by CFM at time of upload but with 750 instead of 755

Permissions and owner for "/user-receipts/user-00000000001/2021-07-06/receipt-000000010566.jpg
Created by CFM at time of upload but with 640 instead of 644

[Julian_Halliwell]

I don’t use Linux, but I blogged about file uploads and permissions just the other day. An issue introduced in Lucee 5.3.4 at some point means that permissions are no longer inherited from the specified destination folder, but from the GetTempDirectory() folder in which Lucee initially stores the uploaded file.

This may not be relevant, but to see if it might be, try changing the permissions on your GetTempDirectory() folder. Then see if uploaded files retain those permissions in their destination.

[Terry_Whitney]

First, I would check your permissions for app-style.

you are saying permission 0644, 0644 a user can not execute.

The permission you are requesting breaks down like this:
usr group other
Read Yes Yes Yes
Write Yes No No
Execute No No No

Try your script first with 777 (anything goes for everyone). If it works its a permissions issue

[andreas]

@derrickpeavy

After CFM uploads the file - which it does, no problem - then I call ImageMagick to do some things. But - and this is key - it doesn’t matter if I try to run (either) CFEXECUTE or FileSetAccessMode() to change file attribtues, before ImageMagick or after ImageMagick - the result is the same - an error.

Additionally to what @Terry_Whitney already said…:

What happens if you simply skip the ImageMagick (for testing) and just try an cfimage or simply copy the raw image without any conversion involved to the public folder? Does the problem continue?

[derrickpeavy]

So, I am not a fan of making everything 777. And 644 is the default for files served by Apache, all of my html, php, coldfusion, etc files are always 644. No problems. That said, I’ll tinker with the permissions of the directories in a bit and see what happens. However, I have done that yesterday I believe, but I’ll try it again. As for the other directory where CFEXECUTE does work, the permissions are the same - 755 for directory, 644 for files.

[Terry_Whitney]

This is not a ColdFusion issue.

This is a permissions issue.

Looking at your screen shots you have 2 sets of permissions
user tomcat, group www-data
and user tomcat, group tomcat.

Your setting up something as 644, which says, User can not EXECUTE. nor groups, nor others.
So your permission if you need something to execute is wrong.

You could fix a number of ways such as:
making tomcat run as www-data
making apache24 run as tomcat
adding tomcat and www-data to each others groups and setting thr group permission correctly
making the other permissions correctly
adding the execute permission: chmod -R directory +x

if you want to print working directory permission out, We can tell you what you have it set at if you do a ls -lt /name of that directory > output.txt and paste the text here.

lastly you will need to tell us how you are using imagemagik, as that is running as ? CFC? A cron job? other? Sorry if I missed it.

[Terry_Whitney]

Last thought, did you create tomcat as a special user or did you just create tomcat as a standard user then add them to the www-data group?

I am asking as if you created the user as a standard user, then you can set the umask setting for the user, which could be the cause of all your problems of why new files are suddenly created with the wrong permissions.

[derrickpeavy]

Julian, you are 100% correct! Well, I think you are anyway. I used Charlie’s test found here… Show Temp File on Server to view the temp file inside /WEB-INF/lucee/temp and indeed it does not honor the intended group and owner and permissions. But, maybe that’s supposed to happen in /WEB-INF/lucee/temp ?

So that said, I retraced my steps as well as tried Terry’s idea below. Neither worked. I did solve it, and will post in a moment at the end of the thread.

EDIT - Adding tomcat to the www-data group did NOT solve the problem after all. Only FileSetAccessMode() seems to solve the issue.

[derrickpeavy]

Terry, I knew you were smart! I’ll post in a few the solution I found, but as you have just noted, it’s not the “right” way to do it because… no, I did not add Tomcat to the www-data group. What I did (do) during set up is:

Set up new tomcat group

`groupadd tomcat`

then add tomcat user

`useradd -s /bin/false -g tomcat -d /opt/tomcat tomcat`

So… I need to add Tomcat to www-data and I need to change my install routine. (not that I do very often, but).

I assume I could just do…

adduser tomcat www-data

EDIT - Adding tomcat to the www-data group did NOT solve the problem after all. Only FileSetAccessMode() seems to solve the issue.

[derrickpeavy]

Terry, I don’t think I can dispute your knowledge of permissions, but I thought it worthy to add this to the thread for others.

I use 755 for directories and 644 for files for anything served to the world as that’s the default setting and from what I have found when searching on the topic:

https://tecadmin.net/set-all-directories-to-755-and-all-files-to-644/

And because 777 is making everything wide open to anyone. Some admins disagree and set permissions for files to 640. But either way.

https://serverfault.com/questions/357108/what-permissions-should-my-website-files-folders-have-on-a-linux-webserver

[derrickpeavy]

And so the bottom line is

FileSetAccessMode() :: Lucee Documentation

is the way to solve my problem. Turns out I was providing the wrong arguments the first time I tried it.

Terry pointed out a problem, which is that I created tomcat as it’s own user and as part of the tomcat group when I need to add tomcat to the www-data group. However, even after adding tomcat to the www-data group, it still does not solve the problem (although I have not restarted Tomcat, so maybe it will - I’ll have to wait to try that.)

But for now, a couple of points:

I can set the group and owner of the directory where I ultimately want my file to either tomcat:tomcat OR tomcat:www-data OR www-data:www-data - turns out it doesn’t matter as long as I apply FileSetAccessMode() correctly.

BUT - I have to first run that on the directory I create for the user and the current date. In other words, if you are uploading a file to a directory which already exists and is set to 755 and available to the web, then no problem. But in my case, I need to fix the directory I create for the uploaded file.

So, in my example …

<!--- set receipt directory --->
<cfset receiptDirectory = "#expandPath('.')#/user-receipts/user-#numberFormat(#session.user.ID#,'00000000000')#/#dateFormat(NOW(), 'yyyy-mm-dd')#/">


<!--- check for the directory, this will ensure the receipt directory is created --->
<cfif (! DirectoryExists("#receiptDirectory#"))>
	<cfdirectory action="create" directory="#receiptDirectory#">
	<cfscript>
		FileSetAccessMode("#receiptDirectory#", "755" );
	</cfscript>
</cfif>

Then, after uploading and processing the file (I do this after calling Image Magick):

<cfscript>
	FileSetAccessMode("#receiptDirectory##newFileName#", "644" );
	FileSetAccessMode("#receiptDirectory#tn-#newFileName#", "644" );
</cfscript>