Permission issues with Lucee REMOTE Methods

dev support
1

Hi Everyone,
I am not sure if something has changed recently (within lucee’s code / tomcat configs) but here are the symptoms…

I have a 403 forbidden response when attempting to call a method that is marked as REMOTE.
The same code works on our production server
which is

  • Lucee 5.3.10.97
  • Tomcat 8.5.39.0
  • Java 11.0.19.7
  • nginx 1.19

On our Test server, we have:

  • Lucee 5.4.3.16
  • Tomcat 9.0.82.0
  • Java 11.0.20.8
  • nginx 1.24.0

Our install process is;

  • install tomcat
    • Create a tomcat user and group
    • create a www-data group
    • add tomcat user to www-data group
  • install mod_cfml_valve (1.1.11)
  • Install nginx
    • add nginx user to www-data group
    • add nginx to tomcat group
    • add tomcat to nginx group
  • update / match the secret between tomcat / nginx for mod_cfml
  • update the ownership of the “/app” diredctory to be tomcat:www-data

both tomcat and nginx users are members of the www-data group

We have code that calls an api endpoint via cfhttp.
We have api keys and secrets defined in the headers.
(
The values are inserted dynamically via ENV vars.
I have verified that the values being passed in - match the values they are being compared against.
I have also changed these to be hardoded values (as a test) - but that does not help
)

local.h = new http(
	method = "post",
	url = this.apiEndpoint & "my.cfc?method=myMethod&returnformat=json",
	charset = "utf-8",
	timeout = 30
);

local.h.addParam(
	type = "header",
	name = "apiKey",
	value = this.apiKey
);

local.h.addParam(
	type='header',
        name='Content-Type',
	value='application/json'
);

local.h.addParam(
	type = "body",
	value = serializeJSON(qry)
);

local.r = local.h.send().getPrefix();

Here is the method signature of myMethod()
remote string function myMethod() {...}

However the result of calling:
local.r = local.h.send().getPrefix();

is 403 Forbidden.

I can call the same URL / using the same headers and body from CURL - and the request finishes successfully…

At first I thought I had a file / directory permissions issue…
And so I added the nginx and tomcat users to each others groups and the www-data group.
Restarted both services
This didn’t change anything.

I then found this forum post
And thought - “YES” that is exactly my issue…
so I added theat header in, too;

local.h.addParam(
	type='header',
        name='Accept',
	value='*/*'
);

But that didn’t make any difference either.

(I of course restarted the Tomcat service ebtween attempts - and made sure I hit another URL, successfully - to ensure that the contexts were created)

Then just to rule it out - I did
chmod -R 777 /app
(restarted the server /hit a different url…)
And that didn’t help either.

I have exhausted all that I can think of and consumed all my google-fo for the the reading all the things that I could find that seemed to resemble something like what I was suffering from, too.

If anyone else has something to share - I would be most grateful!

2

I’d go step by step, tracing/isolating the issue down: first I’d try to figure out if the 403 http status code is being thown by your webserver, tomcat, Lucee or your applications code.

To do that, I’d try to create a request though port 8888. If that works, you have an webserver rule that is throwing the 403? Then you can debug from there. May be the method is being blocked? The .cfc file, the directory or the components filename itself?

If the request gets blocked with a 403 also on port 8888, check also the Lucee Admin. I think there was an admin setting somewhere that blocks remote cfc requests by default. You’d need to allow it on your webcontext.

If the issue persists, try another random remote method name with a very simple cfc file. Does it get blocked also?

I’d also try to see if there are any catalina or web server logs showing the403 status. Where is the error status emitted first?

4

I still have no idea what is causing the issue - but we have side-stepped it.

TL;DR;
We are using CFHTTP to call a TAFFY endpoint - from within a method that is ALSO a CFHTTP request.

This “inner” CFHTTP call is to a separate (CFML) application on the same server - that we manage via NGINX rewrite rules.
It is this “inner” CFHTTP call that is causing a 403/FORBIDDEN result.

It was fixed by swapping out the inner CFHTTP call to

  • Create a (Lucee server config) mapping to the folder that contains APP2
  • Call needed method via myMapping.myCFC.MYMETHOD()

ERROR DETAILS / TROUBLESHOOTING STEPS:
Our application is actually multiple applications.
That is we have multiple app-roots on the same server, with their own Application.cfc files.
(I don’t know if this point changes anything at all)
But the three applications have their own subdomains and all subdomains use the same wildcard SSL certificate)

/myHOME/App1/
https://myDomain.com

/myHOME/App2/
https://app2.myDomain.com

/myHOME/App3/
https://app3.myDomain.com

We use NGINX for rewriting the URLs to match up the differnet subdomains with the app-root.

An external system calls App1/Taffy-API-ENDPOINT
That Method then calls myMethod (It will make sense below)

In the code I shared in the original post, is the initial entryPoint for this process.

local.h = new http(
	method = "post",
	url = this.apiEndpoint & "my.cfc?method=myMethod&returnformat=json",
	charset = "utf-8",
	timeout = 30
);

The content of the body is JSON and contains an element STEP

So we’re in myCFC - executing myMethod with a body content that contains step=1

function myMethod(){}

//do some stuff
if step ==1 {
    try {
        // Each of the below lines are separate methods in their respective CFCs
        // Each of the steps has its own try/catch, transaction, etc.
        // An exception is thrown if any of the steps fail

        // Add new customer to DB
        // Create a new customer - specific database 
        // Create customer specific directories (images / customer uploads / etc)
        // Create 15 ORM (persistent) CFCs for new customer
        // Using adminAPI - create a datasource for new customer DB

        // ormReload() - to make new CFCs available
    } catch {
        // perform cleanup tasks (undo all steps)
    }
} else if step == 2 {
}

Then we use exactly the same CFHTTP call that we used in step 1 - but we alter the body content to have step=2

So we’re calling the same (initial) TAFFY API endpoint
The same function, myMethod
But in a new / extra request - that exercises the “STEP ==2” portion of myMethod()

} else if step == 2 {
    try {
        // populate some "main" tables with data 
        // populate new tables in Customers database
        // assign licences / roles / profiles / etc....

        // vvvv THIS IS WHERE I GET THE ERROR vvvvv
        // Call another Taffy-controlled API, that is part of APP2 - using CFHTTP (https)
    }
}

So using cfhttp within a cfhhtp request - seems to be the issue - and wer’re getting a 403/FOBIDDEN.

The code is unchanged in these methods - between our current PROD database and the TEST server.
The differences are: (upgraded nginx / lucee / tomcat).
And despite the different versions - the NGINX/tomcat and Lucee configs are unchanged.
(Ignoring the fact that the datasources use different hosts)

  • I did find some discussion about changes made to Tomcat for a CVE fix - where the OP complained that their code worked in the old version of tomcat - but the new one failed with a 403. But that was down to differences in “Origin” header processing and the fix was to strip out the “Origin” header. ::: However using curl -v (verbose) - there is no Origin header to strip out.
    The nginx and tomcat logs are not helpful, either.

  • I also found posts about making sure that the SHARED-SECRET is correct in both NGINX / TOMCAT.
    (but they are)

  • And I jumped into every rabbit-holed forum post I could find that seemed like it could be useful - without success

So… I think you know about as much as I do about the sequence / differences.
And although we’re getting around the issue… I would still like to know why it doesn’t work after upgrading
(When / if I get some time - I will build a new test with the same versions as our Prod and upgrade one of nginx / tomcat / lucee - at a time and do some more diagnosis.)

5

Really can’t say anything, bur how you are describing your issue, it looks like you are trying to focusing on your code only. 403 forbiddens can have a lot of causes, from permissions, configuration of webserver, servlet engine, connectors and code. You need to isolate the issue by checking every single instance of those, from a full isolated point of view. And you telling stuff about origin headers and having rewrite rules makes me wonder, that you are also having some sort of added security checks somewhere…

Since you are saying step == 2 is the code that cfhttp requests are getting the 403 error…

  • Have you tried requesting a simple cfml page from within that code? Or even requesting a static file from outside of the internet that just returns soemthing static? Does that work?

  • Have you within step 2 dumped the output of your cfhttp returned data, for example to a file? Since your cfhttp is acting as an “inner” browser, you need to look into those response dumps. Anything there?

  • After that try creating a very simple, lean,minimalistic taffy endpoint with a diffrent URL just to try it out. Access that endpoint directly on port 8888 directly without using the webserver and ajp connector. Dump the output to a file. Does that work?

  • if the above worked, try using that lean minimalistic .cfc on the endpoint URL that caused the 403 (but still on port 8888). Does it work?

  • Then try it through the webserver switching ports.

Please let us know about your finding.