Under what circumstance, a cfhttp call would fail with the current version of Lucee with timeout set to 5 seconds when manual http load of the target URL works perfectly fine?
I ask this question because last night and earlier this morning one of my apps using cfhttp failed while the manual http loading of the target URL worked perfectly. It happened both for the app on my local computer and on a remote server. Another person experienced the same issue for the app on the remote server.
Additional info: this app has been running reliably with cfhttp call for at least 5 months already without a glitch.
How did you do the manual load? Browser vs cfhttp? Postman vs cfhttp? Are these just timeouts, or do they have an http error code? Are the endpoints always the same, same request headers and http body or do they vary? Are the fails constantly happening or only sometimes?
For “manual load”, I meant using a web browser to manual access the target URL, just like a regular web user would do.
"Postman vs cfhttp? "
cfhttp
"Are these just timeouts, or do they have an http error code? "
timeout. I didn’t explore further to track http return code.
“Are the fails constantly happening or only sometimes?”
Described already.
“Additional info: this app has been running reliably with cfhttp call for at least 5 months already without a glitch.”
Additional info: started late this morning, the problem went away by itself. What puzzled me is this behavior defies logic.
Might have nothing to do with cfhttp and could be your upstream provider.
Suggest you log all errors including timeouts.
Try using cfloop to call again if a timeout occurs, delay the loop using cfsleep, be sure to count
the loop iterations so you can abort or return after so many tries.
In my experience the call usually works on the second attempt.
Interesting thoughts. Regarding “upstream provider.”, do you mean the remote server provider?
If so, why would it also happened to my local app running off 127.0.0.1 base URL?
“Try using cfloop to call again if a timeout occurs, delay the loop using cfsleep”,
I like the idea prior to finding its root case, many thanks.
Could also be a type of bot detection with a sort of ban (or request limit rates). Browsers usually request different than cfhttp, curl or similar http requests. I’ve seen http servers setting header cookies just before redirecting to another page that would just look for that cookie and send a 403 it ir wouldn’t be present in the request headers.
I would have waited the request to timeout after more than 5sec, wait for a http exception to check what could be causing it.
@andreas good thought. There’s another possibility. It’s financial data and prior to my data pull, the incident, the specific market sort of crashed. Remember, the NYSE crashed a little while ago and understandably they stopped its operation for a short period of time. So, the data provider in this case could also have taken a similar approach as well, just my speculation for now.
Lucee and the browser also have different SSL certificates. So, a request to a URL could work in the browser, but could then be denied via CFHttp due a missing SSL certificate. That said, if it starts working again (via CFHttp), then this is likely not the issue.
On Sunday Sept 30, 2020 CenturyLink had an incident that took down Cloudflare, Reddit, Hulu, AWS, Blizzard, Steam, Xbox Live, Discord, and dozens more. We have a remote download server on a Level 3 network in St Louis than became unavailable for several hours. My little data center is on AT&T Enterprise network and we couldn’t connect and overall traffic to the sites we host was substantially lower. It was so widespread that it led to a 3.5% drop in global web traffic. Their issues started to become apparent on the previous Friday when our upload speeds to that ST Louis server dropped by half. Although CentryLink says their issue is resolved we are still seeing constant changes to our trace route to the St Louis server with different peering points showing up. We had a tick open for a different issue and the AT&T level 2 tech have told me that on the 28th NTT Global Data shifted it’s routing so that (from the AT&T trouble ticket) “MULTIPLE AT&T CUSTOMERS USING AMAZON, AKAMAI, MICROSOFT AZURE ETC.,. WEB SERVICES WHOSE TRAFFIC WAS GOING THROUGH CONGESTED NTT PEERING LINKS IN NEW YORK AND WASHINGTON, DC WERE IMPACTED”.
So what I am saying is that in the last week or so the Internet has been FUBAR and your issue could be related to any number of routing and traffic problems that have occurred.
That most likely EXPLAINED the problem! Thank you @dennis for such informative post and the closing sarcasm is beautiful though painful.
And by the way, all the tech reporters, tech news staff were total idiots, I read major news media day in and day out, NONE of them covered it, what’s their job?
I went looking for an answer because I was directly impacted in that I couldn’t get there from here , as they say in the northeast. I just happen to be troubleshooting, with AT&T, an issue with a new circuit we had installed and was hyper vigilant about all things “network” related or I might not have been as aware or informed.
I am glad to see that this was solved.
I found this to happen many times based on network level access restrictions. While my logged in user was permitted to make calls outside to internet (i.e. the browser calls would all work), the lucee process running as system account did not have any of those rights and was not permitted to access the internet through the firewalls. All CFHTTP to internet sources would fail.
Ahe, so, @Bilal, it could also have been caused the Lucee server configuration on my hosting company’s network. Good to know. But they have been hosting Lucee service for quite long time already and I would expect some of their customers would have some applications that leverages CFHTTP call. Wouldn’t this hosting company have a standard process when setting each Lucee server?
Edited / Additional Thought:
I’m thinking of “network level access restrictions”, I’m still having some difficulty in fully understanding it because Lucee setup/configuration for my local installation and Lucee on my hosting company’s server seemed to behave the same way ( failed for that particular day ) but not before and after. Does the “… restrictions” refer to the Internet network infrastructure which of course, none of us (its users) have any control.
Correct.
A classic sign is that all CFHTTP outside of your network fails from server.
If a specific CFHTTP fails but others work than this is not the cause.
, as they say in the northeast. I just happen to be troubleshooting, with AT&T, an issue with a new circuit we had installed and was hyper vigilant about all things “network” related or I might not have been as aware or informed.