Scheduled tasks flagged as invalid and won't run any longer

I found out today that somehow my scheduled tasks have not been running for the past several days (not sure how long, but a single day is too long). I don’t know what happened, but I got to looking around and found this issue: https://luceeserver.atlassian.net/browse/LDEV-1951 Which looked like solved the problem. I went out and upgraded the box that these tasks were running on, to the latest version. After the upgrade I went to check the tasks, but they were all gone. Great. I had to get all 11 tasks back in. After that I assume everything should be working, but no. The tasks still do not work, and when I log them they simply say “Task is invalid”. What does that mean and how do I get my tasks to run again???

I’m on: Lucee 5.3.2.77.

Are you adding the tasks manually to the Admin? Can you give us a screenshot of your settings?

Are you adding the tasks programmatically (recommended)? Can you give us the code block you use for that?

Here are the logs showing the invalid tasks:

“Severity”,“ThreadID”,“Date”,“Time”,“Application”,“Message”

“ERROR”,“Thread-226162”,“06/11/2019”,“10:00:00”,“schedule task:Send Customer Cellular Plan Confirm Emails”,“Task is not valid”

“ERROR”,“Thread-226140”,“06/11/2019”,“10:00:00”,“schedule task:Standard Plans from SOs Process”,“Task is not valid”

“ERROR”,“Thread-226120”,“06/11/2019”,“10:00:30”,“schedule task:Process Pending Renewal Subscriptions”,“Task is not valid”

“ERROR”,“Thread-226149”,“06/11/2019”,“15:25:00”,“schedule task:Standard Plans from SOs Process”,“Task is not valid”

“ERROR”,“Thread-231481”,“06/11/2019”,“15:30:00”,“schedule task:Standard Plans from SOs Process”,“Task is not valid”

“ERROR”,“Thread-226170”,“06/11/2019”,“15:30:00”,“schedule task:Send Customer Cellular Plan Confirm Emails”,“Task is not valid”

“ERROR”,“Thread-231543”,“06/11/2019”,“15:30:03”,“schedule task:Standard Plans from SOs Process”,“Task is not valid”

“ERROR”,“Thread-226129”,“06/11/2019”,“15:45:30”,“schedule task:Process Pending Renewal Subscriptions”,“Task is not valid”

“ERROR”,“Thread-226236”,“06/11/2019”,“18:30:00”,“schedule task:Data Limit Exceeded Notice”,“Task is not valid”

“ERROR”,“Thread-226240”,“06/11/2019”,“18:30:00”,“schedule task:Data Limit Exceeded Notice”,“Task is not valid”

“ERROR”,“Thread-226251”,“06/11/2019”,“18:40:00”,“schedule task:Data Usage Rate Warning”,“Task is not valid”

“ERROR”,“Thread-226255”,“06/11/2019”,“18:40:00”,“schedule task:Data Usage Rate Warning”,“Task is not valid”

“ERROR”,“Thread-226271”,“06/11/2019”,“19:30:00”,“schedule task:Cellular Subscription - End of Service Notice”,“Task is not valid”

“ERROR”,“Thread-226277”,“06/11/2019”,“19:30:00”,“schedule task:Cellular Subscription - End of Service Notice”,“Task is not valid”

“ERROR”,“Thread-226199”,“06/11/2019”,“21:00:00”,“schedule task:Cell Renewal Notice (60 day)”,“Task is not valid”

“ERROR”,“Thread-226194”,“06/11/2019”,“21:00:00”,“schedule task:Cell Renewal Notice (60 day)”,“Task is not valid”

“ERROR”,“Thread-226215”,“06/11/2019”,“21:10:00”,“schedule task:Cell Renewal Notice (30 day)”,“Task is not valid”

“ERROR”,“Thread-226221”,“06/11/2019”,“21:10:00”,“schedule task:Cell Renewal Notice (30 day)”,“Task is not valid”

“ERROR”,“Thread-226409”,“06/11/2019”,“22:30:00”,“schedule task:Send Monthly Overage Charge Notifications”,“Task is not valid”

Here are the execution settings on my server. I’ve added these manually. The application is on something of a temporary server for the next few weeks. It will move and then the scheduled tasks will be handled in our standard production way (through code). I haven’t had problems with them running up until recently. I have no idea what has changed to cause them all to become invalid.

Did you try bumping the error level for the scheduler.log file under the web administrator?

No. How do I do this?

Look under Settings - > Logging (ps, let us know what was it currently set to?)

currently set to “Error”

ok, as per that task you linked to at the top, if you set it to INFO, you should see more details

Got it. So, I’m seeing warnings now showing a 403 error: “returned response code 403”

I don’t know why it would return a 403. The tasks are hitting pages on the same server. Those pages are accessible without any authentication or other setup that would send a 403.

good, we are getting somewhere!

what happens if you call one of the problem urls via cfhttp?

200 status code - called via cfhttp from same server.

I just tried restarting Lucee. The scheduler logs have one of these for each of the tasks:

“WARN”,“default task-6”,“06/12/2019”,“09:19:03”,“scheduler”,"task [Standard Plans from SOs Process] could not be stopped: at java.lang.Object.wait(Native Method)

at lucee.commons.io.SystemUtil.wait(SystemUtil.java:680)

at lucee.runtime.schedule.ScheduledTaskThread.sleepEL(ScheduledTaskThread.java:208)

at lucee.runtime.schedule.ScheduledTaskThread._run(ScheduledTaskThread.java:161)

at lucee.runtime.schedule.ScheduledTaskThread.run(ScheduledTaskThread.java:119)

"
The restart doesn’t seem to have helped, however. Still getting a 403 response.

the shutdown errors are a known issue https://luceeserver.atlassian.net/browse/LDEV-2222 but they can be ignored for now, it’s just annoying log spam.

from memory, cfschedule uses a different UA i think than cfhttp does. a 403 is most likely
coming from the webserver rather than lucee/tomcat

I’m beginning to think this as well. I just found out that our server manager recently put NGINX with a web application firewall in front of our servers. This is probably the cause of the 403 errors. I had no idea that had happened, or I would have started there. Thank you much for your time and knowledge, you have been very helpful.

1 Like