Scheduleds tasks fired multiple time

dev
1

Hello,

We have scheduled tasks that are performed at night.
Last night, these tasks were performed FOUR TIMES. Each at their programmed time, but four times, at the same second!
There were no particular issues last night (like a server reboot).
The “Run Exclusive” option is checked for all.

This kind of problem has already happened.
If this mechanism is not reliable, maybe it is better to use Apache CRON?

Version: Lucee 5.3.10.120
Java: 11.0.7 (AdoptOpenJDK) 64bit
OS: Linux (4.15.0-202-generic) 64bit

2

How did you detect that it has run 4 times? Through the scheduler log, the webserver logfiles, or a programatic log within the executed cfm template? I’d cross check those first and also look if there are other web contexts firing a schedule.

3

We have a DB for each project, and a log table in each DB.
A row is inserted by cfm template at the beginning of the task.

4

About web contexts, only three for whole server.
And only one with scheduled tasks (ours sites/applications, one hundred, use Apache directive to group “AJP calls” in the same root directory, and then, share a lot of components)

5

Did you also check the access logs (http request logs) to verify if/how the templates are being requested by the scheduler?

6

Which log file are you talking about?

Nothing in access.log. But i found in error.log this line at the starting time for one task:

[Sat Mar 04 03:24:08.015251 2023] [proxy_ajp:error] [pid 24657] [client 152.89.196.211:48006] AH00877: read zero bytes, expecting 314 bytes

I also found in other_vhost_access.log the four calls for one task:

xxxxx.fr:443 135.125.2.204 - - [04/Mar/2023:03:24:00 +0100] “GET /scripts/importFromZoho.cfm?RequestTimeout=600 HTTP/1.1” 200 5829 “-” “CFSCHEDULE”

xxxxx.fr:443 135.125.2.204 - - [04/Mar/2023:03:24:00 +0100] “GET /scripts/importFromZoho.cfm?RequestTimeout=600 HTTP/1.1” 200 5829 “-” “CFSCHEDULE”

xxxxx.fr:443 135.125.2.204 - - [04/Mar/2023:03:24:00 +0100] “GET /scripts/importFromZoho.cfm?RequestTimeout=600 HTTP/1.1” 200 5829 “-” “CFSCHEDULE”

xxxxx.fr:443 135.125.2.204 - - [04/Mar/2023:03:24:00 +0100] “GET /scripts/importFromZoho.cfm?RequestTimeout=600 HTTP/1.1” 200 5829 “-” “CFSCHEDULE”

This is the second day the Lucee server “stutters”. I can try the heavy solution, with a restart!

7

Yesterday, the Lucee server was restarted.
But the problem remains!

At least, i wrapped all my scripts with a cflock “exclusive” to prevent the code to be run more than one time (a lot of DB operations).

8

Hi, we have the same problem after an update to 5.4.3.2. We went to 5.3.12.1 but the problem with multiple fired scheduler remained.

It is visible in the logs

2023-11-23 00:00:15 xxxx GET /doscheduler/today RequestTimeout=50 443 - xxxx CFSCHEDULE - 200 0 0 458 198 13800 

2023-11-23 00:00:15 xxxx GET /doscheduler/today RequestTimeout=50 443 - xxxx CFSCHEDULE - 200 0 0 458 198 13777 

2023-11-23 00:00:17 xxxx GET /doscheduler/today RequestTimeout=50 443 - xxxx CFSCHEDULE - 200 0 0 458 198 14615 

2023-11-23 00:00:17 xxx GET /doscheduler/today RequestTimeout=50 443 - xxxx CFSCHEDULE - 200 0 0 458 198 14862 

2023-11-23 00:00:18 xxx GET /doscheduler/today RequestTimeout=50 443 - xxxx CFSCHEDULE - 200 0 0 458 198 15819

Did you ever find the root of the problem? Was adding CFLOCK the only workaround?

9

Hello,
Unfortunately, the CFLOCK stay the only protection against multiple calls.

10

If I run the scheduler manually via web admin, it is only run once. The automatic scheduler worked for nicely until the last update to 5.4 or 5.3.12.1.

I am looking for the cause of this behavior. Could it be that the schedule.xml configuration file is defective?

11

We found a scheduler.xml.buggy file in the config files whose task was executed. After removing the file and restarting, the problem was solved.

1 Like
12

I wonder if we could get an admin alert of *.buggy file creation?