Suggestions for good use-case for Task Threads?

language ideas
1

I’ve read-up on Task threads, thread type="task", and I’m struggling to understand what common use-cases for such a thread would be? These threads don’t have access to the contextual application (which means they can’t take advantage of any Components that have been loaded / initialized into memory). They can’t even reference other methods in the same page that they were spawned. It seems that everything they need has to be passed-in as a thread-attribute. This makes them quite a bit more complicated than daemon threads.

So, I am wondering what people are using them for?

I was thinking that making external HTTP calls might be a good use-case. But, even then, doing that in an application context often uses some sort of configured Gateway Component that has API keys and all that jazz. Which, the task thread wouldn’t have access to … so, I am just not sure.

3 Likes
2

wondering the same thing. Maybe doing multiple parallel database requests? Maybe even from different connections to different types of databases?
Besides setting tasks for threads what other events would warrant triggering of a pre-programmed task to execute?

1 Like
3

I’m not even 100% sure you could make database calls inside a Task thread since it probably wouldn’t have access to the per-application data sources. I suppose if you have a DB connection defined in the Lucee Admin, then it would work; but, I know that more-and-more configuration is moving into the app itself, which I’m guessing won’t be available to the Task thread body.

For what it’s worth, in the 4-years since my original post, I have not come up with any real use-cases.

3 Likes
4

I was just looking into this myself.

From what I’ve read here Thread Task :: Lucee Documentation the big advantage seems to be the Retry Interval. From the linked page:

Task thread : Different than daemon threads, task threads can run more than once if they fail. You can schedule a later retry, and you can control it in the administrator.

I haven’t had the chance to try it yet but it could be useful with unresponsive remote 3rd parties or incomplete transactions.

Hope this helps!

1 Like
6

Hi Ben. Big fan, been reading your stuff for ages, you’ve been a great help!

The latest system I wrote does lots of, and all of its, CFHTTP operations using task threads for fully-automatic independent/async-running retries when necessary. For example, trying a CFHTTP every 60 seconds until it works, retrying up to 60 times before giving up, meaning it automatically tries once per minute for an hour, without you having to lift a finger after you launch it. That sort of thing.

You have to pass in everything the task thread will need as a parameter up-front so it’s totally self-contained and doesn’t need any outside data during the course of its life. But it works great and is very convenient!

It’s solved the problem of brief and transient network problems and also brief and transient remote host downtime. It makes my CFHTTP transactions hugely more reliable.

Note that my CFHTTP transmissions can arrive in 1 second or 1 hour, and either way won’t cause any functional problems because it’s within a 1-hour acceptable “transmission window” that’s been established and agreed upon with the remote party/system.

Realtime-critical stuff, well that’s a whole different ball of wax. But if you and the remote party can define acceptable time windows for transmissions (usually meaning the receiving party will receive and store the data in a file (or DB record) whenever it comes in, but won’t READ the DB record or file and USE its data until the hour is up), it works wonderfully.

7

@ToroMaduro really interesting. I would love to hear more about what kind of information you have to provide to the CFHTTP task thread. Also, how are you handling errors? For example, if something eventually does fail; or let’s say you run into an error that can’t be resolved, like a 403 Forbidden due to a missing API key, where does that error go? That’s what I keep coming back to - all these little details that just beg for application context (ie, let me log that to the same ErrorHandler that the rest of the app uses).

1 Like
8

Okay, fair warning Ben, this is going to be a bit long, but I know you’ll like it. It’s a summary of how my application uses task threads to do “fire and forget” CFHTTP sends of information to remote servers with hands-off, fully automatic periodic retries within an agreed-upon “transmission window” (as I describe above), with error handling and logging (which you inquired about!). Here goes:

Think of a task thread as a completely self-contained little thing that’s pre-loaded with every piece of data it needs (usually they’re all passed in as parameters up-front) so that it doesn’t need to access anything external… no variables or data structures from the request which launched it (which, for retries, is usually ended long ago anyway) or from the Application or other shared scope(s)… nothing outside of itself.

Except for files, that is. It can read and write files (provided you either hard-coded the file paths/names in, or you passed the file paths/names in up-front in parameters when spawning the task thread).

So Ben, what I do is spawn a task thread with EVERYTHING that’s needed to, say, send a CFHTTP transmission, and I have designated log files that the task threads can write to (log file paths/names passed in as parameters up-front). The task threads can write to the log file(s) upon successful or failed transmission, or if you prefer, only upon error. Whatever you like. But, whenever they want to put something in the log, a simple FileAppend() call does the trick. I also wrap the FileAppend() in a logfile-specific named lock with CFLOCK, to prevent multithreaded race condition problems, ESPECIALLY when several task threads are sharing the SAME log file (as opposed to them each having their own log file, again your choice).

So what it looks like is:

<cfthread name=“whatever” type=“task” action=“run” retryinterval=“(whatever you decide on!)” …

…followed by a lot of parameters, giving it all the data it will ever need to use as it runs, possibly repeatedly at different times, detached and all by itself, all under control of the Lucee Server which is following your instructions about retry counts and timings between retries… without you having to do anything (which is the main appeal).

Within the task thread code, to do the transmission, you just do a CFTRY, then inside that the CFHTTP, then a bunch of exception/error/problem-detection code, which CFTHROWs as necessary, which is CFCATCH’ed by a block below that namelocks the log file, FileAppend()s a nice informative message to it detailing what problem happened, then the named lock is released. (You can also do whatever other error-handling stuff you might need to do, here).

The next bit is interesting! At this point, having written a description of what happened to the log, you can now decide to either:

  1. Simply end this run/attempt of the task thread WITHOUT an active/unhandled exception, which terminates THIS run of the task thread, but leaves it active and thus eligible for automatic retries (if any still remain) after the proper waiting interval (although that’s all fully automatic and not your problem, ALL you have to do inside that CFCATCH block is just do a CFABORT (since you’re inside a CFCATCH block, the exception that got you into it has been handled and thus “doesn’t count” as an active/unhandled exception, so just doing a CFABORT is considered ending without any active/unhandled exception being in effect).

Or, you can:

  1. End this run/attempt of the task thread DELIBERATELY WITH an active/unhandled exception, which ends the task thread and makes it ineligible for any further retries (if any remain). No further retries will ever occur. To do this, instead of doing CFABORT inside the CFCATCH block, do a CFRETHROW followed by an (optional I guess, but I use it) CFABORT. This takes the exception that got you into the CFCATCH block, which was “handled” by that CFCATCH block, and now re-throws it so it’s now NOT “handled”, resulting in the task thread ending WITH an active/unhandled exception.

There are only two CFHTTP return values which should cause us to cancel all future retries: 200 (yay, success, we’re done, retrying would be pointless and stupid!) and 404 (target URL/module not found, retrying would be pointless and stupid!).

I end the task thread using method #1 when CFHTTP returns a status value of anything other than 200 or 404, because these are the transient problem situations where we DO WANT retries to occur.

I end the task thread using method #2 when CFHTTP returns a status value of 200 or 404, because these are the situations where we DO NOT WANT retries to occur.

Hope that helps! Throw a test task thread together, Ben, as I describe above, and play with it. You might love it like I do, for stuff like agreed-time-windowed CFHTTP transmissions and similar things!

1 Like
9

Writing to physical log files is a good idea. I feel like I’ve always had a blind-spot for log files. And, once we started using 3rd-party logging services, I kind of forgot about log files. But, in this type of case, they make sense since it’s really the only logging you have access to. And, presumably, in a containerized context, you could still have some service slurping up the log data and making it available somehow.

I’m gonna let this sink in. I really want to like the idea of Task threads :laughing: I really appreciate your in-depth response. And you are clearly getting a lot of value out of the mechanics of it.

1 Like
10

Cool! You’ve helped me dozens of times over the years with your blog posts, so I’m happy to provide a tiny bit of help in return!

Yes, I like simple… so much software dev is unnecessary complexity. I don’t want to use a LogFactoryFactoryFactory or a logging third-party framework or classes… I just want to write/append to a regular old file at a known path. Simple is GOOD for so many reasons.

By the way, I edited and improved my big post several times, probably while you were writing your reply, but I’m done now, so read it again if you want the best, final version. :slight_smile:

Anything else I can help with, let me know!

1 Like
11

You might also find this stuff useful:

To clear all tasks:

<cfset myadmin = new Administrator(type:“web”,password:“whatever_the_web_admin_pw_is”)>
<cfset myadmin.removeAllTasks()>

To get a list of tasks:

<cfadmin action=“getSpoolerTasks” type=“web” password=“whatever_the_web_admin_pw_is” returnVariable=“the_scheduled_tasks”>
<cfdump eval=“the_scheduled_tasks”>

Just two random code snippets of interest. There are lots of other things that can be done programatically to view/modify the list of task threads, some Googling should reveal most of it.

12

Ugh. It looks like I needed more coffee when I typed up that long post above.

I inadvertently reversed #1 and #2.

Ending the task thread’s run in error, meaning WITH an unhandled exception in effect, will cause retries to be automatically performed per the “retryinterval” attribute specified in the cfthread tag (if any retries still happen to remain, that is).

Ending the task thread’s run normally, meaning WITHOUT an unhandled exception in effect, will prevent any and all further retries.

Similarly, what I meant to say was that:

When CFHTTP returns a status value of anything other than 200 or 404, these are usually transient problems so I DO want retries, therefore I do a cfrethrow to cause the task thread to end WITH an unhandled exception in effect, which DOES cause retries to occur (if any remain).

When CFHTTP returns a status value of 200 or 404, these are the two situations where I DO NOT want retries (because retrying after success or after being told destination-not-found-or-invalid is pointless) therefore I DO NOT do a cfrethrow, I just do a cfabort to cause the task thread to end WITHOUT an unhandled exception in effect, which prevents any further retries from occurring.

Sorry I mixed them up, I can’t believe I didn’t catch that until now!