How does Lucee web admin work?

I would forget the admin URL bit for a moment and focus on the scheduler. Firstly, the URL configured in the tasks needs to be a URL that when hit from the server itself resolves correctly to the right site. Hitting localhost is probably NOT what you want since you’d be just hitting the Tomcat default web root, not the root defined in IIS and handled by Boncode, etc.

So, if you RDP into the server, open up a Chrome browser, what URL do you need to hit there in order to load the correct page you want to hit as part of the scheduled task? Because that’s the URL that Lucee needs to be using.

btw, I added this to the bottom of the server.xml file, under
<!-- ADD NEW HOSTS HERE -->:

     <Host name="[server_fqdn]" appBase="webapps">
          <Context path="" docBase="c:\web\public" />
          <Alias>localhost</Alias>
     </Host>

But after restarting tomcat, the context link still shows “mediacast:80”…

http://localhost is the url that should work (and does work in the browser). It is handled by IIS… For the default tomcat root, I would need to hit http://localhost:8808

Also, our scheduled task urls are currently being handled by windows Task Scheduler, using curl http://localhost …these work, but we’re trying to ditch windows task scheduler

OK, looks like I screwed up…
I forgot to open up the new test url to anonymous access. It was failing because it’s not coming from a logged-in session, but now it’s working.

However, the original scheduled task is still not running…or at least not successfully. Shouldn’t I get something in the WEB-INF/lucee/logs/scheduler.log if it fails? All I see in there are some pretty old Java exceptions.

…and now the original scheduled task is working again…

Is there a way to log these verbosely? e.g. so I could check the scheduler log and see if a task executed, and what the http status was?

1 Like

@partap great that you are making progress :smiley:

Just like @bdw429s said, I wouldn’t change anything. I was a little confused about your question “how does Lucee web admin work”, when in fact you just wanted to access a particular web context. What I’d do in your case is very simple, I do it quite the same when I need to access the web admin with tunnels, works very similar:

  1. RPD to the server running Lucee (should be the same you access the Lucee Server Admin)
  2. On that machine edit the host file and add:
mediacast 127.0.0.1
  1. open a terminal session on that machine with CMD and ipconfig /flushdns
  2. Then you should be able to access the web context on http://mediacast:8888/lucee/admin/web.cfm from that machine
  3. After finishing your work, change the host file back to what it was.

“mediacast:80” will continue to show up, since this context is generated by mod_cfml comming from IIS (or another machine) accessing it. I would really leave everything as it was. For accessing your web admin in mediacast, do it like I told you above.

We had a similar approach a while back. Some scheduled tasks in our system need System permission. No way you can/should move these to cfml, because Tomcat executes with low level priviliged local service, and you would not want to run Tomcat as local “System”. The windows taks that could run with low privileges and ran in a “global” manner for all websites, we moved as cfschedules to one single web context that acts as the global web context.

I usually check these through the usual access log files for that web context, since these are normal http request that execute a specific cfm/cfc template by calling its URL. These show access time and http status in details. For a more verbose logging, I’d create a custom logging feature from within your scheduler templates. You could even build a monitoring system around that, e.g. adding a the last execution time of a template into a database. The monitoring system could retrieve that information and fire an email notification after a certain amount without an execution.

That’s the workaround I’d been using…thanks.
Weird that you can’t set the name manually though… e.g., like virtual server names in Apache or nginx…

1 Like
  1. This is an OS constraint, not a Lucee / Tomcat constraint.
  2. Even on UNIX / LINUX devices, it is best practice to keep the HOST FILE updated with the vhosts running on the device.

So I finally got some actual data…I have a scheduled task set up to hit an update url every minute.

The endpoint processes and updates a queue, creating and deleting some xml files according to a schedule defined in our system (allcall schedule, to differentiate from lucee task schedule).

This morning the allcall schedule wasn’t working, and then it suddenly started back up again.

Checked the logs, I saw that the update url was being hit once per minute, as expected, until last night at 02:00:01…then nothing.
…then, this morning at 9:56:48, I got another hit, and another one at 9:57:01, and every minute since then on the :01s

The lucee task schedule looks like this

Any ideas about what could be happening?

Edit:
Just noticed that the server has another scheduled task that restarts iis and tomcat every morning at 5am (sysadmin paranoia)…so that happened right in the middle of the blank interval. I would think that if the scheduler got “stuck” that restarting lucee would kick it back into gear…

Partap, what logs are you checking? You’ve mentioned previously that you have your task code log when it’s run, and Andreas had proposed looking in the web server access log.

I’ll note that those focus on “whether the request ran”, not on “whether lucee even attempted to call the task”. And you said you saw no errors in the lucee scheduler log,

I have a couple of other ideas for you.

1: First, do you by any chance run FusionReactor, or would you be open to using its free 14 day trial? It offers a transactions>history page (with a drop down at the top offering httpclient) which would shows any calls out of lucee (or ACF), and under the cover a scheduled task call is just an automated equivalent to a cfhttp.

2: Indeed, yet another proof point for you (without need of fr) would be to setup a cf page that DID do a cfhttp to the url that your task is calling. Yes, I do realize that page sometimes runs for your then sometimes not. (I’m proposing above you make sure lucee is even TRYING to call it.) But now I’m adding that when you feel from your other logs that it’s “NOT being called”, you could at that time also check if the cfhttp call to it somehow ALSO stops working.

If so, now we’d see it’s perhaps NOT about the scheduled task mechanism after all: it may have been TRYING but just not REACHING the url for the task. And in that case, do a cfdump of the cfhttp scope (on that page doing the cfhttp) to see if it reflects whatever is amiss. It could be any of MANY problems.

3: Finally, if you “simply need this to work” and “can’t spend more days and weeks going back and forth here”, I’ll note that we could get together remotely to look at all this. We might even solve it in as little as 15 minutes, my minimum billable increment. I could also help you install/use FusionReactor. You can find more on my rates, approach, satisfaction guarantee, online calendar, and more at Charlie Arehart's Server Troubleshooting Consulting Services.

I know some may think it crass for me to propose that paid help here. I’ve offered a lot above for you to try on your own (and written in a way to help others who may find this thread in the future), as I try usually to do. Again, I just know sometimes people have to/are willing to move on from free help to paid, when it makes more economic sense for them. They just may not know who might even be available. Or Lucee also offers paid support and consulting options, as well.

1 Like

Thanks for the ideas!

I’d actually already set up another test page on our QA server using the same 1-minute schedule, so next time it happens I should be able to see if it’s only the one url that seems to be getting skipped.

We actually do have a dev license for FR…I haven’t used it in a long time and kinda forgot about it, but I will set it up on that server and see if the logs catch anything. Also, I like your idea about calling an indirect page that can log any cfhttp errors from the client side to the allcall page.

Oh, the logs I mentioned before are from the app…it logs every incoming request at OnRequestStart and OnRequestEnd…these were not showing up for around 8 hours yesterday morning.

The good/bad thing about this is that it only seems to happen rarely…good because it usually works, and bad because I have no way to trigger the problem. Right now, the only times I know that it’s happened were yesterday on our QA server, and two weeks ago on a client’s server. It may be happening more often, but these are the only times I know for sure. Also, both times were early Friday mornings, so it’s possible that has something to do with it, too…after all, it’s processing a schedule…only problem with that theory is that the same page has worked for years running curl from Windows Task Scheduler .

I appreciate your offer to look at it…but if it comes down to it, we’ll probably just fall back to using Windows Scheduler…Switching to Lucee’s scheduler was just supposed to make things easier for us :wink: …one less external dependency

Looking forward to hearing what you may find.

It happened again this morning and Monday, as well…

same thing: I see log entries every minute until 2:00:01, then nothing until 9:56:xx, then regular logs resume at 9:57:01

I haven’t found anything special happening on the server on MWF at 2 and/or 9:56, but it doesn’t seem to be random, in any case.

Again, you’re focused on what your log says–from the request having been run. We await your getting evidence (such as via fr) of whether a) it’s BEING called from lucee, or indeed if b) FR shows it TRYING to run when called–which is a separate point–such as if it tries but fails to run.

I appreciate how someone in your situation may want to focus more on what’s broken about Lucee, but we don’t yet know where the problem is.

OK…so this log that I’m talking about is from the new test task that I added last friday…the only thing it does is log a message and exit…

I figured if the allcall schedule page stopped running but this one kept running, I could add a cfhtml call there that could log any problems, but apparently both tasks stopped executing.

Anyways, I just found out how to set scheduler.log level to INFO …looks like that’s what I needed to see the client side of the scheduler request…looks like it’s working so far.

Ok, progress!

Turns out those times match when our VMWare snapshots are scheduled…
We just tried a manual snapshot and …BAM, the system clock jumped ahead 8 hours! …actually, 7 hours, 56 minutes, 49 seconds…The logs look exactly like what we’ve been seeing.

No idea why yet.

Also, don’t know when the clock is getting corrected. I’m not seeing any corresponding backwards time jumps in the logs when ntp server fixes the system time, except for when I ran it manually, just now…

…also doesn’t explain why it would work with Windows task scheduler but not Lucee’s scheduler…

:thinking:

OK, I think I have an explanation for everything now…

  1. The VMWare host clock had the wrong tz set, and no ntp time management (so almost, but not exactly 8 hours fast).
  2. Apparently, running a snapshot syncs the vm time with the host time.
  3. Lucee scheduler continues running every minute, but it thinks it is 8 hours ahead.
  4. Windows eventually checks the ntp time server and corrects the system clock.
  5. Lucee task scheduler looks at our scheduled tasks and checks against the last time they were run. Now it looks like the previous trigger was 8 hours in the future, so it doesn’t run any tasks until 1 minute after the last run time.
  6. Windows task scheduler doesn’t remember/doesn’t care when the task was last triggered…it just keeps running every minute, so once the clock is fixed, the schedule gets back into the correct state.
1 Like

Great sleuthing there, and glad you solved it. Again, with fr we could have confirmed that indeed lucee was not even TRYING to call the task, which at least would have take a LOT of other possible explanations off the table. Often such troubleshooting is about removing suspects, which allows you to focus on remaining ones–or even to discover new ones you’d never considered. Glad you could get there on your own. :slight_smile:

So, I’ve reported the problem (incorrect vm host clock), and the solution (set the damn clock correctly! :stuck_out_tongue: )

But now I’m getting pushback…many of our clients run their own VM hosts that we don’t control…and apparently telling them to set their clocks correctly is not a reasonable request (???)

@bdw429s What if Lucee’s scheduler could treat task execution timestamps that are set in the future as invalid…basically forcing a reset and treating it as a “first execution”?