First, are you sure it’s not that Lucee is unexpectedly restarting on you? Don’t presume that “can’t be happening so frequently” or that “it would be down long enough we’d expects to have seen outage messages”. It can restart fast (shut down and start) in many situations. And there are scenaries that can cause any app server to go down frequently–until the problem is resolved.
Zac offered some suggested diagnosis. For my first point here, simply check the logs (again, don’t presume “it can’t be that”).
Second, you indicate doing logging of requests. For this specific problem, log especially the incoming cookies for each request…or at least when a new session starts.
BTW, don’t look to the cookie scope for this: that can reflect a cookie set at the start of request processing such as one just created for a new session. Instead, look at the cgi.htrp_cookie variable. Often this proves VERY eye-opening in cases like yours.
(And for now I suspect you’re presuming the problem is “with Lucee”, but prove first exactly what’s happening, then figure out why.)
You may find there are NO cookies coming in, for these users “losing sessions”. That would cause creation of an. ew session. And of course it’s natural for a first-time visitor, or an agent that never sends back cookies, even on repeated requests where Lucee sets them. Focus on these users of concern. Then you’d explore how that could be happening.
Or perhaps the cookies (or some aspect) changes from one request to the next (for a given user), making Lucee think it needs to start a new session.
I know this may seem like hard work, but it’s really not. Since you asked for “guidance on further diagnostic steps or recommendations to help identify and resolve this issue”, these steps have helped spot the cause of problems like yours.
If nothing else, you’d have better evidence to present to explain the scenario.
I’ll note also that if you had FR, that can help also. With it you can easily see restarts, incoming cookies, counts of sessions, and more. And these things are logged, not just visible in the ui.
Finally, if indeed the loss of sessions is due to unexpected Lucee restarts, note that beyond preventing those another option to consider is storing sessions persistently, whether to a database or to a cache like redis. Both are options configurable in Lucee. And I did a talk at a recent cfcamp on using redis sessions (in Lucee or ACF), which is available on my site, Presentations by Charlie Arehart, including the pdf and YT recording.
And if you don’t succeed on your own or with guidance here over time, note that I’m available to offer remote screenshare consulting, in as little as 15 minute increments. For more on my rates, approach, satisfaction guarantee, online calendar, email, phone, and more, see Charlie Arehart's Server Troubleshooting Consulting Services.
But I’m sure there will be more offered here for you to consider. Hope you get to resolution soon and easily.
I’m also noting this to help casual readers realize it’s two different people raising session issues here.