Debug execution times- page is taking ~43-60s to load but Lucee reports only ~125ms

Trying to debug a long running page but Lucee doesn’t seem to be reporting
the actual execution time correctly.

Page takes around 43-60 seconds to load. If I remove this block of code it
loads quick, as in maybe 4 seconds- there is a similar routine for US
states that is also slow but not REALLY slow like the countries:

> <option value="#entry.getValue()#" entry.getValue()> > selected="">#HTMLEditFormat(entry.getValue())# >

cftimer and cfstopwatch are both logging 30ms for the block of code in
question, which can’t be right, it seems to take the bulk of a minute.

Debug output:

Execution Time

121.907 ms Application
4.792 ms Startup/Compilation
0.000 ms Query
126.699 ms Total

Total time (ms), Count, Avg TIme (ms), Template, Order
1.094 773 0.001 C:\git_\trunk\models\PropertyValue.cfc$getValue 36
0.059 112 0.001 C:\git_
\trunk\models\PropertyValue.cfc$getCode 35

Any ideas appreciated- I want to know why Lucee is not giving me an actual
page load time anywhere near what I’m experiencing. CPU spikes to around
50% and laptop fan spins up while it is thinking about this and it is the
lucee process.

Version Lucee 4.5.2.018 final
OS Windows 10 (10.0) 64bit
Servlet Container Apache Tomcat/8.0.28
Java 1.8.0_66 (Oracle Corporation) 64bit
MySQL 5.7

Browser makes no difference, same behaviour in both Opera and Edge, page
actually takes 45s-60s but Lucee reports it is running very quickly. I can
see Lucee actually whirring away pegging the CPU in Task Manager so
difficult to see how it could be the browser.

The same code doesn’t seem to have a performance issue, or not as major a
one anyway on our Linux development server running Railo, for whatever
reason, my suspicion here is that it’s something peculiar about the setup
on my laptop, I’d like to figure out what exactly.On Saturday, 9 July 2016 01:37:49 UTC+7, Terry Whitney wrote:

Try a different browser and see if that makes any difference.

Same thing happened here.On Friday, July 8, 2016 at 2:10:47 PM UTC-5, Ivan McAvinchey wrote:

Browser makes no difference, same behaviour in both Opera and Edge, page
actually takes 45s-60s but Lucee reports it is running very quickly. I can
see Lucee actually whirring away pegging the CPU in Task Manager so
difficult to see how it could be the browser.

The same code doesn’t seem to have a performance issue, or not as major a
one anyway on our Linux development server running Railo, for whatever
reason, my suspicion here is that it’s something peculiar about the setup
on my laptop, I’d like to figure out what exactly.

On Saturday, 9 July 2016 01:37:49 UTC+7, Terry Whitney wrote:

Try a different browser and see if that makes any difference.

Try a different browser and see if that makes any difference.On Friday, July 8, 2016 at 9:43:03 AM UTC-4, Ivan McAvinchey wrote:

Trying to debug a long running page but Lucee doesn’t seem to be reporting
the actual execution time correctly.

Page takes around 43-60 seconds to load. If I remove this block of code it
loads quick, as in maybe 4 seconds- there is a similar routine for US
states that is also slow but not REALLY slow like the countries:

> <option value="#entry.getValue()#" entry.getValue()> > selected="">#HTMLEditFormat(entry.getValue())# >

cftimer and cfstopwatch are both logging 30ms for the block of code in
question, which can’t be right, it seems to take the bulk of a minute.

Debug output:

Execution Time

121.907 ms Application
4.792 ms Startup/Compilation
0.000 ms Query
126.699 ms Total

Total time (ms), Count, Avg TIme (ms), Template, Order
1.094 773 0.001 C:\git_\trunk\models\PropertyValue.cfc$getValue 36
0.059 112 0.001 C:\git_
\trunk\models\PropertyValue.cfc$getCode 35

Any ideas appreciated- I want to know why Lucee is not giving me an actual
page load time anywhere near what I’m experiencing. CPU spikes to around
50% and laptop fan spins up while it is thinking about this and it is the
lucee process.

Version Lucee 4.5.2.018 final
OS Windows 10 (10.0) 64bit
Servlet Container Apache Tomcat/8.0.28
Java 1.8.0_66 (Oracle Corporation) 64bit
MySQL 5.7

Anyone have any more ideas as to what this could be?

I am 99% sure the *underlying *issue is something to do with ORM- it always
happens around ORM accesses.

But I can’t even debug it as Lucee is not giving me correct debug
information and is telling me nothing is wrong and the page is loading
quickly! After it has spent MINUTES churning away on my laptop. It is lying
to me!

I’m seeing it again now on another page which loads a few entities. This
page is really inconsistent- sometimes it will run for 10 minutes at 100%
CPU before failing with a Java Out of Memory error

Lucee 4.5.2.018 Error (java.lang.OutOfMemoryError)
Message Java heap space
Cause java.lang.OutOfMemoryError

This I think is to do with the error handling and may involve FW/1 or our
custom code’s error handling- the above error was actually in the “dump”
tag which was failing trying to dump a variable.

On another reload, it pegs the CPU for at over 10 minutes (it was
definitely longer than the 300 seconds reported) before deciding that
entityLoad has timed out:

request (path: …1) has run into a timeout (300 seconds) and has been
stopped.
61: rc.companies = entityLoad(“Company”, {active = 1}, “name asc”);

On another reload, it runs fine and comes back pretty much instantly!

It is completely random, sometimes it loads fine, sometimes it pegs the CPU
for ten minutes and fails. It works 100% of the time on Railo.

OK- so that is the underlying issue. But what I am concerned about here is
the Lucee debug output.

I then turn on debugging and reload, it takes well over a minute but Lucee
is telling me the page processed in a bit over a second. IT DIDN’T! No you
didn’t! Second my ass, you were churning away on this for five minutes. I
could hear the fan! I could see the Lucee process pegged at 100% CPU! 1.3
seconds my ass that is a barefaced lie!

1,232.145 ms Application
129.700 ms Startup/Compilation
0.000 ms Query
1,361.845 ms Total

Looking at the details this is the stuff with repeated ORM accesses- is
this a problem? Is it related? Is the total accurate? I don’t know. But my
hunch is it has something to do with these ORM entities with several
hundred records, I always see this issue in this context.

0.64 45 0.014 \WEB-INF\lucee\context\Component.cfc
1.328 38 0.035 \components\helpers\model.cfc
6.405 942 0.007 \modules\companies\models\Member2.cfc$getId
0.189 249 0.001 \modules\companies\models\Company.cfc$getId
0.646 942 0.001 \modules\companies\models\Member2.cfc$getCompany
0.73 1191 0.001 \modules\companies\models\Company.cfc$getName
0.416 942 0 \modules\companies\models\Member2.cfc$getFirstname
0.331 942 0 \modules\companies\models\Member2.cfc$getLastname
0.279 942 0 \modules\companies\models\Member2.cfc$getMiddleinitial

This page works, no issues, on our production server running Railo.

If anyone had any ideas as to what the issue could be that would be great,
I’m at the point where I am really thinking of just giving up on Lucee
entirely and going back to an old version of Railo to match our production
environment and giving up the idea of transitioning that over to Lucee, it
is very frustrating and I can’t even get information out of Lucee that
seems to gel with the actual behaviour I’m witnessing.

One key difference between the configuration of the two is that I have
this.ormsettings.autoManageSession= “true” on the Railo server and
this.ormsettings.autoManageSession= “false” on Lucee, due to some other bug
with Lucee’s ORM handling which was apparently introduced in Railo just
before the fork (bug #77 I think).

Note I am on Lucee 4.5 because Lucee 5 has an issue with FW/1.

Any ideas appreciated, if there was anything that could even just point me
in the right direction with this that would be fantastic.

Ivan

Looking into this further, Tomcat seems to be logging the duration
accurately, if I add the “%D - Time taken to process the request, in
millis” to the log file it logs the duration I am seeing it take. 86
seconds in this case.

So- why is Tomcat logging 86.9 seconds when Lucee is telling me it is
taking only 1 second? And this only happens on these pages that have ORM
entities with hundreds of records- it’s as if there is something small
happening on each call of a getter on the ORM entity that Lucee isn’t
counting that then gets amplified when I access an ORM entity with hundreds
of records repeatedly.

Pages without problem (although 2.4 seconds DOES seem slow for the home
page- but largest ORM entity there has only 18 records):

127.0.0.1 - - [02/Sep/2016:19:06:49 +0700] “GET /index.cfm HTTP/1.1” 200
89440 2453
127.0.0.1 - - [02/Sep/2016:19:07:04 +0700] “GET /index.cfm/main/news
HTTP/1.1” 200 86904 647
127.0.0.1 - - [02/Sep/2016:19:07:22 +0700] “GET
/index.cfm/main/news/categoryID/1 HTTP/1.1” 200 87084 450

Page with problem (this is the same page with an alias):

127.0.0.1 - - [02/Sep/2016:19:09:00 +0700] “GET /index.cfm/main/offices
HTTP/1.1” 200 233066 86913
127.0.0.1 - - [02/Sep/2016:19:11:11 +0700] “GET /index.cfm/main/searchform
HTTP/1.1” 200 233054 65289

Largest ORM entity here has around a thousand records and there are a few
with hundreds.

That 65 second page Lucee is reporting 1,139.622 ms.

Because Lucee is reporting the time it takes exclusive of debugging
time itself. Retrieving data, building an internal datastructure and
generating the regular HTML only takes 1 second. Generating the debug
output takes another 86 seconds. And with th amount of data the debug
output is sending back even the transfer time may be non-trivial
(which gets counted by Tomcat as well, even though it has nothing to
do with the server if the client is behind a slow connection).

JochemOn Fri, Sep 2, 2016 at 2:35 PM, Ivan McAvinchey wrote:

Looking into this further, Tomcat seems to be logging the duration
accurately, if I add the “%D - Time taken to process the request, in millis”
to the log file it logs the duration I am seeing it take. 86 seconds in this
case.

So- why is Tomcat logging 86.9 seconds when Lucee is telling me it is taking
only 1 second?


Jochem van Dieten
http://jochem.vandieten.net/