Same code 3-4x slower on production

I have an API based Application, no database. When I run it on my dev server is it 3 times faster than when I move it to production. Both environments are windows running 16GB of memory. The production server (Google Cloud Platform 4 CPU build 2.3Ghz) and the dev is a physical box (2 CPU 3.2 GHz) both running Intel Xeon.

Could the difference in CPU speed make the code in production run an average of 2.5s while the code in dev is averaging 600ms for the same call? All of the Lucee admin settings are also the same.

Is there a way to improve the speed in the production environment?

You also have same Lucee/Tomcat/Java Version on dev and prod?

No, everything on prod is much newer
Dev

Prod

There were some java 8 vs java 11 bugs which got addressed since 5.3.2.77, does the problem happen with the latest 5.3.4 RC ?

After update


I want to say it is slightly faster, I think it is just being hopeful but it is still nowhere the speed of the dev envir.

what version of the lucee loader are you running? \lucee\lib\lucee-x-x-x-x.jar ?

There are many key architectural differences in the way processing is handled on the newer processors vs older processors.

Yes, an old processor could be the reason your DEV environment is slow.

My suggestion is if DEV is not online or publicly accessible, you can disable UAC, as well as all the Windows processor mitigation

Link to Disable Windows Processor Mitigation
https://support.microsoft.com/en-us/help/4072698/windows-server-speculative-execution-side-channel-vulnerabilities-prot

@Terry_Whitney the Dev Environment is the FASTER environment, that is the problem

@Zac_Spitzer
Dev = lucee-5.1.0.34.jar
Prod lucee-5.3.2.7.7.jar

The FusionReactor profiler feature springs to mind as a way to identify what parts of your code are slower on production. But I would start by making your Java and Lucee versions the same across the board so you are comparing apples to apples.

see https://luceeserver.atlassian.net/browse/LDEV-2215

@Zac_Spitzer thanks I will give that a shot later today and report back to the group my findings.

1 Like

I have updated the JRE and there is noticeable improvement. Process run on prod from local postman 2.42s, 2.17s, 2.94s from The same process run from dev 836ms, 417ms, 472ms

Guess I will try FusionReactor next

I have uninstalled everything and reinstalled on the production server from https://download.lucee.org/?releases=46DEBB33B51D4B478EB0011B40F31A4B#core and now the Prod server is running as quickly as the dev server

Prod

1 Like

I guess thats because you switched from java 11 back to 8 …

So it looks like there are performance-issues with java 11.

I also tried java 12 (see second to the last post) and it also was not performing as well

did you try with the 5.3.4-RC?

Yes was one of the combo’ I tried with Java v12

Interesting topic,most of the time the application may be slow if we are not writing code properly and we are not using the plugins correctly.From coding perspective everything is same ,there is a chance for having difference in the application server.
It is always better to use some logs to identify if some thing is not going correctly.Apart from that in local system we may have a different network than the network we are using for Production Server.We may also use load balancer in Production Environment.Production environment should be always faster for the users.

Thanks

If you’re using something like postman to test CFML, you aren’t only testing the app server. You’re testing your connection to the server and all of the hops in between.