Lucee 6 "BigDecimal" causes big problems for intensive calculations

[ToroMaduro]

OS: Linux
Java Version: 11.0.11
Tomcat Version: 8/9
Lucee Version: 5.x and 6.x

I recently wrote a .cfm module that does enormous amounts of floating point math very quickly. It spawns multiple threads, each of which handles a subset of the total calculations, and they all run in parallel and saturate the machine’s CPU cores so the overall task gets done very quickly.

Collectively it’s at least a hundred million or more floating point operations, and due to the core-saturating parallelism they all happen very VERY quickly.

Under Lucee 5.x, it works beautifully and shockingly fast, and memory use (heap and stack) is so tiny it doesn’t even register on the admin page’s realtime memory usage graphs. That’s because Lucee 5.x uses the “double” type, which is a mutable primitive type held on the Java stack, and the countless millions of changes to the working variables are done IN-PLACE, with NO new memory being allocated for new copies of the variables.

Under Lucee 6.x, my process dies with an out-of-heap-memory error, after spiking the heap memory use up to the moon. That’s because Lucee 6.x is no longer using “double” but instead “BigDecimal”, which is an immutable non-primitive held in the Java heap, and every one of those millions of changes to the working variables makes NEW copies which eats up the entire heap quickly.

Apparently the garbage collector can’t keep up with this, and the program crashes (with only 1 of its threads, the lightest-workload one, having had time to finish). A minute or so after the module crashes and dies with an out-of-heap-memory exception, the garbage collector seems to run, because the Admin’s heap graph drops back down to near-zero.

In addition to this fatal crashing problem from insane heap memory use, the “BigDecimal” floating point operations are much slower than “double” ones.

I know “BigDecimal” is more accurate, but what I’m doing does not require that extra accuracy at all. These numbers represent just estimates, no extreme precision is needed, in fact the precision of “double” is already far more than enough for this application.

I realize that the majority of Lucee code probably isn’t extreme like this and won’t be significantly harmed by the double-to-BigDecimal switch, but some (like mine) is hugely harmed by it.

Can we please get one or more of the following?

1. Admin (unified simple-mode admin, server admin and/or web admin) setting telling Lucee to use the “double” instead of “BigDecimal” for floating point variables?

2. More granularly, a per-template setting (via the cfsetting tag maybe?) to tell Lucee to use “double” for floating point variables for this specific module only?

3. Most granularly of all, a way to declare only certain variables (the ones that change many millions of times) to be “double” not “BigDecimal”?

If not, I’ll have to stick with Lucee 5.x indefinitely, or switch to another language entirely, which I don’t want to do.

Thanks!

[bennadel]

I have no idea if this would work; but, perhaps using an explicit javaCast( "double", value ) would at least help as stop-gap measure? Perhaps if you apply that to your initial values then Lucee will continue to use it when it does the calculations? I’m not sure when the conversion to BigDecimal happens.

var a = javaCast( "double", a );
var b = javaCast( "double", b );
var c = ( a * b );

The question is, is c a double? Or, has it become a BigDecimal Again, I haven’t tested this - just thinking out loud.

[Ivan]

Hi @bennadel, I did a test and I think it’s as you suspected. c becomes BigDecimal

a = javaCast("double", 1.1);
b = javaCast("double", 2.2);
c = (a * b);

writeOutput("Type of a: " & getMetadata(a).getName() & "<br>"); // Type of a: java.lang.Double
writeOutput("Type of b: " & getMetadata(b).getName() & "<br>"); // Type of a: java.lang.Double
writeOutput("Type of c: " & getMetadata(c).getName() & "<br>"); // Type of c: java.math.BigDecimal

[bennadel]

Ahhh, sorry was worth a try.

[ToroMaduro]

Confirmed, Ivan, I ran your code in Lucee 6 and c is indeed a BigDecimal. This is unfortunate. Thanks to you both, Ivan and Ben, for the ideas and test code!

I hope this will get rectified somehow, soon. This needs to be selectable, as each type is optimal in different situations. BigDecimal can remain the default so most users (who aren’t doing huge calculations) don’t have to take any action at all. However guys like me, who need to do huge amounts of calculations which don’t need BigDecimal precision, can select double instead.

I was absolutely shocked at the high throughput under Lucee 5.x. It was unbelievably fast (and on an older 2017 mid-level “meh” PC!), and used basically no memory. That kind of screamingly-fast performance enables a LOT of additional interesting and powerful use-cases for Lucee!

I don’t want to lose that! Lucee needs to keep that kind of awesome capability!

[David_Anderson]

Would the issue not be solved by simply using javaCast() again?
eg:
c = javaCast(“double”, a * b);

admittedly not the most graceful solution, and a setting would still be nice, but perhaps as a temporary stopgap?

[ToroMaduro]

David, that probably wouldn’t help, because the “a * b” part likely is still done with BigDecimal, so one or more immutable BigDecimals would be created internally during the operation which would cause all the same problems described above. Sure, we can cast the result back into a double afterward, but the “damage” was already done internally by Lucee during the math operation.

If anyone knows a stopgap way to force the actual math operations themselves to be done as double, please tell!

Settings are definitely the way to go though, yes. Default to BigDecimal so most users don’t have to do anything (or even realize the issue exists at all), but those of us who need “big computation” ability can switch the setting to double.

[David_Anderson]

Why would that be, though? If a and b are created the way described above (ie cast to doubles), then aren’t they still doubles when doing “c = javaCast(‘double’, a * b)” ?

Oh, wait – on second thought, I think I get what you’re saying. the a * b is likely going to create a BigDecimal value before it gets cast to a double. That makes sense. Oy.

I hope you get a proper solution from the Lucee team.

[BK_BK]

@ToroMaduro , I think that what you have reported is a show-stopper for Lucee 6. In fact, it makes Lucee 6 not fit for purpose with regard to applications that use floating-point calculations.

Leave the Javacast aside. Then you will see that the problem is deep and far-reaching.

a = 1.1;
b = 2.2;
c = a * b;
writeoutput("<p>Value of a: " & a & "<br>");
writeOutput("Type of a: " & getMetadata(a).getName() & "</p>"); 
writeoutput("<p>Value of b: " & b & "<br>");
writeOutput("Type of b: " & getMetadata(b).getName() & "</p>"); 
writeoutput("<p>Value of c: " & c & "<br>");
writeOutput("Type of c: " & getMetadata(c).getName() & "</p>");

Lucee 6 result:

Value of a: 1.1
Type of a: java.math.BigDecimal

Value of b: 2.2
Type of b: java.math.BigDecimal

Value of c: 2.42
Type of c: java.math.BigDecimal

Lucee 5 result

Value of a: 1.1
Type of a: java.lang.Double

Value of b: 2.2
Type of b: java.lang.Double

Value of c: 2.42
Type of c: java.lang.Double

Lucee 6’s use of the BigDecimal class is excessive. This has disastrous consequences for performance, as you have pointed out. So, your decision to stick with Lucee 5 for the time being is wise.

[David_Raschper]

There is already a setting for that under Settings->Language/Compiler->Precise Math.
When you set it to false, it keeps the double values.

[ToroMaduro]

David, I tried turning that setting off in the administrator. My program still died in the exact same way (heap spikes to the moon, out-of-memory error) just like it does when that setting is turned on.

The setting does change the variable type shown by BK_BK’s test code above. When on, it’s reported as “java.math.BigDecimal”, when off it’s reported as “java.lang.Double”.

I suspect some part(s) of Lucee internally aren’t respecting that checkbox setting and are using BigDecimal internally for some things regardless of the setting’s status (whether on or off).

That setting needs to actually work fully and make things run like they do in Lucee 5.x. So for now, the “show-stopper” nature of this Lucee 6 problem for large numbers of floating point calculations remains in effect, unfortunately.

[carehart]

@ToroMaduro, can you let us know a few more things? First, what’s the heap max is for your lucee 6 instance? I don’t see any mention or or request for that. It might matter. And even if you’d say it’s “the same as lucee 5 was”, please confirm and report that value as well.

Also, is the Java 11 you report the same in both implementations?

And let us know if you see confirming these values in the lucee admin. If not, then by what means?

And the implication is that these are both tested on the same machine. Can you confirm? If not the same, what differs?

And was lucee installed the same way on both? There are of course many ways to run it.

I know these are not answers, and they point away from the presumption that there’s a bug, or that this matter of data types is the crucial thing. Those may well be the problem. I just want to help rule out other reasonable possible explanations for the difference, which should be easy to do here.

[David_Raschper]

Can you create a reproducible testcase?

I tried multiplication, sum etc. in a loop and get the same speed on 5.4 and 6.0 (with precise math disabled). On 6.0 with precises math enabled it´s 2x slower.
But i was not able to reproduce the problem with precise math disabled.

[ToroMaduro]

Careheart, great questions! Getting such details is important! Here are some answers:

Yes, it’s the same 16 GB RAM Linux machine with two different Lucee Express directories, 5 and 6. This is a test machine and nothing else of significance is running on the machine and nothing else (no other requests) are running in Lucee either.

Heap max for Lucee 5 instance: Not shown on admin, how do I find this value?
Non-Heap max for Lucee 5 instance: Not shown on admin, how do I find this value?
Lucee 5 Servlet Container: Tomcat 9.0.11 (came with Lucee 5 Express)
Lucee 5 Java version: 11.0.11+9 AdoptOpenJDK 64-bit
Lucee 5 admin overview’s realtime heap usage graph during module’s run: No significant heap usage at all, for the entire run. Flat line at/near zero.

Heap max for Lucee 6 instance: Shown on admin as 3971 Mb
Non-Heap max for Lucee 6 instance: Shown on admin as 1263 Mb
Lucee 6 Servlet Container: Tomcat 8.0.36 (came with Lucee 6 Express)
Lucee 6 Java version: 11.0.11+9 AdoptOpenJDK 64-bit
Lucee 6 admin overview’s realtime heap usage graph during module’s run: Spikes near the top and stays there until request is killed (makes no difference if the ‘precise math’ admin checkbox is checked or not).

Both my Lucee 5 and 6 environments are “Express” directories and both use the exact same Java, 11.0.11. In fact, when I downloaded Lucee 6 Express, I copied the JRE directory from the Lucee 5 Express directory over to the new Lucee Express 6 directory (because Express .ZIPs don’t come with a JRE anymore). So it’s EXACTLY the same Java version, directory, files, etc.

I didn’t change the heap/non-heap sizes for either the 5 or 6 Express, they’re set however they came in the Express .ZIP file that Lucee distributed. Thus they’re probably identical or very similar. Given that the Lucee 5 heap chart never budges from at/near-zero and the Lucee 6 heap chart blows up near the top, I suspect Lucee 5 is mutably changing the same small set of working doubles millions of times (no memory increase at all) while Lucee 6 is making new immutable BigDecimals (or SOMETHING!) millions of times (huge memory increase and crash).

SIDE NOTE: For some strange reason, Lucee 5 Express came with Tomcat 9, but Lucee 6 Express came with Tomcat 8, which is a backwards step and just makes no sense at all. But Tomcat’s clearly not the issue here, so that’s not too relevant.

David, great idea, I’ll work on a reproducible test case soon, and assuming I can make one, will give it here when it’s done.

Hope that helps! If you need anything else, just ask. Thanks!

[ToroMaduro]

Here is the reproducible test case, guys. It launches 8 cfthreads that run simultaneously, similar to my application. However, I’ve taken out all my application’s processing and I’ve replaced it with just a simple looped summing of random numbers. The effect is the same though!

Note that the loop going from 1 to 50 million in each thread is arbitrary, you can raise or lower it if you want. If you make it low enough it might actually complete without an out-of-heap error on Lucee 6, but that does NOT fix the problem, Lucee 6 is still crushing the heap and greatly limiting how much computation can actually be done, whereas Lucee 5 handles all 50 million x 8 threads fast and perfectly with no heap memory usage increase.

As the module says, try this under 5, 6 with Precise Math off, and 6 with Precise Math on.

Note I’m using a crappy mid-range 2017 PC with a 6th Gen Intel Core i5, results may vary on newer hardware, but I’ll bet they won’t! And simply raising the 50 million to a higher value will probably crash it on new PCs anyway, while Lucee 5 happily works perfectly with no significant heap use no matter how high the number is.

Have fun! Here it is. Note that you may have to increase your request timeout (I have mine set high because I run these sorts of long-running, heavy-computation processes frequently):

<cfset start_ms = GetTickCount()>

<br>Starting up...<br><br>

<b>Try this module in both Lucee 5 and in Lucee 6 with "Precise Math" turned both on and then off.<br><br>
Also make sure to watch the Lucee admin's heap memory graph while this is running!</b><br><br>

<cfloop from="1" to="8" index="thread_loop_i">
   
   <cfoutput>Launching thread #thread_loop_i#...<br></cfoutput><cfflush>
      
   <cfthread action="run" name="t#thread_loop_i#" priority="LOW" thread_number="#thread_loop_i#">

      <cfset thread_total = 0>
      <cfloop from="1" to="50000000" index="i">
         <cfset thread_total = thread_total + Rand()>
      </cfloop>
      
      <cfset cfthread.thread_total = thread_total>
      
   </cfthread>

</cfloop>

<hr>Please wait, doing cfthread join...<hr><cfflush>

<cfthread action="join" name="t1,t2,t3,t4,t5,t6,t7,t8">

cfthread join completed.<hr>

<cfset end_ms = GetTickCount()>
   
<cfset runtime_secs = (end_ms - start_ms) / 1000>

For a successful run, <b>all 8 threads will have their thread scopes dumped below with STATUS=COMPLETED</b>, there will be no exception messages inside any of the dumps, and every dump will have a THREAD_TOTAL value (inside childThreads) that shows the end result of the successfully-completed thread's calculations (a sum value pretty near 25 million).<br><br>

For failed runs, some (or all!) of the 8 thread scopes may <b>not</b> be dumped below, and some or all of them may be STATUS=TERMINATED with the ERROR field's subfields showing out of heap memory errors, and there will be no THREAD_TOTAL value in the terminated ones.<br>
There will also be an exception error (usually at the very bottom) that reads:  <b>"lucee.runtime.exp.NativeException: Java heap space" / "Caused by: java.lang.OutOfMemoryError: Java heap space."</b><hr>

This module runs successfully to completion and fast (60 seconds on my crappy mid-range 2017 Core i5-6500) on Lucee 5 every time, with the admin heap chart showing near-zero heap usage during the entire run.<br><br>

In Lucee 6 it crashes after filling up the heap (spikes the heap graph heavily), likely due to 5 using mutables and 6 using immutables.<br>
This happens <b>even when</b> the "Precise Math" option is <b>disabled!</b><br>
It looks like the "Precise Math" option needs to be fixed so it works FULLY (meaning just like how Lucee 5 works).<hr>

<cfdump eval="runtime_secs">

<cfdump eval="cfthread.t1">
<cfdump eval="cfthread.t2">
<cfdump eval="cfthread.t3">
<cfdump eval="cfthread.t4">
<cfdump eval="cfthread.t5">
<cfdump eval="cfthread.t6">
<cfdump eval="cfthread.t7">
<cfdump eval="cfthread.t8">
<hr>
                   

[gaia-jyh]

If you use <cfloop> with a very large number of iterations in CFML, it can cause Out of Memory errors. This happens because CFML tends to generate a lot of unnecessary line breaks and whitespace, which you can see when you view the source in Chrome. This excessive output can consume a lot of memory, leading to performance issues.

To avoid this problem, it’s better to handle large loops in a way that minimizes unnecessary output, or consider using CFScript instead, which is more efficient for handling such cases

[ToroMaduro]

gaia-jyh, believe it or not, that’s it!

I wrapped the cfloop tag in a cfsilent bufferoutput=“false” tag, and Lucee 6 no longer crashes and it performs just as fast as Lucee 5 does:

Runtime Lucee 5: about 60 seconds.
Runtime Lucee 6: about 60 seconds (Precise Math turned off)
Runtime Lucee 6: about 80 seconds (Precise Math turned on)

Also, with that cfsilent bufferoutput=“false” tag in place, the Lucee 6 heap graph barely moves off of zero, same as with Lucee 5!

CONCLUSION - The problem isn’t with BigDecimal, rather the problem is that without using the cfsilent bufferoutput=“false” tag, Lucee 5 somehow seemingly doesn’t buffer all the loop-induced whitespace, while Lucee 6 does.

SOLUTION - Just use the cfsilent bufferoutput=“false” tag in Lucee 6.

The only question remaining is why, without the cfsilent bufferoutput=“false” tag, Lucee 5 behaves like Lucee 6 does WITH it. That’s very strange.

Other than that odd remaining question, this solves the problem! Thanks very much!

P.S. – check out the precision differences in the THREAD_TOTAL numbers (example numbers shown):

Lucee 5 and Lucee 6 with Precise Math turned off.:
25003231.887363173

Lucee 6 with Precise Math turned on:
24998969.9820738518068593

Look at all those extra decimal places! It ran 80 instead of 60 seconds, but that’s impressively more precision.

I don’t know why I didn’t think of the whitespace… it seems so obvious now that you’ve pointed it out. It probably didn’t occur to me because Lucee 5 and 6 for some reason handle it very differently.

Thanks again!

[ToroMaduro]

Well I feel really dumb now. Turns out I had the admin’s “Output” category’s “Whitespace management” checkbox set to “Smart whitespace management” in Lucee 5, but when I downloaded Lucee 6 I forgot to select that setting and it defaulted to “No whitespace management”. It was the whitespace crushing the heap in Lucee 6.

When I checked “Smart whitespace management” in Lucee 6, it began behaving pretty much just like Lucee 5 in all respects.

Sorry for wasting everyone’s time. It was just the whitespace management setting being different between my 5 and 6 install. That explains everything.

Sorry folks! Thanks for the help in figuring it out, though! Anyone doing huge numbers of calculations, use cfscript, cfsilent, and/or check the whitespace management admin setting! That’s the takeaway message.

[carehart]

Good to see this resolved…and it’s happened before to people, whether using Lucee or CF. And the takeaway I’d assert is “question carefully whether problems in a new version are really caused by that change alone”.

That was the point I was making back on Aug 19 here. I do lament I stopped one short, though it may help some to state it now:

We should also compare the admin settings between a working and a troubled instance, as there may be some unexpected difference, whether due to changes made in one but not the other, or perhaps even a change in defaults with a new version.

And that’s easily done with Commandbox cfconfig–which can be used even for a lucee or cf instance that’s NOT running under Commandbox. Indeed both instances need not be running at all. And cf2021 and up offers a similar cfsetup cli tool–though of course it works only with CF and isn’t quite as powerful. Some know these things, but not “everyone”. And FWIW I’ve got a talk covering more on both tools, available as a pdf or video.)

Just trying to “make lemonade” out of this situation. And Toro, who knows: it may help you spot still other potentially useful differences.

[TonyMonast]

I’m glad to read that it was just that and not a bug with Lucee’s engine. That must be a relief for you. Not everyone would think of whitespaces, this was very well seen by gaia-jyh. Thanks to him.

This discussion was not a waste of time. If I, or someone else, has a similar problem, this topic will help.