Myth busting Boxlang benchmark claim

news
1

The Boxlang team posted this graphic today claiming BoxLang was the fastest

image
image1080×606 73.6 KB

You’ll note they are comparing against Lucee 5 which struck me as a little bit odd, apparently there are compatibility issues with their test suite with Lucee 6, so they omitted Lucee 6.2 from the graphic!

Ortus has just confirmed on slack, that in their testing, Lucee 6.2 is indeed faster than BoxLang.

The Lucee team welcomes competition in the cfml engine performance space, I think this is a little misleading, hence my clarification here :slight_smile:

11 Likes
2

They are a company, out to make money, sad they have to use misleading ads.

From a pure marketing standpoint they should only compare themselves to AFC, I could write volumes upon this, but then I would want a hefty consulting fee :wink:

4 Likes
3

I’d have to say "nay to this, Terry. I’ve known Luis for… almost 20 years? He (and Brad and the rest of the team), I always say that they are single-handedly responsible for keeping CF alive. There is a lot of altruism to what they do, but they also have staff with mouths to feed. I’ll never begrudge someone making money. And I don’t think they set out to mislead anyone.

4 Likes
4

I think you may have misunderstood my point.

They are a company, I want them to succeed, charge people money, feed themselves, continue their charity work (this is really cool of Luis / Ortus). From a marketing standpoint, their info-graphic is not good.

5

Firstly, Luis and the whole Ortus Team are all lovely people, he even offered to update the graphic.

oddly enough, I updated my fork and ran the tests

Turns out currently, it’s BoxLang that was failing, Lucee 6.2.1 does indeed address that regression and Lucee 6 indeed way faster than BoxLang, coming in at 2m31s vs 3m32s

image
image1063×696 63.1 KB

That said, we feel Lucee is still a bit in slow in some places and will be making it even faster with 8!

4 Likes
6

:sparkles: Team Lucee :sparkles:
:checkered_flag:_:racing_car:______

7

Just out of curiosity, Zac, are the benchmarks they and you are running the same ones? And is either avaliable for us to try as well? (Sorry if that should have an obvious answer.)

Also, what does the red x mean in some of the results you show, especially for that quoted boxlang duration…and for the one test showing to be far faster than the rest. Is it that second one (or are both) invalid somehow?

8

Red means the CI test run errored, so yeah invalid

Green means the step/tests completed without any problems

Lucee was indeed previously red with 6.2.0.231, but 6.2.1 passes

You can try yourself, just fork the repo and enable actions for your fork

GitHub actions are a bit variable in performance

All I did was updating my fork, syncing to upstream

1 Like
9

Great, thanks.

10

Oh I’m loving Lucee having competition again!

Sharpen your swords, just don’t cut off your face to spite your face.

Time to put an extra bit of money in the Lucee tip jar, can’t really afford the boxlang ‘Adobe’ licencing model (great for Lucee)… pitty they have some great features I’d use, but using the Adobe ColdFusion, Sencha Extjs pricing model has decades experience of being a waste of money, and small-mid shop developer resentment… Each to their own. Sometimes people dont learn.

2 Likes
11

Please feel free to file posts here under language proposals anything you think we should consider adopting

2 Likes
12

Have submitted some idea for changes already,

My list of updates is long, do we have the time, here’s a few things #lol

From 25+ years of cfml, I’d finally like to see - language / proposals - Lucee Dev

13

It’s certainly admirable/important to encourage Lucee users to consider funding/contributing more to it.

But about boxlang, you say you “can’t really afford the boxlang ‘Adobe’ licencing model”. Um, it’s free and open source–even for production use. While there are paid plans offering support and extended deployment capabilities, it’s as free as Lucee.

I realize my saying that here may rub some the wrong way. I don’t make the observation to pick a fight, nor do I say it to encourage anyone’s movement. As should be clear to folks from my years here, I try to “walk the fence among the engines”. It’s easy for folks in one of the “fields” to stop paying attention to what’s going on in “the other side of the fence”, so I try to help clarify things when I can (and I don’t even press the point often, as it would annoy those who just “don’t care” about any other cfml engine).

But since you say “I’m loving Lucee having competition again”, I’ll note that acf had support of Java 17, tomcat 10/11, and even the ability to run Java code inside cfml pages before Lucee did. Of course, I don’t presume the Lucee team did those things BECAUSE Adobe did. (And the implementation of running Java code in cfml is entirely different between the engines.) They anticipated what they felt the community would want.

And they do sometimes add things that the acf team does first (for compatibility), just as the acf team may add things Lucee does first (either seeing the good idea or just filling the same need they see): such things go in both directions.

And now we have a 3rd engine, also proclaiming general compatibility of cfml. And their innovations may in time be considered by the other two engines, and vice-versa. And we will have new incompatibilities among each, for sure. I suspect many aren’t yet seeing boxlang’s cfml compatibility as a legitimate competitor for acf or Lucee. Time will tell how things shake out.

I just thought some context was warranted given your statements about competition and “cost” for boxlang. As always, I’m just trying to help. (I realize some won’t see it that way despite the disclaimer. Welcome to my life.)

3 Likes
15

I’m a little confused, you went a bit everywhere there Charlie.

On Price:
Boxlang is not free, and at this stage it is unclear which extensions are ‘premium’ in the future.

That said I’d have to invest in licences for 200 instances, in which they would be used for 5 minutes minimum, possibly never depending on pressure on the server cluster.

So yeah, it’s right up there with ACF, ridiculous for SaaS scenarios.

On (my take which means nothing) on Ortus and cfml:
I’m sure Ortus will be invested in cfml for the near future, or until all their products are moved to java or bx platform at least. I can’t see the investment in the cfml ecosystem being thrown away sometime soon, but you cant say never also. It’s not for me to speak on their behalf.

On my business (and the TeamCFML framework @ v52)
Let’s keep it simple, lucee is the only cfml engine I trust for my business to succeed right now. That’s not a debate, it’s a cold hard truth. Our framework is also exclusively lucee for now, but looking at expanding to boxlang. (zero acf support again, that ship has sailed, dont get me wrong ACF is a premium product with some great unique features, just prohibitively expensive) )

I love boxlang is here, it stirs up the community and opens minds to ‘what should be in a product’. Like i did with ACF, people will go to the market and see what product fits on multiple levels.

I’d love to see every lucee cfml dev put $10 a month in the kitty, it’s nothing compared to what you earn and keeps innovation going. Many do, but if we ALL did, then we’d have more money to pay more internal devs.

In 2025 there should be literally hundreds of extensions for lucee. Time me and the rest of us got our act together and delivered back more often.

16

BoxLang is free with the option to subscribe to premium modules/services/support. They have an entire “plans” page showing a matrix of exactly what you get with what. If something isn’t clear, go ask them.

You can spin up a server and hit the ground running just like with Lucee and have a fully functional application at no extra cost beyond infrastructure and time.

But that seems neither here nor there because this is a Lucee forum. The bias is understandable.

Are we myth busting in this thread at this point, or platform bashing. If it doesn’t have what you want, don’t use it.

17

yeah, let’s keep it all nice and respectful

18

We’re clearly busting a myth that dawesi held, in his thinking “Boxlang is not free”. Thanks for stepping in, Tony.

I hope that this clarification might change your perspective, dawesi, in feeling my reply was “all over the place”. It might also lead you to have a different perspective on the next reply you offered. But if not, no worries. I said I wasn’t trying to debate things, just sharing what I felt might be helpful clarifications. Seems I was right, but if you still disagree, we can let it go. The points have been made, hopefully to SOMEONE’S benefit.

2 Likes
19

Even the title of that marketing graphic, “BoxLang vs CFML”, is misleading, because BoxLang is CFML. All they did fundamentally was replace “cf” with “bx:”.

And this from their FAQ: “BoxLang combines many features from different programming languages, including Java, ColdFusion, Python, Ruby, Go, and PHP, to provide developers with a modern, fluent and expressive syntax. BoxLang has been designed to be a highly modular and dynamic language to take advantage of all the modern features of the JVM.”

They make it sound like a brand new language. :joy: It’s like they’re trying to distance themselves from CFML while in fact being rooted in it.

Oh and their FAQ also claims “60% lower memory and disk usage compared to Lucee”. :roll_eyes:

20

The confusion (understandable) stems from the fact that boxlang is BOTH a new language (if you want to do new development with it) AND it supports cfml (if you want to use it to run your existing apps). It is also BOTH free (free to use, in prod, for as many instances and machines as you want) AND it’s got commercial plans. As I hinted at earlier, it may well take time for people to clearly understand even these fundamentals, let alone to see how things go for those who try using it for their cfml apps.

Look, folks, I’m not the boxlang evangelist. And I certainly am not here speaking of it to denigrate Lucee. I’m just responding to the statements of others, with what I sense to be needed objective corrections.

I appreciate this is a touchy subject, just like when I might speak on behalf of acf here. I’m not anyone’s evangelist. Again I’m a fence-rider, and I try to help others from that perspective.

Finally, if anyone here IS interested to get clarification on some of these fundamental boxlang points, I just saw news that they’re offering a live one hour webinar tomorrow (Wed, May 14) at 11a us central time, at https://www.youtube.com/live/G2r-nZ0m3po . See the description there for more details, and bring your questions and concerns to them.

1 Like
21

BoxLang is not CFML. It is a brand new language, with its own dedicated parser for its own syntax. However, it does have a separate parser that supports running CFML syntax. I assure you, when writing pure BL, you can do things we’ll probably never see in CFML (and that’s ok!). The Java interop is superb and capable of things no CFML engine is doing, as an example (no shade on Lucee as they have their own features).

It’s no surprise a lot of what you see in the language is indeed inspired by CFML. There’s a lot to love from it. It’s why we all use it! Including Ortus. But to say all they did was replace “cf” with “bx” is such an ignorant take and the very point I tried to make before about some of the replies in this thread. I get it, the info graphic was misleading because it called out Lucee 5 and not 6. Zac cleared this up with the Ortus folks. I fully support calling out information that doesn’t reflect reality. If anything, a little perf competition is good as it shows there’s always more to be improved upon from everyone.

This is a Lucee forum, I expect everyone here to come to its defense, but put away the torches and pitchforks. BoxLang is gonna BoxLang and Lucee is gonna Lucee. They both have CFML support. They’re gonna have their strengths and weaknesses.

I’m not here to be the BoxLang cheerleader or to change anyones mind. Do what you gotta do, use what gets the job done. I write CFML every day, and I’ve also been getting a lot of exposure to BoxLang since its public release. It’s a wonderful addition to the CFML world and to CF devs looking to stretch into something different, yet familiar.

It feels unfair for it to get picked apart in this forum, but to each their own.

1 Like