Announcing Lucee 5.3.3.62 (final) and 5.3.4.54 (release candidate)

#1

Keep-Calm-Its-the-Silly-Season

Merriam-Webster defines “silly season” as “a period (such as late summer) marked by frivolous, outlandish, or illogical activity or behavior.” Well, we here at Lucee must be out of touch with popular trends, because the past few months have been the exact opposite of silly season! It’s been a dead-serious development push (busiest stretch ever, as I described in the Mid-Year Development Update and other posts). The result is two shipments of Lucee today–a final build (5.3.3.62) and a new release candidate (5.3.4.54). Read on for the deets!

First, here is the final list of tickets covered by 5.3.3.62:

Ticket Summary
LDEV-2385 make all uses of ReferenceMap thread safe
LDEV-2359 CLONE - percentage of total sql execution time is always 0%
LDEV-2346 structEach() on arguments scope sends incorrect values to callback function
LDEV-2340 application component mappings ignored to resolve relative paths
LDEV-2318 Looping a query: concurrency issue with internal call to get current row
LDEV-2302 Massive performance degradation when using Java > 1.8
LDEV-2297 all form fields treated as file uploads with multipart/form-data
LDEV-2265 " Self-attach to current VM error " using Lucee 5.3 (Docker image Lucee-nginx) Java 11
LDEV-2264 warn about incompatible extensions when downgrading
LDEV-2263 NPE in debugging when a query has no source
LDEV-2262 improve debugging execution time readability
LDEV-2261 percentage of total sql execution time is always 0%
LDEV-2260 Queryparam will first check maxLength and only after convert the value to the given type
LDEV-2258 Image functions not working in latest Lucee 5.3.1.103
LDEV-2257 Regression from LDEV-2247 fix
LDEV-2253 xmlParse doesn’t work and then kills Lucee server instance
LDEV-2249 modern debugging template, only load the 1.1mb of echarts javascript when the metrics are shown
LDEV-2247 Client cookies are not marked as secure and httpOnly
LDEV-2240 Regression: felix log growing
LDEV-2239 modern debugging template filter reset doesn’t work
LDEV-2238 duplicate stacktraces in modern debugging output
LDEV-2236 cfquery returnType Array return NULL whereas returnType Query return empty string with Partially NULL Support
LDEV-2229 Deleting a debugging template doesn’t remove it from Lucee-server.xml
LDEV-2227 Parse bundled cfml files as part of the build process
LDEV-2226 improve the missing attribute error message
LDEV-2223 CFZIP: wrong password leaves the zip file locked
LDEV-2221 Unknown XML parsing error after upgrade from 5.2 to 5.3 (CommandBox)
LDEV-2215 lucee’s bundled felix version does not support jrt protocol
LDEV-2204 StructEach is not accessing key-value accurately for arguments
LDEV-2202 key [ACTION2] doesn’t exist
LDEV-2187 snapshots are being suggested as patches for stable releases on overview page
LDEV-2180 Lucee handles files uploads incorrectly when Content-Type is missing
LDEV-2174 ACF Compatibility for this.blockedExtForFileUpload in Application.cfc
LDEV-2173 XmlSearch Performance issues
LDEV-2170 <cfmail with option Folder= not working
LDEV-2169 <cfimap action=“MoveEmail” makes a copy and not move
LDEV-2159 cffile write nameconflict skip - writes an empty file
LDEV-2148 parseDateTime with format argument subtracts one year
LDEV-2142 QueryExecute debugging entries are not consistent with cfquery
LDEV-2102 Invalid call of the function ImageResize, fifth Argument (blurFactor) is invalid, argument blurFactor must be between 0 and 10
LDEV-2100 LuceeAjax.js does not bind empty values
LDEV-2098 cfapplication action=update doesn’t work for javaSettings
LDEV-2007 cfloop doesn’t throw error while using invalid combination of attributes
LDEV-1935 Code of CFCs in componentPaths isn’t flushed by PagePoolClear() or SystemCacheClear()
LDEV-1780 Cannot create MySQL Datasource
LDEV-1723 Extensions with content type of application/octet-stream won’t install
LDEV-1663 Implicit Accessors Do Not Satisy Interface Methods
LDEV-1196 Provide a way to fully “warm up” a Lucee install for containerization

Next, here’s the ticket list so far for the 5.3.4.54 release candidate:

Ticket Summary
LDEV-2437 log spam “com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl is used as DocumentBuilderFactory”
LDEV-2433 Admin blows up when an extension has an invalid icon image and makes ALL extensions unusable
LDEV-2405 Flying Saucer PDF extension leaks debugging lines out to the console
LDEV-2401 LSParseNumber thread unsafe?
LDEV-2385 make all uses of ReferenceMap thread safe
LDEV-2328 Regression: Can’t resize image proportionally in ImageResize
LDEV-2310 Valid base64 string doesn’t convert into image
LDEV-2307 Downgrade version(base) doesn’t exist after upgrade into latest version
LDEV-2304 SerializeJSON not preserving case on query
LDEV-2301 server update crashes when lucee can’t connect to internet or lucee webservices down
LDEV-2290 application log has WARN entries when LOG level set to ERROR
LDEV-2274 undefined tooltips for metrics graphs
LDEV-2273 Replace by struct multiplies when value contains key
LDEV-2270 sameFormFieldsAsArray converts all fields to arrays
LDEV-2150 cfthread object empty inside of a thread
LDEV-2128 Query of Queries UNION returns incorrect results with cfqueryparam
LDEV-2116 executable bit is lost on files when using directoryCopy() on binaries
LDEV-2113 xmlValidate Cannot find the declaration of element
LDEV-2108 SerializeJSON argument error even when supplied argument is a boolean
LDEV-2097 CFQuery Lazy=“true”, i.e. SimpleQuery is able to consume infinite heap memory during request
LDEV-2096 ArrayNew is not default synchronized=true and missing typed arrays documentation
LDEV-2091 CFThread after join logs implicit error that is not fixable
LDEV-2085 add possibility to define a directory containing OSGi bundles with this.javaSettings
LDEV-2035 Event Gateway - Add a Gateway Type to Handle Asynchronous events through CFCs
LDEV-1939 Nested cftry in cfcatch duplicates error logs
LDEV-1933 CFMAIL can’t connect to TLSv1.2 smtp server
LDEV-1850 PDF: cfpdf - remove password from pdf-file
LDEV-1676 Expose XML Parser Configuration to prevent XXE
LDEV-926 Stop allowing unauthenticated people set default passwords for web admins
LDEV-412 CSRF functions are restricted to CF Sessions

Please head over to the downloads site and grab one/both of those, and thanks in advance for any and all testing/feedback.

Next, returning to the topic of the remaining development schedule for 2019, we’ve come up with the following plan:

  1. At least 1 more final release (5.3.4).
  2. At least 1 more release candidate (5.3.5-ish).
  3. Catch-up on any lingering bugs/problems, in particular those related to security, or those that are blockers to anyone wanting to upgrade to the latest production release of Lucee, along with catch-up on pull requests (hears a loud “huzzah!” from the community on this one!).
  4. Progress on Lucee 6 (though not necessarily a beta or even an alpha).

Regarding #1, we’re now in the RC period for 5.3.4, so we’ll make that final depending on how regression testing/community feedback go. Regarding #2, we’ll decide when the next monthly sprint will take place based on how things go with numbers 1, 3 and 4. #3 may push #2 later on the calendar, if we find that we need to make a substantial development push to get caught up/stable/etc. We’ll have more info about #4 as we get closer to CFCamp. We’ll definitely be previewing at least some Lucee 6 goodness at CFCamp, and based on how far we get with that effort, we’ll then decide on the schedule for a formal alpha/beta/release candidate plan for Lucee 6.

Next, a bit more about #3 above. Late last week, and into this week, we put in a really substantial triage effort that covered over 200 tickets, almost all of which were created in recent months. So, if a ticket (or tickets) important to you isn’t baked into either this final release or the RC, then please take a look at Jira to see if there’s an update to your open ticket(s). Again, we’re going to plan the rest of this year’s development efforts around finishing the year strong, and making sure we’re caught up on especially critical items, so comment/upvote, etc., and we’ll be ready to react.

As always, holler with any and all questions/comments/commentary, and thanks for listening!

Best,
Patrick

7 Likes
5.3.4.54 RC Released
#2

The stable release and the RC (to promote testing) have been added to ForgeBox and can be used in CommandBox or the Ortus Docker images as of yesterday.

start cfengine=lucee@5.3.3+62
start cfengine=lucee@5.3.4-rc+54

This morning, I also worked with Micha to create a new functionality on the Lucee update provider feature generates CommandBox CF engines on the fly so it can serve up Lucee Light engines (no extensions) as well if you’d like to try running your site on them. I may start putting these on ForgeBox, but for now, you can start a server using the Light versions like so

start cfengine=http://update.lucee.org/rest/update/provider/forgebox/5.3.3.62?light=true
start cfengine=http://update.lucee.org/rest/update/provider/forgebox/5.3.4.54-rc?light=true

Notice the ?light=true on the end of the URL. That’s the new bit :slight_smile: You can choose exactly what extensions you want to “layer on” to your Light server with JVM args. This example would start up a Light server, but add in the Admin and MySQL JDBC driver.

server set app.cfengine="http://update.lucee.org/rest/update/provider/forgebox/5.3.3.62?light=true"
server set jvm.args="-Dlucee.extensions=CED6227E-0F49-6367-A68D21AACA6B07E8,7E673D15-D87C-41A6-8B5F1956528C605F"
server start
5 Likes
#3

I would like to see these on ForgeBox! We already start servers using -Dlucee.extensions.install=false but I think this would be much better, as we would not have to download the unused extensions in the first place.

1 Like
#4
#5

This is great but it seems that you can’t specify a version or include a name to help identify each extension.

Apparently you can do both with Env Vars, but if I want different commandbox servers in the same OS environment to use different sets of extensions I’d need to use system properties in my individual server.json files rarther than EVs.

#6

The environment variable and system property are equivalent to each other, so you should be able to use the same value in either place to specify your extension version.

1 Like
#7

I’ve tried using the same format in my server.json jvm.args, e.g.

"-Dlucee.extensions=FAD1E8CB-4F45-4184-86359145767C29DE;name=Hibernate ORM Engine,CED6227E-0F49-6367-A68D21AACA6B07E8;name=Lucee Administrator"

but the server startup fails with the following error:

[ERROR] runwar.context: Error: Could not find or load main class ;

#8

@Julian_Halliwell If you’re going to have spaces in the arg, you need to quote it. And if you’re setting it in the server.json, remember that the standard escaping rules for storing the quotes in JSON apply as well.

Another way to accomplish the same thing is to install the CommandBox Dotenv module and place a file called .env in your web root with the contents:

lucee.extensions=FAD1E8CB-4F45-4184-86359145767C29DE;name=Hibernate ORM Engine,CED6227E-0F49-6367-A68D21AACA6B07E8;name=Lucee Administrator

Any key/values in that properties file will get added to the server for you. There’s no quotes needed there and there’s less escaping to worry about since it’s not inside of JSON. And, of course, each site can have it’s own .env file.

#9

I did try an extension without spaces in the name and also one with just the version (no spaces), but got the same error so I don’t think that’s the issue.

Sounds like the Dotenv module is the best approach though. Thanks Brad.

#10

Yes, the lack of quotes was your issue. There’s other reasons to use quotes too, the error message implies the semi colon was involved. All you have to do is run

server start --debug

and you can see the exact JVM args being used. You’ll find that a character such as a space or semicolon which wasn’t quoted broke the parsing of the args.

1 Like
#11

Thanks for all the good work!

Regressions LDEV-2277 and LDEV-2317 are two we’ve been waiting for. Several other people are waiting on these too.

We’ve been running 5.3.3.60-RC for some time and it’s been pretty stable.

Cheers,
Pat

#12

FYI: just fixed a little bug with docs, since commandbox is now running this release and was crashing

2 Likes
#13

interesting !

#14

Thanks, @pat.moody. On my watch list. Stay tuned.

1 Like
#15

Hi Brad,

There appears to be some naming conflict with 5.3.4-rc+54
This works fine for starting the server but fails to stop the sever from the command line
Looks like the stored state of the server has a different specification. i.e. 5.3.4+54-rc requiring the server be stopped from the tray icon.

Cheers
Pat

#16

@pat.moody You are correct, the version in the box,json had the build num in a different place, but that shouldn’t; affect stopping the server. More likely, you have a duplicate server with the same name and the wires are getting crossed. Run server list to verify and forget the duplicates. Also, I updated the version number but you’d need to clear the download from your artifacts to get the new version.,.

#17

Cheers Brad, something bizarre was going on there. I don’t know if it was after installing FusionReactor. I’ve made a fresh start without FR and it is back to normal. Just noticed I was still on CB 4.7 so I’ve updated that too.

#18

@pat.moody Ah, yes FR throws a monkey wrench in the works. The debugger dll library tends to get locked by Windows and they don’t release for a few seconds after the server has stopped. So the restart command sees that the ports are released, but the DLL files are still locked. I haven’t; found a way around this other than to wait a few seconds and run the “start” command again.

#19

There was a big commit (with no linked jira task ) which was meant to address concurrency issues in Lucee which seems to have broken multiple things since 5.3.3.61, this is still an issue with 5.3.4 RC

Concurrency: reflection method lookups using single sychronous map holding locks under concurrent load https://luceeserver.atlassian.net/browse/LDEV-2527

Regression: Lucee gets confused and renders wrong file (with same name)
https://luceeserver.atlassian.net/browse/LDEV-2514

Page not rendered until file saved without changes
https://luceeserver.atlassian.net/browse/LDEV-2376

These look like major regressions which will waste lots of end user (developer) time struggling with these bugs, I think these should be blockers for the 5.3.4 release.

5.3.4 is still at RC, I think these justify a code review of that commit, fixes and a new RC, otherwise, what’s the point of having RC process if major regressions don’t get addressed?

#20

LDEV-2527 just got a fix :slight_smile: