Bundled PDF extension silently breaks ALL outbound TLS on Lucee 7 + Java 21 (BouncyCastle 1.38 registered as a JVM security provider)

dev
1

I want to flag something that cost us two production incidents before we pinned it down, because the failure signature looks nothing like its cause and I suspect others will hit it as Lucee 7 adoption grows.

TL;DR: the PDF extension that ships bundled with Lucee 7 registers its BouncyCastle 1.38 (2007) as a JVM security provider at engine boot, ahead of the JVM’s own providers. On Java 21 its ancient EC/ECDH implementations then poison every outbound TLS handshake the JVM attempts - JDBC with TLS, cfhttp, SMTP, everything - while the server otherwise appears healthy.

Environment

  • Lucee 7.0.4+34 (also reproduced on an earlier 7.0.x), CommandBox 6.3.3
  • Ubuntu 24.04, OpenJDK 21.0.11
  • PDF extension 2.0.1.0 as bundled with the engine (we never installed it explicitly and never call any of its functions - all our PDFs come from wkhtmltopdf)
  • MySQL 8 (AWS RDS) over JDBC with TLS on (Connector/J default sslMode=PREFERRED)

Symptoms

From the moment the engine finishes booting:

  • Every datasource query: Communications link failure - The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
  • Every cfhttp to an https URL: Connection Failure, and with errorDetail exposed: Could not generate ECDH keypair / EC Key Pair Generator not initialised
  • Anything else that leaves the JVM over TLS fails silently the same way.

Nothing in the symptoms points at PDFs. If you go looking for “Communications link failure” you find a decade of threads with a dozen different causes (expired CAs, driver mismatches, keystores, TLS version mismatches) - none of them this.

Diagnosis

  • createObject("java", "java.security.Security").getProviders() showed BC 1.38 sitting FIRST in the provider list, ahead of SunEC / SunJSSE. Lucee core itself carries a modern BC that is perfectly fine - the 1.38 comes from the PDF extension’s bundles (bouncycastle.prov/mail/tsp-1.38.0.jar, alongside icepdf.core-3.1.0.3, org.lucee.pdfbox-3.0.0.RC101 and flyingSaucer).
  • A binary grep across every bundle jar for insertProviderAt matched only icepdf.core-3.1.0.3.jar. The old IceSoft-era IcePDF registers BouncyCastle as a JVM-wide provider for encrypted-PDF support. (The modern IcePDF fork made the provider configurable via the org.icepdf.core.pobjects.security.provider system property - effectively acknowledging that hard-registering it was a problem. Apache PDFBox has the same theme in PDFBOX-2963, “Remove Bouncy Castle Provider Reference”.)
  • BC 1.38 predates most of the named curves and provider plumbing modern JSSE relies on, so once it answers first for EC/ECDH, every handshake dies.
  • Registering a good provider from onApplicationStart does NOT help: the extension registers at engine boot, before any application code, and JSSE caches the poisoned state.

Proof by natural experiment

Removing the extension (the .lex, its installed folder, the icepdf / bouncycastle 1.38 / pdfbox / flyingSaucer bundle jars, and its .CFConfig.json extensions entry), then restarting: 60/60 concurrent TLS JDBC connections clean, cfhttp fine, TLS everywhere restored, with nothing else changed.

Two days later a CommandBox restart resolved our then-floating lucee@7 engine pin to a new patch release; the fresh engine re-deployed its bundled extensions, the PDF extension came back, and the identical failure returned within two minutes of boot. Stripping it again fixed it again. Cause, effect, cause, effect.

Workaround (what we run now)

We pin the exact engine version in server.json so an engine redeploy never happens implicitly, and run an idempotent strip after any engine start:

rm -f  <server-home>/WEB-INF/lucee-server/context/extensions/available/66E312DD-D083-27C0-64189D16753FD6F0*.lex
rm -rf <server-home>/WEB-INF/lucee-server/context/extensions/installed/66E312DD-D083-27C0-64189D16753FD6F0*
rm -f  <server-home>/WEB-INF/lucee-server/bundles/icepdf.core-*.jar \
       <server-home>/WEB-INF/lucee-server/bundles/bouncycastle.*-1.38.0.jar \
       <server-home>/WEB-INF/lucee-server/bundles/org.lucee.pdf*.jar \
       <server-home>/WEB-INF/lucee-server/bundles/org.lucee.flyingSaucerPDF-*.jar \
       <server-home>/WEB-INF/lucee-server/bundles/org.lucee.pdfbox*.jar
# plus remove the 66E312DD-... entry from .CFConfig.json's extensions array

Obviously that only suits apps that do not use cfdocument/cfpdf.

2 Likes
2

@Gary_Stanton , I tried to reproduce this on a clean environment with Lucee 7.0.4.34, PDF Extension 2.0.1.0, and Java 21, but couldn’t reproduce the issue.

I verified that the legacy bouncycastle.*-1.38.0.jar and icepdf.core-3.1.0.3.jar are present, however:

  • Security.getProviders() only shows the default JVM providers (SUN, SunEC, SunJSSE, etc.), with no BC provider registered.
  • Security.getProvider("BC") returns null.
  • cfhttp over HTTPS works correctly.
  • KeyPairGenerator.getInstance("EC") uses SunEC, and SSLContext.getInstance("TLS") uses SunJSSE.
  • Running cfdocument does not change the provider list.

Could you share the output of Security.getProviders() from the affected environment and confirm whether BC appears immediately after startup? Also, are there any custom JVM security settings or additional extensions installed? That would help us narrow down the difference.

3

Apologies - it does in fact seem that we were registering BC manually in the application, so you wouldn’t see this occur on a stock install.

The push library needs BouncyCastle as the first security provider (the JDK’s own EC factory rejects its key specs). So we’re running:

bc = createObject( "java", "org.bouncycastle.jce.provider.BouncyCastleProvider" ).init();
security.removeProvider( "BC" );
security.insertProviderAt( bc, 1 );

The problem is the plain class name. Lucee resolves the name from whatever bundle exports the package - and with the PDF extension deployed, that is bouncycastle.prov 1.38.0 (2007). The same line then puts a 2007 BC first in the JVM, and its broken EC kills every outbound TLS handshake.

Stripping the extension “fixed” it because the name resolved to a modern BC again. On our boxes BC appeared seconds after boot because the first request (the load balancer’s health check) runs Application.cfc - no human involved, which is what made it look like an engine-boot thing.

The information you asked for:

Security.getProviders() on the affected production box (Ubuntu 24.04, OpenJDK 21.0.11, Lucee 7.0.4.34, CommandBox 6.3.3, currently healthy with the extension stripped):

1:  BC 1.7        [org.bouncycastle.jce.provider.BouncyCastleProvider @ file:/opt/xxxxxx/.commandbox/cfml/libExt/runwar-jakarta-6.1.5.jar]
2:  SUN 21        [sun.security.provider.Sun]
3:  SunRsaSign 21 [sun.security.rsa.SunRsaSign]
4:  SunEC 21      [sun.security.ec.SunEC @ jrt:/jdk.crypto.ec]
5:  SunJSSE 21    [sun.security.ssl.SunJSSE]
6:  SunJCE 21     [com.sun.crypto.provider.SunJCE]
7:  SunJGSS 21    [sun.security.jgss.SunProvider @ jrt:/java.security.jgss]
8:  SunSASL 21    [com.sun.security.sasl.Provider]
9:  XMLDSig 21    [org.jcp.xml.dsig.internal.dom.XMLDSigRI @ jrt:/java.xml.crypto]
10: SunPCSC 21    [sun.security.smartcardio.SunPCSC @ jrt:/java.smartcardio]
11: JdkLDAP 21    [sun.security.provider.certpath.ldap.JdkLDAP]
12: JdkSASL 21    [com.sun.security.sasl.gsskerb.JdkSASL @ jrt:/jdk.security.jgss]
13: SunPKCS11 21  [sun.security.pkcs11.SunPKCS11 @ jrt:/jdk.crypto.cryptoki]

Custom JVM security settings: none.
Other extensions: MySQL JDBC 9.6.0, MSSQL 13.2.1, PostgreSQL 42.7.9, jTDS 1.3.1,
Lucee Administrator 1.0.0.7, Lucee Documentation 1.0.0.6, S3 Resource 2.0.3.1, Image
3.0.1.1, Guard 3.0.0.14, Scheduler Classic 1.0.0.1, Compress 2.1.0.2-SNAPSHOT, Redis 4.0.0.0.