If anyone is curious about the Lucee RSA encryption implementation under the hood.
Appears to be PKCS #8, and doesn’t look there is a way to override to change the cipher. Which would be a nice feature for interoperability with other systems/languages for example PKCS1 or OAEP.
Due to server to server interoperability requirements I was not able to rely on Lucee’s RSA encryption implementation (PKCS 8). Being able to specify the cipher for key generation and encrypt/decrypt is required. The system I was interacting was expecting the encrypted value to have used the PKCS #1 v1.5 cipher algorithm.
Here is how I implemented RSA encryption with Java instead.
private any function encryptCardNumber(required string cardNumber, required string publicKey) {
var secureRandom = createObject("java", "java.security.SecureRandom").init();
var cipher = createObject("java", "javax.crypto.Cipher").getInstance("RSA/ECB/PKCS1Padding", "SunJCE"); // This also is equivalent: getInstance("RSA/None/PKCS1Padding", "BC") for BouncyCastle (org.bouncycastle.jce.provider.BouncyCastleProvider)
// Convert public key from string to java Key object
// First need to convert raw string to ByteArray
var publicKeySpec = createObject('java', 'java.security.spec.X509EncodedKeySpec').init(toBinary(arguments.publicKey));
var keyFactory = createObject('java', 'java.security.KeyFactory').getInstance('RSA');
var key = keyFactory.generatePublic(publicKeySpec);
cipher.init(createObject("java", "javax.crypto.Cipher").ENCRYPT_MODE, key, secureRandom);
return cipher.doFinal(toBinary(toBase64(arguments.cardNumber)));
}
That works fine but I had no use for generating RSA keys. And I actually have zero idea of the language of the server API I was interacting with. All I knew was that I was provided with a public key and data was to be encrypted with RSA. And that’s where the mystery began and took some trial and error to solve. I needed finer details to figure out why I was encountering interoperability issues. The API server couldn’t decrypt the ciphertext properly.
I had some implementation examples using same API with JS (JSEncrpyt), PHP (openssl-public_encrypt & Crypt_RSA), and C# (RSA.Encrypt). I ended up digging through the implementations in those languages and they discovered they use commom cipher algorithm PKCS #1 v1.5 so its apples to apples.
With Lucee encryption doesn’t allow you to change the default cipher algorithm (PKCS #8) from what I could tell in the Lucee engine souce code.
Just wanted to share in case anyone ever encounters a similar issue. If the API server language was Lucee 5.3 then there would not be any issues (unless you were working in JS, C#, PHP)
But JS rsa implementations appear to be the commom denominator. Libraries supporting only PKCS #1 v1.5
The cipher should be a parameter in Lucee which exactly match the Java options for most flexibility.
It probably falls more into feature request. Because it does function properly, just incomplete (limited) implementation in my opinion. And Lucee docs should at least mention the default cipher used (PKCS #8 I believe, I know for sure it isn’t RSA/ECB/PKCS1Padding).
And it looks different providers have different defaults to make it even more confusing.
Supposedly this is bad practice because of ambiguity.