Cookie encoding/decoding broken

It seems 5.3.9 (RC1 & 2, last test on 108-RC) breaks cookies for some less than straightforward values. At some point HTML entity encoding has been introduced, which due to the semicolon will never work. This seems to be the case for scope-assigned values and the cfcookie tag.
In some way this breaks decoding as well, and cookies following the problematic value will not be available.

The way cfcookie encodeValue seems to work is also surprising…

Am I missing something, or is cookie encoding/decoding really broken?

Here’s a test case, run once for inspection of cookies getting set, once more to see the result of the parsing/decoding

	try {
	catch(any ex){}

	cfcookie(name="test_tag1", value=serializeJSON({"test_tag1":"somevar",rules:[]}));
	cfcookie(name="test_tag2", value="Plain ASCII text: no problem");
	cfcookie(name="test_tag3", value="Semicolon in ASCII text; problem!");
	cfcookie(name="test_tag4", value='Quotes in ASCII text "encoded"');
	cfcookie(name="test_tag5", value=serializeJSON({"test_tag5":"somevar",rules:[]}), encodevalue=false);
	cfcookie(name="test_tag6", value="Plain ASCII text: no problem", encodevalue=false);
	cfcookie(name="test_tag7", value="Semicolon in ASCII text; problem!", encodevalue=false);
	cfcookie(name="test_tag8", value='Quotes in ASCII text "encoded"', encodevalue=false);
	cfcookie(name="test_tag9", value=serializeJSON({"test_tag9":"somevar",rules:[]}), encodevalue=true);
	cfcookie(name="test_tag10", value="Plain ASCII text: no problem", encodevalue=true);
	cfcookie(name="test_tag11", value="Semicolon in ASCII text; problem!", encodevalue=true);
	cfcookie(name="test_tag12", value='Quotes in ASCII text "encoded"', encodevalue=true);
	cookie["test_str1"] = serializeJSON({"test_str1":"somevar",rules:[]});
	cookie["test_str2"] = "Plain ASCII text: no problem";
	cookie["test_str3"] = "Semicolon in ASCII text; problem!";
	cookie["test_str4"] = 'Quotes in ASCII text "encoded"';
	cfheader(name="Set-Cookie", value="test_cfh1=" & serializeJSON({"test_cfh1":"somevar",rules:[]}));
	cfheader(name="Set-Cookie", value="test_cfh2=Plain ASCII text: no problem");
	cfheader(name="Set-Cookie", value="test_cfh3=Semicolon in ASCII text; problem!");
	cfheader(name="Set-Cookie", value='test_cfh4=Quotes in ASCII text "encoded"');


Received by client (dev tools, Firefox & Chrome on Windows)

Set-Cookie: TEST_TAG1={&quot;test_tag1&quot;:&quot;somevar&quot;,&quot;RULES&quot;:[]};Path=/
Set-Cookie: TEST_TAG2=Plain ASCII text: no problem;Path=/
Set-Cookie: TEST_TAG3=Semicolon in ASCII text; problem!;Path=/
Set-Cookie: TEST_TAG4=Quotes in ASCII text &quot;encoded&quot;;Path=/
Set-Cookie: TEST_TAG5=%7B%22test_tag5%22%3A%22somevar%22%2C%22RULES%22%3A%5B%5D%7D;Path=/
Set-Cookie: TEST_TAG6=Plain%20ASCII%20text%3A%20no%20problem;Path=/
Set-Cookie: TEST_TAG7=Semicolon%20in%20ASCII%20text%3B%20problem%21;Path=/
Set-Cookie: TEST_TAG8=Quotes%20in%20ASCII%20text%20%22encoded%22;Path=/
Set-Cookie: TEST_TAG9={&quot;test_tag9&quot;:&quot;somevar&quot;,&quot;RULES&quot;:[]};Path=/
Set-Cookie: TEST_TAG10=Plain ASCII text: no problem;Path=/
Set-Cookie: TEST_TAG11=Semicolon in ASCII text; problem!;Path=/
Set-Cookie: TEST_TAG12=Quotes in ASCII text &quot;encoded&quot;;Path=/
Set-Cookie: TEST_STR1={&quot;test_str1&quot;:&quot;somevar&quot;,&quot;RULES&quot;:[]};Path=/
Set-Cookie: TEST_STR2=Plain ASCII text: no problem;Path=/
Set-Cookie: TEST_STR3=Semicolon in ASCII text; problem!;Path=/
Set-Cookie: TEST_STR4=Quotes in ASCII text &quot;encoded&quot;;Path=/
Set-Cookie: test_ctx1={"RULES":[],"test_ctx1":"somevar"}
Set-Cookie: test_ctx2=Plain ASCII text: no problem
Set-Cookie: test_ctx3=Semicolon in ASCII text; problem!
Set-Cookie: test_ctx4=Quotes in ASCII text "encoded"
Set-Cookie: test_cfh1={"test_cfh1":"somevar","RULES":[]}
Set-Cookie: test_cfh2=Plain ASCII text: no problem
Set-Cookie: test_cfh3=Semicolon in ASCII text; problem!
Set-Cookie: test_cfh4=Quotes in ASCII text "encoded"

Provided to server as seen in getHTTPRequestData() (newlines in Cookie-header value mine)

test_ctx2=Plain ASCII text: no problem; 
test_ctx3=Semicolon in ASCII text; 
test_ctx4=Quotes in ASCII text "encoded"; 
test_cfh2=Plain ASCII text: no problem; 
test_cfh3=Semicolon in ASCII text; 
test_cfh4=Quotes in ASCII text "encoded"; 
TEST_TAG2=Plain ASCII text: no problem; 
TEST_TAG3=Semicolon in ASCII text; 
TEST_TAG4=Quotes in ASCII text &quot; 
TEST_TAG9={&quot; TEST_TAG10=Plain ASCII text: no problem; 
TEST_TAG11=Semicolon in ASCII text; 
TEST_TAG12=Quotes in ASCII text &quot; 
TEST_STR1={&quot; TEST_STR2=Plain ASCII text: no problem; 
TEST_STR3=Semicolon in ASCII text; 
TEST_STR4=Quotes in ASCII text &quot

Provided to CF as seen in dump

test_str1:	{&quot
test_tag1:	{&quot
test_tag5:	{"test_tag5":"somevar","RULES":[]}
test_tag6:	Plain ASCII text: no problem
test_tag7:	Semicolon in ASCII text; problem!
test_tag8:	Quotes in ASCII text "encoded"
test_tag9:	{&quot

(Using JDK 11 on Linux el7)

looks mostly likely due to [LDEV-3721] - Lucee in

can you file a regression ticket and link back to it?

Filed as [LDEV-3911] 5.3.9 regression, cookie encoding/decoding problems