[Solved, in a way] WebSockets / Tomcat config issue - WebSocket error 1009

Using Lucee 5.3.9.141

I have a Lucee WebSockets test app working properly with small json messages.

Now I am moving to another stage of development, using Y.js CRDT and my messages are getting a lot bigger! eg 12Kb which is not huge, of course. The messages are still pure JSON strings (the ArrayBuffer is encoded as a standard JSON array) and the JS websocket is using the default binaryType of ‘blob’ .

I am experiencing the following java WebSocket error 1009:

'The decoded text message was too big for the output buffer and the endpoint does not support partial messages'

I am somewhat out of my depth here. I think I need to set the following params as per this page:
https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/tomcat/websocket/server/WsServerContainer.html

public class WsServerContainer
extends WsWebSocketContainer
implements ServerContainer
Provides a per class loader (i.e. per web application) instance of a ServerContainer. Web application wide defaults may be configured by setting the following servlet context initialisation parameters to the desired values.
Constants.BINARY_BUFFER_SIZE_SERVLET_CONTEXT_INIT_PARAM
Constants.TEXT_BUFFER_SIZE_SERVLET_CONTEXT_INIT_PARAM

Do they relate to one of these in \usr\local\tomcat\conf\web.xml ?

  <!--   input               Input buffer size (in bytes) when reading      -->
  <!--                       resources to be served.  [2048]                -->
  <!--                                                                      -->
  <!--   output              Output buffer size (in bytes) when writing     -->
  <!--                       resources to be served.  [2048]

If so, how does one set these?
Maybe there is an obvious way to pass the constant?

Can anyone enlighten me about how to fix this issue, please?

Thanks,
Murray

It looks like I have fixed it after a deep dive into the Java.

I am setting a new text buffer size in the onOpen() of my Listener.cfc.

/**
* Handle onOpen.
*/
function onOpen(websocket, endpointConfig, sessionScope, applicationScope) output=true {
    // if (some problem) return false;  // returning false will reject the connection

    // Set an arbitrary bigger buffer. The default is 8192
    arguments.websocket.getWebsocketSession().setMaxTextMessageBufferSize(64000);

    this.notifyChannel(
        arguments.websocket,
        {
            from   : "<server>",
            message: "someone connected"
        }
    );
}

In my notifyChannel() function I do the following so I get the buffer size back in the message along with the other “metadata” - just to confirm while debugging the value that was used.

arguments.data.channel   = chanId;
arguments.data.timestamp = getTickCount();
arguments.data.bufsize = arguments.websocket.getWebsocketSession().getMaxTextMessageBufferSize();

eg from the JS console:

bufsize: 64000
channel: "bab3d6eac3bdc2e530db285f993b1f0d"
from: "YjsListener"
message: "{\"sessionId\":\"session_l78p4ik5\",\"topic\":\"connect\",\"content\":{}}"
timestamp: 1661410852887

See: https://github.com/isapir/lucee-websocket/wiki/WebSocket-API#getwebsocketsession
and
https://github.com/isapir/lucee-websocket/blob/b45cc44e9e1c9efa007cd277b9e7005ef4ab167e/src/main/java/net/twentyonesolutions/lucee/websocket/WebSocket.java line 187.

Onwards…

Murray

3 Likes

A bit more info, relevant where the messages are quite large, as happens with CRDTs.

The solution outlined above does seem to solve the issue, however…

In my test app the Y.js data in the message is encoded into a JSON array of byte values from a javascript Uin8Array. To test the message size limits I made an arbitrary array of bytes and sent the websocket message. I kept incrementing the size of the array being sent until Tomcat threw the “decoded text message was too big” error 1009.

As a “rule of thumb” the Tomcat text buffer size needs to be about 3 times the message size. eg with a buffer set to 128000, the largest message I can send is about 42000. Other combinations of buffer size and message size seem to hold true for that rule of thumb.

Of course, if you make the text buffer size too big you need to increase the Java heap size accordingly to avoid a java.lang.OutOfMemoryError: Java heap space error.

Just FYI.
Murray

2 Likes