Lucee Websockets Extension

[isapir]

Version 1.1.3.1 just released. Added the following methods to the WebSocket API:

close(code, description)

getChannels()

getPathParameters()

getQueryString()

getWebsocketSession()

[isapir]

I have published an updated tutorial on Lucee WebSockets - Getting Started:

Lucee Websockets - Getting Started

[Cliff_Mosdall]

I have installed the Websocket in Lucee 5.2.1.9 and tried the code from here:

Example Echo

When I tried the Websocket the browser says:
“WebSocket connection to ‘ws://localhost:8888/ws/echo’ failed: Error during WebSocket handshake: Unexpected response code: 500”

Which I can understand because that directory does not exist. Here’s the video:

Lucee Websockets - Getting Started

Anyone have an Idea what I’m doing wrong?

[isapir]

@Cliff_Mosdall

“WebSocket connection to ‘ws://localhost:8888/ws/echo’ failed: Error during WebSocket handshake: Unexpected response code: 500”

That usually means that your CFML/cfscript code threw an error.

Please check the Lucee log files, including the websocket log file that you should create as advised in that video.

@Cliff_Mosdall

Which I can understand because that directory does not exist.

Actually, that directory should not exist, so that’s not the reason for the error.

[isapir]

I have published a tutorial that shows how to write a multi-channel Chat Server with the Lucee WebSocket Extension at:

Lucee WebSocket Chat Server Example

The code for the example available at

Example Chat

[dklmuc]

Hi

After waiting a few weeks and checking the github repo from time to time, there are still two big issues for us using this extension:

1. the session problem (see https://lucee.daemonite.io/t/lucee-websockets-extension/2067/8)
For development it wasn’t a big problem only running one local Lucee context, but most time this won’t be the case in a customer environment. Are there any chances this could be fixed (you were talking about some “hacking” because of no Lucee API for getting a session)? Or do we need to find a workaround because we won’t being able to use the session features?

2. specific client messaging
As far as I understand there is already a logic to send websocket messages to a specific client, e.g. returning it in the onMessage() event listener.

Use Case:
e.g. a more complex chat szenario like the facebook chat (1:1 chat, 1:n group chats, same user with different devices)

Are there any chances we will be able to send a websocket messages only to a group of specific clients? Or should we work around this and use rooms here (are there limitations in the rooms number or could there be performance problems)? Like one room per conversation and one room per user (for some “control messages” when he is connected with multiple devices, like e.g. deleting the chat history in device 1 will also delete the history in device 2)?

Thanks a lot in advance

[Redtopia]

Specific to your second point about building a facebook chat, you would need to extend the basic chat example using some back-end logic that maps a user to a thread. For example, the endpoint might look like /ws/chat/{threadid} and then you would need to add some validation and security logic to make sure the current user has access to the thread.

[dklmuc]

Thanks for your answer, but the question was not how to do it, because the code is already there. Based on an older Coldfusion version of it (running an old websocket extension).
So for the first point I would love to kick all that workaround stuff out to get specific user data because of the possibility to access the user session.
And for the second point I see the possibility to use rooms here, but it seems much more complicated than needed to manage all that room stuff (100 users means having a few houndred rooms to take care of) instead of simple call a send function with some specific connection client ids.

[isapir]

@dklmuc

the session problem

You mean in a Cluster? How is your cluster set up?

@dklmuc

specific client messaging

Have you watched the videos that I posted in the past couple of weeks? The second one shows how to set up a Chat Server with multiple channels. You can use it for 1:1, 1:n, n:n.

[dklmuc]

@21Solutions

You mean in a Cluster? How is your cluster set up?

No not in a cluster, standard local installation. In the other thread (sorry it was the wrong link above, this one: https://lucee.daemonite.io/t/websockets-wrong-session-scope-in-listener-component/2188/5?u=dklmuc) I posted some detailed session data what happen when I’m running more than one context (and yes I already checked that they run with a different application name). When I go into server.xml and disable all other context except the one with the chat it’s running like it should. @Redtopia confirmed this.

@21Solutions

Have you watched the videos that I posted in the past couple of weeks? The second one shows how to set up a Chat Server with multiple channels. You can use it for 1:1, 1:n, n:n.

I will have a look thank you. But like I said, it feels a little bit unnecessary complex compared to the possibility to the tell the websocket server to send a message to a couple of specific client connection IDs in a channel.

Anf after thinking more about it’s more complex then I thought yesterday:
We have got for example more than 10 different message types. Some of them are control messages which only must only go to one client connection (like connection ackknowledge with some data, a reconnect order, …), some other control/normal messages must only go to each connection of the same user to keep all his clients in sync (like delete a conversation, the echo of his own new send message, …) and of course control/normal messages which go to all users (and users with multiple clients) in a single conversation.
So I would need at least 2 channels per user (already without an ongoing conversation):

• one channel per each connection (to send connection specific messages)
• one channel per each user (to send user specific messages)

So the running project (with the old Coldfusion Websocket Extension we use there) has around 100 to 200 concurrent users in peak time, which means in the new implementation at least a few hundred channels because of this plus the channels for ongoing conversations. And there will be some more features using a websocket connection for updating data in the future, so again more and more channels.

Compared to the Coldfusion Event Gateway Websocket implementation:

SendGatewayMessage(
	"chat",
	{
		MESSAGE: SerializeJSON( {...} ),
		DESTINATIONWEBSOCKETIDS: arrayWithConnectionIDs
	}
);

[isapir]

@dklmuc

I will have a look thank you. But like I said, it feels a little bit unnecessary complex compared to

I’m not sure how you determine that it’s “unnecessary complex” before you watch the videos?

@dklmuc

We have got for example more than 10 different message types.

Then add a key to a CFML Struct (JavaScript Object) that specifies the type, and use serializeJSON() before you send the message. There is no need for more than one connection per user in that scenario.

[dklmuc]

First let me say I really appreciate you take the time to answer and the work you are doing for this extension. Really love the idea behind using the session of a user also for his websocket connection.

@21Solutions

I’m not sure how you determine that it’s “unnecessary complex” before you watch the videos?

Because I had a look into the documentation and consider how I could achieve a working solution for the requirements. I watched it now But my opinion is still the same.

@21Solutions

Then add a key to a CFML Struct (JavaScript Object) that specifies the type, and use serializeJSON() before you send the message. There is no need for more than one connection per user in that scenario.

This was no question how to do it, it’s already here and working (with a little dirty workaround of client side message filtering, but this isn’t something which can go live). I’ve readed all the discussions about the extension and so I decided to bring the detailed usecase straight away with this feature request.

I already understand that a websocket connection can subscribe to multiple channels. But the usecase is like someone told you to build him the facebook chat, so there isn’t really one single message you can broadcast because of privacy data protection and so on. A user can be online with multiple devices you need to keep in sync, users can be online for all others or limited online only for some other uses, you got simple 1:1 conversations, but also 1:n with invitations/kicks, users can block other users (so they are not shown as online to them anymore), typing notifactions… So a bunch of features which need to be send over the websocket connection and all of them limited to specific connections.

So old implementation (CF):

• client is connected to websocket gateway
• server decides to which websocket connection a message is sent
• client react on message type and further data (e.g. conversation id)

New implementation would be:

• client is connected to websocket gateway
• server must handle subscriptions to multiple channels (a channel for each conversation + a channel for device sync when user is connected with multiple devices + maybe a channel for control message for each single client device (already not sure about it, but should keep client code smaller))
• client react on message type and further data

For me it looks more complex to handle the channel stuff dynamic (e.g. create a channel and subscribe participants when one old archived conversation is opened or a new one is started).

And finally to also bring another use case for this feature:
Some other chat systems I remember from old times which were providing different open rooms (so very similar to this channel feature in the extension) also had a whisper feature, so you could send a message in a room to exactly one recipient.

[isapir]

@dklmuc

it’s already here and working (with a little dirty workaround of client side message filtering, but this isn’t something which can go live)

Sending a type with a message and then deciding on the other end how to interpret the message is not a “dirty workaround”. It’s the only way to do “different” things on the same connection. If you don’t do that then you really are limited to 1-connection-per-type, which would be very inefficient to say the least.

@dklmuc

the usecase is like someone told you to build him the facebook chat, so there isn’t really one single message you can broadcast because of privacy data protection and so on. A user can be online with multiple devices you need to keep in sync, users can be online for all others or limited online only for some other uses, you got simple 1:1 conversations, but also 1:n with invitations/kicks, users can block other users (so they are not shown as online to them anymore), typing notifactions… So a bunch of features which need to be send over the websocket connection and all of them limited to specific connections.

I’m sorry but perhaps I’m missing something simply because I’m not using Facebook. Then again I don’t see any chat capability that you require that can not be done rather easily with the extension.

All these things can be done easily by using a type key when you send a message. You can have a message of type : notification, type : message, type : invitation, type : kick, etc.[quote=“dklmuc, post:15, topic:2067”]
So old implementation (CF):

1. client is connected to websocket gateway
2. <snip/>
3. client react on message type and further data (e.g. conversation id)

New implementation would be:

1. client is connected to websocket gateway
2. <snip/>
3. client react on message type and further data
   [/quote]

These parts are the same so let’s ignore them[quote=“dklmuc, post:15, topic:2067”]
So old implementation (CF):
2) server decides to which websocket connection a message is sent
[/quote]

Really? That’s all you have to do in ACF in order to support all of the features that you mentioned? Can you post actual code please? I’d like to see it.

@dklmuc

New implementation would be:
2) server must handle subscriptions to multiple channels (a channel for each conversation + a channel for device sync when user is connected with multiple devices + maybe a channel for control message for each single client device (already not sure about it, but should keep client code smaller))

The connection manager handle all of the subscriptions for you. If you use the {channel} path parameter then you don’t even need to subscribe to the channel, that happens to you “automagically”.

If you want to be able to send 1:1 messages then you can easily subscribe every user to a channel with its own unique identifier. Then you can send messages to that user directly by broadcasting to its own channel.

@dklmuc

For me it looks more complex to handle the channel stuff dynamic (e.g. create a channel and subscribe participants when one old archived conversation is opened or a new one is started).

Channels are created automatically if they do not exist. If your implementation archives conversations or determines based on your logic when a conversation starts or ends then that is your implementation. The extension does not enforce any of that.

@dklmuc

Some other chat systems I remember from old times which were providing different open rooms (so very similar to this channel feature in the extension) also had a whisper feature, so you could send a message in a room to exactly one recipient.

You can easily support a “whisper” feature. Subscribe each user to a channel with his own unique identifier as described above.

[dklmuc]

@21Solutions

Sending a type with a message and then deciding on the other end how to interpret the message is not a “dirty workaround”. It’s the only way to do “different” things on the same connection. If you don’t do that then you really are limited to 1-connection-per-type, which would be very inefficient to say the least.

The actual workaround is that I send a user ID in each message so the clients can filter it. But like I said just a dirty development workaround to work on the new client code (JS to Angular).

@21Solutions

Really? That’s all you have to do in ACF in order to support all of the features that you mentioned? Can you post actual code please? I’d like to see it.

After the long weekend, I think I understand the base problem of the controversion now. I’ve never worked with CF native websockets, it’s just an old project running with an old websocket extension (Home · nmische/cf-websocket-gateway Wiki · GitHub). And after having a look at the native CF websocket implementation, I see your implementation is very similar.
To answer your question: In the code I’ve got my websockets connection also in the database, when I want to send a message I can do a little lookup and then hand the websocket connection IDs to the SendGatewayMessage function of the extension.

@21Solutions

You can easily support a “whisper” feature. Subscribe each user to a channel with his own unique identifier as described above.

I fully understand your point now and see also the advantage of channels, although I’m still not sure why Adobe (or you) did it that way. I’ve got still the feeling a mixture of channels and the possibility to send messages in a channel to some specific connection would be the better approach.
E.g. there is a chat with the channels movies, weather and cars. I’m in cars and want to whisper to somebody who also is in cars. Why moving it logic wise to a new channel? On client side the incoming message will be something like: channel cars, whisper from user x, message… But just my opinion or problem

So really thanks again for your time to discuss this topic, I was the one missing something. It would be great if you could have a look at this session problem.

[isapir]

@dklmuc

The actual workaround is that I send a user ID in each message so the clients can filter it.

I’m not sure what you mean – you are sending the message only to the client(s) to whom the message is addressed, right? I hope that you’re not sending the message to all of the clients and then rely on the client to filter messages addressed to it – that would be the cause for a major security problem.

@dklmuc

And after having a look at the native CF websocket implementation, I see your implementation is very similar.

I’m actually surprised to hear that because I really didn’t like the ACF API and modeled the extension more similarly to the Socket.io implementation (for Node.js), but my extension is using the Java specification for WebSockets, JSR-356, so if ACF is also utilizing the JSR then there will be some similarities.

@dklmuc

To answer your question: In the code I’ve got my websockets connection also in the database

What do you have stored in the database? The WebSocket connection ID? WebSockets are “fragile” and can close at any moment. I would be very careful of where I store them. That’s the reason that I wrote the ConnectionManager – to do all of that work for you and clean up after dead connections.

@dklmuc

I’ve got still the feeling a mixture of channels and the possibility to send messages in a channel to some specific connection would be the better approach.
E.g. there is a chat with the channels movies, weather and cars. I’m in cars and want to whisper to somebody who also is in cars. Why moving it logic wise to a new channel? On client side the incoming message will be something like: channel cars, whisper from user x, message…

You can name your channels however you want. It doesn’t have to be just the user-id, or cfid. It can be a combination of {channel}+{user-id}. But if I understand you correctly – you want to send the “whisper” to the whole channel and then have the clients filter it? Again, that will be a huge security hole. Anyone would be able to easily rewrite your client code and listen in to all of the “whisper” messages.

@dklmuc

It would be great if you could have a look at this session problem.

I was unable to reproduce this issue as of yet. I will try to look into it soon. AFAIK the Sessions works fine in a non-clustered environment. Not sure why you’re experiencing issues in multi-context.

@Redtopia opened a ticket at Unable to reliably get correct session/application scopes in Listener Components · Issue #3 · isapir/lucee-websocket · GitHub – please see if you can add more useful information there.

[dklmuc]

@21Solutions

I’m not sure what you mean – you are sending the message only to the client(s) to whom the message is addressed, right? I hope that you’re not sending the message to all of the clients and then rely on the client to filter messages addressed to it – that would be the cause for a major security problem.

No fear, just for local development Was the fastest way to be able to rewrite the client code in angular and make changes to the wrapping code around the websocket implementation.

@21Solutions

I’m actually surprised to hear that because I really didn’t like the ACF API and modeled the extension more similarly to the Socket.io implementation (for Node.js), but my extension is using the Java specification for WebSockets, JSR-356, so if ACF is also utilizing the JSR then there will be some similarities.

Wrong wording from me then sorry. Only had a quick look at the Adobe help and see channels and channel broadcast.

@21Solutions

What do you have stored in the database? The WebSocket connection ID? WebSockets are “fragile” and can close at any moment. I would be very careful of where I store them. That’s the reason that I wrote the ConnectionManager – to do all of that work for you and clean up after dead connections.

Yes, there is already a lot of state management going on, mainly because the CF project is also running on a loadbalanced environment.

@21Solutions

You can name your channels however you want. It doesn’t have to be just the user-id, or cfid. It can be a combination of {channel}+{user-id}. But if I understand you correctly – you want to send the “whisper” to the whole channel and then have the clients filter it? Again, that will be a huge security hole. Anyone would be able to easily rewrite your client code and listen in to all of the “whisper” messages.

No not client side, something like:

connMgr.broadcast('cars', serializeJson(message), ToClientIDs)

Same way like the onMessage() listener return sends an answer only to the triggering client. But like I said, maybe I’m wrong or too much into this old CF extension.

@21Solutions

I was unable to reproduce this issue as of yet. I will try to look into it soon. AFAIK the Sessions works fine in a non-clustered environment. Not sure why you’re experiencing issues in multi-context.

@Redtopia opened a ticket at Unable to reliably get correct session/application scopes in Listener Components · Issue #3 · isapir/lucee-websocket · GitHub – please see if you can add more useful information there.

I will do, thanks a lot again.

[dklmuc]

Some further informations:

1. the session problem
Funny thing is that I tried it today at some live server (linux instead of windows and running with real domains) and it worked as it should. No idea what’s going on here. I’ve added this also to the github issue and next monday we will try to get the project running on its live server (also linux).

2. specific client messaging
In the end too much talking about it. When I started to implement the channels, the solution was much easier as I thought. So for people who need to refactor some old project with that cf-websocket-gateway extension, simple put all client connection into their own channel and on sending a message send it to each channel of the client you need to target. Works like a charme now, so thank you again @21Solutions

[isapir]

@dklmuc

Funny thing is that I tried it today at some live server (linux instead of windows and running with real domains) and it worked as it should.

Great to hear. I’ve been testing on Windows properly so no it’s not OS-specific AFAIK.

@dklmuc

I’ve added this also to the github issue

Thanks, I hope to look into it soon.

@dklmuc

In the end too much talking about it. When I started to implement the channels, the solution was much easier as I thought.

Thank you for posting the updated feedback

[Lance_Lake]

I just tried using this and I notice that it is producing the following
json.

Object {MESSAGE: "Hello", TIMESTAMP: 1497640931522, CHANNEL: "default",
FROM: "GM"}

When THIS is what it should be producing to be valid JSON.

Object {"MESSAGE": "Hello", "TIMESTAMP": 1497640931522, "CHANNEL":
"default", "FROM": "GM"}

Please fix this or tell me how to pull just the message from the JSON.
Because evt.data.Message is undefined because of the lack of quotes.

[isapir]

@Lance_Lake

Object {MESSAGE: “Hello”, TIMESTAMP: 1497640931522, CHANNEL: “default”,
FROM: “GM”}

The extension does not produce any JSON. Please post your code that produces this.

Also, please post either here, or on Github, but not on both. Thanks.