[NEW] Support fan-out/multicast (PUBLISH to multiple channels)

[lukehutch]

The problem/use-case that the feature addresses

I need to be able to send a message to multiple channels. Currently I loop through the channels and send a PUBLISH command for each channel, which can result in a lot of round-trips to the server.

To give more specific usecase details: a user can be a member of potentially hundreds of chat channels in the history of using my app. Rather than subscribe every online user to every chat channel in their inbox every time they go online, I have chosen to subscribe users to just one chat channel, associated with their user id. Now when a message is sent to a chat channel with multiple users, I need to multicast the message to all users who are members of that chat channel.

Description of the feature

I would like a PUBLISH_MULTIPLE command that will take a list of channels, and publish to all of them.

Alternatives you've considered

Instead, I could SUBSCRIBE each user to the channel id of all chat channels in their chat history, and just send new messages once to the intended channel, but I am worried about how well that will scale, with potentially millions of users, each with potentially hundreds of chat channels (most of which have not had any new messages in a long time).

[sundb]

what if using lua script to publish message to a channel list? which can avoids the round-trips.

[lukehutch]

Definitely possible, but this should be built in for speed. Most other message queue platforms support fan-out.

[sundb]

@lukehutch in this case i recommend using stream instead of pubsub, as stream supports fan-out.

[lukehutch]

OK, I didn't know that... but streams are a fundamentally different messaging model, so I will probably have a hard time convincing the upstream library I depend upon to switch from pub/sub to streams. Is there a fundamental reason why fan-out is not supported for pub/sub?

[sundb]

@lukehutch pubsub is not a message queue, it is a simple subscribe and publish, and it is more appropriate to use stream in your scenario.