socket.io problem , when having multiple instances

When you're scaling there's a few ways to go about it -

  1. Multiple node processes on a single machine/VM (Cluster mode does exactly this, by spawning a new process per CPU core).
  2. Multiple machines/VMs.

If you're using approach #1 (which you are in this case) then the processes are on the same machine. Therefore you can use the machine itself as a common data layer to communicate between the processes running on different cores.

IPC (Inter Process Communication) is the way to do this. There's a few different methods through which IPC can be achieved, and it's fairly fast at what it does (atleast compared to the socket request time) so you won't take much of a performance hit. And the cluster-adapter linked below uses this to help broadcast the incoming request to all processes.

Wiki - https://en.wikipedia.org/wiki/Inter-process_communication

At some point though if a single machine is no longer enough for you to serve all your requests, you resort to approach #2 and start scaling horizontally (adding more machines into the mix).

Side note - 

* Vertically scaling = Increasing the specs of the existing machine. You can bump up the CPU/memory etc. of your existing machine to help manage the request load.
* Horizontally scaling = Adding more machines to help distribute the load.

Usually when multiple machines are serving requests they have some form of load balancer sitting on top of them directing which machine the request should go to.

Unfortunately most of the algorithms used to dictate the load balancing are tailored for REST APIs where it doesn't matter what server is responding to the request.

So the load balancers might do something like balancing the requests by CPU usage/memory usage where they try and send the request to the machine with the least taxed CPU.

This goes against the principle of how a socket system works (the request needs to go back to the server that has the open TCP connection).

The way to get around this is to use "sticky sessions", i.e. ensure that if first request from client #1 goes to machine #1 then every subsequent request from client #1 also goes to machine #1.

Since these are two separate machines there's no common layer for them to communicate on (unlike in the case with multiple processes on a single machine).

The way around that is to use the redis-adapter - https://socket.io/docs/v4/redis-adapter/

The page I linked goes over the architecture. But the gist of it is that there is a common redis instance that receives each request and helps decide which process on which machine should be responsible for answering said request. It works with a similar broadcast mechanism to the cluster mode, but across multiple machines.

So the TL;DR gist here would be -

  • Read up on how socket connections maintain their TCP connection.
  • Read up on how cluster mode in Node spawns multiple processes - https://nodejs.org/api/cluster.html.
  • If you decide that your application is only ever going to need a single machine then go with the cluster-adapter.
  • If you decide that your application might need to scale beyond a single machine (or if you're doing this to learn about sockets) then it might be worth investing in setting up a redis layer and using the redis-adapter.
/r/node Thread Parent