nginx, node.js and socket.io - is there a working marriage?

haproxy, nginx, node.js, socket.io, websocket

Solution

Here's my (old and for testing purposes) HAProxy config for proxying WebSockets and normal HTTP requests.

global
    maxconn 4096
    nbproc 2
    daemon
    user nobody

defaults
    mode http

frontend app
    bind 0.0.0.0:8000
    mode tcp
    timeout client 86400000
    default_backend www_backend
    acl is_websocket path_beg /sockets

    use_backend socket_backend if is_websocket
    tcp-request inspect-delay 500ms
    tcp-request content accept if HTTP

backend www_backend
    balance roundrobin
    option forwardfor
    mode http
    option httplog
    option httpclose
    timeout server 30000
    timeout connect 4000
    server w1 localhost:8080 weight 1 maxconn 1024 check

backend socket_backend
    balance roundrobin
    mode http
    option httplog
    option forwardfor
    timeout queue 5000
    timeout server 86400000
    timeout connect 86400000
    timeout check 1s
    no option httpclose
    option http-server-close
    option forceclose
    server s1 localhost:8081 weight 1 maxconn 1024 check

Note that I am recognizing whether request is WS or not by looking at path (`acl is_websocket path_beg /sockets` line). This can be replaced with for example this:

acl is_websocket hdr(Upgrade) -i WebSocket

or this:

acl is_websocket hdr_beg(Host) -i ws

or both. Proxying to nginx with this config should work out of the box.

Problem

`nginx` is a killer static file server. it can serve `node.js`, as in this example, but in a limited fashion. but `nginx` is apparently unable to proxy `websockets`. the only thing I found that might work is using HAProxy front end as per this article - but it's from October 6, 2011. this has to be a common problem, but I'm not finding a very common solution. Solution (see https://github.com/bangkok-maco/barebone-node for complete solution and details) ip testing schema: - 127.0.0.12 - www.chat.nit - public, in /etc/hosts and haproxy - 127.0.1.12 - internal nginx web server - 127.0.2.12 - internal chat serving node.js socket.io /etc/haproxy/haproxy.cfg: ``` global maxconn 4096 nbproc 2 daemon # user nobody log 127.0.0.1 local1 notice defaults mode http # listen on 127.0.0.12:80 frontend app bind 127.0.0.12:80 mode tcp timeout client 86400000 default_backend www_backend acl is_chat hdr_dom(Host) chat acl is_websocket path_beg /socket.io use_backend chat_socket_backend if is_websocket is_chat tcp-request inspect-delay 500ms tcp-request content accept if HTTP # ngnix on 127.0.1.12:80 backend www_backend balance roundrobin option forwardfor mode http option httplog option httpclose timeout server 30000 timeout connect 4000 server w1 127.0.1.12:80 weight 1 maxconn 1024 check # node (socket.io) on 127.0.2.12:80 backend chat_socket_backend balance roundrobin mode http option httplog option forwardfor timeout queue 5000 timeout server 86400000 timeout connect 86400000 timeout check 1s no option httpclose option http-server-close option forceclose server s14 127.0.2.12:8000 weight 1 maxconn 1024 check ``` /etc/nginx/sites-enabled/www.chat.nit ``` server { listen 127.0.1.12:80; root /data/node/chat; index client.html; server_name www.chat.nit; # favicon.ico is in /images location = /favicon.ico$ { rewrite /(.*) /images/$1 last; } # standard includes location ^~ /(css|images|scripts)/ { try_files $uri =404; } # html page (only in root dir) location ~ ^/([-_a-z]+).html$ { try_files $uri =404; } error_page 404 /404.html; # redirect server error pages to the static page /50x.html # error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/www; } } ``` chat (node.js): server.js ``` var app = require('http').createServer() , io = require('socket.io').listen(app); app.listen(8000,'127.0.2.12'); io.sockets.on('connection', function(socket) { ... }; ``` chat: client.html ``` <head> <script src="/scripts/socket.io/socket.io.js"></script> <script> var socket = io.connect('http://www.chat.nit:80'); ... </script> </head> ``` notes: link socket.io client js into `scripts/` directory /.../scripts$ ln -s ../node_modules/socket.io/node_modules/socket.io-client/dist/ socket.io /etc/default/haproxy (contrary to text, must set to work at all) ENABLED=1 this version haproxy not logging. found kvz's write up on how to use `rsyslogd` via 127.0.0.1, but could not make it fly. this solution is working - not sysadmin quality to be sure. (enhancements more than welcome.)

Original source

Related problems