Atomic write on an unix socket?

atomic, sockets, unix

Solution

Pipes

Yes the non-blocking capacity is usually 4KB, but for maximum portability you'd probably be better off using the `PIPE_BUF` constant. An alternative is to use non-blocking I/O.

More information than you want to know in `man 7 pipe`.

Unix datagram sockets

Writes using the `send` family of functions on datagram sockets are indeed guaranteed to be atomic. In the case of Linux, they're reliable as well, and preserve ordering. (which makes the recent introduction of `SOCK_SEQPACKET` a bit confusing to me) Much information about this in `man 7 unix`.

The maximum datagram size is socket-dependent. It's accessed using `getsockopt/setsockopt` on `SO_SNDBUF`. On Linux systems, it ranges between 2048 and `wmem_max`, with a default of `wmem_default`. For example on my system, `wmem_default = wmem_max = 112640`. (you can read them from `/proc/sys/net/core`) Most relevant documentation about this is in `man 7 socket` around the `SO_SNDBUF` option. I recommend you read it yourself, as the capacity doubling behavior it describes can be a bit confusing at first.

Practical differences between stream and datagram

Stream sockets only work connected. This mostly means they can only communicate with one peer at a time. As streams, they're not guaranteed to preserve "message boundaries".

Datagram sockets are disconnected. They can (theoretically) communicate with multiple peers at a time. They preserve message boundaries.

[I suppose the new `SOCK_SEQPACKET` is in between: connected and boundary-preserving.]

On Linux, both are reliable and preserve message ordering. If you use them to transmit stream data, they tend to perform similarly. So just use the one that matches your flow, and let the kernel handle buffering for you.

Crude benchmark comparing stream, datagram, and pipes:

# unix stream 0:05.67
socat UNIX-LISTEN:u OPEN:/dev/null &
until [[ -S u ]]; do :;done
time socat OPEN:large-file UNIX-CONNECT:u

# unix datagram 0:05.12
socat UNIX-RECV:u OPEN:/dev/null &
until [[ -S u ]]; do :;done
time socat OPEN:large-file UNIX-SENDTO:u

# pipe 0:05.44
socat PIPE:p,rdonly=1 OPEN:/dev/null &
until [[ -p p ]]; do :;done
time socat OPEN:large-file PIPE:p

Nothing statistically significant here. My bottleneck is likely reading large-file.

Problem

I'm trying to choose between pipes and unix sockets for an IPC mechanism. Both support the `select()` and `epoll()` functions which is great. Now, pipes have a 4kB (as of today) "atomic" write, which is guaranteed by the Linux Kernel. Does such a feature exists in the case of unix sockets? I couldn't find any document stating this explicitely. Say I use a UNIX socket and I write x bytes of data from my client. Am I sure that these x bytes will be written on the server end of the socket when my server's `select()` cracks? On the same subject, would using SOCK_DGRAM ensure that writes are atomic (if such a guarantee is possible), since datagrams are supposed to be single well-defined messages? What would then be the difference using SOCK_STREAM as a transfer mode? Thanks in advance.

Original source

Related problems