Is file append atomic in UNIX?
atomic, file-io, posix, unix
Solution
A write that's under the size of 'PIPE_BUF' is supposed to be atomic. That should be at least 512 bytes, though it could easily be larger (linux seems to have it set to 4096).
This assume that you're talking all fully POSIX-compliant components. For instance, this isn't true on NFS.
But assuming you write to a log file you opened in 'O_APPEND' mode and keep your lines (including newline) under 'PIPE_BUF' bytes long, you should be able to have multiple writers to a log file without any corruption issues. Any interrupts will arrive before or after the write, not in the middle. If you want file integrity to survive a reboot you'll also need to call `fsync(2)` after every write, but that's terrible for performance.
Clarification: read the comments and Oz Solomon's answer. I'm not sure that `O_APPEND` is supposed to have that `PIPE_BUF` size atomicity. It's entirely possible that it's just how Linux implemented `write()`, or it may be due to the underlying filesystem's block sizes.
Problem
In general, what can we take for granted when we append to a file in UNIX from multiple processes? Is it possible to lose data (one process overwriting the other's changes)? Is it possible for data to get mangled? (For example, each process is appending one line per append to a log file, is it possible that two lines get mangled?) If the append is not atomic in the above sense, then what's the best way of ensuring mutual exclusion?