Using `splice` for Linux... what else for other systems?
bsd, c, macos, operating-system, windows
Solution
In the BSDs `sendfile()` fills this role, as long as the destination file descriptor is a socket.
In all POSIX OSes, `memmap()` can be used to achieve this, by mapping a whole file, then calling `write()` on the mapped data. Since you never access this memory in your process, it can still have zero-copy performance.
Almost everywhere, the `read()/write()` combination works well, and is very fast. Zero-copy is cool, but with a page-sized buffer ~4096, you'll be rather close to the same speed, but with code everyone understands.
Problem
On recent Linux kernels, afaict, the fastest way of copying a file or a subset of a file to another file is through the use of the very nice `splice` system call. This system gets the kernel to manage the transfer (almost) directly, without ever having to copy the data to userland memory. Now, I would be interested in finding something similar for other systems, in particular Windows and BSD/MacOS X (I am willing to get some code written in Obj-C in the unlikely case there is an API that cannot be accessed from raw C). Thanks Note I assume that the fastest full-file copy under Windows is `CopyFile`. But what about copying only a range of a file? Note I am working on code that is both performance-sensitive and should work on a variety of mass storage devices (i.e. from Android smartphones to NFS mounts). I am, of course, willing to write different implementations for different OSes. Note At the moment, I use `splice` under Linux/Android, `copyfile` under BSD/Mac OS X, `CopyFile` under Windows, and fallback to `read`/`write` for ranges in files.