Linux: When to use scatter/gather IO (readv, writev) vs a large buffer with fread

io, linux

Solution

The main convenience offered by `readv`, `writev` is:

- It allows working with non contiguous blocks of data. i.e. buffers need not be part of an array, but separately allocated.

- The I/O is 'atomic'. i.e. If you do a `writev`, all the elements in the vector will be written in one contiguous operation, and writes done by other processes will not occur in between them.

e.g. say, your data is naturally segmented, and comes from different sources:

struct foo *my_foo;
struct bar *my_bar;
struct baz *my_baz;

my_foo = get_my_foo();
my_bar = get_my_bar();
my_baz = get_my_baz();

Now, all three 'buffers' are not one big contiguous block. But you want to write them contiguously into a file, for whatever reason (say for example, they are fields in a file header for a file format).

If you use `write` you have to choose between:

- Copying them over into one block of memory using, say, `memcpy` (overhead), followed by a single `write` call. Then the write will be atomic.

- Making three separate calls to `write` (overhead). Also, `write` calls from other processes can intersperse between these writes (not atomic).

If you use `writev` instead, its all good:

- You make exactly one system call, and no `memcpy` to make a single buffer from the three.

- Also, the three buffers are written atomically, as one block write. i.e. if other processes also write, then these writes will not come in between the writes of the three vectors.

So you would do something like:

struct iovec iov[3];

iov[0].iov_base = my_foo;
iov[0].iov_len = sizeof (struct foo);
iov[1].iov_base = my_bar;
iov[1].iov_len = sizeof (struct bar);
iov[2].iov_base = my_baz;
iov[2].iov_len = sizeof (struct baz);

bytes_written = writev (fd, iov, 3);

Sources:

- http://pubs.opengroup.org/onlinepubs/009604499/functions/writev.html

- http://linux.die.net/man/2/readv

Problem

Original source