Copying data from a shared-memory-mapped object using sendfile()/fcopyfile()
c++, file-descriptor, mmap, posix, shared-memory
Solution
Copying data between two "things" mapped in memory - like in the example above - will indeed require copying things from kernel to userspace and then back. And no, you can't really use sendfile(2) system call to send to a file descriptor, I'm afraid.
But you should be able to do it like this:
- Create the shared memory object (or a file, really; due to the second step it will be shared in memory anyway
- Map it in memory, with MAP_SHARED; you'll get a pointer
- Open the destination file
- write(destination_fd, source_pointer, source_length)
In this case, the write syscall won't need to copy the data into your process. Not sure what the actual performance characteristic will be, though. Smart use of madvise(2) might help.
Problem
Is it possible – and if so prudent – to use `sendfile()` (or its Darwin/BSD cousin `fcopyfile()`) to shuttle data directly between a shared-memory object and a file? Functions like `sendfile()` and `fcopyfile()` can perform all of the mechanistic necessities underpinning such transfers of data entirely without leaving kernel-space – you pass along two open descriptors, a source and a destination, when calling these functions, and they take it from there. Other means of copying data will invariably require one to manually maneuver across the boundary between kernel-space and user-space; such context-switches are inherently quite costly, performance-wise. I can’t find anything definitive on the subject of using a shared-memory descriptor as an argument thusly: no articles for or against the practice; nothing in the respective `man`-pages; no tweets publicly considering `sendfile()`-ing shared-memory descriptors harmful; &c… But so, I am thinking I should be able to do something like this: ``` char const* name = "/yo-dogg-i-heard-you-like-shm"; /// only one slash, at zero-index int len = A_REASONABLE_POWER_OF_TWO; /// valid per shm_open() int descriptor = shm_open(name, O_RDWR | O_CREAT, 0600); int destination = open("/tmp/yodogg.block", O_RDWR | O_CREAT, 0644); void* memory = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, descriptor, 0); off_t bytescopied = 0; sendfile(destination, descriptor, &bytescopied, len); /// --> insert other stuff with memset(…), memcopy(…) &c. here, possibly munmap(memory, len); close(descriptor); close(destination); shm_unlink(name); ``` … Is this misguided, or a valid technique? And if the latter, can one adjust the size of the in-memory shared map before copying the data? EDIT: I am developing the project to which this inquiry pertains on macOS 10.12.4; I am aiming for it to work on Linux, with eventual FreeBSD interoperability.