read() from a HDD with O_DIRECT fails with 22 (EINVAL, Invalid Argument)
block-device, hard-drive, io, linux
Solution
Direct DMA transfer typically requires the buffer to be aligned. From `man`:
The O_DIRECT flag may impose alignment restrictions on the length and address of userspace buffers and the file offset of I/Os. ... Under Linux 2.6, alignment to 512-byte boundaries suffices.
So `char buffer[512];` might need to be aligned to a 512-byte address.
It may not be possible to achieve this alignment on the stack, so something like:
`static char buffer[512] __attribute__ ((__aligned__ (512)));`
may work. Or maybe this alignment will work on the stack. Alternatively, if you're using x86, you could use the `<mm_malloc.h>` intrinsic support functions: `_mm_malloc` and `_mm_free`.
Problem
I want to make a basic `read()` from a SATA HDD `/dev/sdd`. A `write()` seems to work. Also `read()` and `write()` works without the `O_DIRECT` Flag. I've read, that it has to be aligned to the blocksize. So I used this to get the blocksize: ``` root$ blockdev --getsize /dev/sdd 488397168 root$ blockdev --getsize64 /dev/sdd 250059350016 root$ python -c "print 250059350016.0/488397168" 512.0 ``` As you can see I have root. The HDD is connected via a PCIe SATA Card and `lspci -vv` shows me, that it uses the basic ahci (`drivers/ata/ahci.c`) driver. I work with the 3.2.0 Linux Kernel on a 64 bit Power Architecture. Here is my code: ``` #define _GNU_SOURCE #include <stdio.h> #include <string.h> #include <fcntl.h> #include <sys/types.h> #include <sys/stat.h> #include <stdlib.h> #include <unistd.h> #include <errno.h> int main() { int r; char *buffer, *tmp_buffer; // alloc more than necesarry to align the real buffer tmp_buffer = malloc(2*512*sizeof(char)); long align = (unsigned long)tmp_buffer%512; printf("tmp_buffer is at: %x \% 512 = %d\n",tmp_buffer,align); buffer = tmp_buffer+(512-align); printf("buffer is at: %x \% 512 = %d\n",buffer,(unsigned long)buffer%512); memset(buffer,0,sizeof(512)); // OPEN int fd = open("/dev/sdd",O_DIRECT | O_RDWR | O_SYNC); if(fd!=3) printf("fd = %d\n",fd); // READ printf("try to read and then dump buffer:\n"); r = read(fd,buffer,sizeof(512)); if(r == -1) printf("Error: %s\n",strerror(errno)); else { // DUMP BUFFER int i; for(i=0; i<sizeof(512); i++) printf("%c",buffer[i]); } printf("\n"); return 0; } ``` The output is: ``` tmp_buffer is at: 1cc80010 % 512 = 16 buffer is at: 1cc80200 % 512 = 0 try to read and then dump buffer: Error: Invalid argument ``` edit: I have updated my source as suggested by Brett Hale's answer. Unfortunately I still get the error. Is my way to find out the blocksize ok? Have I done the aligning right? Thank you very much for reading, Fabian