Is there a non-blocking method to check for data faster than select() and poll()?
c, linux, performance, posix, sockets
Solution
There is no requirement for `poll` or `select` to block for any period of time. If the `timeout` parameters for `poll` or `select` are set to zero both calls return immediately with indications of i/o availability. This eliminates engaging the timer and subsequent rounding.
Note, it is not clear to me why this would be substantially faster than a simple non-blocking polling read if you are only monitoring one file descriptor. I would have expected any benefits from this approach to begin to accrue when multiple FDs were in play so it is interesting that your testing uncovered this.
Problem
I have a C program sending data as fast as it can using the sendto() method from the sender to the receiver which receives with recvfrom() method. The data is encapsulated into layer 2 Ethernet frames and the application is writing Ethernet frames directly on to the wire (no TCP or UDP or even IP). This is on x86_64 Linux (dev machines are just stock Ubuntu 14.04). I have no intention of porting to any other OS, the application design scope is for Linux so other OSes don't matter. Sender: ``` while (true) { sendResult = sendto(sockFD, txBuffer, fSize+headersLength, 0, (struct sockaddr*)&socket_address, sizeof socket_address); } ``` Receiver: ``` while (true) { recvfrom(sockFD, rxBuffer, fSizeTotal, 0, NULL, NULL); } ``` I want the sender to be able to check for received packets; should the receiver application quit it sends data back to the sender saying "I quit" so the sender will stop sending data. I used poll() on the sender to check for received messages as below but this significantly reduces the transmission speed from just shy of 1Gbps (968Mbps) to around 10Mbps. I'm testing with a cross-over cable between two PCs with 1Gbps NICs. The sender counts sent frames and the frame size and the receiver counts received frames and the frame size, so to confirm, the application is actually receiving at wire-rate near enough, I'm not just look at NIC utilization or similar. Poll() Method: ``` int rv; struct pollfd ufds[1]; ufds[0].fd = sockFD; ufds[0].events = POLLIN while (true) { sendResult = sendto(sockFD, txBuffer, fSize+headersLength, 0, (struct sockaddr*)&socket_address, sizeof(socket_address)); // wait for events on the sockets, 1ms timeout rv = poll(ufds, 1, 1); if (rv > 0) { if (ufds[0].revents & POLLIN) { recvfrom(sockFD, rxBuffer, fSizeTotal, 0, NULL, NULL); } } } ``` 1 millisecond is the lowest timeout that can be set for the poll() method. This is why my transmission program could only transmit at 10Mbps. The application can easily saturate a 1Gbps link though with minimal CPU usage, I was getting 968Mbps as I previously stated (I don't mean peak by the way, that is sustained throughput). I removed the poll() call and switched to select() using the below example, but again, using the smallest delay I can here my transmission application could only get 175Mbps. Not close to the original 968Mbps; Select() Method: ``` fd_set readfds; struct timeval tv; int rv, n; FD_ZERO(&readfds); FD_SET(sockFD, &readfds); n = sockFD + 1; while (true) { sendResult = sendto(sockFD, txBuffer, fSize+headersLength, 0, (struct sockaddr*)&socket_address, sizeof(socket_address)); tv.tv_sec = 0; tv.tv_usec = 000001; rv = select(n, &readfds, NULL, NULL, &tv); if (rv > 0) { if (FD_ISSET(sockFD, &readfds)) { recvfrom(sockFD, rxBuffer, fSizeTotal, 0, NULL, NULL); } } ``` It seems that both methods are too slow for today’s systems (my cpu usage was around 2% for all tests above). I wish to move this application on to some 10GigE machines soon and start testing there but I obviously can't using either of these two methods. Is there no faster way I can check? I though these were supposed to be non-blocking but by requiring a timeout they are blocking in a way; I have seen this thread but it isn't the answer I require. Is there no method I can simply call that checks at that moment in time it was called, for data waiting to be read, then returns immediately if no data is waiting to be read? As a side node, I haven't read up on the recvfrom method() to see where the delay is yet before posting this but I did try the following because it only took 30 seconds to change the code, it resulted in the worst outcome which was less than 1Mbps; Sender: ``` while (true) { // Continually send a frame then check for a frame, send a frame then check for a frame... sendResult = sendto(sockFD, txBuffer, fSize+headersLength, 0, (struct sockaddr*)&socket_address, sizeof(socket_address)); recvfrom(sockFD, rxBuffer, fSizeTotal, 0, NULL, NULL); } ```