How to read from a TCPServer socket in ruby using read, readpartial and read_nonblock

httpserver, nonblocking, ruby, tcpserver, unix

Solution

It's done in unicorn here https://github.com/defunkt/unicorn/blob/master/lib/unicorn/http_request.rb#L69-L71

There is `add_parse` method(read the comments above methods) https://github.com/defunkt/unicorn/blob/master/ext/unicorn_http/unicorn_http.rl#L760-L778

Also take a look at some explanations here http://www.ruby-forum.com/topic/2267632#1014288

Here is your working code using http_parser.rb https://gist.github.com/4136962

`gem install http_parser.rb`

require 'socket'
require "http/parser"


READ_CHUNK = 1024 * 4
socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM)
addr = Socket.pack_sockaddr_in(9799, '127.0.0.1')
socket.bind(addr)
socket.listen(Socket::SOMAXCONN)
socket.setsockopt(:SOCKET, :REUSEADDR, true)

puts "Server is listening on port = 9799"
loop do
    connection, addr_info = socket.accept

    parser = Http::Parser.new
    begin
      data = connection.readpartial(READ_CHUNK)
      puts "Buffer = #{data}"
      parser << data
    end until parser.headers

    connection.write("HTTP/1.1 200 \r\n")
    connection.write("Content-Type: text/html\r\n")
    connection.write("Status 200 \r\n")
    connection.write("Connection: close \r\n")
    connection.write("\r\n\r\n")
    connection.write("Hello World \r\n")
    connection.close
end

Problem

I have a 2 part question on reading from sockets and how is it managed on Ruby servers like Unicorn or Mongrel - I've learnt that to read from a socket is different from reading a file and that there are no distinct EOF message sent and the data is an endless stream. So how do you know when to stop reading? My TCPServer for example in this case when I hit my server by accessing `http://localhost:9799` from a browser, it hangs after there is no more data to read and it won't throw the EOFError either. ``` require 'socket' READ_CHUNK = 1024 socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM) addr = Socket.pack_sockaddr_in(9799, '127.0.0.1') socket.bind(addr) socket.listen(Socket::SOMAXCONN) socket.setsockopt(:SOCKET, :REUSEADDR, true) puts "Server is listening on port = 9799" loop do connection, addr_info = socket.accept data_buffer = "" loop do begin connection.read_nonblock(READ_CHUNK, data_buffer) puts "Buffer = #{data_buffer}" rescue Errno::EAGAIN => e IO.select([connection]) retry rescue EOFError break end end connection.write("HTTP/1.1 200 \r\n") connection.write("Content-Type: text/html\r\n") connection.write("Status 200 \r\n") connection.write("Connection: close \r\n") connection.write("Hello World \r\n") connection.close end ``` I'd like to know whats the best practice/standard approach used by Ruby servers. I see the Unicorn uses read_nonblock from kgio library and mongrel uses readpartial (I'm not sure about these but going through the code this is what I feel is the approach adopted.) Even with checks for \r\n how does the server know the input is complete. Could explain how this should be done (and I think `gets` is not the approach - its with `read`, `readpartial`, `read_nonblock`). 2). I would really appreciate a few lines on how this is achieved in servers like unicorn or passenger Thank you.

Original source