How to know when the HTTP headers part is ended?

c, http-headers, sockets

Solution

You are not parsing your input correctly. Here are a couple of things you are doing incorrectly:

- Your code seems to imply that your buffer will contain at most, one line of header data. However recv() does not operate on "lines" of data, but on blocks of binary data. So if you tell it that your buffer is 128 bytes in length, it will attempt to fill your buffer with 128 bytes of data if it is available (even if the 128 bytes of data contain multiple "lines").

- Your code does not take into consideration that the "\r\n" of the header break might be pulled into your buffer by two different calls to recv() which would prevent your code from recognizing the header break.

- If you do find the header break (which may happen if size of the headers is just right), you will end up pushing the last header with the terminating "\r\n" and the header break ("\r\n") into your binary data copy.

I've written a quick function which should find the end of the HTTP headers and write the rest of the server's response to a file stream:

void parse_http_headers(int s, FILE * fp)
{
   int       isnheader;
   ssize_t   readed;
   size_t    len;
   size_t    offset;
   size_t    pos;
   char      buffer[1024];
   char    * eol; // end of line
   char    * bol; // beginning of line

   isnheader = 0;
   len       = 0;

   // read next chunk from socket
   while((readed = read(s, &buffer[len], (1023-len))) > 0)
   {
      // write rest of data to FILE stream
      if (isnheader != 0)
         fwrite(buffer, 1, readed, fp);

      // process headers
      if (isnheader == 0)
      {
         // calculate combined length of unprocessed data and new data
         len += readed;

         // NULL terminate buffer for string functions
         buffer[len] = '\0';

         // checks if the header break happened to be the first line of the
         // buffer
         if (!(strncmp(buffer, "\r\n", 2)))
         {
            if (len > 2)
               fwrite(buffer, 1, (len-2), fp);
            continue;
         };
         if (!(strncmp(buffer, "\n", 1)))
         {
            if (len > 1)
               fwrite(buffer, 1, (len-1), fp);
            continue;
         };

         // process each line in buffer looking for header break
         bol = buffer;
         while((eol = index(bol, '\n')) != NULL)
         {
            // update bol based upon the value of eol
            bol = eol + 1; 

            // test if end of headers has been reached
            if ( (!(strncmp(bol, "\r\n", 2))) || (!(strncmp(bol, "\n", 1))) )
            {
               // note that end of headers has been reached
               isnheader = 1;

               // update the value of bol to reflect the beginning of the line
               // immediately after the headers
               if (bol[0] != '\n')
                  bol += 1;
               bol += 1;

               // calculate the amount of data remaining in the buffer
               len = len - (bol - buffer);

               // write remaining data to FILE stream
               if (len > 0)
                  fwrite(bol, 1, len, fp);

               // reset length of left over data to zero and continue processing
               // non-header information
               len = 0;
            };
         };

         if (isnheader == 0)
         { 
            // shift data remaining in buffer to beginning of buffer
            offset = (bol - buffer);
            for(pos = 0; pos < offset; pos++)
               buffer[pos] = buffer[offset + pos];

            // save amount of unprocessed data remaining in buffer
            len = offset;
         };
      };
   };

   return;
}

I've not tested the code, so it may have simple errors, however it should point you in the correct direction for parsing string data from a buffer in C.

Problem

Original source