Does Google Cloud Storage support SPDY, Keep-Alive, pipelining or other ways to improve small object performance?
google-cloud-storage
Solution
Google Cloud Storage supports a variety of features that improve upload and download performance:
- Use a thread or process pool to issue requests on many connections in parallel.
- Use the Keep-Alive header to enable persistent HTTP connections. This allows you to reuse the same connection after each request, which improves the performance of TCP and SSL.
- Pipeline your requests. Send the second request while you are waiting for the response to your first request. You can do this on the same HTTP connection.
- Use SPDY, which can help reduce latency through compression, multiplexing and prioritization.
Problem
I have ten million tiny objects to transfer with Google Cloud Storage. It is expensive to open a TCP connection perform the SSL handshake, upload the request and wait for the response for each request before issuing the next request. How can I speed things up?