Apache HttpClient Multiple polling connections to server strategy

apache-httpclient-4.x, java

Solution

@Saad's answer was very helpful, but I've experienced delays when trying to load test single url with many concurrent requests.

Here is updated version that resolved the problem.

    DefaultHttpClient client = new DefaultHttpClient();
    ClientConnectionManager mgr = client.getConnectionManager();
    HttpParams params = client.getParams();
    int maxConnections = 100;
    params.setParameter(ConnManagerPNames.MAX_CONNECTIONS_PER_ROUTE, new ConnPerRouteBean(maxConnections));
    params.setIntParameter(ConnManagerPNames.MAX_TOTAL_CONNECTIONS, maxConnections);
    ThreadSafeClientConnManager conman = new ThreadSafeClientConnManager(params, mgr.getSchemeRegistry());
    client = new DefaultHttpClient(conman, params);

Problem

I'm setting up a Java client for a server that I poll periodically and send messages to based on particular response. The strategy I used for the class is as follows: ``` public class PollingClient { private HttpClient client = getHttpClient(); // I get a DefaultHttpClient this way so it's easier to add connection manager and strategy etc to the client later private HttpPost httpPost = getHttpPost(); // same idea, I set headers there public String poll () { List<NameValuePair> formparams = new ArrayList<NameValuePair>(); formparams.add(new BasicNameValuePair("id", someId)); String responseString = null; try { UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(formparams, "UTF-8"); httppost.setURI(polluri)); httppost.setEntity(formEntity); HttpResponse response = httpclient.execute(httppost); HttpEntity entity = response.getEntity(); if (entity != null) { responseString = EntityUtils.toString(entity); } EntityUtils.consume(entity); } catch (Exception e) { e.printStackTrace(); } } public void send(String msg) { List<NameValuePair> formparams = new ArrayList<NameValuePair>(); formparams.add(new BasicNameValuePair("msg", msg)); formparams.add(new BasicNameValuePair("id", someId)); String responseString = null; try { UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(formparams, "UTF-8"); httppost.setURI(new URI(URL + "send")); httppost.setEntity(formEntity); HttpResponse response = httpclient.execute(httppost); HttpEntity entity = response.getEntity(); if (entity != null) { responseString = EntityUtils.toString(entity); } EntityUtils.consume(entity); } catch (Exception e) { e.printStackTrace(); } } } ``` I start a thread that does the polling at around 3 secs. I can send the messages from the main thread based on the polling results. The code works but keeps giving me the following two exceptions but keeps working in between. ``` java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one. org.apache.http.NoHttpResponseException: The target server failed to respond. ``` I could just mute the exceptions but I want to know what's going on. I was unable to find any solutions via Google. I've tried to consume content, make a new `HttpPost` object in the send method, stuff like that but nothing has helped so far. What's a good strategy for a situation like this. I currently set the the `keep-alive` header in the `HttpPost` object in case that matters. Other than that I don't think there is anything I do. I think this has to do with the strategy overall. I don't want to make new object for every connection but I also don't know what level of reuse is recommended. Thanks for any help. Oh.. and this is HttpClient 4.2.2

Original source