HttpClient not Serializable Exception
apache-httpclient-4.x, apache-storm
Solution
I got my answer from the Google Forum for the Storm project. Here is the link to that post:
https://groups.google.com/forum/#!topic/storm-user/vdfamKeR9Lk
The fix is to instantiate the HttpClient instance in the prepare method of the Bolt class.
Problem
I am trying to implement a basic 1 spout - 1 bolt Storm Topology. I have a Storm Bolt to make an HTTP Request using the Apache HttpClient (4.3.1). But, I get the following exception when I run it: ``` [main] ERROR org.apache.zookeeper.server.NIOServerCnxn - Thread Thread[main,5,main] died java.lang.RuntimeException: java.io.NotSerializableException: org.apache.http.impl.client.InternalHttpClient at backtype.storm.utils.Utils.serialize(Utils.java:56) at backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:89) at app.storm.StormTopology.main(StormTopology.java:26) Caused by: java.io.NotSerializableException: org.apache.http.impl.client.InternalHttpClient at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at backtype.storm.utils.Utils.serialize(Utils.java:52) ... 2 more ``` Anyone else seen this? I know Storm itself uses the HTTP Client (4.1.1) internally. I tried to replace the internal library with the 4.3.1 version and I got another error: ``` Exception in thread "main" java.lang.NoSuchFieldError: INSTANCE at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<init>(DefaultHttpRequestWriterFactory.java:52) at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<init>(DefaultHttpRequestWriterFactory.java:56) at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<clinit>(DefaultHttpRequestWriterFactory.java:46) at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<init>(ManagedHttpClientConnectionFactory.java:72) at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<init>(ManagedHttpClientConnectionFactory.java:84) at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<clinit>(ManagedHttpClientConnectionFactory.java:59) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$InternalConnectionFactory.<init>(PoolingHttpClientConnectionManager.java:487) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:147) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:136) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:112) at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:727) at org.apache.http.impl.client.HttpClients.createDefault(HttpClients.java:58) at app.storm.bolts.DataFetcherBolt.<init>(DataFetcherBolt.java:34) at app.storm.StormTopology.main(StormTopology.java:18) ```