Cassandra resets connection under heavy insert workload
cassandra
Solution
I've found the cause: batch mutation size was exceeding the frame size of TFramedTransport. There might two possible solutions: either to increase "thrift_framed_transport_size_in_mb" configuration property in cassandra.yaml, or to use smaller batch size.
Problem
I have a cassandra cluster of 8 nodes, cassandra 1.0.8. I'm trying to do lots of small inserts in a loop with batch_mutate(). After some time (~200K inserts) server resets connection with following exception: ``` org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147) at org.apache.thrift.transport.TFramedTransport.flush(TFramedTransport.java:157) at org.apache.cassandra.thrift.Cassandra$Client.send_batch_mutate(Cassandra.java:998) at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:986) at org.scale7.cassandra.pelops.Mutator$1.execute(Mutator.java:46) at org.scale7.cassandra.pelops.Mutator$1.execute(Mutator.java:42) at org.scale7.cassandra.pelops.Operand.tryOperation(Operand.java:56) at org.scale7.cassandra.pelops.Mutator.execute(Mutator.java:51) ... Caused by: java.net.SocketException: Connection reset at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96) at java.net.SocketOutputStream.write(SocketOutputStream.java:136) at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145) ... 25 more ``` Otherwise than it, the cluster works fine. Server logs are clean. What might be causing this problem? Thanks!