Store and retrieve string arrays in HBase

hbase, serialization

Solution

You may apply the following method to get back the `ArrayWritable` (taken from my earlier answer, see here) .

public static <T extends Writable> T asWritable(byte[] bytes, Class<T> clazz)
            throws IOException {
        T result = null;
        DataInputStream dataIn = null;
        try {
            result = clazz.newInstance();
            ByteArrayInputStream in = new ByteArrayInputStream(bytes);
            dataIn = new DataInputStream(in);
            result.readFields(dataIn);
        }
        catch (InstantiationException e) {
            // should not happen
            assert false;
        }
        catch (IllegalAccessException e) {
            // should not happen
            assert false;
        }
        finally {
            IOUtils.closeQuietly(dataIn);
        }
        return result;
    }

This method just deserializes the byte array to the correct object type, based on the provided class type token. E.g: Let's assume you have a custom ArrayWritable:

public class TextArrayWritable extends ArrayWritable {
    public TextArrayWritable() {
      super(Text.class);
    }
}

Now you issue a single HBase get:

...
Get get = new Get(row);
Result result = htable.get(get);
byte[] value = result.getValue(family, qualifier);
TextArrayWritable tawReturned = asWritable(value, TextArrayWritable.class);
Text[] texts = (Text[]) tawReturned.toArray();
for (Text t : texts) {
  System.out.print(t + " ");
}
...

Note: You may have already found the readCompressedStringArray() and writeCompressedStringArray() methods in WritableUtils which seem to be suitable if you have your own String array-backed Writable class. Before using them, I'd warn you that these can cause serious performance hit due to the overhead caused by the gzip compression/decompression.

Problem

I've read this answer (How to store complex objects into hadoop Hbase?) regarding the storing of string arrays with HBase. There it is said to use the `ArrayWritable` Class to serialize the array. With `WritableUtils.toByteArray(Writable ... writable)` I'll get a `byte[]` which I can store in HBase. When I now try to retrieve the rows again, I get a `byte[]` which I have somehow to transform back again into an `ArrayWritable`. But I don't find a way to do this. Maybe you know an answer or am I doing fundamentally wrong serializing my `String[]`?

Original source