Processing Json through Pig Scripts

Question

I have currently started to work with JSON files and process data using PIG scripts. I am using Pig version 0.9.3.I have come across PiggyBank which i thought will be useful to load and process json file in PIG scripts. I have built piggybank.jar through ANT. Later, I have compiled the Java File and updated the piggybank.jar. Was trying to run the given example json file. I have written a simple PIGSCRIPT and the respective JSON as follows. ``` REGISTER piggybank.jar a = LOAD 'file3.json' using org.apache.pig.piggybank.storage.JsonLoader() AS (json:map[]); b = foreach a GENERATE flatten(json#'menu') AS menu; c = foreach b generate flatten(menu#'popup') as popup; d = foreach c generate flatten(popup#'menuitem') as menu; e = foreach d generate flatten(menu#'value') as val; DUMP e; file3.json { "menu" : { "id" : "file", "value" : "File", "popup": { "menuitem" : [ {"value" : "New", "onclick": "CreateNewDoc()"}, {"value" : "Open", "onclick": "OpenDoc()"}, {"value" : "Close", "onclick": "Cl…

Accepted Answer

You can handle nested json loading with Twitter's Elephant Bird: https://github.com/kevinweil/elephant-bird

```
a = LOAD 'file3.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad')
```

This will parse the JSON into a map http://pig.apache.org/docs/r0.11.1/basic.html#map-schema the JSONArray gets parsed into a DataBag of maps.

Processing Json through Pig Scripts

Solution

Problem

Related problems