collect_set on array type with group by in hive

hadoop, hive

Solution

As the error message says `Only primitive type arguments are accepted but array was passed as parameter 1.`, you need to convert the array to String before using it.

You can achieve the same using `explode()` function. Something like:

select 
  id, 
  collect_set(tokens) 
FROM
  ts LATERAL VIEW explode(values) x AS tokens
group by
  id;

Problem

I have below table which contains duplicates on id with array of values for each id and I want to find out unique values for each id, how to do that? ``` CREATE TABLE test( id string, values array<string>) ``` When I run below command, it throws error as `collect_set` only support primitive type values. ``` select id, collect_set(values) from ts group by id; ``` ERROR: FAILED: UDFArgumentTypeException Only primitive type arguments are accepted but array was passed as parameter 1.

Original source