How to create dynamic Seq<Column> in java

apache-spark, dataset, java

Solution

Use it like below with list.

dataset.select(JavaConversions.asScalaBuffer(filterColumns));

or like this with seq.

dataset.select(JavaConversions.asScalaBuffer(filterColumns).seq());

Problem

I want to create a Dataset out of another one but only with some specific columns and I would like to use the `dataset.select()` function instead of creating a dynamic sql query who select from this dataset the columns I need. I have found that using `JavaConversions.asScalaBuffer(asList(new Column("value1"), new Column("value2")))` I can create a `Set<Column>` but how can I do it dynamic? because I've tried this. ``` List<Column> filterColumns = new ArrayList<>(); for (ColumnMetadata field : fields) { filterColumns.add(new Column(field.getFieldName())); } dataset.select(JavaConversions.asScalaBuffer(asList(filterColumns))) ``` But this doesn't work and the compiler said `no suitable method found for select(Buffer<List<Column>>)`

Original source