Loop through CSV file and create new csv file with while read?

Question

I have ``` while read $field1 $field2 $field3 $field4 do $trimmed=$field2 | sed 's/ *$//g' echo "$trimmed","$field3" >> new.csv done < "$FEEDS"/"$DLFILE" ``` Now the problem is with `read` I can't make it split fields csv style, can I? See the input csv format below. I need to get columns 3 and 4 out, stripping the padding from col 2, and I don't need the quotes. Csv format with col numbers: 12 24(")25(,)26(")/27(Field2values) 42(")/43(,)/44(Field3 decimal values) "Field1_constant_value","Field2values ",Field3,Field4 Field1 is constant and irrelevant. Data is quoted, goes from 2-23 inside the quotes. Field2 fixed with from cols 27-41 inside quotes, with the data at the left and padded by spaces on the right. Field3 is a decimal number with 1,2, or 3 digits before the decimal and 2 after, no padding. Starts at col 74. Field4 is a date and I don't much care about it right now.

Accepted Answer

Yes, you can use read; all you've got to do is reset the environment variable `IFS` -- Internal Field Separator --, so that it won't split lines by its current value (default to whitespace), but by your own delimiter.

Considering an input file "a.csv", with the given contents:

1,2,3,4

2,3,4,5

6,3,2,1

You can do this:

```
IFS=','
while read f1 f2 f3 f4; do
    echo "fields[$f1 $f2 $f3 $f4]"
done < a.csv
```

And the output is:

fields[1 2 3 4]

fields[2 3 4 5]

fields[6 3 2 1]

Loop through CSV file and create new csv file with while read?

Solution

Problem