perl text::csv - filtering specific columns in a csv document and discarding others

perl

Solution

No, not a specific function in Text::CSV. I would do something like this:

use Text::CSV;

my $file = "foo.csv";
my $pattern = "cpu.usage.mhz.average.*";
open(F, $file) or die "Unable to open $file: $!\n";

my $lineCount = 0;
my %desiredColumns;
my %columnContents;

while(<F>) {
  $lineCount++;
  my $csv = Text::CSV->new();
  my $status = $csv->parse($_); # should really check this!
  my @fields = $csv->fields();
  my $colCount = 0;

  if ($lineCount == 1) {
    # Let's look at the column headings.
    foreach my $field (@fields) {
      $colCount++;
      if ($field =~ m/$pattern/) {
        # This heading matches, save the column #.
        $desiredColumns{$colCount} = 1;
      }
    }
  }
  else {
    # Not the header row.  Parse the body of the file.
    foreach my $field (@fields) {
      $colCount++;
      if (exists $desiredColumns{$colCount}) {
        # This is one of the desired columns.
        # Do whatever you want to do with this column!
        push(@{$columnContents{$colCount}}, $field);
      }
    }
  }
}
close(F);

foreach my $key (sort keys %columnContents) {
  print "Column $key: " . join(",", @{$columnContents{$key}}) . "\n\n";
}

Hope that helps! I'm sure someone can write that in a Perl one-liner, but that's easier (for me) to read...

Problem

I would like to filter out particular columns with a regex and discard others. For example, if I had the following column names: date mem_total cpu.usagemhz.average_0 cpu.usagemhz.average_1 cpu.usagemhz.average_2 I would like to capture only columns that begin with "cpu.usage.mhz.average" Is their a particular function of text::csv that will help me do a quick check of the column names? Thanks! JD * Update ** I tried jimtut answer and it is extremely close to what I am looking for. Thanks Again Everyone! Here is the code from jimtut with one small edit on the print statement at the bottom. I added the print $colCount just to see what was going on with the data; ``` use Text::CSV; my $file = "foo.csv"; my $pattern = ".*In"; open(F, $file) or warn "Warning! Unable to open $file\n"; my $lineCount = 0; my %desiredColumns; while(<F>) { $lineCount++; my $csv = Text::CSV->new(); my $status = $csv->parse($_); # should really check this! my @fields = $csv->fields(); my $colCount = 0; if ($lineCount == 1) { # Let's look at the column headings. foreach my $field (@fields) { $colCount++; if ($field =~ m/$pattern/) { # This heading matches, save the column #. $desiredColumns{$colCount} = 1; } } } else { # Not the header row. Parse the body of the file. foreach my $field (@fields) { $colCount++; if (exists $desiredColumns{$colCount}) { # This is one of the desired columns. # Do whatever you want to do with this column! print "$colCount\t$field\n"; } } } } close(F); ``` Here is the results ``` colCount | $field 12 565 13 73 14 36 15 32 16 127 17 40 18 32 19 42 20 171 12 464 13 62 14 32 15 24 16 109 17 21 18 19 19 39 20 150 12 515 13 76 14 28 15 30 16 119 17 15 18 25 19 46 20 169 12 500 13 71 14 30 15 28 16 111 17 20 18 18 19 40 20 167 ``` I would like to add this data to individual arrays or hashes. what do you think? something like... foreach column { check to see if a hash already exists with that column number. If not then create hash. } Then go through each field and add the field data to the appropriate hash. Do you think this is the right way to go about solving this?

Original source