Is There Any Way to Pipe Data from Perl to a Unix Command Line Utility

bash, perl, unix

Solution

If I am reading your question correctly, you are wanting to spawn a process and be able to both write to its stdin and read from its stdout. If that is the case, then IPC::Open2 is exactly what you need. (Also see IPC::Open3 you also need to read from the process' stderr.)

Here is some sample code. I have marked the areas you will have to change.

#!/usr/bin/perl

use strict;
use warnings;

use IPC::Open2;

# Sample data -- ignore this.
my @words = qw(the quick brown fox jumped over the lazy dog);

# Automatically reap child processes.  This is important when forking.
$SIG{'CHLD'} = 'IGNORE';

# Spawn the external process here.  Change this to the process you need.
open2(*READER, *WRITER, "wc -c") or die "wc -c: $!";

# Fork into a child process.  The child process will write the data, while the
# parent process reads data back from the process.  We need to fork in case
# the process' output buffer fills up and it hangs waiting for someone to read
# its output.  This could cause a deadlock.
my $pid;
defined($pid = fork()) or die "fork: $!";

if (!$pid) {
    # This is the child.

    # Close handle to process' stdout; the child doesn't need it.
    close READER;

    # Write out some data.  Change this to print out your data.
    print WRITER $words[rand(@words)], " " for (1..100000);

    # Then close the handle to the process' stdin.
    close WRITER;
    # Terminate the child.
    exit;
}

# Parent closes its handle to the process' stdin immediately!  As long as one
# process has an open handle, the program on the receiving end of the data will
# never see EOF and may continue waiting.
close WRITER;

# Read in data from the process.  Change this to whatever you need to do to
# process the incoming data.
print "READ: $_" while (<READER>);

# Close the handle to the process' stdin.  After this call, the process should
# be finished executing and will terminate on its own.
close READER;

Problem

I have a command line utility from a third party (it's big and written in Java) that I've been using to help me process some data. This utility expects information in a line delimited file and then outputs processed data to STDOUT. In my testing phases, I was fine with writing some Perl to create a file full of information to be processed and then sending that file to this third party utility, but as I'm nearing putting this code in production, I'd really prefer to just pipe data to this utility directly instead of first writing that data to a file as this would save me the overhead of having to write unneeded information to disk. I recently asked on this board how I could do this in Unix, but have since realized that it would be incredibly more convenient to actually run it directly out of a Perl module. Perhaps something like: `system(bin/someapp do-action --option1 some_value --input $piped_in_data)` Currently I call the utility as follows: `bin/someapp do-action --option1 some_value --input some_file` Basically, what I want is to write all my data either to a variable or to STDOUT and then to pipe it to the Java app through a system call in the SAME Perl script or module. This would make my code a lot more fluid. Without it, I'd wind up needing to write a Perl script which calls a bash file half way through which in turn would need to call another Perl script to prep data. If at all possible I'd love to just stay in Perl the whole way through. Any ideas?

Original source