Process substitution
bash, r
Solution
I have written this general purpose function for opening a file connection in my own scripts:
OpenRead <- function(arg) {
if (arg %in% c("-", "/dev/stdin")) {
file("stdin", open = "r")
} else if (grepl("^/dev/fd/", arg)) {
fifo(arg, open = "r")
} else {
file(arg, open = "r")
}
}
In your code, replace `file` with `file <- OpenRead(file)` and it should handle all of the below:
./mean.R test
./mean.R <(cat test)
cat test | ./mean.R -
cat test | ./foo.R /dev/stdin
Problem
I've given a look around about what puzzles me and I only found this: Do some programs not accept process substitution for input files? which is partially helping, but I really would like to understand the full story. I noticed that some of my R scripts give different (ie. wrong) results when I use process substitution. I tried to pinpoint the problem with a test case: This script: ``` #!/usr/bin/Rscript args <- commandArgs(TRUE) file <-args[1] cat(file) cat("\n") data <- read.table(file, header=F) cat(mean(data$V1)) cat("\n") ``` with an input file generated in this way: ``` $ for i in `seq 1 10`; do echo $i >> p; done $ for i in `seq 1 500`; do cat p >> test; done ``` leads me to this: ``` $ ./mean.R test test 5.5 $ ./mean.R <(cat test) /dev/fd/63 5.501476 ``` Further tests reveal that some lines are lost...but I would like to understand why. Does read.table (scan gives the same results) uses seek? Ps. with a smaller test file (100) an error is reported: ``` $./mean.R <(cat test3) /dev/fd/63 Error in read.table(file, header = F) : no lines available in input Execution halted ``` Add #1: with a modified script that uses scan the results are the same.