How to process input and output streams in Steel Bank Common Lisp?

common-lisp, lisp, sbcl

Solution

I got a working answer from Raymond Toy on comp.lang.lisp. His solution was for CMUCL, but it worked with the essentially identical `RUN-PROGRAM` function on the closely related SBCL, and with minor changes it will work on CCL as well, because CCL's `RUN-PROGRAM` is basically a clone of the one from CMUCL/SBCL.

The secret, as it were, is to set up the `ls` process first, and then provide its output stream to the `grep` process as input, like so:

(defun piping-test2 () 
  (let ((ls-process (run-program "/bin/ls" '() 
                                 :wait nil 
                                 :output :stream))) 
    (unwind-protect 
        (with-open-stream (s (process-output ls-process)) 
          (let ((grep-process (run-program "/usr/bin/grep" '("lisp") 
                                          :input s 
                                          :output :stream))) 
            (when grep-process 
              (unwind-protect 
                  (with-open-stream (o (process-output grep-process)) 
                    (loop 
                       :for line := (read-line o nil nil) 
                       :while line 
                       :collect line)) 
                (process-close grep-process))))) 
      (when ls-process (process-close ls-process)))))

I also experimented with omitting the `:WAIT NIL` argument from the `RUN-PROGRAM` call for `ls`, and it worked just as well.

Problem

I'm trying to figure out how to use the output stream of one program I start with `RUN-PROGRAM` so it can be used as the input of another program started with `RUN-PROGRAM` (i.e., the moral and perhaps literal equivalent of piping). I've tried using a number of combinations of the `:INPUT`, `:OUTPUT` and `:WAIT` keyword arguments, but nothing I've hit upon has been productive so far. Any tips would be helpful; for example, how would I go about doing something like `ls | grep lisp` from the shell? One of my attempts is ``` (defun piping-test () (let ((grep-process (run-program "/usr/bin/grep" '("lisp") :input :stream :output :stream))) (unwind-protect (with-open-stream (s (process-input grep-process)) (let ((ls-process (run-program "/bin/ls" '() :output s))) (when ls-process (unwind-protect (with-open-stream (o (process-output grep-process)) (loop :for line := (read-line o nil nil) :while line :collect line)) (process-close ls-process))))) (when grep-process (process-close grep-process))))) ``` Running this in a SLIME REPL causes everything to hang until I break with `C-c C-c`, so it's pretty obviously not the right thing, but I'm not sure how to change it so it is the right thing. EDIT: Adding `:WAIT NIL` to both `RUN-PROGRAM` invocations, or to only the invocation for `grep`, doesn't do the trick. In that case, the function will hang, and breaking with `C-c C-c` gets a stack trace indicating that there's a local function (defined via `FLET`) called `SB-UNIX:SELECT` that has hung.

Original source