subprocess.check_output with grep command fails when grep finds no matches

grep, python, subprocess

Solution

I just want subprocess.check_output to return an empty string if grep doesn't find anything.

Well, too bad. `grep` considers no matches to be failure, and the whole point of the `check` in `check_output` is to check for failure, so you're explicitly asking to do things this way. Here are the relevant docs:

If the return code was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and any output in the output attribute.

And for `grep`:

The following exit values shall be returned:
  0 One or more lines were selected.
  1 No lines were selected.
  >1 An error occurred.

So, if you want to treat "no lines" as success, but actual errors as errors, you have to handle that `1` value differently than other non-zero values. And `check_output` has no idea that you want to do that.

So, either you have to handle the `CalledProcessError`, or you have to do your own checking. In other words, either this:

try:
    tyrs = subprocess.check_output('grep "^A" %s | grep TYR' % pocket_location, shell = True).split('\n')
except subprocess.CalledProcessError as e:
    if e.returncode > 1:
        raise
    tyrs = []

… or this:

p = subprocess.Popen('grep "^A" %s | grep TYR' % pocket_location, shell=True,
                     stdout=subprocess.PIPE)
output, _ = p.communicate()
if p.returncode == 1: # no matches found 
    tyrs = []
elif p.returncode == 0: # matches found
    tyrs = output.split('\n')
else:
    # error, do something with it

Problem

I'm trying to search a text file and retrieve lines containing a specific set of words. This is the code I'm using: ``` tyrs = subprocess.check_output('grep "^A" %s | grep TYR' % pocket_location, shell = True).split('\n') ``` This works fine when the file contains at least one line that grep identifies. But when grep doesn't identify any lines, grep returns exit status 1 and I get the following error: ``` Traceback (most recent call last): File "../../Python_scripts/cbs_wrapper2.py", line 324, in <module> tyrs = subprocess.check_output('grep "^ATOM" %s | grep TYR' % pocket_location, shell = True).split('\n') File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 544, in check_output raise CalledProcessError(retcode, cmd, output=output) subprocess.CalledProcessError: Command 'grep "^ATOM" cbsPrediction_files/1u9c_clean/1u9c_clean_fpocket_out/pockets/pocket0_atm.pdb | grep TYR' returned non-zero exit status 1 ``` How can I avoid this issue? I just want `subprocess.check_output` to return an empty string if grep doesn't find anything. Thanks

Original source

Related problems