Why not just use `shell=True` in subprocess.Popen in Python?

linux, process, python, shell, subprocess

Solution

Using `shell = True` can be a security risk if your input data comes from an untrusted source. E.g. what if the content of your `mid` variable is `"/dev/null; rm -rf /"`. This does not seem to be the case in your scenario, so I would not worry too much about it.

In your code you write the result of `awk` directly to the filename in `mid`. To debug the problem, you might want to use `subprocess.check_output` and read the result from your `awk` invocation in your python program.

cmd = """sort -n -r -k5 %s |
      head -n 500|
      awk 'OFS="\t"{{if($2-{1}>0){{print $1,$2-{1},$3+{1},$4,$5}}}}'""".format(summit, top_count)

subprocess.check_call(cmd, shell=True, stdout=file)

Problem

I have a very long one-line shell command to be called by Python. The codes are like this: ``` # "first way" def run_cmd ( command ): print "Run: %s" % command subprocess.call (command, shell=True) run_cmd('''sort -n -r -k5 {3} |head -n 500|awk 'OFS="\t"{{if($2-{1}>0){{print $1,$2-{1},$3+{1},$4,$5}}}}' > {2}'''.format(top_count,extend/2,mid,summit)) ``` These codes works, but it always complains like this: ``` sort: write failed: standard output: Broken pipe sort: write error awk: (FILENAME=- FNR=132) fatal: print to "standard output" failed (Broken pipe) ``` According to a previous answer, I need to use a longer script to finish this, like: ``` # "second way" p1 = Popen("sort -n -r -k5 %s"%summit, stdout=PIPE) p2 = Popen("head -n 500", stdin=p1.stdout, stdout=PIPE) # and so on .......... ``` My questions are: (1) whether the "second way" will be slower than "first way" (2) if I have to write in "first way" anyway (because it's faster to write), how can I avoid the complain like `broken pipe` (3) what might be the most compelling reason that I shouldn't write in "first way"

Original source

Related problems