Passing a filename with an apostrophe into scp using python

Question

I'm trying to write a python script to copy files from a remote server to a local directory via scp. Because I'm running this on an OpenELEC distribution (minimal HTPC linux distro, read-only filesystem except for userhome makes it impractical to install python ssh module), I'm doing this ugly and just passing the filename to the scp command via os.system. ``` SCPCopy = "scp -c blowfish -C user@host:"" + pipes.quote(file) + "" /storage/downloads/incoming/" SCPCopy = SCPCopy.replace('
','') os.system(SCPCopy) ``` This works, except for filenames containing an apostrophe. Below is an example of what gets passed to os.system in a file with an apostrophe: ``` scp -c blowfish -C user@host:"'/media/sdi1/home/data/bob'"'"'s file.avi'" /storage/downloads/incoming/ ``` And the error: ``` sh: -c: line 0: unexpected EOF while looking for matching `'' sh: -c: line 1: syntax error: unexpected end of file ``` It looks pipes.quote(x) is escaping the apostrophe (as it should), but obviously the sy…

Accepted Answer

As `scp` is based on `SSH`, the filenames you give to it are subject to shell escaping on the remote side as well. Thus you need to escape twice.

A correctly escaped cmdline for the shell:

```
scp -c blowfish -C user@host:"\"/media/sdi1/home/data/bob's file\"" /storage/.../
```

To make a python string, we have to add one more level of escaping. To stay sane, we could use triple-quotes:

```
"""scp -c blowfish -C user@host:"\"/media/sdi1/home/data/bob's file\"" /storage/.../"""
```

If you do it programmatically (e.g. using the deprecated `pipes.quote`), then don't touch the filename at all (in your example above, you added apostrophes around the filename).

```
fp = "/media/sdi1/home/data/bob's file.avi"
fp = "user@host:" + pipes.quote(pipes.quote(fp))

cmdline = "scp -c blowfish -C " + fp + " /storage/downloads/incoming/"
os.system(cmdline)
```

This is admittedly confusing. For a simple model, the whole point of `pipes.quote` is to escape the input so that the input will be parsed by the shell as exactly one word, which is equal to the input.

The following is a more generally correct way (and yields the same result):

```
fp = "/media/sdi1/home/data/bob's file.avi"
# the filepath argument escaped for ssh/scp on the remote side
fp = pipes.quote(fp)
commandargs = ["scp", "-c", "blowfish", "-C", "user@host:"+fp, "/storage/downloads/incoming/"]
# escape all words for the local shell, and then concatenate space-separated
cmdline = " ".join(map(pipes.quote, commandargs))
os.system(cmdline)
```

It expresses more clearly the intent: Controlling what words exactly the shell will parse.

But why start with a shell in the first place? We don't need one and can save the escaping on the local side. To spawn a process with our args, directly, use commands from the `os.exec*` family.

```
fp = pipes.quote("/media/sdi1/home/data/bob's file.avi")
commandargs = ["scp", "-c", "blowfish", "-C", "user@host:"+fp, "/storage/downloads/incoming/"]
if os.fork() == 0:
    os.execvp("scp", commandargs)
```

Passing a filename with an apostrophe into scp using python

Solution

Problem