scp files in a certain order using ls
ls, scp
Solution
To send files from a local machine (e.g. your laptop) to a remote (e.g. your calculation server), you can use Merlin2011's clever solution:
- Go into the folder in your local machine where you want to copy files from.
- Execute the `scp` command, assuming you have an access key for the remote server: `scp -r $(ls -rt) user@foo.bar:/where/you/want/them/.`
Note: if you don't have a public access key it may be better to do something similar using `tar`, then send the tar file, i.e. `tar -zcvf files.tar.gz $(ls -rt)`, and then send that tar file on its own using `scp`.
But to do it the other way around you might not be able to run the `scp` command directly from the remote server to send files to, say, your laptop. Instead, you may need to, let's say bring files into your laptop. My brute-force solution is:
- In the remote server, `cd` into the folder you want to copy files from.
- Create a list of the files in the order you want. For example, for reverse order of creation (most recent copied last): `ls -rt > ../filenames.txt`
- Now you need to add the path to each file name. Before you go up to the directory where the list is, print the path using `pwd`. Now do go up: `cd ..`
- You now need to add this path to each file name in the list. There are many ways to do this, here's one using awk: `cat filenames.txt | awk '{print "path/to/files/" $0}' > delete_me.txt`
- You need the filenames to be in the same line, separated by a space, so change newlines to spaces: `tr '\n' ' ' < delete_me.txt > filenames.txt`
- Get filenames.txt to the local server, and put it in the folder where you want to copy the files into.
- The scp run would be: `scp -r user@foo.bar:"$(cat filenames.txt)" .`
Similarly, this assumes you have a private access key, otherwise it's much simpler to `tar` the file in the remote, and bring that.
Problem
Whenever I try to SCP files (in bash), they end up in a seemingly random(?) order. I've found a simple but not-very-elegant way of keeping a desired order, described below. Is there a clever way of doing it? Edit: deleted my early solution from here, cleaned, adapted using other suggestions, and added as an answer below.