Remove all files older than X days, but keep at least the Y youngest

bash, command-line, find, shell

Solution

Use `find` to get all files that are old enough to delete, filter out the `$KEEP` youngest with `tail`, then pass the rest to `xargs`.

find ${DB_DUMP_DIR} -type f -printf '%T@ %p\n' -mmin +$RETENTION |
  sort -nr | tail -n +$KEEP |
  xargs -r echo

Replace `echo` with `rm` if the reported list of files is the list you want to remove.

(I assume none of the dump files have newlines in their names.)

Problem

I have a script that removes DB dumps that are older than say X=21 days from a backup dir: ``` DB_DUMP_DIR=/var/backups/dbs RETENTION=$((21*24*60)) # 3 weeks find ${DB_DUMP_DIR} -type f -mmin +${RETENTION} -delete ``` But if for whatever reason the DB dump jobs fails to complete for a while, all dumps will eventually be thrown away. So as a safeguard i want to keep at least the youngest Y=7 dumps, even it all or some of them are older than 21 days. I look for something that is more elegant than this spaghetti: ``` DB_DUMP_DIR=/var/backups/dbs RETENTION=$((21*24*60)) # 3 weeks KEEP=7 find ${DB_DUMP_DIR} -type f -printf '%T@ %p\n' | \ # list all dumps with epoch sort -n | \ # sort by epoch, oldest 1st head --lines=-${KEEP} |\ # Remove youngest/bottom 7 dumps while read date filename ; do # loop through the rest find $filename -mmin +${RETENTION} -delete # delete if older than 21 days done ``` (This snippet might have minor bugs - Ignore them. It's to illustrate what i can come up with myself, and why i don't like it) Edit: The find option "-mtime" is one-off: "-mtime +21" means actually "at least 22 days old". That always confused me, so i use -mmin instead. Still one-off, but only a minute.

Original source