bash loop between two given dates

bash, loops

Solution

To process each file between two given date/hours, you can use the following:

#!/usr/bin/bash
#set -x

usage() {
    echo 'Usage: loopscript.sh <from> <to>'
    echo '       <from> MUST be yyyymmdd.hh or empty, meaning 00000000.00'
    echo '       <to> can be shorter and is affected by <from>'
    echo '         e.g., 20091026.00       27.01 becomes'
    echo '               20091026.00 20091027.01'
    echo '         If empty, it is set to 99999999.99'
    echo 'Arguments were:'
    echo "   '${from}'"
    echo "   '${to}'"
}

# Check parameters.

from="00000000.00"
to="99999999.99"
if [[ ! -z "$1" ]] ; then
    from=$1
fi
if [[ ! -z "$2" ]] ; then
    to=$2
fi
## Insert this to default to rest-of-day when first argument
##    but no second argument. Basically just sets second
##    argument to 23 so it will be transformed to end-of-day.
#if [[ ! -z "$1"]] ; then
#    if [[ -z "$2"]] ; then
#        to=23
#    fi
#fi

if [[ ${#from} -ne 11 || ${#to} -gt 11 ]] ; then
    usage
    exit 1
fi

# Sneaky code to modify a short "to" based on the start of "from".
# ${#from} is the length of ${from}.
# $((${#from}-${#to})) is the length difference between ${from} and ${to}
# ${from:0:$((${#from}-${#to}))} is the start of ${from} long enough
#   to make ${to} the same length.
# ${from:0:$((${#from}-${#to}))}${to} is that with ${to} appended.
# Voila! Easy, no?

if [[ ${#to} -lt ${#from} ]] ; then
    to=${from:0:$((${#from}-${#to}))}${to}
fi

# Process all files, checking that they're inside the range.

echo "From ${from} to ${to}"
for file in [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].[0-9][0-9].* ; do
    if [[ ! ( ${file:0:11} < ${from} || ${file:0:11} > ${to} ) ]] ; then
        echo "   ${file}"
    fi
done

When you create the files `20091026.00.${RANDOM}` through `20091028.23.${RANDOM}` inclusive, this is a couple of sample runs:

pax> ./loopscript.sh 20091026.07 9
From 20091026.07 to 20091026.09
   20091026.07.21772
   20091026.08.31390
   20091026.09.9214
pax> ./loopscript.sh 20091027.21 28.02
From 20091027.21 to 20091028.02
   20091027.21.22582
   20091027.22.30063
   20091027.23.29437
   20091028.00.14744
   20091028.01.6827
   20091028.02.10366
pax> ./loopscript.sh 00000000.00 99999999.99 # or just leave off the parameters.
   20091026.00.25772
   20091026.01.25964
   20091026.02.21132
   20091026.03.3116
   20091026.04.6271
   20091026.05.14870
   20091026.06.28826
   : : :
   20091028.17.20089
   20091028.18.13816
   20091028.19.7650
   20091028.20.20927
   20091028.21.13248
   20091028.22.9125
   20091028.23.7870

As you can see, the first argument must be of the correct format `yyyymmdd.hh`. The second argument can be shorter since it inherits the start of the first argument to make it the correct length.

This only attempts to process files that exist (from `ls`) and of the correct format, not every date/hour within the range. This will be more efficient if you have sparse files (including at the start and the end of the range) since it doesn't need to check that the files exist.

By the way, this is the command that created the test files, if you're interested:

pax> for dt in 20091026 20091027 20091028 ; do
         for tm in 00 01 02 ... you get the idea ... 21 22 23 ; do
             touch $dt.$tm.$RANDOM
         done
     done

Please don't type that in verbatim and then complain that it created files like:

20091026.you.12345
20091028.idea.77

I only trimmed down the line so it fits in the code width. :-)

Problem

Original source