UNIX – Remove Files Older Than 5 Days with Date in Filename

bashrmsed

I want to delete log files which are older than 5 days from a directory. But deletion should not be based on the timestamp of file. It should be based on the name of file. For Example todays date is 07/05/2012 and the directory contains 10 files of names like ABC_20120430.log, ABC_20120429.log, ABC_20120502.log, ABC_20120320.log etc. I want to be able to remove the files by extracting the date from the name of the file.

Best Answer

Based on date from filename:

THRESHOLD=$(date -d "5 days ago" +%Y%m%d)
ls -1 ABC_????????.log | 
  sed 'h;s/[_.]/ /g;G;s/\n/ /' | 
  while read A DATE B FILE
  do 
     [[ $DATE -le $THRESHOLD ]] && rm -v $FILE
  done

Related Solutions

Shell Script – Cron Job to Delete Files Older Than 3 Days

This is easy enough (although note that this goes by a modification time more than 3 days ago since a creation time is only available on certain filesystems with special tools):

find /a/b/c/1 /a/b/c/2 -type f -mtime +3 #-delete

Remove the # before the -delete once you are sure that it is finding the files you want to remove.

To have it run by cron, I would probably just create an executable script (add a shebang - #!bin/sh to the top line of the file and make executable with chmod a+x), then put it in an appropriate cron directory like /etc/cron.daily or /etc/cron.weekly. Provided of course that you do not need a more specific schedule and that these directories exist on your distro.

Update

As noted below, the -delete option for find isn't very portable. A POSIX compatible approach would be:

find /a/b/c/1 /a/b/c/2 -type f -mtime +3 #-exec rm {} +

Again remove the # when you are sure you have the right files.

Update2

To quote from Stéphane Chazelas comment below:

Note that -exec rm {} + has race condition vulnerabilities which -delete (where available) doesn't have. So don't use it on directories that are writeable by others. Some finds also have a -execdir that mitigates against those vulnerabilities.

Shell – How to delete the files in one folder which are more than 60 days old in UNIX

OK, I have remade this script, and by sorting it backwards it looks like it should work. It compares the year and month to the previous one, and if it is lower it should be the last entry for that month.

#!/bin/bash

#the tac reverses the listing, so we go from newest to oldest, vital for our logic below
FILES=`ls | tac`

#create a cutoff date by taking current date minus our 60 day limit
CUTOFFDATE=`date --date="60 days ago" +%Y%m%d`

#we are setting this value to month 13 of the current year 
LASTYEARMONTH=`date +%Y`13

for file in $FILES; do

    #get datestamp
    FILEDATE=`expr match "$file" '.*\(20[0-9][0-9][0-9][0-9][0-9][0-9]\)'`

    #get year and month, ie 201410
    THISYEARMONTH=${FILEDATE:0:6}

    if [ ! -z $FILEDATE ] && [ $THISYEARMONTH -lt $LASTYEARMONTH ]; then

            echo "$file IS THE LAST FILE OF THE MONTH. NOT DELETING"

    else

            #now we see if the file is older than 60 days (ie, has a LOWER unix timestamp than our cutoff)
            if [ ! -z $FILEDATE ] && [ $FILEDATE -lt $CUTOFFDATE ]; then

                    echo "$file has a file date of $FILEDATE which is older than 60 days."

                    #uncomment this next line to remove
                    #rm $file
            fi
    fi
    LASTYEARMONTH=$THISYEARMONTH
done