Logrotate Issue – Open File Not Truncating

logrotatelogs

I have a system that is acting simultaneously as a test and demo platform. I'm recording a massive amount of data output by my systems that I wouldn't be recording when this is in deployment and because of that my log files are filling up very rapidly and I eventually run out of partition space on my harddrive.

I tried logrotate, but it seems to only work if my program isn't running. When the programs are down it properly truncates the file, records a compressed version, and discards the rest. However, when my programs that populate the log file are up and running it will properly create the compressed file, but it won't truncate the active file. I'm trying to figure out how to make it truncate.

I'm writing to the log files with a simple redirect. ProgramA>logA, programB>logB etc. My progam is outputting every time I receive an input vector, so it's outputting very rapidly. My assumption is that the logrotate is failing to truncate because of the constant writes to the field, but can anyone confirm rather that is the reason?

Where does logrotate keep it's error file anyways?

Also: I want to modify the behavior of logrotate. If it activates and sees a large, say 100 MB, log file I would like it to create a copy of only the last 1 MB of the file (discarding the older 99 MB of content of the file) before truncating the original log file. I need to keep recent data, but I don't care much about anything that is very old. Can anyone tell me how to do this?

Best Answer

As promised, a "close enough" to ring buffer program, in a few lines of Perl.

To use it, pass it the size of the log files you want, in bytes (give or take one line), and two or more log files. It'll switch between the log files every X bytes. I'm sure there is already a program out there to do this, but I couldn't find it with a quick search. E.g.,

$ your-app | log-splitter 1048576 /var/log/your-app/log1 /var/log/your-app/log2

assuming, of course, that the file log-splitter is executable, in $PATH, and contains:

#!/usr/bin/perl
use warnings qw(all);
use strict;
use autodie;
use Carp;
use Fcntl qw(SEEK_SET);
use IO::Handle;

# yes, we truncate all logs at start. Sorry.
@ARGV >= 2 or croak "Usage: $0 log-size log-file...";
my ($max_size, @lognames) = @ARGV;
my @logs = map {
    open my $fh, '+>', $_;
    $fh->autoflush(1);
    $fh;
} @lognames;

my $current_size = $max_size;
my $current_log  = $#logs;
while (defined(my $line = <STDIN>)) {
    if ($current_size >= $max_size) {
        $current_log  = ($current_log + 1) % @logs;
        $current_size = 0;
        seek($logs[$current_log], 0, SEEK_SET);
        truncate($logs[$current_log], 0);
    }

    $current_size += length($line);
    print {$logs[$current_log]} $line;
}

close($_) foreach @logs;

To clarify any licensing questions on that code:

CC0
To the extent possible under law, Anthony DeRobertis has waived all copyright and related or neighboring rights to the above program (log-splitter). This work is published from: United States.

Related Question