Recovering data written to file that moved

data-recoveryfilesgziplogsrm

I have a Python daemon running on CentOS that opens a file at the beginning of a session, and keeps writing to it.

A cronjob however, gzipped the file that was being written to, and so the file moved from log.txt to log.txt.gz. The daemon however kept writing to log.txt. The daemon then has been stopped and closed the file descriptor to log.txt.

Is there any way to recover the data that was written by the daemon to log.txt after the file was moved to log.txt.gz?

Best Answer

AFAICT, no. The problem is that the gzip process will create a new file and will free the previous (the unzipped) one, including a removal from the directory. If no other hard-link in the filesystem is pointing to the file it will get lost once the last file descriptor refering to it is closed.

For the future you'd be advised to synchronize the access to that file ressource instead of letting two processes simultaneously access the file (for writing and resp. deleting).

Another option is to let gzip create a zipped copy. But then you'd have a race condition where not all written contents in the file might get into the gz file.

Related Solutions

Replace file with hard link to /dev/null

# cp -a /dev/null log.txt

This copies your null device with the right major and minor dev numbers to log.txt so you have another null.

Devices are not known by name at all in the kernel but rather by their major and minor numbers. Since I don't know what OS you have I found it convenient to just copy the numbers from where we already know they are. If you make it with the wrong major and minor numbers, you would most likely have made some other device, perhaps a disk or something else you don't want writing to.

gzip Error – Unexpected End of File, How to Read File Anyway

Apart from the very end of the file, you will be able to see the uncompressed data with zcat (or gzip -dc, or gunzip -c):

zcat log.gz | tail

zcat log.gz | less

zless log.gz

gzip will do buffering for obvious reasons (it needs to compress the data in chunks), so even though the program may have outputted some data, that data may not yet be in the log.gz file.

You may also store the uncompressed log with

zcat log.gz > log

... but that would be silly since there's obviously a reason why you compress the output in the first place.

Best Answer

Related Solutions

Replace file with hard link to /dev/null

gzip Error – Unexpected End of File, How to Read File Anyway

Related Question