Shell – Is it possible to truncate a file (in place, same inode) at the beginning

filesshelltailtruncate

It is possible to remove trailing bytes of a file without writting to a new file (> newfile) and moving it back (mv newfile file). That is done with truncate:

truncate -s -1 file

It is possible to remove leading bytes but by moving it around (which changes the inode) (for some versions of tail):

tail -c +1 file > newfile ; mv newfile file

So: How to do that without moving files around?
Ideally, like truncate, only a few bytes would need to be changed even for very big files.

note: sed -i will change the file inode, so, even if it may be useful, is not an answer to this question IMO.

Best Answer

With ksh93:

tail -c+2 < file 1<>; file

(where <>; is a ksh93 specific variant of the standard <> operator that truncates the file in the end if the command being redirected was successful).

Would remove the first byte (by writing the rest of the file over itself and truncate at the end).

Same can be done with sh with:

{
  tail -c+2 < file &&
    perl -e 'truncate STDOUT, tell STDOUT'
} 1<> file

Note that it would unsparse sparse files (you could still re-dig holes afterwards with fallocate -d though).

Upon read/write errors, tail would likely bail out leaving the file partly overwritten (so for instance, abcdefgh could end up as bcddefgh if it fails after rewriting bcd). You could adapt the above so that it reports the writing offset upon error, so you know how to recover the data. Still with ksh93:

unset -v offset
{ tail -c+2 < file || false >#((offset=CUR)); } 1<>; file

After which if $offset is set, it contains the amount of data that was successfully written.

On Linux (since 3.15) and on ext4 or xfs file systems, one can collapse ranges or bytes of size and offset that are a multiple of the filesystem block size with the fallocate() system call or fallocate utility.

So for instance

fallocate -c -l 8192 file

Would remove the first 8192 bytes of the file (assuming a FS with a block size that is a divisor of 8192) without having to rewrite the rest of the file. But that's of no use if you want to remove a section that is not a multiple of the FS block size.

Related Question