Using tar
in multi-volume mode relies on a ENOSPC
error to detect the end of the first tape and prompt the user for the next tape.
To simulate this behaviour consider the following example by writing to /dev/full
tar -cvf - --multi-volume . > /dev/full
as expected results in
[...]
Prepare volume #2 for ‘-’ and hit return:
A problem arises when piping the output of tar
through an encyption program like aespipe
or gpg
tar -cvf - --multi-volume . | gpg -c --batch -q --passphrase 123 > /dev/full
which causes gpg
to exit with code 2
gpg: [stdout]: write error: No space left on device
gpg: [stdout]: write error: No space left on device
gpg: filter_flush failed on close: No space left on device
The ENOSPC is obviously not propagated to tar, which isn't made aware of the specific errno. Is there a way to catch the error from gpg
and "re-raise" the ENOSPC error to tar
with a bash script?
For example, using tar with a named pipe results in a broken pipe once gpg
fails and tar subsequently exists with SIGPIPE 141 — however ENOSPC
still has to be signaled to tar in some way instead of the broken pipe error.
I would like to avoid the workaround of specifying a fixed tape size.
I am also aware of using mbuffer
to handle tape spanning, which is undesireable because tapes can not be extracted individually.
EDIT: I just realized this is going to be a lot more complicated, as the data that has already left tar and was in the buffer when ENOSPC was encountered is most likely lost. Though most tape driver implementations allow another write operation after that, gpg and aespipe include no retry logic to save the data in the buffer.
EDIT 2: Further research shows that star
on FreeBSD with the -compress-program
option to perform the encryption in conjunction with -multivol
and new-volume-script=...
raises the error
star: Operation not permitted. Cannot lock fifo memory.
star: Can only compress files
when writing to a device instead of a file. So that's a dead end too.
Best Answer
It's not possible to propagate write errors back through a pipeline
And even if it were possible with some kind of hack, the pipes are buffering and by the time the pipe reader tries to "signal" the pipe writer, the latter could've already written the data which is causing the error further down the line, already got a successful status (>0) and updated its state accordingly. For it to work, the writing process would have to go back in time. On top of that, the pipe reader itself may do its own buffering and state keeping which would go out of sync.
The only way out is for
tar
to call the encryption routines directly, instead of passing the data through some kind of channel. Instead of modifying its source code and recompiling it, that could be done by monkey/live patching it with aLD_PRELOAD
hack which overrides thewrite()
library function and processes the data before passing it to the originalwrite()
.How to simulate
ENOSPC
with aLD_PRELOAD
hackThis will cause an write to fd 1 (stdout) to fail with
ENOSPC
as soon as it tries to write more than 40960 bytes to it, after which it resets the counter and succeeds again, etc.If you want it to work with
tar -cf filename
, instead oftar -cf -
, you should probably change thefd == 1
test tofd != 2
.