Linux – Problems reading IBM 3592 JB tape

If you specify a block size (512 bytes) of less than the block size of the disk (often 4096 bytes, but nowadays maybe more), the block will be partially written, so that the contents of the rest of the block must be preserved before writing.

This is because disk blocks cannot be written to with only 512 bytes, but you have to write a full block at once (4096 or larger). When you write this (4096) amount or more, there is no partial write, so it does not have to read.

Ok, I think I've worked this out.

TL;DR

Use dd with a large block size to read from the tape instead:

dd if=/dev/nst0 bs=1M | tar tvf -

Background

When you write to tapes, the data is written in units called blocks. These are like sectors on a hard disk. Where hard disk blocks were fixed at 512-bytes for many years and only recently moved to 4096-byte blocks, tape blocks can be set to any size you like.

The block size you wish to use is set with the setblk subcommand in mt-st:

mt-st -f /dev/nst0 setblk 512    # Use 512-byte blocks
mt-st -f /dev/nst0 setblk 64k    # Use 65536-byte blocks

When you issue a read operation to the drive, it will return data in block-sized chunks. You can't read half a block - the smallest amount of data you can read from a tape is one block, which of course could be any number of actual bytes depending on what the block size is.

This means if the program you are using supplies a 16kB memory buffer, you will be able to read up to 32 blocks at a time from the tape with 512-byte blocks as these fit exactly in the 16kB buffer. However you will not be able to read anything from the tape with 64kB blocks, because you can't fit even one of them into the 16kB buffer, and remember you can't read anything less than one whole block at a time.

Should you attempt to do this, by using a buffer that's too small for one block, the driver (in this case the st SCSI tape driver) will return a memory allocation error code to advise you that your read buffer is too small to hold even a single block.

To further complicate matters, some tape drives (apparently the LTO ones I am using) also support variable-sized blocks. This means the block size is determined by the size of each write operation and each block can be a different size to the last.

This mode is set with a block size of zero:

mt-st -f /dev/nst0 setblk 0    # Use variable-sized blocks

This is also the default option as - presumably, I am guessing here - it wastes less space with an incorrectly configured program. If, for example, you had set 4k blocks but your program only wrote data in units of 512 bytes at a time, there is a risk that each 512-byte chunk of data would take up 4k on the tape.

Cause

If you now put everything together, you will realise that a tape can hypothetically have a 512-byte block followed by a 64kB block. If the program is reading the tape with a 16kB buffer, it will successfully read the first block, but then when it tries to read more, it won't be able to fit the following 64kB block in its buffer so the driver will return an error.

This explains why I was getting Cannot allocate memory errors most of the time, and occasionally I was able to get tar to extract the first few files but then I got the error again. I had not set the block size with mt-st so it had defaulted to variable-sized blocks when the tape was written, and now tar was using too small a buffer to read in some of those blocks.

tar has a couple of options for setting its own internal block sizes, namely --blocking-factor, --read-full-records, and --record-size, however these only work if tar is used to directly read and write to the tape.

Because I wrote to the tape through the mbuffer program to reduce tape shoe-shining, the block size in the tar archive no longer matched the block size on the tape. This meant --blocking-factor had little effect - it would allow the first block on the tape to be read, which includes a header telling tar what the blocking factor is supposed to be, wherein it switches to that and ignores the value given on the command line. This means the second and subsequent blocks can no longer be read!

Solution

The solution is to use another program to read from the tape - one that can have the read buffer size set to a value large enough to hold the biggest block we are likely to see.

dd works for this, and in a pinch this works:

dd if=/dev/nst0 bs=256k | tar tvf -

You may need to increase 256k if your tape has larger blocks on it, but this worked for me. 1M also works fine so it doesn't appear to matter if the value is too large, within reason.