FreeBSD Backup – How to Write Sequentially Several Tar’s to a Tape

backupfreebsdtapetar

I am trying to write several directories to a tape. Each directory with one tar command.

So I have the following sample file/directory structure:

user@host1:~/temp/original % find .
.
./foo1
./foo1/foo1.a
./foo1/foo1.b
./foo1/foo1.c
./foo1/foo1.1
./foo2
./foo2/foo2.a
./foo2/foo2.b
./foo2/foo2.c
./foo2/foo2.2
./foo3
./foo3/foo3.a
./foo3/foo3.b
./foo3/foo3.c
./foo3/foo3.3

I rewind and erase the tape, which I expect it is like using a blank tape.

user@host1:~/temp/original % mt -f /dev/sa0 rewind
user@host1:~/temp/original % mt -f /dev/sa0 erase
user@host1:~/temp/original % mt -f /dev/sa0 rewind
user@host1:~/temp/original % mt -f /dev/sa0 status
Drive: sa0: <SEAGATE DAT    9SP40-000 912L> Serial Number: HN0948V
---------------------------------
Mode      Density              Blocksize      bpi      Compression
Current:  0x24:DDS-2           variable       61000    enabled (DCLZ)
---------------------------------
Current Driver State: at rest.
---------------------------------
Partition:   0      Calc File Number:   0     Calc Record Number: 0
Residual:    0  Reported File Number:   0 Reported Record Number: 0
Flags: BOP

Then I want to write three tar files (I think they are called files when stored to tape) with three tar commands. One command for each directory (foo1, foo2 and foo3). So I do:

user@host1:~/temp/original % tar cvf /dev/nsa0 foo1
a foo1
a foo1/foo1.a
a foo1/foo1.b
a foo1/foo1.c
a foo1/foo1.1
user@host1:~/temp/original % tar cvf /dev/nsa0 foo2
a foo2
a foo2/foo2.a
a foo2/foo2.b
a foo2/foo2.c
a foo2/foo2.2
user@host1:~/temp/original % tar cvf /dev/nsa0 foo3
a foo3
a foo3/foo3.a
a foo3/foo3.b
a foo3/foo3.c
a foo3/foo3.3

As I have been using /dev/nsa0 I expect to have three tar files stored in the tape.

Now I want to recover the three files from the tape into another directory I do:

user@host1:~/temp/original % cd ../backup/
user@host1:~/temp/backup % mt -f /dev/sa0 rewind
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo1/
x foo1/foo1.a
x foo1/foo1.b
x foo1/foo1.c
x foo1/foo1.1
user@host1:~/temp/backup % tar xvf /dev/nsa0
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo2/
x foo2/foo2.a
x foo2/foo2.b
x foo2/foo2.c
x foo2/foo2.2
user@host1:~/temp/backup % tar xvf /dev/nsa0
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo3/
x foo3/foo3.a
x foo3/foo3.b
x foo3/foo3.c
x foo3/foo3.3
user@host1:~/temp/backup % mt -f /dev/nsa0 status
Drive: sa0: <SEAGATE DAT    9SP40-000 912L> Serial Number: HN0948V
---------------------------------
Mode      Density              Blocksize      bpi      Compression
Current:  0x24:DDS-2           variable       61000    enabled (DCLZ)
---------------------------------
Current Driver State: at rest.
---------------------------------
Partition:   0      Calc File Number:   2     Calc Record Number: 1
Residual:    0  Reported File Number:   2 Reported Record Number: 5
Flags: None

Why do I have to type twice tar xvf /dev/nsa0 to extract foo2 and foo3?

If I try to add another directory at the end of the tape I do:

user@host1:~/temp/original % mt -f /dev/nsa0 eom
user@host1:~/temp/original % tar cvf /dev/nsa0 foo4
a foo4
a foo4/foo4.a
a foo4/foo4.b
a foo4/foo4.c
a foo4/foo4.4
user@host1:~/temp/original % cd ..
user@host1:~/temp % cd backup/
user@host1:~/temp/backup % mt -f /dev/nsa0 rewind
user@host1:~/temp/backup % mt -f /dev/nsa0 fsf 3
user@host1:~/temp/backup % tar xvf /dev/nsa0
user@host1:~/temp/backup %

Why foo4 is not extracted?

As an additional test I eject the tape, reinsert it and try to extract the four directories, this is what I have to do:

user@host1:~/temp/backup % mt -f /dev/nsa0 offline
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo1/
x foo1/foo1.a
x foo1/foo1.b
x foo1/foo1.c
x foo1/foo1.1
user@host1:~/temp/backup % tar xvf /dev/nsa0
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo2/
x foo2/foo2.a
x foo2/foo2.b
x foo2/foo2.c
x foo2/foo2.2
user@host1:~/temp/backup % tar xvf /dev/nsa0
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo3/
x foo3/foo3.a
x foo3/foo3.b
x foo3/foo3.c
x foo3/foo3.3
user@host1:~/temp/backup % tar xvf /dev/nsa0
user@host1:~/temp/backup % tar xvf /dev/nsa0
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo4/
x foo4/foo4.a
x foo4/foo4.b
x foo4/foo4.c
x foo4/foo4.4

Why do I have to repeat the tar commands, twice in the case of foo2 and foo3 and three times in the case of foo4?

I am using FreeBSD12.1 and an IBM DDS4 (STD2401LW / Tc4200-236) SCSI Tape Drive.

EDIT>
Following schily's answer, I can get the tar files extracted in order. The only remaining issue would be understanding why mt eom to later add foo4 tar file still requires two mt fsf instead of just one.

After reinserting the tape:

user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo1/
x foo1/foo1.a
x foo1/foo1.b
x foo1/foo1.c
x foo1/foo1.1
user@host1:~/temp/backup % mt fsf
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo2/
x foo2/foo2.a
x foo2/foo2.b
x foo2/foo2.c
x foo2/foo2.2
user@host1:~/temp/backup % mt fsf
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo3/
x foo3/foo3.a
x foo3/foo3.b
x foo3/foo3.c
x foo3/foo3.3
user@host1:~/temp/backup % mt fsf
user@host1:~/temp/backup % tar xvf /dev/nsa0
user@host1:~/temp/backup % tar xvf /dev/nsa0
x foo4/
x foo4/foo4.a
x foo4/foo4.b
x foo4/foo4.c
x foo4/foo4.4
user@host1:~/temp/backup %

EDIT>
This is what mt status return right at the position that allows to extract foo4. The commands are executed right after inserting the tape:

user@host1:~/temp/backup % rm -rf *
user@host1:~/temp/backup % mt status
Drive: sa0: <SEAGATE DAT    9SP40-000 912L> Serial Number: HN0948V
---------------------------------
Mode      Density              Blocksize      bpi      Compression
Current:  0x24:DDS-2           variable       61000    enabled (DCLZ)
---------------------------------
Current Driver State: at rest.
---------------------------------
Partition:   0      Calc File Number:   0     Calc Record Number: 0
Residual:    0  Reported File Number:   0 Reported Record Number: 0
Flags: BOP
user@host1:~/temp/backup % echo $TAPE
/dev/nsa0
user@host1:~/temp/backup % mt fsf 4
user@host1:~/temp/backup % mt status
Drive: sa0: <SEAGATE DAT    9SP40-000 912L> Serial Number: HN0948V
---------------------------------
Mode      Density              Blocksize      bpi      Compression
Current:  0x24:DDS-2           variable       61000    enabled (DCLZ)
---------------------------------
Current Driver State: at rest.
---------------------------------
Partition:   0      Calc File Number:   4     Calc Record Number: 0
Residual:    0  Reported File Number:   4 Reported Record Number: 7
Flags: None
user@host1:~/temp/backup % tar xv
x foo4/
x foo4/foo4.a
x foo4/foo4.b
x foo4/foo4.c
x foo4/foo4.4
user@host1:~/temp/backup %

Best Answer

The behavior is related to the EOF handling of the tape driver.

This handling differs between operating systems and it may help to read the related Solaris man page:

http://schillix.sourceforge.net/man/man7i/mtio.7i.html

that explains a difference between the Solaris handling and the old BSD behavior.

From this explanation, I would expect the old BSD behavior to cause a read after an EOF situation to skip the file mark and to return the first record from the next file on tape. This seems to be what you expect.

It seems that the observed behavior on BSD is between the documented SVr4 behavior and the old BSD behavior, but I guess that there is a way to make things work on both Solaris and current BSD:

  • call tar to read the first tape file

  • after that, the tape is positioned at the end of the first tape file, which is just before the file mark...

  • call mt fsf to skip the file mark

  • call tar to read the next file on tape.

From the rest of the discussion, it seems that FreeBSD writes an additional filemark, when mt rewind is called after a write operation has been applied.

The command mt eom will position the tape after the final double filemark and when another write operation takes place, this happens after the double filemark resultng in an empty tape file before that final write.

A tape with three files looks this way:

data1 FILEMARK data2 FILEMARK data3 FILEMARK FILEMARK

If you like to append a fourth tape file, you need to call:

mt fsf 3

to position the tape after the thirf filemark. If you then start writing, this overwrites the fourth filemark and if you then rewind again, you have this tape layout:

data1 FILEMARK data2 FILEMARK data3 FILEMARK data4 FILEMARK FILEMARK
Related Question