Is there a way to see any tar progress per file

progresstarunix

I have a couple of big files that I would like to compress. I can do this with for example

tar cvfj big-files.tar.bz2 folder-with-big-files

The problem is that I can't see any progress, so I don't have a clue how long it will take or anything like that. Using v I can at least see when each file is completed, but when the files are few and large this isn't the most helpful.

Is there a way I can get tar to show more detailed progress? Like a percentage done or a progress bar or estimated time left or something. Either for each single file or all of them or both.

Best Answer

I prefer oneliners like this:

tar cf - /folder-with-big-files -P | pv -s $(du -sb /folder-with-big-files | awk '{print $1}') | gzip > big-files.tar.gz

It will have output like this:

4.69GB 0:04:50 [16.3MB/s] [==========================>        ] 78% ETA 0:01:21

For OSX (from Kenji's answer)

tar cf - /folder-with-big-files -P | pv -s $(($(du -sk /folder-with-big-files | awk '{print $1}') * 1024)) | gzip > big-files.tar.gz

Explanation:

  • tar tarball tool
  • cf create file
  • - use stdout instead of a file (to be able to pipe the output to the next command)
  • /folder-with-big-files The input folder to zip
  • -P use absolute paths (not necessary, see comments)

pipe to

  • pv progress monitor tool
  • -s use the following size as the total data size to transfer (for % calculation)
    • $(...) evaluate the expression
    • du -sb /folder-with-big-files disk usage summarize in one line with bytes. Returns eg 8367213097 folder-with-big-files
    • pipe (|) to awk '{print $1}' which returns only the first part of the du output (the bytes, removing the foldername)

pipe to

  • gzip gzip compression tool
  • big-files.tar.gz output file name
Related Question