Linux – gzip several files in different directories and copy to new directory

compressiongziplinux

I'm working with several files that are located in different directories that I need to compress into individual .gz files. I also need to move the compressed files to a single directory while leaving the originals alone.

Is there a way to do this using the gzip command and a file that contains a list of all the file paths of the files I want to compress?

Apologies if that's a bit long-winded…I'm fairly new to Linux and can't think of a more efficient way of wording this.

Best Answer

Assuming the list of files is stored in a file called filelist (exactly one file path per line) and you want to store the compressed files in zipdir, this bash script will achieve the desired results:

#!/bin/bash

while IFS= read file; do
    gzip -c "$file" > "zipdir/$(basename "$file").gz"
done < filelist

In bash/dash, you can also convert the above to a one-liner:

while IFS= read file; do gzip -c "$file" > "zipdir/$(basename "$file").gz"; done < filelist

In other shells (e.g., tcsh or zsh)

bash -c 'while IFS= read file; do gzip -c "$file" > "zipdir/$(basename "$file").gz"; done < filelist'

will do the job.

If bash isn't present, it can be substituted with dash.

How it works

  • ... < filelist redirects the contents of filelist to ....

  • while IFS= read file; do ... done goes through the lines in filelist, stores the contents of the currently processed line in the variable file and executes ....

    IFS= modifies the internal file separator. This is needed to handle multiple, leading and trailing spaces properly.

  • gzip -c "$file" > "zipdir/$(basename "$file").gz" compressed the currently processed file and stores the output in a file with the same name plus an .gz extension in the directory zipdir.

    Here basename "$file" extracts the bare filename from the file's path.

Related Question