Copying files from one directory to another

cpdirectoryfiles

I have to copy a set of files from one folder to another folder and I'm using the following command:

cp -rf `head -n 100 /home/tmp/abc.txt` /home/tmp/test/files

the file contents in abc.txt is like:

./folder1/example.tar.gz
./folder2/example.tar.gz
./folder3/example.tar.gz

But while executing the above cp command I'm getting:

cp: will not overwrite just-created `/home/tmp/test/files/example.tar.gz' with `./folder3 /example.tar.gz'
cp: will not overwrite just-created `/home/tmp/test/files/example.tar.gz' with `./folder1/example.tar.gz'

I can understand that because the name of the .gz files are the same that are showing in the error. What I want to do is to have the same folder structure inside /home/tmp/test/files as listed in abc.txt, like:

/home/tmp/test/files/folder1/example.tar.gz
/home/tmp/test/files/folder2/example.tar.gz
/home/tmp/test/files/folder3/example.tar.gz

But I'm getting only one example.tar.gz inside files folder after executing the above cp command.

So what is the way to get what I mentioned above?

Best Answer

cp won't work

Your example as it stands will not work because copy doesn't copy directory structures, it will only copy the files, hence the error message you're encountering. To do a deep copy such as this you can enlist either the tar command and use the construct tar cvf - --files-from=... | (cd /home/tmp/test/files/; tar xvf -) or you can just use rsync.

rsync

If I were you I'd use rsync to do this like so:

$ rsync -avz --files-from=abc.txt /src /home/tmp/test/files/.

If you only want the 1st 100 lines from file abc.txt you can do this:

$ rsync -avz --files-from=<(head -n 100 abc.txt) /src /home/tmp/test/files/.

Example

Sample folder data:

$ tree /home/saml/tmp/folder*
/home/saml/tmp/folder1
`-- example.tar.gz
/home/saml/tmp/folder2
`-- example.tar.gz
/home/saml/tmp/folder3
`-- example.tar.gz

Now copy the files:

$ rsync -avz --files-from=<(head -n 3 /home/saml/tmp/abc.txt) \
         /home/saml/tmp/. /home/saml/tmp/test/files/.
building file list ... done
./
folder1/
folder1/example.tar.gz
folder2/
folder2/example.tar.gz
folder3/
folder3/example.tar.gz

sent 3147093 bytes  received 81 bytes  6294348.00 bytes/sec
total size is 3145728  speedup is 1.00

Confirm they were copied:

$ tree /home/saml/tmp/test/files
/home/saml/tmp/test/files
|-- folder1
|   `-- example.tar.gz
|-- folder2
|   `-- example.tar.gz
`-- folder3
    `-- example.tar.gz

tar

If you interested here's how you do it using just tar.

$ cd /home/saml/tmp
$ tar cvf - --files-from=<(head -n 3 /home/saml/tmp/abc.txt) | (cd /home/saml/tmp/test/files/; tar xvf -)
./folder1/example.tar.gz
./folder1/example.tar.gz
./folder2/example.tar.gz
./folder2/example.tar.gz
./folder3/example.tar.gz
./folder3/example.tar.gz

Confirm that it copied:

$ tree /home/saml/tmp/test/files
/home/saml/tmp/test/files
|-- folder1
|   `-- example.tar.gz
|-- folder2
|   `-- example.tar.gz
`-- folder3
    `-- example.tar.gz

Related Solutions

Shell – Clone directory tree structure and copy files to the corresponding directories modified after a specific date

Assuming you have your desired files in a text file, you could do something like

while IFS= read -r file; do 
    echo mkdir -p ${file%/*}; 
    cp /source/"$file" /target/${file%/*}/${file##*/}; 
done < files.txt

That will read each line of your list, extract the directory and the file name, create the directory and copy the file. You will need to change source and target to the actual parent directories you are using. For example, to copy /foo/a/a.txt to /bar/a/a.txt, change source to foo and target to bar.

I can't tell from your question whether you want to copy all directories and then only specific files or if you just want the directories that will contain files. The solution above will only create the necessary directories. If you want to create all of them, use

find /source -type d -exec mkdir -p {} /target

That will create the directories. Once those are there, just copy the files:

while IFS= read -r file; do 
    cp /source/"$file" /target/"$file"
done

Update

This little script will move all the files modified after September 8. It assumes the GNU versions of find and touch. Assuming you're using Linux, that's what you will have.

#!/usr/bin/env bash    

## Create a file to compare against.
tmp=$(mktemp)
touch -d "September 8" "$tmp"

## Define the source and target parent directories
source=/path/to/source
target=/path/to/target

## move to the source directory
cd "$source"

## Find the files that were modified more recently than $tmp and copy them
find ./ -type f -newer "$tmp" -printf "%h %p\0" | 
    while IFS= read -rd '' path file; do
        mkdir -p "$target"/"$path"
        cp "$file" "$target"/"$path"
    done

Strictly speaking, you don't need the tmp file. However, this way, the same script will work tomorrow. Otherwise, if you use find's -mtime, you would have to calculate the right date every day.

Another approach would be to first find the directories, create them in the target and then copy the files:

Create all directories

 find ./ -type d -exec mkdir -p ../bar/{} \;

Find and copy the relevant files

 find ./ -type f -newer "$tmp" -exec cp {} /path/to/target/bar/{} \;

Remove any empty directories

 find ../bar/ -type d -exec rmdir {} \;