Linux – Bash script to find folders containing a specific subfolder, and then copying to new directory while preserving parent name

bashlinuxscript

I'll try to be as specific as possible. I have a Packages folder containing subfolders with various names, but in each of those subfolders there will always be a dist folder. I want to find the folders containing dist, and then move/copy only dist and its parent folder to a new directory.

Example:

Current path structure is as below:

~/Documents/Packages/core-types/dist
~/Documents/Packages/graphql-utils/dist
~/Documents/Packages/manuscript-assets/dist

NOTE: core-types, graphql, etc., all contain subfolders in addition to dist; otherwise this would be much more straightforward…

I am trying set it up so that when these paths are migrated to a new directory,
say, ~/Documents/Artifacts/, it will look like:

~/Documents/Artifacts/core-types/dist
~/Documents/Artifacts/graphql-utils/dist
~/Documents/Artifacts/manuscript-assets/dist

Here's the script I hacked together in case someone can offer guidance:

#!/bin/bash
find ~/Documents/Packages -name dist -print \
| sort -u \
      | while read olddir
        do 
          moveto="~/Documents/Artifacts/$(echo \"$olddir\" | cut -d\/ -f 6)"
          mv "$olddir" "$moveto"
        done

There has to be a better way to accomplish this…

Best Answer

I don’t mean to be flippant, but you have two issues:

  1. Identifying the subdirectories named dist, and
  2. moving or copying them into the Artifacts directory.

This is quite a bit easier if we use relative pathnames.  If you absolutely need to use absolute pathnames (har har), say so.

Here are two ways to do step 1 (identifying the subdirectories named dist):

find

#!/bin/sh
cd ~/Documents/Packages &&
find . -name dist -type d -exec sh -c 'for d do parent="${d%/*}"; \
                                                mkdir -p "../Artifacts/$parent"; \
                                                mv "$d" "../Artifacts/$parent"; done' \
                                                sh {} +

(You can put the find command all on one line if you want.)  This goes into ~/Documents/Packages, so we can use relative pathnames.  A fairly straightforward use of the -name and -type tests of find to identify (sub)directories named dist.  Use the -exec action to pass their names to a miniature shell script.

The script loops over its arguments.  For each one, it identifies the parent directory by removing / and a pathname level from the right.  For example, the parent of ./core-types/dist is ./core-types.  (Remember; we are in ~/Documents/Packages and looking at ., so we are seeing relative pathnames.)  I could have said parent="${d%/dist}", but I wanted to keep it generic.  Create the destination/parent directory — for example, ../Artifacts/./core-types.  (The extra ./ is ignored; /./ acts like /, so this creates ../Artifacts/core-types, which is ~/Documents/Packages/../Artifacts/core-types, which is ~/Documents/Artifacts/core-types.)  We then move the dist directory into that directory.

bash Pathname Expansion

This depends on a feature of bash that is not present in all shells.  (For that matter, it might not be in all versions of bash.)

#!/bin/bash
cd ~/Documents/Packages && shopt -s globstar &&
for d in **/dist/; do d="${d%/}"; parent="${d%/*}";
                                  mkdir -p "../Artifacts/$parent";
                                  mv "$d" "../Artifacts/$parent"; done

Set the globstar shell option (shopt), which causes ** to do a recursive search.  Then loop through **/dist/.  If we said **/dist, that would find all files and directories named dist; adding the / at the end constrains the search to (sub)directories only.  For each one, execute a bunch of commands.

**/dist/ gives us the directory names in the form core-types/dist/ (with the / at the end, as we specified).  d="${d%/}" strips off that trailing slash.  The rest of the code is the same.

Note: this will not look under top-level directories whose names begin with . (e.g., ~/Documents/Packages/.memories/dist) unless you add the dotglob option.

Step 2

If you want to copy instead of move, just change mv to cp -R in either of the above commands.

Related Question