Fetching all Git repositories in the background

crongit

I'm thinking about setting up a cronjob for fetching all my repositories every once in a while, to have the current status ready in case I'm offline. Something like the following (wrapped for better readability):

find $HOME -name .git -type d -printf "%h\0" |
  parallel --gnu -0 --delay 0.2 --jobs 100 --progress \
  'cd {}; git fetch --all --quiet'

I don't really care what happens if the fetch fails — it might succeed the next time. Perhaps error output could be logged. My questions are:

  • What if the background process fetches into a Git repository while I'm committing to it?
  • Can you recommend other switches to parallel to make this really fail-safe?

Best Answer

I've been fetching my local Git repos in the background for two years now, without any sign of trouble. Currently, crontab contains something like

savelog -n -c 400 ~/log/git-fetch.log
find ~/git -type d -execdir [ -d '{}/.git' ]  \; -print -prune |
    parallel --gnu --keep-order \
    "date; echo {}; cd {}; git fetch --all --verbose" \
    >> ~/log/git-fetch.log 2>&1

(but in one line).

Related Question