Linux – Bash script not waiting for wget to finish download

bashlinuxwget

I am using the following bash script to merge various gzip archives together:

wget -O bluelvl1.gz http://list.iblocklist.com/?list=ydxerpxkpcfqjaybcssw&fileformat=p2p&archiveformat=gz
wget -O bluelvl2.gz http://list.iblocklist.com/?list=gyisgnzbhppbvsphucsw&fileformat=p2p&archiveformat=gz
wget -O badpeer.gz http://list.iblocklist.com/?list=cwworuawihqvocglcoss&fileformat=p2p&archiveformat=gz
wget -O microsoft.gz http://list.iblocklist.com/?list=xshktygkujudfnjfioro&fileformat=p2p&archiveformat=gz
wget -O unallocated.gz http://list.iblocklist.com/?list=gihxqmhyunbxhbmgqrla&fileformat=p2p&archiveformat=gz
cat bluelvl1.gz bluelvl2.gz badpeer.gz microsoft.gz unallocated.gz > blocklist.p2p.gz

The problem I'm having is that the "cat" command is creating "blocklist.p2p.gz" with nothing in it. I believe that the problem is caused by Bash not waiting for wget to complete the download.

I believe this because if I copy-paste each line into the command prompt one at a time (and personally wait for the download to finish), Cat creates a merged archive as I expected. If I copy all six lines above from the script then paste the whole lot into the same command prompt, I see the same problem the script is having.

So is there a way to make Bash wait for wget to finish? Is Bash supposed to wait by default and something is causing this to not occur? I would greatly appreciate a solution.

EDIT:
As per the comments below, the correct script to use is:

wget -O bluelvl1.gz "http://list.iblocklist.com/?list=ydxerpxkpcfqjaybcssw&fileformat=p2p&archiveformat=gz"
wget -O bluelvl2.gz "http://list.iblocklist.com/?list=gyisgnzbhppbvsphucsw&fileformat=p2p&archiveformat=gz"
wget -O badpeer.gz "http://list.iblocklist.com/?list=cwworuawihqvocglcoss&fileformat=p2p&archiveformat=gz"
wget -O microsoft.gz "http://list.iblocklist.com/?list=xshktygkujudfnjfioro&fileformat=p2p&archiveformat=gz"
wget -O unallocated.gz "http://list.iblocklist.com/?list=gihxqmhyunbxhbmgqrla&fileformat=p2p&archiveformat=gz"
cat bluelvl1.gz bluelvl2.gz badpeer.gz microsoft.gz unallocated.gz > blocklist.p2p.gz

Best Answer

Bash is seeing the & symbols in the URL as a terminator of the command and executing it as a background process. If you enter any command in bash and append a & to it, it will run it in the background and return control to the calling script or the terminal immediately. The fields between them are being executed as commands itself.

To keep bash from interpreting the URL this way, enclose the entire thing in quotes. This will make it pass the whole URL as a string parameter, and it will run as a foreground process.

Related Question