I have a link which indludes 12 files and I want to download all of them using just one command in Wget
. And I use Cygwin as a terminal to runWget
.
The link is https://hydro1.gesdisc.eosdis.nasa.gov/data/NLDAS/NLDAS_FORA0125_M.002/1985/ and I only want all .grb
files under this link. I have tired the following code, but it just downloads all .xml
files.
I find some advices from https://disc.sci.gsfc.nasa.gov/recipes/?q=recipes/How-to-Download-Data-Files-from-HTTP-Service-with-wget ,but I still cannot solve the problem. Thanks for any help.
wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies -r -c -nH -nd -np -A nc4,xml "https://hydro1.gesdisc.eosdis.nasa.gov/data/NLDAS/NLDAS_FORA0125_M.002/1985/"
Thanks to all the help, I have fixed it but then it comes to a more complicated question. It requires username and password, which should be the case, but I do not know how to Create a .netrc file
and Create a cookie file
as recommended by https://disc.sci.gsfc.nasa.gov/recipes/?q=recipes/How-to-Download-Data-Files-from-HTTP-Service-with-wget.
Here is a biref description of what I need to do.
To run wget, you need to set up .netrc and create a cookie file:
Create a .netrc file in your home directory.
a. cd ~ or cd $HOME
b. touch .netrc
c. echo "machine urs.earthdata.nasa.gov login <uid> password <password>" >> .netrc
where <uid> is your user name and <password> is your URS password
d. chmod 0600 .netrc (so only you can access it)
Create a cookie file. This file will be used to persist sessions across calls to Wget or Curl. For example:
a. cd ~ or cd $HOME
b. touch .urs_cookies
I wonder how I can do this in Wget
from Cygwin for Windows.
Best Answer
Only looking at your example, it shows
-A nc4,xml
which would explain only downloading .xml files, there must not be any nc4 files at that link.Anyway, here's what
man wget
says about-A
:So for only grb files, try using
-A grb
as in:After Edits:
Username & Password should be fairly easy, try adding these:
again from
man wget
. And read about the--save-cookies file
and ``--read-cookies file` options in there, it may be tricky to get cookies saved from a web browser, if they don't work in wget.