I'd like to download a file using the wget utility. I followed the instruction on "How do I use wget/curl to download from a site I am logged into?" and the download process works; however, the saved cookies expires after a certain time, so I'm unable to continue downloading the file the following day.
Here's the URL I'm trying to download the a file from:
https://frbservices.org/EPaymentsDirectory/FedACHdir.txt
The download page requires that I click the "Agree" button before I can proceed with the download.
Is there a way to include the "Agree" submission with the wget utility?
Thank you.
Best Answer
You can submit a form using
wget
, using the--post-data
option. First, look at the form that page uses:Typically, the value of the
action
attribute is used to get the target URL, which becomeshttps://frbservices.org/EPaymentsDirectory/submitAgreement
. Thename
s of the form elements become the parameters. Then you need to save the cookies that you get from accepting the agreement. From this SO post, we can construct the command needed:Then, we can use these cookies to download the file: