Wget downloading tar.gz file as html

downloadtarwget

I need to download this file http://download.oracle.com/otn-pub/java/jdk/7u67-b01/jdk-7u67-linux-x64.tar.gz using wget.

I use the command

wget http://download.oracle.com/otn-pub/java/jdk/7u67-b01/jdk-7u67-linux-x64.tar.gz

The file gets downloaded but when I try to untar that file I get this

tar -zxvf jdk-7u67-linux-x64.tar.gz
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

So I use the file command to check on the file and I get this

file jdk-7u67-linux-x64.tar.gz
jdk-7u67-linux-x64.tar.gz: HTML document, ASCII text, with very long lines, with CRLF line terminators

I'm on Ubuntu 14.04. Any ideas?

Best Answer

Check the file size. You're probably actually getting back HTML. Oracle does not give you the jdk download unless you check the checkbox accepting their terms. (If you look at the reponse headers, you're probably getting back Content: text/html)

You can accept the terms by providing the following header:

--no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie"

curl

   --limit-rate <speed>
          Specify the maximum transfer rate you want curl to use. 
           This feature is useful  if you  have a limited pipe and 
           you'd like your transfer not to use your entire bandwidth.

          The given speed is measured in bytes/second, unless a suffix 
          is appended.  Appending  'k'  or 'K' will count the number
          as kilobytes, 'm' or M' makes it megabytes, while 'g' or 'G' 
          makes it gigabytes. Examples: 200K, 3m and 1G.

E.g: curl --limit-rate 423K

wget

   --limit-rate=amount
       Limit the download speed to amount bytes per second.  Amount may
       be expressed in bytes, kilobytes with the k suffix, or 
       megabytes with the m suffix.  For example, --limit-rate=20k will limit 
       the retrieval rate to 20KB/s.  This is useful when, for
       whatever reason, you don't want Wget to consume 
       the entire available bandwidth.

E.g: wget --limit-rate=423k

Downloading files using wget

I think your ? gets interpreted by shell (Correction by vinc17: more likely, it's the & which gets interpreted).

Just try with simple quotes around your URL:

wget 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

Note that the file you are requesting is a .tar file but the above command will save it as index.html?acc=GSE48191&format=file. To have it correctly named, you can either rename it to .tar:

mv 'index.html?acc=GSE48191&format=file' GSE4819.tar

Or you can give the name as an option to wget:

wget -O GSE48191.tar 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

The above command will save the downloaded file as GSE48191.tar directly.

Best Answer

Related Solutions

Networking – How to Throttle Download Speed with Wget or Curl

curl

wget

Downloading files using wget

Related Question