Linux – How to download all images from a website (not webpage) using the terminal

downloadimageslinuxterminal

I want a command that I type a URL, for example photos.tumblr.com, and it download all photos on this site in a folder, but not only images of the site's homepage. This command need to download the images from all parts of the site, such as photos.tumblr.com/ph1/1.png / photos.tumblr.com/ph3/4.jpg.

Please show me an example using this url: http://neverending-fairytale.tumblr.com/ and test it before answer the question

Best Answer

You can use:

wget -r -A=.jpg,.png http://website.com

With this command you will get all the JPG and PNG files, but you can get banned from the site.

So, if you use:

wget --random-wait --limit-rate=100k -r -A=.jpg,.png http://website.com

You'll get your images waiting a random time between downloads and setting a speed limit.

Related Solutions

Wget – download all files but not preceding folders

I think what you are looking for is the --cut-dirs option. Used in conjunction with the -nH (no hostname) option, you can specify exactly which level of directory you want to appear in your local output. As an example, I have a .pkg download that I want to write to my local directory, and I don't want all of the parent tree to be included, just the subdirectories. In this case, my starting point to just get the .pkg name as the parent directory is 5 levels down:

wget -np -nH --cut-dirs 5 -r http://www.myhost.org/pub/downloads/My_Drivers/OS_10_5_x/Letter_Format/driver_C123_105.pkg

What you will see, then, is the name driver_C123_105.pkg in your current directory.

% ls -lt | head
drwxr-xr-x   12 rob  rob        408 Feb 22 12:54 driver_C123_105.pkg
-rw-------@   1 rob  rob          0 Feb 16 15:59 1kPSXcUj.pdf.part
-rw-------@   1 rob  rob        842 Feb  3 14:47 WcUuL69s.jnlp.part

[...etc...]

% find driver_C123_105.pkg
driver_C123_105.pkg
driver_C123_105.pkg/Contents
driver_C123_105.pkg/Contents/Archive.bom
driver_C123_105.pkg/Contents/Archive.pax.gz
driver_C123_105.pkg/Contents/index.html
driver_C123_105.pkg/Contents/index.html?C=D;O=A
driver_C123_105.pkg/Contents/index.html?C=D;O=D
driver_C123_105.pkg/Contents/index.html?C=M;O=A
driver_C123_105.pkg/Contents/index.html?C=M;O=D
driver_C123_105.pkg/Contents/index.html?C=N;O=A
driver_C123_105.pkg/Contents/index.html?C=N;O=D
driver_C123_105.pkg/Contents/index.html?C=S;O=A
driver_C123_105.pkg/Contents/index.html?C=S;O=D
driver_C123_105.pkg/Contents/Info.plist
driver_C123_105.pkg/Contents/PkgInfo
driver_C123_105.pkg/Contents/Resources
driver_C123_105.pkg/Contents/Resources/background.jpg

[.....etc....]

You can direct this output to go elsewhere with the -P option.

Best Answer

Related Solutions

Wget – download all files but not preceding folders

Related Question