Bash – How to output what wget retrieves to stdout and suppress all other wget messaging

bashshell-scriptstdoutwget

I'm using bash shell on Amazon Linux. I have a command in a shell script

wget -O - "http://localhost:8088/subco/books/$e_id/segments/$segment_id?product=$product_id&audience=teacher" > /dev/null

Reading this — https://superuser.com/questions/321240/how-do-you-redirect-wget-to-standard-out/321241, I was led to believe I could output the result of wget (what it retrieves) to my screen and suppress all other output. However instead, what I get is

--2018-05-29 18:39:49--  http://localhost:8088/subco/books/C2644BB08F394E209A26175BD2C89F5A/segments/C2F62E7002964DD396E381DB331129A4?product=D399B9C5F6204EDE80A002930CC0D02F&audience=teacher
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8088... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘STDOUT’

How do I output the result of wget to the screen and suppress all other information (e.g. like the "HTTP request sent, awaiting response… 200 OK" part)?

Best Answer

If you want to use wget, the use -qO -

wget -qO - google.com

Or you could use curl with no options

curl google.com

And if you want to go a little further and parse results etc, the package BeautifulSoup for Python is great.

Related Solutions

Bash – wget a file, logging the output and showing the output on prompt

You use the lovely tee command to do this:

wget --no-check-certificate --auth-no-challenge --http-user=$jen_uname --http-password=$jen_psswd 2>&1 | tee -a wget_log

THe 2>&1 means that STDERR goes to the same place as STDOUT, and they're both piped to tee. The -a means append. tee will then send the output both to wget_log and to STDOUT.

Bash – How to Direct Output to Pipe and Stdout

Your assumption:

fortune | tee >(?stdout?) | pbcopy

won't work because the fortune output will be written to standard out twice, so you will double the output to pbcopy.

In OSX (and other systems support /dev/std{out,err,in}), you can check it:

$ echo 1 | tee /dev/stdout | sed 's/1/2/'
2
2

output 2 twice instead of 1 and 2. tee outputs twice to stdout, and tee process's stdout is redirected to sed by the pipe, so all these outputs run through sed and you see double 2 here.

You must use other file descriptors, example standard error through /dev/stderr:

$ echo 1 | tee /dev/stderr | sed 's/1/2/'
1
2

or use tty to get the connected pseudo terminal:

$ echo 1 | tee "$(tty)" | sed 's/1/2/'
1
2

With zsh and multios option set, you don't need tee at all:

$ echo 1 >/dev/stderr | sed 's/1/2/'
1
2

Best Answer

Related Solutions

Bash – wget a file, logging the output and showing the output on prompt

Bash – How to Direct Output to Pipe and Stdout

Related Question