Does a wget showing full request headers, exist on mac? The GNU wget on mac doesn't but the GNU wget on windows (Wget 1.19)
The example output that I want, is what I get on GNU Wget in Windows (This is from the windows binaries listed https://www.gnu.org/software/wget/faq.html#download )
C:\blah>wget -d www.google.com
DEBUG output created by Wget 1.19.4 on mingw32.
Reading HSTS entries from http://www.google.com/.wget-hsts
URI encoding = 'CP1252'
converted 'http://www.google.com' (CP1252) -> 'http://www.google.com' (UTF-8)
Converted file name 'index.html' (UTF-8) -> 'index.html' (CP1252)
--2019-04-10 16:25:48-- http://www.google.com/
Resolving www.google.com (www.google.com)... seconds 0.00, 216.58.204.4
Caching www.google.com => 216.58.204.4
Connecting to www.google.com (www.google.com)|216.58.204.4|:80... seconds 0.00, connected.
Created socket 3.
Releasing 0x00000000002fb820 (new refcount 1).
---request begin---
GET / HTTP/1.1
User-Agent: Wget/1.19.4 (mingw32)
Accept: */*
Accept-Encoding: identity
Host: www.google.com
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
Date: Wed, 10 Apr 2019 15:25:54 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2019-04-10-15; expires=Fri, 10-May-2019 15:25:54 GMT; path=/; domain=.google.com
Set-Cookie: NID=181=eSn6z-R-x0B1ewMV1qhIqoLLQbZFwjf_dq3GeNk8nCpMmG5UIdhoirK8fMGswItGholH9cs-7XSJwAmyJCat0Z24Uff_o9IxGDruiq0ds35p_idqiyY8awPhNQqhrMLe9JODOppcGIMaXSEQjAKlnvhEYa6zuE-e1BKunVlrze
s; expires=Thu, 10-Oct-2019 15:25:54 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked
---response end---
200 OK
cdm: 2
cdm: 3
cdm: 4
cdm: 5
cdm: 6
cdm: 7
cdm: 8
Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-05-10 16:25:54] 1P_JAR 2019-04-10-15
cdm: 2
cdm: 3
cdm: 4
cdm: 5
cdm: 6
cdm: 7
cdm: 8
Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-10-10 16:25:54] NID 181=eSn6z-R-x0B1ewMV1qhIqoLLQbZFwjf_dq3GeNk8nCpMmG5UIdhoirK8fMGswItGholH9cs-7XSJwAmyJCat0Z34Uff_o9
IxGDruiq0ds35p_idqiyY8awPhNQqhrMLe9JODOppcGIMaBSEZjAKlnvhEYa6zuE-e1BKunVlrzes
Registered socket 3 for persistent reuse.
URI content encoding = 'ISO-8859-1'
Length: unspecified [text/html]
Saving to: 'index.html.5'
index.html.5 [ <=> ] 11.69K --.-KB/s in 0s
2019-04-10 16:25:48 (35.6 MB/s) - 'index.html.5' saved [11968]
C:\blah>
Meanwhile on OSX,
~/bleh$ which wget
/usr/local/bin/wget
~/crp$ ls -l /usr/local/bin/wget
lrwxr-xr-x 1 apple admin 32 9 Feb 2018 /usr/local/bin/wget -> ../Cellar/wget/1.19.4_1/bin/wget
~/crp$ wget -d -S www.google.com
Debugging support not compiled in. Ignoring --debug flag.
--2019-04-10 18:17:18-- http://www.google.com/
Resolving www.google.com (www.google.com)... 216.58.201.36, 2a00:1450:4009:814::2004
Connecting to www.google.com (www.google.com)|216.58.201.36|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Date: Wed, 10 Apr 2019 17:17:10 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2019-04-10-17; expires=Fri, 10-May-2019 17:17:10 GMT; path=/; domain=.google.com
Set-Cookie: NID=181=yamm_hb_BvDNDrOORdbgAQgagjGPGAGAMiTYVGhM4BDpDfsZGbwumBM0c75lmUe5-weAvOBX-2Vk6RjJ-fzR5Q4a9xXzO7IYe7_oD7opH8FKZtJ1pTIk_P8eoOVxHmHz-np5sSJPqAuBy4kjejoGLLsALJuri18vvpsvdUc0g4o; expires=Thu, 10-Oct-2019 17:17:10 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked
Length: unspecified [text/html]
Saving to: ‘index.html.1’
index.html.1 [ <=> ] 11.67K --.-KB/s in 0.001s
2019-04-10 18:17:18 (11.1 MB/s) - ‘index.html.1’ saved [11945]
~/bleh$
Added note
I notice on OSX it says
~/bleh$ wget -d www.google.com
Debugging support not compiled in. Ignoring --debug flag.
This link https://discourse.brew.sh/t/how-to-install-wget-with-debug-support/1672
says brew reinstall wget --with-debug . But that doesn't work
~/crp$ brew reinstall wget --with-debug
Usage: brew reinstall [options] formula
Uninstall and then install formula (with existing and any appended install
options).
Unless HOMEBREW_NO_INSTALL_CLEANUP is set, brew cleanup will be run for the
reinstalled formulae or, every 30 days, for all formulae.
-d, --debug If brewing fails, open an interactive
debugging session with access to IRB or a
shell inside the temporary build directory
-s, --build-from-source Compile formula from source even if a
bottle is available.
--force-bottle Install from a bottle if it exists for the
current or newest version of macOS, even if
it would not normally be used for
installation.
--keep-tmp Don't delete the temporary files created
during installation.
-f, --force Install without checking for previously
installed keg-only or non-migrated
versions.
-v, --verbose Print the verification and postinstall
steps.
--display-times Print install times for each formula at the
end of the run.
-h, --help Show this message.
Error: invalid option: --with-debug
And brew info wget doesn't show that option
~/crp$ brew info wget
wget: stable 1.20.3 (bottled), HEAD
Internet file retriever
https://www.gnu.org/software/wget/
/usr/local/Cellar/wget/1.19.4_1 (50 files, 3.7MB) *
Poured from bottle on 2018-02-09 at 16:46:38
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/wget.rb
==> Dependencies
Build: pkg-config ✘
Required: libidn2 ✘, openssl ✘
==> Options
--HEAD
Install HEAD version
==> Analytics
install: 166,419 (30 days), 475,010 (90 days), 1,489,305 (365 days)
install_on_request: 146,838 (30 days), 417,361 (90 days), 1,304,327 (365 days)
build_error: 0 (30 days)
~/crp$
(I'm guessing that maybe HEAD version is not what i'm looking for.. I noticed in man wget mention of making a HEAD request rather than a GET request and maybe that's what that would be)
Best Answer
This link mentions to use macports for this instead of homebrew
https://stackoverflow.com/questions/48734977/how-to-install-wget-with-debug-support-on-homebrew
this is the line
sudo port install wgetAnd you can get macports from https://guide.macports.org/chunked/installing.macports.html
running the relevant pkg file to install macports.
Then it installs wget with debugging support, so -d works
and of course it also installs the latest/latest stable, version GNU Wget 1.20.3 built on darwin16.7.0., and
as opposed to what it was before, before it was
/usr/local/bin/wget.Path was updated from
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/binto/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin