How to Pipe Segment of Heredoc Output to Local File

fifohere-documenttcpdump

I am issuing a command to a remote device in order to run a tcpdump on one of it's interfaces, but capture the output to a local .pcap file on my Ubuntu VM. The command that I'm issuing is as follows:

drew@Drew-Ubuntu:~/Desktop$ sshpass -p $password ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 -T $username@$remote-ip -p $remote-port << EOF \ > /home/drew/Desktop/Example_Capture.pcap
> sh
> tcpdump -i eth5.1 -s 0 -n -v not host $local-ip -w -
> EOF
tcpdump: listening on eth5.1, link-type EN10MB (Ethernet), capture size 65535 bytes
^Ct 23
drew@Drew-Ubuntu:~/Desktop$

The issue that I'm having is that this does allow me to start up a tcpdump. I can see the "Got $counter" values increasing as the tcpdump runs which is great. I can then simply "CTRL + c" in order to stop the capture and my file is generated at my specified location. The issue is that the first command "> sh" shows up in the very first line of this capture file, which stops me from being able to properly open this file for review.

I've tried adding "> sh > /dev/null", but I'm pretty sure that is only going to suppress output on the remote side, and not mine, as it didn't work. I've tried messing with the syntax of this a bunch, but either my command doesn't work entirely, or that first line is included. Hoping there is a proper syntax out there to just populate my local file with the tcpdump information.

Update:

I have since tried to update things to work within a single script that runs my tcpdump capture, and then once cancelled out, sed would clean up the first line as needed. Below is what I have built:

#!/bin/bash

read -p $'\e[32mEnter Remote IP: \e[0m' remote_ip
read -p $'\e[32mEnter Username: \e[0m' username
read -sp $'\e[32mEnter Password: \e[0m' password
echo
read -p $'\e[32mEnter File Save Location and Name: \e[0m' save_file
echo -e '\e[32mEnter tcpdump Options (ie. -i eth1 -s 0 .....)\e[0m'
read -p $'\e[32m*note: You do not need to enter the \"-w\" modifier to save to a file: \e[0m' tcpdump_options

sshpass -p $password ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 -T $username@$remote_ip -p 30007 <<- EOF \ > /tmp/Remote_Capture.pcap
        sh
        tcpdump $tcpdump_options -U -w -
EOF

trap '{ sed 1d /tmp/Remote_Capture.pcap > $save_file; rm /tmp/Remote_Capture.pcap; }' SIGINT SIGTERM

The issue with this is that when I enter a CTRL+C, it seems to terminate out of the entire script and not make it to that final line. My capture file gets created in my /tmp directory with that extra line up top, but the sed operations don't actually happen to clean things up and push the cleaned up file to my set location.

Best Answer

You could strip off the first line before the redirection into your file:

sshpass ... "$remote-port" | sed 1d << EOF > /home/drew/Desktop/Example_Capture.pcap
sh
tcpdump -i eth5.1 -s 0 -n -v not host $local-ip -w -
EOF

Related Solutions

TCPDUMP – How to Process and Pipe Output in Realtime

Straight out of man tcpdump

-l     Make stdout line buffered.  Useful if you want to see the data while 
       capturing it.  E.g.,

              tcpdump -l | tee dat

       or

              tcpdump -l > dat & tail -f dat

       Note that on Windows,``line buffered'' means ``unbuffered'', so that 
       WinDump will write each character individually if -l is specified.

       -U is similar to -l in its behavior, but it will cause output to be 
       ``packet-buffered'', so that the output is written to stdout at the 
       end of each packet rather than at the end of each line; this is 
       buffered on all platforms, including Windows.

here-document – How to Indent an Heredoc Inside Another Heredoc

A here-document is a redirection of the form:

<<[-]DELIMITER
    ....
    ....
    ....
DELIMITER

The optional - (inside the brackets above) changes the way the delimiter is matched and allows indenting each line inside the heredoc content, with tabulations (no spaces allowed).

_{"Matched" means the delimiter is matched to the opener (as when DELIMITER matches <<DELIMITER or <<-DELIMITER, for example).}
_{Note that you may use one or more spaces between << or <<-, and the word that follows).}

So to sum up the basic laws for matching inside a singlar heredoc:

The opener must be placed at the very beginning of the line in an applicable syntax.
The delimiter must be the only word of its line.
All content under the opener (including the delimiter) can be indented with any number of tabulations, with the <<-DELIMITER syntax.

Since with the former syntax, no blanks can precede the heredoc opener, if you want to indent it, your only choice is to use the following syntax and you must exclusively use tabulations at the beginning of each line inside the heredoc's content.

Now you have two options with the <<- syntax.

First option

Use the <<- syntax for the inner heredoc.

bash << APACHE
    ... 
    ... 
    cat <<- MOD_REWRITE
⇨       ...     
⇨       ....    
⇨       MOD_REWRITE
    ... 
    ... 
APACHE

(indentation is 4 spaces, tabulations are symbolized with ⇨)

The code seen by bash will be exactly what is written on your screen (i.e. bash will see the indentation of each line as you see it now). When the inner heredoc is met, owing to the <<- syntax, bash will strip the tabulation characters leading each line until the MOD_REWRITE delimiter.

Second option

Use the <<- syntax for the outer heredoc.

bash <<- APACHE
⇨       ...
⇨       ...
⇨       cat << MOD_REWRITE
⇨       ⇨       ...
⇨       ⇨       ....
⇨       MOD_REWRITE
⇨       ...
⇨       ...
APACHE

This time, the code seen by bash will differ from what you see: it won't contain any leading tabulation. That's why this is not a problem that I use the << syntax for the inner heredoc: the MOD_REWRITE delimiter will be at the beginning of the line.

In both cases, the MOD_REWRITE delimiter is recognized and your Apache configuration file /etc/apache2/apache2.conf is not indented. If you want to indent parts of it, your only option is to use spaces (after the initial tabulations that will be stripped).

Of course, there is a third option: to use the <<- syntax for both heredocs, but that won't change anything from option 2 since all the leading tabulations are removed when the code is sent to bash.