Bash – Redirecting bash loop output to variable specific files

bashlinuxscriptingshellshell-script

this should probably be obvious to me, but I've been stuck for some time.

I'm trying to write a very simple bash loop to take a list of servers, retrieve some specific info from them, and save the output to a file based on that host address on the starting machine.

Code currently looks like:

#!/bin/bash

SERVER_LIST=/path/to/hosts

while read REMOTE_SERVER
do
    {
    ssh user@$REMOTE_SERVER 'show_stat_from_shell_command' 
} > "$REMOTE_SERVER"
done < $SERVER_LIST

The result from the above produces only a single output file for the first host in my list and then exits.

To head off some of the more obvious solutions, Ansible etc. are not an option due to this being a very restricted environment. For the same reason using a multi-shell or tmux is also not an option (I can only log into one system at a time from my host).

So, if someone could tell me exactly how I'm messing this up it would be appreciated!

Best Answer

Replace

ssh user@$REMOTE_SERVER 'show_stat_from_shell_command'

ssh user@$REMOTE_SERVER 'show_stat_from_shell_command' </dev/null

to prevent ssh reading from stdin ($SERVER_LIST) too. Or use ssh's option -n.

-n: Redirects stdin from /dev/null (actually, prevents reading from stdin).

Related Solutions

Bash – Run asynchronous tasks and retrieve their exit code and output in bash

This is a bug/limitation; bash only allows to wait for the last process substitution, no matter if you save the value of $! into another variable.

Simpler testcase:

$ cat script
exec 7< <(sleep .2); pid7=$!
exec 8< <(sleep .2); pid8=$!
echo $pid7 $pid8
echo $(pgrep -P $$)
wait $pid7
wait $pid8

$ bash script
6030 6031
6030 6031
/tmp/sho: line 9: wait: pid 6030 is not a child of this shell

Despite pgrep -P actually finding this as a child of the shell, and strace showing that bash is actually reaping it.

But anyways, $! being also set to the PID of the last process substitution is an undocumented feature (which iirc didn't use to work in older versions), and is subject to some gotchas.

This happens because bash only keeps track of the last process substitution in the last_procsub_child variable. This is where wait will look for the pid:

-- jobs.c --
/* Return the pipeline that PID belongs to.  Note that the pipeline
   doesn't have to belong to a job.  Must be called with SIGCHLD blocked.
   If JOBP is non-null, return the index of the job containing PID.  */
static PROCESS *
find_pipeline (pid, alive_only, jobp)
     pid_t pid;
     int alive_only;
     int *jobp;         /* index into jobs list or NO_JOB */
{
     ...
  /* Now look in the last process substitution pipeline, since that sets $! */
  if (last_procsub_child)
    {

but that will be discarded when a new proc subst is created:

-- subst.c --
static char *
process_substitute (string, open_for_read_in_child)
     char *string;
     int open_for_read_in_child;
{
   ...
      if (last_procsub_child)
        discard_last_procsub_child ();

Best Answer

Related Solutions

Bash – Run asynchronous tasks and retrieve their exit code and output in bash

Related Question