Bash – Running Multiple Commands During an SSH Inside an SSH Session

amazon ec2bashreadssh

I have a local machine which is supposed to make an SSH session to a remote master machine and then another inner SSH session from the master to each of some remote slaves, and then execute 2 commands i.e. to delete a specific directory and recreate it.

Note that the local machine has passwordless SSH to the master and the master has passwordless SSH to the slaves. Also all hostnames are known in .ssh/config of the local/master machines and the hostnames of the slaves are in slaves.txt locally and I read them from there.

So what I do and works is this:

username="ubuntu"
masterHostname="myMaster"
while read line
do

    #Remove previous folders and create new ones.
    ssh -n $username@$masterHostname "ssh -t -t $username@$line "rm -rf Input Output Partition""
    ssh -n $username@$masterHostname "ssh -t -t $username@$line "mkdir -p EC2_WORKSPACE/$project Input Output Partition""


    #Update changed files...
    ssh -n $username@$masterHostname "ssh -t -t $username@$line "rsync --delete -avzh /EC2_NFS/$project/* EC2_WORKSPACE/$project""

done < slaves.txt

This cluster is on Amazon EC2 and I have noticed that there are 6 SSH sessions created at each iteration which induces a significant delay. I would like to combine these 3 commands into 1 to get fewer SSH connections. So I tried to combine the first 2 commands into 

ssh -n $username@$masterHostname "ssh -t -t $username@$line "rm -rf Input Output Partition && mkdir -p EC2_WORKSPACE/$project Input Output Partition""

But it doesn't work as expected. It seems to execute the first one (rm -rf Input Output Partition) and then exits the session and goes on. What can I do?

Best Answer

Consider that && is a logical operator. It does not mean "also run this command" it means "run this command if the other succeeded".

That means if the rm command fails (which will happen if any of the three directories don't exist) then the mkdir won't be executed. This does not sound like the behaviour you want; if the directories don't exist, it's probably fine to create them.

Use ;

The semicolon ; is used to separate commands. The commands are run sequentially, waiting for each before continuing onto the next, but their success or failure has no impact on each other.

Escape inner quotes

Quotes inside other quotes should be escaped, otherwise you're creating an extra end point and start point. Your command:

ssh -n $username@$masterHostname "ssh -t -t $username@$line "rm -rf Input Output Partition && mkdir -p EC2_WORKSPACE/$project Input Output Partition""

Becomes:

ssh -n $username@$masterHostname "ssh -t -t $username@$line \"rm -rf Input Output Partition && mkdir -p EC2_WORKSPACE/$project Input OutputPartition\""

Your current command, because of the lack of escaped quotes should be executing:

ssh -n $username@$masterHostname "ssh -t -t $username@$line "rm -rf Input Output Partition

if that succeeds:

mkdir -p EC2_WORKSPACE/$project Input Output Partition"" # runs on your local machine

You'll notice the syntax highlighting shows the entire command as red on here, which means the whole command is the string being passed to ssh. Check your local machine; you may have the directories Input Output and Partition where you were running this.

Related Question