Shell – bash script executed over ssh returns incorrect exit code 0

exit-statusshellssh

I am trying to automate a process which involves running scripts on various machines via ssh. It is vital to capture both output and the return code (for the detection of errors).

Setting the exit code explicitly works as expected:

~$ ssh host exit 5 && echo OK || echo FAIL
FAIL

However, if there is a shell script signalling an unclean exit, ssh always returns 0 (script simulated by string execution):

~$ ssh host sh -c 'exit 5' && echo OK || echo FAIL
OK

Running the very same script on the host in an interactive shell works just fine:

~$ sh -c 'exit 5' && echo OK || echo FAIL
FAIL

I am confused as to why this happens. How can I tell ssh to propagate bash's return code? I may not change the remote scripts.

I am using public key authentication, the private key is unlocked – there is no need for user interaction. All systems are Ubuntu 18.04. Application versions are:

  • OpenSSH_7.6p1 Ubuntu-4ubuntu0.1, OpenSSL 1.0.2n 7 Dec 2017
  • GNU bash, Version 4.4.19(1)-release (x86_64-pc-linux-gnu)

Note: This question is different from these seemingly similar questions:

Best Answer

I am able to duplicate this using the command you used, and I am able to resolve it by wrapping the remote command in quotes. Here are my test cases:

#!/bin/bash -x

echo 'Unquoted Test:'
ssh evil sh -x -c exit 5 && echo OK || echo FAIL

echo 'Quoted Test 1:'
ssh evil sh -x -c 'exit 5' && echo OK || echo FAIL

echo 'Quoted Test 2:'
ssh evil 'sh -x -c "exit 5"' && echo OK || echo FAIL

Here are the results:

bash-[540]$ bash -x test.sh
+ echo 'Unquoted Test:'
Unquoted Test:
+ ssh evil sh -x -c exit 5
+ exit
+ echo OK
OK
+ echo 'Quoted Test 1:'
Quoted Test 1:
+ ssh evil sh -x -c 'exit 5'
+ exit
+ echo OK
OK
+ echo 'Quoted Test 2:'
Quoted Test 2:
+ ssh evil 'sh -x -c "exit 5"'
+ exit 5
+ echo FAIL
FAIL

In the first test and second tests, it seems the 5 is not being passed to exit as we would expect it to be. It just seems to be disappearing. It's not going to exit, sh isn't complaining about 5: command not found, and ssh isn't complaining about it.

In the third test, exit 5 is quoted within the larger command to run on the remote host, same as in the second test. This ensures that the 5 is passed to exit, and both are executed as the -c option to sh. The difference between the second and third tests is that the whole set of commands and arguments is sent to the remote host quoted as a single command argument to ssh.

Related Question