Shell script: ssh to remote machine, then exit: ‘exit’ does not work

I have a shell script of the following type:

#!/bin/bash

ssh mylogin@myremotemachine.com
echo "Hi"
exit

I run it locally to do something on a remote server (represented by 'echo
"Hi"'). However, when I run it, I see the prompt on the remote server — so the
'exit' command is not executed. When I then manually type 'exit' on the remote
prompt, I then see "Connection to myremotemachine.com" closed and then
"Hi". How can I set up the shell script such that it exits correctly and shows
me the (local) prompt from which I executed it?

https://unix.stackexchange.com/questions/89747/ssh-exits-after-quit-case-in-bash-script and ssh and shell through ssh : how to exit? seem somewhat related, but I couldn't adapt the ideas presented there./

UPDATE

The following, not so minimal version leads to Unmatched '..

#!/bin/bash

date=`date "+%Y-%m-%d"`
rsync -acxzP --delete --link-dest=/u3/mylogin/backup/old_backup /home/mylogin mylogin@myremotemachine.com:/u3/mylogin/backup/$date\_backup
ssh mylogin@myremotemachine.com bash -c "'
rm -f /u3/mylogin/backup/old_backup
ln -s $date\_backup /u3/mylogin/backup/old_backup
exit
'"

If I remove the double quotes in this snippet, I get: bash: -c: option requires an argument and date: Undefined variable.