SFTP – Transferring Files in Binary Mode

Is your machine accessible over the Internet?

The first hurdle is that your machine may not be accessible over the Internet at all!

Most client machines cannot be accessed directly over the Internet because they don't have a public IP address. It's like having a phone that can call out, but can't be called. This came about mainly because there's a limited supply of IP addresses; unless your ISP supports IPv6 or you have a very atypical configuration, you have a single IP address at home, and that's the address of your home router. Your computers can make outgoing connections because the home router provides NAT functionality.

Most home routers can be configured to allow incoming connections to be routed to a particular machine on the local network. To allow incoming SSH connections, route port 22 to your computer. See your router's documentation for how to do this.

If you're unlucky and your ISP doesn't give a public IP address, you won't be able to make incoming connections. To check whether you have a public IP address, connect to your router's administrative interface and check whether its external address is in the private range (internal addresses are in the private range except in atypical configurations).

Giving shell or file access to your machine

The (relatively) easy way to give someone access to your machine is to create a user account for them. With an ordinary user account, they'll be able to see a lot of things, but they won't be able to modify your files (unless you went out of your way to make them world-writable), and they won't be able to see the files that are in a private directory (drwx------ permissions).

For better security, configure the account to be usable only to manipulate files in a particular directory over SFTP. This is a bit more difficult (I kind of expected OSX to provide an easy-to-use GUI for that, but apparently not); see Create a remote only user in OS X? or How to set up an SFTP server on a Mac & then enable a friend to upload files to it from their iPhone, iPad, or other iDevice for instructions.

You'll need to enable remote access. There is an OSX knowledge base entry for that. Enable only the one user who is supposed to have remote access. Do not enable remote access for an account that may have a weak password!

Set a random password on the account and tell them to copy-paste it and save it in a file. Don't expose a machine with weak, human-chosen passwords to the Internet. You can use the following command to generate the password:

</dev/urandom tr -dc A-Za-z0-9 | head -c 16; echo

Transfering files piece by piece

So yeah, sending files over the Internet is still difficult.

The low-tech solution is to use one of the many file sharing websites. They make their money through ads, so don't even think of visiting one without an ad blocker, and be very careful where you click because they're likely to try to serve you malware. After downloading a file, check that it's the right file: calculate its SHA-2 checksum with

sha -a256 /path/to/file

on OSX, sha256sum /path/to/file on Linux.

Use the -q option to tell sftp to be quiet, thereby suppressing most of the output you don't care about:

echo "ls *.txt" | sftp -q user@host.example.com:/path

You will still see the lines for the interactive prompt to which you are echoing, e. g. sftp> ls *.txt, but those can be filtered out with a grep -v:

echo "ls *.txt" | sftp -q user@host.example.com:/path | grep -v '^sftp>'

As an aside, it's probably better practice to use a batch file, and pass it with the -b parameter to sftp rather than echoing a pipeline into it.

If all you really want to do is get a list of files, this might actually be better served with ssh than with sftp (which, after all, is the secure file transfer program):

ssh user@host.example.com ls -1 /path