How to list files and only files via SFTP

sftp

I can list file names actually but a lot of unwanted stuff comes from there also.

> echo "ls *.txt" | sftp user@host:/dir
Connected to 1.1.1.1.
Changing to: /dir
sftp> ls *.txt
1.txt
2.txt
3.txt

Is there a proven/reliable way to list files and only files? I'd like to avoid using head-like filters if possible.

Best Answer

Use the -q option to tell sftp to be quiet, thereby suppressing most of the output you don't care about:

echo "ls *.txt" | sftp -q user@host.example.com:/path

You will still see the lines for the interactive prompt to which you are echoing, e. g. sftp> ls *.txt, but those can be filtered out with a grep -v:

echo "ls *.txt" | sftp -q user@host.example.com:/path | grep -v '^sftp>'

As an aside, it's probably better practice to use a batch file, and pass it with the -b parameter to sftp rather than echoing a pipeline into it.

If all you really want to do is get a list of files, this might actually be better served with ssh than with sftp (which, after all, is the secure file transfer program):

ssh user@host.example.com ls -1 /path

Related Solutions

SFTP – How to Remove Multiple Files

Here is one possible solution that can be added to bash script. This is not ideal as it will make a new connection for each file.

#!/bin/bash
# set variables
USER="username"
HOST="hostname"
file_list="file1 file1 file3 file4"
# delete each file
for file in $file_list; do
    echo "rm $file" | sftp $USER@$HOST
done
exit 0

This one-liner is far better! file1-9 being file names to remove, use a variable if you like, it's the same thing.

for file in file1 file2 file3 file4 file5 file6 file7 file8 file9; do echo -e "rm $file" >> sftp_batch; done; sftp -b sftp_batch username@hostname; rm sftp_batch

Sftp – how to only copy files from folder that don’t exist in destination folder

sftp has limited capabilities. Nonetheless, the get command has an option which may do the trick: get -a completes partial downloads, so if a file is already present on the client and is at least as large as the file on the server, it won't be downloaded. If the file is present but shorter, the end of the file will be transferred, which makes sense if the local file is the product of an interrupted download.

The easiest way to do complex things over SFTP is to use SSHFS. SSHFS is a filesystem that uses SFTP to make a remote filesystem appear as a local filessytem. On the client, SSHFS requires FUSE, which is available on most modern unices. On the server, SSHFS requires SFTP; if the server allows SFTP then you can use SSHFS with it.

mkdir server
sshfs server.example.com:/ server
rsync -a server/remote/path /local/path/
fusermount -u server

Note that rsync over SSHFS can't take advantage of the delta transfer algorithm, because it's unable to compute partial checksums on the remote side. That's irrelevant for a one-time download but wasteful if you're synchronizing files that have been modified. For efficient synchronization of modified files, use rsync -a server:/remote/path /local/path/, but this requires SSH shell access, not just SFTP access. The shell access can be restricted to the rsync command though.

Best Answer

Related Solutions

SFTP – How to Remove Multiple Files

Sftp – how to only copy files from folder that don’t exist in destination folder

Related Question