Best way to transfer huge files over internet

In a Linux environment, for both security and ease of use, ssh is the best way to go. SSH, SSHFS, SCP, and SFTP as you list are all just different services built on top of the SSH protocol. SCP is very easy to use, it works just like CP but you can provide user and machine names in the path. So, we might do a CP like cp ~/music/ ~/newmusic/, but we could just as easily do scp ~/music/ user@host:~/newmusic to send it to the computer named host. That's it - we don't need to set anything up. You'll be prompted for the account password on the other machine if you don't have certificate or some other authentication set up (scp shares those settings with ssh, of course).

SFTP is a tool that makes it easy to do a lot of operations on a remote file system - it works just like FTP, but it runs through SSH so it's secure and requires only an SSH server. man sftp will tell you all about how to use it. I don't use SFTP just to move a folder between two machines, it's more useful when you have a lot of operations to do, like if you're rearranging files on another computer.

SSHFS just extends SFTP in to a file system: it allows you to mount a virtual host in to your file system, so the network stuff happens totally transparently. SSHFS is for semi-permanent setups, not just a one-time file transfer. It takes some more effort to get set up, which you can read about on the project website.

If you need to work in a mixed-OS environment, Samba becomes your next best bet. Windows and OS X support Samba completely automatically, and Linux does as well although it's sometimes rough to use.

1. Not exactly an answer, but conceptually relevant:

Your ISP is selling you a speed from your house to their border router situated at the edge of The Internet. Your ISP has no control over what happens to your packets once they're out 'mongst the tubes.

The metrics you're describing are actually tracking latency between servers well outside your provider's network. Information like that is not at all relevant to the speed your provider is selling you, and can easily be obtained at places like internettrafficreport.com.

I assume the software you're describing was always meant for people managing networks, and not for end users who would confuse latency with last mile speed performance as you have.

2. Not a software solution, but still a way to get the information you want:

To test the health of your connection, run a tracert to some random server on the internet. Find the last hop on your provider's network: that's their border router, and the last point over which they have any control. Run a ping -t to that IP for up to a week: there's your real last mile performance metric.

If you're on a shared resource like cable, expect packet loss during peak hours (when everyone's online) and bursts of awesome performance when all your neighbors are at work or asleep. If you're on a private connection like DSL, expect a fairly uniform response over time.

3. A way to approach your provider with information they'll listen to:

If you think you're not getting the speed you're paying for, find your provider's own speed test (it will be on a server on their network and not out on the internet like the speed tests you mention). Perform this test ten or fifteen times over the course of a week. Calculate the average of all these tests.

Your final number should be roughly ten percent under the speed your provider sold you. (The missing 10% is protocol overhead.) If the end result is much lower, contact your provider and have them fix the problem.