rain-1.github.io

Written Jan 15 2019.

Recommendation: sftp for copying files between machines.

sftp is a better tool than scp for copying files between machines. It doesn’t (mis)interpret filenames and has a more secure design overall. For example it has a built in chroot system.

Disadvantages of sftp: “SCP is usually much faster than SFTP at transferring files, especially on high latency networks”. From [https://superuser.com/questions/134901/whats-the-difference-between-scp-and-sftp]. I timed a copy across my local network of a 104MB file - sftp: 37 seconds. scp: 34 seconds.

A discussion about scp vs sftp from curl dev. [https://ec.haxx.se/usingcurl-scpsftp.html]

Why shouldn’t I use scp?

It interprets the filename as a shell command on the remote host before trying to copy that file. e.g. scp pi:'`touch wow`' . creates a file on the ‘pi’ computer.

Some people have called this a “feature” rather than a bug and discussed how you can do things like scp pi:/db/backup-$(date +%s) . I don’t really buy this, do you know anybody that makes use of this feature on purpose?

So you have to do a bit of awkward/difficult shell sorcery with " and ' to copy filenames with spaces or sigils. It can be complex when there are 4 different possible levels of quoting and interpretation in a single command (local shell processing, command line options processing, parsing of inputs, remote shell processing).

Also there was some interesting research finding even more nasty bugs with scp: [https://blog.f-secure.com/weak-scp-security-vulnerabilities/]. (Security note: This is very minor in terms of security, it can only be exploited if you ssh into a malicious host. For that to happen you would have to accept a bad fingerprint. On the other hand it illustrates the bad design of scp). It’s such an old protocol that it’s really outdated in the way it’s designed. They are sort of forced to keep the unpleasant features in because changing anything would break millions of scripts.

What about rsync

rsync might have some nice theory to the transfer algorithm: [https://rsync.samba.org/tech_report/] but the actual implementation leaves a lot to be desired. It’s got a long long history of quite serious security vulnerabilities involving pathnames. I don’t recommend using this tool at all.

How do I use sftp

copy a file from my local computer to my remote computer

echo put 'sheep.txt' | sftp pi echo put 'sheep.txt' here\\\'s-sheep.txt | sftp pi echo put 'sheep.txt' "'heres sheep.txt'" | sftp pi

Note: You do need to escape the filenames in your ‘put’ commands.

copy a file from my remote computer to my local computer

sftp pi:file-i-want.txt

or

sftp pi:file-i-want.txt mine-now.txt

note: this will go into interactive mode if the filename actualy designates a directory on the remote.

copy a folder from my local computer to my remote computer

echo get -r "'the folder'" | sftp pi

or

echo get -r "'your folder'" "'my folder'" | sftp pi

Note: does not follow symbolic links

copy a folder from my remote computer to my local computer

echo put -r site-backup/ | sftp pi

Note: does not follow symbolic links

Tab completion of remote filenames?

Yes! You can get tab completion. Just make sure to use pubkey auth not password auth.

what commands does sftp let me use?

sftp> help Available commands: bye Quit sftp cd path Change remote directory to 'path' chgrp grp path Change group of file 'path' to 'grp' chmod mode path Change permissions of file 'path' to 'mode' chown own path Change owner of file 'path' to 'own' df [-hi] [path] Display statistics for current directory or filesystem containing 'path' exit Quit sftp get [-afPpRr] remote [local] Download file reget [-fPpRr] remote [local] Resume download file reput [-fPpRr] [local] remote Resume upload file help Display this help text lcd path Change local directory to 'path' lls [ls-options [path]] Display local directory listing lmkdir path Create local directory ln [-s] oldpath newpath Link remote file (-s for symlink) lpwd Print local working directory ls [-1afhlnrSt] [path] Display remote directory listing lumask umask Set local umask to 'umask' mkdir path Create remote directory progress Toggle display of progress meter put [-afPpRr] local [remote] Upload file pwd Display remote working directory quit Quit sftp rename oldpath newpath Rename remote file rm path Delete remote file rmdir path Remove remote directory symlink oldpath newpath Symlink remote file version Show SFTP version !command Execute 'command' in local shell ! Escape to local shell ? Synonym for help

Alternatively

tar cf - . | ssh host 'cd path && tar xf -'

cheers Taylor.