Blog

Posted 2011/04/25

Fast SSH file copy with tar

I have been reinstalling one of our research clusters and had to copy around 200GB of data over the network. The data was mostly tiny files and I half remembered an old trick using tar to speed up the transfer. After some googling I refined it down to this:

$ tar cz dir_name | ssh -c blowfish user@server tar xz -C "/path/to/stash"

According to the documentation using the blowfish cypher increases the speed further.