Improve SFTP Upload With RSYNC And SSHFS

SFTP upload speed isn't the best you can imagine, but I "found" a "trick" to improve this.

Let's assume we have a project folder with dist containing the files you want to upload, troet as SFTP user, example.com as address and public_html/foo/bar/proj as target folder.
The "trick" I found is rsync's batch mode --write-batch=name.

BATCH MODE

Batch mode can be used to apply the same set of updates to many identi-
cal systems. Suppose one has a tree which is replicated on a number of
hosts. Now suppose some changes have been made to this source tree and
those changes need to be propagated to the other hosts. In order to do
this using batch mode, rsync is run with the write-batch option to
apply the changes made to the source tree to one of the destination
trees. The write-batch option causes the rsync client to store in a
"batch file" all the information needed to repeat this operation
against other, identical destination trees.

[...]

-- rsync(1)

First run rsync against any local directory, e.g. mirr. With batch mode, rsync writes a batch file containing all the changes made. It also generates a handy script you can use to apply that to any other directory. Use that to apply it to your mounted SFTP directory (use SSHFS or GVS).

Tada!

$ cd proj
$ ls
dist
$ rsync --write-batch=mybatchname -ru dist/ mirr
$ ls
dist mirr mybatchname mybatchname.sh
$ mkdir target
$ sshfs troet@examle.com:public_html/foo/bar/proj target # Asks for password unless you installed sshkey :)
$ ./mybatchname.sh target

Christoph Schulz

Christoph Schulz
I am Dev. Maybe.

Podman Minikube on Debian 11

Where I can I try to replace docker with podman, rootless podman to be specific.Recently I have been playing around with minikube for kap...… Continue reading

Packaging Fractal as DEB 1/?

Published on January 16, 2020