Using Filespooler for Backups

Filespooler[1] makes an *excellent* tool for handling Backups[2]. In fact, this was the case the prompted me to write it in the first place.

Using Asynchronous Communication[3] for backups has a lot of advantages. It effectively trades latency for reliability; you have a lot of confidence that the backup will actually arrive. What you often don't have is a guarantee that backups will arrive in order. This can be a problem when dealing with incremental backups, which must be applied in order on the destination, whether they're from ZFS[4], tar, dar[5], and so forth.

The general idea would be to pipe a streaming output to `fspl prepare -i -` and send that to the remote, where `fspl process` would use `zfs receive`, `tar -xf -`, or whatever the appropriate command is to unpack and store the archive. Alternatively, perhaps the remote end will just store the file somewhere (though in that case, Filespooler is probably overkill).

--------------------------------------------------------------------------------

Links to this note

• Using Filespooler over Syncthing[6]

Filespooler[7] is a way to execute commands in strict order on a remote machine, and its communication method is by files. This is a perfect mix for Syncthing[8] (and others, but this page is about Filespooler and Syncthing).

• Filespooler[9]

Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing[10], NNCP[11], ssh, UUCP[12], USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler.

Filespooler is particularly suited to distributed and Asynchronous Communication[13].

More on www.complete.org

(c) 2022-2024 John Goerzen