Re: remote mirroring in the works?

[Simon Kirby <sim@hostway.ca>]

On Tue, Aug 31, 2010 at 07:07:29AM +0200, Fred van Zwieten wrote:

> Hmmm, maybe, but rsync would take a lot of time to find the changes.
> the actual blocks of a snap _are_ the changes, that's why SnapMirror
> is very efficient. And, I don't see how rsync will retain the snap's
> between both sites. It would be great if a tool like rsync could have
> access to the changed blocks in a snap. Don't know if btrfs exposes
> these somehow.

rsync doesn't have the hinting required to do this efficiently.  It has
to scan the whole thing every time it is run, and isn't anything like a
continuous replication in this respect.  Also, We've had problems in the
past with very large file systems causing rsync to run out of memory,
because it builds a file list in memory.  This lead us to build a "cpbk"
tool that basically did the same thing without file listsm, which turned
out to be a piece of crap, so some other guy kindly rewrote it, but he
unfortunately missed the original point entirely and rewrote it using
file lists.  Sigh.

Anyway, there _is_ this interface:

	btrfs subvolume find-new <path> <last_gen>
		List the recently modified files in a filesystem.

Eg:

	btrfs sub find-new /mnt 0

This should print all files on the file system, and the last transaction
ID marker.  This can be used to call the interface again, which lists
only new changed things since that ID.

So, it might be pretty easy to glue these tools together, for now, until
something does this automatically and/or in some more efficient or
low-level way.

Simon-