Re: Efficient handling of sparse files

[Jeremy Allison <jra@samba.org>]

On Mon, Feb 28, 2005 at 05:41:49PM +0000, Matthew Wilcox wrote:
> 
> This problem came up with the systemimager program which uses rsync to
> install files from a master server to many clients.  Red Hat has a system
> user with uid 2^32-1 which causes lastlog to grow to 1.2GB in size.
> rsync does understand the concept of sparse files (with the -S flag), but
> it has to read every block to discover that it is indeed empty.  This sucks.
> 
> I was wondering if we could introduce a new system call (or ioctl?) that,
> given an fd would find the next block with data in it.  We could use the
> ->bmap method ... except that has dire warnings about adding new callers
> and viro may soon be in testicle-gouging range.
> 
> One system interface hack would be to introduce lseek(fd, 0, SEEK_DATA)
> ... but without permission to reuse ->bmap for this purpose, it's
> pointless to discuss user interfaces.
> 
> Suggestions?

This is very similar to the Windows ability to do a query to
get the block map of a sparse file. Might be worth looking at
that interface to see what we can learn.

Jeremy.