Re: Why is the actual disk usage of btrfs considered unknowable?

[Martin Steigerwald <Martin@lichtvoll.de>]

[-- Attachment #1: Type: text/plain, Size: 3087 bytes --]

Am Sonntag, 7. Dezember 2014, 18:34:44 schrieb Hugo Mills:
> On Sun, Dec 07, 2014 at 10:20:27AM -0800, ashford@whisperpc.com wrote:
> [snip]
> 
> > > 3) From what I gathered it is planned to allow different raid /
> > > redundancy levels for different subvolumes. BTRFS can´t know
> > > beforehand where applications request to save future data, i.e.
> > > in which subvolume.
> > 
> > Same as above.
> > 
> > If a user will be requesting to use a specific subvolume, it is up to them
> > to verify that adequate free space exists there, or handle the exception.
> 
>    OK, so let's say I've got a filesystem with 100 GiB of unallocated
> space. I have two subvolumes, one configured for RAID-1 and one
> configured for single storage.
> 
>    What number should be shown in the free output of df?
> 
>    100 GiB? But I can only write 50 GiB to the RAID-1 subvolume before
> it runs out of space.
> 
>    50 GiB? I can get twice that much on the single subvolume.
> 
>    *Any* value shown here is going to be inaccurate, and whatever way
> round we show it, someone will complain.

Thats why I pointed out fallocate. If it suceeds, I would expect even BTRFS 
with its special free space challenges guarantees the space is there.

A getfreespacebypath() syscall may yield quite accurate figures as well, cause 
then BTRFS can know which subvolume the application wants to write it, but as 
it cannot predict the future write behavior of all processes, it cannot 
guarantee anything.

So for any guarantee as far as I know, the only thing you can do is fallocate.

I never liked the pre installation check of roughly having 5 GiB of free space 
or so to suceed or fail otherwise. But on the other hand, running 90% through 
an installation and then failing to do not enough free space is also not nice. 
Similarily to copying 4 GiB of a 4,3 GB DVD image to an FAT32 before it fails 
the copy.

But for the FAT32 it is much easier to know it can´t write a file larger 4 GiB 
than for BTRFS or any other filesystem to know whether installing a set of files 
to a set of directories with a certain total size is going to work out.

Only thing that could improve this, would be some kind of more flexible space 
allocation. Or… creating every directory and fallocating each file with the 
exact size before doing the actual copy. And heck, somehow I like this idea. 
It could help to avoid actions that just do not make sense. An rsync could 
abort early if the total free space would not be enough. But especially for 
rsync since version 3 this again doesn´t work, as rsync works incrementally 
since version 3. On the other hand if btrfs send could store size requirements 
in the send data, before the receive side thats working, then the receive side 
could preallocate, but… then that would depend on how easy it would be for the 
send size to get the size difference between two snapshots.

Thanks,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 181 bytes --]