Re: [PATCH] btrfs-progs: add RAID5/6 support to btrfs fi us

[Joshua Houghton <joshua.houghton@yandex.ru>]

On Monday, 13 April 2020 10:08:50 UTC Joshua Houghton wrote:
> On Wednesday, 18 March 2020 21:11:56 UTC Goffredo Baroncelli wrote:
> > Hi all,
> > 
> > this patch adds support for the raid5/6 profiles in the command
> > 'btrfs filesystem usage'.
> > 
> > Until now the problem was that the value r_{data,metadata}_used is not
> > easy to get for a RAID5/6, because it depends by the number of disks.
> > And in a filesystem it is possible to have several raid5/6 chunks with a
> > different number of disks.
> > 
> > In order to bypass this issue, I reworked the code to get rid of these
> > values where possible and to use the l_{data,metadata}_used ones.
> > Notably the biggest differences is in how the free space estimation
> > 
> > is computed. Before it was:
> > 	free_estimated = (r_data_chunks - r_data_used) / data_ratio;
> > 
> > After it is:
> > 	free_estimated = l_data_chunks - l_data_used;
> > 
> > which give the same results when there is no mixed raid level, but a
> > better result in the other case. I have to point out that before in the
> > code there was a comment that said the opposite.
> > 
> > The other place where the r_{data,metadata}_used are use is for the
> > "Used:" field. For this case I estimated these values using the
> > 
> > following formula (only for raid5/6 profiles):
> > 	r_data_used += (double)r_data_chunks * l_data_used /
> > 	
> >                                l_data_chunks;
> > 
> > Note that this is not fully accurate. Eg. suppose to have two raid5
> > chunks,
> > the first one with 3 disks, the second one with 4 disks, and that each
> > chunk is 1GB.
> > r_data_chunks_r56, l_data_used_r56, l_data_chunks_r56 are completely
> > defined, but real r_data_used is completely different in these two cases:
> > - the first chunk is full and the second one id empty
> > - the first chunk is full empty and the second one is full
> > However now this error affect only the "Used:" field.
> > 
> > 
> > So now if you run btrfs fi us in a raid6 filesystem you get:
> > 
> > $ sudo btrfs fi us /
> > 
> > Overall:
> >     Device size:		  40.00GiB
> >     Device allocated:		   8.28GiB
> >     Device unallocated:		  31.72GiB
> >     Device missing:		     0.00B
> >     Used:			   5.00GiB
> >     Free (estimated):		  17.36GiB	(min: 17.36GiB)
> >     Data ratio:			      2.00
> >     Metadata ratio:		      0.00
> >     Global reserve:		   3.25MiB	(used: 0.00B)
> > 
> > Data,RAID6: Size:4.00GiB, Used:2.50GiB (62.53%)
> > [...]
> > 
> > Instead before:
> > 
> > $ sudo btrfs fi us /
> > WARNING: RAID56 detected, not implemented
> > WARNING: RAID56 detected, not implemented
> > WARNING: RAID56 detected, not implemented
> > 
> > Overall:
> >     Device size:		  40.00GiB
> >     Device allocated:		     0.00B
> >     Device unallocated:		  40.00GiB
> >     Device missing:		     0.00B
> >     Used:			     0.00B
> >     Free (estimated):		     0.00B	(min: 8.00EiB)
> >     Data ratio:			      0.00
> >     Metadata ratio:		      0.00
> >     Global reserve:		   3.25MiB	(used: 0.00B)
> > 
> > Data,RAID6: Size:4.00GiB, Used:2.50GiB (62.53%)
> > [...]
> > 
> > 
> > I want to point out that this patch should be compatible with my
> > previous patches set (the ones related to the new ioctl
> > BTRFS_IOC_GET_CHUNK_INFO). If both are merged we will have a 'btrfs fi us'
> > commands with full support a raid5/6 filesystem without needing root
> > capability.
> > 
> > Comments are welcome.
> > BR
> > G.Baroncelli
> 
> Hi Goffredo
> 
> Thanks you for this. It's something I've been wanting for a while. Do
> you know why I get significantly different results in the overall summary
> when I do not run it as root. I'm not sure if this is a bug or a
> limitation.
> 
> When I run it as root it looks to be showing the correct values.
> 
> joshua@r2400g:~/development/btrfs-progs$ colordiff -u <(./btrfs fi us
> /mnt/raid/) <(sudo ./btrfs fi us /mnt/raid/) WARNING: cannot read detailed
> chunk info, per-device usage will not be shown, run as root --- /dev/fd/63 
> 2020-04-13 10:54:26.833747190 +0100
> +++ /dev/fd/62  2020-04-13 10:54:26.843746984 +0100
> @@ -1,17 +1,32 @@
>  Overall:
>      Device size:                 29.11TiB
> -    Device allocated:           284.06GiB
> -    Device unallocated:                  28.83TiB
> -    Device missing:              29.11TiB
> -    Used:                       280.99GiB
> -    Free (estimated):               0.00B      (min: 14.95TiB)
> -    Data ratio:                              0.00
> +    Device allocated:            19.39TiB
> +    Device unallocated:                   9.72TiB
> +    Device missing:                 0.00B
> +    Used:                        18.67TiB
> +    Free (estimated):             7.82TiB      (min: 5.39TiB)
> +    Data ratio:                              1.33
>      Metadata ratio:                  2.00
>      Global reserve:             512.00MiB      (used: 0.00B)
> 
>  Data,RAID5: Size:14.33TiB, Used:13.80TiB (96.27%)
> +   /dev/mapper/traid3     4.78TiB
> +   /dev/mapper/traid1     4.78TiB
> +   /dev/mapper/traid2     4.78TiB
> +   /dev/mapper/traid4     4.78TiB
> 
>  Metadata,RAID1: Size:142.00GiB, Used:140.49GiB (98.94%)
> +   /dev/mapper/traid3    63.00GiB
> +   /dev/mapper/traid1    64.00GiB
> +   /dev/mapper/traid2    63.00GiB
> +   /dev/mapper/traid4    94.00GiB
> 
>  System,RAID1: Size:32.00MiB, Used:1.00MiB (3.12%)
> +   /dev/mapper/traid1    32.00MiB
> +   /dev/mapper/traid4    32.00MiB
> 
> +Unallocated:
> +   /dev/mapper/traid3     2.44TiB
> +   /dev/mapper/traid1     2.44TiB
> +   /dev/mapper/traid2     2.44TiB
> +   /dev/mapper/traid4     2.41TiB
> 
> 
> This is in contrast to raid1 which seems to be mostly correct, irrespective
> of what user I run as.
> 
> 
> joshua@arch:/var/joshua$ colordiff -u <(btrfs fi us raid/) <(sudo btrfs fi
> us raid/) WARNING: cannot read detailed chunk info, per-device usage will
> not be shown, run as root --- /dev/fd/63  2020-04-13 09:52:54.630750079
> +0000
> +++ /dev/fd/62  2020-04-13 09:52:54.637416835 +0000
> @@ -2,7 +2,7 @@
>      Device size:                  8.00GiB
>      Device allocated:             1.32GiB
>      Device unallocated:                   6.68GiB
> -    Device missing:               8.00GiB
> +    Device missing:                 0.00B
>      Used:                       383.40MiB
>      Free (estimated):             3.55GiB      (min: 3.55GiB)
>      Data ratio:                              2.00
> @@ -10,8 +10,17 @@
>      Global reserve:               3.25MiB      (used: 0.00B)
> 
>  Data,RAID1: Size:409.56MiB, Used:191.28MiB (46.70%)
> +   /dev/loop0   409.56MiB
> +   /dev/loop1   409.56MiB
> 
>  Metadata,RAID1: Size:256.00MiB, Used:416.00KiB (0.16%)
> +   /dev/loop0   256.00MiB
> +   /dev/loop1   256.00MiB
> 
>  System,RAID1: Size:8.00MiB, Used:16.00KiB (0.20%)
> +   /dev/loop0     8.00MiB
> +   /dev/loop1     8.00MiB
> 
> +Unallocated:
> +   /dev/loop0     3.34GiB
> +   /dev/loop1     3.34GiB
> 
> Does anyone know if this is something we can fix? I'm happy to take a look.
> 
> Joshua Houghton

Sorry missed this last bit never mind

> If both are merged we will have a 'btrfs fi us'
> commands with full support a raid5/6 filesystem without needing root
> capability.