"Asymmetric" RAID0

"Asymmetric" RAID0

[Slava Barinov]

Hello,

 I've been using a single drive btrfs for some time and when free space
 became too low I've added an additional drive and rebalanced FS with
 RAID0 data and RAID1 System and Metadata storage.

 Now I have the following configuration:

# btrfs fi show /btr
Label: none  uuid: f9d78880-10a7-439b-8ebd-14d815edbc19
    Total devices 2 FS bytes used 415.45GiB
    devid    1 size 931.51GiB used 222.03GiB path /dev/sdc
    devid    2 size 431.51GiB used 222.03GiB path /dev/sdb

# btrfs fi df /btr
Data, RAID0: total=424.00GiB, used=406.81GiB
System, RAID1: total=32.00MiB, used=40.00KiB
Metadata, RAID1: total=10.00GiB, used=8.64GiB

# df -h
Filesystem                               Size  Used Avail Use% Mounted on
/dev/sdb                                 1.4T  424G  437G  50% /btr

 I suppose I should trust to btrfs fi df, not df utility.

 So the main question is if such "asymmetric" RAID0 configuration
 possible at all and why does btrfs ignore ~500 GB of free space on
 /dev/sdc drive?

 Also it's interesting what will happen when I add 20 GB more data to
 my FS. Should I be prepared to usual btrfs low-space problems?

Best regards,
Slava Barinov.

Re: "Asymmetric" RAID0

[Brendan Hide]

On 25/03/14 07:15, Slava Barinov wrote:
> Hello,
>
>   I've been using a single drive btrfs for some time and when free space
>   became too low I've added an additional drive and rebalanced FS with
>   RAID0 data and RAID1 System and Metadata storage.
>
>   Now I have the following configuration:
>
> # btrfs fi show /btr
> Label: none  uuid: f9d78880-10a7-439b-8ebd-14d815edbc19
>      Total devices 2 FS bytes used 415.45GiB
>      devid    1 size 931.51GiB used 222.03GiB path /dev/sdc
>      devid    2 size 431.51GiB used 222.03GiB path /dev/sdb
>
> # btrfs fi df /btr
> Data, RAID0: total=424.00GiB, used=406.81GiB
> System, RAID1: total=32.00MiB, used=40.00KiB
> Metadata, RAID1: total=10.00GiB, used=8.64GiB
>
> # df -h
> Filesystem                               Size  Used Avail Use% Mounted on
> /dev/sdb                                 1.4T  424G  437G  50% /btr
>
>   I suppose I should trust to btrfs fi df, not df utility.
>
>   So the main question is if such "asymmetric" RAID0 configuration
>   possible at all and why does btrfs ignore ~500 GB of free space on
>   /dev/sdc drive?
>
>   Also it's interesting what will happen when I add 20 GB more data to
>   my FS. Should I be prepared to usual btrfs low-space problems?
>
> Best regards,
> Slava Barinov.
The "raid0" will always distribute data to each disk relatively equally. 
There are exceptions of course. The way to have it better utilise the 
diskspace is to use either "single" (which won't get the same 
performance as raid0) or to add a third disk.

In any raided configuration, the largest disk won't be fully utilised 
unless the other disks add up to be equal to or more than that largest disk.

Play around with Hugo's disk usage calculator to get a better idea of 
what the different configurations will do: http://carfax.org.uk/btrfs-usage/

-- 
__________
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97

Re: "Asymmetric" RAID0

[Duncan]

Brendan Hide posted on Tue, 25 Mar 2014 08:42:17 +0200 as excerpted:

> The "raid0" will always distribute data to each disk relatively equally.
> There are exceptions of course. The way to have it better utilise the
> diskspace is to use either "single" (which won't get the same
> performance as raid0) or to add a third disk.
> 
> In any raided configuration, the largest disk won't be fully utilised
> unless the other disks add up to be equal to or more than that largest
> disk.
> 
> Play around with Hugo's disk usage calculator to get a better idea of
> what the different configurations will do:
> http://carfax.org.uk/btrfs-usage/

Agreed.  Adding...

The wiki's multi-device page says this:

"""""
When you have drives with differing sizes and want to use the full 
capacity of each drive, you have to use the single profile for the data 
blocks, rather than raid0. 
"""""
https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices

Further explanation for this reply:

Each raidN mode has a minimum number of devices it can use.  For raid0 
(and raid1), that's two devices, otherwise it's no longer raidN, but 
single (or dup) mode.

That's why btrfs won't be able to use ~500 gig of the larger device, 
since when it fills up the smaller, there's no way to continue raid0 
striping.

If you added a third device the same size as the larger device (931 gig), 
btrfs would stripe across three devices until it filled the smaller, then 
across the two remaining devices.

If, however, the third device was only ~ 500 gig, you'd have roughly the 
same problem, as it would stripe three ways until the smallest was full, 
then two ways until the second-smallest was full, again leaving the 
remember unusable.

Tho I don't see where the 20 gig number comes from, as the smaller device 
still had 230-ish gigs unallocated, so you (OP not Brendan) should be 
safe for 200 gig or so more allocation, which combined with the 200 that 
would also be allocated on the larger drive, would be about 400 gig or so 
of additional data, depending on how that unallocated space gets 
allocated and whether you rebalance as the allocated space on the smaller 
drive drops toward zero, to return any available allocated but not fully 
used chunks to unallocated.

As Brendan says, however, switching to single mode instead of raid0 
should let you use all the space, tho with less performance, because that 
doesn't require the striping.

In theory at least, you /could/ even switch to single mode as you 
approached full on the raid0, doing a rebalance-convert, but then 
canceling it right away, before it has actually converted much to 
single.  That /should/ leave most of the existing chunks (minus the few 
converted before the balance-convert was canceled as raid0, while telling 
the filesystem that new allocations should be single, not raid0.  But I'm 
not sure if that would actually work in practice, as I'm not sure how 
btrfs would do further data block allocations in that case.  Also, you'd 
have to be careful while doing further balances so as not to disturb the 
raid0/single ratio and push over the edge on the smaller device.  So 
unless you decide to be the guinea pig, I'd suggest leaving that as "in 
theory", but if you want to try it, I'm sure I'm not the only one who'd 
be interested in your report of how it goes. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman