Re: "layout" of a six drive raid10

[Hugo Mills <hugo@carfax.org.uk>]

[-- Attachment #1: Type: text/plain, Size: 3205 bytes --]

On Mon, Feb 08, 2016 at 11:19:52PM +0100, boli wrote:
> Hi
> 
> I'm trying to figure out what a six drive btrfs raid10 would look like. The example at <https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_differences_among_MD-RAID_.2F_device_mapper_.2F_btrfs_raid.3F> seems ambiguous to me.
> 
> It could mean that stripes are split over two raid1 sets of three devices each. The sentence "Every stripe is split across to exactly 2 RAID-1 sets" would lead me to believe this.
> 
> However, earlier it says for raid0 that "stripe[s are] split across as many devices as possible". Which for six drives would be: stripes are split over three raid1 sets of two devices each.
> 
> Can anyone enlighten me as to which is correct?

   Both? :)

   You'll find that on six devices, you'll have six chunks allocated
at the same time: A1, B1, C1, A2, B2, C2. The "2" chunks are
duplicates of the corresponding "1" chunks. The "A", "B", "C" chunks
are the alternate stripes. There's no hierarchy of RAID-0-then-RAID-1
or RAID-1-then-RAID-0.

> Reason I'm asking is that I'm deciding on a suitable raid level for a new DIY NAS box. I'd rather not use btrfs raid6 (for now). The first alternative I thought of was raid10. Later I learned how btrfs raid1 works and figured it might be better suited for my use case: Striping the data over multiple raid1 sets doesn't really help, as transfer from/to my box will be limited by gigabit ethernet anyway, and a single drive can saturate that.
> 
> Thoughts on this would also be appreciated.

> As a bonus I was wondering how btrfs raid1 are layed out in general,
> in particular with even and odd numbers of drives. A pair is
> trivial. For three drives I think a "ring setup" with each drive
> sharing half of its data with another drive. But how is it with four
> drives – are they organized as two pairs, or four-way, or …

   The fundamental unit of space allocation at this level is the chunk
-- a 1 GiB unit of storage on one device. (Or 256 MiB for metadata).
Chunks are allocated in block groups to form the RAID behaviour of the
FS.

   So, single mode will allocate one chunk in a block group. RAID-1
and -0 will allocate two chunks in a block group. RAID-10 will
allocate N chunks in a block group, where N is the largest even number
equal to or smaller than the number of devices [with space on]. RAID-5
and -6 will allocate N chunks, where N is the number of devices [with
space on].

   When chunks are to be allocated, they devices are ordered by the
amount of free space on them. The chunks are allocated to devices in
that order.

   So, if you have three equal devices, 1, 2, 3, RAID-1 chunks will be
allocated to them as: 1+2, 3+1, 2+3, repeat.

   With one device larger than the others (say, device 3), it'll start
as: 3+1, 3+2, 3+1, 3+2, repeating until all three devices have equal
free space, and then going back to the pattern above.

   Hugo.

-- 
Hugo Mills             | Well, you don't get to be a kernel hacker simply by
hugo@... carfax.org.uk | looking good in Speedos.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                         Rusty Russell

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]