Re: [SPDK] Replication for SPDK (RAID 1E)

[Luse, Paul E <paul.e.luse at intel.com>]

[-- Attachment #1: Type: text/plain, Size: 4516 bytes --]

Thanks :) I’ll definitely do the function pointer table in the first rev. That’ll be super important for add more levels later...

-from my iPhone 

> On Sep 4, 2019, at 7:59 PM, 松本周平 / MATSUMOTO，SHUUHEI <shuhei.matsumoto.xt(a)hitachi.com> wrote:
> 
> But your idea is surely good start!
> 
> Sent from my iPhone
> 
>> On Sep 5, 2019, at 10:36, 松本周平 / MATSUMOTO，SHUUHEI <shuhei.matsumoto.xt(a)hitachi.com> wrote:
>> 
>> Hi Paul,
>> 
>> It’s great to know you work on raid!
>> 
>> Ziye proposed a patch for RAID1 before RAID bdev module maybe more than a year ago.
>> 
>> IMHO, we will need to have disk replacement feature and degradation mode first. It may be difficult to use raid1 without them. Because raid1 is for RAS
>> 
>> We need to have clean abstraction for RAID level, I.e., extracting common operations and creating function pointer table.
>> 
>> We need to have copy feature between two bdevs.
>> 
>> Thanks
>> Shuhei
>> 
>> Sent from my iPhone
>> 
>>> On Sep 5, 2019, at 8:49, Luse, Paul E <paul.e.luse(a)intel.com> wrote:
>>> 
>>> Hi Everyone,
>>> 
>>> I've got a pretty simple POC working and wanted to solicit any high level input before I get too far.  The idea is to, of course, start very basic but leave room for adding features later.  Here are the broad strokes:
>>> 
>>> 
>>> *       Add a new RAID level to the existing RAID module with level "1E" that requires a "number of replicas" parameter
>>> 
>>> *       The cool thing about 1E is that we can use any number of disks and also pick the number of times the data is replicated so for example:
>>> 
>>> o   2 disk 1E with replication of 2 would be your basic 2 disk RAID1. Mapped as follows (columns are physical disks Dn identifies data copies)
>>> D0         D0
>>> D1         D1
>>> D2         D2
>>> 
>>> *       3 disk 1E with replication of 2:
>>> 
>>> D0         D0         D1
>>> 
>>> D1         D2         D2
>>> 
>>> D3         D3         D4
>>> 
>>> *       3 disk 1E with replication of 3:
>>> 
>>> D0         D0         D0
>>> 
>>> D1         D1         D1
>>> 
>>> D2         D2         D2
>>> 
>>> *       3 disk 1E with replication of 1 (RAID0)
>>> D0         D1         D2
>>> D3         D4         D5
>>> D6         D7         D8
>>> 
>>> *       This scheme is obviously very flexible and can provide basic RAID1 without disk restrictions and also provide for some super paranoid configs
>>> 
>>> *       At the same time we could consider limiting, at least at first, the combinations of disks and replicas to minimize complexity and test but IMHO I think we should leave it wide open
>>> 
>>> *       An even cooler part of this is how well the current implementation lends itself to this.  A RAID0, behind the scenes, becomes a RAID1E with 1 replica
>>> 
>>> 
>>> 
>>> 
>>> Initially we can start with just the RAID level addition (no notification of member failure, no rebuilds, spares, etc). as I don't believe there's really any existing framework to support these kinds of features.  This is the main question I have for interested parties.  Would this be useful without any of the recovery type features or should we at least have some sort of async notification on member disk failure when num_replicas > 1?
>>> 
>>> Trello link: whether it is feasible for 19.10 or not depends on feedback from everyone on features :) https://clicktime.symantec.com/3Ui9u3JQhDqBY9P7euvmPEF7Vc?u=https%3A%2F%2Ftrello.com%2Fc%2FFR4iHAnI
>>> 
>>> Thanks!
>>> Paul
>>> 
>>> PS: My current POC is super raw.  I have hardcoded number of replicas to 3 and have what I believe is the correct block mapping for any # of disks, any # of replicas but have only tested 3 replicas with 2 and 3 member disks using bdevperf w/verify. After I get it in presentable shape and flesh the design and UT out a bit more I'll post something.
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://clicktime.symantec.com/3NBASeDzyXVbw78PP7EaJro7Vc?u=https%3A%2F%2Flists.01.org%2Fmailman%2Flistinfo%2Fspdk
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://clicktime.symantec.com/3PQrvrn5FwxupLWdDkJs9Ef7Vc?u=https%3A%2F%2Flists.01.org%2Fmailman%2Flistinfo%2Fspdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk