From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============4994136311866134615==" MIME-Version: 1.0 From: Luse, Paul E Subject: Re: [SPDK] Replication for SPDK (RAID 1E) Date: Thu, 05 Sep 2019 03:44:14 +0000 Message-ID: In-Reply-To: 47ACA4A7-F74A-413C-A17D-686C900F9D46@hitachi.com List-ID: To: spdk@lists.01.org --===============4994136311866134615== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Of course... not to worry I crank out emails faster than code for sure :) P= lus there no reason at all to rush for 19.10. We=E2=80=99ll do it right! -from my iPhone = > On Sep 4, 2019, at 8:25 PM, =E6=9D=BE=E6=9C=AC=E5=91=A8=E5=B9=B3 / MATSUM= OTO=EF=BC=8CSHUUHEI wrote: > = > Hi Paul, > = > Please give time to read through the whole comment :-) > = > But about copy, my intention is that if we add an new disk, the disk has = to catch up to the existing disks. So copy between two bdevs may be a basic= feature. > = > Thanks, > Shuhei > = > Sent from my iPhone > = >> On Sep 5, 2019, at 10:59, Luse, Paul E wrote: >> = >> Hi Shuhei, >> = >> Thanks for the reply! Yes, I remember Ziye's patch and as I recall there= were two things about it that I didn't want to carry forward (a) it was it= 's own bdev module as opposed to enhancing what's there (I don't believe th= e RAID0 was there yet at the time though, don't remember) (b) it relied hea= vily on common functions used by other basic bdev modules like gpt. >> = >> The first point is worth discussion though, I'm glad you brought it up. = But first, yes I expected feedback on needing some sort of degradation/rep= lacement feature(s) and I don't disagree. By copy I assume you mean someth= ing like "transform an existing RAID0-->RAID1E"? We could make something li= ke as simple or as complex as we wanted as well but that would be very cool= . Anyway, on a separate bdev module vs updating the RAID0, I chatted with J= im about this a bit as well but I think given what we already have it's a m= uch lighter lift (less code, less complex) to add it to the RAID0 module. T= he only pro I can think of to doing it as a separate module is setting the = precedent to stack vbdevs to create more complex RAID levels but I think th= at's more RAID complexity that we want or need for SPDK but certainly and o= pen to feedback on that point. >> = >> All of this stuff will be phased in over a series of patches of course b= ut I guess we can decide at what point it's considered non-experimental bas= ed on feature set. I'll send out a more complete list of proposed features= and include at least some basic stuff in what I'd call the first "producti= on" version and we can go from there. >> = >> Wrt an implantation details like abstracting common operations and using= a function table I can appreciate that input as well. For R/W it may not = be necessary though, at least based on my POC which has very few changes to= how strip locations and physical disk identifiers are calculated. That can= all be part of review feedback on the actual patches. I hope to start post= ing next week if not sooner. Either way at the RPC level I think it's clea= r enough that we'll have very distinct RAID levels. >> = >> Anyway, thanks again and I'll work on a more detailed feature set defini= tion based on your feedback! >> = >> Thx >> Paul >> = >> -----Original Message----- >> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of ???? / MATS= UMOTO,SHUUHEI >> Sent: Wednesday, September 4, 2019 7:28 PM >> To: Storage Performance Development Kit >> Subject: Re: [SPDK] Replication for SPDK (RAID 1E) >> = >> Hi Paul, >> = >> It=E2=80=99s great to know you work on raid! >> = >> Ziye proposed a patch for RAID1 before RAID bdev module maybe more than = a year ago. >> = >> IMHO, we will need to have disk replacement feature and degradation mode= first. It may be difficult to use raid1 without them. Because raid1 is for= RAS >> = >> We need to have clean abstraction for RAID level, I.e., extracting commo= n operations and creating function pointer table. >> = >> We need to have copy feature between two bdevs. >> = >> Thanks >> Shuhei >> = >> Sent from my iPhone >> = >>> On Sep 5, 2019, at 8:49, Luse, Paul E wrote: >>> = >>> Hi Everyone, >>> = >>> I've got a pretty simple POC working and wanted to solicit any high lev= el input before I get too far. The idea is to, of course, start very basic= but leave room for adding features later. Here are the broad strokes: >>> = >>> = >>> * Add a new RAID level to the existing RAID module with level "1E= " that requires a "number of replicas" parameter >>> = >>> * The cool thing about 1E is that we can use any number of disks = and also pick the number of times the data is replicated so for example: >>> = >>> o 2 disk 1E with replication of 2 would be your basic 2 disk RAID1. M= apped as follows (columns are physical disks Dn identifies data copies) >>> D0 D0 >>> D1 D1 >>> D2 D2 >>> = >>> * 3 disk 1E with replication of 2: >>> = >>> D0 D0 D1 >>> = >>> D1 D2 D2 >>> = >>> D3 D3 D4 >>> = >>> * 3 disk 1E with replication of 3: >>> = >>> D0 D0 D0 >>> = >>> D1 D1 D1 >>> = >>> D2 D2 D2 >>> = >>> * 3 disk 1E with replication of 1 (RAID0) >>> D0 D1 D2 >>> D3 D4 D5 >>> D6 D7 D8 >>> = >>> * This scheme is obviously very flexible and can provide basic RA= ID1 without disk restrictions and also provide for some super paranoid conf= igs >>> = >>> * At the same time we could consider limiting, at least at first,= the combinations of disks and replicas to minimize complexity and test but= IMHO I think we should leave it wide open >>> = >>> * An even cooler part of this is how well the current implementat= ion lends itself to this. A RAID0, behind the scenes, becomes a RAID1E wit= h 1 replica >>> = >>> = >>> = >>> = >>> Initially we can start with just the RAID level addition (no notificati= on of member failure, no rebuilds, spares, etc). as I don't believe there's= really any existing framework to support these kinds of features. This is= the main question I have for interested parties. Would this be useful wit= hout any of the recovery type features or should we at least have some sort= of async notification on member disk failure when num_replicas > 1? >>> = >>> Trello link: whether it is feasible for 19.10 or not depends on feedbac= k from everyone on features :) https://clicktime.symantec.com/3Ui9u3JQhDqBY= 9P7euvmPEF7Vc?u=3Dhttps%3A%2F%2Ftrello.com%2Fc%2FFR4iHAnI >>> = >>> Thanks! >>> Paul >>> = >>> PS: My current POC is super raw. I have hardcoded number of replicas t= o 3 and have what I believe is the correct block mapping for any # of disks= , any # of replicas but have only tested 3 replicas with 2 and 3 member dis= ks using bdevperf w/verify. After I get it in presentable shape and flesh t= he design and UT out a bit more I'll post something. >>> = >>> = >>> = >>> _______________________________________________ >>> SPDK mailing list >>> SPDK(a)lists.01.org >>> https://clicktime.symantec.com/3NBASeDzyXVbw78PP7EaJro7Vc?u=3Dhttps%3A%= 2F%2Flists.01.org%2Fmailman%2Flistinfo%2Fspdk >> _______________________________________________ >> SPDK mailing list >> SPDK(a)lists.01.org >> https://clicktime.symantec.com/3DyCUvVSi4zpXYkHUqirtJs7Vc?u=3Dhttps%3A%2= F%2Flists.01.org%2Fmailman%2Flistinfo%2Fspdk >> _______________________________________________ >> SPDK mailing list >> SPDK(a)lists.01.org >> https://clicktime.symantec.com/3DyCUvVSi4zpXYkHUqirtJs7Vc?u=3Dhttps%3A%2= F%2Flists.01.org%2Fmailman%2Flistinfo%2Fspdk > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk --===============4994136311866134615==--