From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============9011913076673663780==" MIME-Version: 1.0 From: Sasha Kotchubievsky Subject: [SPDK] Re: SPDK RAID5 support Date: Wed, 16 Oct 2019 15:19:38 +0300 Message-ID: <04f901d5841b$fb619f00$f224dd00$@dev.mellanox.co.il> In-Reply-To: 9467DF06-B702-489C-95C6-339B3A7728C5@intel.com List-ID: To: spdk@lists.01.org --===============9011913076673663780== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Thank Paul, I understand your point. = In any case, I would suggest to consider to add support for RAID6 along wit= h RAID5. There is demand RAID5/5 solution based on SPDK and I think, this feature wi= ll get great feedback from the field. Last year, we evaluated RAID5/6 implementation in SPDK. In our POC we used = RAID0 vbdev with our extensions. One of challenges was to get max possibl= e performance Read-modify-write scenario. = I'll definitely keep eye on coming patches. And my group, will help as we c= an with the feature. Best regards Sasha = -----Original Message----- From: Luse, Paul E = Sent: Wednesday, October 16, 2019 12:47 AM To: Sasha Kotchubievsky ; Storage Performanc= e Development Kit Cc: Karkra, Kapil ; Baldysiak, Pawel ; Ptak, Slawomir Subject: Re: [SPDK] Re: SPDK RAID5 support Good feedback Sasha. We need to be cautious about how much we "bite off" t= hough. EC is quite different from RAID5 in terms of mapping and rebuild - t= he parity calculations are the easiest part of it since we can rely on ISAL. It should be kept in mind though for sure, I just want to make sure we don'= t try and do too much right away. I've only got half-way through the chain = so far and I think it looks right on the money. Thx Paul =EF=BB=BFOn 10/15/19, 2:21 PM, "Sasha Kotchubievsky" wrote: Hi , = It looks like a massive refactoring. = In case of such big refactoring, I'd suggest to think about more genera= l = solution like erasure coding. RAD5/6 are trivial cases of erasure = coding. I believe, "management": disk degradation and recovery are = similar for EC and for RAID. Actual calculation of parity block can be = done in ISA-lib under an abstraction layer ( which can be replaced by = any other implementation). = = Best regards = Sasha = On 13-Oct-19 9:18 PM, Luse, Paul E wrote: > Note that there's a patch series up there now from Artur also, I'll b= e reviewing it myself for the first tim here shortly ( > > https://review.gerrithub.io/c/spdk/spdk/+/471075/1 > > On 10/13/19, 10:39 AM, "Luse, Paul E" wrote: > > Awesome! Good feedback, keep an eye out for patches and check ou= t trello https://trello.com/b/4HEkWVvF/raid for backlog items... > = > Thx > Paul > = > On 10/13/19, 2:28 AM, "Sasha Kotchubievsky" wrote: > = > Hi, > = > I'm very exiting to progress in RAID development in SPDK. > = > Artur, will you focus on RAID5 only, or RAID6 is also will b= e supported? > = > I think, it's important to keep existing SPDK approach for c= onfiguration. > It would nice to see an abstraction for parity calculation. = I believe, the calculation can be optimized for specific platform, or even = using HW accelerators is possible . > In case of distributed storage, even existing in market netw= ork cards can optimize RAID related operations. I believe, the next, upcomi= ng generation will this capabilities to the next level. That needs some sup= port from bdev layer like events about configuration changes, or recovery/d= egradation. > = > = > Best regards > Sasha > = > -----Original Message----- > From: =E6=9D=BE=E6=9C=AC=E5=91=A8=E5=B9=B3 / MATSUMOTO=EF=BC= =8CSHUUHEI > Sent: Friday, October 4, 2019 1:49 AM > To: Storage Performance Development Kit > Cc: Karkra, Kapil ; Baldysiak, Paw= el ; Ptak, Slawomir > Subject: [SPDK] Re: SPDK RAID5 support > = > Hi Artur, Paul, and All, > = > Thank you so much, I'm excited to know this. > = > Recently SPDK are starting to support DIF feature. > Can we have any possibility to include the extended LBA (blo= ck size =3D 512 + 8, 4096 + 128, or etc) into SPDK RAID? > Do you have any comment? > = > Thanks, > Shuhei > = > ________________________________ > =E5=B7=AE=E5=87=BA=E4=BA=BA: Luse, Paul E > =E9=80=81=E4=BF=A1=E6=97=A5=E6=99=82: 2019=E5=B9=B410=E6=9C= =884=E6=97=A5 4:20 > =E5=AE=9B=E5=85=88: Storage Performance Development Kit > CC: Karkra, Kapil ; Baldysiak, Paw= el ; Ptak, Slawomir > =E4=BB=B6=E5=90=8D: [SPDK] Re: SPDK RAID5 support > = > Hi Artur, > = > Thanks, I think this can be an awesome contribution. A few = other things to consider, you mention some of these already, so I just adde= d some more color. It would be good I think moving forward to put a trello= board up with a backlog of tasks so that others (like me __) can jump in a= nd help also. I'd still like to get the RAID1E in there, but it makes litt= le sense to do it before any major refactoring. > = > I think it makes sense, if you guys are ready, to start putt= ing together the backlog and knocking out a large series of small patches t= o get the refactoring done. > = > * the current RAID0 has no config on disk. We'll need to com= e up with a scheme for handling existing RAID0 configured out of band with = RAID5 using COD. (like not allowing a RAID0 to be built on a set with COD, = etc.) > * the metadata layout, as you knee from working previous RAI= D projects, needs to be thought out carefully to consider not only extensib= ility but version control for backwards compactivity and issues with confli= cting COD that are found. For example you have a 3 disk RAID5, take one of= the disks out and use it in another array somewhere then bring it back lat= er and fire up the original 3, the metadata has to have sufficient info to = know who belongs to what and which volumes to create and which to put in so= me sort of offline state. I don't think we need that kind of capability rig= ht up front (deciding how to deal with conflicts) but the metadata should h= ave enough information it up front. DDF was brought up before in the earlie= r RAID discussions so just to make sure we're all on the same page, I see n= o value in complying with that spec. Open to other thoughts though. > * one thing to keep in mind, we probably want to retain comm= on RPC code > * we should keep migration in mind as well, not that it's so= mething we may ever need/want but lots of reserved space in metadata for tr= acking state wrt migrations and rebuilds is needed. > * similar to the RAID1E discussions earlier, there are some = features you mention as 'future' that most would consider a requirement for= redundant RAID - like degrade operation and rebuild. Those don't all have = to go in at the same time however without that minimum set we need to mark = it experimental, so nobody tries to use it thinking it has those basic thin= gs. > * I can't remember if we talked about unit tests or not, but= be sure the backlog includes getting solid UT coverage in there up front. = The existing UT code will likely need some refactoring as well to support = the function code refactoring. > = > -Paul > = > On 10/3/19, 3:00 AM, "Artur Paszkiewicz" wrote: > = > Hi all, > = > We want to add RAID5 support to SPDK. My team has experi= ence with other RAID > projects, primarily with Linux MD RAID, which we activel= y develop and support > for Intel VROC. We already have an initial SPDK RAID5 im= plementation created > for an internal project. It has working read/write, incl= uding partial-stripe > updates, parity calculation and reconstruct-reads. > = > Currently in SPDK there exists a RAID bdev module, which= has only RAID0 > functionality. This can be used as a basis for a more ge= neric RAID stack. Here > is our idea how to approach this: > = > 1. Refactor the bdev_raid module to separate RAID0-speci= fic I/O handling code > from more generic parts - configuration, bdev creation, = etc. Move the RAID0 > code to a new file. Use RAID level-specific callbacks, s= imilar to existing > struct raid_fn_table. This architecture is also used in = MD RAID drivers, where > different RAID "personalities" work on top of a common l= ayer. > = > 2. Add RAID5 support in another file, similarly to RAID0= . Port our current > RAID5 code to this new framework. > = > 3. Incrementally add new functionalities. At this point,= probably the most > important will be support for member drive failure and d= egraded operation, RAID > rebuild and some form of on-disk metadata. > = > Any comments or suggestions are welcome. > = > Thanks, > Artur > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org > = > = > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org __= _____________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org > = > = > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org > = > > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org > To unsubscribe send an email to spdk-leave(a)lists.01.org = --===============9011913076673663780==--