[SPDK] Change in ...spdk[master]: Initial checkin of the multipath vbdev driver.

[SPDK] Change in ...spdk[master]: Initial checkin of the multipath vbdev driver.

[Andrey Kuzmin]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

I will appreciate an input on if the bdev-level multipathing is of
general interest and is worth inclusion. On positive determination I
can find time to rebase and resubmit.

Notice also that, in this case, decoupling open in write mode from
claims patch (https://review.gerrithub.io/c/spdk/spdk/+/422177) should
be revisited and included, as otherwise implementing multiple paths to
the same base bdev as  path bdevs is rather problematic (as this
obviously involves multiple writers).

Thanks,
Andrey

Re: [SPDK] Change in ...spdk[master]: Initial checkin of the multipath vbdev driver.

[Andrey Kuzmin]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

Ben,

you had indicated that you'd like to have a discussion of the multipath
driver. Let me know what you think.

Thanks,
Andrey

On Fri, Feb 1, 2019, 21:12 Andrey Kuzmin <akuzmin(a)jetstreamsoft.com wrote:

> I will appreciate an input on if the bdev-level multipathing is of
> general interest and is worth inclusion. On positive determination I
> can find time to rebase and resubmit.
>
> Notice also that, in this case, decoupling open in write mode from
> claims patch (https://review.gerrithub.io/c/spdk/spdk/+/422177) should
> be revisited and included, as otherwise implementing multiple paths to
> the same base bdev as  path bdevs is rather problematic (as this
> obviously involves multiple writers).
>
> Thanks,
> Andrey
>

Re: [SPDK] Change in ...spdk[master]: Initial checkin of the multipath vbdev driver.

[Walker, Benjamin]

[-- Attachment #1: Type: text/plain, Size: 2174 bytes --]

On Fri, 2019-02-15 at 19:48 +0300, Andrey Kuzmin wrote:
> Ben,
> 
> you had indicated that you'd like to have a discussion of the multipath
> driver. Let me know what you think.

I haven't had a chance to through the actual implementation of the patch, but
multipathing in general is certainly of interest and warrants inclusion in SPDK.
I'd love to move ahead on this.

The big challenge is figuring out the best place to implement it. There are at
least 3 different options that I see.

1) Implement it as a bdev module. When the module gets a claim callback, it
checks to see if the new bdev is simply a different path to an existing bdev
(using some mechanism that isn't defined yet?) and then claims them both and
exposes a single bdev to the upper layer.

2) Implement it in the generic bdev code so that each individual bdev module can
call some function to add additional paths to a given bdev.

3) Implement it inside the nvme driver. There are a number of really cool
features for multipathing in NVMe (ANA and such) that don't translate well to a
generic bdev. The only way to take advantage of some of these might be to add
support directly into the nvme driver.

It could be that we end up doing a combination of 1 or 2 with number 3 as well.
I'd love some other opinions here from interested parties.

> 
> Thanks,
> Andrey
> 
> On Fri, Feb 1, 2019, 21:12 Andrey Kuzmin <akuzmin(a)jetstreamsoft.com wrote:
> 
> > I will appreciate an input on if the bdev-level multipathing is of
> > general interest and is worth inclusion. On positive determination I
> > can find time to rebase and resubmit.
> > 
> > Notice also that, in this case, decoupling open in write mode from
> > claims patch (https://review.gerrithub.io/c/spdk/spdk/+/422177) should
> > be revisited and included, as otherwise implementing multiple paths to
> > the same base bdev as  path bdevs is rather problematic (as this
> > obviously involves multiple writers).
> > 
> > Thanks,
> > Andrey
> > 
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] Change in ...spdk[master]: Initial checkin of the multipath vbdev driver.

[Andrey Kuzmin]

[-- Attachment #1: Type: text/plain, Size: 3247 bytes --]

On Sat, Feb 16, 2019 at 12:33 AM Walker, Benjamin
<benjamin.walker(a)intel.com> wrote:
>
> On Fri, 2019-02-15 at 19:48 +0300, Andrey Kuzmin wrote:
> > Ben,
> >
> > you had indicated that you'd like to have a discussion of the multipath
> > driver. Let me know what you think.
>
> I haven't had a chance to through the actual implementation of the patch, but
> multipathing in general is certainly of interest and warrants inclusion in SPDK.
> I'd love to move ahead on this.
>
> The big challenge is figuring out the best place to implement it. There are at
> least 3 different options that I see.
>
> 1) Implement it as a bdev module. When the module gets a claim callback, it
> checks to see if the new bdev is simply a different path to an existing bdev
> (using some mechanism that isn't defined yet?) and then claims them both and
> exposes a single bdev to the upper layer.

This is very much how the multipath driver works, and it can easily be
extended to run off the uuids (this will need some extra effort on the
bdev later side, though, as virtual and some base bdevs do not have
uuid defined by default).

>
> 2) Implement it in the generic bdev code so that each individual bdev module can
> call some function to add additional paths to a given bdev.
>

It still would need path failover/failback functionality, so there
needs to be a multipathing component in the bdev layer then.

> 3) Implement it inside the nvme driver. There are a number of really cool
> features for multipathing in NVMe (ANA and such) that don't translate well to a
> generic bdev. The only way to take advantage of some of these might be to add
> support directly into the nvme driver.
>

I'm not very proficient in ANA as it has not been published yet, and
will be glad to get more input on this. One ANA-specific feature from
the upcoming NVMe spec update - path error reporting - could have
definitely improved multipathing if exposed to bdev layer users since
that would allow to differentiate between device and path errors.

Regards,
Andrey

> It could be that we end up doing a combination of 1 or 2 with number 3 as well.
> I'd love some other opinions here from interested parties.
>
> >
> > Thanks,
> > Andrey
> >
> > On Fri, Feb 1, 2019, 21:12 Andrey Kuzmin <akuzmin(a)jetstreamsoft.com wrote:
> >
> > > I will appreciate an input on if the bdev-level multipathing is of
> > > general interest and is worth inclusion. On positive determination I
> > > can find time to rebase and resubmit.
> > >
> > > Notice also that, in this case, decoupling open in write mode from
> > > claims patch (https://review.gerrithub.io/c/spdk/spdk/+/422177) should
> > > be revisited and included, as otherwise implementing multiple paths to
> > > the same base bdev as  path bdevs is rather problematic (as this
> > > obviously involves multiple writers).
> > >
> > > Thanks,
> > > Andrey
> > >
> >
> > _______________________________________________
> > SPDK mailing list
> > SPDK(a)lists.01.org
> > https://lists.01.org/mailman/listinfo/spdk
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk