Re: [RFC PATCH v3 00/13] Basic multipathing support and demonstrator

[Tom Talpey <tom@talpey.com>]

On 2/1/2016 4:12 PM, Trond Myklebust wrote:
> On Mon, Feb 1, 2016 at 3:48 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>
>> On Mon, Feb 01, 2016 at 12:21:04PM -0500, Trond Myklebust wrote:
>>> The following patchset is intended as an RFC to demonstrate how we could
>>> implement multipathing in NFS.
>>> It adds basic support for attaching multiple RPC transports to a client,
>>> an embryonic policy mechanism for switching tasks across those transports
>>> (with a round-robin policy implementation),...
>>
>> Since your prototype is doing round-robin per task, I assume the
>> motivating problem is that you don't want to waste bandwidth when
>> there's multiple links available.  Is that right, or is there some other
>> goal you have in mind?
>>
>
> I did round-robin simply because it is easy to do, and because server
> vendors have been asking for it. That said, the policy is readily
> pluggable, and it should be straightforward to add path failover,
> should you want to.

I think servers probably want load balancing, which is not quite the
same thing as round robin. When we implemented mulitchannel in SMB3,
we quickly discovered many issues at scale, especially when network
links introduced additional variance. We eventually settled on a
selection method that attempted to balance outstanding queue depth,
as well as round robining.

Do you plan to support links of different speed or other characteristics
e.g. mixing TCP and RDMA? That introduces a number of other issues; in
SMB3 we simply avoid ever doing those. Channel selection can also be
used to tune multiprocessor affinity. There were Storage Developer
Conference talks back in 2012 and maybe 2013 that discussed the approaches.

You may also want to talk to the Samba folks, who are implementing SMB3
multichannel (server) recently, and may have insight to share.

All that said, it's great to see this.

Tom.