Re: [PATCH v11 00/20] nfs/nfsd: add support for localio

Mike Snitzer
2024-07-06T03:58:58Z

On Fri, Jul 05, 2024 at 02:59:31PM +0000, Chuck Lever III wrote:
>
>
> > On Jul 5, 2024, at 10:36 AM, Mike Snitzer <snitzer@kernel.org> wrote:
> >
> > On Fri, Jul 05, 2024 at 07:18:29AM -0700, Christoph Hellwig wrote:
> >> On Fri, Jul 05, 2024 at 10:15:46AM -0400, Mike Snitzer wrote:
> >>> NFSv3 is needed because NFSv3 is used to initiate IO to NFSv3 knfsd on
> >>> the same host.
> >>
> >> That doesn't really bring is any further. Why is it required?
> >>
> >> I think we'll just need to stop this discussion until we have reasonable
> >> documentation of the use cases and assumptions, because without that
> >> we'll get hund up in dead loops.
> >
> > It _really_ isn't material to the core capability that localio provides.
> > localio supporting NFSv3 is beneficial for NFSv3 users (NFSv3 in
> > containers).
> >
> > Hammerspace needs localio to work with NFSv3 to assist with its "data
> > movers" that run on the host (using nfs and nfsd).
> >
> > Please just remove yourself from the conversation if you cannot make
> > sense of this. If you'd like to be involved, put the work in to
> > understand the code and be professional.
>
> Sorry, I can't make sense of this either, and I find the
> personal attack here completely inappropriate (and a bit
> hypocritical, to be honest).

Hi Chuck,

I'm out-gunned with this good-cop/bad-cop dynamic. I was replying to
Christoph. Who has taken to feign incapable of understanding localio
yet is perfectly OK with flexing like he is an authority on the topic.

He rallied to your Nacked-By with his chest puffed up and has
proceeded to baselessly shit-talk (did you miss his emails while we
slept last night?). Yes, let's condone and encourage more of that!?
No, I won't abide such toxicity. But thankfully Neil has since called
for him to stop. Alas...

Earlier today I answered the question about "why NFSv3?" in simple
terms. You and Christoph rejected it. I'm _not_ being evassive.
There isn't a lot to it: "More efficient NFS in containers" _is_ the
answer.

But hopefully this email settles "why NFSv3?". If not, please help me
(or others) further your understanding by reframing your NFSv3 concern
in terms other than "why NFSv3?". Getting a bit like having to answer
"why is water wet?"

Why NFSv3?
----------

The localio feature improves IO performance when using NFSv3 and NFSv4
with containers. Hammerspace has immediate need for the NFSv3
support, because its "data movers" use NFSv3, but NFSv4 support is
expected to be useful in the future.

Just because Hammerspace is very invested in pNFS doesn't mean all
aspects are framed in terms of it.

General statement:
------------------

I wrote maybe ~30% of the entire localio code as it stands at "v11"
and that was focused primarily on adding NFSv4 support and developing
the localio protocol, hooking it into NFS's client initialization and
teardown along with the server (and vice-versa, nfsd lifetime due to
container applications: tearing down nfsd in container while nfs
client actively connected from host). Neil helped refine the localio
protocol part, and he has also looked critically at many aspects and
has a great list of improvements that are needed. Jeff provided
top-notch review of my initial use of SRCU and later the percpu refcnt
for interlocking with the client and server.

My point: others wrote the majority of localio (years ago). I'm just
trying to shepherd it upstream in an acceptable form. And yes,
localio supporting both NFSv3 and NFSv4 is important to me,
Hammerspace and anyone who'd like more efficient IO with both NFSv3
and NFSv4 in containers.

Answering "Why NFSv3?" with questions:
--------------------------------------

1) Why wasn't general NFS localio bypass controversial 3 weeks ago?
Instead (given all inputs, NFSv3 support requirement being one of
them) the use of a "localio protocol" got broad consensus and buyin
from you, Jeff, and Neil.

I _thought_ we all agreed localio was a worthwhile _auxilliary_
addition to Linux's NFS client and server (to give them awareness of
each other through nfs_common) regardless of NFS protocol version.
That is why I registered a localio RPC program number with IANA (at
your suggestion, you were cc'd when I applied for it, and you are
named on IANA.org along with Trond and myself for the program number
IANA assigned):
https://www.iana.org/assignments/rpc-program-numbers/rpc-program-numbers.txt

$ cat rpc-program-numbers.txt | egrep 'Snitzer|Myklebust|Lever'
Linux Kernel Organization 400122 nfslocalio [Mike_Snitzer][Trond_Myklebust][Chuck_Lever]
[Chuck_Lever] Chuck Lever mailto:chuck.lever&oracle.com 2024-06-20
[Mike_Snitzer] Mike Snitzer mailto:snitzer&kernel.org 2024-06-20
[Trond_Myklebust] Trond Myklebust mailto:trondmy&hammerspace.com 2024-06-20

2) If we're introducing a general NFS localio bypass feature _and_
NFSv3 is important to the stakeholder proposing the feature _and_
NFSv3 support is easily implemented and supported: then why do you
have such concern about localio supporting NFSv3?

3) Why do you think NFSv3 unworthy? Is this just a bellweather for
broader opposition to flexfiles (and its encouraging more use of
NFSv3)? Flexfiles is at the heart of NFSv3 use at Hammerspace. I've
come to understand from you and Christoph that the lack of flexfiles
support in NFSD helps fuel dislike for flexfiles. That's a lot for me
to unpack, and pretty far removed from "why NFSv3?", so I'd need more
context than I have if Hammerspace's use of flexfiles is what is
fueling your challenge of localio's NFSv3 support.

...

Reiterating and then expanding on my email above:

localio supporting NFSv3 is beneficial for NFSv3 users (NFSv3 in
containers).

Hammerspace needs localio to work with NFSv3 to assist with its
"data movers" that run on the host (using nfs [on host] and nfsd
[within container]).

Now you can ask why _that_ is.. but it really is pretty disjoint from
the simple matter of ensuring localio support both NFSv3 and NFSv4.

I've shared that Hammerspace's "data movers" use NFSv3 currently, in
the future they could use NFSv4 as needed. Hence the desire to
support localio with both NFSv3 and NFSv4. [when I picked up the
localio code NFSv4 wasn't even supported yet].

I _hope_ I've now answered "why NFSv3?" clearly.

> I have nothing else to contribute that you won't either
> dismiss or treat as a personal attack, so I can't continue
> this conversation.

That isn't even a little bit fair... but not taking the bait.

Neil has been wonderful to work with and I look forward to all future
work with him (localio and beyond). I am not trying to do anything
out of line with this feature. I am and have been actively working
with you, Neil and Jeff for over a month now. I've adapted and
learned, _with_ your and others' help, to the best of my ability.

I'm trying here, maybe you could say "I'm trying too hard". Well I
just started a new job with Hammerspace after working for Red Hat for
the past 15 years (much of my time spent as the upstream Linux DM
maintainer -- but you know this). I am a capable engineer and I've
proposed the upstreaming of a localio feature that would do well to
land upstream. I've done so in a proficient way all things
considered, always happy to learn new things and improve. I need to
work with you. Hopefully well, and hopefully I can earn your respect,
please just know I'm merely trying to improve NFS.

Hammerspace would like to get all its Linux kernel NFS innovation
upstream. And I'm trying to do that. localio is my first task and
I've been working on it with focus for the past 2 months since joining
Hammerspace. But you basically know all this, I said all of it to you
at LSF.

So if you know all these things (I _know_ you do), why are you
treating me in this way? I feel like I'm caught in the middle of some
much bigger divide than anything I've been involved with, caused or
made privy to.

Guess the messenger gets shot sometimes.

Mike