Re: [PATCH net-next v3 0/4] Add support to do threaded napi busy poll

[Joe Damato <jdamato@fastly.com>]

On Wed, Feb 05, 2025 at 08:43:48PM -0800, Samiullah Khawaja wrote:
> On Wed, Feb 5, 2025 at 5:15 PM Martin Karsten <mkarsten@uwaterloo.ca> wrote:
> >
> > On 2025-02-05 17:06, Joe Damato wrote:
> > > On Wed, Feb 05, 2025 at 12:35:00PM -0800, Samiullah Khawaja wrote:
> > >> On Tue, Feb 4, 2025 at 5:32 PM Martin Karsten <mkarsten@uwaterloo.ca> wrote:
> > >>>
> > >>> On 2025-02-04 19:10, Samiullah Khawaja wrote:
> >
> > [snip]
> >
> > >>> Note that I don't dismiss the approach out of hand. I just think it's
> > >>> important to properly understand the purported performance improvements.
> > >> I think the performance improvements are apparent with the data I
> > >> provided, I purposefully used more sockets to show the real
> > >> differences in tail latency with this revision.
> > >
> > > Respectfully, I don't agree that the improvements are "apparent." I
> > > think my comments and Martin's comments both suggest that the cover
> > > letter does not make the improvements apparent.
> > >
> > >> Also one thing that you are probably missing here is that the change
> > >> here also has an API aspect, that is it allows a user to drive napi
> > >> independent of the user API or protocol being used.
> > >
> > > I'm not missing that part; I'll let Martin speak for himself but I
> > > suspect he also follows that part.
> >
> > Yes, the API aspect is quite interesting. In fact, Joe has given you
> > pointers how to split this patch into multiple incremental steps, the
> > first of which should be uncontroversial.
> >
> > I also just read your subsequent response to Joe. He has captured the
> > relevant concerns very well and I don't understand why you refuse to
> > document your complete experiment setup for transparency and
> > reproducibility. This shouldn't be hard.
> I think I have provided all the setup details and pointers to
> components. I appreciate that you want to reproduce the results and If
> you really really want to set it up then start by setting up onload on
> your platform. I cannot provide a generic installer script for onload
> that _claims_ to set it up on an arbitrary platform (with arbitrary
> NIC and environment). If it works on your platform (on top of AF_XDP)
> then from that point you can certainly build neper and run it using
> the command I shared.

I'm going to reproduce this on a GCP instance like you illustrated
in your cover letter, but there is a tremendous lack of detail to
get the system into a starting state.

As Martin explains in his follow: I am not asking for a generic
installer script for onload. I can definitely compile that program
on a GCP instance myself.

What we are both asking for is more information on how the system is
setup or, as Martin has repeatedly asked for, a script that gets the
system into its initial state for each experiment to streamline
reproduction of results.

I'm confused why your responses seem so strongly opposed to what we
are asking for?

Neither I nor Martin are against a new mechanism for busy polling
being introduced; we are both saying that a general purpose
mechanism that burns 100% CPU even when the network is idle should
be carefully considered. Wouldn't you agree?

The best way to carefully consider it would be to include more data
and more measurements of important system metrics, like CPU cycle
counts, CPU consumption, etc.

I don't intend to speak for Martin, but I'm asking for (and I think
he'd agree):
  - More rigorous analysis, with more test cases and data (like CPU
    consumption, socket counts, message sizes, etc)
  - Better documentation of how one configures the system, possibly
    using a script. It's OK if the script only works on the exact
    GCP instance you used; I'd like to create that GCP instance and
    know 100% that I have the same starting state as you when I run
    the test.
  - If more test cases are going to be tested and the data extracted
    for a cover letter, a script that generates this in a standard
    format with all of the metrics would be very helpful to ensure
    we are all measuring this the same way, using the same command
    line tools, with the same arguments. In other words, if you
    decide to run "perf" to count cycles or cache misses or whatever
    it is, capturing that in a script ensures we are all measuring
    this the same way.

If you think this is unreasonable, please let me know; your
responses suggest you are opposed to more detailed analysis and I am
trying to understand why that might be.