Re: [PATCH net-next 00/13] Control action percpu counters allocation by netlink flag

[Vlad Buslov <vladbu@mellanox.com>]

On Thu 24 Oct 2019 at 20:31, Marcelo Ricardo Leitner <mleitner@redhat.com> wrote:
> On Thu, Oct 24, 2019 at 03:18:00PM +0000, Vlad Buslov wrote:
>>
>> On Thu 24 Oct 2019 at 18:12, Roopa Prabhu <roopa@cumulusnetworks.com> wrote:
>> > On Tue, Oct 22, 2019 at 10:10 AM Marcelo Ricardo Leitner
>> > <mleitner@redhat.com> wrote:
>> >>
>> >> On Tue, Oct 22, 2019 at 03:52:31PM +0000, Vlad Buslov wrote:
>> >> >
>> >> > On Tue 22 Oct 2019 at 18:15, Marcelo Ricardo Leitner <mleitner@redhat.com> wrote:
>> >> > > On Tue, Oct 22, 2019 at 05:17:51PM +0300, Vlad Buslov wrote:
>> >> > >> - Extend actions that are used for hardware offloads with optional
>> >> > >>   netlink 32bit flags field. Add TCA_ACT_FLAGS_FAST_INIT action flag and
>> >> > >>   update affected actions to not allocate percpu counters when the flag
>> >> > >>   is set.
>> >> > >
>> >> > > I just went over all the patches and they mostly make sense to me. So
>> >> > > far the only point I'm uncertain of is the naming of the flag,
>> >> > > "fast_init".  That is not clear on what it does and can be overloaded
>> >> > > with other stuff later and we probably don't want that.
>> >> >
>> >> > I intentionally named it like that because I do want to overload it with
>> >> > other stuff in future, instead of adding new flag value for every single
>> >> > small optimization we might come up with :)
>> >>
>> >> Hah :-)
>> >>
>> >> >
>> >> > Also, I didn't want to hardcode implementation details into UAPI that we
>> >> > will have to maintain for long time after percpu allocator in kernel is
>> >> > potentially replaced with something new and better (like idr is being
>> >> > replaced with xarray now, for example)
>> >>
>> >> I see. OTOH, this also means that the UAPI here would be unstable
>> >> (different meanings over time for the same call), and hopefully new
>> >> behaviors would always be backwards compatible.
>> >
>> > +1, I also think optimization flags should be specific to what they optimize.
>> > TCA_ACT_FLAGS_NO_PERCPU_STATS seems like a better choice. It allows
>> > user to explicitly request for it.
>>
>> Why would user care about details of optimizations that doesn't produce
>> visible side effects for user land? Again, counters continue to work the
>> same with or without the flag.
>
> It's just just details of optimizations, on whether to use likely() or
> not, and it does produce user visible effects. Not in terms of API but
> of system behavior. Otherwise we wouldn't need the flag, right?
> _FAST_INIT, or the fact that it inits faster, is just one of the
> aspects of it, but one could also want it for being lighther in
> footprint as currently it is really easy to eat Gigs of RAM away on
> these percpu counters. So how should it be called, _FAST_INIT or
> _SLIM_RULES?

Hmm, yes, I think I emphasize insertion rate too much because it was my
main goal, but memory usage is also important.

>
> It may be implementation detail, yes, but we shouldn't be building
> usage profiles and instead let the user pick what they want. Likewise,
> we don't have net.ipv4.fast_tcp, but net.ipv4.tcp_sack, tcp_timestamps
> & cia.
>
> If we can find another name then, without using 'percpu' on it but
> without stablishing profiles, that would be nice.
> Like TCA_ACT_FLAGS_SIMPLE_STATS, or so.
>
> Even though I still prefer the PERCPU, as it's as explicit as it can be. Note
> that bpf does it like that already:
> uapi]$ grep BPF.*HASH -- linux/bpf.h
>         BPF_MAP_TYPE_HASH,
>         BPF_MAP_TYPE_PERCPU_HASH,
>         BPF_MAP_TYPE_LRU_HASH,
>         BPF_MAP_TYPE_LRU_PERCPU_HASH,
> ...

I would argue that all of these produce visible side effect for the
user. Selective acks are clearly visible in packet dumps, BPF maps are
directly accessed from BPF programs loaded from user space, etc. But I
get your point that naming can be improved from FAST_INIT which is too
abstract.