RE: [RFC PATCH 00/13] Ultra Ethernet driver introduction

Bernard Metzler
2025-03-14T14:53:44Z

> -----Original Message-----
> From: Leon Romanovsky <leon@kernel.org>
> Sent: Wednesday, March 12, 2025 4:11 PM
> To: Nikolay Aleksandrov <nikolay@enfabrica.net>
> Cc: netdev@vger.kernel.org; shrijeet@enfabrica.net;
> alex.badea@keysight.com; eric.davis@broadcom.com; rip.sohan@amd.com;
> dsahern@kernel.org; Bernard Metzler <BMT@zurich.ibm.com>;
> roland@enfabrica.net; winston.liu@keysight.com;
> dan.mihailescu@keysight.com; Kamal Heib <kheib@redhat.com>;
> parth.v.parikh@keysight.com; Dave Miller <davem@redhat.com>;
> ian.ziemba@hpe.com; andrew.tauferner@cornelisnetworks.com; welch@hpe.com;
> rakhahari.bhunia@keysight.com; kingshuk.mandal@keysight.com; linux-
> rdma@vger.kernel.org; kuba@kernel.org; Paolo Abeni <pabeni@redhat.com>;
> Jason Gunthorpe <jgg@nvidia.com>
> Subject: [EXTERNAL] Re: [RFC PATCH 00/13] Ultra Ethernet driver
> introduction
>
> On Wed, Mar 12, 2025 at 04:20:08PM +0200, Nikolay Aleksandrov wrote:
> > On 3/12/25 1:29 PM, Leon Romanovsky wrote:
> > > On Wed, Mar 12, 2025 at 11:40:05AM +0200, Nikolay Aleksandrov wrote:
> > >> On 3/8/25 8:46 PM, Leon Romanovsky wrote:
> > >>> On Fri, Mar 07, 2025 at 01:01:50AM +0200, Nikolay Aleksandrov wrote:
> > [snip]
> > >> Also we have the ephemeral PDC connections>> that come and go as
> > needed. There more such objects coming with more
> > >> state, configuration and lifecycle management. That is why we added a
> > >> separate netlink family to cleanly manage them without trying to fit
> > >> a square peg in a round hole so to speak.
> > >
> > > Yeah, I saw that you are planning to use netlink to manage objects,
> > > which is very questionable. It is slow, unreliable, requires sockets,
> > > needs more parsing logic e.t.c
> > >
> > > To avoid all this overhead, RDMA uses netlink-like ioctl calls, which
> > > fits better for object configurations.
> > >
> > > Thanks
> >
> > We'd definitely like to keep using netlink for control path object
> > management. Also please note we're talking about genetlink family. It is
> > fast and reliable enough for us, very easily extensible,
> > has a nice precise object definition with policies to enforce various
> > limitations, has extensive tooling (e.g. ynl), communication can be
> > monitored in realtime for debugging (e.g. nlmon), has a nice human
> > readable error reporting, gives the ability to easily dump large object
> > groups with filters applied, YAML family definitions and so on.
> > Having sockets or parsing are not issues.
>
> Of course it is issue as netlink relies on Netlink sockets, which means
> that you constantly move your configuration data instead of doing
> standard to whole linux kernel pattern of allocating configuration
> structs in user-space and just providing pointer to that through ioctl
> call.
>
> However, this discussion is premature and as an intro it is worth to
> read this cover letter for how object management is done in RDMA
> subsystem.
>
> https://lore.kernel.org/linux%
> 2Drdma_1501765627-2D104860-2D1-2Dgit-2Dsend-2Demail-2Dmatanb-
> 40mellanox.com_&d=DwIBAg&c=BSDicqBQBDjDI9RkVyTcHQ&r=4ynb4Sj_4MUcZXbhvovE4tY
> SbqxyOwdSiLedP4yO55g&m=U78K-khiLd-
> LLkbuNRzBStNppsXFTXdM7br052fwal1mzxpaOcOSQXCnguAK8t3g&s=U9dQl07fp-
> e9380xjR94fW-UGixoMsoxr5HfXKYggLk&e=
>
Nice old stuff. Often history teaches us something. 😉

I assume the correct way forward is to first clarify the
structure of all user-visible objects that need to be
created/controlled/destroyed, and to route them through
this interface. Some will require extensions to given objects,
some may be new, some will be as-is. rdma_netlink will probably
be the right interface to look at for job control.

Best,
Bernard.

> Thanks
>
> >
> > Cheers,
> > Nik
> >
> >