yaml gen NL families support in iproute2?

yaml gen NL families support in iproute2?

[Paolo Abeni]

Hi all,

please allow me to [re?]start this conversation.

I think it would be very useful to bring yaml gennl families support in 
iproute2, so that end-users/admins could consolidated 
administration/setup in a single tool - as opposed to current status 
where something is only doable with iproute2 and something with the 
yml-cli tool bundled in the kernel sources.

Code wise it could be implemented extending a bit the auto-generated 
code generation to provide even text/argument to NL parsing, so that the 
iproute-specific glue (and maintenance effort) could be minimal.

WDYT?

Thanks,

Paolo

Re: yaml gen NL families support in iproute2?

[David Ahern]

On 10/17/24 11:41 AM, Paolo Abeni wrote:
> Hi all,
> 
> please allow me to [re?]start this conversation.
> 
> I think it would be very useful to bring yaml gennl families support in
> iproute2, so that end-users/admins could consolidated
> administration/setup in a single tool - as opposed to current status
> where something is only doable with iproute2 and something with the
> yml-cli tool bundled in the kernel sources.
> 
> Code wise it could be implemented extending a bit the auto-generated
> code generation to provide even text/argument to NL parsing, so that the
> iproute-specific glue (and maintenance effort) could be minimal.
> 
> WDYT?
> 

I would like to see the yaml files integrated into iproute2, but I have
not had time to look into doing it.

I have been using the cli.py tool a lot in the past few months and I
have it klunky and hard to use - especially around valid options for get
and dump requests.

Coping yaml files into iproute2 is the trivial part; we already do that
for uapi header files. The harder part is properly integrating the
commands into the package. To me requirements are along the lines of:
- aligning with iproute2 style of commands,
- a man page per command,
- proper help options to guide users

The last 2 should include attributes that can be added to get and dump
requests.

Re: yaml gen NL families support in iproute2?

[Joe Damato]

On Thu, Oct 17, 2024 at 12:36:47PM -0600, David Ahern wrote:
> On 10/17/24 11:41 AM, Paolo Abeni wrote:
> > Hi all,
> > 
> > please allow me to [re?]start this conversation.
> > 
> > I think it would be very useful to bring yaml gennl families support in
> > iproute2, so that end-users/admins could consolidated
> > administration/setup in a single tool - as opposed to current status
> > where something is only doable with iproute2 and something with the
> > yml-cli tool bundled in the kernel sources.
> > 
> > Code wise it could be implemented extending a bit the auto-generated
> > code generation to provide even text/argument to NL parsing, so that the
> > iproute-specific glue (and maintenance effort) could be minimal.
> > 
> > WDYT?
> > 
> 
> I would like to see the yaml files integrated into iproute2, but I have
> not had time to look into doing it.

I agree with David, but likewise have not had time to look into it.

It would be nice to use one tool instead of a combination of
multiple tools, if that were at all possible.

Re: yaml gen NL families support in iproute2?

[Paolo Abeni]

On 10/21/24 22:58, Joe Damato wrote:
> On Thu, Oct 17, 2024 at 12:36:47PM -0600, David Ahern wrote:
>> On 10/17/24 11:41 AM, Paolo Abeni wrote:
>>> Hi all,
>>>
>>> please allow me to [re?]start this conversation.
>>>
>>> I think it would be very useful to bring yaml gennl families support in
>>> iproute2, so that end-users/admins could consolidated
>>> administration/setup in a single tool - as opposed to current status
>>> where something is only doable with iproute2 and something with the
>>> yml-cli tool bundled in the kernel sources.
>>>
>>> Code wise it could be implemented extending a bit the auto-generated
>>> code generation to provide even text/argument to NL parsing, so that the
>>> iproute-specific glue (and maintenance effort) could be minimal.
>>>
>>> WDYT?
>>
>> I would like to see the yaml files integrated into iproute2, but I have
>> not had time to look into doing it.
> 
> I agree with David, but likewise have not had time to look into it.
> 
> It would be nice to use one tool instead of a combination of
> multiple tools, if that were at all possible.

FTR I'm investigating the idea of using a tool similar to ynl-gen-rst.py
and ynl-gen-c.py to generate the man page and the command line parsing
code to build the NL request and glue libynl.a with an iproute2 like
interface.

Currently I'm stuck at my inferior python skills and -ENOTIME, but
perhaps someone else is interested/willing to step in...

/P

Re: yaml gen NL families support in iproute2?

[Jakub Kicinski]

On Tue, 22 Oct 2024 09:55:08 +0200 Paolo Abeni wrote:
> On 10/21/24 22:58, Joe Damato wrote:
> > On Thu, Oct 17, 2024 at 12:36:47PM -0600, David Ahern wrote:  
> >> On 10/17/24 11:41 AM, Paolo Abeni wrote:  
> >>> Hi all,
> >>>
> >>> please allow me to [re?]start this conversation.
> >>>
> >>> I think it would be very useful to bring yaml gennl families support in
> >>> iproute2, so that end-users/admins could consolidated
> >>> administration/setup in a single tool - as opposed to current status
> >>> where something is only doable with iproute2 and something with the
> >>> yml-cli tool bundled in the kernel sources.
> >>> 
> >>> Code wise it could be implemented extending a bit the auto-generated
> >>> code generation to provide even text/argument to NL parsing, so that the
> >>> iproute-specific glue (and maintenance effort) could be minimal.
> >>>
> >>> WDYT?  

Why integrate with legacy tooling? To avoid the Python dependency?

I was hoping for iproute2 integration a couple of years ago, but
David Ahern convinced me that it's not necessary. Apparently he 
changed his mind now, but I remain convinced that packaging 
YNL CLI is less effort and will ensure complete coverage with
no manual steps.

> >> I would like to see the yaml files integrated into iproute2, but I have
> >> not had time to look into doing it.  
> > 
> > I agree with David, but likewise have not had time to look into it.
> > 
> > It would be nice to use one tool instead of a combination of
> > multiple tools, if that were at all possible.  
> 
> FTR I'm investigating the idea of using a tool similar to ynl-gen-rst.py
> and ynl-gen-c.py to generate the man page and the command line parsing
> code to build the NL request and glue libynl.a with an iproute2 like
> interface.
> 
> Currently I'm stuck at my inferior python skills and -ENOTIME, but
> perhaps someone else is interested/willing to step in...

How do your Python skills compare to your RPM skills?
The main change we need in YNL CLI is to "search" the specs in
a "well known location" so that user can specify:

 ynl --family netdev ...

or even:

 ynl-netdev ...

instead of:

 ynl --spec /usr/bla/bla/netdev.yaml

And make ynl in "distro mode" default to --no-schema and
--process-unknown


That's assuming that by

  so that end-users/admins could consolidated administration/setup 
  in a single tool

you mean that you are aiming to create a single tool capable of
handling arbitrary specs. If you want to make the output
and input more "pretty" than just attrs in / attrs out -- then
indeed building on top of libynl.a makes sense.

Re: yaml gen NL families support in iproute2?

[David Ahern]

On 10/28/24 2:58 PM, Jakub Kicinski wrote:
> I was hoping for iproute2 integration a couple of years ago, but
> David Ahern convinced me that it's not necessary. Apparently he 
> changed his mind now, but I remain convinced that packaging 
> YNL CLI is less effort and will ensure complete coverage with
> no manual steps.

I not recall any comment about it beyond cli.py in its current form is a
total PITA to use as it lacks help and a man page.

Re: yaml gen NL families support in iproute2?

[Jakub Kicinski]

On Mon, 28 Oct 2024 15:29:35 -0600 David Ahern wrote:
> On 10/28/24 2:58 PM, Jakub Kicinski wrote:
> > I was hoping for iproute2 integration a couple of years ago, but
> > David Ahern convinced me that it's not necessary. Apparently he 
> > changed his mind now, but I remain convinced that packaging 
> > YNL CLI is less effort and will ensure complete coverage with
> > no manual steps.  
> 
> I not recall any comment about it beyond cli.py in its current form 
> is a total PITA to use as it lacks help and a man page.

I can only find this thread now:
https://lore.kernel.org/all/20240302193607.36d7a015@kernel.org/
Could be a misunderstanding, but either way, documenting an existing
tool seems like strictly less work than recreating it from scratch.

Re: yaml gen NL families support in iproute2?

[Stephen Hemminger]

On Mon, 28 Oct 2024 15:15:34 -0700
Jakub Kicinski <kuba@kernel.org> wrote:

> On Mon, 28 Oct 2024 15:29:35 -0600 David Ahern wrote:
> > On 10/28/24 2:58 PM, Jakub Kicinski wrote:  
> > > I was hoping for iproute2 integration a couple of years ago, but
> > > David Ahern convinced me that it's not necessary. Apparently he 
> > > changed his mind now, but I remain convinced that packaging 
> > > YNL CLI is less effort and will ensure complete coverage with
> > > no manual steps.    
> > 
> > I not recall any comment about it beyond cli.py in its current form 
> > is a total PITA to use as it lacks help and a man page.  
> 
> I can only find this thread now:
> https://lore.kernel.org/all/20240302193607.36d7a015@kernel.org/
> Could be a misunderstanding, but either way, documenting an existing
> tool seems like strictly less work than recreating it from scratch.

Is the toolset willing to maintain the backward compatibility guarantees
that iproute2 has now? Bpf support was an example of how not to do it.

Re: yaml gen NL families support in iproute2?

[Jakub Kicinski]

On Mon, 28 Oct 2024 16:40:55 -0700 Stephen Hemminger wrote:
> > I can only find this thread now:
> > https://lore.kernel.org/all/20240302193607.36d7a015@kernel.org/
> > Could be a misunderstanding, but either way, documenting an existing
> > tool seems like strictly less work than recreating it from scratch.  
> 
> Is the toolset willing to maintain the backward compatibility guarantees
> that iproute2 has now? Bpf support was an example of how not to do it.

The specs are UAPI.

The Python and CLI tooling are a very thin layer of code basically
converting between JSON and netlink using the specs, so by virtue 
of specs being UAPI they should be fully backward compatible.

The C library is intended to be fully backward compatible, but right
now only supports static linking.

Re: yaml gen NL families support in iproute2?

[Stephen Hemminger]

On Mon, 28 Oct 2024 17:06:47 -0700
Jakub Kicinski <kuba@kernel.org> wrote:

> On Mon, 28 Oct 2024 16:40:55 -0700 Stephen Hemminger wrote:
> > > I can only find this thread now:
> > > https://lore.kernel.org/all/20240302193607.36d7a015@kernel.org/
> > > Could be a misunderstanding, but either way, documenting an existing
> > > tool seems like strictly less work than recreating it from scratch.    
> > 
> > Is the toolset willing to maintain the backward compatibility guarantees
> > that iproute2 has now? Bpf support was an example of how not to do it.  
> 
> The specs are UAPI.
> 
> The Python and CLI tooling are a very thin layer of code basically
> converting between JSON and netlink using the specs, so by virtue 
> of specs being UAPI they should be fully backward compatible.
> 
> The C library is intended to be fully backward compatible, but right
> now only supports static linking.

Right but now you can update kernel and CLI tooling, and not worry about
running on older kernels.

Re: yaml gen NL families support in iproute2?

[Paolo Abeni]

On 10/28/24 21:58, Jakub Kicinski wrote:
> On Tue, 22 Oct 2024 09:55:08 +0200 Paolo Abeni wrote:
>> On 10/21/24 22:58, Joe Damato wrote:
>>> On Thu, Oct 17, 2024 at 12:36:47PM -0600, David Ahern wrote:  
>>>> On 10/17/24 11:41 AM, Paolo Abeni wrote:  
>>>>> Hi all,
>>>>>
>>>>> please allow me to [re?]start this conversation.
>>>>>
>>>>> I think it would be very useful to bring yaml gennl families support in
>>>>> iproute2, so that end-users/admins could consolidated
>>>>> administration/setup in a single tool - as opposed to current status
>>>>> where something is only doable with iproute2 and something with the
>>>>> yml-cli tool bundled in the kernel sources.
>>>>>
>>>>> Code wise it could be implemented extending a bit the auto-generated
>>>>> code generation to provide even text/argument to NL parsing, so that the
>>>>> iproute-specific glue (and maintenance effort) could be minimal.
>>>>>
>>>>> WDYT?  
> 
> Why integrate with legacy tooling? To avoid the Python dependency?

My guestimate is that the majority of the end users would prefer a
single tool (or at least a consistent 'look & feel') and they are
already accustomed to iproute and it's command line.

>>>> I would like to see the yaml files integrated into iproute2, but I have
>>>> not had time to look into doing it.  
>>>
>>> I agree with David, but likewise have not had time to look into it.
>>>
>>> It would be nice to use one tool instead of a combination of
>>> multiple tools, if that were at all possible.  
>>
>> FTR I'm investigating the idea of using a tool similar to ynl-gen-rst.py
>> and ynl-gen-c.py to generate the man page and the command line parsing
>> code to build the NL request and glue libynl.a with an iproute2 like
>> interface.
>>
>> Currently I'm stuck at my inferior python skills and -ENOTIME, but
>> perhaps someone else is interested/willing to step in...
> 
> How do your Python skills compare to your RPM skills?
> The main change we need in YNL CLI is to "search" the specs in
> a "well known location" so that user can specify:
> 
>  ynl --family netdev ...
> 
> or even:
> 
>  ynl-netdev ...
> 
> instead of:
> 
>  ynl --spec /usr/bla/bla/netdev.yaml
> 
> And make ynl in "distro mode" default to --no-schema and
> --process-unknown

Thanks for the hints.

I think that packaging the ynl tool is an orthogonal activity WRT the
goal/discussion here.

> That's assuming that by
> 
>   so that end-users/admins could consolidated administration/setup 
>   in a single tool
> 
> you mean that you are aiming to create a single tool capable of
> handling arbitrary specs. > If you want to make the output
> and input more "pretty" than just attrs in / attrs out -- then
> indeed building on top of libynl.a makes sense.

My understanding is that a similar/consistent command line and man page
based documentation will preferable for the end-user - say to configure
ip addresses and configure tx shaping on a given device.

i.e. libynl would be a good starting point, but IMHO ideally it should
include a few bells and whistles alike ifindex <> interface name
conversion, explicit 'required arguments' list, etc...

Cheers,

Paolo

Re: yaml gen NL families support in iproute2?

[Jakub Kicinski]

On Tue, 29 Oct 2024 10:29:15 +0100 Paolo Abeni wrote:
> > That's assuming that by
> > 
> >   so that end-users/admins could consolidated administration/setup 
> >   in a single tool
> > 
> > you mean that you are aiming to create a single tool capable of
> > handling arbitrary specs. > If you want to make the output
> > and input more "pretty" than just attrs in / attrs out -- then
> > indeed building on top of libynl.a makes sense.  
> 
> My understanding is that a similar/consistent command line and man page
> based documentation will preferable for the end-user - say to configure
> ip addresses and configure tx shaping on a given device.

I see. You got me slightly confused with "single tool" as iproute2
itself contains multiple tools. Sounds like you just want to write
a tool / add to ip the ability to talk net shaper nl.