[ynl] how to fix the order of selector and `_decode_sub_msg`

[ynl] how to fix the order of selector and `_decode_sub_msg`

[Hangbin Liu]

Hi Donald, Jakub,

Recently I have been trying to add encap decoding support for `rt‑route.yaml`,
with formats similar to:

      -
        name: encap-type
        type: u16
        enum: encap-type
      -
        name: encap
        type: sub-message
        sub-message: encap-data
        selector: encap-type

However, sub‑message decoding is broken inside `ynl.py`. The root cause is
that `lwtunnel_fill_encap()` sets the encap‑type *after* the encap payload
data.

In iproute2, `parse_rtattr()` extracts all attributes together. In contrast,
`ynl.py` stores attributes one‑by‑one into `search_attrs` after they get
decoded (_decode() -> self._rsp_add()). As a result, we cannot locate the
`selector` while decoding the "encap" payload.

Should we decode sub‑messages after finishing first‑level attribute decoding?
Do you have any suggestions on how we can fix this?

Thanks
Hangbin

Re: [ynl] how to fix the order of selector and `_decode_sub_msg`

[Jakub Kicinski]

On Tue, 25 Aug 2026 20:48:55 +0800 Hangbin Liu wrote:
> Hi Donald, Jakub,
> 
> Recently I have been trying to add encap decoding support for `rt‑route.yaml`,
> with formats similar to:
> 
>       -
>         name: encap-type
>         type: u16
>         enum: encap-type
>       -
>         name: encap
>         type: sub-message
>         sub-message: encap-data
>         selector: encap-type
> 
> However, sub‑message decoding is broken inside `ynl.py`. The root cause is
> that `lwtunnel_fill_encap()` sets the encap‑type *after* the encap payload
> data.
> 
> In iproute2, `parse_rtattr()` extracts all attributes together. In contrast,
> `ynl.py` stores attributes one‑by‑one into `search_attrs` after they get
> decoded (_decode() -> self._rsp_add()). As a result, we cannot locate the
> `selector` while decoding the "encap" payload.
> 
> Should we decode sub‑messages after finishing first‑level attribute decoding?
> Do you have any suggestions on how we can fix this?

I'd change the kernel to output the type before the data.
The code seems to be trying to cancel out cleanly on EOPNOTSUPP.
Which is a good code pattern in principle but I don't think it makes
sense for dump / get. And no existing ->fill_encap can return EOPNOTSUPP
so instead of complicating things we should just move the type before
the data in the kernel.

Re: [ynl] how to fix the order of selector and `_decode_sub_msg`

[Hangbin Liu]

On Thu, Aug 27, 2026 at 10:14:16AM -0700, Jakub Kicinski wrote:
> On Tue, 25 Aug 2026 20:48:55 +0800 Hangbin Liu wrote:
> > Hi Donald, Jakub,
> > 
> > Recently I have been trying to add encap decoding support for `rt‑route.yaml`,
> > with formats similar to:
> > 
> >       -
> >         name: encap-type
> >         type: u16
> >         enum: encap-type
> >       -
> >         name: encap
> >         type: sub-message
> >         sub-message: encap-data
> >         selector: encap-type
> > 
> > However, sub‑message decoding is broken inside `ynl.py`. The root cause is
> > that `lwtunnel_fill_encap()` sets the encap‑type *after* the encap payload
> > data.
> > 
> > In iproute2, `parse_rtattr()` extracts all attributes together. In contrast,
> > `ynl.py` stores attributes one‑by‑one into `search_attrs` after they get
> > decoded (_decode() -> self._rsp_add()). As a result, we cannot locate the
> > `selector` while decoding the "encap" payload.
> > 
> > Should we decode sub‑messages after finishing first‑level attribute decoding?
> > Do you have any suggestions on how we can fix this?
> 
> I'd change the kernel to output the type before the data.
> The code seems to be trying to cancel out cleanly on EOPNOTSUPP.
> Which is a good code pattern in principle but I don't think it makes
> sense for dump / get. And no existing ->fill_encap can return EOPNOTSUPP
> so instead of complicating things we should just move the type before
> the data in the kernel.

Thanks, that would be an easy fix.

Hangbin