RE: [PATCH V1 net-next 0/5] Add devlink support to ena

["Arinzon, David" <darinzon@amazon.com>]

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Wednesday, January 11, 2023 10:00 PM
> To: Arinzon, David <darinzon@amazon.com>
> Cc: David Miller <davem@davemloft.net>; netdev@vger.kernel.org;
> Machulsky, Zorik <zorik@amazon.com>; Matushevsky, Alexander
> <matua@amazon.com>; Bshara, Saeed <saeedb@amazon.com>; Bshara,
> Nafea <nafea@amazon.com>; Saidi, Ali <alisaidi@amazon.com>;
> Kiyanovski, Arthur <akiyano@amazon.com>; Dagan, Noam
> <ndagan@amazon.com>; Agroskin, Shay <shayagr@amazon.com>; Itzko,
> Shahar <itzko@amazon.com>; Abboud, Osama <osamaabb@amazon.com>
> Subject: RE: [EXTERNAL][PATCH V1 net-next 0/5] Add devlink support to
> ena
> 
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and
> know the content is safe.
> 
> 
> 
> On Wed, 11 Jan 2023 19:31:39 +0000 Arinzon, David wrote:
> > If the packet network headers are not within the size of the LLQ
> > entry, then the packet will be dropped. So I'll say that c) describes
> > the impact the best given that certain types of traffic will not succeed or
> have disruptions due to dropped TX packets.
> 
> I see. Sounds like it could be a fit for
> DEVLINK_ATTR_ESWITCH_INLINE_MODE ? But that one configures the
> depth of the headers copied inline, rather than bytes. We could add a value
> for "tunneled" and have that imply 256B LLQ in case of ena.
> 
> The other option is to introduce the concept of "max length of inline data"
> to ethtool, and add a new netlink attribute to ethtool -g.
> 
> Either way - the length of "inline|push" data is not a ena-specific concept,
> so using private knobs (devlink params or ethtool flags) is not appropriate
> upstream. We should add a bona fide uAPI for it.

I've looked into the original commit of DEVLINK_ATTR_ESWITCH_INLINE_MODE
and it seems to be more related to encapsulation and adding a tunneling concept
here would be misleading, as it doesn't cover all the use-cases affected here.
It is true that in case of tunneling and if there's a need to access the encapsulated
headers, the length of the headers might be larger than the default value in LLQ,
but this may be also the case for non-tunneled traffic, for example,
IPv6 with TCP SACK option.

I'll note again that this is not a configurable value, meaning, the only option is
to have 128B (standard) or 256B (large) LLQ, there's no option to set other values,
but only choose between the modes, therefore, I don't see how having such an
option through ethtool, as you suggested (setting the max length) can be
beneficial in our use-case (might be good overall, as you noted, it's a more
generic concept). Will put more thought into it.