Netdev List
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba@kernel.org>
To: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Cc: netdev@vger.kernel.org, Tony Nguyen <anthony.l.nguyen@intel.com>,
	Aleksandr Loktionov <aleksandr.loktionov@intel.com>,
	Michal Schmidt <mschmidt@redhat.com>,
	edumazet@kernel.org, horms@kernel.org, pabeni@redhat.com,
	davem@davemloft.net, Jiri Pirko <jiri@resnulli.us>,
	Jonathan Corbet <corbet@lwn.net>,
	skhan@linuxfoundation.org, rdunlap@infradead.org,
	andrew+netdev@lunn.ch, saeedm@nvidia.com, tariqt@nvidia.com,
	leon@kernel.org, mbloch@nvidia.com, jacob.e.keller@intel.com,
	jedrzej.jagielski@intel.com, anzaki@gmail.com,
	brett.creeley@amd.com, ohartoov@nvidia.com,
	Petr Oros <poros@redhat.com>
Subject: Re: [PATCH net-next v5 13/15] ice: represent RSS LUTs as devlink resources
Date: Wed, 18 Nov 2026 17:11:21 -0800	[thread overview]
Message-ID: <20261118171121.447b7e6d@kernel.org> (raw)
In-Reply-To: <20261110141534.95614-14-przemyslaw.kitszel@intel.com>

On Tue, 10 Nov 2026 15:07:22 +0100 Przemek Kitszel wrote:
> E800 family offers three kinds of RSS LUTs: VSI LUT (sized 64), GLOBAL LUT
> (sized 512), and PF LUT (sized 2048). Until now the GLOBAL kind was not
> used at all. There are two possible usages for it, subsequent commit will
> give VF option to acquire it, and this one enables PF to switch between PF
> LUT and GLOBAL LUT - switching to smaller one is, again, to make it
> possible for VF to then acquire the former one.
> 
> Devlink resources are used to let user show current usage and change the
> allocation, see examples below.
> 
> Default state on 8-port card, asking for aggregate "whole device" usage,
> note that there are as many PF LUTs as there is PFs, and, for e810, there
> are 16 GLOBAL LUTs:
> $ devlink resource show devlink_index/11
> devlink_index/11:
>   name rss size 8 unit entry size_min 0 size_max 24 size_gran 1 dpipe_tables none
>     resources:
>       name lut_512 size 0 unit entry size_min 0 size_max 16 size_gran 1 dpipe_tables none
>       name lut_2048 size 8 unit entry size_min 0 size_max 8 size_gran 1 dpipe_tables none

The LUTs are not sub-dividable right? The 512 LUT tables "always
exist", they are not collapsed up to form the 2048 entry ones?
If so it'd be more intuitive to me to have:

 lut_512 size $hw-count size_max $hw-count occ $currently-used

size 0 size_max 16 would tell me we configured the device to not have
any 512 LUTs but IIUC it does still have them.

Then I'd have expected per-PF/VF resource which tells the driver
max LUT size allowed for the function:

 lut/pf0 size 1024 (<- we let it use 1024 LUT) 
         size_max 2048 (<- max configurable)
         occ 512 (<- currently only 512 is used)

maybe for you occ == size to day, but we tried to lay the foundation
for dynamic RSS table sizing in Netlink. Note that Netlink accepts RSS
tables smaller than the one reported by the device. I intend it to
mean that user wants the table at least that size. IOW if device
declares it's RSS table as 2048 but user send a table of 512 entries
via devlink - the driver should be free to reallocate the table down
to 512 (or a multiple, so 1024, Bjorn has added the "fold / unfold"
helpers already as part of other work).

What am I missing?

> Now let's add GLOBAL LUT for a single PF (on one-port NIC):
> $ sudo devlink resource set pci/0000:18:00.0 path rss/lut_512 size 1
> And show it's resources after that:
> $ devlink resource show pci/0000:18:00.0
> pci/0000:18:00.0:
>   name rss size 2 unit entry size_min 0 size_max 2 size_gran 1 dpipe_tables none
>     resources:
>       name lut_512 size 1 unit entry size_min 0 size_max 1 size_gran 1 dpipe_tables none
>       name lut_2048 size 1 unit entry size_min 0 size_max 1 size_gran 1 dpipe_tables none
> Let's take the PF LUT out of that PF afterwards:
> $ sudo devlink resource set pci/0000:18:00.0 path rss/lut_2048 size 0
> now `ethtool -x $ifacename` will report smaller RSS table.
> 
> Changing the LUT of a PF resets its RSS indirection table to the default
> spread, as the table size changes, and the ethtool core is told that
> the user table is lost. The PF LUT change is done under rtnl, as
> the ethtool core sizes its RSS buffers by the table size under rtnl.

Why?! 😱️
The resource change should only control the max table size the driver
can allocate *when it allocates a new one*. Increasing the max allowed
table size certainly mustn't cause loss of config.

> FW drops RSS LUT assignments on reset, so the driver gives the PF its PF
> LUT back after each reset, and logs when that discards a devlink change.
> Changes are rejected with -EBUSY while a reset is in progress.

  parent reply	other threads:[~2026-11-19  1:11 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10 14:07 [PATCH net-next v5 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
2026-11-10 14:07 ` [PATCH net-next v5 01/15] devlink, mlx5: add init/fini ops for shared devlink Przemek Kitszel
2026-11-10 14:47   ` Jiri Pirko
2026-11-10 14:07 ` [PATCH net-next v5 02/15] ice: use shared devlink to store ice_adapters instead of custom xarray Przemek Kitszel
2026-11-10 14:07 ` [PATCH net-next v5 03/15] ice: add VF queue ena/dis helper functions Przemek Kitszel
2026-11-10 14:07 ` [PATCH net-next v5 04/15] ice: add helpers for Global RSS LUT alloc, free, vsi_update Przemek Kitszel
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-10 14:07 ` [PATCH net-next v5 05/15] ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1 Przemek Kitszel
2026-11-10 14:07 ` [PATCH net-next v5 06/15] ice: bump to 256qs for VF Przemek Kitszel
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-10 14:07 ` [PATCH net-next v5 07/15] iavf: extend iavf_configure_queues() to support more queues Przemek Kitszel
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-13 12:07   ` Loktionov, Aleksandr
2026-11-10 14:07 ` [PATCH net-next v5 08/15] iavf: temporary rename of IAVF_MAX_REQ_QUEUES to IAVF_MAX_REQ_QUEUES_VCV1 Przemek Kitszel
2026-11-10 14:07 ` [PATCH net-next v5 09/15] iavf: increase max number of queues to 256 Przemek Kitszel
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-10 14:07 ` [PATCH net-next v5 10/15] iavf: use new opcodes to request more than 16 queues Przemek Kitszel
2026-11-12 12:57   ` Petr Oros
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-10 14:07 ` [PATCH net-next v5 11/15] ice: introduce handling of virtchnl LARGE VF opcodes Przemek Kitszel
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-10 14:07 ` [PATCH net-next v5 12/15] devlink: give user option to allocate resources Przemek Kitszel
2026-11-13 12:08   ` Loktionov, Aleksandr
2026-11-19  1:00   ` Jakub Kicinski
2026-11-10 14:07 ` [PATCH net-next v5 13/15] ice: represent RSS LUTs as devlink resources Przemek Kitszel
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-19  1:11   ` Jakub Kicinski [this message]
2026-11-10 14:07 ` [PATCH net-next v5 14/15] devlink: add devlink_alloc_drv() Przemek Kitszel
2026-11-10 14:48   ` Jiri Pirko
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-10 14:07 ` [PATCH net-next v5 15/15] ice: support up to 256 VF queues Przemek Kitszel
2026-11-12 23:07   ` netdev-bot+sashiko
2026-11-12 12:57 ` [PATCH net-next v5 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Petr Oros
2026-11-13 12:10 ` Przemek Kitszel
2026-11-19  1:12 ` Jakub Kicinski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118171121.447b7e6d@kernel.org \
    --to=kuba@kernel.org \
    --cc=aleksandr.loktionov@intel.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=anthony.l.nguyen@intel.com \
    --cc=anzaki@gmail.com \
    --cc=brett.creeley@amd.com \
    --cc=corbet@lwn.net \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=horms@kernel.org \
    --cc=jacob.e.keller@intel.com \
    --cc=jedrzej.jagielski@intel.com \
    --cc=jiri@resnulli.us \
    --cc=leon@kernel.org \
    --cc=mbloch@nvidia.com \
    --cc=mschmidt@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=ohartoov@nvidia.com \
    --cc=pabeni@redhat.com \
    --cc=poros@redhat.com \
    --cc=przemyslaw.kitszel@intel.com \
    --cc=rdunlap@infradead.org \
    --cc=saeedm@nvidia.com \
    --cc=skhan@linuxfoundation.org \
    --cc=tariqt@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox