* [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf
@ 2026-11-17 9:44 Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 01/15] devlink, mlx5: add init/fini ops for shared devlink Przemek Kitszel
` (14 more replies)
0 siblings, 15 replies; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Also available here
https://github.com/pkitszel/linux/tree/xlvf-v6
The purpose of this series is to allow iavf to use more than 16 queue
pairs, in two modes: up to 64 and up to 256 queue pairs.
In order to support more queues for VF, we must give it a bigger RSS
table (GLOBAL LUT or PF LUT). There are 16 GLOBAL LUTs on E810, and one
PF LUT for every PF on given card. Both kinds could be (re)assigned to
a VF, while a PF must hold at least one of them at any given moment.
GLOBAL LUT allows VF to use up to 64 queues, PF LUT lets it use up to
256 queues.
RSS LUTs are exposed to the user for assignment via devlink resources
API, which I have extended to make it possible.
Devlink changes:
1. Extend shared devlink by two callbacks, that give the driver
a constructor/destructor for the priv data attached to the shared
devlink instance. The constructor gets an additional param, used in
"ice: represent RSS LUTs as devlink resources". mlx5 is touched only
to adjust to the API change.
ice uses the shared devlink instance as a "whole device" aggregate
over all PFs on given card, instead of its custom xarray.
2. Extend devlink resources API to allow user to assign resources.
Before, only the driver could assign resources, without any way for
the user to interact. Now the driver could register an occupancy
setter (.occ_set()) for a resource. ice RSS LUTs are exposed that
way. It was proposed as an RFC in Feb 2025, link in the patch.
3. (minor) Add devlink_alloc_drv(), to allocate a devlink instance for
a device on behalf of another driver. ice uses it for the VF devlink
instance, which has to be owned by ice regardless of the driver bound
to the VF device.
There are also patches for Admin Queue extension, GLOBAL RSS LUT
alloc/free, and two rather big "new opcodes" patches, by Ahmed (iavf)
and Brett (ice).
Finally there is a patch that adds devlink instance for VF and
registers devlink resources on it, combined with all the glue code to
make actual use of the whole series and accept the larger number of
queues for VF. This (the last) patch contains usage examples.
There is one resource added that just groups GLOBAL and PF LUTs under
it, so there are 3 rows of data for each device (PF/VF/whole-dev):
$ devlink resource show pci/0000:18:00.0
pci/0000:18:00.0:
name rss size 1 unit entry size_min 0 size_max 2 size_gran 1 dpipe_tables none
resources:
name lut_512 size 0 unit entry size_min 0 size_max 1 size_gran 1 dpipe_tables none
name lut_2048 size 1 unit entry size_min 0 size_max 1 size_gran 1 dpipe_tables none
Technically the aggregate "name rss" line could be eliminated, with
just the two last ones kept (then renamed to "rss_lut_2048" form, from
the current "rss/lut_2048"). I like it as it is, but this is just an
opinion, I'm open to discussion.
Things AI reports but I don't want to change:
* hardening against a malicious or buggy FW: we have to trust the FW
* hardening of the VF against a malicious or buggy PF: likewise, VF has
to trust its PF; in particular, I'm not adding checks for the PF
reporting qregion_width so big that BIT(qregion_width) in
iavf_fill_rss_lut() overflows and yields a modulo by zero
* iavf: ethtool -L max is the PF RSS region in LARGE mode, so iavf
asks only for queues that RSS can spread traffic to; ice never
grants a VF more queues than its RSS region
* iavf: fixes of older bugs that apply to every PF (aq_wait_count
reset, RSS key/LUT reallocation on size change, PF lowering the queue
count) are kept in "iavf: use new opcodes to request more than 16
queues", as the new init and ethtool -L flow depends on them
* non-atomic updates of iavf adapter->flags: pre-existing pattern in
the whole driver, out of scope of this series
* lock inversion between PF devlink lock and pf->vfs.table_lock:
ice_tear_down_devlink_rate_tree() takes them in the opposite order
than ice_start_vfs() -> ice_eswitch_attach_vf(); pre-existing, will
be fixed separately; this series takes table_lock first
* lock inversion between rtnl and PF devlink lock: mqprio in channel
mode calls ice_tear_down_devlink_rate_tree() under rtnl, while
devlink reload takes rtnl under the devlink lock; pre-existing, this
series follows the reload order: devlink RSS LUT change of a PF
takes rtnl under the devlink lock
Some of the pre-existing issues reported by AI are fixed in a separate
series:
https://lore.kernel.org/netdev/20261102105346.30739-1-przemyslaw.kitszel@intel.com
Thanks to Petr Oros for testing v5, the division by zero he has found
is fixed in v6 (the first item below), and the queue range limit he
has hit is now documented in ice.rst:
https://lore.kernel.org/netdev/20261112125721.825296-1-poros@redhat.com
v6:
* iavf: don't refill the RSS LUT when the PF reports new RSS key/LUT
sizes on reinit after iavf_disable_vf(), there are no queues yet,
which gave a division by zero in iavf_fill_rss_lut() (Petr)
* iavf: ethtool -L: report the requested count until the watchdog
sends the queue request, and drop the request on ethtool -L back to
the active count; ethtool core skipped that as no change, and the
stale request went out
* iavf: ethtool -L: request queues from PF only if
VIRTCHNL_VF_OFFLOAD_REQ_QUEUES is negotiated too, as done at init
* iavf: zero adapter->vf_res on a failed VF resources message, instead
of freeing it, as ethtool ops use it while init retries with the
netdev registered
* ice: document in ice.rst that VF queues must fit in one free range
of PF queues, so with many VFs only a few get 64 queues (Petr)
* virtchnl: fix a struct name in a comment, drop a misleading example
from the virtchnl_max_rss_qregion comment
* collect Reviewed-by tags from Jiri and Alex
* rebase on current net-next (v5 had conflict with -net)
v5:
* ice: keep reporting the 64 entry RSS LUT to VFs without
VIRTCHNL_VF_LARGE_NUM_QPAIRS, and repeat their LUT over the bigger
LUT of the VSI; iavf before this series allocates its LUT buffer
only once, and overflowed it after a bigger LUT was assigned (Petr)
* ice: grant REQUEST_QUEUES only when the VF fits in one free range of
Tx and Rx queues, and offer no more than that in the reply to a too
big request; requests of 64 or 256 queues fragmented the PF queue
space, and VFs came up with unprogrammed queues, as scattered
mapping is not implemented for VFs (Petr)
* ice: mark VFs disabled when preparing for PF reset, until
ice_reset_all_vfs() rebuilds them, so devlink can't change their RSS
LUT in between
* ice: register VF devlink resources before devl_register(), and
unregister them after devl_unregister(), so the VF devlink instance
is never visible without them
* ice: set the default rss_size on every VSI config again, as before
this series, and clamp it to what the RSS LUT can address for all
VSI types, under rss_lut_lock
* ice: keep the NAC topology in safe mode caps, secondary NAC PFs
collided with primary ones in the devlink RSS LUT table
* ice: widen num_ring_{rx,tx} of struct ice_q_vector to u16, mapping
all 256 queues of a VF to one vector wrapped them around
* ice: LAG: call ice_aq_cfg_lan_txq() for up to U8_MAX queues at
a time, 256 VF queues were sent as 0
* iavf: ethtool -L: validate rx, tx and other counts before returning
early for an unchanged combined count; keep the queue request
pending until it is sent
* iavf: clear adapter->vsi_res together with adapter->vf_res, and skip
the rest of GET_VF_RESOURCES reply handling when parsing it fails
* iavf: cap the legacy VIRTCHNL_OP_{EN,DIS}ABLE_QUEUES bitmaps at 16
queues
* iavf: report the RX ring count limited to the PF RSS region, the
default LUT made by ethtool core for ethtool -X was rejected
* devlink: document @drv lifetime and device lock rules of
devlink_alloc_drv()
* smaller fixes, mostly after Sashiko review; per-patch changelogs are
under "---"
v4:
* devlink: return -EMSGSIZE directly on
devlink_resource_occ_size_put_legacy() failure (Jakub)
* devlink: document in devlink-info.rst that an instance created for
another device reports the name of the creating driver
* ice: declare rel_err at function scope, a declaration right after
a label does not build with GCC < 11 and breaks sparse (Jakub)
* ice: take rtnl when devlink changes the RSS LUT of the PF, and tell
the ethtool core when the user RSS LUT is dropped, on devlink LUT
change and on reset; drop the -EAGAIN checks from get/set_rxfh
* ice: reject devlink RSS LUT changes while a reset is in progress, drop
global RSS LUTs of the PF from the table already in
ice_prepare_for_reset(), as FW frees them on reset
* ice: log when a reset discards a devlink RSS LUT assignment, keep the
specific extack message and the real errno when a LUT change fails
* ice: take the Q_OPT section from vsi->info under rss_lut_lock when
toggling the RDMA filter
* ice: document the devlink resources and the VF devlink instance in
ice.rst
* ice: limit queue chunks of the LARGE VF opcodes to ICE_MAX_QS_PER_VF;
set the q_vector and ITR fields in VIRTCHNL_OP_MAP_QUEUE_VECTOR, as
VIRTCHNL_OP_CONFIG_IRQ_MAP does
* ice: after a PF reset, lower the queue count only of VFs that lost
a bigger RSS LUT, and only to 16
* ice: restore the VF RSS LUTs after reset outside of rtnl, cap
REQUEST_QUEUES at 16 for VFs without VIRTCHNL_VF_LARGE_NUM_QPAIRS,
report ICE_VF_VSI_ID as vport_id in GET_MAX_RSS_QREGION reply
* ice: do a pending deferred VF reset before handling the next message
from that VF, clear ICE_VF_STATE_NEEDS_RESET on every VF reset, and
re-run the service task at once if a deferred VF reset was scheduled
during its pass
* ice: don't create VF devlink instance without VF pci_dev, e.g. on s390
* ice: report the VF queue allocation in num_queue_pairs also with
VIRTCHNL_VF_LARGE_NUM_QPAIRS, as in legacy mode
* ice: use scoped_guard() instead of guard()
* iavf: send queue configuration and queue to vector map one message per
watchdog run, instead of polling for each reply with a 1s timeout;
stop sending on a PF NACK
* iavf: ethtool -L: let the watchdog send the queue request, instead of
failing with -EBUSY whenever another op is in flight
* iavf: init: wait for the outcome of the queue request before other
extended caps; ask for up to one queue per CPU, take the request as
granted only when PF gives that many, and on a denial go on with what
we have; check IAVF_VF_ARQLEN1 while waiting, as the PF reset event
could be lost
* iavf: reset aq_wait_count when init starts over after the PF stopped
responding, forget the queue request there
* iavf: refill the RSS LUT when the RSS region changes across a reset,
kzalloc() the RSS key and LUT on reallocation, reject ethtool -X LUT
entries outside of the PF RSS region
* iavf: num_queue_pairs is the VF allocation also with
VIRTCHNL_VF_LARGE_NUM_QPAIRS, ethtool -L max is the PF RSS region
* iavf: limit MSI-X budget to 64 queue vectors, program ITR of vectors
16..63 via VFINT_ITRN_64, change ADQ total_qps to u16
* many smaller fixes after Sashiko review; per-patch changelogs are
under "---"
v3:
* ice: fix rollback of VF queue configuration to disable queues by
their IDs, not by qpair index, as the config could be now split into
many virtchnl messages
* ice: register PF devlink resources only in the regular probe path,
not in FW recovery mode
* ice: don't free the user RSS LUT in ice_vsi_decfg(), it was done
outside of pf->rss_lut_lock, and ice_vsi_cfg_def() frees it anyway
when the LUT size changes
* iavf: auto-request queues in LARGE mode only at init, not on every
reset, and only after VSI and RSS fields are initialized
* iavf: schedule a VF reset when RSS key/LUT reallocation fails
* devlink: keep size of .occ_set() resources in sync with the getter;
WARN on registering only one of .occ_set()/.occ_get()
* devlink: add devlink_alloc_drv(), so ice owns the VF devlink
instance, regardless of the driver bound to the VF
* ice: unregister VF devlink on ice_start_vfs() failure
* ice: reject out of range itr_idx in VIRTCHNL_OP_MAP_QUEUE_VECTOR
* ice: fill the new RSS LUT under rss_lut_lock, keep the PF LUT zeroed
when RXHASH is disabled
* ice: restore VF aggregator node accounting in ice_vsi_decfg()
* ice: drop PF RSS LUT ownership also in ice_deinit_pf(), as
ice_vsi_decfg() skips it after a NO_INIT VSI rebuild
* ice: restore default RSS LUT assignment of PF and its VFs after
reset, as FW drops it
* ice: check VF VSI for NULL in ice_vc_request_qs_msg()
* ice: fix RSS LUT tracking on dual-NAC E825C: each NAC numbers its
PFs and global LUTs from 0, so offset secondary NAC PFs by 4, and
keep a separate global LUT table for the secondary NAC
* ice: derive VF rss_size from its queue count and LUT capacity on
each VSI config, instead of keeping a stale value
* ice: wake up service task when scheduling a deferred VF reset
v2:
* drop "ice: simplify ice_vc_dis_qs_msg() a little", already applied
* devlink: propagate the .shd_init() error to the caller, so
devlink_shd_get() returns ERR_PTR() now; register the shared
instance only after successful .shd_init(); document the new ops
* devlink: document the .occ_set() mode of resources
* ice: prefix the shared devlink id with "ice:", handle errors of
devl_nested_devlink_set()
* iavf: stop multi-message virtchnl ops (queue config, queue-vector
mapping) at the first error or PF NACK, keep the op pending on
timeout
* iavf: fix queue and RSS (re)negotiation on init, reset and
ethtool -L
* ice: serialize devlink RSS LUT changes with ethtool -x/-X/-L, RXHASH
toggle and VSI (re)configuration, by new pf->rss_lut_lock; make ADQ
and devlink RSS LUT reassignment mutually exclusive
* ice: limit VF queue count and PF RSS size to the capacity of the
current RSS LUT
* ice: create VF devlink together with the VF, rework deferred VF reset
* many smaller fixes, mostly after Sashiko review; per-patch changelogs
are under "---", together with Sashiko suggestions that I decided
not to apply, and why
v1:
https://lore.kernel.org/netdev/20260508124208.11622-1-przemyslaw.kitszel@intel.com
v2:
https://lore.kernel.org/netdev/20261009121433.30347-1-przemyslaw.kitszel@intel.com
v3:
https://lore.kernel.org/netdev/20261015185454.6332-1-przemyslaw.kitszel@intel.com
v4:
https://lore.kernel.org/netdev/20261103141646.303620-1-przemyslaw.kitszel@intel.com
v5:
https://lore.kernel.org/netdev/20261110141534.95614-1-przemyslaw.kitszel@intel.com
Ahmed Zaki (1):
iavf: use new opcodes to request more than 16 queues
Brett Creeley (2):
ice: add VF queue ena/dis helper functions
ice: introduce handling of virtchnl LARGE VF opcodes
Przemek Kitszel (12):
devlink, mlx5: add init/fini ops for shared devlink
ice: use shared devlink to store ice_adapters instead of custom xarray
ice: add helpers for Global RSS LUT alloc, free, vsi_update
ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1
ice: bump to 256qs for VF
iavf: extend iavf_configure_queues() to support more queues
iavf: temporary rename of IAVF_MAX_REQ_QUEUES to
IAVF_MAX_REQ_QUEUES_VCV1
iavf: increase max number of queues to 256
devlink: give user option to allocate resources
ice: represent RSS LUTs as devlink resources
devlink: add devlink_alloc_drv()
ice: support up to 256 VF queues
.../networking/devlink/devlink-info.rst | 2 +
.../networking/devlink/devlink-resource.rst | 5 +
.../networking/devlink/devlink-shared.rst | 19 +-
Documentation/networking/devlink/ice.rst | 81 ++
drivers/net/ethernet/intel/ice/Makefile | 1 +
drivers/net/ethernet/intel/iavf/iavf.h | 35 +-
.../net/ethernet/intel/iavf/iavf_register.h | 2 +
.../net/ethernet/intel/ice/devlink/devlink.h | 2 +-
.../net/ethernet/intel/ice/devlink/resource.h | 28 +
drivers/net/ethernet/intel/ice/ice.h | 9 +-
drivers/net/ethernet/intel/ice/ice_adapter.h | 55 +-
.../net/ethernet/intel/ice/ice_adminq_cmd.h | 1 +
drivers/net/ethernet/intel/ice/ice_common.h | 1 +
drivers/net/ethernet/intel/ice/ice_lag.h | 2 +-
drivers/net/ethernet/intel/ice/ice_lib.h | 5 +
drivers/net/ethernet/intel/ice/ice_switch.h | 5 +-
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 36 +-
drivers/net/ethernet/intel/ice/virt/queues.h | 3 +
drivers/net/ethernet/intel/ice/virt/rss.h | 1 +
.../net/ethernet/intel/ice/virt/virtchnl.h | 4 +
include/linux/net/intel/virtchnl.h | 129 ++-
include/net/devlink.h | 41 +
.../net/ethernet/intel/iavf/iavf_ethtool.c | 58 +-
drivers/net/ethernet/intel/iavf/iavf_main.c | 272 +++++-
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 291 ++++++-
.../net/ethernet/intel/ice/devlink/devlink.c | 11 +-
.../net/ethernet/intel/ice/devlink/resource.c | 803 ++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_adapter.c | 115 +--
drivers/net/ethernet/intel/ice/ice_common.c | 5 +-
drivers/net/ethernet/intel/ice/ice_ethtool.c | 91 +-
drivers/net/ethernet/intel/ice/ice_idc.c | 25 +-
drivers/net/ethernet/intel/ice/ice_lag.c | 54 +-
drivers/net/ethernet/intel/ice/ice_lib.c | 200 ++++-
drivers/net/ethernet/intel/ice/ice_main.c | 64 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 19 +-
drivers/net/ethernet/intel/ice/ice_switch.c | 52 +-
drivers/net/ethernet/intel/ice/ice_vf_lib.c | 84 +-
.../net/ethernet/intel/ice/virt/allowlist.c | 8 +
drivers/net/ethernet/intel/ice/virt/queues.c | 574 +++++++++++--
drivers/net/ethernet/intel/ice/virt/rss.c | 54 +-
.../net/ethernet/intel/ice/virt/virtchnl.c | 39 +-
.../ethernet/mellanox/mlx5/core/sh_devlink.c | 6 +-
net/devlink/core.c | 24 +
net/devlink/resource.c | 97 ++-
net/devlink/sh_dev.c | 57 +-
45 files changed, 3114 insertions(+), 356 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ice/devlink/resource.h
create mode 100644 drivers/net/ethernet/intel/ice/devlink/resource.c
--
2.54.0
^ permalink raw reply [flat|nested] 25+ messages in thread
* [net-next v6 01/15] devlink, mlx5: add init/fini ops for shared devlink
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 02/15] ice: use shared devlink to store ice_adapters instead of custom xarray Przemek Kitszel
` (13 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel,
Jiri Pirko
Add .shd_init() and .shd_fini() ops, that will be called for the first
devlink_shd_get() (to initialize driver's priv data) and on the last
devlink_shd_put() (to allow for the cleanup). Both ops are optional.
.shd_init() could return an error, which will stop creation of shd
instance. The initializer also gets an additional, optional param,
that driver could use for any needs.
devlink_shd_get() now returns ERR_PTR() on failure instead of NULL, so
the error from .shd_init() reaches the caller. Adjust mlx5, the only
user, to that and to the new param.
If any of the callbacks will need to get devlink instance, it could
be accessed by shd_priv_to_devlink().
Both callbacks are called with devl_lock held, .shd_init() before
devlink is registered, .shd_fini() after it is unregistered.
Next commit will make use of the callbacks, a later one will make use
also of the additional param.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Acked-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
(v0) first discussed at:
https://lore.kernel.org/netdev/20260325063143.261806-3-przemyslaw.kitszel@intel.com
v1: remove redundant added blank line (Jiri)
v2:
* call devl_register() only after .shd_init(), so there is no NEW/DEL
notification pair when the init callback fails; adjust the order of
devl_unregister() in devlink_shd_destroy() (Sashiko)
* return the error from .shd_init() to the caller instead of
collapsing every failure into -ENOMEM, devlink_shd_get() returns
ERR_PTR() now (Sashiko)
* commit message: .shd_init()/.shd_fini() are called outside of
devlink registration; the init param is used within this series
* kdoc: use @shd_init/@shd_fini member tags, fix the "shd_devlink'
priv" wording, document the calling context (locks, before
registration/after unregistration) and that a failed .shd_init()
is retried on the next devlink_shd_get() (Sashiko)
* devlink-shared.rst: document .shd_init()/.shd_fini(), init_param,
ERR_PTR() return and shd_priv_to_devlink() (Sashiko)
---
.../networking/devlink/devlink-shared.rst | 19 ++++++-
include/net/devlink.h | 30 ++++++++++
.../ethernet/mellanox/mlx5/core/sh_devlink.c | 6 +-
net/devlink/sh_dev.c | 57 ++++++++++++++++---
4 files changed, 98 insertions(+), 14 deletions(-)
diff --git a/Documentation/networking/devlink/devlink-shared.rst b/Documentation/networking/devlink/devlink-shared.rst
index 16bf6a7d25d9..f8b2a3d8aa82 100644
--- a/Documentation/networking/devlink/devlink-shared.rst
+++ b/Documentation/networking/devlink/devlink-shared.rst
@@ -45,6 +45,14 @@ The following functions are provided for managing shared devlink instances:
* ``devlink_shd_get()``: Get or create a shared devlink instance identified by a string ID
* ``devlink_shd_put()``: Release a reference on a shared devlink instance
* ``devlink_shd_get_priv()``: Get private data from shared devlink instance
+* ``shd_priv_to_devlink()``: Get shared devlink instance from its private data
+
+``devlink_shd_get()`` returns ``ERR_PTR()`` on failure.
+
+The driver may initialize and clean up its private data of the shared
+instance in the optional ``shd_init()`` and ``shd_fini()`` callbacks of
+``struct devlink_ops``. The ``init_param`` argument of ``devlink_shd_get()``
+is passed to ``shd_init()``.
Initialization Flow
-------------------
@@ -55,7 +63,9 @@ Initialization Flow
* The function looks up existing instance by identifier
* If none exists, creates new instance:
- - Allocates and registers devlink instance
+ - Allocates devlink instance
+ - Calls ``shd_init()`` (if set), on failure frees it and returns the error
+ - Registers devlink instance
- Adds to global shared instances list
- Increments reference count
@@ -67,7 +77,8 @@ Cleanup Flow
1. **Cleanup** when PF is removed
2. **Call** ``devlink_shd_put()`` to release reference (decrements reference count)
-3. **Shared instance is automatically destroyed** when the last PF removes (reference count reaches zero)
+3. **Shared instance is automatically destroyed** when the last PF removes (reference count reaches zero),
+ ``shd_fini()`` (if set) is called after unregistering it
Chip Identification
-------------------
@@ -85,6 +96,10 @@ Locking
A global mutex (``shd_mutex``) protects the shared instances list during registration/deregistration.
+``shd_init()`` and ``shd_fini()`` are called with ``shd_mutex`` and the devlink
+lock of the shared instance held, thus must not call ``devlink_shd_get()`` nor
+``devlink_shd_put()``, and must use ``devl_*()`` variants of devlink API.
+
Similarly to other nested devlink instance relationships, devlink lock of
the shared instance should be always taken after the devlink lock of PF.
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 7abd23376319..98bdde4f72ca 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -1607,6 +1607,34 @@ struct devlink_ops {
* with other operations.
*/
bool supported_cross_device_rate_nodes;
+ /**
+ * @shd_init: Shared devlink instance initializer.
+ *
+ * Gets the driver's priv data of the shared instance (the same as
+ * devlink_shd_get_priv() returns) and the init_param passed to
+ * devlink_shd_get(). Should initialize the priv data. May be NULL.
+ *
+ * Called when devlink_shd_get() creates the shared instance, before
+ * it is registered. If it fails, the instance is freed without calling
+ * @shd_fini, and the next devlink_shd_get() for the same id will try to
+ * create it (and call @shd_init) again.
+ *
+ * Called with the global shared devlink mutex and the devlink lock of
+ * the shared instance held, thus must not call devlink_shd_get() nor
+ * devlink_shd_put(), and must use devl_*() variants of devlink API.
+ *
+ * Return: 0 on success, negative errno to fail the creation.
+ */
+ int (*shd_init)(void *priv, void *init_param);
+ /**
+ * @shd_fini: Shared devlink instance finalizer.
+ *
+ * Gets the driver's priv data of the shared instance. Called when the
+ * last reference is dropped, after the shared instance is unregistered,
+ * in the same locking context as @shd_init. Should clean up the priv
+ * data. May be NULL.
+ */
+ void (*shd_fini)(void *priv);
/**
* selftests_check() - queries if selftest is supported
* @devlink: devlink instance
@@ -1672,9 +1700,11 @@ void devlink_free(struct devlink *devlink);
struct devlink *devlink_shd_get(const char *id,
const struct devlink_ops *ops,
size_t priv_size,
+ void *init_param,
const struct device_driver *driver);
void devlink_shd_put(struct devlink *devlink);
void *devlink_shd_get_priv(struct devlink *devlink);
+struct devlink *shd_priv_to_devlink(void *priv);
/**
* struct devlink_port_ops - Port operations
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c
index b925364765ac..2c58f820b018 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c
@@ -43,10 +43,10 @@ int mlx5_shd_init(struct mlx5_core_dev *dev)
*end = '\0';
/* Get or create shared devlink instance */
- devlink = devlink_shd_get(sn, &mlx5_shd_ops, 0, pdev->dev.driver);
+ devlink = devlink_shd_get(sn, &mlx5_shd_ops, 0, NULL, pdev->dev.driver);
kfree(sn);
- if (!devlink)
- return -ENOMEM;
+ if (IS_ERR(devlink))
+ return PTR_ERR(devlink);
dev->shd = devlink;
return 0;
diff --git a/net/devlink/sh_dev.c b/net/devlink/sh_dev.c
index 85acce97e788..4f1b2f24b755 100644
--- a/net/devlink/sh_dev.c
+++ b/net/devlink/sh_dev.c
@@ -34,43 +34,61 @@ static struct devlink_shd *devlink_shd_lookup(const char *id)
static struct devlink_shd *devlink_shd_create(const char *id,
const struct devlink_ops *ops,
size_t priv_size,
+ void *init_param,
const struct device_driver *driver)
{
struct devlink_shd *shd;
struct devlink *devlink;
+ int err;
devlink = __devlink_alloc(ops, sizeof(struct devlink_shd) + priv_size,
&init_net, NULL, driver);
if (!devlink)
- return NULL;
+ return ERR_PTR(-ENOMEM);
shd = devlink_priv(devlink);
shd->id = kstrdup(id, GFP_KERNEL);
- if (!shd->id)
+ if (!shd->id) {
+ err = -ENOMEM;
goto err_devlink_free;
+ }
shd->priv_size = priv_size;
- refcount_set(&shd->refcount, 1);
devl_lock(devlink);
+
+ if (ops->shd_init) {
+ err = ops->shd_init(shd->priv, init_param);
+ if (err)
+ goto err_unlock;
+ }
devl_register(devlink);
devl_unlock(devlink);
+ refcount_set(&shd->refcount, 1);
list_add_tail(&shd->list, &shd_list);
return shd;
+err_unlock:
+ devl_unlock(devlink);
+ kfree(shd->id);
+
err_devlink_free:
devlink_free(devlink);
- return NULL;
+ return ERR_PTR(err);
}
static void devlink_shd_destroy(struct devlink_shd *shd)
{
struct devlink *devlink = priv_to_devlink(shd);
list_del(&shd->list);
devl_lock(devlink);
devl_unregister(devlink);
+
+ if (devlink->ops->shd_fini)
+ devlink->ops->shd_fini(shd->priv);
+
devl_unlock(devlink);
kfree(shd->id);
devlink_free(devlink);
@@ -81,46 +99,53 @@ static void devlink_shd_destroy(struct devlink_shd *shd)
* @id: Identifier string (e.g., serial number) for the shared instance
* @ops: Devlink operations structure
* @priv_size: Size of private data structure
+ * @init_param: Passed to .shd_init() callback alongside driver's priv;
+ * this value need not match across all users of the shared
+ * instance
* @driver: Driver associated with the shared devlink instance
*
* Get an existing shared devlink instance identified by @id, or create
* a new one if it doesn't exist. Return the devlink instance with a
* reference held. The caller must call devlink_shd_put() when done.
*
+ * On creation, @ops->shd_init() is called (if set), see its description
+ * for the calling context. Its error is returned to the caller.
+ *
* All callers sharing the same @id must pass identical @ops, @priv_size
- * and @driver. A mismatch triggers a warning and returns NULL.
+ * and @driver. A mismatch triggers a warning and returns ERR_PTR(-EINVAL).
*
* Return: Pointer to the shared devlink instance on success,
- * NULL on failure
+ * ERR_PTR() on failure
*/
struct devlink *devlink_shd_get(const char *id,
const struct devlink_ops *ops,
size_t priv_size,
+ void *init_param,
const struct device_driver *driver)
{
struct devlink *devlink;
struct devlink_shd *shd;
mutex_lock(&shd_mutex);
shd = devlink_shd_lookup(id);
if (!shd) {
- shd = devlink_shd_create(id, ops, priv_size, driver);
+ shd = devlink_shd_create(id, ops, priv_size, init_param, driver);
goto unlock;
}
devlink = priv_to_devlink(shd);
if (WARN_ON_ONCE(devlink->ops != ops ||
shd->priv_size != priv_size ||
devlink->dev_driver != driver)) {
- shd = NULL;
+ shd = ERR_PTR(-EINVAL);
goto unlock;
}
refcount_inc(&shd->refcount);
unlock:
mutex_unlock(&shd_mutex);
- return shd ? priv_to_devlink(shd) : NULL;
+ return IS_ERR(shd) ? ERR_CAST(shd) : priv_to_devlink(shd);
}
EXPORT_SYMBOL_GPL(devlink_shd_get);
@@ -159,3 +184,17 @@ void *devlink_shd_get_priv(struct devlink *devlink)
return shd->priv;
}
EXPORT_SYMBOL_GPL(devlink_shd_get_priv);
+
+/**
+ * shd_priv_to_devlink - Get shared devlink instance from its priv data
+ * @priv: Driver's priv data, as returned by devlink_shd_get_priv()
+ *
+ * Return: pointer to shared devlink instance the @priv belongs to.
+ */
+struct devlink *shd_priv_to_devlink(void *priv)
+{
+ struct devlink_shd *shd = container_of(priv, struct devlink_shd, priv);
+
+ return priv_to_devlink(shd);
+}
+EXPORT_SYMBOL_GPL(shd_priv_to_devlink);
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 02/15] ice: use shared devlink to store ice_adapters instead of custom xarray
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 01/15] devlink, mlx5: add init/fini ops for shared devlink Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 03/15] ice: add VF queue ena/dis helper functions Przemek Kitszel
` (12 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel,
Sergey Temerkhanov, Grzegorz Nitka
Refactor our storage and deduplication logic of ice_adapters by moving
it to be handled by shared devlink instance, recently added by
Jiri Pirko [1].
We wanted the devlink instance for whole device anyway - later in the
series I will add devlink resources under it.
Make the shared devlink a parent (wrt. nesting) of the actual PF devices.
[1] commit 411ad0605875 ("Merge branch 'devlink-introduce-shared-devlink-instance-for-pfs-on-same-chip'")
[1] https://lore.kernel.org/all/20260312100407.551173-1-jiri@resnulli.us
Reviewed-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
CC: Sergey Temerkhanov <sergey.temerkhanov@intel.com>
CC: Grzegorz Nitka <grzegorz.nitka@intel.com>
v2:
* propagate the error from devlink_shd_get() with ERR_CAST()
* minor kdoc updates
* check devl_nested_devlink_set() return value, ice_devlink_register()
returns int now (Sashiko)
* prefix the shared devlink id with "ice:", to avoid collisions with
ids used by other drivers (Sashiko)
Sashiko suggestions I don't want to apply:
S: all PFs without DSN get index 0, and so share one instance
me: pre-existing, and DSN 0 is only on pre-production samples
---
.../net/ethernet/intel/ice/devlink/devlink.h | 2 +-
drivers/net/ethernet/intel/ice/ice_adapter.h | 12 +-
.../net/ethernet/intel/ice/devlink/devlink.c | 11 +-
drivers/net/ethernet/intel/ice/ice_adapter.c | 106 ++++++------------
drivers/net/ethernet/intel/ice/ice_main.c | 11 +-
5 files changed, 57 insertions(+), 85 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/devlink/devlink.h b/drivers/net/ethernet/intel/ice/devlink/devlink.h
index 1af3b0763fbb..62af38d11fd9 100644
--- a/drivers/net/ethernet/intel/ice/devlink/devlink.h
+++ b/drivers/net/ethernet/intel/ice/devlink/devlink.h
@@ -7,7 +7,7 @@
struct ice_pf *ice_allocate_pf(struct device *dev);
struct ice_sf_priv *ice_allocate_sf(struct device *dev, struct ice_pf *pf);
-void ice_devlink_register(struct ice_pf *pf);
+int ice_devlink_register(struct ice_pf *pf);
void ice_devlink_unregister(struct ice_pf *pf);
int ice_devlink_register_params(struct ice_pf *pf);
void ice_devlink_unregister_params(struct ice_pf *pf);
diff --git a/drivers/net/ethernet/intel/ice/ice_adapter.h b/drivers/net/ethernet/intel/ice/ice_adapter.h
index fd5585005d91..698a5b9af82f 100644
--- a/drivers/net/ethernet/intel/ice/ice_adapter.h
+++ b/drivers/net/ethernet/intel/ice/ice_adapter.h
@@ -8,7 +8,8 @@
#include <linux/mutex.h>
#include <linux/srcu.h>
#include <linux/spinlock_types.h>
-#include <linux/refcount_types.h>
+
+#include <net/devlink.h>
#include "ice_type.h"
@@ -35,20 +36,20 @@ struct ice_port_list {
/**
* struct ice_adapter - PCI adapter resources shared across PFs
- * @refcount: Reference count. struct ice_pf objects hold the references.
+ * @devlink: ice adapter's devlink (whole dev devlink)
* @ptp_gltsyn_time_lock: Spinlock protecting access to the GLTSYN_TIME
* register of the PTP clock.
* @txq_ctx_lock: Spinlock protecting access to the GLCOMM_QTX_CNTX_CTL register
* @cpi_phy_lock: Per-PHY mutex serializing CPI REQ/ACK transactions.
* Index 0 = PHY0, index 1 = PHY1. Used on E825C devices.
* @cgu_synce_lock: Mutex serializing changes of the CGU registers R10 and R11,
* which all PFs reach. Used on E825C devices.
* @ctrl_pf: Control PF of the adapter
* @ports: Ports list
- * @index: 64-bit index cached for collision detection on 32bit systems
*/
struct ice_adapter {
- refcount_t refcount;
+ struct devlink *devlink;
+
/* For access to the GLTSYN_TIME register */
spinlock_t ptp_gltsyn_time_lock;
/* For access to GLCOMM_QTX_CNTX_CTL register */
@@ -60,10 +61,9 @@ struct ice_adapter {
struct ice_pf *ctrl_pf;
struct ice_port_list ports;
- u64 index;
};
struct ice_adapter *ice_adapter_get(struct pci_dev *pdev);
-void ice_adapter_put(struct pci_dev *pdev);
+void ice_adapter_put(struct ice_adapter *adapter);
#endif /* _ICE_ADAPTER_H */
diff --git a/drivers/net/ethernet/intel/ice/devlink/devlink.c b/drivers/net/ethernet/intel/ice/devlink/devlink.c
index a87a3e0a900c..3ce6162569d6 100644
--- a/drivers/net/ethernet/intel/ice/devlink/devlink.c
+++ b/drivers/net/ethernet/intel/ice/devlink/devlink.c
@@ -1747,11 +1747,20 @@ struct ice_sf_priv *ice_allocate_sf(struct device *dev, struct ice_pf *pf)
*
* Return: zero on success or an error code on failure.
*/
-void ice_devlink_register(struct ice_pf *pf)
+int ice_devlink_register(struct ice_pf *pf)
{
struct devlink *devlink = priv_to_devlink(pf);
+ struct ice_adapter *adapter = pf->adapter;
+ int err;
+ if (adapter) {
+ err = devl_nested_devlink_set(adapter->devlink, devlink);
+ if (err)
+ return err;
+ }
devl_register(devlink);
+
+ return 0;
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_adapter.c b/drivers/net/ethernet/intel/ice/ice_adapter.c
index aa2d8addae5b..8cb61f145d27 100644
--- a/drivers/net/ethernet/intel/ice/ice_adapter.c
+++ b/drivers/net/ethernet/intel/ice/ice_adapter.c
@@ -1,19 +1,14 @@
// SPDX-License-Identifier: GPL-2.0-only
// SPDX-FileCopyrightText: Copyright Red Hat
-#include <linux/cleanup.h>
-#include <linux/mutex.h>
#include <linux/pci.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/srcu.h>
-#include <linux/xarray.h>
+
#include "ice_adapter.h"
#include "ice.h"
-static DEFINE_XARRAY(ice_adapters);
-static DEFINE_MUTEX(ice_adapters_mutex);
-
#define ICE_ADAPTER_FIXED_INDEX BIT_ULL(63)
#define ICE_ADAPTER_INDEX_E825C \
@@ -41,58 +36,43 @@ static u64 ice_adapter_index(struct pci_dev *pdev)
}
}
-static unsigned long ice_adapter_xa_index(struct pci_dev *pdev)
+static int ice_adapter_init(void *priv, void *init_param)
{
- u64 index = ice_adapter_index(pdev);
-
-#if BITS_PER_LONG == 64
- return index;
-#else
- return (u32)index ^ (u32)(index >> 32);
-#endif
-}
+ struct ice_adapter *adapter = priv;
+ struct devlink *devlink;
-static struct ice_adapter *ice_adapter_new(struct pci_dev *pdev)
-{
- struct ice_adapter *adapter;
- int err;
+ devlink = shd_priv_to_devlink(adapter);
+ adapter->devlink = devlink;
- adapter = kzalloc_obj(*adapter);
- if (!adapter)
- return NULL;
-
- err = init_srcu_struct(&adapter->ports.srcu);
- if (err) {
- kfree(adapter);
- return NULL;
- }
-
- adapter->index = ice_adapter_index(pdev);
spin_lock_init(&adapter->ptp_gltsyn_time_lock);
spin_lock_init(&adapter->txq_ctx_lock);
for (int i = 0; i < ARRAY_SIZE(adapter->cpi_phy_lock); i++)
mutex_init(&adapter->cpi_phy_lock[i]);
mutex_init(&adapter->cgu_synce_lock);
- refcount_set(&adapter->refcount, 1);
spin_lock_init(&adapter->ports.lock);
INIT_LIST_HEAD(&adapter->ports.list);
- return adapter;
+ return init_srcu_struct(&adapter->ports.srcu);
}
-static void ice_adapter_free(struct ice_adapter *adapter)
+static void ice_adapter_fini(void *priv)
{
+ struct ice_adapter *adapter = priv;
+
WARN_ON(!list_empty(&adapter->ports.list));
for (int i = 0; i < ARRAY_SIZE(adapter->cpi_phy_lock); i++)
mutex_destroy(&adapter->cpi_phy_lock[i]);
mutex_destroy(&adapter->cgu_synce_lock);
cleanup_srcu_struct(&adapter->ports.srcu);
-
- kfree(adapter);
}
+static const struct devlink_ops ice_adapter_devlink_ops = {
+ .shd_init = ice_adapter_init,
+ .shd_fini = ice_adapter_fini,
+};
+
/**
* ice_adapter_get - Get a shared ice_adapter structure.
* @pdev: Pointer to the pci_dev whose driver is getting the ice_adapter.
@@ -104,59 +84,37 @@ static void ice_adapter_free(struct ice_adapter *adapter)
*
* Context: Process, may sleep.
* Return: Pointer to ice_adapter on success.
- * ERR_PTR() on error. -ENOMEM is the only possible error.
+ * ERR_PTR() on error.
*/
struct ice_adapter *ice_adapter_get(struct pci_dev *pdev)
{
struct ice_adapter *adapter;
- unsigned long index;
- int err;
-
- index = ice_adapter_xa_index(pdev);
- scoped_guard(mutex, &ice_adapters_mutex) {
- adapter = xa_load(&ice_adapters, index);
- if (adapter) {
- refcount_inc(&adapter->refcount);
- WARN_ON_ONCE(adapter->index != ice_adapter_index(pdev));
- return adapter;
- }
- err = xa_reserve(&ice_adapters, index, GFP_KERNEL);
- if (err)
- return ERR_PTR(err);
-
- adapter = ice_adapter_new(pdev);
- if (!adapter) {
- xa_release(&ice_adapters, index);
- return ERR_PTR(-ENOMEM);
- }
- xa_store(&ice_adapters, index, adapter, GFP_KERNEL);
- }
+ struct devlink *devlink;
+ char devlink_id[32];
+ u64 index;
+
+ index = ice_adapter_index(pdev);
+ snprintf(devlink_id, sizeof(devlink_id), "ice:%llx", index);
+ devlink = devlink_shd_get(devlink_id, &ice_adapter_devlink_ops,
+ sizeof(*adapter), NULL, pdev->dev.driver);
+ if (IS_ERR(devlink))
+ return ERR_CAST(devlink);
+
+ adapter = devlink_shd_get_priv(devlink);
+
return adapter;
}
/**
* ice_adapter_put - Release a reference to the shared ice_adapter structure.
- * @pdev: Pointer to the pci_dev whose driver is releasing the ice_adapter.
+ * @adapter: the ice_adapter to release reference to
*
* Releases the reference to ice_adapter previously obtained with
* ice_adapter_get.
*
* Context: Process, may sleep.
*/
-void ice_adapter_put(struct pci_dev *pdev)
+void ice_adapter_put(struct ice_adapter *adapter)
{
- struct ice_adapter *adapter;
- unsigned long index;
-
- index = ice_adapter_xa_index(pdev);
- scoped_guard(mutex, &ice_adapters_mutex) {
- adapter = xa_load(&ice_adapters, index);
- if (WARN_ON(!adapter))
- return;
- if (!refcount_dec_and_test(&adapter->refcount))
- return;
-
- WARN_ON(xa_erase(&ice_adapters, index) != adapter);
- }
- ice_adapter_free(adapter);
+ devlink_shd_put(adapter->devlink);
}
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 47c6798c97e8..c8f06c3d0fdf 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -4945,7 +4945,12 @@ static int ice_init_devlink(struct ice_pf *pf)
return err;
ice_devlink_init_regions(pf);
- ice_devlink_register(pf);
+ err = ice_devlink_register(pf);
+ if (err) {
+ ice_devlink_destroy_regions(pf);
+ ice_devlink_unregister_params(pf);
+ return err;
+ }
ice_health_init(pf);
return 0;
@@ -5290,7 +5295,7 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
unroll_dev_init:
need_dev_deinit = true;
unroll_adapter:
- ice_adapter_put(pdev);
+ ice_adapter_put(adapter);
unroll_hw_init:
ice_deinit_hw(hw);
if (need_dev_deinit)
@@ -5403,7 +5408,7 @@ static void ice_remove(struct pci_dev *pdev)
ice_setup_mc_magic_wake(pf);
ice_set_wake(pf);
- ice_adapter_put(pdev);
+ ice_adapter_put(pf->adapter);
ice_deinit_hw(&pf->hw);
ice_deinit_dev(pf);
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 03/15] ice: add VF queue ena/dis helper functions
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 01/15] devlink, mlx5: add init/fini ops for shared devlink Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 02/15] ice: use shared devlink to store ice_adapters instead of custom xarray Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 04/15] ice: add helpers for Global RSS LUT alloc, free, vsi_update Przemek Kitszel
` (11 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel,
Wojciech Drewek
From: Brett Creeley <brett.creeley@intel.com>
Add three new functions:
- ice_vf_vsi_dis_single_rxq()
- ice_vf_vsi_ena_single_rxq()
- ice_vf_vsi_ena_single_txq()
(ice_vf_vsi_dis_single_txq() was introduced earlier).
Those functions wrap operations needed in the processes of enabling
and disabling single Tx/Rx VF queue, which are:
- check if the queue is not already in desired state
- perform the dis/ena operations
- bookkeeping of the queue's state.
Future commit will use them from another callsite.
Signed-off-by: Brett Creeley <brett.creeley@intel.com>
Signed-off-by: Wojciech Drewek <wojciech.drewek@intel.com>
Reviewed-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v2:
* convert one comment to kdoc (Alex)
* fix typos (Sashiko)
* rebase: move ice_vf_dis_rxq_interrupt(), added to ice_vc_dis_qs_msg()
by commit fb096882095e ("ice: fix VF interrupts cleanup"), into
ice_vf_vsi_dis_single_rxq()
---
drivers/net/ethernet/intel/ice/virt/queues.c | 117 ++++++++++++++-----
1 file changed, 89 insertions(+), 28 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/virt/queues.c b/drivers/net/ethernet/intel/ice/virt/queues.c
index ac7f98479b13..a22780575086 100644
--- a/drivers/net/ethernet/intel/ice/virt/queues.c
+++ b/drivers/net/ethernet/intel/ice/virt/queues.c
@@ -242,6 +242,57 @@ static void ice_vf_dis_rxq_interrupt(struct ice_vsi *vsi, u32 q_idx)
ice_flush(hw);
}
+/**
+ * ice_vf_vsi_ena_single_rxq - enable single Rx queue based on relative q_id
+ * @vf: VF to enable queue for
+ * @vsi: VSI for the VF
+ * @q_id: VSI relative (0-based) queue ID
+ *
+ * Enable the Rx queue passed in.
+ *
+ * Return: 0 on success or negative on error.
+ */
+static int ice_vf_vsi_ena_single_rxq(struct ice_vf *vf, struct ice_vsi *vsi,
+ u16 q_id)
+{
+ int err;
+
+ if (test_bit(q_id, vf->rxq_ena))
+ return 0;
+
+ err = ice_vsi_ctrl_one_rx_ring(vsi, true, q_id, true);
+ if (err) {
+ dev_err(ice_pf_to_dev(vsi->back), "Failed to enable Rx ring %d on VSI %d\n",
+ q_id, vsi->vsi_num);
+ return err;
+ }
+
+ ice_vf_ena_rxq_interrupt(vsi, q_id);
+ set_bit(q_id, vf->rxq_ena);
+
+ return 0;
+}
+
+/**
+ * ice_vf_vsi_ena_single_txq - enable single Tx queue based on relative q_id
+ * @vf: VF to enable queue for
+ * @vsi: VSI for the VF
+ * @q_id: VSI relative (0-based) queue ID
+ *
+ * Enable the Tx queue's interrupt. Note that the Tx queue(s) should have
+ * already been configured/enabled in VIRTCHNL_OP_CONFIG_VSI_QUEUES so this
+ * function only enables the interrupt associated with the q_id.
+ */
+static void ice_vf_vsi_ena_single_txq(struct ice_vf *vf, struct ice_vsi *vsi,
+ u16 q_id)
+{
+ if (test_bit(q_id, vf->txq_ena))
+ return;
+
+ ice_vf_ena_txq_interrupt(vsi, q_id);
+ set_bit(q_id, vf->txq_ena);
+}
+
/**
* ice_vc_ena_qs_msg
* @vf: pointer to the VF info
@@ -290,34 +341,20 @@ int ice_vc_ena_qs_msg(struct ice_vf *vf, u8 *msg)
goto error_param;
}
- /* Skip queue if enabled */
- if (test_bit(vf_q_id, vf->rxq_ena))
- continue;
-
- if (ice_vsi_ctrl_one_rx_ring(vsi, true, vf_q_id, true)) {
- dev_err(ice_pf_to_dev(vsi->back), "Failed to enable Rx ring %d on VSI %d\n",
- vf_q_id, vsi->vsi_num);
+ if (ice_vf_vsi_ena_single_rxq(vf, vsi, vf_q_id)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
-
- ice_vf_ena_rxq_interrupt(vsi, vf_q_id);
- set_bit(vf_q_id, vf->rxq_ena);
}
q_map = vqs->tx_queues;
for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) {
if (!ice_vc_isvalid_q_id(vsi, vf_q_id)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
- /* Skip queue if enabled */
- if (test_bit(vf_q_id, vf->txq_ena))
- continue;
-
- ice_vf_ena_txq_interrupt(vsi, vf_q_id);
- set_bit(vf_q_id, vf->txq_ena);
+ ice_vf_vsi_ena_single_txq(vf, vsi, vf_q_id);
}
/* Set flag to indicate that queues are enabled */
@@ -369,6 +406,41 @@ int ice_vf_vsi_dis_single_txq(struct ice_vf *vf, struct ice_vsi *vsi, u16 q_id)
return 0;
}
+/**
+ * ice_vf_vsi_dis_single_rxq - disable a Rx queue for VF on relative queue ID
+ * @vf: VF to disable queue for
+ * @vsi: VSI for the VF
+ * @q_id: VSI relative (0-based) queue ID
+ *
+ * Attempt to disable the Rx queue passed in. If the Rx queue was successfully
+ * disabled then clear q_id bit in the enabled queues bitmap.
+ *
+ * Return: 0 when queue was already disabled or disabling succeeded, otherwise
+ * negative error code.
+ */
+static int ice_vf_vsi_dis_single_rxq(struct ice_vf *vf, struct ice_vsi *vsi,
+ u16 q_id)
+{
+ int err;
+
+ if (!test_bit(q_id, vf->rxq_ena))
+ return 0;
+
+ err = ice_vsi_ctrl_one_rx_ring(vsi, false, q_id, true);
+ if (err) {
+ dev_err(ice_pf_to_dev(vsi->back), "Failed to stop Rx ring %d on VSI %d\n",
+ q_id, vsi->vsi_num);
+ return err;
+ }
+
+ ice_vf_dis_rxq_interrupt(vsi, q_id);
+
+ /* Clear enabled queues flag */
+ clear_bit(q_id, vf->rxq_ena);
+
+ return 0;
+}
+
/**
* ice_vc_dis_qs_msg
* @vf: pointer to the VF info
@@ -433,21 +505,10 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg)
goto error_param;
}
- /* Skip queue if not enabled */
- if (!test_bit(vf_q_id, vf->rxq_ena))
- continue;
-
- if (ice_vsi_ctrl_one_rx_ring(vsi, false, vf_q_id,
- true)) {
- dev_err(ice_pf_to_dev(vsi->back), "Failed to stop Rx ring %d on VSI %d\n",
- vf_q_id, vsi->vsi_num);
+ if (ice_vf_vsi_dis_single_rxq(vf, vsi, vf_q_id)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
-
- ice_vf_dis_rxq_interrupt(vsi, vf_q_id);
- /* Clear enabled queues flag */
- clear_bit(vf_q_id, vf->rxq_ena);
}
}
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 04/15] ice: add helpers for Global RSS LUT alloc, free, vsi_update
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (2 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 03/15] ice: add VF queue ena/dis helper functions Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 05/15] ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1 Przemek Kitszel
` (10 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Add AQ commands for RSS Global LUT allocation and free operations.
The new functions are used by subsequent commits.
Add programming code for GLOBAL LUT ID of UPDATE VSI AQ,
do the same for RSS LUT "type", also for PF LUT in case of VF VSI.
Co-developed-by: Brett Creeley <brett.creeley@intel.com>
Signed-off-by: Brett Creeley <brett.creeley@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v2: fix whitespace (Sashiko)
---
drivers/net/ethernet/intel/ice/ice.h | 1 +
.../net/ethernet/intel/ice/ice_adminq_cmd.h | 1 +
drivers/net/ethernet/intel/ice/ice_switch.h | 2 +
drivers/net/ethernet/intel/ice/ice_lib.c | 28 +++++++++++--
drivers/net/ethernet/intel/ice/ice_switch.c | 41 +++++++++++++++++++
5 files changed, 69 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 432da05c48c0..89a2bdbecf60 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -367,6 +367,7 @@ struct ice_vsi {
u8 *rss_hkey_user; /* User configured hash keys */
u8 *rss_lut_user; /* User configured lookup table entries */
u8 rss_lut_type; /* used to configure Get/Set RSS LUT AQ call */
+ u8 global_lut_id; /* valid when lut_type == GLOBAL_LUT */
/* aRFS members only allocated for the PF VSI */
#define ICE_MAX_ARFS_LIST 1024
diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
index 42878abac9eb..90c5587642d0 100644
--- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
@@ -169,6 +169,7 @@ struct ice_aqc_set_port_params {
#define ICE_AQC_RES_TYPE_VSI_LIST_PRUNE 0x04
#define ICE_AQC_RES_TYPE_RECIPE 0x05
#define ICE_AQC_RES_TYPE_SWID 0x07
+#define ICE_AQC_RES_TYPE_GLOBAL_RSS_HASH 0x20
#define ICE_AQC_RES_TYPE_FDIR_COUNTER_BLOCK 0x21
#define ICE_AQC_RES_TYPE_FDIR_GUARANTEED_ENTRIES 0x22
#define ICE_AQC_RES_TYPE_FDIR_SHARED_ENTRIES 0x23
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.h b/drivers/net/ethernet/intel/ice/ice_switch.h
index 671d7a5f359f..99b593b68bd0 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.h
+++ b/drivers/net/ethernet/intel/ice/ice_switch.h
@@ -387,6 +387,8 @@ ice_rem_adv_rule_by_id(struct ice_hw *hw,
struct ice_rule_query_data *remove_entry);
int ice_init_def_sw_recp(struct ice_hw *hw);
+int ice_alloc_rss_global_lut(struct ice_hw *hw, u16 *global_lut_id);
+int ice_free_rss_global_lut(struct ice_hw *hw, u16 global_lut_id);
u16 ice_get_hw_vsi_num(struct ice_hw *hw, u16 vsi_handle);
int ice_replay_vsi_all_fltr(struct ice_hw *hw, u16 vsi_handle);
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 977cd8dd6e31..015b1886ba4e 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1244,30 +1244,46 @@ static void ice_set_fd_vsi_ctx(struct ice_vsi_ctx *ctxt, struct ice_vsi *vsi)
ctxt->info.fd_report_opt = cpu_to_le16(val);
}
+/* Translate @lut_type used in most of the places to the Admin Queue
+ * Q_OPT value for RSS.
+ * Used with VSI ADD and VSI UPDATE AQs (opcodes 0x0210, 0x0211).
+ */
+static u8 ice_lut_type_to_aq_qopt_rss_val(enum ice_lut_type lut_type)
+{
+ switch (lut_type) {
+ case ICE_LUT_PF:
+ return ICE_AQ_VSI_Q_OPT_RSS_LUT_PF;
+ case ICE_LUT_GLOBAL:
+ return ICE_AQ_VSI_Q_OPT_RSS_LUT_GBL;
+ case ICE_LUT_VSI:
+ default:
+ return ICE_AQ_VSI_Q_OPT_RSS_LUT_VSI;
+ }
+}
+
/**
* ice_set_rss_vsi_ctx - Set RSS VSI context before adding a VSI
* @ctxt: the VSI context being set
* @vsi: the VSI being configured
*/
static void ice_set_rss_vsi_ctx(struct ice_vsi_ctx *ctxt, struct ice_vsi *vsi)
{
u8 lut_type, hash_type;
+ u8 global_lut_id = 0;
struct device *dev;
struct ice_pf *pf;
pf = vsi->back;
dev = ice_pf_to_dev(pf);
switch (vsi->type) {
case ICE_VSI_CHNL:
- case ICE_VSI_PF:
- /* PF VSI will inherit RSS instance of PF */
lut_type = ICE_AQ_VSI_Q_OPT_RSS_LUT_PF;
break;
+ case ICE_VSI_PF:
case ICE_VSI_VF:
case ICE_VSI_SF:
- /* VF VSI will gets a small RSS table which is a VSI LUT type */
- lut_type = ICE_AQ_VSI_Q_OPT_RSS_LUT_VSI;
+ lut_type = ice_lut_type_to_aq_qopt_rss_val(vsi->rss_lut_type);
break;
default:
dev_dbg(dev, "Unsupported VSI type %s\n",
@@ -1278,8 +1294,12 @@ static void ice_set_rss_vsi_ctx(struct ice_vsi_ctx *ctxt, struct ice_vsi *vsi)
hash_type = ICE_AQ_VSI_Q_OPT_RSS_HASH_TPLZ;
vsi->rss_hfunc = hash_type;
+ if (vsi->rss_lut_type == ICE_LUT_GLOBAL)
+ global_lut_id = vsi->global_lut_id;
+
ctxt->info.q_opt_rss =
FIELD_PREP(ICE_AQ_VSI_Q_OPT_RSS_LUT_M, lut_type) |
+ FIELD_PREP(ICE_AQ_VSI_Q_OPT_RSS_GBL_LUT_M, global_lut_id) |
FIELD_PREP(ICE_AQ_VSI_Q_OPT_RSS_HASH_M, hash_type);
}
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.c b/drivers/net/ethernet/intel/ice/ice_switch.c
index d873eda9af82..567b85bc5a39 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.c
+++ b/drivers/net/ethernet/intel/ice/ice_switch.c
@@ -1527,6 +1527,47 @@ ice_aq_get_sw_cfg(struct ice_hw *hw, struct ice_aqc_get_sw_cfg_resp_elem *buf,
return status;
}
+/* Allocate a new Global LUT for the caller.
+ * LUT ID is returned via @global_lut_id.
+ */
+int ice_alloc_rss_global_lut(struct ice_hw *hw, u16 *global_lut_id)
+{
+ DEFINE_RAW_FLEX(struct ice_aqc_alloc_free_res_elem, buf, elem, 1);
+ u16 buf_len = __struct_size(buf);
+ int err;
+
+ buf->num_elems = cpu_to_le16(1);
+ buf->res_type = cpu_to_le16(ICE_AQC_RES_TYPE_GLOBAL_RSS_HASH);
+
+ err = ice_aq_alloc_free_res(hw, buf, buf_len, ice_aqc_opc_alloc_res);
+ if (err)
+ ice_debug(hw, ICE_DBG_RES, "Failed to allocate RSS global LUT, err %d\n",
+ err);
+ else
+ *global_lut_id = le16_to_cpu(buf->elem[0].e.sw_resp);
+
+ return err;
+}
+
+/* Free Global LUT at @global_lut_id. */
+int ice_free_rss_global_lut(struct ice_hw *hw, u16 global_lut_id)
+{
+ DEFINE_RAW_FLEX(struct ice_aqc_alloc_free_res_elem, buf, elem, 1);
+ u16 buf_len = __struct_size(buf);
+ int err;
+
+ buf->num_elems = cpu_to_le16(1);
+ buf->res_type = cpu_to_le16(ICE_AQC_RES_TYPE_GLOBAL_RSS_HASH);
+ buf->elem[0].e.sw_resp = cpu_to_le16(global_lut_id);
+
+ err = ice_aq_alloc_free_res(hw, buf, buf_len, ice_aqc_opc_free_res);
+ if (err)
+ ice_debug(hw, ICE_DBG_RES, "Failed to free RSS global LUT %d, err %d\n",
+ global_lut_id, err);
+
+ return err;
+}
+
/**
* ice_aq_add_vsi
* @hw: pointer to the HW struct
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 05/15] ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (3 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 04/15] ice: add helpers for Global RSS LUT alloc, free, vsi_update Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:44 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 06/15] ice: bump to 256qs for VF Przemek Kitszel
` (9 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1, in preparation for
the next patch that will extend the max to 256, using old value of 16 for
the "v1" variant of virtchnl opcodes.
Suggested-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
Suggested-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
drivers/net/ethernet/intel/ice/ice_lag.h | 2 +-
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 9 +++---
drivers/net/ethernet/intel/ice/ice_lib.c | 2 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 4 +--
drivers/net/ethernet/intel/ice/ice_vf_lib.c | 12 ++++----
drivers/net/ethernet/intel/ice/virt/queues.c | 30 ++++++++++----------
6 files changed, 30 insertions(+), 29 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_lag.h b/drivers/net/ethernet/intel/ice/ice_lag.h
index f77ebcd61042..4bfffecbdc97 100644
--- a/drivers/net/ethernet/intel/ice/ice_lag.h
+++ b/drivers/net/ethernet/intel/ice/ice_lag.h
@@ -52,7 +52,7 @@ struct ice_lag {
u8 bond_lport_sec; /* lport values for secondary PF */
/* q_home keeps track of which interface the q is currently on */
- u8 q_home[ICE_MAX_SRIOV_VFS][ICE_MAX_RSS_QS_PER_VF];
+ u8 q_home[ICE_MAX_SRIOV_VFS][ICE_MAX_QS_PER_VF_VCV1];
/* placeholder VSI for hanging VF queues from on secondary interface */
struct ice_vsi *sec_vf[ICE_MAX_SRIOV_VFS];
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
index fa436b3b1eac..4959ecee0c21 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
@@ -19,7 +19,8 @@
#define ICE_MAX_SRIOV_VFS 256
/* VF resource constraints */
-#define ICE_MAX_RSS_QS_PER_VF 16
+/* for "old" virtchnl opcodes that accept up to 16 queues */
+#define ICE_MAX_QS_PER_VF_VCV1 16
struct ice_pf;
struct ice_vf;
@@ -161,8 +162,8 @@ struct ice_vf {
u8 dev_lan_addr[ETH_ALEN];
u8 hw_lan_addr[ETH_ALEN];
struct ice_time_mac legacy_last_added_umac;
- DECLARE_BITMAP(txq_ena, ICE_MAX_RSS_QS_PER_VF);
- DECLARE_BITMAP(rxq_ena, ICE_MAX_RSS_QS_PER_VF);
+ DECLARE_BITMAP(txq_ena, ICE_MAX_QS_PER_VF_VCV1);
+ DECLARE_BITMAP(rxq_ena, ICE_MAX_QS_PER_VF_VCV1);
struct ice_vlan port_vlan_info; /* Port VLAN ID, QoS, and TPID */
struct virtchnl_vlan_caps vlan_v2_caps;
struct ice_mbx_vf_info mbx_info;
@@ -205,7 +206,7 @@ struct ice_vf {
u16 lldp_recipe_id;
u16 lldp_rule_id;
- struct ice_vf_qs_bw qs_bw[ICE_MAX_RSS_QS_PER_VF];
+ struct ice_vf_qs_bw qs_bw[ICE_MAX_QS_PER_VF_VCV1];
};
/* Flags for controlling behavior of ice_reset_vf */
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 015b1886ba4e..ae7568cbecd3 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1023,7 +1023,7 @@ static void ice_vsi_set_rss_params(struct ice_vsi *vsi)
* For VSI_LUT, LUT size should be set to 64 bytes.
*/
vsi->rss_table_size = ICE_LUT_VSI_SIZE;
- vsi->rss_size = ICE_MAX_RSS_QS_PER_VF;
+ vsi->rss_size = ICE_MAX_QS_PER_VF_VCV1;
vsi->rss_lut_type = ICE_LUT_VSI;
break;
case ICE_VSI_LB:
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index c49f48b5ae38..dcd6d48ea38d 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -398,15 +398,15 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
}
num_txq = min_t(u16, num_msix_per_vf - ICE_NONQ_VECS_VF,
- ICE_MAX_RSS_QS_PER_VF);
+ ICE_MAX_QS_PER_VF_VCV1);
avail_qs = ice_get_avail_txq_count(pf) / num_vfs;
if (!avail_qs)
num_txq = 0;
else if (num_txq > avail_qs)
num_txq = rounddown_pow_of_two(avail_qs);
num_rxq = min_t(u16, num_msix_per_vf - ICE_NONQ_VECS_VF,
- ICE_MAX_RSS_QS_PER_VF);
+ ICE_MAX_QS_PER_VF_VCV1);
avail_qs = ice_get_avail_rxq_count(pf) / num_vfs;
if (!avail_qs)
num_rxq = 0;
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
index a54cb2b8d3c7..be87a83d2599 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
@@ -528,8 +528,8 @@ static void ice_vf_rebuild_host_cfg(struct ice_vf *vf)
static void ice_set_vf_state_qs_dis(struct ice_vf *vf)
{
/* Clear Rx/Tx enabled queues flag */
- bitmap_zero(vf->txq_ena, ICE_MAX_RSS_QS_PER_VF);
- bitmap_zero(vf->rxq_ena, ICE_MAX_RSS_QS_PER_VF);
+ bitmap_zero(vf->txq_ena, ICE_MAX_QS_PER_VF_VCV1);
+ bitmap_zero(vf->rxq_ena, ICE_MAX_QS_PER_VF_VCV1);
clear_bit(ICE_VF_STATE_QS_ENA, vf->vf_states);
}
@@ -1238,13 +1238,13 @@ bool ice_is_vf_trusted(struct ice_vf *vf)
* ice_vf_has_no_qs_ena - check if the VF has any Rx or Tx queues enabled
* @vf: the VF to check
*
- * Returns true if the VF has no Rx and no Tx queues enabled and returns false
- * otherwise
+ * Return: true if the VF has no Rx and no Tx queues enabled and returns false
+ * otherwise.
*/
bool ice_vf_has_no_qs_ena(struct ice_vf *vf)
{
- return bitmap_empty(vf->rxq_ena, ICE_MAX_RSS_QS_PER_VF) &&
- bitmap_empty(vf->txq_ena, ICE_MAX_RSS_QS_PER_VF);
+ return bitmap_empty(vf->rxq_ena, ICE_MAX_QS_PER_VF_VCV1) &&
+ bitmap_empty(vf->txq_ena, ICE_MAX_QS_PER_VF_VCV1);
}
/**
diff --git a/drivers/net/ethernet/intel/ice/virt/queues.c b/drivers/net/ethernet/intel/ice/virt/queues.c
index a22780575086..2667634e3c8c 100644
--- a/drivers/net/ethernet/intel/ice/virt/queues.c
+++ b/drivers/net/ethernet/intel/ice/virt/queues.c
@@ -171,8 +171,8 @@ static int ice_vf_cfg_q_quanta_profile(struct ice_vf *vf, u16 quanta_size,
static bool ice_vc_validate_vqs_bitmaps(struct virtchnl_queue_select *vqs)
{
if ((!vqs->rx_queues && !vqs->tx_queues) ||
- vqs->rx_queues >= BIT(ICE_MAX_RSS_QS_PER_VF) ||
- vqs->tx_queues >= BIT(ICE_MAX_RSS_QS_PER_VF))
+ vqs->rx_queues >= BIT(ICE_MAX_QS_PER_VF_VCV1) ||
+ vqs->tx_queues >= BIT(ICE_MAX_QS_PER_VF_VCV1))
return false;
return true;
@@ -335,7 +335,7 @@ int ice_vc_ena_qs_msg(struct ice_vf *vf, u8 *msg)
* programmed using ice_vsi_cfg_txqs
*/
q_map = vqs->rx_queues;
- for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) {
+ for_each_set_bit(vf_q_id, &q_map, ICE_MAX_QS_PER_VF_VCV1) {
if (!ice_vc_isvalid_q_id(vsi, vf_q_id)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
@@ -348,7 +348,7 @@ int ice_vc_ena_qs_msg(struct ice_vf *vf, u8 *msg)
}
q_map = vqs->tx_queues;
- for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) {
+ for_each_set_bit(vf_q_id, &q_map, ICE_MAX_QS_PER_VF_VCV1) {
if (!ice_vc_isvalid_q_id(vsi, vf_q_id)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
@@ -484,7 +484,7 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg)
if (vqs->tx_queues) {
q_map = vqs->tx_queues;
- for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) {
+ for_each_set_bit(vf_q_id, &q_map, ICE_MAX_QS_PER_VF_VCV1) {
if (!ice_vc_isvalid_q_id(vsi, vf_q_id)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
@@ -499,7 +499,7 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg)
q_map = vqs->rx_queues;
if (q_map) {
- for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) {
+ for_each_set_bit(vf_q_id, &q_map, ICE_MAX_QS_PER_VF_VCV1) {
if (!ice_vc_isvalid_q_id(vsi, vf_q_id)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
@@ -542,7 +542,7 @@ ice_cfg_interrupt(struct ice_vf *vf, struct ice_vsi *vsi,
q_vector->num_ring_tx = 0;
qmap = map->rxq_map;
- for_each_set_bit(vsi_q_id_idx, &qmap, ICE_MAX_RSS_QS_PER_VF) {
+ for_each_set_bit(vsi_q_id_idx, &qmap, ICE_MAX_QS_PER_VF_VCV1) {
vsi_q_id = vsi_q_id_idx;
if (!ice_vc_isvalid_q_id(vsi, vsi_q_id))
@@ -557,7 +557,7 @@ ice_cfg_interrupt(struct ice_vf *vf, struct ice_vsi *vsi,
}
qmap = map->txq_map;
- for_each_set_bit(vsi_q_id_idx, &qmap, ICE_MAX_RSS_QS_PER_VF) {
+ for_each_set_bit(vsi_q_id_idx, &qmap, ICE_MAX_QS_PER_VF_VCV1) {
vsi_q_id = vsi_q_id_idx;
if (!ice_vc_isvalid_q_id(vsi, vsi_q_id))
@@ -681,7 +681,7 @@ int ice_vc_cfg_q_bw(struct ice_vf *vf, u8 *msg)
goto err;
}
- if (qbw->num_queues > ICE_MAX_RSS_QS_PER_VF ||
+ if (qbw->num_queues > ICE_MAX_QS_PER_VF_VCV1 ||
qbw->num_queues > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
dev_err(ice_pf_to_dev(vf->pf), "VF-%d trying to configure more than allocated number of queues: %d\n",
vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
@@ -773,7 +773,7 @@ int ice_vc_cfg_q_quanta(struct ice_vf *vf, u8 *msg)
goto err;
}
- if (end_qid > ICE_MAX_RSS_QS_PER_VF ||
+ if (end_qid > ICE_MAX_QS_PER_VF_VCV1 ||
end_qid > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
dev_err(ice_pf_to_dev(vf->pf), "VF-%d trying to configure more than allocated number of queues: %d\n",
vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
@@ -841,7 +841,7 @@ int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
if (!vsi)
goto error_param;
- if (qci->num_queue_pairs > ICE_MAX_RSS_QS_PER_VF ||
+ if (qci->num_queue_pairs > ICE_MAX_QS_PER_VF_VCV1 ||
qci->num_queue_pairs > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
dev_err(ice_pf_to_dev(pf), "VF-%d requesting more than supported number of queues: %d\n",
vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
@@ -1019,16 +1019,16 @@ int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg)
if (!req_queues) {
dev_err(dev, "VF %d tried to request 0 queues. Ignoring.\n",
vf->vf_id);
- } else if (req_queues > ICE_MAX_RSS_QS_PER_VF) {
+ } else if (req_queues > ICE_MAX_QS_PER_VF_VCV1) {
dev_err(dev, "VF %d tried to request more than %d queues.\n",
- vf->vf_id, ICE_MAX_RSS_QS_PER_VF);
- vfres->num_queue_pairs = ICE_MAX_RSS_QS_PER_VF;
+ vf->vf_id, ICE_MAX_QS_PER_VF_VCV1);
+ vfres->num_queue_pairs = ICE_MAX_QS_PER_VF_VCV1;
} else if (req_queues > cur_queues &&
req_queues - cur_queues > tx_rx_queue_left) {
dev_warn(dev, "VF %d requested %u more queues, but only %u left.\n",
vf->vf_id, req_queues - cur_queues, tx_rx_queue_left);
vfres->num_queue_pairs = min_t(u16, max_allowed_vf_queues,
- ICE_MAX_RSS_QS_PER_VF);
+ ICE_MAX_QS_PER_VF_VCV1);
} else {
/* request is successful, then reset VF */
vf->num_req_qs = req_queues;
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 06/15] ice: bump to 256qs for VF
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (4 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 05/15] ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1 Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:44 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 07/15] iavf: extend iavf_configure_queues() to support more queues Przemek Kitszel
` (8 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Adjust all bitmaps and arrays in ice to accept 256 VF queues.
Extend struct ice_vf::num_req_qs width to allow 256 queues.
Keep old/legacy size for virtchnl opcodes that were designed to accept
only up to 16 queues.
Raise also the limits for the default VF queue count and for
VIRTCHNL_OP_REQUEST_QUEUES to 256. New virtchnl opcodes, needed to
make use of more than 16 queues, are handled by subsequent commits
of this series, the work is split for easier review.
Reviewed-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v5:
* LAG: call ice_aq_cfg_lan_txq() for up to U8_MAX queues at a time, as
num_qs of the command is u8, 256 VF queues were sent as 0 (Sashiko)
v3:
* fix rollback in ice_vc_cfg_qs_msg() to disable queues by their IDs,
not by qpair index, as config could be now split into many messages;
unwind only the entries that got past validation (Sashiko)
v2:
* change ice_lag allocations to kvzalloc(), as they have grown to
66KB (Sashiko)
* commit message: mention that the default and requested VF queue
limits are raised already here, while more than 16 queues are
enabled by subsequent commits
Sashiko suggestions I don't want to apply:
S: there is an existing error handling issue in ice_init_lag()
me: pre-existing and unrelated, will be fixed separately
---
drivers/net/ethernet/intel/ice/ice_lag.h | 2 +-
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 9 ++--
drivers/net/ethernet/intel/ice/ice_lag.c | 54 ++++++++++++++++----
drivers/net/ethernet/intel/ice/ice_lib.c | 2 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 4 +-
drivers/net/ethernet/intel/ice/ice_vf_lib.c | 8 +--
drivers/net/ethernet/intel/ice/virt/queues.c | 28 +++++-----
7 files changed, 71 insertions(+), 36 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_lag.h b/drivers/net/ethernet/intel/ice/ice_lag.h
index 4bfffecbdc97..39f6a6cc844d 100644
--- a/drivers/net/ethernet/intel/ice/ice_lag.h
+++ b/drivers/net/ethernet/intel/ice/ice_lag.h
@@ -52,7 +52,7 @@ struct ice_lag {
u8 bond_lport_sec; /* lport values for secondary PF */
/* q_home keeps track of which interface the q is currently on */
- u8 q_home[ICE_MAX_SRIOV_VFS][ICE_MAX_QS_PER_VF_VCV1];
+ u8 q_home[ICE_MAX_SRIOV_VFS][ICE_MAX_QS_PER_VF];
/* placeholder VSI for hanging VF queues from on secondary interface */
struct ice_vsi *sec_vf[ICE_MAX_SRIOV_VFS];
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
index 4959ecee0c21..a9e95c85bc98 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
@@ -21,6 +21,7 @@
/* VF resource constraints */
/* for "old" virtchnl opcodes that accept up to 16 queues */
#define ICE_MAX_QS_PER_VF_VCV1 16
+#define ICE_MAX_QS_PER_VF 256
struct ice_pf;
struct ice_vf;
@@ -162,8 +163,8 @@ struct ice_vf {
u8 dev_lan_addr[ETH_ALEN];
u8 hw_lan_addr[ETH_ALEN];
struct ice_time_mac legacy_last_added_umac;
- DECLARE_BITMAP(txq_ena, ICE_MAX_QS_PER_VF_VCV1);
- DECLARE_BITMAP(rxq_ena, ICE_MAX_QS_PER_VF_VCV1);
+ DECLARE_BITMAP(txq_ena, ICE_MAX_QS_PER_VF);
+ DECLARE_BITMAP(rxq_ena, ICE_MAX_QS_PER_VF);
struct ice_vlan port_vlan_info; /* Port VLAN ID, QoS, and TPID */
struct virtchnl_vlan_caps vlan_v2_caps;
struct ice_mbx_vf_info mbx_info;
@@ -185,7 +186,7 @@ struct ice_vf {
DECLARE_BITMAP(vf_states, ICE_VF_STATES_NBITS); /* VF runtime states */
unsigned long vf_caps; /* VF's adv. capabilities */
- u8 num_req_qs; /* num of queue pairs requested by VF */
+ u16 num_req_qs; /* num of queue pairs requested by VF */
u16 num_mac;
u16 num_mac_lldp;
u16 num_vf_qs; /* num of queue configured per VF */
@@ -206,7 +207,7 @@ struct ice_vf {
u16 lldp_recipe_id;
u16 lldp_rule_id;
- struct ice_vf_qs_bw qs_bw[ICE_MAX_QS_PER_VF_VCV1];
+ struct ice_vf_qs_bw qs_bw[ICE_MAX_QS_PER_VF];
};
/* Flags for controlling behavior of ice_reset_vf */
diff --git a/drivers/net/ethernet/intel/ice/ice_lag.c b/drivers/net/ethernet/intel/ice/ice_lag.c
index 08a17ded0ad5..4bb28564c0d6 100644
--- a/drivers/net/ethernet/intel/ice/ice_lag.c
+++ b/drivers/net/ethernet/intel/ice/ice_lag.c
@@ -549,6 +549,41 @@ ice_lag_qbuf_recfg(struct ice_hw *hw, struct ice_aqc_cfg_txqs_buf *qbuf,
return count;
}
+/**
+ * ice_lag_cfg_lan_txq - reconfigure queues for a new port
+ * @hw: HW struct to send the command on
+ * @qbuf: buffer filled by ice_lag_qbuf_recfg(), gets modified
+ * @numq: number of queues in @qbuf
+ * @oldport: lport of the current queues location
+ * @newport: lport of the destination queues location
+ *
+ * One command takes up to U8_MAX queues, a VF could have more.
+ *
+ * Return: 0 on success, negative on failure.
+ */
+static int
+ice_lag_cfg_lan_txq(struct ice_hw *hw, struct ice_aqc_cfg_txqs_buf *qbuf,
+ u16 numq, u8 oldport, u8 newport)
+{
+ while (numq) {
+ int num = min(numq, U8_MAX);
+ int err;
+
+ err = ice_aq_cfg_lan_txq(hw, qbuf,
+ struct_size(qbuf, queue_info, num),
+ num, oldport, newport,
+ ICE_AQC_Q_CFG_TC_CHNG, NULL);
+ if (err)
+ return err;
+
+ numq -= num;
+ memmove(qbuf->queue_info, &qbuf->queue_info[num],
+ numq * sizeof(*qbuf->queue_info));
+ }
+
+ return 0;
+}
+
/**
* ice_lag_get_sched_parent - locate or create a sched node parent
* @hw: HW struct for getting parent in
@@ -689,8 +724,7 @@ ice_lag_move_vf_node_tc(struct ice_lag *lag, u8 oldport, u8 newport,
goto qbuf_none;
}
- if (ice_aq_cfg_lan_txq(&lag->pf->hw, qbuf, qbuf_size, valq, oldport,
- newport, ICE_AQC_Q_CFG_TC_CHNG, NULL)) {
+ if (ice_lag_cfg_lan_txq(&lag->pf->hw, qbuf, valq, oldport, newport)) {
dev_warn(dev, "Failure to configure queues for LAG failover\n");
goto qbuf_err;
}
@@ -1277,9 +1311,8 @@ ice_lag_reclaim_vf_tc(struct ice_lag *lag, struct ice_hw *src_hw, u16 vsi_num,
goto reclaim_none;
}
- if (ice_aq_cfg_lan_txq(hw, qbuf, qbuf_size, numq,
- src_hw->port_info->lport, hw->port_info->lport,
- ICE_AQC_Q_CFG_TC_CHNG, NULL)) {
+ if (ice_lag_cfg_lan_txq(hw, qbuf, numq, src_hw->port_info->lport,
+ hw->port_info->lport)) {
dev_warn(dev, "Failure to configure queues for LAG failover\n");
goto reclaim_qerr;
}
@@ -2495,9 +2528,8 @@ ice_lag_move_vf_nodes_tc_sync(struct ice_lag *lag, struct ice_hw *dest_hw,
goto sync_none;
}
- if (ice_aq_cfg_lan_txq(hw, qbuf, qbuf_size, numq, hw->port_info->lport,
- dest_hw->port_info->lport,
- ICE_AQC_Q_CFG_TC_CHNG, NULL)) {
+ if (ice_lag_cfg_lan_txq(hw, qbuf, numq, hw->port_info->lport,
+ dest_hw->port_info->lport)) {
dev_warn(dev, "Failure to configure queues for LAG reset rebuild\n");
goto sync_qerr;
}
@@ -2577,7 +2609,7 @@ int ice_init_lag(struct ice_pf *pf)
if (!ice_is_feature_supported(pf, ICE_F_SRIOV_LAG))
return 0;
- pf->lag = kzalloc_obj(*lag);
+ pf->lag = kvzalloc_obj(*lag);
if (!pf->lag)
return -ENOMEM;
lag = pf->lag;
@@ -2651,7 +2683,7 @@ int ice_init_lag(struct ice_pf *pf)
ice_free_hw_res(&pf->hw, ICE_AQC_RES_TYPE_RECIPE, 1,
&lag->pf_recipe);
lag_error:
- kfree(lag);
+ kvfree(lag);
pf->lag = NULL;
return err;
}
@@ -2680,7 +2712,7 @@ void ice_deinit_lag(struct ice_pf *pf)
ice_free_hw_res(&pf->hw, ICE_AQC_RES_TYPE_RECIPE, 1,
&pf->lag->lport_recipe);
- kfree(lag);
+ kvfree(lag);
pf->lag = NULL;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index ae7568cbecd3..cbdebb7e1750 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1023,7 +1023,7 @@ static void ice_vsi_set_rss_params(struct ice_vsi *vsi)
* For VSI_LUT, LUT size should be set to 64 bytes.
*/
vsi->rss_table_size = ICE_LUT_VSI_SIZE;
- vsi->rss_size = ICE_MAX_QS_PER_VF_VCV1;
+ vsi->rss_size = ICE_MAX_QS_PER_VF;
vsi->rss_lut_type = ICE_LUT_VSI;
break;
case ICE_VSI_LB:
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index dcd6d48ea38d..d7c4a6540008 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -398,15 +398,15 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
}
num_txq = min_t(u16, num_msix_per_vf - ICE_NONQ_VECS_VF,
- ICE_MAX_QS_PER_VF_VCV1);
+ ICE_MAX_QS_PER_VF);
avail_qs = ice_get_avail_txq_count(pf) / num_vfs;
if (!avail_qs)
num_txq = 0;
else if (num_txq > avail_qs)
num_txq = rounddown_pow_of_two(avail_qs);
num_rxq = min_t(u16, num_msix_per_vf - ICE_NONQ_VECS_VF,
- ICE_MAX_QS_PER_VF_VCV1);
+ ICE_MAX_QS_PER_VF);
avail_qs = ice_get_avail_rxq_count(pf) / num_vfs;
if (!avail_qs)
num_rxq = 0;
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
index be87a83d2599..320b393ddaaf 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
@@ -528,8 +528,8 @@ static void ice_vf_rebuild_host_cfg(struct ice_vf *vf)
static void ice_set_vf_state_qs_dis(struct ice_vf *vf)
{
/* Clear Rx/Tx enabled queues flag */
- bitmap_zero(vf->txq_ena, ICE_MAX_QS_PER_VF_VCV1);
- bitmap_zero(vf->rxq_ena, ICE_MAX_QS_PER_VF_VCV1);
+ bitmap_zero(vf->txq_ena, ICE_MAX_QS_PER_VF);
+ bitmap_zero(vf->rxq_ena, ICE_MAX_QS_PER_VF);
clear_bit(ICE_VF_STATE_QS_ENA, vf->vf_states);
}
@@ -1243,8 +1243,8 @@ bool ice_is_vf_trusted(struct ice_vf *vf)
*/
bool ice_vf_has_no_qs_ena(struct ice_vf *vf)
{
- return bitmap_empty(vf->rxq_ena, ICE_MAX_QS_PER_VF_VCV1) &&
- bitmap_empty(vf->txq_ena, ICE_MAX_QS_PER_VF_VCV1);
+ return bitmap_empty(vf->rxq_ena, ICE_MAX_QS_PER_VF) &&
+ bitmap_empty(vf->txq_ena, ICE_MAX_QS_PER_VF);
}
/**
diff --git a/drivers/net/ethernet/intel/ice/virt/queues.c b/drivers/net/ethernet/intel/ice/virt/queues.c
index 2667634e3c8c..7f2b459dbc83 100644
--- a/drivers/net/ethernet/intel/ice/virt/queues.c
+++ b/drivers/net/ethernet/intel/ice/virt/queues.c
@@ -681,7 +681,7 @@ int ice_vc_cfg_q_bw(struct ice_vf *vf, u8 *msg)
goto err;
}
- if (qbw->num_queues > ICE_MAX_QS_PER_VF_VCV1 ||
+ if (qbw->num_queues > ICE_MAX_QS_PER_VF ||
qbw->num_queues > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
dev_err(ice_pf_to_dev(vf->pf), "VF-%d trying to configure more than allocated number of queues: %d\n",
vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
@@ -773,7 +773,7 @@ int ice_vc_cfg_q_quanta(struct ice_vf *vf, u8 *msg)
goto err;
}
- if (end_qid > ICE_MAX_QS_PER_VF_VCV1 ||
+ if (end_qid > ICE_MAX_QS_PER_VF ||
end_qid > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
dev_err(ice_pf_to_dev(vf->pf), "VF-%d trying to configure more than allocated number of queues: %d\n",
vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
@@ -822,9 +822,9 @@ int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
struct virtchnl_vsi_queue_config_info *qci =
(struct virtchnl_vsi_queue_config_info *)msg;
struct virtchnl_queue_pair_info *qpi;
+ int i, q_idx, last_valid = -1;
struct ice_pf *pf = vf->pf;
struct ice_vsi *vsi;
- int i = -1, q_idx;
bool ena_ts;
u8 act_prt;
@@ -841,7 +841,7 @@ int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
if (!vsi)
goto error_param;
- if (qci->num_queue_pairs > ICE_MAX_QS_PER_VF_VCV1 ||
+ if (qci->num_queue_pairs > ICE_MAX_QS_PER_VF ||
qci->num_queue_pairs > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
dev_err(ice_pf_to_dev(pf), "VF-%d requesting more than supported number of queues: %d\n",
vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
@@ -877,6 +877,7 @@ int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
if (q_idx >= vsi->alloc_txq || q_idx >= vsi->alloc_rxq) {
goto error_param;
}
+ last_valid = i;
/* copy Tx queue info from VF into VSI */
if (qpi->txq.ring_len > 0) {
@@ -967,13 +968,14 @@ int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
VIRTCHNL_STATUS_SUCCESS, NULL, 0);
error_param:
/* disable whatever we can */
- for (; i >= 0; i--) {
- if (ice_vsi_ctrl_one_rx_ring(vsi, false, i, true))
+ for (; last_valid >= 0; last_valid--) {
+ q_idx = qci->qpair[last_valid].rxq.queue_id;
+ if (ice_vsi_ctrl_one_rx_ring(vsi, false, q_idx, true))
dev_err(ice_pf_to_dev(pf), "VF-%d could not disable RX queue %d\n",
- vf->vf_id, i);
- if (ice_vf_vsi_dis_single_txq(vf, vsi, i))
+ vf->vf_id, q_idx);
+ if (ice_vf_vsi_dis_single_txq(vf, vsi, q_idx))
dev_err(ice_pf_to_dev(pf), "VF-%d could not disable TX queue %d\n",
- vf->vf_id, i);
+ vf->vf_id, q_idx);
}
ice_lag_complete_vf_reset(pf->lag, act_prt);
@@ -1019,16 +1021,16 @@ int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg)
if (!req_queues) {
dev_err(dev, "VF %d tried to request 0 queues. Ignoring.\n",
vf->vf_id);
- } else if (req_queues > ICE_MAX_QS_PER_VF_VCV1) {
+ } else if (req_queues > ICE_MAX_QS_PER_VF) {
dev_err(dev, "VF %d tried to request more than %d queues.\n",
- vf->vf_id, ICE_MAX_QS_PER_VF_VCV1);
- vfres->num_queue_pairs = ICE_MAX_QS_PER_VF_VCV1;
+ vf->vf_id, ICE_MAX_QS_PER_VF);
+ vfres->num_queue_pairs = ICE_MAX_QS_PER_VF;
} else if (req_queues > cur_queues &&
req_queues - cur_queues > tx_rx_queue_left) {
dev_warn(dev, "VF %d requested %u more queues, but only %u left.\n",
vf->vf_id, req_queues - cur_queues, tx_rx_queue_left);
vfres->num_queue_pairs = min_t(u16, max_allowed_vf_queues,
- ICE_MAX_QS_PER_VF_VCV1);
+ ICE_MAX_QS_PER_VF);
} else {
/* request is successful, then reset VF */
vf->num_req_qs = req_queues;
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 07/15] iavf: extend iavf_configure_queues() to support more queues
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (5 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 06/15] ice: bump to 256qs for VF Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 08/15] iavf: temporary rename of IAVF_MAX_REQ_QUEUES to IAVF_MAX_REQ_QUEUES_VCV1 Przemek Kitszel
` (7 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Extend iavf_configure_queues() to support more queue pairs than fit
into a single virtchnl message (62).
Virtchnl opcode used was already generic, but we have just not needed
more than one message before.
Send one message per call, and keep IAVF_FLAG_AQ_CONFIGURE_QUEUES set
until all queue pairs are sent, similar to what iavf_add_ether_addrs()
does for MAC filters. The next message goes out on a later watchdog
run, once the reply to the previous one has cleared current_op, as for
any other op. Stop sending on a PF NACK.
Add helper, iavf_max_vc_entries(), that determines how many entries we
could fit into virtchnl message (ending by flex array member). Will be
also used on another op later in this series.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v4:
* send one message per watchdog run, instead of polling for the reply
to each one with a 1s timeout, which could resend the whole config
after a late ACK or NACK, and could take a stale reply for the
awaited one (Sashiko)
* stop sending the rest on a PF NACK, in iavf_virtchnl_completion()
v2:
* fix typo (Sashiko)
* abort iavf_configure_queues() at error (Sashiko)
* report send and poll failures with dev_err(), they are not
recoverable here (me)
* cast via uintptr_t in iavf_match_vc_op_cb(), for clang W=1
-Wpointer-to-enum-cast
* commit message: the limit is the number of queue pairs that fit
into a single virtchnl message (62), not 31
* commit message: describe the switch to a synchronous op (Sashiko)
* stop at PF NACK (Sashiko)
* on timeout keep current_op and IAVF_FLAG_AQ_CONFIGURE_QUEUES set
until the late reply arrives, instead of resending right away;
clear both on other errors (no retry) (Sashiko)
* fix dev_err() continuation alignment (Sashiko)
---
drivers/net/ethernet/intel/iavf/iavf.h | 2 ++
drivers/net/ethernet/intel/iavf/iavf_main.c | 1 +
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 25 ++++++++++++++-----
3 files changed, 22 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index 8c45536fd502..a41d2726ebd8 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -269,6 +269,8 @@ struct iavf_adapter {
u8 rxdid;
int num_active_queues;
int num_req_queues;
+ /* queue pairs sent so far, for ops that take more than one message */
+ int cfg_qps_sent;
/* TX */
struct iavf_ring *tx_rings;
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 2f8a80a11336..d39737dcd425 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -1258,6 +1258,7 @@ static void iavf_configure(struct iavf_adapter *adapter)
iavf_configure_tx(adapter);
iavf_configure_rx(adapter);
+ adapter->cfg_qps_sent = 0;
adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES;
for (i = 0; i < adapter->num_active_queues; i++) {
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index e6b7e8f82c7c..356565aa41c8 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -8,6 +8,11 @@
#include "iavf_ptp.h"
#include "iavf_prototype.h"
+/* how many Flex Array Member entries do fit into VC message of type *ptr */
+#define iavf_max_vc_entries(ptr, flex_member) \
+ ((IAVF_MAX_AQ_BUF_SIZE - virtchnl_struct_size(ptr, flex_member, 1)) / \
+ sizeof(ptr->flex_member[0]) + 1)
+
/**
* iavf_send_pf_msg
* @adapter: adapter structure
@@ -375,11 +380,12 @@ int iavf_get_vf_ptp_caps(struct iavf_adapter *adapter)
void iavf_configure_queues(struct iavf_adapter *adapter)
{
struct virtchnl_vsi_queue_config_info *vqci;
- int pairs = adapter->num_active_queues;
struct virtchnl_queue_pair_info *vqpi;
+ int first = adapter->cfg_qps_sent;
u32 i, max_frame;
u8 rx_flags = 0;
size_t len;
+ int pairs;
max_frame = LIBIE_MAX_RX_FRM_LEN(adapter->rx_rings->pp->p.offset);
max_frame = min_not_zero(adapter->vf_res->max_mtu, max_frame);
@@ -391,6 +397,8 @@ void iavf_configure_queues(struct iavf_adapter *adapter)
return;
}
adapter->current_op = VIRTCHNL_OP_CONFIG_VSI_QUEUES;
+ pairs = min_t(int, adapter->num_active_queues - first,
+ iavf_max_vc_entries(vqci, qpair));
len = virtchnl_struct_size(vqci, qpair, pairs);
vqci = kzalloc(len, GFP_KERNEL);
if (!vqci)
@@ -402,10 +410,7 @@ void iavf_configure_queues(struct iavf_adapter *adapter)
vqci->vsi_id = adapter->vsi_res->vsi_id;
vqci->num_queue_pairs = pairs;
vqpi = vqci->qpair;
- /* Size check is not needed here - HW max is 16 queue pairs, and we
- * can fit info for 31 of them into the AQ buffer before it overflows.
- */
- for (i = 0; i < pairs; i++) {
+ for (i = first; i < first + pairs; i++) {
vqpi->txq.vsi_id = vqci->vsi_id;
vqpi->txq.queue_id = i;
vqpi->txq.ring_len = adapter->tx_rings[i].count;
@@ -425,7 +430,9 @@ void iavf_configure_queues(struct iavf_adapter *adapter)
vqpi++;
}
- adapter->aq_required &= ~IAVF_FLAG_AQ_CONFIGURE_QUEUES;
+ adapter->cfg_qps_sent += pairs;
+ if (adapter->cfg_qps_sent == adapter->num_active_queues)
+ adapter->aq_required &= ~IAVF_FLAG_AQ_CONFIGURE_QUEUES;
iavf_send_pf_msg(adapter, VIRTCHNL_OP_CONFIG_VSI_QUEUES,
(u8 *)vqci, len);
kfree(vqci);
@@ -2551,6 +2558,12 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
dev_warn(&adapter->pdev->dev, "Failed to Config Queue BW, error %s\n",
iavf_stat_str(&adapter->hw, v_retval));
break;
+ case VIRTCHNL_OP_CONFIG_VSI_QUEUES:
+ dev_err(&adapter->pdev->dev, "Failed to configure queues, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ /* don't send the remaining queue pairs */
+ adapter->aq_required &= ~IAVF_FLAG_AQ_CONFIGURE_QUEUES;
+ break;
default:
dev_err(&adapter->pdev->dev, "PF returned error %d (%s) to our request %d\n",
v_retval, iavf_stat_str(&adapter->hw, v_retval),
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 08/15] iavf: temporary rename of IAVF_MAX_REQ_QUEUES to IAVF_MAX_REQ_QUEUES_VCV1
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (6 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 07/15] iavf: extend iavf_configure_queues() to support more queues Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 09/15] iavf: increase max number of queues to 256 Przemek Kitszel
` (6 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Rename IAVF_MAX_REQ_QUEUES to IAVF_MAX_REQ_QUEUES_VCV1, in preparation for
the next patch that will extend the max to 256, using old value of 16 for
the "v1" variant of virtchnl opcodes.
Suggested-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
Suggested-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
drivers/net/ethernet/intel/iavf/iavf.h | 3 ++-
drivers/net/ethernet/intel/iavf/iavf_main.c | 2 +-
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c | 10 +++++-----
3 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index a41d2726ebd8..053f34c4f710 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -87,7 +87,8 @@ struct iavf_vsi {
#define IAVF_TX_DESC(R, i) (&(((struct iavf_tx_desc *)((R)->desc))[i]))
#define IAVF_TX_CTXTDESC(R, i) \
(&(((struct iavf_tx_context_desc *)((R)->desc))[i]))
-#define IAVF_MAX_REQ_QUEUES 16
+/* for "old" virtchnl opcodes that accept up to 16 queues */
+#define IAVF_MAX_REQ_QUEUES_VCV1 16
#define IAVF_HKEY_ARRAY_SIZE ((IAVF_VFQF_HKEY_MAX_INDEX + 1) * 4)
#define IAVF_HLUT_ARRAY_SIZE ((IAVF_VFQF_HLUT_MAX_INDEX + 1) * 4)
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index d39737dcd425..822d6e692d49 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -5371,7 +5371,7 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
pci_set_master(pdev);
netdev = alloc_etherdev_mq(sizeof(struct iavf_adapter),
- IAVF_MAX_REQ_QUEUES);
+ IAVF_MAX_REQ_QUEUES_VCV1);
if (!netdev) {
err = -ENOMEM;
goto err_alloc_etherdev;
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index 356565aa41c8..69241cb146e0 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -260,19 +260,19 @@ int iavf_send_vf_ptp_caps_msg(struct iavf_adapter *adapter)
**/
static void iavf_validate_num_queues(struct iavf_adapter *adapter)
{
- if (adapter->vf_res->num_queue_pairs > IAVF_MAX_REQ_QUEUES) {
+ if (adapter->vf_res->num_queue_pairs > IAVF_MAX_REQ_QUEUES_VCV1) {
struct virtchnl_vsi_resource *vsi_res;
int i;
dev_info(&adapter->pdev->dev, "Received %d queues, but can only have a max of %d\n",
adapter->vf_res->num_queue_pairs,
- IAVF_MAX_REQ_QUEUES);
+ IAVF_MAX_REQ_QUEUES_VCV1);
dev_info(&adapter->pdev->dev, "Fixing by reducing queues to %d\n",
- IAVF_MAX_REQ_QUEUES);
- adapter->vf_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES;
+ IAVF_MAX_REQ_QUEUES_VCV1);
+ adapter->vf_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES_VCV1;
for (i = 0; i < adapter->vf_res->num_vsis; i++) {
vsi_res = &adapter->vf_res->vsi_res[i];
- vsi_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES;
+ vsi_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES_VCV1;
}
}
}
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 09/15] iavf: increase max number of queues to 256
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (7 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 08/15] iavf: temporary rename of IAVF_MAX_REQ_QUEUES to IAVF_MAX_REQ_QUEUES_VCV1 Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:44 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 10/15] iavf: use new opcodes to request more than 16 queues Przemek Kitszel
` (5 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Increase the max number of queues that driver will handle to 256.
Use old legacy limit in the virtchnl handling of iavf_map_queues().
Limit the number of queue vectors to 64, as VF interrupt registers
cover no more.
The netdev is allocated at probe, before the PF tells the VF how many
queues it may use, so it now always has 256 Tx and Rx queues. Whatever
is sized by num_tx_queues grows 16 times: the default qdisc of an up VF
has a child per Tx queue (about 6 MB with pfifo_fast, 17 MB with
fq_codel), and ethtool -S lists 256 queue pairs, unused ones as zeros.
Reviewed-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v5:
* cap the legacy VIRTCHNL_OP_{EN,DIS}ABLE_QUEUES bitmaps at 16 queues,
as done for the IRQ map: PF rejects a wider mask, and BIT() above 63
queues is undefined (me)
v4:
* limit MSI-X budget to 64 queue vectors, VFINT_ITRN_64 and
VFINT_DYN_CTLN1 have no room for more (Sashiko)
* change ADQ total_qps to u16, a sum of 256 queues wrapped to 0 in u8
(Sashiko)
* commit message: mention the cost of the netdev sized for 256 queues
(Sashiko)
* program ITR of vectors 16..63 via VFINT_ITRN_64, VFINT_ITRN1 has room
for 16 vectors only (Sashiko)
* revert dropping of "_i=0...15" from IAVF_Q{R,T}X_TAIL1() comments
v2:
* change iavf_q_vector::num_ringpairs to u16 (Sashiko)
* drop the stale "_i=0...15" range from IAVF_Q{R,T}X_TAIL1()
---
drivers/net/ethernet/intel/iavf/iavf.h | 10 +++++---
.../net/ethernet/intel/iavf/iavf_register.h | 2 ++
drivers/net/ethernet/intel/iavf/iavf_main.c | 24 +++++++++++++------
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 21 +++++++++-------
4 files changed, 38 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index 053f34c4f710..60ee3a195567 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -87,8 +87,10 @@ struct iavf_vsi {
#define IAVF_TX_DESC(R, i) (&(((struct iavf_tx_desc *)((R)->desc))[i]))
#define IAVF_TX_CTXTDESC(R, i) \
(&(((struct iavf_tx_context_desc *)((R)->desc))[i]))
+
/* for "old" virtchnl opcodes that accept up to 16 queues */
#define IAVF_MAX_REQ_QUEUES_VCV1 16
+#define IAVF_MAX_REQ_QUEUES 256
#define IAVF_HKEY_ARRAY_SIZE ((IAVF_VFQF_HKEY_MAX_INDEX + 1) * 4)
#define IAVF_HLUT_ARRAY_SIZE ((IAVF_VFQF_HLUT_MAX_INDEX + 1) * 4)
@@ -108,9 +110,9 @@ struct iavf_q_vector {
struct napi_struct napi;
struct iavf_ring_container rx;
struct iavf_ring_container tx;
- u32 ring_mask;
+ DECLARE_BITMAP(ring_mask, IAVF_MAX_REQ_QUEUES);
u8 itr_countdown; /* when 0 should adjust adaptive ITR */
- u8 num_ringpairs; /* total number of ring pairs in vector */
+ u16 num_ringpairs; /* total number of ring pairs in vector */
u16 v_idx; /* index in the vsi->q_vector array. */
u16 reg_idx; /* register index of the interrupt */
char name[IFNAMSIZ + 15];
@@ -134,6 +136,8 @@ struct iavf_q_vector {
#define MIN_MSIX_Q_VECTORS 1
#define MIN_MSIX_COUNT (MIN_MSIX_Q_VECTORS + NONQ_VECS)
+/* VF interrupt registers cover 64 queue vectors */
+#define MAX_MSIX_Q_VECTORS 64
#define IAVF_QUEUE_END_OF_LIST 0x7FF
#define IAVF_FREE_VECTOR 0x7FFF
@@ -182,7 +186,7 @@ enum iavf_tc_state_t {
struct iavf_channel_config {
struct virtchnl_channel_info ch_info[IAVF_MAX_TRAFFIC_CLASS];
enum iavf_tc_state_t state;
- u8 total_qps;
+ u16 total_qps;
};
/* State of cloud filter */
diff --git a/drivers/net/ethernet/intel/iavf/iavf_register.h b/drivers/net/ethernet/intel/iavf/iavf_register.h
index a19e88898a0b..47fd2d3f5e4b 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_register.h
+++ b/drivers/net/ethernet/intel/iavf/iavf_register.h
@@ -56,6 +56,8 @@
#define IAVF_VFINT_ICR0_ENA1_RSVD_SHIFT 31
#define IAVF_VFINT_ICR01 0x00004800 /* Reset: CORER */
#define IAVF_VFINT_ITRN1(_i, _INTVF) (0x00002800 + ((_i) * 64 + (_INTVF) * 4)) /* _i=0...2, _INTVF=0...15 */ /* Reset: VFR */
+#define IAVF_VFINT_ITRN_64(_i, _INTVF) \
+ (0x00002C00 + ((_i) * 256 + (_INTVF) * 4)) /* _i=0...2, _INTVF=0...63 */
#define IAVF_QRX_TAIL1(_Q) (0x00002000 + ((_Q) * 4)) /* _i=0...15 */ /* Reset: CORER */
#define IAVF_QTX_TAIL1(_Q) (0x00000000 + ((_Q) * 4)) /* _i=0...15 */ /* Reset: PFR */
#define IAVF_VFQF_HENA(_i) (0x0000C400 + ((_i) * 4)) /* _i=0...1 */ /* Reset: CORER */
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 822d6e692d49..db769eead68f 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -419,6 +419,15 @@ static irqreturn_t iavf_msix_clean_rings(int irq, void *data)
return IRQ_HANDLED;
}
+/* VFINT_ITRN1 fits 16 vectors only, VFINT_ITRN_64 aliases it for up to 64 */
+static u32 iavf_itrn_reg(enum iavf_dyn_idx_t itr_idx, u16 reg_idx)
+{
+ if (reg_idx < 16)
+ return IAVF_VFINT_ITRN1(itr_idx, reg_idx);
+
+ return IAVF_VFINT_ITRN_64(itr_idx, reg_idx);
+}
+
/**
* iavf_map_vector_to_rxq - associate irqs with rx queues
* @adapter: board private structure
@@ -439,8 +448,8 @@ iavf_map_vector_to_rxq(struct iavf_adapter *adapter, int v_idx, int r_idx)
q_vector->rx.count++;
q_vector->rx.next_update = jiffies + 1;
q_vector->rx.target_itr = ITR_TO_REG(rx_ring->itr_setting);
- q_vector->ring_mask |= BIT(r_idx);
- wr32(hw, IAVF_VFINT_ITRN1(IAVF_RX_ITR, q_vector->reg_idx),
+ set_bit(r_idx, q_vector->ring_mask);
+ wr32(hw, iavf_itrn_reg(IAVF_RX_ITR, q_vector->reg_idx),
q_vector->rx.current_itr >> 1);
q_vector->rx.current_itr = q_vector->rx.target_itr;
}
@@ -466,7 +475,7 @@ iavf_map_vector_to_txq(struct iavf_adapter *adapter, int v_idx, int t_idx)
q_vector->tx.next_update = jiffies + 1;
q_vector->tx.target_itr = ITR_TO_REG(tx_ring->itr_setting);
q_vector->num_ringpairs++;
- wr32(hw, IAVF_VFINT_ITRN1(IAVF_TX_ITR, q_vector->reg_idx),
+ wr32(hw, iavf_itrn_reg(IAVF_TX_ITR, q_vector->reg_idx),
q_vector->tx.target_itr >> 1);
q_vector->tx.current_itr = q_vector->tx.target_itr;
}
@@ -1661,9 +1670,9 @@ static int iavf_set_interrupt_capability(struct iavf_adapter *adapter)
/* It's easy to be greedy for MSI-X vectors, but it really doesn't do
* us much good if we have more vectors than CPUs. However, we already
* limit the total number of queues by the number of CPUs so we do not
- * need any further limiting here.
+ * need any further limiting here, except for the VF registers.
*/
- v_budget = min_t(int, pairs + NONQ_VECS,
+ v_budget = min_t(int, min(pairs, MAX_MSIX_Q_VECTORS) + NONQ_VECS,
(int)adapter->vf_res->max_vectors);
adapter->msix_entries = kzalloc_objs(struct msix_entry, v_budget);
@@ -3734,9 +3743,10 @@ static int __iavf_setup_tc(struct net_device *netdev, void *type_data)
struct tc_mqprio_qopt_offload *mqprio_qopt = type_data;
struct iavf_adapter *adapter = netdev_priv(netdev);
struct virtchnl_vf_resource *vfres = adapter->vf_res;
- u8 num_tc = 0, total_qps = 0;
int ret = 0, netdev_tc = 0;
+ u16 total_qps = 0;
u64 max_tx_rate;
+ u8 num_tc = 0;
u16 mode;
int i;
@@ -5371,7 +5381,7 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
pci_set_master(pdev);
netdev = alloc_etherdev_mq(sizeof(struct iavf_adapter),
- IAVF_MAX_REQ_QUEUES_VCV1);
+ IAVF_MAX_REQ_QUEUES);
if (!netdev) {
err = -ENOMEM;
goto err_alloc_etherdev;
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index 69241cb146e0..1296ea0f1716 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -260,19 +260,19 @@ int iavf_send_vf_ptp_caps_msg(struct iavf_adapter *adapter)
**/
static void iavf_validate_num_queues(struct iavf_adapter *adapter)
{
- if (adapter->vf_res->num_queue_pairs > IAVF_MAX_REQ_QUEUES_VCV1) {
+ if (adapter->vf_res->num_queue_pairs > IAVF_MAX_REQ_QUEUES) {
struct virtchnl_vsi_resource *vsi_res;
int i;
dev_info(&adapter->pdev->dev, "Received %d queues, but can only have a max of %d\n",
adapter->vf_res->num_queue_pairs,
- IAVF_MAX_REQ_QUEUES_VCV1);
+ IAVF_MAX_REQ_QUEUES);
dev_info(&adapter->pdev->dev, "Fixing by reducing queues to %d\n",
- IAVF_MAX_REQ_QUEUES_VCV1);
- adapter->vf_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES_VCV1;
+ IAVF_MAX_REQ_QUEUES);
+ adapter->vf_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES;
for (i = 0; i < adapter->vf_res->num_vsis; i++) {
vsi_res = &adapter->vf_res->vsi_res[i];
- vsi_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES_VCV1;
+ vsi_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES;
}
}
}
@@ -456,7 +456,8 @@ void iavf_enable_queues(struct iavf_adapter *adapter)
}
adapter->current_op = VIRTCHNL_OP_ENABLE_QUEUES;
vqs.vsi_id = adapter->vsi_res->vsi_id;
- vqs.tx_queues = BIT(adapter->num_active_queues) - 1;
+ vqs.tx_queues = BIT(min(adapter->num_active_queues,
+ IAVF_MAX_REQ_QUEUES_VCV1)) - 1;
vqs.rx_queues = vqs.tx_queues;
adapter->aq_required &= ~IAVF_FLAG_AQ_ENABLE_QUEUES;
iavf_send_pf_msg(adapter, VIRTCHNL_OP_ENABLE_QUEUES,
@@ -481,7 +482,8 @@ void iavf_disable_queues(struct iavf_adapter *adapter)
}
adapter->current_op = VIRTCHNL_OP_DISABLE_QUEUES;
vqs.vsi_id = adapter->vsi_res->vsi_id;
- vqs.tx_queues = BIT(adapter->num_active_queues) - 1;
+ vqs.tx_queues = BIT(min(adapter->num_active_queues,
+ IAVF_MAX_REQ_QUEUES_VCV1)) - 1;
vqs.rx_queues = vqs.tx_queues;
adapter->aq_required &= ~IAVF_FLAG_AQ_DISABLE_QUEUES;
iavf_send_pf_msg(adapter, VIRTCHNL_OP_DISABLE_QUEUES,
@@ -526,8 +528,9 @@ void iavf_map_queues(struct iavf_adapter *adapter)
vecmap->vsi_id = adapter->vsi_res->vsi_id;
vecmap->vector_id = v_idx + NONQ_VECS;
- vecmap->txq_map = q_vector->ring_mask;
- vecmap->rxq_map = q_vector->ring_mask;
+ vecmap->txq_map = bitmap_read(q_vector->ring_mask, 0,
+ IAVF_MAX_REQ_QUEUES_VCV1);
+ vecmap->rxq_map = vecmap->txq_map;
vecmap->rxitr_idx = IAVF_RX_ITR;
vecmap->txitr_idx = IAVF_TX_ITR;
}
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 10/15] iavf: use new opcodes to request more than 16 queues
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (8 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 09/15] iavf: increase max number of queues to 256 Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 11/15] ice: introduce handling of virtchnl LARGE VF opcodes Przemek Kitszel
` (4 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
From: Ahmed Zaki <ahmed.zaki@intel.com>
Legacy virtchnl opcodes select queues by fixed width bitmaps: u32 for
VIRTCHNL_OP_ENABLE_QUEUES and VIRTCHNL_OP_DISABLE_QUEUES, u16 for
VIRTCHNL_OP_CONFIG_IRQ_MAP. Define new OP codes in order to enable
the VF to use a variable number of queues.
First the new capability VIRTCHNL_VF_LARGE_NUM_QPAIRS is defined. Only
if this cap is negotiated, the VF is allowed to use the new OP codes.
The VIRTCHNL_OP_GET_MAX_RSS_QREGION is exchanged right after the VF
resource message is received. This is done similar to the PTP and other
extended cap exchanges.
Enabling and disabling queues, and mapping queues to vectors is done
through the new VIRTCHNL_OP_ENABLE_QUEUES_V2,
VIRTCHNL_OP_DISABLE_QUEUES_V2, and VIRTCHNL_OP_MAP_QUEUE_VECTOR messages.
The queue to vector map may not fit into one message, so it is sent in
parts, one per watchdog run, as done for VIRTCHNL_OP_CONFIG_VSI_QUEUES.
Use the new iavf_request_queues() for ethtool -L.
It's not only for elegance, but it enables our brittle handcoded state
machine to go through requesting the queues before PF driver reallocates
queue-related arrays. This is done only with VIRTCHNL_VF_LARGE_NUM_QPAIRS,
and ethtool -L returns right after queuing the request, the watchdog
sends it once no other op is in flight: PF resets the VF to grant it,
a denial is only logged.
Two changes in iavf_parse_vf_resource_msg() apply to every PF: when the
PF reports fewer queue pairs than the VF has in use (PF has lowered its
limit on reset), reinit queues and MSI-X with that count; when the RSS
key or LUT size changes across a reset, reallocate both and fill them
again. Every PF is also affected by resetting aq_wait_count when init
starts over after the PF stopped responding. It was never reset, so
once it reached IAVF_AQ_MAX_ERR, the first wait for a PF reply sent
init back to __IAVF_COMM_FAILED.
Signed-off-by: Ahmed Zaki <ahmed.zaki@intel.com>
Co-developed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v6:
* init: do not refill RSS LUT in iavf_parse_vf_resource_msg() when the
PF reports new RSS key/LUT sizes on reinit after iavf_disable_vf(),
there are no queues yet (divide by zero in iavf_fill_rss_lut()), and
iavf_init_config_adapter() sets RSS up later anyway (Petr, Sashiko)
* ethtool -L: report the requested count until the watchdog sends the
queue request, and drop the request on ethtool -L back to the active
count; ethtool core skipped that as no change, and the stale request
went out (Sashiko)
* ethtool -L: request queues from PF only if
VIRTCHNL_VF_OFFLOAD_REQ_QUEUES is negotiated too, as done at init;
without it set channels locally, up to num_queue_pairs (Sashiko)
* init: zero adapter->vf_res on a failed VF resources message, as
iavf_disable_vf() does, instead of freeing it and clearing
adapter->vsi_res; ethtool ops use both while init retries with the
netdev registered; find the LAN VSI with a local variable instead,
so a retry does not keep an old adapter->vsi_res (Sashiko)
v5:
* ethtool -L: validate rx, tx and other counts before returning early
for an unchanged combined count; drop the cancel of a not yet sent
queue request there, ethtool core doesn't call us without a change
(Sashiko)
* ethtool -L: keep the queue request pending until it is sent, it was
dropped when sending failed, or on a VF reset that came first (me)
* init: clear adapter->vsi_res when freeing adapter->vf_res that it
points into, a retry would keep using it if the new VF resources
message had no LAN VSI (Sashiko)
* skip the rest of VIRTCHNL_OP_GET_VF_RESOURCES reply handling when
iavf_parse_vf_resource_msg() fails, instead of configuring with
stale queue counts; the reset it has scheduled will redo it (Sashiko)
* report the RX ring count limited to the PF RSS region, as
iavf_fill_rss_lut() does, the default LUT made by ethtool core for
ethtool -X was rejected by iavf_set_rxfh() (me)
v4:
* iavf_map_queue_vector(): send one message per watchdog run, as
iavf_configure_queues() does now, instead of polling for each reply
with a 1s timeout (Sashiko)
* stop sending the queue to vector map on a PF NACK, reported by
iavf_virtchnl_completion()
* refill the RSS LUT when the RSS region changes across a reset, as the
LUT is filled before the VIRTCHNL_OP_GET_MAX_RSS_QREGION reply
(Sashiko)
* kzalloc() the RSS key and LUT on reallocation, as done at init
(Sashiko)
* ethtool -L: let the watchdog send the queue request, instead of
failing with -EBUSY whenever another op is in flight (e.g. GET_STATS);
return -EBUSY only while PF has not answered the previous request
(Sashiko)
* init: check IAVF_VF_ARQLEN1 while waiting for the queue request
outcome, as the event of the PF reset that grants it can be lost;
start over on an error reply, instead of taking the request for
granted after the next reset (Sashiko)
* forget the queue request when init starts over after the PF stopped
responding, and reset aq_wait_count there, so init gets a full retry
budget again (Sashiko)
* treat an error reply to a runtime queue request as a denial; on a VF
reset with the request not answered yet, use the smaller of the
active and requested queue counts, as the reset can drop the PF
answer, and that count fits whatever PF has decided (Sashiko)
* ethtool -X: reject LUT entries outside of the PF RSS region, as
iavf_fill_rss_lut() does; comment that qregion_width 0 means the
region is not known yet (Sashiko)
* commit message: describe the iavf_parse_vf_resource_msg() changes
that apply to every PF, the aq_wait_count reset, and the async
ethtool -L (Sashiko)
* drop num_req_queues on a reset that does not reinit queues, as the
previous reset has set them up already; when PF lowers the limit
right after granting the request (VF RSS LUT taken away just after
ethtool -L), reinit quietly instead of reporting "Requested 64
queues, but PF only gave us 16."
* wait for the outcome of the queue request sent at init before other
extended caps, as their polling dropped a PF denial and VF went on
with the denied queue count; on denial ask for as many queues as PF
can give, and allocate the granted count after the PF reset (Sashiko)
* num_queue_pairs is the VF allocation also with
VIRTCHNL_VF_LARGE_NUM_QPAIRS, as in legacy mode and with the
out-of-tree drivers; ethtool -L max is the PF RSS region (Sashiko)
* init: ask for more queues, up to one per CPU, and take the request as
granted only when PF gives us that many, so an unrelated reset before
the answer is not taken for a grant; on a denial that offers no more
than we have, or on an error reply, go on with what we have instead
of starting over (Sashiko)
* drop the cap of the requested queue count on a VF reset, the
"Requested X queues, but PF only gave us Y" check handles a lost
denial again (Sashiko)
* clarify the qregion_width comment in iavf_fill_rss_lut()
v3:
* auto-request queues in LARGE mode only at init, not on every reset,
so ethtool -L setting is kept; do that after VSI and RSS fields are
initialized, instead of returning early (Sashiko)
* schedule a VF reset when RSS key/LUT reallocation fails, as the
caller on the reset path ignores the error (Sashiko)
v2:
* use proper IAVF_EXTENDED_CAP_SEND_RSS_QREGION flag when handling
iavf_init_recv_max_rss_qregion() error (Sashiko)
* update adapter->rss_{key,lut}_size and adapter->current_op only after
successful allocation (Sashiko)
* use the v2 opcodes to enable queues whenever
VIRTCHNL_VF_LARGE_NUM_QPAIRS is negotiated, as for disabling (it was
based on the queue count) (Sashiko)
* set adapter->current_op in iavf_send_max_rss_qregion() (Sashiko)
* iavf_set_channels(): stop clearing adapter->current_op before
iavf_request_queues(), it was a debugging leftover (Sashiko)
* iavf_request_queues(): undo current_op, num_req_queues and
IAVF_FLAG_REINIT_ITR_NEEDED when sending the request fails (Sashiko)
* iavf_map_queue_vector(): stop at the first send or poll error, and
report it with dev_err()
* replace rss key/lut krealloc() with simple kmalloc()s, to have an
easier job on the error path
* request RSS qregion again after reset, together with other extended
caps: send it from iavf_process_aq_command() and handle the reply in
iavf_virtchnl_completion()
* auto-request queues at init only once, and only with
VIRTCHNL_VF_LARGE_NUM_QPAIRS (and VIRTCHNL_VF_OFFLOAD_REQ_QUEUES),
where num_queue_pairs is the PF limit rather than the allocation;
without it keep the previous behavior, with no extra VF reset
* set channels locally (as before this series) unless
VIRTCHNL_VF_LARGE_NUM_QPAIRS, so ethtool -L down no longer lowers
max_combined for good
* reinit queues when the PF lowers num_queue_pairs below the number of
active queues on reset (e.g. VF RSS LUT switched from GLOBAL back to
VSI, so LARGE_NUM_QPAIRS is gone), instead of using legacy opcodes
with too many queues
* rebase over iavf_poll_virtchnl_response() changes
* commit message: explain the limits of the legacy opcodes
* iavf_map_queue_vector(): stop at PF NACK; on timeout keep current_op
and IAVF_FLAG_AQ_MAP_VECTORS set until the late reply arrives, clear
both on other errors (no retry), as in iavf_configure_queues()
(Sashiko)
Sashiko suggestions I don't want to apply:
S: iavf_fill_rss_lut() will crash if qregion->qregion_width >= 32
me: value is coming from PF, which must be trusted
S: non-atomic adapter->flags update in iavf_request_queues() races with
lockless contexts like iavf_tx_timeout()
me: pre-existing pattern in the whole iavf driver (iavf_set_channels()
did the same before this series), out of scope here
---
drivers/net/ethernet/intel/iavf/iavf.h | 20 ++
include/linux/net/intel/virtchnl.h | 129 ++++++++-
.../net/ethernet/intel/iavf/iavf_ethtool.c | 58 +++-
drivers/net/ethernet/intel/iavf/iavf_main.c | 249 +++++++++++++++--
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 255 +++++++++++++++++-
5 files changed, 676 insertions(+), 35 deletions(-)
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index 60ee3a195567..30a3759e5545 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -274,8 +274,11 @@ struct iavf_adapter {
u8 rxdid;
int num_active_queues;
int num_req_queues;
+ /* count for IAVF_FLAG_AQ_REQUEST_QUEUES */
+ int num_req_queues_next;
/* queue pairs sent so far, for ops that take more than one message */
int cfg_qps_sent;
+ int map_qps_sent;
/* TX */
struct iavf_ring *tx_rings;
@@ -349,13 +352,16 @@ struct iavf_adapter {
#define IAVF_FLAG_AQ_GET_SUPPORTED_RXDIDS BIT_ULL(42)
#define IAVF_FLAG_AQ_GET_PTP_CAPS BIT_ULL(43)
#define IAVF_FLAG_AQ_SEND_PTP_CMD BIT_ULL(44)
+#define IAVF_FLAG_AQ_GET_MAX_RSS_QREGION BIT_ULL(45)
+#define IAVF_FLAG_AQ_REQUEST_QUEUES BIT_ULL(46)
/* AQ messages that must be sent after IAVF_FLAG_AQ_GET_CONFIG, in
* order to negotiated extended capabilities.
*/
#define IAVF_FLAG_AQ_EXTENDED_CAPS \
(IAVF_FLAG_AQ_GET_OFFLOAD_VLAN_V2_CAPS | \
IAVF_FLAG_AQ_GET_SUPPORTED_RXDIDS | \
+ IAVF_FLAG_AQ_GET_MAX_RSS_QREGION | \
IAVF_FLAG_AQ_GET_PTP_CAPS)
/* flags for processing extended capability messages during
@@ -372,12 +378,18 @@ struct iavf_adapter {
#define IAVF_EXTENDED_CAP_RECV_RXDID BIT_ULL(3)
#define IAVF_EXTENDED_CAP_SEND_PTP BIT_ULL(4)
#define IAVF_EXTENDED_CAP_RECV_PTP BIT_ULL(5)
+#define IAVF_EXTENDED_CAP_SEND_RSS_QREGION BIT_ULL(6)
+#define IAVF_EXTENDED_CAP_RECV_RSS_QREGION BIT_ULL(7)
+/* set only when iavf_parse_vf_resource_msg() requests queues */
+#define IAVF_EXTENDED_CAP_RECV_REQ_QUEUES BIT_ULL(8)
#define IAVF_EXTENDED_CAPS \
(IAVF_EXTENDED_CAP_SEND_VLAN_V2 | \
IAVF_EXTENDED_CAP_RECV_VLAN_V2 | \
IAVF_EXTENDED_CAP_SEND_RXDID | \
IAVF_EXTENDED_CAP_RECV_RXDID | \
+ IAVF_EXTENDED_CAP_SEND_RSS_QREGION | \
+ IAVF_EXTENDED_CAP_RECV_RSS_QREGION | \
IAVF_EXTENDED_CAP_SEND_PTP | \
IAVF_EXTENDED_CAP_RECV_PTP)
@@ -417,6 +429,8 @@ struct iavf_adapter {
#define RSS_REG(_a) (!((_a)->vf_res->vf_cap_flags & \
(VIRTCHNL_VF_OFFLOAD_RSS_AQ | \
VIRTCHNL_VF_OFFLOAD_RSS_PF)))
+#define LARGE_NUM_QPAIRS_SUPPORT(_a) \
+ ((_a)->vf_res->vf_cap_flags & VIRTCHNL_VF_LARGE_NUM_QPAIRS)
#define VLAN_ALLOWED(_a) ((_a)->vf_res->vf_cap_flags & \
VIRTCHNL_VF_OFFLOAD_VLAN)
#define VLAN_V2_ALLOWED(_a) ((_a)->vf_res->vf_cap_flags & \
@@ -451,6 +465,7 @@ struct iavf_adapter {
struct virtchnl_vlan_caps vlan_v2_caps;
u64 supp_rxdids;
struct iavf_ptp ptp;
+ struct virtchnl_max_rss_qregion max_rss_qregion;
u16 msg_enable;
struct iavf_eth_stats current_stats;
struct virtchnl_qos_cap_list *qos_caps;
@@ -587,6 +602,9 @@ int iavf_send_vf_supported_rxdids_msg(struct iavf_adapter *adapter);
int iavf_get_vf_supported_rxdids(struct iavf_adapter *adapter);
int iavf_send_vf_ptp_caps_msg(struct iavf_adapter *adapter);
int iavf_get_vf_ptp_caps(struct iavf_adapter *adapter);
+int iavf_send_max_rss_qregion(struct iavf_adapter *adapter);
+int iavf_get_max_rss_qregion(struct iavf_adapter *adapter);
+int iavf_get_req_queues_reply(struct iavf_adapter *adapter);
void iavf_set_queue_vlan_tag_loc(struct iavf_adapter *adapter);
u16 iavf_get_num_vlans_added(struct iavf_adapter *adapter);
void iavf_irq_enable(struct iavf_adapter *adapter, bool flush);
@@ -601,6 +619,7 @@ void iavf_add_vlans(struct iavf_adapter *adapter);
void iavf_del_vlans(struct iavf_adapter *adapter);
void iavf_set_promiscuous(struct iavf_adapter *adapter);
bool iavf_promiscuous_mode_changed(struct iavf_adapter *adapter);
+int iavf_request_queues(struct iavf_adapter *adapter, int num);
void iavf_request_stats(struct iavf_adapter *adapter);
int iavf_request_reset(struct iavf_adapter *adapter);
void iavf_get_rss_hashcfg(struct iavf_adapter *adapter);
@@ -621,6 +640,7 @@ int iavf_poll_virtchnl_response(struct iavf_adapter *adapter,
const void *cond_data,
unsigned int timeout_ms);
int iavf_config_rss(struct iavf_adapter *adapter);
+void iavf_fill_rss_lut(struct iavf_adapter *adapter);
void iavf_cfg_queues_bw(struct iavf_adapter *adapter);
void iavf_cfg_queues_quanta_size(struct iavf_adapter *adapter);
void iavf_get_qos_caps(struct iavf_adapter *adapter);
diff --git a/include/linux/net/intel/virtchnl.h b/include/linux/net/intel/virtchnl.h
index 11bdab5522fd..90c5371c151b 100644
--- a/include/linux/net/intel/virtchnl.h
+++ b/include/linux/net/intel/virtchnl.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-only */
-/* Copyright (c) 2013-2022, Intel Corporation. */
+/* Copyright (c) 2013-2026, Intel Corporation. */
#ifndef _VIRTCHNL_H_
#define _VIRTCHNL_H_
@@ -22,8 +22,7 @@
* we must send all messages as "indirect", i.e. using an external buffer.
*
* All the VSI indexes are relative to the VF. Each VF can have maximum of
- * three VSIs. All the queue indexes are relative to the VSI. Each VF can
- * have a maximum of sixteen queues for all of its VSIs.
+ * three VSIs. All the queue indexes are relative to the VSI.
*
* The PF is required to return a status code in v_retval for all messages
* except RESET_VF, which does not require any response. The returned value
@@ -147,6 +146,7 @@ enum virtchnl_ops {
VIRTCHNL_OP_DEL_RSS_CFG = 46,
VIRTCHNL_OP_ADD_FDIR_FILTER = 47,
VIRTCHNL_OP_DEL_FDIR_FILTER = 48,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION = 50,
VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS = 51,
VIRTCHNL_OP_ADD_VLAN_V2 = 52,
VIRTCHNL_OP_DEL_VLAN_V2 = 53,
@@ -159,7 +159,11 @@ enum virtchnl_ops {
VIRTCHNL_OP_1588_PTP_GET_TIME = 61,
/* opcode 62 - 65 are reserved */
VIRTCHNL_OP_GET_QOS_CAPS = 66,
- /* opcode 68 through 111 are reserved */
+ /* opcodes 67 through 106 are reserved */
+ VIRTCHNL_OP_ENABLE_QUEUES_V2 = 107,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2 = 108,
+ /* opcodes 109 and 110 are reserved */
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR = 111,
VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
VIRTCHNL_OP_CONFIG_QUANTA = 113,
VIRTCHNL_OP_MAX,
@@ -257,6 +261,7 @@ VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
#define VIRTCHNL_VF_OFFLOAD_REQ_QUEUES BIT(6)
/* used to negotiate communicating link speeds in Mbps */
#define VIRTCHNL_VF_CAP_ADV_LINK_SPEED BIT(7)
+#define VIRTCHNL_VF_LARGE_NUM_QPAIRS BIT(9)
#define VIRTCHNL_VF_OFFLOAD_CRC BIT(10)
#define VIRTCHNL_VF_OFFLOAD_TC_U32 BIT(11)
#define VIRTCHNL_VF_OFFLOAD_VLAN_V2 BIT(15)
@@ -500,6 +505,27 @@ struct virtchnl_queue_select {
VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_select);
+/* VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ *
+ * if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES then this op must be supported.
+ *
+ * VF sends this message in order to query the max RSS queue region
+ * size supported by PF, when VIRTCHNL_VF_LARGE_NUM_QPAIRS is enabled.
+ * This information should be used when configuring the RSS LUT and/or
+ * configuring queue region based filters.
+ *
+ * The maximum RSS queue region is 2^qregion_width. So, a qregion_width of 6
+ * would inform the VF that the PF supports a maximum RSS queue region of 64.
+ */
+struct virtchnl_max_rss_qregion {
+ u16 vport_id;
+ u16 qregion_width;
+ u8 pad[4];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_max_rss_qregion);
+
/* VIRTCHNL_OP_ADD_ETH_ADDR
* VF sends this message in order to add one or more unicast or multicast
* address filters for the specified VSI.
@@ -1663,6 +1689,70 @@ struct virtchnl_queue_chunk {
VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+/* VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2
+ *
+ * These opcodes can be used if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated in
+ * VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends virtchnl_del_ena_dis_queues struct to specify the queues to be
+ * enabled/disabled in chunks. Also applicable to single queue RX or
+ * TX. PF performs requested action and returns status.
+ */
+struct virtchnl_del_ena_dis_queues {
+ u16 vport_id;
+ u16 pad;
+ u16 num_chunks;
+ u16 rsvd;
+ struct virtchnl_queue_chunk chunks[];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_del_ena_dis_queues);
+#define virtchnl_del_ena_dis_queues_LEGACY_SIZEOF 16
+
+/* Virtchannel interrupt throttling rate index */
+enum virtchnl_itr_idx {
+ VIRTCHNL_ITR_IDX_0 = 0,
+ VIRTCHNL_ITR_IDX_1 = 1,
+ VIRTCHNL_ITR_IDX_NO_ITR = 3,
+};
+
+/* Queue to vector mapping */
+struct virtchnl_queue_vector {
+ u16 queue_id;
+ u16 vector_id;
+ u8 pad[4];
+
+ /* see enum virtchnl_itr_idx */
+ s32 itr_idx;
+
+ /* see enum virtchnl_queue_type */
+ s32 queue_type;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_queue_vector);
+
+/* VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ *
+ * This opcode can be used only if VIRTCHNL_VF_LARGE_NUM_QPAIRS was negotiated
+ * in VIRTCHNL_OP_GET_VF_RESOURCES
+ *
+ * VF sends this message to map queues to vectors and ITR index registers.
+ * External data buffer contains virtchnl_queue_vector_maps structure
+ * that contains num_qv_maps of virtchnl_queue_vector structures.
+ * PF maps the requested queue vector maps after validating the queue and vector
+ * ids and returns a status code.
+ */
+struct virtchnl_queue_vector_maps {
+ u16 vport_id;
+ u16 num_qv_maps;
+ u8 pad[4];
+ struct virtchnl_queue_vector qv_maps[];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_vector_maps);
+#define virtchnl_queue_vector_maps_LEGACY_SIZEOF 24
+
struct virtchnl_quanta_cfg {
u16 quanta_size;
u16 pad;
@@ -1695,6 +1785,8 @@ VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
__vss(virtchnl_rdma_qvlist_info, __vss_byelem, p, m, c), \
__vss(virtchnl_qos_cap_list, __vss_byelem, p, m, c), \
__vss(virtchnl_queues_bw_cfg, __vss_byelem, p, m, c), \
+ __vss(virtchnl_del_ena_dis_queues, __vss_byelem, p, m, c), \
+ __vss(virtchnl_queue_vector_maps, __vss_byelem, p, m, c), \
__vss(virtchnl_rss_key, __vss_byone, p, m, c), \
__vss(virtchnl_rss_lut, __vss_byone, p, m, c))
@@ -1757,6 +1849,8 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
case VIRTCHNL_OP_DISABLE_QUEUES:
valid_len = sizeof(struct virtchnl_queue_select);
break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ break;
case VIRTCHNL_OP_ADD_ETH_ADDR:
case VIRTCHNL_OP_DEL_ETH_ADDR:
valid_len = virtchnl_ether_addr_list_LEGACY_SIZEOF;
@@ -1929,7 +2023,32 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
case VIRTCHNL_OP_1588_PTP_GET_TIME:
valid_len = sizeof(struct virtchnl_phc_time);
break;
- /* These are always errors coming from the VF. */
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ valid_len = sizeof(struct virtchnl_del_ena_dis_queues);
+ if (msglen >= valid_len) {
+ struct virtchnl_del_ena_dis_queues *qs =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+
+ if (!qs->num_chunks) {
+ err_msg_format = true;
+ break;
+ }
+ valid_len = virtchnl_struct_size(qs, chunks,
+ qs->num_chunks);
+ }
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ valid_len = virtchnl_queue_vector_maps_LEGACY_SIZEOF;
+ if (msglen >= valid_len) {
+ struct virtchnl_queue_vector_maps *v_qp = (void *)msg;
+
+ err_msg_format = !v_qp->num_qv_maps;
+ valid_len = virtchnl_struct_size(v_qp, qv_maps,
+ v_qp->num_qv_maps);
+ }
+ break;
+ /* These are always errors when coming from the VF. */
case VIRTCHNL_OP_EVENT:
case VIRTCHNL_OP_UNKNOWN:
default:
diff --git a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
index e7cf12eaa268..07beff9c1983 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
@@ -1643,6 +1643,11 @@ static int iavf_set_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd)
static u32 iavf_get_rx_ring_count(struct net_device *netdev)
{
struct iavf_adapter *adapter = netdev_priv(netdev);
+ u16 width = adapter->max_rss_qregion.qregion_width;
+
+ /* RSS can use only the PF RSS region, see iavf_fill_rss_lut() */
+ if (LARGE_NUM_QPAIRS_SUPPORT(adapter) && width)
+ return min_t(u32, adapter->num_active_queues, BIT(width));
return adapter->num_active_queues;
}
@@ -1683,6 +1688,19 @@ static int iavf_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd,
return ret;
}
+
+/* in LARGE mode PF gives us up to its RSS region once it has told us that */
+static u32 iavf_max_channels(struct iavf_adapter *adapter)
+{
+ u16 width = adapter->max_rss_qregion.qregion_width;
+
+ if (LARGE_NUM_QPAIRS_SUPPORT(adapter) && width &&
+ adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_REQ_QUEUES)
+ return BIT(width);
+
+ return adapter->vsi_res->num_queue_pairs;
+}
+
/**
* iavf_get_channels - get the number of channels supported by the device
* @netdev: network interface device structure
@@ -1697,12 +1715,15 @@ static void iavf_get_channels(struct net_device *netdev,
struct iavf_adapter *adapter = netdev_priv(netdev);
/* Report maximum channels */
- ch->max_combined = adapter->vsi_res->num_queue_pairs;
+ ch->max_combined = iavf_max_channels(adapter);
ch->max_other = NONQ_VECS;
ch->other_count = NONQ_VECS;
ch->combined_count = adapter->num_active_queues;
+ /* ethtool -L count, until the watchdog sends the request */
+ if (adapter->aq_required & IAVF_FLAG_AQ_REQUEST_QUEUES)
+ ch->combined_count = adapter->num_req_queues_next;
}
/**
@@ -1730,15 +1751,29 @@ static int iavf_set_channels(struct net_device *netdev,
/* All of these should have already been checked by ethtool before this
* even gets to us, but just to be sure.
*/
- if (num_req == 0 || num_req > adapter->vsi_res->num_queue_pairs)
+ if (num_req == 0 || num_req > iavf_max_channels(adapter))
return -EINVAL;
- if (num_req == adapter->num_active_queues)
- return 0;
-
if (ch->rx_count || ch->tx_count || ch->other_count != NONQ_VECS)
return -EINVAL;
+ /* PF has not answered our previous request yet */
+ if (adapter->current_op == VIRTCHNL_OP_REQUEST_QUEUES)
+ return -EBUSY;
+
+ if (num_req == adapter->num_active_queues) {
+ adapter->aq_required &= ~IAVF_FLAG_AQ_REQUEST_QUEUES;
+ return 0;
+ }
+
+ if (LARGE_NUM_QPAIRS_SUPPORT(adapter) &&
+ adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_REQ_QUEUES) {
+ /* the watchdog sends it once no other op is in flight */
+ adapter->num_req_queues_next = num_req;
+ iavf_schedule_aq_request(adapter, IAVF_FLAG_AQ_REQUEST_QUEUES);
+ return 0;
+ }
+
adapter->num_req_queues = num_req;
adapter->flags |= IAVF_FLAG_REINIT_ITR_NEEDED;
adapter->flags |= IAVF_FLAG_RESET_NEEDED;
@@ -1817,13 +1852,26 @@ static int iavf_set_rxfh(struct net_device *netdev,
struct netlink_ext_ack *extack)
{
struct iavf_adapter *adapter = netdev_priv(netdev);
+ u16 width = adapter->max_rss_qregion.qregion_width;
u16 i;
/* Only support toeplitz hash function */
if (rxfh->hfunc != ETH_RSS_HASH_NO_CHANGE &&
rxfh->hfunc != ETH_RSS_HASH_TOP)
return -EOPNOTSUPP;
+ /* PF takes only queues from its RSS region, see iavf_fill_rss_lut() */
+ if (rxfh->indir && LARGE_NUM_QPAIRS_SUPPORT(adapter) && width) {
+ for (i = 0; i < adapter->rss_lut_size; i++) {
+ if (rxfh->indir[i] < BIT(width))
+ continue;
+ NL_SET_ERR_MSG_FMT_MOD(extack,
+ "RSS can use only the first %lu queues",
+ BIT(width));
+ return -EINVAL;
+ }
+ }
+
if ((rxfh->input_xfrm & RXH_XFRM_SYM_XOR) &&
adapter->hfunc != VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC) {
if (!ADV_RSS_SUPPORT(adapter))
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index db769eead68f..b6b6d61ac2ac 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -509,6 +509,7 @@ static void iavf_map_rings_to_vectors(struct iavf_adapter *adapter)
vidx = 0;
}
+ adapter->map_qps_sent = 0;
adapter->aq_required |= IAVF_FLAG_AQ_MAP_VECTORS;
}
@@ -1783,12 +1784,17 @@ int iavf_config_rss(struct iavf_adapter *adapter)
* iavf_fill_rss_lut - Fill the lut with default values
* @adapter: board private structure
**/
-static void iavf_fill_rss_lut(struct iavf_adapter *adapter)
+void iavf_fill_rss_lut(struct iavf_adapter *adapter)
{
- u16 i;
+ struct virtchnl_max_rss_qregion *qregion = &adapter->max_rss_qregion;
+ int max = adapter->num_active_queues;
+
+ /* width == 0: PF has not told us the region yet */
+ if (LARGE_NUM_QPAIRS_SUPPORT(adapter) && qregion->qregion_width)
+ max = min(max, (int)BIT(qregion->qregion_width));
- for (i = 0; i < adapter->rss_lut_size; i++)
- adapter->rss_lut[i] = i % adapter->num_active_queues;
+ for (int i = 0; i < adapter->rss_lut_size; i++)
+ adapter->rss_lut[i] = i % max;
}
/**
@@ -2099,6 +2105,8 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
{
if (adapter->aq_required & IAVF_FLAG_AQ_GET_CONFIG)
return iavf_send_vf_config_msg(adapter);
+ if (adapter->aq_required & IAVF_FLAG_AQ_GET_MAX_RSS_QREGION)
+ return iavf_send_max_rss_qregion(adapter);
if (adapter->aq_required & IAVF_FLAG_AQ_GET_OFFLOAD_VLAN_V2_CAPS)
return iavf_send_vf_offload_vlan_v2_msg(adapter);
if (adapter->aq_required & IAVF_FLAG_AQ_GET_SUPPORTED_RXDIDS)
@@ -2273,6 +2281,12 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
iavf_virtchnl_send_ptp_cmd(adapter);
return IAVF_SUCCESS;
}
+ if (adapter->aq_required & IAVF_FLAG_AQ_REQUEST_QUEUES) {
+ /* ethtool -L has already reported success, retry until sent */
+ if (!iavf_request_queues(adapter, adapter->num_req_queues_next))
+ adapter->aq_required &= ~IAVF_FLAG_AQ_REQUEST_QUEUES;
+ return 0;
+ }
if (adapter->aq_required & IAVF_FLAG_AQ_REQUEST_STATS) {
iavf_request_stats(adapter);
return 0;
@@ -2468,17 +2482,33 @@ static void iavf_init_version_check(struct iavf_adapter *adapter)
*/
int iavf_parse_vf_resource_msg(struct iavf_adapter *adapter)
{
- int i, num_req_queues = adapter->num_req_queues;
+ int i, qnum, num_req_queues = adapter->num_req_queues;
+ struct virtchnl_vsi_resource *vsi_res = NULL;
struct iavf_vsi *vsi = &adapter->vsi;
+ bool reconfig_rss = false;
+ bool init_req = false;
for (i = 0; i < adapter->vf_res->num_vsis; i++) {
if (adapter->vf_res->vsi_res[i].vsi_type == VIRTCHNL_VSI_SRIOV)
- adapter->vsi_res = &adapter->vf_res->vsi_res[i];
+ vsi_res = &adapter->vf_res->vsi_res[i];
}
- if (!adapter->vsi_res) {
+ if (!vsi_res) {
dev_err(&adapter->pdev->dev, "No LAN VSI found\n");
return -ENODEV;
}
+ adapter->vsi_res = vsi_res;
+
+ /* in LARGE mode we ask PF at init for more queues, up to one per CPU */
+ if (adapter->state == __IAVF_INIT_GET_RESOURCES &&
+ LARGE_NUM_QPAIRS_SUPPORT(adapter) &&
+ adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_REQ_QUEUES)
+ init_req = true;
+
+ /* PF resets us to grant the request, but other resets could come
+ * first, then we have a different count and need to ask again
+ */
+ if (init_req && num_req_queues != adapter->vsi_res->num_queue_pairs)
+ num_req_queues = 0;
if (num_req_queues &&
num_req_queues > adapter->vsi_res->num_queue_pairs) {
@@ -2496,21 +2526,72 @@ int iavf_parse_vf_resource_msg(struct iavf_adapter *adapter)
return -EAGAIN;
}
+ /* PF could cut our queues on reset, e.g. when RSS LUT got smaller */
+ if (adapter->num_active_queues > adapter->vsi_res->num_queue_pairs) {
+ adapter->flags |= IAVF_FLAG_REINIT_MSIX_NEEDED;
+ adapter->num_req_queues = adapter->vsi_res->num_queue_pairs;
+ iavf_schedule_reset(adapter, IAVF_FLAG_RESET_NEEDED);
+
+ return -EAGAIN;
+ }
adapter->num_req_queues = 0;
adapter->vsi.id = adapter->vsi_res->vsi_id;
adapter->vsi.back = adapter;
adapter->vsi.base_vector = 1;
vsi->netdev = adapter->netdev;
vsi->qs_handle = adapter->vsi_res->qset_handle;
if (adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
- adapter->rss_key_size = adapter->vf_res->rss_key_size;
- adapter->rss_lut_size = adapter->vf_res->rss_lut_size;
+ if ((adapter->rss_key &&
+ adapter->rss_key_size != adapter->vf_res->rss_key_size) ||
+ (adapter->rss_lut &&
+ adapter->rss_lut_size != adapter->vf_res->rss_lut_size)) {
+ reconfig_rss = true;
+ } else {
+ adapter->rss_key_size = adapter->vf_res->rss_key_size;
+ adapter->rss_lut_size = adapter->vf_res->rss_lut_size;
+ }
} else {
adapter->rss_key_size = IAVF_HKEY_ARRAY_SIZE;
adapter->rss_lut_size = IAVF_HLUT_ARRAY_SIZE;
}
+ if (reconfig_rss) {
+ u8 *rss_key, *rss_lut;
+
+ rss_key = kzalloc(adapter->vf_res->rss_key_size, GFP_KERNEL);
+ rss_lut = kzalloc(adapter->vf_res->rss_lut_size, GFP_KERNEL);
+ if (!rss_lut || !rss_key) {
+ kfree(rss_key);
+ kfree(rss_lut);
+ /* old buffers and sizes are kept, retry via reset */
+ iavf_schedule_reset(adapter, IAVF_FLAG_RESET_NEEDED);
+ return -ENOMEM;
+ }
+
+ kfree(adapter->rss_key);
+ kfree(adapter->rss_lut);
+ adapter->rss_key = rss_key;
+ adapter->rss_lut = rss_lut;
+ adapter->rss_key_size = adapter->vf_res->rss_key_size;
+ adapter->rss_lut_size = adapter->vf_res->rss_lut_size;
+ /* no queues yet on init, iavf_init_config_adapter() sets RSS */
+ if (adapter->state != __IAVF_INIT_GET_RESOURCES)
+ iavf_init_rss(adapter);
+ }
+
+ if (init_req) {
+ qnum = min_t(int, num_online_cpus(), IAVF_MAX_REQ_QUEUES);
+ /* granted, or we have enough */
+ if (num_req_queues || qnum <= adapter->vsi_res->num_queue_pairs)
+ return 0;
+
+ adapter->current_op = VIRTCHNL_OP_UNKNOWN;
+ adapter->extended_caps |= IAVF_EXTENDED_CAP_RECV_REQ_QUEUES;
+
+ return iavf_request_queues(adapter, qnum);
+ }
+
return 0;
}
@@ -2557,28 +2638,70 @@ static void iavf_init_get_resources(struct iavf_adapter *adapter)
goto err_alloc;
}
+ /* Some features require additional messages to negotiate extended
+ * capabilities. These are processed in sequence by the
+ * __IAVF_INIT_EXTENDED_CAPS driver state.
+ */
+ adapter->extended_caps = IAVF_EXTENDED_CAPS;
+
err = iavf_parse_vf_resource_msg(adapter);
if (err) {
dev_err(&pdev->dev, "Failed to parse VF resource message from PF (%d)\n",
err);
goto err_alloc;
}
- /* Some features require additional messages to negotiate extended
- * capabilities. These are processed in sequence by the
- * __IAVF_INIT_EXTENDED_CAPS driver state.
- */
- adapter->extended_caps = IAVF_EXTENDED_CAPS;
iavf_change_state(adapter, __IAVF_INIT_EXTENDED_CAPS);
return;
err_alloc:
- kfree(adapter->vf_res);
- adapter->vf_res = NULL;
+ /* keep it allocated, ethtool ops may run on reinit */
+ memset(adapter->vf_res, 0, IAVF_VIRTCHNL_VF_RESOURCE_SIZE);
err:
iavf_change_state(adapter, __IAVF_INIT_FAILED);
}
+/**
+ * iavf_init_recv_req_queues - part of requesting queues in LARGE mode
+ * @adapter: board private structure
+ *
+ * Function waits for the outcome of the queue request sent by
+ * iavf_parse_vf_resource_msg(). PF resets us when it grants the request and
+ * replies only to deny it, then we ask for as many queues as PF can give,
+ * unless we have that many already.
+ */
+static void iavf_init_recv_req_queues(struct iavf_adapter *adapter)
+{
+ int num = -EALREADY;
+
+ /* PF grants by resetting us, the event may be lost with the ARQ */
+ if (rd32(&adapter->hw, IAVF_VF_ARQLEN1) &
+ IAVF_VF_ARQLEN1_ARQENABLE_MASK)
+ num = iavf_get_req_queues_reply(adapter);
+ else
+ iavf_schedule_reset(adapter, IAVF_FLAG_RESET_PENDING);
+
+ if (num == -EALREADY || adapter->flags & IAVF_FLAG_RESET_PENDING) {
+ /* no reply yet, or PF reset us after granting */
+ iavf_change_state(adapter, __IAVF_INIT_FAILED);
+ return;
+ }
+
+ adapter->current_op = VIRTCHNL_OP_UNKNOWN;
+ if (num > 0) {
+ dev_info(&adapter->pdev->dev,
+ "Requested %d queues, PF can support %d\n",
+ adapter->num_req_queues, num);
+ if (num > adapter->vsi_res->num_queue_pairs &&
+ !iavf_request_queues(adapter, num))
+ return;
+ }
+
+ /* go on with the queues PF has given us */
+ adapter->num_req_queues = 0;
+ adapter->extended_caps &= ~IAVF_EXTENDED_CAP_RECV_REQ_QUEUES;
+}
+
/**
* iavf_init_send_offload_vlan_v2_caps - part of initializing VLAN V2 caps
* @adapter: board private structure
@@ -2636,6 +2759,65 @@ static void iavf_init_recv_offload_vlan_v2_caps(struct iavf_adapter *adapter)
iavf_change_state(adapter, __IAVF_INIT_FAILED);
}
+/*
+ * iavf_init_send_max_rss_qregion - part of querying for RSS max queue region
+ * @adapter: board private structure
+ *
+ * Function processes send of the VIRTCHNL_OP_GET_MAX_RSS_QREGION to the PF.
+ * Must clear IAVF_EXTENDED_CAP_RECV_RSS_QREGION if the message is not sent, e.g.
+ * due to the PF not negotiating VIRTCHNL_VF_LARGE_NUM_QPAIRS.
+ */
+static void iavf_init_send_max_rss_qregion(struct iavf_adapter *adapter)
+{
+ int ret;
+
+ WARN_ON(!(adapter->extended_caps & IAVF_EXTENDED_CAP_SEND_RSS_QREGION));
+
+ ret = iavf_send_max_rss_qregion(adapter);
+ if (ret == -EOPNOTSUPP) {
+ /* PF does not support VIRTCHNL_VF_LARGE_NUM_QPAIRS. In this
+ * case, we did not send the capability exchange message and do
+ * not expect a response.
+ */
+ adapter->extended_caps &= ~IAVF_EXTENDED_CAP_RECV_RSS_QREGION;
+ }
+
+ /* We sent the message, so move on to the next step */
+ adapter->extended_caps &= ~IAVF_EXTENDED_CAP_SEND_RSS_QREGION;
+}
+
+/**
+ * iavf_init_recv_max_rss_qregion - part of querying for RSS max queue region
+ * @adapter: board private structure
+ *
+ * Function processes receipt of the RSS max qregion to be used for the LUT.
+ */
+static void iavf_init_recv_max_rss_qregion(struct iavf_adapter *adapter)
+{
+ int ret;
+
+ WARN_ON(!(adapter->extended_caps & IAVF_EXTENDED_CAP_RECV_RSS_QREGION));
+
+ memset(&adapter->max_rss_qregion, 0, sizeof(adapter->max_rss_qregion));
+
+ ret = iavf_get_max_rss_qregion(adapter);
+ if (ret)
+ goto err;
+
+ /* We've processed the PF response to the VIRTCHNL_OP_GET_MAX_RSS_QREGION
+ * message we sent previously.
+ */
+ adapter->extended_caps &= ~IAVF_EXTENDED_CAP_RECV_RSS_QREGION;
+ return;
+
+err:
+ /* We didn't receive a reply. Make sure we try sending again when
+ * __IAVF_INIT_FAILED attempts to recover.
+ */
+ adapter->extended_caps |= IAVF_EXTENDED_CAP_SEND_RSS_QREGION;
+ iavf_change_state(adapter, __IAVF_INIT_FAILED);
+}
+
/**
* iavf_init_send_supported_rxdids - part of querying for supported RXDID
* formats
@@ -2757,6 +2939,12 @@ static void iavf_init_process_extended_caps(struct iavf_adapter *adapter)
{
WARN_ON(adapter->state != __IAVF_INIT_EXTENDED_CAPS);
+ /* Must be first, other steps would drop the reply while polling */
+ if (adapter->extended_caps & IAVF_EXTENDED_CAP_RECV_REQ_QUEUES) {
+ iavf_init_recv_req_queues(adapter);
+ return;
+ }
+
/* Process capability exchange for VLAN V2 */
if (adapter->extended_caps & IAVF_EXTENDED_CAP_SEND_VLAN_V2) {
iavf_init_send_offload_vlan_v2_caps(adapter);
@@ -2784,6 +2972,16 @@ static void iavf_init_process_extended_caps(struct iavf_adapter *adapter)
return;
}
+ /* Process capability exchange for RSS max qregion */
+ if (adapter->extended_caps & IAVF_EXTENDED_CAP_SEND_RSS_QREGION) {
+ iavf_init_send_max_rss_qregion(adapter);
+ return;
+ }
+ if (adapter->extended_caps & IAVF_EXTENDED_CAP_RECV_RSS_QREGION) {
+ iavf_init_recv_max_rss_qregion(adapter);
+ return;
+ }
+
/* When we reach here, no further extended capabilities exchanges are
* necessary, so we finally transition into __IAVF_INIT_CONFIG_ADAPTER
*/
@@ -2893,6 +3091,10 @@ static void iavf_init_config_adapter(struct iavf_adapter *adapter)
/* Setup initial PTP configuration */
iavf_ptp_init(adapter);
+ /* queues requested at init are set up now */
+ adapter->num_req_queues = 0;
+ adapter->flags &= ~IAVF_FLAG_REINIT_ITR_NEEDED;
+
iavf_schedule_finish_config(adapter);
return;
@@ -2977,6 +3179,9 @@ static int iavf_watchdog_step(struct iavf_adapter *adapter)
*/
iavf_change_state(adapter, __IAVF_STARTUP);
adapter->flags &= ~IAVF_FLAG_PF_COMMS_FAILED;
+ /* fresh retry budget, and init asks for queues again */
+ adapter->aq_wait_count = 0;
+ adapter->num_req_queues = 0;
}
adapter->aq_required = 0;
adapter->current_op = VIRTCHNL_OP_UNKNOWN;
@@ -3015,7 +3220,8 @@ static int iavf_watchdog_step(struct iavf_adapter *adapter)
/* check for hw reset */
reg_val = rd32(hw, IAVF_VF_ARQLEN1) & IAVF_VF_ARQLEN1_ARQENABLE_MASK;
if (!reg_val) {
- adapter->aq_required = 0;
+ /* keep a queue request of ethtool -L that is not sent yet */
+ adapter->aq_required &= IAVF_FLAG_AQ_REQUEST_QUEUES;
adapter->current_op = VIRTCHNL_OP_UNKNOWN;
dev_err(&adapter->pdev->dev, "Hardware reset detected\n");
iavf_schedule_reset(adapter, IAVF_FLAG_RESET_PENDING);
@@ -3251,13 +3457,19 @@ void iavf_reset_step(struct iavf_adapter *adapter)
status);
goto reset_err;
}
- adapter->aq_required = 0;
+ /* keep a queue request of ethtool -L that is not sent yet */
+ adapter->aq_required &= IAVF_FLAG_AQ_REQUEST_QUEUES;
if ((adapter->flags & IAVF_FLAG_REINIT_MSIX_NEEDED) ||
(adapter->flags & IAVF_FLAG_REINIT_ITR_NEEDED)) {
err = iavf_reinit_interrupt_scheme(adapter, running);
if (err)
goto reset_err;
+ } else {
+ /* queues we requested were set up by the previous reset, fewer
+ * of them now is a config change, not a denial
+ */
+ adapter->num_req_queues = 0;
}
if (RSS_AQ(adapter)) {
@@ -3269,6 +3481,7 @@ void iavf_reset_step(struct iavf_adapter *adapter)
}
adapter->aq_required |= IAVF_FLAG_AQ_GET_CONFIG;
+ adapter->map_qps_sent = 0;
adapter->aq_required |= IAVF_FLAG_AQ_MAP_VECTORS;
/* Certain capabilities require an extended negotiation process using
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index 1296ea0f1716..6437fd4681be 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -170,6 +170,7 @@ int iavf_send_vf_config_msg(struct iavf_adapter *adapter)
VIRTCHNL_VF_OFFLOAD_VLAN_V2 |
VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC |
VIRTCHNL_VF_OFFLOAD_CRC |
+ VIRTCHNL_VF_LARGE_NUM_QPAIRS |
VIRTCHNL_VF_OFFLOAD_ENCAP_CSUM |
VIRTCHNL_VF_OFFLOAD_REQ_QUEUES |
VIRTCHNL_VF_CAP_PTP |
@@ -251,28 +252,45 @@ int iavf_send_vf_ptp_caps_msg(struct iavf_adapter *adapter)
(u8 *)&hw_caps, sizeof(hw_caps));
}
+int iavf_send_max_rss_qregion(struct iavf_adapter *adapter)
+{
+ adapter->aq_required &= ~IAVF_FLAG_AQ_GET_MAX_RSS_QREGION;
+
+ if (!LARGE_NUM_QPAIRS_SUPPORT(adapter))
+ return -EOPNOTSUPP;
+
+ adapter->current_op = VIRTCHNL_OP_GET_MAX_RSS_QREGION;
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_GET_MAX_RSS_QREGION, NULL, 0);
+ return 0;
+}
+
/**
* iavf_validate_num_queues
* @adapter: adapter structure
*
* Validate that the number of queues the PF has sent in
* VIRTCHNL_OP_GET_VF_RESOURCES is not larger than the VF can handle.
**/
static void iavf_validate_num_queues(struct iavf_adapter *adapter)
{
- if (adapter->vf_res->num_queue_pairs > IAVF_MAX_REQ_QUEUES) {
+ u32 max_req_queues = IAVF_MAX_REQ_QUEUES;
+
+ if (!LARGE_NUM_QPAIRS_SUPPORT(adapter))
+ max_req_queues = IAVF_MAX_VSI_QP;
+
+ if (adapter->vf_res->num_queue_pairs > max_req_queues) {
struct virtchnl_vsi_resource *vsi_res;
int i;
dev_info(&adapter->pdev->dev, "Received %d queues, but can only have a max of %d\n",
adapter->vf_res->num_queue_pairs,
- IAVF_MAX_REQ_QUEUES);
+ max_req_queues);
dev_info(&adapter->pdev->dev, "Fixing by reducing queues to %d\n",
- IAVF_MAX_REQ_QUEUES);
- adapter->vf_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES;
+ max_req_queues);
+ adapter->vf_res->num_queue_pairs = max_req_queues;
for (i = 0; i < adapter->vf_res->num_vsis; i++) {
vsi_res = &adapter->vf_res->vsi_res[i];
- vsi_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES;
+ vsi_res->num_queue_pairs = max_req_queues;
}
}
}
@@ -371,6 +389,57 @@ int iavf_get_vf_ptp_caps(struct iavf_adapter *adapter)
return err;
}
+int iavf_get_max_rss_qregion(struct iavf_adapter *adapter)
+{
+ struct iavf_arq_event_info event;
+ int err;
+ u16 len;
+
+ len = sizeof(struct virtchnl_max_rss_qregion);
+ event.buf_len = len;
+ event.msg_buf = kzalloc(len, GFP_KERNEL);
+ if (!event.msg_buf)
+ return -ENOMEM;
+
+ err = iavf_poll_virtchnl_msg(&adapter->hw, &event,
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION);
+ if (!err)
+ memcpy(&adapter->max_rss_qregion, event.msg_buf,
+ min(event.msg_len, len));
+
+ adapter->current_op = VIRTCHNL_OP_UNKNOWN;
+
+ kfree(event.msg_buf);
+ return err;
+}
+
+/**
+ * iavf_get_req_queues_reply - poll for PF reply to our queue request
+ * @adapter: private adapter structure
+ *
+ * PF resets us when it grants the request, so a reply means a denial.
+ *
+ * Return: number of queues PF can give us, or negative error code.
+ */
+int iavf_get_req_queues_reply(struct iavf_adapter *adapter)
+{
+ /* poll looks for the reset event in the same buffer */
+ union {
+ struct virtchnl_vf_res_request vfres;
+ struct virtchnl_pf_event vpe;
+ } msg = {};
+ struct iavf_arq_event_info event = {
+ .buf_len = sizeof(msg),
+ .msg_buf = (u8 *)&msg,
+ };
+ int err;
+
+ err = iavf_poll_virtchnl_msg(&adapter->hw, &event,
+ VIRTCHNL_OP_REQUEST_QUEUES);
+
+ return err ?: msg.vfres.num_queue_pairs;
+}
+
/**
* iavf_configure_queues
* @adapter: adapter structure
@@ -438,6 +507,50 @@ void iavf_configure_queues(struct iavf_adapter *adapter)
kfree(vqci);
}
+/**
+ * iavf_enable_disable_queues_v2 - send V2 messages of ENABLE/DISABLE queues ops
+ * @adapter: private adapter structure
+ * @enable: true to enable and false to disable the queues
+ */
+static void iavf_enable_disable_queues_v2(struct iavf_adapter *adapter, bool enable)
+{
+ enum virtchnl_ops op = VIRTCHNL_OP_ENABLE_QUEUES_V2;
+ struct virtchnl_del_ena_dis_queues *msg;
+ u64 flag = IAVF_FLAG_AQ_ENABLE_QUEUES;
+ struct virtchnl_queue_chunk *chunk;
+ int len;
+
+ if (!enable) {
+ op = VIRTCHNL_OP_DISABLE_QUEUES_V2;
+ flag = IAVF_FLAG_AQ_DISABLE_QUEUES;
+ }
+
+ /* We need 2 chunks (one Tx and one Rx). */
+ len = virtchnl_struct_size(msg, chunks, 2);
+ msg = kzalloc(len, GFP_KERNEL);
+ if (!msg)
+ return;
+
+ adapter->current_op = op;
+
+ msg->vport_id = adapter->vsi_res->vsi_id;
+ msg->num_chunks = 2;
+
+ chunk = &msg->chunks[0];
+ chunk->type = VIRTCHNL_QUEUE_TYPE_RX;
+ chunk->start_queue_id = 0;
+ chunk->num_queues = adapter->num_active_queues;
+
+ chunk++;
+ chunk->type = VIRTCHNL_QUEUE_TYPE_TX;
+ chunk->start_queue_id = 0;
+ chunk->num_queues = adapter->num_active_queues;
+
+ adapter->aq_required &= ~flag;
+ iavf_send_pf_msg(adapter, op, (u8 *)msg, len);
+ kfree(msg);
+}
+
/**
* iavf_enable_queues
* @adapter: adapter structure
@@ -454,6 +567,12 @@ void iavf_enable_queues(struct iavf_adapter *adapter)
adapter->current_op);
return;
}
+
+ if (LARGE_NUM_QPAIRS_SUPPORT(adapter)) {
+ iavf_enable_disable_queues_v2(adapter, true);
+ return;
+ }
+
adapter->current_op = VIRTCHNL_OP_ENABLE_QUEUES;
vqs.vsi_id = adapter->vsi_res->vsi_id;
vqs.tx_queues = BIT(min(adapter->num_active_queues,
@@ -480,6 +599,12 @@ void iavf_disable_queues(struct iavf_adapter *adapter)
adapter->current_op);
return;
}
+
+ if (LARGE_NUM_QPAIRS_SUPPORT(adapter)) {
+ iavf_enable_disable_queues_v2(adapter, false);
+ return;
+ }
+
adapter->current_op = VIRTCHNL_OP_DISABLE_QUEUES;
vqs.vsi_id = adapter->vsi_res->vsi_id;
vqs.tx_queues = BIT(min(adapter->num_active_queues,
@@ -490,6 +615,50 @@ void iavf_disable_queues(struct iavf_adapter *adapter)
(u8 *)&vqs, sizeof(vqs));
}
+static void iavf_map_queue_vector(struct iavf_adapter *adapter)
+{
+ struct virtchnl_queue_vector_maps *qvmaps;
+ int first = adapter->map_qps_sent;
+ struct virtchnl_queue_vector *qv;
+ int len, qnum;
+
+ qnum = min_t(int, adapter->num_active_queues - first,
+ iavf_max_vc_entries(qvmaps, qv_maps) / 2);
+ len = virtchnl_struct_size(qvmaps, qv_maps, 2 * qnum);
+ qvmaps = kzalloc(len, GFP_KERNEL);
+ if (!qvmaps)
+ return;
+
+ qvmaps->vport_id = adapter->vsi_res->vsi_id;
+ qvmaps->num_qv_maps = 2 * qnum;
+ qv = qvmaps->qv_maps;
+ for (int qid = first; qid < first + qnum; qid++) {
+ struct iavf_q_vector *q_vector;
+
+ q_vector = adapter->tx_rings[qid].q_vector;
+ qv->queue_id = qid;
+ qv->vector_id = NONQ_VECS + q_vector->v_idx;
+ qv->itr_idx = IAVF_TX_ITR;
+ qv->queue_type = VIRTCHNL_QUEUE_TYPE_TX;
+ qv++;
+
+ q_vector = adapter->rx_rings[qid].q_vector;
+ qv->queue_id = qid;
+ qv->vector_id = NONQ_VECS + q_vector->v_idx;
+ qv->itr_idx = IAVF_RX_ITR;
+ qv->queue_type = VIRTCHNL_QUEUE_TYPE_RX;
+ qv++;
+ }
+
+ adapter->current_op = VIRTCHNL_OP_MAP_QUEUE_VECTOR;
+ adapter->map_qps_sent += qnum;
+ if (adapter->map_qps_sent == adapter->num_active_queues)
+ adapter->aq_required &= ~IAVF_FLAG_AQ_MAP_VECTORS;
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_MAP_QUEUE_VECTOR,
+ (u8 *)qvmaps, len);
+ kfree(qvmaps);
+}
+
/**
* iavf_map_queues
* @adapter: adapter structure
@@ -511,6 +680,12 @@ void iavf_map_queues(struct iavf_adapter *adapter)
adapter->current_op);
return;
}
+
+ if (LARGE_NUM_QPAIRS_SUPPORT(adapter)) {
+ iavf_map_queue_vector(adapter);
+ return;
+ }
+
adapter->current_op = VIRTCHNL_OP_CONFIG_IRQ_MAP;
q_vectors = adapter->num_msix_vectors - NONQ_VECS;
@@ -547,6 +722,40 @@ void iavf_map_queues(struct iavf_adapter *adapter)
kfree(vimi);
}
+/**
+ * iavf_request_queues - request queues via virtchnl
+ * @adapter: adapter structure
+ * @num: number of requested queues
+ *
+ * Return: 0 on success, negative on failure.
+ */
+int iavf_request_queues(struct iavf_adapter *adapter, int num)
+{
+ struct virtchnl_vf_res_request vfres = { num };
+ int err;
+
+ if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
+ /* bail because we already have a command pending */
+ dev_err(&adapter->pdev->dev, "Cannot request queues, command %d pending\n",
+ adapter->current_op);
+ return -EBUSY;
+ }
+
+ adapter->current_op = VIRTCHNL_OP_REQUEST_QUEUES;
+ adapter->flags |= IAVF_FLAG_REINIT_ITR_NEEDED;
+ adapter->num_req_queues = num;
+
+ err = iavf_send_pf_msg(adapter, VIRTCHNL_OP_REQUEST_QUEUES,
+ (u8 *)&vfres, sizeof(vfres));
+ if (err) {
+ adapter->current_op = VIRTCHNL_OP_UNKNOWN;
+ adapter->flags &= ~IAVF_FLAG_REINIT_ITR_NEEDED;
+ adapter->num_req_queues = 0;
+ }
+
+ return err;
+}
+
/**
* iavf_set_mac_addr_type - Set the correct request type from the filter type
* @virtchnl_ether_addr: pointer to requested list element
@@ -2567,6 +2776,18 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
/* don't send the remaining queue pairs */
adapter->aq_required &= ~IAVF_FLAG_AQ_CONFIGURE_QUEUES;
break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ dev_err(&adapter->pdev->dev, "Failed to map queues to vectors, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ /* don't send the remaining queues */
+ adapter->aq_required &= ~IAVF_FLAG_AQ_MAP_VECTORS;
+ break;
+ case VIRTCHNL_OP_REQUEST_QUEUES:
+ dev_err(&adapter->pdev->dev, "Failed to request queues, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ adapter->num_req_queues = 0;
+ adapter->flags &= ~IAVF_FLAG_REINIT_ITR_NEEDED;
+ break;
default:
dev_err(&adapter->pdev->dev, "PF returned error %d (%s) to our request %d\n",
v_retval, iavf_stat_str(&adapter->hw, v_retval),
@@ -2638,7 +2859,9 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
iavf_activate_fdir_filters(adapter);
- iavf_parse_vf_resource_msg(adapter);
+ /* a reset is scheduled on error, it will configure us again */
+ if (iavf_parse_vf_resource_msg(adapter))
+ break;
/* negotiated VIRTCHNL_VF_OFFLOAD_VLAN_V2, so wait for the
* response to VIRTCHNL_OP_GET_OFFLOAD_VLAN_V2_CAPS to finish
@@ -2709,6 +2932,20 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
aq_required;
}
break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION: {
+ u16 width = adapter->max_rss_qregion.qregion_width;
+
+ if (v_retval)
+ break;
+ memcpy(&adapter->max_rss_qregion, msg,
+ sizeof(adapter->max_rss_qregion));
+ /* LUT was filled before this reply, for the old region */
+ if (adapter->max_rss_qregion.qregion_width != width) {
+ iavf_fill_rss_lut(adapter);
+ iavf_config_rss(adapter);
+ }
+ break;
+ }
case VIRTCHNL_OP_GET_SUPPORTED_RXDIDS:
if (msglen != sizeof(u64))
return;
@@ -2729,20 +2966,23 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
iavf_virtchnl_ptp_get_time(adapter, msg, msglen);
break;
case VIRTCHNL_OP_ENABLE_QUEUES:
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
/* enable transmits */
iavf_irq_enable(adapter, true);
adapter->flags &= ~IAVF_FLAG_QUEUES_DISABLED;
break;
case VIRTCHNL_OP_DISABLE_QUEUES:
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
iavf_free_all_tx_resources(adapter);
iavf_free_all_rx_resources(adapter);
if (adapter->state == __IAVF_DOWN_PENDING) {
iavf_change_state(adapter, __IAVF_DOWN);
wake_up(&adapter->down_waitqueue);
}
break;
case VIRTCHNL_OP_VERSION:
case VIRTCHNL_OP_CONFIG_IRQ_MAP:
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
/* Don't display an error if we get these out of sequence.
* If the firmware needed to get kicked, we'll get these and
* it's no problem.
@@ -2765,7 +3005,8 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
struct virtchnl_vf_res_request *vfres =
(struct virtchnl_vf_res_request *)msg;
- if (vfres->num_queue_pairs != adapter->num_req_queues) {
+ if (!v_retval &&
+ vfres->num_queue_pairs != adapter->num_req_queues) {
dev_info(&adapter->pdev->dev,
"Requested %d queues, PF can support %d\n",
adapter->num_req_queues,
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 11/15] ice: introduce handling of virtchnl LARGE VF opcodes
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (9 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 10/15] iavf: use new opcodes to request more than 16 queues Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 12/15] devlink: give user option to allocate resources Przemek Kitszel
` (3 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
From: Brett Creeley <brett.creeley@intel.com>
With new virtchnl offload/capability VFs are able to make use of more than
16 queues. But the old opcodes were designed with a max of 16 queues,
so new ones were added (by iavf/virtchnl commit of this series).
Handle VIRTCHNL_OP_ENABLE_QUEUES_V2, VIRTCHNL_OP_DISABLE_QUEUES_V2 and
VIRTCHNL_OP_MAP_QUEUE_VECTOR here. VIRTCHNL_OP_GET_MAX_RSS_QREGION is
only allowlisted, its handler comes with bigger RSS LUT support later
in the series.
If a VF wishes to request >16 queues it should first make sure that the
PF supports the VIRTCHNL_VF_LARGE_NUM_QPAIRS capability.
Signed-off-by: Brett Creeley <brett.creeley@intel.com>
Co-developed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com> # msglen val
Signed-off-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Co-developed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v5:
* widen num_ring_{rx,tx} of struct ice_q_vector to u16, mapping all
256 queues of a VF to one vector wrapped them around (Sashiko)
v4:
* limit queue chunks to ICE_MAX_QS_PER_VF, the size of
vf->{rx,tx}q_ena, as num_vf_qs could be set higher via
sriov_vf_msix_count (Sashiko)
* VIRTCHNL_OP_MAP_QUEUE_VECTOR: set ring->q_vector, num_ring_{rx,tx}
and {rx,tx}.itr_idx, as VIRTCHNL_OP_CONFIG_IRQ_MAP does (Sashiko)
* look up the VF VSI once per message, make ice_vc_ena_txq_chunk()
void (Sashiko)
v3:
* reject VIRTCHNL_OP_MAP_QUEUE_VECTOR entries with an out of range
itr_idx (me)
v2:
* fix error response in one case of ice_vc_map_q_vector_msg() (Alex)
* use ice_get_vf_vsi() instead of plain array access (Sashiko)
* return VIRTCHNL_STATUS_ERR_PARAM on error (Sashiko)
* commit message: GET_MAX_RSS_QREGION is handled only later
---
drivers/net/ethernet/intel/ice/ice.h | 4 +-
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 1 +
drivers/net/ethernet/intel/ice/virt/queues.h | 3 +
.../net/ethernet/intel/ice/virt/allowlist.c | 8 +
drivers/net/ethernet/intel/ice/virt/queues.c | 339 ++++++++++++++++++
5 files changed, 353 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 89a2bdbecf60..fc2e84136e24 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -474,8 +474,8 @@ struct ice_q_vector {
u16 v_idx; /* index in the vsi->q_vector array. */
u16 reg_idx; /* PF relative register index */
- u8 num_ring_rx; /* total number of Rx rings in vector */
- u8 num_ring_tx; /* total number of Tx rings in vector */
+ u16 num_ring_rx; /* total number of Rx rings in vector */
+ u16 num_ring_tx; /* total number of Tx rings in vector */
u8 wb_on_itr:1; /* if true, WB on ITR is enabled */
/* in usecs, need to use ice_intrl_to_usecs_reg() before writing this
* value to the device
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
index a9e95c85bc98..396dfb2b2d97 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
@@ -125,6 +125,7 @@ struct ice_vf_ops {
void (*clear_reset_trigger)(struct ice_vf *vf);
void (*irq_close)(struct ice_vf *vf);
void (*post_vsi_rebuild)(struct ice_vf *vf);
+ struct ice_q_vector *(*get_q_vector)(struct ice_vsi *vsi, u16 vec_id);
};
/* Virtchnl/SR-IOV config info */
diff --git a/drivers/net/ethernet/intel/ice/virt/queues.h b/drivers/net/ethernet/intel/ice/virt/queues.h
index c4a792cecea1..223f609dd4f3 100644
--- a/drivers/net/ethernet/intel/ice/virt/queues.h
+++ b/drivers/net/ethernet/intel/ice/virt/queues.h
@@ -16,5 +16,8 @@ int ice_vc_cfg_q_bw(struct ice_vf *vf, u8 *msg);
int ice_vc_cfg_q_quanta(struct ice_vf *vf, u8 *msg);
int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg);
int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg);
+int ice_vc_ena_qs_v2_msg(struct ice_vf *vf, u8 *msg, u16 msglen);
+int ice_vc_dis_qs_v2_msg(struct ice_vf *vf, u8 *msg, u16 msglen);
+int ice_vc_map_q_vector_msg(struct ice_vf *vf, u8 *msg, u16 msglen);
#endif /* _ICE_VIRT_QUEUES_H_ */
diff --git a/drivers/net/ethernet/intel/ice/virt/allowlist.c b/drivers/net/ethernet/intel/ice/virt/allowlist.c
index a07efec19c45..ef769b843c6f 100644
--- a/drivers/net/ethernet/intel/ice/virt/allowlist.c
+++ b/drivers/net/ethernet/intel/ice/virt/allowlist.c
@@ -95,6 +95,13 @@ static const u32 tc_allowlist_opcodes[] = {
VIRTCHNL_OP_CONFIG_QUANTA,
};
+static const u32 large_num_qpairs_allowlist_opcodes[] = {
+ VIRTCHNL_OP_GET_MAX_RSS_QREGION,
+ VIRTCHNL_OP_ENABLE_QUEUES_V2,
+ VIRTCHNL_OP_DISABLE_QUEUES_V2,
+ VIRTCHNL_OP_MAP_QUEUE_VECTOR,
+};
+
struct allowlist_opcode_info {
const u32 *opcodes;
size_t size;
@@ -117,6 +124,7 @@ static const struct allowlist_opcode_info allowlist_opcodes[] = {
ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_VLAN_V2, vlan_v2_allowlist_opcodes),
ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_QOS, tc_allowlist_opcodes),
ALLOW_ITEM(VIRTCHNL_VF_CAP_PTP, ptp_allowlist_opcodes),
+ ALLOW_ITEM(VIRTCHNL_VF_LARGE_NUM_QPAIRS, large_num_qpairs_allowlist_opcodes),
};
/**
diff --git a/drivers/net/ethernet/intel/ice/virt/queues.c b/drivers/net/ethernet/intel/ice/virt/queues.c
index 7f2b459dbc83..f87ccf5d370b 100644
--- a/drivers/net/ethernet/intel/ice/virt/queues.c
+++ b/drivers/net/ethernet/intel/ice/virt/queues.c
@@ -1046,3 +1046,342 @@ int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg)
v_ret, (u8 *)vfres, sizeof(*vfres));
}
+static bool ice_vc_supported_queue_type(s32 queue_type)
+{
+ return queue_type == VIRTCHNL_QUEUE_TYPE_RX ||
+ queue_type == VIRTCHNL_QUEUE_TYPE_TX;
+}
+
+/**
+ * ice_vc_validate_qs_v2_msg - validate all qs_msg parameters
+ * @vf: VF the message was received from
+ * @qs_msg: contents of the message from the VF
+ * @msglen: length of @qs_msg
+ *
+ * Used to validate both the VIRTCHNL_OP_ENABLE_QUEUES_V2 and
+ * VIRTCHNL_OP_DISABLE_QUEUES_V2 messages. This should always be called before
+ * attempting to enable and/or disable queues on behalf of a VF in response to
+ * the previously mentioned opcodes.
+ *
+ * Return: If all checks succeed, then return true. Otherwise return
+ * false, indicating to the caller that the qs_msg is invalid.
+ */
+static bool ice_vc_validate_qs_v2_msg(struct ice_vf *vf,
+ struct virtchnl_del_ena_dis_queues *qs_msg,
+ u16 msglen)
+{
+ u32 max_qs = min_t(u32, vf->num_vf_qs, ICE_MAX_QS_PER_VF);
+
+ if (msglen < virtchnl_struct_size(qs_msg, chunks, 0))
+ return false;
+
+ if (msglen < virtchnl_struct_size(qs_msg, chunks, qs_msg->num_chunks))
+ return false;
+
+ if (!qs_msg->num_chunks)
+ return false;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states))
+ return false;
+
+ if (!ice_vc_isvalid_vsi_id(vf, qs_msg->vport_id))
+ return false;
+
+ for (int i = 0; i < qs_msg->num_chunks; i++) {
+ struct virtchnl_queue_chunk *chunk = &qs_msg->chunks[i];
+ u32 end;
+
+ if (!ice_vc_supported_queue_type(chunk->type))
+ return false;
+
+ if (!chunk->num_queues)
+ return false;
+
+ end = chunk->start_queue_id + chunk->num_queues;
+ if (end > max_qs)
+ return false;
+ }
+
+ return true;
+}
+
+#define ice_for_each_q_in_chunk(chunk, q_id) \
+ for ((q_id) = (chunk)->start_queue_id; \
+ (q_id) < (chunk)->start_queue_id + (chunk)->num_queues; \
+ (q_id)++)
+
+static int
+ice_vc_ena_rxq_chunk(struct ice_vf *vf, struct ice_vsi *vsi,
+ struct virtchnl_queue_chunk *chunk)
+{
+ u32 vf_qid;
+
+ ice_for_each_q_in_chunk(chunk, vf_qid) {
+ int err;
+
+ err = ice_vf_vsi_ena_single_rxq(vf, vsi, vf_qid);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static void
+ice_vc_ena_txq_chunk(struct ice_vf *vf, struct ice_vsi *vsi,
+ struct virtchnl_queue_chunk *chunk)
+{
+ u32 vf_qid;
+
+ ice_for_each_q_in_chunk(chunk, vf_qid)
+ ice_vf_vsi_ena_single_txq(vf, vsi, vf_qid);
+}
+
+/**
+ * ice_vc_ena_qs_v2_msg - message handling for VIRTCHNL_OP_ENABLE_QUEUES_V2
+ * @vf: source of the request
+ * @msg: message to handle
+ * @msglen: length of the @msg
+ *
+ * Return: 0 on success or negative on error.
+ */
+int ice_vc_ena_qs_v2_msg(struct ice_vf *vf, u8 *msg, u16 msglen)
+{
+ struct virtchnl_del_ena_dis_queues *ena_qs_msg =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ struct ice_vsi *vsi;
+
+ if (!ice_vc_validate_qs_v2_msg(vf, ena_qs_msg, msglen))
+ goto error_param;
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi)
+ goto error_param;
+
+ for (int i = 0; i < ena_qs_msg->num_chunks; i++) {
+ struct virtchnl_queue_chunk *chunk = &ena_qs_msg->chunks[i];
+
+ if (chunk->type == VIRTCHNL_QUEUE_TYPE_RX &&
+ ice_vc_ena_rxq_chunk(vf, vsi, chunk))
+ goto error_param;
+ else if (chunk->type == VIRTCHNL_QUEUE_TYPE_TX)
+ ice_vc_ena_txq_chunk(vf, vsi, chunk);
+ }
+
+ v_ret = VIRTCHNL_STATUS_SUCCESS;
+ set_bit(ICE_VF_STATE_QS_ENA, vf->vf_states);
+
+error_param:
+ return ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_ENABLE_QUEUES_V2,
+ v_ret, NULL, 0);
+}
+
+static int
+ice_vc_dis_rxq_chunk(struct ice_vf *vf, struct ice_vsi *vsi,
+ struct virtchnl_queue_chunk *chunk)
+{
+ u32 vf_qid;
+
+ ice_for_each_q_in_chunk(chunk, vf_qid) {
+ int err;
+
+ err = ice_vf_vsi_dis_single_rxq(vf, vsi, vf_qid);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int
+ice_vc_dis_txq_chunk(struct ice_vf *vf, struct ice_vsi *vsi,
+ struct virtchnl_queue_chunk *chunk)
+{
+ u32 vf_qid;
+
+ ice_for_each_q_in_chunk(chunk, vf_qid) {
+ int err;
+
+ err = ice_vf_vsi_dis_single_txq(vf, vsi, vf_qid);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_vc_dis_qs_v2_msg - message handling for VIRTCHNL_OP_DISABLE_QUEUES_V2
+ * @vf: source of the request
+ * @msg: message to handle
+ * @msglen: length of @msg
+ *
+ * Return: 0 on success or negative on error.
+ */
+int ice_vc_dis_qs_v2_msg(struct ice_vf *vf, u8 *msg, u16 msglen)
+{
+ struct virtchnl_del_ena_dis_queues *dis_qs_msg =
+ (struct virtchnl_del_ena_dis_queues *)msg;
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ struct ice_vsi *vsi;
+
+ if (!ice_vc_validate_qs_v2_msg(vf, dis_qs_msg, msglen))
+ goto error_param;
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi)
+ goto error_param;
+
+ for (int i = 0; i < dis_qs_msg->num_chunks; i++) {
+ struct virtchnl_queue_chunk *chunk = &dis_qs_msg->chunks[i];
+
+ if (chunk->type == VIRTCHNL_QUEUE_TYPE_RX &&
+ ice_vc_dis_rxq_chunk(vf, vsi, chunk))
+ goto error_param;
+ else if (chunk->type == VIRTCHNL_QUEUE_TYPE_TX &&
+ ice_vc_dis_txq_chunk(vf, vsi, chunk))
+ goto error_param;
+ }
+
+ v_ret = VIRTCHNL_STATUS_SUCCESS;
+ if (ice_vf_has_no_qs_ena(vf))
+ clear_bit(ICE_VF_STATE_QS_ENA, vf->vf_states);
+
+error_param:
+ return ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_DISABLE_QUEUES_V2,
+ v_ret, NULL, 0);
+}
+
+/**
+ * ice_vc_validate_qv_maps - validate parameters sent in the qs_msg structure
+ * @vf: VF the message was received from
+ * @qv_maps: contents of the message from the VF
+ * @msglen: length of the @qv_maps
+ *
+ * Used to validate VIRTCHNL_OP_MAP_QUEUE_VECTOR messages. This should always
+ * be called before attempting map interrupts to queues. If all checks succeed,
+ * then return success indicating to the caller that the qv_maps are valid.
+ * Otherwise return false, indicating to the caller that the qv_maps are
+ * invalid.
+ *
+ * Return: true if parameters are valid, false otherwise.
+ */
+static bool ice_vc_validate_qv_maps(struct ice_vf *vf,
+ struct virtchnl_queue_vector_maps *qv_maps,
+ u16 msglen)
+{
+ struct ice_vsi *vsi;
+ int total_vectors;
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi)
+ return false;
+
+ if (msglen < virtchnl_struct_size(qv_maps, qv_maps, 0))
+ return false;
+
+ if (msglen < virtchnl_struct_size(qv_maps, qv_maps, qv_maps->num_qv_maps))
+ return false;
+
+ if (!qv_maps->num_qv_maps)
+ return false;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states))
+ return false;
+
+ if (!ice_vc_isvalid_vsi_id(vf, qv_maps->vport_id))
+ return false;
+
+ total_vectors = vsi->num_q_vectors + ICE_NONQ_VECS_VF;
+
+ for (int i = 0; i < qv_maps->num_qv_maps; i++) {
+ if (!ice_vc_supported_queue_type(qv_maps->qv_maps[i].queue_type))
+ return false;
+
+ if (qv_maps->qv_maps[i].queue_id >= vf->num_vf_qs)
+ return false;
+
+ if (qv_maps->qv_maps[i].vector_id >= total_vectors ||
+ qv_maps->qv_maps[i].vector_id < ICE_NONQ_VECS_VF)
+ return false;
+
+ if (qv_maps->qv_maps[i].itr_idx < ICE_IDX_ITR0 ||
+ qv_maps->qv_maps[i].itr_idx > ICE_ITR_NONE)
+ return false;
+ }
+
+ return true;
+}
+
+/**
+ * ice_vc_map_q_vector_msg - message handling for VIRTCHNL_OP_MAP_QUEUE_VECTOR
+ * @vf: source of the request
+ * @msg: message to handle
+ * @msglen: length of @msg
+ *
+ * Return: 0 on success or negative on error
+ */
+int ice_vc_map_q_vector_msg(struct ice_vf *vf, u8 *msg, u16 msglen)
+{
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ struct virtchnl_queue_vector_maps *qv_maps;
+ struct ice_vsi *vsi;
+
+ qv_maps = (struct virtchnl_queue_vector_maps *)msg;
+
+ if (!ice_vc_validate_qv_maps(vf, qv_maps, msglen))
+ goto error_param;
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi)
+ goto error_param;
+
+ for (int i = 0; i < qv_maps->num_qv_maps; i++) {
+ struct virtchnl_queue_vector *qv_map = &qv_maps->qv_maps[i];
+ struct ice_q_vector *q_vector;
+ u16 vector_id;
+ int vsi_q_id;
+
+ vsi_q_id = qv_map->queue_id;
+ vector_id = qv_map->vector_id;
+
+ q_vector = vf->vf_ops->get_q_vector(vsi, vector_id);
+ if (!q_vector)
+ goto error_param;
+
+ if (!ice_vc_isvalid_q_id(vsi, vsi_q_id))
+ goto error_param;
+
+ /* keep SW state in sync with HW, as ice_cfg_interrupt() does */
+ if (qv_map->queue_type == VIRTCHNL_QUEUE_TYPE_RX) {
+ struct ice_rx_ring *ring = vsi->rx_rings[vsi_q_id];
+
+ if (ring->q_vector && ring->q_vector->num_ring_rx)
+ ring->q_vector->num_ring_rx--;
+ ring->q_vector = q_vector;
+ q_vector->num_ring_rx++;
+ q_vector->rx.itr_idx = qv_map->itr_idx;
+ ice_cfg_rxq_interrupt(vsi, vsi_q_id,
+ q_vector->vf_reg_idx,
+ qv_map->itr_idx);
+ } else if (qv_map->queue_type == VIRTCHNL_QUEUE_TYPE_TX) {
+ struct ice_tx_ring *ring = vsi->tx_rings[vsi_q_id];
+
+ if (ring->q_vector && ring->q_vector->num_ring_tx)
+ ring->q_vector->num_ring_tx--;
+ ring->q_vector = q_vector;
+ q_vector->num_ring_tx++;
+ q_vector->tx.itr_idx = qv_map->itr_idx;
+ ice_cfg_txq_interrupt(vsi, vsi_q_id,
+ q_vector->vf_reg_idx,
+ qv_map->itr_idx);
+ }
+ }
+
+ v_ret = VIRTCHNL_STATUS_SUCCESS;
+
+error_param:
+ return ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_MAP_QUEUE_VECTOR,
+ v_ret, NULL, 0);
+}
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 12/15] devlink: give user option to allocate resources
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (10 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 11/15] ice: introduce handling of virtchnl LARGE VF opcodes Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 13/15] ice: represent RSS LUTs as devlink resources Przemek Kitszel
` (2 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Current devlink resources are designed as a thing that user could limit,
but there is not much otherwise that could be done with them.
Perhaps that's the reason there is not much adoption despite API being
there for multiple years.
Add new mode of operation, where user could allocate/assign resources
(from a common pool) to specific devices.
That requires a new .occ_set() driver callback, called on
DEVLINK_CMD_RESOURCE_SET with the requested DEVLINK_ATTR_RESOURCE_SIZE,
instead of staging the size until the next reload. Driver registers it
together with .occ_get() by devl_resource_occ_set_get_register(). For
such resources the getter value is reported as DEVLINK_ATTR_RESOURCE_SIZE,
as opposed to DEVLINK_ATTR_RESOURCE_OCC (current occupation) in the
"legacy" mode, and there is no DEVLINK_ATTR_RESOURCE_SIZE_NEW.
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
RFC, Feb '25
https://lore.kernel.org/intel-wired-lan/20250219164410.35665-3-przemyslaw.kitszel@intel.com
I have structured code to just choose "the mode" of operation based on
the presence of .occ_set callback, instead of naming the mode, what,
even if not the most clear code, avoids the need of naming the modes.
v4:
* return -EMSGSIZE directly on devlink_resource_occ_size_put_legacy()
failure (Jakub)
v3:
* keep size and size_new of .occ_set() resources in sync with the
getter, after a successful set and in devl_resource_size_get() (me)
* WARN on registering .occ_set() without .occ_get(), or vice versa
v2:
* avoid setting DEVLINK_ATTR_RESOURCE_SIZE_VALID for leaf resources
(Sashiko)
* minor kdoc update (Sashiko)
* add more WARN_ONs (Sashiko)
* document the occ_set mode in devlink-resource.rst
* commit message: describe the .occ_set() callback and how the reported
attributes differ from the "legacy" mode
---
.../networking/devlink/devlink-resource.rst | 5 +
include/net/devlink.h | 7 ++
net/devlink/resource.c | 97 ++++++++++++++++---
3 files changed, 93 insertions(+), 16 deletions(-)
diff --git a/Documentation/networking/devlink/devlink-resource.rst b/Documentation/networking/devlink/devlink-resource.rst
index b7172d94b2c6..9c48155fcb1e 100644
--- a/Documentation/networking/devlink/devlink-resource.rst
+++ b/Documentation/networking/devlink/devlink-resource.rst
@@ -75,6 +75,11 @@ attribute, which represents the pending change in size. For example:
Note that changes in resource size may require a device reload to properly
take effect.
+Resources registered with ``devl_resource_occ_set_get_register()`` behave
+differently: the new size is passed to the driver and applies immediately,
+there is neither ``size_new`` nor ``occ`` reported, and ``size`` is the value
+returned by the driver's getter.
+
Port-level Resources and Full Dump
==================================
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 98bdde4f72ca..d01128ef393d 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -420,6 +420,8 @@ devlink_resource_size_params_init(struct devlink_resource_size_params *size_para
}
typedef u64 devlink_resource_occ_get_t(void *priv);
+typedef int devlink_resource_occ_set_t(u64 size, struct netlink_ext_ack *extack,
+ void *priv);
#define DEVLINK_RESOURCE_ID_PARENT_TOP 0
@@ -1959,6 +1961,11 @@ void devl_resource_occ_get_register(struct devlink *devlink,
void *occ_get_priv);
void devl_resource_occ_get_unregister(struct devlink *devlink,
u64 resource_id);
+void devl_resource_occ_set_get_register(struct devlink *devlink,
+ u64 resource_id,
+ devlink_resource_occ_set_t *occ_set,
+ devlink_resource_occ_get_t *occ_get,
+ void *occ_priv);
int devl_params_register(struct devlink *devlink,
const struct devlink_param *params,
size_t params_count);
diff --git a/net/devlink/resource.c b/net/devlink/resource.c
index c3cfda7ea070..c4502c480a6b 100644
--- a/net/devlink/resource.c
+++ b/net/devlink/resource.c
@@ -19,7 +19,8 @@
* @list: parent list
* @resource_list: list of child resources
* @occ_get: occupancy getter callback
- * @occ_get_priv: occupancy getter callback priv
+ * @occ_set: occupancy setter callback
+ * @occ_priv: occupancy callbacks priv
*/
struct devlink_resource {
const char *name;
@@ -32,7 +33,8 @@ struct devlink_resource {
struct list_head list;
struct list_head resource_list;
devlink_resource_occ_get_t *occ_get;
- void *occ_get_priv;
+ devlink_resource_occ_set_t *occ_set;
+ void *occ_priv;
};
static struct devlink_resource *
@@ -137,6 +139,16 @@ int devlink_nl_resource_set_doit(struct sk_buff *skb, struct genl_info *info)
if (err)
return err;
+ if (resource->occ_set) {
+ err = resource->occ_set(size, info->extack, resource->occ_priv);
+ if (err)
+ return err;
+
+ resource->size = resource->occ_get(resource->occ_priv);
+ resource->size_new = resource->size;
+ return 0;
+ }
+
resource->size_new = size;
devlink_resource_validate_children(resource);
if (resource->parent)
@@ -162,13 +174,31 @@ devlink_resource_size_params_put(struct devlink_resource *resource,
return 0;
}
-static int devlink_resource_occ_put(struct devlink_resource *resource,
- struct sk_buff *skb)
+static int
+devlink_resource_occ_size_put_legacy(struct devlink_resource *resource,
+ struct sk_buff *skb)
{
- if (!resource->occ_get)
- return 0;
- return devlink_nl_put_u64(skb, DEVLINK_ATTR_RESOURCE_OCC,
- resource->occ_get(resource->occ_get_priv));
+ if (devlink_nl_put_u64(skb, DEVLINK_ATTR_RESOURCE_SIZE, resource->size))
+ return -EMSGSIZE;
+ if (resource->size != resource->size_new &&
+ devlink_nl_put_u64(skb, DEVLINK_ATTR_RESOURCE_SIZE_NEW,
+ resource->size_new))
+ return -EMSGSIZE;
+ if (resource->occ_get &&
+ devlink_nl_put_u64(skb, DEVLINK_ATTR_RESOURCE_OCC,
+ resource->occ_get(resource->occ_priv)))
+ return -EMSGSIZE;
+ return 0;
+}
+
+static int devlink_resource_occ_size_put(struct devlink_resource *resource,
+ struct sk_buff *skb)
+{
+ if (!resource->occ_get || !resource->occ_set)
+ return devlink_resource_occ_size_put_legacy(resource, skb);
+
+ return devlink_nl_put_u64(skb, DEVLINK_ATTR_RESOURCE_SIZE,
+ resource->occ_get(resource->occ_priv));
}
static int devlink_resource_put(struct devlink *devlink, struct sk_buff *skb,
@@ -183,14 +213,9 @@ static int devlink_resource_put(struct devlink *devlink, struct sk_buff *skb,
return -EMSGSIZE;
if (nla_put_string(skb, DEVLINK_ATTR_RESOURCE_NAME, resource->name) ||
- devlink_nl_put_u64(skb, DEVLINK_ATTR_RESOURCE_SIZE, resource->size) ||
devlink_nl_put_u64(skb, DEVLINK_ATTR_RESOURCE_ID, resource->id))
goto nla_put_failure;
- if (resource->size != resource->size_new &&
- devlink_nl_put_u64(skb, DEVLINK_ATTR_RESOURCE_SIZE_NEW,
- resource->size_new))
- goto nla_put_failure;
- if (devlink_resource_occ_put(resource, skb))
+ if (devlink_resource_occ_size_put(resource, skb))
goto nla_put_failure;
if (devlink_resource_size_params_put(resource, skb))
goto nla_put_failure;
@@ -634,12 +659,50 @@ int devl_resource_size_get(struct devlink *devlink,
resource = devlink_resource_find(devlink, NULL, resource_id);
if (!resource)
return -EINVAL;
+ if (resource->occ_set)
+ resource->size_new = resource->occ_get(resource->occ_priv);
*p_resource_size = resource->size_new;
resource->size = resource->size_new;
return 0;
}
EXPORT_SYMBOL_GPL(devl_resource_size_get);
+/**
+ * devl_resource_occ_set_get_register - register occupancy getter and setter
+ *
+ * @devlink: devlink
+ * @resource_id: resource id
+ * @occ_set: occupancy setter callback
+ * @occ_get: occupancy getter callback
+ * @occ_priv: occupancy callbacks priv
+ *
+ * Setter will be called when the user wants to change the resource size,
+ * getter is called to show the user what is current size of the resource.
+ */
+void devl_resource_occ_set_get_register(struct devlink *devlink,
+ u64 resource_id,
+ devlink_resource_occ_set_t *occ_set,
+ devlink_resource_occ_get_t *occ_get,
+ void *occ_priv)
+{
+ struct devlink_resource *resource;
+
+ lockdep_assert_held(&devlink->lock);
+
+ if (WARN_ON(!occ_set || !occ_get))
+ return;
+
+ resource = devlink_resource_find(devlink, NULL, resource_id);
+ if (WARN_ON(!resource))
+ return;
+ WARN_ON(resource->occ_get || resource->occ_set);
+
+ resource->occ_set = occ_set;
+ resource->occ_get = occ_get;
+ resource->occ_priv = occ_priv;
+}
+EXPORT_SYMBOL_GPL(devl_resource_occ_set_get_register);
+
/**
* devl_resource_occ_get_register - register occupancy getter
*
@@ -661,9 +724,10 @@ void devl_resource_occ_get_register(struct devlink *devlink,
if (WARN_ON(!resource))
return;
WARN_ON(resource->occ_get);
+ WARN_ON(resource->occ_set);
resource->occ_get = occ_get;
- resource->occ_get_priv = occ_get_priv;
+ resource->occ_priv = occ_get_priv;
}
EXPORT_SYMBOL_GPL(devl_resource_occ_get_register);
@@ -684,9 +748,10 @@ void devl_resource_occ_get_unregister(struct devlink *devlink,
if (WARN_ON(!resource))
return;
WARN_ON(!resource->occ_get);
+ WARN_ON(resource->occ_set);
resource->occ_get = NULL;
- resource->occ_get_priv = NULL;
+ resource->occ_priv = NULL;
}
EXPORT_SYMBOL_GPL(devl_resource_occ_get_unregister);
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 13/15] ice: represent RSS LUTs as devlink resources
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (11 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 12/15] devlink: give user option to allocate resources Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 14/15] devlink: add devlink_alloc_drv() Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 15/15] ice: support up to 256 VF queues Przemek Kitszel
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
E800 family offers three kinds of RSS LUTs: VSI LUT (sized 64), GLOBAL LUT
(sized 512), and PF LUT (sized 2048). Until now the GLOBAL kind was not
used at all. There are two possible usages for it, subsequent commit will
give VF option to acquire it, and this one enables PF to switch between PF
LUT and GLOBAL LUT - switching to smaller one is, again, to make it
possible for VF to then acquire the former one.
Devlink resources are used to let user show current usage and change the
allocation, see examples below.
Default state on 8-port card, asking for aggregate "whole device" usage,
note that there are as many PF LUTs as there is PFs, and, for e810, there
are 16 GLOBAL LUTs:
$ devlink resource show devlink_index/11
devlink_index/11:
name rss size 8 unit entry size_min 0 size_max 24 size_gran 1 dpipe_tables none
resources:
name lut_512 size 0 unit entry size_min 0 size_max 16 size_gran 1 dpipe_tables none
name lut_2048 size 8 unit entry size_min 0 size_max 8 size_gran 1 dpipe_tables none
Now let's add GLOBAL LUT for a single PF (on one-port NIC):
$ sudo devlink resource set pci/0000:18:00.0 path rss/lut_512 size 1
And show it's resources after that:
$ devlink resource show pci/0000:18:00.0
pci/0000:18:00.0:
name rss size 2 unit entry size_min 0 size_max 2 size_gran 1 dpipe_tables none
resources:
name lut_512 size 1 unit entry size_min 0 size_max 1 size_gran 1 dpipe_tables none
name lut_2048 size 1 unit entry size_min 0 size_max 1 size_gran 1 dpipe_tables none
Let's take the PF LUT out of that PF afterwards:
$ sudo devlink resource set pci/0000:18:00.0 path rss/lut_2048 size 0
now `ethtool -x $ifacename` will report smaller RSS table.
Changing the LUT of a PF resets its RSS indirection table to the default
spread, as the table size changes, and the ethtool core is told that
the user table is lost. The PF LUT change is done under rtnl, as
the ethtool core sizes its RSS buffers by the table size under rtnl.
FW drops RSS LUT assignments on reset, so the driver gives the PF its PF
LUT back after each reset, and logs when that discards a devlink change.
Changes are rejected with -EBUSY while a reset is in progress.
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v5:
* set the default rss_size on every VSI config again, after the queue
counts are known, as before this series; the RSS LUT type and size
are still reset on INIT only; a NO_INIT rebuild left rss_size
clamped (ADQ removal, sriov_vf_msix_count), and new ADQ channel
VSIs got rss_size of 1 (Sashiko)
* keep the NAC topology in safe mode caps, secondary NAC PFs collided
with primary ones in the devlink RSS LUT table (Sashiko)
* fix the kernel-doc opener of ice_rss_lut_resource_state(), the whole
comment was ignored (me)
* assert rss_lut_lock in ice_vsi_update_rss_lut() (me)
v4:
* take rtnl when devlink changes the RSS LUT of the PF, the new LUT is
filled based on NETIF_F_RXHASH, which raced with ndo_set_features
(Sashiko)
* drop the -EAGAIN checks from get/set_rxfh, they broke ethtool -L after
ethtool -X, and were hit after the hash function and key were already
written (Sashiko); the PF RSS LUT size now changes only under rtnl,
also on reset (me)
* tell the ethtool core when the user RSS LUT is dropped, on devlink LUT
change and on reset (Sashiko)
* reject devlink RSS LUT changes while a reset is in progress, they
could leak a global LUT (Sashiko)
* log when a reset discards a devlink RSS LUT assignment (Sashiko)
* drop global RSS LUTs of the PF from the table already in
ice_prepare_for_reset(), FW frees them on reset and could give them
to other PFs before our rebuild cleaned the table, that hit the WARN
in ice_devl_res_take() when LUT changes raced resets of other PFs (me)
* taking adapter devl there also waits for devlink RSS LUT changes that
checked for a reset just before it was requested, they could still
leak a global LUT (Sashiko)
* keep the specific extack message and the real errno when a LUT change
fails (Sashiko)
* declare rel_err at function scope, a declaration right after a label
does not build with GCC < 11 and breaks sparse (Jakub)
* RDMA filter toggle: take the Q_OPT section from vsi->info under
rss_lut_lock, the cached VSI context misses RSS LUT changes (Sashiko)
* document the devlink resources in ice.rst (Sashiko)
* use scoped_guard() instead of guard()
v3:
* don't free the user RSS LUT in ice_vsi_decfg(), it was done outside
of pf->rss_lut_lock, and is redundant anyway: ice_vsi_cfg_def() frees
it under the lock when the LUT size changes (Sashiko)
* fill the new RSS LUT under rss_lut_lock (me)
* keep PF RSS LUT zeroed on change when RXHASH is disabled (me)
* restore default RSS LUT assignment of PF after reset, FW drops it
* dual-NAC: each NAC numbers its PFs from 0, offset secondary NAC PFs by
ICE_PORTS_PER_QUAD in the devlink resource table, reserve
2 * ICE_PORTS_PER_QUAD PF LUT slots
* dual-NAC: FW of each NAC numbers its global LUTs from 0, keep a separate
global LUT table for the secondary NAC
* register PF devlink resources only in the regular probe path, not in
FW recovery mode, where pf->adapter is NULL
* restore VF aggregator node accounting in ice_vsi_decfg(), removed by
mistake (me)
* drop PF RSS LUT ownership also in ice_deinit_pf(), ice_vsi_decfg()
skips it after a NO_INIT VSI rebuild (me)
* drop a stray debug change of the error message in ice_get_rss_lut()
v2:
* replace -ENOANO with proper WARN_ON for "assumption error" (Alex)
* fix goto label (Sashiko)
* avoid queues being 0 in ADQ mode (Sashiko)
* undo ice_take_rss_lut_pf() on error path (Sashiko)
* program the PF VSI context with the new LUT type and global LUT ID
after writing the LUT, so the PF really switches to the new table;
publish the global LUT ID to the VSI only after a successful switch
* drop the cached user RSS LUT when the RSS table size changes, on
devlink LUT switch and on reset
* use logical_pf_id for all PF bookkeeping of devlink resources, as the
PF LUT resource is sized by the number of functions
* warn when FW fails to free a global RSS LUT
* add pf->rss_lut_lock to serialize devlink RSS LUT switch with
ethtool -x/-X/-L, RXHASH toggle and VSI RSS config
* return -EAGAIN from get/set_rxfh when the LUT was resized after
the core sized the indirection table
* reject ADQ when RSS LUTs are reassigned via devlink, and reject
devlink RSS LUT changes when ADQ is configured (-EOPNOTSUPP)
* remove trailing whitespace
* commit message: GLOBAL LUT is sized 512, not 256
---
Documentation/networking/devlink/ice.rst | 58 ++
drivers/net/ethernet/intel/ice/Makefile | 1 +
.../net/ethernet/intel/ice/devlink/resource.h | 25 +
drivers/net/ethernet/intel/ice/ice.h | 3 +-
drivers/net/ethernet/intel/ice/ice_adapter.h | 43 ++
drivers/net/ethernet/intel/ice/ice_common.h | 1 +
drivers/net/ethernet/intel/ice/ice_lib.h | 3 +
drivers/net/ethernet/intel/ice/ice_switch.h | 3 +-
.../net/ethernet/intel/ice/devlink/resource.c | 663 ++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_adapter.c | 19 +-
drivers/net/ethernet/intel/ice/ice_common.c | 5 +-
drivers/net/ethernet/intel/ice/ice_ethtool.c | 79 ++-
drivers/net/ethernet/intel/ice/ice_idc.c | 25 +-
drivers/net/ethernet/intel/ice/ice_lib.c | 150 +++-
drivers/net/ethernet/intel/ice/ice_main.c | 27 +-
drivers/net/ethernet/intel/ice/ice_switch.c | 11 +-
16 files changed, 1030 insertions(+), 86 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ice/devlink/resource.h
create mode 100644 drivers/net/ethernet/intel/ice/devlink/resource.c
diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
index 792e9f8c846a..6e34c107b28d 100644
--- a/Documentation/networking/devlink/ice.rst
+++ b/Documentation/networking/devlink/ice.rst
@@ -485,3 +485,61 @@ and arbitration among them is based on assigned weights.
# assign tx_share to the VF
$ devlink port function rate set pci/0000:4b:00.0/2 tx_share 500Mbps
+
+Resources
+=========
+
+The ``ice`` driver exposes the RSS lookup tables (LUTs) of the device as
+devlink resources. E800 devices have three kinds of RSS LUTs: a VSI LUT
+(64 entries) for each VSI, 16 global LUTs (512 entries) for each NAC of the
+device, and a PF LUT (2048 entries) for each PF. By default each PF uses its
+PF LUT.
+
+.. list-table:: Resources implemented
+ :widths: 15 85
+
+ * - Name
+ - Description
+ * - ``rss``
+ - The sum of its children, read-only.
+ * - ``rss/lut_512``
+ - The number of global LUTs owned.
+ * - ``rss/lut_2048``
+ - The number of PF LUTs owned.
+
+The resources are registered on two kinds of devlink instances:
+
+- The devlink instance of each PF, where ``lut_512`` and ``lut_2048`` are
+ either 0 or 1. They are changed with ``devlink resource set``, and the
+ change applies immediately.
+- The whole device devlink instance, shared by all PFs of the device, which
+ reports the total usage. Its resources are read-only, setting them fails
+ with ``EPERM``.
+
+A PF must own at least one of the two LUTs at all times, so switching a PF
+to a global LUT takes two steps: take a global LUT, then release the PF LUT.
+A PF that owns both uses its PF LUT.
+
+Changing the LUT of a PF resets its RSS indirection table to the default
+spread, a table set with ``ethtool -X`` is lost.
+
+RSS LUTs can't be changed while ADQ is configured, and ADQ can't be
+configured while the RSS LUTs of the PF are reassigned.
+
+Firmware drops the LUT assignments on reset (PFR, CORER or GLOBR). The
+driver then restores the default assignment, and logs that it did so.
+Changes are rejected with ``EBUSY`` while a reset is in progress.
+
+.. code:: shell
+
+ # whole device usage on an 8-port card
+ $ devlink resource show devlink_index/11
+ devlink_index/11:
+ name rss size 8 unit entry size_min 0 size_max 24 size_gran 1 dpipe_tables none
+ resources:
+ name lut_512 size 0 unit entry size_min 0 size_max 16 size_gran 1 dpipe_tables none
+ name lut_2048 size 8 unit entry size_min 0 size_max 8 size_gran 1 dpipe_tables none
+
+ # switch a PF to a global LUT
+ $ devlink resource set pci/0000:18:00.0 path rss/lut_512 size 1
+ $ devlink resource set pci/0000:18:00.0 path rss/lut_2048 size 0
diff --git a/drivers/net/ethernet/intel/ice/Makefile b/drivers/net/ethernet/intel/ice/Makefile
index a952bacb7ec7..e50e02efbf7a 100644
--- a/drivers/net/ethernet/intel/ice/Makefile
+++ b/drivers/net/ethernet/intel/ice/Makefile
@@ -34,6 +34,7 @@ ice-y := ice_main.o \
devlink/devlink.o \
devlink/health.o \
devlink/port.o \
+ devlink/resource.o \
ice_sf_eth.o \
ice_sf_vsi_vlan_ops.o \
ice_ddp.o \
diff --git a/drivers/net/ethernet/intel/ice/devlink/resource.h b/drivers/net/ethernet/intel/ice/devlink/resource.h
new file mode 100644
index 000000000000..a41bda0ff6c8
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/devlink/resource.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2026, Intel Corporation. */
+
+#ifndef _ICE_DEVL_RESOURCE_H_
+#define _ICE_DEVL_RESOURCE_H_
+
+#include <linux/types.h>
+
+struct devlink;
+struct ice_adapter;
+struct ice_hw;
+struct ice_pf;
+
+void ice_devl_pf_resources_register(struct ice_pf *pf);
+void ice_devl_whole_dev_resources_register(const struct ice_hw *hw,
+ struct ice_adapter *adapter);
+
+bool ice_rss_lut_is_reassigned(struct ice_pf *pf);
+int ice_take_rss_lut_pf(struct ice_pf *pf);
+void ice_release_rss_lut_pf(struct ice_pf *pf);
+void ice_prepare_rss_luts_for_reset(struct ice_pf *pf);
+int ice_rebuild_rss_luts(struct ice_pf *pf);
+void ice_free_rss_lut_flr(struct ice_pf *pf);
+
+#endif /* _ICE_DEVL_RESOURCE_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index fc2e84136e24..611f1346b4a7 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -596,6 +596,7 @@ struct ice_pf {
struct mutex tc_mutex; /* lock to protect TC changes */
struct mutex adev_mutex; /* lock to protect aux device access */
struct mutex lag_mutex; /* protect ice_lag struct in PF */
+ struct mutex rss_lut_lock; /* protect VSI RSS LUT config */
u32 msg_enable;
struct ice_ptp ptp;
struct gnss_serial *gnss_serial;
@@ -1113,7 +1114,7 @@ extern const struct xdp_metadata_ops ice_xdp_md_ops;
* Acceleration Complex) configuration variant, false otherwise
* (always false for non-E825 devices).
*/
-static inline bool ice_is_dual(struct ice_hw *hw)
+static inline bool ice_is_dual(const struct ice_hw *hw)
{
return hw->mac_type == ICE_MAC_GENERIC_3K_E825 &&
(hw->dev_caps.nac_topo.mode & ICE_NAC_TOPO_DUAL_M);
diff --git a/drivers/net/ethernet/intel/ice/ice_adapter.h b/drivers/net/ethernet/intel/ice/ice_adapter.h
index 698a5b9af82f..e1bddcec464e 100644
--- a/drivers/net/ethernet/intel/ice/ice_adapter.h
+++ b/drivers/net/ethernet/intel/ice/ice_adapter.h
@@ -16,6 +16,40 @@
struct pci_dev;
struct ice_pf;
+enum ice_devl_resource_id {
+ /* keep parent IDs prior to children, because we register in order */
+ ICE_TOP_RESOURCE = DEVLINK_RESOURCE_ID_PARENT_TOP,
+ ICE_RSS_LUT_BOTH,
+ ICE_RSS_LUT_GLOBAL,
+ ICE_RSS_LUT_PF,
+ ICE_DEVL_RESOURCES_COUNT
+};
+
+#define ICE_MAX_DEVL_RESOURCE_UNITS 16
+
+/**
+ * struct ice_devl_resource - driver data for devlink resource, config & runtime
+ *
+ * @owner: entity that owns given resource (like ptr to ice_pf or ice_vf)
+ * @pf_id: on which PF the VF is (or just PF id when PF is the owner)
+ * @name: name of the resource to register it with
+ * @get: occ getter callback
+ * @set: occ setter callback
+ * @start_size: starting size of the resource
+ * @max_size: max size of the resource, to present in the uAPI/validate against
+ * @parent_id: ID of the parent resource
+ */
+struct ice_devl_resource {
+ void *owner[ICE_MAX_DEVL_RESOURCE_UNITS];
+ u8 pf_id[ICE_MAX_DEVL_RESOURCE_UNITS];
+ const char *name;
+ devlink_resource_occ_get_t *get;
+ devlink_resource_occ_set_t *set;
+ u32 start_size;
+ u32 max_size;
+ u32 parent_id;
+};
+
/**
* struct ice_port_list - data used to store the list of adapter ports
*
@@ -46,6 +80,9 @@ struct ice_port_list {
* which all PFs reach. Used on E825C devices.
* @ctrl_pf: Control PF of the adapter
* @ports: Ports list
+ * @resources: array of ice's data for devlink resources
+ * @sec_nac_global_luts: global RSS LUTs of the secondary NAC, its FW numbers
+ * them from 0 too
*/
struct ice_adapter {
struct devlink *devlink;
@@ -61,9 +98,15 @@ struct ice_adapter {
struct ice_pf *ctrl_pf;
struct ice_port_list ports;
+
+ struct ice_devl_resource resources[ICE_DEVL_RESOURCES_COUNT];
+ struct ice_devl_resource sec_nac_global_luts;
};
struct ice_adapter *ice_adapter_get(struct pci_dev *pdev);
void ice_adapter_put(struct ice_adapter *adapter);
+DEFINE_GUARD(ice_adapter_devl, struct ice_adapter *,
+ devl_lock((_T)->devlink), devl_unlock((_T)->devlink))
+
#endif /* _ICE_ADAPTER_H */
diff --git a/drivers/net/ethernet/intel/ice/ice_common.h b/drivers/net/ethernet/intel/ice/ice_common.h
index d1d674ca644f..d019ec759eda 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.h
+++ b/drivers/net/ethernet/intel/ice/ice_common.h
@@ -125,6 +125,7 @@ int ice_read_txq_ctx(struct ice_hw *hw, struct ice_tlan_ctx *tlan_ctx,
int ice_write_txq_ctx(struct ice_hw *hw, struct ice_tlan_ctx *tlan_ctx,
u32 txq_index);
+enum ice_lut_size ice_lut_type_to_size(enum ice_lut_type type);
int
ice_aq_get_rss_lut(struct ice_hw *hw, struct ice_aq_get_set_rss_lut_params *get_params);
int
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 49454d98dcfe..15b65a757beb 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -47,6 +47,9 @@ int ice_vsi_cfg_tc(struct ice_vsi *vsi, u8 ena_tc);
int ice_vsi_cfg_rss_lut_key(struct ice_vsi *vsi);
+int ice_vsi_update_rss_lut(struct ice_vsi *vsi, enum ice_lut_type lut_type,
+ u8 global_lut_id);
+
void ice_vsi_cfg_netdev_tc(struct ice_vsi *vsi, u8 ena_tc);
struct ice_vsi *
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.h b/drivers/net/ethernet/intel/ice/ice_switch.h
index 99b593b68bd0..991b61b0d26a 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.h
+++ b/drivers/net/ethernet/intel/ice/ice_switch.h
@@ -361,7 +361,8 @@ int ice_remove_mac(struct ice_hw *hw, struct list_head *m_lst);
bool ice_vlan_fltr_exist(struct ice_hw *hw, u16 vlan_id, u16 vsi_handle);
int ice_add_eth_mac(struct ice_hw *hw, struct list_head *em_list);
int ice_remove_eth_mac(struct ice_hw *hw, struct list_head *em_list);
-int ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable);
+int ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle,
+ const struct ice_aqc_vsi_props *info, bool enable);
void ice_remove_vsi_fltr(struct ice_hw *hw, u16 vsi_handle);
/* Promisc/defport setup for VSIs */
diff --git a/drivers/net/ethernet/intel/ice/devlink/resource.c b/drivers/net/ethernet/intel/ice/devlink/resource.c
new file mode 100644
index 000000000000..de9c13e928fa
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/devlink/resource.c
@@ -0,0 +1,663 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026, Intel Corporation. */
+
+#include "resource.h"
+#include "ice_adapter.h"
+#include "ice.h"
+#include "ice_lib.h"
+
+#define ICE_NUM_GLOBAL_LUTS 16
+#define ICE_ANY_SLOT -1
+
+/* each NAC numbers its PFs from 0, secondary NAC ones go after primary ones */
+static int ice_devl_pf_id(struct ice_pf *pf)
+{
+ struct ice_hw *hw = &pf->hw;
+
+ if (ice_is_primary(hw))
+ return hw->logical_pf_id;
+
+ return hw->logical_pf_id + ICE_PORTS_PER_QUAD;
+}
+
+/* each NAC has its own global LUTs, with IDs given by its FW */
+static struct ice_devl_resource *
+ice_devl_res_tbl(struct ice_pf *pf, enum ice_devl_resource_id res_id)
+{
+ if (res_id == ICE_RSS_LUT_GLOBAL && !ice_is_primary(&pf->hw))
+ return &pf->adapter->sec_nac_global_luts;
+
+ return &pf->adapter->resources[res_id];
+}
+
+static u32 ice_devl_res_cnt(const struct ice_devl_resource *res)
+{
+ u32 sum = 0;
+
+ for (int i = 0; i < ARRAY_SIZE(res->owner); i++)
+ sum += res->owner[i] != NULL;
+
+ return sum;
+}
+
+static int ice_devl_res_take(struct ice_pf *pf,
+ enum ice_devl_resource_id res_id, int slot,
+ void *owner)
+{
+ struct ice_devl_resource *res = ice_devl_res_tbl(pf, res_id);
+ int beg, end, new_id = ICE_ANY_SLOT;
+ u16 lut_id;
+
+ if (res_id == ICE_RSS_LUT_GLOBAL) {
+ int err;
+
+ WARN_ON_ONCE(slot != ICE_ANY_SLOT);
+
+ err = ice_alloc_rss_global_lut(&pf->hw, &lut_id);
+ if (err)
+ return err;
+
+ slot = lut_id;
+ }
+
+ end = slot == ICE_ANY_SLOT ? ARRAY_SIZE(res->owner) : slot + 1;
+ beg = slot == ICE_ANY_SLOT ? 0 : slot;
+ for (int id = beg; id < end; id++) {
+ if (!res->owner[id]) {
+ new_id = id;
+ break;
+ }
+ }
+ if (new_id == ICE_ANY_SLOT) {
+ if (WARN_ON(res_id == ICE_RSS_LUT_GLOBAL))
+ ice_free_rss_global_lut(&pf->hw, lut_id);
+
+ return -ENOSPC;
+ }
+
+ res->owner[new_id] = owner;
+ res->pf_id[new_id] = ice_devl_pf_id(pf);
+ return new_id;
+}
+
+static int ice_devl_res_free(struct ice_pf *pf,
+ enum ice_devl_resource_id res_id, void *owner)
+{
+ struct ice_devl_resource *res = ice_devl_res_tbl(pf, res_id);
+ int err = 0, id_to_free = ICE_ANY_SLOT;
+
+ for (int i = 0; i < ARRAY_SIZE(res->owner); i++) {
+ if (res->owner[i] == owner) {
+ id_to_free = i;
+ break;
+ }
+ }
+ if (id_to_free == ICE_ANY_SLOT)
+ return 0;
+
+ if (res_id == ICE_RSS_LUT_GLOBAL) {
+ err = ice_free_rss_global_lut(&pf->hw, id_to_free);
+ if (err)
+ dev_warn(ice_pf_to_dev(pf), "failed to free global RSS LUT %d, leaked until PF reset: %d\n",
+ id_to_free, err);
+ }
+
+ res->owner[id_to_free] = NULL;
+ return err;
+}
+
+static void ice_free_rss_lut_flr_locked(struct ice_pf *pf)
+{
+ struct ice_devl_resource *res, *resources = pf->adapter->resources;
+ int pf_id = ice_devl_pf_id(pf);
+
+ devl_assert_locked(pf->adapter->devlink);
+ resources[ICE_RSS_LUT_PF].owner[pf_id] = NULL;
+
+ res = ice_devl_res_tbl(pf, ICE_RSS_LUT_GLOBAL);
+ for (int i = 0; i < ARRAY_SIZE(res->owner); i++) {
+ /* On FLR/PFR resources assigned to PF are cleared by FW,
+ * reflect that in the SW table.
+ * VFs on given PF must be de-programmed too.
+ */
+ if (res->pf_id[i] == pf_id)
+ res->owner[i] = NULL;
+ }
+}
+
+void ice_free_rss_lut_flr(struct ice_pf *pf)
+{
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ ice_free_rss_lut_flr_locked(pf);
+}
+
+static int ice_devl_res_owned_idx(struct ice_pf *pf,
+ enum ice_devl_resource_id res_id, void *owner)
+{
+ const struct ice_devl_resource *res = ice_devl_res_tbl(pf, res_id);
+
+ for (int i = 0; i < ARRAY_SIZE(res->owner); i++) {
+ if (res->owner[i] == owner)
+ return i;
+ }
+
+ return -ENXIO;
+}
+
+static bool ice_is_devl_res_owned_by(struct ice_pf *pf,
+ enum ice_devl_resource_id res_id,
+ void *owner)
+{
+ return ice_devl_res_owned_idx(pf, res_id, owner) >= 0;
+}
+
+static u64 ice_rss_lut_whole_dev_occ_get_global(void *priv)
+{
+ struct ice_adapter *adapter = priv;
+
+ return ice_devl_res_cnt(&adapter->resources[ICE_RSS_LUT_GLOBAL]) +
+ ice_devl_res_cnt(&adapter->sec_nac_global_luts);
+}
+
+static u64 ice_rss_lut_whole_dev_occ_get_pf(void *priv)
+{
+ struct ice_adapter *adapter = priv;
+
+ return ice_devl_res_cnt(&adapter->resources[ICE_RSS_LUT_PF]);
+}
+
+static u64 ice_rss_lut_whole_dev_occ_get_both(void *priv)
+{
+ return ice_rss_lut_whole_dev_occ_get_global(priv) +
+ ice_rss_lut_whole_dev_occ_get_pf(priv);
+}
+
+static u64 ice_rss_lut_pf_occ_get_global(void *priv)
+{
+ struct ice_pf *pf = priv;
+
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ return ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_GLOBAL, pf);
+}
+
+static u64 ice_rss_lut_pf_occ_get_pf(void *priv)
+{
+ struct ice_pf *pf = priv;
+
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ return ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_PF, pf);
+}
+
+static u64 ice_rss_lut_pf_occ_get_both(void *priv)
+{
+ struct ice_pf *pf = priv;
+
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ return ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_PF, pf) +
+ ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_GLOBAL, pf);
+}
+
+static int ice_devl_resource_deny_occ_set(u64 size,
+ struct netlink_ext_ack *extack,
+ void *priv)
+{
+ NL_SET_ERR_MSG_MOD(extack,
+ "can not change directly, parent/aggregate resource just adds up children data");
+ return -EPERM;
+}
+
+enum ice_rss_lut_resource_state {
+ ICE_HAS_NO_LUT = 0,
+ ICE_HAS_GLOBAL_LUT = BIT(ICE_RSS_LUT_GLOBAL),
+ ICE_HAS_PF_LUT = BIT(ICE_RSS_LUT_PF),
+ ICE_HAS_BOTH_LUTS = ICE_HAS_GLOBAL_LUT | ICE_HAS_PF_LUT,
+};
+
+/**
+ * ice_rss_lut_resource_state - compute opaque resource state for given owner
+ * @pf: the PF the @owner is on
+ * @owner: the entity to compute state of resources for
+ *
+ * Return: computed current state of the RSS resources the @owner has.
+ */
+static enum ice_rss_lut_resource_state
+ice_rss_lut_resource_state(struct ice_pf *pf, void *owner)
+{
+ enum ice_rss_lut_resource_state ret = ICE_HAS_NO_LUT;
+
+ if (ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_GLOBAL, owner))
+ ret |= ICE_HAS_GLOBAL_LUT;
+ if (ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_PF, owner))
+ ret |= ICE_HAS_PF_LUT;
+
+ return ret;
+}
+
+static int ice_maybe_change_rss_lut(struct ice_pf *pf, void *owner,
+ enum ice_rss_lut_resource_state old,
+ enum ice_rss_lut_resource_state new,
+ struct netlink_ext_ack *extack)
+{
+ struct ice_aq_get_set_rss_lut_params params = {};
+ struct ice_hw *hw = &pf->hw;
+ enum ice_lut_type lut_type;
+ int err, lut_size, lut_id;
+ struct ice_vsi *vsi;
+ u8 *lut;
+
+ if (old & new & ICE_HAS_PF_LUT)
+ return 0;
+
+ if (new & ICE_HAS_PF_LUT) {
+ lut_type = ICE_LUT_PF;
+ } else if (new & ICE_HAS_GLOBAL_LUT) {
+ lut_id = ice_devl_res_owned_idx(pf, ICE_RSS_LUT_GLOBAL, owner);
+ if (lut_id < 0)
+ return lut_id;
+
+ lut_type = ICE_LUT_GLOBAL;
+ params.global_lut_id = lut_id;
+ } else {
+ lut_type = ICE_LUT_VSI;
+ if (owner == pf) {
+ NL_SET_ERR_MSG_FMT(extack, "cannot change PF to use LUT_VSI (sized 64)");
+ return -EDOM;
+ }
+ }
+
+ if (pf == owner) {
+ vsi = ice_get_main_vsi(pf);
+ } else {
+ return -EOPNOTSUPP;
+ }
+
+ lut_size = ice_lut_type_to_size(lut_type);
+ lut = kzalloc(lut_size, GFP_KERNEL);
+ if (!lut)
+ return -ENOMEM;
+ params.lut = lut;
+ params.lut_size = lut_size;
+ params.lut_type = lut_type;
+ params.vsi_handle = vsi->idx;
+ mutex_lock(&pf->rss_lut_lock);
+ /* PF with RXHASH disabled keeps its LUT zeroed */
+ if (!vsi->netdev || vsi->netdev->features & NETIF_F_RXHASH)
+ ice_fill_rss_lut(lut, lut_size, vsi->rss_size);
+ err = ice_aq_set_rss_lut(hw, ¶ms);
+ if (err) {
+ NL_SET_ERR_MSG_FMT(extack, "AQ failed: %s", libie_aq_str(hw->adminq.sq_last_status));
+ goto out;
+ }
+
+ err = ice_vsi_update_rss_lut(vsi, lut_type, params.global_lut_id);
+ if (err)
+ goto out;
+
+ devm_kfree(ice_pf_to_dev(pf), vsi->rss_lut_user);
+ vsi->rss_lut_user = NULL;
+ if (lut_type == ICE_LUT_GLOBAL)
+ vsi->global_lut_id = params.global_lut_id;
+ vsi->rss_table_size = lut_size;
+ vsi->rss_lut_type = lut_type;
+out:
+ mutex_unlock(&pf->rss_lut_lock);
+ kfree(lut);
+ /* the user RSS LUT was dropped above, rtnl is held for the PF */
+ if (!err && vsi->netdev && netif_is_rxfh_configured(vsi->netdev))
+ ethtool_rxfh_indir_lost(vsi->netdev);
+ return err;
+}
+
+static int ice_devl_res_change(bool take, enum ice_devl_resource_id res_id,
+ struct ice_pf *pf, void *owner, int slot,
+ struct netlink_ext_ack *extack)
+{
+ enum ice_rss_lut_resource_state old, new, change;
+ int err, rel_err;
+
+ change = BIT(res_id);
+ old = ice_rss_lut_resource_state(pf, owner);
+ new = old;
+ if (take)
+ new |= change;
+ else
+ new &= ~change;
+ if (new == old)
+ return 0;
+
+ /* reset rebuilds the table and the VSIs, see ice_rebuild_rss_luts(),
+ * pairs with adapter devl taken in ice_prepare_rss_luts_for_reset()
+ */
+ if (ice_is_reset_in_progress(pf->state)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "reset in progress, try again later");
+ return -EBUSY;
+ }
+
+ /* pairs with set_bit() under adapter devl in ice_setup_tc_mqprio_qdisc() */
+ if (test_bit(ICE_FLAG_TC_MQPRIO, pf->flags)) {
+ NL_SET_ERR_MSG_MOD(extack, "ADQ configured, can't change RSS LUTs");
+ return -EOPNOTSUPP;
+ }
+
+ if (pf == owner && !take && old != ICE_HAS_BOTH_LUTS) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "at least one of 512+ sized LUTs must be assigned to PF device at all times");
+ return -EDOM;
+ }
+
+ if (take) {
+ int slot_id;
+
+ slot_id = ice_devl_res_take(pf, res_id, slot, owner);
+ if (slot_id < 0)
+ return slot_id;
+ }
+
+ err = ice_maybe_change_rss_lut(pf, owner, old, new, extack);
+ if (err) {
+ NL_SET_ERR_MSG_WEAK_MOD(extack, "failed to change RSS LUT");
+ if (!take)
+ /* We have not released the resource, so we don't free
+ * it, to avoid freeing LUT still in use or assigning it
+ * to other entity.
+ */
+ return err;
+
+ goto undo_res_take;
+ }
+
+ if (!take) {
+undo_res_take:
+ rel_err = ice_devl_res_free(pf, res_id, owner);
+ if (rel_err) {
+ NL_SET_ERR_MSG_FMT(extack,
+ "could not free resource, err: %d", rel_err);
+ if (!err)
+ err = rel_err;
+ }
+ }
+
+ return err;
+}
+
+/* ethtool core sizes RSS buffers by rss_table_size under rtnl, and the new
+ * LUT is filled based on NETIF_F_RXHASH, which is also changed under rtnl
+ */
+static int ice_rss_lut_pf_occ_set(u64 size, enum ice_devl_resource_id res_id,
+ struct ice_pf *pf, int slot,
+ struct netlink_ext_ack *extack)
+{
+ int err;
+
+ rtnl_lock();
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ err = ice_devl_res_change(size, res_id, pf, pf, slot, extack);
+ rtnl_unlock();
+
+ return err;
+}
+
+static int ice_rss_lut_pf_occ_set_pf(u64 size, struct netlink_ext_ack *extack,
+ void *priv)
+{
+ struct ice_pf *pf = priv;
+
+ return ice_rss_lut_pf_occ_set(size, ICE_RSS_LUT_PF, pf,
+ ice_devl_pf_id(pf), extack);
+}
+
+static int ice_rss_lut_pf_occ_set_global(u64 size,
+ struct netlink_ext_ack *extack,
+ void *priv)
+{
+ return ice_rss_lut_pf_occ_set(size, ICE_RSS_LUT_GLOBAL, priv,
+ ICE_ANY_SLOT, extack);
+}
+
+/**
+ * ice_rss_lut_is_reassigned - check if RSS LUTs of PF are in non-default state
+ * @pf: the PF to check
+ *
+ * Return: true if @pf lost its PF LUT, or @pf or any of its VFs owns a
+ * global LUT. Caller must hold the adapter devlink lock.
+ */
+bool ice_rss_lut_is_reassigned(struct ice_pf *pf)
+{
+ struct ice_devl_resource *res, *resources = pf->adapter->resources;
+ int pf_id = ice_devl_pf_id(pf);
+
+ devl_assert_locked(pf->adapter->devlink);
+ if (resources[ICE_RSS_LUT_PF].owner[pf_id] != pf)
+ return true;
+
+ res = ice_devl_res_tbl(pf, ICE_RSS_LUT_GLOBAL);
+ for (int i = 0; i < ARRAY_SIZE(res->owner); i++)
+ if (res->owner[i] && res->pf_id[i] == pf_id)
+ return true;
+
+ return false;
+}
+
+/**
+ * ice_take_rss_lut_pf - allocate PF RSS LUT for PF
+ * @pf: the PF device that PF LUT is physically on, and to allocate it for
+ *
+ * Acquire PF RSS LUT for the caller.
+ *
+ * Return: 0 on success, negative on error.
+ */
+int ice_take_rss_lut_pf(struct ice_pf *pf)
+{
+ int ret, pf_id = ice_devl_pf_id(pf);
+
+ scoped_guard(ice_adapter_devl, pf->adapter) {
+ ret = ice_devl_res_take(pf, ICE_RSS_LUT_PF, pf_id, pf);
+ if (ret >= 0)
+ return 0;
+
+ return ret;
+ }
+}
+
+/**
+ * ice_release_rss_lut_pf - release PF RSS LUT of PF
+ * @pf: the PF device that PF LUT is physically on, and to release it from
+ *
+ * Release PF RSS LUT for the caller.
+ *
+ * No need to call this function in normal operation, provided for error path.
+ */
+void ice_release_rss_lut_pf(struct ice_pf *pf)
+{
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ ice_devl_res_free(pf, ICE_RSS_LUT_PF, pf);
+}
+
+/**
+ * ice_prepare_rss_luts_for_reset - drop global RSS LUTs of PF from the table
+ * @pf: the PF that is about to be reset
+ *
+ * FW frees global RSS LUTs of the PF and its VFs on reset, and other PFs may
+ * get them from FW before ice_rebuild_rss_luts() of @pf, so drop them from
+ * the table now. Taking the lock also waits for RSS LUT changes that checked
+ * for reset just before it was requested.
+ */
+void ice_prepare_rss_luts_for_reset(struct ice_pf *pf)
+{
+ int pf_id = ice_devl_pf_id(pf);
+ struct ice_devl_resource *res;
+
+ scoped_guard(ice_adapter_devl, pf->adapter) {
+ if (ice_rss_lut_is_reassigned(pf))
+ dev_info(ice_pf_to_dev(pf),
+ "reset restores the default RSS LUT assignment\n");
+
+ res = ice_devl_res_tbl(pf, ICE_RSS_LUT_GLOBAL);
+ for (int i = 0; i < ARRAY_SIZE(res->owner); i++)
+ if (res->pf_id[i] == pf_id)
+ res->owner[i] = NULL;
+ }
+}
+
+/**
+ * ice_rebuild_rss_lut_pf - restore default RSS LUT assignment after reset
+ * @pf: the PF that was reset
+ *
+ * FW drops RSS LUT assignments of the PF on reset, so give the PF LUT back to
+ * the PF, and make the PF VSI use it.
+ * Must be called before the VSIs are rebuilt.
+ *
+ * Return: 0 on success, negative on error.
+ */
+static int ice_rebuild_rss_lut_pf(struct ice_pf *pf)
+{
+ struct ice_vsi *vsi = ice_get_main_vsi(pf);
+ struct ice_adapter *adapter = pf->adapter;
+ int err;
+
+ devl_lock(adapter->devlink);
+ ice_free_rss_lut_flr_locked(pf);
+ err = ice_devl_res_take(pf, ICE_RSS_LUT_PF, ice_devl_pf_id(pf), pf);
+ if (err < 0)
+ goto unlock;
+
+ mutex_lock(&pf->rss_lut_lock);
+ if (vsi && vsi->rss_lut_type != ICE_LUT_PF) {
+ devm_kfree(ice_pf_to_dev(pf), vsi->rss_lut_user);
+ vsi->rss_lut_user = NULL;
+ vsi->rss_table_size = ice_lut_type_to_size(ICE_LUT_PF);
+ vsi->rss_lut_type = ICE_LUT_PF;
+ }
+ mutex_unlock(&pf->rss_lut_lock);
+ err = 0;
+unlock:
+ devl_unlock(adapter->devlink);
+
+ return err;
+}
+
+/**
+ * ice_rebuild_rss_luts - restore default RSS LUT assignment after reset
+ * @pf: the PF that was reset
+ *
+ * Wrapper of ice_rebuild_rss_lut_pf() that also tells the ethtool core when
+ * the user RSS LUT was dropped. Must be called without rtnl held.
+ *
+ * Return: 0 on success, negative on error.
+ */
+int ice_rebuild_rss_luts(struct ice_pf *pf)
+{
+ struct ice_vsi *vsi = ice_get_main_vsi(pf);
+ int err;
+
+ /* ethtool core sizes RSS buffers by rss_table_size under rtnl */
+ rtnl_lock();
+ err = ice_rebuild_rss_lut_pf(pf);
+ /* the user RSS LUT is dropped when the PF goes back to the PF LUT */
+ if (vsi && vsi->netdev && !vsi->rss_lut_user &&
+ netif_is_rxfh_configured(vsi->netdev))
+ ethtool_rxfh_indir_lost(vsi->netdev);
+ rtnl_unlock();
+
+ return err;
+}
+
+static void ice_devl_res_register(struct devlink *devlink,
+ struct ice_devl_resource *resources,
+ void *occ_priv)
+{
+ struct devlink_resource_size_params size_params;
+
+ devlink_resource_size_params_init(&size_params, 0, 0, 1,
+ DEVLINK_RESOURCE_UNIT_ENTRY);
+ for (int i = 0; i < ICE_DEVL_RESOURCES_COUNT; i++) {
+ struct ice_devl_resource *res = &resources[i];
+ int err, resource_id = i;
+
+ if (!res->name)
+ continue; /* skip empty entries in config table */
+
+ size_params.size_max = res->max_size;
+ err = devl_resource_register(devlink, res->name,
+ res->start_size, resource_id,
+ res->parent_id, &size_params);
+ if (WARN_ONCE(err, "not all resource handlers registered, err: %d, resname: %s\n",
+ err, res->name))
+ break;
+
+ devl_resource_occ_set_get_register(devlink, resource_id,
+ res->set, res->get, occ_priv);
+ }
+}
+
+void ice_devl_whole_dev_resources_register(const struct ice_hw *hw,
+ struct ice_adapter *adapter)
+{
+ struct devlink *devlink = adapter->devlink;
+ int global_lut_cnt = ICE_NUM_GLOBAL_LUTS;
+ int pf_lut_cnt = hw->dev_caps.num_funcs;
+
+ devl_assert_locked(devlink);
+ /* see ice_devl_pf_id() and ice_devl_res_tbl() */
+ if (ice_is_dual(hw)) {
+ global_lut_cnt = 2 * ICE_NUM_GLOBAL_LUTS;
+ pf_lut_cnt = 2 * ICE_PORTS_PER_QUAD;
+ }
+
+ adapter->resources[ICE_RSS_LUT_GLOBAL] = (struct ice_devl_resource) {
+ .name = "lut_512",
+ .parent_id = ICE_RSS_LUT_BOTH,
+ .max_size = global_lut_cnt,
+ .get = ice_rss_lut_whole_dev_occ_get_global,
+ .set = ice_devl_resource_deny_occ_set,
+ };
+ adapter->resources[ICE_RSS_LUT_PF] = (struct ice_devl_resource) {
+ .name = "lut_2048",
+ .parent_id = ICE_RSS_LUT_BOTH,
+ .max_size = pf_lut_cnt,
+ .get = ice_rss_lut_whole_dev_occ_get_pf,
+ .set = ice_devl_resource_deny_occ_set,
+ };
+ adapter->resources[ICE_RSS_LUT_BOTH] = (struct ice_devl_resource) {
+ .name = "rss",
+ .parent_id = ICE_TOP_RESOURCE,
+ .max_size = pf_lut_cnt + global_lut_cnt,
+ .get = ice_rss_lut_whole_dev_occ_get_both,
+ .set = ice_devl_resource_deny_occ_set,
+ };
+
+ ice_devl_res_register(devlink, adapter->resources, adapter);
+}
+
+void ice_devl_pf_resources_register(struct ice_pf *pf)
+{
+ struct ice_devl_resource pf_resources[ICE_DEVL_RESOURCES_COUNT] = {
+ [ICE_RSS_LUT_GLOBAL] = {
+ .name = "lut_512",
+ .parent_id = ICE_RSS_LUT_BOTH,
+ .max_size = 1,
+ .get = ice_rss_lut_pf_occ_get_global,
+ .set = ice_rss_lut_pf_occ_set_global,
+ },
+ [ICE_RSS_LUT_PF] = {
+ .name = "lut_2048",
+ .parent_id = ICE_RSS_LUT_BOTH,
+ .max_size = 1,
+ .get = ice_rss_lut_pf_occ_get_pf,
+ .set = ice_rss_lut_pf_occ_set_pf,
+ .start_size = 1,
+ },
+ [ICE_RSS_LUT_BOTH] = {
+ .name = "rss",
+ .parent_id = ICE_TOP_RESOURCE,
+ .max_size = 2,
+ .get = ice_rss_lut_pf_occ_get_both,
+ .set = ice_devl_resource_deny_occ_set,
+ },
+ };
+ struct devlink *devlink = priv_to_devlink(pf);
+
+ devl_assert_locked(devlink);
+ ice_devl_res_register(devlink, pf_resources, pf);
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_adapter.c b/drivers/net/ethernet/intel/ice/ice_adapter.c
index 8cb61f145d27..1d12af48f065 100644
--- a/drivers/net/ethernet/intel/ice/ice_adapter.c
+++ b/drivers/net/ethernet/intel/ice/ice_adapter.c
@@ -9,6 +9,8 @@
#include "ice_adapter.h"
#include "ice.h"
+#include "devlink/resource.h"
+
#define ICE_ADAPTER_FIXED_INDEX BIT_ULL(63)
#define ICE_ADAPTER_INDEX_E825C \
@@ -38,8 +40,10 @@ static u64 ice_adapter_index(struct pci_dev *pdev)
static int ice_adapter_init(void *priv, void *init_param)
{
+ const struct ice_hw *hw = init_param;
struct ice_adapter *adapter = priv;
struct devlink *devlink;
+ int err;
devlink = shd_priv_to_devlink(adapter);
adapter->devlink = devlink;
@@ -53,12 +57,22 @@ static int ice_adapter_init(void *priv, void *init_param)
spin_lock_init(&adapter->ports.lock);
INIT_LIST_HEAD(&adapter->ports.list);
- return init_srcu_struct(&adapter->ports.srcu);
+ err = init_srcu_struct(&adapter->ports.srcu);
+ if (err)
+ return err;
+
+ ice_devl_whole_dev_resources_register(hw, adapter);
+
+ return 0;
}
static void ice_adapter_fini(void *priv)
{
struct ice_adapter *adapter = priv;
+ struct devlink *devlink;
+
+ devlink = shd_priv_to_devlink(adapter);
+ devl_resources_unregister(devlink);
WARN_ON(!list_empty(&adapter->ports.list));
for (int i = 0; i < ARRAY_SIZE(adapter->cpi_phy_lock); i++)
@@ -88,15 +102,16 @@ static const struct devlink_ops ice_adapter_devlink_ops = {
*/
struct ice_adapter *ice_adapter_get(struct pci_dev *pdev)
{
+ struct ice_pf *pf = pci_get_drvdata(pdev);
struct ice_adapter *adapter;
struct devlink *devlink;
char devlink_id[32];
u64 index;
index = ice_adapter_index(pdev);
snprintf(devlink_id, sizeof(devlink_id), "ice:%llx", index);
devlink = devlink_shd_get(devlink_id, &ice_adapter_devlink_ops,
- sizeof(*adapter), NULL, pdev->dev.driver);
+ sizeof(*adapter), &pf->hw, pdev->dev.driver);
if (IS_ERR(devlink))
return ERR_CAST(devlink);
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index 90e01eb49ea2..48c3449a2434 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -3184,6 +3184,7 @@ void ice_set_safe_mode_caps(struct ice_hw *hw)
struct ice_hw_func_caps *func_caps = &hw->func_caps;
struct ice_hw_dev_caps *dev_caps = &hw->dev_caps;
struct ice_hw_common_caps cached_caps;
+ struct ice_nac_topology nac_topo;
u32 num_funcs;
/* cache some func_caps values that should be restored after memset */
@@ -3217,6 +3218,7 @@ void ice_set_safe_mode_caps(struct ice_hw *hw)
/* cache some dev_caps values that should be restored after memset */
cached_caps = dev_caps->common_cap;
num_funcs = dev_caps->num_funcs;
+ nac_topo = dev_caps->nac_topo;
/* unset dev capabilities */
memset(dev_caps, 0, sizeof(*dev_caps));
@@ -3235,6 +3237,7 @@ void ice_set_safe_mode_caps(struct ice_hw *hw)
ICE_RESTORE_DEV_CAP(nvm_update_pending_orom);
ICE_RESTORE_DEV_CAP(nvm_update_pending_netlist);
dev_caps->num_funcs = num_funcs;
+ dev_caps->nac_topo = nac_topo;
/* one Tx and one Rx queue per function in safe mode */
dev_caps->common_cap.num_rxq = num_funcs;
@@ -4450,7 +4453,7 @@ ice_aq_sff_eeprom(struct ice_hw *hw, u16 lport, u8 bus_addr,
return status;
}
-static enum ice_lut_size ice_lut_type_to_size(enum ice_lut_type type)
+enum ice_lut_size ice_lut_type_to_size(enum ice_lut_type type)
{
switch (type) {
case ICE_LUT_VSI:
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index dffa213734f1..822b94f87156 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -3645,11 +3645,13 @@ ice_get_rxfh(struct net_device *netdev, struct ethtool_rxfh_param *rxfh)
if (!rxfh->indir)
return 0;
- lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
- if (!lut)
- return -ENOMEM;
+ scoped_guard(mutex, &pf->rss_lut_lock) {
+ lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
+ if (!lut)
+ return -ENOMEM;
- err = ice_get_rss(vsi, rxfh->key, lut, vsi->rss_table_size);
+ err = ice_get_rss(vsi, rxfh->key, lut, vsi->rss_table_size);
+ }
if (err)
goto out;
@@ -3707,7 +3709,9 @@ ice_set_rxfh(struct net_device *netdev, struct ethtool_rxfh_param *rxfh,
if (rxfh->input_xfrm & RXH_XFRM_SYM_XOR)
hfunc = ICE_AQ_VSI_Q_OPT_RSS_HASH_SYM_TPLZ;
- err = ice_set_rss_hfunc(vsi, hfunc);
+ /* q_opt_rss also carries the RSS LUT type, that devlink may change */
+ scoped_guard(mutex, &pf->rss_lut_lock)
+ err = ice_set_rss_hfunc(vsi, hfunc);
if (err)
return err;
@@ -3727,29 +3731,31 @@ ice_set_rxfh(struct net_device *netdev, struct ethtool_rxfh_param *rxfh,
return err;
}
- if (!vsi->rss_lut_user) {
- vsi->rss_lut_user = devm_kzalloc(dev, vsi->rss_table_size,
- GFP_KERNEL);
- if (!vsi->rss_lut_user)
- return -ENOMEM;
- }
+ scoped_guard(mutex, &pf->rss_lut_lock) {
+ if (!vsi->rss_lut_user) {
+ vsi->rss_lut_user = devm_kzalloc(dev,
+ vsi->rss_table_size,
+ GFP_KERNEL);
+ if (!vsi->rss_lut_user)
+ return -ENOMEM;
+ }
- /* Each 32 bits pointed by 'indir' is stored with a lut entry */
- if (rxfh->indir) {
- int i;
+ /* Each 32 bits pointed by 'indir' is stored with a lut entry */
+ if (rxfh->indir) {
+ int i;
- for (i = 0; i < vsi->rss_table_size; i++)
- vsi->rss_lut_user[i] = (u8)(rxfh->indir[i]);
- } else {
- ice_fill_rss_lut(vsi->rss_lut_user, vsi->rss_table_size,
- vsi->rss_size);
- }
+ for (i = 0; i < vsi->rss_table_size; i++)
+ vsi->rss_lut_user[i] = (u8)(rxfh->indir[i]);
+ } else {
+ ice_fill_rss_lut(vsi->rss_lut_user, vsi->rss_table_size,
+ vsi->rss_size);
+ }
- err = ice_set_rss_lut(vsi, vsi->rss_lut_user, vsi->rss_table_size);
- if (err)
- return err;
+ err = ice_set_rss_lut(vsi, vsi->rss_lut_user,
+ vsi->rss_table_size);
+ }
- return 0;
+ return err;
}
static int
@@ -3856,19 +3862,22 @@ static int ice_vsi_set_dflt_rss_lut(struct ice_vsi *vsi, int req_rss_size)
if (!req_rss_size)
return -EINVAL;
- lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
- if (!lut)
- return -ENOMEM;
+ scoped_guard(mutex, &pf->rss_lut_lock) {
+ lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
+ if (!lut)
+ return -ENOMEM;
- /* set RSS LUT parameters */
- if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags))
- vsi->rss_size = 1;
- else
- vsi->rss_size = ice_get_valid_rss_size(hw, req_rss_size);
+ /* set RSS LUT parameters */
+ if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags))
+ vsi->rss_size = 1;
+ else
+ vsi->rss_size = ice_get_valid_rss_size(hw,
+ req_rss_size);
- /* create/set RSS LUT */
- ice_fill_rss_lut(lut, vsi->rss_table_size, vsi->rss_size);
- err = ice_set_rss_lut(vsi, lut, vsi->rss_table_size);
+ /* create/set RSS LUT */
+ ice_fill_rss_lut(lut, vsi->rss_table_size, vsi->rss_size);
+ err = ice_set_rss_lut(vsi, lut, vsi->rss_table_size);
+ }
if (err)
dev_err(dev, "Cannot set RSS lut, err %d aq_err %s\n", err,
libie_aq_str(hw->adminq.sq_last_status));
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index 102d63c3018b..5187916f37be 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -207,15 +207,22 @@ int ice_rdma_update_vsi_filter(struct iidc_rdma_core_dev_info *cdev,
if (!vsi)
return -EINVAL;
- status = ice_cfg_rdma_fltr(&pf->hw, vsi->idx, enable);
- if (status) {
- dev_err(ice_pf_to_dev(pf), "Failed to %sable RDMA filtering\n",
- enable ? "en" : "dis");
- } else {
- if (enable)
- vsi->info.q_opt_flags |= ICE_AQ_VSI_Q_OPT_PE_FLTR_EN;
- else
- vsi->info.q_opt_flags &= ~ICE_AQ_VSI_Q_OPT_PE_FLTR_EN;
+ /* q_opt_rss also carries the RSS LUT type, that devlink may change */
+ scoped_guard(mutex, &pf->rss_lut_lock) {
+ status = ice_cfg_rdma_fltr(&pf->hw, vsi->idx, &vsi->info,
+ enable);
+ if (status) {
+ dev_err(ice_pf_to_dev(pf),
+ "Failed to %sable RDMA filtering\n",
+ enable ? "en" : "dis");
+ } else {
+ if (enable)
+ vsi->info.q_opt_flags |=
+ ICE_AQ_VSI_Q_OPT_PE_FLTR_EN;
+ else
+ vsi->info.q_opt_flags &=
+ ~ICE_AQ_VSI_Q_OPT_PE_FLTR_EN;
+ }
}
return status;
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index cbdebb7e1750..ff16d19a8c27 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -10,6 +10,8 @@
#include "ice_type.h"
#include "ice_vsi_vlan_ops.h"
+#include "devlink/resource.h"
+
/**
* ice_vsi_type_str - maps VSI type enum to string equivalents
* @vsi_type: VSI type enum
@@ -982,6 +984,37 @@ static void ice_rss_clean(struct ice_vsi *vsi)
ice_rem_vsi_rss_list(&pf->hw, vsi->idx);
}
+/**
+ * ice_vsi_set_dflt_rss_lut_type - Setup default RSS LUT per VSI type
+ * @vsi: the VSI being configured
+ */
+static void ice_vsi_set_dflt_rss_lut_type(struct ice_vsi *vsi)
+{
+ struct ice_hw_common_caps *cap = &vsi->back->hw.func_caps.common_cap;
+
+ if (!test_bit(ICE_FLAG_RSS_ENA, vsi->back->flags))
+ return;
+
+ switch (vsi->type) {
+ case ICE_VSI_CHNL:
+ case ICE_VSI_PF:
+ /* PF VSI will inherit RSS instance of PF */
+ vsi->rss_table_size = (u16)cap->rss_table_size;
+ vsi->rss_lut_type = ICE_LUT_PF;
+ break;
+ case ICE_VSI_SF:
+ case ICE_VSI_VF:
+ /* VF and SF VSIs will get a small RSS table.
+ * For VSI_LUT, LUT size should be set to 64 bytes.
+ */
+ vsi->rss_table_size = ICE_LUT_VSI_SIZE;
+ vsi->rss_lut_type = ICE_LUT_VSI;
+ break;
+ default:
+ break;
+ }
+}
+
/**
* ice_vsi_set_rss_params - Setup RSS capabilities per VSI type
* @vsi: the VSI being configured
@@ -1002,29 +1035,19 @@ static void ice_vsi_set_rss_params(struct ice_vsi *vsi)
switch (vsi->type) {
case ICE_VSI_CHNL:
case ICE_VSI_PF:
- /* PF VSI will inherit RSS instance of PF */
- vsi->rss_table_size = (u16)cap->rss_table_size;
if (vsi->type == ICE_VSI_CHNL)
vsi->rss_size = min_t(u16, vsi->num_rxq, max_rss_size);
else
vsi->rss_size = min_t(u16,
netif_get_num_default_rss_queues(),
max_rss_size);
- vsi->rss_lut_type = ICE_LUT_PF;
break;
case ICE_VSI_SF:
- vsi->rss_table_size = ICE_LUT_VSI_SIZE;
vsi->rss_size = min_t(u16, netif_get_num_default_rss_queues(),
max_rss_size);
- vsi->rss_lut_type = ICE_LUT_VSI;
break;
case ICE_VSI_VF:
- /* VF VSI will get a small RSS table.
- * For VSI_LUT, LUT size should be set to 64 bytes.
- */
- vsi->rss_table_size = ICE_LUT_VSI_SIZE;
vsi->rss_size = ICE_MAX_QS_PER_VF;
- vsi->rss_lut_type = ICE_LUT_VSI;
break;
case ICE_VSI_LB:
break;
@@ -1303,6 +1326,34 @@ static void ice_set_rss_vsi_ctx(struct ice_vsi_ctx *ctxt, struct ice_vsi *vsi)
FIELD_PREP(ICE_AQ_VSI_Q_OPT_RSS_HASH_M, hash_type);
}
+int ice_vsi_update_rss_lut(struct ice_vsi *vsi, enum ice_lut_type lut_type,
+ u8 global_lut_id)
+{
+ struct ice_vsi_ctx *ctx;
+ int err;
+
+ lockdep_assert_held(&vsi->back->rss_lut_lock);
+
+ ctx = kzalloc_obj(*ctx);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->info.valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_Q_OPT_VALID);
+ ctx->info.q_opt_rss = vsi->info.q_opt_rss &
+ ~(ICE_AQ_VSI_Q_OPT_RSS_LUT_M | ICE_AQ_VSI_Q_OPT_RSS_GBL_LUT_M);
+ ctx->info.q_opt_rss |= FIELD_PREP(ICE_AQ_VSI_Q_OPT_RSS_LUT_M,
+ ice_lut_type_to_aq_qopt_rss_val(lut_type)) |
+ FIELD_PREP(ICE_AQ_VSI_Q_OPT_RSS_GBL_LUT_M, global_lut_id);
+ ctx->info.q_opt_tc = vsi->info.q_opt_tc;
+ ctx->info.q_opt_flags = vsi->info.q_opt_flags;
+
+ err = ice_update_vsi(&vsi->back->hw, vsi->idx, ctx, NULL);
+ if (!err)
+ vsi->info.q_opt_rss = ctx->info.q_opt_rss;
+ kfree(ctx);
+ return err;
+}
+
static void
ice_chnl_vsi_setup_q_map(struct ice_vsi *vsi, struct ice_vsi_ctx *ctxt)
{
@@ -1578,19 +1629,22 @@ void ice_vsi_manage_rss_lut(struct ice_vsi *vsi, bool ena)
{
u8 *lut;
- lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
- if (!lut)
- return;
+ scoped_guard(mutex, &vsi->back->rss_lut_lock) {
+ lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
+ if (!lut)
+ return;
- if (ena) {
- if (vsi->rss_lut_user)
- memcpy(lut, vsi->rss_lut_user, vsi->rss_table_size);
- else
- ice_fill_rss_lut(lut, vsi->rss_table_size,
- vsi->rss_size);
- }
+ if (ena) {
+ if (vsi->rss_lut_user)
+ memcpy(lut, vsi->rss_lut_user,
+ vsi->rss_table_size);
+ else
+ ice_fill_rss_lut(lut, vsi->rss_table_size,
+ vsi->rss_size);
+ }
- ice_set_rss_lut(vsi, lut, vsi->rss_table_size);
+ ice_set_rss_lut(vsi, lut, vsi->rss_table_size);
+ }
kfree(lut);
}
@@ -1642,16 +1696,19 @@ int ice_vsi_cfg_rss_lut_key(struct ice_vsi *vsi)
}
}
- lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
- if (!lut)
- return -ENOMEM;
+ scoped_guard(mutex, &pf->rss_lut_lock) {
+ lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
+ if (!lut)
+ return -ENOMEM;
- if (vsi->rss_lut_user)
- memcpy(lut, vsi->rss_lut_user, vsi->rss_table_size);
- else
- ice_fill_rss_lut(lut, vsi->rss_table_size, vsi->rss_size);
+ if (vsi->rss_lut_user)
+ memcpy(lut, vsi->rss_lut_user, vsi->rss_table_size);
+ else
+ ice_fill_rss_lut(lut, vsi->rss_table_size,
+ vsi->rss_size);
- err = ice_set_rss_lut(vsi, lut, vsi->rss_table_size);
+ err = ice_set_rss_lut(vsi, lut, vsi->rss_table_size);
+ }
if (err) {
dev_err(dev, "set_rss_lut failed, error %d\n", err);
goto ice_vsi_cfg_rss_exit;
@@ -2424,6 +2481,18 @@ static int ice_vsi_cfg_def(struct ice_vsi *vsi)
vsi->vsw = pf->first_sw;
+ if (vsi->flags & ICE_VSI_FLAG_INIT) {
+ scoped_guard(mutex, &pf->rss_lut_lock) {
+ u16 lut_size = vsi->rss_table_size;
+
+ ice_vsi_set_dflt_rss_lut_type(vsi);
+ if (vsi->rss_table_size != lut_size) {
+ devm_kfree(dev, vsi->rss_lut_user);
+ vsi->rss_lut_user = NULL;
+ }
+ }
+ }
+
ret = ice_vsi_alloc_def(vsi, vsi->ch);
if (ret)
return ret;
@@ -2444,14 +2513,23 @@ static int ice_vsi_cfg_def(struct ice_vsi *vsi)
/* set RSS capabilities */
ice_vsi_set_rss_params(vsi);
+ if ((vsi->flags & ICE_VSI_FLAG_INIT) && vsi->type == ICE_VSI_PF) {
+ ret = ice_take_rss_lut_pf(pf);
+ if (ret) {
+ dev_err(dev, "Failed to allocate RSS LUT for PF: %d\n",
+ ret);
+ goto unroll_get_qs;
+ }
+ }
/* set TC configuration */
ice_vsi_set_tc_cfg(vsi);
- /* create the VSI */
- ret = ice_vsi_init(vsi, vsi->flags);
+ /* create the VSI, its context carries RSS LUT type and global LUT id */
+ scoped_guard(mutex, &pf->rss_lut_lock)
+ ret = ice_vsi_init(vsi, vsi->flags);
if (ret)
- goto unroll_get_qs;
+ goto unroll_pf_rss_lut;
ice_vsi_init_vlan_ops(vsi);
@@ -2563,6 +2641,9 @@ static int ice_vsi_cfg_def(struct ice_vsi *vsi)
ice_vsi_free_q_vectors(vsi);
unroll_vsi_init:
ice_vsi_delete_from_hw(vsi);
+unroll_pf_rss_lut:
+ if ((vsi->flags & ICE_VSI_FLAG_INIT) && vsi->type == ICE_VSI_PF)
+ ice_release_rss_lut_pf(pf);
unroll_get_qs:
ice_vsi_put_qs(vsi);
unroll_vsi_alloc_stat:
@@ -2640,6 +2721,11 @@ void ice_vsi_decfg(struct ice_vsi *vsi)
if (vsi->type == ICE_VSI_VF &&
vsi->agg_node && vsi->agg_node->valid)
vsi->agg_node->num_vsis--;
+
+ if (vsi->flags & ICE_VSI_FLAG_INIT) {
+ if (vsi->type == ICE_VSI_PF)
+ ice_free_rss_lut_flr(pf);
+ }
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index c8f06c3d0fdf..357dc09a0ade 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -15,6 +15,7 @@
#include "ice_dcb_nl.h"
#include "devlink/devlink.h"
#include "devlink/port.h"
+#include "devlink/resource.h"
#include "ice_sf_eth.h"
#include "ice_hwmon.h"
/* Including ice_trace.h with CREATE_TRACE_POINTS defined will generate the
@@ -546,6 +547,8 @@ ice_prepare_for_reset(struct ice_pf *pf, enum ice_reset_req reset_type)
if (test_bit(ICE_PREPARED_FOR_RESET, pf->state))
return;
+ ice_prepare_rss_luts_for_reset(pf);
+
synchronize_irq(pf->oicr_irq.virq);
ice_unplug_aux_dev(pf);
@@ -3873,6 +3876,9 @@ void ice_deinit_pf(struct ice_pf *pf)
{
/* note that we unroll also on ice_init_pf() failure here */
+ /* ice_vsi_decfg() skips it if the last VSI rebuild was NO_INIT */
+ ice_free_rss_lut_flr(pf);
+ mutex_destroy(&pf->rss_lut_lock);
mutex_destroy(&pf->lag_mutex);
mutex_destroy(&pf->adev_mutex);
mutex_destroy(&pf->sw_mutex);
@@ -3978,6 +3984,7 @@ int ice_init_pf(struct ice_pf *pf)
mutex_init(&pf->tc_mutex);
mutex_init(&pf->adev_mutex);
mutex_init(&pf->lag_mutex);
+ mutex_init(&pf->rss_lut_lock);
INIT_HLIST_HEAD(&pf->aq_wait_list);
spin_lock_init(&pf->aq_wait_lock);
@@ -4962,6 +4969,7 @@ static void ice_deinit_devlink(struct ice_pf *pf)
ice_devlink_unregister(pf);
ice_devlink_destroy_regions(pf);
ice_devlink_unregister_params(pf);
+ devl_resources_unregister(priv_to_devlink(pf));
}
static int ice_init(struct ice_pf *pf)
@@ -5283,6 +5291,7 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
err = ice_init_devlink(pf);
if (err)
goto unroll_load;
+ ice_devl_pf_resources_register(pf);
devl_unlock(priv_to_devlink(pf));
return 0;
@@ -7768,6 +7777,12 @@ static void ice_rebuild(struct ice_pf *pf, enum ice_reset_req reset_type)
if (ice_is_feature_supported(pf, ICE_F_GNSS))
ice_gnss_init(pf);
+ err = ice_rebuild_rss_luts(pf);
+ if (err) {
+ dev_err(dev, "RSS LUT rebuild failed: %d\n", err);
+ goto err_vsi_rebuild;
+ }
+
/* rebuild PF VSI */
err = ice_vsi_rebuild_by_type(pf, ICE_VSI_PF);
if (err) {
@@ -7925,6 +7940,8 @@ int ice_set_rss_lut(struct ice_vsi *vsi, u8 *lut, u16 lut_size)
params.lut_size = lut_size;
params.lut_type = vsi->rss_lut_type;
params.lut = lut;
+ if (params.lut_type == ICE_LUT_GLOBAL)
+ params.global_lut_id = vsi->global_lut_id;
status = ice_aq_set_rss_lut(hw, ¶ms);
if (status)
@@ -7978,6 +7995,8 @@ int ice_get_rss_lut(struct ice_vsi *vsi, u8 *lut, u16 lut_size)
params.lut_size = lut_size;
params.lut_type = vsi->rss_lut_type;
params.lut = lut;
+ if (params.lut_type == ICE_LUT_GLOBAL)
+ params.global_lut_id = vsi->global_lut_id;
status = ice_aq_get_rss_lut(hw, ¶ms);
if (status)
@@ -9192,8 +9211,14 @@ static int ice_setup_tc_mqprio_qdisc(struct net_device *netdev, void *type_data)
ret);
return ret;
}
+ scoped_guard(ice_adapter_devl, pf->adapter) {
+ if (ice_rss_lut_is_reassigned(pf)) {
+ dev_err(dev, "RSS LUTs reassigned via devlink, can't configure ADQ\n");
+ return -EOPNOTSUPP;
+ }
+ set_bit(ICE_FLAG_TC_MQPRIO, pf->flags);
+ }
memcpy(&vsi->mqprio_qopt, mqprio_qopt, sizeof(*mqprio_qopt));
- set_bit(ICE_FLAG_TC_MQPRIO, pf->flags);
/* don't assume state of hw_tc_offload during driver load
* and set the flag for TC flower filter if hw_tc_offload
* already ON
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.c b/drivers/net/ethernet/intel/ice/ice_switch.c
index 567b85bc5a39..c9a03baf38df 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.c
+++ b/drivers/net/ethernet/intel/ice/ice_switch.c
@@ -1875,10 +1875,12 @@ ice_update_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
* ice_cfg_rdma_fltr - enable/disable RDMA filtering on VSI
* @hw: pointer to HW struct
* @vsi_handle: VSI SW index
+ * @info: current VSI properties, the rest of the Q_OPT section is kept
* @enable: boolean for enable/disable
*/
int
-ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable)
+ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle,
+ const struct ice_aqc_vsi_props *info, bool enable)
{
struct ice_vsi_ctx *ctx, *cached_ctx;
int status;
@@ -1891,9 +1893,10 @@ ice_cfg_rdma_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable)
if (!ctx)
return -ENOMEM;
- ctx->info.q_opt_rss = cached_ctx->info.q_opt_rss;
- ctx->info.q_opt_tc = cached_ctx->info.q_opt_tc;
- ctx->info.q_opt_flags = cached_ctx->info.q_opt_flags;
+ /* not from the cached ctx, that misses RSS LUT changes */
+ ctx->info.q_opt_rss = info->q_opt_rss;
+ ctx->info.q_opt_tc = info->q_opt_tc;
+ ctx->info.q_opt_flags = info->q_opt_flags;
ctx->info.valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_Q_OPT_VALID);
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 14/15] devlink: add devlink_alloc_drv()
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (12 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 13/15] ice: represent RSS LUTs as devlink resources Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 15/15] ice: support up to 256 VF queues Przemek Kitszel
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel,
Jiri Pirko
devlink_alloc() records the driver bound to @dev at the time of the
call as the owner of the new instance. That fits the common case of
a driver creating an instance for the device it is bound to, but not
a PF driver that creates instances for its VFs, as the VF device could
be:
* not bound to any driver yet (VF driver not loaded, or
sriov_drivers_autoprobe disabled), what triggers the WARN_ON() in
__devlink_alloc();
* bound to another driver (VF driver, or vfio-pci), which could be
unloaded while the instance is still registered, leaving the stored
driver pointer dangling (it is used to report the driver name in
devlink dev info).
Add devlink_alloc_drv(), a variant of devlink_alloc() that takes the
owning driver explicitly, as devlink_shd_get() already does.
The first user is ice, in the next patch, which creates a devlink
instance for each of its VFs.
Such an instance reports the owning driver in devlink dev info, not the
one bound to the device in sysfs. Document that in devlink-info.rst.
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v5:
* document @drv lifetime and device lock rules in kernel-doc (me)
v4:
* document in devlink-info.rst that an instance created for another
device reports the name of the creating driver (Sashiko)
v3:
* new patch
---
.../networking/devlink/devlink-info.rst | 2 ++
include/net/devlink.h | 4 ++++
net/devlink/core.c | 24 +++++++++++++++++++
3 files changed, 30 insertions(+)
diff --git a/Documentation/networking/devlink/devlink-info.rst b/Documentation/networking/devlink/devlink-info.rst
index dd6adc4d0559..7ac5625e2934 100644
--- a/Documentation/networking/devlink/devlink-info.rst
+++ b/Documentation/networking/devlink/devlink-info.rst
@@ -25,6 +25,8 @@ versions is generally discouraged - here, and via any other Linux API.
- Description
* - ``driver``
- Name of the currently used device driver, also available through sysfs.
+ For an instance that a driver creates for another device, e.g. a PF
+ driver for its VFs, the name of the creating driver.
* - ``serial_number``
- Serial number of the device.
diff --git a/include/net/devlink.h b/include/net/devlink.h
index d01128ef393d..5c90edf9a92d 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -1693,6 +1693,10 @@ static inline struct devlink *devlink_alloc(const struct devlink_ops *ops,
return devlink_alloc_ns(ops, priv_size, &init_net, dev);
}
+struct devlink *devlink_alloc_drv(const struct devlink_ops *ops,
+ size_t priv_size, struct device *dev,
+ const struct device_driver *drv);
+
int devl_register(struct devlink *devlink);
void devl_unregister(struct devlink *devlink);
void devlink_register(struct devlink *devlink);
diff --git a/net/devlink/core.c b/net/devlink/core.c
index 9b844e0c1bd2..e2a7169287ec 100644
--- a/net/devlink/core.c
+++ b/net/devlink/core.c
@@ -531,6 +531,30 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops,
}
EXPORT_SYMBOL_GPL(devlink_alloc_ns);
+/**
+ * devlink_alloc_drv - Allocate new devlink instance resources for given driver
+ * @ops: ops
+ * @priv_size: size of user private data
+ * @dev: parent device
+ * @drv: driver that owns the instance
+ *
+ * Variant of devlink_alloc() for a driver that creates an instance for
+ * a device bound to another (or no) driver, e.g. a PF driver for its VFs.
+ *
+ * @drv must not be NULL and must outlive the instance, only a pointer to it
+ * is kept. Operations that take the device lock, like reload, lock @dev,
+ * not a device bound to @drv.
+ *
+ * Return: new devlink instance, NULL on failure.
+ */
+struct devlink *devlink_alloc_drv(const struct devlink_ops *ops,
+ size_t priv_size, struct device *dev,
+ const struct device_driver *drv)
+{
+ return __devlink_alloc(ops, priv_size, &init_net, dev, drv);
+}
+EXPORT_SYMBOL_GPL(devlink_alloc_drv);
+
/**
* devlink_free - Free devlink instance resources
*
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [net-next v6 15/15] ice: support up to 256 VF queues
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
` (13 preceding siblings ...)
2026-11-17 9:44 ` [net-next v6 14/15] devlink: add devlink_alloc_drv() Przemek Kitszel
@ 2026-11-17 9:44 ` Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
14 siblings, 1 reply; 25+ messages in thread
From: Przemek Kitszel @ 2026-11-17 9:44 UTC (permalink / raw)
To: kuba, netdev
Cc: aleksandr.loktionov, andrew+netdev, anthony.l.nguyen, anzaki,
brett.creeley, corbet, davem, edumazet, horms, jacob.e.keller,
jedrzej.jagielski, jiri, leon, mbloch, mschmidt, ohartoov, pabeni,
poros, rdunlap, saeedm, skhan, tariqt, Przemek Kitszel
Add support for up to 256 VF queues. To enable more than usual 16, user
needs to assign GLOBAL LUT (devlink resource named "rss/lut_512") for 64
queues or PF LUT (name "rss/lut_2048") for maximum of 256 queues. There is
a need to assign the GLOBAL LUT to PF first, then release the PF LUT
(initially assigned to PF) from PF, to finally assign it to VF, see
examples further below. Usual default of number of CPU cores does still
apply, but queues could be requested later by usual ethtool -L command.
Add devlink instance of VF device to track RSS LUT resources under
a respective PF devlink instance.
Number of MSI-X vectors assigned to VF is orthogonal to this.
Note that VF reset is deferred to service task.
How to use:
1. Up to 64 queues
a. assign one of 16 GLOBAL LUTs to VF:
sudo devlink resource set pci/0000:18:01.0 path rss/lut_512 size 1
b. if more queues than the default (num of vCPU/CPU cores) are wanted:
sudo ethtool -L $vfiface combined $more
2. Up to 256 queues
a. assign a GLOBAL LUT to PF:
sudo devlink resource set pci/0000:18:00.0 path rss/lut_512 size 1
b. free the PF LUT from PF:
sudo devlink resource set pci/0000:18:00.0 path rss/lut_2048 size 0
c. assign the PF LUT to VF:
sudo devlink resource set pci/0000:18:01.0 path rss/lut_2048 size 1
d. if more queues than the default (num of vCPU/CPU cores) are wanted:
sudo ethtool -L $vfiface combined $more
3. display current RSS LUT or RSS table:
a. see if RSS is mapped correctly (e.g. for lut_512 there are 512
entries expected):
ethtool -x $vfiface
b. see what devlink devices are present:
devlink dev show # note the "whole dev aggregate" device over your pci netdevs
c. see PF, VF, and "whole device aggregate" resources:
devlink resource show pci/0000:18:00.0 # PF
devlink resource show pci/0000:18:01.0 # VF
devlink resource show devlink_index/3 # whole dev
output will look like below, for a device with one PF LUT and one GLOBAL LUT:
pci/0000:18:00.0:
name rss size 2 unit entry size_min 0 size_max 2 size_gran 1 dpipe_tables none
resources:
name lut_512 size 1 unit entry size_min 0 size_max 1 size_gran 1 dpipe_tables none
name lut_2048 size 1 unit entry size_min 0 size_max 1 size_gran 1 dpipe_tables none
Big thanks to Mateusz for working together on the whole story for long
time! Big thanks to Alex Loktionov for pointing the reason of one nasty
bug with the message size!
Co-developed-by: Mateusz Polchlopek <mateusz.polchlopek@intel.com>
Signed-off-by: Mateusz Polchlopek <mateusz.polchlopek@intel.com>
Signed-off-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Reviewed-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
Signed-off-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
---
v6:
* document in ice.rst that VF queues must fit in one free range of PF
queues, so with many VFs only a few get 64 queues, and 256 may not be
available even with a PF LUT (Petr)
v5:
* keep reporting the 64 entry RSS LUT to VFs without
VIRTCHNL_VF_LARGE_NUM_QPAIRS, and repeat their LUT over the bigger
LUT of the VSI; iavf before this series allocates its LUT buffer
only once, and overflowed it after a bigger LUT was assigned (Petr)
* grant REQUEST_QUEUES only when the VF fits in one free range of Tx
and Rx queues, counting its own queues as free; scattered mapping
is not implemented for VFs, and requests of 64 or 256 queues
fragmented the PF queue space, so VFs came up with unprogrammed
queues (Petr)
* offer no more queues than that in the reply to a too big request,
not just the RSS LUT limit (Sashiko)
* drop the check against ICE_MAX_QS_PER_VF, the RSS LUT limit is never
bigger (me)
* mark VFs disabled when preparing for PF reset, until
ice_reset_all_vfs() rebuilds them; devlink could change the RSS LUT
of a VF just after the PF rebuild, and the VF VSI rebuild then went
back to the VSI LUT, with the LUT still taken in the table (Sashiko)
* register VF devlink resources before devl_register(), and unregister
them after devl_unregister(), under one devl lock section, so the
VF devlink instance is never visible without them (Sashiko)
* clamp rss_size to what the RSS LUT can address for all VSI types,
under rss_lut_lock; the VF-only clamp was done before taking the
lock, and raced with devlink LUT changes; the default rss_size is
set on each VSI config again, so no VF special case (Sashiko)
* don't set rss_size of the PF VSI when restoring its PF LUT after
reset, the PF VSI rebuild right after sets the default anyway
(Sashiko)
* drop the second ice_get_vf_vsi() call in ice_vc_config_rss_lut(),
the VSI is now looked up before the LUT size check (Sashiko)
v4:
* after a PF reset, lower the queue count only of VFs that lost a bigger
RSS LUT, and only to 16; ice_vsi_decfg() set every VF back to the
default count, dropping queues granted by REQUEST_QUEUES or set via
sriov_vf_msix_count, and took the PF from vsi->back, which is not the
VF's PF for the SR-IOV LAG placeholder VSIs (Sashiko)
* restore the VF RSS LUTs after reset outside of rtnl, which the PF part
takes now; VF creation takes rtnl under vfs.table_lock (me)
* cap REQUEST_QUEUES at 16 for VFs without VIRTCHNL_VF_LARGE_NUM_QPAIRS,
also when they use a bigger RSS LUT (Sashiko)
* report ICE_VF_VSI_ID as vport_id in GET_MAX_RSS_QREGION reply, not the
HW VSI number (Sashiko)
* clear ICE_VF_STATE_NEEDS_RESET on every VF reset, so a VF is not reset
again for a change that was already applied (Sashiko)
* don't create VF devlink instance without VF pci_dev, e.g. on s390
(Sashiko)
* document the VF devlink instance and its RSS LUT resources in ice.rst
(Sashiko)
* do a pending deferred VF reset before handling the next message from
that VF, instead of answering it from a half-applied RSS LUT change
(GET_VF_RESOURCES offered 64 queues without LARGE_NUM_QPAIRS)
* re-run service task at once if a deferred VF reset was scheduled
during its pass, after ice_handle_deferred_vf_reset(); the wake-up
from ice_schedule_vf_reset() is a no-op while the task runs, so the
reset waited for the service timer, up to 1 s (me)
* report the VF queue allocation in num_queue_pairs also with
VIRTCHNL_VF_LARGE_NUM_QPAIRS, as in legacy mode and as the
out-of-tree ice does, instead of the RSS LUT queue limit; VF learns
the limit from GET_MAX_RSS_QREGION or from a REQUEST_QUEUES denial
(Sashiko)
* declare ret and len in one line in ice_vc_get_vf_res_msg()
v3:
* compute RSS LUT queue count under rss_lut_lock, to not race with
ethtool -L (me)
* allocate VF devlink with ice as the owning driver, via new
devlink_alloc_drv(); VF device could be unbound or bound to iavf at
that point (me)
* create VF devlinks only after all VFs are started, not while
ice_start_vfs() can still fail and release their VSIs (me)
* check VF VSI for NULL in ice_vc_request_qs_msg() (me)
* restore default RSS LUT assignment of VFs after PF reset, FW drops it
* derive VF rss_size from its queue count and LUT capacity on each
VSI config, instead of keeping a stale value (Sashiko)
* wake up service task when scheduling a deferred VF reset (Sashiko)
* lower VF queue count to the capacity of its new RSS LUT when the LUT
shrinks, instead of keeping the queues allocated and offering them
to iavf without VIRTCHNL_VF_LARGE_NUM_QPAIRS (me)
v2:
* drop getting VF ref around vf->needs_deferred_reset, it only opened
a window for resource leak and was not needed anyway (Sashiko)
* replace needs_deferred_reset bitfield with ICE_VF_STATE_NEEDS_RESET
bit, consumed by test_and_clear_bit() before the reset, so a request
coming in during the reset is not lost
* take vfs.table_lock in service task only when a deferred VF reset
is pending, so freshly enabled VFs do not time out on
VIRTCHNL_OP_VERSION
* add !CONFIG_PCI_IOV stub for ice_schedule_vf_reset()
* create VF devlink in ice_start_vfs() under vfs.table_lock instead
of lazily from the virtchnl handler; take PF devlink lock around
devl_nested_devlink_set(); handle VF without devlink on teardown
* take cfg_lock before the adapter devlink lock in VF resource setters,
and reject changes while the VF is not ready (ice_vf_is_ready())
* switch VF VSI context to the new LUT synchronously, so HW no longer
uses the old LUT when it is released
* use logical_pf_id for VF allocation of PF LUT slots
* report the RSS qregion width from LUT capacity, and reject VF queue
requests above it
* keep clamping rss_size to num_rxq for non-VF VSIs, so after
ethtool -L and PF reset the LUT does not point at absent queues
* limit PF rss_size to the queue capacity of its current RSS LUT
(64 for GLOBAL), on devlink LUT switch and on ethtool -L; restore it
from num_rxq when switching back to the PF LUT
* drop ICE_VSI_FLAG_RELOAD, it was set but never tested (and
overwritten on VF reset anyway)
* use ice_get_vf_vsi() instead of open coding it (Sashiko)
* commit message: the whole device aggregate is a shared devlink
instance (devlink_index/N), not a faux device
* check devl_nested_devlink_set() return value (Sashiko)
---
Documentation/networking/devlink/ice.rst | 27 +++-
.../net/ethernet/intel/ice/devlink/resource.h | 3 +
drivers/net/ethernet/intel/ice/ice.h | 1 +
drivers/net/ethernet/intel/ice/ice_lib.h | 2 +
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 23 +++
| 1 +
.../net/ethernet/intel/ice/virt/virtchnl.h | 4 +
.../net/ethernet/intel/ice/devlink/resource.c | 150 +++++++++++++++++-
drivers/net/ethernet/intel/ice/ice_ethtool.c | 14 +-
drivers/net/ethernet/intel/ice/ice_lib.c | 20 +++
drivers/net/ethernet/intel/ice/ice_main.c | 26 +++
drivers/net/ethernet/intel/ice/ice_sriov.c | 15 ++
drivers/net/ethernet/intel/ice/ice_vf_lib.c | 72 +++++++++
drivers/net/ethernet/intel/ice/virt/queues.c | 82 ++++++++--
| 54 ++++++-
.../net/ethernet/intel/ice/virt/virtchnl.c | 39 ++++-
16 files changed, 497 insertions(+), 36 deletions(-)
diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
index 6e34c107b28d..90ec1c7848d8 100644
--- a/Documentation/networking/devlink/ice.rst
+++ b/Documentation/networking/devlink/ice.rst
@@ -507,27 +507,46 @@ PF LUT.
* - ``rss/lut_2048``
- The number of PF LUTs owned.
-The resources are registered on two kinds of devlink instances:
+The resources are registered on three kinds of devlink instances:
- The devlink instance of each PF, where ``lut_512`` and ``lut_2048`` are
either 0 or 1. They are changed with ``devlink resource set``, and the
change applies immediately.
+- The devlink instance of each VF (``pci/<VF BDF>``), nested in the devlink
+ instance of its PF, with the same two resources. A VF can take a global
+ LUT, or the PF LUT of its PF.
- The whole device devlink instance, shared by all PFs of the device, which
reports the total usage. Its resources are read-only, setting them fails
with ``EPERM``.
A PF must own at least one of the two LUTs at all times, so switching a PF
to a global LUT takes two steps: take a global LUT, then release the PF LUT.
A PF that owns both uses its PF LUT.
+A VF uses its VSI LUT by default, which spreads traffic over up to 16
+queues. With a global LUT the VF can use up to 64 queues, and up to 256 with
+a PF LUT (preferred when the VF owns both). Changing the LUT of a VF resets
+the VF, and lowers its queue count if the new LUT can't spread traffic over
+all of them. The VF driver picks the queue count as usual, more queues are
+requested with ``ethtool -L``. More than 16 queues need a VF driver that
+supports ``VIRTCHNL_VF_LARGE_NUM_QPAIRS``, like ``iavf``.
+
+The queues of a VF must be one contiguous range of the PF Tx and Rx queues,
+so a request is granted only when the VF fits in one free range (its own
+queues count as free), and the driver logs how many are free otherwise.
+The queues of other VFs fragment that space, so with many VFs only a few
+of them can get 64 queues, and a VF may not get 256 queues even with a PF
+LUT.
+
Changing the LUT of a PF resets its RSS indirection table to the default
spread, a table set with ``ethtool -X`` is lost.
RSS LUTs can't be changed while ADQ is configured, and ADQ can't be
configured while the RSS LUTs of the PF are reassigned.
Firmware drops the LUT assignments on reset (PFR, CORER or GLOBR). The
-driver then restores the default assignment, and logs that it did so.
+driver then restores the default assignment, and logs that it did so. VFs
+that used a bigger LUT get at most 16 queues after that.
Changes are rejected with ``EBUSY`` while a reset is in progress.
.. code:: shell
@@ -543,3 +562,7 @@ Changes are rejected with ``EBUSY`` while a reset is in progress.
# switch a PF to a global LUT
$ devlink resource set pci/0000:18:00.0 path rss/lut_512 size 1
$ devlink resource set pci/0000:18:00.0 path rss/lut_2048 size 0
+
+ # give the freed PF LUT to a VF of that PF, and use 256 queues there
+ $ devlink resource set pci/0000:18:01.0 path rss/lut_2048 size 1
+ $ ethtool -L $VF_NETDEV combined 256
diff --git a/drivers/net/ethernet/intel/ice/devlink/resource.h b/drivers/net/ethernet/intel/ice/devlink/resource.h
index a41bda0ff6c8..4401ce0171e1 100644
--- a/drivers/net/ethernet/intel/ice/devlink/resource.h
+++ b/drivers/net/ethernet/intel/ice/devlink/resource.h
@@ -10,7 +10,9 @@ struct devlink;
struct ice_adapter;
struct ice_hw;
struct ice_pf;
+struct ice_vf;
+void ice_devlink_vf_resources_register(struct ice_vf *vf);
void ice_devl_pf_resources_register(struct ice_pf *pf);
void ice_devl_whole_dev_resources_register(const struct ice_hw *hw,
struct ice_adapter *adapter);
@@ -21,5 +23,6 @@ void ice_release_rss_lut_pf(struct ice_pf *pf);
void ice_prepare_rss_luts_for_reset(struct ice_pf *pf);
int ice_rebuild_rss_luts(struct ice_pf *pf);
void ice_free_rss_lut_flr(struct ice_pf *pf);
+void ice_free_rss_lut_vf(struct ice_vf *vf);
#endif /* _ICE_DEVL_RESOURCE_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 611f1346b4a7..daa417edc9b7 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -297,6 +297,7 @@ enum ice_pf_state {
ICE_SIDEBANDQ_EVENT_PENDING,
ICE_MDD_EVENT_PENDING,
ICE_VFLR_EVENT_PENDING,
+ ICE_VF_RESET_PENDING,
ICE_FLTR_OVERFLOW_PROMISC,
ICE_VF_DIS,
ICE_CFG_BUSY,
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 15b65a757beb..320edbdece3d 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -18,6 +18,8 @@ enum ice_l2tsel {
ICE_L2TSEL_EXTRACT_FIRST_TAG_L2TAG1,
};
+u16 ice_lut_type_to_qs_num(enum ice_lut_type lut_type);
+
const char *ice_vsi_type_str(enum ice_vsi_type vsi_type);
bool ice_pf_state_is_nominal(struct ice_pf *pf);
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
index 396dfb2b2d97..8abc8c844c96 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
@@ -40,6 +40,7 @@ enum ice_vf_states {
ICE_VF_STATE_DIS,
ICE_VF_STATE_MC_PROMISC,
ICE_VF_STATE_UC_PROMISC,
+ ICE_VF_STATE_NEEDS_RESET, /* reset deferred to service task */
ICE_VF_STATES_NBITS
};
@@ -145,6 +146,7 @@ struct ice_vf {
struct kref refcnt;
struct ice_pf *pf;
struct pci_dev *vfdev;
+ struct devlink *devlink;
/* Used during virtchnl message handling and NDO ops against the VF
* that will trigger a VFR
*/
@@ -243,6 +245,12 @@ static inline bool ice_vf_is_lldp_ena(struct ice_vf *vf)
return vf->num_mac_lldp && vf->trusted;
}
+static inline bool ice_vf_is_ready(struct ice_vf *vf)
+{
+ return test_bit(ICE_VF_STATE_INIT, vf->vf_states) &&
+ !test_bit(ICE_VF_STATE_DIS, vf->vf_states);
+}
+
/* VF Hash Table access functions
*
* These functions provide abstraction for interacting with the VF hash table.
@@ -319,9 +327,12 @@ int
ice_vf_clear_vsi_promisc(struct ice_vf *vf, struct ice_vsi *vsi, u8 promisc_m);
int ice_reset_vf(struct ice_vf *vf, u32 flags);
void ice_reset_all_vfs(struct ice_pf *pf);
+void ice_schedule_vf_reset(struct ice_vf *vf);
struct ice_vsi *ice_get_vf_ctrl_vsi(struct ice_pf *pf, struct ice_vsi *vsi);
void ice_vf_update_mac_lldp_num(struct ice_vf *vf, struct ice_vsi *vsi,
bool incr);
+void ice_init_vf_devlink(struct ice_vf *vf);
+void ice_deinit_vf_devlink(struct ice_vf *vf);
#else /* CONFIG_PCI_IOV */
static inline struct ice_vf *ice_get_vf_by_id(struct ice_pf *pf, u16 vf_id)
{
@@ -393,11 +404,23 @@ static inline void ice_reset_all_vfs(struct ice_pf *pf)
{
}
+static inline void ice_schedule_vf_reset(struct ice_vf *vf)
+{
+}
+
static inline struct ice_vsi *
ice_get_vf_ctrl_vsi(struct ice_pf *pf, struct ice_vsi *vsi)
{
return NULL;
}
+
+static inline void ice_init_vf_devlink(struct ice_vf *vf)
+{
+}
+
+static inline void ice_deinit_vf_devlink(struct ice_vf *vf)
+{
+}
#endif /* !CONFIG_PCI_IOV */
#endif /* _ICE_VF_LIB_H_ */
--git a/drivers/net/ethernet/intel/ice/virt/rss.h b/drivers/net/ethernet/intel/ice/virt/rss.h
index 784d4c43ce8b..388f980b4cdf 100644
--- a/drivers/net/ethernet/intel/ice/virt/rss.h
+++ b/drivers/net/ethernet/intel/ice/virt/rss.h
@@ -14,5 +14,6 @@ int ice_vc_config_rss_lut(struct ice_vf *vf, u8 *msg);
int ice_vc_config_rss_hfunc(struct ice_vf *vf, u8 *msg);
int ice_vc_get_rss_hashcfg(struct ice_vf *vf);
int ice_vc_set_rss_hashcfg(struct ice_vf *vf, u8 *msg);
+int ice_vc_get_max_rss_qregion(struct ice_vf *vf);
#endif /* _ICE_VIRT_RSS_H_ */
diff --git a/drivers/net/ethernet/intel/ice/virt/virtchnl.h b/drivers/net/ethernet/intel/ice/virt/virtchnl.h
index d11789b3ae1f..18024f13fb5d 100644
--- a/drivers/net/ethernet/intel/ice/virt/virtchnl.h
+++ b/drivers/net/ethernet/intel/ice/virt/virtchnl.h
@@ -78,6 +78,10 @@ struct ice_virtchnl_ops {
int (*get_ptp_cap)(struct ice_vf *vf,
const struct virtchnl_ptp_caps *msg);
int (*get_phc_time)(struct ice_vf *vf);
+ int (*get_max_rss_qregion)(struct ice_vf *vf);
+ int (*ena_qs_v2_msg)(struct ice_vf *vf, u8 *msg, u16 msglen);
+ int (*dis_qs_v2_msg)(struct ice_vf *vf, u8 *msg, u16 msglen);
+ int (*map_q_vector_msg)(struct ice_vf *vf, u8 *msg, u16 msglen);
};
#ifdef CONFIG_PCI_IOV
diff --git a/drivers/net/ethernet/intel/ice/devlink/resource.c b/drivers/net/ethernet/intel/ice/devlink/resource.c
index de9c13e928fa..ec3a095deb9f 100644
--- a/drivers/net/ethernet/intel/ice/devlink/resource.c
+++ b/drivers/net/ethernet/intel/ice/devlink/resource.c
@@ -131,6 +131,16 @@ void ice_free_rss_lut_flr(struct ice_pf *pf)
ice_free_rss_lut_flr_locked(pf);
}
+void ice_free_rss_lut_vf(struct ice_vf *vf)
+{
+ struct ice_pf *pf = vf->pf;
+
+ scoped_guard(ice_adapter_devl, pf->adapter) {
+ ice_devl_res_free(pf, ICE_RSS_LUT_GLOBAL, vf);
+ ice_devl_res_free(pf, ICE_RSS_LUT_PF, vf);
+ }
+}
+
static int ice_devl_res_owned_idx(struct ice_pf *pf,
enum ice_devl_resource_id res_id, void *owner)
{
@@ -197,6 +207,34 @@ static u64 ice_rss_lut_pf_occ_get_both(void *priv)
ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_GLOBAL, pf);
}
+static u64 ice_rss_lut_vf_occ_get_global(void *priv)
+{
+ struct ice_vf *vf = priv;
+ struct ice_pf *pf = vf->pf;
+
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ return ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_GLOBAL, vf);
+}
+
+static u64 ice_rss_lut_vf_occ_get_pf(void *priv)
+{
+ struct ice_vf *vf = priv;
+ struct ice_pf *pf = vf->pf;
+
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ return ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_PF, vf);
+}
+
+static u64 ice_rss_lut_vf_occ_get_both(void *priv)
+{
+ struct ice_vf *vf = priv;
+ struct ice_pf *pf = vf->pf;
+
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ return ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_PF, vf) +
+ ice_is_devl_res_owned_by(pf, ICE_RSS_LUT_GLOBAL, vf);
+}
+
static int ice_devl_resource_deny_occ_set(u64 size,
struct netlink_ext_ack *extack,
void *priv)
@@ -242,7 +280,9 @@ static int ice_maybe_change_rss_lut(struct ice_pf *pf, void *owner,
struct ice_hw *hw = &pf->hw;
enum ice_lut_type lut_type;
int err, lut_size, lut_id;
+ struct ice_vf *vf = NULL;
struct ice_vsi *vsi;
+ u16 rss_size;
u8 *lut;
if (old & new & ICE_HAS_PF_LUT)
@@ -268,7 +308,8 @@ static int ice_maybe_change_rss_lut(struct ice_pf *pf, void *owner,
if (pf == owner) {
vsi = ice_get_main_vsi(pf);
} else {
- return -EOPNOTSUPP;
+ vf = owner;
+ vsi = ice_get_vf_vsi(vf);
}
lut_size = ice_lut_type_to_size(lut_type);
@@ -280,9 +321,10 @@ static int ice_maybe_change_rss_lut(struct ice_pf *pf, void *owner,
params.lut_type = lut_type;
params.vsi_handle = vsi->idx;
mutex_lock(&pf->rss_lut_lock);
+ rss_size = min(vsi->num_rxq, ice_lut_type_to_qs_num(lut_type));
/* PF with RXHASH disabled keeps its LUT zeroed */
- if (!vsi->netdev || vsi->netdev->features & NETIF_F_RXHASH)
- ice_fill_rss_lut(lut, lut_size, vsi->rss_size);
+ if (vf || !vsi->netdev || vsi->netdev->features & NETIF_F_RXHASH)
+ ice_fill_rss_lut(lut, lut_size, rss_size);
err = ice_aq_set_rss_lut(hw, ¶ms);
if (err) {
NL_SET_ERR_MSG_FMT(extack, "AQ failed: %s", libie_aq_str(hw->adminq.sq_last_status));
@@ -299,6 +341,15 @@ static int ice_maybe_change_rss_lut(struct ice_pf *pf, void *owner,
vsi->global_lut_id = params.global_lut_id;
vsi->rss_table_size = lut_size;
vsi->rss_lut_type = lut_type;
+ vsi->rss_size = rss_size;
+ if (vf) {
+ u16 max_qs = ice_lut_type_to_qs_num(lut_type);
+
+ /* don't keep queues that the new LUT can't spread traffic to */
+ if ((vf->num_req_qs ?: vf->num_vf_qs) > max_qs)
+ vf->num_req_qs = max_qs;
+ ice_schedule_vf_reset(vf);
+ }
out:
mutex_unlock(&pf->rss_lut_lock);
kfree(lut);
@@ -415,6 +466,41 @@ static int ice_rss_lut_pf_occ_set_global(u64 size,
ICE_ANY_SLOT, extack);
}
+static int ice_rss_lut_vf_occ_set_pf(u64 size, struct netlink_ext_ack *extack,
+ void *occ_priv)
+{
+ struct ice_vf *vf = occ_priv;
+ struct ice_pf *pf = vf->pf;
+ int pf_id;
+
+ pf_id = ice_devl_pf_id(pf);
+ scoped_guard(mutex, &vf->cfg_lock) {
+ if (!ice_vf_is_ready(vf))
+ return -EBUSY;
+
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ return ice_devl_res_change(size, ICE_RSS_LUT_PF, pf, vf,
+ pf_id, extack);
+ }
+}
+
+static int ice_rss_lut_vf_occ_set_global(u64 size,
+ struct netlink_ext_ack *extack,
+ void *occ_priv)
+{
+ struct ice_vf *vf = occ_priv;
+ struct ice_pf *pf = vf->pf;
+
+ scoped_guard(mutex, &vf->cfg_lock) {
+ if (!ice_vf_is_ready(vf))
+ return -EBUSY;
+
+ scoped_guard(ice_adapter_devl, pf->adapter)
+ return ice_devl_res_change(size, ICE_RSS_LUT_GLOBAL, pf,
+ vf, ICE_ANY_SLOT, extack);
+ }
+}
+
/**
* ice_rss_lut_is_reassigned - check if RSS LUTs of PF are in non-default state
* @pf: the PF to check
@@ -542,13 +628,17 @@ static int ice_rebuild_rss_lut_pf(struct ice_pf *pf)
* @pf: the PF that was reset
*
* Wrapper of ice_rebuild_rss_lut_pf() that also tells the ethtool core when
- * the user RSS LUT was dropped. Must be called without rtnl held.
+ * the user RSS LUT was dropped, and makes the VF VSIs use their default LUTs.
+ * Must be called without rtnl held.
*
* Return: 0 on success, negative on error.
*/
int ice_rebuild_rss_luts(struct ice_pf *pf)
{
+ u16 max_qs = ice_lut_type_to_qs_num(ICE_LUT_VSI);
struct ice_vsi *vsi = ice_get_main_vsi(pf);
+ struct ice_vf *vf;
+ unsigned int bkt;
int err;
/* ethtool core sizes RSS buffers by rss_table_size under rtnl */
@@ -559,8 +649,27 @@ int ice_rebuild_rss_luts(struct ice_pf *pf)
netif_is_rxfh_configured(vsi->netdev))
ethtool_rxfh_indir_lost(vsi->netdev);
rtnl_unlock();
+ if (err)
+ return err;
- return err;
+ /* VF creation takes rtnl under table_lock, so don't nest them here */
+ mutex_lock(&pf->vfs.table_lock);
+ mutex_lock(&pf->rss_lut_lock);
+ ice_for_each_vf(pf, bkt, vf) {
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi || vsi->rss_lut_type == ICE_LUT_VSI)
+ continue;
+
+ vsi->rss_table_size = ice_lut_type_to_size(ICE_LUT_VSI);
+ vsi->rss_lut_type = ICE_LUT_VSI;
+ /* don't keep queues that the VSI LUT can't spread traffic to */
+ if ((vf->num_req_qs ?: vf->num_vf_qs) > max_qs)
+ vf->num_req_qs = max_qs;
+ }
+ mutex_unlock(&pf->rss_lut_lock);
+ mutex_unlock(&pf->vfs.table_lock);
+
+ return 0;
}
static void ice_devl_res_register(struct devlink *devlink,
@@ -661,3 +770,34 @@ void ice_devl_pf_resources_register(struct ice_pf *pf)
devl_assert_locked(devlink);
ice_devl_res_register(devlink, pf_resources, pf);
}
+
+void ice_devlink_vf_resources_register(struct ice_vf *vf)
+{
+ struct ice_devl_resource vf_resources[ICE_DEVL_RESOURCES_COUNT] = {
+ [ICE_RSS_LUT_GLOBAL] = {
+ .name = "lut_512",
+ .parent_id = ICE_RSS_LUT_BOTH,
+ .max_size = 1,
+ .get = ice_rss_lut_vf_occ_get_global,
+ .set = ice_rss_lut_vf_occ_set_global,
+ },
+ [ICE_RSS_LUT_PF] = {
+ .name = "lut_2048",
+ .parent_id = ICE_RSS_LUT_BOTH,
+ .max_size = 1,
+ .get = ice_rss_lut_vf_occ_get_pf,
+ .set = ice_rss_lut_vf_occ_set_pf,
+ },
+ [ICE_RSS_LUT_BOTH] = {
+ .name = "rss",
+ .parent_id = ICE_TOP_RESOURCE,
+ .max_size = 2,
+ .get = ice_rss_lut_vf_occ_get_both,
+ .set = ice_devl_resource_deny_occ_set,
+ },
+ };
+ struct devlink *devlink = vf->devlink;
+
+ devl_assert_locked(devlink);
+ ice_devl_res_register(devlink, vf_resources, vf);
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index 822b94f87156..6f393e13b374 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -3831,14 +3831,15 @@ ice_get_channels(struct net_device *dev, struct ethtool_channels *ch)
/**
* ice_get_valid_rss_size - return valid number of RSS queues
- * @hw: pointer to the HW structure
+ * @vsi: VSI to get the RSS LUT type of
* @new_size: requested RSS queues
*/
-static int ice_get_valid_rss_size(struct ice_hw *hw, int new_size)
+static int ice_get_valid_rss_size(struct ice_vsi *vsi, int new_size)
{
- struct ice_hw_common_caps *caps = &hw->func_caps.common_cap;
+ struct ice_hw_common_caps *caps = &vsi->back->hw.func_caps.common_cap;
- return min_t(int, new_size, BIT(caps->rss_table_entry_width));
+ new_size = min_t(int, new_size, BIT(caps->rss_table_entry_width));
+ return min_t(int, new_size, ice_lut_type_to_qs_num(vsi->rss_lut_type));
}
/**
@@ -3871,7 +3872,7 @@ static int ice_vsi_set_dflt_rss_lut(struct ice_vsi *vsi, int req_rss_size)
if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags))
vsi->rss_size = 1;
else
- vsi->rss_size = ice_get_valid_rss_size(hw,
+ vsi->rss_size = ice_get_valid_rss_size(vsi,
req_rss_size);
/* create/set RSS LUT */
@@ -3968,7 +3969,8 @@ static int ice_set_channels(struct net_device *dev, struct ethtool_channels *ch)
}
/* Update rss_size due to change in Rx queues */
- vsi->rss_size = ice_get_valid_rss_size(&pf->hw, new_rx);
+ scoped_guard(mutex, &pf->rss_lut_lock)
+ vsi->rss_size = ice_get_valid_rss_size(vsi, new_rx);
adev_unlock:
if (locked) {
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index ff16d19a8c27..134b4daf1746 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -12,6 +12,23 @@
#include "devlink/resource.h"
+#define ICE_LUT_VSI_MAX_QS 16
+#define ICE_LUT_GLOBAL_MAX_QS 64
+#define ICE_LUT_PF_MAX_QS 256
+
+u16 ice_lut_type_to_qs_num(enum ice_lut_type lut_type)
+{
+ switch (lut_type) {
+ case ICE_LUT_PF:
+ return ICE_LUT_PF_MAX_QS;
+ case ICE_LUT_GLOBAL:
+ return ICE_LUT_GLOBAL_MAX_QS;
+ case ICE_LUT_VSI:
+ default:
+ return ICE_LUT_VSI_MAX_QS;
+ }
+}
+
/**
* ice_vsi_type_str - maps VSI type enum to string equivalents
* @vsi_type: VSI type enum
@@ -1697,6 +1714,9 @@ int ice_vsi_cfg_rss_lut_key(struct ice_vsi *vsi)
}
scoped_guard(mutex, &pf->rss_lut_lock) {
+ /* devlink changes the LUT type under this lock */
+ vsi->rss_size = min(vsi->rss_size,
+ ice_lut_type_to_qs_num(vsi->rss_lut_type));
lut = kzalloc(vsi->rss_table_size, GFP_KERNEL);
if (!lut)
return -ENOMEM;
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 357dc09a0ade..dbf4fdda7998 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -2276,6 +2276,30 @@ static void ice_check_media_subtask(struct ice_pf *pf)
}
}
+static void ice_handle_deferred_vf_reset(struct ice_pf *pf)
+{
+ struct ice_vf *vf;
+ unsigned int bkt;
+ int err;
+
+ if (!test_and_clear_bit(ICE_VF_RESET_PENDING, pf->state))
+ return;
+
+ mutex_lock(&pf->vfs.table_lock);
+ ice_for_each_vf(pf, bkt, vf) {
+ if (!test_and_clear_bit(ICE_VF_STATE_NEEDS_RESET, vf->vf_states))
+ continue;
+
+ dev_info(ice_pf_to_dev(pf), "doing deferred reset of VF %d\n",
+ vf->vf_id);
+ err = ice_reset_vf(vf, ICE_VF_RESET_NOTIFY | ICE_VF_RESET_LOCK);
+ if (err)
+ dev_warn(ice_pf_to_dev(pf), "deferred reset of VF %d failed: %d\n",
+ vf->vf_id, err);
+ }
+ mutex_unlock(&pf->vfs.table_lock);
+}
+
static void ice_service_task_recovery_mode(struct work_struct *work)
{
struct ice_pf *pf = container_of(work, struct ice_pf, serv_task);
@@ -2358,6 +2382,7 @@ static void ice_service_task(struct work_struct *work)
return;
}
+ ice_handle_deferred_vf_reset(pf);
ice_process_vflr_event(pf);
ice_clean_mailboxq_subtask(pf);
ice_clean_sbq_subtask(pf);
@@ -2374,6 +2399,7 @@ static void ice_service_task(struct work_struct *work)
if (time_after(jiffies, (start_time + pf->serv_tmr_period)) ||
test_bit(ICE_MDD_EVENT_PENDING, pf->state) ||
test_bit(ICE_VFLR_EVENT_PENDING, pf->state) ||
+ test_bit(ICE_VF_RESET_PENDING, pf->state) ||
test_bit(ICE_MAILBOXQ_EVENT_PENDING, pf->state) ||
test_bit(ICE_FD_VF_FLUSH_CTX, pf->state) ||
test_bit(ICE_SIDEBANDQ_EVENT_PENDING, pf->state) ||
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index d7c4a6540008..1bafd8b00060 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -501,6 +501,11 @@ static int ice_start_vfs(struct ice_pf *pf)
}
ice_flush(hw);
+
+ /* only now, so the teardown below has no VF devlink to unregister */
+ ice_for_each_vf(pf, bkt, vf)
+ ice_init_vf_devlink(vf);
+
return 0;
teardown:
@@ -657,6 +662,15 @@ static void ice_sriov_post_vsi_rebuild(struct ice_vf *vf)
wr32(&vf->pf->hw, VFGEN_RSTAT(vf->vf_id), VIRTCHNL_VFR_VFACTIVE);
}
+static struct ice_q_vector *ice_sriov_get_q_vector(struct ice_vsi *vsi,
+ u16 vector_id)
+{
+ /* Subtract non queue vector from vector_id passed by VF
+ * to get actual number of VSI queue vector array index
+ */
+ return vsi->q_vectors[vector_id - ICE_NONQ_VECS_VF];
+}
+
static const struct ice_vf_ops ice_sriov_vf_ops = {
.reset_type = ICE_VF_RESET,
.free = ice_sriov_free_vf,
@@ -667,6 +681,7 @@ static const struct ice_vf_ops ice_sriov_vf_ops = {
.clear_reset_trigger = ice_sriov_clear_reset_trigger,
.irq_close = NULL,
.post_vsi_rebuild = ice_sriov_post_vsi_rebuild,
+ .get_q_vector = ice_sriov_get_q_vector,
};
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
index 320b393ddaaf..766936e234c9 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
@@ -6,6 +6,7 @@
#include "ice_lib.h"
#include "ice_fltr.h"
#include "virt/allowlist.h"
+#include "devlink/resource.h"
/* Public functions which may be accessed by all driver files */
@@ -794,6 +795,8 @@ void ice_reset_all_vfs(struct ice_pf *pf)
ice_eswitch_detach_vf(pf, vf);
vf->driver_caps = 0;
ice_vc_set_default_allowlist(vf);
+ /* the VSI rebuild applies what a deferred reset was for */
+ clear_bit(ICE_VF_STATE_NEEDS_RESET, vf->vf_states);
ice_vf_fdir_exit(vf);
ice_vf_fdir_init(vf);
@@ -933,6 +936,8 @@ int ice_reset_vf(struct ice_vf *vf, u32 flags)
/* Set VF disable bit state here, before triggering reset */
set_bit(ICE_VF_STATE_DIS, vf->vf_states);
+ /* this reset applies what a deferred reset was requested for */
+ clear_bit(ICE_VF_STATE_NEEDS_RESET, vf->vf_states);
ice_trigger_vf_reset(vf, flags & ICE_VF_RESET_VFLR, false);
vsi = ice_get_vf_vsi(vf);
@@ -1010,14 +1015,29 @@ int ice_reset_vf(struct ice_vf *vf, u32 flags)
return err;
}
+/**
+ * ice_schedule_vf_reset - reset VF, deferred to next service_task context
+ * @vf: VF to reset
+ */
+void ice_schedule_vf_reset(struct ice_vf *vf)
+{
+ set_bit(ICE_VF_STATE_NEEDS_RESET, vf->vf_states);
+ /* pairs with test_and_clear_bit() in ice_handle_deferred_vf_reset() */
+ smp_mb__after_atomic();
+ set_bit(ICE_VF_RESET_PENDING, vf->pf->state);
+ ice_service_task_schedule(vf->pf);
+}
+
/**
* ice_set_vf_state_dis - Set VF state to disabled
* @vf: pointer to the VF structure
*/
void ice_set_vf_state_dis(struct ice_vf *vf)
{
ice_set_vf_state_qs_dis(vf);
vf->vf_ops->clear_reset_state(vf);
+ /* until ice_reset_all_vfs() rebuilds the VF */
+ set_bit(ICE_VF_STATE_DIS, vf->vf_states);
}
/* Private functions only accessed from other virtualization files */
@@ -1064,6 +1084,9 @@ void ice_deinitialize_vf_entry(struct ice_vf *vf)
{
struct ice_pf *pf = vf->pf;
+ ice_free_rss_lut_vf(vf);
+ ice_deinit_vf_devlink(vf);
+
if (!ice_is_feature_supported(pf, ICE_F_MBX_LIMIT))
list_del(&vf->mbx_info.list_entry);
}
@@ -1450,3 +1473,52 @@ void ice_vf_update_mac_lldp_num(struct ice_vf *vf, struct ice_vsi *vsi,
if (was_ena != is_ena)
ice_vsi_cfg_sw_lldp(vsi, false, is_ena);
}
+
+void ice_init_vf_devlink(struct ice_vf *vf)
+{
+ struct devlink *pf_devlink = priv_to_devlink(vf->pf);
+ static const struct devlink_ops noop = {};
+ struct devlink *devlink;
+ int err;
+
+ lockdep_assert_held(&vf->pf->vfs.table_lock);
+
+ /* no VF pci_dev when the VFs were not scanned, e.g. on s390 */
+ if (!vf->vfdev)
+ return;
+
+ devlink = devlink_alloc_drv(&noop, 0, &vf->vfdev->dev,
+ ice_pf_to_dev(vf->pf)->driver);
+ if (!devlink)
+ return;
+
+ scoped_guard(devl, pf_devlink)
+ err = devl_nested_devlink_set(pf_devlink, devlink);
+ if (err) {
+ devlink_free(devlink);
+ return;
+ }
+
+ vf->devlink = devlink;
+ scoped_guard(devl, devlink) {
+ ice_devlink_vf_resources_register(vf);
+ devl_register(devlink);
+ }
+}
+
+void ice_deinit_vf_devlink(struct ice_vf *vf)
+{
+ struct devlink *devlink = vf->devlink;
+
+ lockdep_assert_held(&vf->pf->vfs.table_lock);
+
+ if (!devlink)
+ return;
+
+ vf->devlink = NULL;
+ scoped_guard(devl, devlink) {
+ devl_unregister(devlink);
+ devl_resources_unregister(devlink);
+ }
+ devlink_free(devlink);
+}
diff --git a/drivers/net/ethernet/intel/ice/virt/queues.c b/drivers/net/ethernet/intel/ice/virt/queues.c
index f87ccf5d370b..76f2ba779053 100644
--- a/drivers/net/ethernet/intel/ice/virt/queues.c
+++ b/drivers/net/ethernet/intel/ice/virt/queues.c
@@ -986,6 +986,48 @@ int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
VIRTCHNL_STATUS_ERR_PARAM, NULL, 0);
}
+/**
+ * ice_vf_max_contig_qs - get the most queues VF could get in one range
+ * @vf: pointer to the VF info
+ * @vsi: VSI of the VF
+ *
+ * Scattered queue mapping is not implemented for VFs. Queues of the VF count
+ * as free, as VF reset releases them before it takes the new range.
+ *
+ * Return: length of the longest range of free Tx queues and Rx queues.
+ */
+static int ice_vf_max_contig_qs(struct ice_vf *vf, struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vf->pf;
+ unsigned long *txqs, *rxqs;
+ int max_tx = 0, max_rx = 0;
+ int b, e;
+
+ txqs = bitmap_alloc(pf->max_pf_txqs, GFP_KERNEL);
+ rxqs = bitmap_alloc(pf->max_pf_rxqs, GFP_KERNEL);
+ if (!txqs || !rxqs)
+ goto out;
+
+ scoped_guard(mutex, &pf->avail_q_mutex) {
+ bitmap_copy(txqs, pf->avail_txqs, pf->max_pf_txqs);
+ bitmap_copy(rxqs, pf->avail_rxqs, pf->max_pf_rxqs);
+ }
+ for (int i = 0; i < vsi->alloc_txq; i++)
+ clear_bit(vsi->txq_map[i], txqs);
+ for (int i = 0; i < vsi->alloc_rxq; i++)
+ clear_bit(vsi->rxq_map[i], rxqs);
+
+ for_each_clear_bitrange(b, e, txqs, pf->max_pf_txqs)
+ max_tx = max(max_tx, e - b);
+ for_each_clear_bitrange(b, e, rxqs, pf->max_pf_rxqs)
+ max_rx = max(max_rx, e - b);
+out:
+ bitmap_free(txqs);
+ bitmap_free(rxqs);
+
+ return min(max_tx, max_rx);
+}
+
/**
* ice_vc_request_qs_msg
* @vf: pointer to the VF info
@@ -1004,33 +1046,41 @@ int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg)
u16 req_queues = vfres->num_queue_pairs;
struct ice_pf *pf = vf->pf;
u16 max_allowed_vf_queues;
- u16 tx_rx_queue_left;
+ struct ice_vsi *vsi;
struct device *dev;
- u16 cur_queues;
+ u16 max_lut_queues;
dev = ice_pf_to_dev(pf);
if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
- cur_queues = vf->num_vf_qs;
- tx_rx_queue_left = min_t(u16, ice_get_avail_txq_count(pf),
- ice_get_avail_rxq_count(pf));
- max_allowed_vf_queues = tx_rx_queue_left + cur_queues;
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto error_param;
+ }
+
+ /* without LARGE_NUM_QPAIRS, VF can only use the v1 queue opcodes */
+ if (vf->driver_caps & VIRTCHNL_VF_LARGE_NUM_QPAIRS)
+ max_lut_queues = ice_lut_type_to_qs_num(vsi->rss_lut_type);
+ else
+ max_lut_queues = ICE_MAX_QS_PER_VF_VCV1;
+ max_allowed_vf_queues = min(ice_vf_max_contig_qs(vf, vsi),
+ max_lut_queues);
+ if (req_queues > max_lut_queues) {
+ vfres->num_queue_pairs = max_allowed_vf_queues;
+ goto error_param;
+ }
+
if (!req_queues) {
dev_err(dev, "VF %d tried to request 0 queues. Ignoring.\n",
vf->vf_id);
- } else if (req_queues > ICE_MAX_QS_PER_VF) {
- dev_err(dev, "VF %d tried to request more than %d queues.\n",
- vf->vf_id, ICE_MAX_QS_PER_VF);
- vfres->num_queue_pairs = ICE_MAX_QS_PER_VF;
- } else if (req_queues > cur_queues &&
- req_queues - cur_queues > tx_rx_queue_left) {
- dev_warn(dev, "VF %d requested %u more queues, but only %u left.\n",
- vf->vf_id, req_queues - cur_queues, tx_rx_queue_left);
- vfres->num_queue_pairs = min_t(u16, max_allowed_vf_queues,
- ICE_MAX_QS_PER_VF);
+ } else if (req_queues > max_allowed_vf_queues) {
+ dev_warn(dev, "VF %d requested %u queues, but only %u are free in one range.\n",
+ vf->vf_id, req_queues, max_allowed_vf_queues);
+ vfres->num_queue_pairs = max_allowed_vf_queues;
} else {
/* request is successful, then reset VF */
vf->num_req_qs = req_queues;
--git a/drivers/net/ethernet/intel/ice/virt/rss.c b/drivers/net/ethernet/intel/ice/virt/rss.c
index 960012ca91b5..99d9295af79c 100644
--- a/drivers/net/ethernet/intel/ice/virt/rss.c
+++ b/drivers/net/ethernet/intel/ice/virt/rss.c
@@ -3,6 +3,7 @@
#include "rss.h"
#include "ice_vf_lib_private.h"
+#include "ice_lib.h"
#include "ice.h"
#define FIELD_SELECTOR(proto_hdr_field) \
@@ -1734,7 +1735,9 @@ int ice_vc_config_rss_lut(struct ice_vf *vf, u8 *msg)
{
struct virtchnl_rss_lut *vrl = (struct virtchnl_rss_lut *)msg;
enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
+ int lut_size = ICE_LUT_VSI_SIZE;
struct ice_vsi *vsi;
+ u8 *lut;
if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
@@ -1746,24 +1749,38 @@ int ice_vc_config_rss_lut(struct ice_vf *vf, u8 *msg)
goto error_param;
}
- if (vrl->lut_entries != ICE_LUT_VSI_SIZE) {
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
- if (!test_bit(ICE_FLAG_RSS_ENA, vf->pf->flags)) {
+ /* the LUT size reported in the GET_VF_RESOURCES reply */
+ if (vf->driver_caps & VIRTCHNL_VF_LARGE_NUM_QPAIRS)
+ lut_size = vsi->rss_table_size;
+ if (vrl->lut_entries != lut_size) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
- vsi = ice_get_vf_vsi(vf);
- if (!vsi) {
+ if (!test_bit(ICE_FLAG_RSS_ENA, vf->pf->flags)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
goto error_param;
}
- if (ice_set_rss_lut(vsi, vrl->lut, ICE_LUT_VSI_SIZE))
+ lut = kmalloc(vsi->rss_table_size, GFP_KERNEL);
+ if (!lut) {
+ v_ret = VIRTCHNL_STATUS_ERR_NO_MEMORY;
+ goto error_param;
+ }
+
+ /* repeat the LUT of the VF over the whole RSS LUT of its VSI */
+ for (int i = 0; i < vsi->rss_table_size; i++)
+ lut[i] = vrl->lut[i % lut_size];
+
+ if (ice_set_rss_lut(vsi, lut, vsi->rss_table_size))
v_ret = VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR;
+ kfree(lut);
error_param:
return ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_CONFIG_RSS_LUT, v_ret,
NULL, 0);
@@ -1920,3 +1937,30 @@ int ice_vc_set_rss_hashcfg(struct ice_vf *vf, u8 *msg)
NULL, 0);
}
+int ice_vc_get_max_rss_qregion(struct ice_vf *vf)
+{
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
+ struct virtchnl_max_rss_qregion max_rss_qregion = {};
+ struct ice_vsi *vsi;
+ int err, len = 0;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto reply;
+ }
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto reply;
+ }
+
+ len = sizeof(max_rss_qregion);
+ max_rss_qregion.vport_id = ICE_VF_VSI_ID;
+ max_rss_qregion.qregion_width =
+ ilog2(ice_lut_type_to_qs_num(vsi->rss_lut_type));
+reply:
+ err = ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_GET_MAX_RSS_QREGION, v_ret,
+ (u8 *)&max_rss_qregion, len);
+ return err;
+}
diff --git a/drivers/net/ethernet/intel/ice/virt/virtchnl.c b/drivers/net/ethernet/intel/ice/virt/virtchnl.c
index ca8018e3dd42..778637f2868c 100644
--- a/drivers/net/ethernet/intel/ice/virt/virtchnl.c
+++ b/drivers/net/ethernet/intel/ice/virt/virtchnl.c
@@ -248,8 +248,7 @@ static int ice_vc_get_vf_res_msg(struct ice_vf *vf, u8 *msg)
struct virtchnl_vf_resource *vfres = NULL;
struct ice_hw *hw = &vf->pf->hw;
struct ice_vsi *vsi;
- int len = 0;
- int ret;
+ int ret, len = 0;
if (ice_check_vf_init(vf)) {
v_ret = VIRTCHNL_STATUS_ERR_PARAM;
@@ -329,12 +328,19 @@ static int ice_vc_get_vf_res_msg(struct ice_vf *vf, u8 *msg)
if (vf->driver_caps & VIRTCHNL_VF_CAP_PTP)
vfres->vf_cap_flags |= VIRTCHNL_VF_CAP_PTP;
+ if (vf->driver_caps & VIRTCHNL_VF_LARGE_NUM_QPAIRS &&
+ vsi->rss_lut_type != ICE_LUT_VSI)
+ vfres->vf_cap_flags |= VIRTCHNL_VF_LARGE_NUM_QPAIRS;
+
vfres->num_vsis = 1;
/* Tx and Rx queue are equal for VF */
vfres->num_queue_pairs = vsi->num_txq;
vfres->max_vectors = vf->num_msix;
vfres->rss_key_size = ICE_VSIQF_HKEY_ARRAY_SIZE;
vfres->rss_lut_size = ICE_LUT_VSI_SIZE;
+ /* older VF drivers allocate their RSS LUT only once, at init */
+ if (vfres->vf_cap_flags & VIRTCHNL_VF_LARGE_NUM_QPAIRS)
+ vfres->rss_lut_size = vsi->rss_table_size;
vfres->max_mtu = ice_vc_get_max_frame_size(vf);
vfres->vsi_res[0].vsi_id = ICE_VF_VSI_ID;
@@ -2533,6 +2539,10 @@ static const struct ice_virtchnl_ops ice_virtchnl_dflt_ops = {
.cfg_q_quanta = ice_vc_cfg_q_quanta,
.get_ptp_cap = ice_vc_get_ptp_cap,
.get_phc_time = ice_vc_get_phc_time,
+ .get_max_rss_qregion = ice_vc_get_max_rss_qregion,
+ .ena_qs_v2_msg = ice_vc_ena_qs_v2_msg,
+ .dis_qs_v2_msg = ice_vc_dis_qs_v2_msg,
+ .map_q_vector_msg = ice_vc_map_q_vector_msg,
/* If you add a new op here please make sure to add it to
* ice_virtchnl_repr_ops as well.
*/
@@ -2670,6 +2680,10 @@ static const struct ice_virtchnl_ops ice_virtchnl_repr_ops = {
.cfg_q_quanta = ice_vc_cfg_q_quanta,
.get_ptp_cap = ice_vc_get_ptp_cap,
.get_phc_time = ice_vc_get_phc_time,
+ .get_max_rss_qregion = ice_vc_get_max_rss_qregion,
+ .ena_qs_v2_msg = ice_vc_ena_qs_v2_msg,
+ .dis_qs_v2_msg = ice_vc_dis_qs_v2_msg,
+ .map_q_vector_msg = ice_vc_map_q_vector_msg,
};
/**
@@ -2766,6 +2780,13 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event,
goto error_handler;
}
+ /* VF must not see a config change before the reset that applies it */
+ if (test_and_clear_bit(ICE_VF_STATE_NEEDS_RESET, vf->vf_states)) {
+ dev_info(dev, "doing deferred reset of VF %d\n", vf_id);
+ ice_reset_vf(vf, ICE_VF_RESET_NOTIFY);
+ goto finish;
+ }
+
ops = vf->virtchnl_ops;
/* Perform basic checks on the msg */
@@ -2901,6 +2922,20 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event,
case VIRTCHNL_OP_GET_QOS_CAPS:
err = ops->get_qos_caps(vf);
break;
+ case VIRTCHNL_OP_GET_MAX_RSS_QREGION:
+ err = ops->get_max_rss_qregion(vf);
+ break;
+ case VIRTCHNL_OP_ENABLE_QUEUES_V2:
+ err = ops->ena_qs_v2_msg(vf, msg, msglen);
+ if (!err)
+ ice_vc_notify_vf_link_state(vf);
+ break;
+ case VIRTCHNL_OP_DISABLE_QUEUES_V2:
+ err = ops->dis_qs_v2_msg(vf, msg, msglen);
+ break;
+ case VIRTCHNL_OP_MAP_QUEUE_VECTOR:
+ err = ops->map_q_vector_msg(vf, msg, msglen);
+ break;
case VIRTCHNL_OP_CONFIG_QUEUE_BW:
err = ops->cfg_q_bw(vf, msg);
break;
--
2.54.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [net-next v6 05/15] ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1
2026-11-17 9:44 ` [net-next v6 05/15] ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1 Przemek Kitszel
@ 2026-11-19 6:44 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:44 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found more severe
issue(s) in other patches of this series. This mail is only an FYI, in case
you want to address the nit picks below when you respin.
Low priority / nit picks for your consideration:
- [Low] The rename marks every user of the old ICE_MAX_RSS_QS_PER_VF as a
legacy virtchnl-v1 opcode limit.
--
ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1
This commit renames ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1 across
the ice driver and keeps the value at 16. It prepares for the next patch,
which raises the per-VF queue maximum to 256. It also adjusts the
kernel-doc of ice_vf_has_no_qs_ena().
> diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
> index fa436b3b1eac2..4959ecee0c21c 100644
> --- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
> +++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
> @@ -19,7 +19,8 @@
> #define ICE_MAX_SRIOV_VFS 256
>
> /* VF resource constraints */
> -#define ICE_MAX_RSS_QS_PER_VF 16
> +/* for "old" virtchnl opcodes that accept up to 16 queues */
> +#define ICE_MAX_QS_PER_VF_VCV1 16
[Severity: Low]
This isn't a bug, but the new comment calls this constant a limit for the
"old" virtchnl opcodes. At this point in the series, several of its users
are not opcode limits.
The txq_ena/rxq_ena bitmaps and qs_bw[] in struct ice_vf, and q_home[][]
in struct ice_lag, use it as a storage size. ice_set_per_vf_res() uses it
as the per-VF Tx/Rx queue cap. ice_vsi_set_rss_params() uses it as the VF
VSI rss_size.
The value is still 16 everywhere, so nothing goes out of bounds here. The
next patch in the series ("ice: bump to 256qs for VF") adds
ICE_MAX_QS_PER_VF and moves the storage, allocation and RSS users over to
it. After that, ICE_MAX_QS_PER_VF_VCV1 remains only at the v1 opcode
sites.
Could ICE_MAX_QS_PER_VF be introduced in this patch, with only the opcode
sites switched to the VCV1 name? That way the intermediate tree would
match the new comment.
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [net-next v6 06/15] ice: bump to 256qs for VF
2026-11-17 9:44 ` [net-next v6 06/15] ice: bump to 256qs for VF Przemek Kitszel
@ 2026-11-19 6:44 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:44 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.
Critical: 0 · High: 0 (1) · Medium: 2 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The commit message does not match the code.
- [Medium] ice_lag_cfg_lan_txq()
(drivers/net/ethernet/intel/ice/ice_lag.c) splits the…
Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but it is worth
reporting.
--
ice: bump to 256qs for VF
This commit resizes the ice per-VF bitmaps and arrays (txq_ena, rxq_ena,
qs_bw, LAG q_home) from 16 to 256 queues and widens num_req_qs. It raises
the VIRTCHNL_OP_REQUEST_QUEUES limit to 256. It also splits the LAG
cfg_txqs admin queue command into chunks of at most U8_MAX queues.
> Raise also the limits for the default VF queue count and for
> VIRTCHNL_OP_REQUEST_QUEUES to 256. New virtchnl opcodes, needed to
> make use of more than 16 queues, are handled by subsequent commits
> of this series, the work is split for easier review.
[Severity: Medium]
Is the default VF queue count actually raised by this patch? In
ice_set_per_vf_res() the count is still bounded by the MSI-X count:
num_txq = min_t(u16, num_msix_per_vf - ICE_NONQ_VECS_VF,
ICE_MAX_QS_PER_VF);
num_msix_per_vf is at most ICE_NUM_VF_MSIX_MED (17) and ICE_NONQ_VECS_VF
is 1, so the default stays at 16 or below.
Several functional changes are described only in the v2/v3/v5 notes
below the --- line. Those notes are dropped when the patch is applied:
- ice_vc_cfg_qs_msg() now rolls back by qpair[].rxq.queue_id, up to
last_valid. Before, it used the qpair index i, which could be left over
from the CRC check loop, so it could disable the wrong queues. That
looks like a separate fix.
- ice_lag_cfg_lan_txq() splits the LAG cfg_txqs command into chunks of
at most U8_MAX queues.
- ice_vsi_set_rss_params() changes vsi->rss_size for VF VSIs from 16
to 256.
- struct ice_lag is now allocated with kvzalloc, since q_home grows
to 64KB.
Could the commit message describe these changes? Could the rollback fix
in ice_vc_cfg_qs_msg() go in its own patch?
> diff --git a/drivers/net/ethernet/intel/ice/ice_lag.c b/drivers/net/ethernet/intel/ice/ice_lag.c
> index 08a17ded0ad55..4bb28564c0d65 100644
> --- a/drivers/net/ethernet/intel/ice/ice_lag.c
> +++ b/drivers/net/ethernet/intel/ice/ice_lag.c
> @@ -549,6 +549,41 @@ ice_lag_qbuf_recfg(struct ice_hw *hw, struct ice_aqc_cfg_txqs_buf *qbuf,
> return count;
> }
>
[ ... ]
> +static int
> +ice_lag_cfg_lan_txq(struct ice_hw *hw, struct ice_aqc_cfg_txqs_buf *qbuf,
> + u16 numq, u8 oldport, u8 newport)
> +{
> + while (numq) {
> + int num = min(numq, U8_MAX);
> + int err;
> +
> + err = ice_aq_cfg_lan_txq(hw, qbuf,
> + struct_size(qbuf, queue_info, num),
> + num, oldport, newport,
> + ICE_AQC_Q_CFG_TC_CHNG, NULL);
> + if (err)
> + return err;
[Severity: Medium]
If the second or a later ice_aq_cfg_lan_txq() call fails, firmware has
already moved the Tx queues of the earlier chunks from oldport to
newport. Does anything move them back?
All three callers, ice_lag_move_vf_node_tc(), ice_lag_reclaim_vf_tc()
and ice_lag_move_vf_nodes_tc_sync(), treat the error as if nothing was
applied. They skip ice_aq_move_sched_elems() for the VF VSI node and
resume the subtree:
ice_lag_move_vf_node_tc()
if (ice_lag_cfg_lan_txq(&lag->pf->hw, qbuf, valq, oldport, newport)) {
dev_warn(dev, "Failure to configure queues for LAG failover\n");
goto qbuf_err;
...
qbuf_err:
kfree(qbuf);
resume_traffic:
ice_sched_suspend_resume_elems(&lag->pf->hw, 1, &tmp_teid, false)
The VF's Tx queues would then be split across the two ports, while the
VSI scheduler node stays under the old port's tree. That would last
until the next failover or VF reset.
Before this change the move was a single command, which firmware either
applied in full or rejected in full.
> +
> + numq -= num;
> + memmove(qbuf->queue_info, &qbuf->queue_info[num],
> + numq * sizeof(*qbuf->queue_info));
> + }
> +
> + return 0;
> +}
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/ice_lag.h b/drivers/net/ethernet/intel/ice/ice_lag.h
> index 4bfffecbdc975..39f6a6cc844d8 100644
> --- a/drivers/net/ethernet/intel/ice/ice_lag.h
> +++ b/drivers/net/ethernet/intel/ice/ice_lag.h
> @@ -52,7 +52,7 @@ struct ice_lag {
> u8 bond_lport_sec; /* lport values for secondary PF */
>
> /* q_home keeps track of which interface the q is currently on */
> - u8 q_home[ICE_MAX_SRIOV_VFS][ICE_MAX_QS_PER_VF_VCV1];
> + u8 q_home[ICE_MAX_SRIOV_VFS][ICE_MAX_QS_PER_VF];
>
> /* placeholder VSI for hanging VF queues from on secondary interface */
> struct ice_vsi *sec_vf[ICE_MAX_SRIOV_VFS];
[Severity: High]
This is a pre-existing issue and was not introduced by this patch. Is
the per-VF queue count capped at ICE_MAX_QS_PER_VF anywhere?
ice_sriov_set_msix_vec_count() takes its value from a write to the
sriov_vf_msix_count sysfs file:
queues = msix_vec_count;
...
if (queues > min(ice_get_avail_txq_count(pf),
ice_get_avail_rxq_count(pf)))
return -EINVAL;
...
vf->num_vf_qs = queues;
The VF VSI then gets alloc_txq = alloc_rxq = num_vf_qs, and
num_lan_q_entries[0] follows alloc_txq.
In ice_lag_aa_failover() the loop is bounded by
num_q = pv_ctx->num_lan_q_entries[0], and it does:
lag->q_home[vf_id][i] = dest;
For i >= 256, can this write into the next VF's row? For the highest
vf_id, can it write into sec_vf[], which follows q_home in struct
ice_lag, or past the end of the allocation?
The same question applies without LAG. ice_vc_cfg_qs_msg() accepts any
queue_id below vsi->alloc_txq and passes it to
ice_vf_vsi_dis_single_txq(). That function does
test_bit()/clear_bit(q_id, vf->txq_ena) on a 256-bit bitmap. Can a
queue_id >= 256 chosen by the guest clear bits in rxq_ena or in later
fields of struct ice_vf?
The same uncapped path existed before, with the 16-entry sizes. The
later patches in the series don't seem to add a cap in
ice_sriov_set_msix_vec_count() either.
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [net-next v6 09/15] iavf: increase max number of queues to 256
2026-11-17 9:44 ` [net-next v6 09/15] iavf: increase max number of queues to 256 Przemek Kitszel
@ 2026-11-19 6:44 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:44 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 1 · Low: 1
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The patch turns q_vector->ring_mask into a fixed 256-bit
bitmap (DECLARE_BITMAP(ring_mask, IAVF_MAX_REQ_QUEUES)).
- [Low] The new helper iavf_itrn_reg() (iavf_main.c:423) picks the ITR
register bank per vector index: IAVF_VFINT_ITRN1 (base 0x2800, 0x40…
--
iavf: increase max number of queues to 256
This patch raises the iavf queue limit from 16 to 256. It turns
q_vector->ring_mask into a 256-bit bitmap and allocates the netdev with
256 Tx/Rx queues. Queue vectors are capped at 64, and the ITR of vectors
16..63 is programmed through a new IAVF_VFINT_ITRN_64 register bank.
> diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
> index 822d6e692d495..db769eead68f3 100644
> --- a/drivers/net/ethernet/intel/iavf/iavf_main.c
> +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
> @@ -419,6 +419,15 @@ static irqreturn_t iavf_msix_clean_rings(int irq, void *data)
> return IRQ_HANDLED;
> }
>
> +/* VFINT_ITRN1 fits 16 vectors only, VFINT_ITRN_64 aliases it for up to 64 */
> +static u32 iavf_itrn_reg(enum iavf_dyn_idx_t itr_idx, u16 reg_idx)
> +{
> + if (reg_idx < 16)
> + return IAVF_VFINT_ITRN1(itr_idx, reg_idx);
> +
> + return IAVF_VFINT_ITRN_64(itr_idx, reg_idx);
> +}
[Severity: Low]
Is the aliasing in this comment documented anywhere? When a VF has more
than 16 queue vectors, iavf_itrn_reg() programs vectors 0..15 through
the 0x2800 bank and vectors 16..63 through the 0x2C00 bank.
idpf_lan_vf_regs.h describes the same VF register space. There, the two
banks are alternatives chosen by the VF's vector mode:
/* For VF with 16 vector support, itrn_reg_spacing is 0x4, itrn_indx_spacing
* is 0x40 and base register offset is 0x00002800
*/
...
/* For VF with 64 vector support, itrn_reg_spacing is 0x4, itrn_indx_spacing
* is 0x100 and base register offset is 0x00002C00
*/
Suppose a VF in 64-vector mode does not decode the 0x2800 bank. Then the
first ITR writes for vectors 0..15 in iavf_map_vector_to_rxq() and
iavf_map_vector_to_txq() are lost. Both functions still set current_itr
to target_itr afterwards.
With static ITR (adaptive turned off with ethtool -C),
iavf_update_enable_itr() only rewrites ITR when target_itr !=
current_itr. Would those vectors then stay at the hardware reset
moderation value?
Using IAVF_VFINT_ITRN_64 for every vector whenever the VF has more than
16 vectors would not depend on the aliasing. Would that be safer?
> +
> /**
> * iavf_map_vector_to_rxq - associate irqs with rx queues
> * @adapter: board private structure
> @@ -439,8 +448,8 @@ iavf_map_vector_to_rxq(struct iavf_adapter *adapter, int v_idx, int r_idx)
> q_vector->rx.count++;
> q_vector->rx.next_update = jiffies + 1;
> q_vector->rx.target_itr = ITR_TO_REG(rx_ring->itr_setting);
> - q_vector->ring_mask |= BIT(r_idx);
> - wr32(hw, IAVF_VFINT_ITRN1(IAVF_RX_ITR, q_vector->reg_idx),
> + set_bit(r_idx, q_vector->ring_mask);
[Severity: Medium]
Can r_idx reach 256 here? ring_mask is now
DECLARE_BITMAP(ring_mask, IAVF_MAX_REQ_QUEUES). iavf_map_rings_to_vectors()
calls this for every ridx up to num_active_queues - 1, and nothing checks
r_idx against IAVF_MAX_REQ_QUEUES.
The only clamp is in iavf_validate_num_queues(). It checks only
vf_res->num_queue_pairs, and it rewrites vsi_res[i].num_queue_pairs only
inside that branch:
if (adapter->vf_res->num_queue_pairs > IAVF_MAX_REQ_QUEUES) {
...
for (i = 0; i < adapter->vf_res->num_vsis; i++) {
vsi_res = &adapter->vf_res->vsi_res[i];
vsi_res->num_queue_pairs = IAVF_MAX_REQ_QUEUES;
}
}
The queue count, however, comes from vsi_res->num_queue_pairs (a u16).
In iavf_alloc_queues():
num_active_queues = min_t(int,
adapter->vsi_res->num_queue_pairs,
(int)(num_online_cpus()));
and in iavf_set_channels():
if (num_req == 0 || num_req > adapter->vsi_res->num_queue_pairs)
return -EINVAL;
Take a PF that reports vf_res->num_queue_pairs <= 256 but
vsi_res->num_queue_pairs > 256. num_active_queues can then go above 256,
either on a guest with more than 256 vCPUs or after
"ethtool -L combined N" once the PF grants N.
If so, set_bit() writes past ring_mask. Small overruns hit the q_vector
fields that follow it (itr_countdown, num_ringpairs, v_idx, reg_idx,
name). Larger r_idx values land past the end of the q_vectors array.
Before this patch, the same input only reached "|= BIT(r_idx)" on a
u32. That shift is undefined, but it stays inside the field.
The in-tree ice PF reports the same value for both counts. Hitting this
needs a buggy or malicious PF, which is the case the driver's own
comment in iavf_validate_num_queues() is meant to guard against.
The final iavf_validate_num_queues() at the end of the series has the
same structure; only the threshold depends on LARGE_NUM_QPAIRS_SUPPORT.
Should iavf_validate_num_queues() clamp vsi_res[i].num_queue_pairs on
its own terms? Or should r_idx be bounded before the set_bit()?
> q_vector->rx.current_itr >> 1);
> q_vector->rx.current_itr = q_vector->rx.target_itr;
> }
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [net-next v6 10/15] iavf: use new opcodes to request more than 16 queues
2026-11-17 9:44 ` [net-next v6 10/15] iavf: use new opcodes to request more than 16 queues Przemek Kitszel
@ 2026-11-19 6:45 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:45 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found 13 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 6 · Low: 7
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The commit message leaves out a major behaviour change and
gets the init ordering wrong.
- [Medium] Possible out-of-bounds write in iavf_map_vector_to_rxq().
- [Medium] The fix for RSS key/LUT size changes across a reset is
incomplete.
- [Medium] qregion_width from VIRTCHNL_OP_GET_MAX_RSS_QREGION is used as
a shift count in BIT(width) with no range check, both at init…
- [Medium] iavf: iavf_get_channels() reports old count while queue
request is in flight
- [Medium] iavf: IAVF_FLAG_REINIT_MSIX_NEEDED not cleared in
iavf_reset_step() when down
- [Low] This feature patch also fixes a bug that affects every PF:
adapter->aq_wait_count is never reset.
- [Low] The commit message lists 'two changes in
iavf_parse_vf_resource_msg()' plus the aq_wait_count reset as the…
- [Low] iavf_send_max_rss_qregion() drops the return value of
iavf_send_pf_msg() and always returns 0 once current_op is set.
- [Low] iavf_virtchnl_completion() now does `if
(iavf_parse_vf_resource_msg(adapter)) break;` with the comment 'a…
- [Low] In LARGE mode, iavf_get_channels() can report combined_count
(num_active_queues) larger than max_combined (iavf_max_channels() =…
- [Low] The VIRTCHNL_OP_GET_MAX_RSS_QREGION case in
iavf_virtchnl_completion() refills the LUT and calls iavf_config_rss()…
- [Low] The new VIRTCHNL_OP_GET_MAX_RSS_QREGION completion handler runs
iavf_fill_rss_lut() and iavf_config_rss() whenever the reported width…
--
iavf: use new opcodes to request more than 16 queues
This adds the VIRTCHNL_VF_LARGE_NUM_QPAIRS capability and the
GET_MAX_RSS_QREGION, ENABLE/DISABLE_QUEUES_V2 and MAP_QUEUE_VECTOR
opcodes, so the VF can use more than 16 queues. ethtool -L goes through
iavf_request_queues() in LARGE mode. The patch also changes
iavf_parse_vf_resource_msg() to handle the PF lowering queue counts or
RSS key/LUT sizes across a reset, and it resets aq_wait_count on recovery.
> The VIRTCHNL_OP_GET_MAX_RSS_QREGION is exchanged right after the VF
> resource message is received. This is done similar to the PTP and other
> extended cap exchanges.
[ ... ]
> Use the new iavf_request_queues() for ethtool -L.
[Severity: Medium]
The commit message only mentions iavf_request_queues() for ethtool -L.
However, iavf_parse_vf_resource_msg() now also sends
VIRTCHNL_OP_REQUEST_QUEUES during probe when both LARGE_NUM_QPAIRS and
OFFLOAD_REQ_QUEUES are negotiated:
if (init_req) {
qnum = min_t(int, num_online_cpus(), IAVF_MAX_REQ_QUEUES);
...
return iavf_request_queues(adapter, qnum);
}
If the PF grants the request, it resets the VF during probe. The new
iavf_init_recv_req_queues() step can send a second request after a
denial. Only the v3/v4 changelog below the --- line describes this, and
that part is dropped when the patch is applied.
Could the commit message describe this init-time request?
Also, is "right after the VF resource message is received" accurate for
the init path? iavf_init_process_extended_caps() handles the RSS qregion
exchange last. It comes after the queue request wait, VLAN V2, RXDID and
PTP.
> Two changes in iavf_parse_vf_resource_msg() apply to every PF: when the
> PF reports fewer queue pairs than the VF has in use (PF has lowered its
> limit on reset), reinit queues and MSI-X with that count; when the RSS
> key or LUT size changes across a reset, reallocate both and fill them
> again.
[Severity: Low]
Is this list complete? Several other changes also affect every PF, or
can be seen by users. Only the changelog below the --- line explains them:
- iavf_validate_num_queues() now clamps PFs without LARGE_NUM_QPAIRS
to IAVF_MAX_VSI_QP (16) instead of IAVF_MAX_REQ_QUEUES
- the iavf_init_get_resources() error path now memsets
adapter->vf_res instead of freeing it
- an error reply to VIRTCHNL_OP_REQUEST_QUEUES is now treated as a
denial
- iavf_reset_step() clears num_req_queues when interrupts are not
reinitialized
- IAVF_FLAG_AQ_REQUEST_QUEUES now survives a hardware reset and
iavf_reset_step()
- in LARGE mode, max_combined becomes BIT(qregion_width), and
ethtool -L can return -EBUSY or finish asynchronously
- virtchnl_vc_validate_vf_msg(), which PF drivers share, gets new
opcode cases
> diff --git a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
> index e7cf12eaa2689..07beff9c1983b 100644
> --- a/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
> +++ b/drivers/net/ethernet/intel/iavf/iavf_ethtool.c
[ ... ]
> @@ -1683,6 +1688,19 @@ static int iavf_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd,
>
> return ret;
> }
> +
> +/* in LARGE mode PF gives us up to its RSS region once it has told us that */
> +static u32 iavf_max_channels(struct iavf_adapter *adapter)
> +{
> + u16 width = adapter->max_rss_qregion.qregion_width;
> +
> + if (LARGE_NUM_QPAIRS_SUPPORT(adapter) && width &&
> + adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_REQ_QUEUES)
> + return BIT(width);
[Severity: Medium]
Can this let ethtool -L request more than IAVF_MAX_REQ_QUEUES queues?
BIT(width) comes straight from the PF and is not clamped.
iavf_set_channels() only checks num_req > iavf_max_channels(adapter).
If the PF reports qregion_width >= 9, combined N > 256 is accepted and
queued:
iavf_process_aq_command()
iavf_request_queues(adapter, N)
adapter->num_req_queues = N;
adapter->flags |= IAVF_FLAG_REINIT_ITR_NEEDED;
Then on the next VF reset:
iavf_reset_step()
iavf_reinit_interrupt_scheme()
iavf_init_interrupt_scheme()
iavf_alloc_queues()
if (adapter->num_req_queues)
num_active_queues = adapter->num_req_queues;
iavf_map_rings_to_vectors()
iavf_map_vector_to_rxq()
set_bit(r_idx, q_vector->ring_mask);
ring_mask is DECLARE_BITMAP(ring_mask, IAVF_MAX_REQ_QUEUES). Does this
write past the end of ring_mask once r_idx >= 256? The netdev also has
only IAVF_MAX_REQ_QUEUES tx/rx queues.
Before this patch, the bound was vsi_res->num_queue_pairs, which
iavf_validate_num_queues() caps. In-tree ice reports a width of at most 8,
so this needs a PF that reports a larger width.
Could the return value be clamped to IAVF_MAX_REQ_QUEUES here?
> +
> + return adapter->vsi_res->num_queue_pairs;
> +}
> +
> /**
> * iavf_get_channels - get the number of channels supported by the device
> * @netdev: network interface device structure
> @@ -1697,12 +1715,15 @@ static void iavf_get_channels(struct net_device *netdev,
> struct iavf_adapter *adapter = netdev_priv(netdev);
>
> /* Report maximum channels */
> - ch->max_combined = adapter->vsi_res->num_queue_pairs;
> + ch->max_combined = iavf_max_channels(adapter);
>
> ch->max_other = NONQ_VECS;
> ch->other_count = NONQ_VECS;
>
> ch->combined_count = adapter->num_active_queues;
[Severity: Low]
In LARGE mode, can combined_count end up larger than max_combined?
The init request asks for min(num_online_cpus(), IAVF_MAX_REQ_QUEUES)
queues before the qregion is known. After that, nothing caps
num_active_queues to BIT(qregion_width).
If a PF grants more queues than its RSS region, ethtool -l shows current
> max. ethnl_set_channels() then rejects any request that leaves combined
unchanged, with "requested channel count exceeds maximum". In-tree ice
caps grants to its LUT queue count, so this needs a different PF.
> + /* ethtool -L count, until the watchdog sends the request */
> + if (adapter->aq_required & IAVF_FLAG_AQ_REQUEST_QUEUES)
> + ch->combined_count = adapter->num_req_queues_next;
> }
[Severity: Medium]
This only covers the time before the watchdog sends the request. When
iavf_process_aq_command() sends VIRTCHNL_OP_REQUEST_QUEUES, it clears
IAVF_FLAG_AQ_REQUEST_QUEUES. combined_count then falls back to
adapter->num_active_queues.
The new count takes effect only after the PF resets the VF and
iavf_reset_step() reallocates the queues. Until then, ethtool -l shows
the old count while a different count is already on its way to the PF.
This also defeats the -EBUSY check this patch adds to
iavf_set_channels(). Suppose the user runs ethtool -L back to the old
count while the request is in flight. ethnl_set_channels() compares
the request against iavf_get_channels(), sees no change, and returns
0 without calling iavf_set_channels(). The PF then grants the earlier
request. The VF ends up with the count the user just tried to undo,
even though ethtool reported success.
The current_op check does not cover the whole window either.
iavf_watchdog_step() resets current_op to VIRTCHNL_OP_UNKNOWN as soon
as it sees ARQLEN1 drop for the grant reset. That happens before
reset_task has applied the new count.
Could iavf_get_channels() report the in-flight count, for example
adapter->num_req_queues, until iavf_parse_vf_resource_msg() confirms
or drops it? The v6 changelog says the requested count is reported
until the watchdog sends the request. The change users see happens
later than that.
[ ... ]
> diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
> index db769eead68f3..b6b6d61ac2ac6 100644
> --- a/drivers/net/ethernet/intel/iavf/iavf_main.c
> +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
[ ... ]
> @@ -1783,12 +1784,17 @@ int iavf_config_rss(struct iavf_adapter *adapter)
> * iavf_fill_rss_lut - Fill the lut with default values
> * @adapter: board private structure
> **/
> -static void iavf_fill_rss_lut(struct iavf_adapter *adapter)
> +void iavf_fill_rss_lut(struct iavf_adapter *adapter)
> {
> - u16 i;
> + struct virtchnl_max_rss_qregion *qregion = &adapter->max_rss_qregion;
> + int max = adapter->num_active_queues;
> +
> + /* width == 0: PF has not told us the region yet */
> + if (LARGE_NUM_QPAIRS_SUPPORT(adapter) && qregion->qregion_width)
> + max = min(max, (int)BIT(qregion->qregion_width));
>
> - for (i = 0; i < adapter->rss_lut_size; i++)
> - adapter->rss_lut[i] = i % adapter->num_active_queues;
> + for (int i = 0; i < adapter->rss_lut_size; i++)
> + adapter->rss_lut[i] = i % max;
[Severity: Medium]
What happens here if the PF reports a qregion_width of 32? On 64-bit,
(int)BIT(32) truncates to 0. max then becomes 0, and i % max divides by
zero.
A width of 31 gives INT_MIN, which puts LUT entries past
num_active_queues. A width >= BITS_PER_LONG is an undefined shift.
Neither iavf_get_max_rss_qregion() nor the VIRTCHNL_OP_GET_MAX_RSS_QREGION
case in iavf_virtchnl_completion() checks the range of qregion_width. The
completion handler calls iavf_fill_rss_lut() whenever the width changes.
The same unchecked BIT(width) also feeds iavf_get_rx_ring_count() (0
rings for width 32), iavf_max_channels() and iavf_set_rxfh().
The changelog says the PF is trusted. Still, the driver already validates
other values from the PF: iavf_validate_num_queues(), the max_mtu range
check, and the msglen checks for the RXDID and PTP caps.
Would a bound such as width <= ilog2(IAVF_MAX_REQ_QUEUES) be acceptable?
> }
[ ... ]
> @@ -2504,13 +2542,56 @@ int iavf_parse_vf_resource_msg(struct iavf_adapter *adapter)
> vsi->netdev = adapter->netdev;
> vsi->qs_handle = adapter->vsi_res->qset_handle;
> if (adapter->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_RSS_PF) {
> - adapter->rss_key_size = adapter->vf_res->rss_key_size;
> - adapter->rss_lut_size = adapter->vf_res->rss_lut_size;
> + if ((adapter->rss_key &&
> + adapter->rss_key_size != adapter->vf_res->rss_key_size) ||
> + (adapter->rss_lut &&
> + adapter->rss_lut_size != adapter->vf_res->rss_lut_size)) {
> + reconfig_rss = true;
> + } else {
> + adapter->rss_key_size = adapter->vf_res->rss_key_size;
> + adapter->rss_lut_size = adapter->vf_res->rss_lut_size;
> + }
> } else {
> adapter->rss_key_size = IAVF_HKEY_ARRAY_SIZE;
> adapter->rss_lut_size = IAVF_HLUT_ARRAY_SIZE;
> }
[Severity: Medium]
iavf_reset_step() calls iavf_reinit_interrupt_scheme() whenever either
REINIT flag is set, whether or not the interface is running. It clears
IAVF_FLAG_REINIT_MSIX_NEEDED only in the running branch, next to
iavf_request_traffic_irqs(). IAVF_FLAG_REINIT_ITR_NEEDED is cleared
unconditionally at the end.
This patch adds a second place that sets IAVF_FLAG_REINIT_MSIX_NEEDED:
the num_active_queues > num_queue_pairs check in
iavf_parse_vf_resource_msg(). That can fire while the VF is down,
which is the state after probe or after ndo_stop. The flag then stays
set. Every later reset while the VF is down tears down and reallocates
MSI-X, q_vectors and rings again. Those resets never reach the new
else branch that drops num_req_queues.
This can also lose user settings. In legacy mode, ethtool -L 4 while
down reallocates 4 queues. iavf_parse_vf_resource_msg() then clears
num_req_queues. On the next reset, the stale flag sends us through
iavf_reinit_interrupt_scheme() again. iavf_alloc_queues() then falls
back to min(vsi_res->num_queue_pairs, num_online_cpus()), so the
channel count the user set is silently replaced.
Clearing the flag only when running predates this patch. The
"Requested %d queues, but PF only gave us %d" path already sets it.
The new setter and the new else branch make it matter more, though.
Could IAVF_FLAG_REINIT_MSIX_NEEDED also be cleared on the !running
path, for example together with IAVF_FLAG_REINIT_ITR_NEEDED? As far
as I can see, iavf_open() requests the traffic IRQs itself.
[Severity: Medium]
The commit message says the key and LUT are reallocated when their sizes
change across a reset, but only the RSS_PF branch checks for that.
Suppose the VF first negotiated RSS_PF with vf_res->rss_key_size and
rss_lut_size smaller than 52/64, and a later reset drops RSS_PF. Does this
else branch then raise the sizes while keeping the smaller buffers that
iavf_init_config_adapter() allocated?
Later, for example on the next reset, iavf_init_rss()->iavf_fill_rss_lut()
writes rss_lut_size bytes. iavf_config_rss_reg() and iavf_config_rss_aq()
read rss_key_size and rss_lut_size worth of data. Can these run past the
end of the old allocations?
In-tree i40e and ice report a key of 52 and a LUT of at least 64, so this
needs an unusual PF.
[ ... ]
> @@ -2977,6 +3179,9 @@ static int iavf_watchdog_step(struct iavf_adapter *adapter)
> */
> iavf_change_state(adapter, __IAVF_STARTUP);
> adapter->flags &= ~IAVF_FLAG_PF_COMMS_FAILED;
> + /* fresh retry budget, and init asks for queues again */
> + adapter->aq_wait_count = 0;
> + adapter->num_req_queues = 0;
[Severity: Low]
This isn't a bug, but the aq_wait_count reset fixes a problem that
affects every PF, as the commit message describes. Before this patch, the
only use of aq_wait_count is the increment in the __IAVF_INIT_FAILED case:
if (++adapter->aq_wait_count > IAVF_AQ_MAX_ERR) {
Could this go out as a separate patch for net, with a Fixes: tag? The
logic comes from 898ef1cb1cb2 ("iavf: Combine init and watchdog state
machines"). As it stands, stable can only take it together with the whole
large queue count feature.
> }
> adapter->aq_required = 0;
> adapter->current_op = VIRTCHNL_OP_UNKNOWN;
[ ... ]
> diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
> index 1296ea0f1716a..6437fd4681be7 100644
> --- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
> +++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
[ ... ]
> @@ -251,6 +252,18 @@ int iavf_send_vf_ptp_caps_msg(struct iavf_adapter *adapter)
> (u8 *)&hw_caps, sizeof(hw_caps));
> }
>
> +int iavf_send_max_rss_qregion(struct iavf_adapter *adapter)
> +{
> + adapter->aq_required &= ~IAVF_FLAG_AQ_GET_MAX_RSS_QREGION;
> +
> + if (!LARGE_NUM_QPAIRS_SUPPORT(adapter))
> + return -EOPNOTSUPP;
> +
> + adapter->current_op = VIRTCHNL_OP_GET_MAX_RSS_QREGION;
> + iavf_send_pf_msg(adapter, VIRTCHNL_OP_GET_MAX_RSS_QREGION, NULL, 0);
> + return 0;
[Severity: Low]
Should this return the status of iavf_send_pf_msg(), as
iavf_send_vf_offload_vlan_v2_msg() and iavf_send_vf_ptp_caps_msg() do?
The header comment of iavf_init_send_max_rss_qregion() says it "Must
clear IAVF_EXTENDED_CAP_RECV_RSS_QREGION if the message is not sent".
That function cannot see a failed send, so the RECV step polls for a
reply that never comes. Init does recover through an INIT_FAILED retry.
That comment also starts with /* instead of the kernel-doc /** that the
neighbouring helpers use.
> +}
> +
[ ... ]
> @@ -2638,7 +2859,9 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
>
> iavf_activate_fdir_filters(adapter);
>
> - iavf_parse_vf_resource_msg(adapter);
> + /* a reset is scheduled on error, it will configure us again */
> + if (iavf_parse_vf_resource_msg(adapter))
> + break;
[Severity: Low]
Does this comment hold for every error path? The -ENODEV return in
iavf_parse_vf_resource_msg() does not schedule a reset:
if (!vsi_res) {
dev_err(&adapter->pdev->dev, "No LAN VSI found\n");
return -ENODEV;
}
In that case the break skips iavf_process_config(),
iavf_schedule_finish_config() and the MAC filter re-add, and nothing
retries them. adapter->vsi_res also keeps pointing into the vf_res buffer
that was just overwritten.
In-tree PFs always report an SRIOV VSI, so this needs a misbehaving PF.
[ ... ]
> @@ -2709,6 +2932,20 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
> aq_required;
> }
> break;
> + case VIRTCHNL_OP_GET_MAX_RSS_QREGION: {
> + u16 width = adapter->max_rss_qregion.qregion_width;
> +
> + if (v_retval)
> + break;
> + memcpy(&adapter->max_rss_qregion, msg,
> + sizeof(adapter->max_rss_qregion));
> + /* LUT was filled before this reply, for the old region */
> + if (adapter->max_rss_qregion.qregion_width != width) {
> + iavf_fill_rss_lut(adapter);
> + iavf_config_rss(adapter);
[Severity: Low]
Does iavf_config_rss() work here for a VF that negotiated RSS_AQ? At this
point current_op is still VIRTCHNL_OP_GET_MAX_RSS_QREGION, so
iavf_config_rss_aq() bails out:
if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
/* bail because we already have a command pending */
dev_err(&adapter->pdev->dev, "Cannot configure RSS, command %d pending\n",
adapter->current_op);
return -EBUSY;
}
The return value is ignored here, so the refilled LUT never reaches the
hardware. With RSS_PF, iavf_config_rss() only sets aq_required flags, so
in-tree ice is not affected.
[Severity: Low]
Can this branch run while rss_lut is NULL or num_active_queues is 0?
Nothing checks either before iavf_fill_rss_lut() is called.
iavf_free_rss() runs on the err_mem path of iavf_init_config_adapter() and
on the register_netdevice() failure path in iavf_finish_config(), while
the adapter stays alive. If a reply with a different width is processed
in that window, rss_lut[i] is written through a NULL pointer. If
num_active_queues is 0, i % max divides by zero.
The window is narrow, because stale replies left by init retries carry
the same width.
> + }
> + break;
> + }
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [net-next v6 11/15] ice: introduce handling of virtchnl LARGE VF opcodes
2026-11-17 9:44 ` [net-next v6 11/15] ice: introduce handling of virtchnl LARGE VF opcodes Przemek Kitszel
@ 2026-11-19 6:45 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:45 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 2 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The patch widens struct ice_q_vector::num_ring_rx/num_ring_tx
from u8 to u16.
- [Medium] ice_vc_validate_qs_v2_msg() bounds each chunk (start_queue_id
+ num_queues <= min(num_vf_qs, 256)), and the only limit on num_chunks…
--
ice: introduce handling of virtchnl LARGE VF opcodes
This adds PF-side handlers for VIRTCHNL_OP_ENABLE_QUEUES_V2,
VIRTCHNL_OP_DISABLE_QUEUES_V2 and VIRTCHNL_OP_MAP_QUEUE_VECTOR. They are
allowlisted under VIRTCHNL_VF_LARGE_NUM_QPAIRS so that VFs can use more
than 16 queues. It also widens num_ring_rx/num_ring_tx in struct
ice_q_vector to u16.
> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> index 89a2bdbecf609..fc2e84136e247 100644
> --- a/drivers/net/ethernet/intel/ice/ice.h
> +++ b/drivers/net/ethernet/intel/ice/ice.h
> @@ -474,8 +474,8 @@ struct ice_q_vector {
>
> u16 v_idx; /* index in the vsi->q_vector array. */
> u16 reg_idx; /* PF relative register index */
> - u8 num_ring_rx; /* total number of Rx rings in vector */
> - u8 num_ring_tx; /* total number of Tx rings in vector */
> + u16 num_ring_rx; /* total number of Rx rings in vector */
> + u16 num_ring_tx; /* total number of Tx rings in vector */
[Severity: Medium]
The v5 notes say these fields were widened because mapping all 256 queues
of a VF to one vector made the counters wrap. The main writer of these
fields still truncates to u8, though:
drivers/net/ethernet/intel/ice/ice_base.c:ice_vsi_map_rings_to_vectors() {
...
u8 tx_rings_per_v, rx_rings_per_v;
u16 q_id, q_base;
/* Tx rings mapping to vector */
tx_rings_per_v = (u8)DIV_ROUND_UP(tx_rings_rem,
q_vectors - v_id);
q_vector->num_ring_tx = tx_rings_per_v;
...
for (q_id = q_base; q_id < (q_base + tx_rings_per_v); q_id++) {
...
}
With 256 rings and one q_vector, DIV_ROUND_UP(256, 1) cast to u8 is 0.
That sets num_ring_tx to 0, links no rings and leaves ring->q_vector
unchanged. The Rx side has the same problem.
This function also runs on VF VSIs. ice_pf_dcb_recfg() calls it for every
VSI except ICE_VSI_CHNL. On resume, ice_reinit_interrupt_scheme() calls it
for every VSI right after it re-allocates the q_vectors.
Readers then use the truncated count. For example, ice_vsi_release_msix(),
which ice_vsi_free_irq() also calls for VF VSIs, does this:
for (q = 0; q < q_vector->num_ring_tx; q++) {
ice_write_itr(&q_vector->tx, 0);
wr32(hw, QINT_TQCTL(vsi->txq_map[txq]), 0);
Wouldn't this skip clearing QINT_TQCTL/QINT_RQCTL for every one of these
queues?
On the resume path, can rings be left with ring->q_vector pointing at
q_vectors that were freed at suspend? ice_free_q_vector() only clears the
pointer for rings on the linked list, and VF rings mapped through virtchnl
are never on that list.
At the end of the series, this needs a VF with the PF LUT
(ICE_LUT_PF_MAX_QS = 256) and ICE_MIN_INTR_PER_VF MSI-X vectors, so
num_q_vectors = 1. The VF requests 256 queues, and then a DCB
reconfiguration or a suspend/resume runs. The u8 locals and the (u8) casts
in ice_vsi_map_rings_to_vectors() are still there at the end of the
series.
Should tx_rings_per_v/rx_rings_per_v and the casts be widened to u16 in
this patch as well?
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/virt/queues.c b/drivers/net/ethernet/intel/ice/virt/queues.c
> index 7f2b459dbc836..f87ccf5d370bb 100644
> --- a/drivers/net/ethernet/intel/ice/virt/queues.c
> +++ b/drivers/net/ethernet/intel/ice/virt/queues.c
> @@ -1046,3 +1046,342 @@ int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg)
[ ... ]
> +static bool ice_vc_validate_qs_v2_msg(struct ice_vf *vf,
> + struct virtchnl_del_ena_dis_queues *qs_msg,
> + u16 msglen)
> +{
> + u32 max_qs = min_t(u32, vf->num_vf_qs, ICE_MAX_QS_PER_VF);
> +
> + if (msglen < virtchnl_struct_size(qs_msg, chunks, 0))
> + return false;
> +
> + if (msglen < virtchnl_struct_size(qs_msg, chunks, qs_msg->num_chunks))
> + return false;
[ ... ]
> + for (int i = 0; i < qs_msg->num_chunks; i++) {
> + struct virtchnl_queue_chunk *chunk = &qs_msg->chunks[i];
> + u32 end;
> +
> + if (!ice_vc_supported_queue_type(chunk->type))
> + return false;
> +
> + if (!chunk->num_queues)
> + return false;
> +
> + end = chunk->start_queue_id + chunk->num_queues;
> + if (end > max_qs)
> + return false;
> + }
[Severity: Medium]
Each chunk is bounds-checked here. Is anything rejecting duplicate or
overlapping chunks? The only limit on num_chunks is msglen.
For VIRTCHNL_OP_DISABLE_QUEUES_V2, ice_vc_dis_txq_chunk() calls
ice_vf_vsi_dis_single_txq() for every queue in every chunk. That helper
does not return early for queues that are already disabled:
drivers/net/ethernet/intel/ice/virt/queues.c:ice_vf_vsi_dis_single_txq() {
if (!test_bit(q_id, vf->txq_ena))
dev_dbg(ice_pf_to_dev(vsi->back), "Queue %u on VSI %u is not enabled, but stopping it anyway\n",
q_id, vsi->vsi_num);
...
err = ice_vsi_stop_tx_ring(vsi, ICE_NO_RESET, vf->vf_id, ring, &txq_meta);
...
}
Each ice_vsi_stop_tx_ring() call does an MMIO read-modify-write of
QINT_TQCTL, an ndelay(100), ice_trigger_sw_intr() and ice_dis_vsi_txq().
ice_dis_vsi_txq() takes pi->sched_lock and searches the Tx scheduler tree
with ice_sched_find_node_by_teid().
A 4 KiB mailbox message can hold the 8 byte header plus about 511 chunks,
each set to {type TX, start 0, num max_qs}. With 256 queues, that is about
130k stop-ring operations from one message. They run in the PF service
task with vf->cfg_lock held.
The v1 DISABLE_QUEUES bitmap limits this to at most 16 queues, and each
one is processed once.
Can a VF keep sending this message to keep the PF service task busy? That
would delay mailbox handling for other VFs, as well as AdminQ events and
resets. The validation and the Tx disable helper are unchanged at the end
of the series.
Could the validation build a bitmap of the requested queues and reject
overlapping chunks? Another option is to skip queues that are not set in
vf->txq_ena on the v2 path.
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [net-next v6 12/15] devlink: give user option to allocate resources
2026-11-17 9:44 ` [net-next v6 12/15] devlink: give user option to allocate resources Przemek Kitszel
@ 2026-11-19 6:45 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:45 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found more severe
issue(s) in other patches of this series. This mail is only an FYI, in case
you want to address the nit picks below when you respin.
Low priority / nit picks for your consideration:
- [Low] API asymmetry in the new occ_set mode, plus an unsafe misuse
path.
- [Low] The occ_set paths change size_new without revalidating the
hierarchy.
--
devlink: give user option to allocate resources
This adds a new mode of operation for devlink resources. In this mode a
driver registers an occ_set() callback together with occ_get() through
devl_resource_occ_set_get_register(). DEVLINK_CMD_RESOURCE_SET then
applies the requested size immediately and does not stage it until
reload. For such resources the getter value is reported as
DEVLINK_ATTR_RESOURCE_SIZE.
> diff --git a/net/devlink/resource.c b/net/devlink/resource.c
> index c3cfda7ea0703..c4502c480a6bb 100644
> --- a/net/devlink/resource.c
> +++ b/net/devlink/resource.c
[ ... ]
> @@ -137,6 +139,16 @@ int devlink_nl_resource_set_doit(struct sk_buff *skb, struct genl_info *info)
> if (err)
> return err;
>
> + if (resource->occ_set) {
> + err = resource->occ_set(size, info->extack, resource->occ_priv);
> + if (err)
> + return err;
> +
> + resource->size = resource->occ_get(resource->occ_priv);
> + resource->size_new = resource->size;
> + return 0;
> + }
> +
> resource->size_new = size;
> devlink_resource_validate_children(resource);
> if (resource->parent)
[Severity: Low]
Should the occ_set branch also revalidate the hierarchy before it returns?
The legacy branch below sets size_new and then calls
devlink_resource_validate_children() on the resource and on its parent.
The occ_set branch changes size_new and returns before doing either.
The occ_set refresh added to devl_resource_size_get() also overwrites
size_new without revalidating.
devlink_resource_validate_children() computes the parent's size_valid
from the sum of the children's size_new. That flag is reported as
DEVLINK_ATTR_RESOURCE_SIZE_VALID. It is also checked by
devlink_resources_validate(), which gates DEVLINK_CMD_RELOAD in
devlink_nl_reload_doit():
err = devlink_resources_validate(devlink, NULL, info);
if (err) {
NL_SET_ERR_MSG(info->extack, "resources size validation failed");
Nothing stops one hierarchy from mixing the two modes. Take a legacy
parent P with an occ_set child A and a legacy child B. Setting B through
the legacy path marks P invalid. If A is then shrunk through occ_set so
the sum fits again, does P stay invalid, so that reload is refused? In
the other direction, growing A through occ_set would leave P reported
as valid.
No in-tree user mixes the modes today. In this series ice registers rss,
lut_512 and lut_2048 all with devl_resource_occ_set_get_register(), so
this is latent. Should this path revalidate mixed hierarchies, or should
they be rejected or documented as unsupported?
[ ... ]
> @@ -684,9 +748,10 @@ void devl_resource_occ_get_unregister(struct devlink *devlink,
> if (WARN_ON(!resource))
> return;
> WARN_ON(!resource->occ_get);
> + WARN_ON(resource->occ_set);
>
> resource->occ_get = NULL;
> - resource->occ_get_priv = NULL;
> + resource->occ_priv = NULL;
> }
> EXPORT_SYMBOL_GPL(devl_resource_occ_get_unregister);
[Severity: Low]
Should this WARN_ON(resource->occ_set) return instead of falling
through?
devl_resource_occ_set_get_register() is exported, but
include/net/devlink.h declares no matching unregister. That leaves
devl_resource_occ_get_unregister() as the only detach helper. If it is
called on a resource registered with
devl_resource_occ_set_get_register(), it warns, clears occ_get and
occ_priv, and leaves occ_set set.
The three mode checks then disagree. devlink_resource_occ_size_put()
requires both callbacks, so it takes the legacy path. But
devlink_nl_resource_set_doit() and devl_resource_size_get() check only
occ_set:
if (resource->occ_set) {
err = resource->occ_set(size, info->extack, resource->occ_priv);
...
resource->size = resource->occ_get(resource->occ_priv);
Would a later DEVLINK_CMD_RESOURCE_SET call occ_set() with a NULL priv
and then call through the NULL occ_get pointer?
devl_resource_size_get() would make the same call to occ_get.
Nothing in-tree reaches this today. The only user in the series is ice,
which tears down with devl_resources_unregister() and never calls
devl_resource_occ_get_unregister(). Drivers that follow the existing
convention keep resources registered across reload and unregister the
callbacks in fini or reload_down. The new mode offers no way to do that.
Would it make sense to add a matching unregister helper, or have this
one also clear occ_set? Could all three places use the same mode check?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [net-next v6 13/15] ice: represent RSS LUTs as devlink resources
2026-11-17 9:44 ` [net-next v6 13/15] ice: represent RSS LUTs as devlink resources Przemek Kitszel
@ 2026-11-19 6:45 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:45 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found 6 potential
issue(s) to consider.
Critical: 0 · High: 1 · Medium: 3 · Low: 2
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [High] ABBA deadlock between the PF devlink instance lock and rtnl.
- [Medium] Cross-PF race on the shared global RSS LUT table during
device-wide resets (CORER/GLOBR/EMPR).
- [Medium] [Baseline: discarded, fixed later in the series] At this
commit, switching the PF from its PF LUT (up to 256 queues) to a…
- [Medium] ice: premature rate teardown and leaked flag in
ice_setup_tc_mqprio_qdisc()
- [Low] ice_set_safe_mode_caps() now keeps dev_caps.nac_topo across the
safe-mode caps reset.
- [Low] [Baseline: discarded as defensive] The concern is that the global
LUT ID returned by firmware from ice_alloc_rss_global_lut() is used…
--
ice: represent RSS LUTs as devlink resources
This patch exposes the E800 RSS LUTs (the 512 entry global LUTs and the
2048 entry PF LUTs) as devlink resources on each PF devlink instance and
on the whole device devlink instance. A PF can switch between its PF LUT
and a global LUT with devlink resource set. LUT ownership is tracked in
a table on the shared ice_adapter, the default assignment is restored
after reset, and a new pf->rss_lut_lock serializes RSS LUT configuration.
> diff --git a/drivers/net/ethernet/intel/ice/devlink/resource.c b/drivers/net/ethernet/intel/ice/devlink/resource.c
> new file mode 100644
> index 0000000000000..de9c13e928fae
> --- /dev/null
> +++ b/drivers/net/ethernet/intel/ice/devlink/resource.c
> @@ -0,0 +1,663 @@
[ ... ]
> +static int ice_devl_res_take(struct ice_pf *pf,
> + enum ice_devl_resource_id res_id, int slot,
> + void *owner)
> +{
> + struct ice_devl_resource *res = ice_devl_res_tbl(pf, res_id);
> + int beg, end, new_id = ICE_ANY_SLOT;
> + u16 lut_id;
> +
> + if (res_id == ICE_RSS_LUT_GLOBAL) {
> + int err;
> +
> + WARN_ON_ONCE(slot != ICE_ANY_SLOT);
> +
> + err = ice_alloc_rss_global_lut(&pf->hw, &lut_id);
> + if (err)
> + return err;
> +
> + slot = lut_id;
[Severity: Low]
This isn't a bug, but lut_id comes straight from firmware through
ice_alloc_rss_global_lut(). It is then used as an index into the 16
entry res->owner[] and res->pf_id[] arrays.
The hardware field ICE_AQ_VSI_Q_OPT_RSS_GBL_LUT_M is 4 bits wide, so a
value above 15 would take a firmware bug. Would a check against
ARRAY_SIZE(res->owner) still be worth adding as hardening?
> + }
> +
> + end = slot == ICE_ANY_SLOT ? ARRAY_SIZE(res->owner) : slot + 1;
> + beg = slot == ICE_ANY_SLOT ? 0 : slot;
> + for (int id = beg; id < end; id++) {
> + if (!res->owner[id]) {
> + new_id = id;
> + break;
> + }
> + }
> + if (new_id == ICE_ANY_SLOT) {
> + if (WARN_ON(res_id == ICE_RSS_LUT_GLOBAL))
> + ice_free_rss_global_lut(&pf->hw, lut_id);
[Severity: Medium]
Can this WARN_ON() fire after a device-wide reset (CORER, GLOBR or
EMPR)?
That kind of reset frees the global LUTs of every PF in firmware at
once. A PF that did not start the reset only removes its stale entries
from adapter->resources[ICE_RSS_LUT_GLOBAL] (or sec_nac_global_luts)
when its own service task gets to them:
ice_reset_subtask()
ice_prepare_for_reset()
ice_prepare_rss_luts_for_reset()
Say PF B finishes its rebuild, and an admin runs devlink resource set on
B before PF A's service task has done that cleanup. ice_devl_res_change()
only checks B's own reset state:
if (ice_is_reset_in_progress(pf->state)) {
Firmware can then hand B an ID that owner[] still records as owned by
A. The loop above finds no free slot, the WARN_ON() fires, and the user
gets -ENOSPC.
The v4 changelog handles this race only for PF-initiated resets. Should
the device-wide case be covered too?
> +
> + return -ENOSPC;
> + }
[ ... ]
> +static int ice_maybe_change_rss_lut(struct ice_pf *pf, void *owner,
> + enum ice_rss_lut_resource_state old,
> + enum ice_rss_lut_resource_state new,
> + struct netlink_ext_ack *extack)
> +{
[ ... ]
> + lut_size = ice_lut_type_to_size(lut_type);
> + lut = kzalloc(lut_size, GFP_KERNEL);
> + if (!lut)
> + return -ENOMEM;
> + params.lut = lut;
> + params.lut_size = lut_size;
> + params.lut_type = lut_type;
> + params.vsi_handle = vsi->idx;
> + mutex_lock(&pf->rss_lut_lock);
> + /* PF with RXHASH disabled keeps its LUT zeroed */
> + if (!vsi->netdev || vsi->netdev->features & NETIF_F_RXHASH)
> + ice_fill_rss_lut(lut, lut_size, vsi->rss_size);
[Severity: Medium]
When the PF moves from its PF LUT (up to 256 queues) to a global LUT (up
to 64 queues), is vsi->rss_size limited anywhere here?
vsi->rss_size was sized for the PF LUT. At this commit,
ice_get_valid_rss_size() only clamps by rss_table_entry_width, and
ice_vsi_cfg_rss_lut_key() has no clamp by LUT type either. A PF with
more than 64 RSS queues would then get queue indices in the global LUT
that it cannot use.
The later patch in this series, "ice: support up to 256 VF queues", adds
ice_lut_type_to_qs_num() and clamps rss_size in
ice_maybe_change_rss_lut(), ice_get_valid_rss_size() and
ice_vsi_cfg_rss_lut_key(). Could that clamping move into this patch, so
this commit is correct on its own?
> + err = ice_aq_set_rss_lut(hw, ¶ms);
[ ... ]
> +/* ethtool core sizes RSS buffers by rss_table_size under rtnl, and the new
> + * LUT is filled based on NETIF_F_RXHASH, which is also changed under rtnl
> + */
> +static int ice_rss_lut_pf_occ_set(u64 size, enum ice_devl_resource_id res_id,
> + struct ice_pf *pf, int slot,
> + struct netlink_ext_ack *extack)
> +{
> + int err;
> +
> + rtnl_lock();
> + scoped_guard(ice_adapter_devl, pf->adapter)
> + err = ice_devl_res_change(size, res_id, pf, pf, slot, extack);
> + rtnl_unlock();
[Severity: High]
Can this deadlock against the tc mqprio path?
ice_rss_lut_pf_occ_set() is reached from DEVLINK_CMD_RESOURCE_SET.
There, devlink_nl_pre_doit() already holds the PF devlink instance lock
before devlink_nl_resource_set_doit() calls occ_set. So this path takes
the PF devl lock, then rtnl.
ice_setup_tc_mqprio_qdisc() runs under rtnl from ndo_setup_tc. In
TC_MQPRIO_MODE_CHANNEL it calls ice_tear_down_devlink_rate_tree(),
which does:
devlink = priv_to_devlink(pf);
devl_lock(devlink);
That path takes rtnl, then the PF devl lock. ice_pf_dcb_cfg() does the
same from dcbnl.
CPU0 CPU1
tc qdisc add ... mqprio mode channel devlink resource set ... lut_512
rtnl_lock() devlink_nl_pre_doit()
ice_setup_tc_mqprio_qdisc() devl_lock(PF devlink)
ice_tear_down_devlink_rate_tree() devlink_nl_resource_set_doit()
devl_lock(PF devlink) ice_rss_lut_pf_occ_set()
rtnl_lock()
The rtnl then devl lock order in ice_tear_down_devlink_rate_tree() was
there before this patch. The new resource set path adds the opposite
order, and both are still in place at the end of the series.
> +
> + return err;
> +}
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
> index 90e01eb49ea21..48c3449a2434d 100644
> --- a/drivers/net/ethernet/intel/ice/ice_common.c
> +++ b/drivers/net/ethernet/intel/ice/ice_common.c
[ ... ]
> @@ -3217,6 +3218,7 @@ void ice_set_safe_mode_caps(struct ice_hw *hw)
> /* cache some dev_caps values that should be restored after memset */
> cached_caps = dev_caps->common_cap;
> num_funcs = dev_caps->num_funcs;
> + nac_topo = dev_caps->nac_topo;
>
> /* unset dev capabilities */
> memset(dev_caps, 0, sizeof(*dev_caps));
> @@ -3235,6 +3237,7 @@ void ice_set_safe_mode_caps(struct ice_hw *hw)
> ICE_RESTORE_DEV_CAP(nvm_update_pending_orom);
> ICE_RESTORE_DEV_CAP(nvm_update_pending_netlist);
> dev_caps->num_funcs = num_funcs;
> + dev_caps->nac_topo = nac_topo;
[Severity: Low]
This changes what ice_is_dual() and ice_is_primary() return in safe mode
for every caller, not only the new devlink resource code.
Before this patch, every safe-mode PF on a dual-NAC E825 counted as
primary and non-dual. Existing users include the PHY lane computation in
ice_common.c:
if (hw->mac_type == ICE_MAC_GENERIC_3K_E825 &&
ice_is_dual(hw) && !ice_is_primary(hw))
lane += ICE_PORTS_PER_QUAD;
Others are the sideband destination selection and the CPI PHY selection
in ice_cpi.c.
The commit message doesn't mention this. Only the v5 changelog below
the --- line does, and that text is dropped when the patch is applied.
Could this go into its own patch, or at least be described in the
commit message?
>
> /* one Tx and one Rx queue per function in safe mode */
> dev_caps->common_cap.num_rxq = num_funcs;
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
> --- a/drivers/net/ethernet/intel/ice/ice_main.c
> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
> @@ -9192,8 +9211,14 @@ static int ice_setup_tc_mqprio_qdisc(struct net_device *netdev, void *type_data)
> ret);
> return ret;
> }
> + scoped_guard(ice_adapter_devl, pf->adapter) {
> + if (ice_rss_lut_is_reassigned(pf)) {
> + dev_err(dev, "RSS LUTs reassigned via devlink, can't configure ADQ\n");
> + return -EOPNOTSUPP;
> + }
> + set_bit(ICE_FLAG_TC_MQPRIO, pf->flags);
> + }
> memcpy(&vsi->mqprio_qopt, mqprio_qopt, sizeof(*mqprio_qopt));
> - set_bit(ICE_FLAG_TC_MQPRIO, pf->flags);
> /* don't assume state of hw_tc_offload during driver load
> * and set the flag for TC flower filter if hw_tc_offload
> * already ON
[Severity: Medium]
Does this check come too late in ice_setup_tc_mqprio_qdisc()?
In the TC_MQPRIO_MODE_CHANNEL case, ice_tear_down_devlink_rate_tree()
has already run by the time ice_rss_lut_is_reassigned() is checked.
Say an admin tries to enable ADQ on a PF whose RSS LUTs were moved via
devlink. The command fails with -EOPNOTSUPP, but the PF's VF devlink
rate leaves and rate nodes have already been destroyed. A rejected
command should not have that side effect.
Could the reassigned check move ahead of the rate tree teardown?
ice_validate_mqprio_qopt() is ordered the same way and has the same
problem.
The flag has a second problem. ICE_FLAG_TC_MQPRIO is now set under the
adapter devlink lock, but nothing clears it if the rest of the ADQ
setup fails:
- the second ice_vsi_rebuild() failure returns directly;
- failures in ice_vsi_cfg_tc(), ice_set_bw_limit() or
ice_create_q_channels() jump to the exit label, which only resets
all_numtc and all_enatc.
This leak predates the patch, but the patch gives it a new effect.
ice_devl_res_change() tests the raw ICE_FLAG_TC_MQPRIO bit, not the
real ADQ state the way ice_is_adq_active() does. After a failed ADQ
setup, every devlink RSS LUT change is refused with "ADQ configured,
can't change RSS LUTs", even though no ADQ is in place.
Should those error paths clear the flag again, under the same adapter
devlink lock? Or should ice_devl_res_change() check whether ADQ is
actually configured?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [net-next v6 14/15] devlink: add devlink_alloc_drv()
2026-11-17 9:44 ` [net-next v6 14/15] devlink: add devlink_alloc_drv() Przemek Kitszel
@ 2026-11-19 6:45 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:45 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found more severe
issue(s) in other patches of this series. This mail is only an FYI, in case
you want to address the nit picks below when you respin.
Low priority / nit picks for your consideration:
- [Low] devlink_alloc_drv() passes @dev straight to __devlink_alloc()
with no WARN_ON(!dev).
--
devlink: add devlink_alloc_drv()
This adds devlink_alloc_drv(), a variant of devlink_alloc() that takes the
owning driver explicitly. It is meant for a PF driver that creates devlink
instances for its VFs, where the VF device may be bound to another driver
or to none. The devlink-info documentation is updated to say the reported
driver is the creating driver in that case.
> diff --git a/net/devlink/core.c b/net/devlink/core.c
> index 9b844e0c1bd2d..e2a7169287ec7 100644
> --- a/net/devlink/core.c
> +++ b/net/devlink/core.c
> @@ -531,6 +531,30 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops,
> }
> EXPORT_SYMBOL_GPL(devlink_alloc_ns);
>
> +/**
> + * devlink_alloc_drv - Allocate new devlink instance resources for given driver
> + * @ops: ops
> + * @priv_size: size of user private data
> + * @dev: parent device
> + * @drv: driver that owns the instance
> + *
> + * Variant of devlink_alloc() for a driver that creates an instance for
> + * a device bound to another (or no) driver, e.g. a PF driver for its VFs.
> + *
> + * @drv must not be NULL and must outlive the instance, only a pointer to it
> + * is kept. Operations that take the device lock, like reload, lock @dev,
> + * not a device bound to @drv.
> + *
> + * Return: new devlink instance, NULL on failure.
> + */
> +struct devlink *devlink_alloc_drv(const struct devlink_ops *ops,
> + size_t priv_size, struct device *dev,
> + const struct device_driver *drv)
> +{
> + return __devlink_alloc(ops, priv_size, &init_net, dev, drv);
> +}
[Severity: Low]
This isn't a bug, but should devlink_alloc_drv() also have the
WARN_ON(!dev) that devlink_alloc_ns() has?
net/devlink/core.c:devlink_alloc_ns() {
WARN_ON(!dev);
return __devlink_alloc(ops, priv_size, net, dev, dev->driver);
}
The kernel-doc calls @dev the parent device and says reload locks @dev.
If @dev is NULL, __devlink_alloc() goes ahead and creates an index-named
instance, like the one devlink_shd_get() creates. Then devl_dev_lock()
skips the device lock without any warning:
net/devlink/devl_internal.h:devl_dev_lock() {
if (dev_lock && devlink->dev)
device_lock(devlink->dev);
devl_lock(devlink);
}
That doesn't match the kernel-doc.
The same applies to @drv. The kernel-doc says it must not be NULL, but a
NULL @drv only hits the WARN_ON(!ops || !dev_driver) in __devlink_alloc(),
and allocation continues.
The only current caller, ice_init_vf_devlink() in the next patch, returns
early when vf->vfdev is NULL, so no current path passes a NULL @dev. Could
a check for dev be added here (and maybe for drv too) so this variant
behaves like devlink_alloc_ns() and the kernel-doc?
> +EXPORT_SYMBOL_GPL(devlink_alloc_drv);
> +
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [net-next v6 15/15] ice: support up to 256 VF queues
2026-11-17 9:44 ` [net-next v6 15/15] ice: support up to 256 VF queues Przemek Kitszel
@ 2026-11-19 6:45 ` netdev-bot+sashiko
0 siblings, 0 replies; 25+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19 6:45 UTC (permalink / raw)
To: przemyslaw.kitszel
Cc: kuba, netdev, aleksandr.loktionov, andrew+netdev,
anthony.l.nguyen, anzaki, brett.creeley, corbet, davem, edumazet,
horms, jacob.e.keller, jedrzej.jagielski, jiri, leon, mbloch,
mschmidt, ohartoov, pabeni, poros, rdunlap, saeedm, skhan, tariqt
Thank you for your contribution! Sashiko AI review found 8 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 4 · Low: 4
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The patch caps the RSS queue count by LUT capacity
(ice_lut_type_to_qs_num(): 64 for ICE_LUT_GLOBAL) only on paths where…
- [Medium] The new ice_vc_get_max_rss_qregion() tells the VF that its RSS
queue region is 2^qregion_width queues:…
- [Medium] The new VF devlink LUT path (ice_rss_lut_vf_occ_set_pf/global
-> ice_devl_res_change -> ice_maybe_change_rss_lut) serializes VF VSI…
- [Medium] ice: lock ordering inversion in ice_init_vf_devlink() under
vfs.table_lock
- [Low] This patch caps rss_size by LUT capacity for every VSI type, the
PF VSI included.
- [Low] The commit message that goes into git history leaves out several
changes made by this patch.
- [Low] In ice_vf_max_contig_qs(), if either bitmap_alloc() fails the
code jumps to 'out' and returns min(max_tx, max_rx) = min(0, 0) = 0,…
- [Low] Goto-based cleanup is mixed with scope-based guards in the same
function, which the cleanup.h rule in the subsystem guide forbids.
--
ice: support up to 256 VF queues
This commit lets a VF use more than 16 queues: up to 64 with a global RSS
LUT, and up to 256 with a PF LUT. To do that it adds per-VF devlink
instances that expose the RSS LUT resources, defers VF resets to the
service task, and wires up the v2 queue virtchnl opcodes. It also changes
VF queue requests so they must fit in one contiguous free range of PF
queues.
> diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
> index 6e34c107b28d2..90ec1c7848d89 100644
> --- a/Documentation/networking/devlink/ice.rst
> +++ b/Documentation/networking/devlink/ice.rst
[ ... ]
> @@ -520,6 +523,21 @@ A PF must own at least one of the two LUTs at all times, so switching a PF
> to a global LUT takes two steps: take a global LUT, then release the PF LUT.
> A PF that owns both uses its PF LUT.
>
> +A VF uses its VSI LUT by default, which spreads traffic over up to 16
> +queues. With a global LUT the VF can use up to 64 queues, and up to 256 with
> +a PF LUT (preferred when the VF owns both). Changing the LUT of a VF resets
[Severity: Low]
This patch also caps the PF VSI by ice_lut_type_to_qs_num(). The cap is
applied in three places, for every VSI type:
- ice_get_valid_rss_size()
- ice_maybe_change_rss_lut()
- ice_vsi_cfg_rss_lut_key()
So a PF that is moved to a global LUT now spreads RSS over at most 64
queues. Step 2 of the 256-queue VF recipe in the commit message requires
exactly that move.
Should this section and the commit message mention the 64-queue limit for
a PF on a global LUT? Only the VF limits are documented here.
Also, ice_set_channels() still accepts "ethtool -L combined N" with N > 64,
up to ice_get_max_rxq(). It then clamps rss_size without any message:
scoped_guard(mutex, &pf->rss_lut_lock)
vsi->rss_size = ice_get_valid_rss_size(vsi, new_rx);
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/devlink/resource.c b/drivers/net/ethernet/intel/ice/devlink/resource.c
> index de9c13e928fae..ec3a095deb9f2 100644
> --- a/drivers/net/ethernet/intel/ice/devlink/resource.c
> +++ b/drivers/net/ethernet/intel/ice/devlink/resource.c
[ ... ]
> @@ -299,6 +341,15 @@ static int ice_maybe_change_rss_lut(struct ice_pf *pf, void *owner,
> vsi->global_lut_id = params.global_lut_id;
> vsi->rss_table_size = lut_size;
> vsi->rss_lut_type = lut_type;
> + vsi->rss_size = rss_size;
> + if (vf) {
> + u16 max_qs = ice_lut_type_to_qs_num(lut_type);
> +
> + /* don't keep queues that the new LUT can't spread traffic to */
> + if ((vf->num_req_qs ?: vf->num_vf_qs) > max_qs)
> + vf->num_req_qs = max_qs;
> + ice_schedule_vf_reset(vf);
> + }
[Severity: Medium]
Is this VF path serialized against ice_sriov_set_msix_vec_count()?
The devlink path holds vf->cfg_lock, the adapter devl lock and
pf->rss_lut_lock, and is gated by ice_vf_is_ready().
The sriov_vf_msix_count path only holds the PF and VF device_lock. It
takes none of the locks above and does not set ICE_VF_STATE_DIS:
ice_sriov_set_msix_vec_count()
vf->num_vf_qs = queues;
...
ice_vsi_rebuild(vsi, ICE_VSI_FLAG_NO_INIT)
ice_vsi_set_rss_params() /* writes vsi->rss_size */
ice_vsi_init()
ice_set_rss_vsi_ctx()
lut_type = ice_lut_type_to_aq_qopt_rss_val(vsi->rss_lut_type);
ice_update_vsi()
vsi->info = ctxt->info;
sriov_vf_msix_count requires that no driver is bound to the VF, but the VF
devlink instance is still registered. So this interleaving looks possible:
CPU1 (msix rebuild) CPU2 (devlink resource set on the VF)
ice_set_rss_vsi_ctx()
reads old rss_lut_type
ice_aq_set_rss_lut()
ice_vsi_update_rss_lut()
updates VSI ctx and vsi->info.q_opt_rss
ice_update_vsi() with old type
vsi->info = ctxt->info
vsi->rss_lut_type = lut_type;
Could the VSI context in hardware and vsi->rss_lut_type then disagree about
which LUT the VF VSI uses? The deferred reset scheduled here might repair
the context later.
In the same window, the num_req_qs clamp above reads vf->num_vf_qs while
the msix path is rewriting it.
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
> index 822b94f87156f..6f393e13b3748 100644
> --- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
> +++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
> @@ -3831,14 +3831,15 @@ ice_get_channels(struct net_device *dev, struct ethtool_channels *ch)
[ ... ]
> -static int ice_get_valid_rss_size(struct ice_hw *hw, int new_size)
> +static int ice_get_valid_rss_size(struct ice_vsi *vsi, int new_size)
> {
> - struct ice_hw_common_caps *caps = &hw->func_caps.common_cap;
> + struct ice_hw_common_caps *caps = &vsi->back->hw.func_caps.common_cap;
>
> - return min_t(int, new_size, BIT(caps->rss_table_entry_width));
> + new_size = min_t(int, new_size, BIT(caps->rss_table_entry_width));
> + return min_t(int, new_size, ice_lut_type_to_qs_num(vsi->rss_lut_type));
> }
[Severity: Medium]
This caps the queue count for LUTs the driver fills itself. Is the same cap
needed for user indirection tables?
ice_set_rxfh() still copies the user entries directly:
for (i = 0; i < vsi->rss_table_size; i++)
vsi->rss_lut_user[i] = (u8)(rxfh->indir[i]);
It then calls ice_set_rss_lut() without checking the entries against
ice_lut_type_to_qs_num(vsi->rss_lut_type). The ethtool core only rejects
entries >= the number of Rx rings.
Take a PF with more than 64 Rx queues that has been moved to a global LUT.
"ethtool -X equal 128" would be accepted and written to a LUT that this
patch treats as able to address only 64 queues.
ice_set_channels() keeps a configured user LUT across ethtool -L and only
updates rss_size, so the table is never checked against the cap later
either.
Moving the PF to a global LUT was added earlier in this series ("ice:
represent RSS LUTs as devlink resources"). What does the hardware do with
entries >= 64 in a global LUT? ethtool -x would report the stored table
either way.
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/virt/queues.c b/drivers/net/ethernet/intel/ice/virt/queues.c
> index f87ccf5d370bb..76f2ba7790537 100644
> --- a/drivers/net/ethernet/intel/ice/virt/queues.c
> +++ b/drivers/net/ethernet/intel/ice/virt/queues.c
> @@ -986,6 +986,48 @@ int ice_vc_cfg_qs_msg(struct ice_vf *vf, u8 *msg)
[ ... ]
> +static int ice_vf_max_contig_qs(struct ice_vf *vf, struct ice_vsi *vsi)
> +{
> + struct ice_pf *pf = vf->pf;
> + unsigned long *txqs, *rxqs;
> + int max_tx = 0, max_rx = 0;
> + int b, e;
> +
> + txqs = bitmap_alloc(pf->max_pf_txqs, GFP_KERNEL);
> + rxqs = bitmap_alloc(pf->max_pf_rxqs, GFP_KERNEL);
> + if (!txqs || !rxqs)
> + goto out;
> +
> + scoped_guard(mutex, &pf->avail_q_mutex) {
> + bitmap_copy(txqs, pf->avail_txqs, pf->max_pf_txqs);
> + bitmap_copy(rxqs, pf->avail_rxqs, pf->max_pf_rxqs);
> + }
[Severity: Low]
This isn't a bug, but this function mixes goto based cleanup (goto out for
the two bitmaps) with scoped_guard(). include/linux/cleanup.h asks that
goto and cleanup helpers are not mixed in the same function.
Would "unsigned long *txqs __free(bitmap) = bitmap_alloc(...)" with plain
returns work here?
The same applies to the scoped_guard(mutex, &pf->rss_lut_lock) added to
ice_set_channels(), which already exits through goto adev_unlock with
manual unlocks.
> + for (int i = 0; i < vsi->alloc_txq; i++)
> + clear_bit(vsi->txq_map[i], txqs);
[ ... ]
> +out:
> + bitmap_free(txqs);
> + bitmap_free(rxqs);
> +
> + return min(max_tx, max_rx);
> +}
[ ... ]
> @@ -1014,23 +1056,31 @@ int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg)
[ ... ]
> + max_allowed_vf_queues = min(ice_vf_max_contig_qs(vf, vsi),
> + max_lut_queues);
[ ... ]
> + } else if (req_queues > max_allowed_vf_queues) {
> + dev_warn(dev, "VF %d requested %u queues, but only %u are free in one range.\n",
> + vf->vf_id, req_queues, max_allowed_vf_queues);
> + vfres->num_queue_pairs = max_allowed_vf_queues;
[Severity: Low]
What happens here if either bitmap_alloc() in ice_vf_max_contig_qs() fails?
It jumps to out with max_tx and max_rx still 0, and returns 0. That is the
same value as "no free range".
max_allowed_vf_queues is then 0. Any req_queues in 1..max_lut_queues takes
this branch, logs "only 0 are free in one range", and replies
VIRTCHNL_STATUS_SUCCESS with num_queue_pairs = 0. That happens even when
the request is not larger than the VF's current queue count.
Should the allocation failure be reported differently, for example with
VIRTCHNL_STATUS_ERR_NO_MEMORY?
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/virt/rss.c b/drivers/net/ethernet/intel/ice/virt/rss.c
> index 960012ca91b53..99d9295af79c0 100644
> --- a/drivers/net/ethernet/intel/ice/virt/rss.c
> +++ b/drivers/net/ethernet/intel/ice/virt/rss.c
[ ... ]
> @@ -1746,24 +1749,38 @@ int ice_vc_config_rss_lut(struct ice_vf *vf, u8 *msg)
[ ... ]
> + /* the LUT size reported in the GET_VF_RESOURCES reply */
> + if (vf->driver_caps & VIRTCHNL_VF_LARGE_NUM_QPAIRS)
> + lut_size = vsi->rss_table_size;
> + if (vrl->lut_entries != lut_size) {
> v_ret = VIRTCHNL_STATUS_ERR_PARAM;
> goto error_param;
> }
[ ... ]
> - if (ice_set_rss_lut(vsi, vrl->lut, ICE_LUT_VSI_SIZE))
> + lut = kmalloc(vsi->rss_table_size, GFP_KERNEL);
> + if (!lut) {
> + v_ret = VIRTCHNL_STATUS_ERR_NO_MEMORY;
> + goto error_param;
> + }
> +
> + /* repeat the LUT of the VF over the whole RSS LUT of its VSI */
> + for (int i = 0; i < vsi->rss_table_size; i++)
> + lut[i] = vrl->lut[i % lut_size];
> +
> + if (ice_set_rss_lut(vsi, lut, vsi->rss_table_size))
> v_ret = VIRTCHNL_STATUS_ERR_ADMIN_QUEUE_ERROR;
> + kfree(lut);
[Severity: Medium]
The new ice_vc_get_max_rss_qregion() advertises a region width of
ilog2(ice_lut_type_to_qs_num(vsi->rss_lut_type)), so 16, 64 or 256 queues.
Does anything here check the VF's entries against that region, or against
vsi->num_rxq?
Only vrl->lut_entries is compared with lut_size. The entries themselves are
repeated over the 512 or 2048 entry LUT and programmed as they are.
A guest driver that ignores the advertised region could then write values
up to 255, including queues the VF does not own. This patch is the first to
let a VF VSI use a global or PF LUT.
For comparison, i40e_vc_config_rss_lut() rejects entries >=
vf->num_queue_pairs. Is a similar check needed here?
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/virt/virtchnl.c b/drivers/net/ethernet/intel/ice/virt/virtchnl.c
> index ca8018e3dd428..778637f2868c2 100644
> --- a/drivers/net/ethernet/intel/ice/virt/virtchnl.c
> +++ b/drivers/net/ethernet/intel/ice/virt/virtchnl.c
[ ... ]
> @@ -2533,6 +2539,10 @@ static const struct ice_virtchnl_ops ice_virtchnl_dflt_ops = {
> .cfg_q_quanta = ice_vc_cfg_q_quanta,
> .get_ptp_cap = ice_vc_get_ptp_cap,
> .get_phc_time = ice_vc_get_phc_time,
> + .get_max_rss_qregion = ice_vc_get_max_rss_qregion,
> + .ena_qs_v2_msg = ice_vc_ena_qs_v2_msg,
> + .dis_qs_v2_msg = ice_vc_dis_qs_v2_msg,
> + .map_q_vector_msg = ice_vc_map_q_vector_msg,
[ ... ]
> @@ -2766,6 +2780,13 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event,
> goto error_handler;
> }
>
> + /* VF must not see a config change before the reset that applies it */
> + if (test_and_clear_bit(ICE_VF_STATE_NEEDS_RESET, vf->vf_states)) {
> + dev_info(dev, "doing deferred reset of VF %d\n", vf_id);
> + ice_reset_vf(vf, ICE_VF_RESET_NOTIFY);
> + goto finish;
> + }
[Severity: Low]
This isn't a bug, but the commit message leaves out several behaviour
changes made by this patch:
- It wires up four virtchnl handlers that a guest can reach for the first
time: get_max_rss_qregion, ena_qs_v2_msg, dis_qs_v2_msg and
map_q_vector_msg. It also adds vf_ops->get_q_vector for them. The
allowlist already permits these opcodes once VIRTCHNL_VF_LARGE_NUM_QPAIRS
is granted, and this patch is the first to grant that capability.
- The VIRTCHNL_OP_REQUEST_QUEUES grant policy changes for every VF. A
request must now fit in the longest contiguous free Tx/Rx range from
ice_vf_max_contig_qs(), not the total free count. VFs without
LARGE_NUM_QPAIRS are capped at ICE_MAX_QS_PER_VF_VCV1. Requests above
the LUT limit are answered without a log line.
- The deferred VF reset can also run synchronously here, on the VF's next
mailbox message. That message is then dropped without a reply through
goto finish.
- ice_set_vf_state_dis() now sets ICE_VF_STATE_DIS on every PF reset
preparation, whether or not RSS LUTs are in use.
The second and fourth points are only described in the v4/v5 changelog
below the --- line, which won't be in git history. Could the commit message
mention these changes?
[ ... ]
> diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
> --- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c
> +++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
> @@ -1450,3 +1473,52 @@ void ice_vf_update_mac_lldp_num(struct ice_vf *vf, struct ice_vsi *vsi,
> if (was_ena != is_ena)
> ice_vsi_cfg_sw_lldp(vsi, false, is_ena);
> }
> +
> +void ice_init_vf_devlink(struct ice_vf *vf)
> +{
> + struct devlink *pf_devlink = priv_to_devlink(vf->pf);
> + static const struct devlink_ops noop = {};
> + struct devlink *devlink;
> + int err;
> +
> + lockdep_assert_held(&vf->pf->vfs.table_lock);
> +
> + /* no VF pci_dev when the VFs were not scanned, e.g. on s390 */
> + if (!vf->vfdev)
> + return;
> +
> + devlink = devlink_alloc_drv(&noop, 0, &vf->vfdev->dev,
> + ice_pf_to_dev(vf->pf)->driver);
> + if (!devlink)
> + return;
> +
> + scoped_guard(devl, pf_devlink)
> + err = devl_nested_devlink_set(pf_devlink, devlink);
> + if (err) {
> + devlink_free(devlink);
> + return;
> + }
> +
> + vf->devlink = devlink;
> + scoped_guard(devl, devlink) {
> + ice_devlink_vf_resources_register(vf);
> + devl_register(devlink);
> + }
> +}
> +
> +void ice_deinit_vf_devlink(struct ice_vf *vf)
> +{
> + struct devlink *devlink = vf->devlink;
> +
> + lockdep_assert_held(&vf->pf->vfs.table_lock);
> +
> + if (!devlink)
> + return;
> +
> + vf->devlink = NULL;
> + scoped_guard(devl, devlink) {
> + devl_unregister(devlink);
> + devl_resources_unregister(devlink);
> + }
> + devlink_free(devlink);
> +}
[Severity: Medium]
Does this create an AB-BA ordering between pf->vfs.table_lock and the
PF devlink instance lock?
ice_ena_vfs() takes pf->vfs.table_lock and calls ice_start_vfs(). That
now calls ice_init_vf_devlink() for every VF, including in the default
legacy eswitch mode. The PF devlink lock is then taken here while
table_lock is held:
ice_ena_vfs()
mutex_lock(&pf->vfs.table_lock)
ice_start_vfs()
ice_init_vf_devlink()
devl_lock(priv_to_devlink(pf))
ice_tear_down_devlink_rate_tree() takes the two locks in the opposite
order:
devl_lock(devlink);
mutex_lock(&pf->vfs.table_lock);
It is reached from two places:
- ice_setup_tc_mqprio_qdisc(), when ADQ is configured
- ice_pf_dcb_cfg(), when DCB is configured with more than one TC
sriov_numvfs_store() only holds the PF device_lock, not rtnl or
pf->tc_mutex. So enabling VFs while ADQ or DCB is being configured
looks like it could deadlock. Even when the two paths run one after
the other, lockdep should report a circular dependency.
Could devl_nested_devlink_set() be done after table_lock is dropped?
Alternatively, could ice_ena_vfs() take the PF devlink lock before
table_lock?
The lockdep_assert_held() on table_lock in ice_init_vf_devlink()
suggests the first option would need some other way to keep the VF
entry alive.
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261117094913.100756-1-przemyslaw.kitszel%40intel.com
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2026-11-19 6:45 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 9:44 [net-next v6 00/15] devlink, mlx5, iavf, ice: XLVF for iavf Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 01/15] devlink, mlx5: add init/fini ops for shared devlink Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 02/15] ice: use shared devlink to store ice_adapters instead of custom xarray Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 03/15] ice: add VF queue ena/dis helper functions Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 04/15] ice: add helpers for Global RSS LUT alloc, free, vsi_update Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 05/15] ice: rename ICE_MAX_RSS_QS_PER_VF to ICE_MAX_QS_PER_VF_VCV1 Przemek Kitszel
2026-11-19 6:44 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 06/15] ice: bump to 256qs for VF Przemek Kitszel
2026-11-19 6:44 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 07/15] iavf: extend iavf_configure_queues() to support more queues Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 08/15] iavf: temporary rename of IAVF_MAX_REQ_QUEUES to IAVF_MAX_REQ_QUEUES_VCV1 Przemek Kitszel
2026-11-17 9:44 ` [net-next v6 09/15] iavf: increase max number of queues to 256 Przemek Kitszel
2026-11-19 6:44 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 10/15] iavf: use new opcodes to request more than 16 queues Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 11/15] ice: introduce handling of virtchnl LARGE VF opcodes Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 12/15] devlink: give user option to allocate resources Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 13/15] ice: represent RSS LUTs as devlink resources Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 14/15] devlink: add devlink_alloc_drv() Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
2026-11-17 9:44 ` [net-next v6 15/15] ice: support up to 256 VF queues Przemek Kitszel
2026-11-19 6:45 ` netdev-bot+sashiko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox