* [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API
@ 2024-07-24 20:24 Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 01/11] netlink: spec: add shaper YAML spec Paolo Abeni
` (11 more replies)
0 siblings, 12 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
We have a plurality of shaping-related drivers API, but none flexible
enough to meet existing demand from vendors[1].
This series introduces new device APIs to configure in a flexible way
TX shaping H/W offload. The new functionality is exposed via a newly
defined generic netlink interface and includes introspection
capabilities. Some self-tests are included, on top of a dummy
netdevsim implementation, and a basic implementation for the iavf
driver.
Some usage examples:
* Configure shaping on a given queue:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do set --json '{"ifindex":'$IFINDEX',
"shaper": {"handle":
{"scope": "queue", "id":'$QUEUEID' },
"bw-max": 2000000 }}'
* Container B/W sharing
The orchestration infrastructure wants to group the
container-related queues under a RR scheduling and limit the aggregate
bandwidth:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do group --json '{"ifindex":'$IFINDEX',
"inputs": [
{"handle": {"scope": "queue", "id":'$QID1' },
"weight": '$W1'},
{"handle": {"scope": "queue", "id":'$QID2' },
"weight": '$W2'}],
{"handle": {"scope": "queue", "id":'$QID3' },
"weight": '$W3'}],
"output": { "handle": {"scope":"netdev"},
"output": { "handle": {"scope":"netdev"},
"bw-max": 10000000}}'
{'handle': {'id': 0, 'scope': 'netdev'}}
* Delegation
A container wants to set a B/W limit on 2 of its own queues:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do group --json '{"ifindex":'$IFINDEX',
"inputs": [
{"handle": {"scope": "queue", "id":'$QID1' },
"weight": '$W1'},
{"handle": {"scope": "queue", "id":'$QID2' },
"weight": '$W2'}],
"output": { "handle": {"scope":"detached"},
"bw-max": 5000000}}'
{'handle': {'id': 0, 'scope': 'detached'}}
* Cleanup:
Deleting a single queue shaper:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do delete --json \
'{"ifindex":'$IFINDEX',
"handle": {"scope": "queue", "id":'$QID1' }}'
Deleting the last shaper under a group deletes the group, too:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do delete --json \
'{"ifindex":'$IFINDEX',
"handle": {"scope": "queue", "id":'$QID2' }}'
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do get --json '{"ifindex":'$IF',
"handle": { "scope": "detached", "id": 0}}'
Netlink error: Invalid argument
nl_len = 80 (64) nl_flags = 0x300 nl_type = 2
error: -22
extack: {'msg': "Can't find shaper for handle 10000000"}
Changes from RFC v1:
- set() and delete() ops operate on a single shaper
- added group() op to allow grouping and nesting
- split the NL implementation into multiple patches to help reviewing
RFC v1: https://lore.kernel.org/netdev/cover.1719518113.git.pabeni@redhat.com/
[1] https://lore.kernel.org/netdev/20240405102313.GA310894@kernel.org/
Paolo Abeni (7):
netlink: spec: add shaper YAML spec
net-shapers: implement NL get operation
net-shapers: implement NL set and delete operations
net-shapers: implement NL group operation
netlink: spec: add shaper introspection support
net: shaper: implement introspection support
testing: net-drv: add basic shaper test
Sudheer Mogilappagari (2):
iavf: Add net_shaper_ops support
iavf: add support to exchange qos capabilities
Wenjun Wu (2):
virtchnl: support queue rate limit and quanta size configuration
ice: Support VF queue rate limit and quanta size configuration
Documentation/netlink/specs/shaper.yaml | 337 ++++++
Documentation/networking/kapi.rst | 3 +
MAINTAINERS | 1 +
drivers/net/Kconfig | 1 +
drivers/net/ethernet/intel/Kconfig | 1 +
drivers/net/ethernet/intel/iavf/iavf.h | 13 +
drivers/net/ethernet/intel/iavf/iavf_main.c | 215 +++-
drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 +
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 157 ++-
drivers/net/ethernet/intel/ice/ice.h | 2 +
drivers/net/ethernet/intel/ice/ice_base.c | 2 +
drivers/net/ethernet/intel/ice/ice_common.c | 21 +
.../net/ethernet/intel/ice/ice_hw_autogen.h | 8 +
drivers/net/ethernet/intel/ice/ice_txrx.h | 1 +
drivers/net/ethernet/intel/ice/ice_type.h | 1 +
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 +
drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++
drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 +
.../intel/ice/ice_virtchnl_allowlist.c | 6 +
drivers/net/netdevsim/netdev.c | 37 +
include/linux/avf/virtchnl.h | 119 +++
include/linux/netdevice.h | 17 +
include/net/net_shaper.h | 169 +++
include/uapi/linux/net_shaper.h | 91 ++
net/Kconfig | 3 +
net/Makefile | 1 +
net/core/dev.c | 2 +
net/core/dev.h | 6 +
net/shaper/Makefile | 9 +
net/shaper/shaper.c | 962 ++++++++++++++++++
net/shaper/shaper_nl_gen.c | 142 +++
net/shaper/shaper_nl_gen.h | 30 +
tools/testing/selftests/drivers/net/Makefile | 1 +
tools/testing/selftests/drivers/net/shaper.py | 267 +++++
.../testing/selftests/net/lib/py/__init__.py | 1 +
tools/testing/selftests/net/lib/py/ynl.py | 5 +
36 files changed, 2983 insertions(+), 2 deletions(-)
create mode 100644 Documentation/netlink/specs/shaper.yaml
create mode 100644 include/net/net_shaper.h
create mode 100644 include/uapi/linux/net_shaper.h
create mode 100644 net/shaper/Makefile
create mode 100644 net/shaper/shaper.c
create mode 100644 net/shaper/shaper_nl_gen.c
create mode 100644 net/shaper/shaper_nl_gen.h
create mode 100755 tools/testing/selftests/drivers/net/shaper.py
--
2.45.2
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH RFC v2 01/11] netlink: spec: add shaper YAML spec
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-25 11:42 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 02/11] net-shapers: implement NL get operation Paolo Abeni
` (10 subsequent siblings)
11 siblings, 1 reply; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Define the user-space visible interface to query, configure and delete
network shapers via yaml definition.
Add dummy implementations for the relevant NL callbacks.
set() and delete() operations touch a single shaper creating/updating or
deleting it.
The group() operation creates a shaper's group, nesting multiple input
shapers under the specified output shaper.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
RFC v1 -> RFC v2:
- u64 -> uint
- net_shapers -> net-shapers
- documented all the attributes
- dropped [ admin-perm ] for get() op
- group op
- set/delete touch a single shaper
---
Documentation/netlink/specs/shaper.yaml | 262 ++++++++++++++++++++++++
MAINTAINERS | 1 +
include/uapi/linux/net_shaper.h | 74 +++++++
net/Kconfig | 3 +
net/Makefile | 1 +
net/shaper/Makefile | 9 +
net/shaper/shaper.c | 34 +++
net/shaper/shaper_nl_gen.c | 117 +++++++++++
net/shaper/shaper_nl_gen.h | 27 +++
9 files changed, 528 insertions(+)
create mode 100644 Documentation/netlink/specs/shaper.yaml
create mode 100644 include/uapi/linux/net_shaper.h
create mode 100644 net/shaper/Makefile
create mode 100644 net/shaper/shaper.c
create mode 100644 net/shaper/shaper_nl_gen.c
create mode 100644 net/shaper/shaper_nl_gen.h
diff --git a/Documentation/netlink/specs/shaper.yaml b/Documentation/netlink/specs/shaper.yaml
new file mode 100644
index 000000000000..7327f5596fdb
--- /dev/null
+++ b/Documentation/netlink/specs/shaper.yaml
@@ -0,0 +1,262 @@
+# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
+
+name: net-shaper
+
+doc: Network device HW Rate Limiting offload
+
+definitions:
+ -
+ type: enum
+ name: scope
+ doc: the different scopes where a shaper can be attached
+ render-max: true
+ entries:
+ - name: unspec
+ doc: The scope is not specified
+ -
+ name: port
+ doc: The root for the whole H/W
+ -
+ name: netdev
+ doc: The main shaper for the given network device.
+ -
+ name: queue
+ doc: The shaper is attached to the given device queue.
+ -
+ name: detached
+ doc: |
+ The shaper is not attached to any user-visible network
+ device component and allows nesting and grouping of
+ queues or others detached shapers.
+ -
+ type: enum
+ name: metric
+ doc: different metric each shaper can support
+ entries:
+ -
+ name: bps
+ doc: Shaper operates on a bits per second basis
+ -
+ name: pps
+ doc: Shaper operates on a packets per second basis
+
+attribute-sets:
+ -
+ name: net-shaper
+ attributes:
+ -
+ name: ifindex
+ type: u32
+ doc: Interface index owing the specified shaper[s]
+ -
+ name: handle
+ type: nest
+ nested-attributes: handle
+ doc: Unique identifier for the given shaper
+ -
+ name: metric
+ type: u32
+ enum: metric
+ doc: Metric used by the given shaper for bw-min, bw-max and burst
+ -
+ name: bw-min
+ type: uint
+ doc: Minimum guaranteed B/W for the given shaper
+ -
+ name: bw-max
+ type: uint
+ doc: Shaping B/W for the given shaper or 0 when unlimited
+ -
+ name: burst
+ type: uint
+ doc: Maximum burst-size for bw-min and bw-max
+ -
+ name: priority
+ type: u32
+ doc: Scheduling priority for the given shaper
+ -
+ name: weight
+ type: u32
+ doc: |
+ Weighted round robin weight for given shaper.
+ The scheduling is applied to all the sibling
+ shapers with the same priority
+ -
+ name: scope
+ type: u32
+ enum: scope
+ doc: The given handle scope
+ -
+ name: id
+ type: u32
+ doc: |
+ The given handle id. The id semantic depends on the actual
+ scope, e.g. for 'queue' scope it's the queue id, for
+ 'detached' scope it's the shaper group identifier.
+ -
+ name: parent
+ type: nest
+ nested-attributes: handle
+ doc: |
+ Identifier for the parent of the affected shaper,
+ The parent handle value is implied by the shaper handle itself,
+ except for the output shaper in the 'group' operation.
+ -
+ name: inputs
+ type: nest
+ multi-attr: true
+ nested-attributes: ns-info
+ doc: |
+ Describes a set of inputs shapers for a @group operation
+ -
+ name: output
+ type: nest
+ nested-attributes: ns-output-info
+ doc: |
+ Describes the output shaper for a @group operation
+ Differently from @inputs and @shaper allow specifying
+ the shaper parent handle, too.
+
+ -
+ name: shaper
+ type: nest
+ nested-attributes: ns-info
+ doc: |
+ Describes a single shaper for a @set operation
+ -
+ name: handle
+ subset-of: net-shaper
+ attributes:
+ -
+ name: scope
+ -
+ name: id
+ -
+ name: ns-info
+ subset-of: net-shaper
+ attributes:
+ -
+ name: handle
+ -
+ name: metric
+ -
+ name: bw-min
+ -
+ name: bw-max
+ -
+ name: burst
+ -
+ name: priority
+ -
+ name: weight
+ -
+ name: ns-output-info
+ subset-of: net-shaper
+ attributes:
+ -
+ name: parent
+ -
+ name: handle
+ -
+ name: metric
+ -
+ name: bw-min
+ -
+ name: bw-max
+ -
+ name: burst
+ -
+ name: priority
+ -
+ name: weight
+
+operations:
+ list:
+ -
+ name: get
+ doc: |
+ Get / Dump information about a/all the shaper for a given device
+ attribute-set: net-shaper
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - handle
+ reply:
+ attributes: &ns-attrs
+ - parent
+ - handle
+ - metric
+ - bw-min
+ - bw-max
+ - burst
+ - priority
+ - weight
+
+ dump:
+ request:
+ attributes:
+ - ifindex
+ reply:
+ attributes: *ns-attrs
+ -
+ name: set
+ doc: |
+ Create or configures the specified shaper.
+ On failures the extack is set accordingly.
+ Can't create @detached scope shaper, use
+ the @group operation instead.
+ attribute-set: net-shaper
+ flags: [ admin-perm ]
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - shaper
+
+ -
+ name: delete
+ doc: |
+ Clear (remove) the specified shaper. If after the removal
+ the parent shaper has no more children and the parent
+ shaper scope is @detached, even the parent is deleted,
+ recursively.
+ On failures the extack is set accordingly.
+ attribute-set: net-shaper
+ flags: [ admin-perm ]
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - handle
+
+ -
+ name: group
+ doc: |
+ Group the specified input shapers under the specified
+ output shaper, eventually creating the latter, if needed.
+ Input shapers scope must be either @queue or @detached.
+ Output shaper scope must be either @detached or @netdev.
+ When using an output @detached scope shaper, if the
+ @handle @id is not specified, a new shaper of such scope
+ is created and, otherwise the specified output shaper
+ must be already existing.
+ The operation is atomic, on failures the extack is set
+ accordingly and no change is applied to the device
+ shaping configuration, otherwise the output shaper
+ handle is provided as reply.
+ attribute-set: net-shaper
+ flags: [ admin-perm ]
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - inputs
+ - output
+ reply:
+ attributes:
+ - handle
diff --git a/MAINTAINERS b/MAINTAINERS
index 4a096207f8b4..1aaa27bb67eb 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15816,6 +15816,7 @@ F: include/linux/netdevice.h
F: include/uapi/linux/cn_proc.h
F: include/uapi/linux/if_*
F: include/uapi/linux/netdevice.h
+F: include/uapu/linux/net_shaper.h
X: drivers/net/wireless/
NETWORKING DRIVERS (WIRELESS)
diff --git a/include/uapi/linux/net_shaper.h b/include/uapi/linux/net_shaper.h
new file mode 100644
index 000000000000..ab3d4eb0e1ab
--- /dev/null
+++ b/include/uapi/linux/net_shaper.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/shaper.yaml */
+/* YNL-GEN uapi header */
+
+#ifndef _UAPI_LINUX_NET_SHAPER_H
+#define _UAPI_LINUX_NET_SHAPER_H
+
+#define NET_SHAPER_FAMILY_NAME "net-shaper"
+#define NET_SHAPER_FAMILY_VERSION 1
+
+/**
+ * enum net_shaper_scope - the different scopes where a shaper can be attached
+ * @NET_SHAPER_SCOPE_UNSPEC: The scope is not specified
+ * @NET_SHAPER_SCOPE_PORT: The root for the whole H/W
+ * @NET_SHAPER_SCOPE_NETDEV: The main shaper for the given network device.
+ * @NET_SHAPER_SCOPE_QUEUE: The shaper is attached to the given device queue.
+ * @NET_SHAPER_SCOPE_DETACHED: The shaper is not attached to any user-visible
+ * network device component and allows nesting and grouping of queues or
+ * others detached shapers.
+ */
+enum net_shaper_scope {
+ NET_SHAPER_SCOPE_UNSPEC,
+ NET_SHAPER_SCOPE_PORT,
+ NET_SHAPER_SCOPE_NETDEV,
+ NET_SHAPER_SCOPE_QUEUE,
+ NET_SHAPER_SCOPE_DETACHED,
+
+ /* private: */
+ __NET_SHAPER_SCOPE_MAX,
+ NET_SHAPER_SCOPE_MAX = (__NET_SHAPER_SCOPE_MAX - 1)
+};
+
+/**
+ * enum net_shaper_metric - different metric each shaper can support
+ * @NET_SHAPER_METRIC_BPS: Shaper operates on a bits per second basis
+ * @NET_SHAPER_METRIC_PPS: Shaper operates on a packets per second basis
+ */
+enum net_shaper_metric {
+ NET_SHAPER_METRIC_BPS,
+ NET_SHAPER_METRIC_PPS,
+};
+
+enum {
+ NET_SHAPER_A_IFINDEX = 1,
+ NET_SHAPER_A_HANDLE,
+ NET_SHAPER_A_METRIC,
+ NET_SHAPER_A_BW_MIN,
+ NET_SHAPER_A_BW_MAX,
+ NET_SHAPER_A_BURST,
+ NET_SHAPER_A_PRIORITY,
+ NET_SHAPER_A_WEIGHT,
+ NET_SHAPER_A_SCOPE,
+ NET_SHAPER_A_ID,
+ NET_SHAPER_A_PARENT,
+ NET_SHAPER_A_INPUTS,
+ NET_SHAPER_A_OUTPUT,
+ NET_SHAPER_A_SHAPER,
+
+ __NET_SHAPER_A_MAX,
+ NET_SHAPER_A_MAX = (__NET_SHAPER_A_MAX - 1)
+};
+
+enum {
+ NET_SHAPER_CMD_GET = 1,
+ NET_SHAPER_CMD_SET,
+ NET_SHAPER_CMD_DELETE,
+ NET_SHAPER_CMD_GROUP,
+
+ __NET_SHAPER_CMD_MAX,
+ NET_SHAPER_CMD_MAX = (__NET_SHAPER_CMD_MAX - 1)
+};
+
+#endif /* _UAPI_LINUX_NET_SHAPER_H */
diff --git a/net/Kconfig b/net/Kconfig
index d27d0deac0bf..31fccfed04f7 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -66,6 +66,9 @@ config SKB_DECRYPTED
config SKB_EXTENSIONS
bool
+config NET_SHAPER
+ bool
+
menu "Networking options"
source "net/packet/Kconfig"
diff --git a/net/Makefile b/net/Makefile
index 65bb8c72a35e..60ed5190eda8 100644
--- a/net/Makefile
+++ b/net/Makefile
@@ -79,3 +79,4 @@ obj-$(CONFIG_XDP_SOCKETS) += xdp/
obj-$(CONFIG_MPTCP) += mptcp/
obj-$(CONFIG_MCTP) += mctp/
obj-$(CONFIG_NET_HANDSHAKE) += handshake/
+obj-$(CONFIG_NET_SHAPER) += shaper/
diff --git a/net/shaper/Makefile b/net/shaper/Makefile
new file mode 100644
index 000000000000..13375884d60e
--- /dev/null
+++ b/net/shaper/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for the Generic HANDSHAKE service
+#
+# Copyright (c) 2024, Red Hat, Inc.
+#
+
+obj-y += shaper.o shaper_nl_gen.o
+
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
new file mode 100644
index 000000000000..49de88c68e2f
--- /dev/null
+++ b/net/shaper/shaper.c
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include <linux/kernel.h>
+#include <linux/skbuff.h>
+
+#include "shaper_nl_gen.h"
+
+int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_get_dumpit(struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return -EOPNOTSUPP;
+}
+
+static int __init shaper_init(void)
+{
+ return genl_register_family(&net_shaper_nl_family);
+}
+
+subsys_initcall(shaper_init);
diff --git a/net/shaper/shaper_nl_gen.c b/net/shaper/shaper_nl_gen.c
new file mode 100644
index 000000000000..b444d1ff55a1
--- /dev/null
+++ b/net/shaper/shaper_nl_gen.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/shaper.yaml */
+/* YNL-GEN kernel source */
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include "shaper_nl_gen.h"
+
+#include <uapi/linux/net_shaper.h>
+
+/* Common nested types */
+const struct nla_policy net_shaper_handle_nl_policy[NET_SHAPER_A_ID + 1] = {
+ [NET_SHAPER_A_SCOPE] = NLA_POLICY_MAX(NLA_U32, 4),
+ [NET_SHAPER_A_ID] = { .type = NLA_U32, },
+};
+
+const struct nla_policy net_shaper_ns_info_nl_policy[NET_SHAPER_A_WEIGHT + 1] = {
+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+ [NET_SHAPER_A_METRIC] = NLA_POLICY_MAX(NLA_U32, 1),
+ [NET_SHAPER_A_BW_MIN] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_BW_MAX] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_BURST] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_PRIORITY] = { .type = NLA_U32, },
+ [NET_SHAPER_A_WEIGHT] = { .type = NLA_U32, },
+};
+
+const struct nla_policy net_shaper_ns_output_info_nl_policy[NET_SHAPER_A_PARENT + 1] = {
+ [NET_SHAPER_A_PARENT] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+ [NET_SHAPER_A_METRIC] = NLA_POLICY_MAX(NLA_U32, 1),
+ [NET_SHAPER_A_BW_MIN] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_BW_MAX] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_BURST] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_PRIORITY] = { .type = NLA_U32, },
+ [NET_SHAPER_A_WEIGHT] = { .type = NLA_U32, },
+};
+
+/* NET_SHAPER_CMD_GET - do */
+static const struct nla_policy net_shaper_get_do_nl_policy[NET_SHAPER_A_HANDLE + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+};
+
+/* NET_SHAPER_CMD_GET - dump */
+static const struct nla_policy net_shaper_get_dump_nl_policy[NET_SHAPER_A_IFINDEX + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+};
+
+/* NET_SHAPER_CMD_SET - do */
+static const struct nla_policy net_shaper_set_nl_policy[NET_SHAPER_A_SHAPER + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_SHAPER] = NLA_POLICY_NESTED(net_shaper_ns_info_nl_policy),
+};
+
+/* NET_SHAPER_CMD_DELETE - do */
+static const struct nla_policy net_shaper_delete_nl_policy[NET_SHAPER_A_HANDLE + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+};
+
+/* NET_SHAPER_CMD_GROUP - do */
+static const struct nla_policy net_shaper_group_nl_policy[NET_SHAPER_A_OUTPUT + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_INPUTS] = NLA_POLICY_NESTED(net_shaper_ns_info_nl_policy),
+ [NET_SHAPER_A_OUTPUT] = NLA_POLICY_NESTED(net_shaper_ns_output_info_nl_policy),
+};
+
+/* Ops table for net_shaper */
+static const struct genl_split_ops net_shaper_nl_ops[] = {
+ {
+ .cmd = NET_SHAPER_CMD_GET,
+ .doit = net_shaper_nl_get_doit,
+ .policy = net_shaper_get_do_nl_policy,
+ .maxattr = NET_SHAPER_A_HANDLE,
+ .flags = GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_GET,
+ .dumpit = net_shaper_nl_get_dumpit,
+ .policy = net_shaper_get_dump_nl_policy,
+ .maxattr = NET_SHAPER_A_IFINDEX,
+ .flags = GENL_CMD_CAP_DUMP,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_SET,
+ .doit = net_shaper_nl_set_doit,
+ .policy = net_shaper_set_nl_policy,
+ .maxattr = NET_SHAPER_A_SHAPER,
+ .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_DELETE,
+ .doit = net_shaper_nl_delete_doit,
+ .policy = net_shaper_delete_nl_policy,
+ .maxattr = NET_SHAPER_A_HANDLE,
+ .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_GROUP,
+ .doit = net_shaper_nl_group_doit,
+ .policy = net_shaper_group_nl_policy,
+ .maxattr = NET_SHAPER_A_OUTPUT,
+ .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
+ },
+};
+
+struct genl_family net_shaper_nl_family __ro_after_init = {
+ .name = NET_SHAPER_FAMILY_NAME,
+ .version = NET_SHAPER_FAMILY_VERSION,
+ .netnsok = true,
+ .parallel_ops = true,
+ .module = THIS_MODULE,
+ .split_ops = net_shaper_nl_ops,
+ .n_split_ops = ARRAY_SIZE(net_shaper_nl_ops),
+};
diff --git a/net/shaper/shaper_nl_gen.h b/net/shaper/shaper_nl_gen.h
new file mode 100644
index 000000000000..00cee4efd21e
--- /dev/null
+++ b/net/shaper/shaper_nl_gen.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/shaper.yaml */
+/* YNL-GEN kernel header */
+
+#ifndef _LINUX_NET_SHAPER_GEN_H
+#define _LINUX_NET_SHAPER_GEN_H
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include <uapi/linux/net_shaper.h>
+
+/* Common nested types */
+extern const struct nla_policy net_shaper_handle_nl_policy[NET_SHAPER_A_ID + 1];
+extern const struct nla_policy net_shaper_ns_info_nl_policy[NET_SHAPER_A_WEIGHT + 1];
+extern const struct nla_policy net_shaper_ns_output_info_nl_policy[NET_SHAPER_A_PARENT + 1];
+
+int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
+int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info);
+
+extern struct genl_family net_shaper_nl_family;
+
+#endif /* _LINUX_NET_SHAPER_GEN_H */
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 02/11] net-shapers: implement NL get operation
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 01/11] netlink: spec: add shaper YAML spec Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 03/11] net-shapers: implement NL set and delete operations Paolo Abeni
` (9 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Introduce the basic infrastructure to implement the net-shaper
core functionality. Each network devices carries a net-shaper cache,
the NL get() operation fetches the data from such cache.
The cache is initially empty, will be fill by the set()/group()
operation implemented later and is destroyed at device cleanup time.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
Documentation/networking/kapi.rst | 3 +
include/linux/netdevice.h | 17 ++
include/net/net_shaper.h | 158 +++++++++++++++++++
net/core/dev.c | 2 +
net/core/dev.h | 6 +
net/shaper/shaper.c | 251 +++++++++++++++++++++++++++++-
6 files changed, 435 insertions(+), 2 deletions(-)
create mode 100644 include/net/net_shaper.h
diff --git a/Documentation/networking/kapi.rst b/Documentation/networking/kapi.rst
index ea55f462cefa..98682b9a13ee 100644
--- a/Documentation/networking/kapi.rst
+++ b/Documentation/networking/kapi.rst
@@ -104,6 +104,9 @@ Driver Support
.. kernel-doc:: include/linux/netdevice.h
:internal:
+.. kernel-doc:: include/net/net_shaper.h
+ :internal:
+
PHY Support
-----------
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 607009150b5f..d3d952be711c 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -81,6 +81,8 @@ struct xdp_frame;
struct xdp_metadata_ops;
struct xdp_md;
struct ethtool_netdev_state;
+struct net_shaper_ops;
+struct net_shaper_data;
typedef u32 xdp_features_t;
@@ -1598,6 +1600,14 @@ struct net_device_ops {
int (*ndo_hwtstamp_set)(struct net_device *dev,
struct kernel_hwtstamp_config *kernel_config,
struct netlink_ext_ack *extack);
+
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+ /**
+ * @net_shaper_ops: Device shaping offload operations
+ * see include/net/net_shapers.h
+ */
+ const struct net_shaper_ops *net_shaper_ops;
+#endif
};
/**
@@ -2408,6 +2418,13 @@ struct net_device {
/** @irq_moder: dim parameters used if IS_ENABLED(CONFIG_DIMLIB). */
struct dim_irq_moder *irq_moder;
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+ /**
+ * @net_shaper_data: data tracking the current shaper status
+ * see include/net/net_shapers.h
+ */
+ struct net_shaper_data *net_shaper_data;
+#endif
u8 priv[] ____cacheline_aligned
__counted_by(priv_len);
} ____cacheline_aligned;
diff --git a/include/net/net_shaper.h b/include/net/net_shaper.h
new file mode 100644
index 000000000000..8cd65d727e52
--- /dev/null
+++ b/include/net/net_shaper.h
@@ -0,0 +1,158 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _NET_SHAPER_H_
+#define _NET_SHAPER_H_
+
+#include <linux/types.h>
+#include <linux/bits.h>
+#include <linux/bitfield.h>
+#include <linux/netdevice.h>
+#include <linux/netlink.h>
+
+#include <uapi/linux/net_shaper.h>
+
+/**
+ * struct net_shaper_info - represents a shaping node on the NIC H/W
+ * zeroed field are considered not set.
+ * @handle: Unique identifier for the shaper, see @net_shaper_make_handle
+ * @parent: Unique identifier for the shaper parent, usually implied. Only
+ * NET_SHAPER_SCOPE_QUEUE, NET_SHAPER_SCOPE_NETDEV and NET_SHAPER_SCOPE_DETACHED
+ * can have the parent handle explicitly set, placing such shaper under
+ * the specified parent.
+ * @metric: Specify if the bw limits refers to PPS or BPS
+ * @bw_min: Minimum guaranteed rate for this shaper
+ * @bw_max: Maximum peak bw allowed for this shaper
+ * @burst: Maximum burst for the peek rate of this shaper
+ * @priority: Scheduling priority for this shaper
+ * @weight: Scheduling weight for this shaper
+ * @children: Number of nested shapers, accounted only for DETACHED scope
+ */
+struct net_shaper_info {
+ u32 handle;
+ u32 parent;
+ enum net_shaper_metric metric;
+ u64 bw_min;
+ u64 bw_max;
+ u64 burst;
+ u32 priority;
+ u32 weight;
+ u32 children;
+};
+
+/**
+ * define NET_SHAPER_SCOPE_VF - Shaper scope
+ *
+ * This shaper scope is not exposed to user-space; the shaper is attached to
+ * the given virtual function.
+ */
+#define NET_SHAPER_SCOPE_VF __NET_SHAPER_SCOPE_MAX
+
+/**
+ * struct net_shaper_ops - Operations on device H/W shapers
+ *
+ * The initial shaping configuration ad device initialization is empty/
+ * a no-op/does not constraint the b/w in any way.
+ * The network core keeps track of the applied user-configuration in
+ * per device storage.
+ *
+ * Each shaper is uniquely identified within the device with an 'handle',
+ * dependent on the shaper scope and other data, see @shaper_make_handle()
+ */
+struct net_shaper_ops {
+ /**
+ * @group: create the specified shapers group
+ *
+ * Nest the specified @inputs shapers under the given @output shaper
+ * on the network device @dev. The @input shaper array size is specified
+ * by @nr_input.
+ * Create either the @inputs and the @output shaper as needed,
+ * otherwise move them as needed. Can't create @inputs shapers with
+ * NET_SHAPER_SCOPE_DETACHED scope, a separate @group call with such
+ * shaper as @output is needed.
+ *
+ * Returns 0 on group successfully created, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*group)(struct net_device *dev, int nr_input,
+ const struct net_shaper_info *inputs,
+ const struct net_shaper_info *output,
+ struct netlink_ext_ack *extack);
+
+ /**
+ * @set: Updates the specified shaper
+ *
+ * Updates or creates the specified @shaper on the given device
+ * @dev. Can't create NET_SHAPER_SCOPE_DETACHED shapers, use @group
+ * instead.
+ *
+ * Returns 0 on group successfully created, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*set)(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack);
+
+ /**
+ * @delete: Removes the specified shaper from the NIC
+ *
+ * Removes the shaper configuration as identified by the given @handle
+ * on the specified device @dev, restoring the default behavior.
+ *
+ * Returns 0 on group successfully created, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*delete)(struct net_device *dev, u32 handle,
+ struct netlink_ext_ack *extack);
+};
+
+#define NET_SHAPER_SCOPE_SHIFT 26
+#define NET_SHAPER_ID_MASK GENMASK(NET_SHAPER_SCOPE_SHIFT - 1, 0)
+#define NET_SHAPER_SCOPE_MASK GENMASK(31, NET_SHAPER_SCOPE_SHIFT)
+
+#define NET_SHAPER_ID_UNSPEC NET_SHAPER_ID_MASK
+
+/**
+ * net_shaper_make_handle - creates an unique shaper identifier
+ * @scope: the shaper scope
+ * @id: the shaper id number
+ *
+ * Return: an unique identifier for the shaper
+ *
+ * Combines the specified arguments to create an unique identifier for
+ * the shaper. The @id argument semantic depends on the
+ * specified scope.
+ * For @NET_SHAPER_SCOPE_QUEUE_GROUP, @id is the queue group id
+ * For @NET_SHAPER_SCOPE_QUEUE, @id is the queue number.
+ * For @NET_SHAPER_SCOPE_VF, @id is virtual function number.
+ */
+static inline u32 net_shaper_make_handle(enum net_shaper_scope scope,
+ int id)
+{
+ return FIELD_PREP(NET_SHAPER_SCOPE_MASK, scope) |
+ FIELD_PREP(NET_SHAPER_ID_MASK, id);
+}
+
+/**
+ * net_shaper_handle_scope - extract the scope from the given handle
+ * @handle: the shaper handle
+ *
+ * Return: the corresponding scope
+ */
+static inline enum net_shaper_scope net_shaper_handle_scope(u32 handle)
+{
+ return FIELD_GET(NET_SHAPER_SCOPE_MASK, handle);
+}
+
+/**
+ * net_shaper_handle_id - extract the id number from the given handle
+ * @handle: the shaper handle
+ *
+ * Return: the corresponding id number
+ */
+static inline int net_shaper_handle_id(u32 handle)
+{
+ return FIELD_GET(NET_SHAPER_ID_MASK, handle);
+}
+
+#endif
+
diff --git a/net/core/dev.c b/net/core/dev.c
index 6ea1d20676fb..3dc1dd428eda 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -11169,6 +11169,8 @@ void free_netdev(struct net_device *dev)
/* Flush device addresses */
dev_addr_flush(dev);
+ dev_shaper_flush(dev);
+
list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
netif_napi_del(p);
diff --git a/net/core/dev.h b/net/core/dev.h
index 5654325c5b71..e376fc1c867b 100644
--- a/net/core/dev.h
+++ b/net/core/dev.h
@@ -35,6 +35,12 @@ void dev_addr_flush(struct net_device *dev);
int dev_addr_init(struct net_device *dev);
void dev_addr_check(struct net_device *dev);
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+void dev_shaper_flush(struct net_device *dev);
+#else
+static inline void dev_shaper_flush(struct net_device *dev) {}
+#endif
+
/* sysctls not referred to from outside net/core/ */
extern int netdev_unregister_timeout_secs;
extern int weight_p;
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 49de88c68e2f..93dd491ac7c2 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -1,19 +1,242 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <linux/kernel.h>
+#include <linux/idr.h>
#include <linux/skbuff.h>
+#include <linux/xarray.h>
+#include <net/net_shaper.h>
#include "shaper_nl_gen.h"
+#include "../core/dev.h"
+
+struct net_shaper_data {
+ struct xarray shapers;
+ struct idr detached_ids;
+};
+
+struct net_shaper_nl_ctx {
+ u32 start_handle;
+};
+
+static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
+ const struct genl_info *info)
+{
+ struct nlattr *handle_attr;
+
+ if (!handle)
+ return 0;
+
+ handle_attr = nla_nest_start_noflag(msg, type);
+ if (!handle_attr)
+ return -EMSGSIZE;
+
+ if (nla_put_u32(msg, NET_SHAPER_A_SCOPE,
+ net_shaper_handle_scope(handle)) ||
+ nla_put_u32(msg, NET_SHAPER_A_ID,
+ net_shaper_handle_id(handle)))
+ goto handle_nest_cancel;
+
+ nla_nest_end(msg, handle_attr);
+ return 0;
+
+handle_nest_cancel:
+ nla_nest_cancel(msg, handle_attr);
+ return -EMSGSIZE;
+}
+
+static int
+net_shaper_fill_one(struct sk_buff *msg, struct net_shaper_info *shaper,
+ const struct genl_info *info)
+{
+ void *hdr;
+
+ hdr = genlmsg_iput(msg, info);
+ if (!hdr)
+ return -EMSGSIZE;
+
+ if (fill_handle(msg, shaper->parent, NET_SHAPER_A_PARENT, info) ||
+ fill_handle(msg, shaper->handle, NET_SHAPER_A_HANDLE, info) ||
+ nla_put_u32(msg, NET_SHAPER_A_METRIC, shaper->metric) ||
+ nla_put_uint(msg, NET_SHAPER_A_BW_MIN, shaper->bw_min) ||
+ nla_put_uint(msg, NET_SHAPER_A_BW_MAX, shaper->bw_max) ||
+ nla_put_uint(msg, NET_SHAPER_A_BURST, shaper->burst) ||
+ nla_put_u32(msg, NET_SHAPER_A_PRIORITY, shaper->priority) ||
+ nla_put_u32(msg, NET_SHAPER_A_WEIGHT, shaper->weight))
+ goto nla_put_failure;
+
+ genlmsg_end(msg, hdr);
+
+ return 0;
+
+nla_put_failure:
+ genlmsg_cancel(msg, hdr);
+ return -EMSGSIZE;
+}
+
+/* On success sets pdev to the relevant device and acquires a reference
+ * to it
+ */
+static int fetch_dev(const struct genl_info *info, struct net_device **pdev)
+{
+ struct net *ns = genl_info_net(info);
+ struct net_device *dev;
+ int ifindex;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_IFINDEX))
+ return -EINVAL;
+
+ ifindex = nla_get_u32(info->attrs[NET_SHAPER_A_IFINDEX]);
+ dev = dev_get_by_index(ns, ifindex);
+ if (!dev) {
+ GENL_SET_ERR_MSG_FMT(info, "device %d not found", ifindex);
+ return -EINVAL;
+ }
+
+ if (!dev->netdev_ops->net_shaper_ops) {
+ GENL_SET_ERR_MSG_FMT(info, "device %s does not support H/W shaper",
+ dev->name);
+ dev_put(dev);
+ return -EOPNOTSUPP;
+ }
+
+ *pdev = dev;
+ return 0;
+}
+
+static struct xarray *__sc_container(struct net_device *dev)
+{
+ return dev->net_shaper_data ? &dev->net_shaper_data->shapers : NULL;
+}
+
+/* lookup the given shaper inside the cache */
+static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle)
+{
+ struct xarray *xa = __sc_container(dev);
+
+ return xa ? xa_load(xa, handle) : NULL;
+}
+
+static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
+ u32 *handle)
+{
+ struct nlattr *tb[NET_SHAPER_A_ID + 1];
+ struct nlattr *scope_attr, *id_attr;
+ enum net_shaper_scope scope;
+ u32 id = 0;
+ int ret;
+
+ ret = nla_parse_nested(tb, NET_SHAPER_A_ID, attr,
+ net_shaper_handle_nl_policy, info->extack);
+ if (ret < 0)
+ return ret;
+
+ scope_attr = tb[NET_SHAPER_A_SCOPE];
+ if (!scope_attr) {
+ GENL_SET_ERR_MSG(info, "Missing 'scope' attribute for handle");
+ return -EINVAL;
+ }
+
+ scope = nla_get_u32(scope_attr);
+
+ /* the default id for detached scope shapers is an invalid one
+ * to help the 'group' operation discriminate between new
+ * detached shaper creation (ID_UNSPEC) and reuse of existing
+ * shaper (any other value)
+ */
+ id_attr = tb[NET_SHAPER_A_ID];
+ if (id_attr)
+ id = nla_get_u32(id_attr);
+ else if (scope == NET_SHAPER_SCOPE_DETACHED)
+ id = NET_SHAPER_ID_UNSPEC;
+
+ *handle = net_shaper_make_handle(scope, id);
+ return 0;
+}
+
int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
{
- return -EOPNOTSUPP;
+ struct net_shaper_info *shaper;
+ struct net_device *dev;
+ struct sk_buff *msg;
+ u32 handle;
+ int ret;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
+ goto put;
+
+ ret = parse_handle(info->attrs[NET_SHAPER_A_HANDLE], info, &handle);
+ if (ret < 0)
+ goto put;
+
+ shaper = sc_lookup(dev, handle);
+ if (!shaper) {
+ GENL_SET_ERR_MSG_FMT(info, "Can't find shaper for handle %x", handle);
+ ret = -EINVAL;
+ goto put;
+ }
+
+ msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg) {
+ ret = -ENOMEM;
+ goto put;
+ }
+
+ ret = net_shaper_fill_one(msg, shaper, info);
+ if (ret)
+ goto free_msg;
+
+ ret = genlmsg_reply(msg, info);
+ if (ret)
+ goto free_msg;
+
+put:
+ dev_put(dev);
+ return ret;
+
+free_msg:
+ nlmsg_free(msg);
+ goto put;
}
int net_shaper_nl_get_dumpit(struct sk_buff *skb,
struct netlink_callback *cb)
{
- return -EOPNOTSUPP;
+ struct net_shaper_nl_ctx *ctx = (struct net_shaper_nl_ctx *)cb->ctx;
+ const struct genl_info *info = genl_info_dump(cb);
+ struct net_shaper_info *shaper;
+ struct net_device *dev;
+ unsigned long handle;
+ int ret;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ BUILD_BUG_ON(sizeof(struct net_shaper_nl_ctx) > sizeof(cb->ctx));
+
+ /* don't error out dumps performed before any set operation */
+ if (!dev->net_shaper_data) {
+ ret = 0;
+ goto put;
+ }
+
+ xa_for_each_range(&dev->net_shaper_data->shapers, handle, shaper,
+ ctx->start_handle, U32_MAX) {
+ ret = net_shaper_fill_one(skb, shaper, info);
+ if (ret)
+ goto put;
+
+ ctx->start_handle = handle;
+ }
+
+put:
+ dev_put(dev);
+ return ret;
}
int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
@@ -26,6 +249,30 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
return -EOPNOTSUPP;
}
+int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return -EOPNOTSUPP;
+}
+
+void dev_shaper_flush(struct net_device *dev)
+{
+ struct xarray *xa = __sc_container(dev);
+ struct net_shaper_info *cur;
+ unsigned long index;
+
+ if (!xa)
+ return;
+
+ xa_lock(xa);
+ xa_for_each(xa, index, cur) {
+ __xa_erase(xa, index);
+ kfree(cur);
+ }
+ idr_destroy(&dev->net_shaper_data->detached_ids);
+ xa_unlock(xa);
+ kfree(xa);
+}
+
static int __init shaper_init(void)
{
return genl_register_family(&net_shaper_nl_family);
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 03/11] net-shapers: implement NL set and delete operations
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 01/11] netlink: spec: add shaper YAML spec Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 02/11] net-shapers: implement NL get operation Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-08-02 16:01 ` Jiri Pirko
2024-07-24 20:24 ` [PATCH RFC v2 04/11] net-shapers: implement NL group operation Paolo Abeni
` (8 subsequent siblings)
11 siblings, 1 reply; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Both NL operations directly map on the homonymous device shaper
callbacks and update accordingly the shapers cache.
Implement the cache modification helpers to additionally deal with
DETACHED scope shaper. That will be needed by the group() operation
implemented in the next patch.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
net/shaper/shaper.c | 323 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 321 insertions(+), 2 deletions(-)
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 93dd491ac7c2..7802c9ba6d9c 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -19,6 +19,35 @@ struct net_shaper_nl_ctx {
u32 start_handle;
};
+static u32 default_parent(u32 handle)
+{
+ enum net_shaper_scope parent, scope = net_shaper_handle_scope(handle);
+
+ switch (scope) {
+ case NET_SHAPER_SCOPE_PORT:
+ case NET_SHAPER_SCOPE_UNSPEC:
+ parent = NET_SHAPER_SCOPE_UNSPEC;
+ break;
+
+ case NET_SHAPER_SCOPE_QUEUE:
+ case NET_SHAPER_SCOPE_DETACHED:
+ parent = NET_SHAPER_SCOPE_NETDEV;
+ break;
+
+ case NET_SHAPER_SCOPE_NETDEV:
+ case NET_SHAPER_SCOPE_VF:
+ parent = NET_SHAPER_SCOPE_PORT;
+ break;
+ }
+
+ return net_shaper_make_handle(parent, 0);
+}
+
+static bool is_detached(u32 handle)
+{
+ return net_shaper_handle_scope(handle) == NET_SHAPER_SCOPE_DETACHED;
+}
+
static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
const struct genl_info *info)
{
@@ -117,6 +146,115 @@ static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle)
return xa ? xa_load(xa, handle) : NULL;
}
+/* allocate on demand the per device shaper's cache */
+static struct xarray *__sc_init(struct net_device *dev,
+ struct netlink_ext_ack *extack)
+{
+ if (!dev->net_shaper_data) {
+ dev->net_shaper_data = kmalloc(sizeof(*dev->net_shaper_data),
+ GFP_KERNEL);
+ if (!dev->net_shaper_data) {
+ NL_SET_ERR_MSG(extack, "Can't allocate memory for shaper data");
+ return NULL;
+ }
+
+ xa_init(&dev->net_shaper_data->shapers);
+ idr_init(&dev->net_shaper_data->detached_ids);
+ }
+ return &dev->net_shaper_data->shapers;
+}
+
+/* prepare the cache to actually insert the given shaper, doing
+ * in advance the needed allocations
+ */
+static int sc_prepare_insert(struct net_device *dev, u32 *handle,
+ struct netlink_ext_ack *extack)
+{
+ enum net_shaper_scope scope = net_shaper_handle_scope(*handle);
+ struct xarray *xa = __sc_init(dev, extack);
+ struct net_shaper_info *prev, *cur;
+ bool id_allocated = false;
+ int ret, id;
+
+ if (!xa)
+ return -ENOMEM;
+
+ cur = xa_load(xa, *handle);
+ if (cur)
+ return 0;
+
+ /* allocated a new id, if needed */
+ if (scope == NET_SHAPER_SCOPE_DETACHED &&
+ net_shaper_handle_id(*handle) == NET_SHAPER_ID_UNSPEC) {
+ xa_lock(xa);
+ id = idr_alloc(&dev->net_shaper_data->detached_ids, NULL,
+ 0, NET_SHAPER_ID_UNSPEC, GFP_ATOMIC);
+ xa_unlock(xa);
+
+ if (id < 0) {
+ NL_SET_ERR_MSG(extack, "Can't allocate new id for detached shaper");
+ return id;
+ }
+
+ *handle = net_shaper_make_handle(scope, id);
+ id_allocated = true;
+ }
+
+ cur = kmalloc(sizeof(*cur), GFP_KERNEL | __GFP_ZERO);
+ if (!cur) {
+ NL_SET_ERR_MSG(extack, "Can't allocate memory for cached shaper");
+ ret = -ENOMEM;
+ goto free_id;
+ }
+
+ /* mark 'tentative' shaper inside the cache */
+ xa_lock(xa);
+ prev = __xa_store(xa, *handle, cur, GFP_KERNEL);
+ __xa_set_mark(xa, *handle, XA_MARK_0);
+ xa_unlock(xa);
+ if (xa_err(prev)) {
+ NL_SET_ERR_MSG(extack, "Can't insert shaper into cache");
+ kfree(cur);
+ ret = xa_err(prev);
+ goto free_id;
+ }
+ return 0;
+
+free_id:
+ if (id_allocated) {
+ xa_lock(xa);
+ idr_remove(&dev->net_shaper_data->detached_ids,
+ net_shaper_handle_id(*handle));
+ xa_unlock(xa);
+ }
+ return ret;
+}
+
+/* commit the tentative insert with the actual values.
+ * Must be called only after a successful sc_prepare_insert()
+ */
+static void sc_commit(struct net_device *dev, int nr_shapers,
+ const struct net_shaper_info *shapers)
+{
+ struct xarray *xa = __sc_container(dev);
+ struct net_shaper_info *cur;
+ int i;
+
+ xa_lock(xa);
+ for (i = 0; i < nr_shapers; ++i) {
+ cur = xa_load(xa, shapers[i].handle);
+ if (WARN_ON_ONCE(!cur))
+ continue;
+
+ /* successful update: drop the tentative mark
+ * and update the cache
+ */
+ __xa_clear_mark(xa, shapers[i].handle, XA_MARK_0);
+ *cur = shapers[i];
+ }
+ xa_unlock(xa);
+}
+
static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
u32 *handle)
{
@@ -154,6 +292,68 @@ static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
return 0;
}
+static int __parse_shaper(struct net_device *dev, struct nlattr **tb,
+ const struct genl_info *info,
+ struct net_shaper_info *shaper)
+{
+ struct net_shaper_info *old;
+ int ret;
+
+ /* the shaper handle is the only mandatory attribute */
+ if (NL_REQ_ATTR_CHECK(info->extack, NULL, tb, NET_SHAPER_A_HANDLE))
+ return -EINVAL;
+
+ ret = parse_handle(tb[NET_SHAPER_A_HANDLE], info, &shaper->handle);
+ if (ret)
+ return ret;
+
+ /* fetch existing data, if any, so that user provide info will
+ * incrementally update the existing shaper configuration
+ */
+ old = sc_lookup(dev, shaper->handle);
+ if (old)
+ *shaper = *old;
+ else
+ shaper->parent = default_parent(shaper->handle);
+
+ if (tb[NET_SHAPER_A_METRIC])
+ shaper->metric = nla_get_u32(tb[NET_SHAPER_A_METRIC]);
+
+ if (tb[NET_SHAPER_A_BW_MIN])
+ shaper->bw_min = nla_get_uint(tb[NET_SHAPER_A_BW_MIN]);
+
+ if (tb[NET_SHAPER_A_BW_MAX])
+ shaper->bw_max = nla_get_uint(tb[NET_SHAPER_A_BW_MAX]);
+
+ if (tb[NET_SHAPER_A_BURST])
+ shaper->burst = nla_get_uint(tb[NET_SHAPER_A_BURST]);
+
+ if (tb[NET_SHAPER_A_PRIORITY])
+ shaper->priority = nla_get_u32(tb[NET_SHAPER_A_PRIORITY]);
+
+ if (tb[NET_SHAPER_A_WEIGHT])
+ shaper->weight = nla_get_u32(tb[NET_SHAPER_A_WEIGHT]);
+ return 0;
+}
+
+/* fetch the cached shaper info and update them with the user-provided
+ * attributes
+ */
+static int parse_shaper(struct net_device *dev, const struct nlattr *attr,
+ const struct genl_info *info,
+ struct net_shaper_info *shaper)
+{
+ struct nlattr *tb[NET_SHAPER_A_WEIGHT + 1];
+ int ret;
+
+ ret = nla_parse_nested(tb, NET_SHAPER_A_WEIGHT, attr,
+ net_shaper_ns_info_nl_policy, info->extack);
+ if (ret < 0)
+ return ret;
+
+ return __parse_shaper(dev, tb, info, shaper);
+}
+
int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net_shaper_info *shaper;
@@ -239,14 +439,133 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb,
return ret;
}
+/* Update the H/W and on success update the local cache, too */
+static int net_shaper_set(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack)
+{
+ enum net_shaper_scope scope;
+ u32 handle = shaper->handle;
+ int ret;
+
+ scope = net_shaper_handle_scope(handle);
+ if (scope == NET_SHAPER_SCOPE_PORT ||
+ scope == NET_SHAPER_SCOPE_UNSPEC) {
+ NL_SET_ERR_MSG_FMT(extack, "Can't set shaper %x with scope %d",
+ handle, scope);
+ return -EINVAL;
+ }
+ if (scope == NET_SHAPER_SCOPE_DETACHED && !sc_lookup(dev, handle)) {
+ NL_SET_ERR_MSG_FMT(extack, "Shaper %x with detached scope does not exist",
+ handle);
+ return -EINVAL;
+ }
+
+ ret = sc_prepare_insert(dev, &handle, extack);
+ if (ret)
+ return ret;
+
+ ret = dev->netdev_ops->net_shaper_ops->set(dev, shaper, extack);
+ sc_commit(dev, 1, shaper);
+ return ret;
+}
+
int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
{
- return -EOPNOTSUPP;
+ struct net_shaper_info shaper;
+ struct net_device *dev;
+ struct nlattr *attr;
+ int ret;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_SHAPER))
+ return -EINVAL;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ attr = info->attrs[NET_SHAPER_A_SHAPER];
+ ret = parse_shaper(dev, attr, info, &shaper);
+ if (ret)
+ goto put;
+
+ ret = net_shaper_set(dev, &shaper, info->extack);
+
+put:
+ dev_put(dev);
+ return ret;
+}
+
+static int net_shaper_delete(struct net_device *dev, u32 handle,
+ struct netlink_ext_ack *extack)
+{
+ struct net_shaper_info *parent, *shaper = sc_lookup(dev, handle);
+ struct xarray *xa = __sc_container(dev);
+ enum net_shaper_scope pscope;
+ u32 parent_handle;
+ int ret;
+
+ if (!xa || !shaper) {
+ NL_SET_ERR_MSG_FMT(extack, "Shaper %x not found", handle);
+ return -EINVAL;
+ }
+
+ if (is_detached(handle) && shaper->children > 0) {
+ NL_SET_ERR_MSG_FMT(extack, "Can't delete detached shaper %d with %d child nodes",
+ handle, shaper->children);
+ return -EINVAL;
+ }
+
+ while (shaper) {
+ parent_handle = shaper->parent;
+ pscope = net_shaper_handle_scope(parent_handle);
+
+ ret = dev->netdev_ops->net_shaper_ops->delete(dev, handle, extack);
+ if (ret < 0)
+ return ret;
+
+ xa_lock(xa);
+ __xa_erase(xa, handle);
+ if (is_detached(handle))
+ idr_remove(&dev->net_shaper_data->detached_ids,
+ net_shaper_handle_id(handle));
+ xa_unlock(xa);
+ kfree(shaper);
+ shaper = NULL;
+
+ if (pscope == NET_SHAPER_SCOPE_DETACHED) {
+ parent = sc_lookup(dev, parent_handle);
+ if (parent && !--parent->children) {
+ shaper = parent;
+ handle = parent_handle;
+ }
+ }
+ }
+ return 0;
}
int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
{
- return -EOPNOTSUPP;
+ struct net_device *dev;
+ u32 handle;
+ int ret;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
+ return -EINVAL;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ ret = parse_handle(info->attrs[NET_SHAPER_A_HANDLE], info, &handle);
+ if (ret)
+ goto put;
+
+ ret = net_shaper_delete(dev, handle, info->extack);
+
+put:
+ dev_put(dev);
+ return ret;
}
int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 04/11] net-shapers: implement NL group operation
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (2 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 03/11] net-shapers: implement NL set and delete operations Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 05/11] netlink: spec: add shaper introspection support Paolo Abeni
` (7 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Alllow grouping multiple inputs shaper under the given output's
one.
Try hard to pre-allocated the needed resources, to avoid non
trivial H/W configuration rollbacks in case of any failure.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
net/shaper/shaper.c | 265 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 264 insertions(+), 1 deletion(-)
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 7802c9ba6d9c..d99d000d7e7a 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -48,6 +48,18 @@ static bool is_detached(u32 handle)
return net_shaper_handle_scope(handle) == NET_SHAPER_SCOPE_DETACHED;
}
+/* count the number of [multi] attributes of the given type */
+static int attr_list_len(struct genl_info *info, int type)
+{
+ struct nlattr *attr;
+ int rem, cnt = 0;
+
+ nla_for_each_attr_type(attr, type, genlmsg_data(info->genlhdr),
+ genlmsg_len(info->genlhdr), rem)
+ cnt++;
+ return cnt;
+}
+
static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
const struct genl_info *info)
{
@@ -255,6 +267,27 @@ static void sc_commit(struct net_device *dev, int nr_shapers,
xa_unlock(xa);
}
+/* rollback all the tentative inserts from the shaper cache */
+static void sc_rollback(struct net_device *dev)
+{
+ struct xarray *xa = __sc_container(dev);
+ struct net_shaper_info *cur;
+ unsigned long index;
+
+ if (!xa)
+ return;
+
+ xa_lock(xa);
+ xa_for_each_marked(xa, index, cur, XA_MARK_0) {
+ if (is_detached(index))
+ idr_remove(&dev->net_shaper_data->detached_ids,
+ net_shaper_handle_id(index));
+ __xa_erase(xa, index);
+ kfree(cur);
+ }
+ xa_unlock(xa);
+}
+
static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
u32 *handle)
{
@@ -354,6 +387,36 @@ static int parse_shaper(struct net_device *dev, const struct nlattr *attr,
return __parse_shaper(dev, tb, info, shaper);
}
+/* alike parse_shaper(), but additionally allow the user specifying
+ * the shaper's parent handle
+ */
+static int parse_output_shaper(struct net_device *dev,
+ const struct nlattr *attr,
+ const struct genl_info *info,
+ struct net_shaper_info *shaper)
+{
+ struct nlattr *tb[NET_SHAPER_A_PARENT + 1];
+ int ret;
+
+ ret = nla_parse_nested(tb, NET_SHAPER_A_PARENT, attr,
+ net_shaper_ns_output_info_nl_policy,
+ info->extack);
+ if (ret < 0)
+ return ret;
+
+ ret = __parse_shaper(dev, tb, info, shaper);
+ if (ret)
+ return ret;
+
+ if (tb[NET_SHAPER_A_PARENT]) {
+ ret = parse_handle(tb[NET_SHAPER_A_PARENT], info,
+ &shaper->parent);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net_shaper_info *shaper;
@@ -568,9 +631,209 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
return ret;
}
+/* Update the H/W and on success update the local cache, too */
+static int net_shaper_group(struct net_device *dev, int nr_inputs,
+ struct net_shaper_info *inputs,
+ struct net_shaper_info *output,
+ struct netlink_ext_ack *extack)
+{
+ enum net_shaper_scope scope, output_scope, output_pscope;
+ struct net_shaper_info *parent = NULL;
+ int i, ret;
+
+ output_scope = net_shaper_handle_scope(output->handle);
+ if (output_scope != NET_SHAPER_SCOPE_DETACHED &&
+ output_scope != NET_SHAPER_SCOPE_NETDEV) {
+ NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for output shaper %x",
+ output_scope, output->handle);
+ return -EINVAL;
+ }
+
+ output_pscope = net_shaper_handle_scope(output->parent);
+ if (output_scope == NET_SHAPER_SCOPE_DETACHED) {
+ if (net_shaper_handle_id(output->handle) !=
+ NET_SHAPER_ID_UNSPEC && !sc_lookup(dev, output->handle)) {
+ NL_SET_ERR_MSG_FMT(extack, "Output shaper %x does not exists",
+ output->handle);
+ return -EINVAL;
+ }
+ if (output_pscope != NET_SHAPER_SCOPE_DETACHED &&
+ output_pscope != NET_SHAPER_SCOPE_NETDEV) {
+ NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for output parent shaper %x",
+ output_pscope, output->parent);
+ return -EINVAL;
+ }
+ }
+
+ if (output_pscope == NET_SHAPER_SCOPE_DETACHED) {
+ parent = sc_lookup(dev, output->parent);
+ if (!parent) {
+ NL_SET_ERR_MSG_FMT(extack, "Output parent shaper %x does not exists",
+ output->parent);
+ return -EINVAL;
+ }
+ }
+
+ /* for newly created detached scope shaper, the following will update the
+ * handle, due to id allocation
+ */
+ ret = sc_prepare_insert(dev, &output->handle, extack);
+ if (ret)
+ goto rollback;
+
+ for (i = 0; i < nr_inputs; ++i) {
+ scope = net_shaper_handle_scope(inputs[i].handle);
+ if (scope != NET_SHAPER_SCOPE_QUEUE &&
+ scope != NET_SHAPER_SCOPE_DETACHED) {
+ ret = -EINVAL;
+ NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for input shaper %d handle %x",
+ scope, i, inputs[i].handle);
+ goto rollback;
+ }
+ if (scope == NET_SHAPER_SCOPE_DETACHED &&
+ !sc_lookup(dev, inputs[i].handle)) {
+ ret = -EINVAL;
+ NL_SET_ERR_MSG_FMT(extack,
+ "Can't create detached shaper as input %d handle %x",
+ i, inputs[i].handle);
+ goto rollback;
+ }
+
+ ret = sc_prepare_insert(dev, &inputs[i].handle, extack);
+ if (ret)
+ goto rollback;
+
+ /* the inputs shapers will be nested to the output */
+ if (inputs[i].parent != output->handle) {
+ inputs[i].parent = output->handle;
+ output->children++;
+ }
+ }
+
+ ret = dev->netdev_ops->net_shaper_ops->group(dev, nr_inputs,
+ inputs, output,
+ extack);
+ if (ret < 0)
+ goto rollback;
+
+ if (parent)
+ parent->children++;
+ sc_commit(dev, 1, output);
+ sc_commit(dev, nr_inputs, inputs);
+ return ret;
+
+rollback:
+ sc_rollback(dev);
+ return ret;
+}
+
+static int nla_handle_total_size(void)
+{
+ return nla_total_size(nla_total_size(sizeof(u32)) +
+ nla_total_size(sizeof(u32)));
+}
+
+static int group_send_reply(struct genl_info *info, u32 handle, int id)
+{
+ struct nlattr *handle_attr;
+ struct sk_buff *msg;
+ int ret = -EMSGSIZE;
+ void *hdr;
+
+/* prepare the msg reply in advance, to avoid device operation rollback */
+ msg = genlmsg_new(nla_handle_total_size(), GFP_KERNEL);
+ if (!msg)
+ return ret;
+
+ hdr = genlmsg_iput(msg, info);
+ if (!hdr)
+ goto free_msg;
+
+ handle_attr = nla_nest_start(msg, NET_SHAPER_A_HANDLE);
+ if (!handle_attr)
+ goto free_msg;
+
+ if (nla_put_u32(msg, NET_SHAPER_A_SCOPE,
+ net_shaper_handle_scope(handle)))
+ goto free_msg;
+
+ if (nla_put_u32(msg, NET_SHAPER_A_ID, id))
+ goto free_msg;
+
+ nla_nest_end(msg, handle_attr);
+ genlmsg_end(msg, hdr);
+
+ ret = genlmsg_reply(msg, info);
+ if (ret)
+ goto free_msg;
+
+ return ret;
+
+free_msg:
+ nlmsg_free(msg);
+ return ret;
+}
+
int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
{
- return -EOPNOTSUPP;
+ struct net_shaper_info *inputs, output;
+ int i, ret, rem, nr_inputs;
+ struct net_device *dev;
+ struct nlattr *attr;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_INPUTS) ||
+ GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_OUTPUT))
+ return -EINVAL;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ nr_inputs = attr_list_len(info, NET_SHAPER_A_INPUTS);
+ inputs = kcalloc(nr_inputs, sizeof(struct net_shaper_info), GFP_KERNEL);
+ if (!inputs) {
+ GENL_SET_ERR_MSG_FMT(info, "Can't allocate memory for %d input shapers",
+ nr_inputs);
+ ret = -ENOMEM;
+ goto put;
+ }
+
+ ret = parse_output_shaper(dev, info->attrs[NET_SHAPER_A_OUTPUT], info,
+ &output);
+ if (ret)
+ goto free_shapers;
+
+ i = 0;
+ nla_for_each_attr_type(attr, NET_SHAPER_A_INPUTS,
+ genlmsg_data(info->genlhdr),
+ genlmsg_len(info->genlhdr), rem) {
+ if (WARN_ON_ONCE(i >= nr_inputs))
+ goto free_shapers;
+
+ ret = parse_shaper(dev, attr, info, &inputs[i++]);
+ if (ret)
+ goto free_shapers;
+ }
+
+ ret = net_shaper_group(dev, nr_inputs, inputs, &output, info->extack);
+ if (ret < 0)
+ goto free_shapers;
+
+ ret = group_send_reply(info, output.handle, ret);
+ if (ret) {
+ /* Error on reply is not fatal to avoid rollback a successful
+ * configuration
+ */
+ GENL_SET_ERR_MSG_FMT(info, "Can't send reply %d", ret);
+ ret = 0;
+ }
+
+free_shapers:
+ kfree(inputs);
+
+put:
+ dev_put(dev);
+ return ret;
}
void dev_shaper_flush(struct net_device *dev)
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 05/11] netlink: spec: add shaper introspection support
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (3 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 04/11] net-shapers: implement NL group operation Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 06/11] net: shaper: implement " Paolo Abeni
` (6 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Allow the user-space to fine-grain query the shaping features
supported by the NIC on each domain.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
Documentation/netlink/specs/shaper.yaml | 75 +++++++++++++++++++++++++
include/uapi/linux/net_shaper.h | 17 ++++++
net/shaper/shaper.c | 11 ++++
net/shaper/shaper_nl_gen.c | 25 +++++++++
net/shaper/shaper_nl_gen.h | 3 +
5 files changed, 131 insertions(+)
diff --git a/Documentation/netlink/specs/shaper.yaml b/Documentation/netlink/specs/shaper.yaml
index 7327f5596fdb..55bf4e49fb2f 100644
--- a/Documentation/netlink/specs/shaper.yaml
+++ b/Documentation/netlink/specs/shaper.yaml
@@ -169,6 +169,52 @@ attribute-sets:
name: priority
-
name: weight
+ -
+ name: capabilities
+ attributes:
+ -
+ name: ifindex
+ type: u32
+ -
+ name: scope
+ type: u32
+ enum: scope
+ doc: The scope to which the queried capabilities apply
+ -
+ name: support-metric-bps
+ type: flag
+ doc: the device accepts 'bps' metric for bw-min, bw-max and burst
+ -
+ name: support-metric-pps
+ type: flag
+ doc: the device accepts 'pps' metric for bw-min, bw-max and burst
+ -
+ name: support-nesting
+ type: flag
+ doc: |
+ the device supports nesting shaper belonging to this scope
+ below 'detached' scoped shapers. Only 'queue' and 'detached'
+ scope and flag 'support-nesting'.
+ -
+ name: support-bw-min
+ type: flag
+ doc: the device supports a minimum guaranteed bw
+ -
+ name: support-bw-max
+ type: flag
+ doc: the device supports maximum bw shaping
+ -
+ name: support-burst
+ type: flag
+ doc: the device supports a maximum burst size
+ -
+ name: support-priority
+ type: flag
+ doc: the device supports priority scheduling
+ -
+ name: support-weight
+ type: flag
+ doc: the device supports weighted round robin scheduling
operations:
list:
@@ -260,3 +306,32 @@ operations:
reply:
attributes:
- handle
+
+ -
+ name: cap-get
+ doc: |
+ Get / Dump the shaper capabilities supported by the given device
+ attribute-set: capabilities
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - scope
+ reply:
+ attributes: &cap-attrs
+ - support-metric-bps
+ - support-metric-pps
+ - support-nesting
+ - support-bw-min
+ - support-bw-max
+ - support-burst
+ - support-priority
+ - support-weight
+
+ dump:
+ request:
+ attributes:
+ - ifindex
+ reply:
+ attributes: *cap-attrs
diff --git a/include/uapi/linux/net_shaper.h b/include/uapi/linux/net_shaper.h
index ab3d4eb0e1ab..12d16d30472e 100644
--- a/include/uapi/linux/net_shaper.h
+++ b/include/uapi/linux/net_shaper.h
@@ -61,11 +61,28 @@ enum {
NET_SHAPER_A_MAX = (__NET_SHAPER_A_MAX - 1)
};
+enum {
+ NET_SHAPER_A_CAPABILITIES_IFINDEX = 1,
+ NET_SHAPER_A_CAPABILITIES_SCOPE,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_BPS,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_PPS,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_NESTING,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MIN,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MAX,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_BURST,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_PRIORITY,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_WEIGHT,
+
+ __NET_SHAPER_A_CAPABILITIES_MAX,
+ NET_SHAPER_A_CAPABILITIES_MAX = (__NET_SHAPER_A_CAPABILITIES_MAX - 1)
+};
+
enum {
NET_SHAPER_CMD_GET = 1,
NET_SHAPER_CMD_SET,
NET_SHAPER_CMD_DELETE,
NET_SHAPER_CMD_GROUP,
+ NET_SHAPER_CMD_CAP_GET,
__NET_SHAPER_CMD_MAX,
NET_SHAPER_CMD_MAX = (__NET_SHAPER_CMD_MAX - 1)
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index d99d000d7e7a..7b7e9a777929 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -836,6 +836,17 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
return ret;
}
+int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return 0;
+}
+
+int net_shaper_nl_cap_get_dumpit(struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ return 0;
+}
+
void dev_shaper_flush(struct net_device *dev)
{
struct xarray *xa = __sc_container(dev);
diff --git a/net/shaper/shaper_nl_gen.c b/net/shaper/shaper_nl_gen.c
index b444d1ff55a1..cc98d58c0862 100644
--- a/net/shaper/shaper_nl_gen.c
+++ b/net/shaper/shaper_nl_gen.c
@@ -67,6 +67,17 @@ static const struct nla_policy net_shaper_group_nl_policy[NET_SHAPER_A_OUTPUT +
[NET_SHAPER_A_OUTPUT] = NLA_POLICY_NESTED(net_shaper_ns_output_info_nl_policy),
};
+/* NET_SHAPER_CMD_CAP_GET - do */
+static const struct nla_policy net_shaper_cap_get_do_nl_policy[NET_SHAPER_A_CAPABILITIES_SCOPE + 1] = {
+ [NET_SHAPER_A_CAPABILITIES_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_CAPABILITIES_SCOPE] = NLA_POLICY_MAX(NLA_U32, 4),
+};
+
+/* NET_SHAPER_CMD_CAP_GET - dump */
+static const struct nla_policy net_shaper_cap_get_dump_nl_policy[NET_SHAPER_A_CAPABILITIES_IFINDEX + 1] = {
+ [NET_SHAPER_A_CAPABILITIES_IFINDEX] = { .type = NLA_U32, },
+};
+
/* Ops table for net_shaper */
static const struct genl_split_ops net_shaper_nl_ops[] = {
{
@@ -104,6 +115,20 @@ static const struct genl_split_ops net_shaper_nl_ops[] = {
.maxattr = NET_SHAPER_A_OUTPUT,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
+ {
+ .cmd = NET_SHAPER_CMD_CAP_GET,
+ .doit = net_shaper_nl_cap_get_doit,
+ .policy = net_shaper_cap_get_do_nl_policy,
+ .maxattr = NET_SHAPER_A_CAPABILITIES_SCOPE,
+ .flags = GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_CAP_GET,
+ .dumpit = net_shaper_nl_cap_get_dumpit,
+ .policy = net_shaper_cap_get_dump_nl_policy,
+ .maxattr = NET_SHAPER_A_CAPABILITIES_IFINDEX,
+ .flags = GENL_CMD_CAP_DUMP,
+ },
};
struct genl_family net_shaper_nl_family __ro_after_init = {
diff --git a/net/shaper/shaper_nl_gen.h b/net/shaper/shaper_nl_gen.h
index 00cee4efd21e..97fbd4502364 100644
--- a/net/shaper/shaper_nl_gen.h
+++ b/net/shaper/shaper_nl_gen.h
@@ -21,6 +21,9 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info);
int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info);
int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_cap_get_dumpit(struct sk_buff *skb,
+ struct netlink_callback *cb);
extern struct genl_family net_shaper_nl_family;
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 06/11] net: shaper: implement introspection support
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (4 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 05/11] netlink: spec: add shaper introspection support Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 07/11] testing: net-drv: add basic shaper test Paolo Abeni
` (5 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
The netlink op is a simple wrapper around the device callback.
Extend the existing fetch_dev() helper adding an attribute argument
for the requested device. Reuse such helper in the newly implemented
operation.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
include/net/net_shaper.h | 11 ++++
net/shaper/shaper.c | 108 +++++++++++++++++++++++++++++++++++----
2 files changed, 109 insertions(+), 10 deletions(-)
diff --git a/include/net/net_shaper.h b/include/net/net_shaper.h
index 8cd65d727e52..402f9d8476ee 100644
--- a/include/net/net_shaper.h
+++ b/include/net/net_shaper.h
@@ -103,6 +103,17 @@ struct net_shaper_ops {
*/
int (*delete)(struct net_device *dev, u32 handle,
struct netlink_ext_ack *extack);
+
+ /**
+ * @capabilities: get the shaper features supported by the NIC
+ *
+ * Fills the bitmask @cap with the supported cababilites for the
+ * specified @scope and device @dev.
+ *
+ * Returns 0 on success or a negative error value otherwise.
+ */
+ int (*capabilities)(struct net_device *dev,
+ enum net_shaper_scope scope, unsigned long *cap);
};
#define NET_SHAPER_SCOPE_SHIFT 26
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 7b7e9a777929..01f34681ad4e 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -118,16 +118,17 @@ net_shaper_fill_one(struct sk_buff *msg, struct net_shaper_info *shaper,
/* On success sets pdev to the relevant device and acquires a reference
* to it
*/
-static int fetch_dev(const struct genl_info *info, struct net_device **pdev)
+static int fetch_dev(const struct genl_info *info, int type,
+ struct net_device **pdev)
{
struct net *ns = genl_info_net(info);
struct net_device *dev;
int ifindex;
- if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_IFINDEX))
+ if (GENL_REQ_ATTR_CHECK(info, type))
return -EINVAL;
- ifindex = nla_get_u32(info->attrs[NET_SHAPER_A_IFINDEX]);
+ ifindex = nla_get_u32(info->attrs[type]);
dev = dev_get_by_index(ns, ifindex);
if (!dev) {
GENL_SET_ERR_MSG_FMT(info, "device %d not found", ifindex);
@@ -425,7 +426,7 @@ int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
u32 handle;
int ret;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -476,7 +477,7 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb,
unsigned long handle;
int ret;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -543,7 +544,7 @@ int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_SHAPER))
return -EINVAL;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -616,7 +617,7 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
return -EINVAL;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -785,7 +786,7 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_OUTPUT))
return -EINVAL;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -836,15 +837,102 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
return ret;
}
-int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info)
+static int
+net_shaper_cap_fill_one(struct sk_buff *msg, unsigned long flags,
+ const struct genl_info *info)
{
+ unsigned long cur;
+ void *hdr;
+
+ hdr = genlmsg_iput(msg, info);
+ if (!hdr)
+ return -EMSGSIZE;
+
+ for (cur = NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_BPS;
+ cur <= NET_SHAPER_A_CAPABILITIES_MAX; ++cur) {
+ if (flags & BIT(cur) && nla_put_flag(msg, cur))
+ goto nla_put_failure;
+ }
+
+ genlmsg_end(msg, hdr);
+
return 0;
+
+nla_put_failure:
+ genlmsg_cancel(msg, hdr);
+ return -EMSGSIZE;
+}
+
+int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ const struct net_shaper_ops *ops;
+ enum net_shaper_scope scope;
+ struct net_device *dev;
+ struct sk_buff *msg;
+ unsigned long flags;
+ int ret;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_CAPABILITIES_SCOPE))
+ return -EINVAL;
+
+ ret = fetch_dev(info, NET_SHAPER_A_CAPABILITIES_IFINDEX, &dev);
+ if (ret)
+ return ret;
+
+ scope = nla_get_u32(info->attrs[NET_SHAPER_A_CAPABILITIES_SCOPE]);
+ ops = dev->netdev_ops->net_shaper_ops;
+ ret = ops->capabilities(dev, scope, &flags);
+ if (ret)
+ goto put;
+
+ msg = genlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg)
+ goto put;
+
+ ret = net_shaper_cap_fill_one(msg, flags, info);
+ if (ret)
+ goto free_msg;
+
+ ret = genlmsg_reply(msg, info);
+ if (ret)
+ goto free_msg;
+
+put:
+ dev_put(dev);
+ return ret;
+
+free_msg:
+ nlmsg_free(msg);
+ goto put;
}
int net_shaper_nl_cap_get_dumpit(struct sk_buff *skb,
struct netlink_callback *cb)
{
- return 0;
+ const struct genl_info *info = genl_info_dump(cb);
+ const struct net_shaper_ops *ops;
+ enum net_shaper_scope scope;
+ struct net_device *dev;
+ unsigned long flags;
+ int ret;
+
+ ret = fetch_dev(info, NET_SHAPER_A_CAPABILITIES_IFINDEX, &dev);
+ if (ret)
+ return ret;
+
+ ops = dev->netdev_ops->net_shaper_ops;
+ for (scope = 0; scope <= NET_SHAPER_SCOPE_MAX; ++scope) {
+ if (ops->capabilities(dev, scope, &flags))
+ continue;
+
+ ret = net_shaper_cap_fill_one(skb, flags, info);
+ if (ret)
+ goto put;
+ }
+
+put:
+ dev_put(dev);
+ return ret;
}
void dev_shaper_flush(struct net_device *dev)
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 07/11] testing: net-drv: add basic shaper test
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (5 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 06/11] net: shaper: implement " Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 08/11] virtchnl: support queue rate limit and quanta size configuration Paolo Abeni
` (4 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Leverage a basic/dummy netdevsim implementation to do functional
coverage for NL interface.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
rfc v1 -> v2:
- added more test-cases WRT nesting and grouping
---
drivers/net/Kconfig | 1 +
drivers/net/netdevsim/netdev.c | 37 +++
tools/testing/selftests/drivers/net/Makefile | 1 +
tools/testing/selftests/drivers/net/shaper.py | 267 ++++++++++++++++++
.../testing/selftests/net/lib/py/__init__.py | 1 +
tools/testing/selftests/net/lib/py/ynl.py | 5 +
6 files changed, 312 insertions(+)
create mode 100755 tools/testing/selftests/drivers/net/shaper.py
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 9920b3a68ed1..1fd5acdc73c6 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -641,6 +641,7 @@ config NETDEVSIM
depends on PTP_1588_CLOCK_MOCK || PTP_1588_CLOCK_MOCK=n
select NET_DEVLINK
select PAGE_POOL
+ select NET_SHAPER
help
This driver is a developer testing tool and software model that can
be used to test various control path networking APIs, especially
diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c
index 017a6102be0a..ab9c3d5055e7 100644
--- a/drivers/net/netdevsim/netdev.c
+++ b/drivers/net/netdevsim/netdev.c
@@ -22,6 +22,7 @@
#include <net/netdev_queues.h>
#include <net/page_pool/helpers.h>
#include <net/netlink.h>
+#include <net/net_shaper.h>
#include <net/pkt_cls.h>
#include <net/rtnetlink.h>
#include <net/udp_tunnel.h>
@@ -475,6 +476,41 @@ static int nsim_stop(struct net_device *dev)
return 0;
}
+static int nsim_shaper_set(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack)
+{
+ return 0;
+}
+
+static int nsim_shaper_del(struct net_device *dev, u32 handles,
+ struct netlink_ext_ack *extack)
+{
+ return 0;
+}
+
+static int nsim_shaper_group(struct net_device *dev, int nr_inputs,
+ const struct net_shaper_info *inputs,
+ const struct net_shaper_info *output,
+ struct netlink_ext_ack *extack)
+{
+ return 0;
+}
+
+static int nsim_shaper_cap(struct net_device *dev, enum net_shaper_scope scope,
+ unsigned long *flags)
+{
+ *flags = ULONG_MAX;
+ return 0;
+}
+
+static const struct net_shaper_ops nsim_shaper_ops = {
+ .set = nsim_shaper_set,
+ .delete = nsim_shaper_del,
+ .group = nsim_shaper_group,
+ .capabilities = nsim_shaper_cap,
+};
+
static const struct net_device_ops nsim_netdev_ops = {
.ndo_start_xmit = nsim_start_xmit,
.ndo_set_rx_mode = nsim_set_rx_mode,
@@ -496,6 +532,7 @@ static const struct net_device_ops nsim_netdev_ops = {
.ndo_bpf = nsim_bpf,
.ndo_open = nsim_open,
.ndo_stop = nsim_stop,
+ .net_shaper_ops = &nsim_shaper_ops,
};
static const struct net_device_ops nsim_vf_netdev_ops = {
diff --git a/tools/testing/selftests/drivers/net/Makefile b/tools/testing/selftests/drivers/net/Makefile
index e54f382bcb02..432306f11664 100644
--- a/tools/testing/selftests/drivers/net/Makefile
+++ b/tools/testing/selftests/drivers/net/Makefile
@@ -6,6 +6,7 @@ TEST_PROGS := \
ping.py \
queues.py \
stats.py \
+ shaper.py
# end of TEST_PROGS
include ../../lib.mk
diff --git a/tools/testing/selftests/drivers/net/shaper.py b/tools/testing/selftests/drivers/net/shaper.py
new file mode 100755
index 000000000000..ed34573bb4f6
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/shaper.py
@@ -0,0 +1,267 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+from lib.py import ksft_run, ksft_exit, ksft_eq, ksft_true, KsftSkipEx
+from lib.py import ShaperFamily
+from lib.py import NetDrvEnv
+from lib.py import NlError
+from lib.py import cmd
+import glob
+import sys
+
+def get_shapers(cfg, nl_shaper) -> None:
+ try:
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ except NlError as e:
+ if e.error == 95:
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+
+ # default configuration, no shapers configured
+ ksft_eq(len(shapers), 0)
+
+def get_caps(cfg, nl_shaper) -> None:
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex}, dump=True)
+ except NlError as e:
+ if e.error == 95:
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+
+ # each device implementing shaper support must support some
+ # features in at least a scope
+ ksft_true(len(caps)> 0)
+
+def set_qshapers(cfg, nl_shaper) -> None:
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'queue'})
+ except NlError as e:
+ if e.error == 95:
+ cfg.queues = False;
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-bw-max' in caps or not 'support-metric-bps' in caps:
+ raise KsftSkipEx("device does not support queue scope shapers with bw_max and metric bps")
+
+ nl_shaper.set({'ifindex': cfg.ifindex,
+ 'shaper': { 'handle': { 'scope': 'queue', 'id': 1 }, 'metric': 'bps', 'bw-max': 10000 }})
+ nl_shaper.set({'ifindex': cfg.ifindex,
+ 'shaper': { 'handle': { 'scope': 'queue', 'id': 2 }, 'metric': 'bps', 'bw-max': 20000 }})
+
+ # querying a specific shaper not yet configured must fail
+ raised = False
+ try:
+ shaper_q0 = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 0}})
+ except (NlError):
+ raised = True
+ ksft_eq(raised, True)
+
+ shaper_q1 = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 1 }})
+ ksft_eq(shaper_q1, { 'parent': { 'scope': 'netdev', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 1 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 10000,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 0 })
+
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(shapers, [{'parent': { 'scope': 'netdev', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 1 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 10000,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 0 },
+ {'parent': { 'scope': 'netdev', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 2 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 20000,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 0 }])
+
+def del_qshapers(cfg, nl_shaper) -> None:
+ if not cfg.queues:
+ raise KsftSkipEx("queue shapers not supported by device, skipping delete")
+
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 2}})
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 1}})
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(len(shapers), 0)
+
+def set_nshapers(cfg, nl_shaper) -> None:
+ # check required features
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'netdev'})
+ except NlError as e:
+ if e.error == 95:
+ cfg.netdev = False;
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-bw-max' in caps or not 'support-metric-bps' in caps:
+ raise KsftSkipEx("device does not support nested netdev scope shapers with weight")
+
+ nl_shaper.set({'ifindex': cfg.ifindex,
+ 'shaper': { 'handle': { 'scope': 'netdev', 'id': 0 }, 'bw-max': 100000 }})
+
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(shapers, [{'parent': { 'scope': 'port', 'id': 0 },
+ 'handle': { 'scope': 'netdev', 'id': 0 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 100000,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 0 }])
+
+def del_nshapers(cfg, nl_shaper) -> None:
+ if not cfg.netdev:
+ raise KsftSkipEx("netdev shaper not supported by device, skipping delete")
+
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'netdev'}})
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(len(shapers), 0)
+
+def basic_groups(cfg, nl_shaper) -> None:
+ if not cfg.netdev:
+ raise KsftSkipEx("netdev shaper not supported by the device")
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'queue'})
+ except NlError as e:
+ if e.error == 95:
+ cfg.queues = False;
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-weight' in caps:
+ raise KsftSkipEx("device does not support queue scope shapers with weight")
+
+ output_handle = nl_shaper.group({'ifindex': cfg.ifindex,
+ 'inputs':[{ 'handle': { 'scope': 'queue', 'id': 1 },'weight': 1 },
+ { 'handle': { 'scope': 'queue', 'id': 2 }, 'weight': 2 }],
+ 'output': { 'handle': {'scope':'netdev'}, 'metric': 'bps', 'bw-max': 10000}})
+ output_id = output_handle['handle']['id']
+
+ shaper = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 1 }})
+ ksft_eq(shaper, {'parent': { 'scope': 'netdev', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 1 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 0,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 1 })
+
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 2}})
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 1}})
+
+ # deleting all the inputs shaper does not affect the output one
+ # when the latter has 'netdev' scope
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(len(shapers), 1)
+
+ nl_shaper.delete({'ifindex': cfg.ifindex, 'handle': { 'scope': 'netdev'}})
+
+def qgroups(cfg, nl_shaper) -> None:
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'detached'})
+ except NlError as e:
+ if e.error == 95:
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-bw-max' in caps or not 'support-metric-bps' in caps:
+ raise KsftSkipEx("device does not support detached scope shapers with bw_max and metric bps")
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'queue'})
+ except NlError as e:
+ if e.error == 95:
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-nesting' in caps or not 'support-weight' in caps or not 'support-metric-bps' in caps:
+ raise KsftSkipEx("device does not support nested queue scope shapers with weight")
+
+ output_handle = nl_shaper.group({'ifindex': cfg.ifindex,
+ 'inputs':[{ 'handle': { 'scope': 'queue', 'id': 1 }, 'metric': 'bps', 'weight': 3 },
+ { 'handle': { 'scope': 'queue', 'id': 2 }, 'metric': 'bps', 'weight': 2 }],
+ 'output': { 'handle': {'scope':'detached'}, 'metric': 'bps', 'bw-max': 10000}})
+ output_id = output_handle['handle']['id']
+
+ shaper = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 1 }})
+ ksft_eq(shaper, {'parent': { 'scope': 'detached', 'id': output_id },
+ 'handle': { 'scope': 'queue', 'id': 1 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 0,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 3 })
+
+ # grouping to a specified, not existing detached scope shaper must fail
+ raised = False
+ try:
+ nl_shaper.group({'ifindex': cfg.ifindex,
+ 'inputs':[ { 'handle': { 'scope': 'queue', 'id': 3 }, 'metric': 'bps', 'weight': 3 }],
+ 'output': { 'handle': {'scope':'detached', 'id': output_id + 1 }, 'metric': 'bps', 'bw-max': 10000}})
+ except (NlError):
+ raised = True
+ ksft_eq(raised, True)
+
+ output_handle = nl_shaper.group({'ifindex': cfg.ifindex,
+ 'inputs':[ { 'handle': { 'scope': 'queue', 'id': 3 }, 'metric': 'bps', 'weight': 4 }],
+ 'output': { 'handle': {'scope':'detached', 'id': output_id} }})
+
+ shaper = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 3 }})
+ ksft_eq(shaper, {'parent': { 'scope': 'detached', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 3 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 0,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 4 })
+
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 2}})
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 1}})
+
+ # deleting a non empty group mast fail
+ raised = False
+ try:
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'detached', 'id': output_id }})
+ except (NlError):
+ raised = True
+ ksft_eq(raised, True)
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 3}})
+
+ # the detached scope shaper deletion is implicit
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(len(shapers), 0)
+
+def main() -> None:
+ with NetDrvEnv(__file__, queue_count=4) as cfg:
+ cfg.queues = True
+ cfg.netdev = True
+ ksft_run([get_shapers,
+ get_caps,
+ set_qshapers,
+ del_qshapers,
+ set_nshapers,
+ del_nshapers,
+ basic_groups,
+ qgroups], args=(cfg, ShaperFamily()))
+ ksft_exit()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/testing/selftests/net/lib/py/__init__.py b/tools/testing/selftests/net/lib/py/__init__.py
index b6d498d125fe..ef1aa52f4761 100644
--- a/tools/testing/selftests/net/lib/py/__init__.py
+++ b/tools/testing/selftests/net/lib/py/__init__.py
@@ -6,3 +6,4 @@ from .netns import NetNS
from .nsim import *
from .utils import *
from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily
+from .ynl import ShaperFamily
diff --git a/tools/testing/selftests/net/lib/py/ynl.py b/tools/testing/selftests/net/lib/py/ynl.py
index 1ace58370c06..42e825905ade 100644
--- a/tools/testing/selftests/net/lib/py/ynl.py
+++ b/tools/testing/selftests/net/lib/py/ynl.py
@@ -47,3 +47,8 @@ class NetdevFamily(YnlFamily):
def __init__(self):
super().__init__((SPEC_PATH / Path('netdev.yaml')).as_posix(),
schema='')
+
+class ShaperFamily(YnlFamily):
+ def __init__(self):
+ super().__init__((SPEC_PATH / Path('shaper.yaml')).as_posix(),
+ schema='')
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 08/11] virtchnl: support queue rate limit and quanta size configuration
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (6 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 07/11] testing: net-drv: add basic shaper test Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 09/11] ice: Support VF " Paolo Abeni
` (3 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
From: Wenjun Wu <wenjun1.wu@intel.com>
This patch adds new virtchnl opcodes and structures for rate limit
and quanta size configuration, which include:
1. VIRTCHNL_OP_CONFIG_QUEUE_BW, to configure max bandwidth for each
VF per queue.
2. VIRTCHNL_OP_CONFIG_QUANTA, to configure quanta size per queue.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such
as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The
configuration is previously set by DCB and PF, and now is the potential
QoS capability of VF. VF can take it as reference to configure queue TC
mapping.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
---
include/linux/avf/virtchnl.h | 119 +++++++++++++++++++++++++++++++++++
1 file changed, 119 insertions(+)
diff --git a/include/linux/avf/virtchnl.h b/include/linux/avf/virtchnl.h
index 8e177b67e82f..8ff31845f685 100644
--- a/include/linux/avf/virtchnl.h
+++ b/include/linux/avf/virtchnl.h
@@ -89,6 +89,9 @@ enum virtchnl_rx_hsplit {
VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
};
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
/* END GENERIC DEFINES */
/* Opcodes for VF-PF communication. These are placed in the v_opcode field
@@ -151,6 +154,11 @@ enum virtchnl_ops {
VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ /* opcode 57 - 65 are reserved */
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ /* opcode 68 through 111 are reserved */
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
VIRTCHNL_OP_MAX,
};
@@ -260,6 +268,7 @@ VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
#define VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC BIT(26)
#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
VIRTCHNL_VF_OFFLOAD_VLAN | \
@@ -1405,6 +1414,85 @@ struct virtchnl_fdir_del {
VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_qos_cap_list);
+#define virtchnl_qos_cap_list_LEGACY_SIZEOF 44
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_queues_bw_cfg);
+#define virtchnl_queues_bw_cfg_LEGACY_SIZEOF 16
+
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
#define __vss_byone(p, member, count, old) \
(struct_size(p, member, count) + (old - 1 - struct_size(p, member, 0)))
@@ -1427,6 +1515,8 @@ VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
__vss(virtchnl_vlan_filter_list_v2, __vss_byelem, p, m, c), \
__vss(virtchnl_tc_info, __vss_byelem, p, m, c), \
__vss(virtchnl_rdma_qvlist_info, __vss_byelem, p, m, c), \
+ __vss(virtchnl_qos_cap_list, __vss_byelem, p, m, c), \
+ __vss(virtchnl_queues_bw_cfg, __vss_byelem, p, m, c), \
__vss(virtchnl_rss_key, __vss_byone, p, m, c), \
__vss(virtchnl_rss_lut, __vss_byone, p, m, c))
@@ -1626,6 +1716,35 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
valid_len = sizeof(struct virtchnl_vlan_setting);
break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = virtchnl_queues_bw_cfg_LEGACY_SIZEOF;
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+
+ valid_len = virtchnl_struct_size(q_bw, cfg,
+ q_bw->num_queues);
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
/* These are always errors coming from the VF. */
case VIRTCHNL_OP_EVENT:
case VIRTCHNL_OP_UNKNOWN:
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 09/11] ice: Support VF queue rate limit and quanta size configuration
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (7 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 08/11] virtchnl: support queue rate limit and quanta size configuration Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 10/11] iavf: Add net_shaper_ops support Paolo Abeni
` (2 subsequent siblings)
11 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
From: Wenjun Wu <wenjun1.wu@intel.com>
Add support to configure VF queue rate limit and quanta size.
For quanta size configuration, the quanta profiles are divided evenly
by PF numbers. For each port, the first quanta profile is reserved for
default. When VF is asked to set queue quanta size, PF will search for
an available profile, change the fields and assigned this profile to the
queue.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 2 +
drivers/net/ethernet/intel/ice/ice_base.c | 2 +
drivers/net/ethernet/intel/ice/ice_common.c | 21 ++
.../net/ethernet/intel/ice/ice_hw_autogen.h | 8 +
drivers/net/ethernet/intel/ice/ice_txrx.h | 1 +
drivers/net/ethernet/intel/ice/ice_type.h | 1 +
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 +
drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 +
.../intel/ice/ice_virtchnl_allowlist.c | 6 +
10 files changed, 393 insertions(+)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 99a75a59078e..ddb31ad284a4 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -659,6 +659,8 @@ struct ice_pf {
struct ice_agg_node vf_agg_node[ICE_MAX_VF_AGG_NODES];
struct ice_dplls dplls;
struct device *hwmon_dev;
+
+ u8 num_quanta_prof_used;
};
extern struct workqueue_struct *ice_lag_wq;
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
index 5d396c1a7731..54cd9fe27d77 100644
--- a/drivers/net/ethernet/intel/ice/ice_base.c
+++ b/drivers/net/ethernet/intel/ice/ice_base.c
@@ -349,6 +349,8 @@ ice_setup_tx_ctx(struct ice_tx_ring *ring, struct ice_tlan_ctx *tlan_ctx, u16 pf
break;
}
+ tlan_ctx->quanta_prof_idx = ring->quanta_prof_id;
+
tlan_ctx->tso_ena = ICE_TX_LEGACY;
tlan_ctx->tso_qnum = pf_q;
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index 009716a12a26..b22e71dc59d4 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -2436,6 +2436,25 @@ ice_parse_func_caps(struct ice_hw *hw, struct ice_hw_func_caps *func_p,
ice_recalc_port_limited_caps(hw, &func_p->common_cap);
}
+/**
+ * ice_func_id_to_logical_id - map from function id to logical pf id
+ * @active_function_bitmap: active function bitmap
+ * @pf_id: function number of device
+ *
+ * Return: logical PF ID.
+ */
+static int ice_func_id_to_logical_id(u32 active_function_bitmap, u8 pf_id)
+{
+ u8 logical_id = 0;
+ u8 i;
+
+ for (i = 0; i < pf_id; i++)
+ if (active_function_bitmap & BIT(i))
+ logical_id++;
+
+ return logical_id;
+}
+
/**
* ice_parse_valid_functions_cap - Parse ICE_AQC_CAPS_VALID_FUNCTIONS caps
* @hw: pointer to the HW struct
@@ -2453,6 +2472,8 @@ ice_parse_valid_functions_cap(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p,
dev_p->num_funcs = hweight32(number);
ice_debug(hw, ICE_DBG_INIT, "dev caps: num_funcs = %d\n",
dev_p->num_funcs);
+
+ hw->logical_pf_id = ice_func_id_to_logical_id(number, hw->pf_id);
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
index 91cbae1eec89..af9302f0e376 100644
--- a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
+++ b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
@@ -6,6 +6,14 @@
#ifndef _ICE_HW_AUTOGEN_H_
#define _ICE_HW_AUTOGEN_H_
+#define GLCOMM_QUANTA_PROF(_i) (0x002D2D68 + ((_i) * 4))
+#define GLCOMM_QUANTA_PROF_MAX_INDEX 15
+#define GLCOMM_QUANTA_PROF_QUANTA_SIZE_S 0
+#define GLCOMM_QUANTA_PROF_QUANTA_SIZE_M ICE_M(0x3FFF, 0)
+#define GLCOMM_QUANTA_PROF_MAX_CMD_S 16
+#define GLCOMM_QUANTA_PROF_MAX_CMD_M ICE_M(0xFF, 16)
+#define GLCOMM_QUANTA_PROF_MAX_DESC_S 24
+#define GLCOMM_QUANTA_PROF_MAX_DESC_M ICE_M(0x3F, 24)
#define QTX_COMM_DBELL(_DBQM) (0x002C0000 + ((_DBQM) * 4))
#define QTX_COMM_HEAD(_DBQM) (0x000E0000 + ((_DBQM) * 4))
#define QTX_COMM_HEAD_HEAD_S 0
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h
index feba314a3fe4..ea2fae9035b5 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h
@@ -406,6 +406,7 @@ struct ice_tx_ring {
#define ICE_TX_FLAGS_RING_VLAN_L2TAG2 BIT(2)
u8 flags;
u8 dcb_tc; /* Traffic class of ring */
+ u16 quanta_prof_id;
} ____cacheline_internodealigned_in_smp;
static inline bool ice_ring_uses_build_skb(struct ice_rx_ring *ring)
diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h
index 96037bef3e78..cad4d68b19dc 100644
--- a/drivers/net/ethernet/intel/ice/ice_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_type.h
@@ -903,6 +903,7 @@ struct ice_hw {
u8 revision_id;
u8 pf_id; /* device profile info */
+ u8 logical_pf_id;
u16 max_burst_size; /* driver sets this value */
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
index fec16919ec19..becf5a009b4e 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
@@ -52,6 +52,13 @@ struct ice_mdd_vf_events {
u16 last_printed;
};
+struct ice_vf_qs_bw {
+ u32 committed;
+ u32 peak;
+ u16 queue_id;
+ u8 tc;
+};
+
/* VF operations */
struct ice_vf_ops {
enum ice_disq_rst_src reset_type;
@@ -132,6 +139,7 @@ struct ice_vf {
struct devlink_port devlink_port;
u16 num_msix; /* num of MSI-X configured on this VF */
+ struct ice_vf_qs_bw qs_bw[ICE_MAX_RSS_QS_PER_VF];
};
/* Flags for controlling behavior of ice_reset_vf */
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl.c b/drivers/net/ethernet/intel/ice/ice_virtchnl.c
index 1c6ce0c4ed4e..4f9dc67741a5 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl.c
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl.c
@@ -491,6 +491,9 @@ static int ice_vc_get_vf_res_msg(struct ice_vf *vf, u8 *msg)
if (vf->driver_caps & VIRTCHNL_VF_OFFLOAD_USO)
vfres->vf_cap_flags |= VIRTCHNL_VF_OFFLOAD_USO;
+ if (vf->driver_caps & VIRTCHNL_VF_OFFLOAD_QOS)
+ vfres->vf_cap_flags |= VIRTCHNL_VF_OFFLOAD_QOS;
+
vfres->num_vsis = 1;
/* Tx and Rx queue are equal for VF */
vfres->num_queue_pairs = vsi->num_txq;
@@ -1030,6 +1033,191 @@ static int ice_vc_config_rss_hfunc(struct ice_vf *vf, u8 *msg)
NULL, 0);
}
+/**
+ * ice_vc_get_qos_caps - Get current QoS caps from PF
+ * @vf: pointer to the VF info
+ *
+ * Get VF's QoS capabilities, such as TC number, arbiter and
+ * bandwidth from PF.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vc_get_qos_caps(struct ice_vf *vf)
+{
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
+ struct virtchnl_qos_cap_list *cap_list = NULL;
+ u8 tc_prio[ICE_MAX_TRAFFIC_CLASS] = { 0 };
+ struct virtchnl_qos_cap_elem *cfg = NULL;
+ struct ice_vsi_ctx *vsi_ctx;
+ struct ice_pf *pf = vf->pf;
+ struct ice_port_info *pi;
+ struct ice_vsi *vsi;
+ u8 numtc, tc;
+ u16 len = 0;
+ int ret, i;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ pi = pf->hw.port_info;
+ numtc = vsi->tc_cfg.numtc;
+
+ vsi_ctx = ice_get_vsi_ctx(pi->hw, vf->lan_vsi_idx);
+ if (!vsi_ctx) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ len = struct_size(cap_list, cap, numtc);
+ cap_list = kzalloc(len, GFP_KERNEL);
+ if (!cap_list) {
+ v_ret = VIRTCHNL_STATUS_ERR_NO_MEMORY;
+ len = 0;
+ goto err;
+ }
+
+ cap_list->vsi_id = vsi->vsi_num;
+ cap_list->num_elem = numtc;
+
+ /* Store the UP2TC configuration from DCB to a user priority bitmap
+ * of each TC. Each element of prio_of_tc represents one TC. Each
+ * bitmap indicates the user priorities belong to this TC.
+ */
+ for (i = 0; i < ICE_MAX_USER_PRIORITY; i++) {
+ tc = pi->qos_cfg.local_dcbx_cfg.etscfg.prio_table[i];
+ tc_prio[tc] |= BIT(i);
+ }
+
+ for (i = 0; i < numtc; i++) {
+ cfg = &cap_list->cap[i];
+ cfg->tc_num = i;
+ cfg->tc_prio = tc_prio[i];
+ cfg->arbiter = pi->qos_cfg.local_dcbx_cfg.etscfg.tsatable[i];
+ cfg->weight = VIRTCHNL_STRICT_WEIGHT;
+ cfg->type = VIRTCHNL_BW_SHAPER;
+ cfg->shaper.committed = vsi_ctx->sched.bw_t_info[i].cir_bw.bw;
+ cfg->shaper.peak = vsi_ctx->sched.bw_t_info[i].eir_bw.bw;
+ }
+
+err:
+ ret = ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_GET_QOS_CAPS, v_ret,
+ (u8 *)cap_list, len);
+ kfree(cap_list);
+ return ret;
+}
+
+/**
+ * ice_vf_cfg_qs_bw - Configure per queue bandwidth
+ * @vf: pointer to the VF info
+ * @num_queues: number of queues to be configured
+ *
+ * Configure per queue bandwidth.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vf_cfg_qs_bw(struct ice_vf *vf, u16 num_queues)
+{
+ struct ice_hw *hw = &vf->pf->hw;
+ struct ice_vsi *vsi;
+ int ret;
+ u16 i;
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi)
+ return -EINVAL;
+
+ for (i = 0; i < num_queues; i++) {
+ u32 p_rate, min_rate;
+ u8 tc;
+
+ p_rate = vf->qs_bw[i].peak;
+ min_rate = vf->qs_bw[i].committed;
+ tc = vf->qs_bw[i].tc;
+ if (p_rate)
+ ret = ice_cfg_q_bw_lmt(hw->port_info, vsi->idx, tc,
+ vf->qs_bw[i].queue_id,
+ ICE_MAX_BW, p_rate);
+ else
+ ret = ice_cfg_q_bw_dflt_lmt(hw->port_info, vsi->idx, tc,
+ vf->qs_bw[i].queue_id,
+ ICE_MAX_BW);
+ if (ret)
+ return ret;
+
+ if (min_rate)
+ ret = ice_cfg_q_bw_lmt(hw->port_info, vsi->idx, tc,
+ vf->qs_bw[i].queue_id,
+ ICE_MIN_BW, min_rate);
+ else
+ ret = ice_cfg_q_bw_dflt_lmt(hw->port_info, vsi->idx, tc,
+ vf->qs_bw[i].queue_id,
+ ICE_MIN_BW);
+
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_vf_cfg_q_quanta_profile - Configure quanta profile
+ * @vf: pointer to the VF info
+ * @quanta_prof_idx: pointer to the quanta profile index
+ * @quanta_size: quanta size to be set
+ *
+ * This function chooses available quanta profile and configures the register.
+ * The quanta profile is evenly divided by the number of device ports, and then
+ * available to the specific PF and VFs. The first profile for each PF is a
+ * reserved default profile. Only quanta size of the rest unused profile can be
+ * modified.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vf_cfg_q_quanta_profile(struct ice_vf *vf, u16 quanta_size,
+ u16 *quanta_prof_idx)
+{
+ const u16 n_desc = calc_quanta_desc(quanta_size);
+ struct ice_hw *hw = &vf->pf->hw;
+ const u16 n_cmd = 2 * n_desc;
+ struct ice_pf *pf = vf->pf;
+ u16 per_pf, begin_id;
+ u8 n_used;
+ u32 reg;
+
+ begin_id = (GLCOMM_QUANTA_PROF_MAX_INDEX + 1) / hw->dev_caps.num_funcs *
+ hw->logical_pf_id;
+
+ if (quanta_size == ICE_DFLT_QUANTA) {
+ *quanta_prof_idx = begin_id;
+ } else {
+ per_pf = (GLCOMM_QUANTA_PROF_MAX_INDEX + 1) /
+ hw->dev_caps.num_funcs;
+ n_used = pf->num_quanta_prof_used;
+ if (n_used < per_pf) {
+ *quanta_prof_idx = begin_id + 1 + n_used;
+ pf->num_quanta_prof_used++;
+ } else {
+ return -EINVAL;
+ }
+ }
+
+ reg = FIELD_PREP(GLCOMM_QUANTA_PROF_QUANTA_SIZE_M, quanta_size) |
+ FIELD_PREP(GLCOMM_QUANTA_PROF_MAX_CMD_M, n_cmd) |
+ FIELD_PREP(GLCOMM_QUANTA_PROF_MAX_DESC_M, n_desc);
+ wr32(hw, GLCOMM_QUANTA_PROF(*quanta_prof_idx), reg);
+
+ return 0;
+}
+
/**
* ice_vc_cfg_promiscuous_mode_msg
* @vf: pointer to the VF info
@@ -1631,6 +1819,139 @@ static int ice_vc_cfg_irq_map_msg(struct ice_vf *vf, u8 *msg)
NULL, 0);
}
+/**
+ * ice_vc_cfg_q_bw - Configure per queue bandwidth
+ * @vf: pointer to the VF info
+ * @msg: pointer to the msg buffer which holds the command descriptor
+ *
+ * Configure VF queues bandwidth.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vc_cfg_q_bw(struct ice_vf *vf, u8 *msg)
+{
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
+ struct virtchnl_queues_bw_cfg *qbw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ struct ice_vsi *vsi;
+ u16 i;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states) ||
+ !ice_vc_isvalid_vsi_id(vf, qbw->vsi_id)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ if (qbw->num_queues > ICE_MAX_RSS_QS_PER_VF ||
+ qbw->num_queues > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
+ dev_err(ice_pf_to_dev(vf->pf), "VF-%d trying to configure more than allocated number of queues: %d\n",
+ vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ for (i = 0; i < qbw->num_queues; i++) {
+ if (qbw->cfg[i].shaper.peak != 0 && vf->max_tx_rate != 0 &&
+ qbw->cfg[i].shaper.peak > vf->max_tx_rate)
+ dev_warn(ice_pf_to_dev(vf->pf), "The maximum queue %d rate limit configuration may not take effect because the maximum TX rate for VF-%d is %d\n",
+ qbw->cfg[i].queue_id, vf->vf_id, vf->max_tx_rate);
+ if (qbw->cfg[i].shaper.committed != 0 && vf->min_tx_rate != 0 &&
+ qbw->cfg[i].shaper.committed < vf->min_tx_rate)
+ dev_warn(ice_pf_to_dev(vf->pf), "The minimum queue %d rate limit configuration may not take effect because the minimum TX rate for VF-%d is %d\n",
+ qbw->cfg[i].queue_id, vf->vf_id, vf->max_tx_rate);
+ }
+
+ for (i = 0; i < qbw->num_queues; i++) {
+ vf->qs_bw[i].queue_id = qbw->cfg[i].queue_id;
+ vf->qs_bw[i].peak = qbw->cfg[i].shaper.peak;
+ vf->qs_bw[i].committed = qbw->cfg[i].shaper.committed;
+ vf->qs_bw[i].tc = qbw->cfg[i].tc;
+ }
+
+ if (ice_vf_cfg_qs_bw(vf, qbw->num_queues))
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+
+err:
+ /* send the response to the VF */
+ return ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_CONFIG_QUEUE_BW,
+ v_ret, NULL, 0);
+}
+
+/**
+ * ice_vc_cfg_q_quanta - Configure per queue quanta
+ * @vf: pointer to the VF info
+ * @msg: pointer to the msg buffer which holds the command descriptor
+ *
+ * Configure VF queues quanta.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vc_cfg_q_quanta(struct ice_vf *vf, u8 *msg)
+{
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
+ u16 quanta_prof_id, quanta_size, start_qid, end_qid, i;
+ struct virtchnl_quanta_cfg *qquanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ struct ice_vsi *vsi;
+ int ret;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ end_qid = qquanta->queue_select.start_queue_id +
+ qquanta->queue_select.num_queues;
+ if (end_qid > ICE_MAX_RSS_QS_PER_VF ||
+ end_qid > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
+ dev_err(ice_pf_to_dev(vf->pf), "VF-%d trying to configure more than allocated number of queues: %d\n",
+ vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ quanta_size = qquanta->quanta_size;
+ if (quanta_size > ICE_MAX_QUANTA_SIZE ||
+ quanta_size < ICE_MIN_QUANTA_SIZE) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ if (quanta_size % 64) {
+ dev_err(ice_pf_to_dev(vf->pf), "quanta size should be the product of 64\n");
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ ret = ice_vf_cfg_q_quanta_profile(vf, quanta_size,
+ &quanta_prof_id);
+ if (ret) {
+ v_ret = VIRTCHNL_STATUS_ERR_NOT_SUPPORTED;
+ goto err;
+ }
+
+ start_qid = qquanta->queue_select.start_queue_id;
+ for (i = start_qid; i < end_qid; i++)
+ vsi->tx_rings[i]->quanta_prof_id = quanta_prof_id;
+
+err:
+ /* send the response to the VF */
+ return ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_CONFIG_QUANTA,
+ v_ret, NULL, 0);
+}
+
/**
* ice_vc_cfg_qs_msg
* @vf: pointer to the VF info
@@ -3817,6 +4138,9 @@ static const struct ice_virtchnl_ops ice_virtchnl_dflt_ops = {
.dis_vlan_stripping_v2_msg = ice_vc_dis_vlan_stripping_v2_msg,
.ena_vlan_insertion_v2_msg = ice_vc_ena_vlan_insertion_v2_msg,
.dis_vlan_insertion_v2_msg = ice_vc_dis_vlan_insertion_v2_msg,
+ .get_qos_caps = ice_vc_get_qos_caps,
+ .cfg_q_bw = ice_vc_cfg_q_bw,
+ .cfg_q_quanta = ice_vc_cfg_q_quanta,
};
/**
@@ -4173,6 +4497,15 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event,
case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
err = ops->dis_vlan_insertion_v2_msg(vf, msg);
break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ err = ops->get_qos_caps(vf);
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ err = ops->cfg_q_bw(vf, msg);
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ err = ops->cfg_q_quanta(vf, msg);
+ break;
case VIRTCHNL_OP_UNKNOWN:
default:
dev_err(dev, "Unsupported opcode %d from VF %d\n", v_opcode,
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl.h b/drivers/net/ethernet/intel/ice/ice_virtchnl.h
index 3a4115869153..0c629aef9baf 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl.h
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl.h
@@ -13,6 +13,13 @@
/* Restrict number of MAC Addr and VLAN that non-trusted VF can programmed */
#define ICE_MAX_VLAN_PER_VF 8
+#define ICE_DFLT_QUANTA 1024
+#define ICE_MAX_QUANTA_SIZE 4096
+#define ICE_MIN_QUANTA_SIZE 256
+
+#define calc_quanta_desc(x) \
+ max_t(u16, 12, min_t(u16, 63, (((x) + 66) / 132) * 2 + 4))
+
/* MAC filters: 1 is reserved for the VF's default/perm_addr/LAA MAC, 1 for
* broadcast, and 16 for additional unicast/multicast filters
*/
@@ -61,6 +68,10 @@ struct ice_virtchnl_ops {
int (*dis_vlan_stripping_v2_msg)(struct ice_vf *vf, u8 *msg);
int (*ena_vlan_insertion_v2_msg)(struct ice_vf *vf, u8 *msg);
int (*dis_vlan_insertion_v2_msg)(struct ice_vf *vf, u8 *msg);
+ int (*get_qos_caps)(struct ice_vf *vf);
+ int (*cfg_q_tc_map)(struct ice_vf *vf, u8 *msg);
+ int (*cfg_q_bw)(struct ice_vf *vf, u8 *msg);
+ int (*cfg_q_quanta)(struct ice_vf *vf, u8 *msg);
};
#ifdef CONFIG_PCI_IOV
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c b/drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c
index d796dbd2a440..c105a82ee136 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c
@@ -84,6 +84,11 @@ static const u32 fdir_pf_allowlist_opcodes[] = {
VIRTCHNL_OP_ADD_FDIR_FILTER, VIRTCHNL_OP_DEL_FDIR_FILTER,
};
+static const u32 tc_allowlist_opcodes[] = {
+ VIRTCHNL_OP_GET_QOS_CAPS, VIRTCHNL_OP_CONFIG_QUEUE_BW,
+ VIRTCHNL_OP_CONFIG_QUANTA,
+};
+
struct allowlist_opcode_info {
const u32 *opcodes;
size_t size;
@@ -104,6 +109,7 @@ static const struct allowlist_opcode_info allowlist_opcodes[] = {
ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF, adv_rss_pf_allowlist_opcodes),
ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_FDIR_PF, fdir_pf_allowlist_opcodes),
ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_VLAN_V2, vlan_v2_allowlist_opcodes),
+ ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_QOS, tc_allowlist_opcodes),
};
/**
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 10/11] iavf: Add net_shaper_ops support
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (8 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 09/11] ice: Support VF " Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-08-02 13:44 ` Jiri Pirko
2024-07-24 20:24 ` [PATCH RFC v2 11/11] iavf: add support to exchange qos capabilities Paolo Abeni
2024-07-29 6:30 ` [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Jiri Pirko
11 siblings, 1 reply; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
From: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Implement net_shaper_ops support for IAVF. This enables configuration
of rate limiting on per queue basis. Customer intends to enforce
bandwidth limit on Tx traffic steered to the queue by configuring
rate limits on the queue.
To set rate limiting for a queue, update shaper object of given queues
in driver and send VIRTCHNL_OP_CONFIG_QUEUE_BW to PF to update HW
configuration.
Deleting shaper configured for queue is nothing but configuring shaper
with bw_max 0. The PF restores the default rate limiting config
when bw_max is zero.
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
---
drivers/net/ethernet/intel/Kconfig | 1 +
drivers/net/ethernet/intel/iavf/iavf.h | 3 +
drivers/net/ethernet/intel/iavf/iavf_main.c | 171 ++++++++++++++++++
drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 +
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 65 +++++++
5 files changed, 242 insertions(+)
diff --git a/drivers/net/ethernet/intel/Kconfig b/drivers/net/ethernet/intel/Kconfig
index 0375c7448a57..20bc40eec487 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -258,6 +258,7 @@ config I40E_DCB
config IAVF
tristate
select LIBIE
+ select NET_SHAPER
config I40EVF
tristate "Intel(R) Ethernet Adaptive Virtual Function support"
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index 23a6557fc3db..f5d1142ea427 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -33,6 +33,7 @@
#include <net/udp.h>
#include <net/tc_act/tc_gact.h>
#include <net/tc_act/tc_mirred.h>
+#include <net/net_shaper.h>
#include "iavf_type.h"
#include <linux/avf/virtchnl.h>
@@ -335,6 +336,7 @@ struct iavf_adapter {
#define IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_INSERTION BIT_ULL(36)
#define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION BIT_ULL(37)
#define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION BIT_ULL(38)
+#define IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW BIT_ULL(39)
/* flags for processing extended capability messages during
* __IAVF_INIT_EXTENDED_CAPS. Each capability exchange requires
@@ -551,6 +553,7 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
int iavf_config_rss(struct iavf_adapter *adapter);
int iavf_lan_add_device(struct iavf_adapter *adapter);
int iavf_lan_del_device(struct iavf_adapter *adapter);
+void iavf_cfg_queues_bw(struct iavf_adapter *adapter);
void iavf_enable_channels(struct iavf_adapter *adapter);
void iavf_disable_channels(struct iavf_adapter *adapter);
void iavf_add_cloud_filter(struct iavf_adapter *adapter);
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index ff11bafb3b4f..3a5ae0cd31c7 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -2085,6 +2085,11 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
return 0;
}
+ if (adapter->aq_required & IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW) {
+ iavf_cfg_queues_bw(adapter);
+ return 0;
+ }
+
if (adapter->aq_required & IAVF_FLAG_AQ_CONFIGURE_QUEUES) {
iavf_configure_queues(adapter);
return 0;
@@ -2918,6 +2923,25 @@ static void iavf_disable_vf(struct iavf_adapter *adapter)
dev_info(&adapter->pdev->dev, "Reset task did not complete, VF disabled\n");
}
+/**
+ * iavf_reconfig_qs_bw - Call-back task to handle hardware reset
+ * @adapter: board private structure
+ *
+ * After a reset, the shaper parameters of queues need to be replayed again.
+ * Since the net_shaper_info object inside TX rings persists across reset,
+ * set the update flag for all queues so that the virtchnl message is triggered
+ * for all queues.
+ **/
+static void iavf_reconfig_qs_bw(struct iavf_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < adapter->num_active_queues; i++)
+ adapter->tx_rings[i].q_shaper_update = true;
+
+ adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+}
+
/**
* iavf_reset_task - Call-back task to handle hardware reset
* @work: pointer to work_struct
@@ -3124,6 +3148,8 @@ static void iavf_reset_task(struct work_struct *work)
iavf_up_complete(adapter);
iavf_irq_enable(adapter, true);
+
+ iavf_reconfig_qs_bw(adapter);
} else {
iavf_change_state(adapter, __IAVF_DOWN);
wake_up(&adapter->down_waitqueue);
@@ -4743,6 +4769,150 @@ static netdev_features_t iavf_fix_features(struct net_device *netdev,
return iavf_fix_strip_features(adapter, features);
}
+static int iavf_verify_handle(struct net_device *dev, u32 handle,
+ struct netlink_ext_ack *extack)
+{
+ struct iavf_adapter *adapter = netdev_priv(dev);
+ enum net_shaper_scope scope;
+ int qid;
+
+ scope = net_shaper_handle_scope(handle);
+ qid = net_shaper_handle_id(handle);
+
+ if (scope != NET_SHAPER_SCOPE_QUEUE) {
+ NL_SET_ERR_MSG_FMT(extack, "Invalid shaper handle %x, unsupported scope %d",
+ handle, scope);
+ return -EOPNOTSUPP;
+ }
+
+ if (qid >= adapter->num_active_queues) {
+ NL_SET_ERR_MSG_FMT(extack, "Invalid shaper handle %x, queued id %d max %d",
+ handle, qid, adapter->num_active_queues);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/**
+ * iavf_verify_shaper_info - check that shaper info received
+ * @dev: pointer to netdev
+ * @shaper: configuration of shaper.
+ * @extack: Netlink extended ACK for reporting errors
+ *
+ * Returns:
+ * * %0 - Success
+ * * %-EOPNOTSUPP - Driver doesn't support this scope.
+ * * %-EINVAL - Invalid queue number in input
+ **/
+static int
+iavf_verify_shaper_info(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack)
+{
+ return iavf_verify_handle(dev, shaper->handle, extack);
+}
+
+/**
+ * iavf_shaper_set - check that shaper info received
+ * @dev: pointer to netdev
+ * @shaper: configuration of shaper.
+ * @extack: Netlink extended ACK for reporting errors
+ *
+ * Returns:
+ * * %0 - Success
+ * * %-EOPNOTSUPP - Driver doesn't support this scope.
+ * * %-EINVAL - Invalid queue number in input
+ **/
+static int
+iavf_shaper_set(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack)
+{
+ struct iavf_adapter *adapter = netdev_priv(dev);
+ bool need_cfg_update = false;
+ enum net_shaper_scope scope;
+ int id, ret = 0;
+
+ ret = iavf_verify_shaper_info(dev, shaper, extack);
+ if (ret)
+ return ret;
+
+ scope = net_shaper_handle_scope(shaper->handle);
+ id = net_shaper_handle_id(shaper->handle);
+
+ if (scope == NET_SHAPER_SCOPE_QUEUE) {
+ struct iavf_ring *tx_ring = &adapter->tx_rings[id];
+
+ tx_ring->q_shaper.bw_min = div_u64(shaper->bw_min, 1000);
+ tx_ring->q_shaper.bw_max = div_u64(shaper->bw_max, 1000);
+ tx_ring->q_shaper_update = true;
+ need_cfg_update = true;
+ }
+
+ if (need_cfg_update)
+ adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+
+ return 0;
+}
+
+static int iavf_shaper_del(struct net_device *dev,
+ const u32 handle,
+ struct netlink_ext_ack *extack)
+{
+ struct iavf_adapter *adapter = netdev_priv(dev);
+ bool need_cfg_update = false;
+ enum net_shaper_scope scope;
+ int qid, ret;
+
+ ret = iavf_verify_handle(dev, handle, extack);
+ if (ret < 0)
+ return ret;
+
+ scope = net_shaper_handle_scope(handle);
+ qid = net_shaper_handle_id(handle);
+
+ if (scope == NET_SHAPER_SCOPE_QUEUE) {
+ struct iavf_ring *tx_ring = &adapter->tx_rings[qid];
+
+ tx_ring->q_shaper.bw_min = 0;
+ tx_ring->q_shaper.bw_max = 0;
+ tx_ring->q_shaper_update = true;
+ need_cfg_update = true;
+ }
+
+ if (need_cfg_update)
+ adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+
+ return 0;
+}
+
+static int iavf_shaper_group(struct net_device *dev, int nr_inputs,
+ const struct net_shaper_info *inputs,
+ const struct net_shaper_info *output,
+ struct netlink_ext_ack *extack)
+{
+ return -EOPNOTSUPP;
+}
+
+static int iavf_shaper_cap(struct net_device *dev, enum net_shaper_scope scope,
+ unsigned long *flags)
+{
+ if (scope != NET_SHAPER_SCOPE_QUEUE)
+ return -EOPNOTSUPP;
+
+ *flags = BIT(NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MIN) |
+ BIT(NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MAX) |
+ BIT(NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_BPS);
+ return 0;
+}
+
+static const struct net_shaper_ops iavf_shaper_ops = {
+ .set = iavf_shaper_set,
+ .delete = iavf_shaper_del,
+ .group = iavf_shaper_group,
+ .capabilities = iavf_shaper_cap,
+};
+
static const struct net_device_ops iavf_netdev_ops = {
.ndo_open = iavf_open,
.ndo_stop = iavf_close,
@@ -4758,6 +4928,7 @@ static const struct net_device_ops iavf_netdev_ops = {
.ndo_fix_features = iavf_fix_features,
.ndo_set_features = iavf_set_features,
.ndo_setup_tc = iavf_setup_tc,
+ .net_shaper_ops = &iavf_shaper_ops,
};
/**
diff --git a/drivers/net/ethernet/intel/iavf/iavf_txrx.h b/drivers/net/ethernet/intel/iavf/iavf_txrx.h
index d7b5587aeb8e..dd503ee50b7f 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_txrx.h
+++ b/drivers/net/ethernet/intel/iavf/iavf_txrx.h
@@ -296,6 +296,8 @@ struct iavf_ring {
*/
u32 rx_buf_len;
+ struct net_shaper_info q_shaper;
+ bool q_shaper_update;
} ____cacheline_internodealigned_in_smp;
#define IAVF_ITR_ADAPTIVE_MIN_INC 0x0002
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index 1e543f6a7c30..c0611608d332 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -1506,6 +1506,60 @@ iavf_set_adapter_link_speed_from_vpe(struct iavf_adapter *adapter,
adapter->link_speed = vpe->event_data.link_event.link_speed;
}
+/**
+ * iavf_cfg_queues_bw - configure bandwidth of allocated queues
+ * @adapter: iavf adapter structure instance
+ *
+ * This function requests PF to configure queue bandwidth of allocated queues
+ */
+void iavf_cfg_queues_bw(struct iavf_adapter *adapter)
+{
+ struct virtchnl_queues_bw_cfg *qs_bw_cfg;
+ struct net_shaper_info *q_shaper;
+ int qs_to_update = 0;
+ int i, inx = 0;
+ size_t len;
+
+ if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
+ /* bail because we already have a command pending */
+ dev_err(&adapter->pdev->dev,
+ "Cannot set tc queue bw, command %d pending\n",
+ adapter->current_op);
+ return;
+ }
+
+ for (i = 0; i < adapter->num_active_queues; i++) {
+ if (adapter->tx_rings[i].q_shaper_update)
+ qs_to_update++;
+ }
+ len = struct_size(qs_bw_cfg, cfg, qs_to_update);
+ qs_bw_cfg = kzalloc(len, GFP_KERNEL);
+ if (!qs_bw_cfg)
+ return;
+
+ qs_bw_cfg->vsi_id = adapter->vsi.id;
+ qs_bw_cfg->num_queues = qs_to_update;
+
+ for (i = 0; i < adapter->num_active_queues; i++) {
+ struct iavf_ring *tx_ring = &adapter->tx_rings[i];
+
+ q_shaper = &tx_ring->q_shaper;
+ if (tx_ring->q_shaper_update) {
+ qs_bw_cfg->cfg[inx].queue_id = i;
+ qs_bw_cfg->cfg[inx].shaper.peak = q_shaper->bw_max;
+ qs_bw_cfg->cfg[inx].shaper.committed = q_shaper->bw_min;
+ qs_bw_cfg->cfg[inx].tc = 0;
+ inx++;
+ }
+ }
+
+ adapter->current_op = VIRTCHNL_OP_CONFIG_QUEUE_BW;
+ adapter->aq_required &= ~IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_CONFIG_QUEUE_BW,
+ (u8 *)qs_bw_cfg, len);
+ kfree(qs_bw_cfg);
+}
+
/**
* iavf_enable_channels
* @adapter: adapter structure
@@ -2226,6 +2280,10 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC;
break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ dev_warn(&adapter->pdev->dev, "Failed to Config Queue BW, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ break;
default:
dev_err(&adapter->pdev->dev, "PF returned error %d (%s) to our request %d\n",
v_retval, iavf_stat_str(&adapter->hw, v_retval),
@@ -2560,6 +2618,13 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
if (!v_retval)
iavf_netdev_features_vlan_strip_set(netdev, false);
break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW: {
+ int i;
+ /* shaper configuration is successful for all queues */
+ for (i = 0; i < adapter->num_active_queues; i++)
+ adapter->tx_rings[i].q_shaper_update = false;
+ }
+ break;
default:
if (adapter->current_op && (v_opcode != adapter->current_op))
dev_warn(&adapter->pdev->dev, "Expected response %d from PF, received %d\n",
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH RFC v2 11/11] iavf: add support to exchange qos capabilities
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (9 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 10/11] iavf: Add net_shaper_ops support Paolo Abeni
@ 2024-07-24 20:24 ` Paolo Abeni
2024-07-29 6:30 ` [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Jiri Pirko
11 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-24 20:24 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
From: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
During driver initialization VF determines QOS capability is allowed
by PF and receives QOS parameters. After which quanta size for queues
is configured which is not configurable and is set to 1KB currently.
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
---
drivers/net/ethernet/intel/iavf/iavf.h | 10 ++
drivers/net/ethernet/intel/iavf/iavf_main.c | 46 +++++++++-
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 92 ++++++++++++++++++-
3 files changed, 145 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index f5d1142ea427..dd9cca067360 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -250,6 +250,9 @@ struct iavf_cloud_filter {
#define IAVF_RESET_WAIT_DETECTED_COUNT 500
#define IAVF_RESET_WAIT_COMPLETE_COUNT 2000
+#define IAVF_MAX_QOS_TC_NUM 8
+#define IAVF_DEFAULT_QUANTA_SIZE 1024
+
/* board specific private data structure */
struct iavf_adapter {
struct workqueue_struct *wq;
@@ -337,6 +340,8 @@ struct iavf_adapter {
#define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION BIT_ULL(37)
#define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION BIT_ULL(38)
#define IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW BIT_ULL(39)
+#define IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE BIT_ULL(40)
+#define IAVF_FLAG_AQ_GET_QOS_CAPS BIT_ULL(41)
/* flags for processing extended capability messages during
* __IAVF_INIT_EXTENDED_CAPS. Each capability exchange requires
@@ -407,6 +412,8 @@ struct iavf_adapter {
VIRTCHNL_VF_OFFLOAD_FDIR_PF)
#define ADV_RSS_SUPPORT(_a) ((_a)->vf_res->vf_cap_flags & \
VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF)
+#define QOS_ALLOWED(_a) ((_a)->vf_res->vf_cap_flags & \
+ VIRTCHNL_VF_OFFLOAD_QOS)
struct virtchnl_vf_resource *vf_res; /* incl. all VSIs */
struct virtchnl_vsi_resource *vsi_res; /* our LAN VSI */
struct virtchnl_version_info pf_version;
@@ -415,6 +422,7 @@ struct iavf_adapter {
struct virtchnl_vlan_caps vlan_v2_caps;
u16 msg_enable;
struct iavf_eth_stats current_stats;
+ struct virtchnl_qos_cap_list *qos_caps;
struct iavf_vsi vsi;
u32 aq_wait_count;
/* RSS stuff */
@@ -554,6 +562,8 @@ int iavf_config_rss(struct iavf_adapter *adapter);
int iavf_lan_add_device(struct iavf_adapter *adapter);
int iavf_lan_del_device(struct iavf_adapter *adapter);
void iavf_cfg_queues_bw(struct iavf_adapter *adapter);
+void iavf_cfg_queues_quanta_size(struct iavf_adapter *adapter);
+void iavf_get_qos_caps(struct iavf_adapter *adapter);
void iavf_enable_channels(struct iavf_adapter *adapter);
void iavf_disable_channels(struct iavf_adapter *adapter);
void iavf_add_cloud_filter(struct iavf_adapter *adapter);
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 3a5ae0cd31c7..0ee128477d59 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -2090,6 +2090,16 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
return 0;
}
+ if (adapter->aq_required & IAVF_FLAG_AQ_GET_QOS_CAPS) {
+ iavf_get_qos_caps(adapter);
+ return 0;
+ }
+
+ if (adapter->aq_required & IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE) {
+ iavf_cfg_queues_quanta_size(adapter);
+ return 0;
+ }
+
if (adapter->aq_required & IAVF_FLAG_AQ_CONFIGURE_QUEUES) {
iavf_configure_queues(adapter);
return 0;
@@ -2675,6 +2685,9 @@ static void iavf_init_config_adapter(struct iavf_adapter *adapter)
/* request initial VLAN offload settings */
iavf_set_vlan_offload_features(adapter, 0, netdev->features);
+ if (QOS_ALLOWED(adapter))
+ adapter->aq_required |= IAVF_FLAG_AQ_GET_QOS_CAPS;
+
iavf_schedule_finish_config(adapter);
return;
@@ -4809,7 +4822,27 @@ iavf_verify_shaper_info(struct net_device *dev,
const struct net_shaper_info *shaper,
struct netlink_ext_ack *extack)
{
- return iavf_verify_handle(dev, shaper->handle, extack);
+ struct iavf_adapter *adapter = netdev_priv(dev);
+ enum net_shaper_scope scope;
+ int ret, qid;
+ u64 vf_max;
+
+ ret = iavf_verify_handle(dev, shaper->handle, extack);
+ if (ret)
+ return ret;
+
+ scope = net_shaper_handle_scope(shaper->handle);
+ qid = net_shaper_handle_id(shaper->handle);
+
+ if (scope == NET_SHAPER_SCOPE_QUEUE) {
+ vf_max = adapter->qos_caps->cap[0].shaper.peak;
+ if (vf_max && shaper->bw_max > vf_max) {
+ NL_SET_ERR_MSG_FMT(extack, "Max rate (%llu) of queue %d can't exceed max TX rate of VF (%llu kbps)",
+ shaper->bw_max, qid,
+ vf_max);
+ }
+ }
+ return 0;
}
/**
@@ -5073,7 +5106,7 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
struct net_device *netdev;
struct iavf_adapter *adapter = NULL;
struct iavf_hw *hw = NULL;
- int err;
+ int err, len;
err = pci_enable_device(pdev);
if (err)
@@ -5141,6 +5174,13 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
hw->bus.func = PCI_FUNC(pdev->devfn);
hw->bus.bus_id = pdev->bus->number;
+ len = struct_size(adapter->qos_caps, cap, IAVF_MAX_QOS_TC_NUM);
+ adapter->qos_caps = kzalloc(len, GFP_KERNEL);
+ if (!adapter->qos_caps) {
+ err = -ENOMEM;
+ goto err_alloc_qos_cap;
+ }
+
/* set up the locks for the AQ, do this only once in probe
* and destroy them only once in remove
*/
@@ -5179,6 +5219,8 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
/* Initialization goes on in the work. Do not add more of it below. */
return 0;
+err_alloc_qos_cap:
+ iounmap(hw->hw_addr);
err_ioremap:
destroy_workqueue(adapter->wq);
err_alloc_wq:
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index c0611608d332..2f1be42ebd58 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -150,7 +150,8 @@ int iavf_send_vf_config_msg(struct iavf_adapter *adapter)
VIRTCHNL_VF_OFFLOAD_USO |
VIRTCHNL_VF_OFFLOAD_FDIR_PF |
VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF |
- VIRTCHNL_VF_CAP_ADV_LINK_SPEED;
+ VIRTCHNL_VF_CAP_ADV_LINK_SPEED |
+ VIRTCHNL_VF_OFFLOAD_QOS;
adapter->current_op = VIRTCHNL_OP_GET_VF_RESOURCES;
adapter->aq_required &= ~IAVF_FLAG_AQ_GET_CONFIG;
@@ -1506,6 +1507,76 @@ iavf_set_adapter_link_speed_from_vpe(struct iavf_adapter *adapter,
adapter->link_speed = vpe->event_data.link_event.link_speed;
}
+/**
+ * iavf_get_qos_caps - get qos caps support
+ * @adapter: iavf adapter struct instance
+ *
+ * This function requests PF for Supported QoS Caps.
+ */
+void iavf_get_qos_caps(struct iavf_adapter *adapter)
+{
+ if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
+ /* bail because we already have a command pending */
+ dev_err(&adapter->pdev->dev,
+ "Cannot get qos caps, command %d pending\n",
+ adapter->current_op);
+ return;
+ }
+
+ adapter->current_op = VIRTCHNL_OP_GET_QOS_CAPS;
+ adapter->aq_required &= ~IAVF_FLAG_AQ_GET_QOS_CAPS;
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_GET_QOS_CAPS, NULL, 0);
+}
+
+/**
+ * iavf_set_quanta_size - set quanta size of queue chunk
+ * @adapter: iavf adapter struct instance
+ * @quanta_size: quanta size in bytes
+ * @queue_index: starting index of queue chunk
+ * @num_queues: number of queues in the queue chunk
+ *
+ * This function requests PF to set quanta size of queue chunk
+ * starting at queue_index.
+ */
+static void
+iavf_set_quanta_size(struct iavf_adapter *adapter, u16 quanta_size,
+ u16 queue_index, u16 num_queues)
+{
+ struct virtchnl_quanta_cfg quanta_cfg;
+
+ if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
+ /* bail because we already have a command pending */
+ dev_err(&adapter->pdev->dev,
+ "Cannot set queue quanta size, command %d pending\n",
+ adapter->current_op);
+ return;
+ }
+
+ adapter->current_op = VIRTCHNL_OP_CONFIG_QUANTA;
+ quanta_cfg.quanta_size = quanta_size;
+ quanta_cfg.queue_select.type = VIRTCHNL_QUEUE_TYPE_TX;
+ quanta_cfg.queue_select.start_queue_id = queue_index;
+ quanta_cfg.queue_select.num_queues = num_queues;
+ adapter->aq_required &= ~IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE;
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_CONFIG_QUANTA,
+ (u8 *)&quanta_cfg, sizeof(quanta_cfg));
+}
+
+/**
+ * iavf_cfg_queues_quanta_size - configure quanta size of queues
+ * @adapter: adapter structure
+ *
+ * Request that the PF configure quanta size of allocated queues.
+ **/
+void iavf_cfg_queues_quanta_size(struct iavf_adapter *adapter)
+{
+ int quanta_size = IAVF_DEFAULT_QUANTA_SIZE;
+
+ /* Set Queue Quanta Size to default */
+ iavf_set_quanta_size(adapter, quanta_size, 0,
+ adapter->num_active_queues);
+}
+
/**
* iavf_cfg_queues_bw - configure bandwidth of allocated queues
* @adapter: iavf adapter structure instance
@@ -2280,6 +2351,14 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC;
break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ dev_warn(&adapter->pdev->dev, "Failed to Get Qos CAPs, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ dev_warn(&adapter->pdev->dev, "Failed to Config Quanta, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ break;
case VIRTCHNL_OP_CONFIG_QUEUE_BW:
dev_warn(&adapter->pdev->dev, "Failed to Config Queue BW, error %s\n",
iavf_stat_str(&adapter->hw, v_retval));
@@ -2618,6 +2697,17 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
if (!v_retval)
iavf_netdev_features_vlan_strip_set(netdev, false);
break;
+ case VIRTCHNL_OP_GET_QOS_CAPS: {
+ u16 len = struct_size(adapter->qos_caps, cap,
+ IAVF_MAX_QOS_TC_NUM);
+
+ memcpy(adapter->qos_caps, msg, min(msglen, len));
+
+ adapter->aq_required |= IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ break;
case VIRTCHNL_OP_CONFIG_QUEUE_BW: {
int i;
/* shaper configuration is successful for all queues */
--
2.45.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH RFC v2 01/11] netlink: spec: add shaper YAML spec
2024-07-24 20:24 ` [PATCH RFC v2 01/11] netlink: spec: add shaper YAML spec Paolo Abeni
@ 2024-07-25 11:42 ` Paolo Abeni
0 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-25 11:42 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 7/24/24 22:24, Paolo Abeni wrote:
> Define the user-space visible interface to query, configure and delete
> network shapers via yaml definition.
>
> Add dummy implementations for the relevant NL callbacks.
>
> set() and delete() operations touch a single shaper creating/updating or
> deleting it.
> The group() operation creates a shaper's group, nesting multiple input
> shapers under the specified output shaper.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
> RFC v1 -> RFC v2:
> - u64 -> uint
> - net_shapers -> net-shapers
> - documented all the attributes
> - dropped [ admin-perm ] for get() op
> - group op
> - set/delete touch a single shaper
FWIW, the CI told me I forgot entirely about the generated user-space
code, and the naming here is not supported by the current tooling. I'll
send something alike the following when net-next re-open.
---
diff --git a/tools/net/ynl/ynl-gen-c.py b/tools/net/ynl/ynl-gen-c.py
index 51529fabd517..717530bc9c52 100755
--- a/tools/net/ynl/ynl-gen-c.py
+++ b/tools/net/ynl/ynl-gen-c.py
@@ -2668,13 +2668,15 @@ def main():
cw.p('#define ' + hdr_prot)
cw.nl()
+ hdr_file=os.path.basename(args.out_file[:-2]) + ".h"
+
if args.mode == 'kernel':
cw.p('#include <net/netlink.h>')
cw.p('#include <net/genetlink.h>')
cw.nl()
if not args.header:
if args.out_file:
- cw.p(f'#include
"{os.path.basename(args.out_file[:-2])}.h"')
+ cw.p(f'#include "{hdr_file}"')
cw.nl()
headers = ['uapi/' + parsed.uapi_header]
headers += parsed.kernel_family.get('headers', [])
@@ -2686,7 +2688,7 @@ def main():
if family_contains_bitfield32(parsed):
cw.p('#include <linux/netlink.h>')
else:
- cw.p(f'#include "{parsed.name}-user.h"')
+ cw.p(f'#include "{hdr_file}"')
cw.p('#include "ynl.h"')
headers = [parsed.uapi_header]
for definition in parsed['definitions']:
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
` (10 preceding siblings ...)
2024-07-24 20:24 ` [PATCH RFC v2 11/11] iavf: add support to exchange qos capabilities Paolo Abeni
@ 2024-07-29 6:30 ` Jiri Pirko
2024-07-29 14:42 ` Paolo Abeni
11 siblings, 1 reply; 19+ messages in thread
From: Jiri Pirko @ 2024-07-29 6:30 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Wed, Jul 24, 2024 at 10:24:46PM CEST, pabeni@redhat.com wrote:
>We have a plurality of shaping-related drivers API, but none flexible
>enough to meet existing demand from vendors[1].
>
>This series introduces new device APIs to configure in a flexible way
>TX shaping H/W offload. The new functionality is exposed via a newly
From what I understand, and please correct me if I'm wrong, this
patchset is about HW shaper configuration. Basically it provides new UAPI
to configure the HW shaper. So Why you say "offload"? I don't see
anything being offloaded here.
Also, from the previous discussions, I gained impression that the goal
of this work is to replace multiple driver apis for the shaper and
consolidate it under new one. I don't see anything like this in this
patchset. Do you plan it as a follow-up? Do you have RFC for that step
as well?
>defined generic netlink interface and includes introspection
>capabilities. Some self-tests are included, on top of a dummy
>netdevsim implementation, and a basic implementation for the iavf
>driver.
>
>Some usage examples:
>
>* Configure shaping on a given queue:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do set --json '{"ifindex":'$IFINDEX',
> "shaper": {"handle":
> {"scope": "queue", "id":'$QUEUEID' },
> "bw-max": 2000000 }}'
>
>* Container B/W sharing
>
>The orchestration infrastructure wants to group the
>container-related queues under a RR scheduling and limit the aggregate
>bandwidth:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do group --json '{"ifindex":'$IFINDEX',
> "inputs": [
> {"handle": {"scope": "queue", "id":'$QID1' },
> "weight": '$W1'},
> {"handle": {"scope": "queue", "id":'$QID2' },
> "weight": '$W2'}],
> {"handle": {"scope": "queue", "id":'$QID3' },
> "weight": '$W3'}],
> "output": { "handle": {"scope":"netdev"},
> "output": { "handle": {"scope":"netdev"},
> "bw-max": 10000000}}'
>{'handle': {'id': 0, 'scope': 'netdev'}}
>
>* Delegation
>
>A container wants to set a B/W limit on 2 of its own queues:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do group --json '{"ifindex":'$IFINDEX',
> "inputs": [
> {"handle": {"scope": "queue", "id":'$QID1' },
> "weight": '$W1'},
> {"handle": {"scope": "queue", "id":'$QID2' },
> "weight": '$W2'}],
> "output": { "handle": {"scope":"detached"},
> "bw-max": 5000000}}'
>{'handle': {'id': 0, 'scope': 'detached'}}
>
>* Cleanup:
>
>Deleting a single queue shaper:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do delete --json \
> '{"ifindex":'$IFINDEX',
> "handle": {"scope": "queue", "id":'$QID1' }}'
>
>Deleting the last shaper under a group deletes the group, too:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do delete --json \
> '{"ifindex":'$IFINDEX',
> "handle": {"scope": "queue", "id":'$QID2' }}'
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do get --json '{"ifindex":'$IF',
> "handle": { "scope": "detached", "id": 0}}'
>Netlink error: Invalid argument
>nl_len = 80 (64) nl_flags = 0x300 nl_type = 2
> error: -22
> extack: {'msg': "Can't find shaper for handle 10000000"}
>
>Changes from RFC v1:
> - set() and delete() ops operate on a single shaper
> - added group() op to allow grouping and nesting
> - split the NL implementation into multiple patches to help reviewing
>
>RFC v1: https://lore.kernel.org/netdev/cover.1719518113.git.pabeni@redhat.com/
>
>[1] https://lore.kernel.org/netdev/20240405102313.GA310894@kernel.org/
>
>Paolo Abeni (7):
> netlink: spec: add shaper YAML spec
> net-shapers: implement NL get operation
> net-shapers: implement NL set and delete operations
> net-shapers: implement NL group operation
> netlink: spec: add shaper introspection support
> net: shaper: implement introspection support
> testing: net-drv: add basic shaper test
>
>Sudheer Mogilappagari (2):
> iavf: Add net_shaper_ops support
> iavf: add support to exchange qos capabilities
>
>Wenjun Wu (2):
> virtchnl: support queue rate limit and quanta size configuration
> ice: Support VF queue rate limit and quanta size configuration
>
> Documentation/netlink/specs/shaper.yaml | 337 ++++++
> Documentation/networking/kapi.rst | 3 +
> MAINTAINERS | 1 +
> drivers/net/Kconfig | 1 +
> drivers/net/ethernet/intel/Kconfig | 1 +
> drivers/net/ethernet/intel/iavf/iavf.h | 13 +
> drivers/net/ethernet/intel/iavf/iavf_main.c | 215 +++-
> drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 +
> .../net/ethernet/intel/iavf/iavf_virtchnl.c | 157 ++-
> drivers/net/ethernet/intel/ice/ice.h | 2 +
> drivers/net/ethernet/intel/ice/ice_base.c | 2 +
> drivers/net/ethernet/intel/ice/ice_common.c | 21 +
> .../net/ethernet/intel/ice/ice_hw_autogen.h | 8 +
> drivers/net/ethernet/intel/ice/ice_txrx.h | 1 +
> drivers/net/ethernet/intel/ice/ice_type.h | 1 +
> drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 +
> drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++
> drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 +
> .../intel/ice/ice_virtchnl_allowlist.c | 6 +
> drivers/net/netdevsim/netdev.c | 37 +
> include/linux/avf/virtchnl.h | 119 +++
> include/linux/netdevice.h | 17 +
> include/net/net_shaper.h | 169 +++
> include/uapi/linux/net_shaper.h | 91 ++
> net/Kconfig | 3 +
> net/Makefile | 1 +
> net/core/dev.c | 2 +
> net/core/dev.h | 6 +
> net/shaper/Makefile | 9 +
> net/shaper/shaper.c | 962 ++++++++++++++++++
> net/shaper/shaper_nl_gen.c | 142 +++
> net/shaper/shaper_nl_gen.h | 30 +
> tools/testing/selftests/drivers/net/Makefile | 1 +
> tools/testing/selftests/drivers/net/shaper.py | 267 +++++
> .../testing/selftests/net/lib/py/__init__.py | 1 +
> tools/testing/selftests/net/lib/py/ynl.py | 5 +
> 36 files changed, 2983 insertions(+), 2 deletions(-)
> create mode 100644 Documentation/netlink/specs/shaper.yaml
> create mode 100644 include/net/net_shaper.h
> create mode 100644 include/uapi/linux/net_shaper.h
> create mode 100644 net/shaper/Makefile
> create mode 100644 net/shaper/shaper.c
> create mode 100644 net/shaper/shaper_nl_gen.c
> create mode 100644 net/shaper/shaper_nl_gen.h
> create mode 100755 tools/testing/selftests/drivers/net/shaper.py
>
>--
>2.45.2
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API
2024-07-29 6:30 ` [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Jiri Pirko
@ 2024-07-29 14:42 ` Paolo Abeni
2024-07-29 15:13 ` Jiri Pirko
0 siblings, 1 reply; 19+ messages in thread
From: Paolo Abeni @ 2024-07-29 14:42 UTC (permalink / raw)
To: Jiri Pirko
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 7/29/24 08:30, Jiri Pirko wrote:
> From what I understand, and please correct me if I'm wrong, this
> patchset is about HW shaper configuration. Basically it provides new UAPI
> to configure the HW shaper. So Why you say "offload"? I don't see
> anything being offloaded here.
The offload part comes from the initial, very old tentative solution. I
guess we can change the title to "net: introduce TX H/W shaping API"
> Also, from the previous discussions, I gained impression that the goal
> of this work is to replace multiple driver apis for the shaper and
> consolidate it under new one. I don't see anything like this in this
> patchset. Do you plan it as a follow-up? Do you have RFC for that step
> as well?
The general idea is, with time, to leverage this API to replace others
H/W shaping related in-kernel interfaces.
At least ndo_set_tx_maxrate() should be quite straight-forward, after
that the relevant device drivers have implemented (very limited) support
for this API.
The latter will need some effort from the drivers' owners.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API
2024-07-29 14:42 ` Paolo Abeni
@ 2024-07-29 15:13 ` Jiri Pirko
2024-07-30 16:01 ` Paolo Abeni
0 siblings, 1 reply; 19+ messages in thread
From: Jiri Pirko @ 2024-07-29 15:13 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Mon, Jul 29, 2024 at 04:42:19PM CEST, pabeni@redhat.com wrote:
>On 7/29/24 08:30, Jiri Pirko wrote:
>> From what I understand, and please correct me if I'm wrong, this
>> patchset is about HW shaper configuration. Basically it provides new UAPI
>> to configure the HW shaper. So Why you say "offload"? I don't see
>> anything being offloaded here.
>
>The offload part comes from the initial, very old tentative solution. I guess
>we can change the title to "net: introduce TX H/W shaping API"
>
>> Also, from the previous discussions, I gained impression that the goal
>> of this work is to replace multiple driver apis for the shaper and
>> consolidate it under new one. I don't see anything like this in this
>> patchset. Do you plan it as a follow-up? Do you have RFC for that step
>> as well?
>
>The general idea is, with time, to leverage this API to replace others H/W
>shaping related in-kernel interfaces.
>
>At least ndo_set_tx_maxrate() should be quite straight-forward, after that
>the relevant device drivers have implemented (very limited) support for this
>API.
Could you try to draft at least one example per each user? I mean, this
is likely to be the tricky part of this work, would be great to make
that click from very beginning.
>
>The latter will need some effort from the drivers' owners.
Let me know what you need exactly. Will try to do my best to help.
Thanks!
>
>Thanks,
>
>Paolo
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API
2024-07-29 15:13 ` Jiri Pirko
@ 2024-07-30 16:01 ` Paolo Abeni
0 siblings, 0 replies; 19+ messages in thread
From: Paolo Abeni @ 2024-07-30 16:01 UTC (permalink / raw)
To: Jiri Pirko
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 7/29/24 17:13, Jiri Pirko wrote:
> Mon, Jul 29, 2024 at 04:42:19PM CEST, pabeni@redhat.com wrote:
>> The general idea is, with time, to leverage this API to replace others H/W
>> shaping related in-kernel interfaces.
>>
>> At least ndo_set_tx_maxrate() should be quite straight-forward, after that
>> the relevant device drivers have implemented (very limited) support for this
>> API.
>
> Could you try to draft at least one example per each user? I mean, this
> is likely to be the tricky part of this work, would be great to make
> that click from very beginning.
I think we need to clarify that we are not going to replace all the
existing in-kernel interfaces that somewhat configure shapers on the
network devices.
Trying to achieve the above would be a (not so) nice way to block this
effort forever.
ndo_set_tx_maxrate() is IMHO a good, feasible example. Others could be
ieee_setmaxrate() or ndo_set_vf_rate() - ignoring the “obsoleted” argument.
>> The latter will need some effort from the drivers' owners.
>
> Let me know what you need exactly. Will try to do my best to help.
Something like what is implemented in this series for the iavf driver
would suffice.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH RFC v2 10/11] iavf: Add net_shaper_ops support
2024-07-24 20:24 ` [PATCH RFC v2 10/11] iavf: Add net_shaper_ops support Paolo Abeni
@ 2024-08-02 13:44 ` Jiri Pirko
0 siblings, 0 replies; 19+ messages in thread
From: Jiri Pirko @ 2024-08-02 13:44 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Wed, Jul 24, 2024 at 10:24:56PM CEST, pabeni@redhat.com wrote:
>From: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
[...]
>+static int
>+iavf_shaper_set(struct net_device *dev,
>+ const struct net_shaper_info *shaper,
>+ struct netlink_ext_ack *extack)
>+{
>+ struct iavf_adapter *adapter = netdev_priv(dev);
>+ bool need_cfg_update = false;
>+ enum net_shaper_scope scope;
>+ int id, ret = 0;
>+
>+ ret = iavf_verify_shaper_info(dev, shaper, extack);
>+ if (ret)
>+ return ret;
>+
>+ scope = net_shaper_handle_scope(shaper->handle);
>+ id = net_shaper_handle_id(shaper->handle);
>+
>+ if (scope == NET_SHAPER_SCOPE_QUEUE) {
>+ struct iavf_ring *tx_ring = &adapter->tx_rings[id];
>+
>+ tx_ring->q_shaper.bw_min = div_u64(shaper->bw_min, 1000);
>+ tx_ring->q_shaper.bw_max = div_u64(shaper->bw_max, 1000);
Isn't this more or less what ndo_set_tx_maxrate() does? I mean, the only
difference I see is ability to set min/max. Do I read this correctly?
If yes, could we have the ndo_set_tx_maxrate() consolidation included in
this patchset (or at least to post follow-up RFC)? I mean, without that,
there are 2 uapis to set queue rate limit for different drivers
(net-shaper for iavf, sysfs tx_maxrate for the rest), which looks messy.
>+ tx_ring->q_shaper_update = true;
>+ need_cfg_update = true;
>+ }
>+
>+ if (need_cfg_update)
>+ adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
>+
>+ return 0;
>+}
[...]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH RFC v2 03/11] net-shapers: implement NL set and delete operations
2024-07-24 20:24 ` [PATCH RFC v2 03/11] net-shapers: implement NL set and delete operations Paolo Abeni
@ 2024-08-02 16:01 ` Jiri Pirko
0 siblings, 0 replies; 19+ messages in thread
From: Jiri Pirko @ 2024-08-02 16:01 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Wed, Jul 24, 2024 at 10:24:49PM CEST, pabeni@redhat.com wrote:
>Both NL operations directly map on the homonymous device shaper
>callbacks and update accordingly the shapers cache.
>Implement the cache modification helpers to additionally deal with
>DETACHED scope shaper. That will be needed by the group() operation
>implemented in the next patch.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
> net/shaper/shaper.c | 323 +++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 321 insertions(+), 2 deletions(-)
>
>diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
>index 93dd491ac7c2..7802c9ba6d9c 100644
>--- a/net/shaper/shaper.c
>+++ b/net/shaper/shaper.c
>@@ -19,6 +19,35 @@ struct net_shaper_nl_ctx {
> u32 start_handle;
> };
>
>+static u32 default_parent(u32 handle)
>+{
>+ enum net_shaper_scope parent, scope = net_shaper_handle_scope(handle);
In the rest of the code you call this "pscope", could you rename?
>+
>+ switch (scope) {
>+ case NET_SHAPER_SCOPE_PORT:
>+ case NET_SHAPER_SCOPE_UNSPEC:
>+ parent = NET_SHAPER_SCOPE_UNSPEC;
>+ break;
>+
>+ case NET_SHAPER_SCOPE_QUEUE:
>+ case NET_SHAPER_SCOPE_DETACHED:
Why default parent of detached is netdev? Detached means "node in tree"
if IIUC. Can't port be parent?
>+ parent = NET_SHAPER_SCOPE_NETDEV;
>+ break;
>+
>+ case NET_SHAPER_SCOPE_NETDEV:
>+ case NET_SHAPER_SCOPE_VF:
>+ parent = NET_SHAPER_SCOPE_PORT;
>+ break;
>+ }
>+
>+ return net_shaper_make_handle(parent, 0);
>+}
>+
>+static bool is_detached(u32 handle)
>+{
>+ return net_shaper_handle_scope(handle) == NET_SHAPER_SCOPE_DETACHED;
>+}
>+
> static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
> const struct genl_info *info)
> {
>@@ -117,6 +146,115 @@ static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle)
> return xa ? xa_load(xa, handle) : NULL;
> }
>
>+/* allocate on demand the per device shaper's cache */
>+static struct xarray *__sc_init(struct net_device *dev,
>+ struct netlink_ext_ack *extack)
>+{
>+ if (!dev->net_shaper_data) {
>+ dev->net_shaper_data = kmalloc(sizeof(*dev->net_shaper_data),
>+ GFP_KERNEL);
>+ if (!dev->net_shaper_data) {
>+ NL_SET_ERR_MSG(extack, "Can't allocate memory for shaper data");
>+ return NULL;
>+ }
>+
>+ xa_init(&dev->net_shaper_data->shapers);
>+ idr_init(&dev->net_shaper_data->detached_ids);
>+ }
>+ return &dev->net_shaper_data->shapers;
>+}
>+
>+/* prepare the cache to actually insert the given shaper, doing
>+ * in advance the needed allocations
In comments (all), would it make sense to start the sentence with
capital letter and end it with "."?
>+ */
>+static int sc_prepare_insert(struct net_device *dev, u32 *handle,
>+ struct netlink_ext_ack *extack)
>+{
>+ enum net_shaper_scope scope = net_shaper_handle_scope(*handle);
>+ struct xarray *xa = __sc_init(dev, extack);
>+ struct net_shaper_info *prev, *cur;
>+ bool id_allocated = false;
>+ int ret, id;
>+
>+ if (!xa)
>+ return -ENOMEM;
>+
>+ cur = xa_load(xa, *handle);
>+ if (cur)
>+ return 0;
>+
>+ /* allocated a new id, if needed */
s/allocated/allocate/ perhaps ?
>+ if (scope == NET_SHAPER_SCOPE_DETACHED &&
>+ net_shaper_handle_id(*handle) == NET_SHAPER_ID_UNSPEC) {
>+ xa_lock(xa);
>+ id = idr_alloc(&dev->net_shaper_data->detached_ids, NULL,
>+ 0, NET_SHAPER_ID_UNSPEC, GFP_ATOMIC);
>+ xa_unlock(xa);
>+
>+ if (id < 0) {
>+ NL_SET_ERR_MSG(extack, "Can't allocate new id for detached shaper");
>+ return id;
>+ }
>+
>+ *handle = net_shaper_make_handle(scope, id);
>+ id_allocated = true;
>+ }
>+
>+ cur = kmalloc(sizeof(*cur), GFP_KERNEL | __GFP_ZERO);
>+ if (!cur) {
>+ NL_SET_ERR_MSG(extack, "Can't allocate memory for cached shaper");
>+ ret = -ENOMEM;
>+ goto free_id;
>+ }
>+
>+ /* mark 'tentative' shaper inside the cache */
>+ xa_lock(xa);
>+ prev = __xa_store(xa, *handle, cur, GFP_KERNEL);
>+ __xa_set_mark(xa, *handle, XA_MARK_0);
I think it is nice to actually have a define for the mark that indicates
what a certain mark represents.
Also, this patch introduces a bug in net_shaper_nl_get_dumpit()/net_shaper_nl_get_doit()
which now work with uncommitted shapers. You have to check the mark.
>+ xa_unlock(xa);
>+ if (xa_err(prev)) {
>+ NL_SET_ERR_MSG(extack, "Can't insert shaper into cache");
>+ kfree(cur);
>+ ret = xa_err(prev);
>+ goto free_id;
>+ }
>+ return 0;
>+
>+free_id:
>+ if (id_allocated) {
>+ xa_lock(xa);
May be worth mentioning you protect detached_ids with the xa lock.
>+ idr_remove(&dev->net_shaper_data->detached_ids,
>+ net_shaper_handle_id(*handle));
>+ xa_unlock(xa);
>+ }
>+ return ret;
>+}
>+
>+/* commit the tentative insert with the actual values.
>+ * Must be called only after a successful sc_prepare_insert()
>+ */
>+static void sc_commit(struct net_device *dev, int nr_shapers,
>+ const struct net_shaper_info *shapers)
>+{
>+ struct xarray *xa = __sc_container(dev);
>+ struct net_shaper_info *cur;
>+ int i;
>+
>+ xa_lock(xa);
>+ for (i = 0; i < nr_shapers; ++i) {
>+ cur = xa_load(xa, shapers[i].handle);
>+ if (WARN_ON_ONCE(!cur))
>+ continue;
>+
>+ /* successful update: drop the tentative mark
>+ * and update the cache
>+ */
>+ __xa_clear_mark(xa, shapers[i].handle, XA_MARK_0);
>+ *cur = shapers[i];
>+ }
>+ xa_unlock(xa);
>+}
>+
> static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
> u32 *handle)
> {
>@@ -154,6 +292,68 @@ static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
> return 0;
> }
>
>+static int __parse_shaper(struct net_device *dev, struct nlattr **tb,
In general, "__" prefix indicates that reader should be careful for any
reason, locking for example. Here, It's a simple helper, isn't it? Why
"__" make sense here?
>+ const struct genl_info *info,
>+ struct net_shaper_info *shaper)
>+{
>+ struct net_shaper_info *old;
>+ int ret;
>+
>+ /* the shaper handle is the only mandatory attribute */
>+ if (NL_REQ_ATTR_CHECK(info->extack, NULL, tb, NET_SHAPER_A_HANDLE))
>+ return -EINVAL;
>+
>+ ret = parse_handle(tb[NET_SHAPER_A_HANDLE], info, &shaper->handle);
>+ if (ret)
>+ return ret;
>+
>+ /* fetch existing data, if any, so that user provide info will
>+ * incrementally update the existing shaper configuration
>+ */
>+ old = sc_lookup(dev, shaper->handle);
>+ if (old)
>+ *shaper = *old;
>+ else
>+ shaper->parent = default_parent(shaper->handle);
>+
>+ if (tb[NET_SHAPER_A_METRIC])
>+ shaper->metric = nla_get_u32(tb[NET_SHAPER_A_METRIC]);
>+
>+ if (tb[NET_SHAPER_A_BW_MIN])
>+ shaper->bw_min = nla_get_uint(tb[NET_SHAPER_A_BW_MIN]);
>+
>+ if (tb[NET_SHAPER_A_BW_MAX])
>+ shaper->bw_max = nla_get_uint(tb[NET_SHAPER_A_BW_MAX]);
>+
>+ if (tb[NET_SHAPER_A_BURST])
>+ shaper->burst = nla_get_uint(tb[NET_SHAPER_A_BURST]);
>+
>+ if (tb[NET_SHAPER_A_PRIORITY])
>+ shaper->priority = nla_get_u32(tb[NET_SHAPER_A_PRIORITY]);
>+
>+ if (tb[NET_SHAPER_A_WEIGHT])
>+ shaper->weight = nla_get_u32(tb[NET_SHAPER_A_WEIGHT]);
>+ return 0;
>+}
>+
>+/* fetch the cached shaper info and update them with the user-provided
>+ * attributes
>+ */
>+static int parse_shaper(struct net_device *dev, const struct nlattr *attr,
>+ const struct genl_info *info,
>+ struct net_shaper_info *shaper)
>+{
>+ struct nlattr *tb[NET_SHAPER_A_WEIGHT + 1];
>+ int ret;
>+
>+ ret = nla_parse_nested(tb, NET_SHAPER_A_WEIGHT, attr,
>+ net_shaper_ns_info_nl_policy, info->extack);
>+ if (ret < 0)
>+ return ret;
>+
>+ return __parse_shaper(dev, tb, info, shaper);
>+}
>+
> int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
> {
> struct net_shaper_info *shaper;
>@@ -239,14 +439,133 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb,
> return ret;
> }
>
>+/* Update the H/W and on success update the local cache, too */
>+static int net_shaper_set(struct net_device *dev,
>+ const struct net_shaper_info *shaper,
>+ struct netlink_ext_ack *extack)
>+{
>+ enum net_shaper_scope scope;
>+ u32 handle = shaper->handle;
>+ int ret;
>+
>+ scope = net_shaper_handle_scope(handle);
>+ if (scope == NET_SHAPER_SCOPE_PORT ||
NET_SHAPER_SCOPE_PORT is not really used in this set. Why do you have
it? Same applies tol NET_SHAPER_SCOPE_NETDEV.
>+ scope == NET_SHAPER_SCOPE_UNSPEC) {
>+ NL_SET_ERR_MSG_FMT(extack, "Can't set shaper %x with scope %d",
>+ handle, scope);
>+ return -EINVAL;
>+ }
>+ if (scope == NET_SHAPER_SCOPE_DETACHED && !sc_lookup(dev, handle)) {
>+ NL_SET_ERR_MSG_FMT(extack, "Shaper %x with detached scope does not exist",
>+ handle);
>+ return -EINVAL;
>+ }
>+
>+ ret = sc_prepare_insert(dev, &handle, extack);
>+ if (ret)
>+ return ret;
>+
>+ ret = dev->netdev_ops->net_shaper_ops->set(dev, shaper, extack);
>+ sc_commit(dev, 1, shaper);
Missing rollback?
>+ return ret;
>+}
>+
> int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
> {
>- return -EOPNOTSUPP;
>+ struct net_shaper_info shaper;
>+ struct net_device *dev;
>+ struct nlattr *attr;
>+ int ret;
>+
>+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_SHAPER))
>+ return -EINVAL;
>+
>+ ret = fetch_dev(info, &dev);
>+ if (ret)
>+ return ret;
>+
>+ attr = info->attrs[NET_SHAPER_A_SHAPER];
>+ ret = parse_shaper(dev, attr, info, &shaper);
>+ if (ret)
>+ goto put;
>+
>+ ret = net_shaper_set(dev, &shaper, info->extack);
Hmm, if I don't miss anything, this is lockless (.parallel_ops==true).
What's stopping user from performing the set action in parallel on a
single shaper? The locking scheme seems to be missing. Another thing to
describe in documentation, perhaps.
>+
>+put:
>+ dev_put(dev);
>+ return ret;
>+}
>+
>+static int net_shaper_delete(struct net_device *dev, u32 handle,
>+ struct netlink_ext_ack *extack)
>+{
>+ struct net_shaper_info *parent, *shaper = sc_lookup(dev, handle);
>+ struct xarray *xa = __sc_container(dev);
>+ enum net_shaper_scope pscope;
>+ u32 parent_handle;
>+ int ret;
>+
>+ if (!xa || !shaper) {
>+ NL_SET_ERR_MSG_FMT(extack, "Shaper %x not found", handle);
>+ return -EINVAL;
>+ }
>+
>+ if (is_detached(handle) && shaper->children > 0) {
>+ NL_SET_ERR_MSG_FMT(extack, "Can't delete detached shaper %d with %d child nodes",
>+ handle, shaper->children);
>+ return -EINVAL;
>+ }
>+
>+ while (shaper) {
>+ parent_handle = shaper->parent;
>+ pscope = net_shaper_handle_scope(parent_handle);
>+
>+ ret = dev->netdev_ops->net_shaper_ops->delete(dev, handle, extack);
>+ if (ret < 0)
>+ return ret;
>+
>+ xa_lock(xa);
>+ __xa_erase(xa, handle);
Hmm, did you think about free of dev->net_shaper_data in case this is
the last item? I think it would be correct to do that.
>+ if (is_detached(handle))
>+ idr_remove(&dev->net_shaper_data->detached_ids,
>+ net_shaper_handle_id(handle));
>+ xa_unlock(xa);
>+ kfree(shaper);
>+ shaper = NULL;
>+
>+ if (pscope == NET_SHAPER_SCOPE_DETACHED) {
>+ parent = sc_lookup(dev, parent_handle);
>+ if (parent && !--parent->children) {
>+ shaper = parent;
>+ handle = parent_handle;
>+ }
>+ }
>+ }
>+ return 0;
> }
>
> int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
> {
>- return -EOPNOTSUPP;
>+ struct net_device *dev;
>+ u32 handle;
>+ int ret;
>+
>+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
>+ return -EINVAL;
>+
>+ ret = fetch_dev(info, &dev);
>+ if (ret)
>+ return ret;
>+
>+ ret = parse_handle(info->attrs[NET_SHAPER_A_HANDLE], info, &handle);
>+ if (ret)
>+ goto put;
>+
>+ ret = net_shaper_delete(dev, handle, info->extack);
>+
>+put:
>+ dev_put(dev);
>+ return ret;
> }
>
> int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
>--
>2.45.2
>
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2024-08-02 16:01 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-24 20:24 [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 01/11] netlink: spec: add shaper YAML spec Paolo Abeni
2024-07-25 11:42 ` Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 02/11] net-shapers: implement NL get operation Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 03/11] net-shapers: implement NL set and delete operations Paolo Abeni
2024-08-02 16:01 ` Jiri Pirko
2024-07-24 20:24 ` [PATCH RFC v2 04/11] net-shapers: implement NL group operation Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 05/11] netlink: spec: add shaper introspection support Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 06/11] net: shaper: implement " Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 07/11] testing: net-drv: add basic shaper test Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 08/11] virtchnl: support queue rate limit and quanta size configuration Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 09/11] ice: Support VF " Paolo Abeni
2024-07-24 20:24 ` [PATCH RFC v2 10/11] iavf: Add net_shaper_ops support Paolo Abeni
2024-08-02 13:44 ` Jiri Pirko
2024-07-24 20:24 ` [PATCH RFC v2 11/11] iavf: add support to exchange qos capabilities Paolo Abeni
2024-07-29 6:30 ` [PATCH RFC v2 00/11] net: introduce TX shaping H/W offload API Jiri Pirko
2024-07-29 14:42 ` Paolo Abeni
2024-07-29 15:13 ` Jiri Pirko
2024-07-30 16:01 ` Paolo Abeni
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).