* [PATCH v3 00/12] net: introduce TX H/W shaping API
@ 2024-07-30 20:39 Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 01/12] tools: ynl: lift an assumption about spec file name Paolo Abeni
` (12 more replies)
0 siblings, 13 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
We have a plurality of shaping-related drivers API, but none flexible
enough to meet existing demand from vendors[1].
This series introduces new device APIs to configure in a flexible way
TX H/W shaping. The new functionalities are exposed via a newly
defined generic netlink interface and include introspection
capabilities. Some self-tests are included, on top of a dummy
netdevsim implementation, and a basic implementation for the iavf
driver.
Some usage examples:
* Configure shaping on a given queue:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do set --json '{"ifindex":'$IFINDEX',
"shaper": {"handle":
{"scope": "queue", "id":'$QUEUEID' },
"bw-max": 2000000 }}'
* Container B/W sharing
The orchestration infrastructure wants to group the
container-related queues under a RR scheduling and limit the aggregate
bandwidth:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do group --json '{"ifindex":'$IFINDEX',
"inputs": [
{"handle": {"scope": "queue", "id":'$QID1' },
"weight": '$W1'},
{"handle": {"scope": "queue", "id":'$QID2' },
"weight": '$W2'}],
{"handle": {"scope": "queue", "id":'$QID3' },
"weight": '$W3'}],
"output": { "handle": {"scope":"netdev"},
"output": { "handle": {"scope":"netdev"},
"bw-max": 10000000}}'
{'handle': {'id': 0, 'scope': 'netdev'}}
* Delegation
A container wants to set a B/W limit on 2 of its own queues:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do group --json '{"ifindex":'$IFINDEX',
"inputs": [
{"handle": {"scope": "queue", "id":'$QID1' },
"weight": '$W1'},
{"handle": {"scope": "queue", "id":'$QID2' },
"weight": '$W2'}],
"output": { "handle": {"scope":"detached"},
"bw-max": 5000000}}'
{'handle': {'id': 0, 'scope': 'detached'}}
* Cleanup:
Deleting a single queue shaper:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do delete --json \
'{"ifindex":'$IFINDEX',
"handle": {"scope": "queue", "id":'$QID1' }}'
deleting the last shaper under a group deletes the group, too:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do delete --json \
'{"ifindex":'$IFINDEX',
"handle": {"scope": "queue", "id":'$QID2' }}'
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do get --json '{"ifindex":'$IF',
"handle": { "scope": "detached", "id": 0}}'
Netlink error: Invalid argument
nl_len = 80 (64) nl_flags = 0x300 nl_type = 2
error: -22
extack: {'msg': "Can't find shaper for handle 10000000"}
Changes from RFC v2:
- added patch 1
- fixed deprecated API usage
RFC v2: https://lore.kernel.org/netdev/cover.1721851988.git.pabeni@redhat.com/
Changes from RFC v1:
- set() and delete() ops operate on a single shaper
- added group() op to allow grouping and nesting
- split the NL implementation into multiple patches to help reviewing
RFC v1: https://lore.kernel.org/netdev/cover.1719518113.git.pabeni@redhat.com/
[1] https://lore.kernel.org/netdev/20240405102313.GA310894@kernel.org/
Paolo Abeni (8):
tools: ynl: lift an assumption about spec file name
netlink: spec: add shaper YAML spec
net-shapers: implement NL get operation
net-shapers: implement NL set and delete operations
net-shapers: implement NL group operation
netlink: spec: add shaper introspection support
net: shaper: implement introspection support
testing: net-drv: add basic shaper test
Sudheer Mogilappagari (2):
iavf: Add net_shaper_ops support
iavf: add support to exchange qos capabilities
Wenjun Wu (2):
virtchnl: support queue rate limit and quanta size configuration
ice: Support VF queue rate limit and quanta size configuration
Documentation/netlink/specs/shaper.yaml | 337 ++++++
Documentation/networking/kapi.rst | 3 +
MAINTAINERS | 1 +
drivers/net/Kconfig | 1 +
drivers/net/ethernet/intel/Kconfig | 1 +
drivers/net/ethernet/intel/iavf/iavf.h | 13 +
drivers/net/ethernet/intel/iavf/iavf_main.c | 215 +++-
drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 +
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 157 ++-
drivers/net/ethernet/intel/ice/ice.h | 2 +
drivers/net/ethernet/intel/ice/ice_base.c | 2 +
drivers/net/ethernet/intel/ice/ice_common.c | 21 +
.../net/ethernet/intel/ice/ice_hw_autogen.h | 8 +
drivers/net/ethernet/intel/ice/ice_txrx.h | 1 +
drivers/net/ethernet/intel/ice/ice_type.h | 1 +
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 +
drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++
drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 +
.../intel/ice/ice_virtchnl_allowlist.c | 6 +
drivers/net/netdevsim/netdev.c | 37 +
include/linux/avf/virtchnl.h | 119 +++
include/linux/netdevice.h | 17 +
include/net/net_shaper.h | 169 +++
include/uapi/linux/net_shaper.h | 91 ++
net/Kconfig | 3 +
net/Makefile | 1 +
net/core/dev.c | 2 +
net/core/dev.h | 6 +
net/shaper/Makefile | 9 +
net/shaper/shaper.c | 963 ++++++++++++++++++
net/shaper/shaper_nl_gen.c | 142 +++
net/shaper/shaper_nl_gen.h | 30 +
tools/net/ynl/ynl-gen-c.py | 6 +-
tools/testing/selftests/drivers/net/Makefile | 1 +
tools/testing/selftests/drivers/net/shaper.py | 267 +++++
.../testing/selftests/net/lib/py/__init__.py | 1 +
tools/testing/selftests/net/lib/py/ynl.py | 5 +
37 files changed, 2988 insertions(+), 4 deletions(-)
create mode 100644 Documentation/netlink/specs/shaper.yaml
create mode 100644 include/net/net_shaper.h
create mode 100644 include/uapi/linux/net_shaper.h
create mode 100644 net/shaper/Makefile
create mode 100644 net/shaper/shaper.c
create mode 100644 net/shaper/shaper_nl_gen.c
create mode 100644 net/shaper/shaper_nl_gen.h
create mode 100755 tools/testing/selftests/drivers/net/shaper.py
--
2.45.2
^ permalink raw reply [flat|nested] 91+ messages in thread
* [PATCH v3 01/12] tools: ynl: lift an assumption about spec file name
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 02/12] netlink: spec: add shaper YAML spec Paolo Abeni
` (11 subsequent siblings)
12 siblings, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Currently the parsing code generator assumes that the yaml
specification file name and the main 'name' attribute carried
inside correspond, that is the field is the c-name representation
of the file basename.
The above assumption held true within the current tree, but will be
hopefully broken soon by the upcoming net shaper specification.
Additionally, it makes the field 'name' itself useless.
Lift the assumption, always computing the generated include file
name from the generated c file name.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
tools/net/ynl/ynl-gen-c.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tools/net/ynl/ynl-gen-c.py b/tools/net/ynl/ynl-gen-c.py
index 51529fabd517..717530bc9c52 100755
--- a/tools/net/ynl/ynl-gen-c.py
+++ b/tools/net/ynl/ynl-gen-c.py
@@ -2668,13 +2668,15 @@ def main():
cw.p('#define ' + hdr_prot)
cw.nl()
+ hdr_file=os.path.basename(args.out_file[:-2]) + ".h"
+
if args.mode == 'kernel':
cw.p('#include <net/netlink.h>')
cw.p('#include <net/genetlink.h>')
cw.nl()
if not args.header:
if args.out_file:
- cw.p(f'#include "{os.path.basename(args.out_file[:-2])}.h"')
+ cw.p(f'#include "{hdr_file}"')
cw.nl()
headers = ['uapi/' + parsed.uapi_header]
headers += parsed.kernel_family.get('headers', [])
@@ -2686,7 +2688,7 @@ def main():
if family_contains_bitfield32(parsed):
cw.p('#include <linux/netlink.h>')
else:
- cw.p(f'#include "{parsed.name}-user.h"')
+ cw.p(f'#include "{hdr_file}"')
cw.p('#include "ynl.h"')
headers = [parsed.uapi_header]
for definition in parsed['definitions']:
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 01/12] tools: ynl: lift an assumption about spec file name Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-07-31 21:13 ` Donald Hunter
` (4 more replies)
2024-07-30 20:39 ` [PATCH v3 03/12] net-shapers: implement NL get operation Paolo Abeni
` (10 subsequent siblings)
12 siblings, 5 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Define the user-space visible interface to query, configure and delete
network shapers via yaml definition.
Add dummy implementations for the relevant NL callbacks.
set() and delete() operations touch a single shaper creating/updating or
deleting it.
The group() operation creates a shaper's group, nesting multiple input
shapers under the specified output shaper.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
RFC v1 -> RFC v2:
- u64 -> uint
- net_shapers -> net-shapers
- documented all the attributes
- dropped [ admin-perm ] for get() op
- group op
- set/delete touch a single shaper
---
Documentation/netlink/specs/shaper.yaml | 262 ++++++++++++++++++++++++
MAINTAINERS | 1 +
include/uapi/linux/net_shaper.h | 74 +++++++
net/Kconfig | 3 +
net/Makefile | 1 +
net/shaper/Makefile | 9 +
net/shaper/shaper.c | 34 +++
net/shaper/shaper_nl_gen.c | 117 +++++++++++
net/shaper/shaper_nl_gen.h | 27 +++
9 files changed, 528 insertions(+)
create mode 100644 Documentation/netlink/specs/shaper.yaml
create mode 100644 include/uapi/linux/net_shaper.h
create mode 100644 net/shaper/Makefile
create mode 100644 net/shaper/shaper.c
create mode 100644 net/shaper/shaper_nl_gen.c
create mode 100644 net/shaper/shaper_nl_gen.h
diff --git a/Documentation/netlink/specs/shaper.yaml b/Documentation/netlink/specs/shaper.yaml
new file mode 100644
index 000000000000..7327f5596fdb
--- /dev/null
+++ b/Documentation/netlink/specs/shaper.yaml
@@ -0,0 +1,262 @@
+# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
+
+name: net-shaper
+
+doc: Network device HW Rate Limiting offload
+
+definitions:
+ -
+ type: enum
+ name: scope
+ doc: the different scopes where a shaper can be attached
+ render-max: true
+ entries:
+ - name: unspec
+ doc: The scope is not specified
+ -
+ name: port
+ doc: The root for the whole H/W
+ -
+ name: netdev
+ doc: The main shaper for the given network device.
+ -
+ name: queue
+ doc: The shaper is attached to the given device queue.
+ -
+ name: detached
+ doc: |
+ The shaper is not attached to any user-visible network
+ device component and allows nesting and grouping of
+ queues or others detached shapers.
+ -
+ type: enum
+ name: metric
+ doc: different metric each shaper can support
+ entries:
+ -
+ name: bps
+ doc: Shaper operates on a bits per second basis
+ -
+ name: pps
+ doc: Shaper operates on a packets per second basis
+
+attribute-sets:
+ -
+ name: net-shaper
+ attributes:
+ -
+ name: ifindex
+ type: u32
+ doc: Interface index owing the specified shaper[s]
+ -
+ name: handle
+ type: nest
+ nested-attributes: handle
+ doc: Unique identifier for the given shaper
+ -
+ name: metric
+ type: u32
+ enum: metric
+ doc: Metric used by the given shaper for bw-min, bw-max and burst
+ -
+ name: bw-min
+ type: uint
+ doc: Minimum guaranteed B/W for the given shaper
+ -
+ name: bw-max
+ type: uint
+ doc: Shaping B/W for the given shaper or 0 when unlimited
+ -
+ name: burst
+ type: uint
+ doc: Maximum burst-size for bw-min and bw-max
+ -
+ name: priority
+ type: u32
+ doc: Scheduling priority for the given shaper
+ -
+ name: weight
+ type: u32
+ doc: |
+ Weighted round robin weight for given shaper.
+ The scheduling is applied to all the sibling
+ shapers with the same priority
+ -
+ name: scope
+ type: u32
+ enum: scope
+ doc: The given handle scope
+ -
+ name: id
+ type: u32
+ doc: |
+ The given handle id. The id semantic depends on the actual
+ scope, e.g. for 'queue' scope it's the queue id, for
+ 'detached' scope it's the shaper group identifier.
+ -
+ name: parent
+ type: nest
+ nested-attributes: handle
+ doc: |
+ Identifier for the parent of the affected shaper,
+ The parent handle value is implied by the shaper handle itself,
+ except for the output shaper in the 'group' operation.
+ -
+ name: inputs
+ type: nest
+ multi-attr: true
+ nested-attributes: ns-info
+ doc: |
+ Describes a set of inputs shapers for a @group operation
+ -
+ name: output
+ type: nest
+ nested-attributes: ns-output-info
+ doc: |
+ Describes the output shaper for a @group operation
+ Differently from @inputs and @shaper allow specifying
+ the shaper parent handle, too.
+
+ -
+ name: shaper
+ type: nest
+ nested-attributes: ns-info
+ doc: |
+ Describes a single shaper for a @set operation
+ -
+ name: handle
+ subset-of: net-shaper
+ attributes:
+ -
+ name: scope
+ -
+ name: id
+ -
+ name: ns-info
+ subset-of: net-shaper
+ attributes:
+ -
+ name: handle
+ -
+ name: metric
+ -
+ name: bw-min
+ -
+ name: bw-max
+ -
+ name: burst
+ -
+ name: priority
+ -
+ name: weight
+ -
+ name: ns-output-info
+ subset-of: net-shaper
+ attributes:
+ -
+ name: parent
+ -
+ name: handle
+ -
+ name: metric
+ -
+ name: bw-min
+ -
+ name: bw-max
+ -
+ name: burst
+ -
+ name: priority
+ -
+ name: weight
+
+operations:
+ list:
+ -
+ name: get
+ doc: |
+ Get / Dump information about a/all the shaper for a given device
+ attribute-set: net-shaper
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - handle
+ reply:
+ attributes: &ns-attrs
+ - parent
+ - handle
+ - metric
+ - bw-min
+ - bw-max
+ - burst
+ - priority
+ - weight
+
+ dump:
+ request:
+ attributes:
+ - ifindex
+ reply:
+ attributes: *ns-attrs
+ -
+ name: set
+ doc: |
+ Create or configures the specified shaper.
+ On failures the extack is set accordingly.
+ Can't create @detached scope shaper, use
+ the @group operation instead.
+ attribute-set: net-shaper
+ flags: [ admin-perm ]
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - shaper
+
+ -
+ name: delete
+ doc: |
+ Clear (remove) the specified shaper. If after the removal
+ the parent shaper has no more children and the parent
+ shaper scope is @detached, even the parent is deleted,
+ recursively.
+ On failures the extack is set accordingly.
+ attribute-set: net-shaper
+ flags: [ admin-perm ]
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - handle
+
+ -
+ name: group
+ doc: |
+ Group the specified input shapers under the specified
+ output shaper, eventually creating the latter, if needed.
+ Input shapers scope must be either @queue or @detached.
+ Output shaper scope must be either @detached or @netdev.
+ When using an output @detached scope shaper, if the
+ @handle @id is not specified, a new shaper of such scope
+ is created and, otherwise the specified output shaper
+ must be already existing.
+ The operation is atomic, on failures the extack is set
+ accordingly and no change is applied to the device
+ shaping configuration, otherwise the output shaper
+ handle is provided as reply.
+ attribute-set: net-shaper
+ flags: [ admin-perm ]
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - inputs
+ - output
+ reply:
+ attributes:
+ - handle
diff --git a/MAINTAINERS b/MAINTAINERS
index c0a3d9e93689..a1136f4dfa24 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15881,6 +15881,7 @@ F: include/linux/inetdevice.h
F: include/linux/netdevice.h
F: include/uapi/linux/cn_proc.h
F: include/uapi/linux/if_*
+F: include/uapi/linux/net_shaper.h
F: include/uapi/linux/netdevice.h
X: drivers/net/wireless/
diff --git a/include/uapi/linux/net_shaper.h b/include/uapi/linux/net_shaper.h
new file mode 100644
index 000000000000..ab3d4eb0e1ab
--- /dev/null
+++ b/include/uapi/linux/net_shaper.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/shaper.yaml */
+/* YNL-GEN uapi header */
+
+#ifndef _UAPI_LINUX_NET_SHAPER_H
+#define _UAPI_LINUX_NET_SHAPER_H
+
+#define NET_SHAPER_FAMILY_NAME "net-shaper"
+#define NET_SHAPER_FAMILY_VERSION 1
+
+/**
+ * enum net_shaper_scope - the different scopes where a shaper can be attached
+ * @NET_SHAPER_SCOPE_UNSPEC: The scope is not specified
+ * @NET_SHAPER_SCOPE_PORT: The root for the whole H/W
+ * @NET_SHAPER_SCOPE_NETDEV: The main shaper for the given network device.
+ * @NET_SHAPER_SCOPE_QUEUE: The shaper is attached to the given device queue.
+ * @NET_SHAPER_SCOPE_DETACHED: The shaper is not attached to any user-visible
+ * network device component and allows nesting and grouping of queues or
+ * others detached shapers.
+ */
+enum net_shaper_scope {
+ NET_SHAPER_SCOPE_UNSPEC,
+ NET_SHAPER_SCOPE_PORT,
+ NET_SHAPER_SCOPE_NETDEV,
+ NET_SHAPER_SCOPE_QUEUE,
+ NET_SHAPER_SCOPE_DETACHED,
+
+ /* private: */
+ __NET_SHAPER_SCOPE_MAX,
+ NET_SHAPER_SCOPE_MAX = (__NET_SHAPER_SCOPE_MAX - 1)
+};
+
+/**
+ * enum net_shaper_metric - different metric each shaper can support
+ * @NET_SHAPER_METRIC_BPS: Shaper operates on a bits per second basis
+ * @NET_SHAPER_METRIC_PPS: Shaper operates on a packets per second basis
+ */
+enum net_shaper_metric {
+ NET_SHAPER_METRIC_BPS,
+ NET_SHAPER_METRIC_PPS,
+};
+
+enum {
+ NET_SHAPER_A_IFINDEX = 1,
+ NET_SHAPER_A_HANDLE,
+ NET_SHAPER_A_METRIC,
+ NET_SHAPER_A_BW_MIN,
+ NET_SHAPER_A_BW_MAX,
+ NET_SHAPER_A_BURST,
+ NET_SHAPER_A_PRIORITY,
+ NET_SHAPER_A_WEIGHT,
+ NET_SHAPER_A_SCOPE,
+ NET_SHAPER_A_ID,
+ NET_SHAPER_A_PARENT,
+ NET_SHAPER_A_INPUTS,
+ NET_SHAPER_A_OUTPUT,
+ NET_SHAPER_A_SHAPER,
+
+ __NET_SHAPER_A_MAX,
+ NET_SHAPER_A_MAX = (__NET_SHAPER_A_MAX - 1)
+};
+
+enum {
+ NET_SHAPER_CMD_GET = 1,
+ NET_SHAPER_CMD_SET,
+ NET_SHAPER_CMD_DELETE,
+ NET_SHAPER_CMD_GROUP,
+
+ __NET_SHAPER_CMD_MAX,
+ NET_SHAPER_CMD_MAX = (__NET_SHAPER_CMD_MAX - 1)
+};
+
+#endif /* _UAPI_LINUX_NET_SHAPER_H */
diff --git a/net/Kconfig b/net/Kconfig
index d27d0deac0bf..31fccfed04f7 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -66,6 +66,9 @@ config SKB_DECRYPTED
config SKB_EXTENSIONS
bool
+config NET_SHAPER
+ bool
+
menu "Networking options"
source "net/packet/Kconfig"
diff --git a/net/Makefile b/net/Makefile
index 65bb8c72a35e..60ed5190eda8 100644
--- a/net/Makefile
+++ b/net/Makefile
@@ -79,3 +79,4 @@ obj-$(CONFIG_XDP_SOCKETS) += xdp/
obj-$(CONFIG_MPTCP) += mptcp/
obj-$(CONFIG_MCTP) += mctp/
obj-$(CONFIG_NET_HANDSHAKE) += handshake/
+obj-$(CONFIG_NET_SHAPER) += shaper/
diff --git a/net/shaper/Makefile b/net/shaper/Makefile
new file mode 100644
index 000000000000..13375884d60e
--- /dev/null
+++ b/net/shaper/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for the Generic HANDSHAKE service
+#
+# Copyright (c) 2024, Red Hat, Inc.
+#
+
+obj-y += shaper.o shaper_nl_gen.o
+
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
new file mode 100644
index 000000000000..49de88c68e2f
--- /dev/null
+++ b/net/shaper/shaper.c
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include <linux/kernel.h>
+#include <linux/skbuff.h>
+
+#include "shaper_nl_gen.h"
+
+int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_get_dumpit(struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return -EOPNOTSUPP;
+}
+
+int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return -EOPNOTSUPP;
+}
+
+static int __init shaper_init(void)
+{
+ return genl_register_family(&net_shaper_nl_family);
+}
+
+subsys_initcall(shaper_init);
diff --git a/net/shaper/shaper_nl_gen.c b/net/shaper/shaper_nl_gen.c
new file mode 100644
index 000000000000..b444d1ff55a1
--- /dev/null
+++ b/net/shaper/shaper_nl_gen.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/shaper.yaml */
+/* YNL-GEN kernel source */
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include "shaper_nl_gen.h"
+
+#include <uapi/linux/net_shaper.h>
+
+/* Common nested types */
+const struct nla_policy net_shaper_handle_nl_policy[NET_SHAPER_A_ID + 1] = {
+ [NET_SHAPER_A_SCOPE] = NLA_POLICY_MAX(NLA_U32, 4),
+ [NET_SHAPER_A_ID] = { .type = NLA_U32, },
+};
+
+const struct nla_policy net_shaper_ns_info_nl_policy[NET_SHAPER_A_WEIGHT + 1] = {
+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+ [NET_SHAPER_A_METRIC] = NLA_POLICY_MAX(NLA_U32, 1),
+ [NET_SHAPER_A_BW_MIN] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_BW_MAX] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_BURST] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_PRIORITY] = { .type = NLA_U32, },
+ [NET_SHAPER_A_WEIGHT] = { .type = NLA_U32, },
+};
+
+const struct nla_policy net_shaper_ns_output_info_nl_policy[NET_SHAPER_A_PARENT + 1] = {
+ [NET_SHAPER_A_PARENT] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+ [NET_SHAPER_A_METRIC] = NLA_POLICY_MAX(NLA_U32, 1),
+ [NET_SHAPER_A_BW_MIN] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_BW_MAX] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_BURST] = { .type = NLA_UINT, },
+ [NET_SHAPER_A_PRIORITY] = { .type = NLA_U32, },
+ [NET_SHAPER_A_WEIGHT] = { .type = NLA_U32, },
+};
+
+/* NET_SHAPER_CMD_GET - do */
+static const struct nla_policy net_shaper_get_do_nl_policy[NET_SHAPER_A_HANDLE + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+};
+
+/* NET_SHAPER_CMD_GET - dump */
+static const struct nla_policy net_shaper_get_dump_nl_policy[NET_SHAPER_A_IFINDEX + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+};
+
+/* NET_SHAPER_CMD_SET - do */
+static const struct nla_policy net_shaper_set_nl_policy[NET_SHAPER_A_SHAPER + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_SHAPER] = NLA_POLICY_NESTED(net_shaper_ns_info_nl_policy),
+};
+
+/* NET_SHAPER_CMD_DELETE - do */
+static const struct nla_policy net_shaper_delete_nl_policy[NET_SHAPER_A_HANDLE + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
+};
+
+/* NET_SHAPER_CMD_GROUP - do */
+static const struct nla_policy net_shaper_group_nl_policy[NET_SHAPER_A_OUTPUT + 1] = {
+ [NET_SHAPER_A_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_INPUTS] = NLA_POLICY_NESTED(net_shaper_ns_info_nl_policy),
+ [NET_SHAPER_A_OUTPUT] = NLA_POLICY_NESTED(net_shaper_ns_output_info_nl_policy),
+};
+
+/* Ops table for net_shaper */
+static const struct genl_split_ops net_shaper_nl_ops[] = {
+ {
+ .cmd = NET_SHAPER_CMD_GET,
+ .doit = net_shaper_nl_get_doit,
+ .policy = net_shaper_get_do_nl_policy,
+ .maxattr = NET_SHAPER_A_HANDLE,
+ .flags = GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_GET,
+ .dumpit = net_shaper_nl_get_dumpit,
+ .policy = net_shaper_get_dump_nl_policy,
+ .maxattr = NET_SHAPER_A_IFINDEX,
+ .flags = GENL_CMD_CAP_DUMP,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_SET,
+ .doit = net_shaper_nl_set_doit,
+ .policy = net_shaper_set_nl_policy,
+ .maxattr = NET_SHAPER_A_SHAPER,
+ .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_DELETE,
+ .doit = net_shaper_nl_delete_doit,
+ .policy = net_shaper_delete_nl_policy,
+ .maxattr = NET_SHAPER_A_HANDLE,
+ .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_GROUP,
+ .doit = net_shaper_nl_group_doit,
+ .policy = net_shaper_group_nl_policy,
+ .maxattr = NET_SHAPER_A_OUTPUT,
+ .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
+ },
+};
+
+struct genl_family net_shaper_nl_family __ro_after_init = {
+ .name = NET_SHAPER_FAMILY_NAME,
+ .version = NET_SHAPER_FAMILY_VERSION,
+ .netnsok = true,
+ .parallel_ops = true,
+ .module = THIS_MODULE,
+ .split_ops = net_shaper_nl_ops,
+ .n_split_ops = ARRAY_SIZE(net_shaper_nl_ops),
+};
diff --git a/net/shaper/shaper_nl_gen.h b/net/shaper/shaper_nl_gen.h
new file mode 100644
index 000000000000..00cee4efd21e
--- /dev/null
+++ b/net/shaper/shaper_nl_gen.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/* Do not edit directly, auto-generated from: */
+/* Documentation/netlink/specs/shaper.yaml */
+/* YNL-GEN kernel header */
+
+#ifndef _LINUX_NET_SHAPER_GEN_H
+#define _LINUX_NET_SHAPER_GEN_H
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include <uapi/linux/net_shaper.h>
+
+/* Common nested types */
+extern const struct nla_policy net_shaper_handle_nl_policy[NET_SHAPER_A_ID + 1];
+extern const struct nla_policy net_shaper_ns_info_nl_policy[NET_SHAPER_A_WEIGHT + 1];
+extern const struct nla_policy net_shaper_ns_output_info_nl_policy[NET_SHAPER_A_PARENT + 1];
+
+int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
+int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info);
+
+extern struct genl_family net_shaper_nl_family;
+
+#endif /* _LINUX_NET_SHAPER_GEN_H */
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 03/12] net-shapers: implement NL get operation
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 01/12] tools: ynl: lift an assumption about spec file name Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 02/12] netlink: spec: add shaper YAML spec Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-08-01 13:42 ` Jiri Pirko
` (2 more replies)
2024-07-30 20:39 ` [PATCH v3 04/12] net-shapers: implement NL set and delete operations Paolo Abeni
` (9 subsequent siblings)
12 siblings, 3 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Introduce the basic infrastructure to implement the net-shaper
core functionality. Each network devices carries a net-shaper cache,
the NL get() operation fetches the data from such cache.
The cache is initially empty, will be fill by the set()/group()
operation implemented later and is destroyed at device cleanup time.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
RFC v2 -> RFC v3:
- dev_put() -> netdev_put()
---
Documentation/networking/kapi.rst | 3 +
include/linux/netdevice.h | 17 ++
include/net/net_shaper.h | 158 +++++++++++++++++++
net/core/dev.c | 2 +
net/core/dev.h | 6 +
net/shaper/shaper.c | 251 +++++++++++++++++++++++++++++-
6 files changed, 435 insertions(+), 2 deletions(-)
create mode 100644 include/net/net_shaper.h
diff --git a/Documentation/networking/kapi.rst b/Documentation/networking/kapi.rst
index ea55f462cefa..98682b9a13ee 100644
--- a/Documentation/networking/kapi.rst
+++ b/Documentation/networking/kapi.rst
@@ -104,6 +104,9 @@ Driver Support
.. kernel-doc:: include/linux/netdevice.h
:internal:
+.. kernel-doc:: include/net/net_shaper.h
+ :internal:
+
PHY Support
-----------
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 607009150b5f..d3d952be711c 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -81,6 +81,8 @@ struct xdp_frame;
struct xdp_metadata_ops;
struct xdp_md;
struct ethtool_netdev_state;
+struct net_shaper_ops;
+struct net_shaper_data;
typedef u32 xdp_features_t;
@@ -1598,6 +1600,14 @@ struct net_device_ops {
int (*ndo_hwtstamp_set)(struct net_device *dev,
struct kernel_hwtstamp_config *kernel_config,
struct netlink_ext_ack *extack);
+
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+ /**
+ * @net_shaper_ops: Device shaping offload operations
+ * see include/net/net_shapers.h
+ */
+ const struct net_shaper_ops *net_shaper_ops;
+#endif
};
/**
@@ -2408,6 +2418,13 @@ struct net_device {
/** @irq_moder: dim parameters used if IS_ENABLED(CONFIG_DIMLIB). */
struct dim_irq_moder *irq_moder;
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+ /**
+ * @net_shaper_data: data tracking the current shaper status
+ * see include/net/net_shapers.h
+ */
+ struct net_shaper_data *net_shaper_data;
+#endif
u8 priv[] ____cacheline_aligned
__counted_by(priv_len);
} ____cacheline_aligned;
diff --git a/include/net/net_shaper.h b/include/net/net_shaper.h
new file mode 100644
index 000000000000..8cd65d727e52
--- /dev/null
+++ b/include/net/net_shaper.h
@@ -0,0 +1,158 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _NET_SHAPER_H_
+#define _NET_SHAPER_H_
+
+#include <linux/types.h>
+#include <linux/bits.h>
+#include <linux/bitfield.h>
+#include <linux/netdevice.h>
+#include <linux/netlink.h>
+
+#include <uapi/linux/net_shaper.h>
+
+/**
+ * struct net_shaper_info - represents a shaping node on the NIC H/W
+ * zeroed field are considered not set.
+ * @handle: Unique identifier for the shaper, see @net_shaper_make_handle
+ * @parent: Unique identifier for the shaper parent, usually implied. Only
+ * NET_SHAPER_SCOPE_QUEUE, NET_SHAPER_SCOPE_NETDEV and NET_SHAPER_SCOPE_DETACHED
+ * can have the parent handle explicitly set, placing such shaper under
+ * the specified parent.
+ * @metric: Specify if the bw limits refers to PPS or BPS
+ * @bw_min: Minimum guaranteed rate for this shaper
+ * @bw_max: Maximum peak bw allowed for this shaper
+ * @burst: Maximum burst for the peek rate of this shaper
+ * @priority: Scheduling priority for this shaper
+ * @weight: Scheduling weight for this shaper
+ * @children: Number of nested shapers, accounted only for DETACHED scope
+ */
+struct net_shaper_info {
+ u32 handle;
+ u32 parent;
+ enum net_shaper_metric metric;
+ u64 bw_min;
+ u64 bw_max;
+ u64 burst;
+ u32 priority;
+ u32 weight;
+ u32 children;
+};
+
+/**
+ * define NET_SHAPER_SCOPE_VF - Shaper scope
+ *
+ * This shaper scope is not exposed to user-space; the shaper is attached to
+ * the given virtual function.
+ */
+#define NET_SHAPER_SCOPE_VF __NET_SHAPER_SCOPE_MAX
+
+/**
+ * struct net_shaper_ops - Operations on device H/W shapers
+ *
+ * The initial shaping configuration ad device initialization is empty/
+ * a no-op/does not constraint the b/w in any way.
+ * The network core keeps track of the applied user-configuration in
+ * per device storage.
+ *
+ * Each shaper is uniquely identified within the device with an 'handle',
+ * dependent on the shaper scope and other data, see @shaper_make_handle()
+ */
+struct net_shaper_ops {
+ /**
+ * @group: create the specified shapers group
+ *
+ * Nest the specified @inputs shapers under the given @output shaper
+ * on the network device @dev. The @input shaper array size is specified
+ * by @nr_input.
+ * Create either the @inputs and the @output shaper as needed,
+ * otherwise move them as needed. Can't create @inputs shapers with
+ * NET_SHAPER_SCOPE_DETACHED scope, a separate @group call with such
+ * shaper as @output is needed.
+ *
+ * Returns 0 on group successfully created, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*group)(struct net_device *dev, int nr_input,
+ const struct net_shaper_info *inputs,
+ const struct net_shaper_info *output,
+ struct netlink_ext_ack *extack);
+
+ /**
+ * @set: Updates the specified shaper
+ *
+ * Updates or creates the specified @shaper on the given device
+ * @dev. Can't create NET_SHAPER_SCOPE_DETACHED shapers, use @group
+ * instead.
+ *
+ * Returns 0 on group successfully created, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*set)(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack);
+
+ /**
+ * @delete: Removes the specified shaper from the NIC
+ *
+ * Removes the shaper configuration as identified by the given @handle
+ * on the specified device @dev, restoring the default behavior.
+ *
+ * Returns 0 on group successfully created, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*delete)(struct net_device *dev, u32 handle,
+ struct netlink_ext_ack *extack);
+};
+
+#define NET_SHAPER_SCOPE_SHIFT 26
+#define NET_SHAPER_ID_MASK GENMASK(NET_SHAPER_SCOPE_SHIFT - 1, 0)
+#define NET_SHAPER_SCOPE_MASK GENMASK(31, NET_SHAPER_SCOPE_SHIFT)
+
+#define NET_SHAPER_ID_UNSPEC NET_SHAPER_ID_MASK
+
+/**
+ * net_shaper_make_handle - creates an unique shaper identifier
+ * @scope: the shaper scope
+ * @id: the shaper id number
+ *
+ * Return: an unique identifier for the shaper
+ *
+ * Combines the specified arguments to create an unique identifier for
+ * the shaper. The @id argument semantic depends on the
+ * specified scope.
+ * For @NET_SHAPER_SCOPE_QUEUE_GROUP, @id is the queue group id
+ * For @NET_SHAPER_SCOPE_QUEUE, @id is the queue number.
+ * For @NET_SHAPER_SCOPE_VF, @id is virtual function number.
+ */
+static inline u32 net_shaper_make_handle(enum net_shaper_scope scope,
+ int id)
+{
+ return FIELD_PREP(NET_SHAPER_SCOPE_MASK, scope) |
+ FIELD_PREP(NET_SHAPER_ID_MASK, id);
+}
+
+/**
+ * net_shaper_handle_scope - extract the scope from the given handle
+ * @handle: the shaper handle
+ *
+ * Return: the corresponding scope
+ */
+static inline enum net_shaper_scope net_shaper_handle_scope(u32 handle)
+{
+ return FIELD_GET(NET_SHAPER_SCOPE_MASK, handle);
+}
+
+/**
+ * net_shaper_handle_id - extract the id number from the given handle
+ * @handle: the shaper handle
+ *
+ * Return: the corresponding id number
+ */
+static inline int net_shaper_handle_id(u32 handle)
+{
+ return FIELD_GET(NET_SHAPER_ID_MASK, handle);
+}
+
+#endif
+
diff --git a/net/core/dev.c b/net/core/dev.c
index 6ea1d20676fb..3dc1dd428eda 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -11169,6 +11169,8 @@ void free_netdev(struct net_device *dev)
/* Flush device addresses */
dev_addr_flush(dev);
+ dev_shaper_flush(dev);
+
list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
netif_napi_del(p);
diff --git a/net/core/dev.h b/net/core/dev.h
index 5654325c5b71..e376fc1c867b 100644
--- a/net/core/dev.h
+++ b/net/core/dev.h
@@ -35,6 +35,12 @@ void dev_addr_flush(struct net_device *dev);
int dev_addr_init(struct net_device *dev);
void dev_addr_check(struct net_device *dev);
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+void dev_shaper_flush(struct net_device *dev);
+#else
+static inline void dev_shaper_flush(struct net_device *dev) {}
+#endif
+
/* sysctls not referred to from outside net/core/ */
extern int netdev_unregister_timeout_secs;
extern int weight_p;
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 49de88c68e2f..5d1d6e600a6a 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -1,19 +1,242 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <linux/kernel.h>
+#include <linux/idr.h>
#include <linux/skbuff.h>
+#include <linux/xarray.h>
+#include <net/net_shaper.h>
#include "shaper_nl_gen.h"
+#include "../core/dev.h"
+
+struct net_shaper_data {
+ struct xarray shapers;
+ struct idr detached_ids;
+};
+
+struct net_shaper_nl_ctx {
+ u32 start_handle;
+};
+
+static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
+ const struct genl_info *info)
+{
+ struct nlattr *handle_attr;
+
+ if (!handle)
+ return 0;
+
+ handle_attr = nla_nest_start_noflag(msg, type);
+ if (!handle_attr)
+ return -EMSGSIZE;
+
+ if (nla_put_u32(msg, NET_SHAPER_A_SCOPE,
+ net_shaper_handle_scope(handle)) ||
+ nla_put_u32(msg, NET_SHAPER_A_ID,
+ net_shaper_handle_id(handle)))
+ goto handle_nest_cancel;
+
+ nla_nest_end(msg, handle_attr);
+ return 0;
+
+handle_nest_cancel:
+ nla_nest_cancel(msg, handle_attr);
+ return -EMSGSIZE;
+}
+
+static int
+net_shaper_fill_one(struct sk_buff *msg, struct net_shaper_info *shaper,
+ const struct genl_info *info)
+{
+ void *hdr;
+
+ hdr = genlmsg_iput(msg, info);
+ if (!hdr)
+ return -EMSGSIZE;
+
+ if (fill_handle(msg, shaper->parent, NET_SHAPER_A_PARENT, info) ||
+ fill_handle(msg, shaper->handle, NET_SHAPER_A_HANDLE, info) ||
+ nla_put_u32(msg, NET_SHAPER_A_METRIC, shaper->metric) ||
+ nla_put_uint(msg, NET_SHAPER_A_BW_MIN, shaper->bw_min) ||
+ nla_put_uint(msg, NET_SHAPER_A_BW_MAX, shaper->bw_max) ||
+ nla_put_uint(msg, NET_SHAPER_A_BURST, shaper->burst) ||
+ nla_put_u32(msg, NET_SHAPER_A_PRIORITY, shaper->priority) ||
+ nla_put_u32(msg, NET_SHAPER_A_WEIGHT, shaper->weight))
+ goto nla_put_failure;
+
+ genlmsg_end(msg, hdr);
+
+ return 0;
+
+nla_put_failure:
+ genlmsg_cancel(msg, hdr);
+ return -EMSGSIZE;
+}
+
+/* On success sets pdev to the relevant device and acquires a reference
+ * to it
+ */
+static int fetch_dev(const struct genl_info *info, struct net_device **pdev)
+{
+ struct net *ns = genl_info_net(info);
+ struct net_device *dev;
+ int ifindex;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_IFINDEX))
+ return -EINVAL;
+
+ ifindex = nla_get_u32(info->attrs[NET_SHAPER_A_IFINDEX]);
+ dev = dev_get_by_index(ns, ifindex);
+ if (!dev) {
+ GENL_SET_ERR_MSG_FMT(info, "device %d not found", ifindex);
+ return -EINVAL;
+ }
+
+ if (!dev->netdev_ops->net_shaper_ops) {
+ GENL_SET_ERR_MSG_FMT(info, "device %s does not support H/W shaper",
+ dev->name);
+ netdev_put(dev, NULL);
+ return -EOPNOTSUPP;
+ }
+
+ *pdev = dev;
+ return 0;
+}
+
+static struct xarray *__sc_container(struct net_device *dev)
+{
+ return dev->net_shaper_data ? &dev->net_shaper_data->shapers : NULL;
+}
+
+/* lookup the given shaper inside the cache */
+static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle)
+{
+ struct xarray *xa = __sc_container(dev);
+
+ return xa ? xa_load(xa, handle) : NULL;
+}
+
+static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
+ u32 *handle)
+{
+ struct nlattr *tb[NET_SHAPER_A_ID + 1];
+ struct nlattr *scope_attr, *id_attr;
+ enum net_shaper_scope scope;
+ u32 id = 0;
+ int ret;
+
+ ret = nla_parse_nested(tb, NET_SHAPER_A_ID, attr,
+ net_shaper_handle_nl_policy, info->extack);
+ if (ret < 0)
+ return ret;
+
+ scope_attr = tb[NET_SHAPER_A_SCOPE];
+ if (!scope_attr) {
+ GENL_SET_ERR_MSG(info, "Missing 'scope' attribute for handle");
+ return -EINVAL;
+ }
+
+ scope = nla_get_u32(scope_attr);
+
+ /* the default id for detached scope shapers is an invalid one
+ * to help the 'group' operation discriminate between new
+ * detached shaper creation (ID_UNSPEC) and reuse of existing
+ * shaper (any other value)
+ */
+ id_attr = tb[NET_SHAPER_A_ID];
+ if (id_attr)
+ id = nla_get_u32(id_attr);
+ else if (scope == NET_SHAPER_SCOPE_DETACHED)
+ id = NET_SHAPER_ID_UNSPEC;
+
+ *handle = net_shaper_make_handle(scope, id);
+ return 0;
+}
+
int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
{
- return -EOPNOTSUPP;
+ struct net_shaper_info *shaper;
+ struct net_device *dev;
+ struct sk_buff *msg;
+ u32 handle;
+ int ret;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
+ goto put;
+
+ ret = parse_handle(info->attrs[NET_SHAPER_A_HANDLE], info, &handle);
+ if (ret < 0)
+ goto put;
+
+ shaper = sc_lookup(dev, handle);
+ if (!shaper) {
+ GENL_SET_ERR_MSG_FMT(info, "Can't find shaper for handle %x", handle);
+ ret = -EINVAL;
+ goto put;
+ }
+
+ msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg) {
+ ret = -ENOMEM;
+ goto put;
+ }
+
+ ret = net_shaper_fill_one(msg, shaper, info);
+ if (ret)
+ goto free_msg;
+
+ ret = genlmsg_reply(msg, info);
+ if (ret)
+ goto free_msg;
+
+put:
+ netdev_put(dev, NULL);
+ return ret;
+
+free_msg:
+ nlmsg_free(msg);
+ goto put;
}
int net_shaper_nl_get_dumpit(struct sk_buff *skb,
struct netlink_callback *cb)
{
- return -EOPNOTSUPP;
+ struct net_shaper_nl_ctx *ctx = (struct net_shaper_nl_ctx *)cb->ctx;
+ const struct genl_info *info = genl_info_dump(cb);
+ struct net_shaper_info *shaper;
+ struct net_device *dev;
+ unsigned long handle;
+ int ret;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ BUILD_BUG_ON(sizeof(struct net_shaper_nl_ctx) > sizeof(cb->ctx));
+
+ /* don't error out dumps performed before any set operation */
+ if (!dev->net_shaper_data) {
+ ret = 0;
+ goto put;
+ }
+
+ xa_for_each_range(&dev->net_shaper_data->shapers, handle, shaper,
+ ctx->start_handle, U32_MAX) {
+ ret = net_shaper_fill_one(skb, shaper, info);
+ if (ret)
+ goto put;
+
+ ctx->start_handle = handle;
+ }
+
+put:
+ netdev_put(dev, NULL);
+ return ret;
}
int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
@@ -26,6 +249,30 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
return -EOPNOTSUPP;
}
+int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return -EOPNOTSUPP;
+}
+
+void dev_shaper_flush(struct net_device *dev)
+{
+ struct xarray *xa = __sc_container(dev);
+ struct net_shaper_info *cur;
+ unsigned long index;
+
+ if (!xa)
+ return;
+
+ xa_lock(xa);
+ xa_for_each(xa, index, cur) {
+ __xa_erase(xa, index);
+ kfree(cur);
+ }
+ idr_destroy(&dev->net_shaper_data->detached_ids);
+ xa_unlock(xa);
+ kfree(xa);
+}
+
static int __init shaper_init(void)
{
return genl_register_family(&net_shaper_nl_family);
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 04/12] net-shapers: implement NL set and delete operations
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (2 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 03/12] net-shapers: implement NL get operation Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-08-01 15:00 ` Jakub Kicinski
2024-07-30 20:39 ` [PATCH v3 05/12] net-shapers: implement NL group operation Paolo Abeni
` (8 subsequent siblings)
12 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Both NL operations directly map on the homonymous device shaper
callbacks and update accordingly the shapers cache.
Implement the cache modification helpers to additionally deal with
DETACHED scope shaper. That will be needed by the group() operation
implemented in the next patch.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
RFC v2 -> RFC v3:
- dev_put() -> netdev_put()
---
net/shaper/shaper.c | 324 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 322 insertions(+), 2 deletions(-)
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 5d1d6e600a6a..8ecbfd9002be 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -19,6 +19,35 @@ struct net_shaper_nl_ctx {
u32 start_handle;
};
+static u32 default_parent(u32 handle)
+{
+ enum net_shaper_scope parent, scope = net_shaper_handle_scope(handle);
+
+ switch (scope) {
+ case NET_SHAPER_SCOPE_PORT:
+ case NET_SHAPER_SCOPE_UNSPEC:
+ parent = NET_SHAPER_SCOPE_UNSPEC;
+ break;
+
+ case NET_SHAPER_SCOPE_QUEUE:
+ case NET_SHAPER_SCOPE_DETACHED:
+ parent = NET_SHAPER_SCOPE_NETDEV;
+ break;
+
+ case NET_SHAPER_SCOPE_NETDEV:
+ case NET_SHAPER_SCOPE_VF:
+ parent = NET_SHAPER_SCOPE_PORT;
+ break;
+ }
+
+ return net_shaper_make_handle(parent, 0);
+}
+
+static bool is_detached(u32 handle)
+{
+ return net_shaper_handle_scope(handle) == NET_SHAPER_SCOPE_DETACHED;
+}
+
static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
const struct genl_info *info)
{
@@ -117,6 +146,115 @@ static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle)
return xa ? xa_load(xa, handle) : NULL;
}
+/* allocate on demand the per device shaper's cache */
+static struct xarray *__sc_init(struct net_device *dev,
+ struct netlink_ext_ack *extack)
+{
+ if (!dev->net_shaper_data) {
+ dev->net_shaper_data = kmalloc(sizeof(*dev->net_shaper_data),
+ GFP_KERNEL);
+ if (!dev->net_shaper_data) {
+ NL_SET_ERR_MSG(extack, "Can't allocate memory for shaper data");
+ return NULL;
+ }
+
+ xa_init(&dev->net_shaper_data->shapers);
+ idr_init(&dev->net_shaper_data->detached_ids);
+ }
+ return &dev->net_shaper_data->shapers;
+}
+
+/* prepare the cache to actually insert the given shaper, doing
+ * in advance the needed allocations
+ */
+static int sc_prepare_insert(struct net_device *dev, u32 *handle,
+ struct netlink_ext_ack *extack)
+{
+ enum net_shaper_scope scope = net_shaper_handle_scope(*handle);
+ struct xarray *xa = __sc_init(dev, extack);
+ struct net_shaper_info *prev, *cur;
+ bool id_allocated = false;
+ int ret, id;
+
+ if (!xa)
+ return -ENOMEM;
+
+ cur = xa_load(xa, *handle);
+ if (cur)
+ return 0;
+
+ /* allocated a new id, if needed */
+ if (scope == NET_SHAPER_SCOPE_DETACHED &&
+ net_shaper_handle_id(*handle) == NET_SHAPER_ID_UNSPEC) {
+ xa_lock(xa);
+ id = idr_alloc(&dev->net_shaper_data->detached_ids, NULL,
+ 0, NET_SHAPER_ID_UNSPEC, GFP_ATOMIC);
+ xa_unlock(xa);
+
+ if (id < 0) {
+ NL_SET_ERR_MSG(extack, "Can't allocate new id for detached shaper");
+ return id;
+ }
+
+ *handle = net_shaper_make_handle(scope, id);
+ id_allocated = true;
+ }
+
+ cur = kmalloc(sizeof(*cur), GFP_KERNEL | __GFP_ZERO);
+ if (!cur) {
+ NL_SET_ERR_MSG(extack, "Can't allocate memory for cached shaper");
+ ret = -ENOMEM;
+ goto free_id;
+ }
+
+ /* mark 'tentative' shaper inside the cache */
+ xa_lock(xa);
+ prev = __xa_store(xa, *handle, cur, GFP_KERNEL);
+ __xa_set_mark(xa, *handle, XA_MARK_0);
+ xa_unlock(xa);
+ if (xa_err(prev)) {
+ NL_SET_ERR_MSG(extack, "Can't insert shaper into cache");
+ kfree(cur);
+ ret = xa_err(prev);
+ goto free_id;
+ }
+ return 0;
+
+free_id:
+ if (id_allocated) {
+ xa_lock(xa);
+ idr_remove(&dev->net_shaper_data->detached_ids,
+ net_shaper_handle_id(*handle));
+ xa_unlock(xa);
+ }
+ return ret;
+}
+
+/* commit the tentative insert with the actual values.
+ * Must be called only after a successful sc_prepare_insert()
+ */
+static void sc_commit(struct net_device *dev, int nr_shapers,
+ const struct net_shaper_info *shapers)
+{
+ struct xarray *xa = __sc_container(dev);
+ struct net_shaper_info *cur;
+ int i;
+
+ xa_lock(xa);
+ for (i = 0; i < nr_shapers; ++i) {
+ cur = xa_load(xa, shapers[i].handle);
+ if (WARN_ON_ONCE(!cur))
+ continue;
+
+ /* successful update: drop the tentative mark
+ * and update the cache
+ */
+ __xa_clear_mark(xa, shapers[i].handle, XA_MARK_0);
+ *cur = shapers[i];
+ }
+ xa_unlock(xa);
+}
+
static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
u32 *handle)
{
@@ -154,6 +292,68 @@ static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
return 0;
}
+static int __parse_shaper(struct net_device *dev, struct nlattr **tb,
+ const struct genl_info *info,
+ struct net_shaper_info *shaper)
+{
+ struct net_shaper_info *old;
+ int ret;
+
+ /* the shaper handle is the only mandatory attribute */
+ if (NL_REQ_ATTR_CHECK(info->extack, NULL, tb, NET_SHAPER_A_HANDLE))
+ return -EINVAL;
+
+ ret = parse_handle(tb[NET_SHAPER_A_HANDLE], info, &shaper->handle);
+ if (ret)
+ return ret;
+
+ /* fetch existing data, if any, so that user provide info will
+ * incrementally update the existing shaper configuration
+ */
+ old = sc_lookup(dev, shaper->handle);
+ if (old)
+ *shaper = *old;
+ else
+ shaper->parent = default_parent(shaper->handle);
+
+ if (tb[NET_SHAPER_A_METRIC])
+ shaper->metric = nla_get_u32(tb[NET_SHAPER_A_METRIC]);
+
+ if (tb[NET_SHAPER_A_BW_MIN])
+ shaper->bw_min = nla_get_uint(tb[NET_SHAPER_A_BW_MIN]);
+
+ if (tb[NET_SHAPER_A_BW_MAX])
+ shaper->bw_max = nla_get_uint(tb[NET_SHAPER_A_BW_MAX]);
+
+ if (tb[NET_SHAPER_A_BURST])
+ shaper->burst = nla_get_uint(tb[NET_SHAPER_A_BURST]);
+
+ if (tb[NET_SHAPER_A_PRIORITY])
+ shaper->priority = nla_get_u32(tb[NET_SHAPER_A_PRIORITY]);
+
+ if (tb[NET_SHAPER_A_WEIGHT])
+ shaper->weight = nla_get_u32(tb[NET_SHAPER_A_WEIGHT]);
+ return 0;
+}
+
+/* fetch the cached shaper info and update them with the user-provided
+ * attributes
+ */
+static int parse_shaper(struct net_device *dev, const struct nlattr *attr,
+ const struct genl_info *info,
+ struct net_shaper_info *shaper)
+{
+ struct nlattr *tb[NET_SHAPER_A_WEIGHT + 1];
+ int ret;
+
+ ret = nla_parse_nested(tb, NET_SHAPER_A_WEIGHT, attr,
+ net_shaper_ns_info_nl_policy, info->extack);
+ if (ret < 0)
+ return ret;
+
+ return __parse_shaper(dev, tb, info, shaper);
+}
+
int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net_shaper_info *shaper;
@@ -239,14 +439,134 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb,
return ret;
}
+/* Update the H/W and on success update the local cache, too */
+static int net_shaper_set(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack)
+{
+ enum net_shaper_scope scope;
+ u32 handle = shaper->handle;
+ int ret;
+
+ scope = net_shaper_handle_scope(handle);
+ if (scope == NET_SHAPER_SCOPE_PORT ||
+ scope == NET_SHAPER_SCOPE_UNSPEC) {
+ NL_SET_ERR_MSG_FMT(extack, "Can't set shaper %x with scope %d",
+ handle, scope);
+ return -EINVAL;
+ }
+ if (scope == NET_SHAPER_SCOPE_DETACHED && !sc_lookup(dev, handle)) {
+ NL_SET_ERR_MSG_FMT(extack, "Shaper %x with detached scope does not exist",
+ handle);
+ return -EINVAL;
+ }
+
+ ret = sc_prepare_insert(dev, &handle, extack);
+ if (ret)
+ return ret;
+
+ ret = dev->netdev_ops->net_shaper_ops->set(dev, shaper, extack);
+ sc_commit(dev, 1, shaper);
+ return ret;
+}
+
int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
{
- return -EOPNOTSUPP;
+ struct net_shaper_info shaper;
+ struct net_device *dev;
+ struct nlattr *attr;
+ int ret;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_SHAPER))
+ return -EINVAL;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ attr = info->attrs[NET_SHAPER_A_SHAPER];
+ ret = parse_shaper(dev, attr, info, &shaper);
+ if (ret)
+ goto put;
+
+ ret = net_shaper_set(dev, &shaper, info->extack);
+
+put:
+ netdev_put(dev, NULL);
+ return ret;
+}
+
+static int net_shaper_delete(struct net_device *dev, u32 handle,
+ struct netlink_ext_ack *extack)
+{
+ struct net_shaper_info *parent, *shaper = sc_lookup(dev, handle);
+ struct xarray *xa = __sc_container(dev);
+ enum net_shaper_scope pscope;
+ u32 parent_handle;
+ int ret;
+
+ if (!xa || !shaper) {
+ NL_SET_ERR_MSG_FMT(extack, "Shaper %x not found", handle);
+ return -EINVAL;
+ }
+
+ if (is_detached(handle) && shaper->children > 0) {
+ NL_SET_ERR_MSG_FMT(extack, "Can't delete detached shaper %d with %d child nodes",
+ handle, shaper->children);
+ return -EINVAL;
+ }
+
+ while (shaper) {
+ parent_handle = shaper->parent;
+ pscope = net_shaper_handle_scope(parent_handle);
+
+ ret = dev->netdev_ops->net_shaper_ops->delete(dev, handle,
+ extack);
+ if (ret < 0)
+ return ret;
+
+ xa_lock(xa);
+ __xa_erase(xa, handle);
+ if (is_detached(handle))
+ idr_remove(&dev->net_shaper_data->detached_ids,
+ net_shaper_handle_id(handle));
+ xa_unlock(xa);
+ kfree(shaper);
+ shaper = NULL;
+
+ if (pscope == NET_SHAPER_SCOPE_DETACHED) {
+ parent = sc_lookup(dev, parent_handle);
+ if (parent && !--parent->children) {
+ shaper = parent;
+ handle = parent_handle;
+ }
+ }
+ }
+ return 0;
}
int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
{
- return -EOPNOTSUPP;
+ struct net_device *dev;
+ u32 handle;
+ int ret;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
+ return -EINVAL;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ ret = parse_handle(info->attrs[NET_SHAPER_A_HANDLE], info, &handle);
+ if (ret)
+ goto put;
+
+ ret = net_shaper_delete(dev, handle, info->extack);
+
+put:
+ netdev_put(dev, NULL);
+ return ret;
}
int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 05/12] net-shapers: implement NL group operation
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (3 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 04/12] net-shapers: implement NL set and delete operations Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 06/12] netlink: spec: add shaper introspection support Paolo Abeni
` (7 subsequent siblings)
12 siblings, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Allow grouping multiple inputs shaper under the given output's
one.
Try hard to pre-allocated the needed resources, to avoid non
trivial H/W configuration rollbacks in case of any failure.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
RFC v2 -> RFC v3:
- dev_put() -> netdev_put()
---
net/shaper/shaper.c | 265 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 264 insertions(+), 1 deletion(-)
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 8ecbfd9002be..87db541de646 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -48,6 +48,18 @@ static bool is_detached(u32 handle)
return net_shaper_handle_scope(handle) == NET_SHAPER_SCOPE_DETACHED;
}
+/* count the number of [multi] attributes of the given type */
+static int attr_list_len(struct genl_info *info, int type)
+{
+ struct nlattr *attr;
+ int rem, cnt = 0;
+
+ nla_for_each_attr_type(attr, type, genlmsg_data(info->genlhdr),
+ genlmsg_len(info->genlhdr), rem)
+ cnt++;
+ return cnt;
+}
+
static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
const struct genl_info *info)
{
@@ -255,6 +267,27 @@ static void sc_commit(struct net_device *dev, int nr_shapers,
xa_unlock(xa);
}
+/* rollback all the tentative inserts from the shaper cache */
+static void sc_rollback(struct net_device *dev)
+{
+ struct xarray *xa = __sc_container(dev);
+ struct net_shaper_info *cur;
+ unsigned long index;
+
+ if (!xa)
+ return;
+
+ xa_lock(xa);
+ xa_for_each_marked(xa, index, cur, XA_MARK_0) {
+ if (is_detached(index))
+ idr_remove(&dev->net_shaper_data->detached_ids,
+ net_shaper_handle_id(index));
+ __xa_erase(xa, index);
+ kfree(cur);
+ }
+ xa_unlock(xa);
+}
+
static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
u32 *handle)
{
@@ -354,6 +387,36 @@ static int parse_shaper(struct net_device *dev, const struct nlattr *attr,
return __parse_shaper(dev, tb, info, shaper);
}
+/* alike parse_shaper(), but additionally allow the user specifying
+ * the shaper's parent handle
+ */
+static int parse_output_shaper(struct net_device *dev,
+ const struct nlattr *attr,
+ const struct genl_info *info,
+ struct net_shaper_info *shaper)
+{
+ struct nlattr *tb[NET_SHAPER_A_PARENT + 1];
+ int ret;
+
+ ret = nla_parse_nested(tb, NET_SHAPER_A_PARENT, attr,
+ net_shaper_ns_output_info_nl_policy,
+ info->extack);
+ if (ret < 0)
+ return ret;
+
+ ret = __parse_shaper(dev, tb, info, shaper);
+ if (ret)
+ return ret;
+
+ if (tb[NET_SHAPER_A_PARENT]) {
+ ret = parse_handle(tb[NET_SHAPER_A_PARENT], info,
+ &shaper->parent);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net_shaper_info *shaper;
@@ -569,9 +632,209 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
return ret;
}
+/* Update the H/W and on success update the local cache, too */
+static int net_shaper_group(struct net_device *dev, int nr_inputs,
+ struct net_shaper_info *inputs,
+ struct net_shaper_info *output,
+ struct netlink_ext_ack *extack)
+{
+ enum net_shaper_scope scope, output_scope, output_pscope;
+ struct net_shaper_info *parent = NULL;
+ int i, ret;
+
+ output_scope = net_shaper_handle_scope(output->handle);
+ if (output_scope != NET_SHAPER_SCOPE_DETACHED &&
+ output_scope != NET_SHAPER_SCOPE_NETDEV) {
+ NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for output shaper %x",
+ output_scope, output->handle);
+ return -EINVAL;
+ }
+
+ output_pscope = net_shaper_handle_scope(output->parent);
+ if (output_scope == NET_SHAPER_SCOPE_DETACHED) {
+ if (net_shaper_handle_id(output->handle) !=
+ NET_SHAPER_ID_UNSPEC && !sc_lookup(dev, output->handle)) {
+ NL_SET_ERR_MSG_FMT(extack, "Output shaper %x does not exists",
+ output->handle);
+ return -EINVAL;
+ }
+ if (output_pscope != NET_SHAPER_SCOPE_DETACHED &&
+ output_pscope != NET_SHAPER_SCOPE_NETDEV) {
+ NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for output parent shaper %x",
+ output_pscope, output->parent);
+ return -EINVAL;
+ }
+ }
+
+ if (output_pscope == NET_SHAPER_SCOPE_DETACHED) {
+ parent = sc_lookup(dev, output->parent);
+ if (!parent) {
+ NL_SET_ERR_MSG_FMT(extack, "Output parent shaper %x does not exists",
+ output->parent);
+ return -EINVAL;
+ }
+ }
+
+ /* for newly created detached scope shaper, the following will update
+ * the handle, due to id allocation
+ */
+ ret = sc_prepare_insert(dev, &output->handle, extack);
+ if (ret)
+ goto rollback;
+
+ for (i = 0; i < nr_inputs; ++i) {
+ scope = net_shaper_handle_scope(inputs[i].handle);
+ if (scope != NET_SHAPER_SCOPE_QUEUE &&
+ scope != NET_SHAPER_SCOPE_DETACHED) {
+ ret = -EINVAL;
+ NL_SET_ERR_MSG_FMT(extack, "Invalid scope %d for input shaper %d handle %x",
+ scope, i, inputs[i].handle);
+ goto rollback;
+ }
+ if (scope == NET_SHAPER_SCOPE_DETACHED &&
+ !sc_lookup(dev, inputs[i].handle)) {
+ ret = -EINVAL;
+ NL_SET_ERR_MSG_FMT(extack,
+ "Can't create detached shaper as input %d handle %x",
+ i, inputs[i].handle);
+ goto rollback;
+ }
+
+ ret = sc_prepare_insert(dev, &inputs[i].handle, extack);
+ if (ret)
+ goto rollback;
+
+ /* the inputs shapers will be nested to the output */
+ if (inputs[i].parent != output->handle) {
+ inputs[i].parent = output->handle;
+ output->children++;
+ }
+ }
+
+ ret = dev->netdev_ops->net_shaper_ops->group(dev, nr_inputs,
+ inputs, output,
+ extack);
+ if (ret < 0)
+ goto rollback;
+
+ if (parent)
+ parent->children++;
+ sc_commit(dev, 1, output);
+ sc_commit(dev, nr_inputs, inputs);
+ return ret;
+
+rollback:
+ sc_rollback(dev);
+ return ret;
+}
+
+static int nla_handle_total_size(void)
+{
+ return nla_total_size(nla_total_size(sizeof(u32)) +
+ nla_total_size(sizeof(u32)));
+}
+
+static int group_send_reply(struct genl_info *info, u32 handle, int id)
+{
+ struct nlattr *handle_attr;
+ struct sk_buff *msg;
+ int ret = -EMSGSIZE;
+ void *hdr;
+
+/* prepare the msg reply in advance, to avoid device operation rollback */
+ msg = genlmsg_new(nla_handle_total_size(), GFP_KERNEL);
+ if (!msg)
+ return ret;
+
+ hdr = genlmsg_iput(msg, info);
+ if (!hdr)
+ goto free_msg;
+
+ handle_attr = nla_nest_start(msg, NET_SHAPER_A_HANDLE);
+ if (!handle_attr)
+ goto free_msg;
+
+ if (nla_put_u32(msg, NET_SHAPER_A_SCOPE,
+ net_shaper_handle_scope(handle)))
+ goto free_msg;
+
+ if (nla_put_u32(msg, NET_SHAPER_A_ID, id))
+ goto free_msg;
+
+ nla_nest_end(msg, handle_attr);
+ genlmsg_end(msg, hdr);
+
+ ret = genlmsg_reply(msg, info);
+ if (ret)
+ goto free_msg;
+
+ return ret;
+
+free_msg:
+ nlmsg_free(msg);
+ return ret;
+}
+
int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
{
- return -EOPNOTSUPP;
+ struct net_shaper_info *inputs, output;
+ int i, ret, rem, nr_inputs;
+ struct net_device *dev;
+ struct nlattr *attr;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_INPUTS) ||
+ GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_OUTPUT))
+ return -EINVAL;
+
+ ret = fetch_dev(info, &dev);
+ if (ret)
+ return ret;
+
+ nr_inputs = attr_list_len(info, NET_SHAPER_A_INPUTS);
+ inputs = kcalloc(nr_inputs, sizeof(struct net_shaper_info), GFP_KERNEL);
+ if (!inputs) {
+ GENL_SET_ERR_MSG_FMT(info, "Can't allocate memory for %d input shapers",
+ nr_inputs);
+ ret = -ENOMEM;
+ goto put;
+ }
+
+ ret = parse_output_shaper(dev, info->attrs[NET_SHAPER_A_OUTPUT], info,
+ &output);
+ if (ret)
+ goto free_shapers;
+
+ i = 0;
+ nla_for_each_attr_type(attr, NET_SHAPER_A_INPUTS,
+ genlmsg_data(info->genlhdr),
+ genlmsg_len(info->genlhdr), rem) {
+ if (WARN_ON_ONCE(i >= nr_inputs))
+ goto free_shapers;
+
+ ret = parse_shaper(dev, attr, info, &inputs[i++]);
+ if (ret)
+ goto free_shapers;
+ }
+
+ ret = net_shaper_group(dev, nr_inputs, inputs, &output, info->extack);
+ if (ret < 0)
+ goto free_shapers;
+
+ ret = group_send_reply(info, output.handle, ret);
+ if (ret) {
+ /* Error on reply is not fatal to avoid rollback a successful
+ * configuration
+ */
+ GENL_SET_ERR_MSG_FMT(info, "Can't send reply %d", ret);
+ ret = 0;
+ }
+
+free_shapers:
+ kfree(inputs);
+
+put:
+ netdev_put(dev, NULL);
+ return ret;
}
void dev_shaper_flush(struct net_device *dev)
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 06/12] netlink: spec: add shaper introspection support
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (4 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 05/12] net-shapers: implement NL group operation Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-08-02 11:21 ` Donald Hunter
2024-07-30 20:39 ` [PATCH v3 07/12] net: shaper: implement " Paolo Abeni
` (6 subsequent siblings)
12 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Allow the user-space to fine-grain query the shaping features
supported by the NIC on each domain.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
Documentation/netlink/specs/shaper.yaml | 75 +++++++++++++++++++++++++
include/uapi/linux/net_shaper.h | 17 ++++++
net/shaper/shaper.c | 11 ++++
net/shaper/shaper_nl_gen.c | 25 +++++++++
net/shaper/shaper_nl_gen.h | 3 +
5 files changed, 131 insertions(+)
diff --git a/Documentation/netlink/specs/shaper.yaml b/Documentation/netlink/specs/shaper.yaml
index 7327f5596fdb..55bf4e49fb2f 100644
--- a/Documentation/netlink/specs/shaper.yaml
+++ b/Documentation/netlink/specs/shaper.yaml
@@ -169,6 +169,52 @@ attribute-sets:
name: priority
-
name: weight
+ -
+ name: capabilities
+ attributes:
+ -
+ name: ifindex
+ type: u32
+ -
+ name: scope
+ type: u32
+ enum: scope
+ doc: The scope to which the queried capabilities apply
+ -
+ name: support-metric-bps
+ type: flag
+ doc: the device accepts 'bps' metric for bw-min, bw-max and burst
+ -
+ name: support-metric-pps
+ type: flag
+ doc: the device accepts 'pps' metric for bw-min, bw-max and burst
+ -
+ name: support-nesting
+ type: flag
+ doc: |
+ the device supports nesting shaper belonging to this scope
+ below 'detached' scoped shapers. Only 'queue' and 'detached'
+ scope and flag 'support-nesting'.
+ -
+ name: support-bw-min
+ type: flag
+ doc: the device supports a minimum guaranteed bw
+ -
+ name: support-bw-max
+ type: flag
+ doc: the device supports maximum bw shaping
+ -
+ name: support-burst
+ type: flag
+ doc: the device supports a maximum burst size
+ -
+ name: support-priority
+ type: flag
+ doc: the device supports priority scheduling
+ -
+ name: support-weight
+ type: flag
+ doc: the device supports weighted round robin scheduling
operations:
list:
@@ -260,3 +306,32 @@ operations:
reply:
attributes:
- handle
+
+ -
+ name: cap-get
+ doc: |
+ Get / Dump the shaper capabilities supported by the given device
+ attribute-set: capabilities
+
+ do:
+ request:
+ attributes:
+ - ifindex
+ - scope
+ reply:
+ attributes: &cap-attrs
+ - support-metric-bps
+ - support-metric-pps
+ - support-nesting
+ - support-bw-min
+ - support-bw-max
+ - support-burst
+ - support-priority
+ - support-weight
+
+ dump:
+ request:
+ attributes:
+ - ifindex
+ reply:
+ attributes: *cap-attrs
diff --git a/include/uapi/linux/net_shaper.h b/include/uapi/linux/net_shaper.h
index ab3d4eb0e1ab..12d16d30472e 100644
--- a/include/uapi/linux/net_shaper.h
+++ b/include/uapi/linux/net_shaper.h
@@ -61,11 +61,28 @@ enum {
NET_SHAPER_A_MAX = (__NET_SHAPER_A_MAX - 1)
};
+enum {
+ NET_SHAPER_A_CAPABILITIES_IFINDEX = 1,
+ NET_SHAPER_A_CAPABILITIES_SCOPE,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_BPS,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_PPS,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_NESTING,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MIN,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MAX,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_BURST,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_PRIORITY,
+ NET_SHAPER_A_CAPABILITIES_SUPPORT_WEIGHT,
+
+ __NET_SHAPER_A_CAPABILITIES_MAX,
+ NET_SHAPER_A_CAPABILITIES_MAX = (__NET_SHAPER_A_CAPABILITIES_MAX - 1)
+};
+
enum {
NET_SHAPER_CMD_GET = 1,
NET_SHAPER_CMD_SET,
NET_SHAPER_CMD_DELETE,
NET_SHAPER_CMD_GROUP,
+ NET_SHAPER_CMD_CAP_GET,
__NET_SHAPER_CMD_MAX,
NET_SHAPER_CMD_MAX = (__NET_SHAPER_CMD_MAX - 1)
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 87db541de646..5363e10606e2 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -837,6 +837,17 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
return ret;
}
+int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ return 0;
+}
+
+int net_shaper_nl_cap_get_dumpit(struct sk_buff *skb,
+ struct netlink_callback *cb)
+{
+ return 0;
+}
+
void dev_shaper_flush(struct net_device *dev)
{
struct xarray *xa = __sc_container(dev);
diff --git a/net/shaper/shaper_nl_gen.c b/net/shaper/shaper_nl_gen.c
index b444d1ff55a1..cc98d58c0862 100644
--- a/net/shaper/shaper_nl_gen.c
+++ b/net/shaper/shaper_nl_gen.c
@@ -67,6 +67,17 @@ static const struct nla_policy net_shaper_group_nl_policy[NET_SHAPER_A_OUTPUT +
[NET_SHAPER_A_OUTPUT] = NLA_POLICY_NESTED(net_shaper_ns_output_info_nl_policy),
};
+/* NET_SHAPER_CMD_CAP_GET - do */
+static const struct nla_policy net_shaper_cap_get_do_nl_policy[NET_SHAPER_A_CAPABILITIES_SCOPE + 1] = {
+ [NET_SHAPER_A_CAPABILITIES_IFINDEX] = { .type = NLA_U32, },
+ [NET_SHAPER_A_CAPABILITIES_SCOPE] = NLA_POLICY_MAX(NLA_U32, 4),
+};
+
+/* NET_SHAPER_CMD_CAP_GET - dump */
+static const struct nla_policy net_shaper_cap_get_dump_nl_policy[NET_SHAPER_A_CAPABILITIES_IFINDEX + 1] = {
+ [NET_SHAPER_A_CAPABILITIES_IFINDEX] = { .type = NLA_U32, },
+};
+
/* Ops table for net_shaper */
static const struct genl_split_ops net_shaper_nl_ops[] = {
{
@@ -104,6 +115,20 @@ static const struct genl_split_ops net_shaper_nl_ops[] = {
.maxattr = NET_SHAPER_A_OUTPUT,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
+ {
+ .cmd = NET_SHAPER_CMD_CAP_GET,
+ .doit = net_shaper_nl_cap_get_doit,
+ .policy = net_shaper_cap_get_do_nl_policy,
+ .maxattr = NET_SHAPER_A_CAPABILITIES_SCOPE,
+ .flags = GENL_CMD_CAP_DO,
+ },
+ {
+ .cmd = NET_SHAPER_CMD_CAP_GET,
+ .dumpit = net_shaper_nl_cap_get_dumpit,
+ .policy = net_shaper_cap_get_dump_nl_policy,
+ .maxattr = NET_SHAPER_A_CAPABILITIES_IFINDEX,
+ .flags = GENL_CMD_CAP_DUMP,
+ },
};
struct genl_family net_shaper_nl_family __ro_after_init = {
diff --git a/net/shaper/shaper_nl_gen.h b/net/shaper/shaper_nl_gen.h
index 00cee4efd21e..97fbd4502364 100644
--- a/net/shaper/shaper_nl_gen.h
+++ b/net/shaper/shaper_nl_gen.h
@@ -21,6 +21,9 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info);
int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info);
int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info);
+int net_shaper_nl_cap_get_dumpit(struct sk_buff *skb,
+ struct netlink_callback *cb);
extern struct genl_family net_shaper_nl_family;
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 07/12] net: shaper: implement introspection support
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (5 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 06/12] netlink: spec: add shaper introspection support Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 08/12] testing: net-drv: add basic shaper test Paolo Abeni
` (5 subsequent siblings)
12 siblings, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
The netlink op is a simple wrapper around the device callback.
Extend the existing fetch_dev() helper adding an attribute argument
for the requested device. Reuse such helper in the newly implemented
operation.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
RFC v2 -> RFC v3:
- dev_put() -> netdev_put()
---
include/net/net_shaper.h | 11 ++++
net/shaper/shaper.c | 108 +++++++++++++++++++++++++++++++++++----
2 files changed, 109 insertions(+), 10 deletions(-)
diff --git a/include/net/net_shaper.h b/include/net/net_shaper.h
index 8cd65d727e52..402f9d8476ee 100644
--- a/include/net/net_shaper.h
+++ b/include/net/net_shaper.h
@@ -103,6 +103,17 @@ struct net_shaper_ops {
*/
int (*delete)(struct net_device *dev, u32 handle,
struct netlink_ext_ack *extack);
+
+ /**
+ * @capabilities: get the shaper features supported by the NIC
+ *
+ * Fills the bitmask @cap with the supported cababilites for the
+ * specified @scope and device @dev.
+ *
+ * Returns 0 on success or a negative error value otherwise.
+ */
+ int (*capabilities)(struct net_device *dev,
+ enum net_shaper_scope scope, unsigned long *cap);
};
#define NET_SHAPER_SCOPE_SHIFT 26
diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
index 5363e10606e2..f9c53c8b7ce7 100644
--- a/net/shaper/shaper.c
+++ b/net/shaper/shaper.c
@@ -118,16 +118,17 @@ net_shaper_fill_one(struct sk_buff *msg, struct net_shaper_info *shaper,
/* On success sets pdev to the relevant device and acquires a reference
* to it
*/
-static int fetch_dev(const struct genl_info *info, struct net_device **pdev)
+static int fetch_dev(const struct genl_info *info, int type,
+ struct net_device **pdev)
{
struct net *ns = genl_info_net(info);
struct net_device *dev;
int ifindex;
- if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_IFINDEX))
+ if (GENL_REQ_ATTR_CHECK(info, type))
return -EINVAL;
- ifindex = nla_get_u32(info->attrs[NET_SHAPER_A_IFINDEX]);
+ ifindex = nla_get_u32(info->attrs[type]);
dev = dev_get_by_index(ns, ifindex);
if (!dev) {
GENL_SET_ERR_MSG_FMT(info, "device %d not found", ifindex);
@@ -425,7 +426,7 @@ int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
u32 handle;
int ret;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -476,7 +477,7 @@ int net_shaper_nl_get_dumpit(struct sk_buff *skb,
unsigned long handle;
int ret;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -543,7 +544,7 @@ int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_SHAPER))
return -EINVAL;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -617,7 +618,7 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
return -EINVAL;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -786,7 +787,7 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_OUTPUT))
return -EINVAL;
- ret = fetch_dev(info, &dev);
+ ret = fetch_dev(info, NET_SHAPER_A_IFINDEX, &dev);
if (ret)
return ret;
@@ -837,15 +838,102 @@ int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
return ret;
}
-int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info)
+static int
+net_shaper_cap_fill_one(struct sk_buff *msg, unsigned long flags,
+ const struct genl_info *info)
{
+ unsigned long cur;
+ void *hdr;
+
+ hdr = genlmsg_iput(msg, info);
+ if (!hdr)
+ return -EMSGSIZE;
+
+ for (cur = NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_BPS;
+ cur <= NET_SHAPER_A_CAPABILITIES_MAX; ++cur) {
+ if (flags & BIT(cur) && nla_put_flag(msg, cur))
+ goto nla_put_failure;
+ }
+
+ genlmsg_end(msg, hdr);
+
return 0;
+
+nla_put_failure:
+ genlmsg_cancel(msg, hdr);
+ return -EMSGSIZE;
+}
+
+int net_shaper_nl_cap_get_doit(struct sk_buff *skb, struct genl_info *info)
+{
+ const struct net_shaper_ops *ops;
+ enum net_shaper_scope scope;
+ struct net_device *dev;
+ struct sk_buff *msg;
+ unsigned long flags;
+ int ret;
+
+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_CAPABILITIES_SCOPE))
+ return -EINVAL;
+
+ ret = fetch_dev(info, NET_SHAPER_A_CAPABILITIES_IFINDEX, &dev);
+ if (ret)
+ return ret;
+
+ scope = nla_get_u32(info->attrs[NET_SHAPER_A_CAPABILITIES_SCOPE]);
+ ops = dev->netdev_ops->net_shaper_ops;
+ ret = ops->capabilities(dev, scope, &flags);
+ if (ret)
+ goto put;
+
+ msg = genlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg)
+ goto put;
+
+ ret = net_shaper_cap_fill_one(msg, flags, info);
+ if (ret)
+ goto free_msg;
+
+ ret = genlmsg_reply(msg, info);
+ if (ret)
+ goto free_msg;
+
+put:
+ netdev_put(dev, NULL);
+ return ret;
+
+free_msg:
+ nlmsg_free(msg);
+ goto put;
}
int net_shaper_nl_cap_get_dumpit(struct sk_buff *skb,
struct netlink_callback *cb)
{
- return 0;
+ const struct genl_info *info = genl_info_dump(cb);
+ const struct net_shaper_ops *ops;
+ enum net_shaper_scope scope;
+ struct net_device *dev;
+ unsigned long flags;
+ int ret;
+
+ ret = fetch_dev(info, NET_SHAPER_A_CAPABILITIES_IFINDEX, &dev);
+ if (ret)
+ return ret;
+
+ ops = dev->netdev_ops->net_shaper_ops;
+ for (scope = 0; scope <= NET_SHAPER_SCOPE_MAX; ++scope) {
+ if (ops->capabilities(dev, scope, &flags))
+ continue;
+
+ ret = net_shaper_cap_fill_one(skb, flags, info);
+ if (ret)
+ goto put;
+ }
+
+put:
+ dev_put(dev);
+ return ret;
}
void dev_shaper_flush(struct net_device *dev)
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (6 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 07/12] net: shaper: implement " Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-07-31 7:52 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 09/12] virtchnl: support queue rate limit and quanta size configuration Paolo Abeni
` (4 subsequent siblings)
12 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Leverage a basic/dummy netdevsim implementation to do functional
coverage for NL interface.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
rfc v1 -> v2:
- added more test-cases WRT nesting and grouping
---
drivers/net/Kconfig | 1 +
drivers/net/netdevsim/netdev.c | 37 +++
tools/testing/selftests/drivers/net/Makefile | 1 +
tools/testing/selftests/drivers/net/shaper.py | 267 ++++++++++++++++++
.../testing/selftests/net/lib/py/__init__.py | 1 +
tools/testing/selftests/net/lib/py/ynl.py | 5 +
6 files changed, 312 insertions(+)
create mode 100755 tools/testing/selftests/drivers/net/shaper.py
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 9920b3a68ed1..1fd5acdc73c6 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -641,6 +641,7 @@ config NETDEVSIM
depends on PTP_1588_CLOCK_MOCK || PTP_1588_CLOCK_MOCK=n
select NET_DEVLINK
select PAGE_POOL
+ select NET_SHAPER
help
This driver is a developer testing tool and software model that can
be used to test various control path networking APIs, especially
diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c
index 017a6102be0a..ab9c3d5055e7 100644
--- a/drivers/net/netdevsim/netdev.c
+++ b/drivers/net/netdevsim/netdev.c
@@ -22,6 +22,7 @@
#include <net/netdev_queues.h>
#include <net/page_pool/helpers.h>
#include <net/netlink.h>
+#include <net/net_shaper.h>
#include <net/pkt_cls.h>
#include <net/rtnetlink.h>
#include <net/udp_tunnel.h>
@@ -475,6 +476,41 @@ static int nsim_stop(struct net_device *dev)
return 0;
}
+static int nsim_shaper_set(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack)
+{
+ return 0;
+}
+
+static int nsim_shaper_del(struct net_device *dev, u32 handles,
+ struct netlink_ext_ack *extack)
+{
+ return 0;
+}
+
+static int nsim_shaper_group(struct net_device *dev, int nr_inputs,
+ const struct net_shaper_info *inputs,
+ const struct net_shaper_info *output,
+ struct netlink_ext_ack *extack)
+{
+ return 0;
+}
+
+static int nsim_shaper_cap(struct net_device *dev, enum net_shaper_scope scope,
+ unsigned long *flags)
+{
+ *flags = ULONG_MAX;
+ return 0;
+}
+
+static const struct net_shaper_ops nsim_shaper_ops = {
+ .set = nsim_shaper_set,
+ .delete = nsim_shaper_del,
+ .group = nsim_shaper_group,
+ .capabilities = nsim_shaper_cap,
+};
+
static const struct net_device_ops nsim_netdev_ops = {
.ndo_start_xmit = nsim_start_xmit,
.ndo_set_rx_mode = nsim_set_rx_mode,
@@ -496,6 +532,7 @@ static const struct net_device_ops nsim_netdev_ops = {
.ndo_bpf = nsim_bpf,
.ndo_open = nsim_open,
.ndo_stop = nsim_stop,
+ .net_shaper_ops = &nsim_shaper_ops,
};
static const struct net_device_ops nsim_vf_netdev_ops = {
diff --git a/tools/testing/selftests/drivers/net/Makefile b/tools/testing/selftests/drivers/net/Makefile
index e54f382bcb02..432306f11664 100644
--- a/tools/testing/selftests/drivers/net/Makefile
+++ b/tools/testing/selftests/drivers/net/Makefile
@@ -6,6 +6,7 @@ TEST_PROGS := \
ping.py \
queues.py \
stats.py \
+ shaper.py
# end of TEST_PROGS
include ../../lib.mk
diff --git a/tools/testing/selftests/drivers/net/shaper.py b/tools/testing/selftests/drivers/net/shaper.py
new file mode 100755
index 000000000000..ed34573bb4f6
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/shaper.py
@@ -0,0 +1,267 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+from lib.py import ksft_run, ksft_exit, ksft_eq, ksft_true, KsftSkipEx
+from lib.py import ShaperFamily
+from lib.py import NetDrvEnv
+from lib.py import NlError
+from lib.py import cmd
+import glob
+import sys
+
+def get_shapers(cfg, nl_shaper) -> None:
+ try:
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ except NlError as e:
+ if e.error == 95:
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+
+ # default configuration, no shapers configured
+ ksft_eq(len(shapers), 0)
+
+def get_caps(cfg, nl_shaper) -> None:
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex}, dump=True)
+ except NlError as e:
+ if e.error == 95:
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+
+ # each device implementing shaper support must support some
+ # features in at least a scope
+ ksft_true(len(caps)> 0)
+
+def set_qshapers(cfg, nl_shaper) -> None:
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'queue'})
+ except NlError as e:
+ if e.error == 95:
+ cfg.queues = False;
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-bw-max' in caps or not 'support-metric-bps' in caps:
+ raise KsftSkipEx("device does not support queue scope shapers with bw_max and metric bps")
+
+ nl_shaper.set({'ifindex': cfg.ifindex,
+ 'shaper': { 'handle': { 'scope': 'queue', 'id': 1 }, 'metric': 'bps', 'bw-max': 10000 }})
+ nl_shaper.set({'ifindex': cfg.ifindex,
+ 'shaper': { 'handle': { 'scope': 'queue', 'id': 2 }, 'metric': 'bps', 'bw-max': 20000 }})
+
+ # querying a specific shaper not yet configured must fail
+ raised = False
+ try:
+ shaper_q0 = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 0}})
+ except (NlError):
+ raised = True
+ ksft_eq(raised, True)
+
+ shaper_q1 = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 1 }})
+ ksft_eq(shaper_q1, { 'parent': { 'scope': 'netdev', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 1 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 10000,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 0 })
+
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(shapers, [{'parent': { 'scope': 'netdev', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 1 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 10000,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 0 },
+ {'parent': { 'scope': 'netdev', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 2 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 20000,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 0 }])
+
+def del_qshapers(cfg, nl_shaper) -> None:
+ if not cfg.queues:
+ raise KsftSkipEx("queue shapers not supported by device, skipping delete")
+
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 2}})
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 1}})
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(len(shapers), 0)
+
+def set_nshapers(cfg, nl_shaper) -> None:
+ # check required features
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'netdev'})
+ except NlError as e:
+ if e.error == 95:
+ cfg.netdev = False;
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-bw-max' in caps or not 'support-metric-bps' in caps:
+ raise KsftSkipEx("device does not support nested netdev scope shapers with weight")
+
+ nl_shaper.set({'ifindex': cfg.ifindex,
+ 'shaper': { 'handle': { 'scope': 'netdev', 'id': 0 }, 'bw-max': 100000 }})
+
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(shapers, [{'parent': { 'scope': 'port', 'id': 0 },
+ 'handle': { 'scope': 'netdev', 'id': 0 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 100000,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 0 }])
+
+def del_nshapers(cfg, nl_shaper) -> None:
+ if not cfg.netdev:
+ raise KsftSkipEx("netdev shaper not supported by device, skipping delete")
+
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'netdev'}})
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(len(shapers), 0)
+
+def basic_groups(cfg, nl_shaper) -> None:
+ if not cfg.netdev:
+ raise KsftSkipEx("netdev shaper not supported by the device")
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'queue'})
+ except NlError as e:
+ if e.error == 95:
+ cfg.queues = False;
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-weight' in caps:
+ raise KsftSkipEx("device does not support queue scope shapers with weight")
+
+ output_handle = nl_shaper.group({'ifindex': cfg.ifindex,
+ 'inputs':[{ 'handle': { 'scope': 'queue', 'id': 1 },'weight': 1 },
+ { 'handle': { 'scope': 'queue', 'id': 2 }, 'weight': 2 }],
+ 'output': { 'handle': {'scope':'netdev'}, 'metric': 'bps', 'bw-max': 10000}})
+ output_id = output_handle['handle']['id']
+
+ shaper = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 1 }})
+ ksft_eq(shaper, {'parent': { 'scope': 'netdev', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 1 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 0,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 1 })
+
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 2}})
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 1}})
+
+ # deleting all the inputs shaper does not affect the output one
+ # when the latter has 'netdev' scope
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(len(shapers), 1)
+
+ nl_shaper.delete({'ifindex': cfg.ifindex, 'handle': { 'scope': 'netdev'}})
+
+def qgroups(cfg, nl_shaper) -> None:
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'detached'})
+ except NlError as e:
+ if e.error == 95:
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-bw-max' in caps or not 'support-metric-bps' in caps:
+ raise KsftSkipEx("device does not support detached scope shapers with bw_max and metric bps")
+ try:
+ caps = nl_shaper.cap_get({'ifindex': cfg.ifindex, 'scope':'queue'})
+ except NlError as e:
+ if e.error == 95:
+ raise KsftSkipEx("shapers not supported by the device")
+ raise
+ if not 'support-nesting' in caps or not 'support-weight' in caps or not 'support-metric-bps' in caps:
+ raise KsftSkipEx("device does not support nested queue scope shapers with weight")
+
+ output_handle = nl_shaper.group({'ifindex': cfg.ifindex,
+ 'inputs':[{ 'handle': { 'scope': 'queue', 'id': 1 }, 'metric': 'bps', 'weight': 3 },
+ { 'handle': { 'scope': 'queue', 'id': 2 }, 'metric': 'bps', 'weight': 2 }],
+ 'output': { 'handle': {'scope':'detached'}, 'metric': 'bps', 'bw-max': 10000}})
+ output_id = output_handle['handle']['id']
+
+ shaper = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 1 }})
+ ksft_eq(shaper, {'parent': { 'scope': 'detached', 'id': output_id },
+ 'handle': { 'scope': 'queue', 'id': 1 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 0,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 3 })
+
+ # grouping to a specified, not existing detached scope shaper must fail
+ raised = False
+ try:
+ nl_shaper.group({'ifindex': cfg.ifindex,
+ 'inputs':[ { 'handle': { 'scope': 'queue', 'id': 3 }, 'metric': 'bps', 'weight': 3 }],
+ 'output': { 'handle': {'scope':'detached', 'id': output_id + 1 }, 'metric': 'bps', 'bw-max': 10000}})
+ except (NlError):
+ raised = True
+ ksft_eq(raised, True)
+
+ output_handle = nl_shaper.group({'ifindex': cfg.ifindex,
+ 'inputs':[ { 'handle': { 'scope': 'queue', 'id': 3 }, 'metric': 'bps', 'weight': 4 }],
+ 'output': { 'handle': {'scope':'detached', 'id': output_id} }})
+
+ shaper = nl_shaper.get({'ifindex': cfg.ifindex, 'handle': { 'scope': 'queue', 'id': 3 }})
+ ksft_eq(shaper, {'parent': { 'scope': 'detached', 'id': 0 },
+ 'handle': { 'scope': 'queue', 'id': 3 },
+ 'metric': 'bps',
+ 'bw-min': 0,
+ 'bw-max': 0,
+ 'burst': 0,
+ 'priority': 0,
+ 'weight': 4 })
+
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 2}})
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 1}})
+
+ # deleting a non empty group mast fail
+ raised = False
+ try:
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'detached', 'id': output_id }})
+ except (NlError):
+ raised = True
+ ksft_eq(raised, True)
+ nl_shaper.delete({'ifindex': cfg.ifindex,
+ 'handle': { 'scope': 'queue', 'id': 3}})
+
+ # the detached scope shaper deletion is implicit
+ shapers = nl_shaper.get({'ifindex': cfg.ifindex}, dump=True)
+ ksft_eq(len(shapers), 0)
+
+def main() -> None:
+ with NetDrvEnv(__file__, queue_count=4) as cfg:
+ cfg.queues = True
+ cfg.netdev = True
+ ksft_run([get_shapers,
+ get_caps,
+ set_qshapers,
+ del_qshapers,
+ set_nshapers,
+ del_nshapers,
+ basic_groups,
+ qgroups], args=(cfg, ShaperFamily()))
+ ksft_exit()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/testing/selftests/net/lib/py/__init__.py b/tools/testing/selftests/net/lib/py/__init__.py
index b6d498d125fe..ef1aa52f4761 100644
--- a/tools/testing/selftests/net/lib/py/__init__.py
+++ b/tools/testing/selftests/net/lib/py/__init__.py
@@ -6,3 +6,4 @@ from .netns import NetNS
from .nsim import *
from .utils import *
from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily
+from .ynl import ShaperFamily
diff --git a/tools/testing/selftests/net/lib/py/ynl.py b/tools/testing/selftests/net/lib/py/ynl.py
index 1ace58370c06..42e825905ade 100644
--- a/tools/testing/selftests/net/lib/py/ynl.py
+++ b/tools/testing/selftests/net/lib/py/ynl.py
@@ -47,3 +47,8 @@ class NetdevFamily(YnlFamily):
def __init__(self):
super().__init__((SPEC_PATH / Path('netdev.yaml')).as_posix(),
schema='')
+
+class ShaperFamily(YnlFamily):
+ def __init__(self):
+ super().__init__((SPEC_PATH / Path('shaper.yaml')).as_posix(),
+ schema='')
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 09/12] virtchnl: support queue rate limit and quanta size configuration
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (7 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 08/12] testing: net-drv: add basic shaper test Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 10/12] ice: Support VF " Paolo Abeni
` (3 subsequent siblings)
12 siblings, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
From: Wenjun Wu <wenjun1.wu@intel.com>
This patch adds new virtchnl opcodes and structures for rate limit
and quanta size configuration, which include:
1. VIRTCHNL_OP_CONFIG_QUEUE_BW, to configure max bandwidth for each
VF per queue.
2. VIRTCHNL_OP_CONFIG_QUANTA, to configure quanta size per queue.
3. VIRTCHNL_OP_GET_QOS_CAPS, VF queries current QoS configuration, such
as enabled TCs, arbiter type, up2tc and bandwidth of VSI node. The
configuration is previously set by DCB and PF, and now is the potential
QoS capability of VF. VF can take it as reference to configure queue TC
mapping.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
---
include/linux/avf/virtchnl.h | 119 +++++++++++++++++++++++++++++++++++
1 file changed, 119 insertions(+)
diff --git a/include/linux/avf/virtchnl.h b/include/linux/avf/virtchnl.h
index 8e177b67e82f..8ff31845f685 100644
--- a/include/linux/avf/virtchnl.h
+++ b/include/linux/avf/virtchnl.h
@@ -89,6 +89,9 @@ enum virtchnl_rx_hsplit {
VIRTCHNL_RX_HSPLIT_SPLIT_SCTP = 8,
};
+enum virtchnl_bw_limit_type {
+ VIRTCHNL_BW_SHAPER = 0,
+};
/* END GENERIC DEFINES */
/* Opcodes for VF-PF communication. These are placed in the v_opcode field
@@ -151,6 +154,11 @@ enum virtchnl_ops {
VIRTCHNL_OP_DISABLE_VLAN_STRIPPING_V2 = 55,
VIRTCHNL_OP_ENABLE_VLAN_INSERTION_V2 = 56,
VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2 = 57,
+ /* opcode 57 - 65 are reserved */
+ VIRTCHNL_OP_GET_QOS_CAPS = 66,
+ /* opcode 68 through 111 are reserved */
+ VIRTCHNL_OP_CONFIG_QUEUE_BW = 112,
+ VIRTCHNL_OP_CONFIG_QUANTA = 113,
VIRTCHNL_OP_MAX,
};
@@ -260,6 +268,7 @@ VIRTCHNL_CHECK_STRUCT_LEN(16, virtchnl_vsi_resource);
#define VIRTCHNL_VF_OFFLOAD_RX_FLEX_DESC BIT(26)
#define VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF BIT(27)
#define VIRTCHNL_VF_OFFLOAD_FDIR_PF BIT(28)
+#define VIRTCHNL_VF_OFFLOAD_QOS BIT(29)
#define VF_BASE_MODE_OFFLOADS (VIRTCHNL_VF_OFFLOAD_L2 | \
VIRTCHNL_VF_OFFLOAD_VLAN | \
@@ -1405,6 +1414,85 @@ struct virtchnl_fdir_del {
VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
+struct virtchnl_shaper_bw {
+ /* Unit is Kbps */
+ u32 committed;
+ u32 peak;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_shaper_bw);
+
+/* VIRTCHNL_OP_GET_QOS_CAPS
+ * VF sends this message to get its QoS Caps, such as
+ * TC number, Arbiter and Bandwidth.
+ */
+struct virtchnl_qos_cap_elem {
+ u8 tc_num;
+ u8 tc_prio;
+#define VIRTCHNL_ABITER_STRICT 0
+#define VIRTCHNL_ABITER_ETS 2
+ u8 arbiter;
+#define VIRTCHNL_STRICT_WEIGHT 1
+ u8 weight;
+ enum virtchnl_bw_limit_type type;
+ union {
+ struct virtchnl_shaper_bw shaper;
+ u8 pad2[32];
+ };
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(40, virtchnl_qos_cap_elem);
+
+struct virtchnl_qos_cap_list {
+ u16 vsi_id;
+ u16 num_elem;
+ struct virtchnl_qos_cap_elem cap[];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_qos_cap_list);
+#define virtchnl_qos_cap_list_LEGACY_SIZEOF 44
+
+/* VIRTCHNL_OP_CONFIG_QUEUE_BW */
+struct virtchnl_queue_bw {
+ u16 queue_id;
+ u8 tc;
+ u8 pad;
+ struct virtchnl_shaper_bw shaper;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_queue_bw);
+
+struct virtchnl_queues_bw_cfg {
+ u16 vsi_id;
+ u16 num_queues;
+ struct virtchnl_queue_bw cfg[];
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(4, virtchnl_queues_bw_cfg);
+#define virtchnl_queues_bw_cfg_LEGACY_SIZEOF 16
+
+enum virtchnl_queue_type {
+ VIRTCHNL_QUEUE_TYPE_TX = 0,
+ VIRTCHNL_QUEUE_TYPE_RX = 1,
+};
+
+/* structure to specify a chunk of contiguous queues */
+struct virtchnl_queue_chunk {
+ /* see enum virtchnl_queue_type */
+ s32 type;
+ u16 start_queue_id;
+ u16 num_queues;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(8, virtchnl_queue_chunk);
+
+struct virtchnl_quanta_cfg {
+ u16 quanta_size;
+ struct virtchnl_queue_chunk queue_select;
+};
+
+VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_quanta_cfg);
+
#define __vss_byone(p, member, count, old) \
(struct_size(p, member, count) + (old - 1 - struct_size(p, member, 0)))
@@ -1427,6 +1515,8 @@ VIRTCHNL_CHECK_STRUCT_LEN(12, virtchnl_fdir_del);
__vss(virtchnl_vlan_filter_list_v2, __vss_byelem, p, m, c), \
__vss(virtchnl_tc_info, __vss_byelem, p, m, c), \
__vss(virtchnl_rdma_qvlist_info, __vss_byelem, p, m, c), \
+ __vss(virtchnl_qos_cap_list, __vss_byelem, p, m, c), \
+ __vss(virtchnl_queues_bw_cfg, __vss_byelem, p, m, c), \
__vss(virtchnl_rss_key, __vss_byone, p, m, c), \
__vss(virtchnl_rss_lut, __vss_byone, p, m, c))
@@ -1626,6 +1716,35 @@ virtchnl_vc_validate_vf_msg(struct virtchnl_version_info *ver, u32 v_opcode,
case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
valid_len = sizeof(struct virtchnl_vlan_setting);
break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ valid_len = virtchnl_queues_bw_cfg_LEGACY_SIZEOF;
+ if (msglen >= valid_len) {
+ struct virtchnl_queues_bw_cfg *q_bw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+
+ valid_len = virtchnl_struct_size(q_bw, cfg,
+ q_bw->num_queues);
+ if (q_bw->num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ valid_len = sizeof(struct virtchnl_quanta_cfg);
+ if (msglen >= valid_len) {
+ struct virtchnl_quanta_cfg *q_quanta =
+ (struct virtchnl_quanta_cfg *)msg;
+
+ if (q_quanta->quanta_size == 0 ||
+ q_quanta->queue_select.num_queues == 0) {
+ err_msg_format = true;
+ break;
+ }
+ }
+ break;
/* These are always errors coming from the VF. */
case VIRTCHNL_OP_EVENT:
case VIRTCHNL_OP_UNKNOWN:
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 10/12] ice: Support VF queue rate limit and quanta size configuration
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (8 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 09/12] virtchnl: support queue rate limit and quanta size configuration Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 11/12] iavf: Add net_shaper_ops support Paolo Abeni
` (2 subsequent siblings)
12 siblings, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
From: Wenjun Wu <wenjun1.wu@intel.com>
Add support to configure VF queue rate limit and quanta size.
For quanta size configuration, the quanta profiles are divided evenly
by PF numbers. For each port, the first quanta profile is reserved for
default. When VF is asked to set queue quanta size, PF will search for
an available profile, change the fields and assigned this profile to the
queue.
Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 2 +
drivers/net/ethernet/intel/ice/ice_base.c | 2 +
drivers/net/ethernet/intel/ice/ice_common.c | 21 ++
.../net/ethernet/intel/ice/ice_hw_autogen.h | 8 +
drivers/net/ethernet/intel/ice/ice_txrx.h | 1 +
drivers/net/ethernet/intel/ice/ice_type.h | 1 +
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 +
drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 +
.../intel/ice/ice_virtchnl_allowlist.c | 6 +
10 files changed, 393 insertions(+)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 99a75a59078e..ddb31ad284a4 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -659,6 +659,8 @@ struct ice_pf {
struct ice_agg_node vf_agg_node[ICE_MAX_VF_AGG_NODES];
struct ice_dplls dplls;
struct device *hwmon_dev;
+
+ u8 num_quanta_prof_used;
};
extern struct workqueue_struct *ice_lag_wq;
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
index 5d396c1a7731..54cd9fe27d77 100644
--- a/drivers/net/ethernet/intel/ice/ice_base.c
+++ b/drivers/net/ethernet/intel/ice/ice_base.c
@@ -349,6 +349,8 @@ ice_setup_tx_ctx(struct ice_tx_ring *ring, struct ice_tlan_ctx *tlan_ctx, u16 pf
break;
}
+ tlan_ctx->quanta_prof_idx = ring->quanta_prof_id;
+
tlan_ctx->tso_ena = ICE_TX_LEGACY;
tlan_ctx->tso_qnum = pf_q;
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index 009716a12a26..b22e71dc59d4 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -2436,6 +2436,25 @@ ice_parse_func_caps(struct ice_hw *hw, struct ice_hw_func_caps *func_p,
ice_recalc_port_limited_caps(hw, &func_p->common_cap);
}
+/**
+ * ice_func_id_to_logical_id - map from function id to logical pf id
+ * @active_function_bitmap: active function bitmap
+ * @pf_id: function number of device
+ *
+ * Return: logical PF ID.
+ */
+static int ice_func_id_to_logical_id(u32 active_function_bitmap, u8 pf_id)
+{
+ u8 logical_id = 0;
+ u8 i;
+
+ for (i = 0; i < pf_id; i++)
+ if (active_function_bitmap & BIT(i))
+ logical_id++;
+
+ return logical_id;
+}
+
/**
* ice_parse_valid_functions_cap - Parse ICE_AQC_CAPS_VALID_FUNCTIONS caps
* @hw: pointer to the HW struct
@@ -2453,6 +2472,8 @@ ice_parse_valid_functions_cap(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p,
dev_p->num_funcs = hweight32(number);
ice_debug(hw, ICE_DBG_INIT, "dev caps: num_funcs = %d\n",
dev_p->num_funcs);
+
+ hw->logical_pf_id = ice_func_id_to_logical_id(number, hw->pf_id);
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
index 91cbae1eec89..af9302f0e376 100644
--- a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
+++ b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
@@ -6,6 +6,14 @@
#ifndef _ICE_HW_AUTOGEN_H_
#define _ICE_HW_AUTOGEN_H_
+#define GLCOMM_QUANTA_PROF(_i) (0x002D2D68 + ((_i) * 4))
+#define GLCOMM_QUANTA_PROF_MAX_INDEX 15
+#define GLCOMM_QUANTA_PROF_QUANTA_SIZE_S 0
+#define GLCOMM_QUANTA_PROF_QUANTA_SIZE_M ICE_M(0x3FFF, 0)
+#define GLCOMM_QUANTA_PROF_MAX_CMD_S 16
+#define GLCOMM_QUANTA_PROF_MAX_CMD_M ICE_M(0xFF, 16)
+#define GLCOMM_QUANTA_PROF_MAX_DESC_S 24
+#define GLCOMM_QUANTA_PROF_MAX_DESC_M ICE_M(0x3F, 24)
#define QTX_COMM_DBELL(_DBQM) (0x002C0000 + ((_DBQM) * 4))
#define QTX_COMM_HEAD(_DBQM) (0x000E0000 + ((_DBQM) * 4))
#define QTX_COMM_HEAD_HEAD_S 0
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.h b/drivers/net/ethernet/intel/ice/ice_txrx.h
index feba314a3fe4..ea2fae9035b5 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.h
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.h
@@ -406,6 +406,7 @@ struct ice_tx_ring {
#define ICE_TX_FLAGS_RING_VLAN_L2TAG2 BIT(2)
u8 flags;
u8 dcb_tc; /* Traffic class of ring */
+ u16 quanta_prof_id;
} ____cacheline_internodealigned_in_smp;
static inline bool ice_ring_uses_build_skb(struct ice_rx_ring *ring)
diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h
index 96037bef3e78..cad4d68b19dc 100644
--- a/drivers/net/ethernet/intel/ice/ice_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_type.h
@@ -903,6 +903,7 @@ struct ice_hw {
u8 revision_id;
u8 pf_id; /* device profile info */
+ u8 logical_pf_id;
u16 max_burst_size; /* driver sets this value */
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
index fec16919ec19..becf5a009b4e 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
@@ -52,6 +52,13 @@ struct ice_mdd_vf_events {
u16 last_printed;
};
+struct ice_vf_qs_bw {
+ u32 committed;
+ u32 peak;
+ u16 queue_id;
+ u8 tc;
+};
+
/* VF operations */
struct ice_vf_ops {
enum ice_disq_rst_src reset_type;
@@ -132,6 +139,7 @@ struct ice_vf {
struct devlink_port devlink_port;
u16 num_msix; /* num of MSI-X configured on this VF */
+ struct ice_vf_qs_bw qs_bw[ICE_MAX_RSS_QS_PER_VF];
};
/* Flags for controlling behavior of ice_reset_vf */
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl.c b/drivers/net/ethernet/intel/ice/ice_virtchnl.c
index 1c6ce0c4ed4e..4f9dc67741a5 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl.c
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl.c
@@ -491,6 +491,9 @@ static int ice_vc_get_vf_res_msg(struct ice_vf *vf, u8 *msg)
if (vf->driver_caps & VIRTCHNL_VF_OFFLOAD_USO)
vfres->vf_cap_flags |= VIRTCHNL_VF_OFFLOAD_USO;
+ if (vf->driver_caps & VIRTCHNL_VF_OFFLOAD_QOS)
+ vfres->vf_cap_flags |= VIRTCHNL_VF_OFFLOAD_QOS;
+
vfres->num_vsis = 1;
/* Tx and Rx queue are equal for VF */
vfres->num_queue_pairs = vsi->num_txq;
@@ -1030,6 +1033,191 @@ static int ice_vc_config_rss_hfunc(struct ice_vf *vf, u8 *msg)
NULL, 0);
}
+/**
+ * ice_vc_get_qos_caps - Get current QoS caps from PF
+ * @vf: pointer to the VF info
+ *
+ * Get VF's QoS capabilities, such as TC number, arbiter and
+ * bandwidth from PF.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vc_get_qos_caps(struct ice_vf *vf)
+{
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
+ struct virtchnl_qos_cap_list *cap_list = NULL;
+ u8 tc_prio[ICE_MAX_TRAFFIC_CLASS] = { 0 };
+ struct virtchnl_qos_cap_elem *cfg = NULL;
+ struct ice_vsi_ctx *vsi_ctx;
+ struct ice_pf *pf = vf->pf;
+ struct ice_port_info *pi;
+ struct ice_vsi *vsi;
+ u8 numtc, tc;
+ u16 len = 0;
+ int ret, i;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ pi = pf->hw.port_info;
+ numtc = vsi->tc_cfg.numtc;
+
+ vsi_ctx = ice_get_vsi_ctx(pi->hw, vf->lan_vsi_idx);
+ if (!vsi_ctx) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ len = struct_size(cap_list, cap, numtc);
+ cap_list = kzalloc(len, GFP_KERNEL);
+ if (!cap_list) {
+ v_ret = VIRTCHNL_STATUS_ERR_NO_MEMORY;
+ len = 0;
+ goto err;
+ }
+
+ cap_list->vsi_id = vsi->vsi_num;
+ cap_list->num_elem = numtc;
+
+ /* Store the UP2TC configuration from DCB to a user priority bitmap
+ * of each TC. Each element of prio_of_tc represents one TC. Each
+ * bitmap indicates the user priorities belong to this TC.
+ */
+ for (i = 0; i < ICE_MAX_USER_PRIORITY; i++) {
+ tc = pi->qos_cfg.local_dcbx_cfg.etscfg.prio_table[i];
+ tc_prio[tc] |= BIT(i);
+ }
+
+ for (i = 0; i < numtc; i++) {
+ cfg = &cap_list->cap[i];
+ cfg->tc_num = i;
+ cfg->tc_prio = tc_prio[i];
+ cfg->arbiter = pi->qos_cfg.local_dcbx_cfg.etscfg.tsatable[i];
+ cfg->weight = VIRTCHNL_STRICT_WEIGHT;
+ cfg->type = VIRTCHNL_BW_SHAPER;
+ cfg->shaper.committed = vsi_ctx->sched.bw_t_info[i].cir_bw.bw;
+ cfg->shaper.peak = vsi_ctx->sched.bw_t_info[i].eir_bw.bw;
+ }
+
+err:
+ ret = ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_GET_QOS_CAPS, v_ret,
+ (u8 *)cap_list, len);
+ kfree(cap_list);
+ return ret;
+}
+
+/**
+ * ice_vf_cfg_qs_bw - Configure per queue bandwidth
+ * @vf: pointer to the VF info
+ * @num_queues: number of queues to be configured
+ *
+ * Configure per queue bandwidth.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vf_cfg_qs_bw(struct ice_vf *vf, u16 num_queues)
+{
+ struct ice_hw *hw = &vf->pf->hw;
+ struct ice_vsi *vsi;
+ int ret;
+ u16 i;
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi)
+ return -EINVAL;
+
+ for (i = 0; i < num_queues; i++) {
+ u32 p_rate, min_rate;
+ u8 tc;
+
+ p_rate = vf->qs_bw[i].peak;
+ min_rate = vf->qs_bw[i].committed;
+ tc = vf->qs_bw[i].tc;
+ if (p_rate)
+ ret = ice_cfg_q_bw_lmt(hw->port_info, vsi->idx, tc,
+ vf->qs_bw[i].queue_id,
+ ICE_MAX_BW, p_rate);
+ else
+ ret = ice_cfg_q_bw_dflt_lmt(hw->port_info, vsi->idx, tc,
+ vf->qs_bw[i].queue_id,
+ ICE_MAX_BW);
+ if (ret)
+ return ret;
+
+ if (min_rate)
+ ret = ice_cfg_q_bw_lmt(hw->port_info, vsi->idx, tc,
+ vf->qs_bw[i].queue_id,
+ ICE_MIN_BW, min_rate);
+ else
+ ret = ice_cfg_q_bw_dflt_lmt(hw->port_info, vsi->idx, tc,
+ vf->qs_bw[i].queue_id,
+ ICE_MIN_BW);
+
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_vf_cfg_q_quanta_profile - Configure quanta profile
+ * @vf: pointer to the VF info
+ * @quanta_prof_idx: pointer to the quanta profile index
+ * @quanta_size: quanta size to be set
+ *
+ * This function chooses available quanta profile and configures the register.
+ * The quanta profile is evenly divided by the number of device ports, and then
+ * available to the specific PF and VFs. The first profile for each PF is a
+ * reserved default profile. Only quanta size of the rest unused profile can be
+ * modified.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vf_cfg_q_quanta_profile(struct ice_vf *vf, u16 quanta_size,
+ u16 *quanta_prof_idx)
+{
+ const u16 n_desc = calc_quanta_desc(quanta_size);
+ struct ice_hw *hw = &vf->pf->hw;
+ const u16 n_cmd = 2 * n_desc;
+ struct ice_pf *pf = vf->pf;
+ u16 per_pf, begin_id;
+ u8 n_used;
+ u32 reg;
+
+ begin_id = (GLCOMM_QUANTA_PROF_MAX_INDEX + 1) / hw->dev_caps.num_funcs *
+ hw->logical_pf_id;
+
+ if (quanta_size == ICE_DFLT_QUANTA) {
+ *quanta_prof_idx = begin_id;
+ } else {
+ per_pf = (GLCOMM_QUANTA_PROF_MAX_INDEX + 1) /
+ hw->dev_caps.num_funcs;
+ n_used = pf->num_quanta_prof_used;
+ if (n_used < per_pf) {
+ *quanta_prof_idx = begin_id + 1 + n_used;
+ pf->num_quanta_prof_used++;
+ } else {
+ return -EINVAL;
+ }
+ }
+
+ reg = FIELD_PREP(GLCOMM_QUANTA_PROF_QUANTA_SIZE_M, quanta_size) |
+ FIELD_PREP(GLCOMM_QUANTA_PROF_MAX_CMD_M, n_cmd) |
+ FIELD_PREP(GLCOMM_QUANTA_PROF_MAX_DESC_M, n_desc);
+ wr32(hw, GLCOMM_QUANTA_PROF(*quanta_prof_idx), reg);
+
+ return 0;
+}
+
/**
* ice_vc_cfg_promiscuous_mode_msg
* @vf: pointer to the VF info
@@ -1631,6 +1819,139 @@ static int ice_vc_cfg_irq_map_msg(struct ice_vf *vf, u8 *msg)
NULL, 0);
}
+/**
+ * ice_vc_cfg_q_bw - Configure per queue bandwidth
+ * @vf: pointer to the VF info
+ * @msg: pointer to the msg buffer which holds the command descriptor
+ *
+ * Configure VF queues bandwidth.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vc_cfg_q_bw(struct ice_vf *vf, u8 *msg)
+{
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
+ struct virtchnl_queues_bw_cfg *qbw =
+ (struct virtchnl_queues_bw_cfg *)msg;
+ struct ice_vsi *vsi;
+ u16 i;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states) ||
+ !ice_vc_isvalid_vsi_id(vf, qbw->vsi_id)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ if (qbw->num_queues > ICE_MAX_RSS_QS_PER_VF ||
+ qbw->num_queues > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
+ dev_err(ice_pf_to_dev(vf->pf), "VF-%d trying to configure more than allocated number of queues: %d\n",
+ vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ for (i = 0; i < qbw->num_queues; i++) {
+ if (qbw->cfg[i].shaper.peak != 0 && vf->max_tx_rate != 0 &&
+ qbw->cfg[i].shaper.peak > vf->max_tx_rate)
+ dev_warn(ice_pf_to_dev(vf->pf), "The maximum queue %d rate limit configuration may not take effect because the maximum TX rate for VF-%d is %d\n",
+ qbw->cfg[i].queue_id, vf->vf_id, vf->max_tx_rate);
+ if (qbw->cfg[i].shaper.committed != 0 && vf->min_tx_rate != 0 &&
+ qbw->cfg[i].shaper.committed < vf->min_tx_rate)
+ dev_warn(ice_pf_to_dev(vf->pf), "The minimum queue %d rate limit configuration may not take effect because the minimum TX rate for VF-%d is %d\n",
+ qbw->cfg[i].queue_id, vf->vf_id, vf->max_tx_rate);
+ }
+
+ for (i = 0; i < qbw->num_queues; i++) {
+ vf->qs_bw[i].queue_id = qbw->cfg[i].queue_id;
+ vf->qs_bw[i].peak = qbw->cfg[i].shaper.peak;
+ vf->qs_bw[i].committed = qbw->cfg[i].shaper.committed;
+ vf->qs_bw[i].tc = qbw->cfg[i].tc;
+ }
+
+ if (ice_vf_cfg_qs_bw(vf, qbw->num_queues))
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+
+err:
+ /* send the response to the VF */
+ return ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_CONFIG_QUEUE_BW,
+ v_ret, NULL, 0);
+}
+
+/**
+ * ice_vc_cfg_q_quanta - Configure per queue quanta
+ * @vf: pointer to the VF info
+ * @msg: pointer to the msg buffer which holds the command descriptor
+ *
+ * Configure VF queues quanta.
+ *
+ * Return: 0 on success or negative error value.
+ */
+static int ice_vc_cfg_q_quanta(struct ice_vf *vf, u8 *msg)
+{
+ enum virtchnl_status_code v_ret = VIRTCHNL_STATUS_SUCCESS;
+ u16 quanta_prof_id, quanta_size, start_qid, end_qid, i;
+ struct virtchnl_quanta_cfg *qquanta =
+ (struct virtchnl_quanta_cfg *)msg;
+ struct ice_vsi *vsi;
+ int ret;
+
+ if (!test_bit(ICE_VF_STATE_ACTIVE, vf->vf_states)) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ vsi = ice_get_vf_vsi(vf);
+ if (!vsi) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ end_qid = qquanta->queue_select.start_queue_id +
+ qquanta->queue_select.num_queues;
+ if (end_qid > ICE_MAX_RSS_QS_PER_VF ||
+ end_qid > min_t(u16, vsi->alloc_txq, vsi->alloc_rxq)) {
+ dev_err(ice_pf_to_dev(vf->pf), "VF-%d trying to configure more than allocated number of queues: %d\n",
+ vf->vf_id, min_t(u16, vsi->alloc_txq, vsi->alloc_rxq));
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ quanta_size = qquanta->quanta_size;
+ if (quanta_size > ICE_MAX_QUANTA_SIZE ||
+ quanta_size < ICE_MIN_QUANTA_SIZE) {
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ if (quanta_size % 64) {
+ dev_err(ice_pf_to_dev(vf->pf), "quanta size should be the product of 64\n");
+ v_ret = VIRTCHNL_STATUS_ERR_PARAM;
+ goto err;
+ }
+
+ ret = ice_vf_cfg_q_quanta_profile(vf, quanta_size,
+ &quanta_prof_id);
+ if (ret) {
+ v_ret = VIRTCHNL_STATUS_ERR_NOT_SUPPORTED;
+ goto err;
+ }
+
+ start_qid = qquanta->queue_select.start_queue_id;
+ for (i = start_qid; i < end_qid; i++)
+ vsi->tx_rings[i]->quanta_prof_id = quanta_prof_id;
+
+err:
+ /* send the response to the VF */
+ return ice_vc_send_msg_to_vf(vf, VIRTCHNL_OP_CONFIG_QUANTA,
+ v_ret, NULL, 0);
+}
+
/**
* ice_vc_cfg_qs_msg
* @vf: pointer to the VF info
@@ -3817,6 +4138,9 @@ static const struct ice_virtchnl_ops ice_virtchnl_dflt_ops = {
.dis_vlan_stripping_v2_msg = ice_vc_dis_vlan_stripping_v2_msg,
.ena_vlan_insertion_v2_msg = ice_vc_ena_vlan_insertion_v2_msg,
.dis_vlan_insertion_v2_msg = ice_vc_dis_vlan_insertion_v2_msg,
+ .get_qos_caps = ice_vc_get_qos_caps,
+ .cfg_q_bw = ice_vc_cfg_q_bw,
+ .cfg_q_quanta = ice_vc_cfg_q_quanta,
};
/**
@@ -4173,6 +4497,15 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event,
case VIRTCHNL_OP_DISABLE_VLAN_INSERTION_V2:
err = ops->dis_vlan_insertion_v2_msg(vf, msg);
break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ err = ops->get_qos_caps(vf);
+ break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ err = ops->cfg_q_bw(vf, msg);
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ err = ops->cfg_q_quanta(vf, msg);
+ break;
case VIRTCHNL_OP_UNKNOWN:
default:
dev_err(dev, "Unsupported opcode %d from VF %d\n", v_opcode,
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl.h b/drivers/net/ethernet/intel/ice/ice_virtchnl.h
index 3a4115869153..0c629aef9baf 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl.h
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl.h
@@ -13,6 +13,13 @@
/* Restrict number of MAC Addr and VLAN that non-trusted VF can programmed */
#define ICE_MAX_VLAN_PER_VF 8
+#define ICE_DFLT_QUANTA 1024
+#define ICE_MAX_QUANTA_SIZE 4096
+#define ICE_MIN_QUANTA_SIZE 256
+
+#define calc_quanta_desc(x) \
+ max_t(u16, 12, min_t(u16, 63, (((x) + 66) / 132) * 2 + 4))
+
/* MAC filters: 1 is reserved for the VF's default/perm_addr/LAA MAC, 1 for
* broadcast, and 16 for additional unicast/multicast filters
*/
@@ -61,6 +68,10 @@ struct ice_virtchnl_ops {
int (*dis_vlan_stripping_v2_msg)(struct ice_vf *vf, u8 *msg);
int (*ena_vlan_insertion_v2_msg)(struct ice_vf *vf, u8 *msg);
int (*dis_vlan_insertion_v2_msg)(struct ice_vf *vf, u8 *msg);
+ int (*get_qos_caps)(struct ice_vf *vf);
+ int (*cfg_q_tc_map)(struct ice_vf *vf, u8 *msg);
+ int (*cfg_q_bw)(struct ice_vf *vf, u8 *msg);
+ int (*cfg_q_quanta)(struct ice_vf *vf, u8 *msg);
};
#ifdef CONFIG_PCI_IOV
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c b/drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c
index d796dbd2a440..c105a82ee136 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c
@@ -84,6 +84,11 @@ static const u32 fdir_pf_allowlist_opcodes[] = {
VIRTCHNL_OP_ADD_FDIR_FILTER, VIRTCHNL_OP_DEL_FDIR_FILTER,
};
+static const u32 tc_allowlist_opcodes[] = {
+ VIRTCHNL_OP_GET_QOS_CAPS, VIRTCHNL_OP_CONFIG_QUEUE_BW,
+ VIRTCHNL_OP_CONFIG_QUANTA,
+};
+
struct allowlist_opcode_info {
const u32 *opcodes;
size_t size;
@@ -104,6 +109,7 @@ static const struct allowlist_opcode_info allowlist_opcodes[] = {
ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF, adv_rss_pf_allowlist_opcodes),
ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_FDIR_PF, fdir_pf_allowlist_opcodes),
ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_VLAN_V2, vlan_v2_allowlist_opcodes),
+ ALLOW_ITEM(VIRTCHNL_VF_OFFLOAD_QOS, tc_allowlist_opcodes),
};
/**
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 11/12] iavf: Add net_shaper_ops support
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (9 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 10/12] ice: Support VF " Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 12/12] iavf: add support to exchange qos capabilities Paolo Abeni
2024-08-01 12:57 ` [PATCH v3 00/12] net: introduce TX H/W shaping API Jiri Pirko
12 siblings, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
From: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Implement net_shaper_ops support for IAVF. This enables configuration
of rate limiting on per queue basis. Customer intends to enforce
bandwidth limit on Tx traffic steered to the queue by configuring
rate limits on the queue.
To set rate limiting for a queue, update shaper object of given queues
in driver and send VIRTCHNL_OP_CONFIG_QUEUE_BW to PF to update HW
configuration.
Deleting shaper configured for queue is nothing but configuring shaper
with bw_max 0. The PF restores the default rate limiting config
when bw_max is zero.
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
---
drivers/net/ethernet/intel/Kconfig | 1 +
drivers/net/ethernet/intel/iavf/iavf.h | 3 +
drivers/net/ethernet/intel/iavf/iavf_main.c | 171 ++++++++++++++++++
drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 +
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 65 +++++++
5 files changed, 242 insertions(+)
diff --git a/drivers/net/ethernet/intel/Kconfig b/drivers/net/ethernet/intel/Kconfig
index 0375c7448a57..20bc40eec487 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -258,6 +258,7 @@ config I40E_DCB
config IAVF
tristate
select LIBIE
+ select NET_SHAPER
config I40EVF
tristate "Intel(R) Ethernet Adaptive Virtual Function support"
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index 23a6557fc3db..f5d1142ea427 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -33,6 +33,7 @@
#include <net/udp.h>
#include <net/tc_act/tc_gact.h>
#include <net/tc_act/tc_mirred.h>
+#include <net/net_shaper.h>
#include "iavf_type.h"
#include <linux/avf/virtchnl.h>
@@ -335,6 +336,7 @@ struct iavf_adapter {
#define IAVF_FLAG_AQ_DISABLE_CTAG_VLAN_INSERTION BIT_ULL(36)
#define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION BIT_ULL(37)
#define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION BIT_ULL(38)
+#define IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW BIT_ULL(39)
/* flags for processing extended capability messages during
* __IAVF_INIT_EXTENDED_CAPS. Each capability exchange requires
@@ -551,6 +553,7 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
int iavf_config_rss(struct iavf_adapter *adapter);
int iavf_lan_add_device(struct iavf_adapter *adapter);
int iavf_lan_del_device(struct iavf_adapter *adapter);
+void iavf_cfg_queues_bw(struct iavf_adapter *adapter);
void iavf_enable_channels(struct iavf_adapter *adapter);
void iavf_disable_channels(struct iavf_adapter *adapter);
void iavf_add_cloud_filter(struct iavf_adapter *adapter);
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index ff11bafb3b4f..3a5ae0cd31c7 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -2085,6 +2085,11 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
return 0;
}
+ if (adapter->aq_required & IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW) {
+ iavf_cfg_queues_bw(adapter);
+ return 0;
+ }
+
if (adapter->aq_required & IAVF_FLAG_AQ_CONFIGURE_QUEUES) {
iavf_configure_queues(adapter);
return 0;
@@ -2918,6 +2923,25 @@ static void iavf_disable_vf(struct iavf_adapter *adapter)
dev_info(&adapter->pdev->dev, "Reset task did not complete, VF disabled\n");
}
+/**
+ * iavf_reconfig_qs_bw - Call-back task to handle hardware reset
+ * @adapter: board private structure
+ *
+ * After a reset, the shaper parameters of queues need to be replayed again.
+ * Since the net_shaper_info object inside TX rings persists across reset,
+ * set the update flag for all queues so that the virtchnl message is triggered
+ * for all queues.
+ **/
+static void iavf_reconfig_qs_bw(struct iavf_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < adapter->num_active_queues; i++)
+ adapter->tx_rings[i].q_shaper_update = true;
+
+ adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+}
+
/**
* iavf_reset_task - Call-back task to handle hardware reset
* @work: pointer to work_struct
@@ -3124,6 +3148,8 @@ static void iavf_reset_task(struct work_struct *work)
iavf_up_complete(adapter);
iavf_irq_enable(adapter, true);
+
+ iavf_reconfig_qs_bw(adapter);
} else {
iavf_change_state(adapter, __IAVF_DOWN);
wake_up(&adapter->down_waitqueue);
@@ -4743,6 +4769,150 @@ static netdev_features_t iavf_fix_features(struct net_device *netdev,
return iavf_fix_strip_features(adapter, features);
}
+static int iavf_verify_handle(struct net_device *dev, u32 handle,
+ struct netlink_ext_ack *extack)
+{
+ struct iavf_adapter *adapter = netdev_priv(dev);
+ enum net_shaper_scope scope;
+ int qid;
+
+ scope = net_shaper_handle_scope(handle);
+ qid = net_shaper_handle_id(handle);
+
+ if (scope != NET_SHAPER_SCOPE_QUEUE) {
+ NL_SET_ERR_MSG_FMT(extack, "Invalid shaper handle %x, unsupported scope %d",
+ handle, scope);
+ return -EOPNOTSUPP;
+ }
+
+ if (qid >= adapter->num_active_queues) {
+ NL_SET_ERR_MSG_FMT(extack, "Invalid shaper handle %x, queued id %d max %d",
+ handle, qid, adapter->num_active_queues);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/**
+ * iavf_verify_shaper_info - check that shaper info received
+ * @dev: pointer to netdev
+ * @shaper: configuration of shaper.
+ * @extack: Netlink extended ACK for reporting errors
+ *
+ * Returns:
+ * * %0 - Success
+ * * %-EOPNOTSUPP - Driver doesn't support this scope.
+ * * %-EINVAL - Invalid queue number in input
+ **/
+static int
+iavf_verify_shaper_info(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack)
+{
+ return iavf_verify_handle(dev, shaper->handle, extack);
+}
+
+/**
+ * iavf_shaper_set - check that shaper info received
+ * @dev: pointer to netdev
+ * @shaper: configuration of shaper.
+ * @extack: Netlink extended ACK for reporting errors
+ *
+ * Returns:
+ * * %0 - Success
+ * * %-EOPNOTSUPP - Driver doesn't support this scope.
+ * * %-EINVAL - Invalid queue number in input
+ **/
+static int
+iavf_shaper_set(struct net_device *dev,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack)
+{
+ struct iavf_adapter *adapter = netdev_priv(dev);
+ bool need_cfg_update = false;
+ enum net_shaper_scope scope;
+ int id, ret = 0;
+
+ ret = iavf_verify_shaper_info(dev, shaper, extack);
+ if (ret)
+ return ret;
+
+ scope = net_shaper_handle_scope(shaper->handle);
+ id = net_shaper_handle_id(shaper->handle);
+
+ if (scope == NET_SHAPER_SCOPE_QUEUE) {
+ struct iavf_ring *tx_ring = &adapter->tx_rings[id];
+
+ tx_ring->q_shaper.bw_min = div_u64(shaper->bw_min, 1000);
+ tx_ring->q_shaper.bw_max = div_u64(shaper->bw_max, 1000);
+ tx_ring->q_shaper_update = true;
+ need_cfg_update = true;
+ }
+
+ if (need_cfg_update)
+ adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+
+ return 0;
+}
+
+static int iavf_shaper_del(struct net_device *dev,
+ const u32 handle,
+ struct netlink_ext_ack *extack)
+{
+ struct iavf_adapter *adapter = netdev_priv(dev);
+ bool need_cfg_update = false;
+ enum net_shaper_scope scope;
+ int qid, ret;
+
+ ret = iavf_verify_handle(dev, handle, extack);
+ if (ret < 0)
+ return ret;
+
+ scope = net_shaper_handle_scope(handle);
+ qid = net_shaper_handle_id(handle);
+
+ if (scope == NET_SHAPER_SCOPE_QUEUE) {
+ struct iavf_ring *tx_ring = &adapter->tx_rings[qid];
+
+ tx_ring->q_shaper.bw_min = 0;
+ tx_ring->q_shaper.bw_max = 0;
+ tx_ring->q_shaper_update = true;
+ need_cfg_update = true;
+ }
+
+ if (need_cfg_update)
+ adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+
+ return 0;
+}
+
+static int iavf_shaper_group(struct net_device *dev, int nr_inputs,
+ const struct net_shaper_info *inputs,
+ const struct net_shaper_info *output,
+ struct netlink_ext_ack *extack)
+{
+ return -EOPNOTSUPP;
+}
+
+static int iavf_shaper_cap(struct net_device *dev, enum net_shaper_scope scope,
+ unsigned long *flags)
+{
+ if (scope != NET_SHAPER_SCOPE_QUEUE)
+ return -EOPNOTSUPP;
+
+ *flags = BIT(NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MIN) |
+ BIT(NET_SHAPER_A_CAPABILITIES_SUPPORT_BW_MAX) |
+ BIT(NET_SHAPER_A_CAPABILITIES_SUPPORT_METRIC_BPS);
+ return 0;
+}
+
+static const struct net_shaper_ops iavf_shaper_ops = {
+ .set = iavf_shaper_set,
+ .delete = iavf_shaper_del,
+ .group = iavf_shaper_group,
+ .capabilities = iavf_shaper_cap,
+};
+
static const struct net_device_ops iavf_netdev_ops = {
.ndo_open = iavf_open,
.ndo_stop = iavf_close,
@@ -4758,6 +4928,7 @@ static const struct net_device_ops iavf_netdev_ops = {
.ndo_fix_features = iavf_fix_features,
.ndo_set_features = iavf_set_features,
.ndo_setup_tc = iavf_setup_tc,
+ .net_shaper_ops = &iavf_shaper_ops,
};
/**
diff --git a/drivers/net/ethernet/intel/iavf/iavf_txrx.h b/drivers/net/ethernet/intel/iavf/iavf_txrx.h
index d7b5587aeb8e..dd503ee50b7f 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_txrx.h
+++ b/drivers/net/ethernet/intel/iavf/iavf_txrx.h
@@ -296,6 +296,8 @@ struct iavf_ring {
*/
u32 rx_buf_len;
+ struct net_shaper_info q_shaper;
+ bool q_shaper_update;
} ____cacheline_internodealigned_in_smp;
#define IAVF_ITR_ADAPTIVE_MIN_INC 0x0002
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index 1e543f6a7c30..c0611608d332 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -1506,6 +1506,60 @@ iavf_set_adapter_link_speed_from_vpe(struct iavf_adapter *adapter,
adapter->link_speed = vpe->event_data.link_event.link_speed;
}
+/**
+ * iavf_cfg_queues_bw - configure bandwidth of allocated queues
+ * @adapter: iavf adapter structure instance
+ *
+ * This function requests PF to configure queue bandwidth of allocated queues
+ */
+void iavf_cfg_queues_bw(struct iavf_adapter *adapter)
+{
+ struct virtchnl_queues_bw_cfg *qs_bw_cfg;
+ struct net_shaper_info *q_shaper;
+ int qs_to_update = 0;
+ int i, inx = 0;
+ size_t len;
+
+ if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
+ /* bail because we already have a command pending */
+ dev_err(&adapter->pdev->dev,
+ "Cannot set tc queue bw, command %d pending\n",
+ adapter->current_op);
+ return;
+ }
+
+ for (i = 0; i < adapter->num_active_queues; i++) {
+ if (adapter->tx_rings[i].q_shaper_update)
+ qs_to_update++;
+ }
+ len = struct_size(qs_bw_cfg, cfg, qs_to_update);
+ qs_bw_cfg = kzalloc(len, GFP_KERNEL);
+ if (!qs_bw_cfg)
+ return;
+
+ qs_bw_cfg->vsi_id = adapter->vsi.id;
+ qs_bw_cfg->num_queues = qs_to_update;
+
+ for (i = 0; i < adapter->num_active_queues; i++) {
+ struct iavf_ring *tx_ring = &adapter->tx_rings[i];
+
+ q_shaper = &tx_ring->q_shaper;
+ if (tx_ring->q_shaper_update) {
+ qs_bw_cfg->cfg[inx].queue_id = i;
+ qs_bw_cfg->cfg[inx].shaper.peak = q_shaper->bw_max;
+ qs_bw_cfg->cfg[inx].shaper.committed = q_shaper->bw_min;
+ qs_bw_cfg->cfg[inx].tc = 0;
+ inx++;
+ }
+ }
+
+ adapter->current_op = VIRTCHNL_OP_CONFIG_QUEUE_BW;
+ adapter->aq_required &= ~IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW;
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_CONFIG_QUEUE_BW,
+ (u8 *)qs_bw_cfg, len);
+ kfree(qs_bw_cfg);
+}
+
/**
* iavf_enable_channels
* @adapter: adapter structure
@@ -2226,6 +2280,10 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC;
break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW:
+ dev_warn(&adapter->pdev->dev, "Failed to Config Queue BW, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ break;
default:
dev_err(&adapter->pdev->dev, "PF returned error %d (%s) to our request %d\n",
v_retval, iavf_stat_str(&adapter->hw, v_retval),
@@ -2560,6 +2618,13 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
if (!v_retval)
iavf_netdev_features_vlan_strip_set(netdev, false);
break;
+ case VIRTCHNL_OP_CONFIG_QUEUE_BW: {
+ int i;
+ /* shaper configuration is successful for all queues */
+ for (i = 0; i < adapter->num_active_queues; i++)
+ adapter->tx_rings[i].q_shaper_update = false;
+ }
+ break;
default:
if (adapter->current_op && (v_opcode != adapter->current_op))
dev_warn(&adapter->pdev->dev, "Expected response %d from PF, received %d\n",
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* [PATCH v3 12/12] iavf: add support to exchange qos capabilities
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (10 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 11/12] iavf: Add net_shaper_ops support Paolo Abeni
@ 2024-07-30 20:39 ` Paolo Abeni
2024-08-01 12:57 ` [PATCH v3 00/12] net: introduce TX H/W shaping API Jiri Pirko
12 siblings, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-07-30 20:39 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
From: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
During driver initialization VF determines QOS capability is allowed
by PF and receives QOS parameters. After which quanta size for queues
is configured which is not configurable and is set to 1KB currently.
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
---
drivers/net/ethernet/intel/iavf/iavf.h | 10 ++
drivers/net/ethernet/intel/iavf/iavf_main.c | 46 +++++++++-
.../net/ethernet/intel/iavf/iavf_virtchnl.c | 92 ++++++++++++++++++-
3 files changed, 145 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h
index f5d1142ea427..dd9cca067360 100644
--- a/drivers/net/ethernet/intel/iavf/iavf.h
+++ b/drivers/net/ethernet/intel/iavf/iavf.h
@@ -250,6 +250,9 @@ struct iavf_cloud_filter {
#define IAVF_RESET_WAIT_DETECTED_COUNT 500
#define IAVF_RESET_WAIT_COMPLETE_COUNT 2000
+#define IAVF_MAX_QOS_TC_NUM 8
+#define IAVF_DEFAULT_QUANTA_SIZE 1024
+
/* board specific private data structure */
struct iavf_adapter {
struct workqueue_struct *wq;
@@ -337,6 +340,8 @@ struct iavf_adapter {
#define IAVF_FLAG_AQ_ENABLE_STAG_VLAN_INSERTION BIT_ULL(37)
#define IAVF_FLAG_AQ_DISABLE_STAG_VLAN_INSERTION BIT_ULL(38)
#define IAVF_FLAG_AQ_CONFIGURE_QUEUES_BW BIT_ULL(39)
+#define IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE BIT_ULL(40)
+#define IAVF_FLAG_AQ_GET_QOS_CAPS BIT_ULL(41)
/* flags for processing extended capability messages during
* __IAVF_INIT_EXTENDED_CAPS. Each capability exchange requires
@@ -407,6 +412,8 @@ struct iavf_adapter {
VIRTCHNL_VF_OFFLOAD_FDIR_PF)
#define ADV_RSS_SUPPORT(_a) ((_a)->vf_res->vf_cap_flags & \
VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF)
+#define QOS_ALLOWED(_a) ((_a)->vf_res->vf_cap_flags & \
+ VIRTCHNL_VF_OFFLOAD_QOS)
struct virtchnl_vf_resource *vf_res; /* incl. all VSIs */
struct virtchnl_vsi_resource *vsi_res; /* our LAN VSI */
struct virtchnl_version_info pf_version;
@@ -415,6 +422,7 @@ struct iavf_adapter {
struct virtchnl_vlan_caps vlan_v2_caps;
u16 msg_enable;
struct iavf_eth_stats current_stats;
+ struct virtchnl_qos_cap_list *qos_caps;
struct iavf_vsi vsi;
u32 aq_wait_count;
/* RSS stuff */
@@ -554,6 +562,8 @@ int iavf_config_rss(struct iavf_adapter *adapter);
int iavf_lan_add_device(struct iavf_adapter *adapter);
int iavf_lan_del_device(struct iavf_adapter *adapter);
void iavf_cfg_queues_bw(struct iavf_adapter *adapter);
+void iavf_cfg_queues_quanta_size(struct iavf_adapter *adapter);
+void iavf_get_qos_caps(struct iavf_adapter *adapter);
void iavf_enable_channels(struct iavf_adapter *adapter);
void iavf_disable_channels(struct iavf_adapter *adapter);
void iavf_add_cloud_filter(struct iavf_adapter *adapter);
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 3a5ae0cd31c7..0ee128477d59 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -2090,6 +2090,16 @@ static int iavf_process_aq_command(struct iavf_adapter *adapter)
return 0;
}
+ if (adapter->aq_required & IAVF_FLAG_AQ_GET_QOS_CAPS) {
+ iavf_get_qos_caps(adapter);
+ return 0;
+ }
+
+ if (adapter->aq_required & IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE) {
+ iavf_cfg_queues_quanta_size(adapter);
+ return 0;
+ }
+
if (adapter->aq_required & IAVF_FLAG_AQ_CONFIGURE_QUEUES) {
iavf_configure_queues(adapter);
return 0;
@@ -2675,6 +2685,9 @@ static void iavf_init_config_adapter(struct iavf_adapter *adapter)
/* request initial VLAN offload settings */
iavf_set_vlan_offload_features(adapter, 0, netdev->features);
+ if (QOS_ALLOWED(adapter))
+ adapter->aq_required |= IAVF_FLAG_AQ_GET_QOS_CAPS;
+
iavf_schedule_finish_config(adapter);
return;
@@ -4809,7 +4822,27 @@ iavf_verify_shaper_info(struct net_device *dev,
const struct net_shaper_info *shaper,
struct netlink_ext_ack *extack)
{
- return iavf_verify_handle(dev, shaper->handle, extack);
+ struct iavf_adapter *adapter = netdev_priv(dev);
+ enum net_shaper_scope scope;
+ int ret, qid;
+ u64 vf_max;
+
+ ret = iavf_verify_handle(dev, shaper->handle, extack);
+ if (ret)
+ return ret;
+
+ scope = net_shaper_handle_scope(shaper->handle);
+ qid = net_shaper_handle_id(shaper->handle);
+
+ if (scope == NET_SHAPER_SCOPE_QUEUE) {
+ vf_max = adapter->qos_caps->cap[0].shaper.peak;
+ if (vf_max && shaper->bw_max > vf_max) {
+ NL_SET_ERR_MSG_FMT(extack, "Max rate (%llu) of queue %d can't exceed max TX rate of VF (%llu kbps)",
+ shaper->bw_max, qid,
+ vf_max);
+ }
+ }
+ return 0;
}
/**
@@ -5073,7 +5106,7 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
struct net_device *netdev;
struct iavf_adapter *adapter = NULL;
struct iavf_hw *hw = NULL;
- int err;
+ int err, len;
err = pci_enable_device(pdev);
if (err)
@@ -5141,6 +5174,13 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
hw->bus.func = PCI_FUNC(pdev->devfn);
hw->bus.bus_id = pdev->bus->number;
+ len = struct_size(adapter->qos_caps, cap, IAVF_MAX_QOS_TC_NUM);
+ adapter->qos_caps = kzalloc(len, GFP_KERNEL);
+ if (!adapter->qos_caps) {
+ err = -ENOMEM;
+ goto err_alloc_qos_cap;
+ }
+
/* set up the locks for the AQ, do this only once in probe
* and destroy them only once in remove
*/
@@ -5179,6 +5219,8 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
/* Initialization goes on in the work. Do not add more of it below. */
return 0;
+err_alloc_qos_cap:
+ iounmap(hw->hw_addr);
err_ioremap:
destroy_workqueue(adapter->wq);
err_alloc_wq:
diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
index c0611608d332..2f1be42ebd58 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c
@@ -150,7 +150,8 @@ int iavf_send_vf_config_msg(struct iavf_adapter *adapter)
VIRTCHNL_VF_OFFLOAD_USO |
VIRTCHNL_VF_OFFLOAD_FDIR_PF |
VIRTCHNL_VF_OFFLOAD_ADV_RSS_PF |
- VIRTCHNL_VF_CAP_ADV_LINK_SPEED;
+ VIRTCHNL_VF_CAP_ADV_LINK_SPEED |
+ VIRTCHNL_VF_OFFLOAD_QOS;
adapter->current_op = VIRTCHNL_OP_GET_VF_RESOURCES;
adapter->aq_required &= ~IAVF_FLAG_AQ_GET_CONFIG;
@@ -1506,6 +1507,76 @@ iavf_set_adapter_link_speed_from_vpe(struct iavf_adapter *adapter,
adapter->link_speed = vpe->event_data.link_event.link_speed;
}
+/**
+ * iavf_get_qos_caps - get qos caps support
+ * @adapter: iavf adapter struct instance
+ *
+ * This function requests PF for Supported QoS Caps.
+ */
+void iavf_get_qos_caps(struct iavf_adapter *adapter)
+{
+ if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
+ /* bail because we already have a command pending */
+ dev_err(&adapter->pdev->dev,
+ "Cannot get qos caps, command %d pending\n",
+ adapter->current_op);
+ return;
+ }
+
+ adapter->current_op = VIRTCHNL_OP_GET_QOS_CAPS;
+ adapter->aq_required &= ~IAVF_FLAG_AQ_GET_QOS_CAPS;
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_GET_QOS_CAPS, NULL, 0);
+}
+
+/**
+ * iavf_set_quanta_size - set quanta size of queue chunk
+ * @adapter: iavf adapter struct instance
+ * @quanta_size: quanta size in bytes
+ * @queue_index: starting index of queue chunk
+ * @num_queues: number of queues in the queue chunk
+ *
+ * This function requests PF to set quanta size of queue chunk
+ * starting at queue_index.
+ */
+static void
+iavf_set_quanta_size(struct iavf_adapter *adapter, u16 quanta_size,
+ u16 queue_index, u16 num_queues)
+{
+ struct virtchnl_quanta_cfg quanta_cfg;
+
+ if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) {
+ /* bail because we already have a command pending */
+ dev_err(&adapter->pdev->dev,
+ "Cannot set queue quanta size, command %d pending\n",
+ adapter->current_op);
+ return;
+ }
+
+ adapter->current_op = VIRTCHNL_OP_CONFIG_QUANTA;
+ quanta_cfg.quanta_size = quanta_size;
+ quanta_cfg.queue_select.type = VIRTCHNL_QUEUE_TYPE_TX;
+ quanta_cfg.queue_select.start_queue_id = queue_index;
+ quanta_cfg.queue_select.num_queues = num_queues;
+ adapter->aq_required &= ~IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE;
+ iavf_send_pf_msg(adapter, VIRTCHNL_OP_CONFIG_QUANTA,
+ (u8 *)&quanta_cfg, sizeof(quanta_cfg));
+}
+
+/**
+ * iavf_cfg_queues_quanta_size - configure quanta size of queues
+ * @adapter: adapter structure
+ *
+ * Request that the PF configure quanta size of allocated queues.
+ **/
+void iavf_cfg_queues_quanta_size(struct iavf_adapter *adapter)
+{
+ int quanta_size = IAVF_DEFAULT_QUANTA_SIZE;
+
+ /* Set Queue Quanta Size to default */
+ iavf_set_quanta_size(adapter, quanta_size, 0,
+ adapter->num_active_queues);
+}
+
/**
* iavf_cfg_queues_bw - configure bandwidth of allocated queues
* @adapter: iavf adapter structure instance
@@ -2280,6 +2351,14 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
VIRTCHNL_RSS_ALG_TOEPLITZ_SYMMETRIC;
break;
+ case VIRTCHNL_OP_GET_QOS_CAPS:
+ dev_warn(&adapter->pdev->dev, "Failed to Get Qos CAPs, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ dev_warn(&adapter->pdev->dev, "Failed to Config Quanta, error %s\n",
+ iavf_stat_str(&adapter->hw, v_retval));
+ break;
case VIRTCHNL_OP_CONFIG_QUEUE_BW:
dev_warn(&adapter->pdev->dev, "Failed to Config Queue BW, error %s\n",
iavf_stat_str(&adapter->hw, v_retval));
@@ -2618,6 +2697,17 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
if (!v_retval)
iavf_netdev_features_vlan_strip_set(netdev, false);
break;
+ case VIRTCHNL_OP_GET_QOS_CAPS: {
+ u16 len = struct_size(adapter->qos_caps, cap,
+ IAVF_MAX_QOS_TC_NUM);
+
+ memcpy(adapter->qos_caps, msg, min(msglen, len));
+
+ adapter->aq_required |= IAVF_FLAG_AQ_CFG_QUEUES_QUANTA_SIZE;
+ }
+ break;
+ case VIRTCHNL_OP_CONFIG_QUANTA:
+ break;
case VIRTCHNL_OP_CONFIG_QUEUE_BW: {
int i;
/* shaper configuration is successful for all queues */
--
2.45.2
^ permalink raw reply related [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-07-30 20:39 ` [PATCH v3 08/12] testing: net-drv: add basic shaper test Paolo Abeni
@ 2024-07-31 7:52 ` Paolo Abeni
2024-08-01 1:55 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-07-31 7:52 UTC (permalink / raw)
To: netdev
Cc: Jakub Kicinski, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 7/30/24 22:39, Paolo Abeni wrote:
> Leverage a basic/dummy netdevsim implementation to do functional
> coverage for NL interface.
>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
FTR, it looks like the CI build went wild around this patch, but the
failures look unrelated to the actual changes here. i.e.:
https://netdev.bots.linux.dev/static/nipa/875223/13747883/build_clang/stderr
Per-file breakdown
--- /tmp/tmp.z9wI8zevoA 2024-07-30 18:26:37.281153512 -0700
+++ /tmp/tmp.pwD35f06q6 2024-07-30 18:26:37.285153598 -0700
@@ -0,0 +1,13 @@
+ 4 ../drivers/block/drbd/drbd_bitmap.c
+ 4 ../drivers/block/drbd/drbd_main.c
+ 2 ../drivers/firmware/broadcom/bcm47xx_sprom.c
+ 6 ../drivers/most/most_usb.c
+ 1 ../drivers/net/ethernet/sfc/ptp.c
+ 1 ../drivers/net/ethernet/sfc/siena/ptp.c
+ 7 ../include/linux/fortify-string.h
+ 2 ../include/linux/kern_levels.h
+ 2 ../include/linux/printk.h
+ 2 ../kernel/audit.c
+ 1 ../lib/vsprintf.c
+ 3 ../net/ipv4/tcp_lp.c
+ 2 ../security/apparmor/lsm.c
Still we hit a similar issue on the previous iteration, so perhaps there
is some correlation I don't see?!?
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-07-30 20:39 ` [PATCH v3 02/12] netlink: spec: add shaper YAML spec Paolo Abeni
@ 2024-07-31 21:13 ` Donald Hunter
2024-08-01 14:31 ` Paolo Abeni
2024-08-01 13:10 ` Jiri Pirko
` (3 subsequent siblings)
4 siblings, 1 reply; 91+ messages in thread
From: Donald Hunter @ 2024-07-31 21:13 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Jiri Pirko, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Paolo Abeni <pabeni@redhat.com> writes:
> diff --git a/Documentation/netlink/specs/shaper.yaml b/Documentation/netlink/specs/shaper.yaml
> new file mode 100644
> index 000000000000..7327f5596fdb
> --- /dev/null
> +++ b/Documentation/netlink/specs/shaper.yaml
It's probably more user-friendly to use the same filename as the spec
name, so net-shaper.yaml
> @@ -0,0 +1,262 @@
> +# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
> +
> +name: net-shaper
> +
> +doc: Network device HW Rate Limiting offload
> +
> +definitions:
> + -
> + type: enum
> + name: scope
> + doc: the different scopes where a shaper can be attached
Nit: upper case 'The' to be consistent with rest of docs.
> + render-max: true
> + entries:
> + - name: unspec
> + doc: The scope is not specified
What are the semantics of 'unspec' ? When can it be used?
> + -
> + name: port
> + doc: The root for the whole H/W
> + -
> + name: netdev
> + doc: The main shaper for the given network device.
What are the semantic differences between netdev and port?
> + -
> + name: queue
> + doc: The shaper is attached to the given device queue.
> + -
> + name: detached
> + doc: |
> + The shaper is not attached to any user-visible network
> + device component and allows nesting and grouping of
> + queues or others detached shapers.
I assume that shapers are always owned by the netdev regardless of
attach status?
> + -
> + type: enum
> + name: metric
> + doc: different metric each shaper can support
Nit: upper case here as well.
> + entries:
> + -
> + name: bps
> + doc: Shaper operates on a bits per second basis
> + -
> + name: pps
> + doc: Shaper operates on a packets per second basis
> +
> +attribute-sets:
> + -
> + name: net-shaper
> + attributes:
> + -
> + name: ifindex
> + type: u32
> + doc: Interface index owing the specified shaper[s]
Typo: this should be 'owning' ?
> + -
> + name: handle
> + type: nest
> + nested-attributes: handle
> + doc: Unique identifier for the given shaper
> + -
> + name: metric
> + type: u32
> + enum: metric
> + doc: Metric used by the given shaper for bw-min, bw-max and burst
> + -
> + name: bw-min
> + type: uint
> + doc: Minimum guaranteed B/W for the given shaper
> + -
> + name: bw-max
> + type: uint
> + doc: Shaping B/W for the given shaper or 0 when unlimited
> + -
> + name: burst
> + type: uint
> + doc: Maximum burst-size for bw-min and bw-max
> + -
> + name: priority
> + type: u32
> + doc: Scheduling priority for the given shaper
> + -
> + name: weight
> + type: u32
> + doc: |
> + Weighted round robin weight for given shaper.
> + The scheduling is applied to all the sibling
> + shapers with the same priority
> + -
> + name: scope
> + type: u32
> + enum: scope
> + doc: The given handle scope
> + -
> + name: id
> + type: u32
> + doc: |
> + The given handle id. The id semantic depends on the actual
> + scope, e.g. for 'queue' scope it's the queue id, for
> + 'detached' scope it's the shaper group identifier.
If scope and id are only ever used as attributes of a handle then they
would be better specified as a separate attribute-set, instead of
mixing them in here and using a subset.
You use 'quoted' references here and @refs elsewhere. It would be good
to be consistent. See note below about @ in htmldocs.
> + -
> + name: parent
> + type: nest
> + nested-attributes: handle
> + doc: |
> + Identifier for the parent of the affected shaper,
> + The parent handle value is implied by the shaper handle itself,
> + except for the output shaper in the 'group' operation.
Nit: quoted ref again here
> + -
> + name: inputs
> + type: nest
> + multi-attr: true
> + nested-attributes: ns-info
> + doc: |
> + Describes a set of inputs shapers for a @group operation
The @group renders exactly as-is in the generated htmldocs. There may be
a more .rst friendly markup you can use that will render better.
> + -
> + name: output
> + type: nest
> + nested-attributes: ns-output-info
> + doc: |
> + Describes the output shaper for a @group operation
> + Differently from @inputs and @shaper allow specifying
> + the shaper parent handle, too.
> +
Nit: remove the extra blank line here
> + -
> + name: shaper
> + type: nest
> + nested-attributes: ns-info
> + doc: |
> + Describes a single shaper for a @set operation
> + -
> + name: handle
> + subset-of: net-shaper
> + attributes:
> + -
> + name: scope
> + -
> + name: id
> + -
> + name: ns-info
> + subset-of: net-shaper
> + attributes:
> + -
> + name: handle
> + -
> + name: metric
> + -
> + name: bw-min
> + -
> + name: bw-max
> + -
> + name: burst
> + -
> + name: priority
> + -
> + name: weight
> + -
> + name: ns-output-info
> + subset-of: net-shaper
> + attributes:
> + -
> + name: parent
> + -
> + name: handle
> + -
> + name: metric
> + -
> + name: bw-min
> + -
> + name: bw-max
> + -
> + name: burst
> + -
> + name: priority
> + -
> + name: weight
> +
> +operations:
> + list:
> + -
> + name: get
> + doc: |
> + Get / Dump information about a/all the shaper for a given device
> + attribute-set: net-shaper
> +
> + do:
> + request:
> + attributes:
> + - ifindex
> + - handle
> + reply:
> + attributes: &ns-attrs
> + - parent
> + - handle
> + - metric
> + - bw-min
> + - bw-max
> + - burst
> + - priority
> + - weight
> +
> + dump:
> + request:
> + attributes:
> + - ifindex
> + reply:
> + attributes: *ns-attrs
> + -
> + name: set
> + doc: |
> + Create or configures the specified shaper.
> + On failures the extack is set accordingly.
> + Can't create @detached scope shaper, use
> + the @group operation instead.
> + attribute-set: net-shaper
> + flags: [ admin-perm ]
> +
> + do:
> + request:
> + attributes:
> + - ifindex
> + - shaper
> +
> + -
> + name: delete
> + doc: |
> + Clear (remove) the specified shaper. If after the removal
> + the parent shaper has no more children and the parent
> + shaper scope is @detached, even the parent is deleted,
> + recursively.
> + On failures the extack is set accordingly.
> + attribute-set: net-shaper
> + flags: [ admin-perm ]
> +
> + do:
> + request:
> + attributes:
> + - ifindex
> + - handle
> +
> + -
> + name: group
> + doc: |
> + Group the specified input shapers under the specified
> + output shaper, eventually creating the latter, if needed.
> + Input shapers scope must be either @queue or @detached.
It says above that you cannot create a detached shaper, so how do you
create one to use as an input shaper here? Is this group op more like a
multi-create op?
> + Output shaper scope must be either @detached or @netdev.
> + When using an output @detached scope shaper, if the
> + @handle @id is not specified, a new shaper of such scope
> + is created and, otherwise the specified output shaper
> + must be already existing.
> + The operation is atomic, on failures the extack is set
> + accordingly and no change is applied to the device
> + shaping configuration, otherwise the output shaper
> + handle is provided as reply.
> + attribute-set: net-shaper
> + flags: [ admin-perm ]
Does there need to be a reciprocal 'ungroup' operation? Without it,
create / group / delete seems like they will have ambiguous semantics.
> + do:
> + request:
> + attributes:
> + - ifindex
> + - inputs
> + - output
> + reply:
> + attributes:
> + - handle
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-07-31 7:52 ` Paolo Abeni
@ 2024-08-01 1:55 ` Jakub Kicinski
2024-08-05 14:22 ` Simon Horman
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-01 1:55 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Wed, 31 Jul 2024 09:52:38 +0200 Paolo Abeni wrote:
> On 7/30/24 22:39, Paolo Abeni wrote:
> > Leverage a basic/dummy netdevsim implementation to do functional
> > coverage for NL interface.
> >
> > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>
> FTR, it looks like the CI build went wild around this patch, but the
> failures look unrelated to the actual changes here. i.e.:
>
> https://netdev.bots.linux.dev/static/nipa/875223/13747883/build_clang/stderr
Could you dig deeper?
The scripts are doing incremental builds, and changes to Kconfig
confuse them. You should be able to run the build script as a normal
bash script, directly, it only needs a small handful of exported
env variables.
I have been trying to massage this for a while, my last change is:
https://github.com/linux-netdev/nipa/commit/5bcb890cbfecd3c1727cec2f026360646a4afc62
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 00/12] net: introduce TX H/W shaping API
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
` (11 preceding siblings ...)
2024-07-30 20:39 ` [PATCH v3 12/12] iavf: add support to exchange qos capabilities Paolo Abeni
@ 2024-08-01 12:57 ` Jiri Pirko
12 siblings, 0 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-01 12:57 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Jul 30, 2024 at 10:39:43PM CEST, pabeni@redhat.com wrote:
>We have a plurality of shaping-related drivers API, but none flexible
>enough to meet existing demand from vendors[1].
>
>This series introduces new device APIs to configure in a flexible way
>TX H/W shaping. The new functionalities are exposed via a newly
>defined generic netlink interface and include introspection
>capabilities. Some self-tests are included, on top of a dummy
>netdevsim implementation, and a basic implementation for the iavf
>driver.
>
>Some usage examples:
>
>* Configure shaping on a given queue:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do set --json '{"ifindex":'$IFINDEX',
> "shaper": {"handle":
> {"scope": "queue", "id":'$QUEUEID' },
> "bw-max": 2000000 }}'
>
>* Container B/W sharing
>
>The orchestration infrastructure wants to group the
>container-related queues under a RR scheduling and limit the aggregate
>bandwidth:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do group --json '{"ifindex":'$IFINDEX',
> "inputs": [
> {"handle": {"scope": "queue", "id":'$QID1' },
> "weight": '$W1'},
> {"handle": {"scope": "queue", "id":'$QID2' },
> "weight": '$W2'}],
> {"handle": {"scope": "queue", "id":'$QID3' },
> "weight": '$W3'}],
> "output": { "handle": {"scope":"netdev"},
> "output": { "handle": {"scope":"netdev"},
I got here and got lost :) What's "input" and "output" meaning here?
In general, I'm missing some documentation file to describe the design
of the shapers. I lookes at the patch descriptions and comments, could
not figure it out :/ Could you please draft it?
> "bw-max": 10000000}}'
>{'handle': {'id': 0, 'scope': 'netdev'}}
>
>* Delegation
>
>A container wants to set a B/W limit on 2 of its own queues:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do group --json '{"ifindex":'$IFINDEX',
> "inputs": [
> {"handle": {"scope": "queue", "id":'$QID1' },
> "weight": '$W1'},
> {"handle": {"scope": "queue", "id":'$QID2' },
> "weight": '$W2'}],
> "output": { "handle": {"scope":"detached"},
> "bw-max": 5000000}}'
>{'handle': {'id': 0, 'scope': 'detached'}}
>
>* Cleanup:
>
>Deleting a single queue shaper:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do delete --json \
> '{"ifindex":'$IFINDEX',
> "handle": {"scope": "queue", "id":'$QID1' }}'
>
>deleting the last shaper under a group deletes the group, too:
>
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do delete --json \
> '{"ifindex":'$IFINDEX',
> "handle": {"scope": "queue", "id":'$QID2' }}'
>./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
> --do get --json '{"ifindex":'$IF',
> "handle": { "scope": "detached", "id": 0}}'
>Netlink error: Invalid argument
>nl_len = 80 (64) nl_flags = 0x300 nl_type = 2
> error: -22
> extack: {'msg': "Can't find shaper for handle 10000000"}
>
>Changes from RFC v2:
> - added patch 1
> - fixed deprecated API usage
>
>RFC v2: https://lore.kernel.org/netdev/cover.1721851988.git.pabeni@redhat.com/
>
>Changes from RFC v1:
> - set() and delete() ops operate on a single shaper
> - added group() op to allow grouping and nesting
> - split the NL implementation into multiple patches to help reviewing
>
>RFC v1: https://lore.kernel.org/netdev/cover.1719518113.git.pabeni@redhat.com/
>
>[1] https://lore.kernel.org/netdev/20240405102313.GA310894@kernel.org/
>
>Paolo Abeni (8):
> tools: ynl: lift an assumption about spec file name
> netlink: spec: add shaper YAML spec
> net-shapers: implement NL get operation
> net-shapers: implement NL set and delete operations
> net-shapers: implement NL group operation
> netlink: spec: add shaper introspection support
> net: shaper: implement introspection support
> testing: net-drv: add basic shaper test
>
>Sudheer Mogilappagari (2):
> iavf: Add net_shaper_ops support
> iavf: add support to exchange qos capabilities
>
>Wenjun Wu (2):
> virtchnl: support queue rate limit and quanta size configuration
> ice: Support VF queue rate limit and quanta size configuration
>
> Documentation/netlink/specs/shaper.yaml | 337 ++++++
> Documentation/networking/kapi.rst | 3 +
> MAINTAINERS | 1 +
> drivers/net/Kconfig | 1 +
> drivers/net/ethernet/intel/Kconfig | 1 +
> drivers/net/ethernet/intel/iavf/iavf.h | 13 +
> drivers/net/ethernet/intel/iavf/iavf_main.c | 215 +++-
> drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 +
> .../net/ethernet/intel/iavf/iavf_virtchnl.c | 157 ++-
> drivers/net/ethernet/intel/ice/ice.h | 2 +
> drivers/net/ethernet/intel/ice/ice_base.c | 2 +
> drivers/net/ethernet/intel/ice/ice_common.c | 21 +
> .../net/ethernet/intel/ice/ice_hw_autogen.h | 8 +
> drivers/net/ethernet/intel/ice/ice_txrx.h | 1 +
> drivers/net/ethernet/intel/ice/ice_type.h | 1 +
> drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 +
> drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++
> drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 +
> .../intel/ice/ice_virtchnl_allowlist.c | 6 +
> drivers/net/netdevsim/netdev.c | 37 +
> include/linux/avf/virtchnl.h | 119 +++
> include/linux/netdevice.h | 17 +
> include/net/net_shaper.h | 169 +++
> include/uapi/linux/net_shaper.h | 91 ++
> net/Kconfig | 3 +
> net/Makefile | 1 +
> net/core/dev.c | 2 +
> net/core/dev.h | 6 +
> net/shaper/Makefile | 9 +
> net/shaper/shaper.c | 963 ++++++++++++++++++
> net/shaper/shaper_nl_gen.c | 142 +++
> net/shaper/shaper_nl_gen.h | 30 +
> tools/net/ynl/ynl-gen-c.py | 6 +-
> tools/testing/selftests/drivers/net/Makefile | 1 +
> tools/testing/selftests/drivers/net/shaper.py | 267 +++++
> .../testing/selftests/net/lib/py/__init__.py | 1 +
> tools/testing/selftests/net/lib/py/ynl.py | 5 +
> 37 files changed, 2988 insertions(+), 4 deletions(-)
> create mode 100644 Documentation/netlink/specs/shaper.yaml
> create mode 100644 include/net/net_shaper.h
> create mode 100644 include/uapi/linux/net_shaper.h
> create mode 100644 net/shaper/Makefile
> create mode 100644 net/shaper/shaper.c
> create mode 100644 net/shaper/shaper_nl_gen.c
> create mode 100644 net/shaper/shaper_nl_gen.h
> create mode 100755 tools/testing/selftests/drivers/net/shaper.py
>
>--
>2.45.2
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-07-30 20:39 ` [PATCH v3 02/12] netlink: spec: add shaper YAML spec Paolo Abeni
2024-07-31 21:13 ` Donald Hunter
@ 2024-08-01 13:10 ` Jiri Pirko
2024-08-01 14:40 ` Jakub Kicinski
2024-08-01 15:12 ` Paolo Abeni
2024-08-02 11:19 ` Jiri Pirko
` (2 subsequent siblings)
4 siblings, 2 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-01 13:10 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
>Define the user-space visible interface to query, configure and delete
>network shapers via yaml definition.
>
>Add dummy implementations for the relevant NL callbacks.
>
>set() and delete() operations touch a single shaper creating/updating or
>deleting it.
>The group() operation creates a shaper's group, nesting multiple input
>shapers under the specified output shaper.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>RFC v1 -> RFC v2:
> - u64 -> uint
> - net_shapers -> net-shapers
> - documented all the attributes
> - dropped [ admin-perm ] for get() op
> - group op
> - set/delete touch a single shaper
>---
> Documentation/netlink/specs/shaper.yaml | 262 ++++++++++++++++++++++++
> MAINTAINERS | 1 +
> include/uapi/linux/net_shaper.h | 74 +++++++
> net/Kconfig | 3 +
> net/Makefile | 1 +
> net/shaper/Makefile | 9 +
> net/shaper/shaper.c | 34 +++
> net/shaper/shaper_nl_gen.c | 117 +++++++++++
> net/shaper/shaper_nl_gen.h | 27 +++
> 9 files changed, 528 insertions(+)
> create mode 100644 Documentation/netlink/specs/shaper.yaml
> create mode 100644 include/uapi/linux/net_shaper.h
> create mode 100644 net/shaper/Makefile
> create mode 100644 net/shaper/shaper.c
> create mode 100644 net/shaper/shaper_nl_gen.c
> create mode 100644 net/shaper/shaper_nl_gen.h
>
>diff --git a/Documentation/netlink/specs/shaper.yaml b/Documentation/netlink/specs/shaper.yaml
>new file mode 100644
>index 000000000000..7327f5596fdb
>--- /dev/null
>+++ b/Documentation/netlink/specs/shaper.yaml
>@@ -0,0 +1,262 @@
>+# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
>+
>+name: net-shaper
>+
>+doc: Network device HW Rate Limiting offload
>+
>+definitions:
>+ -
>+ type: enum
>+ name: scope
>+ doc: the different scopes where a shaper can be attached
>+ render-max: true
>+ entries:
>+ - name: unspec
>+ doc: The scope is not specified
>+ -
>+ name: port
>+ doc: The root for the whole H/W
What is this "port"?
>+ -
>+ name: netdev
>+ doc: The main shaper for the given network device.
>+ -
>+ name: queue
>+ doc: The shaper is attached to the given device queue.
>+ -
>+ name: detached
>+ doc: |
>+ The shaper is not attached to any user-visible network
>+ device component and allows nesting and grouping of
>+ queues or others detached shapers.
What is the purpose of the "detached" thing?
>+ -
>+ type: enum
>+ name: metric
>+ doc: different metric each shaper can support
>+ entries:
>+ -
>+ name: bps
>+ doc: Shaper operates on a bits per second basis
>+ -
>+ name: pps
>+ doc: Shaper operates on a packets per second basis
>+
[..]
>+
>+operations:
>+ list:
>+ -
>+ name: get
>+ doc: |
>+ Get / Dump information about a/all the shaper for a given device
>+ attribute-set: net-shaper
>+
>+ do:
>+ request:
>+ attributes:
>+ - ifindex
>+ - handle
>+ reply:
>+ attributes: &ns-attrs
>+ - parent
>+ - handle
>+ - metric
>+ - bw-min
>+ - bw-max
>+ - burst
>+ - priority
>+ - weight
>+
>+ dump:
>+ request:
>+ attributes:
>+ - ifindex
>+ reply:
>+ attributes: *ns-attrs
>+ -
>+ name: set
>+ doc: |
>+ Create or configures the specified shaper.
>+ On failures the extack is set accordingly.
>+ Can't create @detached scope shaper, use
>+ the @group operation instead.
>+ attribute-set: net-shaper
>+ flags: [ admin-perm ]
>+
>+ do:
>+ request:
>+ attributes:
>+ - ifindex
>+ - shaper
>+
>+ -
>+ name: delete
>+ doc: |
>+ Clear (remove) the specified shaper. If after the removal
>+ the parent shaper has no more children and the parent
>+ shaper scope is @detached, even the parent is deleted,
>+ recursively.
>+ On failures the extack is set accordingly.
>+ attribute-set: net-shaper
>+ flags: [ admin-perm ]
>+
>+ do:
>+ request:
>+ attributes:
>+ - ifindex
>+ - handle
>+
>+ -
>+ name: group
>+ doc: |
>+ Group the specified input shapers under the specified
>+ output shaper, eventually creating the latter, if needed.
>+ Input shapers scope must be either @queue or @detached.
>+ Output shaper scope must be either @detached or @netdev.
>+ When using an output @detached scope shaper, if the
>+ @handle @id is not specified, a new shaper of such scope
>+ is created and, otherwise the specified output shaper
>+ must be already existing.
I'm lost. Could this designt be described in details in the doc I asked
in the cover letter? :/ Please.
>+ The operation is atomic, on failures the extack is set
>+ accordingly and no change is applied to the device
>+ shaping configuration, otherwise the output shaper
>+ handle is provided as reply.
>+ attribute-set: net-shaper
[..]
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-07-30 20:39 ` [PATCH v3 03/12] net-shapers: implement NL get operation Paolo Abeni
@ 2024-08-01 13:42 ` Jiri Pirko
2024-08-13 15:17 ` Paolo Abeni
2024-08-01 15:09 ` Jakub Kicinski
2024-08-02 11:53 ` Jiri Pirko
2 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-01 13:42 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Jul 30, 2024 at 10:39:46PM CEST, pabeni@redhat.com wrote:
>Introduce the basic infrastructure to implement the net-shaper
>core functionality. Each network devices carries a net-shaper cache,
>the NL get() operation fetches the data from such cache.
>
>The cache is initially empty, will be fill by the set()/group()
>operation implemented later and is destroyed at device cleanup time.
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
>RFC v2 -> RFC v3:
> - dev_put() -> netdev_put()
>---
> Documentation/networking/kapi.rst | 3 +
> include/linux/netdevice.h | 17 ++
> include/net/net_shaper.h | 158 +++++++++++++++++++
> net/core/dev.c | 2 +
> net/core/dev.h | 6 +
> net/shaper/shaper.c | 251 +++++++++++++++++++++++++++++-
> 6 files changed, 435 insertions(+), 2 deletions(-)
> create mode 100644 include/net/net_shaper.h
>
>diff --git a/Documentation/networking/kapi.rst b/Documentation/networking/kapi.rst
>index ea55f462cefa..98682b9a13ee 100644
>--- a/Documentation/networking/kapi.rst
>+++ b/Documentation/networking/kapi.rst
>@@ -104,6 +104,9 @@ Driver Support
> .. kernel-doc:: include/linux/netdevice.h
> :internal:
>
>+.. kernel-doc:: include/net/net_shaper.h
>+ :internal:
>+
> PHY Support
> -----------
>
>diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
>index 607009150b5f..d3d952be711c 100644
>--- a/include/linux/netdevice.h
>+++ b/include/linux/netdevice.h
>@@ -81,6 +81,8 @@ struct xdp_frame;
> struct xdp_metadata_ops;
> struct xdp_md;
> struct ethtool_netdev_state;
>+struct net_shaper_ops;
>+struct net_shaper_data;
>
> typedef u32 xdp_features_t;
>
>@@ -1598,6 +1600,14 @@ struct net_device_ops {
> int (*ndo_hwtstamp_set)(struct net_device *dev,
> struct kernel_hwtstamp_config *kernel_config,
> struct netlink_ext_ack *extack);
>+
>+#if IS_ENABLED(CONFIG_NET_SHAPER)
>+ /**
>+ * @net_shaper_ops: Device shaping offload operations
>+ * see include/net/net_shapers.h
>+ */
>+ const struct net_shaper_ops *net_shaper_ops;
>+#endif
> };
>
> /**
>@@ -2408,6 +2418,13 @@ struct net_device {
> /** @irq_moder: dim parameters used if IS_ENABLED(CONFIG_DIMLIB). */
> struct dim_irq_moder *irq_moder;
>
>+#if IS_ENABLED(CONFIG_NET_SHAPER)
>+ /**
>+ * @net_shaper_data: data tracking the current shaper status
>+ * see include/net/net_shapers.h
>+ */
>+ struct net_shaper_data *net_shaper_data;
>+#endif
> u8 priv[] ____cacheline_aligned
> __counted_by(priv_len);
> } ____cacheline_aligned;
>diff --git a/include/net/net_shaper.h b/include/net/net_shaper.h
>new file mode 100644
>index 000000000000..8cd65d727e52
>--- /dev/null
>+++ b/include/net/net_shaper.h
>@@ -0,0 +1,158 @@
>+/* SPDX-License-Identifier: GPL-2.0-or-later */
>+
>+#ifndef _NET_SHAPER_H_
>+#define _NET_SHAPER_H_
>+
>+#include <linux/types.h>
>+#include <linux/bits.h>
>+#include <linux/bitfield.h>
>+#include <linux/netdevice.h>
>+#include <linux/netlink.h>
>+
>+#include <uapi/linux/net_shaper.h>
>+
>+/**
>+ * struct net_shaper_info - represents a shaping node on the NIC H/W
>+ * zeroed field are considered not set.
>+ * @handle: Unique identifier for the shaper, see @net_shaper_make_handle
>+ * @parent: Unique identifier for the shaper parent, usually implied. Only
>+ * NET_SHAPER_SCOPE_QUEUE, NET_SHAPER_SCOPE_NETDEV and NET_SHAPER_SCOPE_DETACHED
>+ * can have the parent handle explicitly set, placing such shaper under
>+ * the specified parent.
>+ * @metric: Specify if the bw limits refers to PPS or BPS
>+ * @bw_min: Minimum guaranteed rate for this shaper
>+ * @bw_max: Maximum peak bw allowed for this shaper
>+ * @burst: Maximum burst for the peek rate of this shaper
>+ * @priority: Scheduling priority for this shaper
>+ * @weight: Scheduling weight for this shaper
>+ * @children: Number of nested shapers, accounted only for DETACHED scope
>+ */
>+struct net_shaper_info {
>+ u32 handle;
>+ u32 parent;
>+ enum net_shaper_metric metric;
>+ u64 bw_min;
>+ u64 bw_max;
>+ u64 burst;
>+ u32 priority;
>+ u32 weight;
>+ u32 children;
>+};
>+
>+/**
>+ * define NET_SHAPER_SCOPE_VF - Shaper scope
>+ *
>+ * This shaper scope is not exposed to user-space; the shaper is attached to
>+ * the given virtual function.
>+ */
>+#define NET_SHAPER_SCOPE_VF __NET_SHAPER_SCOPE_MAX
>+
>+/**
>+ * struct net_shaper_ops - Operations on device H/W shapers
>+ *
>+ * The initial shaping configuration ad device initialization is empty/
>+ * a no-op/does not constraint the b/w in any way.
>+ * The network core keeps track of the applied user-configuration in
>+ * per device storage.
>+ *
>+ * Each shaper is uniquely identified within the device with an 'handle',
>+ * dependent on the shaper scope and other data, see @shaper_make_handle()
>+ */
>+struct net_shaper_ops {
>+ /**
>+ * @group: create the specified shapers group
>+ *
>+ * Nest the specified @inputs shapers under the given @output shaper
>+ * on the network device @dev. The @input shaper array size is specified
>+ * by @nr_input.
>+ * Create either the @inputs and the @output shaper as needed,
>+ * otherwise move them as needed. Can't create @inputs shapers with
>+ * NET_SHAPER_SCOPE_DETACHED scope, a separate @group call with such
>+ * shaper as @output is needed.
>+ *
>+ * Returns 0 on group successfully created, otherwise an negative
>+ * error value and set @extack to describe the failure's reason.
>+ */
>+ int (*group)(struct net_device *dev, int nr_input,
>+ const struct net_shaper_info *inputs,
>+ const struct net_shaper_info *output,
>+ struct netlink_ext_ack *extack);
>+
>+ /**
>+ * @set: Updates the specified shaper
>+ *
>+ * Updates or creates the specified @shaper on the given device
>+ * @dev. Can't create NET_SHAPER_SCOPE_DETACHED shapers, use @group
>+ * instead.
>+ *
>+ * Returns 0 on group successfully created, otherwise an negative
>+ * error value and set @extack to describe the failure's reason.
>+ */
>+ int (*set)(struct net_device *dev,
>+ const struct net_shaper_info *shaper,
>+ struct netlink_ext_ack *extack);
>+
>+ /**
>+ * @delete: Removes the specified shaper from the NIC
>+ *
>+ * Removes the shaper configuration as identified by the given @handle
>+ * on the specified device @dev, restoring the default behavior.
>+ *
>+ * Returns 0 on group successfully created, otherwise an negative
>+ * error value and set @extack to describe the failure's reason.
>+ */
>+ int (*delete)(struct net_device *dev, u32 handle,
>+ struct netlink_ext_ack *extack);
>+};
>+
>+#define NET_SHAPER_SCOPE_SHIFT 26
>+#define NET_SHAPER_ID_MASK GENMASK(NET_SHAPER_SCOPE_SHIFT - 1, 0)
>+#define NET_SHAPER_SCOPE_MASK GENMASK(31, NET_SHAPER_SCOPE_SHIFT)
>+
>+#define NET_SHAPER_ID_UNSPEC NET_SHAPER_ID_MASK
>+
>+/**
>+ * net_shaper_make_handle - creates an unique shaper identifier
>+ * @scope: the shaper scope
>+ * @id: the shaper id number
>+ *
>+ * Return: an unique identifier for the shaper
>+ *
>+ * Combines the specified arguments to create an unique identifier for
>+ * the shaper. The @id argument semantic depends on the
>+ * specified scope.
>+ * For @NET_SHAPER_SCOPE_QUEUE_GROUP, @id is the queue group id
>+ * For @NET_SHAPER_SCOPE_QUEUE, @id is the queue number.
>+ * For @NET_SHAPER_SCOPE_VF, @id is virtual function number.
>+ */
>+static inline u32 net_shaper_make_handle(enum net_shaper_scope scope,
>+ int id)
>+{
>+ return FIELD_PREP(NET_SHAPER_SCOPE_MASK, scope) |
>+ FIELD_PREP(NET_SHAPER_ID_MASK, id);
Perhaps some scopes may find only part of u32 as limitting for id in
the future? I find it elegant to have it in single u32 though. u64 may
be nicer (I know, xarray) :)
>+}
>+
>+/**
>+ * net_shaper_handle_scope - extract the scope from the given handle
>+ * @handle: the shaper handle
>+ *
>+ * Return: the corresponding scope
>+ */
>+static inline enum net_shaper_scope net_shaper_handle_scope(u32 handle)
>+{
>+ return FIELD_GET(NET_SHAPER_SCOPE_MASK, handle);
>+}
>+
>+/**
>+ * net_shaper_handle_id - extract the id number from the given handle
>+ * @handle: the shaper handle
>+ *
>+ * Return: the corresponding id number
>+ */
>+static inline int net_shaper_handle_id(u32 handle)
>+{
>+ return FIELD_GET(NET_SHAPER_ID_MASK, handle);
>+}
>+
>+#endif
>+
>diff --git a/net/core/dev.c b/net/core/dev.c
>index 6ea1d20676fb..3dc1dd428eda 100644
>--- a/net/core/dev.c
>+++ b/net/core/dev.c
>@@ -11169,6 +11169,8 @@ void free_netdev(struct net_device *dev)
> /* Flush device addresses */
> dev_addr_flush(dev);
>
>+ dev_shaper_flush(dev);
>+
> list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
> netif_napi_del(p);
>
>diff --git a/net/core/dev.h b/net/core/dev.h
>index 5654325c5b71..e376fc1c867b 100644
>--- a/net/core/dev.h
>+++ b/net/core/dev.h
>@@ -35,6 +35,12 @@ void dev_addr_flush(struct net_device *dev);
> int dev_addr_init(struct net_device *dev);
> void dev_addr_check(struct net_device *dev);
>
>+#if IS_ENABLED(CONFIG_NET_SHAPER)
>+void dev_shaper_flush(struct net_device *dev);
>+#else
>+static inline void dev_shaper_flush(struct net_device *dev) {}
>+#endif
>+
> /* sysctls not referred to from outside net/core/ */
> extern int netdev_unregister_timeout_secs;
> extern int weight_p;
>diff --git a/net/shaper/shaper.c b/net/shaper/shaper.c
>index 49de88c68e2f..5d1d6e600a6a 100644
>--- a/net/shaper/shaper.c
>+++ b/net/shaper/shaper.c
>@@ -1,19 +1,242 @@
> // SPDX-License-Identifier: GPL-2.0-or-later
>
> #include <linux/kernel.h>
>+#include <linux/idr.h>
> #include <linux/skbuff.h>
>+#include <linux/xarray.h>
>+#include <net/net_shaper.h>
>
> #include "shaper_nl_gen.h"
>
>+#include "../core/dev.h"
>+
>+struct net_shaper_data {
>+ struct xarray shapers;
>+ struct idr detached_ids;
Hmm, I wonder what this will be used for :) Anyway, could you move to
patch which is using it?
>+};
>+
>+struct net_shaper_nl_ctx {
>+ u32 start_handle;
>+};
>+
>+static int fill_handle(struct sk_buff *msg, u32 handle, u32 type,
>+ const struct genl_info *info)
>+{
>+ struct nlattr *handle_attr;
>+
>+ if (!handle)
>+ return 0;
>+
>+ handle_attr = nla_nest_start_noflag(msg, type);
>+ if (!handle_attr)
>+ return -EMSGSIZE;
>+
>+ if (nla_put_u32(msg, NET_SHAPER_A_SCOPE,
>+ net_shaper_handle_scope(handle)) ||
>+ nla_put_u32(msg, NET_SHAPER_A_ID,
>+ net_shaper_handle_id(handle)))
>+ goto handle_nest_cancel;
>+
>+ nla_nest_end(msg, handle_attr);
>+ return 0;
>+
>+handle_nest_cancel:
>+ nla_nest_cancel(msg, handle_attr);
>+ return -EMSGSIZE;
>+}
>+
>+static int
>+net_shaper_fill_one(struct sk_buff *msg, struct net_shaper_info *shaper,
>+ const struct genl_info *info)
>+{
>+ void *hdr;
>+
>+ hdr = genlmsg_iput(msg, info);
>+ if (!hdr)
>+ return -EMSGSIZE;
>+
>+ if (fill_handle(msg, shaper->parent, NET_SHAPER_A_PARENT, info) ||
>+ fill_handle(msg, shaper->handle, NET_SHAPER_A_HANDLE, info) ||
>+ nla_put_u32(msg, NET_SHAPER_A_METRIC, shaper->metric) ||
>+ nla_put_uint(msg, NET_SHAPER_A_BW_MIN, shaper->bw_min) ||
>+ nla_put_uint(msg, NET_SHAPER_A_BW_MAX, shaper->bw_max) ||
>+ nla_put_uint(msg, NET_SHAPER_A_BURST, shaper->burst) ||
>+ nla_put_u32(msg, NET_SHAPER_A_PRIORITY, shaper->priority) ||
>+ nla_put_u32(msg, NET_SHAPER_A_WEIGHT, shaper->weight))
>+ goto nla_put_failure;
>+
>+ genlmsg_end(msg, hdr);
>+
>+ return 0;
>+
>+nla_put_failure:
>+ genlmsg_cancel(msg, hdr);
>+ return -EMSGSIZE;
>+}
>+
>+/* On success sets pdev to the relevant device and acquires a reference
>+ * to it
>+ */
>+static int fetch_dev(const struct genl_info *info, struct net_device **pdev)
>+{
>+ struct net *ns = genl_info_net(info);
>+ struct net_device *dev;
>+ int ifindex;
>+
>+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_IFINDEX))
>+ return -EINVAL;
>+
>+ ifindex = nla_get_u32(info->attrs[NET_SHAPER_A_IFINDEX]);
>+ dev = dev_get_by_index(ns, ifindex);
>+ if (!dev) {
>+ GENL_SET_ERR_MSG_FMT(info, "device %d not found", ifindex);
>+ return -EINVAL;
>+ }
>+
>+ if (!dev->netdev_ops->net_shaper_ops) {
>+ GENL_SET_ERR_MSG_FMT(info, "device %s does not support H/W shaper",
>+ dev->name);
>+ netdev_put(dev, NULL);
>+ return -EOPNOTSUPP;
>+ }
>+
>+ *pdev = dev;
>+ return 0;
>+}
>+
>+static struct xarray *__sc_container(struct net_device *dev)
>+{
>+ return dev->net_shaper_data ? &dev->net_shaper_data->shapers : NULL;
>+}
>+
>+/* lookup the given shaper inside the cache */
>+static struct net_shaper_info *sc_lookup(struct net_device *dev, u32 handle)
>+{
>+ struct xarray *xa = __sc_container(dev);
>+
>+ return xa ? xa_load(xa, handle) : NULL;
>+}
>+
>+static int parse_handle(const struct nlattr *attr, const struct genl_info *info,
>+ u32 *handle)
>+{
>+ struct nlattr *tb[NET_SHAPER_A_ID + 1];
>+ struct nlattr *scope_attr, *id_attr;
>+ enum net_shaper_scope scope;
>+ u32 id = 0;
>+ int ret;
>+
>+ ret = nla_parse_nested(tb, NET_SHAPER_A_ID, attr,
>+ net_shaper_handle_nl_policy, info->extack);
>+ if (ret < 0)
>+ return ret;
>+
>+ scope_attr = tb[NET_SHAPER_A_SCOPE];
>+ if (!scope_attr) {
>+ GENL_SET_ERR_MSG(info, "Missing 'scope' attribute for handle");
>+ return -EINVAL;
>+ }
>+
>+ scope = nla_get_u32(scope_attr);
>+
>+ /* the default id for detached scope shapers is an invalid one
>+ * to help the 'group' operation discriminate between new
>+ * detached shaper creation (ID_UNSPEC) and reuse of existing
>+ * shaper (any other value)
>+ */
>+ id_attr = tb[NET_SHAPER_A_ID];
>+ if (id_attr)
>+ id = nla_get_u32(id_attr);
>+ else if (scope == NET_SHAPER_SCOPE_DETACHED)
>+ id = NET_SHAPER_ID_UNSPEC;
>+
>+ *handle = net_shaper_make_handle(scope, id);
>+ return 0;
>+}
>+
> int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
> {
>- return -EOPNOTSUPP;
>+ struct net_shaper_info *shaper;
>+ struct net_device *dev;
>+ struct sk_buff *msg;
>+ u32 handle;
>+ int ret;
>+
>+ ret = fetch_dev(info, &dev);
This is quite net_device centric. Devlink rate shaper should be
eventually visible throught this api as well, won't they? How do you
imagine that?
Could we have various types of binding? Something like:
NET_SHAPER_A_BINDING nest
NET_SHAPER_A_BINDING_IFINDEX u32
or:
NET_SHAPER_A_BINDING nest
NET_SHAPER_A_BINDING_DEVLINK_PORT nest
DEVLINK_ATTR_BUS_NAME string
DEVLINK_ATTR_DEV_NAME string
DEVLINK_ATTR_PORT_INDEX u32
?
>+ if (ret)
>+ return ret;
>+
>+ if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
>+ goto put;
>+
>+ ret = parse_handle(info->attrs[NET_SHAPER_A_HANDLE], info, &handle);
>+ if (ret < 0)
>+ goto put;
>+
>+ shaper = sc_lookup(dev, handle);
>+ if (!shaper) {
>+ GENL_SET_ERR_MSG_FMT(info, "Can't find shaper for handle %x", handle);
>+ ret = -EINVAL;
>+ goto put;
>+ }
>+
>+ msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
>+ if (!msg) {
>+ ret = -ENOMEM;
>+ goto put;
>+ }
>+
>+ ret = net_shaper_fill_one(msg, shaper, info);
>+ if (ret)
>+ goto free_msg;
>+
>+ ret = genlmsg_reply(msg, info);
>+ if (ret)
>+ goto free_msg;
>+
>+put:
>+ netdev_put(dev, NULL);
>+ return ret;
>+
>+free_msg:
>+ nlmsg_free(msg);
>+ goto put;
> }
>
> int net_shaper_nl_get_dumpit(struct sk_buff *skb,
> struct netlink_callback *cb)
> {
>- return -EOPNOTSUPP;
>+ struct net_shaper_nl_ctx *ctx = (struct net_shaper_nl_ctx *)cb->ctx;
>+ const struct genl_info *info = genl_info_dump(cb);
>+ struct net_shaper_info *shaper;
>+ struct net_device *dev;
>+ unsigned long handle;
>+ int ret;
>+
>+ ret = fetch_dev(info, &dev);
>+ if (ret)
>+ return ret;
>+
>+ BUILD_BUG_ON(sizeof(struct net_shaper_nl_ctx) > sizeof(cb->ctx));
>+
>+ /* don't error out dumps performed before any set operation */
>+ if (!dev->net_shaper_data) {
>+ ret = 0;
>+ goto put;
>+ }
>+
>+ xa_for_each_range(&dev->net_shaper_data->shapers, handle, shaper,
>+ ctx->start_handle, U32_MAX) {
>+ ret = net_shaper_fill_one(skb, shaper, info);
>+ if (ret)
>+ goto put;
>+
>+ ctx->start_handle = handle;
>+ }
>+
>+put:
>+ netdev_put(dev, NULL);
>+ return ret;
> }
>
> int net_shaper_nl_set_doit(struct sk_buff *skb, struct genl_info *info)
>@@ -26,6 +249,30 @@ int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
> return -EOPNOTSUPP;
> }
>
>+int net_shaper_nl_group_doit(struct sk_buff *skb, struct genl_info *info)
>+{
>+ return -EOPNOTSUPP;
>+}
>+
>+void dev_shaper_flush(struct net_device *dev)
>+{
>+ struct xarray *xa = __sc_container(dev);
>+ struct net_shaper_info *cur;
>+ unsigned long index;
>+
>+ if (!xa)
>+ return;
>+
>+ xa_lock(xa);
>+ xa_for_each(xa, index, cur) {
>+ __xa_erase(xa, index);
>+ kfree(cur);
>+ }
>+ idr_destroy(&dev->net_shaper_data->detached_ids);
>+ xa_unlock(xa);
>+ kfree(xa);
>+}
>+
> static int __init shaper_init(void)
fetch_dev
fill_handle
parse_handle
sc_lookup
__sc_container
dev_shaper_flush
shaper_init
Could you perhaps maintain net_shaper_ prefix for all of there?
> {
> return genl_register_family(&net_shaper_nl_family);
>--
>2.45.2
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-07-31 21:13 ` Donald Hunter
@ 2024-08-01 14:31 ` Paolo Abeni
2024-08-02 10:57 ` Jiri Pirko
2024-08-02 11:15 ` Donald Hunter
0 siblings, 2 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-08-01 14:31 UTC (permalink / raw)
To: Donald Hunter
Cc: netdev, Jakub Kicinski, Jiri Pirko, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
On 7/31/24 23:13, Donald Hunter wrote:
> Paolo Abeni <pabeni@redhat.com> writes:
>
>> diff --git a/Documentation/netlink/specs/shaper.yaml b/Documentation/netlink/specs/shaper.yaml
>> new file mode 100644
>> index 000000000000..7327f5596fdb
>> --- /dev/null
>> +++ b/Documentation/netlink/specs/shaper.yaml
>
> It's probably more user-friendly to use the same filename as the spec
> name, so net-shaper.yaml
No big objection on my side, but if we enforce 'Name:' to be $(basename
$file .yaml), the 'Name' field becomes redundant.
[...]
>> + render-max: true
>> + entries:
>> + - name: unspec
>> + doc: The scope is not specified
>
> What are the semantics of 'unspec' ? When can it be used?
I guess at this point it can be dropped. It was introduced in a previous
incarnation to represent the port parent - the port does not have a
parent, being the root of the hierarchy.
>> + -
>> + name: port
>> + doc: The root for the whole H/W
>> + -
>> + name: netdev
>> + doc: The main shaper for the given network device.
>
> What are the semantic differences between netdev and port?
netdev == Linux network device
port == wire plug
>> + -
>> + name: queue
>> + doc: The shaper is attached to the given device queue.
>> + -
>> + name: detached
>> + doc: |
>> + The shaper is not attached to any user-visible network
>> + device component and allows nesting and grouping of
>> + queues or others detached shapers.
>
> I assume that shapers are always owned by the netdev regardless of
> attach status?
If you mean that it's up to the netdev clean them up on (netdev)
removal, yes.
>> +>> + -
>> + name: inputs
>> + type: nest
>> + multi-attr: true
>> + nested-attributes: ns-info
>> + doc: |
>> + Describes a set of inputs shapers for a @group operation
>
> The @group renders exactly as-is in the generated htmldocs. There may be
> a more .rst friendly markup you can use that will render better.
Uhm... AFAICS the problem is the target (e.g. 'group') is outside the
htmldoc section itself, I can't find any existing markup to serve this
purpose well. What about sticking to quotes '' everywhere?
FTR, I used @ following the kdoc style.
[...]
>> + -
>> + name: group
>> + doc: |
>> + Group the specified input shapers under the specified
>> + output shaper, eventually creating the latter, if needed.
>> + Input shapers scope must be either @queue or @detached.
>
> It says above that you cannot create a detached shaper, so how do you
> create one to use as an input shaper here? Is this group op more like a
> multi-create op?
The group operation has the main goal of configuring a single WRR or SP
scheduling group atomically. It can creates the needed shapers as
needed, see below.
The need for such operation sparks from some H/W constraints:
https://lore.kernel.org/netdev/9dd818dc-1fef-4633-b388-6ce7272f9cb4@lunn.ch/
>> + Output shaper scope must be either @detached or @netdev.
>> + When using an output @detached scope shaper, if the
>> + @handle @id is not specified, a new shaper of such scope
>> + is created and, otherwise the specified output shaper
>> + must be already existing.
>> + The operation is atomic, on failures the extack is set
>> + accordingly and no change is applied to the device
>> + shaping configuration, otherwise the output shaper
>> + handle is provided as reply.
>> + attribute-set: net-shaper
>> + flags: [ admin-perm ]
>
> Does there need to be a reciprocal 'ungroup' operation? Without it,
> create / group / delete seems like they will have ambiguous semantics.
I guess we need a better description. Can you please tell where/how the
current one is ambiguous?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-01 13:10 ` Jiri Pirko
@ 2024-08-01 14:40 ` Jakub Kicinski
2024-08-01 15:12 ` Paolo Abeni
1 sibling, 0 replies; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-01 14:40 UTC (permalink / raw)
To: Jiri Pirko
Cc: Paolo Abeni, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Thu, 1 Aug 2024 15:10:50 +0200 Jiri Pirko wrote:
> >+ -
> >+ name: netdev
> >+ doc: The main shaper for the given network device.
> >+ -
> >+ name: queue
> >+ doc: The shaper is attached to the given device queue.
> >+ -
> >+ name: detached
> >+ doc: |
> >+ The shaper is not attached to any user-visible network
> >+ device component and allows nesting and grouping of
> >+ queues or others detached shapers.
>
> What is the purpose of the "detached" thing?
FWIW I also find the proposed model and naming confusing, but didn't
want to object too hard in case it was just me. The model conflates
attachment points with shapers themselves, and (which perhaps is more
subjective) shapers with mux nodes.
This is already visible in the first example of the cover letter:
./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
--do set --json '{"ifindex":'$IFINDEX',
"shaper": {"handle":
{"scope": "queue", "id":'$QUEUEID' },
"bw-max": 2000000 }}'
We are "set"ting a shaper on a queue, not "add"ing it, because
all attachment points implicitly have shapers(?) In my mind attachment
points is where traffic comes from or goes to, shapers must be created
to exist. And _every_ shaper has an ingress and egress, not just the
ones in the middle (i.e. which aren't directly next to an attachment
point) :(
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 04/12] net-shapers: implement NL set and delete operations
2024-07-30 20:39 ` [PATCH v3 04/12] net-shapers: implement NL set and delete operations Paolo Abeni
@ 2024-08-01 15:00 ` Jakub Kicinski
2024-08-01 15:25 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-01 15:00 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Tue, 30 Jul 2024 22:39:47 +0200 Paolo Abeni wrote:
> +static int net_shaper_delete(struct net_device *dev, u32 handle,
> + struct netlink_ext_ack *extack)
> +{
> + struct net_shaper_info *parent, *shaper = sc_lookup(dev, handle);
> + struct xarray *xa = __sc_container(dev);
> + enum net_shaper_scope pscope;
> + u32 parent_handle;
> + int ret;
> +
> + if (!xa || !shaper) {
> + NL_SET_ERR_MSG_FMT(extack, "Shaper %x not found", handle);
below the print format for shaper id is %d
also just point at the attribute with NL_SET_BAD_ATTR(),
we shouldn't bloat the kernel with strings for trivial errors
> + return -EINVAL;
ENOENT
> + }
> +
> + if (is_detached(handle) && shaper->children > 0) {
> + NL_SET_ERR_MSG_FMT(extack, "Can't delete detached shaper %d with %d child nodes",
> + handle, shaper->children);
> + return -EINVAL;
I'd say EBUSY
> + }
> +
> + while (shaper) {
> + parent_handle = shaper->parent;
> + pscope = net_shaper_handle_scope(parent_handle);
> +
> + ret = dev->netdev_ops->net_shaper_ops->delete(dev, handle,
> + extack);
> + if (ret < 0)
> + return ret;
> +
> + xa_lock(xa);
> + __xa_erase(xa, handle);
> + if (is_detached(handle))
> + idr_remove(&dev->net_shaper_data->detached_ids,
> + net_shaper_handle_id(handle));
> + xa_unlock(xa);
> + kfree(shaper);
> + shaper = NULL;
IIUC child is the input / ingress node? (The parentage terminology
is a bit ambiguous in this case, although I must admit I keep catching
myself trying to use it, too).
Does "deleting a queue" return it to the "implicit mux" at the
global level? If we look at the delegation use case - when queue
is "deleted" from a container-controlled mux it should go back to
the group created by the orchestrator, not "escpate" to global scope,
right?
> + if (pscope == NET_SHAPER_SCOPE_DETACHED) {
> + parent = sc_lookup(dev, parent_handle);
> + if (parent && !--parent->children) {
> + shaper = parent;
> + handle = parent_handle;
> + }
> + }
> + }
> + return 0;
> }
>
> int net_shaper_nl_delete_doit(struct sk_buff *skb, struct genl_info *info)
> {
> - return -EOPNOTSUPP;
> + struct net_device *dev;
> + u32 handle;
> + int ret;
> +
> + if (GENL_REQ_ATTR_CHECK(info, NET_SHAPER_A_HANDLE))
> + return -EINVAL;
> +
> + ret = fetch_dev(info, &dev);
> + if (ret)
> + return ret;
Possibly a candidate for a "pre" handler, since multiple ops call it?
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-07-30 20:39 ` [PATCH v3 03/12] net-shapers: implement NL get operation Paolo Abeni
2024-08-01 13:42 ` Jiri Pirko
@ 2024-08-01 15:09 ` Jakub Kicinski
2024-08-02 11:53 ` Jiri Pirko
2 siblings, 0 replies; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-01 15:09 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Tue, 30 Jul 2024 22:39:46 +0200 Paolo Abeni wrote:
> +#include <linux/types.h>
> +#include <linux/bits.h>
> +#include <linux/bitfield.h>
> +#include <linux/netdevice.h>
> +#include <linux/netlink.h>
most of these could be replaced by forward declaration of types
> +#include <uapi/linux/net_shaper.h>
> +
> +/**
> + * struct net_shaper_info - represents a shaping node on the NIC H/W
> + * zeroed field are considered not set.
> + * @handle: Unique identifier for the shaper, see @net_shaper_make_handle
> + * @parent: Unique identifier for the shaper parent, usually implied. Only
> + * NET_SHAPER_SCOPE_QUEUE, NET_SHAPER_SCOPE_NETDEV and NET_SHAPER_SCOPE_DETACHED
> + * can have the parent handle explicitly set, placing such shaper under
> + * the specified parent.
> + * @metric: Specify if the bw limits refers to PPS or BPS
> + * @bw_min: Minimum guaranteed rate for this shaper
> + * @bw_max: Maximum peak bw allowed for this shaper
> + * @burst: Maximum burst for the peek rate of this shaper
> + * @priority: Scheduling priority for this shaper
> + * @weight: Scheduling weight for this shaper
> + * @children: Number of nested shapers, accounted only for DETACHED scope
> + */
> +struct net_shaper_info {
> + u32 handle;
> + u32 parent;
> + enum net_shaper_metric metric;
> + u64 bw_min;
> + u64 bw_max;
> + u64 burst;
> + u32 priority;
> + u32 weight;
> + u32 children;
> +};
> +
> +/**
> + * define NET_SHAPER_SCOPE_VF - Shaper scope
> + *
> + * This shaper scope is not exposed to user-space; the shaper is attached to
> + * the given virtual function.
> + */
> +#define NET_SHAPER_SCOPE_VF __NET_SHAPER_SCOPE_MAX
> +
> +/**
> + * struct net_shaper_ops - Operations on device H/W shapers
> + *
> + * The initial shaping configuration ad device initialization is empty/
> + * a no-op/does not constraint the b/w in any way.
> + * The network core keeps track of the applied user-configuration in
> + * per device storage.
> + *
> + * Each shaper is uniquely identified within the device with an 'handle',
> + * dependent on the shaper scope and other data, see @shaper_make_handle()
we need to document locking, seems like the locking is.. missing?
Or at least I don't see what prevents two deletes from racing.
> + */
> +struct net_shaper_ops {
> + /**
> + * @group: create the specified shapers group
> + *
> + * Nest the specified @inputs shapers under the given @output shaper
> + * on the network device @dev. The @input shaper array size is specified
> + * by @nr_input.
> + * Create either the @inputs and the @output shaper as needed,
> + * otherwise move them as needed. Can't create @inputs shapers with
> + * NET_SHAPER_SCOPE_DETACHED scope, a separate @group call with such
> + * shaper as @output is needed.
> + *
> + * Returns 0 on group successfully created, otherwise an negative
> + * error value and set @extack to describe the failure's reason.
> + */
> + int (*group)(struct net_device *dev, int nr_input,
> + const struct net_shaper_info *inputs,
> + const struct net_shaper_info *output,
> + struct netlink_ext_ack *extack);
> +
> + /* the default id for detached scope shapers is an invalid one
> + * to help the 'group' operation discriminate between new
> + * detached shaper creation (ID_UNSPEC) and reuse of existing
> + * shaper (any other value)
> + */
> + id_attr = tb[NET_SHAPER_A_ID];
> + if (id_attr)
> + id = nla_get_u32(id_attr);
double space
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-01 13:10 ` Jiri Pirko
2024-08-01 14:40 ` Jakub Kicinski
@ 2024-08-01 15:12 ` Paolo Abeni
2024-08-02 10:49 ` Jiri Pirko
1 sibling, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-01 15:12 UTC (permalink / raw)
To: Jiri Pirko
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/1/24 15:10, Jiri Pirko wrote:
> Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
>> + type: enum
>> + name: scope
>> + doc: the different scopes where a shaper can be attached
>> + render-max: true
>> + entries:
>> + - name: unspec
>> + doc: The scope is not specified
>> + -
>> + name: port
>> + doc: The root for the whole H/W
>
> What is this "port"?
~ a wire plug.
>> + -
>> + name: netdev
>> + doc: The main shaper for the given network device.
>> + -
>> + name: queue
>> + doc: The shaper is attached to the given device queue.
>> + -
>> + name: detached
>> + doc: |
>> + The shaper is not attached to any user-visible network
>> + device component and allows nesting and grouping of
>> + queues or others detached shapers.
>
> What is the purpose of the "detached" thing?
I fear I can't escape reusing most of the wording above. 'detached'
nodes goal is to create groups of other shapers. i.e. queue groups,
allowing multiple levels nesting, i.e. to implement this kind of hierarchy:
q1 ----- \
q2 - \SP / RR ------
q3 - / \
q4 - \ SP -> (netdev)
q5 - / /
/
q6 - \ RR
q7 - /
where q1..q7 are queue-level shapers and all the SP/RR are 'detached'
one. The conf. does not necessary make any functional sense, just to
describe the things.
>> + -
>> + name: group
>> + doc: |
>> + Group the specified input shapers under the specified
>> + output shaper, eventually creating the latter, if needed.
>> + Input shapers scope must be either @queue or @detached.
>> + Output shaper scope must be either @detached or @netdev.
>> + When using an output @detached scope shaper, if the
>> + @handle @id is not specified, a new shaper of such scope
>> + is created and, otherwise the specified output shaper
>> + must be already existing.
>
> I'm lost. Could this designt be described in details in the doc I asked
> in the cover letter? :/ Please.
I'm unsure if the context information here and in the previous replies
helped somehow.
The group operation creates and configure a scheduling group, i.e. this
q1 ----- \
q2 - \SP / RR ------
q3 - / \
q4 - \ SP -> (netdev)
q5 - / /
/
q6 - \ RR
q7 - /
can be create with:
group(inputs:[q6, q7], output:[detached,parent:netdev])
group(inputs:[q4, q5], output:[detached,parent:netdev])
group(inputs:[q1], output:[detached,parent:netdev])
group(inputs:[q2,q3], output:[detached,parent:<the detached shaper
create above>])
I'm unsure if this the kind of info you are looking for?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 04/12] net-shapers: implement NL set and delete operations
2024-08-01 15:00 ` Jakub Kicinski
@ 2024-08-01 15:25 ` Paolo Abeni
2024-08-01 15:39 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-01 15:25 UTC (permalink / raw)
To: Jakub Kicinski
Cc: netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/1/24 17:00, Jakub Kicinski wrote:
> On Tue, 30 Jul 2024 22:39:47 +0200 Paolo Abeni wrote:
>> + while (shaper) {
>> + parent_handle = shaper->parent;
>> + pscope = net_shaper_handle_scope(parent_handle);
>> +
>> + ret = dev->netdev_ops->net_shaper_ops->delete(dev, handle,
>> + extack);
>> + if (ret < 0)
>> + return ret;
>> +
>> + xa_lock(xa);
>> + __xa_erase(xa, handle);
>> + if (is_detached(handle))
>> + idr_remove(&dev->net_shaper_data->detached_ids,
>> + net_shaper_handle_id(handle));
>> + xa_unlock(xa);
>> + kfree(shaper);
>> + shaper = NULL;
>
> IIUC child is the input / ingress node?
Yes.
> Does "deleting a queue" return it to the "implicit mux" at the
> global level?
Yes
> If we look at the delegation use case - when queue
> is "deleted" from a container-controlled mux it should go back to
> the group created by the orchestrator, not "escpate" to global scope,
> right?
When deleting a queue-level shaper, the orchestrator is "returning" the
ownership of the queue from the container to the host. If the container
wants to move the queue around e.g. from:
q1 ----- \
q2 - \SP1/ RR1
q3 - / \
q4 - \ RR2 -> RR(root)
q5 - / /
q6 - \ RR3
q7 - /
to:
q1 ----- \
q2 ----- RR1
q3 ---- / \
q4 - \ RR2 -> RR(root)
q5 - / /
q6 - \ RR3
q7 - /
It can do it with a group() operation:
group(inputs:[q2,q3],output:[RR1])
That will implicitly also delete SP1.
Side note, I just noticed that the current code is bugged WRT this last
operation and will not delete SP1.
Cheers,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 04/12] net-shapers: implement NL set and delete operations
2024-08-01 15:25 ` Paolo Abeni
@ 2024-08-01 15:39 ` Jakub Kicinski
2024-08-02 16:15 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-01 15:39 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Thu, 1 Aug 2024 17:25:50 +0200 Paolo Abeni wrote:
> When deleting a queue-level shaper, the orchestrator is "returning" the
> ownership of the queue from the container to the host. If the container
> wants to move the queue around e.g. from:
>
> q1 ----- \
> q2 - \SP1/ RR1
> q3 - / \
> q4 - \ RR2 -> RR(root)
> q5 - / /
> q6 - \ RR3
> q7 - /
>
> to:
>
> q1 ----- \
> q2 ----- RR1
> q3 ---- / \
> q4 - \ RR2 -> RR(root)
> q5 - / /
> q6 - \ RR3
> q7 - /
>
> It can do it with a group() operation:
>
> group(inputs:[q2,q3],output:[RR1])
Isn't that a bit odd? The container was not supposed to know / care
about RR1's existence. We achieve this with group() by implicitly
inheriting the egress node if all grouped entities shared one.
Delete IMO should act here like a "ungroup" operation, meaning that:
1) we're deleting SP1, not q1, q2
2) inputs go "downstream" instead getting ejected into global level
Also, in the first example from the cover letter we "set" a shaper on
the queue, it feels a little ambiguous whether "delete queue" is
purely clearing such per-queue shaping, or also has implications
for the hierarchy.
Coincidentally, others may disagree, but I'd point to tests in patch
8 for examples of how the thing works, instead the cover letter samples.
> That will implicitly also delete SP1.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-01 15:12 ` Paolo Abeni
@ 2024-08-02 10:49 ` Jiri Pirko
2024-08-05 15:11 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-02 10:49 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Thu, Aug 01, 2024 at 05:12:01PM CEST, pabeni@redhat.com wrote:
>On 8/1/24 15:10, Jiri Pirko wrote:
>> Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
>> > + type: enum
>> > + name: scope
>> > + doc: the different scopes where a shaper can be attached
>> > + render-max: true
>> > + entries:
>> > + - name: unspec
>> > + doc: The scope is not specified
>> > + -
>> > + name: port
>> > + doc: The root for the whole H/W
>>
>> What is this "port"?
>
>~ a wire plug.
What's "wire plug"? What of existing kernel objects this relates to? Is
it a devlink port?
>
>> > + -
>> > + name: netdev
>> > + doc: The main shaper for the given network device.
>> > + -
>> > + name: queue
>> > + doc: The shaper is attached to the given device queue.
>> > + -
>> > + name: detached
>> > + doc: |
>> > + The shaper is not attached to any user-visible network
>> > + device component and allows nesting and grouping of
>> > + queues or others detached shapers.
>>
>> What is the purpose of the "detached" thing?
>
>I fear I can't escape reusing most of the wording above. 'detached' nodes
>goal is to create groups of other shapers. i.e. queue groups,
>allowing multiple levels nesting, i.e. to implement this kind of hierarchy:
>
>q1 ----- \
>q2 - \SP / RR ------
>q3 - / \
> q4 - \ SP -> (netdev)
> q5 - / /
> /
> q6 - \ RR
> q7 - /
>
>where q1..q7 are queue-level shapers and all the SP/RR are 'detached' one.
>The conf. does not necessary make any functional sense, just to describe the
>things.
Can you "attach" the "detached" ones? They are "detached" from what?
>
>> > + -
>> > + name: group
>> > + doc: |
>> > + Group the specified input shapers under the specified
>> > + output shaper, eventually creating the latter, if needed.
>> > + Input shapers scope must be either @queue or @detached.
>> > + Output shaper scope must be either @detached or @netdev.
>> > + When using an output @detached scope shaper, if the
>> > + @handle @id is not specified, a new shaper of such scope
>> > + is created and, otherwise the specified output shaper
>> > + must be already existing.
>>
>> I'm lost. Could this designt be described in details in the doc I asked
>> in the cover letter? :/ Please.
>
>I'm unsure if the context information here and in the previous replies helped
>somehow.
>
>The group operation creates and configure a scheduling group, i.e. this
>
>q1 ----- \
>q2 - \SP / RR ------
>q3 - / \
> q4 - \ SP -> (netdev)
> q5 - / /
> /
> q6 - \ RR
> q7 - /
>
>can be create with:
>
>group(inputs:[q6, q7], output:[detached,parent:netdev])
>group(inputs:[q4, q5], output:[detached,parent:netdev])
>group(inputs:[q1], output:[detached,parent:netdev])
>group(inputs:[q2,q3], output:[detached,parent:<the detached shaper create
>above>])
So by "inputs" and "output" you are basically building a tree. In
devlink rate, we have leaf and node, which is in sync with standard tree
terminology.
If what you are building is tree, why don't you use the same
terminology? If you are building tree, you just need to have the link to
upper noded (output in your terminology). Why you have "inputs"? Isn't
that redundant?
If this is not tree, what's that? Can for example q6 be input of 2
different groups? How is that supposed to work?
>
>I'm unsure if this the kind of info you are looking for?
I'm trying to understand the design. Have to say I'm just confused :/
>
>Thanks,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-01 14:31 ` Paolo Abeni
@ 2024-08-02 10:57 ` Jiri Pirko
2024-08-02 11:15 ` Donald Hunter
1 sibling, 0 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-02 10:57 UTC (permalink / raw)
To: Paolo Abeni
Cc: Donald Hunter, netdev, Jakub Kicinski, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Thu, Aug 01, 2024 at 04:31:04PM CEST, pabeni@redhat.com wrote:
>On 7/31/24 23:13, Donald Hunter wrote:
>> Paolo Abeni <pabeni@redhat.com> writes:
>>
>> > diff --git a/Documentation/netlink/specs/shaper.yaml b/Documentation/netlink/specs/shaper.yaml
>> > new file mode 100644
>> > index 000000000000..7327f5596fdb
>> > --- /dev/null
>> > +++ b/Documentation/netlink/specs/shaper.yaml
>>
>> It's probably more user-friendly to use the same filename as the spec
>> name, so net-shaper.yaml
>
>No big objection on my side, but if we enforce 'Name:' to be $(basename $file
>.yaml), the 'Name' field becomes redundant.
I agree with Donald, better to stay consistent.
>
>[...]
>> > + render-max: true
>> > + entries:
>> > + - name: unspec
>> > + doc: The scope is not specified
>>
>> What are the semantics of 'unspec' ? When can it be used?
>
>I guess at this point it can be dropped. It was introduced in a previous
>incarnation to represent the port parent - the port does not have a parent,
>being the root of the hierarchy.
>
>> > + -
>> > + name: port
>> > + doc: The root for the whole H/W
>> > + -
>> > + name: netdev
>> > + doc: The main shaper for the given network device.
>>
>> What are the semantic differences between netdev and port?
I'm happy that I'm not the only one in the dark :)
>
>netdev == Linux network device
>port == wire plug
>
>> > + -
>> > + name: queue
>> > + doc: The shaper is attached to the given device queue.
>> > + -
>> > + name: detached
>> > + doc: |
>> > + The shaper is not attached to any user-visible network
>> > + device component and allows nesting and grouping of
>> > + queues or others detached shapers.
>>
>> I assume that shapers are always owned by the netdev regardless of
>> attach status?
But is it a "status"? It is a scope, can't change. I see you probably
got the same confusion as I got, expecting that this can be attached
somehow.
>
>If you mean that it's up to the netdev clean them up on (netdev) removal,
>yes.
>
[...]
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-01 14:31 ` Paolo Abeni
2024-08-02 10:57 ` Jiri Pirko
@ 2024-08-02 11:15 ` Donald Hunter
2024-08-05 14:35 ` Paolo Abeni
1 sibling, 1 reply; 91+ messages in thread
From: Donald Hunter @ 2024-08-02 11:15 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Jiri Pirko, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Paolo Abeni <pabeni@redhat.com> writes:
> On 7/31/24 23:13, Donald Hunter wrote:
>> Paolo Abeni <pabeni@redhat.com> writes:
>>
>>> + name: inputs
>>> + type: nest
>>> + multi-attr: true
>>> + nested-attributes: ns-info
>>> + doc: |
>>> + Describes a set of inputs shapers for a @group operation
>> The @group renders exactly as-is in the generated htmldocs. There may be
>> a more .rst friendly markup you can use that will render better.
>
> Uhm... AFAICS the problem is the target (e.g. 'group') is outside the htmldoc section itself, I
> can't find any existing markup to serve this purpose well. What about sticking to quotes ''
> everywhere?
>
> FTR, I used @ following the kdoc style.
Yeah, I was just thinking of using .rst markup like ``code`` or
`italics`, but the meaning of @ is pretty obvious when reading the spec.
If you stick with @ then we could always teach ynl-to-rst to render it
as ``code``.
>
> [...]
>>> + -
>>> + name: group
>>> + doc: |
>>> + Group the specified input shapers under the specified
>>> + output shaper, eventually creating the latter, if needed.
>>> + Input shapers scope must be either @queue or @detached.
>> It says above that you cannot create a detached shaper, so how do you
>> create one to use as an input shaper here? Is this group op more like a
>> multi-create op?
>
> The group operation has the main goal of configuring a single WRR or SP scheduling group
> atomically. It can creates the needed shapers as needed, see below.
>
> The need for such operation sparks from some H/W constraints:
>
> https://lore.kernel.org/netdev/9dd818dc-1fef-4633-b388-6ce7272f9cb4@lunn.ch/
>
>>> + Output shaper scope must be either @detached or @netdev.
>>> + When using an output @detached scope shaper, if the
>>> + @handle @id is not specified, a new shaper of such scope
>>> + is created and, otherwise the specified output shaper
>>> + must be already existing.
>>> + The operation is atomic, on failures the extack is set
>>> + accordingly and no change is applied to the device
>>> + shaping configuration, otherwise the output shaper
>>> + handle is provided as reply.
>>> + attribute-set: net-shaper
>>> + flags: [ admin-perm ]
>> Does there need to be a reciprocal 'ungroup' operation? Without it,
>> create / group / delete seems like they will have ambiguous semantics.
>
> I guess we need a better description. Can you please tell where/how the current one is
> ambiguous?
My expectation for 'group' would be to group existing things, with a
reciprocal 'ungroup' operation. I think you intend 'group' to both be
able to group existing shapers/groups and create a group of shapers.
Am I right in saying that delete lets you delete something from a group
(with side-effect of deleting group if it becomes empty), or delete a
whole group?
It feels a lot like each of 'set', 'group' and 'delete' are doing
multiple things and the interaction between them all becomes challenging
to describe, or to handle all the corner cases. I think part of the
problem is the mixed terminology of input, output for groups, handle,
parent for shapers and using detached to differentiate from 'implicitly
attached to a resource'.
Perhaps the API would be better if you had:
- shaper-new
- shaper-delete
- shaper-get/dump
- shaper-set
- group-new
- group-delete
- group-get/dump
- group-set
If you went with Jakub's suggestion to give every shaper n x inputs and
an output, then you could recombine groups and shapers and just have 4
ops. And you could rename 'detached' to 'shaper' so that an attachment
is one of port, netdev, queue or shaper.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-07-30 20:39 ` [PATCH v3 02/12] netlink: spec: add shaper YAML spec Paolo Abeni
2024-07-31 21:13 ` Donald Hunter
2024-08-01 13:10 ` Jiri Pirko
@ 2024-08-02 11:19 ` Jiri Pirko
2024-08-02 11:26 ` Jiri Pirko
2024-08-02 16:04 ` Jiri Pirko
4 siblings, 0 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-02 11:19 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
[...]
>+ -
>+ name: inputs
>+ type: nest
>+ multi-attr: true
>+ nested-attributes: ns-info
Hmm, you sometimes use this "ns-" prefix. Why? Does it mean
"net-shaper"?
This results to:
net_shaper_ns_info_nl_policy
Wouldn't it be better to name it just "info":
net_shaper_info_nl_policy
And for ns-output-info, just "output-info":
net_shaper_output_info_nl_policy
?
>+ doc: |
>+ Describes a set of inputs shapers for a @group operation
>+ -
>+ name: output
>+ type: nest
>+ nested-attributes: ns-output-info
>+ doc: |
>+ Describes the output shaper for a @group operation
>+ Differently from @inputs and @shaper allow specifying
>+ the shaper parent handle, too.
>+
>+ -
>+ name: shaper
>+ type: nest
>+ nested-attributes: ns-info
>+ doc: |
>+ Describes a single shaper for a @set operation
>+ -
>+ name: handle
>+ subset-of: net-shaper
>+ attributes:
>+ -
>+ name: scope
>+ -
>+ name: id
>+ -
>+ name: ns-info
>+ subset-of: net-shaper
>+ attributes:
>+ -
>+ name: handle
>+ -
>+ name: metric
>+ -
>+ name: bw-min
>+ -
>+ name: bw-max
>+ -
>+ name: burst
>+ -
>+ name: priority
>+ -
>+ name: weight
>+ -
>+ name: ns-output-info
>+ subset-of: net-shaper
>+ attributes:
>+ -
>+ name: parent
>+ -
>+ name: handle
>+ -
>+ name: metric
>+ -
>+ name: bw-min
>+ -
>+ name: bw-max
>+ -
>+ name: burst
>+ -
>+ name: priority
>+ -
>+ name: weight
>+
[...]
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 06/12] netlink: spec: add shaper introspection support
2024-07-30 20:39 ` [PATCH v3 06/12] netlink: spec: add shaper introspection support Paolo Abeni
@ 2024-08-02 11:21 ` Donald Hunter
0 siblings, 0 replies; 91+ messages in thread
From: Donald Hunter @ 2024-08-02 11:21 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Jiri Pirko, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Paolo Abeni <pabeni@redhat.com> writes:
> + -
> + name: support-nesting
> + type: flag
> + doc: |
> + the device supports nesting shaper belonging to this scope
Nit: capitalize all the doc strings for consistency.
> + below 'detached' scoped shapers. Only 'queue' and 'detached'
> + scope and flag 'support-nesting'.
'and flag' looks like a typo. Do you mean 'can have flag'?
> + -
> + name: cap-get
> + doc: |
> + Get / Dump the shaper capabilities supported by the given device
> + attribute-set: capabilities
> +
> + do:
> + request:
> + attributes:
> + - ifindex
> + - scope
> + reply:
> + attributes: &cap-attrs
> + - support-metric-bps
> + - support-metric-pps
> + - support-nesting
> + - support-bw-min
> + - support-bw-max
> + - support-burst
> + - support-priority
> + - support-weight
> +
> + dump:
> + request:
> + attributes:
> + - ifindex
> + reply:
> + attributes: *cap-attrs
scope is missing from the dump reply
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-07-30 20:39 ` [PATCH v3 02/12] netlink: spec: add shaper YAML spec Paolo Abeni
` (2 preceding siblings ...)
2024-08-02 11:19 ` Jiri Pirko
@ 2024-08-02 11:26 ` Jiri Pirko
2024-08-02 16:04 ` Jiri Pirko
4 siblings, 0 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-02 11:26 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
[...]
>+const struct nla_policy net_shaper_ns_info_nl_policy[NET_SHAPER_A_WEIGHT + 1] = {
>+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
>+ [NET_SHAPER_A_METRIC] = NLA_POLICY_MAX(NLA_U32, 1),
>+ [NET_SHAPER_A_BW_MIN] = { .type = NLA_UINT, },
>+ [NET_SHAPER_A_BW_MAX] = { .type = NLA_UINT, },
>+ [NET_SHAPER_A_BURST] = { .type = NLA_UINT, },
>+ [NET_SHAPER_A_PRIORITY] = { .type = NLA_U32, },
>+ [NET_SHAPER_A_WEIGHT] = { .type = NLA_U32, },
>+};
>+
>+const struct nla_policy net_shaper_ns_output_info_nl_policy[NET_SHAPER_A_PARENT + 1] = {
>+ [NET_SHAPER_A_PARENT] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
>+ [NET_SHAPER_A_HANDLE] = NLA_POLICY_NESTED(net_shaper_handle_nl_policy),
>+ [NET_SHAPER_A_METRIC] = NLA_POLICY_MAX(NLA_U32, 1),
>+ [NET_SHAPER_A_BW_MIN] = { .type = NLA_UINT, },
>+ [NET_SHAPER_A_BW_MAX] = { .type = NLA_UINT, },
>+ [NET_SHAPER_A_BURST] = { .type = NLA_UINT, },
>+ [NET_SHAPER_A_PRIORITY] = { .type = NLA_U32, },
>+ [NET_SHAPER_A_WEIGHT] = { .type = NLA_U32, },
Since this is the same set as above only extended by parent, wouldn't it
be better to nest it and have it only once?
[...]
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-07-30 20:39 ` [PATCH v3 03/12] net-shapers: implement NL get operation Paolo Abeni
2024-08-01 13:42 ` Jiri Pirko
2024-08-01 15:09 ` Jakub Kicinski
@ 2024-08-02 11:53 ` Jiri Pirko
2 siblings, 0 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-02 11:53 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Jul 30, 2024 at 10:39:46PM CEST, pabeni@redhat.com wrote:
[...]
>+
>+/**
>+ * struct net_shaper_info - represents a shaping node on the NIC H/W
>+ * zeroed field are considered not set.
>+ * @handle: Unique identifier for the shaper, see @net_shaper_make_handle
>+ * @parent: Unique identifier for the shaper parent, usually implied. Only
>+ * NET_SHAPER_SCOPE_QUEUE, NET_SHAPER_SCOPE_NETDEV and NET_SHAPER_SCOPE_DETACHED
>+ * can have the parent handle explicitly set, placing such shaper under
>+ * the specified parent.
>+ * @metric: Specify if the bw limits refers to PPS or BPS
>+ * @bw_min: Minimum guaranteed rate for this shaper
>+ * @bw_max: Maximum peak bw allowed for this shaper
"rate" to be consitent with the text one line above?
>+ * @burst: Maximum burst for the peek rate of this shaper
>+ * @priority: Scheduling priority for this shaper
>+ * @weight: Scheduling weight for this shaper
>+ * @children: Number of nested shapers, accounted only for DETACHED scope
"children" are "inputs"? Or "nested shapers". I'm lost in the
terminology, again.
>+ */
>+struct net_shaper_info {
>+ u32 handle;
I wonder if the handle should be part of this structure. The structure
describes the shaper attributes. The handle is identification. In the
ops below, you sometimes pass the handle as a part of net_shaper_info
structure (group, set), yet for delete op you pass handle directly.
Wouldn't it be nicer to pass the same handle attr every time so it is
clear what it is?
>+ u32 parent;
>+ enum net_shaper_metric metric;
>+ u64 bw_min;
>+ u64 bw_max;
>+ u64 burst;
>+ u32 priority;
>+ u32 weight;
>+ u32 children;
>+};
>+
>+/**
>+ * define NET_SHAPER_SCOPE_VF - Shaper scope
>+ *
>+ * This shaper scope is not exposed to user-space; the shaper is attached to
>+ * the given virtual function.
>+ */
>+#define NET_SHAPER_SCOPE_VF __NET_SHAPER_SCOPE_MAX
This is unused, why do you introduce it here?
>+
>+/**
>+ * struct net_shaper_ops - Operations on device H/W shapers
>+ *
>+ * The initial shaping configuration ad device initialization is empty/
s/as/and ?
>+ * a no-op/does not constraint the b/w in any way.
"bw","b/w","rate". Better to be consistent, I suppose.
>+ * The network core keeps track of the applied user-configuration in
>+ * per device storage.
"per device storage" reads weird. "net_device structure"? IDK.
>+ *
>+ * Each shaper is uniquely identified within the device with an 'handle',
>+ * dependent on the shaper scope and other data, see @shaper_make_handle()
The name of the function is net_shaper_make_handle(). You refer to
"other data". What is that? I see no such thing in the code.
>+ */
>+struct net_shaper_ops {
>+ /**
>+ * @group: create the specified shapers group
>+ *
>+ * Nest the specified @inputs shapers under the given @output shaper
>+ * on the network device @dev. The @input shaper array size is specified
>+ * by @nr_input.
>+ * Create either the @inputs and the @output shaper as needed,
>+ * otherwise move them as needed.
I don't understand the sentense. So this creates either passed inputs
and output shaper, or if they already exist, it links them together in
desired way? Or what do you mean by "move"?
>+ Can't create @inputs shapers with
>+ * NET_SHAPER_SCOPE_DETACHED scope, a separate @group call with such
>+ * shaper as @output is needed.
I don't understand meaning of the sentence. The caller should make sure
that NET_SHAPER_SCOPE_DETACHED is not in the inputs, so drivers do not
have to sanitize it one by one. Then this sentence is not needed here,
is it?
>+ *
>+ * Returns 0 on group successfully created, otherwise an negative
>+ * error value and set @extack to describe the failure's reason.
>+ */
>+ int (*group)(struct net_device *dev, int nr_input,
Perhaps better name would be "nr_inputs" or "inputs_count"?
>+ const struct net_shaper_info *inputs,
>+ const struct net_shaper_info *output,
>+ struct netlink_ext_ack *extack);
>+
>+ /**
>+ * @set: Updates the specified shaper
>+ *
>+ * Updates or creates the specified @shaper on the given device
>+ * @dev. Can't create NET_SHAPER_SCOPE_DETACHED shapers, use @group
>+ * instead.
Again, similar to the NET_SHAPER_SCOPE_DETACHED comment above, the
caller should sanitize this. Why is this needed here then?
>+ *
>+ * Returns 0 on group successfully created, otherwise an negative
Which group? C&P mistake I suppose?
>+ * error value and set @extack to describe the failure's reason.
>+ */
>+ int (*set)(struct net_device *dev,
>+ const struct net_shaper_info *shaper,
>+ struct netlink_ext_ack *extack);
>+
>+ /**
>+ * @delete: Removes the specified shaper from the NIC
>+ *
>+ * Removes the shaper configuration as identified by the given @handle
>+ * on the specified device @dev, restoring the default behavior.
>+ *
>+ * Returns 0 on group successfully created, otherwise an negative
Which group? C&P mistake I suppose?
>+ * error value and set @extack to describe the failure's reason.
>+ */
>+ int (*delete)(struct net_device *dev, u32 handle,
>+ struct netlink_ext_ack *extack);
>+};
>+
>+#define NET_SHAPER_SCOPE_SHIFT 26
>+#define NET_SHAPER_ID_MASK GENMASK(NET_SHAPER_SCOPE_SHIFT - 1, 0)
>+#define NET_SHAPER_SCOPE_MASK GENMASK(31, NET_SHAPER_SCOPE_SHIFT)
>+
>+#define NET_SHAPER_ID_UNSPEC NET_SHAPER_ID_MASK
>+
>+/**
>+ * net_shaper_make_handle - creates an unique shaper identifier
>+ * @scope: the shaper scope
>+ * @id: the shaper id number
>+ *
>+ * Return: an unique identifier for the shaper
>+ *
>+ * Combines the specified arguments to create an unique identifier for
>+ * the shaper. The @id argument semantic depends on the
>+ * specified scope.
>+ * For @NET_SHAPER_SCOPE_QUEUE_GROUP, @id is the queue group id
>+ * For @NET_SHAPER_SCOPE_QUEUE, @id is the queue number.
>+ * For @NET_SHAPER_SCOPE_VF, @id is virtual function number.
>+ */
>+static inline u32 net_shaper_make_handle(enum net_shaper_scope scope,
Why is this in the header? It is used only in shaper.c. Should be
private to it.
>+ int id)
>+{
>+ return FIELD_PREP(NET_SHAPER_SCOPE_MASK, scope) |
>+ FIELD_PREP(NET_SHAPER_ID_MASK, id);
>+}
>+
>+/**
>+ * net_shaper_handle_scope - extract the scope from the given handle
>+ * @handle: the shaper handle
>+ *
>+ * Return: the corresponding scope
>+ */
>+static inline enum net_shaper_scope net_shaper_handle_scope(u32 handle)
>+{
>+ return FIELD_GET(NET_SHAPER_SCOPE_MASK, handle);
>+}
>+
>+/**
>+ * net_shaper_handle_id - extract the id number from the given handle
>+ * @handle: the shaper handle
>+ *
>+ * Return: the corresponding id number
>+ */
>+static inline int net_shaper_handle_id(u32 handle)
I undertand that you use the handle for uapi and xaarray index purposes
as consolidated u32. But, for the driver ops, would it be better to pass
scope and id trough the ops to drivers? Driver then don't need to call
these helpers and they can all stay private to shaper.c
>+{
>+ return FIELD_GET(NET_SHAPER_ID_MASK, handle);
>+}
>+
>+#endif
>+
[...]
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-07-30 20:39 ` [PATCH v3 02/12] netlink: spec: add shaper YAML spec Paolo Abeni
` (3 preceding siblings ...)
2024-08-02 11:26 ` Jiri Pirko
@ 2024-08-02 16:04 ` Jiri Pirko
4 siblings, 0 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-02 16:04 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
[..]
>+ name: handle
I already commented (scope,id) passing to drivers. But in the uapi, why
this has to be a single u32 as well? Why this can't be 2 attributes?
[..]
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 04/12] net-shapers: implement NL set and delete operations
2024-08-01 15:39 ` Jakub Kicinski
@ 2024-08-02 16:15 ` Jiri Pirko
2024-08-02 22:01 ` Jakub Kicinski
2024-08-05 15:23 ` Paolo Abeni
0 siblings, 2 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-02 16:15 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Thu, Aug 01, 2024 at 05:39:24PM CEST, kuba@kernel.org wrote:
>On Thu, 1 Aug 2024 17:25:50 +0200 Paolo Abeni wrote:
>> When deleting a queue-level shaper, the orchestrator is "returning" the
>> ownership of the queue from the container to the host. If the container
What do you meam by "orchestrator" and "container" here? I'm missing
these from the picture.
>> wants to move the queue around e.g. from:
>>
>> q1 ----- \
>> q2 - \SP1/ RR1
What "sp" and "rr" stand for. What are the "scopes" of these?
>> q3 - / \
>> q4 - \ RR2 -> RR(root)
>> q5 - / /
>> q6 - \ RR3
>> q7 - /
>>
>> to:
>>
>> q1 ----- \
>> q2 ----- RR1
>> q3 ---- / \
>> q4 - \ RR2 -> RR(root)
>> q5 - / /
>> q6 - \ RR3
>> q7 - /
>>
>> It can do it with a group() operation:
>>
>> group(inputs:[q2,q3],output:[RR1])
>
>Isn't that a bit odd? The container was not supposed to know / care
>about RR1's existence. We achieve this with group() by implicitly
>inheriting the egress node if all grouped entities shared one.
>
>Delete IMO should act here like a "ungroup" operation, meaning that:
> 1) we're deleting SP1, not q1, q2
Does current code support removing SP1? I mean, if the scope is
detached, I don't think so.
> 2) inputs go "downstream" instead getting ejected into global level
>
>Also, in the first example from the cover letter we "set" a shaper on
>the queue, it feels a little ambiguous whether "delete queue" is
>purely clearing such per-queue shaping, or also has implications
>for the hierarchy.
>
>Coincidentally, others may disagree, but I'd point to tests in patch
>8 for examples of how the thing works, instead the cover letter samples.
Examples in cover letter are generally beneficial. Don't remove them :)
>
>> That will implicitly also delete SP1.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 04/12] net-shapers: implement NL set and delete operations
2024-08-02 16:15 ` Jiri Pirko
@ 2024-08-02 22:01 ` Jakub Kicinski
2024-08-05 15:23 ` Paolo Abeni
1 sibling, 0 replies; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-02 22:01 UTC (permalink / raw)
To: Jiri Pirko
Cc: Paolo Abeni, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Fri, 2 Aug 2024 18:15:32 +0200 Jiri Pirko wrote:
> Thu, Aug 01, 2024 at 05:39:24PM CEST, kuba@kernel.org wrote:
> >On Thu, 1 Aug 2024 17:25:50 +0200 Paolo Abeni wrote:
> >> When deleting a queue-level shaper, the orchestrator is "returning" the
> >> ownership of the queue from the container to the host. If the container
>
> What do you meam by "orchestrator" and "container" here? I'm missing
> these from the picture.
Container (as in docker) and orchestrator.
> >> wants to move the queue around e.g. from:
> >>
> >> q1 ----- \
> >> q2 - \SP1/ RR1
>
> What "sp" and "rr" stand for. What are the "scopes" of these?
"scopes" I agree are confusing, but:
sp = strict priority
rr = round robin
> >> q3 - / \
> >> q4 - \ RR2 -> RR(root)
> >> q5 - / /
> >> q6 - \ RR3
> >> q7 - /
> >>
> >> to:
> >>
> >> q1 ----- \
> >> q2 ----- RR1
> >> q3 ---- / \
> >> q4 - \ RR2 -> RR(root)
> >> q5 - / /
> >> q6 - \ RR3
> >> q7 - /
> >>
> >> It can do it with a group() operation:
> >>
> >> group(inputs:[q2,q3],output:[RR1])
> >
> >Isn't that a bit odd? The container was not supposed to know / care
> >about RR1's existence. We achieve this with group() by implicitly
> >inheriting the egress node if all grouped entities shared one.
> >
> >Delete IMO should act here like a "ungroup" operation, meaning that:
> > 1) we're deleting SP1, not q1, q2
>
> Does current code support removing SP1? I mean, if the scope is
> detached, I don't think so.
that's my reading too, fwiw
> > 2) inputs go "downstream" instead getting ejected into global level
> >
> >Also, in the first example from the cover letter we "set" a shaper on
> >the queue, it feels a little ambiguous whether "delete queue" is
> >purely clearing such per-queue shaping, or also has implications
> >for the hierarchy.
> >
> >Coincidentally, others may disagree, but I'd point to tests in patch
> >8 for examples of how the thing works, instead the cover letter samples.
>
> Examples in cover letter are generally beneficial. Don't remove them :)
They are beneficial, but if I was to order the following three forms of
documentation by priority:
- ReST under Documentation/
- clear selftests with comments
- cover letter
I'm uncertain which will be first, but cover letter is definitely last
:(
With the examples in the cover letter its unclear what the expected
start and end state are. And where the values come from. I feel like
selftest would make it clearer.
But I don't feel strongly. Such newfangled ideas will take a while to
take root :)
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-01 1:55 ` Jakub Kicinski
@ 2024-08-05 14:22 ` Simon Horman
2024-08-05 19:36 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Simon Horman @ 2024-08-05 14:22 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On Wed, Jul 31, 2024 at 06:55:11PM -0700, Jakub Kicinski wrote:
> On Wed, 31 Jul 2024 09:52:38 +0200 Paolo Abeni wrote:
> > On 7/30/24 22:39, Paolo Abeni wrote:
> > > Leverage a basic/dummy netdevsim implementation to do functional
> > > coverage for NL interface.
> > >
> > > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> >
> > FTR, it looks like the CI build went wild around this patch, but the
> > failures look unrelated to the actual changes here. i.e.:
> >
> > https://netdev.bots.linux.dev/static/nipa/875223/13747883/build_clang/stderr
>
> Could you dig deeper?
>
> The scripts are doing incremental builds, and changes to Kconfig
> confuse them. You should be able to run the build script as a normal
> bash script, directly, it only needs a small handful of exported
> env variables.
>
> I have been trying to massage this for a while, my last change is:
> https://github.com/linux-netdev/nipa/commit/5bcb890cbfecd3c1727cec2f026360646a4afc62
>
Thanks Jakub,
I am looking into this.
So far I believe it relate to a Kconfig change activating new code.
But reproducing the problem is proving a little tricky.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-02 11:15 ` Donald Hunter
@ 2024-08-05 14:35 ` Paolo Abeni
2024-08-05 20:37 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-05 14:35 UTC (permalink / raw)
To: Donald Hunter
Cc: netdev, Jakub Kicinski, Jiri Pirko, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Hi all,
My replies this week will be delayed, please allow for some extra latency.
On 8/2/24 13:15, Donald Hunter wrote:
> Paolo Abeni <pabeni@redhat.com> writes:
>
>> On 7/31/24 23:13, Donald Hunter wrote:
>>> Paolo Abeni <pabeni@redhat.com> writes:
>>>
>>>> + name: inputs
>>>> + type: nest
>>>> + multi-attr: true
>>>> + nested-attributes: ns-info
>>>> + doc: |
>>>> + Describes a set of inputs shapers for a @group operation
>>> The @group renders exactly as-is in the generated htmldocs. There may be
>>> a more .rst friendly markup you can use that will render better.
>>
>> Uhm... AFAICS the problem is the target (e.g. 'group') is outside the htmldoc section itself, I
>> can't find any existing markup to serve this purpose well. What about sticking to quotes ''
>> everywhere?
>>
>> FTR, I used @ following the kdoc style.
>
> Yeah, I was just thinking of using .rst markup like ``code`` or
> `italics`, but the meaning of @ is pretty obvious when reading the spec.
> If you stick with @ then we could always teach ynl-to-rst to render it
> as ``code``.
I'm fine with using @ everywhere.
>> [...]
>>>> + -
>>>> + name: group
>>>> + doc: |
>>>> + Group the specified input shapers under the specified
>>>> + output shaper, eventually creating the latter, if needed.
>>>> + Input shapers scope must be either @queue or @detached.
>>> It says above that you cannot create a detached shaper, so how do you
>>> create one to use as an input shaper here? Is this group op more like a
>>> multi-create op?
>>
>> The group operation has the main goal of configuring a single WRR or SP scheduling group
>> atomically. It can creates the needed shapers as needed, see below.
>>
>> The need for such operation sparks from some H/W constraints:
>>
>> https://lore.kernel.org/netdev/9dd818dc-1fef-4633-b388-6ce7272f9cb4@lunn.ch/
>>
>>>> + Output shaper scope must be either @detached or @netdev.
>>>> + When using an output @detached scope shaper, if the
>>>> + @handle @id is not specified, a new shaper of such scope
>>>> + is created and, otherwise the specified output shaper
>>>> + must be already existing.
>>>> + The operation is atomic, on failures the extack is set
>>>> + accordingly and no change is applied to the device
>>>> + shaping configuration, otherwise the output shaper
>>>> + handle is provided as reply.
>>>> + attribute-set: net-shaper
>>>> + flags: [ admin-perm ]
>>> Does there need to be a reciprocal 'ungroup' operation? Without it,
>>> create / group / delete seems like they will have ambiguous semantics.
>>
>> I guess we need a better description. Can you please tell where/how the current one is
>> ambiguous?
>
> My expectation for 'group' would be to group existing things, with a
> reciprocal 'ungroup' operation. I think you intend 'group' to both be
> able to group existing shapers/groups and create a group of shapers.
>
> Am I right in saying that delete lets you delete something from a group
> (with side-effect of deleting group if it becomes empty), or delete a
> whole group?
In the current incarnation, delete() on the whole group is explicitly
forbidden. Jakub suggested we should allow such behavior for the
delegation use-case.
> It feels a lot like each of 'set', 'group' and 'delete' are doing
> multiple things and the interaction between them all becomes challenging
> to describe, or to handle all the corner case > I think part of the
> problem is the mixed terminology of input, output for groups, handle,
> parent for shapers and using detached to differentiate from 'implicitly
> attached to a resource'.
>
> Perhaps the API would be better if you had:
>
> - shaper-new
> - shaper-delete
> - shaper-get/dump
> - shaper-set
> - group-new
> - group-delete
> - group-get/dump
> - group-set
>
> If you went with Jakub's suggestion to give every shaper n x inputs and
> an output, then you could recombine groups and shapers and just have 4
> ops. And you could rename 'detached' to 'shaper' so that an attachment
> is one of port, netdev, queue or shaper.
I'm unsure I read the above correctly, and I'm unsure it's in the same
direction of Jakub's suggestion. AFACS the above is basically the same
interface we proposed in the past iteration and was explicitly nacked
from Jakub,
Additionally, one of the constraint to be addressed here is allowing to
setup/configures all the nodes in a 'group' with a single operation, to
deal with H/W limitations. How would the above address such constraint?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-02 10:49 ` Jiri Pirko
@ 2024-08-05 15:11 ` Paolo Abeni
2024-08-06 7:06 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-05 15:11 UTC (permalink / raw)
To: Jiri Pirko, Jakub Kicinski
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
Hi all,
(same remark of my previous email). My replies this week will be
delayed, please allow for some extra latency.
On 8/2/24 12:49, Jiri Pirko wrote:
> Thu, Aug 01, 2024 at 05:12:01PM CEST, pabeni@redhat.com wrote:
>> On 8/1/24 15:10, Jiri Pirko wrote:
>>> Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
>>>> + type: enum
>>>> + name: scope
>>>> + doc: the different scopes where a shaper can be attached
>>>> + render-max: true
>>>> + entries:
>>>> + - name: unspec
>>>> + doc: The scope is not specified
>>>> + -
>>>> + name: port
>>>> + doc: The root for the whole H/W
>>>
>>> What is this "port"?
>>
>> ~ a wire plug.
>
> What's "wire plug"? What of existing kernel objects this relates to? Is
> it a devlink port?
I'm sorry, my hasty translation of my native language was really
inaccurate. Let me re-phrase from scratch: that is actually the root of
the whole scheduling tree (yes, it's a tree) for a given network device.
One source of confusion is that in a previous iteration we intended to
allow configuring even objects 'above' the network device level, but
such feature has been dropped.
We could probably drop this scope entirely.
>>>> + -
>>>> + name: netdev
>>>> + doc: The main shaper for the given network device.
>>>> + -
>>>> + name: queue
>>>> + doc: The shaper is attached to the given device queue.
>>>> + -
>>>> + name: detached
>>>> + doc: |
>>>> + The shaper is not attached to any user-visible network
>>>> + device component and allows nesting and grouping of
>>>> + queues or others detached shapers.
>>>
>>> What is the purpose of the "detached" thing?
>>
>> I fear I can't escape reusing most of the wording above. 'detached' nodes
>> goal is to create groups of other shapers. i.e. queue groups,
>> allowing multiple levels nesting, i.e. to implement this kind of hierarchy:
>>
>> q1 ----- \
>> q2 - \SP / RR ------
>> q3 - / \
>> q4 - \ SP -> (netdev)
>> q5 - / /
>> /
>> q6 - \ RR
>> q7 - /
>>
>> where q1..q7 are queue-level shapers and all the SP/RR are 'detached' one.
>> The conf. does not necessary make any functional sense, just to describe the
>> things.
>
> Can you "attach" the "detached" ones? They are "detached" from what?
I see such name is very confusing. An alternative one could be 'group',
but IIRC it was explicitly discarded while discussing a previous iteration.
The 'detached' name comes from the fact the such shapers are not a
direct representation of some well-known kernel object (queues, devices),
>>>> + -
>>>> + name: group
>>>> + doc: |
>>>> + Group the specified input shapers under the specified
>>>> + output shaper, eventually creating the latter, if needed.
>>>> + Input shapers scope must be either @queue or @detached.
>>>> + Output shaper scope must be either @detached or @netdev.
>>>> + When using an output @detached scope shaper, if the
>>>> + @handle @id is not specified, a new shaper of such scope
>>>> + is created and, otherwise the specified output shaper
>>>> + must be already existing.
>>>
>>> I'm lost. Could this designt be described in details in the doc I asked
>>> in the cover letter? :/ Please.
>>
>> I'm unsure if the context information here and in the previous replies helped
>> somehow.
>>
>> The group operation creates and configure a scheduling group, i.e. this
>>
>> q1 ----- \
>> q2 - \SP / RR ------
>> q3 - / \
>> q4 - \ SP -> (netdev)
>> q5 - / /
>> /
>> q6 - \ RR
>> q7 - /
>>
>> can be create with:
>>
>> group(inputs:[q6, q7], output:[detached,parent:netdev])
>> group(inputs:[q4, q5], output:[detached,parent:netdev])
>> group(inputs:[q1], output:[detached,parent:netdev])
>> group(inputs:[q2,q3], output:[detached,parent:<the detached shaper create
>> above>])
>
> So by "inputs" and "output" you are basically building a tree. In
> devlink rate, we have leaf and node, which is in sync with standard tree
> terminology.
>
> If what you are building is tree, why don't you use the same
> terminology? If you are building tree, you just need to have the link to
> upper noded (output in your terminology). Why you have "inputs"? Isn't
> that redundant?
The idea behind the inputs/outputs naming is to represent the data flow
towards the wire.
I'm fine with the parent/children naming, but IIRC Jakub was not happy
with it. Is there any intermediate ground that could satisfy both of you?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 04/12] net-shapers: implement NL set and delete operations
2024-08-02 16:15 ` Jiri Pirko
2024-08-02 22:01 ` Jakub Kicinski
@ 2024-08-05 15:23 ` Paolo Abeni
1 sibling, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-08-05 15:23 UTC (permalink / raw)
To: Jiri Pirko, Jakub Kicinski
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim,
Donald Hunter
On 8/2/24 18:15, Jiri Pirko wrote:
> Thu, Aug 01, 2024 at 05:39:24PM CEST, kuba@kernel.org wrote:
>> On Thu, 1 Aug 2024 17:25:50 +0200 Paolo Abeni wrote:
>>> When deleting a queue-level shaper, the orchestrator is "returning" the
>>> ownership of the queue from the container to the host. If the container
>
> What do you meam by "orchestrator" and "container" here? I'm missing
> these from the picture.
>
>
>>> wants to move the queue around e.g. from:
>>>
>>> q1 ----- \
>>> q2 - \SP1/ RR1
>
> What "sp" and "rr" stand for. What are the "scopes" of these?
The scope is 'detached'
>>> q3 - / \
>>> q4 - \ RR2 -> RR(root)
>>> q5 - / /
>>> q6 - \ RR3
>>> q7 - /
>>>
>>> to:
>>>
>>> q1 ----- \
>>> q2 ----- RR1
>>> q3 ---- / \
>>> q4 - \ RR2 -> RR(root)
>>> q5 - / /
>>> q6 - \ RR3
>>> q7 - /
>>>
>>> It can do it with a group() operation:
>>>
>>> group(inputs:[q2,q3],output:[RR1])
>>
>> Isn't that a bit odd? The container was not supposed to know / care
>> about RR1's existence. We achieve this with group() by implicitly
>> inheriting the egress node if all grouped entities shared one.
>>
>> Delete IMO should act here like a "ungroup" operation, meaning that:
>> 1) we're deleting SP1, not q1, q2
>
> Does current code support removing SP1? I mean, if the scope is
> detached, I don't think so.
The current code explicitly prevents the above. We can change such
behavior, if there is agreement.
My understanding is that Donald is against such option.
>> 2) inputs go "downstream" instead getting ejected into global level
>>
>> Also, in the first example from the cover letter we "set" a shaper on
>> the queue, it feels a little ambiguous whether "delete queue" is
>> purely clearing such per-queue shaping, or also has implications
>> for the hierarchy.
>>
>> Coincidentally, others may disagree, but I'd point to tests in patch
>> 8 for examples of how the thing works, instead the cover letter samples.
>
> Examples in cover letter are generally beneficial. Don't remove them :)
No problem to keep both examples and self-tests.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-05 14:22 ` Simon Horman
@ 2024-08-05 19:36 ` Jakub Kicinski
2024-08-06 15:21 ` Simon Horman
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-05 19:36 UTC (permalink / raw)
To: Simon Horman
Cc: Paolo Abeni, netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On Mon, 5 Aug 2024 15:22:53 +0100 Simon Horman wrote:
> On Wed, Jul 31, 2024 at 06:55:11PM -0700, Jakub Kicinski wrote:
> > On Wed, 31 Jul 2024 09:52:38 +0200 Paolo Abeni wrote:
> > > FTR, it looks like the CI build went wild around this patch, but the
> > > failures look unrelated to the actual changes here. i.e.:
> > >
> > > https://netdev.bots.linux.dev/static/nipa/875223/13747883/build_clang/stderr
> >
> > Could you dig deeper?
> >
> > The scripts are doing incremental builds, and changes to Kconfig
> > confuse them. You should be able to run the build script as a normal
> > bash script, directly, it only needs a small handful of exported
> > env variables.
> >
> > I have been trying to massage this for a while, my last change is:
> > https://github.com/linux-netdev/nipa/commit/5bcb890cbfecd3c1727cec2f026360646a4afc62
> >
>
> Thanks Jakub,
>
> I am looking into this.
> So far I believe it relate to a Kconfig change activating new code.
> But reproducing the problem is proving a little tricky.
Have you tried twiddling / exporting FIRST_IN_SERIES ?
See here for the 4 possible exports the test will look at:
https://github.com/linux-netdev/nipa/blob/6112db7d472660450c69457c98ab37b431063301/core/test.py#L124
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-05 14:35 ` Paolo Abeni
@ 2024-08-05 20:37 ` Jakub Kicinski
0 siblings, 0 replies; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-05 20:37 UTC (permalink / raw)
To: Paolo Abeni
Cc: Donald Hunter, netdev, Jiri Pirko, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
On Mon, 5 Aug 2024 16:35:29 +0200 Paolo Abeni wrote:
> > Perhaps the API would be better if you had:
> >
> > - shaper-new
> > - shaper-delete
> > - shaper-get/dump
> > - shaper-set
> > - group-new
> > - group-delete
> > - group-get/dump
> > - group-set
> >
> > If you went with Jakub's suggestion to give every shaper n x inputs and
> > an output, then you could recombine groups and shapers and just have 4
> > ops. And you could rename 'detached' to 'shaper' so that an attachment
> > is one of port, netdev, queue or shaper.
>
> I'm unsure I read the above correctly, and I'm unsure it's in the same
> direction of Jakub's suggestion. AFACS the above is basically the same
> interface we proposed in the past iteration and was explicitly nacked
> from Jakub,
To be clear I was against the low level twiddling APIs, where one has to
separately create a mux/group/scheduler and "move" all its children
under it one by one. (due to the problems it creates with atomic
transitions between configurations).
Having shapers separate from the scheduling hierarchy doesn't seem bad,
tho I haven't gone thru all the considerations in my head.
> Additionally, one of the constraint to be addressed here is allowing to
> setup/configures all the nodes in a 'group' with a single operation, to
> deal with H/W limitations. How would the above address such constraint?
FWIW I think the naming is the major source of confusion :(
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-05 15:11 ` Paolo Abeni
@ 2024-08-06 7:06 ` Jiri Pirko
2024-08-12 14:58 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-06 7:06 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jakub Kicinski, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Mon, Aug 05, 2024 at 05:11:09PM CEST, pabeni@redhat.com wrote:
>Hi all,
>
>(same remark of my previous email). My replies this week will be delayed,
>please allow for some extra latency.
>
>On 8/2/24 12:49, Jiri Pirko wrote:
>> Thu, Aug 01, 2024 at 05:12:01PM CEST, pabeni@redhat.com wrote:
>> > On 8/1/24 15:10, Jiri Pirko wrote:
>> > > Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
>> > > > + type: enum
>> > > > + name: scope
>> > > > + doc: the different scopes where a shaper can be attached
>> > > > + render-max: true
>> > > > + entries:
>> > > > + - name: unspec
>> > > > + doc: The scope is not specified
>> > > > + -
>> > > > + name: port
>> > > > + doc: The root for the whole H/W
>> > >
>> > > What is this "port"?
>> >
>> > ~ a wire plug.
>>
>> What's "wire plug"? What of existing kernel objects this relates to? Is
>> it a devlink port?
>
>
>I'm sorry, my hasty translation of my native language was really inaccurate.
>Let me re-phrase from scratch: that is actually the root of the whole
>scheduling tree (yes, it's a tree) for a given network device.
>
>One source of confusion is that in a previous iteration we intended to allow
>configuring even objects 'above' the network device level, but such feature
>has been dropped.
>
>We could probably drop this scope entirely.
Drop for now, correct? I agree that your patchset now only works on top
of netdev. But all infra should be ready to work on top of something
else, devlink seems like good candidate. I mean, for devlink port
function rate, we will definitelly need something like that.
>
>> > > > + -
>> > > > + name: netdev
>> > > > + doc: The main shaper for the given network device.
>> > > > + -
>> > > > + name: queue
>> > > > + doc: The shaper is attached to the given device queue.
>> > > > + -
>> > > > + name: detached
>> > > > + doc: |
>> > > > + The shaper is not attached to any user-visible network
>> > > > + device component and allows nesting and grouping of
>> > > > + queues or others detached shapers.
>> > >
>> > > What is the purpose of the "detached" thing?
>> >
>> > I fear I can't escape reusing most of the wording above. 'detached' nodes
>> > goal is to create groups of other shapers. i.e. queue groups,
>> > allowing multiple levels nesting, i.e. to implement this kind of hierarchy:
>> >
>> > q1 ----- \
>> > q2 - \SP / RR ------
>> > q3 - / \
>> > q4 - \ SP -> (netdev)
>> > q5 - / /
>> > /
>> > q6 - \ RR
>> > q7 - /
>> >
>> > where q1..q7 are queue-level shapers and all the SP/RR are 'detached' one.
>> > The conf. does not necessary make any functional sense, just to describe the
>> > things.
>>
>> Can you "attach" the "detached" ones? They are "detached" from what?
>
>I see such name is very confusing. An alternative one could be 'group', but
>IIRC it was explicitly discarded while discussing a previous iteration.
>
>The 'detached' name comes from the fact the such shapers are not a direct
>representation of some well-known kernel object (queues, devices),
Understand now. Maybe "node" would make more sense? Leaves are queues
and root is the device? Aligns with the tree terminology...
>
>> > > > + -
>> > > > + name: group
>> > > > + doc: |
>> > > > + Group the specified input shapers under the specified
>> > > > + output shaper, eventually creating the latter, if needed.
>> > > > + Input shapers scope must be either @queue or @detached.
>> > > > + Output shaper scope must be either @detached or @netdev.
>> > > > + When using an output @detached scope shaper, if the
>> > > > + @handle @id is not specified, a new shaper of such scope
>> > > > + is created and, otherwise the specified output shaper
>> > > > + must be already existing.
>> > >
>> > > I'm lost. Could this designt be described in details in the doc I asked
>> > > in the cover letter? :/ Please.
>> >
>> > I'm unsure if the context information here and in the previous replies helped
>> > somehow.
>> >
>> > The group operation creates and configure a scheduling group, i.e. this
>> >
>> > q1 ----- \
>> > q2 - \SP / RR ------
>> > q3 - / \
>> > q4 - \ SP -> (netdev)
>> > q5 - / /
>> > /
>> > q6 - \ RR
>> > q7 - /
>> >
>> > can be create with:
>> >
>> > group(inputs:[q6, q7], output:[detached,parent:netdev])
>> > group(inputs:[q4, q5], output:[detached,parent:netdev])
>> > group(inputs:[q1], output:[detached,parent:netdev])
>> > group(inputs:[q2,q3], output:[detached,parent:<the detached shaper create
>> > above>])
>>
>> So by "inputs" and "output" you are basically building a tree. In
>> devlink rate, we have leaf and node, which is in sync with standard tree
>> terminology.
>>
>> If what you are building is tree, why don't you use the same
>> terminology? If you are building tree, you just need to have the link to
>> upper noded (output in your terminology). Why you have "inputs"? Isn't
>> that redundant?
>
>The idea behind the inputs/outputs naming is to represent the data flow
>towards the wire.
>I'm fine with the parent/children naming, but IIRC Jakub was not happy with
>it. Is there any intermediate ground that could satisfy both of you?
It's a tree, so perhaps just stick with tree terminology, everyone is
used to that. Makes sense? One way or another, this needs to be
properly described in docs, all terminology. That would make things more
clear, I believe.
>
>Thanks,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-05 19:36 ` Jakub Kicinski
@ 2024-08-06 15:21 ` Simon Horman
2024-08-08 12:20 ` Simon Horman
0 siblings, 1 reply; 91+ messages in thread
From: Simon Horman @ 2024-08-06 15:21 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On Mon, Aug 05, 2024 at 12:36:55PM -0700, Jakub Kicinski wrote:
> On Mon, 5 Aug 2024 15:22:53 +0100 Simon Horman wrote:
> > On Wed, Jul 31, 2024 at 06:55:11PM -0700, Jakub Kicinski wrote:
> > > On Wed, 31 Jul 2024 09:52:38 +0200 Paolo Abeni wrote:
> > > > FTR, it looks like the CI build went wild around this patch, but the
> > > > failures look unrelated to the actual changes here. i.e.:
> > > >
> > > > https://netdev.bots.linux.dev/static/nipa/875223/13747883/build_clang/stderr
> > >
> > > Could you dig deeper?
> > >
> > > The scripts are doing incremental builds, and changes to Kconfig
> > > confuse them. You should be able to run the build script as a normal
> > > bash script, directly, it only needs a small handful of exported
> > > env variables.
> > >
> > > I have been trying to massage this for a while, my last change is:
> > > https://github.com/linux-netdev/nipa/commit/5bcb890cbfecd3c1727cec2f026360646a4afc62
> > >
> >
> > Thanks Jakub,
> >
> > I am looking into this.
> > So far I believe it relate to a Kconfig change activating new code.
> > But reproducing the problem is proving a little tricky.
>
> Have you tried twiddling / exporting FIRST_IN_SERIES ?
>
> See here for the 4 possible exports the test will look at:
>
> https://github.com/linux-netdev/nipa/blob/6112db7d472660450c69457c98ab37b431063301/core/test.py#L124
Thanks, I will look into that.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-06 15:21 ` Simon Horman
@ 2024-08-08 12:20 ` Simon Horman
2024-08-08 14:17 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Simon Horman @ 2024-08-08 12:20 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On Tue, Aug 06, 2024 at 04:22:03PM +0100, Simon Horman wrote:
> On Mon, Aug 05, 2024 at 12:36:55PM -0700, Jakub Kicinski wrote:
> > On Mon, 5 Aug 2024 15:22:53 +0100 Simon Horman wrote:
> > > On Wed, Jul 31, 2024 at 06:55:11PM -0700, Jakub Kicinski wrote:
> > > > On Wed, 31 Jul 2024 09:52:38 +0200 Paolo Abeni wrote:
> > > > > FTR, it looks like the CI build went wild around this patch, but the
> > > > > failures look unrelated to the actual changes here. i.e.:
> > > > >
> > > > > https://netdev.bots.linux.dev/static/nipa/875223/13747883/build_clang/stderr
> > > >
> > > > Could you dig deeper?
> > > >
> > > > The scripts are doing incremental builds, and changes to Kconfig
> > > > confuse them. You should be able to run the build script as a normal
> > > > bash script, directly, it only needs a small handful of exported
> > > > env variables.
> > > >
> > > > I have been trying to massage this for a while, my last change is:
> > > > https://github.com/linux-netdev/nipa/commit/5bcb890cbfecd3c1727cec2f026360646a4afc62
> > > >
> > >
> > > Thanks Jakub,
> > >
> > > I am looking into this.
> > > So far I believe it relate to a Kconfig change activating new code.
> > > But reproducing the problem is proving a little tricky.
> >
> > Have you tried twiddling / exporting FIRST_IN_SERIES ?
> >
> > See here for the 4 possible exports the test will look at:
> >
> > https://github.com/linux-netdev/nipa/blob/6112db7d472660450c69457c98ab37b431063301/core/test.py#L124
>
> Thanks, I will look into that.
Hi Jakub,
Thanks again for the information.
I have now taken another look at this problem.
Firstly, my analysis is that the cause of the problem is a combination of
the way the patchset is constricted, and the way that the build tests (I
have focussed on build_allmodconfig_warn.sh [1]).
[1] https://github.com/linux-netdev/nipa/blob/main/tests/patch/build_allmodconfig_warn/build_allmodconfig.sh
What I believe happens is this: The patches 01/12 - 07/12 modify some
header files, adds a new Kconfig entry, and does a bunch of other normal
stuff. Each of those patches is tested in turn, and everything seems fine.
Then we get to patch 08/12. The key thing about this patch is that it
enables the CONFIG_NET_SHAPER Kconfig option, in the context of an
allmodconfig build. That in turn modifies the headers
include/linux/netdevice.h and net/core/dev.h (and net/Makefile). Not in the
in terms of their on-disk contents changing, but rather in the case of the
header files, in terms of preprocessor output. And this is, I believe,
where everything goes wrong.
NIPA arrives at running build_allmodconfig_warn.sh for patch 08/12 with the tree
built for the previous patch, 07/12. It then:
* touches $output_dir/include/generated/utsrelease.h
* checks out HEAD~ (patch 07/12)
* prepares the kernel config
* builds kernel and records incumbent errors (49)
The thing to note here is that the tree has been little perturbed since build
tests were run for patch 07/12, and thus few files are rebuilt.
Moving on, simplifying things, the following then runs:
* touches $output_dir/include/generated/utsrelease.h
* checks out $HEAD (patch 08/12)
* prepares the kernel config
* builds kernel and records current errors (4219)
The key to understanding why the large delta between 49 and 4219 is
that vastly more files have been rebuilt. Because the preprocessor output
of netdevice.h and dev.h have changes since the last build, and those
headers are included, directly or indirectly, by a lot of files (and
compilation results in warnings for many of those files).
I was able to reproduce the result by running build_allmodconfig_warn.sh
over patch 07/12 and then 07/12 with FIRST_IN_SERIES=0.
I was able to get the desired result no new compiler warnings
by doing the same again, but with FIRST_IN_SERIES=1 for the
invocation of build_allmodconfig_warn.sh for 08/12.
I believe this is entirely due to a baseline rebuild being run due to the
FIRST_IN_SERIES=1 parameter. And, FWIIW, I believe the invocation of
build_allmodconfig_warn.sh for 07/12 ensures reproducibility.
My suggestion is that while we may consider reorganising the patch-set,
that is really only a work around. And it would be best to make the CI more
robust in the presence of such constructions.
It may be a bit heavy handed, but my tested solution is to invoke a
baseline rebuild if a Kconfig change is made. At the very last it does
address the problem at hand. (In precisely the same way as manually setting
FIRST_IN_SERIES=1.)
The patch implementing this for build_allmodconfig.sh which I tested is
below. If we want to go ahead with this approach then I expect it is best
to add it to other build tests too. But this seems to be a good point
to report my findings, so here we are.
--- build_allmodconfig.sh.orig 2024-08-08 07:30:56.599372164 +0000
+++ build_allmodconfig.sh 2024-08-08 09:58:22.692206313 +0000
@@ -34,8 +34,10 @@
echo "Tree base:"
git log -1 --pretty='%h ("%s")' HEAD~
-if [ x$FIRST_IN_SERIES == x0 ]; then
- echo "Skip baseline build, not the first patch"
+if [ x$FIRST_IN_SERIES == x0 ] && \
+ ! git diff --name-only HEAD~ | grep -q -E "Kconfig$"
+then
+ echo "Skip baseline build, not the first patch and no Kconfig updates"
else
echo "Baseline building the tree"
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-08 12:20 ` Simon Horman
@ 2024-08-08 14:17 ` Jakub Kicinski
2024-08-08 14:34 ` Simon Horman
2024-08-12 16:03 ` Paolo Abeni
0 siblings, 2 replies; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-08 14:17 UTC (permalink / raw)
To: Simon Horman
Cc: Paolo Abeni, netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On Thu, 8 Aug 2024 13:20:42 +0100 Simon Horman wrote:
> Thanks again for the information.
>
> I have now taken another look at this problem.
>
> Firstly, my analysis is that the cause of the problem is a combination of
> the way the patchset is constricted, and the way that the build tests (I
> have focussed on build_allmodconfig_warn.sh [1]).
>
> [1] https://github.com/linux-netdev/nipa/blob/main/tests/patch/build_allmodconfig_warn/build_allmodconfig.sh
>
> What I believe happens is this: The patches 01/12 - 07/12 modify some
> header files, adds a new Kconfig entry, and does a bunch of other normal
> stuff. Each of those patches is tested in turn, and everything seems fine.
>
> Then we get to patch 08/12. The key thing about this patch is that it
> enables the CONFIG_NET_SHAPER Kconfig option, in the context of an
> allmodconfig build. That in turn modifies the headers
> include/linux/netdevice.h and net/core/dev.h (and net/Makefile). Not in the
> in terms of their on-disk contents changing, but rather in the case of the
> header files, in terms of preprocessor output. And this is, I believe,
> where everything goes wrong.
That's strange, make does not understand preprocessor, does it?
Either file has been modified or it has not.
I guess it doesn't matter, given your solution
> NIPA arrives at running build_allmodconfig_warn.sh for patch 08/12 with the tree
> built for the previous patch, 07/12. It then:
>
> * touches $output_dir/include/generated/utsrelease.h
> * checks out HEAD~ (patch 07/12)
> * prepares the kernel config
> * builds kernel and records incumbent errors (49)
>
> The thing to note here is that the tree has been little perturbed since build
> tests were run for patch 07/12, and thus few files are rebuilt.
>
> Moving on, simplifying things, the following then runs:
>
> * touches $output_dir/include/generated/utsrelease.h
> * checks out $HEAD (patch 08/12)
> * prepares the kernel config
> * builds kernel and records current errors (4219)
>
> The key to understanding why the large delta between 49 and 4219 is
> that vastly more files have been rebuilt. Because the preprocessor output
> of netdevice.h and dev.h have changes since the last build, and those
> headers are included, directly or indirectly, by a lot of files (and
> compilation results in warnings for many of those files).
>
>
> I was able to reproduce the result by running build_allmodconfig_warn.sh
> over patch 07/12 and then 07/12 with FIRST_IN_SERIES=0.
>
> I was able to get the desired result no new compiler warnings
> by doing the same again, but with FIRST_IN_SERIES=1 for the
> invocation of build_allmodconfig_warn.sh for 08/12.
>
> I believe this is entirely due to a baseline rebuild being run due to the
> FIRST_IN_SERIES=1 parameter. And, FWIIW, I believe the invocation of
> build_allmodconfig_warn.sh for 07/12 ensures reproducibility.
>
> My suggestion is that while we may consider reorganising the patch-set,
> that is really only a work around. And it would be best to make the CI more
> robust in the presence of such constructions.
>
> It may be a bit heavy handed, but my tested solution is to invoke a
> baseline rebuild if a Kconfig change is made. At the very last it does
> address the problem at hand. (In precisely the same way as manually setting
> FIRST_IN_SERIES=1.)
>
> The patch implementing this for build_allmodconfig.sh which I tested is
> below. If we want to go ahead with this approach then I expect it is best
> to add it to other build tests too. But this seems to be a good point
> to report my findings, so here we are.
>
> --- build_allmodconfig.sh.orig 2024-08-08 07:30:56.599372164 +0000
> +++ build_allmodconfig.sh 2024-08-08 09:58:22.692206313 +0000
> @@ -34,8 +34,10 @@
> echo "Tree base:"
> git log -1 --pretty='%h ("%s")' HEAD~
>
> -if [ x$FIRST_IN_SERIES == x0 ]; then
> - echo "Skip baseline build, not the first patch"
> +if [ x$FIRST_IN_SERIES == x0 ] && \
> + ! git diff --name-only HEAD~ | grep -q -E "Kconfig$"
> +then
> + echo "Skip baseline build, not the first patch and no Kconfig updates"
> else
> echo "Baseline building the tree"
Excellent idea, let's try it! Could you send a PR to NIPA?
Note that the code is copied 3 times for each flavor of building :(
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-08 14:17 ` Jakub Kicinski
@ 2024-08-08 14:34 ` Simon Horman
2024-08-11 12:40 ` Simon Horman
2024-08-12 16:03 ` Paolo Abeni
1 sibling, 1 reply; 91+ messages in thread
From: Simon Horman @ 2024-08-08 14:34 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On Thu, Aug 08, 2024 at 07:17:54AM -0700, Jakub Kicinski wrote:
> On Thu, 8 Aug 2024 13:20:42 +0100 Simon Horman wrote:
> > Thanks again for the information.
> >
> > I have now taken another look at this problem.
> >
> > Firstly, my analysis is that the cause of the problem is a combination of
> > the way the patchset is constricted, and the way that the build tests (I
> > have focussed on build_allmodconfig_warn.sh [1]).
> >
> > [1] https://github.com/linux-netdev/nipa/blob/main/tests/patch/build_allmodconfig_warn/build_allmodconfig.sh
> >
> > What I believe happens is this: The patches 01/12 - 07/12 modify some
> > header files, adds a new Kconfig entry, and does a bunch of other normal
> > stuff. Each of those patches is tested in turn, and everything seems fine.
> >
> > Then we get to patch 08/12. The key thing about this patch is that it
> > enables the CONFIG_NET_SHAPER Kconfig option, in the context of an
> > allmodconfig build. That in turn modifies the headers
> > include/linux/netdevice.h and net/core/dev.h (and net/Makefile). Not in the
> > in terms of their on-disk contents changing, but rather in the case of the
> > header files, in terms of preprocessor output. And this is, I believe,
> > where everything goes wrong.
>
> That's strange, make does not understand preprocessor, does it?
> Either file has been modified or it has not.
True, that is a good point.
I would say there is something deeper going on than
I have been able to discover at this time: probably something very obvious.
> I guess it doesn't matter, given your solution
Right, in any case I think the script needs to be enhanced.
My solution may prove heavy handed, but it can be improved over time.
> > NIPA arrives at running build_allmodconfig_warn.sh for patch 08/12 with the tree
> > built for the previous patch, 07/12. It then:
> >
> > * touches $output_dir/include/generated/utsrelease.h
> > * checks out HEAD~ (patch 07/12)
> > * prepares the kernel config
> > * builds kernel and records incumbent errors (49)
> >
> > The thing to note here is that the tree has been little perturbed since build
> > tests were run for patch 07/12, and thus few files are rebuilt.
> >
> > Moving on, simplifying things, the following then runs:
> >
> > * touches $output_dir/include/generated/utsrelease.h
> > * checks out $HEAD (patch 08/12)
> > * prepares the kernel config
> > * builds kernel and records current errors (4219)
> >
> > The key to understanding why the large delta between 49 and 4219 is
> > that vastly more files have been rebuilt. Because the preprocessor output
> > of netdevice.h and dev.h have changes since the last build, and those
> > headers are included, directly or indirectly, by a lot of files (and
> > compilation results in warnings for many of those files).
> >
> >
> > I was able to reproduce the result by running build_allmodconfig_warn.sh
> > over patch 07/12 and then 07/12 with FIRST_IN_SERIES=0.
Correction: this should have read "07/12 and then 08/12"
> > I was able to get the desired result no new compiler warnings
> > by doing the same again, but with FIRST_IN_SERIES=1 for the
> > invocation of build_allmodconfig_warn.sh for 08/12.
> >
> > I believe this is entirely due to a baseline rebuild being run due to the
> > FIRST_IN_SERIES=1 parameter. And, FWIIW, I believe the invocation of
> > build_allmodconfig_warn.sh for 07/12 ensures reproducibility.
> >
> > My suggestion is that while we may consider reorganising the patch-set,
> > that is really only a work around. And it would be best to make the CI more
> > robust in the presence of such constructions.
> >
> > It may be a bit heavy handed, but my tested solution is to invoke a
> > baseline rebuild if a Kconfig change is made. At the very last it does
> > address the problem at hand. (In precisely the same way as manually setting
> > FIRST_IN_SERIES=1.)
> >
> > The patch implementing this for build_allmodconfig.sh which I tested is
> > below. If we want to go ahead with this approach then I expect it is best
> > to add it to other build tests too. But this seems to be a good point
> > to report my findings, so here we are.
> >
> > --- build_allmodconfig.sh.orig 2024-08-08 07:30:56.599372164 +0000
> > +++ build_allmodconfig.sh 2024-08-08 09:58:22.692206313 +0000
> > @@ -34,8 +34,10 @@
> > echo "Tree base:"
> > git log -1 --pretty='%h ("%s")' HEAD~
> >
> > -if [ x$FIRST_IN_SERIES == x0 ]; then
> > - echo "Skip baseline build, not the first patch"
> > +if [ x$FIRST_IN_SERIES == x0 ] && \
> > + ! git diff --name-only HEAD~ | grep -q -E "Kconfig$"
> > +then
> > + echo "Skip baseline build, not the first patch and no Kconfig updates"
> > else
> > echo "Baseline building the tree"
>
> Excellent idea, let's try it! Could you send a PR to NIPA?
Yes, can do.
> Note that the code is copied 3 times for each flavor of building :(
I assumed so :)
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-08 14:34 ` Simon Horman
@ 2024-08-11 12:40 ` Simon Horman
2024-08-12 15:31 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Simon Horman @ 2024-08-11 12:40 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On Thu, Aug 08, 2024 at 03:34:43PM +0100, Simon Horman wrote:
> On Thu, Aug 08, 2024 at 07:17:54AM -0700, Jakub Kicinski wrote:
> > On Thu, 8 Aug 2024 13:20:42 +0100 Simon Horman wrote:
...
> > > It may be a bit heavy handed, but my tested solution is to invoke a
> > > baseline rebuild if a Kconfig change is made. At the very last it does
> > > address the problem at hand. (In precisely the same way as manually setting
> > > FIRST_IN_SERIES=1.)
> > >
> > > The patch implementing this for build_allmodconfig.sh which I tested is
> > > below. If we want to go ahead with this approach then I expect it is best
> > > to add it to other build tests too. But this seems to be a good point
> > > to report my findings, so here we are.
> > >
> > > --- build_allmodconfig.sh.orig 2024-08-08 07:30:56.599372164 +0000
> > > +++ build_allmodconfig.sh 2024-08-08 09:58:22.692206313 +0000
> > > @@ -34,8 +34,10 @@
> > > echo "Tree base:"
> > > git log -1 --pretty='%h ("%s")' HEAD~
> > >
> > > -if [ x$FIRST_IN_SERIES == x0 ]; then
> > > - echo "Skip baseline build, not the first patch"
> > > +if [ x$FIRST_IN_SERIES == x0 ] && \
> > > + ! git diff --name-only HEAD~ | grep -q -E "Kconfig$"
> > > +then
> > > + echo "Skip baseline build, not the first patch and no Kconfig updates"
> > > else
> > > echo "Baseline building the tree"
> >
> > Excellent idea, let's try it! Could you send a PR to NIPA?
>
> Yes, can do.
For reference, the PR is here:
https://github.com/linux-netdev/nipa/pull/35
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-06 7:06 ` Jiri Pirko
@ 2024-08-12 14:58 ` Paolo Abeni
2024-08-12 15:25 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-12 14:58 UTC (permalink / raw)
To: Jiri Pirko, Jakub Kicinski, Donald Hunter
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On 8/6/24 09:06, Jiri Pirko wrote:
> Mon, Aug 05, 2024 at 05:11:09PM CEST, pabeni@redhat.com wrote:
>> Hi all,
>>
>> (same remark of my previous email). My replies this week will be delayed,
>> please allow for some extra latency.
>>
>> On 8/2/24 12:49, Jiri Pirko wrote:
>>> Thu, Aug 01, 2024 at 05:12:01PM CEST, pabeni@redhat.com wrote:
>>>> On 8/1/24 15:10, Jiri Pirko wrote:
>>>>> Tue, Jul 30, 2024 at 10:39:45PM CEST, pabeni@redhat.com wrote:
>>>>>> + type: enum
>>>>>> + name: scope
>>>>>> + doc: the different scopes where a shaper can be attached
>>>>>> + render-max: true
>>>>>> + entries:
>>>>>> + - name: unspec
>>>>>> + doc: The scope is not specified
>>>>>> + -
>>>>>> + name: port
>>>>>> + doc: The root for the whole H/W
>>>>>
>>>>> What is this "port"?
>>>>
>>>> ~ a wire plug.
>>>
>>> What's "wire plug"? What of existing kernel objects this relates to? Is
>>> it a devlink port?
>>
>>
>> I'm sorry, my hasty translation of my native language was really inaccurate.
>> Let me re-phrase from scratch: that is actually the root of the whole
>> scheduling tree (yes, it's a tree) for a given network device.
>>
>> One source of confusion is that in a previous iteration we intended to allow
>> configuring even objects 'above' the network device level, but such feature
>> has been dropped.
>>
>> We could probably drop this scope entirely.
>
> Drop for now, correct? I agree that your patchset now only works on top
> of netdev. But all infra should be ready to work on top of something
> else, devlink seems like good candidate. I mean, for devlink port
> function rate, we will definitelly need something like that.
>
>
>>
>>>>>> + -
>>>>>> + name: netdev
>>>>>> + doc: The main shaper for the given network device.
>>>>>> + -
>>>>>> + name: queue
>>>>>> + doc: The shaper is attached to the given device queue.
>>>>>> + -
>>>>>> + name: detached
>>>>>> + doc: |
>>>>>> + The shaper is not attached to any user-visible network
>>>>>> + device component and allows nesting and grouping of
>>>>>> + queues or others detached shapers.
>>>>>
>>>>> What is the purpose of the "detached" thing?
>>>>
>>>> I fear I can't escape reusing most of the wording above. 'detached' nodes
>>>> goal is to create groups of other shapers. i.e. queue groups,
>>>> allowing multiple levels nesting, i.e. to implement this kind of hierarchy:
>>>>
>>>> q1 ----- \
>>>> q2 - \SP / RR ------
>>>> q3 - / \
>>>> q4 - \ SP -> (netdev)
>>>> q5 - / /
>>>> /
>>>> q6 - \ RR
>>>> q7 - /
>>>>
>>>> where q1..q7 are queue-level shapers and all the SP/RR are 'detached' one.
>>>> The conf. does not necessary make any functional sense, just to describe the
>>>> things.
>>>
>>> Can you "attach" the "detached" ones? They are "detached" from what?
>>
>> I see such name is very confusing. An alternative one could be 'group', but
>> IIRC it was explicitly discarded while discussing a previous iteration.
>>
>> The 'detached' name comes from the fact the such shapers are not a direct
>> representation of some well-known kernel object (queues, devices),
>
> Understand now. Maybe "node" would make more sense? Leaves are queues
> and root is the device? Aligns with the tree terminology...
>
>>
>>>>>> + -
>>>>>> + name: group
>>>>>> + doc: |
>>>>>> + Group the specified input shapers under the specified
>>>>>> + output shaper, eventually creating the latter, if needed.
>>>>>> + Input shapers scope must be either @queue or @detached.
>>>>>> + Output shaper scope must be either @detached or @netdev.
>>>>>> + When using an output @detached scope shaper, if the
>>>>>> + @handle @id is not specified, a new shaper of such scope
>>>>>> + is created and, otherwise the specified output shaper
>>>>>> + must be already existing.
>>>>>
>>>>> I'm lost. Could this designt be described in details in the doc I asked
>>>>> in the cover letter? :/ Please.
>>>>
>>>> I'm unsure if the context information here and in the previous replies helped
>>>> somehow.
>>>>
>>>> The group operation creates and configure a scheduling group, i.e. this
>>>>
>>>> q1 ----- \
>>>> q2 - \SP / RR ------
>>>> q3 - / \
>>>> q4 - \ SP -> (netdev)
>>>> q5 - / /
>>>> /
>>>> q6 - \ RR
>>>> q7 - /
>>>>
>>>> can be create with:
>>>>
>>>> group(inputs:[q6, q7], output:[detached,parent:netdev])
>>>> group(inputs:[q4, q5], output:[detached,parent:netdev])
>>>> group(inputs:[q1], output:[detached,parent:netdev])
>>>> group(inputs:[q2,q3], output:[detached,parent:<the detached shaper create
>>>> above>])
>>>
>>> So by "inputs" and "output" you are basically building a tree. In
>>> devlink rate, we have leaf and node, which is in sync with standard tree
>>> terminology.
>>>
>>> If what you are building is tree, why don't you use the same
>>> terminology? If you are building tree, you just need to have the link to
>>> upper noded (output in your terminology). Why you have "inputs"? Isn't
>>> that redundant?
>>
>> The idea behind the inputs/outputs naming is to represent the data flow
>> towards the wire.
>> I'm fine with the parent/children naming, but IIRC Jakub was not happy with
>> it. Is there any intermediate ground that could satisfy both of you?
>
> It's a tree, so perhaps just stick with tree terminology, everyone is
> used to that. Makes sense? One way or another, this needs to be
> properly described in docs, all terminology. That would make things more
> clear, I believe.
@Jakub, would you be ok with:
'inputs' -> 'leaves'
'output' -> 'node'
?
Also while at it, I think renaming the 'group()' operation as
'node_set()' could be clearer (or at least less unclear), WDYT?
Note: I think it's would be more user-friendly to keep a single
delete/get/dump operation for 'nodes' and leaves.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-12 14:58 ` Paolo Abeni
@ 2024-08-12 15:25 ` Jakub Kicinski
2024-08-12 16:50 ` Jiri Pirko
2024-08-13 17:12 ` Donald Hunter
0 siblings, 2 replies; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-12 15:25 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jiri Pirko, Donald Hunter, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
On Mon, 12 Aug 2024 16:58:33 +0200 Paolo Abeni wrote:
> > It's a tree, so perhaps just stick with tree terminology, everyone is
> > used to that. Makes sense? One way or another, this needs to be
> > properly described in docs, all terminology. That would make things more
> > clear, I believe.
>
> @Jakub, would you be ok with:
>
> 'inputs' -> 'leaves'
> 'output' -> 'node'
> ?
I think the confusion is primarily about the parent / child.
input and output should be very clear, IMO.
> Also while at it, I think renaming the 'group()' operation as
> 'node_set()' could be clearer (or at least less unclear), WDYT?
No idea how we arrived at node_set(), and how it can possibly
represent a grouping operation.
The operations is grouping inputs and creating a scheduler node.
> Note: I think it's would be more user-friendly to keep a single
> delete/get/dump operation for 'nodes' and leaves.
Are you implying that nodes and leaves are different types of objects?
Aren't leaves nodes without any inputs?
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-11 12:40 ` Simon Horman
@ 2024-08-12 15:31 ` Jakub Kicinski
0 siblings, 0 replies; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-12 15:31 UTC (permalink / raw)
To: Simon Horman
Cc: Paolo Abeni, netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On Sun, 11 Aug 2024 13:40:53 +0100 Simon Horman wrote:
> > > Excellent idea, let's try it! Could you send a PR to NIPA?
> >
> > Yes, can do.
>
> For reference, the PR is here:
> https://github.com/linux-netdev/nipa/pull/35
Deployed now, thank you!
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 08/12] testing: net-drv: add basic shaper test
2024-08-08 14:17 ` Jakub Kicinski
2024-08-08 14:34 ` Simon Horman
@ 2024-08-12 16:03 ` Paolo Abeni
1 sibling, 0 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-08-12 16:03 UTC (permalink / raw)
To: Jakub Kicinski, Simon Horman
Cc: netdev, Jiri Pirko, Madhu Chittim, Sridhar Samudrala,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On 8/8/24 16:17, Jakub Kicinski wrote:
> On Thu, 8 Aug 2024 13:20:42 +0100 Simon Horman wrote:
>> Thanks again for the information.
>>
>> I have now taken another look at this problem.
>>
>> Firstly, my analysis is that the cause of the problem is a combination of
>> the way the patchset is constricted, and the way that the build tests (I
>> have focussed on build_allmodconfig_warn.sh [1]).
>>
>> [1] https://github.com/linux-netdev/nipa/blob/main/tests/patch/build_allmodconfig_warn/build_allmodconfig.sh
>>
>> What I believe happens is this: The patches 01/12 - 07/12 modify some
>> header files, adds a new Kconfig entry, and does a bunch of other normal
>> stuff. Each of those patches is tested in turn, and everything seems fine.
>>
>> Then we get to patch 08/12. The key thing about this patch is that it
>> enables the CONFIG_NET_SHAPER Kconfig option, in the context of an
>> allmodconfig build. That in turn modifies the headers
>> include/linux/netdevice.h and net/core/dev.h (and net/Makefile). Not in the
>> in terms of their on-disk contents changing, but rather in the case of the
>> header files, in terms of preprocessor output. And this is, I believe,
>> where everything goes wrong.
>
> That's strange, make does not understand preprocessor, does it?
AFICS kbuild creates a file for each enabled knob under include/config/.
Then, for each kernel object target, it creates a .cmd file including
the list of all dependencies. Such list comprises all the included files
_and_ the relevant, mentioned "knob" file under include/config/
scripts/basic/fixdep is responsible for including the "knob files" in
the dependency list.
To test the above:
make drivers/net/ethernet/intel/ice/ice_main.o
touch include/config/NET_SHAPER
make V=2 drivers/net/ethernet/intel/ice/ice_main.o
CALL scripts/checksyscalls.sh - due to target is PHONY
DESCEND objtool
INSTALL libsubcmd_headers
CC drivers/net/ethernet/intel/ice/ice_main.o - due to:
include/config/NET_SHAPER
Cheers,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-12 15:25 ` Jakub Kicinski
@ 2024-08-12 16:50 ` Jiri Pirko
2024-08-12 17:42 ` Jakub Kicinski
2024-08-13 17:12 ` Donald Hunter
1 sibling, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-12 16:50 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, Donald Hunter, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Mon, Aug 12, 2024 at 05:25:44PM CEST, kuba@kernel.org wrote:
>On Mon, 12 Aug 2024 16:58:33 +0200 Paolo Abeni wrote:
>> > It's a tree, so perhaps just stick with tree terminology, everyone is
>> > used to that. Makes sense? One way or another, this needs to be
>> > properly described in docs, all terminology. That would make things more
>> > clear, I believe.
>>
>> @Jakub, would you be ok with:
>>
>> 'inputs' -> 'leaves'
>> 'output' -> 'node'
>> ?
>
>I think the confusion is primarily about the parent / child.
>input and output should be very clear, IMO.
For me, "inputs" and "output" in this context sounds very odd. It should
be children and parent, isn't it. Confused...
>
>> Also while at it, I think renaming the 'group()' operation as
>> 'node_set()' could be clearer (or at least less unclear), WDYT?
>
>No idea how we arrived at node_set(), and how it can possibly
subtree_set() ?
>represent a grouping operation.
>The operations is grouping inputs and creating a scheduler node.
>
>> Note: I think it's would be more user-friendly to keep a single
>> delete/get/dump operation for 'nodes' and leaves.
>
>Are you implying that nodes and leaves are different types of objects?
>Aren't leaves nodes without any inputs?
Agree. Same op would be nice for both.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-12 16:50 ` Jiri Pirko
@ 2024-08-12 17:42 ` Jakub Kicinski
2024-08-13 5:38 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-12 17:42 UTC (permalink / raw)
To: Jiri Pirko
Cc: Paolo Abeni, Donald Hunter, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
On Mon, 12 Aug 2024 18:50:06 +0200 Jiri Pirko wrote:
> Mon, Aug 12, 2024 at 05:25:44PM CEST, kuba@kernel.org wrote:
> >I think the confusion is primarily about the parent / child.
> >input and output should be very clear, IMO.
>
> For me, "inputs" and "output" in this context sounds very odd. It should
> be children and parent, isn't it. Confused...
Parent / child is completely confusing. Let's not.
User will classify traffic based on 'leaf' attributes.
Therefore in my mind traffic enters the tree at the "leaves",
and travels towards the root (whether or not that's how HW
evaluates the hierarchy).
This is opposite to how trees as an data structure are normally
traversed. Hence I find the tree analogy to be imperfect.
But yes, root and leaf are definitely better than parent / child.
> >> Also while at it, I think renaming the 'group()' operation as
> >> 'node_set()' could be clearer (or at least less unclear), WDYT?
> >
> >No idea how we arrived at node_set(), and how it can possibly
>
> subtree_set() ?
The operation is grouping inputs and creating a scheduler node.
> >represent a grouping operation.
> >The operations is grouping inputs and creating a scheduler node.
> >
> >> Note: I think it's would be more user-friendly to keep a single
> >> delete/get/dump operation for 'nodes' and leaves.
> >
> >Are you implying that nodes and leaves are different types of objects?
> >Aren't leaves nodes without any inputs?
>
> Agree. Same op would be nice for both.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-12 17:42 ` Jakub Kicinski
@ 2024-08-13 5:38 ` Jiri Pirko
2024-08-13 14:12 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-13 5:38 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, Donald Hunter, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Mon, Aug 12, 2024 at 07:42:21PM CEST, kuba@kernel.org wrote:
>On Mon, 12 Aug 2024 18:50:06 +0200 Jiri Pirko wrote:
>> Mon, Aug 12, 2024 at 05:25:44PM CEST, kuba@kernel.org wrote:
>> >I think the confusion is primarily about the parent / child.
>> >input and output should be very clear, IMO.
>>
>> For me, "inputs" and "output" in this context sounds very odd. It should
>> be children and parent, isn't it. Confused...
>
>Parent / child is completely confusing. Let's not.
>
>User will classify traffic based on 'leaf' attributes.
>Therefore in my mind traffic enters the tree at the "leaves",
>and travels towards the root (whether or not that's how HW
>evaluates the hierarchy).
>
>This is opposite to how trees as an data structure are normally
>traversed. Hence I find the tree analogy to be imperfect.
Normally? Tree as a datastructure could be traversed freely, why it
can't? In this case, it is traversed from leaf to root. It's still a
tree. Why the tree analogy is imperfect. From what I see, it fits 100%.
>But yes, root and leaf are definitely better than parent / child.
Node has 0-n children and 0-1 parents. In case it has 0 children, it's a
leaf, in case it has 0 parents, it's a root.
This is the common tree terminology, isn't it?
>
>> >> Also while at it, I think renaming the 'group()' operation as
>> >> 'node_set()' could be clearer (or at least less unclear), WDYT?
>> >
>> >No idea how we arrived at node_set(), and how it can possibly
>>
>> subtree_set() ?
>
>The operation is grouping inputs and creating a scheduler node.
Creating a node inside a tree, isn't it? Therefore subtree.
But it could be unified to node_set() as Paolo suggested. That would
work for any node, including leaf, tree, non-existent internal node.
>
>> >represent a grouping operation.
>> >The operations is grouping inputs and creating a scheduler node.
>> >
>> >> Note: I think it's would be more user-friendly to keep a single
>> >> delete/get/dump operation for 'nodes' and leaves.
>> >
>> >Are you implying that nodes and leaves are different types of objects?
>> >Aren't leaves nodes without any inputs?
>>
>> Agree. Same op would be nice for both.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-13 5:38 ` Jiri Pirko
@ 2024-08-13 14:12 ` Jakub Kicinski
2024-08-13 14:47 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-13 14:12 UTC (permalink / raw)
To: Jiri Pirko
Cc: Paolo Abeni, Donald Hunter, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
On Tue, 13 Aug 2024 07:38:46 +0200 Jiri Pirko wrote:
> >Parent / child is completely confusing. Let's not.
> >
> >User will classify traffic based on 'leaf' attributes.
> >Therefore in my mind traffic enters the tree at the "leaves",
> >and travels towards the root (whether or not that's how HW
> >evaluates the hierarchy).
> >
> >This is opposite to how trees as an data structure are normally
> >traversed. Hence I find the tree analogy to be imperfect.
>
> Normally?
Yes, normally, in sort and/or lookup algorithms the owner of the tree
has a pointer to root, and walks from root.
> Tree as a datastructure could be traversed freely, why it
> can't?
I didn't say it can't.
> In this case, it is traversed from leaf to root. It's still a
> tree. Why the tree analogy is imperfect. From what I see, it fits 100%.
>
> >But yes, root and leaf are definitely better than parent / child.
>
> Node has 0-n children and 0-1 parents. In case it has 0 children, it's a
> leaf, in case it has 0 parents, it's a root.
> This is the common tree terminology, isn't it?
You're using tree terminology and then you're asking if it's the tree
terminology. What are you trying to prove?
To me using input / output is more intuitive, as it matches direction
of traffic flow. I'm fine with root / leaf tho, as I said.
> >> subtree_set() ?
> >
> >The operation is grouping inputs and creating a scheduler node.
>
> Creating a node inside a tree, isn't it? Therefore subtree.
All nodes are inside the tree.
> But it could be unified to node_set() as Paolo suggested. That would
> work for any node, including leaf, tree, non-existent internal node.
A "set" operation which creates a node.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-13 14:12 ` Jakub Kicinski
@ 2024-08-13 14:47 ` Paolo Abeni
2024-08-13 14:58 ` Jakub Kicinski
2024-08-14 8:56 ` Donald Hunter
0 siblings, 2 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-08-13 14:47 UTC (permalink / raw)
To: Jakub Kicinski, Jiri Pirko
Cc: Donald Hunter, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/13/24 16:12, Jakub Kicinski wrote:
> To me using input / output is more intuitive, as it matches direction
> of traffic flow. I'm fine with root / leaf tho, as I said.
Can we converge on root / leaf ?
>>>> subtree_set() ?
>>>
>>> The operation is grouping inputs and creating a scheduler node.
>>
>> Creating a node inside a tree, isn't it? Therefore subtree.
>
> All nodes are inside the tree.
>
>> But it could be unified to node_set() as Paolo suggested. That would
>> work for any node, including leaf, tree, non-existent internal node.
>
> A "set" operation which creates a node.
Here the outcome is unclear to me. My understanding is that group() does
not fit Jiri nor Donald and and node_set() or subtree_set() do not fit
Jakub.
Did I misread something? As a trade-off, what about, group_set()?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-13 14:47 ` Paolo Abeni
@ 2024-08-13 14:58 ` Jakub Kicinski
2024-08-13 15:31 ` Paolo Abeni
2024-08-14 8:56 ` Donald Hunter
1 sibling, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-13 14:58 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jiri Pirko, Donald Hunter, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
On Tue, 13 Aug 2024 16:47:34 +0200 Paolo Abeni wrote:
> >> Creating a node inside a tree, isn't it? Therefore subtree.
> >
> > All nodes are inside the tree.
> >
> >> But it could be unified to node_set() as Paolo suggested. That would
> >> work for any node, including leaf, tree, non-existent internal node.
> >
> > A "set" operation which creates a node.
>
> Here the outcome is unclear to me. My understanding is that group() does
> not fit Jiri nor Donald and and node_set() or subtree_set() do not fit
> Jakub.
>
> Did I misread something? As a trade-off, what about, group_set()?
"set" is not a sensible verb for creating something. "group" in
the original was the verb.
Why are both saying "set" and not "create"? What am I missing?
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-01 13:42 ` Jiri Pirko
@ 2024-08-13 15:17 ` Paolo Abeni
2024-08-14 8:37 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-13 15:17 UTC (permalink / raw)
To: Jiri Pirko
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/1/24 15:42, Jiri Pirko wrote:
> Tue, Jul 30, 2024 at 10:39:46PM CEST, pabeni@redhat.com wrote:
>> +/**
>> + * net_shaper_make_handle - creates an unique shaper identifier
>> + * @scope: the shaper scope
>> + * @id: the shaper id number
>> + *
>> + * Return: an unique identifier for the shaper
>> + *
>> + * Combines the specified arguments to create an unique identifier for
>> + * the shaper. The @id argument semantic depends on the
>> + * specified scope.
>> + * For @NET_SHAPER_SCOPE_QUEUE_GROUP, @id is the queue group id
>> + * For @NET_SHAPER_SCOPE_QUEUE, @id is the queue number.
>> + * For @NET_SHAPER_SCOPE_VF, @id is virtual function number.
>> + */
>> +static inline u32 net_shaper_make_handle(enum net_shaper_scope scope,
>> + int id)
>> +{
>> + return FIELD_PREP(NET_SHAPER_SCOPE_MASK, scope) |
>> + FIELD_PREP(NET_SHAPER_ID_MASK, id);
>
> Perhaps some scopes may find only part of u32 as limitting for id in
> the future? I find it elegant to have it in single u32 though. u64 may
> be nicer (I know, xarray) :)
With this code the id limit is 2^26 for each scope. The most capable H/W
I'm aware of supports at most 64K shapers, overall. Are you aware of any
specific constraint we need to address?
[...]
>> int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
>> {
>> - return -EOPNOTSUPP;
>> + struct net_shaper_info *shaper;
>> + struct net_device *dev;
>> + struct sk_buff *msg;
>> + u32 handle;
>> + int ret;
>> +
>> + ret = fetch_dev(info, &dev);
>
> This is quite net_device centric. Devlink rate shaper should be
> eventually visible throught this api as well, won't they? How do you
> imagine that?
I'm unsure we are on the same page. Do you foresee this to replace and
obsoleted the existing devlink rate API? It was not our so far.
> Could we have various types of binding? Something like:
>
> NET_SHAPER_A_BINDING nest
> NET_SHAPER_A_BINDING_IFINDEX u32
>
> or:
> NET_SHAPER_A_BINDING nest
> NET_SHAPER_A_BINDING_DEVLINK_PORT nest
> DEVLINK_ATTR_BUS_NAME string
> DEVLINK_ATTR_DEV_NAME string
> DEVLINK_ATTR_PORT_INDEX u32
>
> ?
Somewhat related, the current get()/dump() operations currently don't
return the shaper ifindex. I guess we can include 'scope' and 'id' under
NET_SHAPER_A_BINDING and replace the existing handle attribute with it.
It should cover eventual future devlink extensions and provide all the
relevant info for get/dump sake.
>> +
>> static int __init shaper_init(void)
>
>
>
> fetch_dev
> fill_handle
> parse_handle
> sc_lookup
> __sc_container
> dev_shaper_flush
> shaper_init
>
>
> Could you perhaps maintain net_shaper_ prefix for all of there?
Most of the helpers are static and should never be visible outside this
compilation unit, so I did not bother with a prefix, I'll add it in the
next revision.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-13 14:58 ` Jakub Kicinski
@ 2024-08-13 15:31 ` Paolo Abeni
2024-08-13 15:43 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-13 15:31 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Jiri Pirko, Donald Hunter, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
On 8/13/24 16:58, Jakub Kicinski wrote:
> On Tue, 13 Aug 2024 16:47:34 +0200 Paolo Abeni wrote:
>>>> Creating a node inside a tree, isn't it? Therefore subtree.
>>>
>>> All nodes are inside the tree.
>>>
>>>> But it could be unified to node_set() as Paolo suggested. That would
>>>> work for any node, including leaf, tree, non-existent internal node.
>>>
>>> A "set" operation which creates a node.
>>
>> Here the outcome is unclear to me. My understanding is that group() does
>> not fit Jiri nor Donald and and node_set() or subtree_set() do not fit
>> Jakub.
>>
>> Did I misread something? As a trade-off, what about, group_set()?
>
> "set" is not a sensible verb for creating something. "group" in
> the original was the verb.
> Why are both saying "set" and not "create"? What am I missing?
Please, don't read too much in my limited English skills!
I'm fine with group_create() - or create_group()
Still WRT naming, I almost forgot about the much blamed 'detached'
scope. Would 'node' or 'group' be a better name? (the latter only if we
rename the homonymous operation)
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-13 15:31 ` Paolo Abeni
@ 2024-08-13 15:43 ` Jakub Kicinski
0 siblings, 0 replies; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-13 15:43 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jiri Pirko, Donald Hunter, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
On Tue, 13 Aug 2024 17:31:17 +0200 Paolo Abeni wrote:
> > "set" is not a sensible verb for creating something. "group" in
> > the original was the verb.
> > Why are both saying "set" and not "create"? What am I missing?
>
> Please, don't read too much in my limited English skills!
> I'm fine with group_create() - or create_group()
Again, group was a verb :)
I don't think anyone suggested group as a noun / object.
> Still WRT naming, I almost forgot about the much blamed 'detached'
> scope. Would 'node' or 'group' be a better name? (the latter only if we
> rename the homonymous operation)
I vote 'node', given the above.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-12 15:25 ` Jakub Kicinski
2024-08-12 16:50 ` Jiri Pirko
@ 2024-08-13 17:12 ` Donald Hunter
2024-08-14 14:21 ` Paolo Abeni
1 sibling, 1 reply; 91+ messages in thread
From: Donald Hunter @ 2024-08-13 17:12 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Jakub Kicinski <kuba@kernel.org> writes:
> On Mon, 12 Aug 2024 16:58:33 +0200 Paolo Abeni wrote:
>> > It's a tree, so perhaps just stick with tree terminology, everyone is
>> > used to that. Makes sense? One way or another, this needs to be
>> > properly described in docs, all terminology. That would make things more
>> > clear, I believe.
>>
>> @Jakub, would you be ok with:
>>
>> 'inputs' -> 'leaves'
>> 'output' -> 'node'
>> ?
>
> I think the confusion is primarily about the parent / child.
> input and output should be very clear, IMO.
input / output seems the most intuitive of the different terms that have
been suggested.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-13 15:17 ` Paolo Abeni
@ 2024-08-14 8:37 ` Jiri Pirko
2024-08-16 8:52 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-14 8:37 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Aug 13, 2024 at 05:17:12PM CEST, pabeni@redhat.com wrote:
>On 8/1/24 15:42, Jiri Pirko wrote:
>> Tue, Jul 30, 2024 at 10:39:46PM CEST, pabeni@redhat.com wrote:
>> > +/**
>> > + * net_shaper_make_handle - creates an unique shaper identifier
>> > + * @scope: the shaper scope
>> > + * @id: the shaper id number
>> > + *
>> > + * Return: an unique identifier for the shaper
>> > + *
>> > + * Combines the specified arguments to create an unique identifier for
>> > + * the shaper. The @id argument semantic depends on the
>> > + * specified scope.
>> > + * For @NET_SHAPER_SCOPE_QUEUE_GROUP, @id is the queue group id
>> > + * For @NET_SHAPER_SCOPE_QUEUE, @id is the queue number.
>> > + * For @NET_SHAPER_SCOPE_VF, @id is virtual function number.
>> > + */
>> > +static inline u32 net_shaper_make_handle(enum net_shaper_scope scope,
>> > + int id)
>> > +{
>> > + return FIELD_PREP(NET_SHAPER_SCOPE_MASK, scope) |
>> > + FIELD_PREP(NET_SHAPER_ID_MASK, id);
>>
>> Perhaps some scopes may find only part of u32 as limitting for id in
>> the future? I find it elegant to have it in single u32 though. u64 may
>> be nicer (I know, xarray) :)
>
>With this code the id limit is 2^26 for each scope. The most capable H/W I'm
>aware of supports at most 64K shapers, overall. Are you aware of any specific
>constraint we need to address?
Nope. Just thinking out loud.
>
>[...]
>> > int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
>> > {
>> > - return -EOPNOTSUPP;
>> > + struct net_shaper_info *shaper;
>> > + struct net_device *dev;
>> > + struct sk_buff *msg;
>> > + u32 handle;
>> > + int ret;
>> > +
>> > + ret = fetch_dev(info, &dev);
>>
>> This is quite net_device centric. Devlink rate shaper should be
>> eventually visible throught this api as well, won't they? How do you
>> imagine that?
>
>I'm unsure we are on the same page. Do you foresee this to replace and
>obsoleted the existing devlink rate API? It was not our so far.
Driver-api-wise, yes. I believe that was the goal, to have drivers to
implement one rate api.
>
>> Could we have various types of binding? Something like:
>>
>> NET_SHAPER_A_BINDING nest
>> NET_SHAPER_A_BINDING_IFINDEX u32
>>
>> or:
>> NET_SHAPER_A_BINDING nest
>> NET_SHAPER_A_BINDING_DEVLINK_PORT nest
>> DEVLINK_ATTR_BUS_NAME string
>> DEVLINK_ATTR_DEV_NAME string
>> DEVLINK_ATTR_PORT_INDEX u32
>>
>> ?
>
>Somewhat related, the current get()/dump() operations currently don't return
>the shaper ifindex. I guess we can include 'scope' and 'id' under
>NET_SHAPER_A_BINDING and replace the existing handle attribute with it.
>
>It should cover eventual future devlink extensions and provide all the
>relevant info for get/dump sake.
Sounds fine.
>
>> > +
>> > static int __init shaper_init(void)
>>
>>
>>
>> fetch_dev
>> fill_handle
>> parse_handle
>> sc_lookup
>> __sc_container
>> dev_shaper_flush
>> shaper_init
>>
>>
>> Could you perhaps maintain net_shaper_ prefix for all of there?
>
>Most of the helpers are static and should never be visible outside this
>compilation unit, so I did not bother with a prefix, I'll add it in the next
>revision.
Thanks.
>
>Thanks,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-13 14:47 ` Paolo Abeni
2024-08-13 14:58 ` Jakub Kicinski
@ 2024-08-14 8:56 ` Donald Hunter
1 sibling, 0 replies; 91+ messages in thread
From: Donald Hunter @ 2024-08-14 8:56 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jakub Kicinski, Jiri Pirko, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Paolo Abeni <pabeni@redhat.com> writes:
> On 8/13/24 16:12, Jakub Kicinski wrote:
>> To me using input / output is more intuitive, as it matches direction
>> of traffic flow. I'm fine with root / leaf tho, as I said.
>
> Can we converge on root / leaf ?
>
>>>>> subtree_set() ?
>>>>
>>>> The operation is grouping inputs and creating a scheduler node.
>>>
>>> Creating a node inside a tree, isn't it? Therefore subtree.
>> All nodes are inside the tree.
>>
>>> But it could be unified to node_set() as Paolo suggested. That would
>>> work for any node, including leaf, tree, non-existent internal node.
>> A "set" operation which creates a node.
>
> Here the outcome is unclear to me. My understanding is that group() does not fit Jiri nor Donald
> and and node_set() or subtree_set() do not fit Jakub.
I found group() confusing because it does not imply creation. So
create-group() would be fine. But it seems like creating a group is a
step towards creating a scheduler node?
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-13 17:12 ` Donald Hunter
@ 2024-08-14 14:21 ` Paolo Abeni
2024-08-15 9:07 ` Donald Hunter
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-14 14:21 UTC (permalink / raw)
To: Donald Hunter, Jakub Kicinski
Cc: Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/13/24 19:12, Donald Hunter wrote:
> Jakub Kicinski <kuba@kernel.org> writes:
>
>> On Mon, 12 Aug 2024 16:58:33 +0200 Paolo Abeni wrote:
>>>> It's a tree, so perhaps just stick with tree terminology, everyone is
>>>> used to that. Makes sense? One way or another, this needs to be
>>>> properly described in docs, all terminology. That would make things more
>>>> clear, I believe.
>>>
>>> @Jakub, would you be ok with:
>>>
>>> 'inputs' -> 'leaves'
>>> 'output' -> 'node'
>>> ?
>>
>> I think the confusion is primarily about th parent / child.
>> input and output should be very clear, IMO.
>
> input / output seems the most intuitive of the different terms that have
> been suggested.
Since a sort of agreement was reached about using root / leaf instead,
and (quoting someone smarter then me) a good compromise makes every
party equally unhappy, would you consider such other option?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 02/12] netlink: spec: add shaper YAML spec
2024-08-14 14:21 ` Paolo Abeni
@ 2024-08-15 9:07 ` Donald Hunter
0 siblings, 0 replies; 91+ messages in thread
From: Donald Hunter @ 2024-08-15 9:07 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jakub Kicinski, Jiri Pirko, netdev, Madhu Chittim,
Sridhar Samudrala, Simon Horman, John Fastabend,
Sunil Kovvuri Goutham, Jamal Hadi Salim
Paolo Abeni <pabeni@redhat.com> writes:
> On 8/13/24 19:12, Donald Hunter wrote:
>> Jakub Kicinski <kuba@kernel.org> writes:
>>
>>> On Mon, 12 Aug 2024 16:58:33 +0200 Paolo Abeni wrote:
>>>>> It's a tree, so perhaps just stick with tree terminology, everyone is
>>>>> used to that. Makes sense? One way or another, this needs to be
>>>>> properly described in docs, all terminology. That would make things more
>>>>> clear, I believe.
>>>>
>>>> @Jakub, would you be ok with:
>>>>
>>>> 'inputs' -> 'leaves'
>>>> 'output' -> 'node'
>>>> ?
>>>
>>> I think the confusion is primarily about th parent / child.
>>> input and output should be very clear, IMO.
>> input / output seems the most intuitive of the different terms that have
>> been suggested.
>
> Since a sort of agreement was reached about using root / leaf instead, and (quoting someone
> smarter then me) a good compromise makes every party equally unhappy, would you consider such
> other option?
Sure, if that is the general consensus. Though I do find it confusing
because there are not any tree-like ops. Does that mean you are shifting
away from using 'inputs' and 'output' for the group op?
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-14 8:37 ` Jiri Pirko
@ 2024-08-16 8:52 ` Paolo Abeni
2024-08-16 9:16 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-16 8:52 UTC (permalink / raw)
To: Jiri Pirko
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/14/24 10:37, Jiri Pirko wrote:
> Tue, Aug 13, 2024 at 05:17:12PM CEST, pabeni@redhat.com wrote:
>> On 8/1/24 15:42, Jiri Pirko wrote:
>> [...]
>>>> int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
>>>> {
>>>> - return -EOPNOTSUPP;
>>>> + struct net_shaper_info *shaper;
>>>> + struct net_device *dev;
>>>> + struct sk_buff *msg;
>>>> + u32 handle;
>>>> + int ret;
>>>> +
>>>> + ret = fetch_dev(info, &dev);
>>>
>>> This is quite net_device centric. Devlink rate shaper should be
>>> eventually visible throught this api as well, won't they? How do you
>>> imagine that?
>>
>> I'm unsure we are on the same page. Do you foresee this to replace and
>> obsoleted the existing devlink rate API? It was not our so far.
>
> Driver-api-wise, yes. I believe that was the goal, to have drivers to
> implement one rate api.
I initially underlooked at this point, I'm sorry.
Re-reading this I think we are not on the same page.
The net_shaper_ops are per network device operations: they are aimed
(also) at consolidating network device shaping related callbacks, but
they can't operate on non-network device objects (devlink port).
Cheers,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-16 8:52 ` Paolo Abeni
@ 2024-08-16 9:16 ` Jiri Pirko
2024-08-19 9:33 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-16 9:16 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Jakub Kicinski, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Fri, Aug 16, 2024 at 10:52:58AM CEST, pabeni@redhat.com wrote:
>On 8/14/24 10:37, Jiri Pirko wrote:
>> Tue, Aug 13, 2024 at 05:17:12PM CEST, pabeni@redhat.com wrote:
>> > On 8/1/24 15:42, Jiri Pirko wrote:
>> > [...]
>> > > > int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
>> > > > {
>> > > > - return -EOPNOTSUPP;
>> > > > + struct net_shaper_info *shaper;
>> > > > + struct net_device *dev;
>> > > > + struct sk_buff *msg;
>> > > > + u32 handle;
>> > > > + int ret;
>> > > > +
>> > > > + ret = fetch_dev(info, &dev);
>> > >
>> > > This is quite net_device centric. Devlink rate shaper should be
>> > > eventually visible throught this api as well, won't they? How do you
>> > > imagine that?
>> >
>> > I'm unsure we are on the same page. Do you foresee this to replace and
>> > obsoleted the existing devlink rate API? It was not our so far.
>>
>> Driver-api-wise, yes. I believe that was the goal, to have drivers to
>> implement one rate api.
>
>I initially underlooked at this point, I'm sorry.
>
>Re-reading this I think we are not on the same page.
>
>The net_shaper_ops are per network device operations: they are aimed (also)
>at consolidating network device shaping related callbacks, but they can't
>operate on non-network device objects (devlink port).
Why not?
>
>Cheers,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-16 9:16 ` Jiri Pirko
@ 2024-08-19 9:33 ` Paolo Abeni
2024-08-19 11:53 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-19 9:33 UTC (permalink / raw)
To: Jiri Pirko
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim,
Jakub Kicinski
On 8/16/24 11:16, Jiri Pirko wrote:
> Fri, Aug 16, 2024 at 10:52:58AM CEST, pabeni@redhat.com wrote:
>> On 8/14/24 10:37, Jiri Pirko wrote:
>>> Tue, Aug 13, 2024 at 05:17:12PM CEST, pabeni@redhat.com wrote:
>>>> On 8/1/24 15:42, Jiri Pirko wrote:
>>>> [...]
>>>>>> int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
>>>>>> {
>>>>>> - return -EOPNOTSUPP;
>>>>>> + struct net_shaper_info *shaper;
>>>>>> + struct net_device *dev;
>>>>>> + struct sk_buff *msg;
>>>>>> + u32 handle;
>>>>>> + int ret;
>>>>>> +
>>>>>> + ret = fetch_dev(info, &dev);
>>>>>
>>>>> This is quite net_device centric. Devlink rate shaper should be
>>>>> eventually visible throught this api as well, won't they? How do you
>>>>> imagine that?
>>>>
>>>> I'm unsure we are on the same page. Do you foresee this to replace and
>>>> obsoleted the existing devlink rate API? It was not our so far.
>>>
>>> Driver-api-wise, yes. I believe that was the goal, to have drivers to
>>> implement one rate api.
>>
>> I initially underlooked at this point, I'm sorry.
>>
>> Re-reading this I think we are not on the same page.
>>
>> The net_shaper_ops are per network device operations: they are aimed (also)
>> at consolidating network device shaping related callbacks, but they can't
>> operate on non-network device objects (devlink port).
>
> Why not?
Isn't the whole point of devlink to configure objects that are directly
related to any network device? Would be somewhat awkward accessing
devlink port going through some net_device?
Side note: I experimented adding the 'binging' abstraction to this API
and gives a quite significant uglification to the user syntax (due to
the additional nesting required) and the code.
Still, if there is a very strong need for controlling devlink rate via
this API _and_ we can assume that each net_device "relates"
(/references/is connected to) at most a single devlink object (out of
sheer ignorance on my side I'm unsure about this point, but skimming
over the existing implementations it looks so), the current API
definition would be IMHO sufficient and clean enough to reach for both
devlink port rate objects and devlink node rate objects.
We could define additional scopes for each of such objects and use the
id to discriminate the specific port or node within the relevant devlink.
I think such scopes definition should come with related implementation,
e.g. not with this series.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-19 9:33 ` Paolo Abeni
@ 2024-08-19 11:53 ` Jiri Pirko
2024-08-19 16:52 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-19 11:53 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim,
Jakub Kicinski
Mon, Aug 19, 2024 at 11:33:28AM CEST, pabeni@redhat.com wrote:
>
>
>On 8/16/24 11:16, Jiri Pirko wrote:
>> Fri, Aug 16, 2024 at 10:52:58AM CEST, pabeni@redhat.com wrote:
>> > On 8/14/24 10:37, Jiri Pirko wrote:
>> > > Tue, Aug 13, 2024 at 05:17:12PM CEST, pabeni@redhat.com wrote:
>> > > > On 8/1/24 15:42, Jiri Pirko wrote:
>> > > > [...]
>> > > > > > int net_shaper_nl_get_doit(struct sk_buff *skb, struct genl_info *info)
>> > > > > > {
>> > > > > > - return -EOPNOTSUPP;
>> > > > > > + struct net_shaper_info *shaper;
>> > > > > > + struct net_device *dev;
>> > > > > > + struct sk_buff *msg;
>> > > > > > + u32 handle;
>> > > > > > + int ret;
>> > > > > > +
>> > > > > > + ret = fetch_dev(info, &dev);
>> > > > >
>> > > > > This is quite net_device centric. Devlink rate shaper should be
>> > > > > eventually visible throught this api as well, won't they? How do you
>> > > > > imagine that?
>> > > >
>> > > > I'm unsure we are on the same page. Do you foresee this to replace and
>> > > > obsoleted the existing devlink rate API? It was not our so far.
>> > >
>> > > Driver-api-wise, yes. I believe that was the goal, to have drivers to
>> > > implement one rate api.
>> >
>> > I initially underlooked at this point, I'm sorry.
>> >
>> > Re-reading this I think we are not on the same page.
>> >
>> > The net_shaper_ops are per network device operations: they are aimed (also)
>> > at consolidating network device shaping related callbacks, but they can't
>> > operate on non-network device objects (devlink port).
>>
>> Why not?
>
>Isn't the whole point of devlink to configure objects that are directly
>related to any network device? Would be somewhat awkward accessing devlink
Yeah, not even network. Just "a device".
>port going through some net_device?
I'm not sure why you are asking that. I didn't suggest anything like
that. On contrary, as your API is netdev centric, I suggested to
disconnect from netdev to the shapers could be used not only with them.
This is what I understood was a plan from very beginning. I may be wrong
though....
>
>Side note: I experimented adding the 'binging' abstraction to this API and
>gives a quite significant uglification to the user syntax (due to the
>additional nesting required) and the code.
>
>Still, if there is a very strong need for controlling devlink rate via this
>API _and_ we can assume that each net_device "relates" (/references/is
>connected to) at most a single devlink object (out of sheer ignorance on my
>side I'm unsure about this point, but skimming over the existing
>implementations it looks so), the current API definition would be IMHO
>sufficient and clean enough to reach for both devlink port rate objects and
>devlink node rate objects.
Don't assume this. Not always true.
>
>We could define additional scopes for each of such objects and use the id to
>discriminate the specific port or node within the relevant devlink.
But you still want to use some netdevice as a handle IIUC, is that
right?
>
>I think such scopes definition should come with related implementation, e.g.
>not with this series.
>
>Thanks,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-19 11:53 ` Jiri Pirko
@ 2024-08-19 16:52 ` Paolo Abeni
2024-08-22 12:02 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-19 16:52 UTC (permalink / raw)
To: Jiri Pirko
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim,
Jakub Kicinski
On 8/19/24 13:53, Jiri Pirko wrote:
> Mon, Aug 19, 2024 at 11:33:28AM CEST, pabeni@redhat.com wrote:
>> Isn't the whole point of devlink to configure objects that are directly
>> related to any network device? Would be somewhat awkward accessing devlink
>> port going through some net_device?
>
> I'm not sure why you are asking that. I didn't suggest anything like
> that. On contrary, as your API is netdev centric, I suggested to
> disconnect from netdev to the shapers could be used not only with them.
ndo_shaper_ops are basically net_device ndo. Any implementation of them
will operate 'trough some net_device'.
I'm still not sure which one of the following you mean:
1) the shaper NL interface must be able to manage devlink (rate)
objects. The core will lookup the devlink obj and use devlink_ops to do
the change.
2) the shaper NL interface must be able to manage devlink (rate)
objects, the core will use ndo_shaper_ops to do the actual change.
3) something else?
In my previous reply, I assumed you wanted option 2). If so, which kind
of object should implement the ndo_shaper_ops callbacks? net_device?
devlink? other?
> This is what I understood was a plan from very beginning.
Originally the scope was much more limited than what defined here. Jakub
asked to implement an interface capable to unify the network device
shaping/rate related callbacks.
In a previous revision, I stretched that to cover objects "above" the
network device level (e.g. PF/VF/SFs groups), but then I left them out
because:
- it caused several inconsistencies (among other thing we can't use the
'shaper cache' there and Jakub wants the cache in place).
- we already have devlink for that.
>> We could define additional scopes for each of such objects and use the id to
>> discriminate the specific port or node within the relevant devlink.
>
> But you still want to use some netdevice as a handle IIUC, is that
> right?
The revision of the series that I hope to share soon still used
net_device ndos. Shapers are identified within the network device by an
handle.
Cheers,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-19 16:52 ` Paolo Abeni
@ 2024-08-22 12:02 ` Jiri Pirko
2024-08-22 14:41 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-22 12:02 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim,
Jakub Kicinski
Mon, Aug 19, 2024 at 06:52:22PM CEST, pabeni@redhat.com wrote:
>On 8/19/24 13:53, Jiri Pirko wrote:
>> Mon, Aug 19, 2024 at 11:33:28AM CEST, pabeni@redhat.com wrote:
>> > Isn't the whole point of devlink to configure objects that are directly
>> > related to any network device? Would be somewhat awkward accessing devlink
>> > port going through some net_device?
>>
>> I'm not sure why you are asking that. I didn't suggest anything like
>> that. On contrary, as your API is netdev centric, I suggested to
>> disconnect from netdev to the shapers could be used not only with them.
>
>ndo_shaper_ops are basically net_device ndo. Any implementation of them will
>operate 'trough some net_device'.
I know, I see that in the code. But the question is, does it have to be
like that?
>
>I'm still not sure which one of the following you mean:
>
>1) the shaper NL interface must be able to manage devlink (rate) objects. The
>core will lookup the devlink obj and use devlink_ops to do the change.
>
>2) the shaper NL interface must be able to manage devlink (rate) objects, the
>core will use ndo_shaper_ops to do the actual change.
>
>3) something else?
I don't care about the shaper NL in case of devlink rate objects. I care
more about in-kernel api. I see shaper NL as one of the UAPIs to consume
the shaper infrastructure. The devlink rate is another one. If the
devlink rate shapers are visible over shaper NL, IDK. They may be RO
perhaps.
>
>In my previous reply, I assumed you wanted option 2). If so, which kind of
>object should implement the ndo_shaper_ops callbacks? net_device? devlink?
>other?
Whoever implements the shaper in driver. If that is net_device tight
shaper, driver should work with net_device. If that is devlink port
related shaper, driver should work on top of devlink port based api.
>
>> This is what I understood was a plan from very beginning.
>
>Originally the scope was much more limited than what defined here. Jakub
>asked to implement an interface capable to unify the network device
>shaping/rate related callbacks.
I'm not saying this is deal breaker for me. I just think that if the api
is designed to be independent of the object shaper is bound to
(netdev/devlink_port/etc), it would be much much easier to extend in the
future. If you do everything netdev-centric from start, I'm sure no
shaper consolidation will ever happen. And that I thought was one of the
goals.
Perhaps Jakub has opinion.
>
>In a previous revision, I stretched that to cover objects "above" the network
>device level (e.g. PF/VF/SFs groups), but then I left them out because:
>
>- it caused several inconsistencies (among other thing we can't use the
>'shaper cache' there and Jakub wants the cache in place).
>- we already have devlink for that.
>
>> > We could define additional scopes for each of such objects and use the id to
>> > discriminate the specific port or node within the relevant devlink.
>>
>> But you still want to use some netdevice as a handle IIUC, is that
>> right?
>
>The revision of the series that I hope to share soon still used net_device
>ndos. Shapers are identified within the network device by an handle.
>
>Cheers,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-22 12:02 ` Jiri Pirko
@ 2024-08-22 14:41 ` Jakub Kicinski
2024-08-22 20:30 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-22 14:41 UTC (permalink / raw)
To: Jiri Pirko
Cc: Paolo Abeni, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Thu, 22 Aug 2024 14:02:54 +0200 Jiri Pirko wrote:
>>> This is what I understood was a plan from very beginning.
>>
>> Originally the scope was much more limited than what defined here. Jakub
>> asked to implement an interface capable to unify the network device
>> shaping/rate related callbacks.
>
> I'm not saying this is deal breaker for me. I just think that if the api
> is designed to be independent of the object shaper is bound to
> (netdev/devlink_port/etc), it would be much much easier to extend in the
> future. If you do everything netdev-centric from start, I'm sure no
> shaper consolidation will ever happen. And that I thought was one of the
> goals.
>
> Perhaps Jakub has opinion.
I think you and I are on the same page :) Other than the "reference
object" (netdev / devlink port) the driver facing API should be
identical. Making it possible for the same driver code to handle
translating the parameters into HW config / FW requests, whether
they shape at the device (devlink) or port (netdev) level.
Shaper NL for netdevs is separate from internal representation and
driver API in my mind. My initial ask was to create the internal
representation first, make sure it can express devlink and handful of
exiting netdev APIs, and only once that's merged worry about exposing
it via a new NL.
I'm not opposed to showing devlink shapers in netdev NL (RO as you say)
but talking about it now strikes me as cart before the horse.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-22 14:41 ` Jakub Kicinski
@ 2024-08-22 20:30 ` Paolo Abeni
2024-08-22 22:56 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-22 20:30 UTC (permalink / raw)
To: Jakub Kicinski, Jiri Pirko
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On 8/22/24 16:41, Jakub Kicinski wrote:
> On Thu, 22 Aug 2024 14:02:54 +0200 Jiri Pirko wrote:
>>>> This is what I understood was a plan from very beginning.
>>>
>>> Originally the scope was much more limited than what defined here. Jakub
>>> asked to implement an interface capable to unify the network device
>>> shaping/rate related callbacks.
>>
>> I'm not saying this is deal breaker for me. I just think that if the api
>> is designed to be independent of the object shaper is bound to
>> (netdev/devlink_port/etc), it would be much much easier to extend in the
>> future. If you do everything netdev-centric from start, I'm sure no
>> shaper consolidation will ever happen. And that I thought was one of the
>> goals.
>>
>> Perhaps Jakub has opinion.
>
> I think you and I are on the same page :) Other than the "reference
> object" (netdev / devlink port) the driver facing API should be
> identical. Making it possible for the same driver code to handle
> translating the parameters into HW config / FW requests, whether
> they shape at the device (devlink) or port (netdev) level.
>
> Shaper NL for netdevs is separate from internal representation and
> driver API in my mind. My initial ask was to create the internal
> representation first, make sure it can express devlink and handful of
> exiting netdev APIs, and only once that's merged worry about exposing
> it via a new NL.
>
> I'm not opposed to showing devlink shapers in netdev NL (RO as you say)
> but talking about it now strikes me as cart before the horse.
FTR, I don't see both of you on the same page ?!?
I read the above as Jiri's preference is a single ndo set to control
both devlink and device shapers, while I read Jakub's preference as for
different sets of operations that will use the same arguments to specify
the shaper informations.
Or to phrase the above differently, Jiri is focusing on the shaper
"binding" (how to locate/access it) while Jakub is focusing on the
shaper "info" (content/definition/attributes). Please correct me If I
misread something.
Still for the record, I interpret the current proposal as not clashing
with Jakub's preference, and being tolerated from Jiri, again please
correct me if I read too far.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-22 20:30 ` Paolo Abeni
@ 2024-08-22 22:56 ` Jakub Kicinski
2024-08-23 11:50 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-22 22:56 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Thu, 22 Aug 2024 22:30:35 +0200 Paolo Abeni wrote:
> >> I'm not saying this is deal breaker for me. I just think that if the api
> >> is designed to be independent of the object shaper is bound to
> >> (netdev/devlink_port/etc), it would be much much easier to extend in the
> >> future. If you do everything netdev-centric from start, I'm sure no
> >> shaper consolidation will ever happen. And that I thought was one of the
> >> goals.
> >>
> >> Perhaps Jakub has opinion.
> >
> > I think you and I are on the same page :) Other than the "reference
> > object" (netdev / devlink port) the driver facing API should be
> > identical. Making it possible for the same driver code to handle
> > translating the parameters into HW config / FW requests, whether
> > they shape at the device (devlink) or port (netdev) level.
> >
> > Shaper NL for netdevs is separate from internal representation and
> > driver API in my mind. My initial ask was to create the internal
> > representation first, make sure it can express devlink and handful of
> > exiting netdev APIs, and only once that's merged worry about exposing
> > it via a new NL.
> >
> > I'm not opposed to showing devlink shapers in netdev NL (RO as you say)
> > but talking about it now strikes me as cart before the horse.
>
> FTR, I don't see both of you on the same page ?!?
>
> I read the above as Jiri's preference is a single ndo set to control
> both devlink and device shapers, while I read Jakub's preference as for
> different sets of operations that will use the same arguments to specify
> the shaper informations.
Jiri replied:
> which kind of object should implement the ndo_shaper_ops callbacks?
Whoever implements the shaper in driver. If that is net_device tight
shaper, driver should work with net_device. If that is devlink port
related shaper, driver should work on top of devlink port based api.
I interpret this as having two almost identical versions of shaper ops,
the only difference is that one takes netdev and the other devlink port.
We could simplify it slightly, and call the ndo for getting devlink
port from netdev, and always pass devlink port in?
I _think_ (but I'm not 100% sure) that Jiri does _not_ mean that we
would be able to render the internal shaper tree as ops for the
existing devlink rate API. Because that may cause scope creep,
inconsistencies and duplication.
> Or to phrase the above differently, Jiri is focusing on the shaper
> "binding" (how to locate/access it) while Jakub is focusing on the
> shaper "info" (content/definition/attributes). Please correct me If I
> misread something.
>
> Still for the record, I interpret the current proposal as not clashing
> with Jakub's preference, and being tolerated from Jiri, again please
> correct me if I read too far.
One more thing, Jiri said:
If you do everything netdev-centric from start, I'm sure no shaper
consolidation will ever happen. And that I thought was one of the goals.
Consolidation was indeed one of the goals, and I share Jiri's concern :(
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-22 22:56 ` Jakub Kicinski
@ 2024-08-23 11:50 ` Jiri Pirko
2024-08-23 12:58 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-23 11:50 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Paolo Abeni, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Fri, Aug 23, 2024 at 12:56:08AM CEST, kuba@kernel.org wrote:
>On Thu, 22 Aug 2024 22:30:35 +0200 Paolo Abeni wrote:
>> >> I'm not saying this is deal breaker for me. I just think that if the api
>> >> is designed to be independent of the object shaper is bound to
>> >> (netdev/devlink_port/etc), it would be much much easier to extend in the
>> >> future. If you do everything netdev-centric from start, I'm sure no
>> >> shaper consolidation will ever happen. And that I thought was one of the
>> >> goals.
>> >>
>> >> Perhaps Jakub has opinion.
>> >
>> > I think you and I are on the same page :) Other than the "reference
>> > object" (netdev / devlink port) the driver facing API should be
>> > identical. Making it possible for the same driver code to handle
>> > translating the parameters into HW config / FW requests, whether
>> > they shape at the device (devlink) or port (netdev) level.
>> >
>> > Shaper NL for netdevs is separate from internal representation and
>> > driver API in my mind. My initial ask was to create the internal
>> > representation first, make sure it can express devlink and handful of
>> > exiting netdev APIs, and only once that's merged worry about exposing
>> > it via a new NL.
>> >
>> > I'm not opposed to showing devlink shapers in netdev NL (RO as you say)
>> > but talking about it now strikes me as cart before the horse.
>>
>> FTR, I don't see both of you on the same page ?!?
>>
>> I read the above as Jiri's preference is a single ndo set to control
"Ndo" stands for netdev op and they are all tightly coupled with
netdevices. So, "single ndo set to control both devlink and netdev
shapers" sounds like nonsense to me.
>> both devlink and device shapers, while I read Jakub's preference as for
>> different sets of operations that will use the same arguments to specify
>> the shaper informations.
>
>Jiri replied:
>
> > which kind of object should implement the ndo_shaper_ops callbacks?
>
> Whoever implements the shaper in driver. If that is net_device tight
> shaper, driver should work with net_device. If that is devlink port
> related shaper, driver should work on top of devlink port based api.
>
>I interpret this as having two almost identical versions of shaper ops,
>the only difference is that one takes netdev and the other devlink port.
Could be done like that. But see more below.
>We could simplify it slightly, and call the ndo for getting devlink
>port from netdev, and always pass devlink port in?
No please. Keep that separate. You can't always rely on devlink port
having netdev paired with it. Plus, it would be odd callpath from
devlink port code to netdev op. Not to mention locking :)
>
>I _think_ (but I'm not 100% sure) that Jiri does _not_ mean that we
>would be able to render the internal shaper tree as ops for the
>existing devlink rate API. Because that may cause scope creep,
>inconsistencies and duplication.
Not sure what you mean by this. Devlink rate UAPI will stay the same.
Only the backend will use new driver API instead of the existing one.
>
>> Or to phrase the above differently, Jiri is focusing on the shaper
>> "binding" (how to locate/access it) while Jakub is focusing on the
>> shaper "info" (content/definition/attributes). Please correct me If I
>> misread something.
Two(or more) similar ops structs looks odd to me. I think that the ops
should should be shared and just the "binding point" should be somehow
abstracted out. Code speaks, let me draft how it could be done:
enum net_shaper_binding_type {
NET_SHAPER_BINDING_TYPE_NETDEV,
NET_SHAPER_BINDING_TYPE_DEVLINK_PORT,
};
struct net_shaper_binding {
enum net_shaper_binding_type type;
union {
struct net_device *netdev;
struct devlink_port *devlink_port;
};
};
struct net_shaper_ops {
+ /**
+ * @group: create the specified shapers scheduling group
+ *
+ * Nest the @leaves shapers identified by @leaves_handles under the
+ * @root shaper identified by @root_handle. All the shapers belong
+ * to the network device @dev. The @leaves and @leaves_handles shaper
+ * arrays size is specified by @leaves_count.
+ * Create either the @leaves and the @root shaper; or if they already
+ * exists, links them together in the desired way.
+ * @leaves scope must be NET_SHAPER_SCOPE_QUEUE.
+ *
+ * Returns 0 on group successfully created, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*group)(const struct net_shaper_binding *binding, int leaves_count,
+ const struct net_shaper_handle *leaves_handles,
+ const struct net_shaper_info *leaves,
+ const struct net_shaper_handle *root_handle,
+ const struct net_shaper_info *root,
+ struct netlink_ext_ack *extack);
+
+ /**
+ * @set: Updates the specified shaper
+ *
+ * Updates or creates the @shaper identified by the provided @handle
+ * on the given device @dev.
+ *
+ * Returns 0 on success, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*set)(const struct net_shaper_binding *binding,
+ const struct net_shaper_handle *handle,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack);
+
+ /**
+ * @delete: Removes the specified shaper from the NIC
+ *
+ * Removes the shaper configuration as identified by the given @handle
+ * on the specified device @dev, restoring the default behavior.
+ *
+ * Returns 0 on success, otherwise an negative
+ * error value and set @extack to describe the failure's reason.
+ */
+ int (*delete)(const struct net_shaper_binding *binding,
+ const struct net_shaper_handle *handle,
+ struct netlink_ext_ack *extack);
+};
static inline struct net_device *
net_shaper_binding_netdev(struct net_shaper_binding *binding)
{
WARN_ON(binding->type != NET_SHAPER_BINDING_TYPE_NETDEV)
return binding->netdev;
}
static inline struct devlink_port *
net_shaper_binding_devlink_port(struct net_shaper_binding *binding)
{
WARN_ON(binding->type != NET_SHAPER_BINDING_TYPE_DEVLINK_PORT)
return binding->devlink_port;
}
Then whoever calls the op fills-up the binding structure accordingly.
drivers can implement ops, for netdev-bound shaper like this:
static int driverx_shaper_set(const struct net_shaper_binding *binding,
const struct net_shaper_handle *handle,
const struct net_shaper_info *shaper,
struct netlink_ext_ack *extack);
{
struct net_device *netdev = net_shaper_binding_netdev(binding);
......
}
struct net_shaper_ops driverx_shaper_ops {
.set = driverx_shaper_set;
......
};
static const struct net_device_ops driverx_netdev_ops = {
.net_shaper_ops = &driverx_shaper_ops,
......
};
drivers can implement ops, for devlink_port-bound shaper like this:
static int drivery_shaper_set(const struct net_shaper_binding *binding,
const struct net_shaper_handle *handle,
const struct net_shaper_info *shaper,
struct netlink_ext_ack *extack);
{
struct devlink_port *devlink_port = net_shaper_binding_devlink_port(binding);
......
}
struct net_shaper_ops drivery_shaper_ops {
.set = drivery_shaper_set;
......
};
static const struct devlink_port_ops drivery_devlink_port_ops = {
.port_shaper_ops = &drivery_shaper_ops,
};
Some driver can even have one ops implementation for both,
and distinguish just by looking at binding->type.
>>
>> Still for the record, I interpret the current proposal as not clashing
>> with Jakub's preference, and being tolerated from Jiri, again please
>> correct me if I read too far.
>
>One more thing, Jiri said:
>
> If you do everything netdev-centric from start, I'm sure no shaper
> consolidation will ever happen. And that I thought was one of the goals.
>
>Consolidation was indeed one of the goals, and I share Jiri's concern :(
Good.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-23 11:50 ` Jiri Pirko
@ 2024-08-23 12:58 ` Paolo Abeni
2024-08-23 13:36 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-23 12:58 UTC (permalink / raw)
To: Jiri Pirko, Jakub Kicinski
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On 8/23/24 13:50, Jiri Pirko wrote:
> Fri, Aug 23, 2024 at 12:56:08AM CEST, kuba@kernel.org wrote:
>> On Thu, 22 Aug 2024 22:30:35 +0200 Paolo Abeni wrote:
>>>>> I'm not saying this is deal breaker for me. I just think that if the api
>>>>> is designed to be independent of the object shaper is bound to
>>>>> (netdev/devlink_port/etc), it would be much much easier to extend in the
>>>>> future. If you do everything netdev-centric from start, I'm sure no
>>>>> shaper consolidation will ever happen. And that I thought was one of the
>>>>> goals.
>>>>>
>>>>> Perhaps Jakub has opinion.
>>>>
>>>> I think you and I are on the same page :) Other than the "reference
>>>> object" (netdev / devlink port) the driver facing API should be
>>>> identical. Making it possible for the same driver code to handle
>>>> translating the parameters into HW config / FW requests, whether
>>>> they shape at the device (devlink) or port (netdev) level.
>>>>
>>>> Shaper NL for netdevs is separate from internal representation and
>>>> driver API in my mind. My initial ask was to create the internal
>>>> representation first, make sure it can express devlink and handful of
>>>> exiting netdev APIs, and only once that's merged worry about exposing
>>>> it via a new NL.
>>>>
>>>> I'm not opposed to showing devlink shapers in netdev NL (RO as you say)
>>>> but talking about it now strikes me as cart before the horse.
>>>
>>> FTR, I don't see both of you on the same page ?!?
>>>
>>> I read the above as Jiri's preference is a single ndo set to control
>
> "Ndo" stands for netdev op and they are all tightly coupled with
> netdevices. So, "single ndo set to control both devlink and netdev
> shapers" sounds like nonsense to me.
In this context, "NDOs" == set of function pointers operating on the
same object.
>>> Or to phrase the above differently, Jiri is focusing on the shaper
>>> "binding" (how to locate/access it) while Jakub is focusing on the
>>> shaper "info" (content/definition/attributes). Please correct me If I
>>> misread something.
>
> Two(or more) similar ops structs looks odd to me. I think that the ops
> should should be shared and just the "binding point" should be somehow
> abstracted out. Code speaks, let me draft how it could be done:
>
> enum net_shaper_binding_type {
> NET_SHAPER_BINDING_TYPE_NETDEV,
> NET_SHAPER_BINDING_TYPE_DEVLINK_PORT,
> };
>
> struct net_shaper_binding {
> enum net_shaper_binding_type type;
> union {
> struct net_device *netdev;
> struct devlink_port *devlink_port;
> };
> };
>
> struct net_shaper_ops {
> + /**
> + * @group: create the specified shapers scheduling group
> + *
> + * Nest the @leaves shapers identified by @leaves_handles under the
> + * @root shaper identified by @root_handle. All the shapers belong
> + * to the network device @dev. The @leaves and @leaves_handles shaper
> + * arrays size is specified by @leaves_count.
> + * Create either the @leaves and the @root shaper; or if they already
> + * exists, links them together in the desired way.
> + * @leaves scope must be NET_SHAPER_SCOPE_QUEUE.
> + *
> + * Returns 0 on group successfully created, otherwise an negative
> + * error value and set @extack to describe the failure's reason.
> + */
> + int (*group)(const struct net_shaper_binding *binding, int leaves_count,
> + const struct net_shaper_handle *leaves_handles,
> + const struct net_shaper_info *leaves,
> + const struct net_shaper_handle *root_handle,
> + const struct net_shaper_info *root,
> + struct netlink_ext_ack *extack);
> +
> + /**
> + * @set: Updates the specified shaper
> + *
> + * Updates or creates the @shaper identified by the provided @handle
> + * on the given device @dev.
> + *
> + * Returns 0 on success, otherwise an negative
> + * error value and set @extack to describe the failure's reason.
> + */
> + int (*set)(const struct net_shaper_binding *binding,
> + const struct net_shaper_handle *handle,
> + const struct net_shaper_info *shaper,
> + struct netlink_ext_ack *extack);
> +
> + /**
> + * @delete: Removes the specified shaper from the NIC
> + *
> + * Removes the shaper configuration as identified by the given @handle
> + * on the specified device @dev, restoring the default behavior.
> + *
> + * Returns 0 on success, otherwise an negative
> + * error value and set @extack to describe the failure's reason.
> + */
> + int (*delete)(const struct net_shaper_binding *binding,
> + const struct net_shaper_handle *handle,
> + struct netlink_ext_ack *extack);
> +};
>
>
> static inline struct net_device *
> net_shaper_binding_netdev(struct net_shaper_binding *binding)
> {
> WARN_ON(binding->type != NET_SHAPER_BINDING_TYPE_NETDEV)
> return binding->netdev;
> }
>
> static inline struct devlink_port *
> net_shaper_binding_devlink_port(struct net_shaper_binding *binding)
> {
> WARN_ON(binding->type != NET_SHAPER_BINDING_TYPE_DEVLINK_PORT)
> return binding->devlink_port;
> }
>
> Then whoever calls the op fills-up the binding structure accordingly.
>
>
> drivers can implement ops, for netdev-bound shaper like this:
>
> static int driverx_shaper_set(const struct net_shaper_binding *binding,
> const struct net_shaper_handle *handle,
> const struct net_shaper_info *shaper,
> struct netlink_ext_ack *extack);
> {
> struct net_device *netdev = net_shaper_binding_netdev(binding);
> ......
> }
>
> struct net_shaper_ops driverx_shaper_ops {
> .set = driverx_shaper_set;
> ......
> };
>
> static const struct net_device_ops driverx_netdev_ops = {
> .net_shaper_ops = &driverx_shaper_ops,
> ......
> };
If I read correctly, the net_shaper_ops caller will have to discriminate
between net_device and devlink, do the object-type-specific lookup to
get the relevant net_device (or devlink) object and then pass to such
net_device (or devlink) a "generic" binding. Did I misread something? If
so I must admit I really dislike such interface.
I personally think it would be much cleaner to have 2 separate set of
operations, with exactly the same semantic and argument list, except for
the first argument (struct net_device or struct devlink).
The driver implementation could still de-duplicate a lot of code, as far
as the shaper-related arguments are the same.
Side note, if the intention is to allow the user to touch/modify the
queue-level and queue-group-level shapers via the devlink object? if
that is the intention, we will need to drop the shaper cache and
(re-)introduce a get() callback, as the same shaper could be reached via
multiple binding/handle pairs and the core will not know all of such
pairs for a given shaper.
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-23 12:58 ` Paolo Abeni
@ 2024-08-23 13:36 ` Jiri Pirko
2024-08-23 14:23 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-23 13:36 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jakub Kicinski, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Fri, Aug 23, 2024 at 02:58:27PM CEST, pabeni@redhat.com wrote:
>On 8/23/24 13:50, Jiri Pirko wrote:
>> Fri, Aug 23, 2024 at 12:56:08AM CEST, kuba@kernel.org wrote:
>> > On Thu, 22 Aug 2024 22:30:35 +0200 Paolo Abeni wrote:
>> > > > > I'm not saying this is deal breaker for me. I just think that if the api
>> > > > > is designed to be independent of the object shaper is bound to
>> > > > > (netdev/devlink_port/etc), it would be much much easier to extend in the
>> > > > > future. If you do everything netdev-centric from start, I'm sure no
>> > > > > shaper consolidation will ever happen. And that I thought was one of the
>> > > > > goals.
>> > > > >
>> > > > > Perhaps Jakub has opinion.
>> > > >
>> > > > I think you and I are on the same page :) Other than the "reference
>> > > > object" (netdev / devlink port) the driver facing API should be
>> > > > identical. Making it possible for the same driver code to handle
>> > > > translating the parameters into HW config / FW requests, whether
>> > > > they shape at the device (devlink) or port (netdev) level.
>> > > >
>> > > > Shaper NL for netdevs is separate from internal representation and
>> > > > driver API in my mind. My initial ask was to create the internal
>> > > > representation first, make sure it can express devlink and handful of
>> > > > exiting netdev APIs, and only once that's merged worry about exposing
>> > > > it via a new NL.
>> > > >
>> > > > I'm not opposed to showing devlink shapers in netdev NL (RO as you say)
>> > > > but talking about it now strikes me as cart before the horse.
>> > >
>> > > FTR, I don't see both of you on the same page ?!?
>> > >
>> > > I read the above as Jiri's preference is a single ndo set to control
>>
>> "Ndo" stands for netdev op and they are all tightly coupled with
>> netdevices. So, "single ndo set to control both devlink and netdev
>> shapers" sounds like nonsense to me.
>
>In this context, "NDOs" == set of function pointers operating on the same
>object.
>
>> > > Or to phrase the above differently, Jiri is focusing on the shaper
>> > > "binding" (how to locate/access it) while Jakub is focusing on the
>> > > shaper "info" (content/definition/attributes). Please correct me If I
>> > > misread something.
>>
>> Two(or more) similar ops structs looks odd to me. I think that the ops
>> should should be shared and just the "binding point" should be somehow
>> abstracted out. Code speaks, let me draft how it could be done:
>>
>> enum net_shaper_binding_type {
>> NET_SHAPER_BINDING_TYPE_NETDEV,
>> NET_SHAPER_BINDING_TYPE_DEVLINK_PORT,
>> };
>>
>> struct net_shaper_binding {
>> enum net_shaper_binding_type type;
>> union {
>> struct net_device *netdev;
>> struct devlink_port *devlink_port;
>> };
>> };
>>
>> struct net_shaper_ops {
>> + /**
>> + * @group: create the specified shapers scheduling group
>> + *
>> + * Nest the @leaves shapers identified by @leaves_handles under the
>> + * @root shaper identified by @root_handle. All the shapers belong
>> + * to the network device @dev. The @leaves and @leaves_handles shaper
>> + * arrays size is specified by @leaves_count.
>> + * Create either the @leaves and the @root shaper; or if they already
>> + * exists, links them together in the desired way.
>> + * @leaves scope must be NET_SHAPER_SCOPE_QUEUE.
>> + *
>> + * Returns 0 on group successfully created, otherwise an negative
>> + * error value and set @extack to describe the failure's reason.
>> + */
>> + int (*group)(const struct net_shaper_binding *binding, int leaves_count,
>> + const struct net_shaper_handle *leaves_handles,
>> + const struct net_shaper_info *leaves,
>> + const struct net_shaper_handle *root_handle,
>> + const struct net_shaper_info *root,
>> + struct netlink_ext_ack *extack);
>> +
>> + /**
>> + * @set: Updates the specified shaper
>> + *
>> + * Updates or creates the @shaper identified by the provided @handle
>> + * on the given device @dev.
>> + *
>> + * Returns 0 on success, otherwise an negative
>> + * error value and set @extack to describe the failure's reason.
>> + */
>> + int (*set)(const struct net_shaper_binding *binding,
>> + const struct net_shaper_handle *handle,
>> + const struct net_shaper_info *shaper,
>> + struct netlink_ext_ack *extack);
>> +
>> + /**
>> + * @delete: Removes the specified shaper from the NIC
>> + *
>> + * Removes the shaper configuration as identified by the given @handle
>> + * on the specified device @dev, restoring the default behavior.
>> + *
>> + * Returns 0 on success, otherwise an negative
>> + * error value and set @extack to describe the failure's reason.
>> + */
>> + int (*delete)(const struct net_shaper_binding *binding,
>> + const struct net_shaper_handle *handle,
>> + struct netlink_ext_ack *extack);
>> +};
>>
>>
>> static inline struct net_device *
>> net_shaper_binding_netdev(struct net_shaper_binding *binding)
>> {
>> WARN_ON(binding->type != NET_SHAPER_BINDING_TYPE_NETDEV)
>> return binding->netdev;
>> }
>>
>> static inline struct devlink_port *
>> net_shaper_binding_devlink_port(struct net_shaper_binding *binding)
>> {
>> WARN_ON(binding->type != NET_SHAPER_BINDING_TYPE_DEVLINK_PORT)
>> return binding->devlink_port;
>> }
>>
>> Then whoever calls the op fills-up the binding structure accordingly.
>>
>>
>> drivers can implement ops, for netdev-bound shaper like this:
>>
>> static int driverx_shaper_set(const struct net_shaper_binding *binding,
>> const struct net_shaper_handle *handle,
>> const struct net_shaper_info *shaper,
>> struct netlink_ext_ack *extack);
>> {
>> struct net_device *netdev = net_shaper_binding_netdev(binding);
>> ......
>> }
>>
>> struct net_shaper_ops driverx_shaper_ops {
>> .set = driverx_shaper_set;
>> ......
>> };
>>
>> static const struct net_device_ops driverx_netdev_ops = {
>> .net_shaper_ops = &driverx_shaper_ops,
>> ......
>> };
>
>If I read correctly, the net_shaper_ops caller will have to discriminate
>between net_device and devlink, do the object-type-specific lookup to get the
>relevant net_device (or devlink) object and then pass to such net_device (or
>devlink) a "generic" binding. Did I misread something? If so I must admit I
>really dislike such interface.
You are right. For example devlink rate set command handler will call
the op with devlink_port binding set.
>
>I personally think it would be much cleaner to have 2 separate set of
>operations, with exactly the same semantic and argument list, except for the
>first argument (struct net_device or struct devlink).
I think it is totally subjective. You like something, I like something
else. Both works. The amount of duplicity and need to change same
things on multiple places in case of bugfixes and extensions is what I
dislike on the 2 separate sets. Plus, there might be another binding in
the future, will you copy the ops struct again then?
>
>The driver implementation could still de-duplicate a lot of code, as far as
>the shaper-related arguments are the same.
>
>Side note, if the intention is to allow the user to touch/modify the
>queue-level and queue-group-level shapers via the devlink object? if that is
>the intention, we will need to drop the shaper cache and (re-)introduce a
>get() callback, as the same shaper could be reached via multiple
>binding/handle pairs and the core will not know all of such pairs for a given
>shaper.
That is a good question, I don't know. But gut feeling is "no".
>
>Thanks,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-23 13:36 ` Jiri Pirko
@ 2024-08-23 14:23 ` Paolo Abeni
2024-08-26 9:31 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-23 14:23 UTC (permalink / raw)
To: Jiri Pirko
Cc: Jakub Kicinski, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/23/24 15:36, Jiri Pirko wrote:
> Fri, Aug 23, 2024 at 02:58:27PM CEST, pabeni@redhat.com wrote:
>> I personally think it would be much cleaner to have 2 separate set of
>> operations, with exactly the same semantic and argument list, except for the
>> first argument (struct net_device or struct devlink).
>
> I think it is totally subjective. You like something, I like something
> else. Both works. The amount of duplicity and need to change same
> things on multiple places in case of bugfixes and extensions is what I
> dislike on the 2 separate sets.
My guestimate is that the amount of deltas caused by bugfixes and
extensions will be much different in practice with the two approaches.
I guess that even with the net_shaper_ops between devlink and
net_device, there will be different callbacks implementation for devlink
and net_device, right?
If so, the differentiated operation list between devlink and net_device
will trade a:
{
struct {net_device, netlink} =
net_shaper_binding_{netdevice_netlink}(binding);
preamble in every callback of every driver for a single additional
operations set definition.
It will at least scale better with the number of driver implementing the
interface.
> Plus, there might be another binding in
> the future, will you copy the ops struct again then?
Yes. Same reasons of the above.
>> The driver implementation could still de-duplicate a lot of code, as far as
>> the shaper-related arguments are the same.
>>
>> Side note, if the intention is to allow the user to touch/modify the
>> queue-level and queue-group-level shapers via the devlink object? if that is
>> the intention, we will need to drop the shaper cache and (re-)introduce a
>> get() callback, as the same shaper could be reached via multiple
>> binding/handle pairs and the core will not know all of such pairs for a given
>> shaper.
>
> That is a good question, I don't know. But gut feeling is "no".
Well, at least that is not in the direction of unlimited amount of
additional time and pain ;)
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-23 14:23 ` Paolo Abeni
@ 2024-08-26 9:31 ` Jiri Pirko
2024-08-27 14:37 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-26 9:31 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jakub Kicinski, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Fri, Aug 23, 2024 at 04:23:30PM CEST, pabeni@redhat.com wrote:
>On 8/23/24 15:36, Jiri Pirko wrote:
>> Fri, Aug 23, 2024 at 02:58:27PM CEST, pabeni@redhat.com wrote:
>> > I personally think it would be much cleaner to have 2 separate set of
>> > operations, with exactly the same semantic and argument list, except for the
>> > first argument (struct net_device or struct devlink).
>>
>> I think it is totally subjective. You like something, I like something
>> else. Both works. The amount of duplicity and need to change same
>> things on multiple places in case of bugfixes and extensions is what I
>> dislike on the 2 separate sets.
>
>My guestimate is that the amount of deltas caused by bugfixes and extensions
>will be much different in practice with the two approaches.
>
>I guess that even with the net_shaper_ops between devlink and net_device,
>there will be different callbacks implementation for devlink and net_device,
>right?
>
>If so, the differentiated operation list between devlink and net_device will
>trade a:
>
>{
> struct {net_device, netlink} =
>net_shaper_binding_{netdevice_netlink}(binding);
>
>preamble in every callback of every driver for a single additional operations
>set definition.
So?
>
>It will at least scale better with the number of driver implementing the
>interface.
>
>> Plus, there might be another binding in
>> the future, will you copy the ops struct again then?
>
>Yes. Same reasons of the above.
What's stopping anyone from diverging these 2-n sets? I mean, the whole
purpose it unification and finding common ground. Once you have ops
duplicated, sooner then later someone does change in A but ignore B.
Having the "preamble" in every callback seems like very good tradeoff
to prevent this scenario.
>
>> > The driver implementation could still de-duplicate a lot of code, as far as
>> > the shaper-related arguments are the same.
>> >
>> > Side note, if the intention is to allow the user to touch/modify the
>> > queue-level and queue-group-level shapers via the devlink object? if that is
>> > the intention, we will need to drop the shaper cache and (re-)introduce a
>> > get() callback, as the same shaper could be reached via multiple
>> > binding/handle pairs and the core will not know all of such pairs for a given
>> > shaper.
>>
>> That is a good question, I don't know. But gut feeling is "no".
>
>Well, at least that is not in the direction of unlimited amount of additional
>time and pain ;)
>
>Thanks,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-26 9:31 ` Jiri Pirko
@ 2024-08-27 14:37 ` Paolo Abeni
2024-08-27 14:54 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-27 14:37 UTC (permalink / raw)
To: Jiri Pirko, Jakub Kicinski
Cc: netdev, Madhu Chittim, Sridhar Samudrala, Simon Horman,
John Fastabend, Sunil Kovvuri Goutham, Jamal Hadi Salim
On 8/26/24 11:31, Jiri Pirko wrote:
> Fri, Aug 23, 2024 at 04:23:30PM CEST, pabeni@redhat.com wrote:
>> On 8/23/24 15:36, Jiri Pirko wrote:
>>> Fri, Aug 23, 2024 at 02:58:27PM CEST, pabeni@redhat.com wrote:
>>>> I personally think it would be much cleaner to have 2 separate set of
>>>> operations, with exactly the same semantic and argument list, except for the
>>>> first argument (struct net_device or struct devlink).
>>>
>>> I think it is totally subjective. You like something, I like something
>>> else. Both works. The amount of duplicity and need to change same
>>> things on multiple places in case of bugfixes and extensions is what I
>>> dislike on the 2 separate sets.
>>
>> My guestimate is that the amount of deltas caused by bugfixes and extensions
>> will be much different in practice with the two approaches.
>>
>> I guess that even with the net_shaper_ops between devlink and net_device,
>> there will be different callbacks implementation for devlink and net_device,
>> right?
>>
>> If so, the differentiated operation list between devlink and net_device will
>> trade a:
>>
>> {
>> struct {net_device, netlink} =
>> net_shaper_binding_{netdevice_netlink}(binding);
>>
>> preamble in every callback of every driver for a single additional operations
>> set definition.
>
> So?
The amount of code we would need to change in case of core changes would
probably be similar with either the differentiated operations list or not.
>> It will at least scale better with the number of driver implementing the
>> interface.
>>
>>> Plus, there might be another binding in
>>> the future, will you copy the ops struct again then?
>>
>> Yes. Same reasons of the above.
>
> What's stopping anyone from diverging these 2-n sets? I mean, the whole
> purpose it unification and finding common ground. Once you have ops
> duplicated, sooner then later someone does change in A but ignore B.
> Having the "preamble" in every callback seems like very good tradeoff
> to prevent this scenario.
The main fact is that we do not agree on the above point - unify the
shaper_ops between struct net_device and struct devlink.
I think a 3rd party opinion could help moving forward.
@Jakub could you please share your view here?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-27 14:37 ` Paolo Abeni
@ 2024-08-27 14:54 ` Jakub Kicinski
2024-08-27 20:43 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-27 14:54 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Tue, 27 Aug 2024 16:37:38 +0200 Paolo Abeni wrote:
> > What's stopping anyone from diverging these 2-n sets? I mean, the whole
> > purpose it unification and finding common ground. Once you have ops
> > duplicated, sooner then later someone does change in A but ignore B.
> > Having the "preamble" in every callback seems like very good tradeoff
> > to prevent this scenario.
>
> The main fact is that we do not agree on the above point - unify the
> shaper_ops between struct net_device and struct devlink.
>
> I think a 3rd party opinion could help moving forward.
> @Jakub could you please share your view here?
I don't mind Jiri's suggestion. Driver can declare its own helper:
static struct drv_port *
drv_shaper_binding_to_prot(const struct net_shaper_binding *binding)
{
if (binding->type == NET_SHAPER_BINDING_TYPE_NETDEV)
return /* netdev_priv() ? */;
if (binding->type == NET_SHAPER_BINDING_TYPE_DEVLINK_PORT)
return /* container_of() ? */;
WARN_ONCE();
return NULL;
}
And call that instead of netdev_priv()?
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-27 14:54 ` Jakub Kicinski
@ 2024-08-27 20:43 ` Paolo Abeni
2024-08-27 21:03 ` Jakub Kicinski
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-27 20:43 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/27/24 16:54, Jakub Kicinski wrote:
> On Tue, 27 Aug 2024 16:37:38 +0200 Paolo Abeni wrote:
>>> What's stopping anyone from diverging these 2-n sets? I mean, the whole
>>> purpose it unification and finding common ground. Once you have ops
>>> duplicated, sooner then later someone does change in A but ignore B.
>>> Having the "preamble" in every callback seems like very good tradeoff
>>> to prevent this scenario.
>>
>> The main fact is that we do not agree on the above point - unify the
>> shaper_ops between struct net_device and struct devlink.
>>
>> I think a 3rd party opinion could help moving forward.
>> @Jakub could you please share your view here?
>
> I don't mind Jiri's suggestion. Driver can declare its own helper:
>
> static struct drv_port *
> drv_shaper_binding_to_prot(const struct net_shaper_binding *binding)
> {
> if (binding->type == NET_SHAPER_BINDING_TYPE_NETDEV)
> return /* netdev_priv() ? */;
> if (binding->type == NET_SHAPER_BINDING_TYPE_DEVLINK_PORT)
> return /* container_of() ? */;
> WARN_ONCE();
> return NULL;
> }
>
> And call that instead of netdev_priv()?
As I wrote this does not look like something that would help
de-deuplicate any code, but since you both seem to agree...
Double checking before I rewrote a significant amount of the core code:
In the NL API, I will replace ifindex with binding, the latter will
include nested attributes ifindex, bus_name
and dev_name.
Note that 'struct net_shaper_binding' must include a ‘struct devlink
*’ as opposed to a
‘struct devlink_port *’, as mentioned in the code snippet so far, or
we will have to drop the
shaper cache and re-introduce a get() operation.
The ops will try to fetch the net_device or devlink according to the
provided attributes.
At the moment the ops will error out with ENOTSUP for netlink object.
Most core helpers
will be re-factored to accept a 'binding *' argument in place of a
'struct net_device *'.
Full netlink support (for the brave that will implement it later) will
likely require at least an
additional scope (devlink_port), include the net_shaper_ops and
net_shaper_data inside
struct devlink, and likely code adaptation inside the core.
/P
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-27 20:43 ` Paolo Abeni
@ 2024-08-27 21:03 ` Jakub Kicinski
2024-08-27 21:54 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-27 21:03 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Tue, 27 Aug 2024 22:43:09 +0200 Paolo Abeni wrote:
> >> The main fact is that we do not agree on the above point - unify the
> >> shaper_ops between struct net_device and struct devlink.
> >>
> >> I think a 3rd party opinion could help moving forward.
> >> @Jakub could you please share your view here?
> >
> > I don't mind Jiri's suggestion. Driver can declare its own helper:
> >
> > static struct drv_port *
> > drv_shaper_binding_to_port(const struct net_shaper_binding *binding)
> > {
> > if (binding->type == NET_SHAPER_BINDING_TYPE_NETDEV)
> > return /* netdev_priv() ? */;
> > if (binding->type == NET_SHAPER_BINDING_TYPE_DEVLINK_PORT)
> > return /* container_of() ? */;
> > WARN_ONCE();
> > return NULL;
> > }
> >
> > And call that instead of netdev_priv()?
>
> As I wrote this does not look like something that would help
> de-deuplicate any code, but since you both seem to agree...
To be clear. Given the helper above you can replace:
struct iavf_adapter *adapter = netdev_priv(dev);
with:
struct iavf_adapter *adapter = iavf_from_shaper_handle(handle);
In all the callbacks, and callbacks can now take devlink or other
"handles".
> Double checking before I rewrote a significant amount of the core code:
>
> In the NL API, I will replace ifindex with binding, the latter will
> include nested attributes ifindex, bus_name
> and dev_name.
Any mention of the netlink API makes me worried that the "internal
representation should be separate from the uAPI" point is not
agreed on or at least not understood :( The NL API does not need other
object types. And there's currently no uAPI gap for devlink, AFAIK.
So the conversation about "handles" if quite forward-looking.
> Note that 'struct net_shaper_binding' must include a ‘struct devlink
> *’ as opposed to a
> ‘struct devlink_port *’, as mentioned in the code snippet so far, or
> we will have to drop the
> shaper cache and re-introduce a get() operation.
>
> The ops will try to fetch the net_device or devlink according to the
> provided attributes.
> At the moment the ops will error out with ENOTSUP for netlink object.
No need to error out. Driver must not receive calls with object types
they don't support (we can add capabilities for this, later on, and
core should check, or do what Jiri suggests and just hook the ops into
different structs).
> Most core helpers
> will be re-factored to accept a 'binding *' argument in place of a
> 'struct net_device *'.
>
> Full netlink support (for the brave that will implement it later) will
> likely require at least an
> additional scope (devlink_port), include the net_shaper_ops and
> net_shaper_data inside
> struct devlink, and likely code adaptation inside the core.
Back to uAPI worries. FWIW I think that how we pass the handle to
drivers is of relatively little importance. It's small amount of
mechanical work to change it later. But you asked me about the proposal
so I answered. To me representing the other shaper APIs in terms of
the new driver facing API *within netdev* is more important.
Literally the only change I would have expected based on this branch of
the conversation is slight adjustment to the parameters to ops. Nothing
else. No netlink changes. Only core change would be to wrap the ops.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-27 21:03 ` Jakub Kicinski
@ 2024-08-27 21:54 ` Paolo Abeni
2024-08-28 6:40 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-27 21:54 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Tue, Aug 27, 2024 at 11:10 PM Jakub Kicinski <kuba@kernel.org> wrote:
> Literally the only change I would have expected based on this branch of
> the conversation is slight adjustment to the parameters to ops. Nothing
> else. No netlink changes. Only core change would be to wrap the ops.
FTR, the above is quite the opposite of my understanding of the whole
conversation so far. If the whole delta under discussion is just that,
we could make such a change at any given time (and I would strongly
support the idea to make it only when the devlink bits will be ready)
and its presence (or absence) will not block the net_device/devlink
shaper callback consolidation in any way.
I thought Jiri intended the whole core bits to be adapted to handle
'binding' instead of net_device, to allow a more seamless plug of
devlink.
@Jiri: did I misread your words?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-27 21:54 ` Paolo Abeni
@ 2024-08-28 6:40 ` Jiri Pirko
2024-08-28 10:55 ` Paolo Abeni
0 siblings, 1 reply; 91+ messages in thread
From: Jiri Pirko @ 2024-08-28 6:40 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jakub Kicinski, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Tue, Aug 27, 2024 at 11:54:48PM CEST, pabeni@redhat.com wrote:
>On Tue, Aug 27, 2024 at 11:10 PM Jakub Kicinski <kuba@kernel.org> wrote:
>> Literally the only change I would have expected based on this branch of
>> the conversation is slight adjustment to the parameters to ops. Nothing
>> else. No netlink changes. Only core change would be to wrap the ops.
>
>FTR, the above is quite the opposite of my understanding of the whole
>conversation so far. If the whole delta under discussion is just that,
>we could make such a change at any given time (and I would strongly
>support the idea to make it only when the devlink bits will be ready)
>and its presence (or absence) will not block the net_device/devlink
>shaper callback consolidation in any way.
>
>I thought Jiri intended the whole core bits to be adapted to handle
>'binding' instead of net_device, to allow a more seamless plug of
>devlink.
>@Jiri: did I misread your words?
Yep that is correct. But you don't need the UAPI extension for that, as
UAPI for devlink rate already exists.
Regarding your shaper UAPI. I think that you just need to make sure it
is easily extendable by another binding in the future. The current
ifindex binding is fine for your shaper UAPI. Just have it in nest and
allow to extend by another attr later on.
Code speaks:
enum net_shaper_a_binding {
NET_SHAPER_A_BINDING_IFINDEX = 1;
};
enum {
NET_SHAPER_A_BINDING = 1, /*nested enum net_shaper_a_binding*/
NET_SHAPER_A_.....
..................
..................
};
Later on we can easily extend the UAPI by:
enum net_shaper_a_binding {
NET_SHAPER_A_BINDING_IFINDEX = 1;
NET_SHAPER_A_BINDING_DEVLINK_PORT;
}
Makes sense?
>
>Thanks,
>
>Paolo
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-28 6:40 ` Jiri Pirko
@ 2024-08-28 10:55 ` Paolo Abeni
2024-08-28 13:02 ` Jiri Pirko
2024-08-28 20:30 ` Jakub Kicinski
0 siblings, 2 replies; 91+ messages in thread
From: Paolo Abeni @ 2024-08-28 10:55 UTC (permalink / raw)
To: Jiri Pirko
Cc: Jakub Kicinski, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/28/24 08:40, Jiri Pirko wrote:
> Makes sense?
Almost! Tacking aside the (very significant) differences between your
proposition and Jakub’s, we can't use devlink port here, just devlink,
or we will have to drop the cache too[1]. Specific devlink port shapers
will be reached via different handles (scope/id).
Additionally, I think we don't need strictly the ‘binding’ nested
attribute to extend the NL API with different binding objects (devlink),
we could append the new attributes needed to support (identify) devlink
at the end of the net shaper attributes list. I agree that would be
likely less ‘nice’.
What about:
- Refactor the core and the driver api to support the ‘binding’ thing
- Update the NL definition to nest the ‘ifindex’ attribute under the
‘binding’ one. No mention/reference to devlink yet, so most of the
documentation will be unchanged.
- devlink support will not be included, but there should be enough
ground paved for it.
?
Thanks,
Paolo
[1] the cache container belongs to the ‘entry point’ inside the shaper
hierarchy - i.e. currently, the struct net_device. If we add a
devlink_port ‘entry point’, the cache there will have to manage even the
shaper for devlink ports group. When accessing a group containing
multiple ports, we will get multiple inconsistent cache values.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-28 10:55 ` Paolo Abeni
@ 2024-08-28 13:02 ` Jiri Pirko
2024-08-28 20:30 ` Jakub Kicinski
1 sibling, 0 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-28 13:02 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jakub Kicinski, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Wed, Aug 28, 2024 at 12:55:31PM CEST, pabeni@redhat.com wrote:
>On 8/28/24 08:40, Jiri Pirko wrote:
>> Makes sense?
>
>Almost! Tacking aside the (very significant) differences between your
>proposition and Jakub’s, we can't use devlink port here, just devlink, or we
>will have to drop the cache too[1]. Specific devlink port shapers will be
>reached via different handles (scope/id).
Ok, I guess. Need to see the code.
>
>Additionally, I think we don't need strictly the ‘binding’ nested attribute
>to extend the NL API with different binding objects (devlink), we could
>append the new attributes needed to support (identify) devlink at the end of
>the net shaper attributes list. I agree that would be likely less ‘nice’.
True and true.
>
>What about:
>- Refactor the core and the driver api to support the ‘binding’ thing
Ack.
>- Update the NL definition to nest the ‘ifindex’ attribute under the
>‘binding’ one. No mention/reference to devlink yet, so most of the
>documentation will be unchanged.
Ack.
>- devlink support will not be included, but there should be enough ground
>paved for it.
Ack.
>
>?
Thanks!
>
>Thanks,
>
>Paolo
>
>[1] the cache container belongs to the ‘entry point’ inside the shaper
>hierarchy - i.e. currently, the struct net_device. If we add a devlink_port
>‘entry point’, the cache there will have to manage even the shaper for
>devlink ports group. When accessing a group containing multiple ports, we
>will get multiple inconsistent cache values.
>
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-28 10:55 ` Paolo Abeni
2024-08-28 13:02 ` Jiri Pirko
@ 2024-08-28 20:30 ` Jakub Kicinski
2024-08-28 21:13 ` Paolo Abeni
1 sibling, 1 reply; 91+ messages in thread
From: Jakub Kicinski @ 2024-08-28 20:30 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On Wed, 28 Aug 2024 12:55:31 +0200 Paolo Abeni wrote:
> - Update the NL definition to nest the ‘ifindex’ attribute under the
> ‘binding’ one. No mention/reference to devlink yet, so most of the
> documentation will be unchanged.
Sorry but I think that's a bad idea. Nesting attributes in netlink
with no semantic implications, just to "organize" attributes which
are somehow related just complicates the code and wastes space.
Netlink is not JSON.
Especially in this case, where we would do it for future uAPI extension
which I really hope we won't need, since (1) devlink already has one,
(2) the point of this API is to reduce the uAPI surface, not extend it,
(3) user requirements for devlink and netdev config are different.
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-28 20:30 ` Jakub Kicinski
@ 2024-08-28 21:13 ` Paolo Abeni
2024-08-29 11:45 ` Jiri Pirko
0 siblings, 1 reply; 91+ messages in thread
From: Paolo Abeni @ 2024-08-28 21:13 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Jiri Pirko, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
On 8/28/24 22:30, Jakub Kicinski wrote:
> On Wed, 28 Aug 2024 12:55:31 +0200 Paolo Abeni wrote:
>> - Update the NL definition to nest the ‘ifindex’ attribute under the
>> ‘binding’ one. No mention/reference to devlink yet, so most of the
>> documentation will be unchanged.
>
> Sorry but I think that's a bad idea. Nesting attributes in netlink
> with no semantic implications, just to "organize" attributes which
> are somehow related just complicates the code and wastes space.
> Netlink is not JSON.
>
> Especially in this case, where we would do it for future uAPI extension
> which I really hope we won't need, since (1) devlink already has one,
> (2) the point of this API is to reduce the uAPI surface, not extend it,
> (3) user requirements for devlink and netdev config are different.
FTR I was no more than 60" from posting the new revision including that
when I read that. I hope Jiri would agree...
Well, I guess I have to roll-back a lot of changes... Not sure if I'll
be able to post tomorrow evening my time...
/P
^ permalink raw reply [flat|nested] 91+ messages in thread
* Re: [PATCH v3 03/12] net-shapers: implement NL get operation
2024-08-28 21:13 ` Paolo Abeni
@ 2024-08-29 11:45 ` Jiri Pirko
0 siblings, 0 replies; 91+ messages in thread
From: Jiri Pirko @ 2024-08-29 11:45 UTC (permalink / raw)
To: Paolo Abeni
Cc: Jakub Kicinski, netdev, Madhu Chittim, Sridhar Samudrala,
Simon Horman, John Fastabend, Sunil Kovvuri Goutham,
Jamal Hadi Salim
Wed, Aug 28, 2024 at 11:13:04PM CEST, pabeni@redhat.com wrote:
>
>
>On 8/28/24 22:30, Jakub Kicinski wrote:
>> On Wed, 28 Aug 2024 12:55:31 +0200 Paolo Abeni wrote:
>> > - Update the NL definition to nest the ‘ifindex’ attribute under the
>> > ‘binding’ one. No mention/reference to devlink yet, so most of the
>> > documentation will be unchanged.
>>
>> Sorry but I think that's a bad idea. Nesting attributes in netlink
>> with no semantic implications, just to "organize" attributes which
>> are somehow related just complicates the code and wastes space.
>> Netlink is not JSON.
>>
>> Especially in this case, where we would do it for future uAPI extension
>> which I really hope we won't need, since (1) devlink already has one,
>> (2) the point of this API is to reduce the uAPI surface, not extend it,
>> (3) user requirements for devlink and netdev config are different.
>
>FTR I was no more than 60" from posting the new revision including that when
>I read that. I hope Jiri would agree...
Sure.
>
>Well, I guess I have to roll-back a lot of changes... Not sure if I'll be
>able to post tomorrow evening my time...
>
>/P
>
>
^ permalink raw reply [flat|nested] 91+ messages in thread
end of thread, other threads:[~2024-08-29 11:45 UTC | newest]
Thread overview: 91+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-30 20:39 [PATCH v3 00/12] net: introduce TX H/W shaping API Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 01/12] tools: ynl: lift an assumption about spec file name Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 02/12] netlink: spec: add shaper YAML spec Paolo Abeni
2024-07-31 21:13 ` Donald Hunter
2024-08-01 14:31 ` Paolo Abeni
2024-08-02 10:57 ` Jiri Pirko
2024-08-02 11:15 ` Donald Hunter
2024-08-05 14:35 ` Paolo Abeni
2024-08-05 20:37 ` Jakub Kicinski
2024-08-01 13:10 ` Jiri Pirko
2024-08-01 14:40 ` Jakub Kicinski
2024-08-01 15:12 ` Paolo Abeni
2024-08-02 10:49 ` Jiri Pirko
2024-08-05 15:11 ` Paolo Abeni
2024-08-06 7:06 ` Jiri Pirko
2024-08-12 14:58 ` Paolo Abeni
2024-08-12 15:25 ` Jakub Kicinski
2024-08-12 16:50 ` Jiri Pirko
2024-08-12 17:42 ` Jakub Kicinski
2024-08-13 5:38 ` Jiri Pirko
2024-08-13 14:12 ` Jakub Kicinski
2024-08-13 14:47 ` Paolo Abeni
2024-08-13 14:58 ` Jakub Kicinski
2024-08-13 15:31 ` Paolo Abeni
2024-08-13 15:43 ` Jakub Kicinski
2024-08-14 8:56 ` Donald Hunter
2024-08-13 17:12 ` Donald Hunter
2024-08-14 14:21 ` Paolo Abeni
2024-08-15 9:07 ` Donald Hunter
2024-08-02 11:19 ` Jiri Pirko
2024-08-02 11:26 ` Jiri Pirko
2024-08-02 16:04 ` Jiri Pirko
2024-07-30 20:39 ` [PATCH v3 03/12] net-shapers: implement NL get operation Paolo Abeni
2024-08-01 13:42 ` Jiri Pirko
2024-08-13 15:17 ` Paolo Abeni
2024-08-14 8:37 ` Jiri Pirko
2024-08-16 8:52 ` Paolo Abeni
2024-08-16 9:16 ` Jiri Pirko
2024-08-19 9:33 ` Paolo Abeni
2024-08-19 11:53 ` Jiri Pirko
2024-08-19 16:52 ` Paolo Abeni
2024-08-22 12:02 ` Jiri Pirko
2024-08-22 14:41 ` Jakub Kicinski
2024-08-22 20:30 ` Paolo Abeni
2024-08-22 22:56 ` Jakub Kicinski
2024-08-23 11:50 ` Jiri Pirko
2024-08-23 12:58 ` Paolo Abeni
2024-08-23 13:36 ` Jiri Pirko
2024-08-23 14:23 ` Paolo Abeni
2024-08-26 9:31 ` Jiri Pirko
2024-08-27 14:37 ` Paolo Abeni
2024-08-27 14:54 ` Jakub Kicinski
2024-08-27 20:43 ` Paolo Abeni
2024-08-27 21:03 ` Jakub Kicinski
2024-08-27 21:54 ` Paolo Abeni
2024-08-28 6:40 ` Jiri Pirko
2024-08-28 10:55 ` Paolo Abeni
2024-08-28 13:02 ` Jiri Pirko
2024-08-28 20:30 ` Jakub Kicinski
2024-08-28 21:13 ` Paolo Abeni
2024-08-29 11:45 ` Jiri Pirko
2024-08-01 15:09 ` Jakub Kicinski
2024-08-02 11:53 ` Jiri Pirko
2024-07-30 20:39 ` [PATCH v3 04/12] net-shapers: implement NL set and delete operations Paolo Abeni
2024-08-01 15:00 ` Jakub Kicinski
2024-08-01 15:25 ` Paolo Abeni
2024-08-01 15:39 ` Jakub Kicinski
2024-08-02 16:15 ` Jiri Pirko
2024-08-02 22:01 ` Jakub Kicinski
2024-08-05 15:23 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 05/12] net-shapers: implement NL group operation Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 06/12] netlink: spec: add shaper introspection support Paolo Abeni
2024-08-02 11:21 ` Donald Hunter
2024-07-30 20:39 ` [PATCH v3 07/12] net: shaper: implement " Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 08/12] testing: net-drv: add basic shaper test Paolo Abeni
2024-07-31 7:52 ` Paolo Abeni
2024-08-01 1:55 ` Jakub Kicinski
2024-08-05 14:22 ` Simon Horman
2024-08-05 19:36 ` Jakub Kicinski
2024-08-06 15:21 ` Simon Horman
2024-08-08 12:20 ` Simon Horman
2024-08-08 14:17 ` Jakub Kicinski
2024-08-08 14:34 ` Simon Horman
2024-08-11 12:40 ` Simon Horman
2024-08-12 15:31 ` Jakub Kicinski
2024-08-12 16:03 ` Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 09/12] virtchnl: support queue rate limit and quanta size configuration Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 10/12] ice: Support VF " Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 11/12] iavf: Add net_shaper_ops support Paolo Abeni
2024-07-30 20:39 ` [PATCH v3 12/12] iavf: add support to exchange qos capabilities Paolo Abeni
2024-08-01 12:57 ` [PATCH v3 00/12] net: introduce TX H/W shaping API Jiri Pirko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).