[PATCH net 0/3] net: sched: Make qdisc offload uapi uniform

[PATCH net 0/3] net: sched: Make qdisc offload uapi uniform

[Yuval Mintz]

Several qdiscs can already be offloaded to hardware, but there's an
inconsistecy in regard to the uapi through which they indicate such
an offload is taking place - indication is passed to the user via
TCA_OPTIONS where each qdisc retains private logic for setting it.

The recent addition of offloading to RED in
602f3baf2218 ("net_sch: red: Add offload ability to RED qdisc") caused
the addition of yet another uapi field for this purpose -
TC_RED_OFFLOADED.

For clarity and prevention of bloat in the uapi we want to eliminate
said added uapi, replacing it with a common mechanism that can be used
to reflect offload status of the various qdiscs.

The first patch introduces TCA_HW_OFFLOAD as the generic message meant
for this purpose. The second changes the current RED implementation into
setting the internal bits necessary for passing it, and the third removes
TC_RED_OFFLOADED as its no longer needed.

Dave,

A bit unorthodox as it's not a fix per-se, but it's the last chance
for killing the unneeded uapi and replacing it with something better
before getting stuck with it forever.

Cheers,
Yuval

Yuval Mintz (3):
  net: sched: Add TCA_HW_OFFLOAD
  net: sched: Move to new offload indication in RED
  pkt_sched: Remove TC_RED_OFFLOADED from uapi

 include/net/sch_generic.h      |  1 +
 include/uapi/linux/pkt_sched.h |  1 -
 include/uapi/linux/rtnetlink.h |  1 +
 net/sched/sch_api.c            |  2 ++
 net/sched/sch_red.c            | 31 +++++++++++++++----------------
 5 files changed, 19 insertions(+), 17 deletions(-)

-- 
2.4.3

[PATCH net 1/3] net: sched: Add TCA_HW_OFFLOAD

[Yuval Mintz]

Qdiscs can be offloaded to HW, but current implementation isn't uniform.
Instead, qdiscs either pass information about offload status via their
TCA_OPTIONS or omit it altogether.

Introduce a new attribute - TCA_HW_OFFLOAD that would form a uniform
uAPI for the offloading status of qdiscs.

Signed-off-by: Yuval Mintz <yuvalm@mellanox.com>
---
Do Notice this is going to create [easy-to-solve-]conflicts with net-next,
Due to 6b3ba9146fe6 ("net: sched: allow qdiscs to handle locking").
That's also why the numbering here are apparently inconsistent [skipping
0x100].
---

 include/net/sch_generic.h      | 1 +
 include/uapi/linux/rtnetlink.h | 1 +
 net/sched/sch_api.c            | 2 ++
 3 files changed, 4 insertions(+)

diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index 65d0d25..83a3e47 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -71,6 +71,7 @@ struct Qdisc {
 				      * qdisc_tree_decrease_qlen() should stop.
 				      */
 #define TCQ_F_INVISIBLE		0x80 /* invisible by default in dump */
+#define TCQ_F_OFFLOADED		0x200 /* qdisc is offloaded to HW */
 	u32			limit;
 	const struct Qdisc_ops	*ops;
 	struct qdisc_size_table	__rcu *stab;
diff --git a/include/uapi/linux/rtnetlink.h b/include/uapi/linux/rtnetlink.h
index d8b5f80..843e29a 100644
--- a/include/uapi/linux/rtnetlink.h
+++ b/include/uapi/linux/rtnetlink.h
@@ -557,6 +557,7 @@ enum {
 	TCA_PAD,
 	TCA_DUMP_INVISIBLE,
 	TCA_CHAIN,
+	TCA_HW_OFFLOAD,
 	__TCA_MAX
 };
 
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index b6c4f53..0f1eab9 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -795,6 +795,8 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct Qdisc *q, u32 clid,
 	tcm->tcm_info = refcount_read(&q->refcnt);
 	if (nla_put_string(skb, TCA_KIND, q->ops->id))
 		goto nla_put_failure;
+	if (nla_put_u8(skb, TCA_HW_OFFLOAD, !!(q->flags & TCQ_F_OFFLOADED)))
+		goto nla_put_failure;
 	if (q->ops->dump && q->ops->dump(q, skb) < 0)
 		goto nla_put_failure;
 	qlen = q->q.qlen;
-- 
2.4.3

Re: [PATCH net 1/3] net: sched: Add TCA_HW_OFFLOAD

[Jiri Pirko]

Thu, Dec 14, 2017 at 02:54:29PM CET, yuvalm@mellanox.com wrote:
>Qdiscs can be offloaded to HW, but current implementation isn't uniform.
>Instead, qdiscs either pass information about offload status via their
>TCA_OPTIONS or omit it altogether.
>
>Introduce a new attribute - TCA_HW_OFFLOAD that would form a uniform
>uAPI for the offloading status of qdiscs.
>
>Signed-off-by: Yuval Mintz <yuvalm@mellanox.com>

Acked-by: Jiri Pirko <jiri@mellanox.com>

[PATCH net 2/3] net: sched: Move to new offload indication in RED

[Yuval Mintz]

Let RED utilize the new internal flag, TCQ_F_OFFLOADED,
to mark a given qdisc as offloaded instead of using a dedicated
indication.

Also, change internal logic into looking at said flag when possible.

Fixes: 602f3baf2218 ("net_sch: red: Add offload ability to RED qdisc")
Signed-off-by: Yuval Mintz <yuvalm@mellanox.com>
---
 net/sched/sch_red.c | 31 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/net/sched/sch_red.c b/net/sched/sch_red.c
index 9d874e6..f0747eb 100644
--- a/net/sched/sch_red.c
+++ b/net/sched/sch_red.c
@@ -157,6 +157,7 @@ static int red_offload(struct Qdisc *sch, bool enable)
 		.handle = sch->handle,
 		.parent = sch->parent,
 	};
+	int err;
 
 	if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc)
 		return -EOPNOTSUPP;
@@ -171,7 +172,14 @@ static int red_offload(struct Qdisc *sch, bool enable)
 		opt.command = TC_RED_DESTROY;
 	}
 
-	return dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_RED, &opt);
+	err = dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_RED, &opt);
+
+	if (!err && enable)
+		sch->flags |= TCQ_F_OFFLOADED;
+	else
+		sch->flags &= ~TCQ_F_OFFLOADED;
+
+	return err;
 }
 
 static void red_destroy(struct Qdisc *sch)
@@ -274,7 +282,7 @@ static int red_init(struct Qdisc *sch, struct nlattr *opt)
 	return red_change(sch, opt);
 }
 
-static int red_dump_offload(struct Qdisc *sch, struct tc_red_qopt *opt)
+static int red_dump_offload_stats(struct Qdisc *sch, struct tc_red_qopt *opt)
 {
 	struct net_device *dev = qdisc_dev(sch);
 	struct tc_red_qopt_offload hw_stats = {
@@ -286,21 +294,12 @@ static int red_dump_offload(struct Qdisc *sch, struct tc_red_qopt *opt)
 			.stats.qstats = &sch->qstats,
 		},
 	};
-	int err;
 
-	opt->flags &= ~TC_RED_OFFLOADED;
-	if (!tc_can_offload(dev) || !dev->netdev_ops->ndo_setup_tc)
-		return 0;
-
-	err = dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_RED,
-					    &hw_stats);
-	if (err == -EOPNOTSUPP)
+	if (!(sch->flags & TCQ_F_OFFLOADED))
 		return 0;
 
-	if (!err)
-		opt->flags |= TC_RED_OFFLOADED;
-
-	return err;
+	return dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_QDISC_RED,
+					     &hw_stats);
 }
 
 static int red_dump(struct Qdisc *sch, struct sk_buff *skb)
@@ -319,7 +318,7 @@ static int red_dump(struct Qdisc *sch, struct sk_buff *skb)
 	int err;
 
 	sch->qstats.backlog = q->qdisc->qstats.backlog;
-	err = red_dump_offload(sch, &opt);
+	err = red_dump_offload_stats(sch, &opt);
 	if (err)
 		goto nla_put_failure;
 
@@ -347,7 +346,7 @@ static int red_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
 		.marked	= q->stats.prob_mark + q->stats.forced_mark,
 	};
 
-	if (tc_can_offload(dev) &&  dev->netdev_ops->ndo_setup_tc) {
+	if (sch->flags & TCQ_F_OFFLOADED) {
 		struct red_stats hw_stats = {0};
 		struct tc_red_qopt_offload hw_stats_request = {
 			.command = TC_RED_XSTATS,
-- 
2.4.3

Re: [PATCH net 2/3] net: sched: Move to new offload indication in RED

[Jiri Pirko]

Thu, Dec 14, 2017 at 02:54:30PM CET, yuvalm@mellanox.com wrote:
>Let RED utilize the new internal flag, TCQ_F_OFFLOADED,
>to mark a given qdisc as offloaded instead of using a dedicated
>indication.
>
>Also, change internal logic into looking at said flag when possible.
>
>Fixes: 602f3baf2218 ("net_sch: red: Add offload ability to RED qdisc")
>Signed-off-by: Yuval Mintz <yuvalm@mellanox.com>

Acked-by: Jiri Pirko <jiri@mellanox.com>

Note that it would be good to add red offload presence in drivers
(mlxsw) and forbid to disable tc offload feature in that case.

[PATCH net 3/3] pkt_sched: Remove TC_RED_OFFLOADED from uapi

[Yuval Mintz]

Following the previous patch, RED is now using the new uniform uapi
for indicating it's offloaded. As a result, TC_RED_OFFLOADED is no
longer utilized by kernel and can be removed [as it's still not
part of any stable release].

Fixes: 602f3baf2218 ("net_sch: red: Add offload ability to RED qdisc")
Signed-off-by: Yuval Mintz <yuvalm@mellanox.com>
---
 include/uapi/linux/pkt_sched.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index af3cc2f..37b5096 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -256,7 +256,6 @@ struct tc_red_qopt {
 #define TC_RED_ECN		1
 #define TC_RED_HARDDROP		2
 #define TC_RED_ADAPTATIVE	4
-#define TC_RED_OFFLOADED	8
 };
 
 struct tc_red_xstats {
-- 
2.4.3

Re: [PATCH net 3/3] pkt_sched: Remove TC_RED_OFFLOADED from uapi

[Jiri Pirko]

Thu, Dec 14, 2017 at 02:54:31PM CET, yuvalm@mellanox.com wrote:
>Following the previous patch, RED is now using the new uniform uapi
>for indicating it's offloaded. As a result, TC_RED_OFFLOADED is no
>longer utilized by kernel and can be removed [as it's still not
>part of any stable release].
>
>Fixes: 602f3baf2218 ("net_sch: red: Add offload ability to RED qdisc")
>Signed-off-by: Yuval Mintz <yuvalm@mellanox.com>

Acked-by: Jiri Pirko <jiri@mellanox.com>

Re: [PATCH net 0/3] net: sched: Make qdisc offload uapi uniform

[David Miller]

From: Yuval Mintz <yuvalm@mellanox.com>
Date: Thu, 14 Dec 2017 15:54:28 +0200

> Several qdiscs can already be offloaded to hardware, but there's an
> inconsistecy in regard to the uapi through which they indicate such
> an offload is taking place - indication is passed to the user via
> TCA_OPTIONS where each qdisc retains private logic for setting it.
> 
> The recent addition of offloading to RED in
> 602f3baf2218 ("net_sch: red: Add offload ability to RED qdisc") caused
> the addition of yet another uapi field for this purpose -
> TC_RED_OFFLOADED.
> 
> For clarity and prevention of bloat in the uapi we want to eliminate
> said added uapi, replacing it with a common mechanism that can be used
> to reflect offload status of the various qdiscs.
> 
> The first patch introduces TCA_HW_OFFLOAD as the generic message meant
> for this purpose. The second changes the current RED implementation into
> setting the internal bits necessary for passing it, and the third removes
> TC_RED_OFFLOADED as its no longer needed.
> 
> Dave,
> 
> A bit unorthodox as it's not a fix per-se, but it's the last chance
> for killing the unneeded uapi and replacing it with something better
> before getting stuck with it forever.

I agree, let's take care of this now while we can.

Series applied, thanks.