* RE: [PATCH net v2 2/3] xfrm: Add an activate() offload dev op
From: Yossi Kuperman @ 2017-12-03 22:16 UTC (permalink / raw)
To: Shannon Nelson, Aviv Heller, Steffen Klassert
Cc: Herbert Xu, Boris Pismenny, Yevgeny Kliteynik,
netdev@vger.kernel.org
In-Reply-To: <f5014a6b-082e-3526-0978-ab1c063f2879@oracle.com>
> -----Original Message-----
> From: Shannon Nelson [mailto:shannon.nelson@oracle.com]
> Sent: Sunday, December 3, 2017 12:11 AM
> To: Aviv Heller <avivh@mellanox.com>; Steffen Klassert
> <steffen.klassert@secunet.com>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>; Boris Pismenny
> <borisp@mellanox.com>; Yossi Kuperman <yossiku@mellanox.com>;
> Yevgeny Kliteynik <kliteyn@mellanox.com>; netdev@vger.kernel.org
> Subject: Re: [PATCH net v2 2/3] xfrm: Add an activate() offload dev op
>
> On 12/1/2017 11:47 AM, Shannon Nelson wrote:
> > On 11/28/2017 9:55 AM, avivh@mellanox.com wrote:
> >> From: Aviv Heller <avivh@mellanox.com>
> >>
> >> Adding the state to the offload device prior to replay init in
> >> xfrm_state_construct() will result in NULL dereference if a matching
> >> ESP packet is received in between.
> >>
> >> In order to inhibit driver offload logic from processing the state's
> >> packets prior to the xfrm_state object being completely initialized
> >> and added to the SADBs, a new activate() operation was added to
> >> inform the driver the aforementioned conditions have been met.
> >
> > Are there also conditions where you would want to temporarily
> > deactivate, or pause, the incoming driver offload, followed then by
> > another activate?
> >
Nope.
> > sln
>
> Instead of setting up a half-ready state that needs the activate() operation to
> finish, can we instead just move the xfrm_dev_state_add() call to after the
> xfrm_init_replay()? Especially since this really only makes sense for the
> inbound, and makes no sense for the outbound path.
>
It might solve the problem this time, but still the state is not fully initialized and it might break
in the future. Adding an SA to hardware can fail, and if we were to place xfrm_dev_state_add()
after the SA is fully initialized, we will be forced to roll-back. Let alone the possible things that could
go wrong once an SA is fully active while the hardware is oblivious. The core issue here is that we
can't construct the SA and configure the hardware atomically, hence the two steps. The first step is
to ensure the hardware can be configured properly, if not abort before the SA is initialized, and the
second step is to just "mark" the state as active. Please note that the we don't explicitly mark it, it
is active just by its existence in the hash table, which we already have and maintain.
Out of curiosity, what's wrong with adding yet another callback (not on the critical-path)?
Thanks for your comments.
> sln
>
> >
> >>
> >> Signed-off-by: Aviv Heller <avivh@mellanox.com>
> >> Signed-off-by: Yossi Kuperman <yossiku@mellanox.com>
> >> ---
> >> v1 -> v2:
> >> - Separate to state addition and then activation, instead
> >> of relocating dev state addition call.
> >> ---
> >> include/linux/netdevice.h | 1 +
> >> include/net/xfrm.h | 12 ++++++++++++
> >> net/xfrm/xfrm_user.c | 5 +++++
> >> 3 files changed, 18 insertions(+)
> >>
> >> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> >> index 2eaac7d..c6ca356 100644
> >> --- a/include/linux/netdevice.h
> >> +++ b/include/linux/netdevice.h
> >> @@ -819,6 +819,7 @@ struct netdev_xdp {
> >> #ifdef CONFIG_XFRM_OFFLOAD
> >> struct xfrmdev_ops {
> >> int (*xdo_dev_state_add) (struct xfrm_state *x);
> >> + void (*xdo_dev_state_activate) (struct xfrm_state *x);
> >> void (*xdo_dev_state_delete) (struct xfrm_state *x);
> >> void (*xdo_dev_state_free) (struct xfrm_state *x);
> >> bool (*xdo_dev_offload_ok) (struct sk_buff *skb, diff --git
> >> a/include/net/xfrm.h b/include/net/xfrm.h index e015e16..324374e
> >> 100644
> >> --- a/include/net/xfrm.h
> >> +++ b/include/net/xfrm.h
> >> @@ -1877,6 +1877,14 @@ static inline bool xfrm_dst_offload_ok(struct
> >> dst_entry *dst)
> >> return false;
> >> }
> >> +static inline void xfrm_dev_state_activate(struct xfrm_state *x) {
> >> + struct xfrm_state_offload *xso = &x->xso;
> >> +
> >> + if (xso->dev && xso->dev->xfrmdev_ops->xdo_dev_state_activate)
> >> + xso->dev->xfrmdev_ops->xdo_dev_state_activate(x);
> >> +}
> >> +
> >> static inline void xfrm_dev_state_delete(struct xfrm_state *x)
> >> {
> >> struct xfrm_state_offload *xso = &x->xso; @@ -1907,6 +1915,10
> >> @@ static inline int xfrm_dev_state_add(struct net *net, struct
> >> xfrm_state *x, stru
> >> return 0;
> >> }
> >> +static inline void xfrm_dev_state_activate(struct xfrm_state *x) { }
> >> +
> >> static inline void xfrm_dev_state_delete(struct xfrm_state *x)
> >> {
> >> }
> >> diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c index
> >> e44a0fe..d06f579 100644
> >> --- a/net/xfrm/xfrm_user.c
> >> +++ b/net/xfrm/xfrm_user.c
> >> @@ -662,6 +662,11 @@ static int xfrm_add_sa(struct sk_buff *skb,
> >> struct nlmsghdr *nlh,
> >> goto out;
> >> }
> >> + spin_lock_bh(&x->lock);
> >> + if (x->km.state == XFRM_STATE_VALID)
> >> + xfrm_dev_state_activate(x);
> >> + spin_unlock_bh(&x->lock);
> >> +
> >> c.seq = nlh->nlmsg_seq;
> >> c.portid = nlh->nlmsg_pid;
> >> c.event = nlh->nlmsg_type;
> >>
^ permalink raw reply
* [PATCH net-next 2/3] Add Common Applications Kept Enhanced (cake) qdisc
From: Dave Taht @ 2017-12-03 22:06 UTC (permalink / raw)
To: netdev
Cc: Dave Taht, Toke Høiland-Jørgensen, Sebastian Moeller,
Ryan Mounce, Jonathan Morton, Kevin Darbyshire-Bryant,
Nils Andreas Svee, Dean Scarff, Loganaden Velvindron
In-Reply-To: <1512338775-3270-1-git-send-email-dave.taht@gmail.com>
sch_cake is intended to squeeze the most bandwidth and latency out of even
the slowest ISP links and routers, while presenting an API simple enough
that even an ISP can configure it.
Example of use on a cable ISP uplink:
tc qdisc add dev eth0 cake bandwidth 20Mbit nat docsis ack-filter
To shape a cable download link (ifb and tc-mirred setup elided)
tc qdisc add dev ifb0 cake bandwidth 200mbit nat docsis ingress wash besteffort
Cake is filled with:
* A hybrid Codel/Blue AQM algorithm, "Cobalt", tied to an FQ_Codel
derived Flow Queuing system, which autoconfigures based on the bandwidth.
* A novel "triple-isolate" mode (the default) which balances per-host
and per-flow FQ even through NAT.
* An deficit based shaper, that can also be used in an unlimited mode.
* 8 way set associative hashing to reduce flow collisions to a minimum.
* A reasonable interpretation of various diffserv latency/loss tradeoffs.
* Support for zeroing diffserv markings for entering and exiting traffic.
* Support for interacting well with Docsis 3.0 shaper framing.
* Extensive support for DSL framing types.
* (New) Support for ack filtering.
* Extensive statistics for measuring, loss, ecn markings, latency variation.
There are some features still considered experimental, notably the
ingress_autorate bandwidth estimator and cobalt itself.
Various versions baking have been available as an out of tree build for
kernel versions going back to 3.10, as the embedded router world has been
running a few years behind mainline Linux. A stable version has been
generally available on lede-17.01 and later.
sch_cake replaces a combination of iptables, tc filter, htb and fq_codel
in the sqm-scripts, with sane defaults and vastly simpler configuration.
Cake's principal author is Jonathan Morton, with contributions from
Kevin Darbyshire-Bryant, Toke Høiland-Jørgensen, Sebastian Moeller,
Ryan Mounce, Guido Sarducci, Dean Scarff, Nils Andreas Svee, Dave Täht,
and Loganaden Velvindron.
Testing from Pete Heist, Georgios Amanakis, and the many other members of
the cake@lists.bufferbloat.net mailing list.
tc -s qdisc show dev eth0
qdisc cake 1: dev eth0 root refcnt 2 bandwidth 20Mbit diffserv3
triple-isolate nat ack-filter rtt 100.0ms noatm overhead 18 via-ethernet
total_overhead 18 hard_header_len 14 mpu 64
Sent 119336682 bytes 390608 pkt (dropped 126463, overlimits 882857 requeues 0)
backlog 23620b 29p requeues 0
memory used: 158598b of 4Mb
capacity estimate: 20Mbit
Bulk Best Effort Voice
thresh 1250Kbit 20Mbit 5Mbit
target 14.5ms 5.0ms 5.0ms
interval 109.5ms 100.0ms 100.0ms
pk_delay 6.8ms 12.8ms 9.8ms
av_delay 2.5ms 2.6ms 2.6ms
sp_delay 538us 256us 350us
pkts 80534 339655 96911
bytes 12165915 90314487 31167998
way_inds 0 0 0
way_miss 6 79 7
way_cols 0 0 0
drops 1161 2303 480
marks 0 0 0
ack_drop 45420 69090 8009
sp_flows 2 5 2
bk_flows 4 16 3
un_flows 0 0 0
max_len 3028 3028 3028
Signed-off-by: Dave Taht <dave.taht@gmail.com>
Tested-by: Pete Heist <peteheist@gmail.com>
Tested-by: Georgios Amanakis <gamanakis@gmail.com>
---
include/net/cobalt.h | 152 +++
net/sched/sch_cake.c | 2561 ++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 2713 insertions(+)
create mode 100644 include/net/cobalt.h
create mode 100644 net/sched/sch_cake.c
diff --git a/include/net/cobalt.h b/include/net/cobalt.h
new file mode 100644
index 0000000..b2559df
--- /dev/null
+++ b/include/net/cobalt.h
@@ -0,0 +1,152 @@
+#ifndef __NET_SCHED_COBALT_H
+#define __NET_SCHED_COBALT_H
+
+/* COBALT - Codel-BLUE Alternate AQM algorithm.
+ *
+ * Copyright (C) 2011-2012 Kathleen Nichols <nichols@pollere.com>
+ * Copyright (C) 2011-2012 Van Jacobson <van@pollere.net>
+ * Copyright (C) 2012 Eric Dumazet <edumazet@google.com>
+ * Copyright (C) 2016-2017 Michael D. Täht <dave.taht@gmail.com>
+ * Copyright (c) 2015-2017 Jonathan Morton <chromatix99@gmail.com>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions, and the following disclaimer,
+ * without modification.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * 3. The names of the authors may not be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ * Alternatively, provided that this notice is retained in full, this
+ * software may be distributed under the terms of the GNU General
+ * Public License ("GPL") version 2, in which case the provisions of the
+ * GPL apply INSTEAD OF those given above.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
+ * DAMAGE.
+ */
+
+/* COBALT operates the Codel and BLUE algorithms in parallel, in order to
+ * obtain the best features of each. Codel is excellent on flows which
+ * respond to congestion signals in a TCP-like way. BLUE is more effective on
+ * unresponsive flows.
+ */
+
+#include <linux/version.h>
+#include <linux/types.h>
+#include <linux/ktime.h>
+#include <linux/skbuff.h>
+#include <net/pkt_sched.h>
+#include <net/inet_ecn.h>
+#include <linux/reciprocal_div.h>
+
+typedef u64 cobalt_time_t;
+typedef s64 cobalt_tdiff_t;
+
+#define MS2TIME(a) (a * (u64) NSEC_PER_MSEC)
+#define US2TIME(a) (a * (u64) NSEC_PER_USEC)
+
+struct cobalt_skb_cb {
+ cobalt_time_t enqueue_time;
+};
+
+static inline cobalt_time_t cobalt_get_time(void)
+{
+ return ktime_get_ns();
+}
+
+static inline u32 cobalt_time_to_us(cobalt_time_t val)
+{
+ do_div(val, NSEC_PER_USEC);
+ return (u32)val;
+}
+
+static inline struct cobalt_skb_cb *get_cobalt_cb(const struct sk_buff *skb)
+{
+ qdisc_cb_private_validate(skb, sizeof(struct cobalt_skb_cb));
+ return (struct cobalt_skb_cb *)qdisc_skb_cb(skb)->data;
+}
+
+static inline cobalt_time_t cobalt_get_enqueue_time(const struct sk_buff *skb)
+{
+ return get_cobalt_cb(skb)->enqueue_time;
+}
+
+static inline void cobalt_set_enqueue_time(struct sk_buff *skb,
+ cobalt_time_t now)
+{
+ get_cobalt_cb(skb)->enqueue_time = now;
+}
+
+/**
+ * struct cobalt_params - contains codel and blue parameters
+ * @interval: codel initial drop rate
+ * @target: maximum persistent sojourn time & blue update rate
+ * @p_inc: increment of blue drop probability (0.32 fxp)
+ * @p_dec: decrement of blue drop probability (0.32 fxp)
+ */
+struct cobalt_params {
+ cobalt_time_t interval;
+ cobalt_time_t target;
+ u32 p_inc;
+ u32 p_dec;
+};
+
+/* struct cobalt_vars - contains codel and blue variables
+ * @count: codel dropping frequency
+ * @rec_inv_sqrt: reciprocal value of sqrt(count) >> 1
+ * @drop_next: time to drop next packet, or when we dropped last
+ * @blue_timer: Blue time to next drop
+ * @p_drop: BLUE drop probability (0.32 fxp)
+ * @dropping: set if in dropping state
+ * @ecn_marked: set if marked
+ */
+struct cobalt_vars {
+ u32 count;
+ u32 rec_inv_sqrt;
+ cobalt_time_t drop_next;
+ cobalt_time_t blue_timer;
+ u32 p_drop;
+ bool dropping;
+ bool ecn_marked;
+};
+
+/* Initialise visible and internal data. */
+static void cobalt_vars_init(struct cobalt_vars *vars);
+
+static struct cobalt_skb_cb *get_cobalt_cb(const struct sk_buff *skb);
+static cobalt_time_t cobalt_get_enqueue_time(const struct sk_buff *skb);
+
+/* Call this when a packet had to be dropped due to queue overflow. */
+static bool cobalt_queue_full(struct cobalt_vars *vars,
+ struct cobalt_params *p,
+ cobalt_time_t now);
+
+/* Call this when the queue was serviced but turned out to be empty. */
+static bool cobalt_queue_empty(struct cobalt_vars *vars,
+ struct cobalt_params *p,
+ cobalt_time_t now);
+
+/* Call this with a freshly dequeued packet for possible congestion marking.
+ * Returns true as an instruction to drop the packet, false for delivery.
+ */
+static bool cobalt_should_drop(struct cobalt_vars *vars,
+ struct cobalt_params *p,
+ cobalt_time_t now,
+ struct sk_buff *skb);
+
+#endif
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
new file mode 100644
index 0000000..79964d7
--- /dev/null
+++ b/net/sched/sch_cake.c
@@ -0,0 +1,2561 @@
+/* COMMON Applications Kept Enhanced (CAKE) discipline - version 5
+ *
+ * Copyright (C) 2014-2017 Jonathan Morton <chromatix99@gmail.com>
+ * Copyright (C) 2015-2017 Toke Høiland-Jørgensen <toke@toke.dk>
+ * Copyright (C) 2014-2017 Dave Täht <dave.taht@gmail.com>
+ * Copyright (C) 2015-2017 Sebastian Moeller <moeller0@gmx.de>
+ * (C) 2015-2017 Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk>
+ * Copyright (C) 2017 Ryan Mounce <ryan@mounce.com.au>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions, and the following disclaimer,
+ * without modification.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * 3. The names of the authors may not be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ * Alternatively, provided that this notice is retained in full, this
+ * software may be distributed under the terms of the GNU General
+ * Public License ("GPL") version 2, in which case the provisions of the
+ * GPL apply INSTEAD OF those given above.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
+ * DAMAGE.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/jiffies.h>
+#include <linux/string.h>
+#include <linux/in.h>
+#include <linux/errno.h>
+#include <linux/init.h>
+#include <linux/skbuff.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+#include <linux/reciprocal_div.h>
+#include <linux/version.h>
+#include <net/netlink.h>
+#include <net/pkt_sched.h>
+#include <linux/if_vlan.h>
+#include <net/tcp.h>
+#include <net/flow_dissector.h>
+#include <net/cobalt.h>
+
+#if IS_ENABLED(CONFIG_NF_CONNTRACK)
+#include <net/netfilter/nf_conntrack_core.h>
+#include <net/netfilter/nf_conntrack_zones.h>
+#include <net/netfilter/nf_conntrack.h>
+#endif
+
+/* The CAKE Principles:
+ * (or, how to have your cake and eat it too)
+ *
+ * This is a combination of several shaping, AQM and FQ techniques into one
+ * easy-to-use package:
+ *
+ * - An overall bandwidth shaper, to move the bottleneck away from dumb CPE
+ * equipment and bloated MACs. This operates in deficit mode (as in sch_fq),
+ * eliminating the need for any sort of burst parameter (eg. token bucket
+ * depth). Burst support is limited to that necessary to overcome scheduling
+ * latency.
+ *
+ * - A Diffserv-aware priority queue, giving more priority to certain classes,
+ * up to a specified fraction of bandwidth. Above that bandwidth threshold,
+ * the priority is reduced to avoid starving other tins.
+ *
+ * - Each priority tin has a separate Flow Queue system, to isolate traffic
+ * flows from each other. This prevents a burst on one flow from increasing
+ * the delay to another. Flows are distributed to queues using a
+ * set-associative hash function.
+ *
+ * - Each queue is actively managed by Cobalt, which is a combination of the
+ * Codel and Blue AQM algorithms. This serves flows fairly, and signals
+ * congestion early via ECN (if available) and/or packet drops, to keep
+ * latency low. The codel parameters are auto-tuned based on the bandwidth
+ * setting, as is necessary at low bandwidths.
+ *
+ * The configuration parameters are kept deliberately simple for ease of use.
+ * Everything has sane defaults. Complete generality of configuration is *not*
+ * a goal.
+ *
+ * The priority queue operates according to a weighted DRR scheme, combined with
+ * a bandwidth tracker which reuses the shaper logic to detect which side of the
+ * bandwidth sharing threshold the tin is operating. This determines whether a
+ * priority-based weight (high) or a bandwidth-based weight (low) is used for
+ * that tin in the current pass.
+ *
+ * This qdisc was inspired by Eric Dumazet's fq_codel code, which he kindly
+ * granted us permission to leverage.
+ */
+
+#define CAKE_SET_WAYS (8)
+#define CAKE_MAX_TINS (8)
+#define CAKE_QUEUES (1024)
+
+#ifndef CAKE_VERSION
+#define CAKE_VERSION "unknown"
+#endif
+static char *cake_version __attribute__((used)) = "Cake version: "
+ CAKE_VERSION;
+
+enum {
+ CAKE_SET_NONE = 0,
+ CAKE_SET_SPARSE,
+ CAKE_SET_SPARSE_WAIT, /* counted in SPARSE, actually in BULK */
+ CAKE_SET_BULK,
+ CAKE_SET_DECAYING
+};
+
+struct cake_flow {
+ /* this stuff is all needed per-flow at dequeue time */
+ struct sk_buff *head;
+ struct sk_buff *tail;
+ struct sk_buff *ackcheck;
+ struct list_head flowchain;
+ s32 deficit;
+ struct cobalt_vars cvars;
+ u16 srchost; /* index into cake_host table */
+ u16 dsthost;
+ u8 set;
+}; /* please try to keep this structure <= 64 bytes */
+
+struct cake_host {
+ u32 srchost_tag;
+ u32 dsthost_tag;
+ u16 srchost_refcnt;
+ u16 dsthost_refcnt;
+};
+
+struct cake_heap_entry {
+ u16 t:3, b:10;
+};
+
+struct cake_tin_data {
+ struct cake_flow flows[CAKE_QUEUES];
+ u32 backlogs[CAKE_QUEUES];
+ u32 tags[CAKE_QUEUES]; /* for set association */
+ u16 overflow_idx[CAKE_QUEUES];
+ struct cake_host hosts[CAKE_QUEUES]; /* for triple isolation */
+ u16 flow_quantum;
+
+ struct cobalt_params cparams;
+ u32 drop_overlimit;
+ u16 bulk_flow_count;
+ u16 sparse_flow_count;
+ u16 decaying_flow_count;
+ u16 unresponsive_flow_count;
+
+ u32 max_skblen;
+
+ struct list_head new_flows;
+ struct list_head old_flows;
+ struct list_head decaying_flows;
+
+ /* time_next = time_this + ((len * rate_ns) >> rate_shft) */
+ u64 tin_time_next_packet;
+ u32 tin_rate_ns;
+ u32 tin_rate_bps;
+ u16 tin_rate_shft;
+
+ u16 tin_quantum_prio;
+ u16 tin_quantum_band;
+ s32 tin_deficit;
+ u32 tin_backlog;
+ u32 tin_dropped;
+ u32 tin_ecn_mark;
+
+ u32 packets;
+ u64 bytes;
+
+ u32 ack_drops;
+
+ /* moving averages */
+ cobalt_time_t avge_delay;
+ cobalt_time_t peak_delay;
+ cobalt_time_t base_delay;
+
+ /* hash function stats */
+ u32 way_directs;
+ u32 way_hits;
+ u32 way_misses;
+ u32 way_collisions;
+}; /* number of tins is small, so size of this struct doesn't matter much */
+
+struct cake_sched_data {
+ struct cake_tin_data *tins;
+
+ struct cake_heap_entry overflow_heap[CAKE_QUEUES * CAKE_MAX_TINS];
+ u16 overflow_timeout;
+
+ u16 tin_cnt;
+ u8 tin_mode;
+ u8 flow_mode;
+
+ /* time_next = time_this + ((len * rate_ns) >> rate_shft) */
+ u16 rate_shft;
+ u64 time_next_packet;
+ u64 failsafe_next_packet;
+ u32 rate_ns;
+ u32 rate_bps;
+ u16 rate_flags;
+ s16 rate_overhead;
+ u16 rate_mpu;
+ u32 interval;
+ u32 target;
+
+ /* resource tracking */
+ u32 buffer_used;
+ u32 buffer_max_used;
+ u32 buffer_limit;
+ u32 buffer_config_limit;
+
+ /* indices for dequeue */
+ u16 cur_tin;
+ u16 cur_flow;
+
+ struct qdisc_watchdog watchdog;
+ const u8 *tin_index;
+ const u8 *tin_order;
+
+ /* bandwidth capacity estimate */
+ u64 last_packet_time;
+ u64 avg_packet_interval;
+ u64 avg_window_begin;
+ u32 avg_window_bytes;
+ u32 avg_peak_bandwidth;
+ u64 last_reconfig_time;
+};
+
+enum {
+ CAKE_MODE_BESTEFFORT = 1,
+ CAKE_MODE_PRECEDENCE,
+ CAKE_MODE_DIFFSERV8,
+ CAKE_MODE_DIFFSERV4,
+ CAKE_MODE_LLT,
+ CAKE_MODE_DIFFSERV3,
+ CAKE_MODE_MAX
+};
+
+enum {
+ CAKE_FLAG_ATM = 0x0001,
+ CAKE_FLAG_PTM = 0x0002,
+ CAKE_FLAG_AUTORATE_INGRESS = 0x0010,
+ CAKE_FLAG_INGRESS = 0x0040,
+ CAKE_FLAG_WASH = 0x0100,
+ CAKE_FLAG_ACK_FILTER = 0x0200,
+ CAKE_FLAG_ACK_AGGRESSIVE = 0x0400
+};
+
+enum {
+ CAKE_FLOW_NONE = 0,
+ CAKE_FLOW_SRC_IP,
+ CAKE_FLOW_DST_IP,
+ CAKE_FLOW_HOSTS, /* = CAKE_FLOW_SRC_IP | CAKE_FLOW_DST_IP */
+ CAKE_FLOW_FLOWS,
+ CAKE_FLOW_DUAL_SRC, /* = CAKE_FLOW_SRC_IP | CAKE_FLOW_FLOWS */
+ CAKE_FLOW_DUAL_DST, /* = CAKE_FLOW_DST_IP | CAKE_FLOW_FLOWS */
+ CAKE_FLOW_TRIPLE, /* = CAKE_FLOW_HOSTS | CAKE_FLOW_FLOWS */
+ CAKE_FLOW_MAX,
+ CAKE_FLOW_NAT_FLAG = 64
+};
+
+static u16 quantum_div[CAKE_QUEUES + 1] = {0};
+
+/* Diffserv lookup tables */
+
+static const u8 precedence[] = {0, 0, 0, 0, 0, 0, 0, 0,
+ 1, 1, 1, 1, 1, 1, 1, 1,
+ 2, 2, 2, 2, 2, 2, 2, 2,
+ 3, 3, 3, 3, 3, 3, 3, 3,
+ 4, 4, 4, 4, 4, 4, 4, 4,
+ 5, 5, 5, 5, 5, 5, 5, 5,
+ 6, 6, 6, 6, 6, 6, 6, 6,
+ 7, 7, 7, 7, 7, 7, 7, 7,
+ };
+
+static const u8 diffserv_llt[] = {1, 0, 0, 1, 2, 2, 1, 1,
+ 3, 1, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 1, 1, 1, 1,
+ 1, 1, 1, 1, 2, 1, 2, 1,
+ 4, 1, 1, 1, 1, 1, 1, 1,
+ 4, 1, 1, 1, 1, 1, 1, 1,
+ };
+
+static const u8 diffserv8[] = {2, 5, 1, 2, 4, 2, 2, 2,
+ 0, 2, 1, 2, 1, 2, 1, 2,
+ 5, 2, 4, 2, 4, 2, 4, 2,
+ 3, 2, 3, 2, 3, 2, 3, 2,
+ 6, 2, 3, 2, 3, 2, 3, 2,
+ 6, 2, 2, 2, 6, 2, 6, 2,
+ 7, 2, 2, 2, 2, 2, 2, 2,
+ 7, 2, 2, 2, 2, 2, 2, 2,
+ };
+
+static const u8 diffserv4[] = {0, 2, 0, 0, 2, 0, 0, 0,
+ 1, 0, 0, 0, 0, 0, 0, 0,
+ 2, 0, 2, 0, 2, 0, 2, 0,
+ 2, 0, 2, 0, 2, 0, 2, 0,
+ 3, 0, 2, 0, 2, 0, 2, 0,
+ 3, 0, 0, 0, 3, 0, 3, 0,
+ 3, 0, 0, 0, 0, 0, 0, 0,
+ 3, 0, 0, 0, 0, 0, 0, 0,
+ };
+
+static const u8 diffserv3[] = {0, 0, 0, 0, 2, 0, 0, 0,
+ 1, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 2, 0, 2, 0,
+ 2, 0, 0, 0, 0, 0, 0, 0,
+ 2, 0, 0, 0, 0, 0, 0, 0,
+ };
+
+static const u8 besteffort[] = {0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ 0, 0, 0, 0, 0, 0, 0, 0,
+ };
+
+/* tin priority order for stats dumping */
+
+static const u8 normal_order[] = {0, 1, 2, 3, 4, 5, 6, 7};
+static const u8 bulk_order[] = {1, 0, 2, 3};
+
+#define REC_INV_SQRT_CACHE (16)
+static u32 cobalt_rec_inv_sqrt_cache[REC_INV_SQRT_CACHE] = {0};
+
+/* http://en.wikipedia.org/wiki/Methods_of_computing_square_roots
+ * new_invsqrt = (invsqrt / 2) * (3 - count * invsqrt^2)
+ *
+ * Here, invsqrt is a fixed point number (< 1.0), 32bit mantissa, aka Q0.32
+ */
+
+static void cobalt_newton_step(struct cobalt_vars *vars)
+{
+ u32 invsqrt = vars->rec_inv_sqrt;
+ u32 invsqrt2 = ((u64)invsqrt * invsqrt) >> 32;
+ u64 val = (3LL << 32) - ((u64)vars->count * invsqrt2);
+
+ val >>= 2; /* avoid overflow in following multiply */
+ val = (val * invsqrt) >> (32 - 2 + 1);
+
+ vars->rec_inv_sqrt = val;
+}
+
+static void cobalt_invsqrt(struct cobalt_vars *vars)
+{
+ if (vars->count < REC_INV_SQRT_CACHE)
+ vars->rec_inv_sqrt = cobalt_rec_inv_sqrt_cache[vars->count];
+ else
+ cobalt_newton_step(vars);
+}
+
+/* There is a big difference in timing between the accurate values placed in
+ * the cache and the approximations given by a single Newton step for small
+ * count values, particularly when stepping from count 1 to 2 or vice versa.
+ * Above 16, a single Newton step gives sufficient accuracy in either
+ * direction, given the precision stored.
+ *
+ * The magnitude of the error when stepping up to count 2 is such as to give
+ * the value that *should* have been produced at count 4.
+ */
+
+static void cobalt_cache_init(void)
+{
+ struct cobalt_vars v;
+
+ memset(&v, 0, sizeof(v));
+ v.rec_inv_sqrt = ~0U;
+ cobalt_rec_inv_sqrt_cache[0] = v.rec_inv_sqrt;
+
+ for (v.count = 1; v.count < REC_INV_SQRT_CACHE; v.count++) {
+ cobalt_newton_step(&v);
+ cobalt_newton_step(&v);
+ cobalt_newton_step(&v);
+ cobalt_newton_step(&v);
+
+ cobalt_rec_inv_sqrt_cache[v.count] = v.rec_inv_sqrt;
+ }
+}
+
+static void cobalt_vars_init(struct cobalt_vars *vars)
+{
+ memset(vars, 0, sizeof(*vars));
+
+ if (!cobalt_rec_inv_sqrt_cache[0]) {
+ cobalt_cache_init();
+ cobalt_rec_inv_sqrt_cache[0] = ~0;
+ }
+}
+
+/* CoDel control_law is t + interval/sqrt(count)
+ * We maintain in rec_inv_sqrt the reciprocal value of sqrt(count) to avoid
+ * both sqrt() and divide operation.
+ */
+static cobalt_time_t cobalt_control(cobalt_time_t t,
+ cobalt_time_t interval,
+ u32 rec_inv_sqrt)
+{
+ return t + reciprocal_scale(interval, rec_inv_sqrt);
+}
+
+/* Call this when a packet had to be dropped due to queue overflow. Returns
+ * true if the BLUE state was quiescent before but active after this call.
+ */
+static bool cobalt_queue_full(struct cobalt_vars *vars,
+ struct cobalt_params *p,
+ cobalt_time_t now)
+{
+ bool up = false;
+
+ if ((now - vars->blue_timer) > p->target) {
+ up = !vars->p_drop;
+ vars->p_drop += p->p_inc;
+ if (vars->p_drop < p->p_inc)
+ vars->p_drop = ~0;
+ vars->blue_timer = now;
+ }
+ vars->dropping = true;
+ vars->drop_next = now;
+ if (!vars->count)
+ vars->count = 1;
+
+ return up;
+}
+
+/* Call this when the queue was serviced but turned out to be empty. Returns
+ * true if the BLUE state was active before but quiescent after this call.
+ */
+static bool cobalt_queue_empty(struct cobalt_vars *vars,
+ struct cobalt_params *p,
+ cobalt_time_t now)
+{
+ bool down = false;
+
+ if (vars->p_drop && (now - vars->blue_timer) > p->target) {
+ if (vars->p_drop < p->p_dec)
+ vars->p_drop = 0;
+ else
+ vars->p_drop -= p->p_dec;
+ vars->blue_timer = now;
+ down = !vars->p_drop;
+ }
+ vars->dropping = false;
+
+ if (vars->count && (now - vars->drop_next) >= 0) {
+ vars->count--;
+ cobalt_invsqrt(vars);
+ vars->drop_next = cobalt_control(vars->drop_next,
+ p->interval,
+ vars->rec_inv_sqrt);
+ }
+
+ return down;
+}
+
+/* Call this with a freshly dequeued packet for possible congestion marking.
+ * Returns true as an instruction to drop the packet, false for delivery.
+ */
+static bool cobalt_should_drop(struct cobalt_vars *vars,
+ struct cobalt_params *p,
+ cobalt_time_t now,
+ struct sk_buff *skb)
+{
+ bool drop = false;
+
+ /* Simplified Codel implementation */
+ cobalt_tdiff_t sojourn = now - cobalt_get_enqueue_time(skb);
+
+/* The 'schedule' variable records, in its sign, whether 'now' is before or
+ * after 'drop_next'. This allows 'drop_next' to be updated before the next
+ * scheduling decision is actually branched, without destroying that
+ * information. Similarly, the first 'schedule' value calculated is preserved
+ * in the boolean 'next_due'.
+ *
+ * As for 'drop_next', we take advantage of the fact that 'interval' is both
+ * the delay between first exceeding 'target' and the first signalling event,
+ * *and* the scaling factor for the signalling frequency. It's therefore very
+ * natural to use a single mechanism for both purposes, and eliminates a
+ * significant amount of reference Codel's spaghetti code. To help with this,
+ * both the '0' and '1' entries in the invsqrt cache are 0xFFFFFFFF, as close
+ * as possible to 1.0 in fixed-point.
+ */
+
+ cobalt_tdiff_t schedule = now - vars->drop_next;
+ bool over_target = sojourn > p->target;
+ bool next_due = vars->count && schedule >= 0;
+
+ vars->ecn_marked = false;
+
+ if (over_target) {
+ if (!vars->dropping) {
+ vars->dropping = true;
+ vars->drop_next = cobalt_control(now,
+ p->interval,
+ vars->rec_inv_sqrt);
+ }
+ if (!vars->count)
+ vars->count = 1;
+ } else if (vars->dropping) {
+ vars->dropping = false;
+ }
+
+ if (next_due && vars->dropping) {
+ /* Use ECN mark if possible, otherwise drop */
+ drop = !(vars->ecn_marked = INET_ECN_set_ce(skb));
+
+ vars->count++;
+ if (!vars->count)
+ vars->count--;
+ cobalt_invsqrt(vars);
+ vars->drop_next = cobalt_control(vars->drop_next,
+ p->interval,
+ vars->rec_inv_sqrt);
+ schedule = now - vars->drop_next;
+ } else {
+ while (next_due) {
+ vars->count--;
+ cobalt_invsqrt(vars);
+ vars->drop_next = cobalt_control(vars->drop_next,
+ p->interval,
+ vars->rec_inv_sqrt);
+ schedule = now - vars->drop_next;
+ next_due = vars->count && schedule >= 0;
+ }
+ }
+
+ /* Simple BLUE implementation. Lack of ECN is deliberate. */
+ if (vars->p_drop)
+ drop |= (prandom_u32() < vars->p_drop);
+
+ /* Overload the drop_next field as an activity timeout */
+ if (!vars->count)
+ vars->drop_next = now + p->interval;
+ else if (schedule > 0 && !drop)
+ vars->drop_next = now;
+
+ return drop;
+}
+
+#if IS_ENABLED(CONFIG_NF_CONNTRACK)
+
+static inline void cake_update_flowkeys(struct flow_keys *keys,
+ const struct sk_buff *skb)
+{
+ enum ip_conntrack_info ctinfo;
+ bool rev = false;
+
+ struct nf_conn *ct;
+ const struct nf_conntrack_tuple *tuple;
+
+ if (tc_skb_protocol(skb) != htons(ETH_P_IP))
+ return;
+
+ ct = nf_ct_get(skb, &ctinfo);
+ if (ct) {
+ tuple = nf_ct_tuple(ct, CTINFO2DIR(ctinfo));
+ } else {
+ const struct nf_conntrack_tuple_hash *hash;
+ struct nf_conntrack_tuple srctuple;
+
+ if (!nf_ct_get_tuplepr(skb, skb_network_offset(skb),
+ NFPROTO_IPV4, dev_net(skb->dev),
+ &srctuple))
+ return;
+
+ hash = nf_conntrack_find_get(dev_net(skb->dev),
+ &nf_ct_zone_dflt,
+ &srctuple);
+ if (!hash)
+ return;
+
+ rev = true;
+ ct = nf_ct_tuplehash_to_ctrack(hash);
+ tuple = nf_ct_tuple(ct, !hash->tuple.dst.dir);
+ }
+
+ keys->addrs.v4addrs.src = rev ? tuple->dst.u3.ip : tuple->src.u3.ip;
+ keys->addrs.v4addrs.dst = rev ? tuple->src.u3.ip : tuple->dst.u3.ip;
+
+ if (keys->ports.ports) {
+ keys->ports.src = rev ? tuple->dst.u.all : tuple->src.u.all;
+ keys->ports.dst = rev ? tuple->src.u.all : tuple->dst.u.all;
+ }
+
+ if (rev)
+ nf_ct_put(ct);
+}
+#else
+static inline void cake_update_flowkeys(struct flow_keys *keys,
+ const struct sk_buff *skb)
+{
+ /* There is nothing we can do here without CONNTRACK */
+}
+#endif
+
+/* Cake has several subtle multiple bit settings. In these cases you
+ * would be matching triple isolate mode as well.
+ */
+
+static inline bool cake_dsrc(int flow_mode)
+{
+ return (flow_mode & CAKE_FLOW_DUAL_SRC) == CAKE_FLOW_DUAL_SRC;
+}
+
+static inline bool cake_ddst(int flow_mode)
+{
+ return (flow_mode & CAKE_FLOW_DUAL_DST) == CAKE_FLOW_DUAL_DST;
+}
+
+static inline u32
+cake_hash(struct cake_tin_data *q, const struct sk_buff *skb, int flow_mode)
+{
+ struct flow_keys keys, host_keys;
+ u32 flow_hash = 0, srchost_hash, dsthost_hash;
+ u16 reduced_hash, srchost_idx, dsthost_idx;
+
+ if (unlikely(flow_mode == CAKE_FLOW_NONE))
+ return 0;
+
+ skb_flow_dissect_flow_keys(skb, &keys,
+ FLOW_DISSECTOR_F_STOP_AT_FLOW_LABEL);
+
+ if (flow_mode & CAKE_FLOW_NAT_FLAG)
+ cake_update_flowkeys(&keys, skb);
+
+ /* flow_hash_from_keys() sorts the addresses by value, so we have
+ * to preserve their order in a separate data structure to treat
+ * src and dst host addresses as independently selectable.
+ */
+ host_keys = keys;
+ host_keys.ports.ports = 0;
+ host_keys.basic.ip_proto = 0;
+ host_keys.keyid.keyid = 0;
+ host_keys.tags.flow_label = 0;
+
+ switch (host_keys.control.addr_type) {
+ case FLOW_DISSECTOR_KEY_IPV4_ADDRS:
+ host_keys.addrs.v4addrs.src = 0;
+ dsthost_hash = flow_hash_from_keys(&host_keys);
+ host_keys.addrs.v4addrs.src = keys.addrs.v4addrs.src;
+ host_keys.addrs.v4addrs.dst = 0;
+ srchost_hash = flow_hash_from_keys(&host_keys);
+ break;
+
+ case FLOW_DISSECTOR_KEY_IPV6_ADDRS:
+ memset(&host_keys.addrs.v6addrs.src, 0,
+ sizeof(host_keys.addrs.v6addrs.src));
+ dsthost_hash = flow_hash_from_keys(&host_keys);
+ host_keys.addrs.v6addrs.src = keys.addrs.v6addrs.src;
+ memset(&host_keys.addrs.v6addrs.dst, 0,
+ sizeof(host_keys.addrs.v6addrs.dst));
+ srchost_hash = flow_hash_from_keys(&host_keys);
+ break;
+
+ default:
+ dsthost_hash = 0;
+ srchost_hash = 0;
+ };
+
+ /* This *must* be after the above switch, since as a
+ * side-effect it sorts the src and dst addresses.
+ */
+ if (flow_mode & CAKE_FLOW_FLOWS)
+ flow_hash = flow_hash_from_keys(&keys);
+
+ if (!(flow_mode & CAKE_FLOW_FLOWS)) {
+ if (flow_mode & CAKE_FLOW_SRC_IP)
+ flow_hash ^= srchost_hash;
+
+ if (flow_mode & CAKE_FLOW_DST_IP)
+ flow_hash ^= dsthost_hash;
+ }
+
+ reduced_hash = flow_hash % CAKE_QUEUES;
+
+ /* set-associative hashing */
+ /* fast path if no hash collision (direct lookup succeeds) */
+ if (likely(q->tags[reduced_hash] == flow_hash &&
+ q->flows[reduced_hash].set)) {
+ q->way_directs++;
+ } else {
+ u32 inner_hash = reduced_hash % CAKE_SET_WAYS;
+ u32 outer_hash = reduced_hash - inner_hash;
+ u32 i, k;
+ bool allocate_src = false;
+ bool allocate_dst = false;
+
+ /* check if any active queue in the set is reserved for
+ * this flow.
+ */
+ for (i = 0, k = inner_hash; i < CAKE_SET_WAYS;
+ i++, k = (k + 1) % CAKE_SET_WAYS) {
+ if (q->tags[outer_hash + k] == flow_hash) {
+ if (i)
+ q->way_hits++;
+
+ if (!q->flows[outer_hash + k].set) {
+ /* need to increment host refcnts */
+ allocate_src = cake_dsrc(flow_mode);
+ allocate_dst = cake_ddst(flow_mode);
+ }
+
+ goto found;
+ }
+ }
+
+ /* no queue is reserved for this flow, look for an
+ * empty one.
+ */
+ for (i = 0; i < CAKE_SET_WAYS;
+ i++, k = (k + 1) % CAKE_SET_WAYS) {
+ if (!q->flows[outer_hash + k].set) {
+ q->way_misses++;
+ allocate_src = cake_dsrc(flow_mode);
+ allocate_dst = cake_ddst(flow_mode);
+ goto found;
+ }
+ }
+
+ /* With no empty queues, default to the original
+ * queue, accept the collision, update the host tags.
+ */
+ q->way_collisions++;
+ q->hosts[q->flows[reduced_hash].srchost].srchost_refcnt--;
+ q->hosts[q->flows[reduced_hash].dsthost].dsthost_refcnt--;
+ allocate_src = cake_dsrc(flow_mode);
+ allocate_dst = cake_ddst(flow_mode);
+found:
+ /* reserve queue for future packets in same flow */
+ reduced_hash = outer_hash + k;
+ q->tags[reduced_hash] = flow_hash;
+
+ if (allocate_src) {
+ srchost_idx = srchost_hash % CAKE_QUEUES;
+ inner_hash = srchost_idx % CAKE_SET_WAYS;
+ outer_hash = srchost_idx - inner_hash;
+ for (i = 0, k = inner_hash; i < CAKE_SET_WAYS;
+ i++, k = (k + 1) % CAKE_SET_WAYS) {
+ if (q->hosts[outer_hash + k].srchost_tag ==
+ srchost_hash)
+ goto found_src;
+ }
+ for (i = 0; i < CAKE_SET_WAYS;
+ i++, k = (k + 1) % CAKE_SET_WAYS) {
+ if (!q->hosts[outer_hash + k].srchost_refcnt)
+ break;
+ }
+ q->hosts[outer_hash + k].srchost_tag = srchost_hash;
+found_src:
+ srchost_idx = outer_hash + k;
+ q->hosts[srchost_idx].srchost_refcnt++;
+ q->flows[reduced_hash].srchost = srchost_idx;
+ }
+
+ if (allocate_dst) {
+ dsthost_idx = dsthost_hash % CAKE_QUEUES;
+ inner_hash = dsthost_idx % CAKE_SET_WAYS;
+ outer_hash = dsthost_idx - inner_hash;
+ for (i = 0, k = inner_hash; i < CAKE_SET_WAYS;
+ i++, k = (k + 1) % CAKE_SET_WAYS) {
+ if (q->hosts[outer_hash + k].dsthost_tag ==
+ dsthost_hash)
+ goto found_dst;
+ }
+ for (i = 0; i < CAKE_SET_WAYS;
+ i++, k = (k + 1) % CAKE_SET_WAYS) {
+ if (!q->hosts[outer_hash + k].dsthost_refcnt)
+ break;
+ }
+ q->hosts[outer_hash + k].dsthost_tag = dsthost_hash;
+found_dst:
+ dsthost_idx = outer_hash + k;
+ q->hosts[dsthost_idx].dsthost_refcnt++;
+ q->flows[reduced_hash].dsthost = dsthost_idx;
+ }
+ }
+
+ return reduced_hash;
+}
+
+/* helper functions : might be changed when/if skb use a standard list_head */
+/* remove one skb from head of slot queue */
+
+static inline struct sk_buff *dequeue_head(struct cake_flow *flow)
+{
+ struct sk_buff *skb = flow->head;
+
+ if (skb) {
+ flow->head = skb->next;
+ skb->next = NULL;
+
+ if (skb == flow->ackcheck)
+ flow->ackcheck = NULL;
+ }
+
+ return skb;
+}
+
+/* add skb to flow queue (tail add) */
+
+static inline void
+flow_queue_add(struct cake_flow *flow, struct sk_buff *skb)
+{
+ if (!flow->head)
+ flow->head = skb;
+ else
+ flow->tail->next = skb;
+ flow->tail = skb;
+ skb->next = NULL;
+}
+
+static struct sk_buff *cake_ack_filter(struct cake_sched_data *q,
+ struct cake_flow *flow)
+{
+ int seglen;
+ struct sk_buff *skb = flow->tail, *skb_check, *skb_check_prev;
+ struct iphdr *iph, *iph_check;
+ struct ipv6hdr *ipv6h, *ipv6h_check;
+ struct tcphdr *tcph, *tcph_check;
+ bool otherconn_ack_seen = false;
+ struct sk_buff *otherconn_checked_to = NULL;
+ bool thisconn_redundant_seen = false, thisconn_seen_last = false;
+ struct sk_buff *thisconn_checked_to = NULL, *thisconn_ack = NULL;
+ bool aggressive = q->rate_flags & CAKE_FLAG_ACK_AGGRESSIVE;
+
+ /* no other possible ACKs to filter */
+ if (flow->head == skb)
+ return NULL;
+
+ iph = skb->encapsulation ? inner_ip_hdr(skb) : ip_hdr(skb);
+ ipv6h = skb->encapsulation ? inner_ipv6_hdr(skb) : ipv6_hdr(skb);
+
+ /* check that the innermost network header is v4/v6, and contains TCP */
+ if (iph->version == 4) {
+ if (iph->protocol != IPPROTO_TCP)
+ return NULL;
+ seglen = ntohs(iph->tot_len) - (4 * iph->ihl);
+ tcph = (struct tcphdr *)((void *)iph + (4 * iph->ihl));
+ } else if (ipv6h->version == 6) {
+ if (ipv6h->nexthdr != IPPROTO_TCP)
+ return NULL;
+ seglen = ntohs(ipv6h->payload_len);
+ tcph = (struct tcphdr *)((void *)ipv6h + sizeof(struct ipv6hdr));
+ } else {
+ return NULL;
+ }
+
+ /* the 'triggering' packet need only have the ACK flag set.
+ * also check that SYN is not set, as there won't be any previous ACKs.
+ */
+ if ((tcp_flag_word(tcph) &
+ (TCP_FLAG_ACK | TCP_FLAG_SYN)) != TCP_FLAG_ACK)
+ return NULL;
+
+ /* the 'triggering' ACK is at the end of the queue,
+ * we have already returned if it is the only packet in the flow.
+ * stop before last packet in queue, don't compare trigger ACK to itself
+ * start where we finished last time if recorded in ->ackcheck
+ * otherwise start from the the head of the flow queue.
+ */
+ skb_check_prev = flow->ackcheck;
+ skb_check = flow->ackcheck ?: flow->head;
+
+ while (skb_check->next) {
+ bool pure_ack, thisconn;
+
+ /* don't increment if at head of flow queue (_prev == NULL) */
+ if (skb_check_prev) {
+ skb_check_prev = skb_check;
+ skb_check = skb_check->next;
+ if (!skb_check->next)
+ break;
+ } else {
+ skb_check_prev = ERR_PTR(-1);
+ }
+
+ iph_check = skb_check->encapsulation ?
+ inner_ip_hdr(skb_check) : ip_hdr(skb_check);
+ ipv6h_check = skb_check->encapsulation ?
+ inner_ipv6_hdr(skb_check) : ipv6_hdr(skb_check);
+
+ if (iph_check->version == 4) {
+ if (iph_check->protocol != IPPROTO_TCP)
+ continue;
+ seglen = ntohs(iph_check->tot_len) - (4 * iph_check->ihl);
+ tcph_check = (struct tcphdr *)((void *)iph_check
+ + (4 * iph_check->ihl));
+ if (iph->version == 4 &&
+ iph_check->saddr == iph->saddr &&
+ iph_check->daddr == iph->daddr) {
+ thisconn = true;
+ } else {
+ thisconn = false;
+ }
+ } else if (ipv6h_check->version == 6) {
+ if (ipv6h_check->nexthdr != IPPROTO_TCP)
+ continue;
+ seglen = ntohs(ipv6h_check->payload_len);
+ tcph_check = (struct tcphdr *)((void *)ipv6h_check +
+ sizeof(struct ipv6hdr));
+ if (ipv6h->version == 6 &&
+ ipv6_addr_cmp(&ipv6h_check->saddr, &ipv6h->saddr) &&
+ ipv6_addr_cmp(&ipv6h_check->daddr, &ipv6h->daddr)) {
+ thisconn = true;
+ } else {
+ thisconn = false;
+ }
+ } else {
+ continue;
+ }
+
+ /* stricter criteria apply to ACKs that we may filter
+ * 3 reserved flags must be unset to avoid future breakage
+ * ECE/CWR/NS can be safely ignored
+ * ACK must be set
+ * All other flags URG/PSH/RST/SYN/FIN must be unset
+ * 0x0FFF0000 = all TCP flags (confirm ACK=1, others zero)
+ * 0x01C00000 = NS/CWR/ECE (safe to ignore)
+ * 0x0E3F0000 = 0x0FFF0000 & ~0x01C00000
+ * must be 'pure' ACK, contain zero bytes of segment data
+ * options are ignored
+ */
+ if ((tcp_flag_word(tcph) &
+ (TCP_FLAG_ACK | TCP_FLAG_SYN)) != TCP_FLAG_ACK) {
+ continue;
+ } else if (((tcp_flag_word(tcph_check) &
+ cpu_to_be32(0x0E3F0000)) != TCP_FLAG_ACK) ||
+ ((seglen - 4 * tcph_check->doff) != 0)) {
+ pure_ack = false;
+ } else {
+ pure_ack = true;
+ }
+
+ /* if we find an ACK belonging to a different connection
+ * continue checking for other ACKs this round however
+ * restart checking from the other connection next time.
+ */
+ if (thisconn && (tcph_check->source != tcph->source ||
+ tcph_check->dest != tcph->dest)) {
+ thisconn = false;
+ }
+
+ /* new ack sequence must be greater
+ */
+ if (thisconn &&
+ (ntohl(tcph_check->ack_seq) > ntohl(tcph->ack_seq)))
+ continue;
+
+ /* DupACKs with an equal sequence number shouldn't be filtered,
+ * but we can filter if the triggering packet is a SACK
+ */
+ if (thisconn &&
+ (ntohl(tcph_check->ack_seq) == ntohl(tcph->ack_seq))) {
+ /* inspired by tcp_parse_options in tcp_input.c */
+ bool sack = false;
+ int length = (tcph->doff * 4) - sizeof(struct tcphdr);
+ const u8 *ptr = (const u8 *)(tcph + 1);
+
+ while (length > 0) {
+ int opcode = *ptr++;
+ int opsize;
+
+ if (opcode == TCPOPT_EOL)
+ break;
+ if (opcode == TCPOPT_NOP) {
+ length--;
+ continue;
+ }
+ opsize = *ptr++;
+ if (opsize < 2 || opsize > length)
+ break;
+ if (opcode == TCPOPT_SACK) {
+ sack = true;
+ break;
+ }
+ ptr += opsize - 2;
+ length -= opsize;
+ }
+ if (!sack)
+ continue;
+ }
+
+ /* somewhat complicated control flow for 'conservative'
+ * ACK filtering that aims to be more polite to slow-start and
+ * in the presence of packet loss.
+ * does not filter if there is one 'redundant' ACK in the queue.
+ * 'data' ACKs won't be filtered but do count as redundant ACKs.
+ */
+ if (thisconn) {
+ thisconn_seen_last = true;
+ /* if aggressive and this is a data ack we can skip
+ * checking it next time.
+ */
+ thisconn_checked_to = (aggressive && !pure_ack) ?
+ skb_check : skb_check_prev;
+ /* the first pure ack for this connection.
+ * record where it is, but only break if aggressive
+ * or already seen data ack from the same connection
+ */
+ if (pure_ack && !thisconn_ack) {
+ thisconn_ack = skb_check_prev;
+ if (aggressive || thisconn_redundant_seen)
+ break;
+ /* data ack or subsequent pure ack */
+ } else {
+ thisconn_redundant_seen = true;
+ /* this is the second ack for this connection
+ * break to filter the first pure ack
+ */
+ if (thisconn_ack)
+ break;
+ }
+ /* track packets from non-matching tcp connections that will
+ * need evaluation on the next run.
+ * if there are packets from both the matching connection and
+ * others that requre checking next run, track which was updated
+ * last and return the older of the two to ensure full coverage.
+ * if a non-matching pure ack has been seen, cannot skip any
+ * further on the next run so don't update.
+ */
+ } else if (!otherconn_ack_seen) {
+ thisconn_seen_last = false;
+ if (pure_ack) {
+ otherconn_ack_seen = true;
+ /* if aggressive we don't care about old data,
+ * start from the pure ack.
+ * otherwise if there is a previous data ack,
+ * start checking from it next time.
+ */
+ if (aggressive || !otherconn_checked_to)
+ otherconn_checked_to = skb_check_prev;
+ } else {
+ otherconn_checked_to = aggressive ?
+ skb_check : skb_check_prev;
+ }
+ }
+ }
+
+ /* skb_check is reused at this point
+ * it is the pure ACK to be filtered (if any)
+ */
+ skb_check = NULL;
+
+ /* next time start checking from the older/nearest to head of unfiltered
+ * but important tcp packets from this connection and other connections.
+ * if none seen, start after the last packet evaluated in the loop.
+ */
+ if (thisconn_checked_to && otherconn_checked_to)
+ flow->ackcheck = thisconn_seen_last ?
+ otherconn_checked_to : thisconn_checked_to;
+ else if (thisconn_checked_to)
+ flow->ackcheck = thisconn_checked_to;
+ else if (otherconn_checked_to)
+ flow->ackcheck = otherconn_checked_to;
+ else
+ flow->ackcheck = skb_check_prev;
+
+ /* if filtering, the pure ACK from the flow queue */
+ if (thisconn_ack && (aggressive || thisconn_redundant_seen)) {
+ if (PTR_ERR(thisconn_ack) == -1) {
+ skb_check = flow->head;
+ flow->head = flow->head->next;
+ } else {
+ skb_check = thisconn_ack->next;
+ thisconn_ack->next = thisconn_ack->next->next;
+ }
+ }
+
+ /* we just filtered that ack, fix up the list */
+ if (flow->ackcheck == skb_check)
+ flow->ackcheck = thisconn_ack;
+ /* check the entire flow queue next time */
+ if (PTR_ERR(flow->ackcheck) == -1)
+ flow->ackcheck = NULL;
+
+ return skb_check;
+}
+
+static inline u32 cake_overhead(struct cake_sched_data *q, u32 in)
+{
+ u32 out = in + q->rate_overhead;
+
+ if (q->rate_mpu && out < q->rate_mpu)
+ out = q->rate_mpu;
+
+ if (q->rate_flags & CAKE_FLAG_ATM) {
+ out += 47;
+ out /= 48;
+ out *= 53;
+ } else if (q->rate_flags & CAKE_FLAG_PTM) {
+ /* Add one byte per 64 bytes or part thereof.
+ * This is conservative and easier to calculate than the
+ * precise value.
+ */
+ out += (out / 64) + !!(out % 64);
+ }
+
+ return out;
+}
+
+static inline cobalt_time_t cake_ewma(cobalt_time_t avg, cobalt_time_t sample,
+ u32 shift)
+{
+ avg -= avg >> shift;
+ avg += sample >> shift;
+ return avg;
+}
+
+static inline void cake_heap_swap(struct cake_sched_data *q, u16 i, u16 j)
+{
+ struct cake_heap_entry ii = q->overflow_heap[i];
+ struct cake_heap_entry jj = q->overflow_heap[j];
+
+ q->overflow_heap[i] = jj;
+ q->overflow_heap[j] = ii;
+
+ q->tins[ii.t].overflow_idx[ii.b] = j;
+ q->tins[jj.t].overflow_idx[jj.b] = i;
+}
+
+static inline u32 cake_heap_get_backlog(const struct cake_sched_data *q, u16 i)
+{
+ struct cake_heap_entry ii = q->overflow_heap[i];
+
+ return q->tins[ii.t].backlogs[ii.b];
+}
+
+static void cake_heapify(struct cake_sched_data *q, u16 i)
+{
+ static const u32 a = CAKE_MAX_TINS * CAKE_QUEUES;
+ u32 m = i;
+ u32 mb = cake_heap_get_backlog(q, m);
+
+ while (m < a) {
+ u32 l = m + m + 1;
+ u32 r = l + 1;
+
+ if (l < a) {
+ u32 lb = cake_heap_get_backlog(q, l);
+
+ if (lb > mb) {
+ m = l;
+ mb = lb;
+ }
+ }
+
+ if (r < a) {
+ u32 rb = cake_heap_get_backlog(q, r);
+
+ if (rb > mb) {
+ m = r;
+ mb = rb;
+ }
+ }
+
+ if (m != i) {
+ cake_heap_swap(q, i, m);
+ i = m;
+ } else {
+ break;
+ }
+ }
+}
+
+static void cake_heapify_up(struct cake_sched_data *q, u16 i)
+{
+ while (i > 0 && i < CAKE_MAX_TINS * CAKE_QUEUES) {
+ u16 p = (i - 1) >> 1;
+ u32 ib = cake_heap_get_backlog(q, i);
+ u32 pb = cake_heap_get_backlog(q, p);
+
+ if (ib > pb) {
+ cake_heap_swap(q, i, p);
+ i = p;
+ } else {
+ break;
+ }
+ }
+}
+
+static int cake_advance_shaper(struct cake_sched_data *q,
+ struct cake_tin_data *b,
+ u32 len, u64 now, bool drop)
+{
+ s64 tdiff1, tdiff2, tdiff3, tdiff4;
+ /* charge packet bandwidth to this tin
+ * and to the global shaper.
+ */
+ if (q->rate_ns) {
+ len = cake_overhead(q, len);
+ tdiff1 = b->tin_time_next_packet - now;
+ tdiff2 = (len * (u64)b->tin_rate_ns) >> b->tin_rate_shft;
+ tdiff3 = (len * (u64)q->rate_ns) >> q->rate_shft;
+ tdiff4 = tdiff3 + (tdiff3 >> 1);
+
+ if (tdiff1 < 0)
+ b->tin_time_next_packet += tdiff2;
+ else if (tdiff1 < tdiff2)
+ b->tin_time_next_packet = now + tdiff2;
+
+ q->time_next_packet += tdiff3;
+ if (!drop)
+ q->failsafe_next_packet += tdiff4;
+ }
+ return len;
+}
+
+static unsigned int cake_drop(struct Qdisc *sch, struct sk_buff **to_free)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ struct sk_buff *skb;
+ u32 idx = 0, tin = 0, len;
+ struct cake_tin_data *b;
+ struct cake_flow *flow;
+ struct cake_heap_entry qq;
+ u64 now = cobalt_get_time();
+
+ if (!q->overflow_timeout) {
+ int i;
+ /* Build fresh max-heap */
+ for (i = CAKE_MAX_TINS * CAKE_QUEUES / 2; i >= 0; i--)
+ cake_heapify(q, i);
+ }
+ q->overflow_timeout = 65535;
+
+ /* select longest queue for pruning */
+ qq = q->overflow_heap[0];
+ tin = qq.t;
+ idx = qq.b;
+
+ b = &q->tins[tin];
+ flow = &b->flows[idx];
+ skb = dequeue_head(flow);
+ if (unlikely(!skb)) {
+ /* heap has gone wrong, rebuild it next time */
+ q->overflow_timeout = 0;
+ return idx + (tin << 16);
+ }
+
+ if (cobalt_queue_full(&flow->cvars, &b->cparams, now))
+ b->unresponsive_flow_count++;
+
+ len = qdisc_pkt_len(skb);
+ q->buffer_used -= skb->truesize;
+ b->backlogs[idx] -= len;
+ b->tin_backlog -= len;
+ sch->qstats.backlog -= len;
+ qdisc_tree_reduce_backlog(sch, 1, len);
+
+ b->tin_dropped++;
+ sch->qstats.drops++;
+
+ if (q->rate_flags & CAKE_FLAG_INGRESS)
+ cake_advance_shaper(q, b, len, now, true);
+
+ __qdisc_drop(skb, to_free);
+ sch->q.qlen--;
+ cake_heapify(q, 0);
+
+ return idx + (tin << 16);
+}
+
+static inline void cake_wash_diffserv(struct sk_buff *skb)
+{
+ switch (skb->protocol) {
+ case htons(ETH_P_IP):
+ ipv4_change_dsfield(ip_hdr(skb), INET_ECN_MASK, 0);
+ break;
+ case htons(ETH_P_IPV6):
+ ipv6_change_dsfield(ipv6_hdr(skb), INET_ECN_MASK, 0);
+ break;
+ default:
+ break;
+ };
+}
+
+static inline u8 cake_handle_diffserv(struct sk_buff *skb, u16 wash)
+{
+ u8 dscp;
+
+ switch (skb->protocol) {
+ case htons(ETH_P_IP):
+ dscp = ipv4_get_dsfield(ip_hdr(skb)) >> 2;
+ if (wash && dscp)
+ ipv4_change_dsfield(ip_hdr(skb), INET_ECN_MASK, 0);
+ return dscp;
+
+ case htons(ETH_P_IPV6):
+ dscp = ipv6_get_dsfield(ipv6_hdr(skb)) >> 2;
+ if (wash && dscp)
+ ipv6_change_dsfield(ipv6_hdr(skb), INET_ECN_MASK, 0);
+ return dscp;
+
+ case htons(ETH_P_ARP):
+ return 0x38; /* CS7 - Net Control */
+
+ default:
+ /* If there is no Diffserv field, treat as best-effort */
+ return 0;
+ };
+}
+
+static void cake_reconfigure(struct Qdisc *sch);
+
+static s32 cake_enqueue(struct sk_buff *skb, struct Qdisc *sch,
+ struct sk_buff **to_free)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ u32 idx, tin;
+ struct cake_tin_data *b;
+ struct cake_flow *flow;
+ /* signed len to handle corner case filtered ACK larger than trigger */
+ int len = qdisc_pkt_len(skb);
+ u64 now = cobalt_get_time();
+ struct sk_buff *ack = NULL;
+
+ /* extract the Diffserv Precedence field, if it exists */
+ /* and clear DSCP bits if washing */
+ if (q->tin_mode != CAKE_MODE_BESTEFFORT) {
+ tin = q->tin_index[cake_handle_diffserv(skb,
+ q->rate_flags & CAKE_FLAG_WASH)];
+ if (unlikely(tin >= q->tin_cnt))
+ tin = 0;
+ } else {
+ tin = 0;
+ if (q->rate_flags & CAKE_FLAG_WASH)
+ cake_wash_diffserv(skb);
+ }
+
+ b = &q->tins[tin];
+
+ /* choose flow to insert into */
+ idx = cake_hash(b, skb, q->flow_mode);
+ flow = &b->flows[idx];
+
+ /* ensure shaper state isn't stale */
+ if (!b->tin_backlog) {
+ if (b->tin_time_next_packet < now)
+ b->tin_time_next_packet = now;
+
+ if (!sch->q.qlen) {
+ if (q->time_next_packet < now) {
+ q->failsafe_next_packet = now;
+ q->time_next_packet = now;
+ } else if (q->time_next_packet > now && q->failsafe_next_packet > now) {
+ u64 next = min(q->time_next_packet,
+ q->failsafe_next_packet);
+ sch->qstats.overlimits++;
+ qdisc_watchdog_schedule_ns(&q->watchdog, next);
+ }
+ }
+ }
+
+ if (unlikely(len > b->max_skblen))
+ b->max_skblen = len;
+
+ /* Split GSO aggregates if they're likely to impair flow isolation
+ * or if we need to know individual packet sizes for framing overhead.
+ */
+
+ if (skb_is_gso(skb)) {
+ struct sk_buff *segs, *nskb;
+ netdev_features_t features = netif_skb_features(skb);
+ /* signed slen to handle corner case
+ * suppressed ACK larger than trigger
+ */
+ int slen = 0;
+
+ segs = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK);
+ if (IS_ERR_OR_NULL(segs))
+ return qdisc_drop(skb, sch, to_free);
+
+ while (segs) {
+ nskb = segs->next;
+ segs->next = NULL;
+ qdisc_skb_cb(segs)->pkt_len = segs->len;
+ cobalt_set_enqueue_time(segs, now);
+ flow_queue_add(flow, segs);
+
+ if (q->rate_flags & CAKE_FLAG_ACK_FILTER)
+ ack = cake_ack_filter(q, flow);
+
+ if (ack) {
+ b->ack_drops++;
+ sch->qstats.drops++;
+ b->bytes += ack->len;
+ slen += segs->len - ack->len;
+ q->buffer_used += segs->truesize -
+ ack->truesize;
+ if (q->rate_flags & CAKE_FLAG_INGRESS)
+ cake_advance_shaper(q, b,
+ qdisc_pkt_len(ack),
+ now, true);
+
+ qdisc_tree_reduce_backlog(sch, 1,
+ qdisc_pkt_len(ack));
+ consume_skb(ack);
+ } else {
+ sch->q.qlen++;
+ slen += segs->len;
+ q->buffer_used += segs->truesize;
+ }
+ b->packets++;
+ segs = nskb;
+ }
+ /* stats */
+ b->bytes += slen;
+ b->backlogs[idx] += slen;
+ b->tin_backlog += slen;
+ sch->qstats.backlog += slen;
+ q->avg_window_bytes += slen;
+
+ qdisc_tree_reduce_backlog(sch, 1, len);
+ consume_skb(skb);
+ } else {
+ /* not splitting */
+ cobalt_set_enqueue_time(skb, now);
+ flow_queue_add(flow, skb);
+
+ if (q->rate_flags & CAKE_FLAG_ACK_FILTER)
+ ack = cake_ack_filter(q, flow);
+
+ if (ack) {
+ b->ack_drops++;
+ sch->qstats.drops++;
+ b->bytes += qdisc_pkt_len(ack);
+ len -= qdisc_pkt_len(ack);
+ q->buffer_used += skb->truesize - ack->truesize;
+ if (q->rate_flags & CAKE_FLAG_INGRESS)
+ cake_advance_shaper(q, b, ack->len, now, true);
+
+ qdisc_tree_reduce_backlog(sch, 1, qdisc_pkt_len(ack));
+ consume_skb(ack);
+ } else {
+ sch->q.qlen++;
+ q->buffer_used += skb->truesize;
+ }
+ /* stats */
+ b->packets++;
+ b->bytes += len;
+ b->backlogs[idx] += len;
+ b->tin_backlog += len;
+ sch->qstats.backlog += len;
+ q->avg_window_bytes += len;
+ }
+
+ if (q->overflow_timeout)
+ cake_heapify_up(q, b->overflow_idx[idx]);
+
+ /* incoming bandwidth capacity estimate */
+ if (q->rate_flags & CAKE_FLAG_AUTORATE_INGRESS) {
+ u64 packet_interval = now - q->last_packet_time;
+
+ if (packet_interval > NSEC_PER_SEC)
+ packet_interval = NSEC_PER_SEC;
+
+ /* filter out short-term bursts, eg. wifi aggregation */
+ q->avg_packet_interval = cake_ewma(q->avg_packet_interval,
+ packet_interval,
+ packet_interval > q->avg_packet_interval ? 2 : 8);
+
+ q->last_packet_time = now;
+
+ if (packet_interval > q->avg_packet_interval) {
+ u64 window_interval = now - q->avg_window_begin;
+ u64 b = q->avg_window_bytes * (u64)NSEC_PER_SEC;
+
+ do_div(b, window_interval);
+ q->avg_peak_bandwidth =
+ cake_ewma(q->avg_peak_bandwidth, b,
+ b > q->avg_peak_bandwidth ? 2 : 8);
+ q->avg_window_bytes = 0;
+ q->avg_window_begin = now;
+
+ if (q->rate_flags & CAKE_FLAG_AUTORATE_INGRESS &&
+ now - q->last_reconfig_time >
+ (NSEC_PER_SEC / 4)) {
+ q->rate_bps = (q->avg_peak_bandwidth * 15) >> 4;
+ cake_reconfigure(sch);
+ }
+ }
+ } else {
+ q->avg_window_bytes = 0;
+ q->last_packet_time = now;
+ }
+
+ /* flowchain */
+ if (!flow->set || flow->set == CAKE_SET_DECAYING) {
+ struct cake_host *srchost = &b->hosts[flow->srchost];
+ struct cake_host *dsthost = &b->hosts[flow->dsthost];
+ u16 host_load = 1;
+
+ if (!flow->set) {
+ list_add_tail(&flow->flowchain, &b->new_flows);
+ } else {
+ b->decaying_flow_count--;
+ list_move_tail(&flow->flowchain, &b->new_flows);
+ }
+ flow->set = CAKE_SET_SPARSE;
+ b->sparse_flow_count++;
+
+ if (cake_dsrc(q->flow_mode))
+ host_load = max(host_load, srchost->srchost_refcnt);
+
+ if (cake_ddst(q->flow_mode))
+ host_load = max(host_load, dsthost->dsthost_refcnt);
+
+ flow->deficit = (b->flow_quantum * quantum_div[host_load]) >> 16;
+ } else if (flow->set == CAKE_SET_SPARSE_WAIT) {
+ /* this flow was empty, accounted as a sparse flow, but actually
+ * in the bulk rotation.
+ */
+ flow->set = CAKE_SET_BULK;
+ b->sparse_flow_count--;
+ b->bulk_flow_count++;
+ }
+
+ if (q->buffer_used > q->buffer_max_used)
+ q->buffer_max_used = q->buffer_used;
+
+ if (q->buffer_used > q->buffer_limit) {
+ u32 dropped = 0;
+
+ while (q->buffer_used > q->buffer_limit) {
+ dropped++;
+ cake_drop(sch, to_free);
+ }
+ b->drop_overlimit += dropped;
+ }
+ return NET_XMIT_SUCCESS;
+}
+
+static struct sk_buff *cake_dequeue_one(struct Qdisc *sch)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ struct cake_tin_data *b = &q->tins[q->cur_tin];
+ struct cake_flow *flow = &b->flows[q->cur_flow];
+ struct sk_buff *skb = NULL;
+ u32 len;
+
+ /* WARN_ON(flow != container_of(vars, struct cake_flow, cvars)); */
+
+ if (flow->head) {
+ skb = dequeue_head(flow);
+ len = qdisc_pkt_len(skb);
+ b->backlogs[q->cur_flow] -= len;
+ b->tin_backlog -= len;
+ sch->qstats.backlog -= len;
+ q->buffer_used -= skb->truesize;
+ sch->q.qlen--;
+
+ if (q->overflow_timeout)
+ cake_heapify(q, b->overflow_idx[q->cur_flow]);
+ }
+ return skb;
+}
+
+/* Discard leftover packets from a tin no longer in use. */
+static void cake_clear_tin(struct Qdisc *sch, u16 tin)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ struct sk_buff *skb;
+
+ q->cur_tin = tin;
+ for (q->cur_flow = 0; q->cur_flow < CAKE_QUEUES; q->cur_flow++)
+ while (!!(skb = cake_dequeue_one(sch)))
+ kfree_skb(skb);
+}
+
+static struct sk_buff *cake_dequeue(struct Qdisc *sch)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ struct sk_buff *skb;
+ struct cake_tin_data *b = &q->tins[q->cur_tin];
+ struct cake_flow *flow;
+ struct cake_host *srchost, *dsthost;
+ struct list_head *head;
+ u32 len;
+ u16 host_load;
+ cobalt_time_t now = ktime_get_ns();
+ cobalt_time_t delay;
+ bool first_flow = true;
+
+begin:
+ if (!sch->q.qlen)
+ return NULL;
+
+ /* global hard shaper */
+ if (q->time_next_packet > now && q->failsafe_next_packet > now) {
+ u64 next = min(q->time_next_packet, q->failsafe_next_packet);
+ sch->qstats.overlimits++;
+ qdisc_watchdog_schedule_ns(&q->watchdog, next);
+ return NULL;
+ }
+
+ /* Choose a class to work on. */
+ if (!q->rate_ns) {
+ /* In unlimited mode, can't rely on shaper timings, just balance
+ * with DRR
+ */
+ while (b->tin_deficit < 0 ||
+ !(b->sparse_flow_count + b->bulk_flow_count)) {
+ if (b->tin_deficit <= 0)
+ b->tin_deficit += b->tin_quantum_band;
+
+ q->cur_tin++;
+ b++;
+ if (q->cur_tin >= q->tin_cnt) {
+ q->cur_tin = 0;
+ b = q->tins;
+ }
+ }
+ } else {
+ /* In shaped mode, choose:
+ * - Highest-priority tin with queue and meeting schedule, or
+ * - The earliest-scheduled tin with queue.
+ */
+ int tin, best_tin = 0;
+ s64 best_time = 0xFFFFFFFFFFFFUL;
+
+ for (tin = 0; tin < q->tin_cnt; tin++) {
+ b = q->tins + tin;
+ if ((b->sparse_flow_count + b->bulk_flow_count) > 0) {
+ s64 tdiff = b->tin_time_next_packet - now;
+
+ if (tdiff <= 0 || tdiff <= best_time) {
+ best_time = tdiff;
+ best_tin = tin;
+ }
+ }
+ }
+
+ q->cur_tin = best_tin;
+ b = q->tins + best_tin;
+ }
+
+retry:
+ /* service this class */
+ head = &b->decaying_flows;
+ if (!first_flow || list_empty(head)) {
+ head = &b->new_flows;
+ if (list_empty(head)) {
+ head = &b->old_flows;
+ if (unlikely(list_empty(head))) {
+ head = &b->decaying_flows;
+ if (unlikely(list_empty(head)))
+ goto begin;
+ }
+ }
+ }
+ flow = list_first_entry(head, struct cake_flow, flowchain);
+ q->cur_flow = flow - b->flows;
+ first_flow = false;
+
+ /* triple isolation (modified DRR++) */
+ srchost = &b->hosts[flow->srchost];
+ dsthost = &b->hosts[flow->dsthost];
+ host_load = 1;
+
+ if (cake_dsrc(q->flow_mode))
+ host_load = max(host_load, srchost->srchost_refcnt);
+
+ if (cake_ddst(q->flow_mode))
+ host_load = max(host_load, dsthost->dsthost_refcnt);
+
+ WARN_ON(host_load > CAKE_QUEUES);
+
+ /* flow isolation (DRR++) */
+ if (flow->deficit <= 0) {
+ flow->deficit += (b->flow_quantum * quantum_div[host_load] +
+ (prandom_u32() >> 16)) >> 16;
+ list_move_tail(&flow->flowchain, &b->old_flows);
+
+ /* Keep all flows with deficits out of the sparse and decaying
+ * rotations. No non-empty flow can go into the decaying
+ * rotation, so they can't get deficits
+ */
+ if (flow->set == CAKE_SET_SPARSE) {
+ if (flow->head) {
+ b->sparse_flow_count--;
+ b->bulk_flow_count++;
+ flow->set = CAKE_SET_BULK;
+ } else {
+ /* we've moved it to the bulk rotation for
+ * correct deficit accounting but we still want
+ * to count it as a sparse flow, not a bulk one.
+ */
+ flow->set = CAKE_SET_SPARSE_WAIT;
+ }
+ }
+ goto retry;
+ }
+
+ /* Retrieve a packet via the AQM */
+ while (1) {
+ skb = cake_dequeue_one(sch);
+ if (!skb) {
+ /* this queue was actually empty */
+ if (cobalt_queue_empty(&flow->cvars, &b->cparams, now))
+ b->unresponsive_flow_count--;
+
+ if (flow->cvars.p_drop || flow->cvars.count ||
+ (now - flow->cvars.drop_next) < 0) {
+ /* keep in the flowchain until the state has
+ * decayed to rest
+ */
+ list_move_tail(&flow->flowchain,
+ &b->decaying_flows);
+ if (flow->set == CAKE_SET_BULK) {
+ b->bulk_flow_count--;
+ b->decaying_flow_count++;
+ } else if (flow->set == CAKE_SET_SPARSE ||
+ flow->set == CAKE_SET_SPARSE_WAIT) {
+ b->sparse_flow_count--;
+ b->decaying_flow_count++;
+ }
+ flow->set = CAKE_SET_DECAYING;
+ } else {
+ /* remove empty queue from the flowchain */
+ list_del_init(&flow->flowchain);
+ if (flow->set == CAKE_SET_SPARSE ||
+ flow->set == CAKE_SET_SPARSE_WAIT)
+ b->sparse_flow_count--;
+ else if (flow->set == CAKE_SET_BULK)
+ b->bulk_flow_count--;
+ else
+ b->decaying_flow_count--;
+
+ flow->set = CAKE_SET_NONE;
+ srchost->srchost_refcnt--;
+ dsthost->dsthost_refcnt--;
+ }
+ goto begin;
+ }
+
+ /* Last packet in queue may be marked, shouldn't be dropped */
+ if (!cobalt_should_drop(&flow->cvars, &b->cparams, now, skb) ||
+ !flow->head)
+ break;
+
+ /* drop this packet, get another one */
+ if (q->rate_flags & CAKE_FLAG_INGRESS) {
+ len = cake_advance_shaper(q, b,
+ qdisc_pkt_len(skb),
+ now, true);
+ flow->deficit -= len;
+ b->tin_deficit -= len;
+ }
+ b->tin_dropped++;
+ qdisc_tree_reduce_backlog(sch, 1, qdisc_pkt_len(skb));
+ qdisc_qstats_drop(sch);
+ kfree_skb(skb);
+ if (q->rate_flags & CAKE_FLAG_INGRESS)
+ goto retry;
+ }
+
+ b->tin_ecn_mark += !!flow->cvars.ecn_marked;
+ qdisc_bstats_update(sch, skb);
+
+ len = cake_overhead(q, qdisc_pkt_len(skb));
+ flow->deficit -= len;
+ b->tin_deficit -= len;
+
+ /* collect delay stats */
+ delay = now - cobalt_get_enqueue_time(skb);
+ b->avge_delay = cake_ewma(b->avge_delay, delay, 8);
+ b->peak_delay = cake_ewma(b->peak_delay, delay,
+ delay > b->peak_delay ? 2 : 8);
+ b->base_delay = cake_ewma(b->base_delay, delay,
+ delay < b->base_delay ? 2 : 8);
+
+ cake_advance_shaper(q, b, len, now, false);
+ if (q->time_next_packet > now && sch->q.qlen) {
+ u64 next = min(q->time_next_packet, q->failsafe_next_packet);
+ qdisc_watchdog_schedule_ns(&q->watchdog, next);
+ } else if (!sch->q.qlen) {
+ int i;
+
+ for (i = 0; i < q->tin_cnt; i++) {
+ if (q->tins[i].decaying_flow_count) {
+ u64 next = now + q->tins[i].cparams.target;
+ qdisc_watchdog_schedule_ns(&q->watchdog, next);
+ break;
+ }
+ }
+ }
+
+ if (q->overflow_timeout)
+ q->overflow_timeout--;
+
+ return skb;
+}
+
+static void cake_reset(struct Qdisc *sch)
+{
+ u32 c;
+
+ for (c = 0; c < CAKE_MAX_TINS; c++)
+ cake_clear_tin(sch, c);
+}
+
+static const struct nla_policy cake_policy[TCA_CAKE_MAX + 1] = {
+ [TCA_CAKE_BASE_RATE] = { .type = NLA_U32 },
+ [TCA_CAKE_DIFFSERV_MODE] = { .type = NLA_U32 },
+ [TCA_CAKE_ATM] = { .type = NLA_U32 },
+ [TCA_CAKE_FLOW_MODE] = { .type = NLA_U32 },
+ [TCA_CAKE_OVERHEAD] = { .type = NLA_S32 },
+ [TCA_CAKE_RTT] = { .type = NLA_U32 },
+ [TCA_CAKE_TARGET] = { .type = NLA_U32 },
+ [TCA_CAKE_AUTORATE] = { .type = NLA_U32 },
+ [TCA_CAKE_MEMORY] = { .type = NLA_U32 },
+ [TCA_CAKE_NAT] = { .type = NLA_U32 },
+ [TCA_CAKE_ETHERNET] = { .type = NLA_U32 },
+ [TCA_CAKE_WASH] = { .type = NLA_U32 },
+ [TCA_CAKE_MPU] = { .type = NLA_U32 },
+ [TCA_CAKE_INGRESS] = { .type = NLA_U32 },
+ [TCA_CAKE_ACK_FILTER] = { .type = NLA_U32 },
+};
+
+static void cake_set_rate(struct cake_tin_data *b, u64 rate, u32 mtu,
+ cobalt_time_t ns_target, cobalt_time_t rtt_est_ns)
+{
+ /* convert byte-rate into time-per-byte
+ * so it will always unwedge in reasonable time.
+ */
+ static const u64 MIN_RATE = 64;
+ u64 rate_ns = 0;
+ u8 rate_shft = 0;
+ cobalt_time_t byte_target_ns;
+ u32 byte_target = mtu + (mtu >> 1);
+
+ b->flow_quantum = 1514;
+ if (rate) {
+ b->flow_quantum = max(min(rate >> 12, 1514ULL), 300ULL);
+ rate_shft = 32;
+ rate_ns = ((u64)NSEC_PER_SEC) << rate_shft;
+ do_div(rate_ns, max(MIN_RATE, rate));
+ while (!!(rate_ns >> 32)) {
+ rate_ns >>= 1;
+ rate_shft--;
+ }
+ } /* else unlimited, ie. zero delay */
+
+ b->tin_rate_bps = rate;
+ b->tin_rate_ns = rate_ns;
+ b->tin_rate_shft = rate_shft;
+
+ byte_target_ns = (byte_target * rate_ns) >> rate_shft;
+
+ b->cparams.target = max(byte_target_ns, ns_target);
+ b->cparams.interval = max(rtt_est_ns +
+ b->cparams.target - ns_target,
+ b->cparams.target * 2);
+ b->cparams.p_inc = 1 << 24; /* 1/256 */
+ b->cparams.p_dec = 1 << 20; /* 1/4096 */
+}
+
+static int cake_config_besteffort(struct Qdisc *sch)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ struct cake_tin_data *b = &q->tins[0];
+ u32 rate = q->rate_bps;
+ u32 mtu = psched_mtu(qdisc_dev(sch));
+
+ q->tin_cnt = 1;
+
+ q->tin_index = besteffort;
+ q->tin_order = normal_order;
+
+ cake_set_rate(b, rate, mtu, US2TIME(q->target), US2TIME(q->interval));
+ b->tin_quantum_band = 65535;
+ b->tin_quantum_prio = 65535;
+
+ return 0;
+}
+
+static int cake_config_precedence(struct Qdisc *sch)
+{
+ /* convert high-level (user visible) parameters into internal format */
+ struct cake_sched_data *q = qdisc_priv(sch);
+ u32 rate = q->rate_bps;
+ u32 mtu = psched_mtu(qdisc_dev(sch));
+ u32 quantum1 = 256;
+ u32 quantum2 = 256;
+ u32 i;
+
+ q->tin_cnt = 8;
+ q->tin_index = precedence;
+ q->tin_order = normal_order;
+
+ for (i = 0; i < q->tin_cnt; i++) {
+ struct cake_tin_data *b = &q->tins[i];
+
+ cake_set_rate(b, rate, mtu, US2TIME(q->target),
+ US2TIME(q->interval));
+
+ b->tin_quantum_prio = max_t(u16, 1U, quantum1);
+ b->tin_quantum_band = max_t(u16, 1U, quantum2);
+
+ /* calculate next class's parameters */
+ rate *= 7;
+ rate >>= 3;
+
+ quantum1 *= 3;
+ quantum1 >>= 1;
+
+ quantum2 *= 7;
+ quantum2 >>= 3;
+ }
+
+ return 0;
+}
+
+/* List of known Diffserv codepoints:
+ *
+ * Least Effort (CS1)
+ * Best Effort (CS0)
+ * Max Reliability & LLT "Lo" (TOS1)
+ * Max Throughput (TOS2)
+ * Min Delay (TOS4)
+ * LLT "La" (TOS5)
+ * Assured Forwarding 1 (AF1x) - x3
+ * Assured Forwarding 2 (AF2x) - x3
+ * Assured Forwarding 3 (AF3x) - x3
+ * Assured Forwarding 4 (AF4x) - x3
+ * Precedence Class 2 (CS2)
+ * Precedence Class 3 (CS3)
+ * Precedence Class 4 (CS4)
+ * Precedence Class 5 (CS5)
+ * Precedence Class 6 (CS6)
+ * Precedence Class 7 (CS7)
+ * Voice Admit (VA)
+ * Expedited Forwarding (EF)
+
+ * Total 25 codepoints.
+ */
+
+/* List of traffic classes in RFC 4594:
+ * (roughly descending order of contended priority)
+ * (roughly ascending order of uncontended throughput)
+ *
+ * Network Control (CS6,CS7) - routing traffic
+ * Telephony (EF,VA) - aka. VoIP streams
+ * Signalling (CS5) - VoIP setup
+ * Multimedia Conferencing (AF4x) - aka. video calls
+ * Realtime Interactive (CS4) - eg. games
+ * Multimedia Streaming (AF3x) - eg. YouTube, NetFlix, Twitch
+ * Broadcast Video (CS3)
+ * Low Latency Data (AF2x,TOS4) - eg. database
+ * Ops, Admin, Management (CS2,TOS1) - eg. ssh
+ * Standard Service (CS0 & unrecognised codepoints)
+ * High Throughput Data (AF1x,TOS2) - eg. web traffic
+ * Low Priority Data (CS1) - eg. BitTorrent
+
+ * Total 12 traffic classes.
+ */
+
+static int cake_config_diffserv8(struct Qdisc *sch)
+{
+/* Pruned list of traffic classes for typical applications:
+ *
+ * Network Control (CS6, CS7)
+ * Minimum Latency (EF, VA, CS5, CS4)
+ * Interactive Shell (CS2, TOS1)
+ * Low Latency Transactions (AF2x, TOS4)
+ * Video Streaming (AF4x, AF3x, CS3)
+ * Bog Standard (CS0 etc.)
+ * High Throughput (AF1x, TOS2)
+ * Background Traffic (CS1)
+ *
+ * Total 8 traffic classes.
+ */
+
+ struct cake_sched_data *q = qdisc_priv(sch);
+ u32 rate = q->rate_bps;
+ u32 mtu = psched_mtu(qdisc_dev(sch));
+ u32 quantum1 = 256;
+ u32 quantum2 = 256;
+ u32 i;
+
+ q->tin_cnt = 8;
+
+ /* codepoint to class mapping */
+ q->tin_index = diffserv8;
+ q->tin_order = normal_order;
+
+ /* class characteristics */
+ for (i = 0; i < q->tin_cnt; i++) {
+ struct cake_tin_data *b = &q->tins[i];
+
+ cake_set_rate(b, rate, mtu, US2TIME(q->target),
+ US2TIME(q->interval));
+
+ b->tin_quantum_prio = max_t(u16, 1U, quantum1);
+ b->tin_quantum_band = max_t(u16, 1U, quantum2);
+
+ /* calculate next class's parameters */
+ rate *= 7;
+ rate >>= 3;
+
+ quantum1 *= 3;
+ quantum1 >>= 1;
+
+ quantum2 *= 7;
+ quantum2 >>= 3;
+ }
+
+ return 0;
+}
+
+static int cake_config_diffserv4(struct Qdisc *sch)
+{
+/* Further pruned list of traffic classes for four-class system:
+ *
+ * Latency Sensitive (CS7, CS6, EF, VA, CS5, CS4)
+ * Streaming Media (AF4x, AF3x, CS3, AF2x, TOS4, CS2, TOS1)
+ * Best Effort (CS0, AF1x, TOS2, and those not specified)
+ * Background Traffic (CS1)
+ *
+ * Total 4 traffic classes.
+ */
+
+ struct cake_sched_data *q = qdisc_priv(sch);
+ u32 rate = q->rate_bps;
+ u32 mtu = psched_mtu(qdisc_dev(sch));
+ u32 quantum = 1024;
+
+ q->tin_cnt = 4;
+
+ /* codepoint to class mapping */
+ q->tin_index = diffserv4;
+ q->tin_order = bulk_order;
+
+ /* class characteristics */
+ cake_set_rate(&q->tins[0], rate, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+ cake_set_rate(&q->tins[1], rate >> 4, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+ cake_set_rate(&q->tins[2], rate >> 1, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+ cake_set_rate(&q->tins[3], rate >> 2, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+
+ /* priority weights */
+ q->tins[0].tin_quantum_prio = quantum;
+ q->tins[1].tin_quantum_prio = quantum >> 4;
+ q->tins[2].tin_quantum_prio = quantum << 2;
+ q->tins[3].tin_quantum_prio = quantum << 4;
+
+ /* bandwidth-sharing weights */
+ q->tins[0].tin_quantum_band = quantum;
+ q->tins[1].tin_quantum_band = quantum >> 4;
+ q->tins[2].tin_quantum_band = quantum >> 1;
+ q->tins[3].tin_quantum_band = quantum >> 2;
+
+ return 0;
+}
+
+static int cake_config_diffserv3(struct Qdisc *sch)
+{
+/* Simplified Diffserv structure with 3 tins.
+ * Low Priority (CS1)
+ * Best Effort
+ * Latency Sensitive (TOS4, VA, EF, CS6, CS7)
+ */
+ struct cake_sched_data *q = qdisc_priv(sch);
+ u32 rate = q->rate_bps;
+ u32 mtu = psched_mtu(qdisc_dev(sch));
+ u32 quantum = 1024;
+
+ q->tin_cnt = 3;
+
+ /* codepoint to class mapping */
+ q->tin_index = diffserv3;
+ q->tin_order = bulk_order;
+
+ /* class characteristics */
+ cake_set_rate(&q->tins[0], rate, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+ cake_set_rate(&q->tins[1], rate >> 4, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+ cake_set_rate(&q->tins[2], rate >> 2, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+
+ /* priority weights */
+ q->tins[0].tin_quantum_prio = quantum;
+ q->tins[1].tin_quantum_prio = quantum >> 4;
+ q->tins[2].tin_quantum_prio = quantum << 4;
+
+ /* bandwidth-sharing weights */
+ q->tins[0].tin_quantum_band = quantum;
+ q->tins[1].tin_quantum_band = quantum >> 4;
+ q->tins[2].tin_quantum_band = quantum >> 2;
+
+ return 0;
+}
+
+static int cake_config_diffserv_llt(struct Qdisc *sch)
+{
+/* Diffserv structure specialised for Latency-Loss-Tradeoff spec.
+ * Loss Sensitive (TOS1, TOS2)
+ * Best Effort
+ * Latency Sensitive (TOS4, TOS5, VA, EF)
+ * Low Priority (CS1)
+ * Network Control (CS6, CS7)
+ */
+ struct cake_sched_data *q = qdisc_priv(sch);
+ u32 rate = q->rate_bps;
+ u32 mtu = psched_mtu(qdisc_dev(sch));
+
+ q->tin_cnt = 5;
+
+ /* codepoint to class mapping */
+ q->tin_index = diffserv_llt;
+ q->tin_order = normal_order;
+
+ /* class characteristics */
+ cake_set_rate(&q->tins[5], rate, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+
+ cake_set_rate(&q->tins[0], rate / 3, mtu,
+ US2TIME(q->target * 4), US2TIME(q->interval * 4));
+ cake_set_rate(&q->tins[1], rate / 3, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+ cake_set_rate(&q->tins[2], rate / 3, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+ cake_set_rate(&q->tins[3], rate >> 4, mtu,
+ US2TIME(q->target), US2TIME(q->interval));
+ cake_set_rate(&q->tins[4], rate >> 4, mtu,
+ US2TIME(q->target * 4), US2TIME(q->interval * 4));
+
+ /* priority weights */
+ q->tins[0].tin_quantum_prio = 2048;
+ q->tins[1].tin_quantum_prio = 2048;
+ q->tins[2].tin_quantum_prio = 2048;
+ q->tins[3].tin_quantum_prio = 16384;
+ q->tins[4].tin_quantum_prio = 32768;
+
+ /* bandwidth-sharing weights */
+ q->tins[0].tin_quantum_band = 2048;
+ q->tins[1].tin_quantum_band = 2048;
+ q->tins[2].tin_quantum_band = 2048;
+ q->tins[3].tin_quantum_band = 256;
+ q->tins[4].tin_quantum_band = 16;
+
+ return 5;
+}
+
+static void cake_reconfigure(struct Qdisc *sch)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ int c, ft;
+
+ switch (q->tin_mode) {
+ case CAKE_MODE_BESTEFFORT:
+ ft = cake_config_besteffort(sch);
+ break;
+
+ case CAKE_MODE_PRECEDENCE:
+ ft = cake_config_precedence(sch);
+ break;
+
+ case CAKE_MODE_DIFFSERV8:
+ ft = cake_config_diffserv8(sch);
+ break;
+
+ case CAKE_MODE_DIFFSERV4:
+ ft = cake_config_diffserv4(sch);
+ break;
+
+ case CAKE_MODE_LLT:
+ ft = cake_config_diffserv_llt(sch);
+ break;
+
+ case CAKE_MODE_DIFFSERV3:
+ default:
+ ft = cake_config_diffserv3(sch);
+ break;
+ };
+
+ for (c = q->tin_cnt; c < CAKE_MAX_TINS; c++)
+ cake_clear_tin(sch, c);
+
+ q->rate_ns = q->tins[ft].tin_rate_ns;
+ q->rate_shft = q->tins[ft].tin_rate_shft;
+
+ if (q->buffer_config_limit) {
+ q->buffer_limit = q->buffer_config_limit;
+ } else if (q->rate_bps) {
+ u64 t = (u64)q->rate_bps * q->interval;
+
+ do_div(t, USEC_PER_SEC / 4);
+ q->buffer_limit = max_t(u32, t, 4U << 20);
+ } else {
+ q->buffer_limit = ~0;
+ }
+
+ sch->flags &= ~TCQ_F_CAN_BYPASS;
+
+ q->buffer_limit = min(q->buffer_limit, max(sch->limit * psched_mtu(qdisc_dev(sch)), q->buffer_config_limit));
+}
+
+static int cake_change(struct Qdisc *sch, struct nlattr *opt)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ struct nlattr *tb[TCA_CAKE_MAX + 1];
+ int err;
+
+ if (!opt)
+ return -EINVAL;
+
+ err = nla_parse_nested(tb, TCA_CAKE_MAX, opt, cake_policy, NULL);
+ if (err < 0)
+ return err;
+
+ if (tb[TCA_CAKE_BASE_RATE])
+ q->rate_bps = nla_get_u32(tb[TCA_CAKE_BASE_RATE]);
+
+ if (tb[TCA_CAKE_DIFFSERV_MODE])
+ q->tin_mode = nla_get_u32(tb[TCA_CAKE_DIFFSERV_MODE]);
+
+ if (tb[TCA_CAKE_ATM]) {
+ q->rate_flags &= ~(CAKE_FLAG_ATM | CAKE_FLAG_PTM);
+ q->rate_flags |= nla_get_u32(tb[TCA_CAKE_ATM]) &
+ (CAKE_FLAG_ATM | CAKE_FLAG_PTM);
+ }
+
+ if (tb[TCA_CAKE_WASH]) {
+ if (!!nla_get_u32(tb[TCA_CAKE_WASH]))
+ q->rate_flags |= CAKE_FLAG_WASH;
+ else
+ q->rate_flags &= ~CAKE_FLAG_WASH;
+ }
+
+ if (tb[TCA_CAKE_FLOW_MODE])
+ q->flow_mode = nla_get_u32(tb[TCA_CAKE_FLOW_MODE]);
+
+ if (tb[TCA_CAKE_NAT]) {
+ q->flow_mode &= ~CAKE_FLOW_NAT_FLAG;
+ q->flow_mode |= CAKE_FLOW_NAT_FLAG *
+ !!nla_get_u32(tb[TCA_CAKE_NAT]);
+ }
+
+ if (tb[TCA_CAKE_OVERHEAD]) {
+ if (tb[TCA_CAKE_ETHERNET])
+ q->rate_overhead = -(nla_get_s32(tb[TCA_CAKE_ETHERNET]));
+ else
+ q->rate_overhead = -(qdisc_dev(sch)->hard_header_len);
+
+ q->rate_overhead += nla_get_s32(tb[TCA_CAKE_OVERHEAD]);
+ }
+
+ if (tb[TCA_CAKE_MPU])
+ q->rate_mpu = nla_get_u32(tb[TCA_CAKE_MPU]);
+
+ if (tb[TCA_CAKE_RTT]) {
+ q->interval = nla_get_u32(tb[TCA_CAKE_RTT]);
+
+ if (!q->interval)
+ q->interval = 1;
+ }
+
+ if (tb[TCA_CAKE_TARGET]) {
+ q->target = nla_get_u32(tb[TCA_CAKE_TARGET]);
+
+ if (!q->target)
+ q->target = 1;
+ }
+
+ if (tb[TCA_CAKE_AUTORATE]) {
+ if (!!nla_get_u32(tb[TCA_CAKE_AUTORATE]))
+ q->rate_flags |= CAKE_FLAG_AUTORATE_INGRESS;
+ else
+ q->rate_flags &= ~CAKE_FLAG_AUTORATE_INGRESS;
+ }
+
+ if (tb[TCA_CAKE_INGRESS]) {
+ if (!!nla_get_u32(tb[TCA_CAKE_INGRESS]))
+ q->rate_flags |= CAKE_FLAG_INGRESS;
+ else
+ q->rate_flags &= ~CAKE_FLAG_INGRESS;
+ }
+
+ if (tb[TCA_CAKE_ACK_FILTER]) {
+ q->rate_flags &= ~(CAKE_FLAG_ACK_FILTER |
+ CAKE_FLAG_ACK_AGGRESSIVE);
+ q->rate_flags |= nla_get_u32(tb[TCA_CAKE_ACK_FILTER]) &
+ (CAKE_FLAG_ACK_FILTER |
+ CAKE_FLAG_ACK_AGGRESSIVE);
+ }
+
+ if (tb[TCA_CAKE_MEMORY])
+ q->buffer_config_limit = nla_get_s32(tb[TCA_CAKE_MEMORY]);
+
+ if (q->tins) {
+ sch_tree_lock(sch);
+ cake_reconfigure(sch);
+ sch_tree_unlock(sch);
+ }
+
+ return 0;
+}
+
+static void cake_free(void *addr)
+{
+ if (addr)
+ kvfree(addr);
+}
+
+static void cake_destroy(struct Qdisc *sch)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+
+ qdisc_watchdog_cancel(&q->watchdog);
+
+ if (q->tins)
+ cake_free(q->tins);
+}
+
+static int cake_init(struct Qdisc *sch, struct nlattr *opt)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ int i, j;
+
+ sch->limit = 10240;
+ q->tin_mode = CAKE_MODE_DIFFSERV3;
+ q->flow_mode = CAKE_FLOW_TRIPLE;
+
+ q->rate_bps = 0; /* unlimited by default */
+
+ q->interval = 100000; /* 100ms default */
+ q->target = 5000; /* 5ms: codel RFC argues
+ * for 5 to 10% of interval
+ */
+
+ q->cur_tin = 0;
+ q->cur_flow = 0;
+
+ if (opt) {
+ int err = cake_change(sch, opt);
+
+ if (err)
+ return err;
+ }
+
+ qdisc_watchdog_init(&q->watchdog, sch);
+
+ quantum_div[0] = ~0;
+ for (i = 1; i <= CAKE_QUEUES; i++)
+ quantum_div[i] = 65535 / i;
+
+ q->tins = kvzalloc(CAKE_MAX_TINS * sizeof(struct cake_tin_data),
+ GFP_KERNEL);
+ if (!q->tins)
+ goto nomem;
+
+ for (i = 0; i < CAKE_MAX_TINS; i++) {
+ struct cake_tin_data *b = q->tins + i;
+
+ INIT_LIST_HEAD(&b->new_flows);
+ INIT_LIST_HEAD(&b->old_flows);
+ INIT_LIST_HEAD(&b->decaying_flows);
+ b->sparse_flow_count = 0;
+ b->bulk_flow_count = 0;
+ b->decaying_flow_count = 0;
+
+ for (j = 0; j < CAKE_QUEUES; j++) {
+ struct cake_flow *flow = b->flows + j;
+ u32 k = j * CAKE_MAX_TINS + i;
+
+ INIT_LIST_HEAD(&flow->flowchain);
+ cobalt_vars_init(&flow->cvars);
+
+ q->overflow_heap[k].t = i;
+ q->overflow_heap[k].b = j;
+ b->overflow_idx[j] = k;
+ }
+ }
+
+ cake_reconfigure(sch);
+ q->avg_peak_bandwidth = q->rate_bps;
+ return 0;
+
+nomem:
+ cake_destroy(sch);
+ return -ENOMEM;
+}
+
+static int cake_dump(struct Qdisc *sch, struct sk_buff *skb)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ struct nlattr *opts;
+
+ opts = nla_nest_start(skb, TCA_OPTIONS);
+ if (!opts)
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_BASE_RATE, q->rate_bps))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_DIFFSERV_MODE, q->tin_mode))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_ATM, (q->rate_flags &
+ (CAKE_FLAG_ATM | CAKE_FLAG_PTM))))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_FLOW_MODE, q->flow_mode))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_WASH,
+ !!(q->rate_flags & CAKE_FLAG_WASH)))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_OVERHEAD, q->rate_overhead +
+ qdisc_dev(sch)->hard_header_len))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_MPU, q->rate_mpu))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_ETHERNET,
+ qdisc_dev(sch)->hard_header_len))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_RTT, q->interval))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_TARGET, q->target))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_AUTORATE,
+ !!(q->rate_flags & CAKE_FLAG_AUTORATE_INGRESS)))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_INGRESS,
+ !!(q->rate_flags & CAKE_FLAG_INGRESS)))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_ACK_FILTER,
+ (q->rate_flags &
+ (CAKE_FLAG_ACK_FILTER | CAKE_FLAG_ACK_AGGRESSIVE))))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_MEMORY, q->buffer_config_limit))
+ goto nla_put_failure;
+
+ return nla_nest_end(skb, opts);
+
+nla_put_failure:
+ return -1;
+}
+
+static int cake_dump_stats(struct Qdisc *sch, struct gnet_dump *d)
+{
+ struct cake_sched_data *q = qdisc_priv(sch);
+ struct tc_cake_xstats *st = kvzalloc(sizeof(*st), GFP_KERNEL);
+ int i;
+
+ if (!st)
+ return -ENOMEM;
+
+ st->version = 5;
+ st->max_tins = TC_CAKE_MAX_TINS;
+ st->tin_cnt = q->tin_cnt;
+
+ for (i = 0; i < q->tin_cnt; i++) {
+ struct cake_tin_data *b = &q->tins[q->tin_order[i]];
+
+ st->threshold_rate[i] = b->tin_rate_bps;
+ st->target_us[i] = cobalt_time_to_us(b->cparams.target);
+ st->interval_us[i] = cobalt_time_to_us(b->cparams.interval);
+
+ /* TODO FIXME: add missing aspects of these composite stats */
+ st->sent[i].packets = b->packets;
+ st->sent[i].bytes = b->bytes;
+ st->dropped[i].packets = b->tin_dropped;
+ st->ecn_marked[i].packets = b->tin_ecn_mark;
+ st->backlog[i].bytes = b->tin_backlog;
+ st->ack_drops[i].packets = b->ack_drops;
+
+ st->peak_delay_us[i] = cobalt_time_to_us(b->peak_delay);
+ st->avge_delay_us[i] = cobalt_time_to_us(b->avge_delay);
+ st->base_delay_us[i] = cobalt_time_to_us(b->base_delay);
+
+ st->way_indirect_hits[i] = b->way_hits;
+ st->way_misses[i] = b->way_misses;
+ st->way_collisions[i] = b->way_collisions;
+
+ st->sparse_flows[i] = b->sparse_flow_count +
+ b->decaying_flow_count;
+ st->bulk_flows[i] = b->bulk_flow_count;
+ st->unresponse_flows[i] = b->unresponsive_flow_count;
+ st->spare[i] = 0;
+ st->max_skblen[i] = b->max_skblen;
+ }
+ st->capacity_estimate = q->avg_peak_bandwidth;
+ st->memory_limit = q->buffer_limit;
+ st->memory_used = q->buffer_max_used;
+
+ i = gnet_stats_copy_app(d, st, sizeof(*st));
+ cake_free(st);
+ return i;
+}
+
+static struct Qdisc_ops cake_qdisc_ops __read_mostly = {
+ .id = "cake",
+ .priv_size = sizeof(struct cake_sched_data),
+ .enqueue = cake_enqueue,
+ .dequeue = cake_dequeue,
+ .peek = qdisc_peek_dequeued,
+ .init = cake_init,
+ .reset = cake_reset,
+ .destroy = cake_destroy,
+ .change = cake_change,
+ .dump = cake_dump,
+ .dump_stats = cake_dump_stats,
+ .owner = THIS_MODULE,
+};
+
+static int __init cake_module_init(void)
+{
+ return register_qdisc(&cake_qdisc_ops);
+}
+
+static void __exit cake_module_exit(void)
+{
+ unregister_qdisc(&cake_qdisc_ops);
+}
+
+module_init(cake_module_init)
+module_exit(cake_module_exit)
+MODULE_AUTHOR("Jonathan Morton");
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_DESCRIPTION("The Cake shaper. Version: " CAKE_VERSION);
--
2.7.4
^ permalink raw reply related
* [PATCH net-next 3/3] Add support for building the new cake qdisc
From: Dave Taht @ 2017-12-03 22:06 UTC (permalink / raw)
To: netdev
Cc: Dave Taht, Toke Høiland-Jørgensen, Sebastian Moeller,
Ryan Mounce, Jonathan Morton, Kevin Darbyshire-Bryant,
Nils Andreas Svee, Dean Scarff, Loganaden Velvindron
In-Reply-To: <1512338775-3270-1-git-send-email-dave.taht@gmail.com>
Hook up sch_cake to the build system.
Signed-off-by: Dave Taht <dave.taht@gmail.com>
---
net/sched/Kconfig | 11 +++++++++++
net/sched/Makefile | 1 +
2 files changed, 12 insertions(+)
diff --git a/net/sched/Kconfig b/net/sched/Kconfig
index c03d86a..3ea22e5 100644
--- a/net/sched/Kconfig
+++ b/net/sched/Kconfig
@@ -284,6 +284,17 @@ config NET_SCH_FQ_CODEL
If unsure, say N.
+config NET_SCH_CAKE
+ tristate "Common Applicatons Kept Enhanced (CAKE)"
+ help
+ Say Y here if you want to use the CAKE
+ packet scheduling algorithm.
+
+ To compile this driver as a module, choose M here: the module
+ will be called sch_cake.
+
+ If unsure, say N.
+
config NET_SCH_FQ
tristate "Fair Queue"
help
diff --git a/net/sched/Makefile b/net/sched/Makefile
index 5b63544..b8dd962 100644
--- a/net/sched/Makefile
+++ b/net/sched/Makefile
@@ -50,6 +50,7 @@ obj-$(CONFIG_NET_SCH_CHOKE) += sch_choke.o
obj-$(CONFIG_NET_SCH_QFQ) += sch_qfq.o
obj-$(CONFIG_NET_SCH_CODEL) += sch_codel.o
obj-$(CONFIG_NET_SCH_FQ_CODEL) += sch_fq_codel.o
+obj-$(CONFIG_NET_SCH_CAKE) += sch_cake.o
obj-$(CONFIG_NET_SCH_FQ) += sch_fq.o
obj-$(CONFIG_NET_SCH_HHF) += sch_hhf.o
obj-$(CONFIG_NET_SCH_PIE) += sch_pie.o
--
2.7.4
^ permalink raw reply related
* [PATCH net-next 1/3] pkt_sched.h: add support for sch_cake API
From: Dave Taht @ 2017-12-03 22:06 UTC (permalink / raw)
To: netdev
Cc: Dave Taht, Toke Høiland-Jørgensen, Sebastian Moeller,
Ryan Mounce, Jonathan Morton, Kevin Darbyshire-Bryant,
Nils Andreas Svee, Dean Scarff, Loganaden Velvindron
In-Reply-To: <1512338775-3270-1-git-send-email-dave.taht@gmail.com>
Signed-off-by: Dave Taht <dave.taht@gmail.com>
---
include/uapi/linux/pkt_sched.h | 58 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 58 insertions(+)
diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index af3cc2f..ed7c111 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -935,4 +935,62 @@ enum {
#define TCA_CBS_MAX (__TCA_CBS_MAX - 1)
+/* CAKE */
+enum {
+ TCA_CAKE_UNSPEC,
+ TCA_CAKE_BASE_RATE,
+ TCA_CAKE_DIFFSERV_MODE,
+ TCA_CAKE_ATM,
+ TCA_CAKE_FLOW_MODE,
+ TCA_CAKE_OVERHEAD,
+ TCA_CAKE_RTT,
+ TCA_CAKE_TARGET,
+ TCA_CAKE_AUTORATE,
+ TCA_CAKE_MEMORY,
+ TCA_CAKE_NAT,
+ TCA_CAKE_ETHERNET,
+ TCA_CAKE_WASH,
+ TCA_CAKE_MPU,
+ TCA_CAKE_INGRESS,
+ TCA_CAKE_ACK_FILTER,
+ __TCA_CAKE_MAX
+};
+#define TCA_CAKE_MAX (__TCA_CAKE_MAX - 1)
+
+struct tc_cake_traffic_stats {
+ __u32 packets;
+ __u32 link_ms;
+ __u64 bytes;
+};
+
+#define TC_CAKE_MAX_TINS (8)
+struct tc_cake_xstats {
+ __u16 version; /* == 5, increments when struct extended */
+ __u8 max_tins; /* == TC_CAKE_MAX_TINS */
+ __u8 tin_cnt; /* <= TC_CAKE_MAX_TINS */
+
+ __u32 threshold_rate[TC_CAKE_MAX_TINS];
+ __u32 target_us[TC_CAKE_MAX_TINS];
+ struct tc_cake_traffic_stats sent[TC_CAKE_MAX_TINS];
+ struct tc_cake_traffic_stats dropped[TC_CAKE_MAX_TINS];
+ struct tc_cake_traffic_stats ecn_marked[TC_CAKE_MAX_TINS];
+ struct tc_cake_traffic_stats backlog[TC_CAKE_MAX_TINS];
+ __u32 interval_us[TC_CAKE_MAX_TINS];
+ __u32 way_indirect_hits[TC_CAKE_MAX_TINS];
+ __u32 way_misses[TC_CAKE_MAX_TINS];
+ __u32 way_collisions[TC_CAKE_MAX_TINS];
+ __u32 peak_delay_us[TC_CAKE_MAX_TINS]; /* ~= bulk flow delay */
+ __u32 avge_delay_us[TC_CAKE_MAX_TINS];
+ __u32 base_delay_us[TC_CAKE_MAX_TINS]; /* ~= sparse flows delay */
+ __u16 sparse_flows[TC_CAKE_MAX_TINS];
+ __u16 bulk_flows[TC_CAKE_MAX_TINS];
+ __u16 unresponse_flows[TC_CAKE_MAX_TINS]; /* v4 - was u32 last_len */
+ __u16 spare[TC_CAKE_MAX_TINS]; /* v4 - split last_len */
+ __u32 max_skblen[TC_CAKE_MAX_TINS];
+ __u32 capacity_estimate; /* version 2 */
+ __u32 memory_limit; /* version 3 */
+ __u32 memory_used; /* version 3 */
+ struct tc_cake_traffic_stats ack_drops[TC_CAKE_MAX_TINS]; /* v5 */
+};
+
#endif
--
2.7.4
^ permalink raw reply related
* [PATCH net-next 0/3] Add Common Applications Kept Enhanced (cake) qdisc
From: Dave Taht @ 2017-12-03 22:06 UTC (permalink / raw)
To: netdev
Cc: Dave Taht, Toke Høiland-Jørgensen, Sebastian Moeller,
Ryan Mounce, Jonathan Morton, Kevin Darbyshire-Bryant,
Nils Andreas Svee, Dean Scarff, Loganaden Velvindron
sch_cake is intended to squeeze the most bandwidth and latency out of even
the slowest ISP links and routers, while presenting an API simple enough
that even an ISP can configure it.
Example of use on a cable ISP uplink:
tc qdisc add dev eth0 cake bandwidth 20Mbit nat docsis ack-filter
To shape a cable download link (ifb and tc-mirred setup elided)
tc qdisc add dev ifb0 cake bandwidth 200mbit nat docsis ingress wash besteffort
Cake is filled with:
* A hybrid Codel/Blue AQM algorithm, "Cobalt", tied to an FQ_Codel
derived Flow Queuing system, which autoconfigures based on the bandwidth.
* A novel "triple-isolate" mode (the default) which balances per-host
and per-flow FQ even through NAT.
* An deficit based shaper, that can also be used in an unlimited mode.
* 8 way set associative hashing to reduce flow collisions to a minimum.
* A reasonable interpretation of various diffserv latency/loss tradeoffs.
* Support for zeroing diffserv markings for entering and exiting traffic.
* Support for interacting well with Docsis 3.0 shaper framing.
* Support for DSL framing types and shapers.
* (New) Support for ack filtering.
* Extensive statistics for measuring, loss, ecn markings, latency variation.
There are some features still considered experimental, notably the
ingress_autorate bandwidth estimator and cobalt itself.
Various versions baking have been available as an out of tree build for
kernel versions going back to 3.10, as the embedded router world has been
running a few years behind mainline Linux. A stable version has been
generally available on lede-17.01 and later.
sch_cake replaces a combination of iptables, tc filter, htb and fq_codel
in the sqm-scripts, with sane defaults and vastly simpler configuration.
Cake's principal author is Jonathan Morton, with contributions from
Kevin Darbyshire-Bryant, Toke Høiland-Jørgensen, Sebastian Moeller,
Ryan Mounce, Guido Sarducci, Dean Scarff, Nils Andreas Svee, Dave Täht,
and Loganaden Velvindron.
Testing from Pete Heist, Georgios Amanakis, and the many other members of
the cake@lists.bufferbloat.net mailing list.
Dave Taht (3):
pkt_sched.h: add support for sch_cake API
Add Common Applications Kept Enhanced (cake) qdisc
Add support for building the new cake qdisc
include/net/cobalt.h | 152 +++
include/uapi/linux/pkt_sched.h | 58 +
net/sched/Kconfig | 11 +
net/sched/Makefile | 1 +
net/sched/sch_cake.c | 2561 ++++++++++++++++++++++++++++++++++++++++
5 files changed, 2783 insertions(+)
create mode 100644 include/net/cobalt.h
create mode 100644 net/sched/sch_cake.c
--
2.7.4
^ permalink raw reply
* [PATCH] cfg80211: fix kernel-doc warnings
From: Randy Dunlap @ 2017-12-03 21:17 UTC (permalink / raw)
To: netdev@vger.kernel.org, linux-doc@vger.kernel.org, David Miller
Cc: Arend Van Spriel
From: Randy Dunlap <rdunlap@infradead.org>
Drop kernel-doc comment for this value since this value was dropped
in a previous commit.
Fixes >100 of these warnings:
../include/net/cfg80211.h:3278: warning: Excess enum value 'WIPHY_FLAG_SUPPORTS_SCHED_SCAN' description in 'wiphy_flags'
Fixes: ca986ad9bcd3 ("nl80211: allow multiple active scheduled scan requests")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Arend Van Spriel <arend.vanspriel@broadcom.com>
---
include/net/cfg80211.h | 1 -
1 file changed, 1 deletion(-)
--- lnx-415-rc2.orig/include/net/cfg80211.h
+++ lnx-415-rc2/include/net/cfg80211.h
@@ -3226,7 +3226,6 @@ struct cfg80211_ops {
* @WIPHY_FLAG_IBSS_RSN: The device supports IBSS RSN.
* @WIPHY_FLAG_MESH_AUTH: The device supports mesh authentication by routing
* auth frames to userspace. See @NL80211_MESH_SETUP_USERSPACE_AUTH.
- * @WIPHY_FLAG_SUPPORTS_SCHED_SCAN: The device supports scheduled scans.
* @WIPHY_FLAG_SUPPORTS_FW_ROAM: The device supports roaming feature in the
* firmware.
* @WIPHY_FLAG_AP_UAPSD: The device supports uapsd on AP.
^ permalink raw reply
* RE: [PATCH net v2 2/3] xfrm: Add an activate() offload dev op
From: Yossi Kuperman @ 2017-12-03 21:09 UTC (permalink / raw)
To: Shannon Nelson, Steffen Klassert
Cc: Aviv Heller, Herbert Xu, Boris Pismenny, Yevgeny Kliteynik,
netdev@vger.kernel.org
In-Reply-To: <d9e31dd8-233d-ecf8-b93d-53b1b2d26a8c@oracle.com>
> -----Original Message-----
> From: Shannon Nelson [mailto:shannon.nelson@oracle.com]
> Sent: Sunday, December 3, 2017 2:38 AM
> To: Yossi Kuperman <yossiku@mellanox.com>; Steffen Klassert
> <steffen.klassert@secunet.com>
> Cc: Aviv Heller <avivh@mellanox.com>; Herbert Xu
> <herbert@gondor.apana.org.au>; Boris Pismenny <borisp@mellanox.com>;
> Yevgeny Kliteynik <kliteyn@mellanox.com>; netdev@vger.kernel.org
> Subject: Re: [PATCH net v2 2/3] xfrm: Add an activate() offload dev op
>
> On 12/2/2017 2:33 PM, Yossi Kuperman wrote:
> >
> >
> >>> On 1 Dec 2017, at 9:09, Steffen Klassert <steffen.klassert@secunet.com>
> wrote:
> >>>
> >>> On Tue, Nov 28, 2017 at 07:55:41PM +0200, avivh@mellanox.com wrote:
> >>> From: Aviv Heller <avivh@mellanox.com>
> >>>
> >>> Adding the state to the offload device prior to replay init in
> >>> xfrm_state_construct() will result in NULL dereference if a matching
> >>> ESP packet is received in between.
> >>>
> >>> In order to inhibit driver offload logic from processing the state's
> >>> packets prior to the xfrm_state object being completely initialized
> >>> and added to the SADBs, a new activate() operation was added to
> >>> inform the driver the aforementioned conditions have been met.
> >>
> >> We discussed this already some time ago, and I still think that we
> >> should fix this by setting XFRM_STATE_VALID only after the state is
> >> fully initialized.
> >
> > An upcoming patch will refactor the if statement (encap_type < 0) in
> xfrm_input, in order to support crypto offload with GRO disabled. Currently it
> doesn’t work. This entails yet another check for the validity of the state.
> Resulting in total of 3 copies: 1) for normal traffic, 2) GRO and 3) crypto offload.
> >
> > Anyway, IMO it is not right that we (the driver) allow an incoming packet to be
> delivered while the SA is not yet ready. Rather than checking for an invalid input I
> prefer to make sure that such a case won’t happen in the first place.
> >
> > To complete the picture, there is another patch to the driver which simply drop
> incoming packets that underwent successful decryption and haven’t been
> activated yet. Active state merely means that the SA is present in the driver’s
> hash table.
> >
> > We can make a separate patch to set the state to valid once it is fully
> initialized, it make sense on its own.
> >
> > What do you think?
> >
>
> If the SA isn't ready, just don't tell the driver about it. Please don't add yet
> another state for the driver to track. This should be as simple as possible, and
> shouldn't be any more complex than the model already used by
> ndo_vlan_rx_add_vid and ndo_vlan_rx_kill_vid.
It is not something new, the driver already "tracks" any SA that uses crypto offload, we just
do it in two phases; please see other answers.
Anyway it is symmetric with xdo_dev_state_delete and xdo_dev_state_free, J
>
> sln
^ permalink raw reply
* Re: [PATCH] leds: trigger: Introduce a NETDEV trigger
From: Jacek Anaszewski @ 2017-12-03 21:09 UTC (permalink / raw)
To: Ben Whitten, rpurdie, pavel; +Cc: linux-leds, linux-kernel, netdev
In-Reply-To: <1511906058-30649-2-git-send-email-ben.whitten@gmail.com>
Hi Ben,
Thanks for the patch. I have some comments in the code below.
Please take a look.
On 11/28/2017 10:54 PM, Ben Whitten wrote:
> This commit introduces a NETDEV trigger for named device
> activity. Available triggers are link, rx, and tx.
>
> Signed-off-by: Ben Whitten <ben.whitten@gmail.com>
> ---
> .../ABI/testing/sysfs-class-led-trigger-netdev | 45 +++
> drivers/leds/trigger/Kconfig | 7 +
> drivers/leds/trigger/Makefile | 1 +
> drivers/leds/trigger/ledtrig-netdev.c | 428 +++++++++++++++++++++
> 4 files changed, 481 insertions(+)
> create mode 100644 Documentation/ABI/testing/sysfs-class-led-trigger-netdev
> create mode 100644 drivers/leds/trigger/ledtrig-netdev.c
>
> diff --git a/Documentation/ABI/testing/sysfs-class-led-trigger-netdev b/Documentation/ABI/testing/sysfs-class-led-trigger-netdev
> new file mode 100644
> index 0000000..2c917df
> --- /dev/null
> +++ b/Documentation/ABI/testing/sysfs-class-led-trigger-netdev
> @@ -0,0 +1,45 @@
> +What: /sys/class/leds/<led>/device_name
> +Date: Nov 2017
> +KernelVersion: 4.15
Now it will be 4.16 presumably.
> +Contact: linux-leds@vger.kernel.org
> +Description:
> + Specifies the network device name to monitor.
> +
> +What: /sys/class/leds/<led>/interval
> +Date: Nov 2017
> +KernelVersion: 4.15
> +Contact: linux-leds@vger.kernel.org
> +Description:
> + Specifies the duration of the LED blink in milliseconds.
> + Defaults to 50 ms.
> +
> +What: /sys/class/leds/<led>/link
> +Date: Nov 2017
> +KernelVersion: 4.15
> +Contact: linux-leds@vger.kernel.org
> +Description:
> + Signal the link state of the named network device.
> + If set to 0 (default), the LED's normal state is off.
> + If set to 1, the LED's normal state reflects the link state
> + of the named network device.
> + Setting this value also immediately changes the LED state.
> +
> +What: /sys/class/leds/<led>/tx
> +Date: Nov 2017
> +KernelVersion: 4.15
> +Contact: linux-leds@vger.kernel.org
> +Description:
> + Signal transmission of data on the named network device.
> + If set to 0 (default), the LED will not blink on transmission.
> + If set to 1, the LED will blink for the milliseconds specified
> + in interval to signal transmission.
> +
> +What: /sys/class/leds/<led>/rx
> +Date: Nov 2017
> +KernelVersion: 4.15
> +Contact: linux-leds@vger.kernel.org
> +Description:
> + Signal reception of data on the named network device.
> + If set to 0 (default), the LED will not blink on reception.
> + If set to 1, the LED will blink for the milliseconds specified
> + in interval to signal reception.
> diff --git a/drivers/leds/trigger/Kconfig b/drivers/leds/trigger/Kconfig
> index 3f9ddb9..4ec1853 100644
> --- a/drivers/leds/trigger/Kconfig
> +++ b/drivers/leds/trigger/Kconfig
> @@ -126,4 +126,11 @@ config LEDS_TRIGGER_PANIC
> a different trigger.
> If unsure, say Y.
>
> +config LEDS_TRIGGER_NETDEV
> + tristate "LED Netdev Trigger"
> + depends on NET && LEDS_TRIGGERS
> + help
> + This allows LEDs to be controlled by network device activity.
> + If unsure, say Y.
> +
> endif # LEDS_TRIGGERS
> diff --git a/drivers/leds/trigger/Makefile b/drivers/leds/trigger/Makefile
> index 9f2e868..59e163d 100644
> --- a/drivers/leds/trigger/Makefile
> +++ b/drivers/leds/trigger/Makefile
> @@ -11,3 +11,4 @@ obj-$(CONFIG_LEDS_TRIGGER_DEFAULT_ON) += ledtrig-default-on.o
> obj-$(CONFIG_LEDS_TRIGGER_TRANSIENT) += ledtrig-transient.o
> obj-$(CONFIG_LEDS_TRIGGER_CAMERA) += ledtrig-camera.o
> obj-$(CONFIG_LEDS_TRIGGER_PANIC) += ledtrig-panic.o
> +obj-$(CONFIG_LEDS_TRIGGER_NETDEV) += ledtrig-netdev.o
> diff --git a/drivers/leds/trigger/ledtrig-netdev.c b/drivers/leds/trigger/ledtrig-netdev.c
> new file mode 100644
> index 0000000..2953b41
> --- /dev/null
> +++ b/drivers/leds/trigger/ledtrig-netdev.c
> @@ -0,0 +1,428 @@
> +/*
> + * LED Kernel Netdev Trigger
> + *
> + * Toggles the LED to reflect the link and traffic state of a named net device
> + *
> + * Copyright 2017 Ben Whitten <ben.whitten@gmail.com>
> + *
> + * Copyright 2007 Oliver Jowett <oliver@opencloud.com>
> + *
> + * Derived from ledtrig-timer.c which is:
> + * Copyright 2005-2006 Openedhand Ltd.
> + * Author: Richard Purdie <rpurdie@openedhand.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + */
> +
> +#include <linux/module.h>
> +#include <linux/jiffies.h>
> +#include <linux/kernel.h>
> +#include <linux/init.h>
> +#include <linux/list.h>
> +#include <linux/spinlock.h>
> +#include <linux/device.h>
> +#include <linux/netdevice.h>
> +#include <linux/timer.h>
> +#include <linux/ctype.h>
> +#include <linux/atomic.h>
> +#include <linux/leds.h>
Please sort includes lexicographically.
> +/*
> + * Configurable sysfs attributes:
> + *
> + * device_name - network device name to monitor
> + *
Redundant empty line.
> + * interval - duration of LED blink, in milliseconds
> + *
Ditto.
> + * link - LED's normal state reflects whether the link is up
> + * (has carrier) or not
> + * tx - LED blinks on transmitted data
> + * rx - LED blinks on receive data
> + *
> + */
> +
> +struct led_netdev_data {
> + spinlock_t lock;
> +
> + struct delayed_work work;
> + struct notifier_block notifier;
> +
> + struct led_classdev *led_cdev;
> + struct net_device *net_dev;
> +
> + char device_name[IFNAMSIZ];
> + atomic_t interval;
> + unsigned int last_activity;
> +
> + unsigned long mode;
> +#define LED_BLINK_link 0
> +#define LED_BLINK_tx 1
> +#define LED_BLINK_rx 2
LED core already has LED_BLINK* family of macros. Please come up
with the prefix specific for this trigger e.g. NETDEV_LED.
Also let's use uppercase for the whole macro name.
> +#define LED_MODE_LINKUP 3
s/LED/NETDEV_LED/
> +};
> +
> +static void set_baseline_state(struct led_netdev_data *trigger_data)
> +{
> + if (!test_bit(LED_MODE_LINKUP, &trigger_data->mode))
> + led_set_brightness(trigger_data->led_cdev, LED_OFF);
> + else {
> + if (test_bit(LED_BLINK_link, &trigger_data->mode))
> + led_set_brightness(trigger_data->led_cdev, LED_FULL);
> +
> + if (test_bit(LED_BLINK_tx, &trigger_data->mode) ||
> + test_bit(LED_BLINK_rx, &trigger_data->mode))
> + schedule_delayed_work(&trigger_data->work,
> + atomic_read(&trigger_data->interval));
Now we have blink support in the LED core. Please use
led_blink_set_oneshot() instead.
Generally you can compare how ledtrig-timer is now implemented.
> + }
> +}
> +
> +static ssize_t device_name_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + struct led_classdev *led_cdev = dev_get_drvdata(dev);
> + struct led_netdev_data *trigger_data = led_cdev->trigger_data;
> + ssize_t len;
> +
> + spin_lock_bh(&trigger_data->lock);
> + len = sprintf(buf, "%s\n", trigger_data->device_name);
> + spin_unlock_bh(&trigger_data->lock);
> +
> + return len;
> +}
> +
> +static ssize_t device_name_store(struct device *dev,
> + struct device_attribute *attr, const char *buf,
> + size_t size)
> +{
> + struct led_classdev *led_cdev = dev_get_drvdata(dev);
> + struct led_netdev_data *trigger_data = led_cdev->trigger_data;
> +
> + if (size >= IFNAMSIZ)
> + return -EINVAL;
> +
> + cancel_delayed_work_sync(&trigger_data->work);
> +
> + spin_lock_bh(&trigger_data->lock);
> +
> + if (trigger_data->net_dev) {
> + dev_put(trigger_data->net_dev);
> + trigger_data->net_dev = NULL;
> + }
> +
> + strncpy(trigger_data->device_name, buf, size);
> + if (size > 0 && trigger_data->device_name[size - 1] == '\n')
> + trigger_data->device_name[size - 1] = 0;
> +
> + if (trigger_data->device_name[0] != 0)
> + trigger_data->net_dev =
> + dev_get_by_name(&init_net, trigger_data->device_name);
> +
> + clear_bit(LED_MODE_LINKUP, &trigger_data->mode);
> + if (trigger_data->net_dev != NULL)
> + if (netif_carrier_ok(trigger_data->net_dev))
> + set_bit(LED_MODE_LINKUP, &trigger_data->mode);
> +
> + trigger_data->last_activity = 0;
> +
> + set_baseline_state(trigger_data);
> + spin_unlock_bh(&trigger_data->lock);
> +
> + return size;
> +}
> +
> +static DEVICE_ATTR_RW(device_name);
> +
> +#define led_mode_flags_attr(field) \
> +static ssize_t field##_show(struct device *dev, \
> + struct device_attribute *attr, char *buf) \
> +{ \
> + struct led_classdev *led_cdev = dev_get_drvdata(dev); \
> + struct led_netdev_data *trigger_data = led_cdev->trigger_data; \
> + \
> + return sprintf(buf, "%u\n", test_bit(LED_BLINK_##field, \
> + &trigger_data->mode)); \
> +} \
> +static ssize_t field##_store(struct device *dev, \
> + struct device_attribute *attr, const char *buf, size_t size) \
> +{ \
> + struct led_classdev *led_cdev = dev_get_drvdata(dev); \
> + struct led_netdev_data *trigger_data = led_cdev->trigger_data; \
> + unsigned long state; \
> + int ret; \
> + \
> + ret = kstrtoul(buf, 0, &state); \
> + if (ret) \
> + return ret; \
> + \
> + cancel_delayed_work_sync(&trigger_data->work); \
> + \
> + if (state) \
> + set_bit(LED_BLINK_##field, &trigger_data->mode); \
> + else \
> + clear_bit(LED_BLINK_##field, &trigger_data->mode); \
> + \
> + set_baseline_state(trigger_data); \
> + \
> + return size; \
> +} \
> +static DEVICE_ATTR_RW(field);
In order to get rid of this macro and avoid the need for camel case
macro name we could have one function that would accept an enum e.g.
enum netdev_led_attr {
NETDEV_ATTR_LINK,
NETDEV_ATTR_RX,
NETDEV_ATTR_TX
};
The function would be called from each sysfs attr callback with the
related enum, and would alter the state of the related bit.
> +led_mode_flags_attr(link);
> +led_mode_flags_attr(rx);
> +led_mode_flags_attr(tx);
> +
> +static ssize_t interval_show(struct device *dev,
> + struct device_attribute *attr, char *buf)
> +{
> + struct led_classdev *led_cdev = dev_get_drvdata(dev);
> + struct led_netdev_data *trigger_data = led_cdev->trigger_data;
> +
> + return sprintf(buf, "%u\n",
> + jiffies_to_msecs(atomic_read(&trigger_data->interval)));
> +}
> +
> +static ssize_t interval_store(struct device *dev,
> + struct device_attribute *attr, const char *buf,
> + size_t size)
> +{
> + struct led_classdev *led_cdev = dev_get_drvdata(dev);
> + struct led_netdev_data *trigger_data = led_cdev->trigger_data;
> + unsigned long value;
> + int ret;
> +
> + ret = kstrtoul(buf, 0, &value);
> + if (ret)
> + return ret;
> +
> + /* impose some basic bounds on the timer interval */
> + if (value >= 5 && value <= 10000) {
> + cancel_delayed_work_sync(&trigger_data->work);
> +
> + atomic_set(&trigger_data->interval, msecs_to_jiffies(value));
> + set_baseline_state(trigger_data); /* resets timer */
> + }
> +
> + return size;
> +}
> +
> +static DEVICE_ATTR_RW(interval);
> +
> +static int netdev_trig_notify(struct notifier_block *nb,
> + unsigned long evt, void *dv)
> +{
> + struct net_device *dev =
> + netdev_notifier_info_to_dev((struct netdev_notifier_info *)dv);
> + struct led_netdev_data *trigger_data = container_of(nb,
> + struct
> + led_netdev_data,
> + notifier);
> +
> + if (evt != NETDEV_UP && evt != NETDEV_DOWN && evt != NETDEV_CHANGE
> + && evt != NETDEV_REGISTER && evt != NETDEV_UNREGISTER
> + && evt != NETDEV_CHANGENAME)
> + return NOTIFY_DONE;
> +
> + if (strcmp(dev->name, trigger_data->device_name))
> + return NOTIFY_DONE;
> +
> + cancel_delayed_work_sync(&trigger_data->work);
> +
> + spin_lock_bh(&trigger_data->lock);
> +
> + clear_bit(LED_MODE_LINKUP, &trigger_data->mode);
> + switch (evt) {
> + case NETDEV_REGISTER:
> + if (trigger_data->net_dev)
> + dev_put(trigger_data->net_dev);
> + dev_hold(dev);
> + trigger_data->net_dev = dev;
> + break;
> + case NETDEV_CHANGENAME:
> + case NETDEV_UNREGISTER:
> + if (trigger_data->net_dev) {
> + dev_put(trigger_data->net_dev);
> + trigger_data->net_dev = NULL;
> + }
> + break;
> + case NETDEV_UP:
> + case NETDEV_CHANGE:
> + if (netif_carrier_ok(dev))
> + set_bit(LED_MODE_LINKUP, &trigger_data->mode);
> + break;
> + }
> +
> + set_baseline_state(trigger_data);
> +
> + spin_unlock_bh(&trigger_data->lock);
> +
> + return NOTIFY_DONE;
> +}
> +
> +/* here's the real work! */
> +static void netdev_trig_work(struct work_struct *work)
> +{
> + struct led_netdev_data *trigger_data = container_of(work,
> + struct
> + led_netdev_data,
> + work.work);
> + struct rtnl_link_stats64 *dev_stats;
> + unsigned int new_activity;
> + struct rtnl_link_stats64 temp;
> +
> + if (!test_bit(LED_MODE_LINKUP, &trigger_data->mode) ||
> + !trigger_data->net_dev ||
> + (!test_bit(LED_BLINK_tx, &trigger_data->mode) &&
> + !test_bit(LED_BLINK_rx, &trigger_data->mode))) {
> + /* we don't need to do timer work, just reflect link state. */
> + led_set_brightness(trigger_data->led_cdev,
> + (test_bit
> + (LED_BLINK_link, &trigger_data->mode)
> + && test_bit(LED_MODE_LINKUP,
> + &trigger_data->mode
> + ) ? LED_FULL : LED_OFF));
> + return;
> + }
> +
> + dev_stats = dev_get_stats(trigger_data->net_dev, &temp);
> + new_activity =
> + (test_bit(LED_BLINK_tx, &trigger_data->mode) ?
> + dev_stats->tx_packets : 0) +
> + (test_bit(LED_BLINK_rx, &trigger_data->mode) ?
> + dev_stats->rx_packets : 0);
> +
> + if (test_bit(LED_BLINK_link, &trigger_data->mode)) {
> + /* base state is ON (link present) */
> + /* if there's no link, we don't get this far and
> + * the LED is off
> + */
> +
> + /* OFF -> ON always */
> + /* ON -> OFF on activity */
> + if (trigger_data->led_cdev->brightness == LED_OFF)
> + led_set_brightness(trigger_data->led_cdev, LED_FULL);
> + else if (trigger_data->last_activity != new_activity)
> + led_set_brightness(trigger_data->led_cdev, LED_OFF);
> + } else {
> + /* base state is OFF */
> + /* ON -> OFF always */
> + /* OFF -> ON on activity */
> + if (trigger_data->led_cdev->brightness == LED_FULL)
> + led_set_brightness(trigger_data->led_cdev, LED_OFF);
> + else if (trigger_data->last_activity != new_activity)
> + led_set_brightness(trigger_data->led_cdev, LED_FULL);
> + }
> +
> + trigger_data->last_activity = new_activity;
> + schedule_delayed_work(&trigger_data->work,
> + atomic_read(&trigger_data->interval));
> +}
> +
> +static void netdev_trig_activate(struct led_classdev *led_cdev)
> +{
> + struct led_netdev_data *trigger_data;
> + int rc;
> +
> + trigger_data = kzalloc(sizeof(struct led_netdev_data), GFP_KERNEL);
> + if (!trigger_data)
> + return;
> +
> + spin_lock_init(&trigger_data->lock);
> +
> + trigger_data->notifier.notifier_call = netdev_trig_notify;
> + trigger_data->notifier.priority = 10;
> +
> + INIT_DELAYED_WORK(&trigger_data->work, netdev_trig_work);
> +
> + trigger_data->led_cdev = led_cdev;
> + trigger_data->net_dev = NULL;
> + trigger_data->device_name[0] = 0;
> +
> + trigger_data->mode = 0;
> + atomic_set(&trigger_data->interval, msecs_to_jiffies(50));
> + trigger_data->last_activity = 0;
> +
> + led_cdev->trigger_data = trigger_data;
> +
> + rc = device_create_file(led_cdev->dev, &dev_attr_device_name);
> + if (rc)
> + goto err_out;
> + rc = device_create_file(led_cdev->dev, &dev_attr_link);
> + if (rc)
> + goto err_out_device_name;
> + rc = device_create_file(led_cdev->dev, &dev_attr_rx);
> + if (rc)
> + goto err_out_link;
> + rc = device_create_file(led_cdev->dev, &dev_attr_tx);
> + if (rc)
> + goto err_out_rx;
> + rc = device_create_file(led_cdev->dev, &dev_attr_interval);
> + if (rc)
> + goto err_out_tx;
> + rc = register_netdevice_notifier(&trigger_data->notifier);
> + if (rc)
> + goto err_out_interval;
> + return;
> +
> +err_out_interval:
> + device_remove_file(led_cdev->dev, &dev_attr_interval);
> +err_out_tx:
> + device_remove_file(led_cdev->dev, &dev_attr_tx);
> +err_out_rx:
> + device_remove_file(led_cdev->dev, &dev_attr_rx);
> +err_out_link:
> + device_remove_file(led_cdev->dev, &dev_attr_link);
> +err_out_device_name:
> + device_remove_file(led_cdev->dev, &dev_attr_device_name);
> +err_out:
> + led_cdev->trigger_data = NULL;
> + kfree(trigger_data);
> +}
> +
> +static void netdev_trig_deactivate(struct led_classdev *led_cdev)
> +{
> + struct led_netdev_data *trigger_data = led_cdev->trigger_data;
> +
> + if (trigger_data) {
> + unregister_netdevice_notifier(&trigger_data->notifier);
> +
> + device_remove_file(led_cdev->dev, &dev_attr_device_name);
> + device_remove_file(led_cdev->dev, &dev_attr_link);
> + device_remove_file(led_cdev->dev, &dev_attr_rx);
> + device_remove_file(led_cdev->dev, &dev_attr_tx);
> + device_remove_file(led_cdev->dev, &dev_attr_interval);
> +
> + cancel_delayed_work_sync(&trigger_data->work);
> +
> + if (trigger_data->net_dev)
> + dev_put(trigger_data->net_dev);
> +
> + kfree(trigger_data);
> + }
> +}
> +
> +static struct led_trigger netdev_led_trigger = {
> + .name = "netdev",
> + .activate = netdev_trig_activate,
> + .deactivate = netdev_trig_deactivate,
> +};
> +
> +static int __init netdev_trig_init(void)
> +{
> + return led_trigger_register(&netdev_led_trigger);
> +}
> +
> +static void __exit netdev_trig_exit(void)
> +{
> + led_trigger_unregister(&netdev_led_trigger);
> +}
> +
> +module_init(netdev_trig_init);
> +module_exit(netdev_trig_exit);
> +
> +MODULE_AUTHOR("Ben Whitten <ben.whitten@gmail.com>");
> +MODULE_AUTHOR("Oliver Jowett <oliver@opencloud.com>");
> +MODULE_DESCRIPTION("Netdev LED trigger");
> +MODULE_LICENSE("GPL");
"GPL v2" according to the licensing information from the top
of the file.
--
Best regards,
Jacek Anaszewski
^ permalink raw reply
* Re: kernel BUG at net/key/af_key.c:LINE!
From: Eric Biggers @ 2017-12-03 20:28 UTC (permalink / raw)
To: Steffen Klassert
Cc: Herbert Xu, Dmitry Vyukov, syzbot, David Miller, LKML, netdev,
syzkaller-bugs
In-Reply-To: <20171115112919.GT11292@secunet.com>
On Wed, Nov 15, 2017 at 12:29:19PM +0100, Steffen Klassert wrote:
> On Fri, Nov 10, 2017 at 02:14:06PM +1100, Herbert Xu wrote:
> > On Fri, Nov 10, 2017 at 01:30:38PM +1100, Herbert Xu wrote:
> > >
> > > I found the problem. This crap is coming from clone_policy. Now
> > > let me where this code came from.
> >
> > ---8<---
> > Subject: xfrm: Copy policy family in clone_policy
> >
> > The syzbot found an ancient bug in the IPsec code. When we cloned
> > a socket policy (for example, for a child TCP socket derived from a
> > listening socket), we did not copy the family field. This results
> > in a live policy with a zero family field. This triggers a BUG_ON
> > check in the af_key code when the cloned policy is retrieved.
> >
> > This patch fixes it by copying the family field over.
> >
> > Reported-by: syzbot <syzkaller@googlegroups.com>
> > Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
>
> Patch applied, thanks Herbert!
And to tell the bot what fixes this:
#syz fix: xfrm: Copy policy family in clone_policy
Also, does this fix need to go to stable? The commit doesn't have Cc: stable.
^ permalink raw reply
* Re: [Patch net-next] net_sched: get rid of rcu_barrier() in tcf_block_put_ext()
From: Cong Wang @ 2017-12-03 19:32 UTC (permalink / raw)
To: Jiri Pirko
Cc: Linux Kernel Network Developers, Paolo Abeni, Eric Dumazet,
Jiri Pirko, Jamal Hadi Salim
In-Reply-To: <20171202092157.GF1821@nanopsycho>
On Sat, Dec 2, 2017 at 1:21 AM, Jiri Pirko <jiri@resnulli.us> wrote:
> Sat, Dec 02, 2017 at 01:18:04AM CET, xiyou.wangcong@gmail.com wrote:
>>Both Eric and Paolo noticed the rcu_barrier() we use in
>>tcf_block_put_ext() could be a performance bottleneck when
>>we have lots of filters.
>
> The problem is not a lots of filters, the problem is lots of classes and
> therefore tcf_blocks
Fixed.
[...]
>>@@ -218,8 +219,12 @@ static void tcf_chain_flush(struct tcf_chain *chain)
>>
>> static void tcf_chain_destroy(struct tcf_chain *chain)
>> {
>>+ struct tcf_block *block = chain->block;
>>+
>> list_del(&chain->list);
>> kfree(chain);
>>+ if (!--block->nr_chains)
>
> You don't need this counter. You can just check
> list_empty(block->chain_list);
Makes sense! Done.
[...]
>>@@ -364,13 +355,9 @@ void tcf_block_put_ext(struct tcf_block *block, struct Qdisc *q,
>>
>> tcf_block_offload_unbind(block, q, ei);
>>
>>- INIT_WORK(&block->work, tcf_block_put_final);
>>- /* Wait for existing RCU callbacks to cool down, make sure their works
>>- * have been queued before this. We can not flush pending works here
>>- * because we are holding the RTNL lock.
>>- */
>>- rcu_barrier();
>>- tcf_queue_work(&block->work);
>>+ /* At this point, all the chains should have refcnt >= 1. */
>>+ list_for_each_entry_safe(chain, tmp, &block->chain_list, list)
>>+ tcf_chain_put(chain);
>
> I think this is correct. Would be probably good to elaborate a bit more
> about what is happening. Perhaps a comment?
OK, I will add a comment about this refcnt.
Thanks!
^ permalink raw reply
* Re: BUG: unable to handle kernel NULL pointer dereference
From: Eric Biggers @ 2017-12-03 19:28 UTC (permalink / raw)
To: syzbot; +Cc: davem, linux-kernel, netdev, syzkaller-bugs
In-Reply-To: <001a113ed15ec1d449055f6edbcf@google.com>
On Sun, Dec 03, 2017 at 04:37:01AM -0800, syzbot wrote:
> BUG: KASAN: use-after-free in skcipher_request_set_tfm
> include/crypto/skcipher.h:499 [inline]
> BUG: KASAN: use-after-free in crypto_aead_copy_sgl crypto/algif_aead.c:85
> [inline]
> BUG: KASAN: use-after-free in _aead_recvmsg crypto/algif_aead.c:210 [inline]
> BUG: KASAN: use-after-free in aead_recvmsg+0x1552/0x1970
> crypto/algif_aead.c:313
> Read of size 4 at addr ffff8801cabcfd9c by task syzkaller687795/3634
[...]
> Allocated by task 3442:
> save_stack+0x43/0xd0 mm/kasan/kasan.c:447
> set_track mm/kasan/kasan.c:459 [inline]
> kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:551
> __do_kmalloc mm/slab.c:3711 [inline]
> __kmalloc+0x162/0x760 mm/slab.c:3720
> kmalloc include/linux/slab.h:504 [inline]
> kzalloc include/linux/slab.h:688 [inline]
> crypto_create_tfm+0x82/0x2e0 crypto/api.c:455
> crypto_alloc_tfm+0x10e/0x2f0 crypto/api.c:540
> crypto_alloc_skcipher+0x2c/0x40 crypto/skcipher.c:926
> crypto_get_default_null_skcipher+0x5f/0x80 crypto/crypto_null.c:164
> crypto_get_default_null_skcipher2 include/crypto/null.h:17 [inline]
> aead_bind+0x89/0x140 crypto/algif_aead.c:472
> alg_bind+0x1ab/0x440 crypto/af_alg.c:179
> SYSC_bind+0x1b4/0x3f0 net/socket.c:1475
> SyS_bind+0x24/0x30 net/socket.c:1461
> entry_SYSCALL_64_fastpath+0x1f/0x96
#syz dup: KASAN: use-after-free Read in aead_recvmsg
^ permalink raw reply
* Re: [PATCH net-next] bpf: Add access to snd_cwnd and others in sock_ops
From: Alexei Starovoitov @ 2017-12-03 18:26 UTC (permalink / raw)
To: Lawrence Brakmo; +Cc: netdev, Daniel Borkmann, Kernel Team, Vlad Dumitrescu
In-Reply-To: <20171201181504.1040223-1-brakmo@fb.com>
On Fri, Dec 01, 2017 at 10:15:04AM -0800, Lawrence Brakmo wrote:
> Adds read access to snd_cwnd and srtt_us fields of tcp_sock. Since these
> fields are only valid if the socket associated with the sock_ops program
> call is a full socket, the field is_fullsock is also added to the
> bpf_sock_ops struct. If the socket is not a full socket, reading these
> fields returns 0.
>
> Note that in most cases it will not be necessary to check is_fullsock to
> know if there is a full socket. The context of the call, as specified by
> the 'op' field, can sometimes determine whether there is a full socket.
>
> The struct bpf_sock_ops has the following fields added:
>
> __u32 is_fullsock; /* Some TCP fields are only valid if
> * there is a full socket. If not, the
> * fields read as zero.
> */
> __u32 snd_cwnd;
> __u32 srtt_us; /* Averaged RTT << 3 in usecs */
>
> There is a new macro, SOCK_OPS_GET_TCP32(NAME), to make it easier to add
> read access to more 32 bit tcp_sock fields.
>
> Signed-off-by: Lawrence Brakmo <brakmo@fb.com>
lgtm
Acked-by: Alexei Starovoitov <ast@kernel.org>
will wait for Daniel to either ack it or apply it.
^ permalink raw reply
* [GIT] Networking
From: David Miller @ 2017-12-03 18:14 UTC (permalink / raw)
To: torvalds; +Cc: akpm, netdev, linux-kernel
1) Various TCP control block fixes, including one that crashes with
SELINUX, from David Ahern and Eric Dumazet.
2) Fix ACK generation in rxrpc, from David Howells.
3) ipvlan doesn't set the mark properly in the ipv4 route lookup
key, from Gao Feng.
4) SIT configuration doesn't take on the frag_off ipv4 field
configuration properly, fix from Hangbin Liu.
5) TSO can fail after device down/up on stmmac, fix from Lars Persson.
6) Various bpftool fixes (mostly in JSON handling) from Quentin
Monnet.
7) Various SKB leak fixes in vhost/tun/tap (mostly observed as
performance problems). From Wei Xu.
8) mvpps's TX descriptors were not zero initialized, from Yan Markman.
Please pull, thanks a lot!
The following changes since commit b9151761021e25c024a6670df4e7c43ffbab0e1d:
Merge tag 'nfsd-4.15-1' of git://linux-nfs.org/~bfields/linux (2017-11-29 14:49:26 -0800)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git
for you to fetch changes up to c2eb6d07a63cb01f0ef978b28927335198c544ce:
Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf (2017-12-03 13:08:30 -0500)
----------------------------------------------------------------
Colin Ian King (1):
liquidio: fix incorrect indentation of assignment statement
Cong Wang (1):
act_sample: get rid of tcf_sample_cleanup_rcu()
Corentin Labbe (1):
net: stmmac: dwmac-sun8i: fix allwinner,leds-active-low handling
Daniel Borkmann (1):
Merge branch 'bpftool-misc-fixes'
David Ahern (1):
tcp: use IPCB instead of TCP_SKB_CB in inet_exact_dif_match()
David Howells (4):
rxrpc: Clean up whitespace
rxrpc: Fix ACK generation from the connection event processor
rxrpc: Use correct netns source in rxrpc_release_sock()
rxrpc: Fix the MAINTAINERS record
David S. Miller (9):
Merge tag 'rxrpc-fixes-20171129' of git://git.kernel.org/.../dhowells/linux-fs
Merge branch 'sctp-prsctp-chunk-fixes'
Merge branch 'sfp-phylink-fixes'
Merge branch 'bnxt_en-fixes'
Merge branch 'vhost-skb-leaks'
Merge branch 's390-qeth-fixes'
Merge tag 'linux-can-fixes-for-4.15-20171201' of git://git.kernel.org/.../mkl/linux-can
Merge branch 'tcp-cb-selinux-corruption'
Merge git://git.kernel.org/.../bpf/bpf
Eric Dumazet (3):
tcp: remove buggy call to tcp_v6_restore_cb()
tcp/dccp: block bh before arming time_wait timer
tcp: add tcp_v4_fill_cb()/tcp_v4_restore_cb()
Florian Fainelli (1):
net: dsa: bcm_sf2: Set correct CHAIN_ID and slice number mask
Gao Feng (1):
ipvlan: Add the skb->mark as flow4's member to lookup route
Geert Uytterhoeven (1):
skbuff: Grammar s/are can/can/, s/change/changes/
Gustavo A. R. Silva (1):
rxrpc: Fix variable overwrite
Hangbin Liu (1):
sit: update frag_off info
Jakub Kicinski (1):
bpf: offload: add a license header
Jimmy Assarsson (3):
can: kvaser_usb: free buf in error paths
can: kvaser_usb: Fix comparison bug in kvaser_usb_read_bulk_callback()
can: kvaser_usb: ratelimit errors if incomplete messages are received
Julian Wiedmann (3):
s390/qeth: fix thinko in IPv4 multicast address tracking
s390/qeth: fix GSO throughput regression
s390/qeth: build max size GSO skbs on L2 devices
Lars Persson (1):
stmmac: reset last TSO segment size after device open
Marc Kleine-Budde (2):
can: flexcan: Update IRQ Err Passive information
can: flexcan: fix VF610 state transition issue
Martin Kelly (2):
can: mcba_usb: fix typo
can: mcba_usb: fix device disconnect bug
Max Uvarov (1):
net: phy-micrel: check return code in flp center function
Oliver Stäbler (1):
can: ti_hecc: Fix napi poll return value for repoll
Quentin Monnet (6):
tools: bpftool: fix crash on bad parameters with JSON
tools: bpftool: clean up the JSON writer before exiting in usage()
tools: bpftool: make error message from getopt_long() JSON-friendly
tools: bpftool: remove spurious line break from error message
tools: bpftool: unify installation directories
tools: bpftool: declare phony targets as such
Ray Jui (1):
bnxt_en: Need to unconditionally shut down RoCE in bnxt_shutdown
Russell King (4):
sfp: fix RX_LOS signal handling
sfp: improve RX_LOS handling
sfp: warn about modules requiring address change sequence
phylink: ensure we take the link down when phylink_stop() is called
Sathya Perla (1):
bnxt_en: fix dst/src fid for vxlan encap/decap actions
Stephane Grosjean (1):
can: peak/pci: fix potential bug when probe() fails
Sunil Challa (1):
bnxt_en: wildcard smac while creating tunnel decap filter
Tommi Rantala (1):
tipc: call tipc_rcv() only if bearer is up in tipc_udp_recv()
Vasundhara Volam (1):
bnxt_en: Fix a variable scoping in bnxt_hwrm_do_send_msg()
Wei Xu (3):
vhost: fix skb leak in handle_rx()
tun: free skb in early errors
tap: free skb if flags error
Xie XiuQi (1):
trace/xdp: fix compile warning: 'struct bpf_map' declared inside parameter list
Xin Long (3):
sctp: only update outstanding_bytes for transmitted queue when doing prsctp_prune
sctp: abandon the whole msg if one part of a fragmented message is abandoned
sctp: do not abandon the other frags in unsent outq if one msg has outstanding frags
Yan Markman (1):
net: mvpp2: allocate zeroed tx descriptors
Yonghong Song (3):
tools/bpf: adjust rlimit RLIMIT_MEMLOCK for test_verifier_log
bpf: set maximum number of attached progs to 64 for a single perf tp
samples/bpf: add error checking for perf ioctl calls in bpf loader
MAINTAINERS | 18 +++++++++++++++---
drivers/net/can/flexcan.c | 9 +++++----
drivers/net/can/peak_canfd/peak_pciefd_main.c | 5 ++++-
drivers/net/can/sja1000/peak_pci.c | 5 ++++-
drivers/net/can/ti_hecc.c | 3 +++
drivers/net/can/usb/kvaser_usb.c | 11 +++++++----
drivers/net/can/usb/mcba_usb.c | 3 ++-
drivers/net/dsa/bcm_sf2_cfp.c | 4 ++--
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 5 +++--
drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c | 55 ++++++++++++++++++++++++++++---------------------------
drivers/net/ethernet/cavium/liquidio/lio_main.c | 2 +-
drivers/net/ethernet/marvell/mvpp2.c | 2 +-
drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c | 3 +--
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 1 +
drivers/net/ipvlan/ipvlan_core.c | 1 +
drivers/net/phy/micrel.c | 6 ++++--
drivers/net/phy/phylink.c | 1 +
drivers/net/phy/sfp.c | 41 ++++++++++++++++++++++++++++++-----------
drivers/net/tap.c | 14 ++++++++++----
drivers/net/tun.c | 24 ++++++++++++++++++------
drivers/s390/net/qeth_core.h | 3 +++
drivers/s390/net/qeth_core_main.c | 31 +++++++++++++++++++++++++++++++
drivers/s390/net/qeth_l2_main.c | 4 ++--
drivers/s390/net/qeth_l3_main.c | 7 +++++--
drivers/vhost/net.c | 20 ++++++++++----------
include/linux/skbuff.h | 3 +--
include/net/sctp/structs.h | 3 ++-
include/net/tc_act/tc_sample.h | 1 -
include/net/tcp.h | 3 +--
include/trace/events/xdp.h | 1 +
kernel/bpf/core.c | 3 ++-
kernel/bpf/offload.c | 15 +++++++++++++++
kernel/trace/bpf_trace.c | 8 ++++++++
net/dccp/minisocks.c | 6 ++++++
net/ipv4/tcp_ipv4.c | 59 ++++++++++++++++++++++++++++++++++++++++-------------------
net/ipv4/tcp_minisocks.c | 6 ++++++
net/ipv6/sit.c | 1 +
net/ipv6/tcp_ipv6.c | 11 ++++++-----
net/rxrpc/af_rxrpc.c | 5 +++--
net/rxrpc/call_event.c | 4 ++--
net/rxrpc/conn_event.c | 50 +++++++++++++++++++++++++++++---------------------
net/rxrpc/conn_object.c | 2 +-
net/rxrpc/input.c | 4 ++--
net/rxrpc/sendmsg.c | 2 +-
net/sched/act_sample.c | 14 +++-----------
net/sctp/chunk.c | 11 +++++++++++
net/sctp/outqueue.c | 19 +++++++++++++------
net/tipc/udp_media.c | 4 ----
samples/bpf/bpf_load.c | 14 ++++++++++++--
tools/bpf/bpftool/Documentation/Makefile | 2 +-
tools/bpf/bpftool/Makefile | 7 ++++---
tools/bpf/bpftool/main.c | 36 ++++++++++++++++++++++++------------
tools/bpf/bpftool/main.h | 5 +++--
tools/testing/selftests/bpf/test_verifier_log.c | 7 +++++++
54 files changed, 397 insertions(+), 187 deletions(-)
^ permalink raw reply
* Re: pull-request: bpf 2017-12-02
From: David Miller @ 2017-12-03 18:08 UTC (permalink / raw)
To: daniel; +Cc: ast, netdev
In-Reply-To: <20171202010504.21516-1-daniel@iogearbox.net>
From: Daniel Borkmann <daniel@iogearbox.net>
Date: Sat, 2 Dec 2017 02:05:04 +0100
> The following pull-request contains BPF updates for your *net* tree.
Pulled, thanks a lot Daniel.
^ permalink raw reply
* Re: [PATCH 1/5] bpf: correct broken uapi for BPF_PROG_TYPE_PERF_EVENT program type
From: Alexei Starovoitov @ 2017-12-03 17:51 UTC (permalink / raw)
To: Hendrik Brueckner
Cc: Arnaldo Carvalho de Melo, Daniel Borkmann, Martin Schwidefsky,
Will Deacon, Ingo Molnar, linux-kernel, linux-s390, netdev,
Peter Zijlstra, David S. Miller
In-Reply-To: <1512137948-31729-2-git-send-email-brueckner@linux.vnet.ibm.com>
On Fri, Dec 01, 2017 at 03:19:04PM +0100, Hendrik Brueckner wrote:
> Commit 0515e5999a466dfe ("bpf: introduce BPF_PROG_TYPE_PERF_EVENT
> program type") introduced the bpf_perf_event_data structure which
> exports the pt_regs structure. This is OK for multiple architectures
> but fail for s390 and arm64 which do not export pt_regs. Programs
> using them, for example, the bpf selftest fail to compile on these
> architectures.
>
> For s390, exporting the pt_regs is not an option because s390 wants
> to allow changes to it. For arm64, there is a user_pt_regs structure
> that covers parts of the pt_regs structure for use by user space.
>
> To solve the broken uapi for s390 and arm64, introduce an abstract
> type for pt_regs and add an asm/bpf_perf_event.h file that concretes
> the type. An asm-generic header file covers the architectures that
> export pt_regs today.
>
> The arch-specific enablement for s390 and arm64 follows in separate
> commits.
>
> Reported-by: Thomas Richter <tmricht@linux.vnet.ibm.com>
> Fixes: 0515e5999a466dfe ("bpf: introduce BPF_PROG_TYPE_PERF_EVENT program type")
> Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com>
> Reviewed-and-tested-by: Thomas Richter <tmricht@linux.vnet.ibm.com>
> Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
> Cc: Jiri Olsa <jolsa@redhat.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> ---
> include/linux/perf_event.h | 6 +++++-
> include/uapi/asm-generic/bpf_perf_event.h | 9 +++++++++
> include/uapi/linux/bpf_perf_event.h | 5 ++---
> kernel/events/core.c | 2 +-
> 4 files changed, 17 insertions(+), 5 deletions(-)
> create mode 100644 include/uapi/asm-generic/bpf_perf_event.h
>
> diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
> index 2c9c87d..7546822 100644
> --- a/include/linux/perf_event.h
> +++ b/include/linux/perf_event.h
> @@ -15,6 +15,7 @@
> #define _LINUX_PERF_EVENT_H
>
> #include <uapi/linux/perf_event.h>
> +#include <uapi/linux/bpf_perf_event.h>
>
> /*
> * Kernel-internal data types and definitions:
> @@ -787,7 +788,7 @@ struct perf_output_handle {
> };
>
> struct bpf_perf_event_data_kern {
> - struct pt_regs *regs;
> + bpf_user_pt_regs_t *regs;
> struct perf_sample_data *data;
> struct perf_event *event;
> };
> @@ -1177,6 +1178,9 @@ extern void perf_tp_event(u16 event_type, u64 count, void *record,
> (user_mode(regs) ? PERF_RECORD_MISC_USER : PERF_RECORD_MISC_KERNEL)
> # define perf_instruction_pointer(regs) instruction_pointer(regs)
> #endif
> +#ifndef perf_arch_bpf_user_pt_regs
> +# define perf_arch_bpf_user_pt_regs(regs) regs
> +#endif
>
> static inline bool has_branch_stack(struct perf_event *event)
> {
> diff --git a/include/uapi/asm-generic/bpf_perf_event.h b/include/uapi/asm-generic/bpf_perf_event.h
> new file mode 100644
> index 0000000..53815d2
> --- /dev/null
> +++ b/include/uapi/asm-generic/bpf_perf_event.h
> @@ -0,0 +1,9 @@
> +#ifndef _UAPI__ASM_GENERIC_BPF_PERF_EVENT_H__
> +#define _UAPI__ASM_GENERIC_BPF_PERF_EVENT_H__
> +
> +#include <linux/ptrace.h>
> +
> +/* Export kernel pt_regs structure */
> +typedef struct pt_regs bpf_user_pt_regs_t;
> +
> +#endif /* _UAPI__ASM_GENERIC_BPF_PERF_EVENT_H__ */
> diff --git a/include/uapi/linux/bpf_perf_event.h b/include/uapi/linux/bpf_perf_event.h
> index af549d4..8f95303 100644
> --- a/include/uapi/linux/bpf_perf_event.h
> +++ b/include/uapi/linux/bpf_perf_event.h
> @@ -8,11 +8,10 @@
> #ifndef _UAPI__LINUX_BPF_PERF_EVENT_H__
> #define _UAPI__LINUX_BPF_PERF_EVENT_H__
>
> -#include <linux/types.h>
> -#include <linux/ptrace.h>
> +#include <asm/bpf_perf_event.h>
>
> struct bpf_perf_event_data {
> - struct pt_regs regs;
> + bpf_user_pt_regs_t regs;
> __u64 sample_period;
> };
Thank you for working on this problem.
The fix looks great to me.
While applying it I noticed few nits:
Applying: selftests/bpf: sync kernel headers and introduce arch support in Makefile
/w/bpf/.git/rebase-apply/patch:253: trailing whitespace.
freg_t fprs[NUM_FPRS];
/w/bpf/.git/rebase-apply/patch:262: trailing whitespace.
typedef struct
/w/bpf/.git/rebase-apply/patch:439: trailing whitespace.
} lowcore;
/w/bpf/.git/rebase-apply/patch:490: trailing whitespace.
} ptprot_area;
warning: 4 lines add whitespace errors.
Could you please fix those and resubmit ?
With that fixed feel free to add my
Acked-by: Alexei Starovoitov <ast@kernel.org>
to the patches.
I've tested it on arm64 and don't see any issues.
When resubmitting could you please reduce cc-list, since this set
went into spam folder for me and I noticed it only in patchworks.
Thanks
^ permalink raw reply
* Re: [PATCH net 0/2] tcp: add tcp_v4_fill_cb()/tcp_v4_restore_cb()
From: David Miller @ 2017-12-03 17:39 UTC (permalink / raw)
To: edumazet; +Cc: netdev, eric.dumazet, dsa, james.l.morris, casey
In-Reply-To: <20171203173300.21149-1-edumazet@google.com>
From: Eric Dumazet <edumazet@google.com>
Date: Sun, 3 Dec 2017 09:32:58 -0800
> James Morris reported kernel stack corruption bug that
> we tracked back to commit 971f10eca186 ("tcp: better TCP_SKB_CB
> layout to reduce cache line misses")
>
> First patch needs to be backported to kernels >= 3.18,
> while second patch needs to be backported to kernels >= 4.9, since
> this was the time when inet_exact_dif_match appeared.
Series applied and queued up for -stable, thanks Eric.
^ permalink raw reply
* Re: [RFC] virtio-net: help live migrate SR-IOV devices
From: Stephen Hemminger @ 2017-12-03 17:35 UTC (permalink / raw)
To: achiad shochat
Cc: Michael S. Tsirkin, Jakub Kicinski, Hannes Frederic Sowa,
Sridhar Samudrala, netdev, virtualization, Achiad,
Peter Waskiewicz Jr, Singhai, Anjali, Shannon Nelson,
Andy Gospodarek, Or Gerlitz
In-Reply-To: <CAEHy93+yhRtTQub0xk5W1jKRNisrcRBtNexcwRfWdSKPm5a4Gw@mail.gmail.com>
On Sun, 3 Dec 2017 11:14:37 +0200
achiad shochat <achiad.mellanox@gmail.com> wrote:
> On 3 December 2017 at 07:05, Michael S. Tsirkin <mst@redhat.com> wrote:
> > On Fri, Dec 01, 2017 at 12:08:59PM -0800, Shannon Nelson wrote:
> >> On 11/30/2017 6:11 AM, Michael S. Tsirkin wrote:
> >> > On Thu, Nov 30, 2017 at 10:08:45AM +0200, achiad shochat wrote:
> >> > > Re. problem #2:
> >> > > Indeed the best way to address it seems to be to enslave the VF driver
> >> > > netdev under a persistent anchor netdev.
> >> > > And it's indeed desired to allow (but not enforce) PV netdev and VF
> >> > > netdev to work in conjunction.
> >> > > And it's indeed desired that this enslavement logic work out-of-the box.
> >> > > But in case of PV+VF some configurable policies must be in place (and
> >> > > they'd better be generic rather than differ per PV technology).
> >> > > For example - based on which characteristics should the PV+VF coupling
> >> > > be done? netvsc uses MAC address, but that might not always be the
> >> > > desire.
> >> >
> >> > It's a policy but not guest userspace policy.
> >> >
> >> > The hypervisor certainly knows.
> >> >
> >> > Are you concerned that someone might want to create two devices with the
> >> > same MAC for an unrelated reason? If so, hypervisor could easily set a
> >> > flag in the virtio device to say "this is a backup, use MAC to find
> >> > another device".
> >>
> >> This is something I was going to suggest: a flag or other configuration on
> >> the virtio device to help control how this new feature is used. I can
> >> imagine this might be useful to control from either the hypervisor side or
> >> the VM side.
> >>
> >> The hypervisor might want to (1) disable it (force it off), (2) enable it
> >> for VM choice, or (3) force it on for the VM. In case (2), the VM might be
> >> able to chose whether it wants to make use of the feature, or stick with the
> >> bonding solution.
> >>
> >> Either way, the kernel is making a feature available, and the user (VM or
> >> hypervisor) is able to control it by selecting the feature based on the
> >> policy desired.
> >>
> >> sln
> >
> > I'm not sure what's the feature that is available here.
> >
> > I saw this as a flag that says "this device shares backend with another
> > network device which can be found using MAC, and that backend should be
> > preferred". kernel then forces configuration which uses that other
> > backend - as long as it exists.
> >
> > However, please Cc virtio-dev mailing list if we are doing this since
> > this is a spec extension.
> >
> > --
> > MST
>
>
> Can someone please explain why assume a virtio device is there at all??
> I specified a case where there isn't any.
>
> I second Jacob - having a netdev of one device driver enslave a netdev
> of another device driver is an awkward a-symmetric model.
> Regardless of whether they share the same backend device.
> Only I am not sure the Linux Bond is the right choice.
> e.g one may well want to use the virtio device also when the
> pass-through device is available, e.g for multicasts, east-west
> traffic, etc.
> I'm not sure the Linux Bond fits that functionality.
> And, as I hear in this thread, it is hard to make it work out of the box.
> So I think the right thing would be to write a new dedicated module
> for this purpose.
>
> Re policy -
> Indeed the HV can request a policy from the guest but that's not a
> claim for the virtio device enslaving the pass-through device.
> Any policy can be queried by the upper enslaving device.
>
> Bottom line - I do not see a single reason to have the virtio netdev
> (nor netvsc or any other PV netdev) enslave another netdev by itself.
> If we'd do it right with netvsc from the beginning we wouldn't need
> this discussion at all...
There are several issues with transparent migration.
The first is that the SR-IOV device needs to be shut off for earlier
in the migration process.
Next, the SR-IOV device in the migrated go guest environment maybe different.
It might not exist at all, it might be at a different PCI address, or it
could even be a different vendor/speed/model.
Keeping a virtual network device around allows persisting the connectivity,
during the process.
^ permalink raw reply
* Re: [PATCH net 0/2] tcp: fix SELinux/Smack corruptions
From: Eric Dumazet @ 2017-12-03 17:33 UTC (permalink / raw)
To: David Miller, edumazet; +Cc: netdev, dsa, james.l.morris, casey
In-Reply-To: <20171203.104228.2271114416251083134.davem@davemloft.net>
On Sun, 2017-12-03 at 10:42 -0500, David Miller wrote:
> From: Eric Dumazet <edumazet@google.com>
> Date: Fri, 1 Dec 2017 15:08:11 -0800
>
> > James Morris reported kernel stack corruption bug that
> > we tracked back to commit 971f10eca186 ("tcp: better TCP_SKB_CB
> > layout to reduce cache line misses")
> >
> > First patch needs to be backported to kernels >= 3.18,
> > while second patch needs to be backported to kernels >= 4.9, since
> > this was the time when inet_exact_dif_match appeared.
>
> I don't know why but patch #2 never made it into patchwork which
> makes
> it really difficult for me to queue it up for -stable properly and
> reliably.
>
> Can you please repost this series? Hopefully it will work.
>
> If it doesn't work we'll ask the patchwork folks to take a look.
>
> Thanks!
Sure, I have rebased and resent.
Thanks.
^ permalink raw reply
* [PATCH net 2/2] tcp: use IPCB instead of TCP_SKB_CB in inet_exact_dif_match()
From: Eric Dumazet @ 2017-12-03 17:33 UTC (permalink / raw)
To: David S . Miller
Cc: netdev, Eric Dumazet, David Ahern, James Morris, Casey Schaufler,
David Ahern, Eric Dumazet
In-Reply-To: <20171203173300.21149-1-edumazet@google.com>
From: David Ahern <dsahern@gmail.com>
After this fix : ("tcp: add tcp_v4_fill_cb()/tcp_v4_restore_cb()"),
socket lookups happen while skb->cb[] has not been mangled yet by TCP.
Fixes: a04a480d4392 ("net: Require exact match for TCP socket lookups if dif is l3mdev")
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
include/net/tcp.h | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 4e09398009c10a72478b43d3cffc24ba01612b91..6998707e81f343ef8d893c0b2ba16db541082230 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -844,12 +844,11 @@ static inline int tcp_v6_sdif(const struct sk_buff *skb)
}
#endif
-/* TCP_SKB_CB reference means this can not be used from early demux */
static inline bool inet_exact_dif_match(struct net *net, struct sk_buff *skb)
{
#if IS_ENABLED(CONFIG_NET_L3_MASTER_DEV)
if (!net->ipv4.sysctl_tcp_l3mdev_accept &&
- skb && ipv4_l3mdev_skb(TCP_SKB_CB(skb)->header.h4.flags))
+ skb && ipv4_l3mdev_skb(IPCB(skb)->flags))
return true;
#endif
return false;
--
2.15.0.531.g2ccb3012c9-goog
^ permalink raw reply related
* [PATCH net 1/2] tcp: add tcp_v4_fill_cb()/tcp_v4_restore_cb()
From: Eric Dumazet @ 2017-12-03 17:32 UTC (permalink / raw)
To: David S . Miller
Cc: netdev, Eric Dumazet, David Ahern, James Morris, Casey Schaufler,
Eric Dumazet
In-Reply-To: <20171203173300.21149-1-edumazet@google.com>
James Morris reported kernel stack corruption bug [1] while
running the SELinux testsuite, and bisected to a recent
commit bffa72cf7f9d ("net: sk_buff rbnode reorg")
We believe this commit is fine, but exposes an older bug.
SELinux code runs from tcp_filter() and might send an ICMP,
expecting IP options to be found in skb->cb[] using regular IPCB placement.
We need to defer TCP mangling of skb->cb[] after tcp_filter() calls.
This patch adds tcp_v4_fill_cb()/tcp_v4_restore_cb() in a very
similar way we added them for IPv6.
[1]
[ 339.806024] SELinux: failure in selinux_parse_skb(), unable to parse packet
[ 339.822505] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: ffffffff81745af5
[ 339.822505]
[ 339.852250] CPU: 4 PID: 3642 Comm: client Not tainted 4.15.0-rc1-test #15
[ 339.868498] Hardware name: LENOVO 10FGS0VA1L/30BC, BIOS FWKT68A 01/19/2017
[ 339.885060] Call Trace:
[ 339.896875] <IRQ>
[ 339.908103] dump_stack+0x63/0x87
[ 339.920645] panic+0xe8/0x248
[ 339.932668] ? ip_push_pending_frames+0x33/0x40
[ 339.946328] ? icmp_send+0x525/0x530
[ 339.958861] ? kfree_skbmem+0x60/0x70
[ 339.971431] __stack_chk_fail+0x1b/0x20
[ 339.984049] icmp_send+0x525/0x530
[ 339.996205] ? netlbl_skbuff_err+0x36/0x40
[ 340.008997] ? selinux_netlbl_err+0x11/0x20
[ 340.021816] ? selinux_socket_sock_rcv_skb+0x211/0x230
[ 340.035529] ? security_sock_rcv_skb+0x3b/0x50
[ 340.048471] ? sk_filter_trim_cap+0x44/0x1c0
[ 340.061246] ? tcp_v4_inbound_md5_hash+0x69/0x1b0
[ 340.074562] ? tcp_filter+0x2c/0x40
[ 340.086400] ? tcp_v4_rcv+0x820/0xa20
[ 340.098329] ? ip_local_deliver_finish+0x71/0x1a0
[ 340.111279] ? ip_local_deliver+0x6f/0xe0
[ 340.123535] ? ip_rcv_finish+0x3a0/0x3a0
[ 340.135523] ? ip_rcv_finish+0xdb/0x3a0
[ 340.147442] ? ip_rcv+0x27c/0x3c0
[ 340.158668] ? inet_del_offload+0x40/0x40
[ 340.170580] ? __netif_receive_skb_core+0x4ac/0x900
[ 340.183285] ? rcu_accelerate_cbs+0x5b/0x80
[ 340.195282] ? __netif_receive_skb+0x18/0x60
[ 340.207288] ? process_backlog+0x95/0x140
[ 340.218948] ? net_rx_action+0x26c/0x3b0
[ 340.230416] ? __do_softirq+0xc9/0x26a
[ 340.241625] ? do_softirq_own_stack+0x2a/0x40
[ 340.253368] </IRQ>
[ 340.262673] ? do_softirq+0x50/0x60
[ 340.273450] ? __local_bh_enable_ip+0x57/0x60
[ 340.285045] ? ip_finish_output2+0x175/0x350
[ 340.296403] ? ip_finish_output+0x127/0x1d0
[ 340.307665] ? nf_hook_slow+0x3c/0xb0
[ 340.318230] ? ip_output+0x72/0xe0
[ 340.328524] ? ip_fragment.constprop.54+0x80/0x80
[ 340.340070] ? ip_local_out+0x35/0x40
[ 340.350497] ? ip_queue_xmit+0x15c/0x3f0
[ 340.361060] ? __kmalloc_reserve.isra.40+0x31/0x90
[ 340.372484] ? __skb_clone+0x2e/0x130
[ 340.382633] ? tcp_transmit_skb+0x558/0xa10
[ 340.393262] ? tcp_connect+0x938/0xad0
[ 340.403370] ? ktime_get_with_offset+0x4c/0xb0
[ 340.414206] ? tcp_v4_connect+0x457/0x4e0
[ 340.424471] ? __inet_stream_connect+0xb3/0x300
[ 340.435195] ? inet_stream_connect+0x3b/0x60
[ 340.445607] ? SYSC_connect+0xd9/0x110
[ 340.455455] ? __audit_syscall_entry+0xaf/0x100
[ 340.466112] ? syscall_trace_enter+0x1d0/0x2b0
[ 340.476636] ? __audit_syscall_exit+0x209/0x290
[ 340.487151] ? SyS_connect+0xe/0x10
[ 340.496453] ? do_syscall_64+0x67/0x1b0
[ 340.506078] ? entry_SYSCALL64_slow_path+0x25/0x25
Fixes: 971f10eca186 ("tcp: better TCP_SKB_CB layout to reduce cache line misses")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Reported-by: James Morris <james.l.morris@oracle.com>
Tested-by: James Morris <james.l.morris@oracle.com>
Tested-by: Casey Schaufler <casey@schaufler-ca.com>
---
net/ipv4/tcp_ipv4.c | 59 ++++++++++++++++++++++++++++++++++++-----------------
net/ipv6/tcp_ipv6.c | 10 +++++----
2 files changed, 46 insertions(+), 23 deletions(-)
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index c6bc0c4d19c624888b0d0b5a4246c7183edf63f5..77ea45da0fe9c746907a312989658af3ad3b198d 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1591,6 +1591,34 @@ int tcp_filter(struct sock *sk, struct sk_buff *skb)
}
EXPORT_SYMBOL(tcp_filter);
+static void tcp_v4_restore_cb(struct sk_buff *skb)
+{
+ memmove(IPCB(skb), &TCP_SKB_CB(skb)->header.h4,
+ sizeof(struct inet_skb_parm));
+}
+
+static void tcp_v4_fill_cb(struct sk_buff *skb, const struct iphdr *iph,
+ const struct tcphdr *th)
+{
+ /* This is tricky : We move IPCB at its correct location into TCP_SKB_CB()
+ * barrier() makes sure compiler wont play fool^Waliasing games.
+ */
+ memmove(&TCP_SKB_CB(skb)->header.h4, IPCB(skb),
+ sizeof(struct inet_skb_parm));
+ barrier();
+
+ TCP_SKB_CB(skb)->seq = ntohl(th->seq);
+ TCP_SKB_CB(skb)->end_seq = (TCP_SKB_CB(skb)->seq + th->syn + th->fin +
+ skb->len - th->doff * 4);
+ TCP_SKB_CB(skb)->ack_seq = ntohl(th->ack_seq);
+ TCP_SKB_CB(skb)->tcp_flags = tcp_flag_byte(th);
+ TCP_SKB_CB(skb)->tcp_tw_isn = 0;
+ TCP_SKB_CB(skb)->ip_dsfield = ipv4_get_dsfield(iph);
+ TCP_SKB_CB(skb)->sacked = 0;
+ TCP_SKB_CB(skb)->has_rxtstamp =
+ skb->tstamp || skb_hwtstamps(skb)->hwtstamp;
+}
+
/*
* From tcp_input.c
*/
@@ -1631,24 +1659,6 @@ int tcp_v4_rcv(struct sk_buff *skb)
th = (const struct tcphdr *)skb->data;
iph = ip_hdr(skb);
- /* This is tricky : We move IPCB at its correct location into TCP_SKB_CB()
- * barrier() makes sure compiler wont play fool^Waliasing games.
- */
- memmove(&TCP_SKB_CB(skb)->header.h4, IPCB(skb),
- sizeof(struct inet_skb_parm));
- barrier();
-
- TCP_SKB_CB(skb)->seq = ntohl(th->seq);
- TCP_SKB_CB(skb)->end_seq = (TCP_SKB_CB(skb)->seq + th->syn + th->fin +
- skb->len - th->doff * 4);
- TCP_SKB_CB(skb)->ack_seq = ntohl(th->ack_seq);
- TCP_SKB_CB(skb)->tcp_flags = tcp_flag_byte(th);
- TCP_SKB_CB(skb)->tcp_tw_isn = 0;
- TCP_SKB_CB(skb)->ip_dsfield = ipv4_get_dsfield(iph);
- TCP_SKB_CB(skb)->sacked = 0;
- TCP_SKB_CB(skb)->has_rxtstamp =
- skb->tstamp || skb_hwtstamps(skb)->hwtstamp;
-
lookup:
sk = __inet_lookup_skb(&tcp_hashinfo, skb, __tcp_hdrlen(th), th->source,
th->dest, sdif, &refcounted);
@@ -1679,14 +1689,19 @@ int tcp_v4_rcv(struct sk_buff *skb)
sock_hold(sk);
refcounted = true;
nsk = NULL;
- if (!tcp_filter(sk, skb))
+ if (!tcp_filter(sk, skb)) {
+ th = (const struct tcphdr *)skb->data;
+ iph = ip_hdr(skb);
+ tcp_v4_fill_cb(skb, iph, th);
nsk = tcp_check_req(sk, skb, req, false);
+ }
if (!nsk) {
reqsk_put(req);
goto discard_and_relse;
}
if (nsk == sk) {
reqsk_put(req);
+ tcp_v4_restore_cb(skb);
} else if (tcp_child_process(sk, nsk, skb)) {
tcp_v4_send_reset(nsk, skb);
goto discard_and_relse;
@@ -1712,6 +1727,7 @@ int tcp_v4_rcv(struct sk_buff *skb)
goto discard_and_relse;
th = (const struct tcphdr *)skb->data;
iph = ip_hdr(skb);
+ tcp_v4_fill_cb(skb, iph, th);
skb->dev = NULL;
@@ -1742,6 +1758,8 @@ int tcp_v4_rcv(struct sk_buff *skb)
if (!xfrm4_policy_check(NULL, XFRM_POLICY_IN, skb))
goto discard_it;
+ tcp_v4_fill_cb(skb, iph, th);
+
if (tcp_checksum_complete(skb)) {
csum_error:
__TCP_INC_STATS(net, TCP_MIB_CSUMERRORS);
@@ -1768,6 +1786,8 @@ int tcp_v4_rcv(struct sk_buff *skb)
goto discard_it;
}
+ tcp_v4_fill_cb(skb, iph, th);
+
if (tcp_checksum_complete(skb)) {
inet_twsk_put(inet_twsk(sk));
goto csum_error;
@@ -1784,6 +1804,7 @@ int tcp_v4_rcv(struct sk_buff *skb)
if (sk2) {
inet_twsk_deschedule_put(inet_twsk(sk));
sk = sk2;
+ tcp_v4_restore_cb(skb);
refcounted = false;
goto process;
}
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index be11dc13aa705145a83177e17d23594e9416e11a..1f04ec0e4a7aa2c11b8ee27cbdd4067b5bcf32e5 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1454,7 +1454,6 @@ static int tcp_v6_rcv(struct sk_buff *skb)
struct sock *nsk;
sk = req->rsk_listener;
- tcp_v6_fill_cb(skb, hdr, th);
if (tcp_v6_inbound_md5_hash(sk, skb)) {
sk_drops_add(sk, skb);
reqsk_put(req);
@@ -1467,8 +1466,12 @@ static int tcp_v6_rcv(struct sk_buff *skb)
sock_hold(sk);
refcounted = true;
nsk = NULL;
- if (!tcp_filter(sk, skb))
+ if (!tcp_filter(sk, skb)) {
+ th = (const struct tcphdr *)skb->data;
+ hdr = ipv6_hdr(skb);
+ tcp_v6_fill_cb(skb, hdr, th);
nsk = tcp_check_req(sk, skb, req, false);
+ }
if (!nsk) {
reqsk_put(req);
goto discard_and_relse;
@@ -1492,8 +1495,6 @@ static int tcp_v6_rcv(struct sk_buff *skb)
if (!xfrm6_policy_check(sk, XFRM_POLICY_IN, skb))
goto discard_and_relse;
- tcp_v6_fill_cb(skb, hdr, th);
-
if (tcp_v6_inbound_md5_hash(sk, skb))
goto discard_and_relse;
@@ -1501,6 +1502,7 @@ static int tcp_v6_rcv(struct sk_buff *skb)
goto discard_and_relse;
th = (const struct tcphdr *)skb->data;
hdr = ipv6_hdr(skb);
+ tcp_v6_fill_cb(skb, hdr, th);
skb->dev = NULL;
--
2.15.0.531.g2ccb3012c9-goog
^ permalink raw reply related
* [PATCH net 0/2] tcp: add tcp_v4_fill_cb()/tcp_v4_restore_cb()
From: Eric Dumazet @ 2017-12-03 17:32 UTC (permalink / raw)
To: David S . Miller
Cc: netdev, Eric Dumazet, David Ahern, James Morris, Casey Schaufler,
Eric Dumazet
James Morris reported kernel stack corruption bug that
we tracked back to commit 971f10eca186 ("tcp: better TCP_SKB_CB
layout to reduce cache line misses")
First patch needs to be backported to kernels >= 3.18,
while second patch needs to be backported to kernels >= 4.9, since
this was the time when inet_exact_dif_match appeared.
David Ahern (1):
tcp: use IPCB instead of TCP_SKB_CB in inet_exact_dif_match()
Eric Dumazet (1):
tcp: add tcp_v4_fill_cb()/tcp_v4_restore_cb()
include/net/tcp.h | 3 +--
net/ipv4/tcp_ipv4.c | 59 ++++++++++++++++++++++++++++++++++++-----------------
net/ipv6/tcp_ipv6.c | 10 +++++----
3 files changed, 47 insertions(+), 25 deletions(-)
--
2.15.0.531.g2ccb3012c9-goog
^ permalink raw reply
* Re: [PATCH v3 1/6] perf: prepare perf_event.h for new types perf_kprobe and perf_uprobe
From: Alexei Starovoitov @ 2017-12-03 17:03 UTC (permalink / raw)
To: Song Liu
Cc: peterz, rostedt, mingo, davem, netdev, linux-kernel, daniel,
kernel-team
In-Reply-To: <20171130235023.1414663-4-songliubraving@fb.com>
On Thu, Nov 30, 2017 at 03:50:18PM -0800, Song Liu wrote:
> Two new perf types, perf_kprobe and perf_uprobe, will be added to allow
> creating [k,u]probe with perf_event_open. These [k,u]probe are associated
> with the file decriptor created by perf_event_open, thus are easy to
> clean when the file descriptor is destroyed.
>
> kprobe_func and uprobe_path are added to union config1 for pointers to
> function name for kprobe or binary path for uprobe.
>
> kprobe_addr and probe_offset are added to union config2 for kernel
> address (when kprobe_func is NULL), or [k,u]probe offset.
>
> Signed-off-by: Song Liu <songliubraving@fb.com>
> Reviewed-by: Yonghong Song <yhs@fb.com>
> Reviewed-by: Josef Bacik <jbacik@fb.com>
> Acked-by: Alexei Starovoitov <ast@kernel.org>
> ---
> include/uapi/linux/perf_event.h | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
> index 362493a..247c6cb 100644
> --- a/include/uapi/linux/perf_event.h
> +++ b/include/uapi/linux/perf_event.h
> @@ -299,6 +299,8 @@ enum perf_event_read_format {
> #define PERF_ATTR_SIZE_VER4 104 /* add: sample_regs_intr */
> #define PERF_ATTR_SIZE_VER5 112 /* add: aux_watermark */
>
> +#define MAX_PROBE_FUNC_NAME_LEN 64
I think we have to remove this restriction.
There are already functions with names longer than 64 characters
in the current vmlinux:
trace_event_define_fields_ext4_ext_convert_to_initialized_fastpath
trace_event_define_fields_mm_vmscan_direct_reclaim_begin_template
How about we drop this restriction and use NAME_MAX internally
without adding new uapi defines ?
^ permalink raw reply
* Re: [PATCH v5 net-next,mips 6/7] netdev: octeon-ethernet: Add Cavium Octeon III support.
From: David Miller @ 2017-12-03 16:26 UTC (permalink / raw)
To: david.daney-YGCgFSpz5w/QT0dZR+AlfA
Cc: linux-mips-6z/3iImG2C8G8FEW9MqTrA, ralf-6z/3iImG2C8G8FEW9MqTrA,
james.hogan-8NJIiSa5LzA, netdev-u79uwXL29TY76Z2rM5mHXA,
robh+dt-DgEjT+Ai2ygdnm+yROfE0A, mark.rutland-5wv7dgnIgG8,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
steven.hill-YGCgFSpz5w/QT0dZR+AlfA,
devicetree-u79uwXL29TY76Z2rM5mHXA, andrew-g2DYL2Zd6BY,
f.fainelli-Re5JQEeQqe8AvxtiuMwx3w, pombredanne-od1rfyK75/E,
cmunoz-YGCgFSpz5w/QT0dZR+AlfA
In-Reply-To: <20171201231807.25266-7-david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
From: David Daney <david.daney-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Date: Fri, 1 Dec 2017 15:18:06 -0800
> +static char *mix_port;
> +module_param(mix_port, charp, 0444);
> +MODULE_PARM_DESC(mix_port, "Specifies which ports connect to MIX interfaces.");
> +
> +static char *pki_port;
> +module_param(pki_port, charp, 0444);
> +MODULE_PARM_DESC(pki_port, "Specifies which ports connect to the PKI.");
Please no module parameters.
Please instead find a way to determine or configure these elements
at run time with generic configuration interfaces.
> +
> +static int bgx_probe(struct platform_device *pdev)
> +{
> + struct mac_platform_data platform_data;
> + const __be32 *reg;
> + u32 port;
> + u64 addr;
> + struct device_node *child;
> + struct platform_device *new_dev;
> + struct platform_device *pki_dev;
> + int numa_node, interface;
> + int i;
> + int r = 0;
> + char id[64];
> + u64 data;
Please use reverse-christmas-tree ordering (longest to shortest line) for
local variable declarations.
Please fix this in your entire submission.
> +static int bgx_mix_init_from_fdt(void)
> +{
> + struct device_node *node;
> + struct device_node *parent = NULL;
> + int mix = 0;
> +
Please do not use tabs like this when declaring local variables.
Much worse, some functions use this style whereas others do not,
be consistent otherwise your code is very hard to read.
Please fix this for your entire submission not just this specific
case I am pointing out.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH v2 net-next] netlink: optimize err assignment
From: David Miller @ 2017-12-03 16:22 UTC (permalink / raw)
To: eric.dumazet; +Cc: cugyly, netdev, Linyu.Yuan
In-Reply-To: <1512314409.19682.51.camel@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Sun, 03 Dec 2017 07:20:09 -0800
> On Sun, 2017-12-03 at 21:10 +0800, yuan linyu wrote:
>> From: yuan linyu <Linyu.Yuan@alcatel-sbell.com.cn>
>>
>> Signed-off-by: yuan linyu <Linyu.Yuan@alcatel-sbell.com.cn>
>> ---
>> v2: fix kbuild test warning
>> ---
>> net/netlink/af_netlink.c | 52 ++++++++++++++++++++----------------
>> ------------
>> 1 file changed, 22 insertions(+), 30 deletions(-)
>>
>
> I see no reason why we should accept this code churn.
>
> This kind of change makes future fix backports harder.
I agree, and I've been trying to discourage these "improvements"
as much as possible.
^ permalink raw reply
* Re: [PATCH net-next V2 1/2] net-next: use five-tuple hash for sk_txhash
From: Tom Herbert @ 2017-12-03 16:02 UTC (permalink / raw)
To: David Miller
Cc: Shaohua Li, Linux Kernel Network Developers, Martin Lau,
Eric Dumazet, flo, Cong Wang, Shaohua Li
In-Reply-To: <20171203.103801.90553633657825548.davem@davemloft.net>
On Sun, Dec 3, 2017 at 7:38 AM, David Miller <davem@davemloft.net> wrote:
> From: Shaohua Li <shli@kernel.org>
> Date: Fri, 1 Dec 2017 13:00:43 -0800
>
>> This causes our router doesn't correctly close tcp connection.
>
> Then please fix your router.
>
> How many times do I have to say this... The flowlabel is not part of
> the socket connection identity, therefore you cannot use it for
> connection state.
>
> The more of these kinds of patches with this kind of nonsense in the
> commit message I let into the tree the more this illusion of the
> flowlabel meaning something on the connection level is made to seem
> like reality.
>
> Can we please stop pretending that the flowlabel is part of the
> saddr/sport/daddr/dport socket identity? Please???
>
> I don't mind the flowlabel being set correctly, but your justification
> stinks.
Dave,
The problem isn't us, it's the rest of the world. There are countless
network devices that maintain connection state (load balancers,
firewalls, NAT, etc.). They force a requirement that all packets for a
flow follow the same path route through their device. This is
fundamentally incorrect per the architecture of Internet protocols,
but nevertheless it is pervasive and not going away anytime soon. If
the flow label is not persistent during a flow and used for ECMP then
flows through these devices can be broken. This is precisely why there
are some network operators running around now telling people to turn
off the flow label for ECMP (and continue doing DPI). We're not going
to win the argument that they need to fix their architecture, making
flow labels persistent as a default is a pragmatic solution.
Tom
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox