* [PATCH 17/20] can: rx-offload: introduce can_rx_offload_get_echo_skb() and can_rx_offload_queue_sorted() functions
From: Marc Kleine-Budde @ 2018-11-12 11:57 UTC (permalink / raw)
To: netdev
Cc: davem, linux-can, kernel, Oleksij Rempel, linux-stable,
Marc Kleine-Budde
In-Reply-To: <20181112115728.18331-1-mkl@pengutronix.de>
From: Oleksij Rempel <o.rempel@pengutronix.de>
Current CAN framework can't guarantee proper/chronological order
of RX and TX-ECHO messages. To make this possible, drivers should use
this functions instead of can_get_echo_skb().
Signed-off-by: Oleksij Rempel <o.rempel@pengutronix.de>
Cc: linux-stable <stable@vger.kernel.org>
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
drivers/net/can/rx-offload.c | 46 ++++++++++++++++++++++++++++++++++
include/linux/can/rx-offload.h | 4 +++
2 files changed, 50 insertions(+)
diff --git a/drivers/net/can/rx-offload.c b/drivers/net/can/rx-offload.c
index c7d05027a7a0..c368686e2164 100644
--- a/drivers/net/can/rx-offload.c
+++ b/drivers/net/can/rx-offload.c
@@ -211,6 +211,52 @@ int can_rx_offload_irq_offload_fifo(struct can_rx_offload *offload)
}
EXPORT_SYMBOL_GPL(can_rx_offload_irq_offload_fifo);
+int can_rx_offload_queue_sorted(struct can_rx_offload *offload,
+ struct sk_buff *skb, u32 timestamp)
+{
+ struct can_rx_offload_cb *cb;
+ unsigned long flags;
+
+ if (skb_queue_len(&offload->skb_queue) >
+ offload->skb_queue_len_max)
+ return -ENOMEM;
+
+ cb = can_rx_offload_get_cb(skb);
+ cb->timestamp = timestamp;
+
+ spin_lock_irqsave(&offload->skb_queue.lock, flags);
+ __skb_queue_add_sort(&offload->skb_queue, skb, can_rx_offload_compare);
+ spin_unlock_irqrestore(&offload->skb_queue.lock, flags);
+
+ can_rx_offload_schedule(offload);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(can_rx_offload_queue_sorted);
+
+unsigned int can_rx_offload_get_echo_skb(struct can_rx_offload *offload,
+ unsigned int idx, u32 timestamp)
+{
+ struct net_device *dev = offload->dev;
+ struct net_device_stats *stats = &dev->stats;
+ struct sk_buff *skb;
+ u8 len;
+ int err;
+
+ skb = __can_get_echo_skb(dev, idx, &len);
+ if (!skb)
+ return 0;
+
+ err = can_rx_offload_queue_sorted(offload, skb, timestamp);
+ if (err) {
+ stats->rx_errors++;
+ stats->tx_fifo_errors++;
+ }
+
+ return len;
+}
+EXPORT_SYMBOL_GPL(can_rx_offload_get_echo_skb);
+
int can_rx_offload_irq_queue_err_skb(struct can_rx_offload *offload, struct sk_buff *skb)
{
if (skb_queue_len(&offload->skb_queue) >
diff --git a/include/linux/can/rx-offload.h b/include/linux/can/rx-offload.h
index cb31683bbe15..01a7c9e5d8d8 100644
--- a/include/linux/can/rx-offload.h
+++ b/include/linux/can/rx-offload.h
@@ -41,6 +41,10 @@ int can_rx_offload_add_timestamp(struct net_device *dev, struct can_rx_offload *
int can_rx_offload_add_fifo(struct net_device *dev, struct can_rx_offload *offload, unsigned int weight);
int can_rx_offload_irq_offload_timestamp(struct can_rx_offload *offload, u64 reg);
int can_rx_offload_irq_offload_fifo(struct can_rx_offload *offload);
+int can_rx_offload_queue_sorted(struct can_rx_offload *offload,
+ struct sk_buff *skb, u32 timestamp);
+unsigned int can_rx_offload_get_echo_skb(struct can_rx_offload *offload,
+ unsigned int idx, u32 timestamp);
int can_rx_offload_irq_queue_err_skb(struct can_rx_offload *offload, struct sk_buff *skb);
void can_rx_offload_reset(struct can_rx_offload *offload);
void can_rx_offload_del(struct can_rx_offload *offload);
--
2.19.1
^ permalink raw reply related
* [PATCH 18/20] can: flexcan: handle tx-complete CAN frames via rx-offload infrastructure
From: Marc Kleine-Budde @ 2018-11-12 11:57 UTC (permalink / raw)
To: netdev
Cc: davem, linux-can, kernel, Oleksij Rempel, linux-stable,
Marc Kleine-Budde
In-Reply-To: <20181112115728.18331-1-mkl@pengutronix.de>
From: Oleksij Rempel <o.rempel@pengutronix.de>
Current flexcan driver will put TX-ECHO in regular unsorted way, in
this case TX-ECHO can come after the response to the same TXed message.
In some cases, for example for J1939 stack, things will break.
This patch is using new rx-offload API to put the messages just in the
right place.
Signed-off-by: Oleksij Rempel <o.rempel@pengutronix.de>
Cc: linux-stable <stable@vger.kernel.org>
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
drivers/net/can/flexcan.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/net/can/flexcan.c b/drivers/net/can/flexcan.c
index 68b46395c580..41a175f80c4b 100644
--- a/drivers/net/can/flexcan.c
+++ b/drivers/net/can/flexcan.c
@@ -787,8 +787,11 @@ static irqreturn_t flexcan_irq(int irq, void *dev_id)
/* transmission complete interrupt */
if (reg_iflag2 & FLEXCAN_IFLAG_MB(FLEXCAN_TX_MB)) {
+ u32 reg_ctrl = priv->read(®s->mb[FLEXCAN_TX_MB].can_ctrl);
+
handled = IRQ_HANDLED;
- stats->tx_bytes += can_get_echo_skb(dev, 0);
+ stats->tx_bytes += can_rx_offload_get_echo_skb(&priv->offload,
+ 0, reg_ctrl << 16);
stats->tx_packets++;
can_led_event(dev, CAN_LED_EVENT_TX);
--
2.19.1
^ permalink raw reply related
* [PATCH 19/20] can: rx-offload: rename can_rx_offload_irq_queue_err_skb() to can_rx_offload_queue_tail()
From: Marc Kleine-Budde @ 2018-11-12 11:57 UTC (permalink / raw)
To: netdev
Cc: davem, linux-can, kernel, Oleksij Rempel, linux-stable,
Marc Kleine-Budde
In-Reply-To: <20181112115728.18331-1-mkl@pengutronix.de>
From: Oleksij Rempel <o.rempel@pengutronix.de>
This function has nothing todo with error.
Signed-off-by: Oleksij Rempel <o.rempel@pengutronix.de>
Cc: linux-stable <stable@vger.kernel.org>
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
drivers/net/can/flexcan.c | 4 ++--
drivers/net/can/rx-offload.c | 5 +++--
include/linux/can/rx-offload.h | 3 ++-
3 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/drivers/net/can/flexcan.c b/drivers/net/can/flexcan.c
index 41a175f80c4b..5923bd0ec118 100644
--- a/drivers/net/can/flexcan.c
+++ b/drivers/net/can/flexcan.c
@@ -610,7 +610,7 @@ static void flexcan_irq_bus_err(struct net_device *dev, u32 reg_esr)
if (tx_errors)
dev->stats.tx_errors++;
- can_rx_offload_irq_queue_err_skb(&priv->offload, skb);
+ can_rx_offload_queue_tail(&priv->offload, skb);
}
static void flexcan_irq_state(struct net_device *dev, u32 reg_esr)
@@ -650,7 +650,7 @@ static void flexcan_irq_state(struct net_device *dev, u32 reg_esr)
if (unlikely(new_state == CAN_STATE_BUS_OFF))
can_bus_off(dev);
- can_rx_offload_irq_queue_err_skb(&priv->offload, skb);
+ can_rx_offload_queue_tail(&priv->offload, skb);
}
static inline struct flexcan_priv *rx_offload_to_priv(struct can_rx_offload *offload)
diff --git a/drivers/net/can/rx-offload.c b/drivers/net/can/rx-offload.c
index c368686e2164..2ce4fa8698c7 100644
--- a/drivers/net/can/rx-offload.c
+++ b/drivers/net/can/rx-offload.c
@@ -257,7 +257,8 @@ unsigned int can_rx_offload_get_echo_skb(struct can_rx_offload *offload,
}
EXPORT_SYMBOL_GPL(can_rx_offload_get_echo_skb);
-int can_rx_offload_irq_queue_err_skb(struct can_rx_offload *offload, struct sk_buff *skb)
+int can_rx_offload_queue_tail(struct can_rx_offload *offload,
+ struct sk_buff *skb)
{
if (skb_queue_len(&offload->skb_queue) >
offload->skb_queue_len_max)
@@ -268,7 +269,7 @@ int can_rx_offload_irq_queue_err_skb(struct can_rx_offload *offload, struct sk_b
return 0;
}
-EXPORT_SYMBOL_GPL(can_rx_offload_irq_queue_err_skb);
+EXPORT_SYMBOL_GPL(can_rx_offload_queue_tail);
static int can_rx_offload_init_queue(struct net_device *dev, struct can_rx_offload *offload, unsigned int weight)
{
diff --git a/include/linux/can/rx-offload.h b/include/linux/can/rx-offload.h
index 01a7c9e5d8d8..8268811a697e 100644
--- a/include/linux/can/rx-offload.h
+++ b/include/linux/can/rx-offload.h
@@ -45,7 +45,8 @@ int can_rx_offload_queue_sorted(struct can_rx_offload *offload,
struct sk_buff *skb, u32 timestamp);
unsigned int can_rx_offload_get_echo_skb(struct can_rx_offload *offload,
unsigned int idx, u32 timestamp);
-int can_rx_offload_irq_queue_err_skb(struct can_rx_offload *offload, struct sk_buff *skb);
+int can_rx_offload_queue_tail(struct can_rx_offload *offload,
+ struct sk_buff *skb);
void can_rx_offload_reset(struct can_rx_offload *offload);
void can_rx_offload_del(struct can_rx_offload *offload);
void can_rx_offload_enable(struct can_rx_offload *offload);
--
2.19.1
^ permalink raw reply related
* [PATCH 20/20] can: flexcan: use can_rx_offload_queue_sorted() for flexcan_irq_bus_*()
From: Marc Kleine-Budde @ 2018-11-12 11:57 UTC (permalink / raw)
To: netdev
Cc: davem, linux-can, kernel, Oleksij Rempel, linux-stable,
Marc Kleine-Budde
In-Reply-To: <20181112115728.18331-1-mkl@pengutronix.de>
From: Oleksij Rempel <o.rempel@pengutronix.de>
Currently, in case of bus error, driver will generate error message and put
in the tail of the message queue. To avoid confusions, this change should
place the bus related messages in proper order.
Signed-off-by: Oleksij Rempel <o.rempel@pengutronix.de>
Cc: linux-stable <stable@vger.kernel.org>
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
---
drivers/net/can/flexcan.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/net/can/flexcan.c b/drivers/net/can/flexcan.c
index 5923bd0ec118..75ce11395ee8 100644
--- a/drivers/net/can/flexcan.c
+++ b/drivers/net/can/flexcan.c
@@ -561,9 +561,13 @@ static netdev_tx_t flexcan_start_xmit(struct sk_buff *skb, struct net_device *de
static void flexcan_irq_bus_err(struct net_device *dev, u32 reg_esr)
{
struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
struct sk_buff *skb;
struct can_frame *cf;
bool rx_errors = false, tx_errors = false;
+ u32 timestamp;
+
+ timestamp = priv->read(®s->timer) << 16;
skb = alloc_can_err_skb(dev, &cf);
if (unlikely(!skb))
@@ -610,17 +614,21 @@ static void flexcan_irq_bus_err(struct net_device *dev, u32 reg_esr)
if (tx_errors)
dev->stats.tx_errors++;
- can_rx_offload_queue_tail(&priv->offload, skb);
+ can_rx_offload_queue_sorted(&priv->offload, skb, timestamp);
}
static void flexcan_irq_state(struct net_device *dev, u32 reg_esr)
{
struct flexcan_priv *priv = netdev_priv(dev);
+ struct flexcan_regs __iomem *regs = priv->regs;
struct sk_buff *skb;
struct can_frame *cf;
enum can_state new_state, rx_state, tx_state;
int flt;
struct can_berr_counter bec;
+ u32 timestamp;
+
+ timestamp = priv->read(®s->timer) << 16;
flt = reg_esr & FLEXCAN_ESR_FLT_CONF_MASK;
if (likely(flt == FLEXCAN_ESR_FLT_CONF_ACTIVE)) {
@@ -650,7 +658,7 @@ static void flexcan_irq_state(struct net_device *dev, u32 reg_esr)
if (unlikely(new_state == CAN_STATE_BUS_OFF))
can_bus_off(dev);
- can_rx_offload_queue_tail(&priv->offload, skb);
+ can_rx_offload_queue_sorted(&priv->offload, skb, timestamp);
}
static inline struct flexcan_priv *rx_offload_to_priv(struct can_rx_offload *offload)
--
2.19.1
^ permalink raw reply related
* [PATCH RFC net-next] net: SAIL based FIB lookup for XDP
From: Md. Islam @ 2018-11-12 2:25 UTC (permalink / raw)
To: Netdev, David Ahern, Jesper Dangaard Brouer, David Miller,
Stephen Hemminger, john fastabend, Alexey Kuznetsov
This patch implements SAIL[1] based routing table lookup for XDP. I
however made some changes from the original proposal (details are
described in the patch). This changes decreased the memory consumption
from 21.94 MB to 4.97 MB for my example routing table with 400K
routes.
This patch can perform FIB lookup in 50 CPU cycles for the example
routing table (with 400K routes) whereas LC-trie takes around 350 CPU
cycles.
I tried to follow all the advice I got from my last patch. Looking
forward to your review.
1. Yang, Tong, et al. "Guarantee IP lookup performance with FIB
explosion." ACM SIGCOMM Computer Communication Review. Vol. 44. No. 4.
ACM, 2014.
Thanks
Tamim
Signed-off-by: MD Iftakharul Islam (Tamim) <mislam4@kent.edu>
---
MAINTAINERS | 5 +
include/net/ip_fib.h | 60 ++++
net/core/filter.c | 49 +++
net/ipv4/Kconfig | 11 +
net/ipv4/Makefile | 1 +
net/ipv4/fib_sail_xdp.c | 939 ++++++++++++++++++++++++++++++++++++++++++++++++
net/ipv4/fib_trie.c | 12 +
7 files changed, 1077 insertions(+)
create mode 100644 net/ipv4/fib_sail_xdp.c
diff --git a/MAINTAINERS b/MAINTAINERS
index b22e7fd..969e625 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7630,6 +7630,11 @@ M: Juanjo Ciarlante <jjciarla@raiz.uncu.edu.ar>
S: Maintained
F: net/ipv4/netfilter/ipt_MASQUERADE.c
+SAIL FIB LOOKUP
+M: MD Iftakharul Islam (Tamim) <tamim@csebuet.org>
+S: Maintained
+F: net/ipv4/fib_sail_xdp.c
+
IPMI SUBSYSTEM
M: Corey Minyard <minyard@acm.org>
L: openipmi-developer@lists.sourceforge.net (moderated for non-subscribers)
diff --git a/include/net/ip_fib.h b/include/net/ip_fib.h
index 69c91d1..cc275c7 100644
--- a/include/net/ip_fib.h
+++ b/include/net/ip_fib.h
@@ -197,6 +197,62 @@ struct fib_entry_notifier_info {
u32 tb_id;
};
+#if IS_ENABLED(CONFIG_FIB_SAIL_XDP)
+/*
+ * The router can have upto 255 ports. This limitation
+ * allows us to represent netdev_index as an u8
+ */
+#define NETDEV_COUNT_MAX 255
+
+struct chunk {
+ /*256-bit bitmap. Here i-th bit (from LSB) is set to 1 if C24[i] > 0 */
+ u64 bitmap[4];
+ /*
+ * Index to C24 where chunk is started. A chunk corresponds
+ * to 256 elements. Instead of having just one start index for the
+ * whole chunk, we divide the chunk into 4 parts and save start
+ * index for each part.
+ */
+ u64 start_index[4];
+};
+
+struct sail {
+ /*default next-hop (Level 0)*/
+ u8 def_nh;
+
+ /*Level 16*/
+ u8 __rcu *N16;
+ u8 __rcu *P16;
+ u16 __rcu *C16;
+
+ /*Level 24*/
+ u8 __rcu *N24;
+ u8 __rcu *P24;
+ struct chunk __rcu *CK24;
+ u32 __rcu *C24;
+ u32 cnk24_count;/*Number of chunks in level 24*/
+
+ /*Level 32*/
+ u8 __rcu *N32;
+ u8 __rcu *P32;
+ u32 cnk32_count;/*Number of chunks in level 32*/
+
+ /*Index to this array is stored in N16, N24 and N32*/
+ struct net_device *netdevs[NETDEV_COUNT_MAX];
+ u8 netdev_count;/*Number of netdevs*/
+
+ spinlock_t lock;
+};
+
+int sail_insert(struct sail *s, u32 key,
+ u8 prefix_len, struct net_device *dev);
+int sail_delete(struct sail *s, u32 key,
+ u8 prefix_len);
+int sail_flush(struct sail *s);
+int sail_lookup(const struct sail *s, const __be32 dest,
+ struct net_device **dev);
+#endif
+
struct fib_nh_notifier_info {
struct fib_notifier_info info; /* must be first */
struct fib_nh *fib_nh;
@@ -219,6 +275,10 @@ struct fib_table {
int tb_num_default;
struct rcu_head rcu;
unsigned long *tb_data;
+#if IS_ENABLED(CONFIG_FIB_SAIL_XDP)
+ /*Each FIB table will have its own SAIL structure.*/
+ struct sail sail;
+#endif
unsigned long __data[0];
};
diff --git a/net/core/filter.c b/net/core/filter.c
index 5e00f2b..e89b4bb 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -4236,6 +4236,49 @@ static int bpf_fib_set_fwd_params(struct
bpf_fib_lookup *params,
}
#endif
+#if IS_ENABLED(CONFIG_FIB_SAIL_XDP)
+static int sail_fib_lookup(struct net *net, struct bpf_fib_lookup *params,
+ u32 flags, bool check_mtu)
+{
+ struct net_device *dev_in, *dev_out;
+ struct fib_table *tb;
+ struct neighbour *neigh;
+ u32 tbid;
+ int err;
+ u32 mtu;
+
+ if (flags & BPF_FIB_LOOKUP_DIRECT) {
+ dev_in = dev_get_by_index_rcu(net, params->ifindex);
+ if (unlikely(!dev_in))
+ return -ENODEV;
+ tbid = l3mdev_fib_table_rcu(dev_in) ? : RT_TABLE_MAIN;
+ } else {
+ tbid = RT_TABLE_MAIN;
+ }
+
+ tb = fib_get_table(net, tbid);
+ if (unlikely(!tb))
+ return BPF_FIB_LKUP_RET_NOT_FWDED;
+
+ err = sail_lookup(&tb->sail, params->ipv4_dst, &dev_out);
+ if (err)
+ return -ENOENT;
+
+ if (check_mtu) {
+ mtu = min(READ_ONCE(dev_out->mtu), IP_MAX_MTU);
+ if (params->tot_len > mtu)
+ return BPF_FIB_LKUP_RET_FRAG_NEEDED;
+ }
+
+ neigh = __ipv4_neigh_lookup_noref(dev_out,
+ (__force u32)params->ipv4_dst);
+ if (!neigh)
+ return BPF_FIB_LKUP_RET_NO_NEIGH;
+
+ return bpf_fib_set_fwd_params(params, neigh, dev_out);
+}
+#endif
+
#if IS_ENABLED(CONFIG_INET)
static int bpf_ipv4_fib_lookup(struct net *net, struct bpf_fib_lookup *params,
u32 flags, bool check_mtu)
@@ -4468,9 +4511,15 @@ BPF_CALL_4(bpf_xdp_fib_lookup, struct xdp_buff *, ctx,
switch (params->family) {
#if IS_ENABLED(CONFIG_INET)
case AF_INET:
+#if IS_ENABLED(CONFIG_FIB_SAIL_XDP)
+ return sail_fib_lookup(dev_net(ctx->rxq->dev), params,
+ flags, true);
+#else
return bpf_ipv4_fib_lookup(dev_net(ctx->rxq->dev), params,
flags, true);
#endif
+
+#endif
#if IS_ENABLED(CONFIG_IPV6)
case AF_INET6:
return bpf_ipv6_fib_lookup(dev_net(ctx->rxq->dev), params,
diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig
index 32cae39..5c41071 100644
--- a/net/ipv4/Kconfig
+++ b/net/ipv4/Kconfig
@@ -52,6 +52,17 @@ config IP_ADVANCED_ROUTER
If unsure, say N here.
+config FIB_SAIL_XDP
+ bool "SAIL based FIB lookup for XDP"
+ depends on !IP_ROUTE_MULTIPATH && !LWTUNNEL
+ default y
+ help
+ This option will enable SAIL based routing table lookup for XDP
+ forwarding. This will store FIB table as a SAIL (along with
+ LC-trie). Currently it only supports up to 255 ports. Currently
+ it does not support multi-path routing and light-weight
+ tunnels such as MPLS.
+
config IP_FIB_TRIE_STATS
bool "FIB TRIE statistics"
depends on IP_ADVANCED_ROUTER
diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile
index 7446b98..55ff5e7 100644
--- a/net/ipv4/Makefile
+++ b/net/ipv4/Makefile
@@ -64,6 +64,7 @@ obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
+obj-$(CONFIG_FIB_SAIL_XDP) += fib_sail_xdp.o
obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \
xfrm4_output.o xfrm4_protocol.o
diff --git a/net/ipv4/fib_sail_xdp.c b/net/ipv4/fib_sail_xdp.c
new file mode 100644
index 0000000..f3f56c5
--- /dev/null
+++ b/net/ipv4/fib_sail_xdp.c
@@ -0,0 +1,939 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018-19 MD Iftakharul Islam (Tamim) <mislam4@kent.edu>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ *
+ * This is SAIL_L based routing table lookup which was initially proposed in:
+ *
+ * Yang, Tong, Gaogang Xie, YanBiao Li, Qiaobin Fu, Alex X. Liu, Qi Li,
+ * and Laurent Mathy. "Guarantee IP lookup performance with FIB explosion."
+ * In ACM SIGCOMM Computer Communication Review, vol. 44, no. 4, pp. 39-50.
+ * ACM, 2014.
+ *
+ * It however deviates from the SAIL_L in three ways:
+ *
+ * 1. It pushes all the solid nodes in level 1~15 to level 16 whereas SAIL_L
+ * pushes them to either level 16, level 24 or level 32.
+ *
+ * 2. It pushes all the solid nodes in level 17~23 to level 24 whereas SAIL_L
+ * pushes them to either level 24 or level 32.
+ *
+ * 3. It adds a bitmap array, CK24 in addition to C24. This reduces the memory
+ * memory requirement of original C24 from 17.08 MB to 110KB for our example
+ * routing table.
+ */
+
+#include <net/ip_fib.h>
+
+/*The length of N16, P16 and C16 is 2^16*/
+#define LEVEL16_SIZE 65536
+
+/*Length of C24.*/
+#define C24_SIZE 1048576
+
+/*chunk size is 2^8*/
+#define CHUNK_SIZE 256
+
+/*Total number of chunks preallocated for level 24 and 32*/
+#define NUM_CHUNKS 16384
+
+/*Calculates the number of bits set to 1*/
+#define POPCNT(X) (hweight64(X))
+
+/*POPCNT of left-most N bits of X*/
+#define POPCNT_OFF(X, N) (hweight64(((1ULL << (N)) - 1) & (X)))
+
+/*Calculate index to C24 from CK26 chunk and chunk offset */
+static u64 calc_c24_idx(struct chunk c, u32 cnk_off)
+{
+ u8 part_idx, part_off;
+
+ part_idx = cnk_off / 64;
+ part_off = cnk_off % 64;
+
+ return c.start_index[part_idx] +
+ POPCNT_OFF(c.bitmap[part_idx], part_off);
+}
+
+/*Converts a net_device to corresponding netdev_index*/
+static u8 get_netdev_index(struct sail *s, struct net_device *dev)
+{
+ u8 i;
+
+ /*checks if the net_device is already seen; if yes then return the
+ *corresponding index
+ */
+ for (i = 0; i < s->netdev_count; i++) {
+ if (s->netdevs[i] == dev)
+ return i;
+ }
+ /*If the net_device is not previously seen, then add it to the array*/
+ s->netdevs[s->netdev_count++] = dev;
+ return s->netdev_count - 1;
+}
+
+/* Insert a new chunk to N24 and P24 at index chunk_id-1*/
+static int N24_insert(struct sail *s, u16 chunk_id)
+{
+ long long m;
+
+ if (chunk_id > (s->cnk24_count + 1) || chunk_id < 1) {
+ pr_err("Invalid chunk_id for level 26");
+ return -EINVAL;
+ }
+
+ if (s->cnk24_count >= NUM_CHUNKS) {
+ pr_err("Cannot insert a chunk. The chunk array is full");
+ return -EINVAL;
+ }
+
+ /*shift each element one step right to make
+ *space for the new one
+ */
+ m = (long long)s->cnk24_count * CHUNK_SIZE - 1;
+ for (; m >= (chunk_id - 1) * CHUNK_SIZE; m--) {
+ s->N24[m + CHUNK_SIZE] = s->N24[m];
+ s->P24[m + CHUNK_SIZE] = s->P24[m];
+ }
+
+ /*Reset the newly created chunk*/
+ m = (chunk_id - 1) * CHUNK_SIZE;
+ for (; m < chunk_id * CHUNK_SIZE; m++) {
+ s->N24[m] = 0;
+ s->P24[m] = 0;
+ }
+ return 0;
+}
+
+/* Insert a new chunk to CK24 at index chunk_id-1*/
+static int CK24_insert(struct sail *s, u16 chunk_id)
+{
+ long long m;
+
+ if (chunk_id > (s->cnk24_count + 1) || chunk_id < 1) {
+ pr_err("Invalid chunk_id for level 26");
+ return -EINVAL;
+ }
+
+ if (s->cnk24_count >= NUM_CHUNKS) {
+ pr_err("Cannot insert a chunk. The chunk array is full");
+ return -EINVAL;
+ }
+
+ /*shift each chunk one step right to make
+ *space for the new one
+ */
+ m = (long long)s->cnk24_count - 1;
+ for (; m >= (chunk_id - 1); m--)
+ s->CK24[m + 1] = s->CK24[m];
+
+ /*Reset the newly created empty chunk*/
+ for (m = 0; m < 4; m++) {
+ s->CK24[chunk_id - 1].bitmap[m] = 0;
+ s->CK24[chunk_id - 1].start_index[m] = 0;
+ }
+ return 0;
+}
+
+/* Insert a new chunk to level 24 at chunk_id-1. Note that Chunk ID
+ * starts from 1, not 0
+ */
+static int chunk24_insert(struct sail *s, u16 chunk_id)
+{
+ int err = 0;
+
+ err = N24_insert(s, chunk_id);
+ if (!err)
+ err = CK24_insert(s, chunk_id);
+ if (!err)
+ ++s->cnk24_count;
+ else
+ pr_err("Error in level 24 insertion");
+ return err;
+}
+
+/* Insert a new chunk to level 32 at chunk_id-1. Note that Chunk ID
+ * starts from 1, not 0
+ */
+static int chunk32_insert(struct sail *s, u32 chunk_id)
+{
+ long long m;
+
+ if (chunk_id > (s->cnk32_count + 1) || chunk_id < 1) {
+ pr_err("Invalid chunk_id for level 32");
+ return -EINVAL;
+ }
+
+ if (s->cnk32_count >= NUM_CHUNKS) {
+ pr_err("Cannot insert a new chunk. The chunk array is full");
+ return -EINVAL;
+ }
+
+ /*shift each element one step right to make
+ *space for the new one
+ */
+ m = (long long)s->cnk32_count * CHUNK_SIZE - 1;
+ for (; m >= (chunk_id - 1) * CHUNK_SIZE; m--) {
+ s->N32[m + CHUNK_SIZE] = s->N32[m];
+ s->P32[m + CHUNK_SIZE] = s->P32[m];
+ }
+
+ /*Reset the newly created empty chunk*/
+ m = (chunk_id - 1) * CHUNK_SIZE;
+ for (; m < chunk_id * CHUNK_SIZE; m++) {
+ s->N32[m] = 0;
+ s->P32[m] = 0;
+ }
+
+ ++s->cnk32_count;
+ return 0;
+}
+
+static int N24_delete(struct sail *s, u16 chunk_id)
+{
+ long long m;
+ u64 end_idx;
+
+ if (chunk_id > s->cnk24_count) {
+ pr_err("Invalid chunk_id to level 26");
+ return -EINVAL;
+ }
+
+ if (chunk_id < s->cnk24_count) {
+ /*shift each chunk one step left*/
+ m = (long long)chunk_id * CHUNK_SIZE;
+ end_idx = s->cnk24_count * CHUNK_SIZE - 1;
+ for (; m <= end_idx; m++) {
+ s->N24[m - CHUNK_SIZE] = s->N24[m];
+ s->P24[m - CHUNK_SIZE] = s->P24[m];
+ }
+ }
+
+ /*Reset the the last chunk*/
+ end_idx = s->cnk24_count * CHUNK_SIZE - 1;
+ m = (s->cnk24_count - 1) * CHUNK_SIZE;
+ for (; m <= end_idx; m++) {
+ s->N24[m] = 0;
+ s->P24[m] = 0;
+ }
+
+ return 0;
+}
+
+static int CK24_delete(struct sail *s, u16 chunk_id)
+{
+ long long m;
+
+ if (chunk_id > s->cnk24_count) {
+ pr_err("Invalid chunk_id to level 24");
+ return -EINVAL;
+ }
+
+ if (chunk_id < s->cnk24_count) {
+ /*shift each chunk one step left*/
+ m = (long long)chunk_id;
+ for (; m <= s->cnk24_count - 1; m++)
+ s->CK24[m - 1] = s->CK24[m];
+ }
+
+ /*Reset the the last chunk*/
+ for (m = 0; m < 4; m++) {
+ s->CK24[s->cnk24_count - 1].start_index[m] = 0;
+ s->CK24[s->cnk24_count - 1].bitmap[m] = 0;
+ }
+
+ return 0;
+}
+
+static int chunk24_delete(struct sail *s, u16 chunk_id)
+{
+ N24_delete(s, chunk_id);
+ CK24_delete(s, chunk_id);
+ --s->cnk24_count;
+ return 0;
+}
+
+static int chunk32_delete(struct sail *s, u32 chunk_id)
+{
+ long long m;
+ u64 end_idx;
+
+ if (chunk_id > s->cnk32_count) {
+ pr_err("Invalid chunk_id to level 32");
+ return -EINVAL;
+ }
+
+ if (chunk_id < s->cnk32_count) {
+ /*shift each chunk one step left*/
+ m = (long long)chunk_id * CHUNK_SIZE;
+ end_idx = s->cnk32_count * CHUNK_SIZE - 1;
+ for (; m <= end_idx; m++) {
+ s->N32[m - CHUNK_SIZE] = s->N32[m];
+ s->P32[m - CHUNK_SIZE] = s->P32[m];
+ }
+ }
+
+ /*Reset the the last chunk*/
+ end_idx = s->cnk32_count * CHUNK_SIZE - 1;
+ m = (s->cnk32_count - 1) * CHUNK_SIZE;
+ for (; m <= end_idx; m++) {
+ s->N32[m] = 0;
+ s->P32[m] = 0;
+ }
+
+ --s->cnk32_count;
+ return 0;
+}
+
+/*Calculate the chunk ID for level 24 based on C16*/
+static u16 calc_ckid24_from_C16(u16 *c16, u32 c16_size, u16 idx16)
+{
+ long long i;
+
+ if (idx16 >= c16_size) {
+ pr_err("Index needs to be smaller than arr_size");
+ return 0;
+ }
+
+ /*Find the first chunk ID to the left and increment that by 1*/
+ for (i = (long long)idx16 - 1; i >= 0; i--) {
+ if (c16[i] > 0)
+ return c16[i] + 1;
+ }
+
+ /*If there is no chunk to the left, then this is the first chunk*/
+ return 1;
+}
+
+/*Update C16 based on the newly inserted chunk*/
+static int update_C16(u16 *c16, u32 c16_size,
+ u16 idx16, u16 chunk_id)
+{
+ long long i;
+
+ if (idx16 >= c16_size) {
+ pr_err("Invalid index");
+ return -EINVAL;
+ }
+
+ c16[idx16] = chunk_id;
+
+ /* Increment chunk ID to the right */
+ for (i = idx16 + 1; i < c16_size; i++) {
+ if (c16[i] > 0)
+ c16[i]++;
+ }
+
+ return 0;
+}
+
+/*Remove Chunk ID for level 16*/
+static int C16_remove_chunkid(u16 *c16, u32 c16_size, u16 chunk_id)
+{
+ long long i;
+ bool found = false;
+
+ /*Check if the chunk ID already exists*/
+ for (i = 0; i < c16_size; i++) {
+ if (found && c16[i] > 0) {
+ c16[i]--;
+ } else if (c16[i] == chunk_id) {
+ c16[i] = 0;
+ found = true;
+ }
+ }
+
+ return 0;
+}
+
+static int C24_remove_chunkid(struct sail *s, u16 cnk_idx, u64 cnk_off)
+{
+ long long i;
+ bool found = false;
+ u32 chunk_id;
+ u8 part_idx, part_off;
+
+ part_idx = cnk_off / 64;
+ part_off = cnk_off % 64;
+
+ chunk_id = s->C24[calc_c24_idx(s->CK24[cnk_idx], cnk_off)];
+ s->CK24[cnk_idx].bitmap[part_idx] &= ~(1ULL << part_off);
+
+ for (i = 0; i < C24_SIZE; i++) {
+ if (s->C24[i] == chunk_id)
+ found = true;
+ else if (found && s->C24[i] > 0)
+ s->C24[i - 1] = s->C24[i];
+ else if (found && !s->C24[i])
+ break;
+ }
+
+ return 0;
+}
+
+ /* Check if a chunk in level 32 is being unused */
+static bool is_N32_chunk_unused(u8 *nh, u32 chunk_id)
+{
+ long long i;
+ long long start_index, end_index;
+
+ if (chunk_id < 1) {
+ pr_err("Invalid chunk ID");
+ return -EINVAL;
+ }
+
+ start_index = (long long)(chunk_id - 1) * CHUNK_SIZE;
+ end_index = (long long)chunk_id * CHUNK_SIZE - 1;
+ for (i = start_index; i <= end_index; i++) {
+ /*The chunk is being used*/
+ if (nh[i] > 0)
+ return false;
+ }
+
+ return true;
+}
+
+ /* Check if a chunk in level 24 is being unused*/
+static bool is_CK24_N24_chunk_unused(u8 *nh, struct chunk *ck, u16 chunk_id)
+{
+ long long i;
+ long long start_index, end_index;
+
+ if (chunk_id < 1) {
+ pr_err("Invalid chunk ID");
+ return -EINVAL;
+ }
+
+ start_index = (long long)(chunk_id - 1) * CHUNK_SIZE;
+ end_index = (long long)chunk_id * CHUNK_SIZE - 1;
+ for (i = start_index; i <= end_index; i++) {
+ /*The chunk is being used*/
+ if (nh[i] > 0)
+ return false;
+ }
+
+ if (ck->bitmap[0] || ck->bitmap[1] || ck->bitmap[2] || ck->bitmap[3])
+ return false;
+
+ return true;
+}
+
+/*Calculate the chunk ID for level 32 based on CK24 and C24*/
+static u32 calc_ckid32_from_ck24(struct chunk *ck24,
+ u32 ck24_idx, u32 ck24_off, u32 *c24)
+{
+ long long i, j;
+ long long index = 0;
+ u8 part_idx, part_off;
+ struct chunk c;
+
+ part_idx = ck24_off / 64;
+ part_off = ck24_off % 64;
+ c = ck24[ck24_idx];
+
+ /*find the index to C24 where previous chunk ID would be found*/
+ if (c.bitmap[part_idx]) {
+ index = c.start_index[part_idx] +
+ POPCNT_OFF(c.bitmap[part_idx], part_off) - 1;
+ if (index >= 0)
+ goto index_found;
+ }
+
+ j = part_idx - 1;
+ for (i = (long long)ck24_idx; i >= 0; i--) {
+ for (; j >= 0; j--) {
+ if (ck24[i].bitmap[j]) {
+ index = ck24[i].start_index[j] +
+ POPCNT(ck24[i].bitmap[j]) - 1;
+ goto index_found;
+ }
+ }
+ j = 3;
+ }
+
+ /*If there is no chunk to the left, then this is the first chunk*/
+ return 1;
+index_found:
+ if (index >= (C24_SIZE - 1)) {
+ pr_err("CK24 array is full. Cannot insert");
+ return 0;
+ } else if (c24[index] <= 0) {
+ pr_err("Invalid chunk ID");
+ return 0;
+ } else {
+ return c24[index] + 1;
+ }
+}
+
+/*Update CK24 and C24 based on the newly inserted chunk*/
+static int update_ck24_c24(struct chunk *ck24, u32 ck24_idx, u32 ck24_off,
+ u32 *c24, u32 cnk24_count, u32 chunk_id)
+{
+ long long i, j;
+ u64 index = 0;
+ u8 part_idx, part_off;
+
+ part_idx = ck24_off / 64;
+ part_off = ck24_off % 64;
+
+ if (ck24[ck24_idx].bitmap[part_idx] & (1ULL << part_off)) {
+ pr_err("Error: bitmap is already set");
+ return -EINVAL;
+ }
+
+ /*find the index where Chunk ID should be copied to*/
+ if (ck24[ck24_idx].bitmap[part_idx]) {
+ index = ck24[ck24_idx].start_index[part_idx] +
+ POPCNT_OFF(ck24[ck24_idx].bitmap[part_idx], part_off);
+ goto index_found;
+ }
+
+ /*Find a chunk to the left which is not empty*/
+ j = part_idx - 1;
+ for (i = (long long)ck24_idx; i >= 0; i--) {
+ for (; j >= 0; j--) {
+ if (ck24[i].bitmap[j]) {
+ index = ck24[i].start_index[j] +
+ POPCNT(ck24[i].bitmap[j]);
+ goto index_found;
+ }
+ }
+ j = 3;
+ }
+
+index_found:
+ /* Move each element one step to the right to create a space for
+ * the new element. Also increment each element by 1
+ */
+ for (i = C24_SIZE - 2 ; i >= (long long)index; i--)
+ c24[i + 1] = (c24[i] > 0) ? c24[i] + 1 : c24[i];
+ /*Set the new Chunk ID*/
+ c24[index] = chunk_id;
+
+ /*This is the first element of this chunk*/
+ if (!ck24[ck24_idx].bitmap[part_idx])
+ ck24[ck24_idx].start_index[part_idx] = index;
+
+ ck24[ck24_idx].bitmap[part_idx] |= (1ULL << part_off);
+
+ /*Update offset of the chunks to the right*/
+ j = part_idx + 1;
+ for (i = (long long)ck24_idx; i < cnk24_count; i++) {
+ for (; j < 4; j++) {
+ if (ck24[i].bitmap[j])
+ ck24[i].start_index[j]++;
+ }
+ j = 0;
+ }
+
+ return 0;
+}
+
+int sail_insert(struct sail *s, u32 key, u8 prefix_len,
+ struct net_device *dev)
+{
+ int i;
+ u8 *n16, *p16, *n24, *p24, *n32, *p32;
+ u16 *c16;
+ u32 *c24;
+ struct chunk *ck24;
+ u16 chunk_id;
+ u16 n16_idx;/*Index to N16, P16 and C16*/
+ u64 n24_idx;/*Index to N24 and P24*/
+ u64 c24_idx;/*Index to C24*/
+ u32 ck24_idx;/*Index to CK24*/
+ u64 ck24_off;/*offset inside a chunk*/
+ u8 part_idx, part_off;
+ u64 n32_idx;/*Index to N32 and P32*/
+ u32 num_leafs;/*Number of leafs need to be inserted for this prefix*/
+ u8 netdev_index = get_netdev_index(s, dev);
+ int err = 0;
+
+ spin_lock(&s->lock);
+
+ /* Default route */
+ if (prefix_len == 0) {
+ s->def_nh = netdev_index;
+ goto finish;
+ }
+
+ /* Preallocate all the arrays at once*/
+ if (!s->N16) {
+ n16 = kcalloc(LEVEL16_SIZE, sizeof(*n16), GFP_ATOMIC);
+ p16 = kcalloc(LEVEL16_SIZE, sizeof(*p16), GFP_ATOMIC);
+ c16 = kcalloc(LEVEL16_SIZE, sizeof(*c16), GFP_ATOMIC);
+ n24 = kcalloc(NUM_CHUNKS * CHUNK_SIZE, sizeof(*n24),
+ GFP_ATOMIC);
+ p24 = kcalloc(NUM_CHUNKS * CHUNK_SIZE, sizeof(*p24),
+ GFP_ATOMIC);
+ ck24 = kcalloc(NUM_CHUNKS, sizeof(*ck24), GFP_ATOMIC);
+ c24 = kcalloc(C24_SIZE, sizeof(*c24), GFP_ATOMIC);
+ n32 = kcalloc(NUM_CHUNKS * CHUNK_SIZE, sizeof(*n32),
+ GFP_ATOMIC);
+ p32 = kcalloc(NUM_CHUNKS * CHUNK_SIZE, sizeof(*p32),
+ GFP_ATOMIC);
+
+ if (!n16 || !c16 || !p16 || !n24 || !p24 || !ck24 ||
+ !c24 || !n32 || !p32) {
+ kfree(n16);
+ kfree(c16);
+ kfree(p16);
+ kfree(n24);
+ kfree(p24);
+ kfree(ck24);
+ kfree(c24);
+ kfree(n32);
+ kfree(p32);
+ pr_err("Out of memory while preallocating SAIL");
+ goto error;
+ }
+
+ RCU_INIT_POINTER(s->N16, n16);
+ RCU_INIT_POINTER(s->P16, p16);
+ RCU_INIT_POINTER(s->C16, c16);
+ RCU_INIT_POINTER(s->N24, n24);
+ RCU_INIT_POINTER(s->P24, p24);
+ RCU_INIT_POINTER(s->CK24, ck24);
+ RCU_INIT_POINTER(s->C24, c24);
+ RCU_INIT_POINTER(s->N32, n32);
+ RCU_INIT_POINTER(s->P32, p32);
+
+ synchronize_rcu();
+ }
+
+ /*Eextract 16 bits from LSB.*/
+ n16_idx = key >> 16;
+
+ if (prefix_len <= 16) {
+ /*All the leafs in level 1~16 will be stored in level 16.*/
+ num_leafs = 1U << (16 - prefix_len);
+ for (i = 0; i < num_leafs; i++) {
+ /*Longer prefix exists*/
+ if (s->P16[n16_idx + i] > prefix_len)
+ continue;
+ s->N16[n16_idx + i] = netdev_index;
+ s->P16[n16_idx + i] = prefix_len;
+ }
+ goto finish;
+ }
+
+ /* The length of the prefix is 21~32. So need to check if there is a
+ * chunk for this prefix in level 24. C16[..] = 0 indicates that there
+ * is no chunk, so need to insert one. The insertion works as
+ * following:
+ * 1. Calculate the chunk ID to level 24 from C16. The Chunk ID
+ * indicates where the new chunk should be inserted.
+ * 2. Insert a new chunk in level 24
+ * 3. Update the C16[] based on the newly inserted chunk
+ */
+ if (s->C16[n16_idx] == 0) {
+ /*Step 1*/
+ chunk_id = calc_ckid24_from_C16(s->C16, LEVEL16_SIZE, n16_idx);
+ if (!chunk_id)
+ goto error;
+ /*Step 2*/
+ err = chunk24_insert(s, chunk_id);
+ if (err)
+ goto error;
+ /*Step 3*/
+ err = update_C16(s->C16, LEVEL16_SIZE, n16_idx, chunk_id);
+ if (err)
+ goto error;
+ }
+
+ /*Extract bit 17~24 and calculate index to level 24*/
+ n24_idx = (s->C16[n16_idx] - 1) * CHUNK_SIZE + ((key & 65280) >> 8);
+ ck24_idx = s->C16[n16_idx] - 1;
+ ck24_off = (key & 65280) >> 8;
+
+ if (prefix_len <= 24) {
+ /*All the leafs in level 17~24 will be stored in level 24.*/
+ num_leafs = 1U << (24 - prefix_len);
+ for (i = 0; i < num_leafs; i++) {
+ /*Longer prefix exists*/
+ if (s->P24[n24_idx + i] > prefix_len)
+ continue;
+ s->N24[n24_idx + i] = netdev_index;
+ s->P24[n24_idx + i] = prefix_len;
+ }
+ goto finish;
+ }
+
+ /* The length of the prefix is 25~32, but there is no chunk for
+ * this prefix in level 32. So need to insert a new one.
+ * The insertion works as following:
+ * 1. Calculate the chunk ID to level 32 from CK24 and C24. The
+ * Chunk ID indicates where the new chunk should be inserted.
+ * 2. Insert a new chunk in level 32
+ * 3. Update the CK24 and C24 based on the newly inserted chunk
+ */
+ part_idx = ck24_off / 64;
+ part_off = ck24_off % 64;
+ if (!(s->CK24[ck24_idx].bitmap[part_idx] & (1ULL << part_off))) {
+ /*Step 1*/
+ chunk_id = calc_ckid32_from_ck24(s->CK24, ck24_idx,
+ ck24_off, s->C24);
+ if (!chunk_id)
+ goto error;
+ /*Step 2*/
+ err = chunk32_insert(s, chunk_id);
+ if (err)
+ goto error;
+ /*Step 3*/
+ err = update_ck24_c24(s->CK24, ck24_idx, ck24_off,
+ s->C24, s->cnk24_count, chunk_id);
+ if (err)
+ goto error;
+ }
+
+ c24_idx = calc_c24_idx(s->CK24[ck24_idx], ck24_off);
+ /*Extract 8 bit from MSB and calculate index to level 32*/
+ n32_idx = (s->C24[c24_idx] - 1) * CHUNK_SIZE + (key & 255);
+
+ if (prefix_len <= 32) {
+ /*All the leafs in level 25~32 will be stored in level 32.*/
+ num_leafs = 1U << (32 - prefix_len);
+ for (i = 0; i < num_leafs; i++) {
+ /*Longer prefix exists*/
+ if (s->P32[n32_idx + i] > prefix_len)
+ continue;
+ s->N32[n32_idx + i] = netdev_index;
+ s->P32[n32_idx + i] = prefix_len;
+ }
+ goto finish;
+ }
+
+error:
+ pr_err("Something went wrong in route insertion");
+finish:
+ spin_unlock(&s->lock);
+ return err;
+}
+
+int sail_delete(struct sail *s, u32 key, u8 prefix_len)
+{
+ int i;
+ u32 n16_idx;/*Index to N16, P16 and C16*/
+ u64 n24_idx;/*Index to level N24 and P24*/
+ u16 ck24_idx;/*Index to CK24*/
+ u64 ck24_off;/*Offset inside chunk in level 24*/
+ u64 n32_idx;/*Index to level N32 and P32*/
+ u32 consecutive_leafs;
+ int err = 0;
+ u32 chunkid_level32;
+ u8 part_idx, part_off;
+
+ spin_lock(&s->lock);
+
+ /* Simply ignore */
+ if (prefix_len == 0 || !s->N16)
+ goto error;
+
+ /*Eextract 16 bits from LSB.*/
+ n16_idx = key >> 16;
+
+ if (prefix_len <= 16) {
+ /*Level pushing*/
+ consecutive_leafs = 1U << (16 - prefix_len);
+ for (i = 0; i < consecutive_leafs; i++) {
+ /*Prefix len doesn't match*/
+ if (s->P16[n16_idx + i] != prefix_len)
+ continue;
+ s->N16[n16_idx + i] = 0;
+ s->P16[n16_idx + i] = 0;
+ }
+ goto finish;
+ }
+
+ /* The prefix_len is 17~32 but no chunk for the prefix*/
+ if (s->C16[n16_idx] == 0)
+ goto error;
+
+ ck24_idx = s->C16[n16_idx] - 1;
+ ck24_off = (key & 65280) >> 8;/*Extract bit 17~24 from the prefix*/
+ n24_idx = ck24_idx * CHUNK_SIZE + ck24_off;
+
+ if (prefix_len <= 24) {
+ /*Level pushing*/
+ consecutive_leafs = 1U << (24 - prefix_len);
+ for (i = 0; i < consecutive_leafs; i++) {
+ /*Prefix len doesn't match*/
+ if (s->P24[n24_idx + i] != prefix_len)
+ continue;
+ s->N24[n24_idx + i] = 0;
+ s->P24[n24_idx + i] = 0;
+ }
+ if (is_CK24_N24_chunk_unused(s->N24, &s->CK24[ck24_idx],
+ s->C16[n16_idx])) {
+ chunk24_delete(s, s->C16[n16_idx]);
+ C16_remove_chunkid(s->C16, LEVEL16_SIZE,
+ s->C16[n16_idx]);
+ }
+ goto finish;
+ }
+
+ part_idx = ck24_off / 64;
+ part_off = ck24_off % 64;
+ /* The prefix_len is 25~32. but no chunk for the prefix */
+ if (!(s->CK24[ck24_idx].bitmap[part_idx] & (1ULL << part_off)))
+ goto error;
+
+ chunkid_level32 = s->C24[calc_c24_idx(s->CK24[ck24_idx], ck24_off)];
+
+ /*Extract 8 bit from MSB and calculate index to level 32*/
+ n32_idx = (chunkid_level32 - 1) * CHUNK_SIZE + (key & 255);
+
+ if (prefix_len <= 32) {
+ /*Level pushing*/
+ consecutive_leafs = 1U << (32 - prefix_len);
+
+ for (i = 0; i < consecutive_leafs; i++) {
+ /*Prefix len doesn't match*/
+ if (s->P32[n32_idx + i] != prefix_len)
+ continue;
+ s->N32[n32_idx + i] = 0;
+ s->P32[n32_idx + i] = 0;
+ }
+ if (is_N32_chunk_unused(s->N32, chunkid_level32)) {
+ chunk32_delete(s, chunkid_level32);
+ C24_remove_chunkid(s, ck24_idx, ck24_off);
+ }
+ goto finish;
+ }
+
+/*The prefix was not found*/
+error:
+ err = -ENOENT;
+finish:
+ spin_unlock(&s->lock);
+ return err;
+}
+
+int sail_flush(struct sail *s)
+{
+ u8 *n16_old, *p16_old, *n24_old, *p24_old, *n32_old, *p32_old;
+ u16 *c16_old;
+ u32 *c24_old;
+ struct chunk *ck24_old;
+
+ spin_lock(&s->lock);
+
+ /*Save old pointers*/
+ n16_old = s->N16;
+ p16_old = s->P16;
+ c16_old = s->C16;
+ n24_old = s->N24;
+ p24_old = s->P24;
+ c24_old = s->C24;
+ ck24_old = s->CK24;
+ n32_old = s->N32;
+ p32_old = s->P32;
+
+ /*Set the counter before the chunk are deleted*/
+ s->cnk32_count = 0;
+ s->cnk24_count = 0;
+
+ /*Set the pointers to NULL*/
+ rcu_assign_pointer(s->N16, NULL);
+ rcu_assign_pointer(s->P16, NULL);
+ rcu_assign_pointer(s->C16, NULL);
+
+ rcu_assign_pointer(s->N24, NULL);
+ rcu_assign_pointer(s->P24, NULL);
+ rcu_assign_pointer(s->C24, NULL);
+ rcu_assign_pointer(s->CK24, NULL);
+
+ rcu_assign_pointer(s->N32, NULL);
+ rcu_assign_pointer(s->P32, NULL);
+
+ /* Wait for all references to be released */
+ synchronize_rcu();
+
+ /* Deallocate old references after setting them NULL*/
+ kfree(n16_old);
+ kfree(p16_old);
+ kfree(c16_old);
+ kfree(n24_old);
+ kfree(p24_old);
+ kfree(c24_old);
+ kfree(ck24_old);
+ kfree(n32_old);
+ kfree(p32_old);
+
+ spin_unlock(&s->lock);
+ return 0;
+}
+
+int sail_lookup(const struct sail *s,
+ const __be32 dest, struct net_device **dev)
+{
+ u8 *n16, *n24, *n32;
+ u16 *c16;
+ u32 *c24;
+ struct chunk *ck24;
+ const u32 key = ntohl(dest);
+ u8 netdev_index = s->def_nh;
+ u16 n16_idx;/*Index to N16 and C16*/
+ u64 n24_idx;/*Index to N24*/
+ u32 ck24_idx;/*Index to CK24*/
+ u64 ck24_off;/*Offset inside a chunk*/
+ u64 c24_idx;/*Index to C24*/
+ u64 n32_idx;/*Index to N32*/
+ u8 part_idx, part_off;
+
+ rcu_read_lock();
+
+ /*extract 16 bits from LSB*/
+ n16_idx = key >> 16;
+ n16 = rcu_dereference(s->N16);
+ if (unlikely(!n16))
+ goto finish;
+
+ if (likely(n16[n16_idx] != 0))
+ netdev_index = n16[n16_idx];
+
+ /*Check if there is a longer prefix; if yes, extract bit 17~24
+ * and calculate index to N24
+ */
+ c16 = rcu_dereference(s->C16);
+ if (likely(c16[n16_idx] != 0)) {
+ ck24_idx = c16[n16_idx] - 1;
+ ck24_off = (key & 65280) >> 8;
+ n24_idx = ck24_idx * CHUNK_SIZE + ck24_off;
+ } else {
+ goto finish;
+ }
+
+ /*Find corresponding next-hop in level 24*/
+ n24 = rcu_dereference(s->N24);
+ if (likely(n24[n24_idx] != 0))
+ netdev_index = n24[n24_idx];
+
+ /*Check if there is a longer prefix; if yes, extract 8 bits
+ * from MSB and calculate index to N32
+ */
+ ck24 = rcu_dereference(s->CK24);
+ part_idx = ck24_off / 64;
+ part_off = ck24_off % 64;
+ if (likely(ck24[ck24_idx].bitmap[part_idx] & (1ULL << part_off))) {
+ c24 = rcu_dereference(s->C24);
+ c24_idx = ck24[ck24_idx].start_index[part_idx] +
+ POPCNT_OFF(ck24[ck24_idx].bitmap[part_idx], part_off);
+ n32_idx = (c24[c24_idx] - 1) * CHUNK_SIZE + (key & 255);
+ } else {
+ goto finish;
+ }
+
+ n32 = rcu_dereference(s->N32);
+ if (likely(n32[n32_idx] != 0))
+ netdev_index = n32[n32_idx];
+
+finish:
+ *dev = s->netdevs[netdev_index];
+ rcu_read_unlock();
+ return 0;
+}
diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c
index 5bc0c89..d60fa45 100644
--- a/net/ipv4/fib_trie.c
+++ b/net/ipv4/fib_trie.c
@@ -1280,6 +1280,10 @@ int fib_table_insert(struct net *net, struct
fib_table *tb,
if (err)
goto out_fib_notif;
+#if IS_ENABLED(CONFIG_FIB_SAIL_XDP)
+ sail_insert(&tb->sail, key, plen, fi->fib_dev);
+#endif
+
if (!plen)
tb->tb_num_default++;
@@ -1568,6 +1572,10 @@ int fib_table_delete(struct net *net, struct
fib_table *tb,
pr_debug("Deleting %08x/%d tos=%d t=%p\n", key, plen, tos, t);
+#if IS_ENABLED(CONFIG_FIB_SAIL_XDP)
+ sail_delete(&tb->sail, key, plen);
+#endif
+
fa_to_delete = NULL;
hlist_for_each_entry_from(fa, fa_list) {
struct fib_info *fi = fa->fa_info;
@@ -1929,6 +1937,10 @@ int fib_table_flush(struct net *net, struct
fib_table *tb)
}
}
+#if IS_ENABLED(CONFIG_FIB_SAIL_XDP)
+ sail_flush(&tb->sail);
+#endif
+
pr_debug("trie_flush found=%d\n", found);
return found;
}
--
2.7.4
^ permalink raw reply related
* [PATCH net v2 1/1] bnx2x: Assign unique DMAE channel number for FW DMAE transactions.
From: Sudarsana Reddy Kalluru @ 2018-11-12 2:27 UTC (permalink / raw)
To: davem; +Cc: netdev, Michal.Kalderon
Driver assigns DMAE channel 0 for FW as part of START_RAMROD command. FW
uses this channel for DMAE operations (e.g., TIME_SYNC implementation).
Driver also uses the same channel 0 for DMAE operations for some of the PFs
(e.g., PF0 on Port0). This could lead to concurrent access to the DMAE
channel by FW and driver which is not legal. Hence need to assign unique
DMAE id for FW.
Currently following DMAE channels are used by the clients,
MFW - OCBB/OCSD functionality uses DMAE channel 14/15
Driver 0-3 and 8-11 (for PF dmae operations)
4 and 12 (for stats requests)
Assigning unique dmae_id '13' to the FW.
Changes from previous version:
------------------------------
v2: Incorporated the review comments.
Signed-off-by: Sudarsana Reddy Kalluru <Sudarsana.Kalluru@cavium.com>
Signed-off-by: Michal Kalderon <Michal.Kalderon@cavium.com>
---
drivers/net/ethernet/broadcom/bnx2x/bnx2x.h | 7 +++++++
drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.c | 1 +
2 files changed, 8 insertions(+)
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
index be15061..0de487a 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
@@ -2191,6 +2191,13 @@ void bnx2x_igu_clear_sb_gen(struct bnx2x *bp, u8 func, u8 idu_sb_id,
#define PMF_DMAE_C(bp) (BP_PORT(bp) * MAX_DMAE_C_PER_PORT + \
E1HVN_MAX)
+/* Following is the DMAE channel number allocation for the clients.
+ * MFW: OCBB/OCSD implementations use DMAE channels 14/15 respectively.
+ * Driver: 0-3 and 8-11 (for PF dmae operations)
+ * 4 and 12 (for stats requests)
+ */
+#define BNX2X_FW_DMAE_C 13 /* Channel for FW DMAE operations */
+
/* PCIE link and speed */
#define PCICFG_LINK_WIDTH 0x1f00000
#define PCICFG_LINK_WIDTH_SHIFT 20
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.c
index 3f4d2c8..a9eaaf3 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.c
@@ -6149,6 +6149,7 @@ static inline int bnx2x_func_send_start(struct bnx2x *bp,
rdata->sd_vlan_tag = cpu_to_le16(start_params->sd_vlan_tag);
rdata->path_id = BP_PATH(bp);
rdata->network_cos_mode = start_params->network_cos_mode;
+ rdata->dmae_cmd_id = BNX2X_FW_DMAE_C;
rdata->vxlan_dst_port = cpu_to_le16(start_params->vxlan_dst_port);
rdata->geneve_dst_port = cpu_to_le16(start_params->geneve_dst_port);
--
1.8.3.1
^ permalink raw reply related
* Re: [RFC PATCH 0/3] acpi: Add acpi mdio support code
From: Wang, Dongsheng @ 2018-11-12 2:52 UTC (permalink / raw)
To: Andrew Lunn
Cc: timur@kernel.org, Zheng, Joey, f.fainelli@gmail.com,
rjw@rjwysocki.net, linux-acpi@vger.kernel.org,
netdev@vger.kernel.org
In-Reply-To: <20181108232353.GL5259@lunn.ch>
On 2018/11/9 7:23, Andrew Lunn wrote:
> I'm just trying to ensure whatever is defined is flexible enough that
> we really can later support everything which DT does. We have PHYs on
> MDIO busses, inside switches, which are on MDIO busses, which are
> inside Ethernet interfaces, etc.
>
> An MDIO bus is very similar to an i2c bus. How is that described in
> ACPI? Anything we can learn from that?
All the documents I've seen are from UEFII/ACPICA and I have not seen any
independent description of MDIO in the ACPI documents. Maybe I missed a
document, too.
I think not all devices need to be described in ACPI. The ACPI defines what
should be included in a device, as well as some special definitions. Our
MDIO
doesn't need a special definition, because it's enough to use a generic ACPI
description. I think that's why MDIO doesn't have a separate description.
Cheers,
Dongsheng
^ permalink raw reply
* Re: [PATCH tip/core/rcu 2/7] sfc: Replace spin_is_locked() with lockdep
From: Edward Cree @ 2018-11-12 13:02 UTC (permalink / raw)
To: Paul E. McKenney, linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg, joel,
Lance Roy, Solarflare linux maintainers, Bert Kenward,
David S. Miller, netdev
In-Reply-To: <20181111200443.10772-2-paulmck@linux.ibm.com>
On 11/11/18 20:04, Paul E. McKenney wrote:
> From: Lance Roy <ldr709@gmail.com>
>
> lockdep_assert_held() is better suited to checking locking requirements,
> since it only checks if the current thread holds the lock regardless of
> whether someone else does. This is also a step towards possibly removing
> spin_is_locked().
>
> Signed-off-by: Lance Roy <ldr709@gmail.com>
> Cc: Solarflare linux maintainers <linux-net-drivers@solarflare.com>
> Cc: Edward Cree <ecree@solarflare.com>
> Cc: Bert Kenward <bkenward@solarflare.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: <netdev@vger.kernel.org>
> Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
> ---
Acked-by: Edward Cree <ecree@solarflare.com>
> drivers/net/ethernet/sfc/efx.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c
> index 98fe7e762e17..3643015a55cf 100644
> --- a/drivers/net/ethernet/sfc/efx.c
> +++ b/drivers/net/ethernet/sfc/efx.c
> @@ -3167,7 +3167,7 @@ struct hlist_head *efx_rps_hash_bucket(struct efx_nic *efx,
> {
> u32 hash = efx_filter_spec_hash(spec);
>
> - WARN_ON(!spin_is_locked(&efx->rps_hash_lock));
> + lockdep_assert_held(&efx->rps_hash_lock);
> if (!efx->rps_hash_table)
> return NULL;
> return &efx->rps_hash_table[hash % EFX_ARFS_HASH_TABLE_SIZE];
^ permalink raw reply
* Re: [PATCH net-next] NFC: nci: remove redundant variable 'status'
From: YueHaibing @ 2018-11-12 13:20 UTC (permalink / raw)
To: davem, sameo; +Cc: linux-kernel, netdev
In-Reply-To: <c168a8ff-0736-d21d-9c5c-bde001ad018c@huawei.com>
Ping.
On 2018/9/13 20:06, YueHaibing wrote:
> Ping.
>
> On 2018/8/1 15:20, YueHaibing wrote:
>> After commit d8cd37ed2fc8 ("NFC: nci: Fix improper management of HCI return code")
>> variable 'status' is being assigned but never used,
>> so can be removed. Also make a trival cleanup.
>>
>> Signed-off-by: YueHaibing <yuehaibing@huawei.com>
>> ---
>> net/nfc/nci/hci.c | 10 ++--------
>> 1 file changed, 2 insertions(+), 8 deletions(-)
>>
>> diff --git a/net/nfc/nci/hci.c b/net/nfc/nci/hci.c
>> index ddfc52a..4debba8 100644
>> --- a/net/nfc/nci/hci.c
>> +++ b/net/nfc/nci/hci.c
>> @@ -370,17 +370,11 @@ static void nci_hci_resp_received(struct nci_dev *ndev, u8 pipe,
>> u8 result, struct sk_buff *skb)
>> {
>> struct nci_conn_info *conn_info;
>> - u8 status = result;
>>
>> conn_info = ndev->hci_dev->conn_info;
>> - if (!conn_info) {
>> - status = NCI_STATUS_REJECTED;
>> - goto exit;
>> - }
>> -
>> - conn_info->rx_skb = skb;
>> + if (conn_info)
>> + conn_info->rx_skb = skb;
>>
>> -exit:
>> nci_req_complete(ndev, NCI_STATUS_OK);
>> }
>>
>>
>
>
> .
>
^ permalink raw reply
* Re: (2) [PATCH net] ip: hash fragments consistently
From: Eric Dumazet @ 2018-11-12 3:37 UTC (permalink / raw)
To: soukjin.bae, Paolo Abeni, netdev@vger.kernel.org; +Cc: David S. Miller
In-Reply-To: <20181112004054epcms1p3db41e04dad77c59e675a769e07f4615a@epcms1p3>
On 11/11/2018 04:40 PM, 배석진 wrote:
>> Also we might note that flow dissector itself is buggy as
>> found by Soukjin Bae ( https://patchwork.ozlabs.org/patch/994601/ )
>>
>> I will send a v2 of his patch with a different changelog.
>>
>> Defrag is fixed [1] but the bug in flow dissector is adding
>> extra work and hash inconsistencies.
>>
>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=0d5b9311baf27bb545f187f12ecfd558220c607d
>
>
> Dear all,
>
> it's good news to mee too.
> thanks for accept my work :)
Sure thing ;)
^ permalink raw reply
* [PATCH net-next 0/4] net: ethernet: ti: cpsw: allow vlan h/w timestamping
From: Ivan Khoronzhuk @ 2018-11-12 14:00 UTC (permalink / raw)
To: grygorii.strashko, davem
Cc: linux-omap, netdev, linux-kernel, Ivan Khoronzhuk
The patchset adds several improvements and allows vlan h/w ts.
Based on net-next/master
Ivan Khoronzhuk (4):
net: ethernet: ti: cpts: correct debug for expired txq skb
net: ethernet: ti: cpts: purge staled skbs from txq
net: ethernet: ti: cpts: move enable/disable flags outside of cpts
module
net: ethernet: ti: cpsw: allow vlan tagged packets to be timestamped
drivers/net/ethernet/ti/cpsw.c | 51 ++++++++++++++-------------
drivers/net/ethernet/ti/cpts.c | 32 +++++++++++++----
drivers/net/ethernet/ti/cpts.h | 38 --------------------
drivers/net/ethernet/ti/netcp_ethss.c | 32 +++++++++--------
4 files changed, 69 insertions(+), 84 deletions(-)
--
2.17.1
^ permalink raw reply
* [PATCH net-next 1/4] net: ethernet: ti: cpts: correct debug for expired txq skb
From: Ivan Khoronzhuk @ 2018-11-12 14:00 UTC (permalink / raw)
To: grygorii.strashko, davem
Cc: linux-omap, netdev, linux-kernel, Ivan Khoronzhuk
In-Reply-To: <20181112140023.12407-1-ivan.khoronzhuk@linaro.org>
The msgtype and seqid that is smth that belongs to event for
comparison but not for staled txq skb.
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
drivers/net/ethernet/ti/cpts.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpts.c b/drivers/net/ethernet/ti/cpts.c
index b96b93c686bf..dac4c528a1ff 100644
--- a/drivers/net/ethernet/ti/cpts.c
+++ b/drivers/net/ethernet/ti/cpts.c
@@ -119,9 +119,7 @@ static bool cpts_match_tx_ts(struct cpts *cpts, struct cpts_event *event)
if (time_after(jiffies, skb_cb->tmo)) {
/* timeout any expired skbs over 1s */
- dev_dbg(cpts->dev,
- "expiring tx timestamp mtype %u seqid %04x\n",
- mtype, seqid);
+ dev_dbg(cpts->dev, "expiring tx timestamp from txq\n");
__skb_unlink(skb, &cpts->txq);
dev_consume_skb_any(skb);
}
--
2.17.1
^ permalink raw reply related
* [PATCH net-next 3/4] net: ethernet: ti: cpts: move enable/disable flags outside of cpts module
From: Ivan Khoronzhuk @ 2018-11-12 14:00 UTC (permalink / raw)
To: grygorii.strashko, davem
Cc: linux-omap, netdev, linux-kernel, Ivan Khoronzhuk
In-Reply-To: <20181112140023.12407-1-ivan.khoronzhuk@linaro.org>
Each slave has it's own receive timestamp filter. But cpts rx/tx
timestamp enable flags are used to allow ts retrieve only for one
user. This limitation causes data path redundancy and setting overlap
if cpsw module is in dual-mac mode for instance.
If rx ts is enabled only for one port - the second interface must expect
every incoming packet to be PTP packet w/o absolutely any reason, and if
it's PTP - do unneeded stuff, as rx filter for second port is not set
and cpts fifo is not supposed to contain appropriate ts event.
That's not correct.
So, to fix control overlap and avoid redundant CPU cycles, the patch
splits rx/tx ts enable flags between network devices. After the patch,
PTP timestamping still should be used for only one port (or PTP id
counter has to be different for both ports as cpts IP is common).
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
drivers/net/ethernet/ti/cpsw.c | 46 ++++++++++++++-------------
drivers/net/ethernet/ti/cpts.c | 2 --
drivers/net/ethernet/ti/cpts.h | 38 ----------------------
drivers/net/ethernet/ti/netcp_ethss.c | 32 ++++++++++---------
4 files changed, 41 insertions(+), 77 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index 9434fd5a5477..48ea21e7e0cb 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -466,6 +466,8 @@ struct cpsw_priv {
bool mqprio_hw;
int fifo_bw[CPSW_TC_NUM];
int shp_cfg_speed;
+ int tx_ts_enabled;
+ int rx_ts_enabled;
u32 emac_port;
struct cpsw_common *cpsw;
};
@@ -905,6 +907,7 @@ static void cpsw_rx_handler(void *token, int len, int status)
struct net_device *ndev = skb->dev;
int ret = 0, port;
struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
+ struct cpsw_priv *priv;
if (cpsw->data.dual_emac) {
port = CPDMA_RX_SOURCE_PORT(status);
@@ -939,7 +942,9 @@ static void cpsw_rx_handler(void *token, int len, int status)
skb_put(skb, len);
if (status & CPDMA_RX_VLAN_ENCAP)
cpsw_rx_vlan_encap(skb);
- cpts_rx_timestamp(cpsw->cpts, skb);
+ priv = netdev_priv(ndev);
+ if (priv->rx_ts_enabled)
+ cpts_rx_timestamp(cpsw->cpts, skb);
skb->protocol = eth_type_trans(skb, ndev);
netif_receive_skb(skb);
ndev->stats.rx_bytes += len;
@@ -2126,7 +2131,7 @@ static netdev_tx_t cpsw_ndo_start_xmit(struct sk_buff *skb,
}
if (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP &&
- cpts_is_tx_enabled(cpts) && cpts_can_timestamp(cpts, skb))
+ priv->tx_ts_enabled && cpts_can_timestamp(cpts, skb))
skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
q_idx = skb_get_queue_mapping(skb);
@@ -2170,13 +2175,13 @@ static netdev_tx_t cpsw_ndo_start_xmit(struct sk_buff *skb,
#if IS_ENABLED(CONFIG_TI_CPTS)
-static void cpsw_hwtstamp_v1(struct cpsw_common *cpsw)
+static void cpsw_hwtstamp_v1(struct cpsw_priv *priv)
{
+ struct cpsw_common *cpsw = priv->cpsw;
struct cpsw_slave *slave = &cpsw->slaves[cpsw->data.active_slave];
u32 ts_en, seq_id;
- if (!cpts_is_tx_enabled(cpsw->cpts) &&
- !cpts_is_rx_enabled(cpsw->cpts)) {
+ if (!priv->tx_ts_enabled && !priv->rx_ts_enabled) {
slave_write(slave, 0, CPSW1_TS_CTL);
return;
}
@@ -2184,10 +2189,10 @@ static void cpsw_hwtstamp_v1(struct cpsw_common *cpsw)
seq_id = (30 << CPSW_V1_SEQ_ID_OFS_SHIFT) | ETH_P_1588;
ts_en = EVENT_MSG_BITS << CPSW_V1_MSG_TYPE_OFS;
- if (cpts_is_tx_enabled(cpsw->cpts))
+ if (priv->tx_ts_enabled)
ts_en |= CPSW_V1_TS_TX_EN;
- if (cpts_is_rx_enabled(cpsw->cpts))
+ if (priv->rx_ts_enabled)
ts_en |= CPSW_V1_TS_RX_EN;
slave_write(slave, ts_en, CPSW1_TS_CTL);
@@ -2207,20 +2212,20 @@ static void cpsw_hwtstamp_v2(struct cpsw_priv *priv)
case CPSW_VERSION_2:
ctrl &= ~CTRL_V2_ALL_TS_MASK;
- if (cpts_is_tx_enabled(cpsw->cpts))
+ if (priv->tx_ts_enabled)
ctrl |= CTRL_V2_TX_TS_BITS;
- if (cpts_is_rx_enabled(cpsw->cpts))
+ if (priv->rx_ts_enabled)
ctrl |= CTRL_V2_RX_TS_BITS;
break;
case CPSW_VERSION_3:
default:
ctrl &= ~CTRL_V3_ALL_TS_MASK;
- if (cpts_is_tx_enabled(cpsw->cpts))
+ if (priv->tx_ts_enabled)
ctrl |= CTRL_V3_TX_TS_BITS;
- if (cpts_is_rx_enabled(cpsw->cpts))
+ if (priv->rx_ts_enabled)
ctrl |= CTRL_V3_RX_TS_BITS;
break;
}
@@ -2237,7 +2242,6 @@ static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
struct cpsw_priv *priv = netdev_priv(dev);
struct hwtstamp_config cfg;
struct cpsw_common *cpsw = priv->cpsw;
- struct cpts *cpts = cpsw->cpts;
if (cpsw->version != CPSW_VERSION_1 &&
cpsw->version != CPSW_VERSION_2 &&
@@ -2256,7 +2260,7 @@ static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
switch (cfg.rx_filter) {
case HWTSTAMP_FILTER_NONE:
- cpts_rx_enable(cpts, 0);
+ priv->rx_ts_enabled = 0;
break;
case HWTSTAMP_FILTER_ALL:
case HWTSTAMP_FILTER_NTP_ALL:
@@ -2264,7 +2268,7 @@ static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
case HWTSTAMP_FILTER_PTP_V1_L4_EVENT:
case HWTSTAMP_FILTER_PTP_V1_L4_SYNC:
case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ:
- cpts_rx_enable(cpts, HWTSTAMP_FILTER_PTP_V1_L4_EVENT);
+ priv->rx_ts_enabled = HWTSTAMP_FILTER_PTP_V1_L4_EVENT;
cfg.rx_filter = HWTSTAMP_FILTER_PTP_V1_L4_EVENT;
break;
case HWTSTAMP_FILTER_PTP_V2_L4_EVENT:
@@ -2276,18 +2280,18 @@ static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
case HWTSTAMP_FILTER_PTP_V2_EVENT:
case HWTSTAMP_FILTER_PTP_V2_SYNC:
case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
- cpts_rx_enable(cpts, HWTSTAMP_FILTER_PTP_V2_EVENT);
+ priv->rx_ts_enabled = HWTSTAMP_FILTER_PTP_V2_EVENT;
cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT;
break;
default:
return -ERANGE;
}
- cpts_tx_enable(cpts, cfg.tx_type == HWTSTAMP_TX_ON);
+ priv->tx_ts_enabled = cfg.tx_type == HWTSTAMP_TX_ON;
switch (cpsw->version) {
case CPSW_VERSION_1:
- cpsw_hwtstamp_v1(cpsw);
+ cpsw_hwtstamp_v1(priv);
break;
case CPSW_VERSION_2:
case CPSW_VERSION_3:
@@ -2303,7 +2307,7 @@ static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
static int cpsw_hwtstamp_get(struct net_device *dev, struct ifreq *ifr)
{
struct cpsw_common *cpsw = ndev_to_cpsw(dev);
- struct cpts *cpts = cpsw->cpts;
+ struct cpsw_priv *priv = netdev_priv(dev);
struct hwtstamp_config cfg;
if (cpsw->version != CPSW_VERSION_1 &&
@@ -2312,10 +2316,8 @@ static int cpsw_hwtstamp_get(struct net_device *dev, struct ifreq *ifr)
return -EOPNOTSUPP;
cfg.flags = 0;
- cfg.tx_type = cpts_is_tx_enabled(cpts) ?
- HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
- cfg.rx_filter = (cpts_is_rx_enabled(cpts) ?
- cpts->rx_enable : HWTSTAMP_FILTER_NONE);
+ cfg.tx_type = priv->tx_ts_enabled ? HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
+ cfg.rx_filter = priv->rx_ts_enabled;
return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ? -EFAULT : 0;
}
diff --git a/drivers/net/ethernet/ti/cpts.c b/drivers/net/ethernet/ti/cpts.c
index 63232b35024e..054f78295d1d 100644
--- a/drivers/net/ethernet/ti/cpts.c
+++ b/drivers/net/ethernet/ti/cpts.c
@@ -430,8 +430,6 @@ void cpts_rx_timestamp(struct cpts *cpts, struct sk_buff *skb)
u64 ns;
struct skb_shared_hwtstamps *ssh;
- if (!cpts->rx_enable)
- return;
ns = cpts_find_ts(cpts, skb, CPTS_EV_RX);
if (!ns)
return;
diff --git a/drivers/net/ethernet/ti/cpts.h b/drivers/net/ethernet/ti/cpts.h
index 73d73faf0f38..d2c7decd59b6 100644
--- a/drivers/net/ethernet/ti/cpts.h
+++ b/drivers/net/ethernet/ti/cpts.h
@@ -136,26 +136,6 @@ struct cpts *cpts_create(struct device *dev, void __iomem *regs,
struct device_node *node);
void cpts_release(struct cpts *cpts);
-static inline void cpts_rx_enable(struct cpts *cpts, int enable)
-{
- cpts->rx_enable = enable;
-}
-
-static inline bool cpts_is_rx_enabled(struct cpts *cpts)
-{
- return !!cpts->rx_enable;
-}
-
-static inline void cpts_tx_enable(struct cpts *cpts, int enable)
-{
- cpts->tx_enable = enable;
-}
-
-static inline bool cpts_is_tx_enabled(struct cpts *cpts)
-{
- return !!cpts->tx_enable;
-}
-
static inline bool cpts_can_timestamp(struct cpts *cpts, struct sk_buff *skb)
{
unsigned int class = ptp_classify_raw(skb);
@@ -197,24 +177,6 @@ static inline void cpts_unregister(struct cpts *cpts)
{
}
-static inline void cpts_rx_enable(struct cpts *cpts, int enable)
-{
-}
-
-static inline bool cpts_is_rx_enabled(struct cpts *cpts)
-{
- return false;
-}
-
-static inline void cpts_tx_enable(struct cpts *cpts, int enable)
-{
-}
-
-static inline bool cpts_is_tx_enabled(struct cpts *cpts)
-{
- return false;
-}
-
static inline bool cpts_can_timestamp(struct cpts *cpts, struct sk_buff *skb)
{
return false;
diff --git a/drivers/net/ethernet/ti/netcp_ethss.c b/drivers/net/ethernet/ti/netcp_ethss.c
index 0397ccb6597e..20d81e0b1c29 100644
--- a/drivers/net/ethernet/ti/netcp_ethss.c
+++ b/drivers/net/ethernet/ti/netcp_ethss.c
@@ -763,6 +763,8 @@ struct gbe_priv {
int cpts_registered;
struct cpts *cpts;
+ int rx_ts_enabled;
+ int tx_ts_enabled;
};
struct gbe_intf {
@@ -2564,7 +2566,7 @@ static int gbe_txtstamp_mark_pkt(struct gbe_intf *gbe_intf,
struct gbe_priv *gbe_dev = gbe_intf->gbe_dev;
if (!(skb_shinfo(p_info->skb)->tx_flags & SKBTX_HW_TSTAMP) ||
- !cpts_is_tx_enabled(gbe_dev->cpts))
+ !gbe_dev->tx_ts_enabled)
return 0;
/* If phy has the txtstamp api, assume it will do it.
@@ -2598,7 +2600,9 @@ static int gbe_rxtstamp(struct gbe_intf *gbe_intf, struct netcp_packet *p_info)
return 0;
}
- cpts_rx_timestamp(gbe_dev->cpts, p_info->skb);
+ if (gbe_dev->rx_ts_enabled)
+ cpts_rx_timestamp(gbe_dev->cpts, p_info->skb);
+
p_info->rxtstamp_complete = true;
return 0;
@@ -2614,10 +2618,8 @@ static int gbe_hwtstamp_get(struct gbe_intf *gbe_intf, struct ifreq *ifr)
return -EOPNOTSUPP;
cfg.flags = 0;
- cfg.tx_type = cpts_is_tx_enabled(cpts) ?
- HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
- cfg.rx_filter = (cpts_is_rx_enabled(cpts) ?
- cpts->rx_enable : HWTSTAMP_FILTER_NONE);
+ cfg.tx_type = gbe_dev->tx_ts_enabled ? HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF;
+ cfg.rx_filter = gbe_dev->rx_ts_enabled;
return copy_to_user(ifr->ifr_data, &cfg, sizeof(cfg)) ? -EFAULT : 0;
}
@@ -2628,8 +2630,8 @@ static void gbe_hwtstamp(struct gbe_intf *gbe_intf)
struct gbe_slave *slave = gbe_intf->slave;
u32 ts_en, seq_id, ctl;
- if (!cpts_is_rx_enabled(gbe_dev->cpts) &&
- !cpts_is_tx_enabled(gbe_dev->cpts)) {
+ if (!gbe_dev->rx_ts_enabled &&
+ !gbe_dev->tx_ts_enabled) {
writel(0, GBE_REG_ADDR(slave, port_regs, ts_ctl));
return;
}
@@ -2641,10 +2643,10 @@ static void gbe_hwtstamp(struct gbe_intf *gbe_intf)
(slave->ts_ctl.uni ? TS_UNI_EN :
slave->ts_ctl.maddr_map << TS_CTL_MADDR_SHIFT);
- if (cpts_is_tx_enabled(gbe_dev->cpts))
+ if (gbe_dev->tx_ts_enabled)
ts_en |= (TS_TX_ANX_ALL_EN | TS_TX_VLAN_LT1_EN);
- if (cpts_is_rx_enabled(gbe_dev->cpts))
+ if (gbe_dev->rx_ts_enabled)
ts_en |= (TS_RX_ANX_ALL_EN | TS_RX_VLAN_LT1_EN);
writel(ts_en, GBE_REG_ADDR(slave, port_regs, ts_ctl));
@@ -2670,10 +2672,10 @@ static int gbe_hwtstamp_set(struct gbe_intf *gbe_intf, struct ifreq *ifr)
switch (cfg.tx_type) {
case HWTSTAMP_TX_OFF:
- cpts_tx_enable(cpts, 0);
+ gbe_dev->tx_ts_enabled = 0;
break;
case HWTSTAMP_TX_ON:
- cpts_tx_enable(cpts, 1);
+ gbe_dev->tx_ts_enabled = 1;
break;
default:
return -ERANGE;
@@ -2681,12 +2683,12 @@ static int gbe_hwtstamp_set(struct gbe_intf *gbe_intf, struct ifreq *ifr)
switch (cfg.rx_filter) {
case HWTSTAMP_FILTER_NONE:
- cpts_rx_enable(cpts, 0);
+ gbe_dev->rx_ts_enabled = HWTSTAMP_FILTER_NONE;
break;
case HWTSTAMP_FILTER_PTP_V1_L4_EVENT:
case HWTSTAMP_FILTER_PTP_V1_L4_SYNC:
case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ:
- cpts_rx_enable(cpts, HWTSTAMP_FILTER_PTP_V1_L4_EVENT);
+ gbe_dev->rx_ts_enabled = HWTSTAMP_FILTER_PTP_V1_L4_EVENT;
cfg.rx_filter = HWTSTAMP_FILTER_PTP_V1_L4_EVENT;
break;
case HWTSTAMP_FILTER_PTP_V2_L4_EVENT:
@@ -2698,7 +2700,7 @@ static int gbe_hwtstamp_set(struct gbe_intf *gbe_intf, struct ifreq *ifr)
case HWTSTAMP_FILTER_PTP_V2_EVENT:
case HWTSTAMP_FILTER_PTP_V2_SYNC:
case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
- cpts_rx_enable(cpts, HWTSTAMP_FILTER_PTP_V2_EVENT);
+ gbe_dev->rx_ts_enabled = HWTSTAMP_FILTER_PTP_V2_EVENT;
cfg.rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT;
break;
default:
--
2.17.1
^ permalink raw reply related
* [PATCH net-next 4/4] net: ethernet: ti: cpsw: allow vlan tagged packets to be timestamped
From: Ivan Khoronzhuk @ 2018-11-12 14:00 UTC (permalink / raw)
To: grygorii.strashko, davem
Cc: linux-omap, netdev, linux-kernel, Ivan Khoronzhuk
In-Reply-To: <20181112140023.12407-1-ivan.khoronzhuk@linaro.org>
Allow vlan tagged packets to be timestamped, as no any restrictions
for this.
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
drivers/net/ethernet/ti/cpsw.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index 48ea21e7e0cb..e4aa030f1726 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -283,7 +283,7 @@ struct cpsw_ss_regs {
#define CTRL_V2_TS_BITS \
(TS_320 | TS_319 | TS_132 | TS_131 | TS_130 | TS_129 |\
- TS_TTL_NONZERO | TS_ANNEX_D_EN | TS_LTYPE1_EN)
+ TS_TTL_NONZERO | TS_ANNEX_D_EN | TS_LTYPE1_EN | VLAN_LTYPE1_EN)
#define CTRL_V2_ALL_TS_MASK (CTRL_V2_TS_BITS | TS_TX_EN | TS_RX_EN)
#define CTRL_V2_TX_TS_BITS (CTRL_V2_TS_BITS | TS_TX_EN)
@@ -293,7 +293,7 @@ struct cpsw_ss_regs {
#define CTRL_V3_TS_BITS \
(TS_107 | TS_320 | TS_319 | TS_132 | TS_131 | TS_130 | TS_129 |\
TS_TTL_NONZERO | TS_ANNEX_F_EN | TS_ANNEX_D_EN |\
- TS_LTYPE1_EN)
+ TS_LTYPE1_EN | VLAN_LTYPE1_EN)
#define CTRL_V3_ALL_TS_MASK (CTRL_V3_TS_BITS | TS_TX_EN | TS_RX_EN)
#define CTRL_V3_TX_TS_BITS (CTRL_V3_TS_BITS | TS_TX_EN)
@@ -2235,6 +2235,7 @@ static void cpsw_hwtstamp_v2(struct cpsw_priv *priv)
slave_write(slave, mtype, CPSW2_TS_SEQ_MTYPE);
slave_write(slave, ctrl, CPSW2_CONTROL);
writel_relaxed(ETH_P_1588, &cpsw->regs->ts_ltype);
+ writel_relaxed(ETH_P_8021Q, &cpsw->regs->vlan_ltype);
}
static int cpsw_hwtstamp_set(struct net_device *dev, struct ifreq *ifr)
--
2.17.1
^ permalink raw reply related
* Re: [PATCH net-next 0/6] net: sched: indirect tc block cb registration
From: Jakub Kicinski @ 2018-11-12 4:13 UTC (permalink / raw)
To: David Miller, gerlitz.or
Cc: oss-drivers, netdev, jiri, xiyou.wangcong, jhs, ozsh, vladbu
In-Reply-To: <20181111.095535.170428974627804709.davem@davemloft.net>
On Sun, 11 Nov 2018 09:55:35 -0800 (PST), David Miller wrote:
> From: Jakub Kicinski <jakub.kicinski@netronome.com>
> Date: Fri, 9 Nov 2018 21:21:25 -0800
>
> > John says:
> >
> > This patchset introduces an alternative to egdev offload by allowing a
> > driver to register for block updates when an external device (e.g. tunnel
> > netdev) is bound to a TC block. Drivers can track new netdevs or register
> > to existing ones to receive information on such events. Based on this,
> > they may register for block offload rules using already existing
> > functions.
> >
> > The patchset also implements this new indirect block registration in the
> > NFP driver to allow the offloading of tunnel rules. The use of egdev
> > offload (which is currently only used for tunnel offload) is subsequently
> > removed.
>
> Really nice. Series applied.
>
> Can the Mellanox folks use this too so that we can remove egdev altogether?
> mlx5 is the only remaining user.
I believe Or and Oz are working on mlx5 counterpart, hopefully to land
in this cycle? :)
^ permalink raw reply
* Re: [RFC] mm: Replace all open encodings for NUMA_NO_NODE
From: Anshuman Khandual @ 2018-11-12 4:14 UTC (permalink / raw)
To: Joseph Qi, linux-mm, linux-kernel
Cc: ocfs2-devel, linux-fbdev, dri-devel, netdev, linux-media, iommu,
linux-rdma, dmaengine, linux-block, sparclinux, linuxppc-dev,
linux-ia64, linux-alpha
In-Reply-To: <b92e3275-7a04-a148-bb5b-38658c270583@arm.com>
On 11/12/2018 09:40 AM, Anshuman Khandual wrote:
>
>
> On 11/12/2018 09:27 AM, Joseph Qi wrote:
>> For ocfs2 part, node means host in the cluster, not NUMA node.
>>
>
> Does not -1 indicate an invalid node which can never be present ?
>
My bad, got it wrong. Seems like this is nothing to do with NUMA node
at all. Will drop the changes from ocfs2.
^ permalink raw reply
* Re: [PATCH 1/2] mm/page_alloc: free order-0 pages through PCP in page_frag_free()
From: Alexander Duyck @ 2018-11-12 15:30 UTC (permalink / raw)
To: Paweł Staszewski
Cc: aaron.lu, linux-mm, LKML, Netdev, Andrew Morton,
Jesper Dangaard Brouer, Eric Dumazet, Tariq Toukan,
ilias.apalodimas, yoel, Mel Gorman, Saeed Mahameed, Michal Hocko,
Vlastimil Babka, dave.hansen
In-Reply-To: <bd33633b-2f6c-0034-a130-38a8468531db@itcare.pl>
On Sun, Nov 11, 2018 at 4:39 PM Paweł Staszewski <pstaszewski@itcare.pl> wrote:
>
>
> W dniu 12.11.2018 o 00:05, Alexander Duyck pisze:
> > On Sat, Nov 10, 2018 at 3:54 PM Paweł Staszewski <pstaszewski@itcare.pl> wrote:
> >>
> >>
> >> W dniu 05.11.2018 o 16:44, Alexander Duyck pisze:
> >>> On Mon, Nov 5, 2018 at 12:58 AM Aaron Lu <aaron.lu@intel.com> wrote:
> >>>> page_frag_free() calls __free_pages_ok() to free the page back to
> >>>> Buddy. This is OK for high order page, but for order-0 pages, it
> >>>> misses the optimization opportunity of using Per-Cpu-Pages and can
> >>>> cause zone lock contention when called frequently.
> >>>>
> >>>> Paweł Staszewski recently shared his result of 'how Linux kernel
> >>>> handles normal traffic'[1] and from perf data, Jesper Dangaard Brouer
> >>>> found the lock contention comes from page allocator:
> >>>>
> >>>> mlx5e_poll_tx_cq
> >>>> |
> >>>> --16.34%--napi_consume_skb
> >>>> |
> >>>> |--12.65%--__free_pages_ok
> >>>> | |
> >>>> | --11.86%--free_one_page
> >>>> | |
> >>>> | |--10.10%--queued_spin_lock_slowpath
> >>>> | |
> >>>> | --0.65%--_raw_spin_lock
> >>>> |
> >>>> |--1.55%--page_frag_free
> >>>> |
> >>>> --1.44%--skb_release_data
> >>>>
> >>>> Jesper explained how it happened: mlx5 driver RX-page recycle
> >>>> mechanism is not effective in this workload and pages have to go
> >>>> through the page allocator. The lock contention happens during
> >>>> mlx5 DMA TX completion cycle. And the page allocator cannot keep
> >>>> up at these speeds.[2]
> >>>>
> >>>> I thought that __free_pages_ok() are mostly freeing high order
> >>>> pages and thought this is an lock contention for high order pages
> >>>> but Jesper explained in detail that __free_pages_ok() here are
> >>>> actually freeing order-0 pages because mlx5 is using order-0 pages
> >>>> to satisfy its page pool allocation request.[3]
> >>>>
> >>>> The free path as pointed out by Jesper is:
> >>>> skb_free_head()
> >>>> -> skb_free_frag()
> >>>> -> skb_free_frag()
> >>>> -> page_frag_free()
> >>>> And the pages being freed on this path are order-0 pages.
> >>>>
> >>>> Fix this by doing similar things as in __page_frag_cache_drain() -
> >>>> send the being freed page to PCP if it's an order-0 page, or
> >>>> directly to Buddy if it is a high order page.
> >>>>
> >>>> With this change, Paweł hasn't noticed lock contention yet in
> >>>> his workload and Jesper has noticed a 7% performance improvement
> >>>> using a micro benchmark and lock contention is gone.
> >>>>
> >>>> [1]: https://www.spinics.net/lists/netdev/msg531362.html
> >>>> [2]: https://www.spinics.net/lists/netdev/msg531421.html
> >>>> [3]: https://www.spinics.net/lists/netdev/msg531556.html
> >>>> Reported-by: Paweł Staszewski <pstaszewski@itcare.pl>
> >>>> Analysed-by: Jesper Dangaard Brouer <brouer@redhat.com>
> >>>> Signed-off-by: Aaron Lu <aaron.lu@intel.com>
> >>>> ---
> >>>> mm/page_alloc.c | 10 ++++++++--
> >>>> 1 file changed, 8 insertions(+), 2 deletions(-)
> >>>>
> >>>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> >>>> index ae31839874b8..91a9a6af41a2 100644
> >>>> --- a/mm/page_alloc.c
> >>>> +++ b/mm/page_alloc.c
> >>>> @@ -4555,8 +4555,14 @@ void page_frag_free(void *addr)
> >>>> {
> >>>> struct page *page = virt_to_head_page(addr);
> >>>>
> >>>> - if (unlikely(put_page_testzero(page)))
> >>>> - __free_pages_ok(page, compound_order(page));
> >>>> + if (unlikely(put_page_testzero(page))) {
> >>>> + unsigned int order = compound_order(page);
> >>>> +
> >>>> + if (order == 0)
> >>>> + free_unref_page(page);
> >>>> + else
> >>>> + __free_pages_ok(page, order);
> >>>> + }
> >>>> }
> >>>> EXPORT_SYMBOL(page_frag_free);
> >>>>
> >>> One thing I would suggest for Pawel to try would be to reduce the Tx
> >>> qdisc size on his transmitting interfaces, Reduce the Tx ring size,
> >>> and possibly increase the Tx interrupt rate. Ideally we shouldn't have
> >>> too many packets in-flight and I suspect that is the issue that Pawel
> >>> is seeing that is leading to the page pool allocator freeing up the
> >>> memory. I know we like to try to batch things but the issue is
> >>> processing too many Tx buffers in one batch leads to us eating up too
> >>> much memory and causing evictions from the cache. Ideally the Rx and
> >>> Tx rings and queues should be sized as small as possible while still
> >>> allowing us to process up to our NAPI budget. Usually I run things
> >>> with a 128 Rx / 128 Tx setup and then reduce the Tx queue length so we
> >>> don't have more buffers stored there than we can place in the Tx ring.
> >>> Then we can avoid the extra thrash of having to pull/push memory into
> >>> and out of the freelists. Essentially the issue here ends up being
> >>> another form of buffer bloat.
> >> Thanks Aleksandar - yes it can be - but in my scenario setting RX buffer
> >> <4096 producing more interface rx drops - and no_rx_buffer on network
> >> controller that is receiving more packets
> >> So i need to stick with 3000-4000 on RX - and yes i was trying to lower
> >> the TX buff on connectx4 - but that changed nothing before Aaron patch
> >>
> >> After Aaron patch - decreasing TX buffer influencing total bandwidth
> >> that can be handled by the router/server
> >> Dono why before this patch there was no difference there no matter what
> >> i set there there was always page_alloc/slowpath on top in perf
> >>
> >>
> >> Currently testing RX4096/TX256 - this helps with bandwidth like +10%
> >> more bandwidth with less interrupts...
> > The problem is if you are going for less interrupts you are setting
> > yourself up for buffer bloat. Basically you are going to use much more
> > cache and much more memory then you actually need and if things are
> > properly configured NAPI should take care of the interrupts anyway
> > since under maximum load you shouldn't stop polling normally.
>
> Im trying to balance here - there is problem cause server is forwarding
> all kingd of protocols packets/different size etc
>
> The problem is im trying to go in high interrupt rate - but
>
> Setting coalescence to adaptative for rx killing cpu's at 22Gbit/s RX
> and 22Gbit with rly high interrupt rate
I wouldn't recommend adaptive just because the behavior would be hard
to predict.
> So adding a little more latency i can turn off adaptative rx and setup
> rx-usecs from range 16-64 - and this gives me more or less interrupts -
> but the problem is - always same bandwidth as maximum
What about the tx-usecs, is that a functional thing for the adapter
you are using?
The Rx side logic should be pretty easy to figure out. Essentially you
want to keep the Rx ring size as small as possible while at the same
time avoiding storming the system with interrupts. I know for 10Gb/s I
have used a value of 25us in the past. What you want to watch for is
if you are dropping packets on the Rx side or not. Ideally you want
enough buffers that you can capture any burst while you wait for the
interrupt routine to catch up.
> >
> > One issue I have seen is people delay interrupts for as long as
> > possible which isn't really a good thing since most network
> > controllers will use NAPI which will disable the interrupts and leave
> > them disabled whenever the system is under heavy stress so you should
> > be able to get the maximum performance by configuring an adapter with
> > small ring sizes and for high interrupt rates.
>
> Sure this is bad to setup rx-usec for high values - cause at some point
> this will add high latency for packet traversing both sides - and start
> to hurt buffers
>
> But my problem is a little different now i have no problems with RX side
> - cause i can setup anything like:
>
> coalescence from 16 to 64
>
> rx ring from 3000 to max 8192
>
> And it does not change my max bw - only produces less or more interrupts.
Right so the issue itself isn't Rx, you aren't throttled there. We are
probably looking at an issue of PCIe bandwidth or Tx slowing things
down. The fact that you are still filing interrupts is a bit
surprising though. Are the Tx and Rx interrupts linked for the device
you are using or are they firing them seperately? Normally Rx traffic
won't generate many interrupts under a stress test as NAPI will leave
the interrupts disabled unless it can keep up. Anyway, my suggestion
would be to look at tuning things for as small a ring size as
possible.
> So I start to change params for TX side - and for now i know that the
> best for me is
>
> coalescence adaptative on
>
> TX buffer 128
>
> This helps with max BW that for now is close to 70Gbit/s RX and 70Gbit
> TX but after this change i have increasing DROPS on TX side for vlan
> interfaces.
So this sounds like you are likely bottlenecked due to either PCIe
bandwidth or latency. When you start putting back-pressure on the Tx
like you have described it starts pushing packets onto the Qdisc
layer. One thing that happens when packets are on the qdisc layer is
that they can start to perform a bulk dequeue. The side effect of this
is that you write multiple packets to the descriptor ring and then
update the hardware doorbell only once for the entire group of packets
instead of once per packet.
> And only 50% cpu (max was 50% for 70Gbit/s)
>
>
> > It is easiest to think of it this way. Your total packet rate is equal
> > to your interrupt rate times the number of buffers you will store in
> > the ring. So if you have some fixed rate "X" for packets and an
> > interrupt rate of "i" then your optimal ring size should be "X/i". So
> > if you lower the interrupt rate you end up hurting the throughput
> > unless you increase the buffer size. However at a certain point the
> > buffer size starts becoming an issue. For example with UDP flows I
> > often see massive packet drops if you tune the interrupt rate too low
> > and then put the system under heavy stress.
>
> Yes - in normal life traffic - most of ddos'es are like this many pps
> with small frames.
It sounds to me like XDP would probably be your best bet. With that
you could probably get away with smaller ring sizes, higher interrupt
rates, and get the advantage of it batching the Tx without having to
drop packets.
^ permalink raw reply
* Re: [PATCH] mt76: fix uninitialized mutex access setting rts threshold
From: Kalle Valo @ 2018-11-12 5:50 UTC (permalink / raw)
To: Lorenzo Bianconi; +Cc: nbd, sgruszka, linux-wireless, netdev
In-Reply-To: <1cfa6dca73f1d81595f061ded6275293c6e96b75.1541847014.git.lorenzo.bianconi@redhat.com>
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
> Fix following crash due to a leftover uninitialized mutex access
> in mt76x2_set_rts_threshold routine.
>
> [ 31.018059] Call Trace:
> [ 31.018341] register_lock_class+0x51f/0x530
> [ 31.018828] __lock_acquire+0x6c/0x1580
> [ 31.019247] lock_acquire+0x88/0x120
> [ 31.021089] __mutex_lock+0x4a/0x4f0
> [ 31.023343] mt76x2_set_rts_threshold+0x28/0x50
> [ 31.023831] ieee80211_set_wiphy_params+0x16d/0x4e0
> [ 31.024344] nl80211_set_wiphy+0x72b/0xbc0
> [ 31.024781] genl_family_rcv_msg+0x192/0x3a0
> [ 31.025233] genl_rcv_msg+0x42/0x89
> [ 31.026079] netlink_rcv_skb+0x38/0x100
> [ 31.026475] genl_rcv+0x1f/0x30
> [ 31.026804] netlink_unicast+0x19c/0x250
> [ 31.027212] netlink_sendmsg+0x1ed/0x390
> [ 31.027615] sock_sendmsg+0x31/0x40
> [ 31.027973] ___sys_sendmsg+0x23c/0x280
> [ 31.030414] __sys_sendmsg+0x42/0x80
> [ 31.030783] do_syscall_64+0x4a/0x170
> [ 31.031160] entry_SYSCALL_64_after_hwframe+0x49/0xbe
> [ 31.031677] RIP: 0033:0x7f3498b39ba7
> [ 31.033953] RSP: 002b:00007fffe19675b8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
> [ 31.034883] RAX: ffffffffffffffda RBX: 00000000012d5350 RCX: 00007f3498b39ba7
> [ 31.035756] RDX: 0000000000000000 RSI: 00007fffe19675f0 RDI: 0000000000000003
> [ 31.036587] RBP: 00000000012da740 R08: 0000000000000002 R09: 0000000000000000
> [ 31.037422] R10: 0000000000000006 R11: 0000000000000246 R12: 00000000012da880
> [ 31.038252] R13: 00007fffe19675f0 R14: 00007fffe19678c0 R15: 00000000012da880
>
> Fixes: 108a4861ef19 ("mt76: create new mt76x02-lib module for common
> mt76x{0,2} code")
> Reported-by: lorenzo.trisolini@fluidmesh.com
> Reported-by: luca.bisti@fluidmesh.com
> Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com>
This should be v2, but no need to resend.
> This patch is for 4.20
Ok, I'll queue this for that release.
--
Kalle Valo
^ permalink raw reply
* KASAN: use-after-free Read in nbp_vlan_rcu_free
From: syzbot @ 2018-11-12 5:51 UTC (permalink / raw)
To: bridge, davem, linux-kernel, netdev, nikolay, roopa,
syzkaller-bugs
Hello,
syzbot found the following crash on:
HEAD commit: e12e00e388de Merge tag 'kbuild-fixes-v4.20' of git://git.k..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=14cdb6f5400000
kernel config: https://syzkaller.appspot.com/x/.config?x=8f215f21f041a0d7
dashboard link: https://syzkaller.appspot.com/bug?extid=04681da557a0e49a52e5
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
Unfortunately, I don't have any reproducer for this crash yet.
IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+04681da557a0e49a52e5@syzkaller.appspotmail.com
device bridge_slave_1 left promiscuous mode
bridge0: port 2(bridge_slave_1) entered disabled state
device bridge_slave_0 left promiscuous mode
bridge0: port 1(bridge_slave_0) entered disabled state
==================================================================
BUG: KASAN: use-after-free in nbp_vlan_rcu_free+0x152/0x160
net/bridge/br_vlan.c:200
Read of size 8 at addr ffff8881bbc44a18 by task swapper/1/0
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.20.0-rc1+ #332
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x244/0x39d lib/dump_stack.c:113
print_address_description.cold.7+0x9/0x1ff mm/kasan/report.c:256
kasan_report_error mm/kasan/report.c:354 [inline]
kasan_report.cold.8+0x242/0x309 mm/kasan/report.c:412
__asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433
nbp_vlan_rcu_free+0x152/0x160 net/bridge/br_vlan.c:200
__rcu_reclaim kernel/rcu/rcu.h:240 [inline]
rcu_do_batch kernel/rcu/tree.c:2437 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2716 [inline]
rcu_process_callbacks+0x100a/0x1ac0 kernel/rcu/tree.c:2697
__do_softirq+0x308/0xb7e kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x17f/0x1c0 kernel/softirq.c:413
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x1cb/0x760 arch/x86/kernel/apic/apic.c:1061
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:804
</IRQ>
RIP: 0010:native_safe_halt+0x6/0x10 arch/x86/include/asm/irqflags.h:57
Code: e9 2c ff ff ff 48 89 c7 48 89 45 d8 e8 d3 43 f3 f9 48 8b 45 d8 e9 ca
fe ff ff 48 89 df e8 c2 43 f3 f9 eb 82 55 48 89 e5 fb f4 <5d> c3 0f 1f 84
00 00 00 00 00 55 48 89 e5 f4 5d c3 90 90 90 90 90
RSP: 0018:ffff8881d9b27cb8 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
RAX: dffffc0000000000 RBX: 1ffff1103b364f9b RCX: 0000000000000000
RDX: 1ffffffff12a3f71 RSI: 0000000000000001 RDI: ffffffff8951fb88
RBP: ffff8881d9b27cb8 R08: ffff8881d9b14340 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8881d9b27d78
R13: ffffffff8a14e1e0 R14: 0000000000000000 R15: 0000000000000001
arch_safe_halt arch/x86/include/asm/paravirt.h:151 [inline]
default_idle+0xbf/0x490 arch/x86/kernel/process.c:498
arch_cpu_idle+0x10/0x20 arch/x86/kernel/process.c:489
default_idle_call+0x6d/0x90 kernel/sched/idle.c:93
cpuidle_idle_call kernel/sched/idle.c:153 [inline]
do_idle+0x49b/0x5c0 kernel/sched/idle.c:262
cpu_startup_entry+0x18/0x20 kernel/sched/idle.c:353
start_secondary+0x487/0x5f0 arch/x86/kernel/smpboot.c:271
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243
Allocated by task 13858:
save_stack+0x43/0xd0 mm/kasan/kasan.c:448
set_track mm/kasan/kasan.c:460 [inline]
kasan_kmalloc+0xc7/0xe0 mm/kasan/kasan.c:553
kmem_cache_alloc_trace+0x152/0x750 mm/slab.c:3620
kmalloc include/linux/slab.h:546 [inline]
kzalloc include/linux/slab.h:741 [inline]
br_vlan_add+0x6e5/0x1340 net/bridge/br_vlan.c:650
br_vlan_init+0x339/0x3e0 net/bridge/br_vlan.c:1047
br_dev_init+0x10d/0x2a0 net/bridge/br_device.c:134
register_netdevice+0x332/0x11d0 net/core/dev.c:8459
br_dev_newlink+0x2a/0x130 net/bridge/br_netlink.c:1309
rtnl_newlink+0xf08/0x1da0 net/core/rtnetlink.c:3175
rtnetlink_rcv_msg+0x46a/0xc20 net/core/rtnetlink.c:4947
netlink_rcv_skb+0x172/0x440 net/netlink/af_netlink.c:2477
rtnetlink_rcv+0x1c/0x20 net/core/rtnetlink.c:4965
netlink_unicast_kernel net/netlink/af_netlink.c:1310 [inline]
netlink_unicast+0x5a5/0x760 net/netlink/af_netlink.c:1336
netlink_sendmsg+0xa18/0xfc0 net/netlink/af_netlink.c:1917
sock_sendmsg_nosec net/socket.c:621 [inline]
sock_sendmsg+0xd5/0x120 net/socket.c:631
___sys_sendmsg+0x7fd/0x930 net/socket.c:2116
__sys_sendmsg+0x11d/0x280 net/socket.c:2154
__do_sys_sendmsg net/socket.c:2163 [inline]
__se_sys_sendmsg net/socket.c:2161 [inline]
__x64_sys_sendmsg+0x78/0xb0 net/socket.c:2161
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
Freed by task 9:
save_stack+0x43/0xd0 mm/kasan/kasan.c:448
set_track mm/kasan/kasan.c:460 [inline]
__kasan_slab_free+0x102/0x150 mm/kasan/kasan.c:521
kasan_slab_free+0xe/0x10 mm/kasan/kasan.c:528
__cache_free mm/slab.c:3498 [inline]
kfree+0xcf/0x230 mm/slab.c:3817
br_master_vlan_rcu_free+0xa8/0xe0 net/bridge/br_vlan.c:174
__rcu_reclaim kernel/rcu/rcu.h:240 [inline]
rcu_do_batch kernel/rcu/tree.c:2437 [inline]
invoke_rcu_callbacks kernel/rcu/tree.c:2716 [inline]
rcu_process_callbacks+0x100a/0x1ac0 kernel/rcu/tree.c:2697
__do_softirq+0x308/0xb7e kernel/softirq.c:292
The buggy address belongs to the object at ffff8881bbc44a00
which belongs to the cache kmalloc-96 of size 96
The buggy address is located 24 bytes inside of
96-byte region [ffff8881bbc44a00, ffff8881bbc44a60)
The buggy address belongs to the page:
page:ffffea0006ef1100 count:1 mapcount:0 mapping:ffff8881da8004c0 index:0x0
flags: 0x2fffc0000000200(slab)
raw: 02fffc0000000200 ffffea0007471a08 ffffea000757b908 ffff8881da8004c0
raw: 0000000000000000 ffff8881bbc44000 0000000100000020 0000000000000000
page dumped because: kasan: bad access detected
Memory state around the buggy address:
ffff8881bbc44900: fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
ffff8881bbc44980: fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
> ffff8881bbc44a00: fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
^
ffff8881bbc44a80: 00 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc
ffff8881bbc44b00: fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
==================================================================
---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.
^ permalink raw reply
* Re: [PATCH 1/2] mm/page_alloc: free order-0 pages through PCP in page_frag_free()
From: Eric Dumazet @ 2018-11-12 15:44 UTC (permalink / raw)
To: Alexander Duyck, Paweł Staszewski
Cc: aaron.lu, linux-mm, LKML, Netdev, Andrew Morton,
Jesper Dangaard Brouer, Eric Dumazet, Tariq Toukan,
ilias.apalodimas, yoel, Mel Gorman, Saeed Mahameed, Michal Hocko,
Vlastimil Babka, dave.hansen
In-Reply-To: <CAKgT0UeOBF0yPJLOTBBb3m7nTkmSDxzkCur+iGzJ++Y-jWaw9g@mail.gmail.com>
On 11/12/2018 07:30 AM, Alexander Duyck wrote:
> It sounds to me like XDP would probably be your best bet. With that
> you could probably get away with smaller ring sizes, higher interrupt
> rates, and get the advantage of it batching the Tx without having to
> drop packets.
Add to this that with XDP (or anything lowering per packet processing costs)
you can reduce number of cpus/queues, get better latencies, and bigger TX batches.
^ permalink raw reply
* 答复: [PATCH][RFC] udp: cache sock to avoid searching it twice
From: Li,Rongqing @ 2018-11-12 5:39 UTC (permalink / raw)
To: Eric Dumazet, netdev@vger.kernel.org
In-Reply-To: <0c6a070f-d2ac-2ffc-00d5-097fe84ae42c@gmail.com>
On Sat, Nov 10, 2018 at 1:29 AM Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
>
>
> On 11/08/2018 10:21 PM, Li RongQing wrote:
> > GRO for UDP needs to lookup socket twice, first is in gro receive,
> > second is gro complete, so if store sock to skb to avoid looking up
> > twice, this can give small performance boost
> >
> > netperf -t UDP_RR -l 10
> >
> > Before:
> > Rate per sec: 28746.01
> > After:
> > Rate per sec: 29401.67
> >
> > Signed-off-by: Li RongQing <lirongqing@baidu.com>
> > ---
> > net/ipv4/udp_offload.c | 18 +++++++++++++++++-
> > 1 file changed, 17 insertions(+), 1 deletion(-)
> >
> > diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c
> > index 0646d61f4fa8..429570112a33 100644
> > --- a/net/ipv4/udp_offload.c
> > +++ b/net/ipv4/udp_offload.c
> > @@ -408,6 +408,11 @@ struct sk_buff *udp_gro_receive(struct list_head *head, struct sk_buff *skb,
> >
> > if (udp_sk(sk)->gro_enabled) {
> > pp = call_gro_receive(udp_gro_receive_segment, head, skb);
> > +
> > + if (!IS_ERR(pp) && NAPI_GRO_CB(pp)->count > 1) {
> > + sock_hold(sk);
> > + pp->sk = sk;
>
>
> You also have to set pp->destructor to sock_edemux
>
> flush_gro_hash -> kfree_skb()
>
> If there is no destructor, the reference on pp->sk will never be released.
>
>
Ok, thanks,
does it need to reset sk in udp_gro_complete, ip early demuxing will lookup udp socket again, if we can keep it, we can avoid to lookup socket again
-RongQing
>
>
> > + }
> > rcu_read_unlock();
> > return pp;
> > }
> > @@ -444,6 +449,10 @@ struct sk_buff *udp_gro_receive(struct list_head *head, struct sk_buff *skb,
> > skb_gro_postpull_rcsum(skb, uh, sizeof(struct udphdr));
> > pp = call_gro_receive_sk(udp_sk(sk)->gro_receive, sk, head, skb);
> >
> > + if (!IS_ERR(pp) && NAPI_GRO_CB(pp)->count > 1) {
> > + sock_hold(sk);
> > + pp->sk = sk;
> > + }
> > out_unlock:
> > rcu_read_unlock();
> > skb_gro_flush_final(skb, pp, flush);
> > @@ -502,7 +511,9 @@ int udp_gro_complete(struct sk_buff *skb, int nhoff,
> > uh->len = newlen;
> >
> > rcu_read_lock();
> > - sk = (*lookup)(skb, uh->source, uh->dest);
> > + sk = skb->sk;
> > + if (!sk)
> > + sk = (*lookup)(skb, uh->source, uh->dest);
> > if (sk && udp_sk(sk)->gro_enabled) {
> > err = udp_gro_complete_segment(skb);
> > } else if (sk && udp_sk(sk)->gro_complete) {
> > @@ -516,6 +527,11 @@ int udp_gro_complete(struct sk_buff *skb, int nhoff,
> > err = udp_sk(sk)->gro_complete(sk, skb,
> > nhoff + sizeof(struct udphdr));
> > }
> > +
> > + if (skb->sk) {
> > + sock_put(skb->sk);
> > + skb->sk = NULL;
> > + }
> > rcu_read_unlock();
> >
> > if (skb->remcsum_offload)
> >
^ permalink raw reply
* 答复: [PATCH][RFC] udp: cache sock to avoid searching it twice
From: Li,Rongqing @ 2018-11-12 5:46 UTC (permalink / raw)
To: Paolo Abeni, netdev@vger.kernel.org; +Cc: Willem de Bruijn
In-Reply-To: <2b32d67c09bbdec46b97a3cf2898145ce60a6836.camel@redhat.com>
> > return pp;
> > }
>
> What if 'pp' is NULL?
>
> Aside from that, this replace a lookup with 2 atomic ops, and only when
> such lookup is amortized on multiple aggregated packets: I'm unsure if
> it's worthy and I don't understand how that improves RR tests (where
> the socket can't see multiple, consecutive skbs, AFAIK).
>
> Cheers,
>
> Paolo
>
If we not release the socket in udp_gro_complete , we can reduce a udp socket
Lookup when do ip demux again, it maybe more worthy.
I test UDP_STREAM, find no difference, both can reach NIC's limit, 10G;
so Test RR, I will do more tests
-RongQing
^ permalink raw reply
* Re: [PATCH 2/2] dt-binding: mediatek-dwmac: add binding document for MediaTek MT2712 DWMAC
From: Rob Herring @ 2018-11-12 16:22 UTC (permalink / raw)
To: Biao Huang
Cc: mark.rutland, devicetree, nelson.chang, andrew, netdev, sean.wang,
liguo.zhang, linux-kernel, matthias.bgg, joabreu, linux-mediatek,
honghui.zhang, yt.shen, davem, linux-arm-kernel
In-Reply-To: <1540782294-27721-3-git-send-email-biao.huang@mediatek.com>
On Mon, Oct 29, 2018 at 11:04:54AM +0800, Biao Huang wrote:
> The commit adds the device tree binding documentation for the MediaTek DWMAC
> found on MediaTek MT2712.
>
> Signed-off-by: Biao Huang <biao.huang@mediatek.com>
> ---
> .../devicetree/bindings/net/mediatek-dwmac.txt | 49 ++++++++++++++++++++
> 1 file changed, 49 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/net/mediatek-dwmac.txt
>
> diff --git a/Documentation/devicetree/bindings/net/mediatek-dwmac.txt b/Documentation/devicetree/bindings/net/mediatek-dwmac.txt
> new file mode 100644
> index 0000000..8baf0e4
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/net/mediatek-dwmac.txt
> @@ -0,0 +1,49 @@
> +MediaTek DWMAC glue layer controller
> +
> +This file documents platform glue layer for stmmac.
> +Please see stmmac.txt for the other unchanged properties.
> +
> +The device node has following properties.
> +
> +Required properties:
> +- compatible: Should be "mediatek,mt2712-gmac" for MT2712 SoC
> +- reg: Address and length of the register set for the device
> +- interrupts: Should contain the MAC interrupts
> +- interrupt-names: Should contain a list of interrupt names corresponding to
> + the interrupts in the interrupts property, if available.
> + Should be "macirq" for the main MAC IRQ
> +- clocks: Must contain a phandle for each entry in clock-names.
> +- clock-names: the name of the clock listed in the clocks property. These are
> + "axi", "apb", "mac_ext", "mac_parent", "ptp_ref", "ptp_parent", "ptp_top"
> + for MT2712 SoC
Clocks should represent the physical clocks connected to a block. Parent
clocks are not in that category.
> +- mac-address: See ethernet.txt in the same directory
> +- phy-mode: See ethernet.txt in the same directory
> +
> +Example:
> +
> + eth: ethernet@1101c000 {
> + compatible = "mediatek,mt2712-gmac";
> + reg = <0 0x1101c000 0 0x1300>;
> + interrupts = <GIC_SPI 237 IRQ_TYPE_LEVEL_LOW>;
> + interrupt-names = "macirq";
> + phy-mode ="rgmii-id";
> + mac-address = [00 55 7b b5 7d f7];
> + clock-names = "axi",
> + "apb",
> + "mac_ext",
> + "mac_parent",
> + "ptp_ref",
> + "ptp_parent",
> + "ptp_top";
> + clocks = <&pericfg CLK_PERI_GMAC>,
> + <&pericfg CLK_PERI_GMAC_PCLK>,
> + <&topckgen CLK_TOP_ETHER_125M_SEL>,
> + <&topckgen CLK_TOP_ETHERPLL_125M>,
> + <&topckgen CLK_TOP_ETHER_50M_SEL>,
> + <&topckgen CLK_TOP_APLL1_D3>,
> + <&topckgen CLK_TOP_APLL1>;
> + snps,txpbl = <32>;
> + snps,rxpbl = <32>;
> + snps,reset-gpio = <&pio 87 GPIO_ACTIVE_LOW>;
> + snps,reset-active-low;
> + };
> --
> 1.7.9.5
>
^ permalink raw reply
* Re: [PATCH net-next 1/6] net: sched: register callbacks for indirect tc block binds
From: Jiri Pirko @ 2018-11-12 6:37 UTC (permalink / raw)
To: Jakub Kicinski
Cc: davem, oss-drivers, netdev, xiyou.wangcong, jhs, gerlitz.or, ozsh,
vladbu, John Hurley
In-Reply-To: <20181110052131.3306-2-jakub.kicinski@netronome.com>
Sat, Nov 10, 2018 at 06:21:26AM CET, jakub.kicinski@netronome.com wrote:
>From: John Hurley <john.hurley@netronome.com>
>
>Currently drivers can register to receive TC block bind/unbind callbacks
>by implementing the setup_tc ndo in any of their given netdevs. However,
>drivers may also be interested in binds to higher level devices (e.g.
>tunnel drivers) to potentially offload filters applied to them.
>
>Introduce indirect block devs which allows drivers to register callbacks
>for block binds on other devices. The callback is triggered when the
>device is bound to a block, allowing the driver to register for rules
>applied to that block using already available functions.
>
>Freeing an indirect block callback will trigger an unbind event (if
>necessary) to direct the driver to remove any offloaded rules and unreg
>any block rule callbacks. It is the responsibility of the implementing
>driver to clean any registered indirect block callbacks before exiting,
>if the block it still active at such a time.
>
>Allow registering an indirect block dev callback for a device that is
>already bound to a block. In this case (if it is an ingress block),
>register and also trigger the callback meaning that any already installed
>rules can be replayed to the calling driver.
>
>Signed-off-by: John Hurley <john.hurley@netronome.com>
>Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Acked-by: Jiri Pirko <jiri@mellanox.com>
^ permalink raw reply
* Re: [PATCH 1/2] mm/page_alloc: free order-0 pages through PCP in page_frag_free()
From: Paweł Staszewski @ 2018-11-12 17:01 UTC (permalink / raw)
To: Alexander Duyck
Cc: aaron.lu, linux-mm, LKML, Netdev, Andrew Morton,
Jesper Dangaard Brouer, Eric Dumazet, Tariq Toukan,
ilias.apalodimas, yoel, Mel Gorman, Saeed Mahameed, Michal Hocko,
Vlastimil Babka, dave.hansen
In-Reply-To: <CAKgT0UeOBF0yPJLOTBBb3m7nTkmSDxzkCur+iGzJ++Y-jWaw9g@mail.gmail.com>
W dniu 12.11.2018 o 16:30, Alexander Duyck pisze:
> On Sun, Nov 11, 2018 at 4:39 PM Paweł Staszewski <pstaszewski@itcare.pl> wrote:
>>
>> W dniu 12.11.2018 o 00:05, Alexander Duyck pisze:
>>> On Sat, Nov 10, 2018 at 3:54 PM Paweł Staszewski <pstaszewski@itcare.pl> wrote:
>>>>
>>>> W dniu 05.11.2018 o 16:44, Alexander Duyck pisze:
>>>>> On Mon, Nov 5, 2018 at 12:58 AM Aaron Lu <aaron.lu@intel.com> wrote:
>>>>>> page_frag_free() calls __free_pages_ok() to free the page back to
>>>>>> Buddy. This is OK for high order page, but for order-0 pages, it
>>>>>> misses the optimization opportunity of using Per-Cpu-Pages and can
>>>>>> cause zone lock contention when called frequently.
>>>>>>
>>>>>> Paweł Staszewski recently shared his result of 'how Linux kernel
>>>>>> handles normal traffic'[1] and from perf data, Jesper Dangaard Brouer
>>>>>> found the lock contention comes from page allocator:
>>>>>>
>>>>>> mlx5e_poll_tx_cq
>>>>>> |
>>>>>> --16.34%--napi_consume_skb
>>>>>> |
>>>>>> |--12.65%--__free_pages_ok
>>>>>> | |
>>>>>> | --11.86%--free_one_page
>>>>>> | |
>>>>>> | |--10.10%--queued_spin_lock_slowpath
>>>>>> | |
>>>>>> | --0.65%--_raw_spin_lock
>>>>>> |
>>>>>> |--1.55%--page_frag_free
>>>>>> |
>>>>>> --1.44%--skb_release_data
>>>>>>
>>>>>> Jesper explained how it happened: mlx5 driver RX-page recycle
>>>>>> mechanism is not effective in this workload and pages have to go
>>>>>> through the page allocator. The lock contention happens during
>>>>>> mlx5 DMA TX completion cycle. And the page allocator cannot keep
>>>>>> up at these speeds.[2]
>>>>>>
>>>>>> I thought that __free_pages_ok() are mostly freeing high order
>>>>>> pages and thought this is an lock contention for high order pages
>>>>>> but Jesper explained in detail that __free_pages_ok() here are
>>>>>> actually freeing order-0 pages because mlx5 is using order-0 pages
>>>>>> to satisfy its page pool allocation request.[3]
>>>>>>
>>>>>> The free path as pointed out by Jesper is:
>>>>>> skb_free_head()
>>>>>> -> skb_free_frag()
>>>>>> -> skb_free_frag()
>>>>>> -> page_frag_free()
>>>>>> And the pages being freed on this path are order-0 pages.
>>>>>>
>>>>>> Fix this by doing similar things as in __page_frag_cache_drain() -
>>>>>> send the being freed page to PCP if it's an order-0 page, or
>>>>>> directly to Buddy if it is a high order page.
>>>>>>
>>>>>> With this change, Paweł hasn't noticed lock contention yet in
>>>>>> his workload and Jesper has noticed a 7% performance improvement
>>>>>> using a micro benchmark and lock contention is gone.
>>>>>>
>>>>>> [1]: https://www.spinics.net/lists/netdev/msg531362.html
>>>>>> [2]: https://www.spinics.net/lists/netdev/msg531421.html
>>>>>> [3]: https://www.spinics.net/lists/netdev/msg531556.html
>>>>>> Reported-by: Paweł Staszewski <pstaszewski@itcare.pl>
>>>>>> Analysed-by: Jesper Dangaard Brouer <brouer@redhat.com>
>>>>>> Signed-off-by: Aaron Lu <aaron.lu@intel.com>
>>>>>> ---
>>>>>> mm/page_alloc.c | 10 ++++++++--
>>>>>> 1 file changed, 8 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>>>>>> index ae31839874b8..91a9a6af41a2 100644
>>>>>> --- a/mm/page_alloc.c
>>>>>> +++ b/mm/page_alloc.c
>>>>>> @@ -4555,8 +4555,14 @@ void page_frag_free(void *addr)
>>>>>> {
>>>>>> struct page *page = virt_to_head_page(addr);
>>>>>>
>>>>>> - if (unlikely(put_page_testzero(page)))
>>>>>> - __free_pages_ok(page, compound_order(page));
>>>>>> + if (unlikely(put_page_testzero(page))) {
>>>>>> + unsigned int order = compound_order(page);
>>>>>> +
>>>>>> + if (order == 0)
>>>>>> + free_unref_page(page);
>>>>>> + else
>>>>>> + __free_pages_ok(page, order);
>>>>>> + }
>>>>>> }
>>>>>> EXPORT_SYMBOL(page_frag_free);
>>>>>>
>>>>> One thing I would suggest for Pawel to try would be to reduce the Tx
>>>>> qdisc size on his transmitting interfaces, Reduce the Tx ring size,
>>>>> and possibly increase the Tx interrupt rate. Ideally we shouldn't have
>>>>> too many packets in-flight and I suspect that is the issue that Pawel
>>>>> is seeing that is leading to the page pool allocator freeing up the
>>>>> memory. I know we like to try to batch things but the issue is
>>>>> processing too many Tx buffers in one batch leads to us eating up too
>>>>> much memory and causing evictions from the cache. Ideally the Rx and
>>>>> Tx rings and queues should be sized as small as possible while still
>>>>> allowing us to process up to our NAPI budget. Usually I run things
>>>>> with a 128 Rx / 128 Tx setup and then reduce the Tx queue length so we
>>>>> don't have more buffers stored there than we can place in the Tx ring.
>>>>> Then we can avoid the extra thrash of having to pull/push memory into
>>>>> and out of the freelists. Essentially the issue here ends up being
>>>>> another form of buffer bloat.
>>>> Thanks Aleksandar - yes it can be - but in my scenario setting RX buffer
>>>> <4096 producing more interface rx drops - and no_rx_buffer on network
>>>> controller that is receiving more packets
>>>> So i need to stick with 3000-4000 on RX - and yes i was trying to lower
>>>> the TX buff on connectx4 - but that changed nothing before Aaron patch
>>>>
>>>> After Aaron patch - decreasing TX buffer influencing total bandwidth
>>>> that can be handled by the router/server
>>>> Dono why before this patch there was no difference there no matter what
>>>> i set there there was always page_alloc/slowpath on top in perf
>>>>
>>>>
>>>> Currently testing RX4096/TX256 - this helps with bandwidth like +10%
>>>> more bandwidth with less interrupts...
>>> The problem is if you are going for less interrupts you are setting
>>> yourself up for buffer bloat. Basically you are going to use much more
>>> cache and much more memory then you actually need and if things are
>>> properly configured NAPI should take care of the interrupts anyway
>>> since under maximum load you shouldn't stop polling normally.
>> Im trying to balance here - there is problem cause server is forwarding
>> all kingd of protocols packets/different size etc
>>
>> The problem is im trying to go in high interrupt rate - but
>>
>> Setting coalescence to adaptative for rx killing cpu's at 22Gbit/s RX
>> and 22Gbit with rly high interrupt rate
> I wouldn't recommend adaptive just because the behavior would be hard
> to predict.
>
>> So adding a little more latency i can turn off adaptative rx and setup
>> rx-usecs from range 16-64 - and this gives me more or less interrupts -
>> but the problem is - always same bandwidth as maximum
> What about the tx-usecs, is that a functional thing for the adapter
> you are using?
Yes tx-usecs is not used now cause of adaptative mode on tx side:
ethtool -c enp175s0
Coalesce parameters for enp175s0:
Adaptive RX: off TX: on
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0
dmac: 32551
rx-usecs: 64
rx-frames: 128
rx-usecs-irq: 0
rx-frames-irq: 0
tx-usecs: 8
tx-frames: 64
tx-usecs-irq: 0
tx-frames-irq: 0
rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0
rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0
>
> The Rx side logic should be pretty easy to figure out. Essentially you
> want to keep the Rx ring size as small as possible while at the same
> time avoiding storming the system with interrupts. I know for 10Gb/s I
> have used a value of 25us in the past. What you want to watch for is
> if you are dropping packets on the Rx side or not. Ideally you want
> enough buffers that you can capture any burst while you wait for the
> interrupt routine to catch up.
>
>>> One issue I have seen is people delay interrupts for as long as
>>> possible which isn't really a good thing since most network
>>> controllers will use NAPI which will disable the interrupts and leave
>>> them disabled whenever the system is under heavy stress so you should
>>> be able to get the maximum performance by configuring an adapter with
>>> small ring sizes and for high interrupt rates.
>> Sure this is bad to setup rx-usec for high values - cause at some point
>> this will add high latency for packet traversing both sides - and start
>> to hurt buffers
>>
>> But my problem is a little different now i have no problems with RX side
>> - cause i can setup anything like:
>>
>> coalescence from 16 to 64
>>
>> rx ring from 3000 to max 8192
>>
>> And it does not change my max bw - only produces less or more interrupts.
> Right so the issue itself isn't Rx, you aren't throttled there. We are
> probably looking at an issue of PCIe bandwidth or Tx slowing things
> down. The fact that you are still filing interrupts is a bit
> surprising though. Are the Tx and Rx interrupts linked for the device
> you are using or are they firing them seperately? Normally Rx traffic
> won't generate many interrupts under a stress test as NAPI will leave
> the interrupts disabled unless it can keep up. Anyway, my suggestion
> would be to look at tuning things for as small a ring size as
> possible.
PCIe bw was eliminated - previously there was one 2 port 100G card
installed in one pciex16 (max bw for pcie x16 gen3 is 32GB/s 16/16GB
bidirectional)
Currently there are two separate nic's installed in two separate x16
slots - so can't be problem with pcie bandwidth
But i think I reach memory bandwidth limit now for 70Gbit/70Gbit :)
But wondering if there is any counter that can help me to diagnose
problems with memory bandwidth ?
stream app tests gives me results like:
./stream_c.exe
-------------------------------------------------------------
STREAM version $Revision: 5.10 $
-------------------------------------------------------------
This system uses 8 bytes per array element.
-------------------------------------------------------------
Array size = 10000000 (elements), Offset = 0 (elements)
Memory per array = 76.3 MiB (= 0.1 GiB).
Total memory required = 228.9 MiB (= 0.2 GiB).
Each kernel will be executed 10 times.
The *best* time for each kernel (excluding the first iteration)
will be used to compute the reported bandwidth.
-------------------------------------------------------------
Number of Threads requested = 56
Number of Threads counted = 56
-------------------------------------------------------------
Your clock granularity/precision appears to be 1 microseconds.
Each test below will take on the order of 4081 microseconds.
(= 4081 clock ticks)
Increase the size of the arrays if this shows that
you are not getting at least 20 clock ticks per test.
-------------------------------------------------------------
WARNING -- The above is only a rough guideline.
For best results, please be sure you know the
precision of your system timer.
-------------------------------------------------------------
Function Best Rate MB/s Avg time Min time Max time
Copy: 29907.2 0.005382 0.005350 0.005405
Scale: 28787.3 0.005611 0.005558 0.005650
Add: 34153.3 0.007037 0.007027 0.007055
Triad: 34944.0 0.006880 0.006868 0.006887
-------------------------------------------------------------
Solution Validates: avg error less than 1.000000e-13 on all three arrays
But this is for node 0+1
When limiting test to one node and cores used by network controllers:
-------------------------------------------------------------
STREAM version $Revision: 5.10 $
-------------------------------------------------------------
This system uses 8 bytes per array element.
-------------------------------------------------------------
Array size = 10000000 (elements), Offset = 0 (elements)
Memory per array = 76.3 MiB (= 0.1 GiB).
Total memory required = 228.9 MiB (= 0.2 GiB).
Each kernel will be executed 10 times.
The *best* time for each kernel (excluding the first iteration)
will be used to compute the reported bandwidth.
-------------------------------------------------------------
Number of Threads requested = 28
Number of Threads counted = 28
-------------------------------------------------------------
Your clock granularity/precision appears to be 1 microseconds.
Each test below will take on the order of 6107 microseconds.
(= 6107 clock ticks)
Increase the size of the arrays if this shows that
you are not getting at least 20 clock ticks per test.
-------------------------------------------------------------
WARNING -- The above is only a rough guideline.
For best results, please be sure you know the
precision of your system timer.
-------------------------------------------------------------
Function Best Rate MB/s Avg time Min time Max time
Copy: 20156.4 0.007946 0.007938 0.007958
Scale: 19436.1 0.008237 0.008232 0.008243
Add: 20184.7 0.011896 0.011890 0.011904
Triad: 20687.9 0.011607 0.011601 0.011613
-------------------------------------------------------------
Solution Validates: avg error less than 1.000000e-13 on all three arrays
-------------------------------------------------------------
Close to the limit but still some place - there can be some doubled
operations like for RX/TX side and network controllers can use more
bandwidth or just can't do this more optimally - cause of
bulking/buffers etc.
So currently there are only four from six channels used - i will upgrade
also memory and populate all six channels left/right side for two memory
controllers that cpu have.
>> So I start to change params for TX side - and for now i know that the
>> best for me is
>>
>> coalescence adaptative on
>>
>> TX buffer 128
>>
>> This helps with max BW that for now is close to 70Gbit/s RX and 70Gbit
>> TX but after this change i have increasing DROPS on TX side for vlan
>> interfaces.
> So this sounds like you are likely bottlenecked due to either PCIe
> bandwidth or latency. When you start putting back-pressure on the Tx
> like you have described it starts pushing packets onto the Qdisc
> layer. One thing that happens when packets are on the qdisc layer is
> that they can start to perform a bulk dequeue. The side effect of this
> is that you write multiple packets to the descriptor ring and then
> update the hardware doorbell only once for the entire group of packets
> instead of once per packet.
yes the problem is i just can't find any place where counters will shows
me why nic's start to drop packets
it does not reflect in cpu load or any other counter besides rx_phy
drops and tx_vlan drop packets
>> And only 50% cpu (max was 50% for 70Gbit/s)
>>
>>
>>> It is easiest to think of it this way. Your total packet rate is equal
>>> to your interrupt rate times the number of buffers you will store in
>>> the ring. So if you have some fixed rate "X" for packets and an
>>> interrupt rate of "i" then your optimal ring size should be "X/i". So
>>> if you lower the interrupt rate you end up hurting the throughput
>>> unless you increase the buffer size. However at a certain point the
>>> buffer size starts becoming an issue. For example with UDP flows I
>>> often see massive packet drops if you tune the interrupt rate too low
>>> and then put the system under heavy stress.
>> Yes - in normal life traffic - most of ddos'es are like this many pps
>> with small frames.
> It sounds to me like XDP would probably be your best bet. With that
> you could probably get away with smaller ring sizes, higher interrupt
> rates, and get the advantage of it batching the Tx without having to
> drop packets.
Yes im testing in lab xdp_fwd currently - but have some problems with
random drops that occuring randomly where server forwards only 1/10
packet and after some time it starts to work normally.
Currently trying to eliminate nic's offloading that can cause this - so
turning off one by one and running tests.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox