* Re: Kernel 4.19 network performance - forwarding/routing normal users traffic
From: David Ahern @ 2018-11-08 16:06 UTC (permalink / raw)
To: Paweł Staszewski, Jesper Dangaard Brouer; +Cc: netdev, Yoel Caspersen
In-Reply-To: <8dde3b32-59ce-38f3-5913-2ce08264e9dc@itcare.pl>
On 11/8/18 6:33 AM, Paweł Staszewski wrote:
>
>
> W dniu 07.11.2018 o 22:06, David Ahern pisze:
>> On 11/3/18 6:24 PM, Paweł Staszewski wrote:
>>>> Does your setup have any other device types besides physical ports with
>>>> VLANs (e.g., any macvlans or bonds)?
>>>>
>>>>
>>> no.
>>> just
>>> phy(mlnx)->vlans only config
>> VLAN and non-VLAN (and a mix) seem to work ok. Patches are here:
>> https://github.com/dsahern/linux.git bpf/kernel-tables-wip
>>
>> I got lazy with the vlan exports; right now it requires 8021q to be
>> builtin (CONFIG_VLAN_8021Q=y)
>>
>> You can use the xdp_fwd sample:
>> make O=kbuild -C samples/bpf -j 8
>>
>> Copy samples/bpf/xdp_fwd_kern.o and samples/bpf/xdp_fwd to the server
>> and run:
>> ./xdp_fwd <list of NIC ports>
>>
>> e.g., in my testing I run:
>> xdp_fwd eth1 eth2 eth3 eth4
>>
>> All of the relevant forwarding ports need to be on the same command
>> line. This version populates a second map to verify the egress port has
>> XDP enabled.
> Installed today on some lab server with mellanox connectx4
>
> And trying some simple static routing first - but after enabling xdp
> program - receiver is not receiving frames
>
> Route table is simple as possible for tests :)
>
> icmp ping test send from 192.168.22.237 to 172.16.0.2 - incomming
> packets on vlan 4081
>
> ip r
> default via 192.168.22.236 dev vlan4081
> 172.16.0.0/30 dev vlan1740 proto kernel scope link src 172.16.0.1
> 192.168.22.0/24 dev vlan4081 proto kernel scope link src 192.168.22.205
>
> neigh table:
> ip neigh ls
>
> 192.168.22.237 dev vlan4081 lladdr 00:25:90:fb:a6:8d REACHABLE
> 172.16.0.2 dev vlan1740 lladdr ac:1f:6b:2c:2e:5a REACHABLE
>
> and interfaces:
> 4: enp175s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
> UP mode DEFAULT group default qlen 1000
> link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff
> 5: enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
> UP mode DEFAULT group default qlen 1000
> link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff
> 6: vlan4081@enp175s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP mode DEFAULT group default qlen 1000
> link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff
> 7: vlan1740@enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP mode DEFAULT group default qlen 1000
> link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff
>
> 5: enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp/id:5 qdisc
> mq state UP group default qlen 1000
> link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff
> inet6 fe80::ae1f:6bff:fe07:c891/64 scope link
> valid_lft forever preferred_lft forever
> 6: vlan4081@enp175s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP group default qlen 1000
> link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff
> inet 192.168.22.205/24 scope global vlan4081
> valid_lft forever preferred_lft forever
> inet6 fe80::ae1f:6bff:fe07:c890/64 scope link
> valid_lft forever preferred_lft forever
> 7: vlan1740@enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP group default qlen 1000
> link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff
> inet 172.16.0.1/30 scope global vlan1740
> valid_lft forever preferred_lft forever
> inet6 fe80::ae1f:6bff:fe07:c891/64 scope link
> valid_lft forever preferred_lft forever
>
>
> xdp program detached:
> Receiving side tcpdump:
> 14:28:09.141233 IP 192.168.22.237 > 172.16.0.2: ICMP echo request, id
> 30227, seq 487, length 64
>
> I can see icmp requests
>
>
> enabling xdp
> ./xdp_fwd enp175s0f1 enp175s0f0
>
> 4: enp175s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp qdisc mq
> state UP mode DEFAULT group default qlen 1000
> link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff
> prog/xdp id 5 tag 3c231ff1e5e77f3f
> 5: enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp qdisc mq
> state UP mode DEFAULT group default qlen 1000
> link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff
> prog/xdp id 5 tag 3c231ff1e5e77f3f
> 6: vlan4081@enp175s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP mode DEFAULT group default qlen 1000
> link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff
> 7: vlan1740@enp175s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP mode DEFAULT group default qlen 1000
> link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff
>
What hardware is this?
Start with:
echo 1 > /sys/kernel/debug/tracing/events/xdp/enable
cat /sys/kernel/debug/tracing/trace_pipe
>From there, you can check the FIB lookups:
sysctl -w kernel.perf_event_max_stack=16
perf record -e fib:* -a -g -- sleep 5
perf script
^ permalink raw reply
* Re: [PATCH net-next 0/7] net: sched: prepare for more Qdisc offloads
From: Jakub Kicinski @ 2018-11-08 16:02 UTC (permalink / raw)
To: Toke Høiland-Jørgensen
Cc: davem, netdev, oss-drivers, jiri, xiyou.wangcong, jhs,
nogah.frankel
In-Reply-To: <877ehnvlh0.fsf@toke.dk>
On Thu, 08 Nov 2018 12:48:27 +0100, Toke Høiland-Jørgensen wrote:
> Jakub Kicinski <jakub.kicinski@netronome.com> writes:
> > Hi!
> >
> > This series refactors the "switchdev" Qdisc offloads a little. We have
> > a few Qdiscs which can be fully offloaded today to the forwarding plane
> > of switching devices.
> >
> > First patch adds a helper for handing statistic dumps, the code seems
> > to be copy pasted between PRIO and RED.
>
> Hi Jakub
>
> I didn't know there was offload capabilities for AQMs, that's cool! Do
> you have any plans to add offloads for any of the modern AQMs (CoDel or
> PIE)?
I'd really like to add CoDel offload, but it's not a plan at this
point :(
^ permalink raw reply
* Re: [PATCH][net-hns3-next] net: hns3: fix spelling mistake, "assertting" -> "asserting"
From: David Miller @ 2018-11-09 1:08 UTC (permalink / raw)
To: colin.king
Cc: yisen.zhuang, salil.mehta, lipeng321, netdev, kernel-janitors,
linux-kernel
In-Reply-To: <20181108103202.1212-1-colin.king@canonical.com>
From: Colin King <colin.king@canonical.com>
Date: Thu, 8 Nov 2018 10:32:02 +0000
> From: Colin Ian King <colin.king@canonical.com>
>
> Trivial fix to spelling mistake in dev_err error message
>
> Signed-off-by: Colin Ian King <colin.king@canonical.com>
Applied.
^ permalink raw reply
* [net-next, PATCH 2/2] net: socionext: refactor netsec_alloc_dring()
From: Ilias Apalodimas @ 2018-11-08 15:19 UTC (permalink / raw)
To: netdev, jaswinder.singh, ard.biesheuvel
Cc: masami.hiramatsu, masahisa.kojima, brouer, arnd, Ilias Apalodimas
In-Reply-To: <1541690395-8337-1-git-send-email-ilias.apalodimas@linaro.org>
return -ENOMEM directly instead of assigning it in a variable
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
---
drivers/net/ethernet/socionext/netsec.c | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c
index 9d24f29..bba9733 100644
--- a/drivers/net/ethernet/socionext/netsec.c
+++ b/drivers/net/ethernet/socionext/netsec.c
@@ -996,26 +996,21 @@ static void netsec_free_dring(struct netsec_priv *priv, int id)
static int netsec_alloc_dring(struct netsec_priv *priv, enum ring_id id)
{
struct netsec_desc_ring *dring = &priv->desc_ring[id];
- int ret = 0;
dring->vaddr = dma_zalloc_coherent(priv->dev, DESC_SZ * DESC_NUM,
&dring->desc_dma, GFP_KERNEL);
- if (!dring->vaddr) {
- ret = -ENOMEM;
+ if (!dring->vaddr)
goto err;
- }
dring->desc = kcalloc(DESC_NUM, sizeof(*dring->desc), GFP_KERNEL);
- if (!dring->desc) {
- ret = -ENOMEM;
+ if (!dring->desc)
goto err;
- }
return 0;
err:
netsec_free_dring(priv, id);
- return ret;
+ return -ENOMEM;
}
static int netsec_setup_rx_dring(struct netsec_priv *priv)
--
2.7.4
^ permalink raw reply related
* [net-next, PATCH 1/2] net: socionext: different approach on DMA
From: Ilias Apalodimas @ 2018-11-08 15:19 UTC (permalink / raw)
To: netdev, jaswinder.singh, ard.biesheuvel
Cc: masami.hiramatsu, masahisa.kojima, brouer, arnd, Ilias Apalodimas
Current driver dynamically allocates an skb and maps it as DMA Rx
buffer. In order to prepare for upcoming XDP changes, let's introduce a
different allocation scheme.
Buffers are allocated dynamically and mapped into hardware.
During the Rx operation the driver uses build_skb() to produce the
necessary buffers for the network stack.
This change increases performance ~15% on 64b packets with smmu disabled
and ~5% with smmu enabled
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
---
drivers/net/ethernet/socionext/netsec.c | 227 ++++++++++++++++++--------------
1 file changed, 128 insertions(+), 99 deletions(-)
diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c
index d9d0d03..9d24f29 100644
--- a/drivers/net/ethernet/socionext/netsec.c
+++ b/drivers/net/ethernet/socionext/netsec.c
@@ -234,6 +234,9 @@
#define DESC_NUM 256
+#define NETSEC_SKB_PAD (NET_SKB_PAD + NET_IP_ALIGN)
+#define NETSEC_RX_BUF_SZ 1536
+
#define DESC_SZ sizeof(struct netsec_de)
#define NETSEC_F_NETSEC_VER_MAJOR_NUM(x) ((x) & 0xffff0000)
@@ -571,34 +574,10 @@ static const struct ethtool_ops netsec_ethtool_ops = {
/************* NETDEV_OPS FOLLOW *************/
-static struct sk_buff *netsec_alloc_skb(struct netsec_priv *priv,
- struct netsec_desc *desc)
-{
- struct sk_buff *skb;
-
- if (device_get_dma_attr(priv->dev) == DEV_DMA_COHERENT) {
- skb = netdev_alloc_skb_ip_align(priv->ndev, desc->len);
- } else {
- desc->len = L1_CACHE_ALIGN(desc->len);
- skb = netdev_alloc_skb(priv->ndev, desc->len);
- }
- if (!skb)
- return NULL;
-
- desc->addr = skb->data;
- desc->dma_addr = dma_map_single(priv->dev, desc->addr, desc->len,
- DMA_FROM_DEVICE);
- if (dma_mapping_error(priv->dev, desc->dma_addr)) {
- dev_kfree_skb_any(skb);
- return NULL;
- }
- return skb;
-}
static void netsec_set_rx_de(struct netsec_priv *priv,
struct netsec_desc_ring *dring, u16 idx,
- const struct netsec_desc *desc,
- struct sk_buff *skb)
+ const struct netsec_desc *desc)
{
struct netsec_de *de = dring->vaddr + DESC_SZ * idx;
u32 attr = (1 << NETSEC_RX_PKT_OWN_FIELD) |
@@ -617,59 +596,6 @@ static void netsec_set_rx_de(struct netsec_priv *priv,
dring->desc[idx].dma_addr = desc->dma_addr;
dring->desc[idx].addr = desc->addr;
dring->desc[idx].len = desc->len;
- dring->desc[idx].skb = skb;
-}
-
-static struct sk_buff *netsec_get_rx_de(struct netsec_priv *priv,
- struct netsec_desc_ring *dring,
- u16 idx,
- struct netsec_rx_pkt_info *rxpi,
- struct netsec_desc *desc, u16 *len)
-{
- struct netsec_de de = {};
-
- memcpy(&de, dring->vaddr + DESC_SZ * idx, DESC_SZ);
-
- *len = de.buf_len_info >> 16;
-
- rxpi->err_flag = (de.attr >> NETSEC_RX_PKT_ER_FIELD) & 1;
- rxpi->rx_cksum_result = (de.attr >> NETSEC_RX_PKT_CO_FIELD) & 3;
- rxpi->err_code = (de.attr >> NETSEC_RX_PKT_ERR_FIELD) &
- NETSEC_RX_PKT_ERR_MASK;
- *desc = dring->desc[idx];
- return desc->skb;
-}
-
-static struct sk_buff *netsec_get_rx_pkt_data(struct netsec_priv *priv,
- struct netsec_rx_pkt_info *rxpi,
- struct netsec_desc *desc,
- u16 *len)
-{
- struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX];
- struct sk_buff *tmp_skb, *skb = NULL;
- struct netsec_desc td;
- int tail;
-
- *rxpi = (struct netsec_rx_pkt_info){};
-
- td.len = priv->ndev->mtu + 22;
-
- tmp_skb = netsec_alloc_skb(priv, &td);
-
- tail = dring->tail;
-
- if (!tmp_skb) {
- netsec_set_rx_de(priv, dring, tail, &dring->desc[tail],
- dring->desc[tail].skb);
- } else {
- skb = netsec_get_rx_de(priv, dring, tail, rxpi, desc, len);
- netsec_set_rx_de(priv, dring, tail, &td, tmp_skb);
- }
-
- /* move tail ahead */
- dring->tail = (dring->tail + 1) % DESC_NUM;
-
- return skb;
}
static int netsec_clean_tx_dring(struct netsec_priv *priv, int budget)
@@ -736,19 +662,65 @@ static int netsec_process_tx(struct netsec_priv *priv, int budget)
return done;
}
+static void *netsec_alloc_rx_data(struct netsec_priv *priv,
+ dma_addr_t *dma_handle, u16 *desc_len)
+{
+ size_t total_len = SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+ size_t payload_len = NETSEC_RX_BUF_SZ;
+ dma_addr_t mapping;
+ void *buf;
+
+ total_len += SKB_DATA_ALIGN(payload_len + NETSEC_SKB_PAD);
+
+ buf = napi_alloc_frag(total_len);
+ if (!buf)
+ return NULL;
+
+ mapping = dma_map_single(priv->dev, buf + NETSEC_SKB_PAD, payload_len,
+ DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(priv->dev, mapping)))
+ goto err_out;
+
+ *dma_handle = mapping;
+ *desc_len = payload_len;
+
+ return buf;
+
+err_out:
+ skb_free_frag(buf);
+ return NULL;
+}
+
+static void netsec_rx_fill(struct netsec_priv *priv, u16 from, u16 num)
+{
+ struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX];
+ u16 idx = from;
+
+ while (num) {
+ netsec_set_rx_de(priv, dring, idx, &dring->desc[idx]);
+ idx++;
+ if (idx >= DESC_NUM)
+ idx = 0;
+ num--;
+ }
+}
+
static int netsec_process_rx(struct netsec_priv *priv, int budget)
{
struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX];
struct net_device *ndev = priv->ndev;
struct netsec_rx_pkt_info rx_info;
- int done = 0;
- struct netsec_desc desc;
struct sk_buff *skb;
- u16 len;
+ int done = 0;
while (done < budget) {
u16 idx = dring->tail;
struct netsec_de *de = dring->vaddr + (DESC_SZ * idx);
+ struct netsec_desc *desc = &dring->desc[idx];
+ u16 pkt_len, desc_len;
+ dma_addr_t dma_handle;
+ void *buf_addr;
+ u32 truesize;
if (de->attr & (1U << NETSEC_RX_PKT_OWN_FIELD)) {
/* reading the register clears the irq */
@@ -762,18 +734,59 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget)
*/
dma_rmb();
done++;
- skb = netsec_get_rx_pkt_data(priv, &rx_info, &desc, &len);
- if (unlikely(!skb) || rx_info.err_flag) {
+
+ pkt_len = de->buf_len_info >> 16;
+ rx_info.err_code = (de->attr >> NETSEC_RX_PKT_ERR_FIELD) &
+ NETSEC_RX_PKT_ERR_MASK;
+ rx_info.err_flag = (de->attr >> NETSEC_RX_PKT_ER_FIELD) & 1;
+ if (rx_info.err_flag) {
netif_err(priv, drv, priv->ndev,
- "%s: rx fail err(%d)\n",
- __func__, rx_info.err_code);
+ "%s: rx fail err(%d)\n", __func__,
+ rx_info.err_code);
ndev->stats.rx_dropped++;
+ dring->tail = (dring->tail + 1) % DESC_NUM;
+ /* reuse buffer page frag */
+ netsec_rx_fill(priv, idx, 1);
continue;
}
+ rx_info.rx_cksum_result =
+ (de->attr >> NETSEC_RX_PKT_CO_FIELD) & 3;
- dma_unmap_single(priv->dev, desc.dma_addr, desc.len,
- DMA_FROM_DEVICE);
- skb_put(skb, len);
+ /* allocate a fresh buffer and map it to the hardware.
+ * This will eventually replace the old buffer in the hardware
+ */
+ buf_addr = netsec_alloc_rx_data(priv, &dma_handle, &desc_len);
+ if (unlikely(!buf_addr))
+ break;
+
+ dma_sync_single_for_cpu(priv->dev, desc->dma_addr, pkt_len,
+ DMA_FROM_DEVICE);
+ prefetch(desc->addr);
+
+ truesize = SKB_DATA_ALIGN(desc->len + NETSEC_SKB_PAD) +
+ SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+ skb = build_skb(desc->addr, truesize);
+ if (unlikely(!skb)) {
+ /* free the newly allocated buffer, we are not going to
+ * use it
+ */
+ dma_unmap_single(priv->dev, dma_handle, desc_len,
+ DMA_FROM_DEVICE);
+ skb_free_frag(buf_addr);
+ netif_err(priv, drv, priv->ndev,
+ "rx failed to build skb\n");
+ break;
+ }
+ dma_unmap_single_attrs(priv->dev, desc->dma_addr, desc->len,
+ DMA_FROM_DEVICE, DMA_ATTR_SKIP_CPU_SYNC);
+
+ /* Update the descriptor with the new buffer we allocated */
+ desc->len = desc_len;
+ desc->dma_addr = dma_handle;
+ desc->addr = buf_addr;
+
+ skb_reserve(skb, NETSEC_SKB_PAD);
+ skb_put(skb, pkt_len);
skb->protocol = eth_type_trans(skb, priv->ndev);
if (priv->rx_cksum_offload_flag &&
@@ -782,8 +795,11 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget)
if (napi_gro_receive(&priv->napi, skb) != GRO_DROP) {
ndev->stats.rx_packets++;
- ndev->stats.rx_bytes += len;
+ ndev->stats.rx_bytes += pkt_len;
}
+
+ netsec_rx_fill(priv, idx, 1);
+ dring->tail = (dring->tail + 1) % DESC_NUM;
}
return done;
@@ -946,7 +962,10 @@ static void netsec_uninit_pkt_dring(struct netsec_priv *priv, int id)
dma_unmap_single(priv->dev, desc->dma_addr, desc->len,
id == NETSEC_RING_RX ? DMA_FROM_DEVICE :
DMA_TO_DEVICE);
- dev_kfree_skb(desc->skb);
+ if (id == NETSEC_RING_RX)
+ skb_free_frag(desc->addr);
+ else if (id == NETSEC_RING_TX)
+ dev_kfree_skb(desc->skb);
}
memset(dring->desc, 0, sizeof(struct netsec_desc) * DESC_NUM);
@@ -1002,22 +1021,30 @@ static int netsec_alloc_dring(struct netsec_priv *priv, enum ring_id id)
static int netsec_setup_rx_dring(struct netsec_priv *priv)
{
struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_RX];
- struct netsec_desc desc;
- struct sk_buff *skb;
- int n;
+ int i;
- desc.len = priv->ndev->mtu + 22;
+ for (i = 0; i < DESC_NUM; i++) {
+ struct netsec_desc *desc = &dring->desc[i];
+ dma_addr_t dma_handle;
+ void *buf;
+ u16 len;
- for (n = 0; n < DESC_NUM; n++) {
- skb = netsec_alloc_skb(priv, &desc);
- if (!skb) {
+ buf = netsec_alloc_rx_data(priv, &dma_handle, &len);
+ if (!buf) {
netsec_uninit_pkt_dring(priv, NETSEC_RING_RX);
- return -ENOMEM;
+ goto err_out;
}
- netsec_set_rx_de(priv, dring, n, &desc, skb);
+ desc->dma_addr = dma_handle;
+ desc->addr = buf;
+ desc->len = len;
}
+ netsec_rx_fill(priv, 0, DESC_NUM);
+
return 0;
+
+err_out:
+ return -ENOMEM;
}
static int netsec_netdev_load_ucode_region(struct netsec_priv *priv, u32 reg,
@@ -1377,6 +1404,8 @@ static int netsec_netdev_init(struct net_device *ndev)
int ret;
u16 data;
+ BUILD_BUG_ON_NOT_POWER_OF_2(DESC_NUM);
+
ret = netsec_alloc_dring(priv, NETSEC_RING_TX);
if (ret)
return ret;
--
2.7.4
^ permalink raw reply related
* Re: [PATCH v3 net-next] sock: Reset dst when changing sk_mark via setsockopt
From: Eric Dumazet @ 2018-11-08 15:16 UTC (permalink / raw)
To: David Barmann, netdev
In-Reply-To: <20181108141335.GA29455@konacove.com>
On 11/08/2018 06:13 AM, David Barmann wrote:
> When setting the SO_MARK socket option, if the mark changes, the dst
> needs to be reset so that a new route lookup is performed.
>
> This fixes the case where an application wants to change routing by
> setting a new sk_mark. If this is done after some packets have already
> been sent, the dst is cached and has no effect.
>
> Signed-off-by: David Barmann <david.barmann@stackpath.com>
> ---
Reviewed-by: Eric Dumazet <edumazet@google.com>
Thanks David !
^ permalink raw reply
* Re: SACK compression patch causing performance drop
From: Eric Dumazet @ 2018-11-08 15:14 UTC (permalink / raw)
To: Jean-Louis Dupond, netdev, edumazet
In-Reply-To: <ce731718-3f9d-04df-d6d2-a300d3b4fc29@dupond.be>
On 11/08/2018 12:23 AM, Jean-Louis Dupond wrote:
> Hi,
>
> Was somebody able to check this?
> Really think this should be fixed :)
>
> Thanks
> Jean-Louis
>
I somehow missed this email.
Packet captures might help, please send me
tcpdump -s 128 -i ethX -w sack.pcap
of some samples with or without the sack compression enabled.
> On 3/11/18 16:59, Jean-Louis Dupond wrote:
>> Hi All,
>>
>> On recent kernels we noticed a way lower throughput to our SAN system than before.
>> While on pre 4.18 kernels we had 400-700MB/sec read speed, on 4.18+ we only had 70-120MB/sec.
>>
>> The SAN is connected via iSCSI over a 10G network (ixgbe/X520 NICS if it matters).
>>
>> After some debugging, I tried to bisect between 4.17 and 4.18 to see what commit caused the slowdown.
>> It showed that the addition of the SACK compression (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5d9f4262b7ea41ca9981cc790e37cca6e37c789e) was the cause.
>>
>> And indeed, if I set net.ipv4.tcp_comp_sack_nr to 0 on 4.19 for example, the throughput is (almost) back to normal again.
>> So it seems like this change causes quite some performance issues.
>>
>> Any ideas?
>>
>> Thanks
>> Jean-Louis
>>
^ permalink raw reply
* Re: (2) (2) (2) [Kernel][NET] Bug report on packet defragmenting
From: Eric Dumazet @ 2018-11-08 15:12 UTC (permalink / raw)
To: soukjin.bae, Eric Dumazet, netdev@vger.kernel.org
In-Reply-To: <20181108075837epcms1p2747d212aee83ba0df60cc14ffac316aa@epcms1p2>
On 11/07/2018 11:58 PM, 배석진 wrote:
>> --------- Original Message ---------
>> Sender : Eric Dumazet <eric.dumazet@gmail.com>
>> Date : 2018-11-08 15:13 (GMT+9)
>> Title : Re: (2) (2) [Kernel][NET] Bug report on packet defragmenting
>>
>> On 11/07/2018 08:26 PM, Eric Dumazet wrote:
>>>
>>>
>>> On 11/07/2018 08:10 PM, 배석진 wrote:
>>>>> --------- Original Message ---------
>>>>> Sender : Eric Dumazet <eric.dumazet@gmail.com>
>>>>> Date : 2018-11-08 12:57 (GMT+9)
>>>>> Title : Re: (2) [Kernel][NET] Bug report on packet defragmenting
>>>>>
>>>>> On 11/07/2018 07:24 PM, Eric Dumazet wrote:
>>>>>
>>>>>> Sure, it is better if RPS is smarter, but if there is a bug in IPv6 defrag unit
>>>>>> we must investigate and root-cause it.
>>>>>
>>>>> BTW, IPv4 defrag seems to have the same issue.
>>>>
>>>>
>>>> yes, it could be.
>>>> key point isn't limitted to ipv6.
>>>>
>>>> maybe because of faster air-network and modem,
>>>> it looks like occure more often and we got recognized that.
>>>>
>>>> anyway,
>>>> we'll apply our patch to resolve this problem.
>>>
>>> Yeah, and I will fix the defrag units.
>>>
>>> We can not rely on other layers doing proper no-reorder logic for us.
>>>
>>> Problem here is that multiple cpus attempt concurrent rhashtable_insert_fast()
>>> and do not properly recover in case -EEXIST is returned.
>>>
>>> This is silly, of course :/
>>
>> Patch would be https://patchwork.ozlabs.org/patch/994658/
>
>
> Dear Dumazet,
>
> with your patch, kernel got the panic when packet recieved.
> I double checked after disable your patch, then no problem.
>
>
> <6>[ 119.702054] I[3: kworker/u18:1: 1705] LNK-RX(1464): 6b 80 00 00 05 90 2c 3e 20 01 44 30 00 05 04 01 ...
> <6>[ 119.702120] I[3: kworker/u18:1: 1705] __skb_flow_dissect: ports: 77500000
> <6>[ 119.702153] I[3: kworker/u18:1: 1705] get_rps_cpu: cpu:2, hash:2055028308
> <6>[ 119.702203] I[3: kworker/u18:1: 1705] LNK-RX(1212): 6b 80 00 00 04 94 2c 3e 20 01 44 30 00 05 04 01 ...
> <6>[ 119.702231] I[3: kworker/u18:1: 1705] __skb_flow_dissect: ports: 3c7e2c6b
> <6>[ 119.702258] I[3: kworker/u18:1: 1705] get_rps_cpu: cpu:1, hash:671343869
> <6>[ 119.702365] I[1: Binder:11369_2:11382] ipv6_rcv +++
> <6>[ 119.702375] I[2: swapper/2: 0] ipv6_rcv +++
> <6>[ 119.702406] I[2: swapper/2: 0] ipv6_defrag +++
> <6>[ 119.702425] I[1: Binder:11369_2:11382] ipv6_defrag +++
> <6>[ 119.702494] I[2: swapper/2: 0] ipv6_defrag: EINPROGRESS
> <6>[ 119.702522] I[2: swapper/2: 0] ipv6_rcv ---
> <6>[ 119.702628] I[1: Binder:11369_2:11382] ipv6_defrag ---
> <6>[ 119.702892] I[1: Binder:11369_2:11382] ipv6_defrag +++
> <6>[ 119.702922] I[1: Binder:11369_2:11382] ipv6_defrag ---
> <6>[ 119.702966] I[1: Binder:11369_2:11382] ipv6_rcv ---
> <0>[ 119.703792] [1: Binder:11369_2:11382] BUG: sleeping function called from invalid context at arch/arm64/mm/fault.c:518
> <3>[ 119.703826] [1: Binder:11369_2:11382] in_atomic(): 0, irqs_disabled(): 0, pid: 11382, name: Binder:11369_2
> <3>[ 119.703854] [1: Binder:11369_2:11382] Preemption disabled at:
> <4>[ 119.703888] [1: Binder:11369_2:11382] [<ffffff80080b13d4>] __do_softirq+0x68/0x3c4
> <4>[ 119.703934] [1: Binder:11369_2:11382] CPU: 1 PID: 11382 Comm: Binder:11369_2 Tainted: G S W 4.14.75-20181108-163447-eng #0
> <4>[ 119.703960] [1: Binder:11369_2:11382] Hardware name: Samsung BEYOND2LTE KOR SINGLE 19 board based on EXYNOS9820 (DT)
> <4>[ 119.703987] [1: Binder:11369_2:11382] Call trace:
> <4>[ 119.704015] [1: Binder:11369_2:11382] [<ffffff80080bd87c>] dump_backtrace+0x0/0x280
> <4>[ 119.704045] [1: Binder:11369_2:11382] [<ffffff80080bddd4>] show_stack+0x18/0x24
> <4>[ 119.704074] [1: Binder:11369_2:11382] [<ffffff80090bb3f8>] dump_stack+0xb8/0xf8
> <4>[ 119.704104] [1: Binder:11369_2:11382] [<ffffff800811f180>] ___might_sleep+0x16c/0x178
> <4>[ 119.704132] [1: Binder:11369_2:11382] [<ffffff800811efdc>] __might_sleep+0x4c/0x84
> <4>[ 119.704164] [1: Binder:11369_2:11382] [<ffffff80090dcf60>] do_page_fault+0x2e8/0x4b8
> <4>[ 119.704193] [1: Binder:11369_2:11382] [<ffffff80090dcbf4>] do_translation_fault+0x7c/0x100
> <4>[ 119.704219] [1: Binder:11369_2:11382] [<ffffff80080b0d70>] do_mem_abort+0x4c/0x12c
> <4>[ 119.704243] [1: Binder:11369_2:11382] Exception stack(0xffffff8038bf3ec0 to 0xffffff8038bf4000)
> <4>[ 119.704266] [1: Binder:11369_2:11382] 3ec0: 00000077b8262600 00000077b1bd0800 00000000708fcae0 0000000000000018
> ...
> <4>[ 119.704459] [1: Binder:11369_2:11382] 3fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> <4>[ 119.704480] [1: Binder:11369_2:11382] [<ffffff80080b33d0>] el0_da+0x20/0x24
> <4>[ 119.704509] [1: Binder:11369_2:11382] ------------[ cut here ]------------
> <0>[ 119.704541] [1: Binder:11369_2:11382] kernel BUG at kernel/sched/core.c:6152!
> <2>[ 119.704563] [1: Binder:11369_2:11382] sec_debug_set_extra_info_fault = BUG / 0xffffff800811f180
> <0>[ 119.704603] [1: Binder:11369_2:11382] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>
>
Thanks for testing.
This is not a pristine net-next tree, this dump seems unrelated to the patch ?
^ permalink raw reply
* [PATCH bpf-next] filter: add BPF_ADJ_ROOM_DATA mode to bpf_skb_adjust_room()
From: Nicolas Dichtel @ 2018-11-08 15:11 UTC (permalink / raw)
To: ast, daniel, davem; +Cc: netdev, Nicolas Dichtel
This new mode enables to add or remove an l2 header in a programmatic way
with cls_bpf.
For example, it enables to play with mpls headers.
Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
---
include/uapi/linux/bpf.h | 3 ++
net/core/filter.c | 54 ++++++++++++++++++++++++++++++++++
tools/include/uapi/linux/bpf.h | 3 ++
3 files changed, 60 insertions(+)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 852dc17ab47a..47407fd5162b 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1467,6 +1467,8 @@ union bpf_attr {
*
* * **BPF_ADJ_ROOM_NET**: Adjust room at the network layer
* (room space is added or removed below the layer 3 header).
+ * * **BPF_ADJ_ROOM_DATA**: Adjust room at the beginning of the
+ * packet (room space is added or removed below skb->data).
*
* All values for *flags* are reserved for future usage, and must
* be left at zero.
@@ -2408,6 +2410,7 @@ enum bpf_func_id {
/* Mode for BPF_FUNC_skb_adjust_room helper. */
enum bpf_adj_room_mode {
BPF_ADJ_ROOM_NET,
+ BPF_ADJ_ROOM_DATA,
};
/* Mode for BPF_FUNC_skb_load_bytes_relative helper. */
diff --git a/net/core/filter.c b/net/core/filter.c
index e521c5ebc7d1..e699849b269d 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2884,6 +2884,58 @@ static int bpf_skb_adjust_net(struct sk_buff *skb, s32 len_diff)
return ret;
}
+static int bpf_skb_data_shrink(struct sk_buff *skb, u32 len)
+{
+ unsigned short hhlen = skb->dev->header_ops ?
+ skb->dev->hard_header_len : 0;
+ int ret;
+
+ ret = skb_unclone(skb, GFP_ATOMIC);
+ if (unlikely(ret < 0))
+ return ret;
+
+ __skb_pull(skb, len);
+ skb_reset_mac_header(skb);
+ skb_reset_network_header(skb);
+ skb->network_header += hhlen;
+ skb_reset_transport_header(skb);
+ return 0;
+}
+
+static int bpf_skb_data_grow(struct sk_buff *skb, u32 len)
+{
+ unsigned short hhlen = skb->dev->header_ops ?
+ skb->dev->hard_header_len : 0;
+ int ret;
+
+ ret = skb_cow(skb, len);
+ if (unlikely(ret < 0))
+ return ret;
+
+ skb_push(skb, len);
+ skb_reset_mac_header(skb);
+ return 0;
+}
+
+static int bpf_skb_adjust_data(struct sk_buff *skb, s32 len_diff)
+{
+ u32 len_diff_abs = abs(len_diff);
+ bool shrink = len_diff < 0;
+ int ret;
+
+ if (unlikely(len_diff_abs > 0xfffU))
+ return -EFAULT;
+
+ if (shrink && len_diff_abs >= skb_headlen(skb))
+ return -EFAULT;
+
+ ret = shrink ? bpf_skb_data_shrink(skb, len_diff_abs) :
+ bpf_skb_data_grow(skb, len_diff_abs);
+
+ bpf_compute_data_pointers(skb);
+ return ret;
+}
+
BPF_CALL_4(bpf_skb_adjust_room, struct sk_buff *, skb, s32, len_diff,
u32, mode, u64, flags)
{
@@ -2891,6 +2943,8 @@ BPF_CALL_4(bpf_skb_adjust_room, struct sk_buff *, skb, s32, len_diff,
return -EINVAL;
if (likely(mode == BPF_ADJ_ROOM_NET))
return bpf_skb_adjust_net(skb, len_diff);
+ if (likely(mode == BPF_ADJ_ROOM_DATA))
+ return bpf_skb_adjust_data(skb, len_diff);
return -ENOTSUPP;
}
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 852dc17ab47a..47407fd5162b 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1467,6 +1467,8 @@ union bpf_attr {
*
* * **BPF_ADJ_ROOM_NET**: Adjust room at the network layer
* (room space is added or removed below the layer 3 header).
+ * * **BPF_ADJ_ROOM_DATA**: Adjust room at the beginning of the
+ * packet (room space is added or removed below skb->data).
*
* All values for *flags* are reserved for future usage, and must
* be left at zero.
@@ -2408,6 +2410,7 @@ enum bpf_func_id {
/* Mode for BPF_FUNC_skb_adjust_room helper. */
enum bpf_adj_room_mode {
BPF_ADJ_ROOM_NET,
+ BPF_ADJ_ROOM_DATA,
};
/* Mode for BPF_FUNC_skb_load_bytes_relative helper. */
--
2.18.0
^ permalink raw reply related
* Re: drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.c:96: undefined reference to `thermal_zone_device_register'
From: Randy Dunlap @ 2018-11-09 0:46 UTC (permalink / raw)
To: kbuild test robot, Arnd Bergmann
Cc: kbuild-all, linux-kernel, Ganesh Goudar, David Miller,
netdev@vger.kernel.org
In-Reply-To: <201811081531.m0SEgZ58%fengguang.wu@intel.com>
[adding netdev + maintainers]
On 11/7/18 11:08 PM, kbuild test robot wrote:
> tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
> head: 85758777c2a227fd1541b6dd122a08ab79c347ce
> commit: e70a57fa59bb7fefe063780a49e063d0d0f61863 cxgb4: fix thermal configuration dependencies
> date: 4 weeks ago
> config: x86_64-randconfig-s0-11081213 (attached as .config)
> compiler: gcc-6 (Debian 6.4.0-9) 6.4.0 20171026
> reproduce:
> git checkout e70a57fa59bb7fefe063780a49e063d0d0f61863
> # save the attached .config to linux build tree
> make ARCH=x86_64
>
> All errors (new ones prefixed by >>):
>
> drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.o: In function `cxgb4_thermal_init':
>>> drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.c:96: undefined reference to `thermal_zone_device_register'
> drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.o: In function `cxgb4_thermal_remove':
>>> drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.c:112: undefined reference to `thermal_zone_device_unregister'
>
Hi,
There was a Kconfig WARNING (clue!) before the build errors:
WARNING: unmet direct dependencies detected for CHELSIO_T4
Depends on [m]: NETDEVICES [=y] && ETHERNET [=y] && NET_VENDOR_CHELSIO [=y] && PCI [=y] && (IPV6 [=n] || IPV6 [=n]=n) && (THERMAL [=m] || !THERMAL [=m])
Selected by [y]:
- SCSI_CXGB4_ISCSI [=y] && SCSI_LOWLEVEL [=y] && SCSI [=y] && PCI [=y] && INET [=y] && (IPV6 [=n] || IPV6 [=n]=n)
That says that the problem is with CONFIG_THERMAL=m while CONFIG_CHELSIO_T4=y.
This is caused by drivers/scsi/cxgbi/cxgb4i/Kconfig and its use of "select"
instead of "depends on":
config SCSI_CXGB4_ISCSI
tristate "Chelsio T4 iSCSI support"
depends on PCI && INET && (IPV6 || IPV6=n)
select NETDEVICES
select ETHERNET
select NET_VENDOR_CHELSIO
select CHELSIO_T4
select CHELSIO_LIB
select SCSI_ISCSI_ATTRS
---help---
This driver supports iSCSI offload for the Chelsio T4 devices.
> vim +96 drivers/net/ethernet/chelsio/cxgb4/cxgb4_thermal.c
>
> b1871915 Ganesh Goudar 2018-10-09 72
> b1871915 Ganesh Goudar 2018-10-09 73 int cxgb4_thermal_init(struct adapter *adap)
> b1871915 Ganesh Goudar 2018-10-09 74 {
> b1871915 Ganesh Goudar 2018-10-09 75 struct ch_thermal *ch_thermal = &adap->ch_thermal;
> b1871915 Ganesh Goudar 2018-10-09 76 int num_trip = CXGB4_NUM_TRIPS;
> b1871915 Ganesh Goudar 2018-10-09 77 u32 param, val;
> b1871915 Ganesh Goudar 2018-10-09 78 int ret;
> b1871915 Ganesh Goudar 2018-10-09 79
> b1871915 Ganesh Goudar 2018-10-09 80 /* on older firmwares we may not get the trip temperature,
> b1871915 Ganesh Goudar 2018-10-09 81 * set the num of trips to 0.
> b1871915 Ganesh Goudar 2018-10-09 82 */
> b1871915 Ganesh Goudar 2018-10-09 83 param = (FW_PARAMS_MNEM_V(FW_PARAMS_MNEM_DEV) |
> b1871915 Ganesh Goudar 2018-10-09 84 FW_PARAMS_PARAM_X_V(FW_PARAMS_PARAM_DEV_DIAG) |
> b1871915 Ganesh Goudar 2018-10-09 85 FW_PARAMS_PARAM_Y_V(FW_PARAM_DEV_DIAG_MAXTMPTHRESH));
> b1871915 Ganesh Goudar 2018-10-09 86
> b1871915 Ganesh Goudar 2018-10-09 87 ret = t4_query_params(adap, adap->mbox, adap->pf, 0, 1,
> b1871915 Ganesh Goudar 2018-10-09 88 ¶m, &val);
> b1871915 Ganesh Goudar 2018-10-09 89 if (ret < 0) {
> b1871915 Ganesh Goudar 2018-10-09 90 num_trip = 0; /* could not get trip temperature */
> b1871915 Ganesh Goudar 2018-10-09 91 } else {
> b1871915 Ganesh Goudar 2018-10-09 92 ch_thermal->trip_temp = val * 1000;
> b1871915 Ganesh Goudar 2018-10-09 93 ch_thermal->trip_type = THERMAL_TRIP_CRITICAL;
> b1871915 Ganesh Goudar 2018-10-09 94 }
> b1871915 Ganesh Goudar 2018-10-09 95
> b1871915 Ganesh Goudar 2018-10-09 @96 ch_thermal->tzdev = thermal_zone_device_register("cxgb4", num_trip,
> b1871915 Ganesh Goudar 2018-10-09 97 0, adap,
> b1871915 Ganesh Goudar 2018-10-09 98 &cxgb4_thermal_ops,
> b1871915 Ganesh Goudar 2018-10-09 99 NULL, 0, 0);
> b1871915 Ganesh Goudar 2018-10-09 100 if (IS_ERR(ch_thermal->tzdev)) {
> b1871915 Ganesh Goudar 2018-10-09 101 ret = PTR_ERR(ch_thermal->tzdev);
> b1871915 Ganesh Goudar 2018-10-09 102 dev_err(adap->pdev_dev, "Failed to register thermal zone\n");
> b1871915 Ganesh Goudar 2018-10-09 103 ch_thermal->tzdev = NULL;
> b1871915 Ganesh Goudar 2018-10-09 104 return ret;
> b1871915 Ganesh Goudar 2018-10-09 105 }
> b1871915 Ganesh Goudar 2018-10-09 106 return 0;
> b1871915 Ganesh Goudar 2018-10-09 107 }
> b1871915 Ganesh Goudar 2018-10-09 108
> b1871915 Ganesh Goudar 2018-10-09 109 int cxgb4_thermal_remove(struct adapter *adap)
> b1871915 Ganesh Goudar 2018-10-09 110 {
> b1871915 Ganesh Goudar 2018-10-09 111 if (adap->ch_thermal.tzdev)
> b1871915 Ganesh Goudar 2018-10-09 @112 thermal_zone_device_unregister(adap->ch_thermal.tzdev);
>
> :::::: The code at line 96 was first introduced by commit
> :::::: b187191577629b5358acf4e234809ee8d441ceb4 cxgb4: Add thermal zone support
>
> :::::: TO: Ganesh Goudar <ganeshgr@chelsio.com>
> :::::: CC: David S. Miller <davem@davemloft.net>
>
> ---
> 0-DAY kernel test infrastructure Open Source Technology Center
> https://lists.01.org/pipermail/kbuild-all Intel Corporation
>
ciao.
--
~Randy
^ permalink raw reply
* Re: [Patch net] net: hns3: bugfix for not checking return value
From: David Miller @ 2018-11-09 0:24 UTC (permalink / raw)
To: tanhuazhong; +Cc: netdev, linux-kernel, salil.mehta, yisen.zhuang, linuxarm
In-Reply-To: <1541643199-57792-1-git-send-email-tanhuazhong@huawei.com>
From: Huazhong Tan <tanhuazhong@huawei.com>
Date: Thu, 8 Nov 2018 10:13:19 +0800
> hns3_reset_notify_init_enet() only return error early if the return
> value of hns3_restore_vlan() is not 0.
>
> This patch adds checking for the return value of hns3_restore_vlan.
>
> Fixes: 7fa6be4fd2f6 ("net: hns3: fix incorrect return value/type of some functions")
> Signed-off-by: Huazhong Tan <tanhuazhong@huawei.com>
Applied, thank you.
^ permalink raw reply
* [PATCH net 4/4] qed: Fix potential memory corruption
From: Denis Bolotin @ 2018-11-08 14:46 UTC (permalink / raw)
To: davem, netdev; +Cc: ariel.elior, michal.kalderon, Sagiv Ozeri, Denis Bolotin
In-Reply-To: <20181108144611.28598-1-denis.bolotin@cavium.com>
From: Sagiv Ozeri <sagiv.ozeri@cavium.com>
A stuck ramrod should be deleted from the completion_pending list,
otherwise it will be added again in the future and corrupt the list.
Return error value to inform that ramrod is stuck and should be deleted.
Signed-off-by: Sagiv Ozeri <sagiv.ozeri@cavium.com>
Signed-off-by: Denis Bolotin <denis.bolotin@cavium.com>
---
drivers/net/ethernet/qlogic/qed/qed_spq.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_spq.c b/drivers/net/ethernet/qlogic/qed/qed_spq.c
index c1a81ec..0a9c5bb 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_spq.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_spq.c
@@ -142,6 +142,7 @@ static int qed_spq_block(struct qed_hwfn *p_hwfn,
DP_INFO(p_hwfn, "Ramrod is stuck, requesting MCP drain\n");
rc = qed_mcp_drain(p_hwfn, p_ptt);
+ qed_ptt_release(p_hwfn, p_ptt);
if (rc) {
DP_NOTICE(p_hwfn, "MCP drain failed\n");
goto err;
@@ -150,18 +151,15 @@ static int qed_spq_block(struct qed_hwfn *p_hwfn,
/* Retry after drain */
rc = __qed_spq_block(p_hwfn, p_ent, p_fw_ret, true);
if (!rc)
- goto out;
+ return 0;
comp_done = (struct qed_spq_comp_done *)p_ent->comp_cb.cookie;
- if (comp_done->done == 1)
+ if (comp_done->done == 1) {
if (p_fw_ret)
*p_fw_ret = comp_done->fw_return_code;
-out:
- qed_ptt_release(p_hwfn, p_ptt);
- return 0;
-
+ return 0;
+ }
err:
- qed_ptt_release(p_hwfn, p_ptt);
DP_NOTICE(p_hwfn,
"Ramrod is stuck [CID %08x cmd %02x protocol %02x echo %04x]\n",
le32_to_cpu(p_ent->elem.hdr.cid),
--
1.8.3.1
^ permalink raw reply related
* [PATCH net 3/4] qed: Fix SPQ entries not returned to pool in error flows
From: Denis Bolotin @ 2018-11-08 14:46 UTC (permalink / raw)
To: davem, netdev; +Cc: ariel.elior, michal.kalderon, Denis Bolotin
In-Reply-To: <20181108144611.28598-1-denis.bolotin@cavium.com>
qed_sp_destroy_request() API was added for SPQ users that need to
free/return the entry they acquired in their error flows.
Signed-off-by: Denis Bolotin <denis.bolotin@cavium.com>
Signed-off-by: Michal Kalderon <michal.kalderon@cavium.com>
---
drivers/net/ethernet/qlogic/qed/qed_fcoe.c | 11 ++++++++---
drivers/net/ethernet/qlogic/qed/qed_iscsi.c | 1 +
drivers/net/ethernet/qlogic/qed/qed_l2.c | 12 ++++++++----
drivers/net/ethernet/qlogic/qed/qed_rdma.c | 1 +
drivers/net/ethernet/qlogic/qed/qed_roce.c | 1 +
drivers/net/ethernet/qlogic/qed/qed_sp.h | 11 +++++++++++
drivers/net/ethernet/qlogic/qed/qed_sp_commands.c | 22 ++++++++++++++--------
drivers/net/ethernet/qlogic/qed/qed_sriov.c | 1 +
8 files changed, 45 insertions(+), 15 deletions(-)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_fcoe.c b/drivers/net/ethernet/qlogic/qed/qed_fcoe.c
index cc1b373..46dc93d 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_fcoe.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_fcoe.c
@@ -147,7 +147,8 @@ struct qed_fcoe_conn {
"Cannot satisfy CQ amount. CQs requested %d, CQs available %d. Aborting function start\n",
fcoe_pf_params->num_cqs,
p_hwfn->hw_info.feat_num[QED_FCOE_CQ]);
- return -EINVAL;
+ rc = -EINVAL;
+ goto err;
}
p_data->mtu = cpu_to_le16(fcoe_pf_params->mtu);
@@ -156,14 +157,14 @@ struct qed_fcoe_conn {
rc = qed_cxt_acquire_cid(p_hwfn, PROTOCOLID_FCOE, &dummy_cid);
if (rc)
- return rc;
+ goto err;
cxt_info.iid = dummy_cid;
rc = qed_cxt_get_cid_info(p_hwfn, &cxt_info);
if (rc) {
DP_NOTICE(p_hwfn, "Cannot find context info for dummy cid=%d\n",
dummy_cid);
- return rc;
+ goto err;
}
p_cxt = cxt_info.p_cxt;
SET_FIELD(p_cxt->tstorm_ag_context.flags3,
@@ -240,6 +241,10 @@ struct qed_fcoe_conn {
rc = qed_spq_post(p_hwfn, p_ent, NULL);
return rc;
+
+err:
+ qed_sp_destroy_request(p_hwfn, p_ent);
+ return rc;
}
static int
diff --git a/drivers/net/ethernet/qlogic/qed/qed_iscsi.c b/drivers/net/ethernet/qlogic/qed/qed_iscsi.c
index 1135387..4f8a685 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_iscsi.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_iscsi.c
@@ -200,6 +200,7 @@ struct qed_iscsi_conn {
"Cannot satisfy CQ amount. Queues requested %d, CQs available %d. Aborting function start\n",
p_params->num_queues,
p_hwfn->hw_info.feat_num[QED_ISCSI_CQ]);
+ qed_sp_destroy_request(p_hwfn, p_ent);
return -EINVAL;
}
diff --git a/drivers/net/ethernet/qlogic/qed/qed_l2.c b/drivers/net/ethernet/qlogic/qed/qed_l2.c
index 82a1bd1..67c02ea93 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_l2.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_l2.c
@@ -740,8 +740,7 @@ int qed_sp_vport_update(struct qed_hwfn *p_hwfn,
rc = qed_sp_vport_update_rss(p_hwfn, p_ramrod, p_rss_params);
if (rc) {
- /* Return spq entry which is taken in qed_sp_init_request()*/
- qed_spq_return_entry(p_hwfn, p_ent);
+ qed_sp_destroy_request(p_hwfn, p_ent);
return rc;
}
@@ -1355,6 +1354,7 @@ static enum eth_filter_action qed_filter_action(enum qed_filter_opcode opcode)
DP_NOTICE(p_hwfn,
"%d is not supported yet\n",
p_filter_cmd->opcode);
+ qed_sp_destroy_request(p_hwfn, *pp_ent);
return -EINVAL;
}
@@ -2056,13 +2056,13 @@ void qed_arfs_mode_configure(struct qed_hwfn *p_hwfn,
} else {
rc = qed_fw_vport(p_hwfn, p_params->vport_id, &abs_vport_id);
if (rc)
- return rc;
+ goto err;
if (p_params->qid != QED_RFS_NTUPLE_QID_RSS) {
rc = qed_fw_l2_queue(p_hwfn, p_params->qid,
&abs_rx_q_id);
if (rc)
- return rc;
+ goto err;
p_ramrod->rx_qid_valid = 1;
p_ramrod->rx_qid = cpu_to_le16(abs_rx_q_id);
@@ -2083,6 +2083,10 @@ void qed_arfs_mode_configure(struct qed_hwfn *p_hwfn,
(u64)p_params->addr, p_params->length);
return qed_spq_post(p_hwfn, p_ent, NULL);
+
+err:
+ qed_sp_destroy_request(p_hwfn, p_ent);
+ return rc;
}
int qed_get_rxq_coalesce(struct qed_hwfn *p_hwfn,
diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
index c71391b..6211343 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
@@ -1514,6 +1514,7 @@ static int qed_rdma_modify_qp(void *rdma_cxt,
default:
rc = -EINVAL;
DP_VERBOSE(p_hwfn, QED_MSG_RDMA, "rc = %d\n", rc);
+ qed_sp_destroy_request(p_hwfn, p_ent);
return rc;
}
SET_FIELD(p_ramrod->flags1,
diff --git a/drivers/net/ethernet/qlogic/qed/qed_roce.c b/drivers/net/ethernet/qlogic/qed/qed_roce.c
index f9167d1..e49fada 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_roce.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_roce.c
@@ -745,6 +745,7 @@ static int qed_roce_sp_destroy_qp_responder(struct qed_hwfn *p_hwfn,
DP_NOTICE(p_hwfn,
"qed destroy responder failed: cannot allocate memory (ramrod). rc = %d\n",
rc);
+ qed_sp_destroy_request(p_hwfn, p_ent);
return rc;
}
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sp.h b/drivers/net/ethernet/qlogic/qed/qed_sp.h
index 04259df..3157c0d 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sp.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_sp.h
@@ -399,6 +399,17 @@ struct qed_sp_init_data {
struct qed_spq_comp_cb *p_comp_data;
};
+/**
+ * @brief Returns a SPQ entry to the pool / frees the entry if allocated.
+ * Should be called on in error flows after initializing the SPQ entry
+ * and before posting it.
+ *
+ * @param p_hwfn
+ * @param p_ent
+ */
+void qed_sp_destroy_request(struct qed_hwfn *p_hwfn,
+ struct qed_spq_entry *p_ent);
+
int qed_sp_init_request(struct qed_hwfn *p_hwfn,
struct qed_spq_entry **pp_ent,
u8 cmd,
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c b/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
index e86a1ea..888274f 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
@@ -47,6 +47,19 @@
#include "qed_sp.h"
#include "qed_sriov.h"
+void qed_sp_destroy_request(struct qed_hwfn *p_hwfn,
+ struct qed_spq_entry *p_ent)
+{
+ /* qed_spq_get_entry() can either get an entry from the free_pool,
+ * or, if no entries are left, allocate a new entry and add it to
+ * the unlimited_pending list.
+ */
+ if (p_ent->queue == &p_hwfn->p_spq->unlimited_pending)
+ kfree(p_ent);
+ else
+ qed_spq_return_entry(p_hwfn, p_ent);
+}
+
int qed_sp_init_request(struct qed_hwfn *p_hwfn,
struct qed_spq_entry **pp_ent,
u8 cmd, u8 protocol, struct qed_sp_init_data *p_data)
@@ -111,14 +124,7 @@ int qed_sp_init_request(struct qed_hwfn *p_hwfn,
return 0;
err:
- /* qed_spq_get_entry() can either get an entry from the free_pool,
- * or, if no entries are left, allocate a new entry and add it to
- * the unlimited_pending list.
- */
- if (p_ent->queue == &p_hwfn->p_spq->unlimited_pending)
- kfree(p_ent);
- else
- qed_spq_return_entry(p_hwfn, p_ent);
+ qed_sp_destroy_request(p_hwfn, p_ent);
return -EINVAL;
}
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sriov.c b/drivers/net/ethernet/qlogic/qed/qed_sriov.c
index 9b08a9d..ca6290f 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sriov.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_sriov.c
@@ -101,6 +101,7 @@ static int qed_sp_vf_start(struct qed_hwfn *p_hwfn, struct qed_vf_info *p_vf)
default:
DP_NOTICE(p_hwfn, "Unknown VF personality %d\n",
p_hwfn->hw_info.personality);
+ qed_sp_destroy_request(p_hwfn, p_ent);
return -EINVAL;
}
--
1.8.3.1
^ permalink raw reply related
* [PATCH net 2/4] qed: Fix blocking/unlimited SPQ entries leak
From: Denis Bolotin @ 2018-11-08 14:46 UTC (permalink / raw)
To: davem, netdev; +Cc: ariel.elior, michal.kalderon, Denis Bolotin
In-Reply-To: <20181108144611.28598-1-denis.bolotin@cavium.com>
When there are no SPQ entries left in the free_pool, new entries are
allocated and are added to the unlimited list. When an entry in the pool
is available, the content is copied from the original entry, and the new
entry is sent to the device. qed_spq_post() is not aware of that, so the
additional entry is stored in the original entry as p_post_ent, which can
later be returned to the pool.
Signed-off-by: Denis Bolotin <denis.bolotin@cavium.com>
Signed-off-by: Michal Kalderon <michal.kalderon@cavium.com>
---
drivers/net/ethernet/qlogic/qed/qed_sp.h | 3 ++
drivers/net/ethernet/qlogic/qed/qed_spq.c | 57 ++++++++++++++++---------------
2 files changed, 33 insertions(+), 27 deletions(-)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sp.h b/drivers/net/ethernet/qlogic/qed/qed_sp.h
index e95431f..04259df 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sp.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_sp.h
@@ -167,6 +167,9 @@ struct qed_spq_entry {
enum spq_mode comp_mode;
struct qed_spq_comp_cb comp_cb;
struct qed_spq_comp_done comp_done; /* SPQ_MODE_EBLOCK */
+
+ /* Posted entry for unlimited list entry in EBLOCK mode */
+ struct qed_spq_entry *post_ent;
};
struct qed_eq {
diff --git a/drivers/net/ethernet/qlogic/qed/qed_spq.c b/drivers/net/ethernet/qlogic/qed/qed_spq.c
index c4a6274..c1a81ec 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_spq.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_spq.c
@@ -685,6 +685,8 @@ static int qed_spq_add_entry(struct qed_hwfn *p_hwfn,
/* EBLOCK responsible to free the allocated p_ent */
if (p_ent->comp_mode != QED_SPQ_MODE_EBLOCK)
kfree(p_ent);
+ else
+ p_ent->post_ent = p_en2;
p_ent = p_en2;
}
@@ -767,6 +769,25 @@ static int qed_spq_pend_post(struct qed_hwfn *p_hwfn)
SPQ_HIGH_PRI_RESERVE_DEFAULT);
}
+/* Avoid overriding of SPQ entries when getting out-of-order completions, by
+ * marking the completions in a bitmap and increasing the chain consumer only
+ * for the first successive completed entries.
+ */
+static void qed_spq_comp_bmap_update(struct qed_hwfn *p_hwfn, __le16 echo)
+{
+ u16 pos = le16_to_cpu(echo) % SPQ_RING_SIZE;
+ struct qed_spq *p_spq = p_hwfn->p_spq;
+
+ __set_bit(pos, p_spq->p_comp_bitmap);
+ while (test_bit(p_spq->comp_bitmap_idx,
+ p_spq->p_comp_bitmap)) {
+ __clear_bit(p_spq->comp_bitmap_idx,
+ p_spq->p_comp_bitmap);
+ p_spq->comp_bitmap_idx++;
+ qed_chain_return_produced(&p_spq->chain);
+ }
+}
+
int qed_spq_post(struct qed_hwfn *p_hwfn,
struct qed_spq_entry *p_ent, u8 *fw_return_code)
{
@@ -824,11 +845,12 @@ int qed_spq_post(struct qed_hwfn *p_hwfn,
p_ent->queue == &p_spq->unlimited_pending);
if (p_ent->queue == &p_spq->unlimited_pending) {
- /* This is an allocated p_ent which does not need to
- * return to pool.
- */
+ struct qed_spq_entry *p_post_ent = p_ent->post_ent;
+
kfree(p_ent);
- return rc;
+
+ /* Return the entry which was actually posted */
+ p_ent = p_post_ent;
}
if (rc)
@@ -842,7 +864,7 @@ int qed_spq_post(struct qed_hwfn *p_hwfn,
spq_post_fail2:
spin_lock_bh(&p_spq->lock);
list_del(&p_ent->list);
- qed_chain_return_produced(&p_spq->chain);
+ qed_spq_comp_bmap_update(p_hwfn, p_ent->elem.hdr.echo);
spq_post_fail:
/* return to the free pool */
@@ -874,25 +896,8 @@ int qed_spq_completion(struct qed_hwfn *p_hwfn,
spin_lock_bh(&p_spq->lock);
list_for_each_entry_safe(p_ent, tmp, &p_spq->completion_pending, list) {
if (p_ent->elem.hdr.echo == echo) {
- u16 pos = le16_to_cpu(echo) % SPQ_RING_SIZE;
-
list_del(&p_ent->list);
-
- /* Avoid overriding of SPQ entries when getting
- * out-of-order completions, by marking the completions
- * in a bitmap and increasing the chain consumer only
- * for the first successive completed entries.
- */
- __set_bit(pos, p_spq->p_comp_bitmap);
-
- while (test_bit(p_spq->comp_bitmap_idx,
- p_spq->p_comp_bitmap)) {
- __clear_bit(p_spq->comp_bitmap_idx,
- p_spq->p_comp_bitmap);
- p_spq->comp_bitmap_idx++;
- qed_chain_return_produced(&p_spq->chain);
- }
-
+ qed_spq_comp_bmap_update(p_hwfn, echo);
p_spq->comp_count++;
found = p_ent;
break;
@@ -931,11 +936,9 @@ int qed_spq_completion(struct qed_hwfn *p_hwfn,
QED_MSG_SPQ,
"Got a completion without a callback function\n");
- if ((found->comp_mode != QED_SPQ_MODE_EBLOCK) ||
- (found->queue == &p_spq->unlimited_pending))
+ if (found->comp_mode != QED_SPQ_MODE_EBLOCK)
/* EBLOCK is responsible for returning its own entry into the
- * free list, unless it originally added the entry into the
- * unlimited pending list.
+ * free list.
*/
qed_spq_return_entry(p_hwfn, found);
--
1.8.3.1
^ permalink raw reply related
* [PATCH net 1/4] qed: Fix memory/entry leak in qed_init_sp_request()
From: Denis Bolotin @ 2018-11-08 14:46 UTC (permalink / raw)
To: davem, netdev; +Cc: ariel.elior, michal.kalderon, Denis Bolotin
In-Reply-To: <20181108144611.28598-1-denis.bolotin@cavium.com>
Free the allocated SPQ entry or return the acquired SPQ entry to the free
list in error flows.
Signed-off-by: Denis Bolotin <denis.bolotin@cavium.com>
Signed-off-by: Michal Kalderon <michal.kalderon@cavium.com>
---
drivers/net/ethernet/qlogic/qed/qed_sp_commands.c | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c b/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
index 77b6248..e86a1ea 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_sp_commands.c
@@ -80,7 +80,7 @@ int qed_sp_init_request(struct qed_hwfn *p_hwfn,
case QED_SPQ_MODE_BLOCK:
if (!p_data->p_comp_data)
- return -EINVAL;
+ goto err;
p_ent->comp_cb.cookie = p_data->p_comp_data->cookie;
break;
@@ -95,7 +95,7 @@ int qed_sp_init_request(struct qed_hwfn *p_hwfn,
default:
DP_NOTICE(p_hwfn, "Unknown SPQE completion mode %d\n",
p_ent->comp_mode);
- return -EINVAL;
+ goto err;
}
DP_VERBOSE(p_hwfn, QED_MSG_SPQ,
@@ -109,6 +109,18 @@ int qed_sp_init_request(struct qed_hwfn *p_hwfn,
memset(&p_ent->ramrod, 0, sizeof(p_ent->ramrod));
return 0;
+
+err:
+ /* qed_spq_get_entry() can either get an entry from the free_pool,
+ * or, if no entries are left, allocate a new entry and add it to
+ * the unlimited_pending list.
+ */
+ if (p_ent->queue == &p_hwfn->p_spq->unlimited_pending)
+ kfree(p_ent);
+ else
+ qed_spq_return_entry(p_hwfn, p_ent);
+
+ return -EINVAL;
}
static enum tunnel_clss qed_tunn_clss_to_fw_clss(u8 type)
--
1.8.3.1
^ permalink raw reply related
* [PATCH net 0/4] Slowpath Queue bug fixes
From: Denis Bolotin @ 2018-11-08 14:46 UTC (permalink / raw)
To: davem, netdev; +Cc: ariel.elior, michal.kalderon, Denis Bolotin
Hi Dave,
This patch series fixes several bugs in the SPQ mechanism.
It deals with SPQ entries management, preventing resource leaks, memory
corruptions and handles error cases throughout the driver.
Please consider applying to net.
Thanks,
Denis
Denis Bolotin (4):
qed: Fix memory/entry leak in qed_init_sp_request()
qed: Fix blocking/unlimited SPQ entries leak
qed: Fix SPQ entries not returned to pool in error flows
qed: Fix potential memory corruption
drivers/net/ethernet/qlogic/qed/qed_fcoe.c | 11 +++-
drivers/net/ethernet/qlogic/qed/qed_iscsi.c | 1 +
drivers/net/ethernet/qlogic/qed/qed_l2.c | 12 ++--
drivers/net/ethernet/qlogic/qed/qed_rdma.c | 1 +
drivers/net/ethernet/qlogic/qed/qed_roce.c | 1 +
drivers/net/ethernet/qlogic/qed/qed_sp.h | 14 +++++
drivers/net/ethernet/qlogic/qed/qed_sp_commands.c | 22 +++++++-
drivers/net/ethernet/qlogic/qed/qed_spq.c | 69 ++++++++++++-----------
drivers/net/ethernet/qlogic/qed/qed_sriov.c | 1 +
9 files changed, 89 insertions(+), 43 deletions(-)
--
1.8.3.1
^ permalink raw reply
* Re: [PATCH net-next] net: bcmgenet: return correct value 'ret' from bcmgenet_power_down
From: David Miller @ 2018-11-09 0:22 UTC (permalink / raw)
To: yuehaibing; +Cc: opendmb, f.fainelli, netdev, kernel-janitors, linux-kernel
In-Reply-To: <1541642923-81463-1-git-send-email-yuehaibing@huawei.com>
From: YueHaibing <yuehaibing@huawei.com>
Date: Thu, 8 Nov 2018 02:08:43 +0000
> Fixes gcc '-Wunused-but-set-variable' warning:
>
> drivers/net/ethernet/broadcom/genet/bcmgenet.c: In function 'bcmgenet_power_down':
> drivers/net/ethernet/broadcom/genet/bcmgenet.c:1136:6: warning:
> variable 'ret' set but not used [-Wunused-but-set-variable]
>
> bcmgenet_power_down should return 'ret' instead of 0.
>
> Fixes: ca8cf341903f ("net: bcmgenet: propagate errors from bcmgenet_power_down")
> Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Applied, thanks.
^ permalink raw reply
* Re: Kernel 4.19 network performance - forwarding/routing normal users traffic
From: Paweł Staszewski @ 2018-11-08 14:43 UTC (permalink / raw)
To: Jesper Dangaard Brouer; +Cc: David Ahern, netdev, Yoel Caspersen
In-Reply-To: <795357b6-04b8-dbc2-acfe-d561f10d4a2a@itcare.pl>
W dniu 08.11.2018 o 01:59, Paweł Staszewski pisze:
>
>
> W dniu 05.11.2018 o 21:17, Jesper Dangaard Brouer pisze:
>> On Sun, 4 Nov 2018 01:24:03 +0100 Paweł Staszewski
>> <pstaszewski@itcare.pl> wrote:
>>
>>> And today again after allpy patch for page allocator - reached again
>>> 64/64 Gbit/s
>>>
>>> with only 50-60% cpu load
>> Great.
>>
>>> today no slowpath hit for netwoking :)
>>>
>>> But again dropped pckt at 64GbitRX and 64TX ....
>>> And as it should not be pcie express limit -i think something more is
>> Well, this does sounds like a PCIe bandwidth limit to me.
>>
>> See the PCIe BW here: https://en.wikipedia.org/wiki/PCI_Express
>>
>> You likely have PCIe v3, where 1-lane have 984.6 MBytes/s or 7.87 Gbit/s
>> Thus, x16-lanes have 15.75 GBytes or 126 Gbit/s. It does say "in each
>> direction", but you are also forwarding this RX->TX on both (dual) ports
>> NIC that is sharing the same PCIe slot.
> Network controller changed from 2-port 100G connectx4 to 2 separate
> cards 100G connectx5
>
>
> PerfTop: 92239 irqs/sec kernel:99.4% exact: 0.0% [4000Hz
> cycles], (all, 56 CPUs)
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>
> 6.65% [kernel] [k] irq_entries_start
> 5.57% [kernel] [k] tasklet_action_common.isra.21
> 4.60% [kernel] [k] mlx5_eq_int
> 4.04% [kernel] [k] mlx5e_skb_from_cqe_mpwrq_linear
> 3.66% [kernel] [k] _raw_spin_lock_irqsave
> 3.58% [kernel] [k] mlx5e_sq_xmit
> 2.66% [kernel] [k] fib_table_lookup
> 2.52% [kernel] [k] _raw_spin_lock
> 2.51% [kernel] [k] build_skb
> 2.50% [kernel] [k] _raw_spin_lock_irq
> 2.04% [kernel] [k] try_to_wake_up
> 1.83% [kernel] [k] queued_spin_lock_slowpath
> 1.81% [kernel] [k] mlx5e_poll_tx_cq
> 1.65% [kernel] [k] do_idle
> 1.50% [kernel] [k] mlx5e_poll_rx_cq
> 1.34% [kernel] [k] __sched_text_start
> 1.32% [kernel] [k] cmd_exec
> 1.30% [kernel] [k] cmd_work_handler
> 1.16% [kernel] [k] vlan_do_receive
> 1.15% [kernel] [k] memcpy_erms
> 1.15% [kernel] [k] __dev_queue_xmit
> 1.07% [kernel] [k] mlx5_cmd_comp_handler
> 1.06% [kernel] [k] sched_ttwu_pending
> 1.00% [kernel] [k] ipt_do_table
> 0.98% [kernel] [k] ip_finish_output2
> 0.92% [kernel] [k] pfifo_fast_dequeue
> 0.88% [kernel] [k] mlx5e_handle_rx_cqe_mpwrq
> 0.78% [kernel] [k] dev_gro_receive
> 0.78% [kernel] [k] mlx5e_napi_poll
> 0.76% [kernel] [k] mlx5e_post_rx_mpwqes
> 0.70% [kernel] [k] process_one_work
> 0.67% [kernel] [k] __netif_receive_skb_core
> 0.65% [kernel] [k] __build_skb
> 0.63% [kernel] [k] llist_add_batch
> 0.62% [kernel] [k] tcp_gro_receive
> 0.60% [kernel] [k] inet_gro_receive
> 0.59% [kernel] [k] ip_route_input_rcu
> 0.59% [kernel] [k] rcu_irq_exit
> 0.56% [kernel] [k] napi_complete_done
> 0.52% [kernel] [k] kmem_cache_alloc
> 0.48% [kernel] [k] __softirqentry_text_start
> 0.48% [kernel] [k] mlx5e_xmit
> 0.47% [kernel] [k] __queue_work
> 0.46% [kernel] [k] memset_erms
> 0.46% [kernel] [k] dev_hard_start_xmit
> 0.45% [kernel] [k] insert_work
> 0.45% [kernel] [k] enqueue_task_fair
> 0.44% [kernel] [k] __wake_up_common
> 0.43% [kernel] [k] finish_task_switch
> 0.43% [kernel] [k] kmem_cache_free_bulk
> 0.42% [kernel] [k] ip_forward
> 0.42% [kernel] [k] worker_thread
> 0.41% [kernel] [k] schedule
> 0.41% [kernel] [k] _raw_spin_unlock_irqrestore
> 0.40% [kernel] [k] netif_skb_features
> 0.40% [kernel] [k] queue_work_on
> 0.40% [kernel] [k] pfifo_fast_enqueue
> 0.39% [kernel] [k] vlan_dev_hard_start_xmit
> 0.39% [kernel] [k] page_frag_free
> 0.36% [kernel] [k] swiotlb_map_page
> 0.36% [kernel] [k] update_cfs_rq_h_load
> 0.35% [kernel] [k] validate_xmit_skb.isra.142
> 0.35% [kernel] [k] dev_ifconf
> 0.35% [kernel] [k] check_preempt_curr
> 0.34% [kernel] [k] _raw_spin_trylock
> 0.34% [kernel] [k] rcu_idle_exit
> 0.33% [kernel] [k] ip_rcv_core.isra.20.constprop.25
> 0.33% [kernel] [k] __qdisc_run
> 0.33% [kernel] [k] skb_release_data
> 0.32% [kernel] [k] native_sched_clock
> 0.30% [kernel] [k] add_interrupt_randomness
> 0.29% [kernel] [k] interrupt_entry
> 0.28% [kernel] [k] skb_gro_receive
> 0.26% [kernel] [k] read_tsc
> 0.26% [kernel] [k] __get_xps_queue_idx
> 0.26% [kernel] [k] inet_gifconf
> 0.26% [kernel] [k] skb_segment
> 0.25% [kernel] [k] __tasklet_schedule_common
> 0.25% [kernel] [k] smpboot_thread_fn
> 0.23% [kernel] [k] __update_load_avg_se
> 0.22% [kernel] [k] tcp4_gro_receive
>
>
> Not much traffic now:
> bwm-ng v0.6.1 (probing every 0.500s), press 'h' for help
> input: /proc/net/dev type: rate
> | iface Rx Tx Total
> ==============================================================================
>
> enp175s0: 6.95 Gb/s 4.20 Gb/s
> 11.15 Gb/s
> enp216s0: 4.23 Gb/s 6.98 Gb/s
> 11.21 Gb/s
> ------------------------------------------------------------------------------
>
> total: 11.18 Gb/s 11.18 Gb/s
> 22.37 Gb/s
>
> bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
> input: /proc/net/dev type: rate
> | iface Rx Tx Total
> ==============================================================================
>
> enp175s0: 700264.50 P/s 923890.25 P/s 1624154.75
> P/s
> enp216s0: 932598.81 P/s 708771.50 P/s 1641370.25
> P/s
> ------------------------------------------------------------------------------
>
> total: 1632863.38 P/s 1632661.75 P/s 3265525.00
> P/s
>
>
>
updated perf top - more traffic 37Gbit/37Gbit total traffic
bwm-ng v0.6.1 (probing every 0.500s), press 'h' for help
input: /proc/net/dev type: rate
/ iface Rx Tx Total
==============================================================================
enp175s0: 28.91 Gb/s 8.89 Gb/s
37.80 Gb/s
enp216s0: 8.91 Gb/s 28.95 Gb/s
37.86 Gb/s
------------------------------------------------------------------------------
total: 37.82 Gb/s 37.84 Gb/s
75.67 Gb/s
bwm-ng v0.6.1 (probing every 0.500s), press 'h' for help
input: /proc/net/dev type: rate
- iface Rx Tx Total
==============================================================================
enp175s0: 2721518.75 P/s 2460930.50 P/s 5182449.50 P/s
enp216s0: 2471451.25 P/s 2731946.25 P/s 5203397.50 P/s
------------------------------------------------------------------------------
total: 5192970.00 P/s 5192876.50 P/s 10385847.00 P/s
PerfTop: 56488 irqs/sec kernel:99.4% exact: 0.0% [4000Hz
cycles], (all, 56 CPUs)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10.41% [kernel] [k] mlx5e_skb_from_cqe_mpwrq_linear
7.73% [kernel] [k] mlx5e_sq_xmit
6.05% [kernel] [k] build_skb
5.63% [kernel] [k] fib_table_lookup
2.75% [kernel] [k] mlx5e_poll_rx_cq
2.74% [kernel] [k] memcpy_erms
2.33% [kernel] [k] vlan_do_receive
2.00% [kernel] [k] __dev_queue_xmit
1.81% [kernel] [k] ip_finish_output2
1.79% [kernel] [k] dev_gro_receive
1.78% [kernel] [k] ipt_do_table
1.78% [kernel] [k] mlx5e_handle_rx_cqe_mpwrq
1.76% [kernel] [k] pfifo_fast_dequeue
1.70% [kernel] [k] mlx5e_post_rx_mpwqes
1.52% [kernel] [k] mlx5e_poll_tx_cq
1.49% [kernel] [k] irq_entries_start
1.47% [kernel] [k] _raw_spin_lock
1.45% [kernel] [k] inet_gro_receive
1.42% [kernel] [k] __netif_receive_skb_core
1.39% [kernel] [k] mlx5_eq_int
1.39% [kernel] [k] tcp_gro_receive
1.23% [kernel] [k] __build_skb
1.14% [kernel] [k] ip_route_input_rcu
1.00% [kernel] [k] vlan_dev_hard_start_xmit
0.92% [kernel] [k] _raw_spin_lock_irqsave
0.89% [kernel] [k] kmem_cache_alloc
0.88% [kernel] [k] dev_hard_start_xmit
0.88% [kernel] [k] swiotlb_map_page
0.86% [kernel] [k] mlx5e_xmit
0.81% [kernel] [k] ip_forward
0.80% [kernel] [k] tasklet_action_common.isra.21
0.79% [kernel] [k] netif_skb_features
0.77% [kernel] [k] pfifo_fast_enqueue
0.66% [kernel] [k] validate_xmit_skb.isra.142
0.64% [kernel] [k] ip_rcv_core.isra.20.constprop.25
0.63% [kernel] [k] find_busiest_group
0.60% [kernel] [k] __qdisc_run
0.59% [kernel] [k] skb_release_data
0.59% [kernel] [k] skb_gro_receive
0.58% [kernel] [k] page_frag_free
0.53% [kernel] [k] skb_segment
0.52% [kernel] [k] try_to_wake_up
0.52% [kernel] [k] _raw_spin_lock_irq
0.50% [kernel] [k] tcp4_gro_receive
0.47% [kernel] [k] kmem_cache_free_bulk
0.45% [kernel] [k] mlx5e_page_release
0.43% [kernel] [k] _raw_spin_trylock
0.39% [kernel] [k] kmem_cache_free
0.38% [kernel] [k] __sched_text_start
0.38% [kernel] [k] sch_direct_xmit
0.38% [kernel] [k] do_idle
0.34% [kernel] [k] vlan_passthru_hard_header
0.34% [kernel] [k] cmd_exec
0.34% [kernel] [k] __local_bh_enable_ip
0.33% [kernel] [k] inet_lookup_ifaddr_rcu
0.33% [kernel] [k] skb_network_protocol
0.33% [kernel] [k] netdev_pick_tx
0.33% [kernel] [k] eth_type_trans
0.32% [kernel] [k] __get_xps_queue_idx
0.31% [kernel] [k] __slab_free.isra.79
0.29% [kernel] [k] mlx5e_xdp_handle
0.27% [kernel] [k] sched_ttwu_pending
0.26% [kernel] [k] cmd_work_handler
0.24% [kernel] [k] ip_finish_output
0.23% [kernel] [k] neigh_connected_output
0.23% [kernel] [k] napi_gro_receive
0.23% [kernel] [k] mlx5e_napi_poll
0.23% [kernel] [k] mlx5e_features_check
0.22% [kernel] [k] ip_output
0.21% [kernel] [k] ip_rcv_finish_core.isra.17
0.21% [kernel] [k] fib_validate_source
0.20% [kernel] [k] dev_ifconf
0.20% [kernel] [k] eth_header
0.20% [kernel] [k] __netdev_pick_tx
0.20% [kernel] [k] mlx5_cmd_comp_handler
0.19% [kernel] [k] memset_erms
0.18% [kernel] [k] __netif_receive_skb_one_core
0.18% [kernel] [k] __memcpy
0.18% [kernel] [k] queued_spin_lock_slowpath
0.18% [kernel] [k] nf_hook_slow
0.17% [kernel] [k] enqueue_task_fair
Also modified a little coal settings for connectx5 compared to connectx4
ethtool -c enp175s0
Coalesce parameters for enp175s0:
Adaptive RX: off TX: on
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0
dmac: 32588
rx-usecs: 128
rx-frames: 128
rx-usecs-irq: 0
rx-frames-irq: 0
tx-usecs: 8
tx-frames: 128
tx-usecs-irq: 0
tx-frames-irq: 0
rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0
rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0
So far cpu load (looks better than in previous configuration with 2 port
100G connectx4):
Average: CPU %usr %nice %sys %iowait %irq %soft %steal
%guest %gnice %idle
Average: all 0.05 0.00 0.64 0.01 0.00 8.79 0.00
0.00 0.00 90.51
Average: 0 0.00 0.00 0.10 0.00 0.00 0.00 0.00
0.00 0.00 99.90
Average: 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 2 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 4 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 5 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 6 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 7 0.10 0.00 1.30 0.00 0.00 0.00 0.00
0.00 0.00 98.60
Average: 8 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 9 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 10 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 11 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 12 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 13 0.00 0.00 1.00 0.00 0.00 0.00 0.00
0.00 0.00 99.00
Average: 14 0.10 0.00 0.80 0.00 0.00 22.80 0.00
0.00 0.00 76.30
Average: 15 0.10 0.00 0.70 0.00 0.00 21.20 0.00
0.00 0.00 78.00
Average: 16 0.00 0.00 0.80 0.00 0.00 17.70 0.00
0.00 0.00 81.50
Average: 17 0.00 0.00 0.50 0.00 0.00 15.10 0.00
0.00 0.00 84.40
Average: 18 0.00 0.00 0.70 0.00 0.00 20.90 0.00
0.00 0.00 78.40
Average: 19 0.10 0.00 0.70 0.00 0.00 20.50 0.00
0.00 0.00 78.70
Average: 20 0.50 0.00 1.70 0.00 0.00 18.80 0.00
0.00 0.00 79.00
Average: 21 0.10 0.00 1.30 0.00 0.00 20.90 0.00
0.00 0.00 77.70
Average: 22 0.00 0.00 0.70 0.00 0.00 19.40 0.00
0.00 0.00 79.90
Average: 23 0.00 0.00 0.90 0.00 0.00 18.50 0.00
0.00 0.00 80.60
Average: 24 0.10 0.00 1.00 0.00 0.00 15.80 0.00
0.00 0.00 83.10
Average: 25 0.00 0.00 0.70 0.00 0.00 19.50 0.00
0.00 0.00 79.80
Average: 26 0.00 0.00 0.50 0.00 0.00 18.30 0.00
0.00 0.00 81.20
Average: 27 0.00 0.00 0.70 0.00 0.00 17.60 0.00
0.00 0.00 81.70
Average: 28 0.00 0.00 0.70 0.00 0.00 0.00 0.00
0.00 0.00 99.30
Average: 29 0.00 0.00 2.00 0.00 0.00 0.00 0.00
0.00 0.00 98.00
Average: 30 0.00 0.00 0.10 0.00 0.00 0.00 0.00
0.00 0.00 99.90
Average: 31 0.00 0.00 2.50 0.00 0.00 0.00 0.00
0.00 0.00 97.50
Average: 32 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 33 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 34 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 35 0.00 0.00 0.70 0.00 0.00 0.00 0.00
0.00 0.00 99.30
Average: 36 0.00 0.00 2.00 0.00 0.00 0.00 0.00
0.00 0.00 98.00
Average: 37 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 38 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 39 0.00 0.00 1.40 0.00 0.00 0.00 0.00
0.00 0.00 98.60
Average: 40 0.60 0.00 0.40 0.00 0.00 0.00 0.00
0.00 0.00 99.00
Average: 41 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 100.00
Average: 42 0.00 0.00 1.20 0.00 0.00 17.70 0.00
0.00 0.00 81.10
Average: 43 0.00 0.00 0.70 0.00 0.00 20.00 0.00
0.00 0.00 79.30
Average: 44 0.00 0.00 0.50 0.00 0.00 16.10 0.00
0.00 0.00 83.40
Average: 45 0.30 0.00 1.10 0.00 0.00 16.10 0.00
0.00 0.00 82.50
Average: 46 0.00 0.00 0.80 0.00 0.00 14.80 0.00
0.00 0.00 84.40
Average: 47 0.10 0.00 1.60 0.00 0.00 17.20 0.00
0.00 0.00 81.10
Average: 48 0.00 0.00 0.60 0.00 0.00 15.00 0.00
0.00 0.00 84.40
Average: 49 0.10 0.00 0.80 0.00 0.00 14.90 0.00
0.00 0.00 84.20
Average: 50 0.20 0.00 0.50 0.70 0.00 13.60 0.00
0.00 0.00 85.00
Average: 51 0.00 0.00 0.70 0.00 0.00 14.10 0.00
0.00 0.00 85.20
Average: 52 0.20 0.00 1.60 0.00 0.00 16.80 0.00
0.00 0.00 81.40
Average: 53 0.00 0.00 0.80 0.00 0.00 13.20 0.00
0.00 0.00 86.00
Average: 54 0.20 0.00 0.50 0.00 0.00 17.20 0.00
0.00 0.00 82.10
Average: 55 0.00 0.00 0.40 0.00 0.00 18.30 0.00
0.00 0.00 81.30
>
>
>
>>
>>
>>> going on there - and hard to catch - cause perf top doestn chenged
>>> besides there is no queued slowpath hit now
>>>
>>> I ordered now also intel cards to compare - but 3 weeks eta
>>> Faster - cause 3 days - i will have mellanox connectx 5 - so can
>>> separate traffic to two different x16 pcie busses
>> I do think you need to separate traffic to two different x16 PCIe
>> slots. I have found that the ConnectX-5 is significantly faster
>> packet-per-sec performance than ConnectX-4, but that is not your
>> use-case (max BW). I've not tested these NICs for maximum
>> _bidirectional_ bandwidth limits, I've only made sure I can do 100G
>> unidirectional, which can hit some funny motherboard memory limits
>> (remember to equip motherboard with 4 RAM blocks for full memory BW).
>>
> Yes memory channels are separated and there are 4 modules per cpu :)
>
>
^ permalink raw reply
* Re: [PATCH 3/3] iwlwifi: Load firmware exclusively for Intel WiFi
From: João Paulo Rechi Vita @ 2018-11-09 0:11 UTC (permalink / raw)
To: Marcel Holtmann
Cc: kai.heng.feng, Luca Coelho, Kalle Valo, Emmanuel Grumbach,
Johannes Berg, David S. Miller, Intel Linux Wireless,
linux-wireless, Network Development, LKML, linux,
João Paulo Rechi Vita, linux-bluetooth
In-Reply-To: <20181109000800.15431-1-jprvita@gmail.com>
+ linux-bluetooth
On Thu, Nov 8, 2018 at 4:08 PM João Paulo Rechi Vita <jprvita@gmail.com> wrote:
>
> Hello Marcel,
>
> > On Oct 4, 2018, at 2:25 AM, Marcel Holtmann <marcel@holtmann.org> wrote:
> >
> > Hi Kai-Heng,
> >
> >>> I think Canonical were facing some wifi fw load error from some 8260
> >>> earlier module during the BT still loading the fw.
> >>> I believe we had later 8260 sku that fixed this issue.
> >>
> >> But there are already 8260 that is affected by this bug in the wild.
> >>
> >> Search "Bluetooth: hci0: Failed to send firmware data (-38)” and there are lots of user are affected.
> >
> > which SKUs are these actually. What are the initial details about the boot loader. For the Bluetooth side, you should be able to grab them from dmesg or by running btmon.
> >
> > So I am not in favor of this kind of hack and creating dependencies between drivers. If you only have a hammer, then everything looks like a nail. And this is a massive hammer trying to squash everything. This problem needs to be debugged. And this starts by providing affected SKU information and firmware information. So get the details about the SKU and its Bluetooth and WiFi boot loaders.
> >
>
> I have a Lenovo Yoga 900 which presents this problem and has the same bootloader / firmware information as Kai-Heng already posted:
>
> [ 5.992426] Bluetooth: Core ver 2.22
> [ 5.992438] Bluetooth: HCI device and connection manager initialized
> [ 5.992442] Bluetooth: HCI socket layer initialized
> [ 5.992444] Bluetooth: L2CAP socket layer initialized
> [ 5.992450] Bluetooth: SCO socket layer initialized
> [ 6.004941] Bluetooth: hci0: Bootloader revision 0.0 build 2 week 52 2014
> [ 6.010922] Bluetooth: hci0: Device revision is 5
> [ 6.010923] Bluetooth: hci0: Secure boot is enabled
> [ 6.010924] Bluetooth: hci0: OTP lock is enabled
> [ 6.010925] Bluetooth: hci0: API lock is enabled
> [ 6.010926] Bluetooth: hci0: Debug lock is disabled
> [ 6.010927] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
> [ 6.014253] bluetooth hci0: firmware: direct-loading firmware intel/ibt-11-5.sfi
> [ 6.014256] Bluetooth: hci0: Found device firmware: intel/ibt-11-5.sfi
> [ 6.613961] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
> [ 6.613966] Bluetooth: BNEP filters: protocol multicast
> [ 6.613974] Bluetooth: BNEP socket layer initialized
> [ 6.983804] Bluetooth: hci0: Failed to send firmware data (-38)
>
> And the following product id and revision, from usb-devices:
>
> T: Bus=01 Lev=01 Prnt=01 Port=06 Cnt=02 Dev#= 3 Spd=12 MxCh= 0
> D: Ver= 2.00 Cls=e0(wlcon) Sub=01 Prot=01 MxPS=64 #Cfgs= 1
> P: Vendor=8087 ProdID=0a2b Rev=00.01
> C: #Ifs= 2 Cfg#= 1 Atr=e0 MxPwr=100mA
> I: If#= 0 Alt= 0 #EPs= 3 Cls=e0(wlcon) Sub=01 Prot=01 Driver=btusb
> I: If#= 1 Alt= 0 #EPs= 2 Cls=e0(wlcon) Sub=01 Prot=01 Driver=btusb
>
> I understand the drawbacks with the approach presented here and lack of clear explanation of the problem, but I can confirm these patches work around the problem on my machine. Is there any extra info or test result I can provide to help debug this? I can also dedicate time to help write a different solution if some guidance is provided.
>
> Kai-Heng, did you end up filling a Bugzilla entry for this?
>
> Please CC-me on the replies as I'm not receiving emails from linux-bluetooth or linux-wireless anymore.
>
> Thanks,
>
> --
> João Paulo Rechi Vita
> http://about.me/jprvita
^ permalink raw reply
* Re: [PATCH 3/3] iwlwifi: Load firmware exclusively for Intel WiFi
From: João Paulo Rechi Vita @ 2018-11-09 0:08 UTC (permalink / raw)
To: Marcel Holtmann
Cc: João Paulo Rechi Vita, Kai-Heng Feng, Luca Coelho,
Kalle Valo, Emmanuel Grumbach, Johannes Berg, David S. Miller,
Intel Linux Wireless, linux-wireless, netdev, linux-kernel, linux,
João Paulo Rechi Vita
In-Reply-To: <910F8A29-3141-422F-87EE-7BFBCDE9060C@holtmann.org>
Hello Marcel,
> On Oct 4, 2018, at 2:25 AM, Marcel Holtmann <marcel@holtmann.org> wrote:
>
> Hi Kai-Heng,
>
>>> I think Canonical were facing some wifi fw load error from some 8260
>>> earlier module during the BT still loading the fw.
>>> I believe we had later 8260 sku that fixed this issue.
>>
>> But there are already 8260 that is affected by this bug in the wild.
>>
>> Search "Bluetooth: hci0: Failed to send firmware data (-38)” and there are lots of user are affected.
>
> which SKUs are these actually. What are the initial details about the boot loader. For the Bluetooth side, you should be able to grab them from dmesg or by running btmon.
>
> So I am not in favor of this kind of hack and creating dependencies between drivers. If you only have a hammer, then everything looks like a nail. And this is a massive hammer trying to squash everything. This problem needs to be debugged. And this starts by providing affected SKU information and firmware information. So get the details about the SKU and its Bluetooth and WiFi boot loaders.
>
I have a Lenovo Yoga 900 which presents this problem and has the same bootloader / firmware information as Kai-Heng already posted:
[ 5.992426] Bluetooth: Core ver 2.22
[ 5.992438] Bluetooth: HCI device and connection manager initialized
[ 5.992442] Bluetooth: HCI socket layer initialized
[ 5.992444] Bluetooth: L2CAP socket layer initialized
[ 5.992450] Bluetooth: SCO socket layer initialized
[ 6.004941] Bluetooth: hci0: Bootloader revision 0.0 build 2 week 52 2014
[ 6.010922] Bluetooth: hci0: Device revision is 5
[ 6.010923] Bluetooth: hci0: Secure boot is enabled
[ 6.010924] Bluetooth: hci0: OTP lock is enabled
[ 6.010925] Bluetooth: hci0: API lock is enabled
[ 6.010926] Bluetooth: hci0: Debug lock is disabled
[ 6.010927] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[ 6.014253] bluetooth hci0: firmware: direct-loading firmware intel/ibt-11-5.sfi
[ 6.014256] Bluetooth: hci0: Found device firmware: intel/ibt-11-5.sfi
[ 6.613961] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 6.613966] Bluetooth: BNEP filters: protocol multicast
[ 6.613974] Bluetooth: BNEP socket layer initialized
[ 6.983804] Bluetooth: hci0: Failed to send firmware data (-38)
And the following product id and revision, from usb-devices:
T: Bus=01 Lev=01 Prnt=01 Port=06 Cnt=02 Dev#= 3 Spd=12 MxCh= 0
D: Ver= 2.00 Cls=e0(wlcon) Sub=01 Prot=01 MxPS=64 #Cfgs= 1
P: Vendor=8087 ProdID=0a2b Rev=00.01
C: #Ifs= 2 Cfg#= 1 Atr=e0 MxPwr=100mA
I: If#= 0 Alt= 0 #EPs= 3 Cls=e0(wlcon) Sub=01 Prot=01 Driver=btusb
I: If#= 1 Alt= 0 #EPs= 2 Cls=e0(wlcon) Sub=01 Prot=01 Driver=btusb
I understand the drawbacks with the approach presented here and lack of clear explanation of the problem, but I can confirm these patches work around the problem on my machine. Is there any extra info or test result I can provide to help debug this? I can also dedicate time to help write a different solution if some guidance is provided.
Kai-Heng, did you end up filling a Bugzilla entry for this?
Please CC-me on the replies as I'm not receiving emails from linux-bluetooth or linux-wireless anymore.
Thanks,
--
João Paulo Rechi Vita
http://about.me/jprvita
^ permalink raw reply
* [PATCH v3 net-next] sock: Reset dst when changing sk_mark via setsockopt
From: David Barmann @ 2018-11-08 14:13 UTC (permalink / raw)
To: netdev
When setting the SO_MARK socket option, if the mark changes, the dst
needs to be reset so that a new route lookup is performed.
This fixes the case where an application wants to change routing by
setting a new sk_mark. If this is done after some packets have already
been sent, the dst is cached and has no effect.
Signed-off-by: David Barmann <david.barmann@stackpath.com>
---
net/core/sock.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/net/core/sock.c b/net/core/sock.c
index 7b304e454a38..6d7e189e3cd9 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -952,10 +952,12 @@ int sock_setsockopt(struct socket *sock, int level, int optname,
clear_bit(SOCK_PASSSEC, &sock->flags);
break;
case SO_MARK:
- if (!ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN))
+ if (!ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN)) {
ret = -EPERM;
- else
+ } else if (val != sk->sk_mark) {
sk->sk_mark = val;
+ sk_dst_reset(sk);
+ }
break;
case SO_RXQ_OVFL:
--
2.14.5
^ permalink raw reply related
* [PATCH net-next 5/8] s390/qeth: replace open-coded cmd setup
From: Julian Wiedmann @ 2018-11-08 14:06 UTC (permalink / raw)
To: David Miller
Cc: netdev, linux-s390, Martin Schwidefsky, Heiko Carstens,
Stefan Raspl, Ursula Braun, Julian Wiedmann
In-Reply-To: <20181108140622.35244-1-jwi@linux.ibm.com>
Call qeth_prepare_ipa_cmd() during setup of a new IPA cmd buffer, so
that it is used for all commands. Thus ARP and SNMP requests don't have
to do their own initialization.
This will now also set the proper MPC protocol version for SNMP requests
on L2 devices.
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
---
drivers/s390/net/qeth_core_main.c | 27 ++++++++++++---------------
drivers/s390/net/qeth_core_mpc.c | 1 -
drivers/s390/net/qeth_l3_main.c | 3 ---
3 files changed, 12 insertions(+), 19 deletions(-)
diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c
index 508bad2dbd0b..c1315ac63041 100644
--- a/drivers/s390/net/qeth_core_main.c
+++ b/drivers/s390/net/qeth_core_main.c
@@ -2835,6 +2835,17 @@ static void qeth_fill_ipacmd_header(struct qeth_card *card,
cmd->hdr.prot_version = prot;
}
+void qeth_prepare_ipa_cmd(struct qeth_card *card, struct qeth_cmd_buffer *iob)
+{
+ u8 prot_type = qeth_mpc_select_prot_type(card);
+
+ memcpy(iob->data, IPA_PDU_HEADER, IPA_PDU_HEADER_SIZE);
+ memcpy(QETH_IPA_CMD_PROT_TYPE(iob->data), &prot_type, 1);
+ memcpy(QETH_IPA_CMD_DEST_ADDR(iob->data),
+ &card->token.ulp_connection_r, QETH_MPC_TOKEN_LENGTH);
+}
+EXPORT_SYMBOL_GPL(qeth_prepare_ipa_cmd);
+
struct qeth_cmd_buffer *qeth_get_ipacmd_buffer(struct qeth_card *card,
enum qeth_ipa_cmds ipacmd, enum qeth_prot_versions prot)
{
@@ -2842,6 +2853,7 @@ struct qeth_cmd_buffer *qeth_get_ipacmd_buffer(struct qeth_card *card,
iob = qeth_get_buffer(&card->write);
if (iob) {
+ qeth_prepare_ipa_cmd(card, iob);
qeth_fill_ipacmd_header(card, __ipa_cmd(iob), ipacmd, prot);
} else {
dev_warn(&card->gdev->dev,
@@ -2854,17 +2866,6 @@ struct qeth_cmd_buffer *qeth_get_ipacmd_buffer(struct qeth_card *card,
}
EXPORT_SYMBOL_GPL(qeth_get_ipacmd_buffer);
-void qeth_prepare_ipa_cmd(struct qeth_card *card, struct qeth_cmd_buffer *iob)
-{
- u8 prot_type = qeth_mpc_select_prot_type(card);
-
- memcpy(iob->data, IPA_PDU_HEADER, IPA_PDU_HEADER_SIZE);
- memcpy(QETH_IPA_CMD_PROT_TYPE(iob->data), &prot_type, 1);
- memcpy(QETH_IPA_CMD_DEST_ADDR(iob->data),
- &card->token.ulp_connection_r, QETH_MPC_TOKEN_LENGTH);
-}
-EXPORT_SYMBOL_GPL(qeth_prepare_ipa_cmd);
-
/**
* qeth_send_ipa_cmd() - send an IPA command
*
@@ -2879,7 +2880,6 @@ int qeth_send_ipa_cmd(struct qeth_card *card, struct qeth_cmd_buffer *iob,
int rc;
QETH_CARD_TEXT(card, 4, "sendipa");
- qeth_prepare_ipa_cmd(card, iob);
rc = qeth_send_control_data(card, IPA_CMD_LENGTH,
iob, reply_cb, reply_param);
if (rc == -ETIME) {
@@ -4497,9 +4497,6 @@ static int qeth_send_ipa_snmp_cmd(struct qeth_card *card,
QETH_CARD_TEXT(card, 4, "sendsnmp");
- memcpy(iob->data, IPA_PDU_HEADER, IPA_PDU_HEADER_SIZE);
- memcpy(QETH_IPA_CMD_DEST_ADDR(iob->data),
- &card->token.ulp_connection_r, QETH_MPC_TOKEN_LENGTH);
/* adjust PDU length fields in IPA_PDU_HEADER */
s1 = (u32) IPA_PDU_HEADER_SIZE + len;
s2 = (u32) len;
diff --git a/drivers/s390/net/qeth_core_mpc.c b/drivers/s390/net/qeth_core_mpc.c
index e891c0b52f4c..16fc51ad0514 100644
--- a/drivers/s390/net/qeth_core_mpc.c
+++ b/drivers/s390/net/qeth_core_mpc.c
@@ -144,7 +144,6 @@ unsigned char IPA_PDU_HEADER[] = {
sizeof(struct qeth_ipa_cmd) % 256,
0x00, 0x00, 0x00, 0x40,
};
-EXPORT_SYMBOL_GPL(IPA_PDU_HEADER);
struct ipa_rc_msg {
enum qeth_ipa_return_codes rc;
diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c
index 61d847d41cab..ce5ef6aca891 100644
--- a/drivers/s390/net/qeth_l3_main.c
+++ b/drivers/s390/net/qeth_l3_main.c
@@ -1693,9 +1693,6 @@ static int qeth_l3_send_ipa_arp_cmd(struct qeth_card *card,
{
QETH_CARD_TEXT(card, 4, "sendarp");
- memcpy(iob->data, IPA_PDU_HEADER, IPA_PDU_HEADER_SIZE);
- memcpy(QETH_IPA_CMD_DEST_ADDR(iob->data),
- &card->token.ulp_connection_r, QETH_MPC_TOKEN_LENGTH);
return qeth_send_control_data(card, IPA_PDU_HEADER_SIZE + len, iob,
reply_cb, reply_param);
}
--
2.16.4
^ permalink raw reply related
* [PATCH net-next 7/8] s390/qeth: remove unused fallback in Layer3's MAC code
From: Julian Wiedmann @ 2018-11-08 14:06 UTC (permalink / raw)
To: David Miller
Cc: netdev, linux-s390, Martin Schwidefsky, Heiko Carstens,
Stefan Raspl, Ursula Braun, Julian Wiedmann
In-Reply-To: <20181108140622.35244-1-jwi@linux.ibm.com>
If the CREATE ADDR sent by qeth_l3_iqd_read_initial_mac() fails, its
callback sets a random MAC address on the net_device. The error then
propagates back, and qeth_l3_setup_netdev() bails out without
registering the net_device.
Any subsequent call to qeth_l3_setup_netdev() will then attempt a fresh
CREATE ADDR which either 1) also fails, or 2) sets a proper MAC address
on the net_device. Consequently, the net_device will never be registered
with a random MAC and we can drop the fallback code.
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
---
drivers/s390/net/qeth_l3_main.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c
index 89d5e6fb21ac..674ff3344ff3 100644
--- a/drivers/s390/net/qeth_l3_main.c
+++ b/drivers/s390/net/qeth_l3_main.c
@@ -949,9 +949,6 @@ static int qeth_l3_iqd_read_initial_mac_cb(struct qeth_card *card,
if (cmd->hdr.return_code == 0)
ether_addr_copy(card->dev->dev_addr,
cmd->data.create_destroy_addr.unique_id);
- else
- eth_random_addr(card->dev->dev_addr);
-
return 0;
}
--
2.16.4
^ permalink raw reply related
* [PATCH net-next 8/8] s390/qeth: don't process hsuid in qeth_l3_setup_netdev()
From: Julian Wiedmann @ 2018-11-08 14:06 UTC (permalink / raw)
To: David Miller
Cc: netdev, linux-s390, Martin Schwidefsky, Heiko Carstens,
Stefan Raspl, Ursula Braun, Julian Wiedmann
In-Reply-To: <20181108140622.35244-1-jwi@linux.ibm.com>
qeth_l3_setup_netdev() checks if the hsuid attribute is set on the qeth
device, and propagates it to the net_device. In the past this was needed
to pick up any hsuid that was set before allocation of the net_device.
With commit d3d1b205e89f ("s390/qeth: allocate netdevice early") this
is no longer necessary, qeth_l3_dev_hsuid_store() always stores the
hsuid straight into dev->perm_addr.
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
---
drivers/s390/net/qeth_l3_main.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c
index 674ff3344ff3..eca68da39d05 100644
--- a/drivers/s390/net/qeth_l3_main.c
+++ b/drivers/s390/net/qeth_l3_main.c
@@ -2279,9 +2279,6 @@ static int qeth_l3_setup_netdev(struct qeth_card *card, bool carrier_ok)
rc = qeth_l3_iqd_read_initial_mac(card);
if (rc)
goto out;
-
- if (card->options.hsuid[0])
- memcpy(card->dev->perm_addr, card->options.hsuid, 9);
} else
return -ENODEV;
--
2.16.4
^ permalink raw reply related
* [PATCH net-next 6/8] s390/qeth: remove two IPA command helpers
From: Julian Wiedmann @ 2018-11-08 14:06 UTC (permalink / raw)
To: David Miller
Cc: netdev, linux-s390, Martin Schwidefsky, Heiko Carstens,
Stefan Raspl, Ursula Braun, Julian Wiedmann
In-Reply-To: <20181108140622.35244-1-jwi@linux.ibm.com>
qeth_l3_send_ipa_arp_cmd() is merely a wrapper around
qeth_send_control_data() now. So push the length adjustment into
QETH_SETASS_BASE_LEN, and remove the wrapper. While at it, also remove
some redundant 0-initializations.
qeth_send_setassparms() requires that callers prepare their command
parameters, so that they can be copied into the parameter area in one
go. Skip the indirection, and just let callers set up the command
themselves.
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
---
drivers/s390/net/qeth_core_main.c | 51 ++++++++++-----------------------------
drivers/s390/net/qeth_core_mpc.h | 7 +++---
drivers/s390/net/qeth_l3_main.c | 20 +++------------
3 files changed, 19 insertions(+), 59 deletions(-)
diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c
index c1315ac63041..ae972461860e 100644
--- a/drivers/s390/net/qeth_core_main.c
+++ b/drivers/s390/net/qeth_core_main.c
@@ -5475,34 +5475,11 @@ struct qeth_cmd_buffer *qeth_get_setassparms_cmd(struct qeth_card *card,
}
EXPORT_SYMBOL_GPL(qeth_get_setassparms_cmd);
-static int qeth_send_setassparms(struct qeth_card *card,
- struct qeth_cmd_buffer *iob, u16 len,
- long data, int (*reply_cb)(struct qeth_card *,
- struct qeth_reply *,
- unsigned long),
- void *reply_param)
-{
- int rc;
- struct qeth_ipa_cmd *cmd;
-
- QETH_CARD_TEXT(card, 4, "sendassp");
-
- cmd = __ipa_cmd(iob);
- if (len <= sizeof(__u32))
- cmd->data.setassparms.data.flags_32bit = (__u32) data;
- else /* (len > sizeof(__u32)) */
- memcpy(&cmd->data.setassparms.data, (void *) data, len);
-
- rc = qeth_send_ipa_cmd(card, iob, reply_cb, reply_param);
- return rc;
-}
-
int qeth_send_simple_setassparms_prot(struct qeth_card *card,
enum qeth_ipa_funcs ipa_func,
u16 cmd_code, long data,
enum qeth_prot_versions prot)
{
- int rc;
int length = 0;
struct qeth_cmd_buffer *iob;
@@ -5512,9 +5489,9 @@ int qeth_send_simple_setassparms_prot(struct qeth_card *card,
iob = qeth_get_setassparms_cmd(card, ipa_func, cmd_code, length, prot);
if (!iob)
return -ENOMEM;
- rc = qeth_send_setassparms(card, iob, length, data,
- qeth_setassparms_cb, NULL);
- return rc;
+
+ __ipa_cmd(iob)->data.setassparms.data.flags_32bit = (__u32) data;
+ return qeth_send_ipa_cmd(card, iob, qeth_setassparms_cb, NULL);
}
EXPORT_SYMBOL_GPL(qeth_send_simple_setassparms_prot);
@@ -6385,16 +6362,16 @@ static int qeth_ipa_checksum_run_cmd(struct qeth_card *card,
enum qeth_prot_versions prot)
{
struct qeth_cmd_buffer *iob;
- int rc = -ENOMEM;
QETH_CARD_TEXT(card, 4, "chkdocmd");
iob = qeth_get_setassparms_cmd(card, ipa_func, cmd_code,
sizeof(__u32), prot);
- if (iob)
- rc = qeth_send_setassparms(card, iob, sizeof(__u32), data,
- qeth_ipa_checksum_run_cmd_cb,
- chksum_cb);
- return rc;
+ if (!iob)
+ return -ENOMEM;
+
+ __ipa_cmd(iob)->data.setassparms.data.flags_32bit = (__u32) data;
+ return qeth_send_ipa_cmd(card, iob, qeth_ipa_checksum_run_cmd_cb,
+ chksum_cb);
}
static int qeth_send_checksum_on(struct qeth_card *card, int cstype,
@@ -6492,8 +6469,7 @@ static int qeth_set_tso_on(struct qeth_card *card,
if (!iob)
return -ENOMEM;
- rc = qeth_send_setassparms(card, iob, 0, 0 /* unused */,
- qeth_start_tso_cb, &tso_data);
+ rc = qeth_send_ipa_cmd(card, iob, qeth_start_tso_cb, &tso_data);
if (rc)
return rc;
@@ -6510,10 +6486,9 @@ static int qeth_set_tso_on(struct qeth_card *card,
}
/* enable TSO capability */
- caps.supported = 0;
- caps.enabled = QETH_IPA_LARGE_SEND_TCP;
- rc = qeth_send_setassparms(card, iob, sizeof(caps), (long) &caps,
- qeth_setassparms_get_caps_cb, &caps);
+ __ipa_cmd(iob)->data.setassparms.data.caps.enabled =
+ QETH_IPA_LARGE_SEND_TCP;
+ rc = qeth_send_ipa_cmd(card, iob, qeth_setassparms_get_caps_cb, &caps);
if (rc) {
qeth_set_tso_off(card, prot);
return rc;
diff --git a/drivers/s390/net/qeth_core_mpc.h b/drivers/s390/net/qeth_core_mpc.h
index 10cc487c16d6..1ab321926f64 100644
--- a/drivers/s390/net/qeth_core_mpc.h
+++ b/drivers/s390/net/qeth_core_mpc.h
@@ -833,10 +833,9 @@ enum qeth_ipa_arp_return_codes {
extern const char *qeth_get_ipa_msg(enum qeth_ipa_return_codes rc);
extern const char *qeth_get_ipa_cmd_name(enum qeth_ipa_cmds cmd);
-#define QETH_SETASS_BASE_LEN (sizeof(struct qeth_ipacmd_hdr) + \
- sizeof(struct qeth_ipacmd_setassparms_hdr))
-#define QETH_IPA_ARP_DATA_POS(buffer) (buffer + IPA_PDU_HEADER_SIZE + \
- QETH_SETASS_BASE_LEN)
+#define QETH_SETASS_BASE_LEN (IPA_PDU_HEADER_SIZE + \
+ sizeof(struct qeth_ipacmd_hdr) + \
+ sizeof(struct qeth_ipacmd_setassparms_hdr))
#define QETH_SETADP_BASE_LEN (sizeof(struct qeth_ipacmd_hdr) + \
sizeof(struct qeth_ipacmd_setadpparms_hdr))
#define QETH_SNMP_SETADP_CMDLENGTH 16
diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c
index ce5ef6aca891..89d5e6fb21ac 100644
--- a/drivers/s390/net/qeth_l3_main.c
+++ b/drivers/s390/net/qeth_l3_main.c
@@ -1685,18 +1685,6 @@ static int qeth_l3_arp_query_cb(struct qeth_card *card,
return 0;
}
-static int qeth_l3_send_ipa_arp_cmd(struct qeth_card *card,
- struct qeth_cmd_buffer *iob, int len,
- int (*reply_cb)(struct qeth_card *, struct qeth_reply *,
- unsigned long),
- void *reply_param)
-{
- QETH_CARD_TEXT(card, 4, "sendarp");
-
- return qeth_send_control_data(card, IPA_PDU_HEADER_SIZE + len, iob,
- reply_cb, reply_param);
-}
-
static int qeth_l3_query_arp_cache_info(struct qeth_card *card,
enum qeth_prot_versions prot,
struct qeth_arp_query_info *qinfo)
@@ -1716,11 +1704,9 @@ static int qeth_l3_query_arp_cache_info(struct qeth_card *card,
return -ENOMEM;
cmd = __ipa_cmd(iob);
cmd->data.setassparms.data.query_arp.request_bits = 0x000F;
- cmd->data.setassparms.data.query_arp.reply_bits = 0;
- cmd->data.setassparms.data.query_arp.no_entries = 0;
- rc = qeth_l3_send_ipa_arp_cmd(card, iob,
- QETH_SETASS_BASE_LEN+QETH_ARP_CMD_LEN,
- qeth_l3_arp_query_cb, (void *)qinfo);
+ rc = qeth_send_control_data(card,
+ QETH_SETASS_BASE_LEN + QETH_ARP_CMD_LEN,
+ iob, qeth_l3_arp_query_cb, qinfo);
if (rc)
QETH_DBF_MESSAGE(2, "Error while querying ARP cache on device %x: %#x\n",
CARD_DEVID(card), rc);
--
2.16.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox