Netdev List
 help / color / mirror / Atom feed
* [PATCH mlx5-next 1/4] net/mlx5: Use debug message instead of warn
From: Leon Romanovsky @ 2019-08-08  8:43 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe
  Cc: Leon Romanovsky, RDMA mailing list, Edward Srouji, Saeed Mahameed,
	Yishai Hadas, linux-netdev
In-Reply-To: <20190808084358.29517-1-leon@kernel.org>

From: Yishai Hadas <yishaih@mellanox.com>

As QP may be created by DEVX, it may be valid to not find the rsn in
mlx5 core tree, change the level to be debug.

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/qp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/qp.c b/drivers/net/ethernet/mellanox/mlx5/core/qp.c
index b8ba74de9555..f0f3abe331da 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/qp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/qp.c
@@ -162,7 +162,7 @@ static int rsc_event_notifier(struct notifier_block *nb,
 
 	common = mlx5_get_rsc(table, rsn);
 	if (!common) {
-		mlx5_core_warn(dev, "Async event for bogus resource 0x%x\n", rsn);
+		mlx5_core_dbg(dev, "Async event for unknown resource 0x%x\n", rsn);
 		return NOTIFY_OK;
 	}
 
-- 
2.20.1


^ permalink raw reply related

* [PATCH mlx5-next 2/4] net/mlx5: Add XRQ legacy commands opcodes
From: Leon Romanovsky @ 2019-08-08  8:43 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe
  Cc: Leon Romanovsky, RDMA mailing list, Edward Srouji, Saeed Mahameed,
	Yishai Hadas, linux-netdev
In-Reply-To: <20190808084358.29517-1-leon@kernel.org>

From: Yishai Hadas <yishaih@mellanox.com>

Add XRQ legacy commands opcodes, will be used via the DEVX interface.

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 4 ++++
 include/linux/mlx5/mlx5_ifc.h                 | 2 ++
 2 files changed, 6 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
index 8cdd7e66f8df..53d09620e215 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
@@ -446,6 +446,8 @@ static int mlx5_internal_err_ret_value(struct mlx5_core_dev *dev, u16 op,
 	case MLX5_CMD_OP_CREATE_UMEM:
 	case MLX5_CMD_OP_DESTROY_UMEM:
 	case MLX5_CMD_OP_ALLOC_MEMIC:
+	case MLX5_CMD_OP_MODIFY_XRQ:
+	case MLX5_CMD_OP_RELEASE_XRQ_ERROR:
 		*status = MLX5_DRIVER_STATUS_ABORTED;
 		*synd = MLX5_DRIVER_SYND;
 		return -EIO;
@@ -637,6 +639,8 @@ const char *mlx5_command_str(int command)
 	MLX5_COMMAND_STR_CASE(DESTROY_UCTX);
 	MLX5_COMMAND_STR_CASE(CREATE_UMEM);
 	MLX5_COMMAND_STR_CASE(DESTROY_UMEM);
+	MLX5_COMMAND_STR_CASE(RELEASE_XRQ_ERROR);
+	MLX5_COMMAND_STR_CASE(MODIFY_XRQ);
 	default: return "unknown command opcode";
 	}
 }
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 82470b670cb7..080bba20b129 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -172,6 +172,8 @@ enum {
 	MLX5_CMD_OP_QUERY_XRQ_DC_PARAMS_ENTRY     = 0x725,
 	MLX5_CMD_OP_SET_XRQ_DC_PARAMS_ENTRY       = 0x726,
 	MLX5_CMD_OP_QUERY_XRQ_ERROR_PARAMS        = 0x727,
+	MLX5_CMD_OP_RELEASE_XRQ_ERROR             = 0x729,
+	MLX5_CMD_OP_MODIFY_XRQ                    = 0x72a,
 	MLX5_CMD_OP_QUERY_ESW_FUNCTIONS           = 0x740,
 	MLX5_CMD_OP_QUERY_VPORT_STATE             = 0x750,
 	MLX5_CMD_OP_MODIFY_VPORT_STATE            = 0x751,
-- 
2.20.1


^ permalink raw reply related

* [PATCH rdma-next 4/4] IB/mlx5: Expose XRQ legacy commands over the DEVX interface
From: Leon Romanovsky @ 2019-08-08  8:43 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe
  Cc: Leon Romanovsky, RDMA mailing list, Edward Srouji, Saeed Mahameed,
	Yishai Hadas, linux-netdev
In-Reply-To: <20190808084358.29517-1-leon@kernel.org>

From: Yishai Hadas <yishaih@mellanox.com>

Expose missing XRQ legacy commands over the DEVX interface.

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/devx.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
index 3dbdfe0eb5e4..04b4e937c198 100644
--- a/drivers/infiniband/hw/mlx5/devx.c
+++ b/drivers/infiniband/hw/mlx5/devx.c
@@ -546,6 +546,8 @@ static u64 devx_get_obj_id(const void *in)
 		break;
 	case MLX5_CMD_OP_ARM_XRQ:
 	case MLX5_CMD_OP_SET_XRQ_DC_PARAMS_ENTRY:
+	case MLX5_CMD_OP_RELEASE_XRQ_ERROR:
+	case MLX5_CMD_OP_MODIFY_XRQ:
 		obj_id = get_enc_obj_id(MLX5_CMD_OP_CREATE_XRQ,
 					MLX5_GET(arm_xrq_in, in, xrqn));
 		break;
@@ -822,6 +824,8 @@ static bool devx_is_obj_modify_cmd(const void *in)
 	case MLX5_CMD_OP_ARM_DCT_FOR_KEY_VIOLATION:
 	case MLX5_CMD_OP_ARM_XRQ:
 	case MLX5_CMD_OP_SET_XRQ_DC_PARAMS_ENTRY:
+	case MLX5_CMD_OP_RELEASE_XRQ_ERROR:
+	case MLX5_CMD_OP_MODIFY_XRQ:
 		return true;
 	case MLX5_CMD_OP_SET_FLOW_TABLE_ENTRY:
 	{
-- 
2.20.1


^ permalink raw reply related

* [PATCH mlx5-next 3/4] IB/mlx5: Add legacy events to DEVX list
From: Leon Romanovsky @ 2019-08-08  8:43 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe
  Cc: Leon Romanovsky, RDMA mailing list, Edward Srouji, Saeed Mahameed,
	Yishai Hadas, linux-netdev
In-Reply-To: <20190808084358.29517-1-leon@kernel.org>

From: Yishai Hadas <yishaih@mellanox.com>

Add two events that were defined in the device specification but were
not exposed in the driver list.

Post this patch those events can be read over the DEVX events interface
once be reported by the firmware.

Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Edward Srouji <edwards@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/hw/mlx5/devx.c | 8 ++++++++
 include/linux/mlx5/device.h       | 9 +++++++++
 2 files changed, 17 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
index fd577ffd7864..3dbdfe0eb5e4 100644
--- a/drivers/infiniband/hw/mlx5/devx.c
+++ b/drivers/infiniband/hw/mlx5/devx.c
@@ -233,6 +233,8 @@ static bool is_legacy_obj_event_num(u16 event_num)
 	case MLX5_EVENT_TYPE_SRQ_CATAS_ERROR:
 	case MLX5_EVENT_TYPE_DCT_DRAINED:
 	case MLX5_EVENT_TYPE_COMP:
+	case MLX5_EVENT_TYPE_DCT_KEY_VIOLATION:
+	case MLX5_EVENT_TYPE_XRQ_ERROR:
 		return true;
 	default:
 		return false;
@@ -315,8 +317,10 @@ static u16 get_event_obj_type(unsigned long event_type, struct mlx5_eqe *eqe)
 	case MLX5_EVENT_TYPE_SRQ_CATAS_ERROR:
 		return eqe->data.qp_srq.type;
 	case MLX5_EVENT_TYPE_CQ_ERROR:
+	case MLX5_EVENT_TYPE_XRQ_ERROR:
 		return 0;
 	case MLX5_EVENT_TYPE_DCT_DRAINED:
+	case MLX5_EVENT_TYPE_DCT_KEY_VIOLATION:
 		return MLX5_EVENT_QUEUE_TYPE_DCT;
 	default:
 		return MLX5_GET(affiliated_event_header, &eqe->data, obj_type);
@@ -2300,7 +2304,11 @@ static u32 devx_get_obj_id_from_event(unsigned long event_type, void *data)
 	case MLX5_EVENT_TYPE_WQ_ACCESS_ERROR:
 		obj_id = be32_to_cpu(eqe->data.qp_srq.qp_srq_n) & 0xffffff;
 		break;
+	case MLX5_EVENT_TYPE_XRQ_ERROR:
+		obj_id = be32_to_cpu(eqe->data.xrq_err.type_xrqn) & 0xffffff;
+		break;
 	case MLX5_EVENT_TYPE_DCT_DRAINED:
+	case MLX5_EVENT_TYPE_DCT_KEY_VIOLATION:
 		obj_id = be32_to_cpu(eqe->data.dct.dctn) & 0xffffff;
 		break;
 	case MLX5_EVENT_TYPE_CQ_ERROR:
diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index ce9839c8bc1a..e427af260ebe 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -328,6 +328,7 @@ enum mlx5_event {
 	MLX5_EVENT_TYPE_GPIO_EVENT	   = 0x15,
 	MLX5_EVENT_TYPE_PORT_MODULE_EVENT  = 0x16,
 	MLX5_EVENT_TYPE_TEMP_WARN_EVENT    = 0x17,
+	MLX5_EVENT_TYPE_XRQ_ERROR	   = 0x18,
 	MLX5_EVENT_TYPE_REMOTE_CONFIG	   = 0x19,
 	MLX5_EVENT_TYPE_GENERAL_EVENT	   = 0x22,
 	MLX5_EVENT_TYPE_MONITOR_COUNTER    = 0x24,
@@ -345,6 +346,7 @@ enum mlx5_event {
 	MLX5_EVENT_TYPE_ESW_FUNCTIONS_CHANGED = 0xe,
 
 	MLX5_EVENT_TYPE_DCT_DRAINED        = 0x1c,
+	MLX5_EVENT_TYPE_DCT_KEY_VIOLATION  = 0x1d,
 
 	MLX5_EVENT_TYPE_FPGA_ERROR         = 0x20,
 	MLX5_EVENT_TYPE_FPGA_QP_ERROR      = 0x21,
@@ -584,6 +586,12 @@ struct mlx5_eqe_cq_err {
 	u8	syndrome;
 };
 
+struct mlx5_eqe_xrq_err {
+	__be32	reserved1[5];
+	__be32	type_xrqn;
+	__be32	reserved2;
+};
+
 struct mlx5_eqe_port_state {
 	u8	reserved0[8];
 	u8	port;
@@ -698,6 +706,7 @@ union ev_data {
 	struct mlx5_eqe_pps		pps;
 	struct mlx5_eqe_dct             dct;
 	struct mlx5_eqe_temp_warning	temp_warning;
+	struct mlx5_eqe_xrq_err		xrq_err;
 } __packed;
 
 struct mlx5_eqe {
-- 
2.20.1


^ permalink raw reply related

* RE: [PATCH net-next 5/5] r8152: change rx_frag_head_sz and rx_max_agg_num dynamically
From: Hayes Wang @ 2019-08-08  8:52 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: netdev@vger.kernel.org, nic_swsd, linux-kernel@vger.kernel.org,
	linux-usb@vger.kernel.org
In-Reply-To: <20190806151007.75a8dd2c@cakuba.netronome.com>

Jakub Kicinski [mailto:jakub.kicinski@netronome.com]
> Sent: Wednesday, August 07, 2019 6:10 AM
[...]
> On Tue, 6 Aug 2019 19:18:04 +0800, Hayes Wang wrote:
> > Let rx_frag_head_sz and rx_max_agg_num could be modified dynamically
> > through the sysfs.
> >
> > Signed-off-by: Hayes Wang <hayeswang@realtek.com>
> 
> Please don't expose those via sysfs. Ethtool's copybreak and descriptor
> count should be applicable here, I think.

Excuse me again.
I find the kernel supports the copybreak of Ethtool.
However, I couldn't find a command of Ethtool to use it.
Do I miss something?

Best Regards,
Hayes


^ permalink raw reply

* RE: Clause 73 and USXGMII
From: Jose Abreu @ 2019-08-08  9:02 UTC (permalink / raw)
  To: Russell King - ARM Linux admin
  Cc: netdev@vger.kernel.org, Andrew Lunn, Florian Fainelli,
	Heiner Kallweit
In-Reply-To: <20190808082626.GB5193@shell.armlinux.org.uk>

From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
Date: Aug/08/2019, 09:26:26 (UTC+00:00)

> Hi,
> 
> Have you tried enabling debug mode in phylink (add #define DEBUG at the
> top of the file) ?

Yes:

[ With > 2.5G modes removed ]
# dmesg | grep -i phy
libphy: stmmac: probed
stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
00,00000000,0002e040 advertising 00,00000000,0002e040
stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,0002e040 pause=10 
link=0 an=1
stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown

[ Without any limit ]
# dmesg | grep -i phy
libphy: stmmac: probed
stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
00,00000000,000ee040 advertising 00,00000000,000ee040
stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,000ee040 pause=10 
link=0 an=1
stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
stmmaceth 0000:04:00.0 enp4s0: phy link up usxgmii/2.5Gbps/Full
stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
mode=phy/usxgmii/2.5Gbps/Full adv=00,00000000,00000000 pause=0f link=1 
an=0

I'm thinking on whether this can be related with USXGMII. As link is 
operating in 10G but I configure USXGMII for 2.5G maybe autoneg outcome 
should always be 10G ?

> On Thu, Aug 08, 2019 at 08:17:29AM +0000, Jose Abreu wrote:
> > ++ PHY Experts
> > 
> > From: Jose Abreu <joabreu@synopsys.com>
> > Date: Aug/07/2019, 16:46:23 (UTC+00:00)
> > 
> > > Hello,
> > > 
> > > I've some sample code for Clause 73 support using Synopsys based XPCS 
> > > but I would like to clarify some things that I noticed.
> > > 
> > > I'm using USXGMII as interface and a single SERDES that operates at 10G 
> > > rate but MAC side is working at 2.5G. Maximum available bandwidth is 
> > > therefore 2.5Gbps.
> > > 
> > > So, I configure USXGMII for 2.5G mode and it works but if I try to limit 
> > > the autoneg abilities to 2.5G max then it never finishes:
> > > # ethtool enp4s0
> > > Settings for enp4s0:
> > > 	Supported ports: [ ]
> > > 	Supported link modes:   1000baseKX/Full 
> > > 	                        2500baseX/Full 
> > > 	Supported pause frame use: Symmetric Receive-only
> > > 	Supports auto-negotiation: Yes
> > > 	Supported FEC modes: Not reported
> > > 	Advertised link modes:  1000baseKX/Full 
> > > 	                        2500baseX/Full 
> > > 	Advertised pause frame use: Symmetric Receive-only
> > > 	Advertised auto-negotiation: Yes
> > > 	Advertised FEC modes: Not reported
> > > 	Speed: Unknown!
> > > 	Duplex: Unknown! (255)
> > > 	Port: MII
> > > 	PHYAD: 0
> > > 	Transceiver: internal
> > > 	Auto-negotiation: on
> > > 	Supports Wake-on: ug
> > > 	Wake-on: d
> > > 	Current message level: 0x0000003f (63)
> > > 			       drv probe link timer ifdown ifup
> > > 	Link detected: no
> > > 
> > > When I do not limit autoneg and I say that maximum limit is 10G then I 
> > > get Link Up and autoneg finishes with this outcome:
> > > # ethtool enp4s0
> > > Settings for enp4s0:
> > > 	Supported ports: [ ]
> > > 	Supported link modes:   1000baseKX/Full 
> > > 	                        2500baseX/Full 
> > > 	                        10000baseKX4/Full 
> > > 	                        10000baseKR/Full 
> > > 	Supported pause frame use: Symmetric Receive-only
> > > 	Supports auto-negotiation: Yes
> > > 	Supported FEC modes: Not reported
> > > 	Advertised link modes:  1000baseKX/Full 
> > > 	                        2500baseX/Full 
> > > 	                        10000baseKX4/Full 
> > > 	                        10000baseKR/Full 
> > > 	Advertised pause frame use: Symmetric Receive-only
> > > 	Advertised auto-negotiation: Yes
> > > 	Advertised FEC modes: Not reported
> > > 	Link partner advertised link modes:  1000baseKX/Full 
> > > 	                                     2500baseX/Full 
> > > 	                                     10000baseKX4/Full 
> > > 	                                     10000baseKR/Full 
> > > 	Link partner advertised pause frame use: Symmetric Receive-only
> > > 	Link partner advertised auto-negotiation: Yes
> > > 	Link partner advertised FEC modes: Not reported
> > > 	Speed: 2500Mb/s
> > > 	Duplex: Full
> > > 	Port: MII <- Never mind this, it's a SW issue
> > > 	PHYAD: 0
> > > 	Transceiver: internal
> > > 	Auto-negotiation: on
> > > 	Supports Wake-on: ug
> > > 	Wake-on: d
> > > 	Current message level: 0x0000003f (63)
> > > 			       drv probe link timer ifdown ifup
> > > 	Link detected: yes
> > > 
> > > I was expecting that, as MAC side is limited to 2.5G, I should set in 
> > > phylink the correct capabilities and then outcome of autoneg would only 
> > > have up to 2.5G modes. Am I wrong ?
> > > 
> > > ---
> > > Thanks,
> > > Jose Miguel Abreu
> > 
> > 
> > ---
> > Thanks,
> > Jose Miguel Abreu
> > 
> 
> -- 
> RMK's Patch system: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.armlinux.org.uk_developer_patches_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=WHDsc6kcWAl4i96Vm5hJ_19IJiuxx_p_Rzo2g-uHDKw&m=1MdSlPrmzsMMCJbbLcDYTNuPq1njfusBRjcRz3UD4Dg&s=_30hwSYkGf9DfyCG48mnh7lXP8iiULXpfAP_6agUJno&e= 
> FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> According to speedtest.net: 11.9Mbps down 500kbps up


---
Thanks,
Jose Miguel Abreu

^ permalink raw reply

* Re: [PATCH AUTOSEL 4.19 04/42] netfilter: conntrack: always store window size un-scaled
From: Thomas Jarosch @ 2019-08-08  9:02 UTC (permalink / raw)
  To: Sasha Levin
  Cc: linux-kernel, stable, Florian Westphal, Jakub Jankowski,
	Jozsef Kadlecsik, Pablo Neira Ayuso, netfilter-devel, coreteam,
	netdev
In-Reply-To: <20190802132302.13537-4-sashal@kernel.org>

Hello together,

You wrote on Fri, Aug 02, 2019 at 09:22:24AM -0400:
> From: Florian Westphal <fw@strlen.de>
> 
> [ Upstream commit 959b69ef57db00cb33e9c4777400ae7183ebddd3 ]
> 
> Jakub Jankowski reported following oddity:
> 
> After 3 way handshake completes, timeout of new connection is set to
> max_retrans (300s) instead of established (5 days).
> 
> shortened excerpt from pcap provided:
> 25.070622 IP (flags [DF], proto TCP (6), length 52)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [S], seq 11, win 64240, [wscale 8]
> 26.070462 IP (flags [DF], proto TCP (6), length 48)
> 10.8.1.2.80 > 10.8.5.4.1025: Flags [S.], seq 82, ack 12, win 65535, [wscale 3]
> 27.070449 IP (flags [DF], proto TCP (6), length 40)
> 10.8.5.4.1025 > 10.8.1.2.80: Flags [.], ack 83, win 512, length 0
> 
> Turns out the last_win is of u16 type, but we store the scaled value:
> 512 << 8 (== 0x20000) becomes 0 window.
> 
> The Fixes tag is not correct, as the bug has existed forever, but
> without that change all that this causes might cause is to mistake a
> window update (to-nonzero-from-zero) for a retransmit.
> 
> Fixes: fbcd253d2448b8 ("netfilter: conntrack: lower timeout to RETRANS seconds if window is 0")
> Reported-by: Jakub Jankowski <shasta@toxcorp.com>
> Tested-by: Jakub Jankowski <shasta@toxcorp.com>
> Signed-off-by: Florian Westphal <fw@strlen.de>
> Acked-by: Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>
> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> Signed-off-by: Sasha Levin <sashal@kernel.org>

Also:
Tested-by: Thomas Jarosch <thomas.jarosch@intra2net.com>

;)

We've hit the issue with the wrong conntrack timeout at two different sites,
long-lived connections to a SAP server over IPSec VPN were constantly dropping.

For us this was a regression after updating from kernel 3.14 to 4.19.
Yesterday I've applied the patch to kernel 4.19.57 and the problem is fixed.

The issue was extra hard to debug as we could just boot the new kernel
for twenty minutes in the evening on these productive systems.

The stable kernel patch from last Friday came right on time. I was just
about the replay the TCP connection with tcpreplay, so this saved
me from another week of debugging. Thanks everyone!

Best regards,
Thomas Jarosch

^ permalink raw reply

* Re: [PATCH v2 0/3] arm/arm64: Add support for function error injection
From: Leo Yan @ 2019-08-08  9:09 UTC (permalink / raw)
  To: Will Deacon
  Cc: Russell King, Oleg Nesterov, Catalin Marinas,
	Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov, H. Peter Anvin,
	x86, Arnd Bergmann, Alexei Starovoitov, Daniel Borkmann,
	Martin KaFai Lau, Song Liu, Yonghong Song, Naveen N. Rao,
	linux-arm-kernel, linux-kernel, linuxppc-dev, linux-arch, netdev,
	bpf, clang-built-linux, Masami Hiramatsu
In-Reply-To: <20190807160703.pe4jxak7hs7ptvde@willie-the-truck>

On Wed, Aug 07, 2019 at 05:07:03PM +0100, Will Deacon wrote:
> On Tue, Aug 06, 2019 at 06:00:12PM +0800, Leo Yan wrote:
> > This small patch set is to add support for function error injection;
> > this can be used to eanble more advanced debugging feature, e.g.
> > CONFIG_BPF_KPROBE_OVERRIDE.
> > 
> > The patch 01/03 is to consolidate the function definition which can be
> > suared cross architectures, patches 02,03/03 are used for enabling
> > function error injection on arm64 and arm architecture respectively.
> > 
> > I tested on arm64 platform Juno-r2 and one of my laptop with x86
> > architecture with below steps; I don't test for Arm architecture so
> > only pass compilation.
> 
> Thanks. I've queued the first two patches up here:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/error-injection

Thank you, Will.

Leo.

^ permalink raw reply

* Re: Clause 73 and USXGMII
From: Russell King - ARM Linux admin @ 2019-08-08  9:23 UTC (permalink / raw)
  To: Jose Abreu
  Cc: netdev@vger.kernel.org, Andrew Lunn, Florian Fainelli,
	Heiner Kallweit
In-Reply-To: <BN8PR12MB32665E5465A83D5E11C7B5D6D3D70@BN8PR12MB3266.namprd12.prod.outlook.com>

On Thu, Aug 08, 2019 at 09:02:57AM +0000, Jose Abreu wrote:
> From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
> Date: Aug/08/2019, 09:26:26 (UTC+00:00)
> 
> > Hi,
> > 
> > Have you tried enabling debug mode in phylink (add #define DEBUG at the
> > top of the file) ?
> 
> Yes:
> 
> [ With > 2.5G modes removed ]
> # dmesg | grep -i phy
> libphy: stmmac: probed
> stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> 00,00000000,0002e040 advertising 00,00000000,0002e040
> stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,0002e040 pause=10 
> link=0 an=1
> stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown

This shows that the PHY isn't reporting that the link came up.  Did
the PHY negotiate link?  If so, why isn't it reporting that the link
came up?  Maybe something is mis-programming the capability bits in
the PHY?  Maybe disabling the 10G speeds disables everything faster
than 1G?

> [ Without any limit ]
> # dmesg | grep -i phy
> libphy: stmmac: probed
> stmmaceth 0000:04:00.0 enp4s0: PHY [stmmac-1:00] driver [Synopsys 10G]
> stmmaceth 0000:04:00.0 enp4s0: phy: setting supported 
> 00,00000000,000ee040 advertising 00,00000000,000ee040
> stmmaceth 0000:04:00.0 enp4s0: configuring for phy/usxgmii link mode
> stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> mode=phy/usxgmii/Unknown/Unknown adv=00,00000000,000ee040 pause=10 
> link=0 an=1
> stmmaceth 0000:04:00.0 enp4s0: phy link down usxgmii/Unknown/Unknown
> stmmaceth 0000:04:00.0 enp4s0: phy link up usxgmii/2.5Gbps/Full
> stmmaceth 0000:04:00.0 enp4s0: phylink_mac_config: 
> mode=phy/usxgmii/2.5Gbps/Full adv=00,00000000,00000000 pause=0f link=1 
> an=0
> 
> I'm thinking on whether this can be related with USXGMII. As link is 
> operating in 10G but I configure USXGMII for 2.5G maybe autoneg outcome 
> should always be 10G ?

As I understand USXGMII (which isn't very well, because the spec isn't
available) I believe that it operates in a similar way to SGMII where
data is replicated the appropriate number of times to achieve the link
speed.  So, the USXGMII link always operates at a bit rate equivalent
to 10G, but data is replicated twice for 5G, four times for 2.5G, ten
times for 1G, etc.

I notice that you don't say that you support any copper speeds, which
brings up the question about what the PHY's media is...

> > On Thu, Aug 08, 2019 at 08:17:29AM +0000, Jose Abreu wrote:
> > > ++ PHY Experts
> > > 
> > > From: Jose Abreu <joabreu@synopsys.com>
> > > Date: Aug/07/2019, 16:46:23 (UTC+00:00)
> > > 
> > > > Hello,
> > > > 
> > > > I've some sample code for Clause 73 support using Synopsys based XPCS 
> > > > but I would like to clarify some things that I noticed.
> > > > 
> > > > I'm using USXGMII as interface and a single SERDES that operates at 10G 
> > > > rate but MAC side is working at 2.5G. Maximum available bandwidth is 
> > > > therefore 2.5Gbps.
> > > > 
> > > > So, I configure USXGMII for 2.5G mode and it works but if I try to limit 
> > > > the autoneg abilities to 2.5G max then it never finishes:
> > > > # ethtool enp4s0
> > > > Settings for enp4s0:
> > > > 	Supported ports: [ ]
> > > > 	Supported link modes:   1000baseKX/Full 
> > > > 	                        2500baseX/Full 
> > > > 	Supported pause frame use: Symmetric Receive-only
> > > > 	Supports auto-negotiation: Yes
> > > > 	Supported FEC modes: Not reported
> > > > 	Advertised link modes:  1000baseKX/Full 
> > > > 	                        2500baseX/Full 
> > > > 	Advertised pause frame use: Symmetric Receive-only
> > > > 	Advertised auto-negotiation: Yes
> > > > 	Advertised FEC modes: Not reported
> > > > 	Speed: Unknown!
> > > > 	Duplex: Unknown! (255)
> > > > 	Port: MII
> > > > 	PHYAD: 0
> > > > 	Transceiver: internal
> > > > 	Auto-negotiation: on
> > > > 	Supports Wake-on: ug
> > > > 	Wake-on: d
> > > > 	Current message level: 0x0000003f (63)
> > > > 			       drv probe link timer ifdown ifup
> > > > 	Link detected: no
> > > > 
> > > > When I do not limit autoneg and I say that maximum limit is 10G then I 
> > > > get Link Up and autoneg finishes with this outcome:
> > > > # ethtool enp4s0
> > > > Settings for enp4s0:
> > > > 	Supported ports: [ ]
> > > > 	Supported link modes:   1000baseKX/Full 
> > > > 	                        2500baseX/Full 
> > > > 	                        10000baseKX4/Full 
> > > > 	                        10000baseKR/Full 
> > > > 	Supported pause frame use: Symmetric Receive-only
> > > > 	Supports auto-negotiation: Yes
> > > > 	Supported FEC modes: Not reported
> > > > 	Advertised link modes:  1000baseKX/Full 
> > > > 	                        2500baseX/Full 
> > > > 	                        10000baseKX4/Full 
> > > > 	                        10000baseKR/Full 
> > > > 	Advertised pause frame use: Symmetric Receive-only
> > > > 	Advertised auto-negotiation: Yes
> > > > 	Advertised FEC modes: Not reported
> > > > 	Link partner advertised link modes:  1000baseKX/Full 
> > > > 	                                     2500baseX/Full 
> > > > 	                                     10000baseKX4/Full 
> > > > 	                                     10000baseKR/Full 
> > > > 	Link partner advertised pause frame use: Symmetric Receive-only
> > > > 	Link partner advertised auto-negotiation: Yes
> > > > 	Link partner advertised FEC modes: Not reported
> > > > 	Speed: 2500Mb/s
> > > > 	Duplex: Full
> > > > 	Port: MII <- Never mind this, it's a SW issue
> > > > 	PHYAD: 0
> > > > 	Transceiver: internal
> > > > 	Auto-negotiation: on
> > > > 	Supports Wake-on: ug
> > > > 	Wake-on: d
> > > > 	Current message level: 0x0000003f (63)
> > > > 			       drv probe link timer ifdown ifup
> > > > 	Link detected: yes
> > > > 
> > > > I was expecting that, as MAC side is limited to 2.5G, I should set in 
> > > > phylink the correct capabilities and then outcome of autoneg would only 
> > > > have up to 2.5G modes. Am I wrong ?
> > > > 
> > > > ---
> > > > Thanks,
> > > > Jose Miguel Abreu
> > > 
> > > 
> > > ---
> > > Thanks,
> > > Jose Miguel Abreu
> > > 
> > 
> > -- 
> > RMK's Patch system: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.armlinux.org.uk_developer_patches_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=WHDsc6kcWAl4i96Vm5hJ_19IJiuxx_p_Rzo2g-uHDKw&m=1MdSlPrmzsMMCJbbLcDYTNuPq1njfusBRjcRz3UD4Dg&s=_30hwSYkGf9DfyCG48mnh7lXP8iiULXpfAP_6agUJno&e= 
> > FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
> > According to speedtest.net: 11.9Mbps down 500kbps up
> 
> 
> ---
> Thanks,
> Jose Miguel Abreu
> 

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

^ permalink raw reply

* Re: [PATCH] liquidio: Use pcie_flr() instead of reimplementing it
From: Andrew Murray @ 2019-08-08  9:25 UTC (permalink / raw)
  To: Denis Efremov
  Cc: Bjorn Helgaas, Derek Chickles, Satanand Burla, Felix Manlunas,
	netdev, linux-pci, linux-kernel
In-Reply-To: <20190808045753.5474-1-efremov@linux.com>

On Thu, Aug 08, 2019 at 07:57:53AM +0300, Denis Efremov wrote:
> octeon_mbox_process_cmd() directly writes the PCI_EXP_DEVCTL_BCR_FLR
> bit, which bypasses timing requirements imposed by the PCIe spec.
> This patch fixes the function to use the pcie_flr() interface instead.
> 
> Signed-off-by: Denis Efremov <efremov@linux.com>
> ---
>  drivers/net/ethernet/cavium/liquidio/octeon_mailbox.c | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_mailbox.c b/drivers/net/ethernet/cavium/liquidio/octeon_mailbox.c
> index 021d99cd1665..614d07be7181 100644
> --- a/drivers/net/ethernet/cavium/liquidio/octeon_mailbox.c
> +++ b/drivers/net/ethernet/cavium/liquidio/octeon_mailbox.c
> @@ -260,9 +260,7 @@ static int octeon_mbox_process_cmd(struct octeon_mbox *mbox,
>  		dev_info(&oct->pci_dev->dev,
>  			 "got a request for FLR from VF that owns DPI ring %u\n",
>  			 mbox->q_no);
> -		pcie_capability_set_word(
> -			oct->sriov_info.dpiring_to_vfpcidev_lut[mbox->q_no],
> -			PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR);
> +		pcie_flr(oct->sriov_info.dpiring_to_vfpcidev_lut[mbox->q_no]);
>  		break;

It's possible for pcie_flr to fail if the device doesn't become ready soon
enough after reset in which case it returns ENOTTY. I think it's OK not to
test the return value here though, as pci_dev_wait will print a warning
anyway, and I'm not sure what you'd do with it anyway.

Reviewed-by: Andrew Murray <andrew.murray@arm.com>

>  
>  	case OCTEON_PF_CHANGED_VF_MACADDR:
> -- 
> 2.21.0
> 

^ permalink raw reply

* Re: [bpf-next PATCH 0/3] bpf: improvements to xdp_fwd sample
From: Jesper Dangaard Brouer @ 2019-08-08  9:29 UTC (permalink / raw)
  To: Zvi Effron
  Cc: Xdp, Anton Protopopov, dsahern, Toke Høiland-Jørgensen,
	brouer, netdev@vger.kernel.org
In-Reply-To: <CAC1LvL29KS9CKcXYwR4EHeNo7++i4hYQuXfY5OLtbPFDVUO2mw@mail.gmail.com>

On Wed, 7 Aug 2019 15:09:09 -0700
Zvi Effron <zeffron@riotgames.com> wrote:

> On Wed, Aug 7, 2019 at 6:00 AM Jesper Dangaard Brouer <brouer@redhat.com> wrote:
> >
> > Toke's devmap lookup improvement is first avail in kernel v5.3.
> > Thus, not part of XDP-tutorial yet.
> >  
> I probably missed this in an earlier email, but what are Toke's devmap
> improvements? Performance? Capability?

Toke's devmap and redirect improvements are primarily about usability.

Currently, from BPF-context (kernel-side) you cannot read the contents
of devmap (or cpumap or xskmap(AF_XDP)).  Because for devmap you get
the real pointer to the net_device ifindex, and we cannot allow you to
write/change that from BPF (kernel would likely crash or be inconsistent).

The work-around, is to keep a shadow map, that contains the "config" of
the devmap, which you check/validate against instead.  It is just a pain
to maintain this shadow map.  Toke's change allow you to read devmap
from BPF-context.  Thus, you can avoid this shadow map.

Another improvement from Toke, is that the bpf_redirect_map() helper,
now also check if the redirect index is valid in the map.  If not, then
it returns another value than XDP_REDIRECT.  You can choose the
alternative return value yourself, via "flags" e.g. XDP_PASS.  Thus,
you don't even need to check/validate devmap in your BPF-code, as it is
part of the bpf_redirect_map() call now.

 action = bpf_redirect_map(&map, &index, flags_as_xdp_value) 

The default flags used in most programs today is 0, which maps to
XDP_ABORTED.  This is sort of a small UAPI change, but for the better.
As today, the packet is dropped later, only diagnose/seen via
tracepoint xdp:xdp_redirect_map_err.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer
 

^ permalink raw reply

* [PATCH v2 bpf-next] xdp: xdp_umem: fix umem pages mapping for 32bits systems
From: Ivan Khoronzhuk @ 2019-08-08  9:38 UTC (permalink / raw)
  To: bjorn.topel, magnus.karlsson
  Cc: davem, ast, daniel, john.fastabend, hawk, netdev, bpf,
	xdp-newbies, linux-kernel, Ivan Khoronzhuk

Use kmap instead of page_address as it's not always in low memory.

Acked-by: Björn Töpel <bjorn.topel@intel.com>
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---

Based on bpf-next/master
v2..v1:
	included highmem.h

v1: https://lkml.org/lkml/2019/6/26/693

 net/xdp/xdp_umem.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/net/xdp/xdp_umem.c b/net/xdp/xdp_umem.c
index 83de74ca729a..a0607969f8c0 100644
--- a/net/xdp/xdp_umem.c
+++ b/net/xdp/xdp_umem.c
@@ -14,6 +14,7 @@
 #include <linux/netdevice.h>
 #include <linux/rtnetlink.h>
 #include <linux/idr.h>
+#include <linux/highmem.h>
 
 #include "xdp_umem.h"
 #include "xsk_queue.h"
@@ -164,6 +165,14 @@ void xdp_umem_clear_dev(struct xdp_umem *umem)
 	umem->zc = false;
 }
 
+static void xdp_umem_unmap_pages(struct xdp_umem *umem)
+{
+	unsigned int i;
+
+	for (i = 0; i < umem->npgs; i++)
+		kunmap(umem->pgs[i]);
+}
+
 static void xdp_umem_unpin_pages(struct xdp_umem *umem)
 {
 	unsigned int i;
@@ -207,6 +216,7 @@ static void xdp_umem_release(struct xdp_umem *umem)
 
 	xsk_reuseq_destroy(umem);
 
+	xdp_umem_unmap_pages(umem);
 	xdp_umem_unpin_pages(umem);
 
 	kfree(umem->pages);
@@ -369,7 +379,7 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr)
 	}
 
 	for (i = 0; i < umem->npgs; i++)
-		umem->pages[i].addr = page_address(umem->pgs[i]);
+		umem->pages[i].addr = kmap(umem->pgs[i]);
 
 	return 0;
 
-- 
2.17.1


^ permalink raw reply related

* [PATCH] pcan_usb_fd: zero out the common command buffer
From: Oliver Neukum @ 2019-08-08  9:28 UTC (permalink / raw)
  To: davem, netdev; +Cc: Oliver Neukum

Lest we leak kernel memory to a device we better zero out buffers.

Reported-by: syzbot+513e4d0985298538bf9b@syzkaller.appspotmail.com
Signed-off-by: Oliver Neukum <oneukum@suse.com>
---
 drivers/net/can/usb/peak_usb/pcan_usb_fd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
index 34761c3a6286..47cc1ff5b88e 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
@@ -841,7 +841,7 @@ static int pcan_usb_fd_init(struct peak_usb_device *dev)
 			goto err_out;
 
 		/* allocate command buffer once for all for the interface */
-		pdev->cmd_buffer_addr = kmalloc(PCAN_UFD_CMD_BUFFER_SIZE,
+		pdev->cmd_buffer_addr = kzalloc(PCAN_UFD_CMD_BUFFER_SIZE,
 						GFP_KERNEL);
 		if (!pdev->cmd_buffer_addr)
 			goto err_out_1;
-- 
2.16.4


^ permalink raw reply related

* [PATCH] zd1211rw: remove false assertion from zd_mac_clear()
From: Oliver Neukum @ 2019-08-08  9:32 UTC (permalink / raw)
  To: davem, netdev; +Cc: Oliver Neukum

The function is called before the lock which is asserted was ever used.
Just remove it.

Reported-by: syzbot+74c65761783d66a9c97c@syzkaller.appspotmail.com
Signed-off-by: Oliver Neukum <oneukum@suse.com>
---
 drivers/net/wireless/zydas/zd1211rw/zd_mac.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/net/wireless/zydas/zd1211rw/zd_mac.c b/drivers/net/wireless/zydas/zd1211rw/zd_mac.c
index da7e63fca9f5..a9999d10ae81 100644
--- a/drivers/net/wireless/zydas/zd1211rw/zd_mac.c
+++ b/drivers/net/wireless/zydas/zd1211rw/zd_mac.c
@@ -223,7 +223,6 @@ void zd_mac_clear(struct zd_mac *mac)
 {
 	flush_workqueue(zd_workqueue);
 	zd_chip_clear(&mac->chip);
-	lockdep_assert_held(&mac->lock);
 	ZD_MEMCLEAR(mac, sizeof(struct zd_mac));
 }
 
-- 
2.16.4


^ permalink raw reply related

* [PATCH net 0/2] Fix collisions in socket cookie generation
From: Daniel Borkmann @ 2019-08-08  9:49 UTC (permalink / raw)
  To: davem; +Cc: netdev, bpf, m, edumazet, ast, willemb, Daniel Borkmann

This change makes the socket cookie generator as a global counter
instead of per netns in order to fix cookie collisions for BPF use
cases we ran into. See main patch #1 for more details.

Given the change is small/trivial and fixes an issue we're seeing
my preference would be net tree (though it cleanly applies to
net-next as well). Went for net tree instead of bpf tree here given
the main change is in net/core/sock_diag.c, but either way would be
fine with me.

Thanks a lot!

Daniel Borkmann (2):
  sock: make cookie generation global instead of per netns
  bpf: sync bpf.h to tools infrastructure

 include/net/net_namespace.h    |  1 -
 include/uapi/linux/bpf.h       |  4 ++--
 net/core/sock_diag.c           |  3 ++-
 tools/include/uapi/linux/bpf.h | 11 +++++++----
 4 files changed, 11 insertions(+), 8 deletions(-)

-- 
2.17.1


^ permalink raw reply

* [PATCH net 2/2] bpf: sync bpf.h to tools infrastructure
From: Daniel Borkmann @ 2019-08-08  9:49 UTC (permalink / raw)
  To: davem; +Cc: netdev, bpf, m, edumazet, ast, willemb, Daniel Borkmann
In-Reply-To: <20190808094937.26918-1-daniel@iogearbox.net>

Pull in updates in BPF helper function description.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
 tools/include/uapi/linux/bpf.h | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 4e455018da65..a5aa7d3ac6a1 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1466,8 +1466,8 @@ union bpf_attr {
  * 		If no cookie has been set yet, generate a new cookie. Once
  * 		generated, the socket cookie remains stable for the life of the
  * 		socket. This helper can be useful for monitoring per socket
- * 		networking traffic statistics as it provides a unique socket
- * 		identifier per namespace.
+ * 		networking traffic statistics as it provides a global socket
+ * 		identifier that can be assumed unique.
  * 	Return
  * 		A 8-byte long non-decreasing number on success, or 0 if the
  * 		socket field is missing inside *skb*.
@@ -1571,8 +1571,11 @@ union bpf_attr {
  * 		but this is only implemented for native XDP (with driver
  * 		support) as of this writing).
  *
- * 		All values for *flags* are reserved for future usage, and must
- * 		be left at zero.
+ * 		The lower two bits of *flags* are used as the return code if
+ * 		the map lookup fails. This is so that the return value can be
+ * 		one of the XDP program return codes up to XDP_TX, as chosen by
+ * 		the caller. Any higher bits in the *flags* argument must be
+ * 		unset.
  *
  * 		When used to redirect packets to net devices, this helper
  * 		provides a high performance increase over **bpf_redirect**\ ().
-- 
2.17.1


^ permalink raw reply related

* [PATCH net 1/2] sock: make cookie generation global instead of per netns
From: Daniel Borkmann @ 2019-08-08  9:49 UTC (permalink / raw)
  To: davem; +Cc: netdev, bpf, m, edumazet, ast, willemb, Daniel Borkmann
In-Reply-To: <20190808094937.26918-1-daniel@iogearbox.net>

Generating and retrieving socket cookies are a useful feature that is
exposed to BPF for various program types through bpf_get_socket_cookie()
helper.

The fact that the cookie counter is per netns is quite a limitation
for BPF in practice in particular for programs in host namespace that
use socket cookies as part of a map lookup key since they will be
causing socket cookie collisions e.g. when attached to BPF cgroup hooks
or cls_bpf on tc egress in host namespace handling container traffic
from veths or ipvlan slaves. Change the counter to be global instead.

Socket cookie consumers must assume the value as opqaue in any case.
The cookie does not guarantee an always unique identifier since it
could wrap in fabricated corner cases where two sockets could end up
holding the same cookie, but is good enough to be used as a hint for
many use cases; not every socket must have a cookie generated hence
knowledge of the counter value does not provide much value either way.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Willem de Bruijn <willemb@google.com>
Cc: Martynas Pumputis <m@lambda.lt>
---
 include/net/net_namespace.h | 1 -
 include/uapi/linux/bpf.h    | 4 ++--
 net/core/sock_diag.c        | 3 ++-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index 4a9da951a794..cb668bc2692d 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -61,7 +61,6 @@ struct net {
 	spinlock_t		rules_mod_lock;
 
 	u32			hash_mix;
-	atomic64_t		cookie_gen;
 
 	struct list_head	list;		/* list of network namespaces */
 	struct list_head	exit_list;	/* To linked to call pernet exit
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index fa1c753dcdbc..a5aa7d3ac6a1 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1466,8 +1466,8 @@ union bpf_attr {
  * 		If no cookie has been set yet, generate a new cookie. Once
  * 		generated, the socket cookie remains stable for the life of the
  * 		socket. This helper can be useful for monitoring per socket
- * 		networking traffic statistics as it provides a unique socket
- * 		identifier per namespace.
+ * 		networking traffic statistics as it provides a global socket
+ * 		identifier that can be assumed unique.
  * 	Return
  * 		A 8-byte long non-decreasing number on success, or 0 if the
  * 		socket field is missing inside *skb*.
diff --git a/net/core/sock_diag.c b/net/core/sock_diag.c
index 3312a5849a97..c13ffbd33d8d 100644
--- a/net/core/sock_diag.c
+++ b/net/core/sock_diag.c
@@ -19,6 +19,7 @@ static const struct sock_diag_handler *sock_diag_handlers[AF_MAX];
 static int (*inet_rcv_compat)(struct sk_buff *skb, struct nlmsghdr *nlh);
 static DEFINE_MUTEX(sock_diag_table_mutex);
 static struct workqueue_struct *broadcast_wq;
+static atomic64_t cookie_gen;
 
 u64 sock_gen_cookie(struct sock *sk)
 {
@@ -27,7 +28,7 @@ u64 sock_gen_cookie(struct sock *sk)
 
 		if (res)
 			return res;
-		res = atomic64_inc_return(&sock_net(sk)->cookie_gen);
+		res = atomic64_inc_return(&cookie_gen);
 		atomic64_cmpxchg(&sk->sk_cookie, 0, res);
 	}
 }
-- 
2.17.1


^ permalink raw reply related

* Re: [PATCH rdma-next 0/4] Add XRQ and SRQ support to DEVX interface
From: Leon Romanovsky @ 2019-08-08 10:11 UTC (permalink / raw)
  To: Doug Ledford, Jason Gunthorpe
  Cc: RDMA mailing list, Edward Srouji, Saeed Mahameed, Yishai Hadas,
	linux-netdev
In-Reply-To: <20190808084358.29517-1-leon@kernel.org>

On Thu, Aug 08, 2019 at 11:43:54AM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
>
> Hi,
>
> This small series extends DEVX interface with SRQ and XRQ legacy commands.

Sorry for typo in cover letter, there is no SRQ here.

Thanks

>
> Thanks
>
> Yishai Hadas (4):
>   net/mlx5: Use debug message instead of warn
>   net/mlx5: Add XRQ legacy commands opcodes
>   IB/mlx5: Add legacy events to DEVX list
>   IB/mlx5: Expose XRQ legacy commands over the DEVX interface
>
>  drivers/infiniband/hw/mlx5/devx.c             | 12 ++++++++++++
>  drivers/net/ethernet/mellanox/mlx5/core/cmd.c |  4 ++++
>  drivers/net/ethernet/mellanox/mlx5/core/qp.c  |  2 +-
>  include/linux/mlx5/device.h                   |  9 +++++++++
>  include/linux/mlx5/mlx5_ifc.h                 |  2 ++
>  5 files changed, 28 insertions(+), 1 deletion(-)
>
> --
> 2.20.1
>

^ permalink raw reply

* Re: [PATCH] net/netfilter/nf_nat_proto.c - make tables static
From: Florian Westphal @ 2019-08-08 10:33 UTC (permalink / raw)
  To: Valdis Klētnieks
  Cc: Pablo Neira Ayuso, Jozsef Kadlecsik, Florian Westphal,
	netfilter-devel, coreteam, netdev, linux-kernel
In-Reply-To: <55481.1565243002@turing-police>

Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:
> Sparse warns about two tables not being declared.
> 
>   CHECK   net/netfilter/nf_nat_proto.c
> net/netfilter/nf_nat_proto.c:725:26: warning: symbol 'nf_nat_ipv4_ops' was not declared. Should it be static?
> net/netfilter/nf_nat_proto.c:964:26: warning: symbol 'nf_nat_ipv6_ops' was not declared. Should it be static?
> 
> And in fact they can indeed be static.

Acked-by: Florian Westphal <fw@strlen.de>

Seems i removed the static qualifier when i added inet nat support,
but the patch that was merged doesn't use them outside of
nf_nat_proto.c.

Thanks for fixing this.

^ permalink raw reply

* Re: [PATCH net 1/2] sock: make cookie generation global instead of per netns
From: Eric Dumazet @ 2019-08-08 10:45 UTC (permalink / raw)
  To: Daniel Borkmann
  Cc: David Miller, netdev, bpf, m, Alexei Starovoitov,
	Willem de Bruijn
In-Reply-To: <20190808094937.26918-2-daniel@iogearbox.net>

On Thu, Aug 8, 2019 at 11:50 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>

> Socket cookie consumers must assume the value as opqaue in any case.
> The cookie does not guarantee an always unique identifier since it
> could wrap in fabricated corner cases where two sockets could end up
> holding the same cookie,

What do you mean by this ?

Cookie is guaranteed to be unique, it is from a 64bit counter...

There should be no collision.

> but is good enough to be used as a hint for
> many use cases; not every socket must have a cookie generated hence
> knowledge of the counter value does not provide much value either way.
>
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> Cc: Eric Dumazet <edumazet@google.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Willem de Bruijn <willemb@google.com>
> Cc: Martynas Pumputis <m@lambda.lt>
> ---
>  include/net/net_namespace.h | 1 -
>  include/uapi/linux/bpf.h    | 4 ++--
>  net/core/sock_diag.c        | 3 ++-
>  3 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
> index 4a9da951a794..cb668bc2692d 100644
> --- a/include/net/net_namespace.h
> +++ b/include/net/net_namespace.h
> @@ -61,7 +61,6 @@ struct net {
>         spinlock_t              rules_mod_lock;
>
>         u32                     hash_mix;
> -       atomic64_t              cookie_gen;
>
>         struct list_head        list;           /* list of network namespaces */
>         struct list_head        exit_list;      /* To linked to call pernet exit
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index fa1c753dcdbc..a5aa7d3ac6a1 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -1466,8 +1466,8 @@ union bpf_attr {
>   *             If no cookie has been set yet, generate a new cookie. Once
>   *             generated, the socket cookie remains stable for the life of the
>   *             socket. This helper can be useful for monitoring per socket
> - *             networking traffic statistics as it provides a unique socket
> - *             identifier per namespace.
> + *             networking traffic statistics as it provides a global socket
> + *             identifier that can be assumed unique.
>   *     Return
>   *             A 8-byte long non-decreasing number on success, or 0 if the
>   *             socket field is missing inside *skb*.
> diff --git a/net/core/sock_diag.c b/net/core/sock_diag.c
> index 3312a5849a97..c13ffbd33d8d 100644
> --- a/net/core/sock_diag.c
> +++ b/net/core/sock_diag.c
> @@ -19,6 +19,7 @@ static const struct sock_diag_handler *sock_diag_handlers[AF_MAX];
>  static int (*inet_rcv_compat)(struct sk_buff *skb, struct nlmsghdr *nlh);
>  static DEFINE_MUTEX(sock_diag_table_mutex);
>  static struct workqueue_struct *broadcast_wq;
> +static atomic64_t cookie_gen;
>
>  u64 sock_gen_cookie(struct sock *sk)
>  {
> @@ -27,7 +28,7 @@ u64 sock_gen_cookie(struct sock *sk)
>
>                 if (res)
>                         return res;
> -               res = atomic64_inc_return(&sock_net(sk)->cookie_gen);
> +               res = atomic64_inc_return(&cookie_gen);
>                 atomic64_cmpxchg(&sk->sk_cookie, 0, res);
>         }
>  }
> --
> 2.17.1
>

^ permalink raw reply

* Re: [PATCH bpf-next] btf: expose BTF info through sysfs
From: Jiri Olsa @ 2019-08-08 10:55 UTC (permalink / raw)
  To: Andrii Nakryiko
  Cc: bpf, netdev, ast, daniel, yhs, andrii.nakryiko, kernel-team,
	Masahiro Yamada, Arnaldo Carvalho de Melo, Jiri Olsa
In-Reply-To: <20190807183821.138728-1-andriin@fb.com>

On Wed, Aug 07, 2019 at 11:38:21AM -0700, Andrii Nakryiko wrote:
> Make .BTF section allocated and expose its contents through sysfs.
> 
> /sys/kernel/btf directory is created to contain all the BTFs present
> inside kernel. Currently there is only kernel's main BTF, represented as
> /sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
> each module will expose its BTF as /sys/kernel/btf/<module-name> file.
> 
> Current approach relies on a few pieces coming together:
> 1. pahole is used to take almost final vmlinux image (modulo .BTF and
>    kallsyms) and generate .BTF section by converting DWARF info into
>    BTF. This section is not allocated and not mapped to any segment,
>    though, so is not yet accessible from inside kernel at runtime.
> 2. objcopy dumps .BTF contents into binary file and subsequently
>    convert binary file into linkable object file with automatically
>    generated symbols _binary__btf_kernel_bin_start and
>    _binary__btf_kernel_bin_end, pointing to start and end, respectively,
>    of BTF raw data.
> 3. final vmlinux image is generated by linking this object file (and
>    kallsyms, if necessary). sysfs_btf.c then creates
>    /sys/kernel/btf/kernel file and exposes embedded BTF contents through
>    it. This allows, e.g., libbpf and bpftool access BTF info at
>    well-known location, without resorting to searching for vmlinux image
>    on disk (location of which is not standardized and vmlinux image
>    might not be even available in some scenarios, e.g., inside qemu
>    during testing).
> 
> Alternative approach using .incbin assembler directive to embed BTF
> contents directly was attempted but didn't work, because sysfs_proc.o is
> not re-compiled during link-vmlinux.sh stage. This is required, though,
> to update embedded BTF data (initially empty data is embedded, then
> pahole generates BTF info and we need to regenerate sysfs_btf.o with
> updated contents, but it's too late at that point).
> 
> If BTF couldn't be generated due to missing or too old pahole,
> sysfs_btf.c handles that gracefully by detecting that
> _binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
> /sys/kernel/btf at all.
> 
> Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Signed-off-by: Andrii Nakryiko <andriin@fb.com>
> ---
>  kernel/bpf/Makefile     |  3 +++
>  kernel/bpf/sysfs_btf.c  | 52 ++++++++++++++++++++++++++++++++++++++++
>  scripts/link-vmlinux.sh | 53 ++++++++++++++++++++++++++---------------
>  3 files changed, 89 insertions(+), 19 deletions(-)
>  create mode 100644 kernel/bpf/sysfs_btf.c
> 
> diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile
> index 29d781061cd5..e1d9adb212f9 100644
> --- a/kernel/bpf/Makefile
> +++ b/kernel/bpf/Makefile
> @@ -22,3 +22,6 @@ obj-$(CONFIG_CGROUP_BPF) += cgroup.o
>  ifeq ($(CONFIG_INET),y)
>  obj-$(CONFIG_BPF_SYSCALL) += reuseport_array.o
>  endif
> +ifeq ($(CONFIG_SYSFS),y)
> +obj-$(CONFIG_DEBUG_INFO_BTF) += sysfs_btf.o
> +endif
> diff --git a/kernel/bpf/sysfs_btf.c b/kernel/bpf/sysfs_btf.c
> new file mode 100644
> index 000000000000..ac06ce1d62e8
> --- /dev/null
> +++ b/kernel/bpf/sysfs_btf.c
> @@ -0,0 +1,52 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Provide kernel BTF information for introspection and use by eBPF tools.
> + */
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/kobject.h>
> +#include <linux/init.h>
> +
> +/* See scripts/link-vmlinux.sh, gen_btf() func for details */
> +extern char __weak _binary__btf_kernel_bin_start[];
> +extern char __weak _binary__btf_kernel_bin_end[];
> +
> +static ssize_t
> +btf_kernel_read(struct file *file, struct kobject *kobj,
> +		struct bin_attribute *bin_attr,
> +		char *buf, loff_t off, size_t len)
> +{
> +	memcpy(buf, _binary__btf_kernel_bin_start + off, len);

hum, should you check the end of the btf data?
maybe use the memory_read_from_buffer function instead

jirka

^ permalink raw reply

* Re: [PATCH bpf-next] btf: expose BTF info through sysfs
From: Jiri Olsa @ 2019-08-08 10:59 UTC (permalink / raw)
  To: Andrii Nakryiko
  Cc: bpf, netdev, ast, daniel, yhs, andrii.nakryiko, kernel-team,
	Masahiro Yamada, Arnaldo Carvalho de Melo, Jiri Olsa
In-Reply-To: <20190808105558.GB31775@krava>

On Thu, Aug 08, 2019 at 12:56:01PM +0200, Jiri Olsa wrote:
> On Wed, Aug 07, 2019 at 11:38:21AM -0700, Andrii Nakryiko wrote:
> > Make .BTF section allocated and expose its contents through sysfs.
> > 
> > /sys/kernel/btf directory is created to contain all the BTFs present
> > inside kernel. Currently there is only kernel's main BTF, represented as
> > /sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
> > each module will expose its BTF as /sys/kernel/btf/<module-name> file.
> > 
> > Current approach relies on a few pieces coming together:
> > 1. pahole is used to take almost final vmlinux image (modulo .BTF and
> >    kallsyms) and generate .BTF section by converting DWARF info into
> >    BTF. This section is not allocated and not mapped to any segment,
> >    though, so is not yet accessible from inside kernel at runtime.
> > 2. objcopy dumps .BTF contents into binary file and subsequently
> >    convert binary file into linkable object file with automatically
> >    generated symbols _binary__btf_kernel_bin_start and
> >    _binary__btf_kernel_bin_end, pointing to start and end, respectively,
> >    of BTF raw data.
> > 3. final vmlinux image is generated by linking this object file (and
> >    kallsyms, if necessary). sysfs_btf.c then creates
> >    /sys/kernel/btf/kernel file and exposes embedded BTF contents through
> >    it. This allows, e.g., libbpf and bpftool access BTF info at
> >    well-known location, without resorting to searching for vmlinux image
> >    on disk (location of which is not standardized and vmlinux image
> >    might not be even available in some scenarios, e.g., inside qemu
> >    during testing).
> > 
> > Alternative approach using .incbin assembler directive to embed BTF
> > contents directly was attempted but didn't work, because sysfs_proc.o is
> > not re-compiled during link-vmlinux.sh stage. This is required, though,
> > to update embedded BTF data (initially empty data is embedded, then
> > pahole generates BTF info and we need to regenerate sysfs_btf.o with
> > updated contents, but it's too late at that point).
> > 
> > If BTF couldn't be generated due to missing or too old pahole,
> > sysfs_btf.c handles that gracefully by detecting that
> > _binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
> > /sys/kernel/btf at all.
> > 
> > Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> > Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> > Cc: Jiri Olsa <jolsa@kernel.org>
> > Signed-off-by: Andrii Nakryiko <andriin@fb.com>
> > ---
> >  kernel/bpf/Makefile     |  3 +++
> >  kernel/bpf/sysfs_btf.c  | 52 ++++++++++++++++++++++++++++++++++++++++
> >  scripts/link-vmlinux.sh | 53 ++++++++++++++++++++++++++---------------
> >  3 files changed, 89 insertions(+), 19 deletions(-)
> >  create mode 100644 kernel/bpf/sysfs_btf.c
> > 
> > diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile
> > index 29d781061cd5..e1d9adb212f9 100644
> > --- a/kernel/bpf/Makefile
> > +++ b/kernel/bpf/Makefile
> > @@ -22,3 +22,6 @@ obj-$(CONFIG_CGROUP_BPF) += cgroup.o
> >  ifeq ($(CONFIG_INET),y)
> >  obj-$(CONFIG_BPF_SYSCALL) += reuseport_array.o
> >  endif
> > +ifeq ($(CONFIG_SYSFS),y)
> > +obj-$(CONFIG_DEBUG_INFO_BTF) += sysfs_btf.o
> > +endif
> > diff --git a/kernel/bpf/sysfs_btf.c b/kernel/bpf/sysfs_btf.c
> > new file mode 100644
> > index 000000000000..ac06ce1d62e8
> > --- /dev/null
> > +++ b/kernel/bpf/sysfs_btf.c
> > @@ -0,0 +1,52 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Provide kernel BTF information for introspection and use by eBPF tools.
> > + */
> > +#include <linux/kernel.h>
> > +#include <linux/module.h>
> > +#include <linux/kobject.h>
> > +#include <linux/init.h>
> > +
> > +/* See scripts/link-vmlinux.sh, gen_btf() func for details */
> > +extern char __weak _binary__btf_kernel_bin_start[];
> > +extern char __weak _binary__btf_kernel_bin_end[];
> > +
> > +static ssize_t
> > +btf_kernel_read(struct file *file, struct kobject *kobj,
> > +		struct bin_attribute *bin_attr,
> > +		char *buf, loff_t off, size_t len)
> > +{
> > +	memcpy(buf, _binary__btf_kernel_bin_start + off, len);
> 
> hum, should you check the end of the btf data?
> maybe use the memory_read_from_buffer function instead

nah, looks like that size settings will do that for you
in sysfs_kf_bin_read, nice.. nevermind then ;-)

jirka

^ permalink raw reply

* Re: [PATCH net 1/2] sock: make cookie generation global instead of per netns
From: Daniel Borkmann @ 2019-08-08 11:09 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: David Miller, netdev, bpf, m, Alexei Starovoitov,
	Willem de Bruijn
In-Reply-To: <CANn89iKzaxxyC=6s45PEnTsKfz7GN4HHOw3wtpb6-ozrJSRP=g@mail.gmail.com>

On 8/8/19 12:45 PM, Eric Dumazet wrote:
> On Thu, Aug 8, 2019 at 11:50 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> 
>> Socket cookie consumers must assume the value as opqaue in any case.
>> The cookie does not guarantee an always unique identifier since it
>> could wrap in fabricated corner cases where two sockets could end up
>> holding the same cookie,
> 
> What do you mean by this ?
> 
> Cookie is guaranteed to be unique, it is from a 64bit counter...
> 
> There should be no collision.

I meant the [theoretical] corner case where socket_1 has cookie X and
we'd create, trigger sock_gen_cookie() to increment, close socket in a
loop until we wrap and get another cookie X for socket_2; agree it's
impractical and for little gain anyway. So in practice there should be
no collision which is what I tried to say.

Thanks,
Daniel

^ permalink raw reply

* Re: [PATCH net-next v1 1/8] netfilter: inlined four headers files into another one.
From: Pablo Neira Ayuso @ 2019-08-08 11:23 UTC (permalink / raw)
  To: Jeremy Sowden; +Cc: Netfilter Devel, Net Dev, Masahiro Yamada, kadlec
In-Reply-To: <20190807141705.4864-2-jeremy@azazel.net>

Hi Jeremy,

Thanks for working on this.

Cc'ing Jozsef.

On Wed, Aug 07, 2019 at 03:16:58PM +0100, Jeremy Sowden wrote:
[...]
> +/* Called from uadd only, protected by the set spinlock.
> + * The kadt functions don't use the comment extensions in any way.
> + */
> +static inline void
> +ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment,
> +		    const struct ip_set_ext *ext)

Not related to this patch, but I think the number of inline functions
could be reduced a bit by exporting symbols? Specifically for
functions that are called from the netlink control plane, ie. _uadd()
functions. I think forcing the compiler to inline this is not useful.
This could be done in a follow up patchset.

Thanks.

^ permalink raw reply

* Re: [PATCH] net: ethernet: et131x: Use GFP_KERNEL instead of GFP_ATOMIC when allocating tx_ring->tcb_ring
From: Matthew Wilcox @ 2019-08-08 11:24 UTC (permalink / raw)
  To: Jesse Brandeburg
  Cc: Christophe JAILLET, mark.einon, davem, f.fainelli, andrew, netdev,
	linux-kernel, kernel-janitors
In-Reply-To: <20190807222346.00002ba7@intel.com>

On Wed, Aug 07, 2019 at 10:23:46PM -0700, Jesse Brandeburg wrote:
> On Wed, 31 Jul 2019 09:38:42 +0200
> Christophe JAILLET <christophe.jaillet@wanadoo.fr> wrote:
> 
> > There is no good reason to use GFP_ATOMIC here. Other memory allocations
> > are performed with GFP_KERNEL (see other 'dma_alloc_coherent()' below and
> > 'kzalloc()' in 'et131x_rx_dma_memory_alloc()')
> > 
> > Use GFP_KERNEL which should be enough.
> > 
> > Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
> 
> Sure, but generally I'd say GFP_ATOMIC is ok if you're in an init path
> and you can afford to have the allocation thread sleep while memory is
> being found by the kernel.

That's not what GFP_ATOMIC means.  GFP_ATOMIC _will not_ sleep.  GFP_KERNEL
will.

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox