Netdev List
 help / color / mirror / Atom feed
* [PATCH net 2/3] tls: clear key material from kernel memory when do_tls_setsockopt_conf fails
From: Sabrina Dubroca @ 2018-09-05 13:21 UTC (permalink / raw)
  To: netdev
  Cc: Sabrina Dubroca, Boris Pismenny, Ilya Lesokhin, Aviad Yehezkel,
	Dave Watson
In-Reply-To: <cover.1536152698.git.sd@queasysnail.net>

Fixes: 3c4d7559159b ("tls: kernel TLS support")
Signed-off-by: Sabrina Dubroca <sd@queasysnail.net>
---
 net/tls/tls_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
index 180b6640e531..0d432d025471 100644
--- a/net/tls/tls_main.c
+++ b/net/tls/tls_main.c
@@ -499,7 +499,7 @@ static int do_tls_setsockopt_conf(struct sock *sk, char __user *optval,
 	goto out;
 
 err_crypto_info:
-	memset(crypto_info, 0, sizeof(*crypto_info));
+	memzero_explicit(crypto_info, sizeof(struct tls12_crypto_info_aes_gcm_128));
 out:
 	return rc;
 }
-- 
2.18.0

^ permalink raw reply related

* [PATCH net 3/3] tls: zero the crypto information from tls_context before freeing
From: Sabrina Dubroca @ 2018-09-05 13:21 UTC (permalink / raw)
  To: netdev
  Cc: Sabrina Dubroca, Boris Pismenny, Ilya Lesokhin, Aviad Yehezkel,
	Dave Watson
In-Reply-To: <cover.1536152698.git.sd@queasysnail.net>

This contains key material in crypto_send_aes_gcm_128 and
crypto_recv_aes_gcm_128.

Fixes: 3c4d7559159b ("tls: kernel TLS support")
Signed-off-by: Sabrina Dubroca <sd@queasysnail.net>
---
 include/net/tls.h  |  1 +
 net/tls/tls_main.c | 14 ++++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/include/net/tls.h b/include/net/tls.h
index d5c683e8bb22..2010d23112f9 100644
--- a/include/net/tls.h
+++ b/include/net/tls.h
@@ -180,6 +180,7 @@ struct tls_context {
 		struct tls_crypto_info crypto_recv;
 		struct tls12_crypto_info_aes_gcm_128 crypto_recv_aes_gcm_128;
 	};
+	char tls_crypto_ctx_end[0];
 
 	struct list_head list;
 	struct net_device *netdev;
diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
index 0d432d025471..d3a57c0b2182 100644
--- a/net/tls/tls_main.c
+++ b/net/tls/tls_main.c
@@ -241,6 +241,16 @@ static void tls_write_space(struct sock *sk)
 	ctx->sk_write_space(sk);
 }
 
+static void tls_ctx_free(struct tls_context *ctx)
+{
+	if (!ctx)
+		return;
+
+	memzero_explicit(&ctx->crypto_send,
+			 offsetof(struct tls_context, tls_crypto_ctx_end));
+	kfree(ctx);
+}
+
 static void tls_sk_proto_close(struct sock *sk, long timeout)
 {
 	struct tls_context *ctx = tls_get_ctx(sk);
@@ -294,7 +304,7 @@ static void tls_sk_proto_close(struct sock *sk, long timeout)
 #else
 	{
 #endif
-		kfree(ctx);
+		tls_ctx_free(ctx);
 		ctx = NULL;
 	}
 
@@ -305,7 +315,7 @@ static void tls_sk_proto_close(struct sock *sk, long timeout)
 	 * for sk->sk_prot->unhash [tls_hw_unhash]
 	 */
 	if (free_ctx)
-		kfree(ctx);
+		tls_ctx_free(ctx);
 }
 
 static int do_tls_getsockopt_tx(struct sock *sk, char __user *optval,
-- 
2.18.0

^ permalink raw reply related

* Re: [oss-drivers] Re: [RFC bpf-next PATCH] samples/bpf: xdp1 add XDP hardware offload option
From: Jesper Dangaard Brouer @ 2018-09-05 13:41 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: Nick Viljoen, oss-drivers, netdev, John W. Linville, jhsiao,
	Quentin Monnet, brouer
In-Reply-To: <20180904185957.0d344534@cakuba>

On Tue, 4 Sep 2018 18:59:57 +0200
Jakub Kicinski <jakub.kicinski@netronome.com> wrote:

> On Tue, 4 Sep 2018 18:49:33 +0200, Jesper Dangaard Brouer wrote:
> > On Tue, 4 Sep 2018 17:09:12 +0200
> > Jakub Kicinski <jakub.kicinski@netronome.com> wrote:
> >   
> > > On Tue, 04 Sep 2018 16:59:19 +0200, Jesper Dangaard Brouer wrote:    
> > > > Trying to use XDP hardware offloading via XDP_FLAGS_HW_MODE
> > > > and setting the ifindex in prog_load_attr.ifindex before
> > > > loading the BPF code via bpf_prog_load_xattr().
> > > > 
> > > > This unfortunately does not seem to work...
> > > > - Am I doing something wrong?
> > > > 
> > > > Notice, I also disable the map BPF_MAP_TYPE_PERCPU_ARRAY
> > > > to make sure it was not related to the map (not supporting
> > > > offloading).
> > > > 
> > > > Failed with:
> > > >  # ./xdp1 -O $(</sys/class/net/enp130s0np1/ifindex)
> > > >  libbpf: load bpf program failed: Invalid argument
> > > >  libbpf: failed to load program 'xdp1'
> > > >  libbpf: failed to load object './xdp1_kern.o'
> > > > 
> > > > Tested on kernel 4.18.0-2.el8.x86_64 with driver nfp
> > > >  Ethernet controller: Netronome Systems, Inc. Device 4000      
> > > 
> > > Are you running the BPF capable FW?
> > > 
> > > https://help.netronome.com/support/solutions/articles/36000050009-agilio-ebpf-2-0-6-extended-berkeley-packet-filter    
> > 
> > I'm likely not running the correct firmware...
> > 
> > Can you tell me, with the ethtool -i output, if I'm running the
> > appropriate firmware?
> > 
> > # ethtool -i enp129s0np1
> > driver: nfp
> > version: 4.18.0-2.el8.x86_64 SMP mod_unl
> > firmware-version: 0.0.3.5 0.21 nic-2.0.7 nic
> > expansion-rom-version: 
> > bus-info: 0000:81:00.0
> > supports-statistics: yes
> > supports-test: no
> > supports-eeprom-access: no
> > supports-register-dump: yes
> > supports-priv-flags: no  
> 
> Yup, the BPF firmware says bpf in firmware version.

Downloaded: agilio-bpf-firmware-2.0.6.121-1.noarch.rpm

RPM install it:
 rpm -hiv  agilio-bpf-firmware-2.0.6.121-1.noarch.rpm

# rpm -ql agilio-bpf-firmware
/opt/netronome/firmware/agilio-bpf
/opt/netronome/firmware/agilio-bpf/nic_AMDA0058-0011_2x40.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0058-0012_2x40.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0078-0011_1x100.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0081-0001_1x40.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0081-0001_4x10.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0096-0001_2x10.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0097-0001_2x40.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0097-0001_4x10_1x40.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0097-0001_8x10.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0099-0001_1x10_1x25.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0099-0001_2x10.nffw
/opt/netronome/firmware/agilio-bpf/nic_AMDA0099-0001_2x25.nffw

Netronome: Basic Firmware User Guide
 https://help.netronome.com/support/solutions/articles/36000049975-basic-firmware-user-guide

A section says after installing the firmware, unload and reload the
driver kernel module will upgrade the firmware.

 ## reload driver to load new firmware
 rmmod nfp; modprobe nfp

Firmware upgrade worked! :-) This is by-far the easiest firmware
upgrade I've experienced.  And I don't have to reboot the machine :-)

 # ethtool -i enp130s0np1  | grep firmware-version
 firmware-version: 0.0.3.5 0.21 bpf-2.0.6.121 ebpf

Notice the bpf/ebpf strings in firmware-version.

 
> > If this is a firmware version case, then we should really improve the
> > errors we are giving the user, the -EINVAL can be anything.
> > 
> >  "libbpf: load bpf program failed: Invalid argument"  
> 
> That is true.

Same goes for improving the error message, when loading a map type that
offload cannot handle.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer

^ permalink raw reply

* Re: [PATCH v3 2/3] IB/ipoib: Use dev_port to expose network interface port numbers
From: Leon Romanovsky @ 2018-09-05 13:41 UTC (permalink / raw)
  To: Arseny Maslennikov; +Cc: linux-rdma, Doug Ledford, Jason Gunthorpe, netdev
In-Reply-To: <20180903161316.25121-3-ar@cs.msu.ru>

[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]

On Mon, Sep 03, 2018 at 07:13:15PM +0300, Arseny Maslennikov wrote:
> Some InfiniBand network devices have multiple ports on the same PCI
> function. This initializes the `dev_port' sysfs field of those
> network interfaces with their port number.
>
> Prior to this the kernel erroneously used the `dev_id' sysfs
> field of those network interfaces to convey the port number to userspace.
>
> The use of `dev_id' was considered correct until Linux 3.15,
> when another field, `dev_port', was defined for this particular
> purpose and `dev_id' was reserved for distinguishing stacked ifaces
> (e.g: VLANs) with the same hardware address as their parent device.
>
> Similar fixes to net/mlx4_en and many other drivers, which started
> exporting this information through `dev_id' before 3.15, were accepted
> into the kernel 4 years ago.
> See 76a066f2a2a0 (`net/mlx4_en: Expose port number through sysfs').
>
> Signed-off-by: Arseny Maslennikov <ar@cs.msu.ru>
> ---
>  drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 ++
>  1 file changed, 2 insertions(+)
>

Thanks,
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

^ permalink raw reply

* Re: [PATCH bpf-next] bpf/verifier: properly clear union members after a ctx read
From: Edward Cree @ 2018-09-05 13:47 UTC (permalink / raw)
  To: Alexei Starovoitov; +Cc: daniel, ast, netdev
In-Reply-To: <20180905022323.6lkmq2kmv5ejwy3c@ast-mbp.dhcp.thefacebook.com>

On 05/09/18 03:23, Alexei Starovoitov wrote:
> So would you agree it's fair to add
> Fixes: f1174f77b50c ("bpf/verifier: rework value tracking")
> ?
Sure.  Though I don't think it needs backporting, as it's a conservative
 bug (i.e. it merely prevents pruning, but that's safe security-wise).
> How about patch like the following:
> ------------
> From 422fd975ed78645ab67d2eb50ff6e1ff6fb3de32 Mon Sep 17 00:00:00 2001
> From: Alexei Starovoitov <ast@kernel.org>
> Date: Tue, 4 Sep 2018 19:13:44 -0700
> Subject: [PATCH] bpf/verifier: fix verifier instability
>
> Fixes: f1174f77b50c ("bpf/verifier: rework value tracking")
> Debugged-by: Edward Cree <ecree@solarflare.com> 
> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Edward Cree <ecree@solarflare.com>

^ permalink raw reply

* RE: [PATCH net 3/3] tls: zero the crypto information from tls_context before freeing
From: Vakul Garg @ 2018-09-05 13:48 UTC (permalink / raw)
  To: Sabrina Dubroca, netdev@vger.kernel.org
  Cc: Boris Pismenny, Ilya Lesokhin, Aviad Yehezkel, Dave Watson
In-Reply-To: <c2b629a06486cd9c17a42cf2efe11ad26d8f9ef6.1536152698.git.sd@queasysnail.net>



> -----Original Message-----
> From: netdev-owner@vger.kernel.org <netdev-owner@vger.kernel.org> On
> Behalf Of Sabrina Dubroca
> Sent: Wednesday, September 5, 2018 6:52 PM
> To: netdev@vger.kernel.org
> Cc: Sabrina Dubroca <sd@queasysnail.net>; Boris Pismenny
> <borisp@mellanox.com>; Ilya Lesokhin <ilyal@mellanox.com>; Aviad
> Yehezkel <aviadye@mellanox.com>; Dave Watson <davejwatson@fb.com>
> Subject: [PATCH net 3/3] tls: zero the crypto information from tls_context
> before freeing
> 
> This contains key material in crypto_send_aes_gcm_128 and
> crypto_recv_aes_gcm_128.
> 
> Fixes: 3c4d7559159b ("tls: kernel TLS support")
> Signed-off-by: Sabrina Dubroca <sd@queasysnail.net>
> ---
>  include/net/tls.h  |  1 +
>  net/tls/tls_main.c | 14 ++++++++++++--
>  2 files changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/include/net/tls.h b/include/net/tls.h index
> d5c683e8bb22..2010d23112f9 100644
> --- a/include/net/tls.h
> +++ b/include/net/tls.h
> @@ -180,6 +180,7 @@ struct tls_context {
>  		struct tls_crypto_info crypto_recv;
>  		struct tls12_crypto_info_aes_gcm_128
> crypto_recv_aes_gcm_128;
>  	};
> +	char tls_crypto_ctx_end[0];

This makes the key zeroization dependent upon the position of unions in structure.
Can you zeroise the crypto_send, crypto_recv separately using two memzero_explicit calls? 

> 
>  	struct list_head list;
>  	struct net_device *netdev;
> diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index
> 0d432d025471..d3a57c0b2182 100644
> --- a/net/tls/tls_main.c
> +++ b/net/tls/tls_main.c
> @@ -241,6 +241,16 @@ static void tls_write_space(struct sock *sk)
>  	ctx->sk_write_space(sk);
>  }
> 
> +static void tls_ctx_free(struct tls_context *ctx) {
> +	if (!ctx)
> +		return;
> +
> +	memzero_explicit(&ctx->crypto_send,
> +			 offsetof(struct tls_context, tls_crypto_ctx_end));
> +	kfree(ctx);
> +}
> +
>  static void tls_sk_proto_close(struct sock *sk, long timeout)  {
>  	struct tls_context *ctx = tls_get_ctx(sk); @@ -294,7 +304,7 @@
> static void tls_sk_proto_close(struct sock *sk, long timeout)  #else
>  	{
>  #endif
> -		kfree(ctx);
> +		tls_ctx_free(ctx);
>  		ctx = NULL;
>  	}
> 
> @@ -305,7 +315,7 @@ static void tls_sk_proto_close(struct sock *sk, long
> timeout)
>  	 * for sk->sk_prot->unhash [tls_hw_unhash]
>  	 */
>  	if (free_ctx)
> -		kfree(ctx);
> +		tls_ctx_free(ctx);
>  }
> 
>  static int do_tls_getsockopt_tx(struct sock *sk, char __user *optval,
> --
> 2.18.0

^ permalink raw reply

* Re: [PATCH v3 3/3] IB/ipoib: Log sysfs 'dev_id' accesses from userspace
From: Leon Romanovsky @ 2018-09-05 13:50 UTC (permalink / raw)
  To: Arseny Maslennikov; +Cc: linux-rdma, Doug Ledford, Jason Gunthorpe, netdev
In-Reply-To: <20180903161316.25121-4-ar@cs.msu.ru>

[-- Attachment #1: Type: text/plain, Size: 2324 bytes --]

On Mon, Sep 03, 2018 at 07:13:16PM +0300, Arseny Maslennikov wrote:
> Signed-off-by: Arseny Maslennikov <ar@cs.msu.ru>
> ---
>  drivers/infiniband/ulp/ipoib/ipoib_main.c | 38 +++++++++++++++++++++++
>  1 file changed, 38 insertions(+)
>
> diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
> index 30f840f874b3..7386e5bde3d3 100644
> --- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
> +++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
> @@ -2386,6 +2386,42 @@ int ipoib_add_pkey_attr(struct net_device *dev)
>  	return device_create_file(&dev->dev, &dev_attr_pkey);
>  }
>
> +/*
> + * We erroneously exposed the iface's port number in the dev_id
> + * sysfs field long after dev_port was introduced for that purpose[1],
> + * and we need to stop everyone from relying on that.
> + * Let's overload the shower routine for the dev_id file here
> + * to gently bring the issue up.
> + *
> + * [1] https://www.spinics.net/lists/netdev/msg272123.html
> + */
> +static ssize_t dev_id_show(struct device *dev,
> +			   struct device_attribute *attr, char *buf)
> +{
> +	struct net_device *ndev = to_net_dev(dev);
> +	ssize_t ret = -EINVAL;
> +
> +	if (ndev->dev_id == ndev->dev_port) {
> +		netdev_info_once(ndev,
> +			"\"%s\" wants to know my dev_id. "
> +			"Should it look at dev_port instead?\n",
> +			current->comm);
> +		netdev_info_once(ndev,
> +			"See Documentation/ABI/testing/sysfs-class-net for more info.\n");
> +	}
> +
> +	ret = sprintf(buf, "%#x\n", ndev->dev_id);
> +
> +	return ret;
> +}
> +static DEVICE_ATTR_RO(dev_id);
> +

I don't see this field among exposed by IPoIB, why should we expose it now?

> +int ipoib_intercept_dev_id_attr(struct net_device *dev)
> +{
> +	device_remove_file(&dev->dev, &dev_attr_dev_id);
> +	return device_create_file(&dev->dev, &dev_attr_dev_id);

Why isn't enough to rely on netdev code?

> +}
> +
>  static struct net_device *ipoib_add_port(const char *format,
>  					 struct ib_device *hca, u8 port)
>  {
> @@ -2427,6 +2463,8 @@ static struct net_device *ipoib_add_port(const char *format,
>  	 */
>  	ndev->priv_destructor = ipoib_intf_free;
>
> +	if (ipoib_intercept_dev_id_attr(ndev))
> +		goto sysfs_failed;
>  	if (ipoib_cm_add_mode_attr(ndev))
>  		goto sysfs_failed;
>  	if (ipoib_add_pkey_attr(ndev))
> --
> 2.19.0.rc1
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

^ permalink raw reply

* Re: [PATCH net 2/3] tls: clear key material from kernel memory when do_tls_setsockopt_conf fails
From: Boris Pismenny @ 2018-09-05 13:53 UTC (permalink / raw)
  To: Sabrina Dubroca, netdev; +Cc: Ilya Lesokhin, Aviad Yehezkel, Dave Watson
In-Reply-To: <695fc3449c2ba927e97abd48b452a31ec73d6c61.1536152698.git.sd@queasysnail.net>

Hi Sabrina,

On 9/5/2018 4:21 PM, Sabrina Dubroca wrote:
> Fixes: 3c4d7559159b ("tls: kernel TLS support")
> Signed-off-by: Sabrina Dubroca <sd@queasysnail.net>
> ---
>   net/tls/tls_main.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
> index 180b6640e531..0d432d025471 100644
> --- a/net/tls/tls_main.c
> +++ b/net/tls/tls_main.c
> @@ -499,7 +499,7 @@ static int do_tls_setsockopt_conf(struct sock *sk, char __user *optval,
>   	goto out;
>   
>   err_crypto_info:
> -	memset(crypto_info, 0, sizeof(*crypto_info));
> +	memzero_explicit(crypto_info, sizeof(struct tls12_crypto_info_aes_gcm_128));

Besides the key, there are other (not secret) information in 
tls12_crypto_info_aes_gcm_128. I'd prefer you do not delete it to enable 
users to obtain it (using getsockopt) in case we decide to implement a 
fallback to userspace in the future. Such a fallback must obtain the 
kernel's iv, and record sequence number.

Thanks,
Boris.

^ permalink raw reply

* Re: pull-request: mac80211-next 2018-09-05
From: David Miller @ 2018-09-05 14:49 UTC (permalink / raw)
  To: johannes; +Cc: netdev, linux-wireless
In-Reply-To: <20180905090542.21790-1-johannes@sipsolutions.net>

From: Johannes Berg <johannes@sipsolutions.net>
Date: Wed,  5 Sep 2018 11:05:41 +0200

> Here's a pull request for net-next. Thanks for including
> net, that way I could include Stanislaw's patch to check
> the regulatory WMM data.
> 
> Please pull and let me know if there's any problem.

Pulled, thanks!

^ permalink raw reply

* [PATCH net 0/3] net/iucv: fixes 2018-09-05
From: Julian Wiedmann @ 2018-09-05 14:55 UTC (permalink / raw)
  To: David Miller
  Cc: netdev, linux-s390, Martin Schwidefsky, Heiko Carstens,
	Stefan Raspl, Ursula Braun, Julian Wiedmann

Hi Dave,

please apply three straight-forward fixes for iucv. One that prevents
leaking the skb on malformed inbound packets, one to fix the error
handling on transmit error, and one to get rid of a compile warning.

Thanks,
Julian


Julian Wiedmann (3):
  net/af_iucv: drop inbound packets with invalid flags
  net/af_iucv: fix skb handling on HiperTransport xmit error
  net/iucv: declare iucv_path_table_empty() as static

 net/iucv/af_iucv.c | 38 +++++++++++++++++++++++++-------------
 net/iucv/iucv.c    |  2 +-
 2 files changed, 26 insertions(+), 14 deletions(-)

-- 
2.16.4

^ permalink raw reply

* [PATCH net 1/3] net/af_iucv: drop inbound packets with invalid flags
From: Julian Wiedmann @ 2018-09-05 14:55 UTC (permalink / raw)
  To: David Miller
  Cc: netdev, linux-s390, Martin Schwidefsky, Heiko Carstens,
	Stefan Raspl, Ursula Braun, Julian Wiedmann
In-Reply-To: <20180905145512.30165-1-jwi@linux.ibm.com>

Inbound packets may have any combination of flag bits set in their iucv
header. If we don't know how to handle a specific combination, drop the
skb instead of leaking it.

To clarify what error is returned in this case, replace the hard-coded
0 with the corresponding macro.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
---
 net/iucv/af_iucv.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/iucv/af_iucv.c b/net/iucv/af_iucv.c
index a21d8ed0a325..01000c14417f 100644
--- a/net/iucv/af_iucv.c
+++ b/net/iucv/af_iucv.c
@@ -2155,8 +2155,8 @@ static int afiucv_hs_rcv(struct sk_buff *skb, struct net_device *dev,
 	struct sock *sk;
 	struct iucv_sock *iucv;
 	struct af_iucv_trans_hdr *trans_hdr;
+	int err = NET_RX_SUCCESS;
 	char nullstring[8];
-	int err = 0;
 
 	if (skb->len < (ETH_HLEN + sizeof(struct af_iucv_trans_hdr))) {
 		WARN_ONCE(1, "AF_IUCV too short skb, len=%d, min=%d",
@@ -2254,7 +2254,7 @@ static int afiucv_hs_rcv(struct sk_buff *skb, struct net_device *dev,
 		err = afiucv_hs_callback_rx(sk, skb);
 		break;
 	default:
-		;
+		kfree_skb(skb);
 	}
 
 	return err;
-- 
2.16.4

^ permalink raw reply related

* [PATCH net 2/3] net/af_iucv: fix skb handling on HiperTransport xmit error
From: Julian Wiedmann @ 2018-09-05 14:55 UTC (permalink / raw)
  To: David Miller
  Cc: netdev, linux-s390, Martin Schwidefsky, Heiko Carstens,
	Stefan Raspl, Ursula Braun, Julian Wiedmann
In-Reply-To: <20180905145512.30165-1-jwi@linux.ibm.com>

When sending an skb, afiucv_hs_send() bails out on various error
conditions. But currently the caller has no way of telling whether the
skb was freed or not - resulting in potentially either
a) leaked skbs from iucv_send_ctrl(), or
b) double-free's from iucv_sock_sendmsg().

As dev_queue_xmit() will always consume the skb (even on error), be
consistent and also free the skb from all other error paths. This way
callers no longer need to care about managing the skb.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: Ursula Braun <ubraun@linux.ibm.com>
---
 net/iucv/af_iucv.c | 34 +++++++++++++++++++++++-----------
 1 file changed, 23 insertions(+), 11 deletions(-)

diff --git a/net/iucv/af_iucv.c b/net/iucv/af_iucv.c
index 01000c14417f..e2f16a0173a9 100644
--- a/net/iucv/af_iucv.c
+++ b/net/iucv/af_iucv.c
@@ -351,20 +351,28 @@ static int afiucv_hs_send(struct iucv_message *imsg, struct sock *sock,
 		memcpy(&phs_hdr->iucv_hdr, imsg, sizeof(struct iucv_message));
 
 	skb->dev = iucv->hs_dev;
-	if (!skb->dev)
-		return -ENODEV;
-	if (!(skb->dev->flags & IFF_UP) || !netif_carrier_ok(skb->dev))
-		return -ENETDOWN;
+	if (!skb->dev) {
+		err = -ENODEV;
+		goto err_free;
+	}
+	if (!(skb->dev->flags & IFF_UP) || !netif_carrier_ok(skb->dev)) {
+		err = -ENETDOWN;
+		goto err_free;
+	}
 	if (skb->len > skb->dev->mtu) {
-		if (sock->sk_type == SOCK_SEQPACKET)
-			return -EMSGSIZE;
-		else
-			skb_trim(skb, skb->dev->mtu);
+		if (sock->sk_type == SOCK_SEQPACKET) {
+			err = -EMSGSIZE;
+			goto err_free;
+		}
+		skb_trim(skb, skb->dev->mtu);
 	}
 	skb->protocol = cpu_to_be16(ETH_P_AF_IUCV);
 	nskb = skb_clone(skb, GFP_ATOMIC);
-	if (!nskb)
-		return -ENOMEM;
+	if (!nskb) {
+		err = -ENOMEM;
+		goto err_free;
+	}
+
 	skb_queue_tail(&iucv->send_skb_q, nskb);
 	err = dev_queue_xmit(skb);
 	if (net_xmit_eval(err)) {
@@ -375,6 +383,10 @@ static int afiucv_hs_send(struct iucv_message *imsg, struct sock *sock,
 		WARN_ON(atomic_read(&iucv->msg_recv) < 0);
 	}
 	return net_xmit_eval(err);
+
+err_free:
+	kfree_skb(skb);
+	return err;
 }
 
 static struct sock *__iucv_get_sock_by_name(char *nm)
@@ -1167,7 +1179,7 @@ static int iucv_sock_sendmsg(struct socket *sock, struct msghdr *msg,
 		err = afiucv_hs_send(&txmsg, sk, skb, 0);
 		if (err) {
 			atomic_dec(&iucv->msg_sent);
-			goto fail;
+			goto out;
 		}
 	} else { /* Classic VM IUCV transport */
 		skb_queue_tail(&iucv->send_skb_q, skb);
-- 
2.16.4

^ permalink raw reply related

* [PATCH net 3/3] net/iucv: declare iucv_path_table_empty() as static
From: Julian Wiedmann @ 2018-09-05 14:55 UTC (permalink / raw)
  To: David Miller
  Cc: netdev, linux-s390, Martin Schwidefsky, Heiko Carstens,
	Stefan Raspl, Ursula Braun, Julian Wiedmann
In-Reply-To: <20180905145512.30165-1-jwi@linux.ibm.com>

Fixes a compile warning.

Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
---
 net/iucv/iucv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/iucv/iucv.c b/net/iucv/iucv.c
index 8f7ef167c45a..eb502c6290c2 100644
--- a/net/iucv/iucv.c
+++ b/net/iucv/iucv.c
@@ -1874,7 +1874,7 @@ static void iucv_pm_complete(struct device *dev)
  * Returns 0 if there are still iucv pathes defined
  *	   1 if there are no iucv pathes defined
  */
-int iucv_path_table_empty(void)
+static int iucv_path_table_empty(void)
 {
 	int i;
 
-- 
2.16.4

^ permalink raw reply related

* Re: [PATCH net] mlxsw: spectrum_buffers: Set up a dedicated pool for BUM traffic
From: David Miller @ 2018-09-05 14:56 UTC (permalink / raw)
  To: petrm; +Cc: netdev, jiri, idosch
In-Reply-To: <e6061a9fa329ba8de7663a9c1be4ece3b2c4100b.1536142130.git.petrm@mellanox.com>

From: Petr Machata <petrm@mellanox.com>
Date: Wed, 05 Sep 2018 12:16:00 +0200

> MC-aware mode was recently enabled by mlxsw on Spectrum switches in
> commit 7b8195306694 ("mlxsw: spectrum: Configure MC-aware mode on mlxsw
> ports"). Unfortunately, testing has shown that the fix is incomplete and
> in the presented form actually makes the problem even worse, because any
> amount of MC traffic causes UC disruption.
> 
> The reason for this is that currently, mlxsw configures the MC-specific
> TCs (8..15) to map to pool 0. It also configures a maximum buffer size
> of 0, but for MC traffic that maximum is disregarded and not part of the
> quota. Therefore MC traffic is always admitted to the egress buffer.
> 
> Fix the configuration by directing the MC TCs into pool 15, which is
> dedicated to MC traffic and recognized as such by the silicon.
> 
> Fixes: 7b8195306694 ("mlxsw: spectrum: Configure MC-aware mode on mlxsw ports")
> Signed-off-by: Petr Machata <petrm@mellanox.com>
> Acked-by: Jiri Pirko <jiri@mellanox.com>

Applied, thank you.

^ permalink raw reply

* Re: [PATCH net-next] net/mlx5e: Make function mlx5i_grp_sw_update_stats() static
From: David Miller @ 2018-09-05 15:06 UTC (permalink / raw)
  To: weiyongjun1
  Cc: saeedm, leon, jgg, ogerlitz, tariqt, ferasda, valex, erezsh,
	netdev, linux-rdma, kernel-janitors
In-Reply-To: <1536146162-132732-1-git-send-email-weiyongjun1@huawei.com>

From: Wei Yongjun <weiyongjun1@huawei.com>
Date: Wed, 5 Sep 2018 11:16:02 +0000

> Fixes the following sparse warning:
> 
> drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c:119:6: warning:
>  symbol 'mlx5i_grp_sw_update_stats' was not declared. Should it be static?
> 
> Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>

Applied.

^ permalink raw reply

* Photo replacing
From: Jesse Kurtz @ 2018-09-05 13:06 UTC (permalink / raw)
  To: netdev

Just want to check with you to see if you have any photos for editing?

We are providing below services to you.
Clipping path for the images.
Cut out for the photos
Masking for the images
All kinds of retouching for the beauty and model.

We provide testing for your photos if you need.

Thanks,
Jesse

^ permalink raw reply

* [PATCH net-next] qed*: Utilize FW 8.37.7.0
From: Denis Bolotin @ 2018-09-05 15:35 UTC (permalink / raw)
  To: davem, netdev; +Cc: Denis Bolotin, Ariel Elior

This patch adds a new qed firmware with fixes and support for new features.

Fixes:
- Fix a rare case of device crash with iWARP, iSCSI or FCoE offload.
- Fix GRE tunneled traffic when iWARP offload is enabled.
- Fix RoCE failure in ib_send_bw when using inline data.
- Fix latency optimization flow for inline WQEs.
- BigBear 100G fix

RDMA:
- Reduce task context size.
- Application page sizes above 2GB support.
- Performance improvements.

ETH:
- Tenant DCB support.
- Replace RSS indirection table update interface.

Misc:
- Debug Tools changes.

Signed-off-by: Denis Bolotin <denis.bolotin@cavium.com>
Signed-off-by: Ariel Elior <ariel.elior@cavium.com>
---
 drivers/net/ethernet/qlogic/qed/qed.h       |   1 +
 drivers/net/ethernet/qlogic/qed/qed_debug.c | 248 +++++++++++++----------
 drivers/net/ethernet/qlogic/qed/qed_dev.c   |  11 ++
 drivers/net/ethernet/qlogic/qed/qed_hsi.h   | 297 +++++++++++++++++++---------
 include/linux/qed/common_hsi.h              |  10 +-
 include/linux/qed/iscsi_common.h            |   2 +-
 6 files changed, 367 insertions(+), 202 deletions(-)

diff --git a/drivers/net/ethernet/qlogic/qed/qed.h b/drivers/net/ethernet/qlogic/qed/qed.h
index a60e1c8..5f0962d 100644
--- a/drivers/net/ethernet/qlogic/qed/qed.h
+++ b/drivers/net/ethernet/qlogic/qed/qed.h
@@ -623,6 +623,7 @@ struct qed_hwfn {
 	void				*unzip_buf;
 
 	struct dbg_tools_data		dbg_info;
+	void				*dbg_user_info;
 
 	/* PWM region specific data */
 	u16				wid_count;
diff --git a/drivers/net/ethernet/qlogic/qed/qed_debug.c b/drivers/net/ethernet/qlogic/qed/qed_debug.c
index 1aa9fc1..78a638e 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_debug.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_debug.c
@@ -3454,7 +3454,8 @@ static u32 qed_grc_dump_iors(struct qed_hwfn *p_hwfn,
 			addr = BYTES_TO_DWORDS(storm->sem_fast_mem_addr +
 					       SEM_FAST_REG_STORM_REG_FILE) +
 			       IOR_SET_OFFSET(set_id);
-			buf[strlen(buf) - 1] = '0' + set_id;
+			if (strlen(buf) > 0)
+				buf[strlen(buf) - 1] = '0' + set_id;
 			offset += qed_grc_dump_mem(p_hwfn,
 						   p_ptt,
 						   dump_buf + offset,
@@ -5563,35 +5564,6 @@ struct block_info {
 	enum block_id id;
 };
 
-struct mcp_trace_format {
-	u32 data;
-#define MCP_TRACE_FORMAT_MODULE_MASK	0x0000ffff
-#define MCP_TRACE_FORMAT_MODULE_SHIFT	0
-#define MCP_TRACE_FORMAT_LEVEL_MASK	0x00030000
-#define MCP_TRACE_FORMAT_LEVEL_SHIFT	16
-#define MCP_TRACE_FORMAT_P1_SIZE_MASK	0x000c0000
-#define MCP_TRACE_FORMAT_P1_SIZE_SHIFT	18
-#define MCP_TRACE_FORMAT_P2_SIZE_MASK	0x00300000
-#define MCP_TRACE_FORMAT_P2_SIZE_SHIFT	20
-#define MCP_TRACE_FORMAT_P3_SIZE_MASK	0x00c00000
-#define MCP_TRACE_FORMAT_P3_SIZE_SHIFT	22
-#define MCP_TRACE_FORMAT_LEN_MASK	0xff000000
-#define MCP_TRACE_FORMAT_LEN_SHIFT	24
-
-	char *format_str;
-};
-
-/* Meta data structure, generated by a perl script during MFW build. therefore,
- * the structs mcp_trace_meta and mcp_trace_format are duplicated in the perl
- * script.
- */
-struct mcp_trace_meta {
-	u32 modules_num;
-	char **modules;
-	u32 formats_num;
-	struct mcp_trace_format *formats;
-};
-
 /* REG fifo element */
 struct reg_fifo_element {
 	u64 data;
@@ -5714,6 +5686,20 @@ struct igu_fifo_addr_data {
 	enum igu_fifo_addr_types type;
 };
 
+struct mcp_trace_meta {
+	u32 modules_num;
+	char **modules;
+	u32 formats_num;
+	struct mcp_trace_format *formats;
+	bool is_allocated;
+};
+
+/* Debug Tools user data */
+struct dbg_tools_user_data {
+	struct mcp_trace_meta mcp_trace_meta;
+	const u32 *mcp_trace_user_meta_buf;
+};
+
 /******************************** Constants **********************************/
 
 #define MAX_MSG_LEN				1024
@@ -6137,15 +6123,6 @@ struct user_dbg_array {
 
 /******************************** Variables **********************************/
 
-/* MCP Trace meta data array - used in case the dump doesn't contain the
- * meta data (e.g. due to no NVRAM access).
- */
-static struct user_dbg_array s_mcp_trace_meta_arr = { NULL, 0 };
-
-/* Parsed MCP Trace meta data info, based on MCP trace meta array */
-static struct mcp_trace_meta s_mcp_trace_meta;
-static bool s_mcp_trace_meta_valid;
-
 /* Temporary buffer, used for print size calculations */
 static char s_temp_buf[MAX_MSG_LEN];
 
@@ -6311,6 +6288,12 @@ static u32 qed_print_section_params(u32 *dump_buf,
 	return dump_offset;
 }
 
+static struct dbg_tools_user_data *
+qed_dbg_get_user_data(struct qed_hwfn *p_hwfn)
+{
+	return (struct dbg_tools_user_data *)p_hwfn->dbg_user_info;
+}
+
 /* Parses the idle check rules and returns the number of characters printed.
  * In case of parsing error, returns 0.
  */
@@ -6570,43 +6553,26 @@ static enum dbg_status qed_parse_idle_chk_dump(u32 *dump_buf,
 	return DBG_STATUS_OK;
 }
 
-/* Frees the specified MCP Trace meta data */
-static void qed_mcp_trace_free_meta(struct qed_hwfn *p_hwfn,
-				    struct mcp_trace_meta *meta)
-{
-	u32 i;
-
-	s_mcp_trace_meta_valid = false;
-
-	/* Release modules */
-	if (meta->modules) {
-		for (i = 0; i < meta->modules_num; i++)
-			kfree(meta->modules[i]);
-		kfree(meta->modules);
-	}
-
-	/* Release formats */
-	if (meta->formats) {
-		for (i = 0; i < meta->formats_num; i++)
-			kfree(meta->formats[i].format_str);
-		kfree(meta->formats);
-	}
-}
-
 /* Allocates and fills MCP Trace meta data based on the specified meta data
  * dump buffer.
  * Returns debug status code.
  */
-static enum dbg_status qed_mcp_trace_alloc_meta(struct qed_hwfn *p_hwfn,
-						const u32 *meta_buf,
-						struct mcp_trace_meta *meta)
+static enum dbg_status
+qed_mcp_trace_alloc_meta_data(struct qed_hwfn *p_hwfn,
+			      const u32 *meta_buf)
 {
-	u8 *meta_buf_bytes = (u8 *)meta_buf;
+	struct dbg_tools_user_data *dev_user_data;
 	u32 offset = 0, signature, i;
+	struct mcp_trace_meta *meta;
+	u8 *meta_buf_bytes;
+
+	dev_user_data = qed_dbg_get_user_data(p_hwfn);
+	meta = &dev_user_data->mcp_trace_meta;
+	meta_buf_bytes = (u8 *)meta_buf;
 
 	/* Free the previous meta before loading a new one. */
-	if (s_mcp_trace_meta_valid)
-		qed_mcp_trace_free_meta(p_hwfn, meta);
+	if (meta->is_allocated)
+		qed_mcp_trace_free_meta_data(p_hwfn);
 
 	memset(meta, 0, sizeof(*meta));
 
@@ -6674,7 +6640,7 @@ static enum dbg_status qed_mcp_trace_alloc_meta(struct qed_hwfn *p_hwfn,
 				      format_len, format_ptr->format_str);
 	}
 
-	s_mcp_trace_meta_valid = true;
+	meta->is_allocated = true;
 	return DBG_STATUS_OK;
 }
 
@@ -6687,21 +6653,26 @@ static enum dbg_status qed_mcp_trace_alloc_meta(struct qed_hwfn *p_hwfn,
  *               buffer.
  * data_size - size in bytes of data to parse.
  * parsed_buf - destination buffer for parsed data.
- * parsed_bytes - size of parsed data in bytes.
+ * parsed_results_bytes - size of parsed data in bytes.
  */
-static enum dbg_status qed_parse_mcp_trace_buf(u8 *trace_buf,
+static enum dbg_status qed_parse_mcp_trace_buf(struct qed_hwfn *p_hwfn,
+					       u8 *trace_buf,
 					       u32 trace_buf_size,
 					       u32 data_offset,
 					       u32 data_size,
 					       char *parsed_buf,
-					       u32 *parsed_bytes)
+					       u32 *parsed_results_bytes)
 {
+	struct dbg_tools_user_data *dev_user_data;
+	struct mcp_trace_meta *meta;
 	u32 param_mask, param_shift;
 	enum dbg_status status;
 
-	*parsed_bytes = 0;
+	dev_user_data = qed_dbg_get_user_data(p_hwfn);
+	meta = &dev_user_data->mcp_trace_meta;
+	*parsed_results_bytes = 0;
 
-	if (!s_mcp_trace_meta_valid)
+	if (!meta->is_allocated)
 		return DBG_STATUS_MCP_TRACE_BAD_DATA;
 
 	status = DBG_STATUS_OK;
@@ -6723,7 +6694,7 @@ static enum dbg_status qed_parse_mcp_trace_buf(u8 *trace_buf,
 		format_idx = header & MFW_TRACE_EVENTID_MASK;
 
 		/* Skip message if its index doesn't exist in the meta data */
-		if (format_idx >= s_mcp_trace_meta.formats_num) {
+		if (format_idx >= meta->formats_num) {
 			u8 format_size =
 				(u8)((header & MFW_TRACE_PRM_SIZE_MASK) >>
 				     MFW_TRACE_PRM_SIZE_SHIFT);
@@ -6738,7 +6709,7 @@ static enum dbg_status qed_parse_mcp_trace_buf(u8 *trace_buf,
 			continue;
 		}
 
-		format_ptr = &s_mcp_trace_meta.formats[format_idx];
+		format_ptr = &meta->formats[format_idx];
 
 		for (i = 0,
 		     param_mask = MCP_TRACE_FORMAT_P1_SIZE_MASK,
@@ -6783,19 +6754,20 @@ static enum dbg_status qed_parse_mcp_trace_buf(u8 *trace_buf,
 			return DBG_STATUS_MCP_TRACE_BAD_DATA;
 
 		/* Print current message to results buffer */
-		*parsed_bytes +=
-			sprintf(qed_get_buf_ptr(parsed_buf, *parsed_bytes),
+		*parsed_results_bytes +=
+			sprintf(qed_get_buf_ptr(parsed_buf,
+						*parsed_results_bytes),
 				"%s %-8s: ",
 				s_mcp_trace_level_str[format_level],
-				s_mcp_trace_meta.modules[format_module]);
-		*parsed_bytes +=
-		    sprintf(qed_get_buf_ptr(parsed_buf, *parsed_bytes),
+				meta->modules[format_module]);
+		*parsed_results_bytes +=
+		    sprintf(qed_get_buf_ptr(parsed_buf, *parsed_results_bytes),
 			    format_ptr->format_str,
 			    params[0], params[1], params[2]);
 	}
 
 	/* Add string NULL terminator */
-	(*parsed_bytes)++;
+	(*parsed_results_bytes)++;
 
 	return status;
 }
@@ -6803,24 +6775,25 @@ static enum dbg_status qed_parse_mcp_trace_buf(u8 *trace_buf,
 /* Parses an MCP Trace dump buffer.
  * If result_buf is not NULL, the MCP Trace results are printed to it.
  * In any case, the required results buffer size is assigned to
- * parsed_bytes.
+ * parsed_results_bytes.
  * The parsing status is returned.
  */
 static enum dbg_status qed_parse_mcp_trace_dump(struct qed_hwfn *p_hwfn,
 						u32 *dump_buf,
-						char *parsed_buf,
-						u32 *parsed_bytes)
+						char *results_buf,
+						u32 *parsed_results_bytes,
+						bool free_meta_data)
 {
 	const char *section_name, *param_name, *param_str_val;
 	u32 data_size, trace_data_dwords, trace_meta_dwords;
-	u32 offset, results_offset, parsed_buf_bytes;
+	u32 offset, results_offset, results_buf_bytes;
 	u32 param_num_val, num_section_params;
 	struct mcp_trace *trace;
 	enum dbg_status status;
 	const u32 *meta_buf;
 	u8 *trace_buf;
 
-	*parsed_bytes = 0;
+	*parsed_results_bytes = 0;
 
 	/* Read global_params section */
 	dump_buf += qed_read_section_hdr(dump_buf,
@@ -6831,7 +6804,7 @@ static enum dbg_status qed_parse_mcp_trace_dump(struct qed_hwfn *p_hwfn,
 	/* Print global params */
 	dump_buf += qed_print_section_params(dump_buf,
 					     num_section_params,
-					     parsed_buf, &results_offset);
+					     results_buf, &results_offset);
 
 	/* Read trace_data section */
 	dump_buf += qed_read_section_hdr(dump_buf,
@@ -6846,6 +6819,9 @@ static enum dbg_status qed_parse_mcp_trace_dump(struct qed_hwfn *p_hwfn,
 
 	/* Prepare trace info */
 	trace = (struct mcp_trace *)dump_buf;
+	if (trace->signature != MFW_TRACE_SIGNATURE || !trace->size)
+		return DBG_STATUS_MCP_TRACE_BAD_DATA;
+
 	trace_buf = (u8 *)dump_buf + sizeof(*trace);
 	offset = trace->trace_oldest;
 	data_size = qed_cyclic_sub(trace->trace_prod, offset, trace->size);
@@ -6865,31 +6841,39 @@ static enum dbg_status qed_parse_mcp_trace_dump(struct qed_hwfn *p_hwfn,
 	/* Choose meta data buffer */
 	if (!trace_meta_dwords) {
 		/* Dump doesn't include meta data */
-		if (!s_mcp_trace_meta_arr.ptr)
+		struct dbg_tools_user_data *dev_user_data =
+			qed_dbg_get_user_data(p_hwfn);
+
+		if (!dev_user_data->mcp_trace_user_meta_buf)
 			return DBG_STATUS_MCP_TRACE_NO_META;
-		meta_buf = s_mcp_trace_meta_arr.ptr;
+
+		meta_buf = dev_user_data->mcp_trace_user_meta_buf;
 	} else {
 		/* Dump includes meta data */
 		meta_buf = dump_buf;
 	}
 
 	/* Allocate meta data memory */
-	status = qed_mcp_trace_alloc_meta(p_hwfn, meta_buf, &s_mcp_trace_meta);
+	status = qed_mcp_trace_alloc_meta_data(p_hwfn, meta_buf);
 	if (status != DBG_STATUS_OK)
 		return status;
 
-	status = qed_parse_mcp_trace_buf(trace_buf,
+	status = qed_parse_mcp_trace_buf(p_hwfn,
+					 trace_buf,
 					 trace->size,
 					 offset,
 					 data_size,
-					 parsed_buf ?
-					 parsed_buf + results_offset :
+					 results_buf ?
+					 results_buf + results_offset :
 					 NULL,
-					 &parsed_buf_bytes);
+					 &results_buf_bytes);
 	if (status != DBG_STATUS_OK)
 		return status;
 
-	*parsed_bytes = results_offset + parsed_buf_bytes;
+	if (free_meta_data)
+		qed_mcp_trace_free_meta_data(p_hwfn);
+
+	*parsed_results_bytes = results_offset + results_buf_bytes;
 
 	return DBG_STATUS_OK;
 }
@@ -7361,6 +7345,16 @@ enum dbg_status qed_dbg_user_set_bin_ptr(const u8 * const bin_ptr)
 	return DBG_STATUS_OK;
 }
 
+enum dbg_status qed_dbg_alloc_user_data(struct qed_hwfn *p_hwfn)
+{
+	p_hwfn->dbg_user_info = kzalloc(sizeof(struct dbg_tools_user_data),
+					GFP_KERNEL);
+	if (!p_hwfn->dbg_user_info)
+		return DBG_STATUS_VIRT_MEM_ALLOC_FAILED;
+
+	return DBG_STATUS_OK;
+}
+
 const char *qed_dbg_get_status_str(enum dbg_status status)
 {
 	return (status <
@@ -7397,10 +7391,13 @@ enum dbg_status qed_print_idle_chk_results(struct qed_hwfn *p_hwfn,
 				       num_errors, num_warnings);
 }
 
-void qed_dbg_mcp_trace_set_meta_data(u32 *data, u32 size)
+void qed_dbg_mcp_trace_set_meta_data(struct qed_hwfn *p_hwfn,
+				     const u32 *meta_buf)
 {
-	s_mcp_trace_meta_arr.ptr = data;
-	s_mcp_trace_meta_arr.size_in_dwords = size;
+	struct dbg_tools_user_data *dev_user_data =
+		qed_dbg_get_user_data(p_hwfn);
+
+	dev_user_data->mcp_trace_user_meta_buf = meta_buf;
 }
 
 enum dbg_status qed_get_mcp_trace_results_buf_size(struct qed_hwfn *p_hwfn,
@@ -7409,7 +7406,7 @@ enum dbg_status qed_get_mcp_trace_results_buf_size(struct qed_hwfn *p_hwfn,
 						   u32 *results_buf_size)
 {
 	return qed_parse_mcp_trace_dump(p_hwfn,
-					dump_buf, NULL, results_buf_size);
+					dump_buf, NULL, results_buf_size, true);
 }
 
 enum dbg_status qed_print_mcp_trace_results(struct qed_hwfn *p_hwfn,
@@ -7421,20 +7418,61 @@ enum dbg_status qed_print_mcp_trace_results(struct qed_hwfn *p_hwfn,
 
 	return qed_parse_mcp_trace_dump(p_hwfn,
 					dump_buf,
-					results_buf, &parsed_buf_size);
+					results_buf, &parsed_buf_size, true);
+}
+
+enum dbg_status qed_print_mcp_trace_results_cont(struct qed_hwfn *p_hwfn,
+						 u32 *dump_buf,
+						 char *results_buf)
+{
+	u32 parsed_buf_size;
+
+	return qed_parse_mcp_trace_dump(p_hwfn, dump_buf, results_buf,
+					&parsed_buf_size, false);
 }
 
-enum dbg_status qed_print_mcp_trace_line(u8 *dump_buf,
+enum dbg_status qed_print_mcp_trace_line(struct qed_hwfn *p_hwfn,
+					 u8 *dump_buf,
 					 u32 num_dumped_bytes,
 					 char *results_buf)
 {
-	u32 parsed_bytes;
+	u32 parsed_results_bytes;
 
-	return qed_parse_mcp_trace_buf(dump_buf,
+	return qed_parse_mcp_trace_buf(p_hwfn,
+				       dump_buf,
 				       num_dumped_bytes,
 				       0,
 				       num_dumped_bytes,
-				       results_buf, &parsed_bytes);
+				       results_buf, &parsed_results_bytes);
+}
+
+/* Frees the specified MCP Trace meta data */
+void qed_mcp_trace_free_meta_data(struct qed_hwfn *p_hwfn)
+{
+	struct dbg_tools_user_data *dev_user_data;
+	struct mcp_trace_meta *meta;
+	u32 i;
+
+	dev_user_data = qed_dbg_get_user_data(p_hwfn);
+	meta = &dev_user_data->mcp_trace_meta;
+	if (!meta->is_allocated)
+		return;
+
+	/* Release modules */
+	if (meta->modules) {
+		for (i = 0; i < meta->modules_num; i++)
+			kfree(meta->modules[i]);
+		kfree(meta->modules);
+	}
+
+	/* Release formats */
+	if (meta->formats) {
+		for (i = 0; i < meta->formats_num; i++)
+			kfree(meta->formats[i].format_str);
+		kfree(meta->formats);
+	}
+
+	meta->is_allocated = false;
 }
 
 enum dbg_status qed_get_reg_fifo_results_buf_size(struct qed_hwfn *p_hwfn,
diff --git a/drivers/net/ethernet/qlogic/qed/qed_dev.c b/drivers/net/ethernet/qlogic/qed/qed_dev.c
index 016ca8a..128eb63 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_dev.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_dev.c
@@ -144,6 +144,12 @@ static void qed_qm_info_free(struct qed_hwfn *p_hwfn)
 	qm_info->wfq_data = NULL;
 }
 
+static void qed_dbg_user_data_free(struct qed_hwfn *p_hwfn)
+{
+	kfree(p_hwfn->dbg_user_info);
+	p_hwfn->dbg_user_info = NULL;
+}
+
 void qed_resc_free(struct qed_dev *cdev)
 {
 	int i;
@@ -183,6 +189,7 @@ void qed_resc_free(struct qed_dev *cdev)
 		qed_l2_free(p_hwfn);
 		qed_dmae_info_free(p_hwfn);
 		qed_dcbx_info_free(p_hwfn);
+		qed_dbg_user_data_free(p_hwfn);
 	}
 }
 
@@ -1083,6 +1090,10 @@ int qed_resc_alloc(struct qed_dev *cdev)
 		rc = qed_dcbx_info_alloc(p_hwfn);
 		if (rc)
 			goto alloc_err;
+
+		rc = qed_dbg_alloc_user_data(p_hwfn);
+		if (rc)
+			goto alloc_err;
 	}
 
 	cdev->reset_stats = kzalloc(sizeof(*cdev->reset_stats), GFP_KERNEL);
diff --git a/drivers/net/ethernet/qlogic/qed/qed_hsi.h b/drivers/net/ethernet/qlogic/qed/qed_hsi.h
index 8faceb6..21ec809 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_hsi.h
+++ b/drivers/net/ethernet/qlogic/qed/qed_hsi.h
@@ -274,7 +274,8 @@ struct core_rx_start_ramrod_data {
 	u8 mf_si_mcast_accept_all;
 	struct core_rx_action_on_error action_on_error;
 	u8 gsi_offload_flag;
-	u8 reserved[6];
+	u8 wipe_inner_vlan_pri_en;
+	u8 reserved[5];
 };
 
 /* Ramrod data for rx queue stop ramrod */
@@ -351,7 +352,8 @@ struct core_tx_start_ramrod_data {
 	__le16 pbl_size;
 	__le16 qm_pq_id;
 	u8 gsi_offload_flag;
-	u8 resrved[3];
+	u8 vport_id;
+	u8 resrved[2];
 };
 
 /* Ramrod data for tx queue stop ramrod */
@@ -914,6 +916,16 @@ struct eth_rx_rate_limit {
 	__le16 reserved1;
 };
 
+/* Update RSS indirection table entry command */
+struct eth_tstorm_rss_update_data {
+	u8 valid;
+	u8 vport_id;
+	u8 ind_table_index;
+	u8 reserved;
+	__le16 ind_table_value;
+	__le16 reserved1;
+};
+
 struct eth_ustorm_per_pf_stat {
 	struct regpair rcv_lb_ucast_bytes;
 	struct regpair rcv_lb_mcast_bytes;
@@ -1241,6 +1253,10 @@ struct rl_update_ramrod_data {
 	u8 rl_id_first;
 	u8 rl_id_last;
 	u8 rl_dc_qcn_flg;
+	u8 dcqcn_reset_alpha_on_idle;
+	u8 rl_bc_stage_th;
+	u8 rl_timer_stage_th;
+	u8 reserved1;
 	__le32 rl_bc_rate;
 	__le16 rl_max_rate;
 	__le16 rl_r_ai;
@@ -1249,7 +1265,7 @@ struct rl_update_ramrod_data {
 	__le32 dcqcn_k_us;
 	__le32 dcqcn_timeuot_us;
 	__le32 qcn_timeuot_us;
-	__le32 reserved[2];
+	__le32 reserved2;
 };
 
 /* Slowpath Element (SPQE) */
@@ -3322,6 +3338,25 @@ enum dbg_status qed_dbg_read_attn(struct qed_hwfn *p_hwfn,
 enum dbg_status qed_dbg_print_attn(struct qed_hwfn *p_hwfn,
 				   struct dbg_attn_block_result *results);
 
+/******************************* Data Types **********************************/
+
+struct mcp_trace_format {
+	u32 data;
+#define MCP_TRACE_FORMAT_MODULE_MASK	0x0000ffff
+#define MCP_TRACE_FORMAT_MODULE_SHIFT	0
+#define MCP_TRACE_FORMAT_LEVEL_MASK	0x00030000
+#define MCP_TRACE_FORMAT_LEVEL_SHIFT	16
+#define MCP_TRACE_FORMAT_P1_SIZE_MASK	0x000c0000
+#define MCP_TRACE_FORMAT_P1_SIZE_SHIFT	18
+#define MCP_TRACE_FORMAT_P2_SIZE_MASK	0x00300000
+#define MCP_TRACE_FORMAT_P2_SIZE_SHIFT	20
+#define MCP_TRACE_FORMAT_P3_SIZE_MASK	0x00c00000
+#define MCP_TRACE_FORMAT_P3_SIZE_SHIFT	22
+#define MCP_TRACE_FORMAT_LEN_MASK	0xff000000
+#define MCP_TRACE_FORMAT_LEN_SHIFT	24
+	char *format_str;
+};
+
 /******************************** Constants **********************************/
 
 #define MAX_NAME_LEN	16
@@ -3337,6 +3372,13 @@ enum dbg_status qed_dbg_print_attn(struct qed_hwfn *p_hwfn,
 enum dbg_status qed_dbg_user_set_bin_ptr(const u8 * const bin_ptr);
 
 /**
+ * @brief qed_dbg_alloc_user_data - Allocates user debug data.
+ *
+ * @param p_hwfn -		 HW device data
+ */
+enum dbg_status qed_dbg_alloc_user_data(struct qed_hwfn *p_hwfn);
+
+/**
  * @brief qed_dbg_get_status_str - Returns a string for the specified status.
  *
  * @param status - a debug status code.
@@ -3381,8 +3423,7 @@ enum dbg_status qed_print_idle_chk_results(struct qed_hwfn *p_hwfn,
 					   u32 *num_warnings);
 
 /**
- * @brief qed_dbg_mcp_trace_set_meta_data - Sets a pointer to the MCP Trace
- *	meta data.
+ * @brief qed_dbg_mcp_trace_set_meta_data - Sets the MCP Trace meta data.
  *
  * Needed in case the MCP Trace dump doesn't contain the meta data (e.g. due to
  * no NVRAM access).
@@ -3390,7 +3431,8 @@ enum dbg_status qed_print_idle_chk_results(struct qed_hwfn *p_hwfn,
  * @param data - pointer to MCP Trace meta data
  * @param size - size of MCP Trace meta data in dwords
  */
-void qed_dbg_mcp_trace_set_meta_data(u32 *data, u32 size);
+void qed_dbg_mcp_trace_set_meta_data(struct qed_hwfn *p_hwfn,
+				     const u32 *meta_buf);
 
 /**
  * @brief qed_get_mcp_trace_results_buf_size - Returns the required buffer size
@@ -3425,19 +3467,45 @@ enum dbg_status qed_print_mcp_trace_results(struct qed_hwfn *p_hwfn,
 					    char *results_buf);
 
 /**
+ * @brief qed_print_mcp_trace_results_cont - Prints MCP Trace results, and
+ * keeps the MCP trace meta data allocated, to support continuous MCP Trace
+ * parsing. After the continuous parsing ends, mcp_trace_free_meta_data should
+ * be called to free the meta data.
+ *
+ * @param p_hwfn -	      HW device data
+ * @param dump_buf -	      mcp trace dump buffer, starting from the header.
+ * @param results_buf -	      buffer for printing the mcp trace results.
+ *
+ * @return error if the parsing fails, ok otherwise.
+ */
+enum dbg_status qed_print_mcp_trace_results_cont(struct qed_hwfn *p_hwfn,
+						 u32 *dump_buf,
+						 char *results_buf);
+
+/**
  * @brief print_mcp_trace_line - Prints MCP Trace results for a single line
  *
+ * @param p_hwfn -	      HW device data
  * @param dump_buf -	      mcp trace dump buffer, starting from the header.
  * @param num_dumped_bytes -  number of bytes that were dumped.
  * @param results_buf -	      buffer for printing the mcp trace results.
  *
  * @return error if the parsing fails, ok otherwise.
  */
-enum dbg_status qed_print_mcp_trace_line(u8 *dump_buf,
+enum dbg_status qed_print_mcp_trace_line(struct qed_hwfn *p_hwfn,
+					 u8 *dump_buf,
 					 u32 num_dumped_bytes,
 					 char *results_buf);
 
 /**
+ * @brief mcp_trace_free_meta_data - Frees the MCP Trace meta data.
+ * Should be called after continuous MCP Trace parsing.
+ *
+ * @param p_hwfn - HW device data
+ */
+void qed_mcp_trace_free_meta_data(struct qed_hwfn *p_hwfn);
+
+/**
  * @brief qed_get_reg_fifo_results_buf_size - Returns the required buffer size
  *	for reg_fifo results (in bytes).
  *
@@ -4303,154 +4371,161 @@ void qed_set_rdma_error_level(struct qed_hwfn *p_hwfn,
 	(IRO[29].base + ((pf_id) * IRO[29].m1))
 #define ETH_RX_RATE_LIMIT_SIZE				(IRO[29].size)
 
+/* RSS indirection table entry update command per PF offset in TSTORM PF BAR0.
+ * Use eth_tstorm_rss_update_data for update.
+ */
+#define TSTORM_ETH_RSS_UPDATE_OFFSET(pf_id) \
+	(IRO[30].base + ((pf_id) * IRO[30].m1))
+#define TSTORM_ETH_RSS_UPDATE_SIZE			(IRO[30].size)
+
 /* Xstorm queue zone */
 #define XSTORM_ETH_QUEUE_ZONE_OFFSET(queue_id) \
-	(IRO[30].base + ((queue_id) * IRO[30].m1))
-#define XSTORM_ETH_QUEUE_ZONE_SIZE			(IRO[30].size)
+	(IRO[31].base + ((queue_id) * IRO[31].m1))
+#define XSTORM_ETH_QUEUE_ZONE_SIZE			(IRO[31].size)
 
 /* Ystorm cqe producer */
 #define YSTORM_TOE_CQ_PROD_OFFSET(rss_id) \
-	(IRO[31].base + ((rss_id) * IRO[31].m1))
-#define YSTORM_TOE_CQ_PROD_SIZE				(IRO[31].size)
+	(IRO[32].base + ((rss_id) * IRO[32].m1))
+#define YSTORM_TOE_CQ_PROD_SIZE				(IRO[32].size)
 
 /* Ustorm cqe producer */
 #define USTORM_TOE_CQ_PROD_OFFSET(rss_id) \
-	(IRO[32].base + ((rss_id) * IRO[32].m1))
-#define USTORM_TOE_CQ_PROD_SIZE				(IRO[32].size)
+	(IRO[33].base + ((rss_id) * IRO[33].m1))
+#define USTORM_TOE_CQ_PROD_SIZE				(IRO[33].size)
 
 /* Ustorm grq producer */
 #define USTORM_TOE_GRQ_PROD_OFFSET(pf_id) \
-	(IRO[33].base + ((pf_id) * IRO[33].m1))
-#define USTORM_TOE_GRQ_PROD_SIZE			(IRO[33].size)
+	(IRO[34].base + ((pf_id) * IRO[34].m1))
+#define USTORM_TOE_GRQ_PROD_SIZE			(IRO[34].size)
 
 /* Tstorm cmdq-cons of given command queue-id */
 #define TSTORM_SCSI_CMDQ_CONS_OFFSET(cmdq_queue_id) \
-	(IRO[34].base + ((cmdq_queue_id) * IRO[34].m1))
-#define TSTORM_SCSI_CMDQ_CONS_SIZE			(IRO[34].size)
+	(IRO[35].base + ((cmdq_queue_id) * IRO[35].m1))
+#define TSTORM_SCSI_CMDQ_CONS_SIZE			(IRO[35].size)
 
 /* Tstorm (reflects M-Storm) bdq-external-producer of given function ID,
  * BDqueue-id.
  */
 #define TSTORM_SCSI_BDQ_EXT_PROD_OFFSET(func_id, bdq_id) \
-	(IRO[35].base + ((func_id) * IRO[35].m1) + ((bdq_id) * IRO[35].m2))
-#define TSTORM_SCSI_BDQ_EXT_PROD_SIZE			(IRO[35].size)
+	(IRO[36].base + ((func_id) * IRO[36].m1) + ((bdq_id) * IRO[36].m2))
+#define TSTORM_SCSI_BDQ_EXT_PROD_SIZE			(IRO[36].size)
 
 /* Mstorm bdq-external-producer of given BDQ resource ID, BDqueue-id */
 #define MSTORM_SCSI_BDQ_EXT_PROD_OFFSET(func_id, bdq_id) \
-	(IRO[36].base + ((func_id) * IRO[36].m1) + ((bdq_id) * IRO[36].m2))
-#define MSTORM_SCSI_BDQ_EXT_PROD_SIZE			(IRO[36].size)
+	(IRO[37].base + ((func_id) * IRO[37].m1) + ((bdq_id) * IRO[37].m2))
+#define MSTORM_SCSI_BDQ_EXT_PROD_SIZE			(IRO[37].size)
 
 /* Tstorm iSCSI RX stats */
 #define TSTORM_ISCSI_RX_STATS_OFFSET(pf_id) \
-	(IRO[37].base + ((pf_id) * IRO[37].m1))
-#define TSTORM_ISCSI_RX_STATS_SIZE			(IRO[37].size)
+	(IRO[38].base + ((pf_id) * IRO[38].m1))
+#define TSTORM_ISCSI_RX_STATS_SIZE			(IRO[38].size)
 
 /* Mstorm iSCSI RX stats */
 #define MSTORM_ISCSI_RX_STATS_OFFSET(pf_id) \
-	(IRO[38].base + ((pf_id) * IRO[38].m1))
-#define MSTORM_ISCSI_RX_STATS_SIZE			(IRO[38].size)
+	(IRO[39].base + ((pf_id) * IRO[39].m1))
+#define MSTORM_ISCSI_RX_STATS_SIZE			(IRO[39].size)
 
 /* Ustorm iSCSI RX stats */
 #define USTORM_ISCSI_RX_STATS_OFFSET(pf_id) \
-	(IRO[39].base + ((pf_id) * IRO[39].m1))
-#define USTORM_ISCSI_RX_STATS_SIZE			(IRO[39].size)
+	(IRO[40].base + ((pf_id) * IRO[40].m1))
+#define USTORM_ISCSI_RX_STATS_SIZE			(IRO[40].size)
 
 /* Xstorm iSCSI TX stats */
 #define XSTORM_ISCSI_TX_STATS_OFFSET(pf_id) \
-	(IRO[40].base + ((pf_id) * IRO[40].m1))
-#define XSTORM_ISCSI_TX_STATS_SIZE			(IRO[40].size)
+	(IRO[41].base + ((pf_id) * IRO[41].m1))
+#define XSTORM_ISCSI_TX_STATS_SIZE			(IRO[41].size)
 
 /* Ystorm iSCSI TX stats */
 #define YSTORM_ISCSI_TX_STATS_OFFSET(pf_id) \
-	(IRO[41].base + ((pf_id) * IRO[41].m1))
-#define YSTORM_ISCSI_TX_STATS_SIZE			(IRO[41].size)
+	(IRO[42].base + ((pf_id) * IRO[42].m1))
+#define YSTORM_ISCSI_TX_STATS_SIZE			(IRO[42].size)
 
 /* Pstorm iSCSI TX stats */
 #define PSTORM_ISCSI_TX_STATS_OFFSET(pf_id) \
-	(IRO[42].base + ((pf_id) * IRO[42].m1))
-#define PSTORM_ISCSI_TX_STATS_SIZE			(IRO[42].size)
+	(IRO[43].base + ((pf_id) * IRO[43].m1))
+#define PSTORM_ISCSI_TX_STATS_SIZE			(IRO[43].size)
 
 /* Tstorm FCoE RX stats */
 #define TSTORM_FCOE_RX_STATS_OFFSET(pf_id) \
-	(IRO[43].base + ((pf_id) * IRO[43].m1))
-#define TSTORM_FCOE_RX_STATS_SIZE			(IRO[43].size)
+	(IRO[44].base + ((pf_id) * IRO[44].m1))
+#define TSTORM_FCOE_RX_STATS_SIZE			(IRO[44].size)
 
 /* Pstorm FCoE TX stats */
 #define PSTORM_FCOE_TX_STATS_OFFSET(pf_id) \
-	(IRO[44].base + ((pf_id) * IRO[44].m1))
-#define PSTORM_FCOE_TX_STATS_SIZE			(IRO[44].size)
+	(IRO[45].base + ((pf_id) * IRO[45].m1))
+#define PSTORM_FCOE_TX_STATS_SIZE			(IRO[45].size)
 
 /* Pstorm RDMA queue statistics */
 #define PSTORM_RDMA_QUEUE_STAT_OFFSET(rdma_stat_counter_id) \
-	(IRO[45].base + ((rdma_stat_counter_id) * IRO[45].m1))
-#define PSTORM_RDMA_QUEUE_STAT_SIZE			(IRO[45].size)
+	(IRO[46].base + ((rdma_stat_counter_id) * IRO[46].m1))
+#define PSTORM_RDMA_QUEUE_STAT_SIZE			(IRO[46].size)
 
 /* Tstorm RDMA queue statistics */
 #define TSTORM_RDMA_QUEUE_STAT_OFFSET(rdma_stat_counter_id) \
-	(IRO[46].base + ((rdma_stat_counter_id) * IRO[46].m1))
-#define TSTORM_RDMA_QUEUE_STAT_SIZE			(IRO[46].size)
+	(IRO[47].base + ((rdma_stat_counter_id) * IRO[47].m1))
+#define TSTORM_RDMA_QUEUE_STAT_SIZE			(IRO[47].size)
 
 /* Xstorm error level for assert */
 #define XSTORM_RDMA_ASSERT_LEVEL_OFFSET(pf_id) \
-	(IRO[47].base +	((pf_id) * IRO[47].m1))
-#define XSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[47].size)
+	(IRO[48].base +	((pf_id) * IRO[48].m1))
+#define XSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[48].size)
 
 /* Ystorm error level for assert */
 #define YSTORM_RDMA_ASSERT_LEVEL_OFFSET(pf_id) \
-	(IRO[48].base + ((pf_id) * IRO[48].m1))
-#define YSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[48].size)
+	(IRO[49].base + ((pf_id) * IRO[49].m1))
+#define YSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[49].size)
 
 /* Pstorm error level for assert */
 #define PSTORM_RDMA_ASSERT_LEVEL_OFFSET(pf_id) \
-	(IRO[49].base +	((pf_id) * IRO[49].m1))
-#define PSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[49].size)
+	(IRO[50].base +	((pf_id) * IRO[50].m1))
+#define PSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[50].size)
 
 /* Tstorm error level for assert */
 #define TSTORM_RDMA_ASSERT_LEVEL_OFFSET(pf_id) \
-	(IRO[50].base +	((pf_id) * IRO[50].m1))
-#define TSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[50].size)
+	(IRO[51].base +	((pf_id) * IRO[51].m1))
+#define TSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[51].size)
 
 /* Mstorm error level for assert */
 #define MSTORM_RDMA_ASSERT_LEVEL_OFFSET(pf_id) \
-	(IRO[51].base + ((pf_id) * IRO[51].m1))
-#define MSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[51].size)
+	(IRO[52].base + ((pf_id) * IRO[52].m1))
+#define MSTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[52].size)
 
 /* Ustorm error level for assert */
 #define USTORM_RDMA_ASSERT_LEVEL_OFFSET(pf_id) \
-	(IRO[52].base + ((pf_id) * IRO[52].m1))
-#define USTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[52].size)
+	(IRO[53].base + ((pf_id) * IRO[53].m1))
+#define USTORM_RDMA_ASSERT_LEVEL_SIZE			(IRO[53].size)
 
 /* Xstorm iWARP rxmit stats */
 #define XSTORM_IWARP_RXMIT_STATS_OFFSET(pf_id) \
-	(IRO[53].base +	((pf_id) * IRO[53].m1))
-#define XSTORM_IWARP_RXMIT_STATS_SIZE			(IRO[53].size)
+	(IRO[54].base +	((pf_id) * IRO[54].m1))
+#define XSTORM_IWARP_RXMIT_STATS_SIZE			(IRO[54].size)
 
 /* Tstorm RoCE Event Statistics */
 #define TSTORM_ROCE_EVENTS_STAT_OFFSET(roce_pf_id) \
-	(IRO[54].base + ((roce_pf_id) * IRO[54].m1))
-#define TSTORM_ROCE_EVENTS_STAT_SIZE			(IRO[54].size)
+	(IRO[55].base + ((roce_pf_id) * IRO[55].m1))
+#define TSTORM_ROCE_EVENTS_STAT_SIZE			(IRO[55].size)
 
 /* DCQCN Received Statistics */
 #define YSTORM_ROCE_DCQCN_RECEIVED_STATS_OFFSET(roce_pf_id) \
-	(IRO[55].base + ((roce_pf_id) * IRO[55].m1))
-#define YSTORM_ROCE_DCQCN_RECEIVED_STATS_SIZE		(IRO[55].size)
+	(IRO[56].base + ((roce_pf_id) * IRO[56].m1))
+#define YSTORM_ROCE_DCQCN_RECEIVED_STATS_SIZE		(IRO[56].size)
 
 /* RoCE Error Statistics */
 #define YSTORM_ROCE_ERROR_STATS_OFFSET(roce_pf_id) \
-	(IRO[56].base + ((roce_pf_id) * IRO[56].m1))
-#define YSTORM_ROCE_ERROR_STATS_SIZE			(IRO[56].size)
+	(IRO[57].base + ((roce_pf_id) * IRO[57].m1))
+#define YSTORM_ROCE_ERROR_STATS_SIZE			(IRO[57].size)
 
 /* DCQCN Sent Statistics */
 #define PSTORM_ROCE_DCQCN_SENT_STATS_OFFSET(roce_pf_id) \
-	(IRO[57].base + ((roce_pf_id) * IRO[57].m1))
-#define PSTORM_ROCE_DCQCN_SENT_STATS_SIZE		(IRO[57].size)
+	(IRO[58].base + ((roce_pf_id) * IRO[58].m1))
+#define PSTORM_ROCE_DCQCN_SENT_STATS_SIZE		(IRO[58].size)
 
 /* RoCE CQEs Statistics */
 #define USTORM_ROCE_CQE_STATS_OFFSET(roce_pf_id) \
-	(IRO[58].base + ((roce_pf_id) * IRO[58].m1))
-#define USTORM_ROCE_CQE_STATS_SIZE			(IRO[58].size)
+	(IRO[59].base + ((roce_pf_id) * IRO[59].m1))
+#define USTORM_ROCE_CQE_STATS_SIZE			(IRO[59].size)
 
-static const struct iro iro_arr[59] = {
+static const struct iro iro_arr[60] = {
 	{0x0, 0x0, 0x0, 0x0, 0x8},
 	{0x4cb8, 0x88, 0x0, 0x0, 0x88},
 	{0x6530, 0x20, 0x0, 0x0, 0x20},
@@ -4461,14 +4536,14 @@ void qed_set_rdma_error_level(struct qed_hwfn *p_hwfn,
 	{0x84, 0x8, 0x0, 0x0, 0x2},
 	{0x4c48, 0x0, 0x0, 0x0, 0x78},
 	{0x3e38, 0x0, 0x0, 0x0, 0x78},
-	{0x2b78, 0x0, 0x0, 0x0, 0x78},
+	{0x3ef8, 0x0, 0x0, 0x0, 0x78},
 	{0x4c40, 0x0, 0x0, 0x0, 0x78},
 	{0x4998, 0x0, 0x0, 0x0, 0x78},
 	{0x7f50, 0x0, 0x0, 0x0, 0x78},
 	{0xa28, 0x8, 0x0, 0x0, 0x8},
 	{0x6210, 0x10, 0x0, 0x0, 0x10},
 	{0xb820, 0x30, 0x0, 0x0, 0x30},
-	{0x96c0, 0x30, 0x0, 0x0, 0x30},
+	{0xa990, 0x30, 0x0, 0x0, 0x30},
 	{0x4b68, 0x80, 0x0, 0x0, 0x40},
 	{0x1f8, 0x4, 0x0, 0x0, 0x4},
 	{0x53a8, 0x80, 0x4, 0x0, 0x4},
@@ -4476,11 +4551,12 @@ void qed_set_rdma_error_level(struct qed_hwfn *p_hwfn,
 	{0x4ba8, 0x80, 0x0, 0x0, 0x20},
 	{0x8158, 0x40, 0x0, 0x0, 0x30},
 	{0xe770, 0x60, 0x0, 0x0, 0x60},
-	{0x2d10, 0x80, 0x0, 0x0, 0x38},
-	{0xf2b8, 0x78, 0x0, 0x0, 0x78},
+	{0x4090, 0x80, 0x0, 0x0, 0x38},
+	{0xfea8, 0x78, 0x0, 0x0, 0x78},
 	{0x1f8, 0x4, 0x0, 0x0, 0x4},
 	{0xaf20, 0x0, 0x0, 0x0, 0xf0},
 	{0xb010, 0x8, 0x0, 0x0, 0x8},
+	{0xc00, 0x8, 0x0, 0x0, 0x8},
 	{0x1f8, 0x8, 0x0, 0x0, 0x8},
 	{0xac0, 0x8, 0x0, 0x0, 0x8},
 	{0x2578, 0x8, 0x0, 0x0, 0x8},
@@ -4492,23 +4568,23 @@ void qed_set_rdma_error_level(struct qed_hwfn *p_hwfn,
 	{0x12908, 0x18, 0x0, 0x0, 0x10},
 	{0x11aa8, 0x40, 0x0, 0x0, 0x18},
 	{0xa588, 0x50, 0x0, 0x0, 0x20},
-	{0x8700, 0x40, 0x0, 0x0, 0x28},
-	{0x10300, 0x18, 0x0, 0x0, 0x10},
+	{0x8f00, 0x40, 0x0, 0x0, 0x28},
+	{0x10e30, 0x18, 0x0, 0x0, 0x10},
 	{0xde48, 0x48, 0x0, 0x0, 0x38},
-	{0x10768, 0x20, 0x0, 0x0, 0x20},
-	{0x2d48, 0x80, 0x0, 0x0, 0x10},
+	{0x11298, 0x20, 0x0, 0x0, 0x20},
+	{0x40c8, 0x80, 0x0, 0x0, 0x10},
 	{0x5048, 0x10, 0x0, 0x0, 0x10},
 	{0xc748, 0x8, 0x0, 0x0, 0x1},
-	{0xa128, 0x8, 0x0, 0x0, 0x1},
-	{0x10f00, 0x8, 0x0, 0x0, 0x1},
+	{0xa928, 0x8, 0x0, 0x0, 0x1},
+	{0x11a30, 0x8, 0x0, 0x0, 0x1},
 	{0xf030, 0x8, 0x0, 0x0, 0x1},
 	{0x13028, 0x8, 0x0, 0x0, 0x1},
 	{0x12c58, 0x8, 0x0, 0x0, 0x1},
 	{0xc9b8, 0x30, 0x0, 0x0, 0x10},
 	{0xed90, 0x28, 0x0, 0x0, 0x28},
-	{0xa520, 0x18, 0x0, 0x0, 0x18},
-	{0xa6a0, 0x8, 0x0, 0x0, 0x8},
-	{0x13108, 0x8, 0x0, 0x0, 0x8},
+	{0xad20, 0x18, 0x0, 0x0, 0x18},
+	{0xaea0, 0x8, 0x0, 0x0, 0x8},
+	{0x13c38, 0x8, 0x0, 0x0, 0x8},
 	{0x13c50, 0x18, 0x0, 0x0, 0x18},
 };
 
@@ -5661,6 +5737,14 @@ enum eth_filter_type {
 	MAX_ETH_FILTER_TYPE
 };
 
+/* inner to inner vlan priority translation configurations */
+struct eth_in_to_in_pri_map_cfg {
+	u8 inner_vlan_pri_remap_en;
+	u8 reserved[7];
+	u8 non_rdma_in_to_in_pri_map[8];
+	u8 rdma_in_to_in_pri_map[8];
+};
+
 /* Eth IPv4 Fragment Type */
 enum eth_ipv4_frag_type {
 	ETH_IPV4_NOT_FRAG,
@@ -6018,6 +6102,14 @@ struct tx_queue_update_ramrod_data {
 	struct regpair reserved1[5];
 };
 
+/* Inner to Inner VLAN priority map update mode */
+enum update_in_to_in_pri_map_mode_enum {
+	ETH_IN_TO_IN_PRI_MAP_UPDATE_DISABLED,
+	ETH_IN_TO_IN_PRI_MAP_UPDATE_NON_RDMA_TBL,
+	ETH_IN_TO_IN_PRI_MAP_UPDATE_RDMA_TBL,
+	MAX_UPDATE_IN_TO_IN_PRI_MAP_MODE_ENUM
+};
+
 /* Ramrod data for vport update ramrod */
 struct vport_filter_update_ramrod_data {
 	struct eth_filter_cmd_header filter_cmd_hdr;
@@ -6048,7 +6140,8 @@ struct vport_start_ramrod_data {
 	u8 zero_placement_offset;
 	u8 ctl_frame_mac_check_en;
 	u8 ctl_frame_ethtype_check_en;
-	u8 reserved[1];
+	u8 wipe_inner_vlan_pri_en;
+	struct eth_in_to_in_pri_map_cfg in_to_in_vlan_pri_map_cfg;
 };
 
 /* Ramrod data for vport stop ramrod */
@@ -6100,7 +6193,9 @@ struct vport_update_ramrod_data_cmn {
 	u8 update_ctl_frame_checks_en_flg;
 	u8 ctl_frame_mac_check_en;
 	u8 ctl_frame_ethtype_check_en;
-	u8 reserved[15];
+	u8 update_in_to_in_pri_map_mode;
+	u8 in_to_in_pri_map[8];
+	u8 reserved[6];
 };
 
 struct vport_update_ramrod_mcast {
@@ -6929,11 +7024,6 @@ struct mstorm_rdma_task_st_ctx {
 	struct regpair temp[4];
 };
 
-/* The roce task context of Ustorm */
-struct ustorm_rdma_task_st_ctx {
-	struct regpair temp[2];
-};
-
 struct e4_ustorm_rdma_task_ag_ctx {
 	u8 reserved;
 	u8 state;
@@ -7007,8 +7097,6 @@ struct e4_rdma_task_context {
 	struct e4_mstorm_rdma_task_ag_ctx mstorm_ag_context;
 	struct mstorm_rdma_task_st_ctx mstorm_st_context;
 	struct rdif_task_context rdif_context;
-	struct ustorm_rdma_task_st_ctx ustorm_st_context;
-	struct regpair ustorm_st_padding[2];
 	struct e4_ustorm_rdma_task_ag_ctx ustorm_ag_context;
 };
 
@@ -7388,7 +7476,7 @@ struct e4_ustorm_rdma_conn_ag_ctx {
 #define E4_USTORM_RDMA_CONN_AG_CTX_RULE8EN_MASK		0x1
 #define E4_USTORM_RDMA_CONN_AG_CTX_RULE8EN_SHIFT	7
 	u8 byte2;
-	u8 byte3;
+	u8 nvmf_only;
 	__le16 conn_dpi;
 	__le16 word1;
 	__le32 cq_cons;
@@ -7831,7 +7919,12 @@ struct roce_create_qp_req_ramrod_data {
 	struct regpair qp_handle_for_cqe;
 	struct regpair qp_handle_for_async;
 	u8 stats_counter_id;
-	u8 reserved3[7];
+	u8 reserved3[6];
+	u8 flags2;
+#define ROCE_CREATE_QP_REQ_RAMROD_DATA_EDPM_MODE_MASK			0x1
+#define ROCE_CREATE_QP_REQ_RAMROD_DATA_EDPM_MODE_SHIFT			0
+#define ROCE_CREATE_QP_REQ_RAMROD_DATA_RESERVED_MASK			0x7F
+#define ROCE_CREATE_QP_REQ_RAMROD_DATA_RESERVED_SHIFT			1
 	__le16 regular_latency_phy_queue;
 	__le16 dpi;
 };
@@ -7954,6 +8047,7 @@ enum roce_event_opcode {
 	ROCE_EVENT_DESTROY_QP,
 	ROCE_EVENT_CREATE_UD_QP,
 	ROCE_EVENT_DESTROY_UD_QP,
+	ROCE_EVENT_FUNC_UPDATE,
 	MAX_ROCE_EVENT_OPCODE
 };
 
@@ -7962,7 +8056,13 @@ struct roce_init_func_params {
 	u8 ll2_queue_id;
 	u8 cnp_vlan_priority;
 	u8 cnp_dscp;
-	u8 reserved;
+	u8 flags;
+#define ROCE_INIT_FUNC_PARAMS_DCQCN_NP_EN_MASK		0x1
+#define ROCE_INIT_FUNC_PARAMS_DCQCN_NP_EN_SHIFT		0
+#define ROCE_INIT_FUNC_PARAMS_DCQCN_RP_EN_MASK		0x1
+#define ROCE_INIT_FUNC_PARAMS_DCQCN_RP_EN_SHIFT		1
+#define ROCE_INIT_FUNC_PARAMS_RESERVED0_MASK		0x3F
+#define ROCE_INIT_FUNC_PARAMS_RESERVED0_SHIFT		2
 	__le32 cnp_send_timeout;
 	__le16 rl_offset;
 	u8 rl_count_log;
@@ -8109,9 +8209,24 @@ enum roce_ramrod_cmd_id {
 	ROCE_RAMROD_DESTROY_QP,
 	ROCE_RAMROD_CREATE_UD_QP,
 	ROCE_RAMROD_DESTROY_UD_QP,
+	ROCE_RAMROD_FUNC_UPDATE,
 	MAX_ROCE_RAMROD_CMD_ID
 };
 
+/* RoCE func init ramrod data */
+struct roce_update_func_params {
+	u8 cnp_vlan_priority;
+	u8 cnp_dscp;
+	__le16 flags;
+#define ROCE_UPDATE_FUNC_PARAMS_DCQCN_NP_EN_MASK	0x1
+#define ROCE_UPDATE_FUNC_PARAMS_DCQCN_NP_EN_SHIFT	0
+#define ROCE_UPDATE_FUNC_PARAMS_DCQCN_RP_EN_MASK	0x1
+#define ROCE_UPDATE_FUNC_PARAMS_DCQCN_RP_EN_SHIFT	1
+#define ROCE_UPDATE_FUNC_PARAMS_RESERVED0_MASK		0x3FFF
+#define ROCE_UPDATE_FUNC_PARAMS_RESERVED0_SHIFT		2
+	__le32 cnp_send_timeout;
+};
+
 struct e4_xstorm_roce_conn_ag_ctx_dq_ext_ld_part {
 	u8 reserved0;
 	u8 state;
diff --git a/include/linux/qed/common_hsi.h b/include/linux/qed/common_hsi.h
index 0081fa6..03f59a2 100644
--- a/include/linux/qed/common_hsi.h
+++ b/include/linux/qed/common_hsi.h
@@ -110,7 +110,7 @@
 
 #define FW_MAJOR_VERSION	8
 #define FW_MINOR_VERSION        37
-#define FW_REVISION_VERSION     2
+#define FW_REVISION_VERSION     7
 #define FW_ENGINEERING_VERSION	0
 
 /***********************/
@@ -931,12 +931,12 @@ struct db_rdma_dpm_params {
 #define DB_RDMA_DPM_PARAMS_WQE_SIZE_SHIFT		16
 #define DB_RDMA_DPM_PARAMS_RESERVED0_MASK		0x1
 #define DB_RDMA_DPM_PARAMS_RESERVED0_SHIFT		27
-#define DB_RDMA_DPM_PARAMS_COMPLETION_FLG_MASK		0x1
-#define DB_RDMA_DPM_PARAMS_COMPLETION_FLG_SHIFT		28
+#define DB_RDMA_DPM_PARAMS_ACK_REQUEST_MASK		0x1
+#define DB_RDMA_DPM_PARAMS_ACK_REQUEST_SHIFT		28
 #define DB_RDMA_DPM_PARAMS_S_FLG_MASK			0x1
 #define DB_RDMA_DPM_PARAMS_S_FLG_SHIFT			29
-#define DB_RDMA_DPM_PARAMS_RESERVED1_MASK		0x1
-#define DB_RDMA_DPM_PARAMS_RESERVED1_SHIFT		30
+#define DB_RDMA_DPM_PARAMS_COMPLETION_FLG_MASK		0x1
+#define DB_RDMA_DPM_PARAMS_COMPLETION_FLG_SHIFT		30
 #define DB_RDMA_DPM_PARAMS_CONN_TYPE_IS_IWARP_MASK	0x1
 #define DB_RDMA_DPM_PARAMS_CONN_TYPE_IS_IWARP_SHIFT	31
 };
diff --git a/include/linux/qed/iscsi_common.h b/include/linux/qed/iscsi_common.h
index b34c573..66aba50 100644
--- a/include/linux/qed/iscsi_common.h
+++ b/include/linux/qed/iscsi_common.h
@@ -896,7 +896,7 @@ struct e4_ustorm_iscsi_task_ag_ctx {
 	__le32 exp_cont_len;
 	__le32 total_data_acked;
 	__le32 exp_data_acked;
-	u8 next_tid_valid;
+	u8 byte2;
 	u8 byte3;
 	__le16 word1;
 	__le16 next_tid;
-- 
1.8.3.1

^ permalink raw reply related

* Re: [PATCH v3 3/3] IB/ipoib: Log sysfs 'dev_id' accesses from userspace
From: Stephen Hemminger @ 2018-09-05 15:47 UTC (permalink / raw)
  To: Arseny Maslennikov; +Cc: linux-rdma, Doug Ledford, Jason Gunthorpe, netdev
In-Reply-To: <20180903161316.25121-4-ar@cs.msu.ru>

On Mon,  3 Sep 2018 19:13:16 +0300
Arseny Maslennikov <ar@cs.msu.ru> wrote:

> +	if (ndev->dev_id == ndev->dev_port) {
> +		netdev_info_once(ndev,
> +			"\"%s\" wants to know my dev_id. "
> +			"Should it look at dev_port instead?\n",
> +			current->comm);
> +		netdev_info_once(ndev,
> +			"See Documentation/ABI/testing/sysfs-class-net for more info.\n");
> +	}

Single line message is sufficient.
Also don't break strings in messages.

> +	}
> +
> +	ret = sprintf(buf, "%#x\n", ndev->dev_id);
> +
> +	return ret;

Why not?
	return sprintf...

^ permalink raw reply

* [PATCH] Subject: [PATCH] rtl_bt: Add firmware and configuration files for the Bluetooth part of RTL8822CU
From: Larry Finger @ 2018-09-05 15:51 UTC (permalink / raw)
  To: linux-firmware
  Cc: linux-wireless, Larry Finger, netdev, 陆朱伟

These devices are new models from Realtek. Updates to driver btrtl will
soon be submitted to the kernel.

These files were provided by the Realtek developer.

Signed-off-by: 陆朱伟 <alex_lu@realsil.com.cn>
Signed-off-by: Larry Finger <Larry.Finger@lwfinger.net>
---
 WHENCE                  |   2 ++
 rtl_bt/rtl8822cu_fw.bin | Bin 0 -> 22412 bytes
 2 files changed, 2 insertions(+)
 create mode 100644 rtl_bt/rtl8822cu_fw.bin

diff --git a/WHENCE b/WHENCE
index cda8a6d..30be952 100644
--- a/WHENCE
+++ b/WHENCE
@@ -3149,6 +3149,7 @@ File: rtl_bt/rtl8723d_fw.bin
 File: rtl_bt/rtl8723d_config.bin
 File: rtl_bt/rtl8821c_fw.bin
 File: rtl_bt/rtl8821c_config.bin
+File: rtl_bt/rtl8822cu_fw.bin
 
 Licence: Redistributable. See LICENCE.rtlwifi_firmware.txt for details.
 
@@ -3157,6 +3158,7 @@ From https://github.com/troy-tan/driver_store
 Files rtl_bt/rtl8822b_* came directly from Realtek. These files are
 updated on April 14, 2017.
 Files rtl_bt/rtl8821c_* and rtl_bt/rtl8723d_* came directly from Realtek.
+File rtl_bt/rtl8822cu_fw.bin came directly from Realtek.
 
 --------------------------------------------------------------------------
 
diff --git a/rtl_bt/rtl8822cu_fw.bin b/rtl_bt/rtl8822cu_fw.bin
new file mode 100644
index 0000000000000000000000000000000000000000..558037c14cb51243528cfb433c6ad4a829b716f7
GIT binary patch
literal 22412
zcmeHv3wTu3z3<xd%FN`oGYMpX2tyKHAz(-XsKvSo(ejueDk_57JqhE4nLUXHY;AjN
zH;C_`ARrLvxx-9A8iENk33~2n&xrx6jnP5?^+0VSpeM`&1`J@^TXKK@Ju@MA)Sh!s
z&;8DKzdPTz_F8+*dawWbzt(^M@3z~dRradtqLn|<s~E>rtmxLYBZ?Sf9k=4~Z+?HN
z(pHxGg@)2VS?QWm`Mr{WfKciePn5==(~+JYh&^W@JtLqz{k>6}w>K(##gcWQ-sl>y
zTq>8k!@bceFS|^6`Y0#87_JXhmi9))&fe(iPIkGnWDVZd;H|22LfeFZd~!nDL_8-d
zPdWOsSM73z>O*q0wsTxy9C{lU7?0=p_;Ui%69N<bvibgrluN0c8R7|M{?&JxUAu*y
zS64B6;U~O&A2Z2|5e(hDyd%RTM-KBcpJkFSeay?{sV4c!4|#cgno0gu4==x&Zjz6D
zz{}Nm7g6U)q~AqZH{yE4E4;isE88UB@LOI!HryoNjJO|lRv=!o2<6cB-=U9jh@U{b
zAMqxWNzM!8W4!tCXKXgg6KiM3+O3VR-7|x0=ZLS}=(csrx-G{_Jw3Q?&*06`$)s$*
z+?CyC_A{4!BHJ%^a?S<8;cY^9EQp7$rG-Po{gytlliA0$@bXQV!|Z*$T!{JKzKfSj
z5r49qmw$<v<_+MGv)(kx#~ZOGyh+v|tw+!y7!gtsc!YEW6G9T;##WY;m*aXOs12Bc
z^G<66RVOTc%+Bm@CS(0uO!8Yzyo?;@|Efx=)E&_t9_AOD2Cj^8O2MpvCEDPtbg1&r
zUm5AjXnU?v^`OQ!aAjDv)TiQ&)$YEuU}`Q~k%C_GOW8$}RS32^xOoFk>sieM{kEhn
zx((X(?7XJRq+$m=2TbxNj7hIUKp*FoK9T}Lu~6&|>Z8o%4r&6HKH(&@H=`WQ^_L#j
zszK}J#X^5sv8As*$n2Cy<7*CjE}AhfqcSf~^nS0ZjyVlBvH8!JM$MOkstYQ#D0Fkq
z*nAiL&LpEx7jvSXv^DtMc2ju&nU3hojohk9w(`>Nv~T!KZqCo2JN=de?Vo#o-Vosm
zKkGAv*MvCnDF^TRahRL@Szl?}?&i`qx!DxjcUmkFOD4LjN?s3FmFNo#56qq3{<-|g
zqyyy?zdyOH<3ejACpCvB=BF$jFIr0{wr~Dy&91Te^1d<oI|tjZ40EC>#7hk!>MsZF
z=d@Alo@S+esjB3qNL7h$inY{yV8iF0?w!B$9^&Q?NcBPE(gxeW%1nQKNM*|mh<n8@
z-Y>qu`#mqbak}G*uk@wJ#?RD?H8#$j(&p(FyRNN+fb}&a<lJkL8xe~$0Sk!z2(Kfw
zA`sq5@~d1zLs@ZIu@qG2GV`8U*Um8Q1C>ekiw!DgWl3d;HDvG`^F*(4na0M<Dc@ms
z4bLqA^z&k8?Br7&7H_+{d0=I(nB})bTfL*lVNcv|hBDesA@^tPO~S>y_r4y?ESTSz
z7O)o2J1uxMxmArCz@(wT=oh@krCOVz-FSeR|9p!|Lfg(5ULhExeSUW;;0OKO%yG=3
zedJymXJuo#WAe&$v2dAMELfQ&YWOKD3&g1_jUsc-;W5w4!)i+S+(5@(mFq^ns!_Ev
zS=6k|6uFgZQR8AY<M^Dwj=d-MYWYzC&xPmqdM^BWZ^w*|8SBnzT-p5afDnCduMqw9
z-b>k;ZJrB<8inZEM$RRjlfuuH9zv;<HcNC<V|~bdj&rT{K3Y=M$t#%D*6C@8&4Y6$
z@MAvkV;FOxeNoQkVS;$A!y49TE$09ELu(&%jk0SBo^;IJyReU2f|Is>iGxlWv(sFe
zd3kwM(g##-CW+f=Z<<5a5SPbctp=PpMZm(e7k{oYRO%0>w5lXlLu0_{b!vJzN$<bv
z7QV=?(O$`})g88&hj(dv3|sU#%i5~hoQkbqnIy;R4pi6ov7!Oz^!co)qOa9~me3BX
ziM5lYw4*^g=6}-;wehE_<Vxca!(m~E#=-6`5UEzSU)wXg87FmxMk)+mb$47+*HmB5
zs-;ordSr9ML}s78$9h%`ETf;4>KDD*JWFSapP4_r#d0D!Fd)fp!M8V3AKVwL4+;m9
zw-`4V)^mUmT^02?2DsFM@GH#!eAU=W%aJjMX-)J!Zve8m6&XDT{Byn8J?(y>iRCQG
zAKudz5HHmDw37pYjtfOT?CIGS<ZSS13dIYxzT{)-JkFhbe4L{xaArHJF@C7}q1N_`
zfav3vZFLxry@r#SUX#8|B@Wa`=XM2{Bp+Pk82KT~`F{SWkGu-}TcT-ROY}}}`te5{
zsXeKi_JszV>q48_o@z`xJo43H$2Pagjia{k$F{V|^^Fc|#Bf|wh`i?dprL1zAFxqf
z-!3$*aioN^j(IMW?^@p`L>qS%*e%h;jYj)(j(LuCSPjnI9DK1|c>Be6&#twA#)ihs
z<3B+<<=Eax#})7N*6F6>Ir$@c%tuFlNb5Z6qt)%4<k^=JSr?Jp<Q*xI6uaKRE{>E0
zq9D@BcC_*Ks4XL3HE$g8N-bJ^xp6{1b3g5H+8wRG=$&hqFJyQJ9yN)c3-i3%YIE>#
zRDZjsXuiX6iZz*ns-UWx6S?ZtuzHnV<SGWLQzE*DPT3YXH2ECAz_GktetS7)UTCkv
z(`py-OtCL)Gc<Y|2c(=L$zI=gH+o&)xX*6PUxt6{?ahv#0pq5Ild%>W{%2BZc(0>5
ztl!KovugL*eSO0=FD_=6<<i#aZhI&ap?S~jo4MB!iMZ{lo7>{4^v!R^Q)!!jsidTt
zkx25EDUsnD&Xl+#11>4znaOYNof+wCo5{IfKg&H}+<cRxXxfzXzeITdJiFXGv&#|b
zJL}lv=$>|F#`|Ym5#D!vJbm9;>8xRkWtwGL#;cLDDO=v1t~-2YM&_3HBVW$Q##8nE
zj~!g?xpP@temCR&$mJPaZTMU_QWX)?D{}wnnu-YeMk^}ky0e;|XC0&c+ACRiUUerO
ziJj9{#}P*ipN!pN*)RrYHls=lTu8K+cn8Y^F}&hKdJn>G1j3sM+`pS+2MG6Y$WKEU
z3EE9GTZJ_77)=Oj1UgHEYX+Q2f%>5ATNu{lPmpBuN0Wqu2|Pyi|5%k;nQ|og@H3$K
zM(45+SMXUMALQ!NY)|>`Zc-Pl!THeUtqE{A7e51U%Un<T>$-jh-U(b0(Q456zXL^d
zpYcTNeA<<uXRaTDr_%ADf}rOT=wR3;n~wr7tc>w4D@`J|0RKt#p+(EYu4G${a_TtX
zTd_;>i(Q=4(4*p++i=_&ufbne!(g)yslhq*N@@LZu~pcEcI%47MBl2V!n>1f%L0OL
zK;q`t2l0OJfk2W?<&@1^uoA@i>3FqMI4GNUPvRuz6y8+<%iRSmcPHT+>8e}Tvj8d2
zMIG>5wu0x9dgcdgKIR0*cFqlWd`wyxQH?`u(jET%2Eu&YSZ>7}V4>go#0&d<MZhBI
z-uBPAd*5_$kH6{A6z%IHx@w6Y^X{2W>DRqe&ov`8v%P6oE8+onbKm%>gx4(5d%QJ4
zOTjdZAuX6wP~@K$nD#~o>h`||T;da=%YBicI|#fNYz}@9OfHxV%&0HWzoIKJY}QOs
z7iOZSI+u2Sf;3vem81QvGZ|xA*=dQEb~4v!tho*=J#i(sfW!WYU8!4AHY5W!ld6dS
zAoxqbBEcomaN=tTZi%-gzLsE*cw6FYiN7WIs|TGYcqEu6`1T?Y+!Ea+7$!c3=#<zf
z?6?ikP_%NYm<I@=vF1tR`O=l+ZNmbfjLy6~f{si`D+i{DU9vg7nAv6Xucpy+_ar8z
z`89w?qE9K<IZo0zue#+g(y(L0<CKYWay<A?NI2Rm2DvW5hYzK51+9XITVXvrW-N0}
zB)I_aFd=Z&&Au3gmdv%<qne%=ovZh}?m6rUdEA~mJ#wjMddG}N*d4*Rr&XLQ;75Tk
zGq)x<Gkq@Lt0M+qkS4m4d71@{{MsxuD3KUFaZm(zmM_wWj6X&viApki8`dqc#>6Mk
zTw?S-Cy?X+$qAJ!2Rlk>t~$ZHj|>pa5KfLDdgLN1;c3bS?=D4BBSKJ$4D)wfxYL^w
z<a3!bGqB!q)!lnx1o|H~WY)6zvB}Jy;53;1N04(m_L<~_Jis6=(|8g%iDV*L-`LnJ
z=0AMQ>|^~M(QV$2=o66H+~JPskC9#mIgTV=c9~?2kpU@KkUOGH*ul*wuJ+yWpk_`^
z8rDASz$r&&D>HvGUeg0u*vicN$M0z{`Pqd9-n0T*KOVa;Y*k_P*u^n5pIz3*ci&?E
z4f;-uJT9vVBx8|$Mer>a#m1_FyjSjJ_UHbA<e&jpeQ0gdoF=VxNyz9VEVop_kd{6Z
zWF0EQ2QvKfr9Iv`d)MxLGpMzif_I%UI+unCkCywKPrY$!xziMWGi(aS@=U=Mrv%7U
zb09Cx0TfdkG#*`>cyURi%4H2Prz%gp_`uqjJV5^qRB?x=;{PWaRp7CiOO-2L{L$LU
zep~0HfP7K*4OGh)Py5z%$`@Dr<V$-Ix_wXKZH@1<&V8M0ky`8fpz|5TFZiCt^F`lt
zcs}cU9?z$JFW~u%@5N5}(mLPEc)sLY-#K6fuhJ@a42HJeL*m69zWPqjCDHej&n?g|
zu*)4YYCmI_56&Qc#XB>=<6W*Q>F9Kn80qY+xAbv<RT}>$%&`qLsvTjU9`J&21>qku
zV{;pDS|iqQcks<%#(~t`HM`&1P4t;vK0WiunG0u3!TZm!%bw}%vTwS#Pc4F1n6BX~
zBO0VcC6`-20C_XkAN8fc+mR&V%B#yA&3#ELrkonM^3T2wUur<yesvzTWtq0#q3L0l
z^QH+QQ1!WVD!99(KFup<MZn31B!!!1cA~8#l(Rt?c1(fLIN;h-_CbXdQ9q;-ug)LF
z?<so^ZyLM}%unOLSEsVgbsCY|5ig^fujUslRoT?@g+qG(KCjrg&|4KziIzUkp*fIV
zg+sK{iFG2HtvSr9liFnSj|wx+aC5ObeK#z*V$*KXU`(W+*^N0;o$(6fjrUhIt!;WU
zv}Zb?FZ9?MV_X(q?iDVL0qi{yHiZcOUMPK8k%_BHs!EoGG*@`3;|hO|DcBHHB<Wxb
zD(4hyZL{Ym((LXOTk-tmt{(?Ux(3YSgtE&7yUhN<H{{3WLU5e|8gr+2z$Jz6IU_}+
zux<)?yTST8;JJISv)6KH@7_T$0+QrXs>?1vRXX5WTRM;G2Q}lg;3A(tC1I~gN0KO?
z(2ek<>2TD#)shh8YH0VLy~5R|oDH3~o}V@C*(;1>63D|-&ZnGLOSNtx#MNBWf?%(c
zFlK~7_;E5<b5QNbad$f22@$84H2z0tI-WTcdB5*}RSeX0N5IF_bcgiI&f8M=i`&qn
zGwQIO*Yv2RN~LwpE8;fQ%u_S1=&wr2846p}S3<_3(WZpcoliRIgNqJEA@ST(eAht^
zW3CU*DZcyQJpY<fq2ZO%jR%DwFY)e*$Q_tTN<@Hef>oDc=0Y$-(m-Qq4Xa(Z;XNg^
zE`(Oy67W0zQ5jB&-lUZ(n+}w6#(rU&5Hv|Ycch+K?>miB)<_z9`rZr<&v~EU2))nZ
z;*RYV5t@@VVv>Y&x}z~z)GT?|MzwiKw{hS~I%<mBl~Fh~>Wb5{FKHF}0Jo^o+;zqS
z8JnBm9MZaLKTC2qhn622z$wx!4cdtIsHF$ov{R}mt9{ZzyL7+r2fm*>gyJVk%0UTM
zH9p|xYTUuc8`Y9j;tqbQ@RJqi4VXENO9)xd52cF?>#0Hb>OTIoRoH4g_S5}BFinbS
zvXs5T$c0dvBy1Ie(<E`59%~w#tq_`~<fb9_in2C|a<?Jpe_UZo&D-K-zP`m0$+)@q
zi!oAtP~1hkP$G#X)tH004Wo`wpI_Iy3@%3sc5E9itqW2u%~6y+m*}Zu8~Td>L;10>
zmq_Tx-5fM-?%1|k=}Rtsu5?|HN|eOM*pZk=yv}WsW@E>;LP-k7`uUIbvgSY33(Z3c
zeiz!lp}qXIRjc}jRim96xRR~x!1cR?)xgRQJzoiUh~ZSygOWZ(2u_j!Noau;5jNn9
zXxCFCla%^{ST+tz30#n;&;a5GIPB@z*0KA?=L%c3M`<5>X<yuE1=3%4MDw9=KDILx
zCGEyB?!6IfWO>PGr!}N`)g2MHbpz@q4#HQOL=cwF3tKI~TH2%9je4a<5-7d6ts-KP
zXmr=ljA%t-Zi(>~T{pg|tyA|;`m5ucf$^nBIrJx%SR+zNdK4IP$S7xgWt8JE%EXK{
zUpwPcYB4_JwCma`9n#jHG2`oJ6r&l5IVJG9u{GZzZ1wCEf=1Nd8IQe6JQQ!!nNO_4
z|H(d0#i(NtQ3pO$^Xj9}A;k7+>X1<<_9^#k_bK<U>=RL}#27~eWFHSatI9R`^gLlg
zAvjtpD@rRG0IkmiZXS(3vWj}YU=kmw@4GEJ#%c`<+t>BcSxSj$0Cgw<9Gru5{0HRa
zK#zF+ijs_wSWg}Lv!Fjs?n<8qJVSM7jDAuXf+ad1nSSx&R<8>1os=hD-0l@G{#fDA
zO?+Blrw`Or>W<V$m}DGBazZ!8{@_y_fA3Qmkz@_aW`4>w-7~dGZBx5Czs<`5mNmJc
z-DbyNDf<88cp2!Zy)q<mDM-{IIaeR&x02eVAXh+~(4|7=j%h6tV?KhqUGe%!fz>`e
z=BLfIM7R57^Vz{(VzLsAgUV>)Wg1YXa_Y75CdQiVU+opJ0%>_h=n`vvIO!pEo+bLC
zSH)YRFL-71mxC=Q#zAc-`(<V2zFO~DrQTN5vkdl@s7E^xt84q_x#!Z{@BKG*dol0N
z5eS}c4-A*&OV1rF_w(*vudkEG>GO8#-K-`O@pP{4?CiSDzqXU>PnOnp$(LU6(P{Ub
zIN0?<C-%{IRr-s&K8O@FA)21)itNmsW22ch^Ddj>vJ;c)rx*D-Yx^nY%6L%xr1JeM
zwkd6EPVmw*U4?Bgop@0C`>x6DliSvx@So601?@sq>S{Y7L|1h|$KG{AY$e^zq3p@|
z0r5oGRoRsn`0)v|OGLTczz<I32A)8Wqv8pxONj315>ZQzF7HZm-Jr<eseaBSUwWkT
z#qh|$hJ#t)EVOv9=`;hX5f0wuk}ujI!&G+0rF(^!{u1j$JSfSZ4DrO@@$y^*2Hkxk
z2>$|6{3P-b79)rV#D5NzR}%WM$G}4{X{o}m=pkd)>`%ObqJP0^E6x+1ww_mV!B<l0
z+m+J2SH{$W{{wCnw=D=J%5gOe^yYT=Lpc39;8G!ochtFGsUvRFTo~FWa^r0#{#V6&
zq#9g}^}G;b)kTiEXpKofQ`*3IAWhyuunD;5p@ks6bdLWoZp$ST_E%S%bkuNE8`EeU
zHf)S>cjHd;OQ@!F_FrstuN`Y&2pWzWHco5(^W__Aq0zGoTWS0S_POA8=Q)Jn{cdQT
zP*w;wIt&~2TZFB*6iLcjU?qY-n;UC859gdD*wkEtHnn8IScFR)#*$Q<7{^F|l~>hO
z)u*>rdTB**vlU$8SP-=K{S3GfoT5v8+S-?KT}xW%f2gG_w4@(wNj864xZZJ1lXZ>1
zCwf8fL9jt*P!C8V{GoxC?9;*xJ~hDNi5suS4|LWnxHTAXTEo`J8>hKd)<~+T&R4A(
zC0cOjbf<4gqzu}+TAwMh_f#@&oG51w-Y9SHDc3370s3605)#xf;8iM7juPjxX+3nu
zlKm>2M|LT#W)kKR(-$uTUQEC>$<PUzhsOS8ReELGk<`N^Es>;DXlJe${ENJL_x+z$
zMwnY8?kQ_OjS)wl-v8Fl=N)v9vSw$R{awd~&wsmf!>QDrA4ksZG@UyMDXarh{J|NW
zG5LCihCg+V<m+A~_woHte%kZS>Yb@O&+SaHKeaPa|KrGCulq>rlf5V2S?$U9L_8mO
z(mg5md3JN8!10`8+L@<MnIdm|Iw;$pC)vIivVEr_+jl6k{e1_Xgto!^4kRt$5dE^3
zZJ+M8@7}ljrQPm-bz?Qf64E+UMyRf*M8mHM(YR$L_j?BhYNhks?nhAG5)J;^-8;0#
zOS`|W&MmEU4>Z=}@a*$gqVILMKfSJw=j-ZB**^d&%ZBx_M4R7AwpZ_bY3D@8bv0w_
z`jzpG**_pj;Tomhy0<3VZ96CJOvd^+9TOe)PbUq+N?D)UW^ky2G&82i&jUEwJfv$L
zdf=K)l`|!nk*g&b`Y73-^r8N^;n-i?8j1C11Clc1C&G9%eM{PgROk@&&>;{U4eI8g
zE6E36aEJG{+`LxUTkIFM>ZHfr8sIsuMhcD%;HLg#;y0LmOhDew0r#ZXWB5*cbgtuZ
zU?3r=!`<S6aUA#(s!MX67V_a3aLeNMfh(hd1C?^2M7cCz3?Zli2P1xb8|IVjUk(hO
z(7Y%-E_8_l0|Qq`o@0#ZNnR$NhVXg<FDs*nq#a8-E?m&(3m0_bn2YqZw`OQJrELaR
zw_<Jayw7w!9Zls9Zhkq^tZ6iGad$8+%Idn$dZHu5v?zCXjtDGVCr34ixx4iuFiqVe
z5nOKF55!rye1=i^H$zNO;u}P8{3xvw)fvN)@<h|b;Ze?28B7&T%0F3RmJ*K^c_r41
z=lGFGJ?ij8d2ystRx6t0WqbJ?q>3CJQ6^$^bsf=co96Iwe!vY4fYcpsJ|{TenNe}B
z7HifWSyM{u9d*3T53G2`;qn)rLn)U~D>S?_T?$7cg%QJ1{YTIjU=MVg2p@Zzxtius
zRjFoUVdS;xkx1=%sn2?TnLqVBQzYraaA8D1&JKQwe?Sr%3d3gpe)KiqDhxdZ{BAg^
zK|3!)l3C=h@9SttRU~EnbIuxB!&ji>21hH(G5b&W1%AyY?22pNr=F&lgY|t9YNbZX
zl`>K!l3ngs^K<-Kgxf!5zp3J9Bi3vdenE9w!cwq4*xZ*2UQqbWy*xqGddFRe{}FKo
zUrb{Hl_?AhiVhVT)Vs?>`j?XiNQVa+5t(>1uX|;nhE~qgBsiPH-I#MlWGw2gDecDE
z(7L39YaN##t7#xObq*}BhE7i&#p^awO~L8eJK(~qg(anS25L(YA&9jHtm-Gn`<$or
zdG@nOTAH&{1wLJSOuIShl{t}rzC7A?jrtJm5*!olAs8mTB4GLe;}HPo3LWx4(;%CF
zU-WvDY;G+A{qZmNYN6##+}9BwL$FV@i^^*bC$-93sfKsY8YSMnXHb4cIVxjRcxhvR
zgN8ufZYa?E>x05c$i2~K^z!?nhK5+{bl#ek^LBoK`k}g7zYzVUQtH)sDQMjW{}ZE)
z%70t`&3_vIT4f$hP4SV(>lpkmDf$1@l;H3o7Zm8g<Nm(L9Ov(;eHzP`;KdU160PiV
zhcz_Lk=<%NJ67rrGWS8;qCR*=(<5%>q(hDi4k0wh4Q-q9b_#FhZs0EbGaH!5QX5O(
zfi$nAzvn(g&{}&9-wf6JX&lMN$4bo2%wL>N`A~Q_3%nn-HMSPASr9F5qc<V=Q+LWH
z=w2YR2;EddbM%k{eZ@=s6uljCXf{Hk*(khsttVzrz`~WPj7t7U`r*k#cuR~OI5*`Z
z;-wyRj|X2YZ<nKud-Gdql~Q|BwuoD?9xI5?1QrMto=?TBnRLogc>Y0!OH9^TqDHN3
z{$iR@V+Pd5xUz>Fo0{6OUms2TunDrBR}aoAQ{f3S!1X`rSiDyY-b2@O?_STP9Nhb)
z)h6}ax0iSjeNV;~enUFwh5>X#!Nu4kn!{K5s8sNp9OS3;JcoC6!ya!L;g*^;rDesQ
z>D}@BG3z<vuIT0~1{(?4OLKSt`cM6XDYh?h_b}SIBuu+*3a42u=AYbY4eR(U<e9=2
zv;EFAYgw_{bswyrQi2J1Pw+xB1EBuXV0GYAIZ3-;9~_Ze;@5a>PnBRTR9IExQH^BW
zs5w^DrUjJ*z3RUKR5Ak=FiA)BM`^}}p7E`B$y0-6;0H$(PQ%^Zn4%31*}SQc*$3Qt
zfLD9r%!*vt5#_ik&FmIflW7v^XazV2$wPNpqi|zxi7vpq)5EGMCmT7Q3n|VeeZyfH
zx8hWlA|30|(_O{u9VXmQtOxXZ9WjnMdC6JZHL6WCo6bMs52D?45AZs|0fZp%NFT!D
zn_-g#+(PFUdeRHayhfaPqU}2L@jozD$iBsE90{DF*c-%k05PnCo65o6VRq8Rq_zt`
z=jJCBa8k+vRvWv2OM#`Ad}*FnpWFL|-u)(Q1Lf^>A7_dr;f_^PMdSQU7i225eW_|#
zW!90*!)4XXg<X)+v~uTqoZs(E<&W|kf90@5x9*y{f5oTVf;F9(H}?IuW!Ao+t$b(I
zPEJ|1JA+9s;WwJ2DZz08?NMEiuoHap7s)kul%|A}jxEGff9!5N4ae@o(|Bwtp2^1^
zE>=tP!2j^duBP<x8+G83puyUy2CP3^tm~P*lYK$5ZdPqdFbQvWl%$08N9W^dIJyu|
z<I$yfK3tp}Z()&R*p{pfnK%o{I18D;mju7G7qs58`8UeF^cctvviUXTjyHDG`}az^
z3r{|-+kjkCAe+mS@^p_q;7*Rim2n7M>4(5oY{zIlQy{};A><&y0%TTi^3n;QCHT#5
z(Xg@>J)6^#!pd8Cc6&<}yQ9U-zSok&=C)Yayq0`6zhxp@&{D{(Ek&%NrIamfxryD`
z!V8R<=Awy5jlwLps6}9RwUo2FTjsKRTC8kw%OZAf%VKt4%Mx~f%Y$r5i^v{mS<W78
zsb&wgNbKR3N7?sVR-un4(8u4Q4;qacW2?Av=>)@<f|}klWaF&z_vepe7Q?J@i{^29
zCUBI`TJ^K^E&Az<8P?G&$0kvFU*etq7d)~5e>q^BHNfUi&!1bsF~K-%9K|TBX34YI
z&s@`omZi4x=T7F-%rcD9kCbTi6jQsu{O2L<P+uP7tikc1gr4;(V<z~A&FzJS-rU@*
zm^BJvPT1Z)>RaO|D>mfKR_p|S?s)VN*<M>iI$m(C3(>pERS|mZkiq|?LulBum-v<&
ztg+JpjPU^MJ7LR1chAHNEXHlOA(wM$q4QX^1J4s2B)`SKBR$c(I7{6KkbUtla-5*U
zJ8-rdVSnRX5=vJv3G8pY=_`lXI-;|^rUK~HE<EN<n@l5DEj8NokQNpJ2PBa-NP+Nc
z+?JB%t&?myXe=}iYEx+D`qi$QpuRu{n;+rei(xJ7%e4XFV65N4v7jt-YU5>a`cG=e
zI{4tw|8H+x-bgqv#;a9d=3R4&Yl7TTOWzVl3_szly9U6ev3_WstWu;R0-gg>ffO0U
zPgo;bACg_ICCP2>GxA=}DiaQcm9W0e6ny@a7+wFu?CtZ;%s$CE%i`l)jB`o2kosX}
zoi#yk=)JtT_X}g)c$=*;-Aya#g#G1`Q04(s@HeOA=({gu_<I$dTsSyT8xEQZVD0#W
zbTQSB`<qk3_J29madLK}F)+WlIS6^;wAjQgfc38V=Nl4z(3<9y#%y@WmY4bz8t*Pv
z)n(X{{pAg+x>TEytUQ!;hCL^=SbETfn2io*9=DgJeIXliva=z38)~Prkss8QT2Mcw
zzz|rzom&YDJt+z8z+(Gr+ex&M(5++TtMeULci4n=Ozeimsy3Hs+}jgjWlB)k71yw1
zEM&Kuh5q}ivMYxl8HU^G<qp%~`+4p$gQy;>dMsH?=co7$r#dby^&0reO6p;+hM$O3
zW&d4XQ;%s=@~RnPzJGn`hEvPkr~2RQHf>IkG{?q*1}i%mNjk1O{3kcppH*YKq^~uF
zH=JS@cQhML>5p|>Xz&{F{CltQ7`4Vp$vu{+;7jSbV{dBD-M+M*dwl6Vi+z00y}nEZ
zPiORGZj9Nvs~^~B<K|lXG)q(>Tk%?U_LkurhOMXdO~IVV4r!<ioDRNdw3Ai__UU!`
zj_y7q_Dk?4<*FskapL&3AA#T4yaBXpf5BnLP@XPlgh~EU%C&n4=EAoNooH(XS*g~a
zRV?MsdL3i@tlAW+I6d0Gv>})OmmP~rH+;Uf+44oos&w1Pwr85BhLa0Nv}ul#-HZ0P
zW}|`yPHH#vys7PXz!BWsh3DQ2zaQ?n@>FSka6qc>OKJOLM`zy&K-5DG3;Wa)0a*=;
zPm=xW(?do`=aS&Ew;8rq5619Gf@cz&$MlbIY8*AtpntzVV`WG51~I}{Hj=LRMf*<!
zvk%TYIRD`8_YO9e9jtf!#Q*L=-+S#{2fEt2p6@O5GS?&gk9v3QPOp7@e>iexw`cmV
zcSj<<JB<_S-jG{f59<o$x40FP&pz5_>9+iqlYTvI#T%3C^=IeVb6=C}FE|YLiO>bD
zd1tbtr?fDfVqg8*M28jmPk%ZVdEI9Xu#g?+;0p6vmc5bp+exs#EIdEXp}O&bH%eQU
zw3T*$Fq2zBIoypinisz@<F{SPyLQ6wr^RZj@)z!({)>vX>D_Brbe~=OnAXOvpnvA>
zfBr4igx}9lO})K+_rCH2yFXAXI;LjF)e3LuRoKC;aY*6s9(d~4ca%!uTVYN6^4s&L
zH;0&{&RemYthMPja7LT+(2O=!9;|YB&eaSpEhxjGg(cJ4yF<{;ynSEE(&FyWl(z2R
z@<YNQcM#gZUG%*3?XpHyK_T$!1O+pz3JQ>p*-w)uX9BqLF@ZV0Ke>DRwKck)8&K+H
z@1%<(YLkwEyY=evT!^z^=z-P|JgX75*JR;0il5UsLcz+MOLO#3?qLCTX=Kf$iz8vj
zMdfroMxJvDZ!@RxYv#PhTgrPBdFPjJ!^XTn8~P!fccKXi8$ZH7WMfKp`*dc=#x#LT
z%9}NgffXGWQ|mHqnf~!jgi}p{C1;7hUDdF9jy|7n<J_J@>22cgIMB#d_}-}K>Usy^
zz@*&rJycT-tG~6S8yrLL^$1Q0W)n1p@So-|^=1M7PPYj=X1U6}cYvlquEpw6+eyG&
zv6X<uIr>P*2V|c}c8b@=jJw5tQ(c9P?GQW}b+bj&QjJKIZ%pnSKC^DDn7LGK`)-|H
zoW<v2oHv78dkq}$biT}^sw=jQZe`|6<5zDbO`LjJfw&shXn!2<**1~K?H*a$9klS3
z9@x_H%cr;ZuGy7d(;a?mS7v+rKDqS$@athsp=_Qo2^52meJxTI3<1|X-c;tPQ8+{T
zkC{v5p({f6x^Iq)(FEEXnv2lP`n9;WWc@mfpY?MXaT(A0N7NOIWBbR{<%-|oErWau
zzDRg)G<cWw_--gEU#n1(46GXDa9-nd2YKAmKj~O>@JYuoV38QlGS)g0kZl4s8%}!h
z5@zRIJS;IACW@V@JhXV&L-YsjU)2v&+?-PrFU6dLyv`6lhj<L4X*r;6*Y;EHXLTA;
z?l;ukEY|SCj<N%;rnTN(u(~Dw=9yCUDx%G4V&<yv+X$i_%)do}s9Odh>K;Dx@w;rB
z8g`cJ$Cb2mH63r4w9B7xu!q@n0<B0}aR$kLuLIinTk|~8yS-_31){NTrfmZHTkd_@
zyP)Zj67@=rILo;(G&XNCpNa2|6s#Q_jhgFDZ`w|6=O}IGB-%b1D?zf{ywe@@jR2&q
z*9w@F%x6Iw*5|_#4(p56i$kTwyyqX+JBX?z1M+FcYRnt6w+YbTtSx;JIulLY&TA;_
zC*U`s6EO!y_~(Si+suBs!5nxgG<LG8DZBj!zU=nwwmIREwhp>IshxAjoW3mE?Vsg^
zGi<AOiMz<=G930?;vO`Jb9z(#>I%?V^Lsak`}R9dO&m(kqPJzp^c`Rk!o7R*o}Gbs
z0C^;T(7qY{$$>h2&mkf_f*>KtX7A)DP8b75aIXpejqF2j{fL)$;#<GJ9OLEv#}SZz
z`v@<$0+uIOG`Y;ec<|pW$1;{NPcg$gvsheT=334e@+=?B;w}0542y-CEW+<IPzLcI
zkjK(2o$?HnMa*JlitwLjAj5Ja#aA<M?TYv>8I~!CW#ku9jQq(IBfkK#i2O+uBYz^r
z$e(~%K>m1&kw1=N<mV%{AU_YW_=gNjF2%o(jY<3@K1Sq^q4LNdO)>IsKx{#N4we6H
zhGi7RA7ofYQv7~~WdvdY`DTidpG`6Hha<Kie;CEc&q6HTnPJJKcwvUcL~%uiB?Ga5
ze4b+Dr&EmlG{hF<ry>@=mtjev_>K%qGR3!NSd54T<QphPzMf*_Cn2^VUq><WwTQ)}
z42y<hU4}(Xu{Oh^LM$MkqZs*+AuWQI_J&3Avi?qWEUratWUBSUzGP7q;_NR=8N=n$
z$tP}McS$#~YLP`n<p1!?<vNJ6N;(_q@l2(=X{U-~4CPPX#9Sh?WGvz=Or^50qT4Lv
z)hfIQ_!hAYVq-aD><+e;&1D^IKGIf%g{+7zVtn`AjPZ-{ejh+<3Cmct6m=fN`@>8-
zF^$7_zMR;BwmMjAKQ|u$NNE)}mhw;oGgaO#EQRA)8pczm`e$vq>gMIc)zAJVOU?LI
zYOS_T-J^O`y+vK4zCkUicd4D~+gLTiBM3zn_5_**oWII$voK9Q)|j=v%kt4Y^5gBz
zLVG8!qCK`s{VdwMOY^AusHR5!2eh|OGqk<o?4Nco<vtw$UA0avsGn0$Rq?D?^$NR1
zwVTaS{j@_=%@|v$`XF1co;0ac-Nnk(oMEnd(m1T=*pa$tkCyA!-BF`^;hw1+t1iWS
zZbHCk6XPqn2gWbw%D?jn*YksFuKX??TVAARt?Uu@fMC>}T$G}__!C}Nf3HbroS3DX
zz|lHT8&u||n{FzaXfg9&ja4e=-~C__W0UYrHD>!?`Xk&l6|`$={Cn&vkcXxOrUD~P
zjX!Th`o=))S%h?v|3CXBwk0qY?Tn2-bCJ#s#GbhEK9LuQJs}UA$oJD7N&LI!SXz0;
zu%-MDy!<fm>w^eqeZ0I1_R;&BdASX8#SO4iXKDCG8s8gF#JAHG`2P6gRFhndvY+h4
zH_7<U`hCPv)VY9flpBYe<Z1ZsdO6D45a--wk{?2wVSJaIfw&LxAJO(Mq`$;B*OQQb
z?9WW$DgG<;9cMuAN#C*j4e*beZ`XIwHzLF%T&M5A``=36L1UqD5HCycVo<(``0IKI
z8ux!x53vM&m~V;o^IxHNfIbHh6@wq*7oID@I-}2WgwI5sGQWS1?m>@sCJgyjm)3~*
z<x~XvzT}U7&~L2OzeoQ-WBO`aL)%I4;4vLUvY)gNLNK9&pplRcf;jo>br6?c=Ve0+
z*h`G(zg`#d))(xr>LR{P6OqxAikWcIzfTvDf0IdGh4X*wocQ-LxRd{8eT3$?_9G)&
zS7XOWA3<w&y*^^C88`&rx7@ph$>a7iW$&0%AJ^AGQ*;uVqJu-B2Yfx&RXKNv-v$n$
z{Z{mWkj^ydL4mWlZfXSZPl6Y{rt8Q<scX6pOI+76=4*5vqvE;_&C!u<BT?&mT}S3O
z=sHs4x{l<yt|M8|b>JBss_US6Y}&vS*|8D)XX`_;c|E_K$*su$|9X%AZhDV`7<57J
zF>z4u;psPjquztG9>$*0sKrV2jOjgc2K64<0Uhpndizrq%|{kW{K@^zdXIGcpU``3
z1?&)QAbdsgHsL;!y-5E-bTx5C2_GgSZ(F}=h)&@@st1Ae9Q>*tgwFL)sh`;iehIfH
z&gYLZHL@G$kIv_}>qGQ@`ldF9kLh_$A41=0l1`WAkuh8|WDYUCh&8Sk5r*hR%Agmq
z4C+O`qv%C$D#%5<oK&jlMW)%tv|8|<(2GohUgQS6d$vu0e+J=Tj(_ymXhpJzXhooR
zAUrcfE5Z+IMQF8{ou1dP&v4A6;sE@lK%TsH2a_uhh<4F_(OzzD<>l2}Y@fbPUou<K
zmrRT6OK#!ET&FL&;oqe%0gMgmOGZFn0t>~?VLYrxFk&7StYRncPv-H360{}hpm-xd
z@nYH%!rjpI#I+@sAv_ZCC$quRCFar#ntbc;xgq=h-%W2~`A_Igm^939<cC3PVo<au
zSqkh<kTR9^G2t20o1|Q)H_<|G@?sOd?I5U$Yfgrq6@rVQu>O!CNjW<S+V!8&qgeVA
zdXx>P)D=YSzg~|*eEnJk(xbe9nDi*bxBuo}V{{pQS)fZXLXTpE9>oZIXu<*bUUi5j
zW$o+W=O6<-zK54@{sk{@15f`D_@TG)4Xzn*|BW&Tq#L{U+k7RB`c_{_&|-CEwGC5z
zCE@<FlY`DM6TXsSK|F+;&^~wtu<w5p-_b(mcpU4!1M<r4`yqoMUIF|<{*q|E5d0M0
z?Fi1}M93cm*MwgQ&nET=w~TgRs&Dj*R0^KP+-j14PUrpqf?p)UA4C&~_a}Ut5iluV
zSmS(mQ9AK`H6IX8;&dyg9ei#43u}D&fICh4@@VLtrW^C8{cJ_Mv?ht~rU_rsx6{-Y
zEVRDW2UDUheQyok?Em{<`WyRTBKe8rs9yMB+Jl(*bCTr_Aovm55C|Tw_rau%<DuAh
zy0kZEFUHRJ*ZX0L;r_4l!$kbX`{7vKzlAR*!YD(1F%eu7o-CRg<82bYn4XXK`QO<e
z6LB&s>6$;LCxBC1F(29^8owf64*F!W_R%V9wBPKL>G&VD%3c~!E-D$m)i2XyCQk8t
zmiTTr<{N3U;v2~f-u0SqBwdeiX*4Y9vTBKfU3JIkSi(1wVG_K7d3AY~&UC-!1aY9?
zY`)by60;|GPr?%~1Glb6xD;aF;43NNDd_<Gfea^qNkMK!d)!x2+n}$c*01%Iw0DTF
zq^3b%NqesIm9%@%S5hO|9`#WpZYas8$+!DTdKF)Aldq(0LwqG|iTg@=Rq>Uy?mAyd
zuY9AgB$Ru59sDCb)AsGYlAdlCBBMT9JLoIvsju;s^f+#JulY)P3@u*sl~nyV@Rj6%
zuOtU%UTA*=PpjfF>8rky7NgggucU`j=KpnHNtXT*U-6Zci_?>I<XH##OClNKs+)Wz
zk#wh$#t!;QBDsv{u^EA6e9}|Vxql7s%?M2hB;yl)OW<nqnMC-(fd5EV7)obJwj<u-
zbp(><5@#%d2Z#Dkg6AZRM$v33ei*2&gkDT&ybX51MRiGHUY*uP<??``I36}qidRi4
zm(Y8;LEEqQwd7QMEv-;|Ev<kQD<6CWdTZud4qr=;T<2@4>MOpM9tKZ#y|1MQ(KoX%
zi^(u6Gp_TTWVGEk#B<VO@GzVkV|3pWFP(l}X}xXXkkVWYO4eNSd^9i63(rSZ`@-|;
z30$pk@J{>fL%biA0k6F5D+NwS_vYBiAKRDkd$i$Sxy4%@+^R3(d&?4y0$XhI_RXa9
zId4HYjMVfug0Dpp!}6+clI@xYq&#>&A`eJo3T{MvBhJdE#;F0LoqQk_+Q|#j^YEe;
z^Me!)_BqZ4jRg~c`H~8BQ?*m(JJbchzg~4NX{ewrQ1IZWulPY4h4s*3C9e5Fx@L_)
zJOIH|0>%j5h<?t7K9YFYguZ}eYJ#W!DKVHL_#}8GKVAfX2|7jm1HptMha9F+F?+7R
z<FEL}8KwBf85#GDGeW7woTuQ)WLSIl0eneJ`o`Ab{yh!Dk(Rd(_h0jklZEp|zHxLt
z?BX!RHxBWuSKTqL6?eb5b=Fqdewviv)dk|UNoPj9_9uvG{!<XYd*iipVnF%XU}EWx
X=Skf0O#K_&@eHJL>@Lk;e)zuu-ts=k

literal 0
HcmV?d00001

-- 
2.18.0

^ permalink raw reply related

* Re: phys_port_id in switchdev mode?
From: Samudrala, Sridhar @ 2018-09-05 16:20 UTC (permalink / raw)
  To: Jakub Kicinski, Or Gerlitz
  Cc: Florian Fainelli, Simon Horman, Andy Gospodarek,
	mchan@broadcom.com, Jiri Pirko, Alexander Duyck, Frederick Botha,
	nick viljoen, Linux Netdev List
In-Reply-To: <20180904122057.46fce83a@cakuba>

On 9/4/2018 3:20 AM, Jakub Kicinski wrote:
> On Mon, 3 Sep 2018 12:40:22 +0300, Or Gerlitz wrote:
>> On Tue, Aug 28, 2018 at 9:05 PM, Jakub Kicinski wrote:
>>> Hi!
>> Hi Jakub and sorry for the late reply, this crazigly hot summer refuses to die,
>>
>> Note I replied couple of minutes ago but it didn't get to the list, so
>> lets take it from this one:
>>
>>> I wonder if we can use phys_port_id in switchdev to group together
>>> interfaces of a single PCI PF?  Here is the problem:
>>>
>>> With a mix of PF and VF interfaces it gets increasingly difficult to
>>> figure out which one corresponds to which PF.  We can identify which
>>> *representor* is which, by means of phys_port_name and devlink
>>> flavours.  But if the actual VF/PF interfaces are also present on the
>>> same host, it gets confusing when one tries to identify the PF they
>>> came from.  Generally one has to resort of matching between PCI DBDF of
>>> the PF and VFs or read relevant info out of ethtool -i.
>>>
>>> In multi host scenario this is particularly painful, as there seems to
>>> be no immediately obvious way to match PCI interface ID of a card (0,
>>> 1, 2, 3, 4...) to the DBDF we have connected.
>>>
>>> Another angle to this is legacy SR-IOV NDOs.  User space picks a netdev
>>> from /sys/bus/pci/$VF_DBDF/physfn/net/ to run the NDOs on in somehow
>>> random manner, which means we have to provide those for all devices with
>>> link to the PF (all reprs).  And we have to link them (a) because it's
>>> right (tm) and (b) to get correct naming.
>> wait, as you commented in later, not only the mellanox vf reprs but rather also
>> the nfp vf reprs are not linked to the PF, because ip link output
>> grows quadratically.
> Right, correct.  If we set phys_port_id libvirt will reliably pick the
> correct netdev to run NDOs on (PF/PF repr) so we can remove them from
> the other netdevs and therefore limit the size of ip link show output.
>
>>> The only reliable way to make
>>> user space (libvirt) choose the repr it should run the NDOs on (which is
>>> IMHO the corresponding PF repr) is to set phys_port_id on actual VFs,
>>> VF reprs, PFs and PF reprs to a value corresponding to the *PCI PF*,
>>> not the external/Ethernet port when in switchdev mode.  User space
>>> should understand phys_port_id in this context, given it was originally
>>> introduced for matching VFs to ports.
>> Using phy_port_id to match/group VFs to PFs makes sense to me.
>>
>> So what would be the libvirt use case you envision that needs
>> the VF and PF reprs to support that as well? or maybe you were
>> not referring to libvirt but to some other provisioning element? I need
>> to refresh my memory on that area.
> Ugh, you're right!  Libvirt is our primary target here.  IIUC we need
> phys_port_id on the actual VF and then *a* netdev linked to physfn in
> sysfs which will have the legacy NDOs.
>
> We can't set the phys_port_id on the VF reprs because then we're back
> to the problem of ip link output growing.  Perhaps we shouldn't set it
> on PF repr either?
>
> Let's make a table (assuming bare metal cloud scenario where Host0 is
> controlling the network, while Host1 is the actual server):
>
> [act - actual; rpr - representor; SN -serial number]
>
> Today:
>
>    dev     | host | sysfs | phys_-  | switch- | phys_-    | NDOs
>            |      | link  | port_id | dev_id  | port_name |
> ---------------------------------------------------------------
> uplink    |   0  |   PF0 |   -     | ASIC SN | p0        | PF0
> act PF0   |   0  |   PF0 |   -     |   -     |  -        |  -
> act VF0/0 |   0  | VF0/0 |   -     |   -     |  -        |  -
> rpr PF0   |   0  |    -  |   -     | ASIC SN | pf0       |  -
> rpr VF0/0 |   0  |    -  |   -     | ASIC SN | pf0vf0    |  -
> act PF1   |   1  |   PF1 |   -     |   -     |  -        | PF1
> act VF1/0 |   1  | VF1/0 |   -     |   -     |  -        |  -
> rpr PF1   |   0  |    -  |   -     | ASIC SN | pf1       |  -
> rpr VF1/0 |   0  |    -  |   -     | ASIC SN | pf1vf0    |  -
>
> Proposed:
>
>    dev     | host | sysfs | phys_-  | switch- | phys_-    | NDOs
>            |      | link  | port_id | dev_id  | port_name |
> ---------------------------------------------------------------
> uplink    |   0  |   PF0 |   -     | ASIC SN | p0        |  -
> act PF0   |   0  |   PF0 | PF0 SN  |   -     |  -        | PF0
> act VF0/0 |   0  | VF0/0 | PF0 SN  |   -     |  -        |  -
> rpr PF0   |   0  |   PF0 |   -     | ASIC SN | pf0       |  -
> rpr VF0/0 |   0  |   PF0 |   -     | ASIC SN | pf0vf0    |  -
> act PF1   |   1  |   PF1 | PF1 SN  |   -     |  -        | PF1
> act VF1/0 |   1  | VF1/0 | PF1 SN  |   -     |  -        |  -
> rpr PF1   |   0  |   PF0 |   -     | ASIC SN | pf1       |  -
> rpr VF1/0 |   0  |   PF0 |   -     | ASIC SN | pf1vf0    |  -
>
> With this libvirt on Host0 should easily find the actual PF0 netdev to
> run the NDO on, if it wants to use VFs:
>   - libvrit finds act VF0/0 to plug into the VF;
>   - reads its phys_port_id -> "PF0 SN";
>   - finds netdev with "PF0 SN" linked to physfn -> "act PF0";
>   - runs NDOs on "act PF0" for PF0's VF correctly.

I think Host0 corresponds to embedded OS on the NIC. Is this correct?
I guess in this setup, only PF0's PCI interface on Host0 is in switchdev mode and
the representors for PF0 and its VFs are created on Host0 when they come up
on Host1. I would think PF0 on Host0 acts as a Control PF for PF1 on Host1.

Isn't hypervisor running only on Host1?


>
>
> The other problem remains unsolved - Host0 can't be sure without
> vendor-specific knowledge whether it's connected to PF0 or PF1.
> That's why I was thinking maybe we should provide phys_port_id
> on PF representors as well.  That means we'd have to provide the
> legacy NDOs on PF reprs too because libvirt may now find PF repr...
> Would it be cleaner to add a new attribute?
>
> Should Host0 in bare metal cloud have access to SR-IOV NDOs of Host1?

Do you mean the legacy VF ndo ops on the PF?  I think it is possible to configure
the VFs on Host1 via the port representors except for the MAC address.

^ permalink raw reply

* Re: [PATCH mlx5-next v1 05/15] net/mlx5: Break encap/decap into two separated flow table creation flags
From: Jason Gunthorpe @ 2018-09-05 16:38 UTC (permalink / raw)
  To: Leon Romanovsky
  Cc: Doug Ledford, RDMA mailing list, Ariel Levkovich, Mark Bloch,
	Or Gerlitz, Saeed Mahameed, linux-netdev
In-Reply-To: <20180905051025.GC2977@mtr-leonro.mtl.com>

On Wed, Sep 05, 2018 at 08:10:25AM +0300, Leon Romanovsky wrote:
> > > -	int en_encap_decap = !!(flags & MLX5_FLOW_TABLE_TUNNEL_EN);
> > > +	int en_encap = !!(flags & MLX5_FLOW_TABLE_TUNNEL_EN_ENCAP);
> > > +	int en_decap = !!(flags & MLX5_FLOW_TABLE_TUNNEL_EN_DECAP);
> >
> > Yuk, please don't use !!.
> >
> > 	bool en_decap = flags & MLX5_FLOW_TABLE_TUNNEL_EN_DECAP;
> 
> We need to provide en_encap and en_decap as an input to MLX5_SET(...)
> which is passed to FW as 0 or 1. 
>
> Boolean type is declared in C as int and treated as zero for false
> and any other value for true,

No, that isn't right, the kernel uses C99's _Bool intrinsic type, which
is guaranteed to only hold 0 or 1 by the compiler.

See types.h:

typedef _Bool                   bool;

Jason

^ permalink raw reply

* Re: [PATCH rdma-next v1 12/15] RDMA/mlx5: Add a new flow action verb - modify header
From: Jason Gunthorpe @ 2018-09-05 16:38 UTC (permalink / raw)
  To: Leon Romanovsky
  Cc: Doug Ledford, RDMA mailing list, Ariel Levkovich, Mark Bloch,
	Or Gerlitz, Saeed Mahameed, linux-netdev
In-Reply-To: <20180905052044.GE2977@mtr-leonro.mtl.com>

On Wed, Sep 05, 2018 at 08:20:44AM +0300, Leon Romanovsky wrote:
> On Tue, Sep 04, 2018 at 03:58:23PM -0600, Jason Gunthorpe wrote:
> > On Tue, Aug 28, 2018 at 02:18:51PM +0300, Leon Romanovsky wrote:
> >
> > > +static int UVERBS_HANDLER(MLX5_IB_METHOD_FLOW_ACTION_CREATE_MODIFY_HEADER)(
> > > +	struct ib_uverbs_file *file,
> > > +	struct uverbs_attr_bundle *attrs)
> > > +{
> > > +	struct ib_uobject *uobj = uverbs_attr_get_uobject(
> > > +		attrs, MLX5_IB_ATTR_CREATE_MODIFY_HEADER_HANDLE);
> > > +	struct mlx5_ib_dev *mdev = to_mdev(uobj->context->device);
> > > +	enum mlx5_ib_uapi_flow_table_type ft_type;
> > > +	struct ib_flow_action *action;
> > > +	size_t num_actions;
> > > +	void *in;
> > > +	int len;
> > > +	int ret;
> > > +
> > > +	if (!mlx5_ib_modify_header_supported(mdev))
> > > +		return -EOPNOTSUPP;
> > > +
> > > +	in = uverbs_attr_get_alloced_ptr(attrs,
> > > +		MLX5_IB_ATTR_CREATE_MODIFY_HEADER_ACTIONS_PRM);
> > > +	len = uverbs_attr_get_len(attrs,
> > > +		MLX5_IB_ATTR_CREATE_MODIFY_HEADER_ACTIONS_PRM);
> > > +
> > > +	if (len % MLX5_UN_SZ_BYTES(set_action_in_add_action_in_auto))
> > > +		return -EINVAL;
> > > +
> > > +	ret = uverbs_get_const(&ft_type, attrs,
> > > +			       MLX5_IB_ATTR_CREATE_MODIFY_HEADER_FT_TYPE);
> > > +	if (ret)
> > > +		return -EINVAL;
> >
> > This should be
> >
> > 	if (ret)
> > 		return ret;
> >
> > Every call to uverbs_get_const is wrong in this same way..
> 
> Right, from technical point of view uverbs_get_const can return EINVAL
> only, and it is correct for now, but need to be changed to proper
> "return ret".

No, it can return ENOENT as well.

Jason

^ permalink raw reply

* Re: phys_port_id in switchdev mode?
From: Jakub Kicinski @ 2018-09-05 16:43 UTC (permalink / raw)
  To: Or Gerlitz
  Cc: Florian Fainelli, Simon Horman, Andy Gospodarek,
	mchan@broadcom.com, Jiri Pirko, Alexander Duyck, Frederick Botha,
	nick viljoen, Linux Netdev List
In-Reply-To: <CAJ3xEMj79J0F9dpBNNLGzq1Mub7ig1itv7_Wgpsi6PxVgmRqsg@mail.gmail.com>

On Tue, 4 Sep 2018 23:37:29 +0300, Or Gerlitz wrote:
> On Tue, Sep 4, 2018 at 1:20 PM, Jakub Kicinski wrote:
> > On Mon, 3 Sep 2018 12:40:22 +0300, Or Gerlitz wrote:  
> >> On Tue, Aug 28, 2018 at 9:05 PM, Jakub Kicinski wrote:  
> >> > Hi!  
> >>
> >> Hi Jakub and sorry for the late reply, this crazigly hot summer refuses to die,
> >>
> >> Note I replied couple of minutes ago but it didn't get to the list, so
> >> lets take it from this one:
> >>  
> >> > I wonder if we can use phys_port_id in switchdev to group together
> >> > interfaces of a single PCI PF?  Here is the problem:
> >> >
> >> > With a mix of PF and VF interfaces it gets increasingly difficult to
> >> > figure out which one corresponds to which PF.  We can identify which
> >> > *representor* is which, by means of phys_port_name and devlink
> >> > flavours.  But if the actual VF/PF interfaces are also present on the
> >> > same host, it gets confusing when one tries to identify the PF they
> >> > came from.  Generally one has to resort of matching between PCI DBDF of
> >> > the PF and VFs or read relevant info out of ethtool -i.
> >> >
> >> > In multi host scenario this is particularly painful, as there seems to
> >> > be no immediately obvious way to match PCI interface ID of a card (0,
> >> > 1, 2, 3, 4...) to the DBDF we have connected.
> >> >
> >> > Another angle to this is legacy SR-IOV NDOs.  User space picks a netdev
> >> > from /sys/bus/pci/$VF_DBDF/physfn/net/ to run the NDOs on in somehow
> >> > random manner, which means we have to provide those for all devices with
> >> > link to the PF (all reprs).  And we have to link them (a) because it's
> >> > right (tm) and (b) to get correct naming.  
> >>
> >> wait, as you commented in later, not only the mellanox vf reprs but rather also
> >> the nfp vf reprs are not linked to the PF, because ip link output
> >> grows quadratically.  
> >
> > Right, correct.  If we set phys_port_id libvirt will reliably pick the
> > correct netdev to run NDOs on (PF/PF repr) so we can remove them from
> > the other netdevs and therefore limit the size of ip link show output.  
> 
> just to make sure, this is suggested/future not existing flow of libvirt?

Mm..  admittedly I haven't investigated in depth, but my colleague did
and indicated this is the current flow.  It matches phys_port_id right
here:

https://github.com/libvirt/libvirt/blob/master/src/util/virpci.c#L2793

Are we wrong?

> > Ugh, you're right!  Libvirt is our primary target here.  IIUC we need
> > phys_port_id on the actual VF and then *a* netdev linked to physfn in
> > sysfs which will have the legacy NDOs.
> >
> > We can't set the phys_port_id on the VF reprs because then we're back
> > to the problem of ip link output growing.  Perhaps we shouldn't set it
> > on PF repr either?
> >
> > Let's make a table (assuming bare metal cloud scenario where Host0 is
> > controlling the network, while Host1 is the actual server):  
> 
> yeah, this would be a super-set the non-smartnic case where
> we have only one host.
> 
> [...]
> 
> > With this libvirt on Host0 should easily find the actual PF0 netdev to
> > run the NDO on, if it wants to use VFs:
> >  - libvrit finds act VF0/0 to plug into the VM;
> >  - reads its phys_port_id -> "PF0 SN";
> >  - finds netdev with "PF0 SN" linked to physfn -> "act PF0";
> >  - runs NDOs on "act PF0" for PF0's VF correctly.  
> 
> What you describe here doesn't seem to be networking
> configuration, as it deals only with VFs and PF but not with reprs,
> and hence AFAIK runs on host host1

No, hm, depends on your definition of SmartNIC.  ARM64 control CPU is
capable of running VMs.  Why would you not run VMs on your controller?
Or one day we will need reprs for containers, people are definitely
going to run containers on the controller...  I wouldn't design this
assuming there is no advanced switching a'la service chains on the
control CPU...

> > Should Host0 in bare metal cloud have access to SR-IOV NDOs of Host1?  
> 
> I need to think on that

Okay :)

^ permalink raw reply

* Re: phys_port_id in switchdev mode?
From: Jakub Kicinski @ 2018-09-05 16:47 UTC (permalink / raw)
  To: Samudrala, Sridhar
  Cc: Or Gerlitz, Florian Fainelli, Simon Horman, Andy Gospodarek,
	mchan@broadcom.com, Jiri Pirko, Alexander Duyck, Frederick Botha,
	nick viljoen, Linux Netdev List
In-Reply-To: <ee10a22c-3605-e60e-391e-4eee322021dc@intel.com>

On Wed, 5 Sep 2018 09:20:43 -0700, Samudrala, Sridhar wrote:
> > With this libvirt on Host0 should easily find the actual PF0 netdev to
> > run the NDO on, if it wants to use VFs:
> >   - libvrit finds act VF0/0 to plug into the VF;
> >   - reads its phys_port_id -> "PF0 SN";
> >   - finds netdev with "PF0 SN" linked to physfn -> "act PF0";
> >   - runs NDOs on "act PF0" for PF0's VF correctly.  
> 
> I think Host0 corresponds to embedded OS on the NIC. Is this correct?
> I guess in this setup, only PF0's PCI interface on Host0 is in switchdev mode and
> the representors for PF0 and its VFs are created on Host0 when they come up
> on Host1. I would think PF0 on Host0 acts as a Control PF for PF1 on Host1.
> 
> Isn't hypervisor running only on Host1?

The main hypervisor is, but Host0 can very easily want to run some DPI
or some such on flows before it allows them though, and people like
running DPI-like apps in VMs/containers..

> > The other problem remains unsolved - Host0 can't be sure without
> > vendor-specific knowledge whether it's connected to PF0 or PF1.
> > That's why I was thinking maybe we should provide phys_port_id
> > on PF representors as well.  That means we'd have to provide the
> > legacy NDOs on PF reprs too because libvirt may now find PF repr...
> > Would it be cleaner to add a new attribute?
> >
> > Should Host0 in bare metal cloud have access to SR-IOV NDOs of Host1?  
> 
> Do you mean the legacy VF ndo ops on the PF?  I think it is possible to configure
> the VFs on Host1 via the port representors except for the MAC address.

Yes, the MAC address would be the only one.  Could Host0 care about
which MACs Host1 assigned to its VFs?  I'm not sure..

^ permalink raw reply

* Re: bpfilter causes a leftover kernel process
From: Olivier Brunel @ 2018-09-05 15:52 UTC (permalink / raw)
  To: Alexei Starovoitov; +Cc: netdev, daniel
In-Reply-To: <20180828033500.g3siwst5h2ckewwb@ast-mbp>

Hi,

Quick follow-up on this:

- first off, Arch devs have updated their kernel config so the next
  kernel will not have bpfilter enabled anymore, thus avoiding any
  issue.

- having said that, I've found a neasy way to reproduce it in an Arch
  VM, in case you're interested :

Boot the latest Arch ISO[1] which now contains a kernel 4.18.5 and do a
very basic installation, pretty much just:

# pacstrap /mnt base
# genfstab -U /mnt >> /mnt/etc/fstab

And of course install your boot loader of choice.

Then boot the brand new system, log in and make sure the helper is
actually started, i.e. `modprobe bpfilter` -- Now halt.

You'll see in the end that systemd complains that it can't
unmount /oldroot (EBUSY), aka the root fs; and that's because of the
bpfilter helper, which wasn't killed because it's seen as a kernel
thread due to its empty command line and therefore not signaled.


Cheers,



[1] https://www.archlinux.org/download/

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox