Netdev List
 help / color / mirror / Atom feed
* Re: [PATCH 3/4] net: lantiq: Add Lantiq / Intel vrx200 Ethernet driver
From: Andrew Lunn @ 2018-07-25 15:28 UTC (permalink / raw)
  To: Hauke Mehrtens
  Cc: davem, netdev, vivien.didelot, f.fainelli, john, linux-mips, dev,
	hauke.mehrtens
In-Reply-To: <20180721191358.13952-4-hauke@hauke-m.de>

> +	/* Make sure the firmware of the embedded GPHY is loaded before,
> +	 * otherwise they will not be detectable on the MDIO bus.
> +	 */
> +	of_for_each_phandle(&it, err, np, "lantiq,phys", NULL, 0) {
> +		phy_np = it.node;
> +		if (phy_np) {
> +			struct platform_device *phy = of_find_device_by_node(phy_np);
> +
> +			of_node_put(phy_np);
> +			if (!platform_get_drvdata(phy))
> +				return -EPROBE_DEFER;
> +		}
> +	}

Is there a device tree binding document for this somewhere?

   Andrew

^ permalink raw reply

* Re: [PATCH net-next v3 4/5] net/tc: introduce TC_ACT_REINJECT.
From: Jamal Hadi Salim @ 2018-07-25 15:26 UTC (permalink / raw)
  To: Paolo Abeni, Cong Wang
  Cc: Linux Kernel Network Developers, Jiri Pirko, Daniel Borkmann,
	Marcelo Ricardo Leitner, Eyal Birger, David Miller
In-Reply-To: <1db747cb97bc11dae4a946c489141a970ad053c5.camel@redhat.com>

On 25/07/18 10:24 AM, Paolo Abeni wrote:
> On Wed, 2018-07-25 at 08:27 -0400, Jamal Hadi Salim wrote:

>> Those changes were there from the beginning (above patch did
>> not introduce them).
>> IIRC, the reason was to distinguish between policy intended
>> drops and drops because of errors.
> 
> Double-checking to avoid misinterepration on my side: you are ok with
> keeping the 'overlimits' increment, right?

Yes.

cheers,
jamal

^ permalink raw reply

* Re: [net-next 10/16] net/mlx5: Support PCIe buffer congestion handling via Devlink
From: Alexander Duyck @ 2018-07-25 15:23 UTC (permalink / raw)
  To: Eran Ben Elisha
  Cc: Jakub Kicinski, Saeed Mahameed, Jiri Pirko, David S. Miller,
	netdev@vger.kernel.org
In-Reply-To: <98071d31-bf99-d713-7fd1-4b1cc156377e@mellanox.com>

On Wed, Jul 25, 2018 at 5:31 AM, Eran Ben Elisha <eranbe@mellanox.com> wrote:
>
>
> On 7/24/2018 10:51 PM, Jakub Kicinski wrote:
>>>>
>>>>
>>>> The devlink params haven't been upstream even for a full cycle and
>>>> already you guys are starting to use them to configure standard
>>>> features like queuing.
>>>
>>>
>>> We developed the devlink params in order to support non-standard
>>> configuration only. And for non-standard, there are generic and vendor
>>> specific options.
>>
>>
>> I thought it was developed for performing non-standard and possibly
>> vendor specific configuration.  Look at DEVLINK_PARAM_GENERIC_* for
>> examples of well justified generic options for which we have no
>> other API.  The vendor mlx4 options look fairly vendor specific if you
>> ask me, too.
>>
>> Configuring queuing has an API.  The question is it acceptable to enter
>> into the risky territory of controlling offloads via devlink parameters
>> or would we rather make vendors take the time and effort to model
>> things to (a subset) of existing APIs.  The HW never fits the APIs
>> perfectly.
>
>
> I understand what you meant here, I would like to highlight that this
> mechanism was not meant to handle SRIOV, Representors, etc.
> The vendor specific configuration suggested here is to handle a congestion
> state in Multi Host environment (which includes PF and multiple VFs per
> host), where one host is not aware to the other hosts, and each is running
> on its own pci/driver. It is a device working mode configuration.
>
> This  couldn't fit into any existing API, thus creating this vendor specific
> unique API is needed.

If we are just going to start creating devlink interfaces in for every
one-off option a device wants to add why did we even bother with
trying to prevent drivers from using sysfs? This just feels like we
are back to the same arguments we had back in the day with it.

I feel like the bigger question here is if devlink is how we are going
to deal with all PCIe related features going forward, or should we
start looking at creating a new interface/tool for PCI/PCIe related
features? My concern is that we have already had features such as DMA
Coalescing that didn't really fit into anything and now we are
starting to see other things related to DMA and PCIe bus credits. I'm
wondering if we shouldn't start looking at a tool/interface to
configure all the PCIe related features such as interrupts, error
reporting, DMA configuration, power management, etc. Maybe we could
even look at sharing it across subsystems and include things like
storage, graphics, and other subsystems in the conversation.

- Alex

^ permalink raw reply

* Re: [PATCH 2/5] rhashtable: don't hold lock on first table throughout insertion.
From: Paul E. McKenney @ 2018-07-25 15:22 UTC (permalink / raw)
  To: NeilBrown; +Cc: Herbert Xu, Thomas Graf, netdev, linux-kernel
In-Reply-To: <87in53oqzz.fsf@notabene.neil.brown.name>

On Wed, Jul 25, 2018 at 02:53:36PM +1000, NeilBrown wrote:
> On Tue, Jul 24 2018, Paul E. McKenney wrote:
> 
> > On Tue, Jul 24, 2018 at 07:52:03AM +1000, NeilBrown wrote:
> >> On Mon, Jul 23 2018, Paul E. McKenney wrote:
> >> 
> >> > On Mon, Jul 23, 2018 at 09:13:43AM +1000, NeilBrown wrote:
> >> >> On Sun, Jul 22 2018, Paul E. McKenney wrote:
> >> >> >
> >> >> > One issue is that the ->func pointer can legitimately be NULL while on
> >> >> > RCU's callback lists.  This happens when someone invokes kfree_rcu()
> >> >> > with the rcu_head structure at the beginning of the enclosing structure.
> >> >> > I could add an offset to avoid this, or perhaps the kmalloc() folks
> >> >> > could be persuaded Rao Shoaib's patch moving kfree_rcu() handling to
> >> >> > the slab allocators, so that RCU only ever sees function pointers in
> >> >> > the ->func field.
> >> >> >
> >> >> > Either way, this should be hidden behind an API to allow adjustments
> >> >> > to be made if needed.  Maybe something like is_after_call_rcu()?
> >> >> > This would (for example) allow debug-object checks to be used to catch
> >> >> > check-after-free bugs.
> >> >> >
> >> >> > Would something of that sort work for you?
> >> >> 
> >> >> Yes, if you could provide an is_after_call_rcu() API, that would
> >> >> perfectly suit my use-case.
> >> >
> >> > After beating my head against the object-debug code a bit, I have to ask
> >> > if it would be OK for you if the is_after_call_rcu() API also takes the
> >> > function that was passed to RCU.
> >> 
> >> Sure.  It feels a bit clumsy, but I can see it could be easier to make
> >> robust.
> >> So yes: I'm fine with pass the same function and rcu_head to both
> >> call_rcu() and is_after_call_rcu().  Actually, when I say it like that,
> >> it seems less clumsy :-)
> >
> > How about like this?  (It needs refinements, like lockdep, but should
> > get the gist.)
> >
> 
> Looks good ... except ... naming is hard.
> 
>  is_after_call_rcu_init()  asserts where in the lifecycle we are,
>  is_after_call_rcu() tests where in the lifecycle we are.
> 
>  The names are similar but the purpose is quite different.
>  Maybe s/is_after_call_rcu_init/call_rcu_init/ ??

How about rcu_head_init() and rcu_head_after_call_rcu()?

							Thanx, Paul

> Thanks,
> NeilBrown
> 
> 
> > 							Thanx, Paul
> >
> > ------------------------------------------------------------------------
> >
> > commit 5aa0ebf4799b8bddbbd0124db1c008526e99fc7c
> > Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> > Date:   Tue Jul 24 15:28:09 2018 -0700
> >
> >     rcu: Provide functions for determining if call_rcu() has been invoked
> >     
> >     This commit adds is_after_call_rcu() and is_after_call_rcu_init()
> >     functions to help RCU users detect when another CPU has passed
> >     the specified rcu_head structure and function to call_rcu().
> >     The is_after_call_rcu_init() should be invoked before making the
> >     structure visible to RCU readers, and then the is_after_call_rcu() may
> >     be invoked from within an RCU read-side critical section on an rcu_head
> >     structure that was obtained during a traversal of the data structure
> >     in question.  The is_after_call_rcu() function will return true if the
> >     rcu_head structure has already been passed (with the specified function)
> >     to call_rcu(), otherwise it will return false.
> >     
> >     If is_after_call_rcu_init() has not been invoked on the rcu_head
> >     structure or if the rcu_head (AKA callback) has already been invoked,
> >     then is_after_call_rcu() will do WARN_ON_ONCE().
> >     
> >     Reported-by: NeilBrown <neilb@suse.com>
> >     Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> >
> > diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
> > index e4f821165d0b..82e5a91539b5 100644
> > --- a/include/linux/rcupdate.h
> > +++ b/include/linux/rcupdate.h
> > @@ -857,6 +857,45 @@ static inline notrace void rcu_read_unlock_sched_notrace(void)
> >  #endif /* #else #ifdef CONFIG_ARCH_WEAK_RELEASE_ACQUIRE */
> >  
> >  
> > +/* Has the specified rcu_head structure been handed to call_rcu()? */
> > +
> > +/*
> > + * is_after_call_rcu_init - Initialize rcu_head for is_after_call_rcu()
> > + * @rhp: The rcu_head structure to initialize.
> > + *
> > + * If you intend to invoke is_after_call_rcu() to test whether a given
> > + * rcu_head structure has already been passed to call_rcu(), then you must
> > + * also invoke this is_after_call_rcu_init() function on it just after
> > + * allocating that structure.  Calls to this function must not race with
> > + * calls to call_rcu(), is_after_call_rcu(), or callback invocation.
> > + */
> > +static inline void is_after_call_rcu_init(struct rcu_head *rhp)
> > +{
> > +	rhp->func = (rcu_callback_t)~0L;
> > +}
> > +
> > +/*
> > + * is_after_call_rcu - Has this rcu_head been passed to call_rcu()?
> > + * @rhp: The rcu_head structure to test.
> > + * @func: The function passed to call_rcu() along with @rhp.
> > + *
> > + * Returns @true if the @rhp has been passed to call_rcu() with @func, and
> > + * @false otherwise.  Emits a warning in any other case, including the
> > + * case where @rhp has already been invoked after a grace period.
> > + * Calls to this function must not race with callback invocation.  One
> > + * way to avoid such races is to enclose the call to is_after_call_rcu()
> > + * in an RCU read-side critical section that includes a read-side fetch
> > + * of the pointer to the structure containing @rhp.
> > + */
> > +static inline bool is_after_call_rcu(struct rcu_head *rhp, rcu_callback_t f)
> > +{
> > +	if (READ_ONCE(rhp->func) == f)
> > +		return true;
> > +	WARN_ON_ONCE(READ_ONCE(rhp->func) != (rcu_callback_t)~0L);
> > +	return false;
> > +}
> > +
> > +
> >  /* Transitional pre-consolidation compatibility definitions. */
> >  
> >  static inline void synchronize_rcu_bh(void)
> > diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
> > index 5dec94509a7e..4c56c1d98fb3 100644
> > --- a/kernel/rcu/rcu.h
> > +++ b/kernel/rcu/rcu.h
> > @@ -224,6 +224,7 @@ void kfree(const void *);
> >   */
> >  static inline bool __rcu_reclaim(const char *rn, struct rcu_head *head)
> >  {
> > +	rcu_callback_t f;
> >  	unsigned long offset = (unsigned long)head->func;
> >  
> >  	rcu_lock_acquire(&rcu_callback_map);
> > @@ -234,7 +235,9 @@ static inline bool __rcu_reclaim(const char *rn, struct rcu_head *head)
> >  		return true;
> >  	} else {
> >  		RCU_TRACE(trace_rcu_invoke_callback(rn, head);)
> > -		head->func(head);
> > +		f = head->func;
> > +		WRITE_ONCE(head->func, (rcu_callback_t)0L);
> > +		f(head);
> >  		rcu_lock_release(&rcu_callback_map);
> >  		return false;
> >  	}

^ permalink raw reply

* [PATCH] samples/bpf: Add BTF build flags to Makefile
From: Taeung Song @ 2018-07-25 16:30 UTC (permalink / raw)
  To: Daniel Borkmann, Alexei Starovoitov
  Cc: netdev, linux-kernel, Martin KaFai Lau

To smoothly test BTF supported binary on samples/bpf,
let samples/bpf/Makefile probe llc, pahole and
llvm-objcopy for BPF support and use them
like tools/testing/selftests/bpf/Makefile
changed from the commit c0fa1b6c3efc ("bpf: btf:
 Add BTF tests")

Cc: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Taeung Song <treeze.taeung@gmail.com>
---
 samples/bpf/Makefile | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 1303af10e54d..e079266360a3 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -191,6 +191,8 @@ HOSTLOADLIBES_xdpsock		+= -pthread
 #  make samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
 LLC ?= llc
 CLANG ?= clang
+LLVM_OBJCOPY ?= llvm-objcopy
+BTF_PAHOLE ?= pahole
 
 # Detect that we're cross compiling and use the cross compiler
 ifdef CROSS_COMPILE
@@ -198,6 +200,20 @@ HOSTCC = $(CROSS_COMPILE)gcc
 CLANG_ARCH_ARGS = -target $(ARCH)
 endif
 
+BTF_LLC_PROBE := $(shell $(LLC) -march=bpf -mattr=help 2>&1 | grep dwarfris)
+BTF_PAHOLE_PROBE := $(shell $(BTF_PAHOLE) --help 2>&1 | grep BTF)
+BTF_OBJCOPY_PROBE := $(shell $(LLVM_OBJCOPY) --help 2>&1 | grep -i 'usage.*llvm')
+
+ifneq ($(BTF_LLC_PROBE),)
+ifneq ($(BTF_PAHOLE_PROBE),)
+ifneq ($(BTF_OBJCOPY_PROBE),)
+	EXTRA_CFLAGS += -g
+	LLC_FLAGS += -mattr=dwarfris
+	DWARF2BTF = y
+endif
+endif
+endif
+
 # Trick to allow make to be run from this directory
 all:
 	$(MAKE) -C ../../ $(CURDIR)/ BPF_SAMPLES_PATH=$(CURDIR)
@@ -256,4 +272,7 @@ $(obj)/%.o: $(src)/%.c
 		-Wno-gnu-variable-sized-type-not-at-end \
 		-Wno-address-of-packed-member -Wno-tautological-compare \
 		-Wno-unknown-warning-option $(CLANG_ARCH_ARGS) \
-		-O2 -emit-llvm -c $< -o -| $(LLC) -march=bpf -filetype=obj -o $@
+		-O2 -emit-llvm -c $< -o -| $(LLC) -march=bpf $(LLC_FLAGS) -filetype=obj -o $@
+ifeq ($(DWARF2BTF),y)
+	$(BTF_PAHOLE) -J $@
+endif
-- 
2.17.1

^ permalink raw reply related

* [PATCH V2 bpf] xdp: add NULL pointer check in __xdp_return()
From: Taehee Yoo @ 2018-07-25 15:09 UTC (permalink / raw)
  To: daniel, ast, bjorn.topel; +Cc: brouer, netdev, Taehee Yoo

rhashtable_lookup() can return NULL. so that NULL pointer
check routine should be added.

Fixes: 02b55e5657c3 ("xdp: add MEM_TYPE_ZERO_COPY")
Signed-off-by: Taehee Yoo <ap420073@gmail.com>
---
V2 : add WARN_ON_ONCE when xa is NULL.

 net/core/xdp.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/net/core/xdp.c b/net/core/xdp.c
index 9d1f220..786fdbe 100644
--- a/net/core/xdp.c
+++ b/net/core/xdp.c
@@ -345,7 +345,10 @@ static void __xdp_return(void *data, struct xdp_mem_info *mem, bool napi_direct,
 		rcu_read_lock();
 		/* mem->id is valid, checked in xdp_rxq_info_reg_mem_model() */
 		xa = rhashtable_lookup(mem_id_ht, &mem->id, mem_id_rht_params);
-		xa->zc_alloc->free(xa->zc_alloc, handle);
+		if (!xa)
+			WARN_ON_ONCE(1);
+		else
+			xa->zc_alloc->free(xa->zc_alloc, handle);
 		rcu_read_unlock();
 	default:
 		/* Not possible, checked in xdp_rxq_info_reg_mem_model() */
-- 
2.9.3

^ permalink raw reply related

* [PATCH net-next 3/4] net/smc: provide fallback reason code
From: Ursula Braun @ 2018-07-25 14:35 UTC (permalink / raw)
  To: davem; +Cc: netdev, linux-s390, schwidefsky, heiko.carstens, raspl,
	linux-kernel
In-Reply-To: <20180725143533.7082-1-ubraun@linux.ibm.com>

From: Karsten Graul <kgraul@linux.ibm.com>

Remember the fallback reason code and the peer diagnosis code for
smc sockets, and provide them in smc_diag.c to the netlink interface.
And add more detailed reason codes.

Signed-off-by: Karsten Graul <kgraul@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
---
 include/uapi/linux/smc_diag.h |  6 +++++
 net/smc/af_smc.c              | 52 +++++++++++++++++++++++++------------------
 net/smc/smc.h                 |  2 ++
 net/smc/smc_clc.c             |  6 ++++-
 net/smc/smc_clc.h             | 18 ++++++++++-----
 net/smc/smc_diag.c            |  6 +++++
 6 files changed, 61 insertions(+), 29 deletions(-)

diff --git a/include/uapi/linux/smc_diag.h b/include/uapi/linux/smc_diag.h
index 48ae3ee22b2d..ac9e8c96d9bd 100644
--- a/include/uapi/linux/smc_diag.h
+++ b/include/uapi/linux/smc_diag.h
@@ -43,6 +43,7 @@ enum {
 	SMC_DIAG_LGRINFO,
 	SMC_DIAG_SHUTDOWN,
 	SMC_DIAG_DMBINFO,
+	SMC_DIAG_FALLBACK,
 	__SMC_DIAG_MAX,
 };
 
@@ -92,6 +93,11 @@ struct smc_diag_lgrinfo {
 	__u8				role;
 };
 
+struct smc_diag_fallback {
+	__u32 reason;
+	__u32 peer_diagnosis;
+};
+
 struct smcd_diag_dmbinfo {		/* SMC-D Socket internals */
 	__u32 linkid;			/* Link identifier */
 	__u64 peer_gid;			/* Peer GID */
diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c
index b81797103260..fce7e4751151 100644
--- a/net/smc/af_smc.c
+++ b/net/smc/af_smc.c
@@ -344,17 +344,17 @@ static int smc_clnt_conf_first_link(struct smc_sock *smc)
 
 	rc = smc_ib_modify_qp_rts(link);
 	if (rc)
-		return SMC_CLC_DECL_INTERR;
+		return SMC_CLC_DECL_ERR_RDYLNK;
 
 	smc_wr_remember_qp_attr(link);
 
 	if (smc_reg_rmb(link, smc->conn.rmb_desc, false))
-		return SMC_CLC_DECL_INTERR;
+		return SMC_CLC_DECL_ERR_REGRMB;
 
 	/* send CONFIRM LINK response over RoCE fabric */
 	rc = smc_llc_send_confirm_link(link, SMC_LLC_RESP);
 	if (rc < 0)
-		return SMC_CLC_DECL_TCL;
+		return SMC_CLC_DECL_TIMEOUT_CL;
 
 	/* receive ADD LINK request from server over RoCE fabric */
 	rest = wait_for_completion_interruptible_timeout(&link->llc_add,
@@ -372,7 +372,7 @@ static int smc_clnt_conf_first_link(struct smc_sock *smc)
 				   link->smcibdev->mac[link->ibport - 1],
 				   link->gid, SMC_LLC_RESP);
 	if (rc < 0)
-		return SMC_CLC_DECL_TCL;
+		return SMC_CLC_DECL_TIMEOUT_AL;
 
 	smc_llc_link_active(link, net->ipv4.sysctl_tcp_keepalive_time);
 
@@ -424,9 +424,10 @@ static void smc_link_save_peer_info(struct smc_link *link,
 }
 
 /* fall back during connect */
-static int smc_connect_fallback(struct smc_sock *smc)
+static int smc_connect_fallback(struct smc_sock *smc, int reason_code)
 {
 	smc->use_fallback = true;
+	smc->fallback_rsn = reason_code;
 	smc_copy_sock_settings_to_clc(smc);
 	if (smc->sk.sk_state == SMC_INIT)
 		smc->sk.sk_state = SMC_ACTIVE;
@@ -443,7 +444,7 @@ static int smc_connect_decline_fallback(struct smc_sock *smc, int reason_code)
 			sock_put(&smc->sk); /* passive closing */
 		return reason_code;
 	}
-	if (reason_code != SMC_CLC_DECL_REPLY) {
+	if (reason_code != SMC_CLC_DECL_PEERDECL) {
 		rc = smc_clc_send_decline(smc, reason_code);
 		if (rc < 0) {
 			if (smc->sk.sk_state == SMC_INIT)
@@ -451,7 +452,7 @@ static int smc_connect_decline_fallback(struct smc_sock *smc, int reason_code)
 			return rc;
 		}
 	}
-	return smc_connect_fallback(smc);
+	return smc_connect_fallback(smc, reason_code);
 }
 
 /* abort connecting */
@@ -568,7 +569,7 @@ static int smc_connect_rdma(struct smc_sock *smc,
 		smc_link_save_peer_info(link, aclc);
 
 	if (smc_rmb_rtoken_handling(&smc->conn, aclc))
-		return smc_connect_abort(smc, SMC_CLC_DECL_INTERR,
+		return smc_connect_abort(smc, SMC_CLC_DECL_ERR_RTOK,
 					 local_contact);
 
 	smc_close_init(smc);
@@ -576,12 +577,12 @@ static int smc_connect_rdma(struct smc_sock *smc,
 
 	if (local_contact == SMC_FIRST_CONTACT) {
 		if (smc_ib_ready_link(link))
-			return smc_connect_abort(smc, SMC_CLC_DECL_INTERR,
+			return smc_connect_abort(smc, SMC_CLC_DECL_ERR_RDYLNK,
 						 local_contact);
 	} else {
 		if (!smc->conn.rmb_desc->reused &&
 		    smc_reg_rmb(link, smc->conn.rmb_desc, true))
-			return smc_connect_abort(smc, SMC_CLC_DECL_INTERR,
+			return smc_connect_abort(smc, SMC_CLC_DECL_ERR_REGRMB,
 						 local_contact);
 	}
 	smc_rmb_sync_sg_for_device(&smc->conn);
@@ -659,11 +660,11 @@ static int __smc_connect(struct smc_sock *smc)
 	sock_hold(&smc->sk); /* sock put in passive closing */
 
 	if (smc->use_fallback)
-		return smc_connect_fallback(smc);
+		return smc_connect_fallback(smc, smc->fallback_rsn);
 
 	/* if peer has not signalled SMC-capability, fall back */
 	if (!tcp_sk(smc->clcsock->sk)->syn_smc)
-		return smc_connect_fallback(smc);
+		return smc_connect_fallback(smc, SMC_CLC_DECL_PEERNOSMC);
 
 	/* IPSec connections opt out of SMC-R optimizations */
 	if (using_ipsec(smc))
@@ -693,7 +694,7 @@ static int __smc_connect(struct smc_sock *smc)
 
 	/* if neither ISM nor RDMA are supported, fallback */
 	if (!rdma_supported && !ism_supported)
-		return smc_connect_decline_fallback(smc, SMC_CLC_DECL_CNFERR);
+		return smc_connect_decline_fallback(smc, SMC_CLC_DECL_NOSMCDEV);
 
 	/* perform CLC handshake */
 	rc = smc_connect_clc(smc, smc_type, &aclc, ibdev, ibport, gid, ismdev);
@@ -708,7 +709,7 @@ static int __smc_connect(struct smc_sock *smc)
 	else if (ism_supported && aclc.hdr.path == SMC_TYPE_D)
 		rc = smc_connect_ism(smc, &aclc, ismdev);
 	else
-		rc = SMC_CLC_DECL_CNFERR;
+		rc = SMC_CLC_DECL_MODEUNSUPP;
 	if (rc) {
 		smc_connect_ism_vlan_cleanup(smc, ism_supported, ismdev, vlan);
 		return smc_connect_decline_fallback(smc, rc);
@@ -946,12 +947,12 @@ static int smc_serv_conf_first_link(struct smc_sock *smc)
 	link = &lgr->lnk[SMC_SINGLE_LINK];
 
 	if (smc_reg_rmb(link, smc->conn.rmb_desc, false))
-		return SMC_CLC_DECL_INTERR;
+		return SMC_CLC_DECL_ERR_REGRMB;
 
 	/* send CONFIRM LINK request to client over the RoCE fabric */
 	rc = smc_llc_send_confirm_link(link, SMC_LLC_REQ);
 	if (rc < 0)
-		return SMC_CLC_DECL_TCL;
+		return SMC_CLC_DECL_TIMEOUT_CL;
 
 	/* receive CONFIRM LINK response from client over the RoCE fabric */
 	rest = wait_for_completion_interruptible_timeout(
@@ -973,7 +974,7 @@ static int smc_serv_conf_first_link(struct smc_sock *smc)
 				   link->smcibdev->mac[link->ibport - 1],
 				   link->gid, SMC_LLC_REQ);
 	if (rc < 0)
-		return SMC_CLC_DECL_TCL;
+		return SMC_CLC_DECL_TIMEOUT_AL;
 
 	/* receive ADD LINK response from client over the RoCE fabric */
 	rest = wait_for_completion_interruptible_timeout(&link->llc_add_resp,
@@ -1048,7 +1049,8 @@ static void smc_listen_decline(struct smc_sock *new_smc, int reason_code,
 	}
 	smc_conn_free(&new_smc->conn);
 	new_smc->use_fallback = true;
-	if (reason_code && reason_code != SMC_CLC_DECL_REPLY) {
+	new_smc->fallback_rsn = reason_code;
+	if (reason_code && reason_code != SMC_CLC_DECL_PEERDECL) {
 		if (smc_clc_send_decline(new_smc, reason_code) < 0) {
 			smc_listen_out_err(new_smc);
 			return;
@@ -1139,7 +1141,7 @@ static int smc_listen_rdma_reg(struct smc_sock *new_smc, int local_contact)
 	if (local_contact != SMC_FIRST_CONTACT) {
 		if (!new_smc->conn.rmb_desc->reused) {
 			if (smc_reg_rmb(link, new_smc->conn.rmb_desc, true))
-				return SMC_CLC_DECL_INTERR;
+				return SMC_CLC_DECL_ERR_REGRMB;
 		}
 	}
 	smc_rmb_sync_sg_for_device(&new_smc->conn);
@@ -1159,13 +1161,13 @@ static void smc_listen_rdma_finish(struct smc_sock *new_smc,
 		smc_link_save_peer_info(link, cclc);
 
 	if (smc_rmb_rtoken_handling(&new_smc->conn, cclc)) {
-		reason_code = SMC_CLC_DECL_INTERR;
+		reason_code = SMC_CLC_DECL_ERR_RTOK;
 		goto decline;
 	}
 
 	if (local_contact == SMC_FIRST_CONTACT) {
 		if (smc_ib_ready_link(link)) {
-			reason_code = SMC_CLC_DECL_INTERR;
+			reason_code = SMC_CLC_DECL_ERR_RDYLNK;
 			goto decline;
 		}
 		/* QP confirmation over RoCE fabric */
@@ -1206,6 +1208,7 @@ static void smc_listen_work(struct work_struct *work)
 	/* check if peer is smc capable */
 	if (!tcp_sk(newclcsock->sk)->syn_smc) {
 		new_smc->use_fallback = true;
+		new_smc->fallback_rsn = SMC_CLC_DECL_PEERNOSMC;
 		smc_listen_out_connected(new_smc);
 		return;
 	}
@@ -1250,7 +1253,8 @@ static void smc_listen_work(struct work_struct *work)
 	     smc_listen_rdma_reg(new_smc, local_contact))) {
 		/* SMC not supported, decline */
 		mutex_unlock(&smc_create_lgr_pending);
-		smc_listen_decline(new_smc, SMC_CLC_DECL_CNFERR, local_contact);
+		smc_listen_decline(new_smc, SMC_CLC_DECL_MODEUNSUPP,
+				   local_contact);
 		return;
 	}
 
@@ -1297,6 +1301,7 @@ static void smc_tcp_listen_work(struct work_struct *work)
 
 		new_smc->listen_smc = lsmc;
 		new_smc->use_fallback = lsmc->use_fallback;
+		new_smc->fallback_rsn = lsmc->fallback_rsn;
 		sock_hold(lsk); /* sock_put in smc_listen_work */
 		INIT_WORK(&new_smc->smc_listen_work, smc_listen_work);
 		smc_copy_sock_settings_to_smc(new_smc);
@@ -1451,6 +1456,7 @@ static int smc_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)
 	if (msg->msg_flags & MSG_FASTOPEN) {
 		if (sk->sk_state == SMC_INIT) {
 			smc->use_fallback = true;
+			smc->fallback_rsn = SMC_CLC_DECL_OPTUNSUPP;
 		} else {
 			rc = -EINVAL;
 			goto out;
@@ -1648,6 +1654,7 @@ static int smc_setsockopt(struct socket *sock, int level, int optname,
 		/* option not supported by SMC */
 		if (sk->sk_state == SMC_INIT) {
 			smc->use_fallback = true;
+			smc->fallback_rsn = SMC_CLC_DECL_OPTUNSUPP;
 		} else {
 			if (!smc->use_fallback)
 				rc = -EINVAL;
@@ -1885,6 +1892,7 @@ static int smc_create(struct net *net, struct socket *sock, int protocol,
 	/* create internal TCP socket for CLC handshake and fallback */
 	smc = smc_sk(sk);
 	smc->use_fallback = false; /* assume rdma capability first */
+	smc->fallback_rsn = 0;
 	rc = sock_create_kern(net, family, SOCK_STREAM, IPPROTO_TCP,
 			      &smc->clcsock);
 	if (rc) {
diff --git a/net/smc/smc.h b/net/smc/smc.h
index be20acd7b5ab..08786ace6010 100644
--- a/net/smc/smc.h
+++ b/net/smc/smc.h
@@ -208,6 +208,8 @@ struct smc_sock {				/* smc sock container */
 	struct list_head	accept_q;	/* sockets to be accepted */
 	spinlock_t		accept_q_lock;	/* protects accept_q */
 	bool			use_fallback;	/* fallback to tcp */
+	int			fallback_rsn;	/* reason for fallback */
+	u32			peer_diagnosis; /* decline reason from peer */
 	int			sockopt_defer_accept;
 						/* sockopt TCP_DEFER_ACCEPT
 						 * value
diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c
index 78d74938a9d9..83aba9ade060 100644
--- a/net/smc/smc_clc.c
+++ b/net/smc/smc_clc.c
@@ -334,7 +334,11 @@ int smc_clc_wait_msg(struct smc_sock *smc, void *buf, int buflen,
 		goto out;
 	}
 	if (clcm->type == SMC_CLC_DECLINE) {
-		reason_code = SMC_CLC_DECL_REPLY;
+		struct smc_clc_msg_decline *dclc;
+
+		dclc = (struct smc_clc_msg_decline *)clcm;
+		reason_code = SMC_CLC_DECL_PEERDECL;
+		smc->peer_diagnosis = ntohl(dclc->peer_diagnosis);
 		if (((struct smc_clc_msg_decline *)buf)->hdr.flag) {
 			smc->conn.lgr->sync_err = 1;
 			smc_lgr_terminate(smc->conn.lgr);
diff --git a/net/smc/smc_clc.h b/net/smc/smc_clc.h
index 6bdc63352d6a..18da89b681c2 100644
--- a/net/smc/smc_clc.h
+++ b/net/smc/smc_clc.h
@@ -28,15 +28,21 @@
 #define SMC_TYPE_B		3		/* SMC-R and SMC-D	      */
 #define CLC_WAIT_TIME		(6 * HZ)	/* max. wait time on clcsock  */
 #define SMC_CLC_DECL_MEM	0x01010000  /* insufficient memory resources  */
-#define SMC_CLC_DECL_TIMEOUT	0x02000000  /* timeout                        */
+#define SMC_CLC_DECL_TIMEOUT_CL	0x02010000  /* timeout w4 QP confirm link     */
+#define SMC_CLC_DECL_TIMEOUT_AL	0x02020000  /* timeout w4 QP add link	      */
 #define SMC_CLC_DECL_CNFERR	0x03000000  /* configuration error            */
-#define SMC_CLC_DECL_IPSEC	0x03030000  /* IPsec usage                    */
+#define SMC_CLC_DECL_PEERNOSMC	0x03010000  /* peer did not indicate SMC      */
+#define SMC_CLC_DECL_IPSEC	0x03020000  /* IPsec usage		      */
+#define SMC_CLC_DECL_NOSMCDEV	0x03030000  /* no SMC device found	      */
+#define SMC_CLC_DECL_MODEUNSUPP	0x03040000  /* smc modes do not match (R or D)*/
+#define SMC_CLC_DECL_RMBE_EC	0x03050000  /* peer has eyecatcher in RMBE    */
+#define SMC_CLC_DECL_OPTUNSUPP	0x03060000  /* fastopen sockopt not supported */
 #define SMC_CLC_DECL_SYNCERR	0x04000000  /* synchronization error          */
-#define SMC_CLC_DECL_REPLY	0x06000000  /* reply to a received decline    */
+#define SMC_CLC_DECL_PEERDECL	0x05000000  /* peer declined during handshake */
 #define SMC_CLC_DECL_INTERR	0x99990000  /* internal error                 */
-#define SMC_CLC_DECL_TCL	0x02040000  /* timeout w4 QP confirm          */
-#define SMC_CLC_DECL_SEND	0x07000000  /* sending problem                */
-#define SMC_CLC_DECL_RMBE_EC	0x08000000  /* peer has eyecatcher in RMBE    */
+#define SMC_CLC_DECL_ERR_RTOK	0x99990001  /*	 rtoken handling failed       */
+#define SMC_CLC_DECL_ERR_RDYLNK	0x99990002  /*	 ib ready link failed	      */
+#define SMC_CLC_DECL_ERR_REGRMB	0x99990003  /*	 reg rmb failed		      */
 
 struct smc_clc_msg_hdr {	/* header1 of clc messages */
 	u8 eyecatcher[4];	/* eye catcher */
diff --git a/net/smc/smc_diag.c b/net/smc/smc_diag.c
index a3cf7313a2d3..dbf64a93d68a 100644
--- a/net/smc/smc_diag.c
+++ b/net/smc/smc_diag.c
@@ -79,6 +79,7 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb,
 			   struct nlattr *bc)
 {
 	struct smc_sock *smc = smc_sk(sk);
+	struct smc_diag_fallback fallback;
 	struct user_namespace *user_ns;
 	struct smc_diag_msg *r;
 	struct nlmsghdr *nlh;
@@ -101,6 +102,11 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb,
 	if (smc_diag_msg_attrs_fill(sk, skb, r, user_ns))
 		goto errout;
 
+	fallback.reason = smc->fallback_rsn;
+	fallback.peer_diagnosis = smc->peer_diagnosis;
+	if (nla_put(skb, SMC_DIAG_FALLBACK, sizeof(fallback), &fallback) < 0)
+		goto errout;
+
 	if ((req->diag_ext & (1 << (SMC_DIAG_CONNINFO - 1))) &&
 	    smc->conn.alert_token_local) {
 		struct smc_connection *conn = &smc->conn;
-- 
2.16.4

^ permalink raw reply related

* [PATCH net-next 4/4] net/smc: improve delete link processing
From: Ursula Braun @ 2018-07-25 14:35 UTC (permalink / raw)
  To: davem; +Cc: netdev, linux-s390, schwidefsky, heiko.carstens, raspl,
	linux-kernel
In-Reply-To: <20180725143533.7082-1-ubraun@linux.ibm.com>

From: Karsten Graul <kgraul@linux.ibm.com>

Send an orderly DELETE LINK request before termination of a link group,
add support for client triggered DELETE LINK processing. And send a
disorderly DELETE LINK before module is unloaded.

Signed-off-by: Karsten Graul <kgraul@linux.ibm.com>
Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
---
 net/smc/smc_core.c | 47 ++++++++++++++++++++++++++++++++++++++++++-----
 net/smc/smc_core.h |  4 +++-
 net/smc/smc_llc.c  | 30 +++++++++++++++++++-----------
 net/smc/smc_llc.h  |  3 ++-
 net/smc/smc_wr.c   |  7 ++-----
 5 files changed, 68 insertions(+), 23 deletions(-)

diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c
index 90c10ae9ae09..a46418f45ecd 100644
--- a/net/smc/smc_core.c
+++ b/net/smc/smc_core.c
@@ -30,6 +30,7 @@
 #define SMC_LGR_NUM_INCR		256
 #define SMC_LGR_FREE_DELAY_SERV		(600 * HZ)
 #define SMC_LGR_FREE_DELAY_CLNT		(SMC_LGR_FREE_DELAY_SERV + 10 * HZ)
+#define SMC_LGR_FREE_DELAY_FAST		(8 * HZ)
 
 static struct smc_lgr_list smc_lgr_list = {	/* established link groups */
 	.lock = __SPIN_LOCK_UNLOCKED(smc_lgr_list.lock),
@@ -51,6 +52,11 @@ static void smc_lgr_schedule_free_work(struct smc_link_group *lgr)
 			 SMC_LGR_FREE_DELAY_CLNT : SMC_LGR_FREE_DELAY_SERV);
 }
 
+void smc_lgr_schedule_free_work_fast(struct smc_link_group *lgr)
+{
+	mod_delayed_work(system_wq, &lgr->free_work, SMC_LGR_FREE_DELAY_FAST);
+}
+
 /* Register connection's alert token in our lookup structure.
  * To use rbtrees we have to implement our own insert core.
  * Requires @conns_lock
@@ -133,6 +139,20 @@ static void smc_lgr_unregister_conn(struct smc_connection *conn)
 	smc_lgr_schedule_free_work(lgr);
 }
 
+/* Send delete link, either as client to request the initiation
+ * of the DELETE LINK sequence from server; or as server to
+ * initiate the delete processing. See smc_llc_rx_delete_link().
+ */
+static int smc_link_send_delete(struct smc_link *lnk)
+{
+	if (lnk->state == SMC_LNK_ACTIVE &&
+	    !smc_llc_send_delete_link(lnk, SMC_LLC_REQ, true)) {
+		smc_llc_link_deleting(lnk);
+		return 0;
+	}
+	return -ENOTCONN;
+}
+
 static void smc_lgr_free_work(struct work_struct *work)
 {
 	struct smc_link_group *lgr = container_of(to_delayed_work(work),
@@ -153,10 +173,21 @@ static void smc_lgr_free_work(struct work_struct *work)
 	list_del_init(&lgr->list); /* remove from smc_lgr_list */
 free:
 	spin_unlock_bh(&smc_lgr_list.lock);
+
+	if (!lgr->is_smcd && !lgr->terminating)	{
+		/* try to send del link msg, on error free lgr immediately */
+		if (!smc_link_send_delete(&lgr->lnk[SMC_SINGLE_LINK])) {
+			/* reschedule in case we never receive a response */
+			smc_lgr_schedule_free_work(lgr);
+			return;
+		}
+	}
+
 	if (!delayed_work_pending(&lgr->free_work)) {
-		if (!lgr->is_smcd &&
-		    lgr->lnk[SMC_SINGLE_LINK].state != SMC_LNK_INACTIVE)
-			smc_llc_link_inactive(&lgr->lnk[SMC_SINGLE_LINK]);
+		struct smc_link *lnk = &lgr->lnk[SMC_SINGLE_LINK];
+
+		if (!lgr->is_smcd && lnk->state != SMC_LNK_INACTIVE)
+			smc_llc_link_inactive(lnk);
 		smc_lgr_free(lgr);
 	}
 }
@@ -984,8 +1015,14 @@ void smc_core_exit(void)
 	spin_unlock_bh(&smc_lgr_list.lock);
 	list_for_each_entry_safe(lgr, lg, &lgr_freeing_list, list) {
 		list_del_init(&lgr->list);
-		if (!lgr->is_smcd)
-			smc_llc_link_inactive(&lgr->lnk[SMC_SINGLE_LINK]);
+		if (!lgr->is_smcd) {
+			struct smc_link *lnk = &lgr->lnk[SMC_SINGLE_LINK];
+
+			if (lnk->state == SMC_LNK_ACTIVE)
+				smc_llc_send_delete_link(lnk, SMC_LLC_REQ,
+							 false);
+			smc_llc_link_inactive(lnk);
+		}
 		cancel_delayed_work_sync(&lgr->free_work);
 		smc_lgr_free(lgr); /* free link group */
 	}
diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h
index a4f0cc4e0270..c156674733c9 100644
--- a/net/smc/smc_core.h
+++ b/net/smc/smc_core.h
@@ -34,7 +34,8 @@ enum smc_lgr_role {		/* possible roles of a link group */
 enum smc_link_state {			/* possible states of a link */
 	SMC_LNK_INACTIVE,	/* link is inactive */
 	SMC_LNK_ACTIVATING,	/* link is being activated */
-	SMC_LNK_ACTIVE		/* link is active */
+	SMC_LNK_ACTIVE,		/* link is active */
+	SMC_LNK_DELETING,	/* link is being deleted */
 };
 
 #define SMC_WR_BUF_SIZE		48	/* size of work request buffer */
@@ -265,6 +266,7 @@ int smc_conn_create(struct smc_sock *smc, bool is_smcd, int srv_first_contact,
 		    struct smc_clc_msg_local *lcl, struct smcd_dev *smcd,
 		    u64 peer_gid);
 void smcd_conn_free(struct smc_connection *conn);
+void smc_lgr_schedule_free_work_fast(struct smc_link_group *lgr);
 void smc_core_exit(void);
 
 static inline struct smc_link_group *smc_get_lgr(struct smc_link *link)
diff --git a/net/smc/smc_llc.c b/net/smc/smc_llc.c
index a88c01029fa6..9c916c709ca7 100644
--- a/net/smc/smc_llc.c
+++ b/net/smc/smc_llc.c
@@ -278,7 +278,7 @@ int smc_llc_send_add_link(struct smc_link *link, u8 mac[], u8 gid[],
 /* prepare a delete link message */
 static void smc_llc_prep_delete_link(struct smc_llc_msg_del_link *delllc,
 				     struct smc_link *link,
-				     enum smc_llc_reqresp reqresp)
+				     enum smc_llc_reqresp reqresp, bool orderly)
 {
 	memset(delllc, 0, sizeof(*delllc));
 	delllc->hd.common.type = SMC_LLC_DELETE_LINK;
@@ -287,13 +287,14 @@ static void smc_llc_prep_delete_link(struct smc_llc_msg_del_link *delllc,
 		delllc->hd.flags |= SMC_LLC_FLAG_RESP;
 	/* DEL_LINK_ALL because only 1 link supported */
 	delllc->hd.flags |= SMC_LLC_FLAG_DEL_LINK_ALL;
-	delllc->hd.flags |= SMC_LLC_FLAG_DEL_LINK_ORDERLY;
+	if (orderly)
+		delllc->hd.flags |= SMC_LLC_FLAG_DEL_LINK_ORDERLY;
 	delllc->link_num = link->link_id;
 }
 
 /* send DELETE LINK request or response */
 int smc_llc_send_delete_link(struct smc_link *link,
-			     enum smc_llc_reqresp reqresp)
+			     enum smc_llc_reqresp reqresp, bool orderly)
 {
 	struct smc_llc_msg_del_link *delllc;
 	struct smc_wr_tx_pend_priv *pend;
@@ -304,7 +305,7 @@ int smc_llc_send_delete_link(struct smc_link *link,
 	if (rc)
 		return rc;
 	delllc = (struct smc_llc_msg_del_link *)wr_buf;
-	smc_llc_prep_delete_link(delllc, link, reqresp);
+	smc_llc_prep_delete_link(delllc, link, reqresp, orderly);
 	/* send llc message */
 	rc = smc_wr_tx_send(link, pend);
 	return rc;
@@ -438,17 +439,19 @@ static void smc_llc_rx_delete_link(struct smc_link *link,
 
 	if (llc->hd.flags & SMC_LLC_FLAG_RESP) {
 		if (lgr->role == SMC_SERV)
-			smc_lgr_terminate(lgr);
+			smc_lgr_schedule_free_work_fast(lgr);
 	} else {
+		smc_lgr_forget(lgr);
+		smc_llc_link_deleting(link);
 		if (lgr->role == SMC_SERV) {
-			smc_lgr_forget(lgr);
-			smc_llc_prep_delete_link(llc, link, SMC_LLC_REQ);
-			smc_llc_send_message(link, llc, sizeof(*llc));
+			/* client asks to delete this link, send request */
+			smc_llc_prep_delete_link(llc, link, SMC_LLC_REQ, true);
 		} else {
-			smc_llc_prep_delete_link(llc, link, SMC_LLC_RESP);
-			smc_llc_send_message(link, llc, sizeof(*llc));
-			smc_lgr_terminate(lgr);
+			/* server requests to delete this link, send response */
+			smc_llc_prep_delete_link(llc, link, SMC_LLC_RESP, true);
 		}
+		smc_llc_send_message(link, llc, sizeof(*llc));
+		smc_lgr_schedule_free_work_fast(lgr);
 	}
 }
 
@@ -622,6 +625,11 @@ void smc_llc_link_active(struct smc_link *link, int testlink_time)
 	}
 }
 
+void smc_llc_link_deleting(struct smc_link *link)
+{
+	link->state = SMC_LNK_DELETING;
+}
+
 /* called in tasklet context */
 void smc_llc_link_inactive(struct smc_link *link)
 {
diff --git a/net/smc/smc_llc.h b/net/smc/smc_llc.h
index 95a7f3662e59..9e2ff088e301 100644
--- a/net/smc/smc_llc.h
+++ b/net/smc/smc_llc.h
@@ -41,9 +41,10 @@ int smc_llc_send_confirm_link(struct smc_link *lnk,
 int smc_llc_send_add_link(struct smc_link *link, u8 mac[], u8 gid[],
 			  enum smc_llc_reqresp reqresp);
 int smc_llc_send_delete_link(struct smc_link *link,
-			     enum smc_llc_reqresp reqresp);
+			     enum smc_llc_reqresp reqresp, bool orderly);
 int smc_llc_link_init(struct smc_link *link);
 void smc_llc_link_active(struct smc_link *link, int testlink_time);
+void smc_llc_link_deleting(struct smc_link *link);
 void smc_llc_link_inactive(struct smc_link *link);
 void smc_llc_link_clear(struct smc_link *link);
 int smc_llc_do_confirm_rkey(struct smc_link *link,
diff --git a/net/smc/smc_wr.c b/net/smc/smc_wr.c
index b6df69756bef..f856b8402b3f 100644
--- a/net/smc/smc_wr.c
+++ b/net/smc/smc_wr.c
@@ -182,17 +182,14 @@ int smc_wr_tx_get_free_slot(struct smc_link *link,
 		if (rc)
 			return rc;
 	} else {
-		struct smc_link_group *lgr;
-
-		lgr = smc_get_lgr(link);
 		rc = wait_event_timeout(
 			link->wr_tx_wait,
-			list_empty(&lgr->list) || /* lgr terminated */
+			link->state == SMC_LNK_INACTIVE ||
 			(smc_wr_tx_get_free_slot_index(link, &idx) != -EBUSY),
 			SMC_WR_TX_WAIT_FREE_SLOT_TIME);
 		if (!rc) {
 			/* timeout - terminate connections */
-			smc_lgr_terminate(lgr);
+			smc_lgr_terminate(smc_get_lgr(link));
 			return -EPIPE;
 		}
 		if (idx == link->wr_tx_cnt)
-- 
2.16.4

^ permalink raw reply related

* [PATCH net-next 2/4] net/smc: use correct vlan gid of RoCE device
From: Ursula Braun @ 2018-07-25 14:35 UTC (permalink / raw)
  To: davem; +Cc: netdev, linux-s390, schwidefsky, heiko.carstens, raspl,
	linux-kernel
In-Reply-To: <20180725143533.7082-1-ubraun@linux.ibm.com>

SMC code uses the base gid for VLAN traffic. The gids exchanged in
the CLC handshake and the gid index used for the QP have to switch
from the base gid to the appropriate vlan gid.

When searching for a matching IB device port for a certain vlan
device, it does not make sense to return an IB device port, which
is not enabled for the used vlan_id. Add another check whether a
vlan gid exists for a certain IB device port.

Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
---
 net/smc/af_smc.c   | 24 +++++++++++++-----------
 net/smc/smc_clc.c  | 10 ++++------
 net/smc/smc_clc.h  |  2 +-
 net/smc/smc_core.c | 37 ++++---------------------------------
 net/smc/smc_core.h |  5 +++--
 net/smc/smc_diag.c |  2 +-
 net/smc/smc_ib.c   | 41 ++++++++++++++++++++++++++++++++++++-----
 net/smc/smc_ib.h   |  3 ++-
 net/smc/smc_llc.c  | 15 +++++----------
 net/smc/smc_llc.h  |  2 +-
 net/smc/smc_pnet.c | 30 +++++++++++++++++++++---------
 net/smc/smc_pnet.h |  3 ++-
 12 files changed, 93 insertions(+), 81 deletions(-)

diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c
index 7883f70f7c6d..b81797103260 100644
--- a/net/smc/af_smc.c
+++ b/net/smc/af_smc.c
@@ -370,8 +370,7 @@ static int smc_clnt_conf_first_link(struct smc_sock *smc)
 	/* send add link reject message, only one link supported for now */
 	rc = smc_llc_send_add_link(link,
 				   link->smcibdev->mac[link->ibport - 1],
-				   &link->smcibdev->gid[link->ibport - 1],
-				   SMC_LLC_RESP);
+				   link->gid, SMC_LLC_RESP);
 	if (rc < 0)
 		return SMC_CLC_DECL_TCL;
 
@@ -469,7 +468,7 @@ static int smc_connect_abort(struct smc_sock *smc, int reason_code,
 /* check if there is a rdma device available for this connection. */
 /* called for connect and listen */
 static int smc_check_rdma(struct smc_sock *smc, struct smc_ib_device **ibdev,
-			  u8 *ibport)
+			  u8 *ibport, unsigned short vlan_id, u8 gid[])
 {
 	int reason_code = 0;
 
@@ -477,7 +476,8 @@ static int smc_check_rdma(struct smc_sock *smc, struct smc_ib_device **ibdev,
 	 * within same PNETID that also contains the ethernet device
 	 * used for the internal TCP socket
 	 */
-	smc_pnet_find_roce_resource(smc->clcsock->sk, ibdev, ibport);
+	smc_pnet_find_roce_resource(smc->clcsock->sk, ibdev, ibport, vlan_id,
+				    gid);
 	if (!(*ibdev))
 		reason_code = SMC_CLC_DECL_CNFERR; /* configuration error */
 
@@ -523,12 +523,12 @@ static int smc_connect_ism_vlan_cleanup(struct smc_sock *smc, bool is_smcd,
 static int smc_connect_clc(struct smc_sock *smc, int smc_type,
 			   struct smc_clc_msg_accept_confirm *aclc,
 			   struct smc_ib_device *ibdev, u8 ibport,
-			   struct smcd_dev *ismdev)
+			   u8 gid[], struct smcd_dev *ismdev)
 {
 	int rc = 0;
 
 	/* do inband token exchange */
-	rc = smc_clc_send_proposal(smc, smc_type, ibdev, ibport, ismdev);
+	rc = smc_clc_send_proposal(smc, smc_type, ibdev, ibport, gid, ismdev);
 	if (rc)
 		return rc;
 	/* receive SMC Accept CLC message */
@@ -650,6 +650,7 @@ static int __smc_connect(struct smc_sock *smc)
 	struct smc_clc_msg_accept_confirm aclc;
 	struct smc_ib_device *ibdev;
 	struct smcd_dev *ismdev;
+	u8 gid[SMC_GID_SIZE];
 	unsigned short vlan;
 	int smc_type;
 	int rc = 0;
@@ -681,7 +682,7 @@ static int __smc_connect(struct smc_sock *smc)
 	}
 
 	/* check if there is a rdma device available */
-	if (!smc_check_rdma(smc, &ibdev, &ibport)) {
+	if (!smc_check_rdma(smc, &ibdev, &ibport, vlan, gid)) {
 		/* RDMA is supported for this connection */
 		rdma_supported = true;
 		if (ism_supported)
@@ -695,7 +696,7 @@ static int __smc_connect(struct smc_sock *smc)
 		return smc_connect_decline_fallback(smc, SMC_CLC_DECL_CNFERR);
 
 	/* perform CLC handshake */
-	rc = smc_connect_clc(smc, smc_type, &aclc, ibdev, ibport, ismdev);
+	rc = smc_connect_clc(smc, smc_type, &aclc, ibdev, ibport, gid, ismdev);
 	if (rc) {
 		smc_connect_ism_vlan_cleanup(smc, ism_supported, ismdev, vlan);
 		return smc_connect_decline_fallback(smc, rc);
@@ -970,8 +971,7 @@ static int smc_serv_conf_first_link(struct smc_sock *smc)
 	/* send ADD LINK request to client over the RoCE fabric */
 	rc = smc_llc_send_add_link(link,
 				   link->smcibdev->mac[link->ibport - 1],
-				   &link->smcibdev->gid[link->ibport - 1],
-				   SMC_LLC_REQ);
+				   link->gid, SMC_LLC_REQ);
 	if (rc < 0)
 		return SMC_CLC_DECL_TCL;
 
@@ -1193,6 +1193,7 @@ static void smc_listen_work(struct work_struct *work)
 	struct smcd_dev *ismdev;
 	u8 buf[SMC_CLC_MAX_LEN];
 	int local_contact = 0;
+	unsigned short vlan;
 	int reason_code = 0;
 	int rc = 0;
 	u8 ibport;
@@ -1241,7 +1242,8 @@ static void smc_listen_work(struct work_struct *work)
 	/* check if RDMA is available */
 	if (!ism_supported &&
 	    ((pclc->hdr.path != SMC_TYPE_R && pclc->hdr.path != SMC_TYPE_B) ||
-	     smc_check_rdma(new_smc, &ibdev, &ibport) ||
+	     smc_vlan_by_tcpsk(new_smc->clcsock, &vlan) ||
+	     smc_check_rdma(new_smc, &ibdev, &ibport, vlan, NULL) ||
 	     smc_listen_rdma_check(new_smc, pclc) ||
 	     smc_listen_rdma_init(new_smc, pclc, ibdev, ibport,
 				  &local_contact) ||
diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c
index ad39efdb4f1c..78d74938a9d9 100644
--- a/net/smc/smc_clc.c
+++ b/net/smc/smc_clc.c
@@ -378,7 +378,7 @@ int smc_clc_send_decline(struct smc_sock *smc, u32 peer_diag_info)
 
 /* send CLC PROPOSAL message across internal TCP socket */
 int smc_clc_send_proposal(struct smc_sock *smc, int smc_type,
-			  struct smc_ib_device *ibdev, u8 ibport,
+			  struct smc_ib_device *ibdev, u8 ibport, u8 gid[],
 			  struct smcd_dev *ismdev)
 {
 	struct smc_clc_ipv6_prefix ipv6_prfx[SMC_CLC_MAX_V6_PREFIX];
@@ -409,7 +409,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, int smc_type,
 		/* add SMC-R specifics */
 		memcpy(pclc.lcl.id_for_peer, local_systemid,
 		       sizeof(local_systemid));
-		memcpy(&pclc.lcl.gid, &ibdev->gid[ibport - 1], SMC_GID_SIZE);
+		memcpy(&pclc.lcl.gid, gid, SMC_GID_SIZE);
 		memcpy(&pclc.lcl.mac, &ibdev->mac[ibport - 1], ETH_ALEN);
 		pclc.iparea_offset = htons(0);
 	}
@@ -492,8 +492,7 @@ int smc_clc_send_confirm(struct smc_sock *smc)
 		cclc.hdr.length = htons(SMCR_CLC_ACCEPT_CONFIRM_LEN);
 		memcpy(cclc.lcl.id_for_peer, local_systemid,
 		       sizeof(local_systemid));
-		memcpy(&cclc.lcl.gid, &link->smcibdev->gid[link->ibport - 1],
-		       SMC_GID_SIZE);
+		memcpy(&cclc.lcl.gid, link->gid, SMC_GID_SIZE);
 		memcpy(&cclc.lcl.mac, &link->smcibdev->mac[link->ibport - 1],
 		       ETH_ALEN);
 		hton24(cclc.qpn, link->roce_qp->qp_num);
@@ -566,8 +565,7 @@ int smc_clc_send_accept(struct smc_sock *new_smc, int srv_first_contact)
 		link = &conn->lgr->lnk[SMC_SINGLE_LINK];
 		memcpy(aclc.lcl.id_for_peer, local_systemid,
 		       sizeof(local_systemid));
-		memcpy(&aclc.lcl.gid, &link->smcibdev->gid[link->ibport - 1],
-		       SMC_GID_SIZE);
+		memcpy(&aclc.lcl.gid, link->gid, SMC_GID_SIZE);
 		memcpy(&aclc.lcl.mac, link->smcibdev->mac[link->ibport - 1],
 		       ETH_ALEN);
 		hton24(aclc.qpn, link->roce_qp->qp_num);
diff --git a/net/smc/smc_clc.h b/net/smc/smc_clc.h
index 100e988ad1a8..6bdc63352d6a 100644
--- a/net/smc/smc_clc.h
+++ b/net/smc/smc_clc.h
@@ -179,7 +179,7 @@ int smc_clc_wait_msg(struct smc_sock *smc, void *buf, int buflen,
 		     u8 expected_type);
 int smc_clc_send_decline(struct smc_sock *smc, u32 peer_diag_info);
 int smc_clc_send_proposal(struct smc_sock *smc, int smc_type,
-			  struct smc_ib_device *smcibdev, u8 ibport,
+			  struct smc_ib_device *smcibdev, u8 ibport, u8 gid[],
 			  struct smcd_dev *ismdev);
 int smc_clc_send_confirm(struct smc_sock *smc);
 int smc_clc_send_accept(struct smc_sock *smc, int srv_first_contact);
diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c
index 66741e61a3b0..90c10ae9ae09 100644
--- a/net/smc/smc_core.c
+++ b/net/smc/smc_core.c
@@ -219,6 +219,10 @@ static int smc_lgr_create(struct smc_sock *smc, bool is_smcd,
 		get_random_bytes(rndvec, sizeof(rndvec));
 		lnk->psn_initial = rndvec[0] + (rndvec[1] << 8) +
 			(rndvec[2] << 16);
+		rc = smc_ib_determine_gid(lnk->smcibdev, lnk->ibport,
+					  vlan_id, lnk->gid, &lnk->sgid_index);
+		if (rc)
+			goto free_lgr;
 		rc = smc_llc_link_init(lnk);
 		if (rc)
 			goto free_lgr;
@@ -522,37 +526,6 @@ int smc_vlan_by_tcpsk(struct socket *clcsock, unsigned short *vlan_id)
 	return rc;
 }
 
-/* determine the link gid matching the vlan id of the link group */
-static int smc_link_determine_gid(struct smc_link_group *lgr)
-{
-	struct smc_link *lnk = &lgr->lnk[SMC_SINGLE_LINK];
-	struct ib_gid_attr gattr;
-	union ib_gid gid;
-	int i;
-
-	if (!lgr->vlan_id) {
-		lnk->gid = lnk->smcibdev->gid[lnk->ibport - 1];
-		return 0;
-	}
-
-	for (i = 0; i < lnk->smcibdev->pattr[lnk->ibport - 1].gid_tbl_len;
-	     i++) {
-		if (ib_query_gid(lnk->smcibdev->ibdev, lnk->ibport, i, &gid,
-				 &gattr))
-			continue;
-		if (gattr.ndev) {
-			if (is_vlan_dev(gattr.ndev) &&
-			    vlan_dev_vlan_id(gattr.ndev) == lgr->vlan_id) {
-				lnk->gid = gid;
-				dev_put(gattr.ndev);
-				return 0;
-			}
-			dev_put(gattr.ndev);
-		}
-	}
-	return -ENODEV;
-}
-
 static bool smcr_lgr_match(struct smc_link_group *lgr,
 			   struct smc_clc_msg_local *lcl,
 			   enum smc_lgr_role role)
@@ -631,8 +604,6 @@ int smc_conn_create(struct smc_sock *smc, bool is_smcd, int srv_first_contact,
 		if (rc)
 			goto out;
 		smc_lgr_register_conn(conn); /* add smc conn to lgr */
-		if (!is_smcd)
-			rc = smc_link_determine_gid(conn->lgr);
 	}
 	conn->local_tx_ctrl.common.type = SMC_CDC_MSG_TYPE;
 	conn->local_tx_ctrl.len = SMC_WR_TX_SIZE;
diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h
index 1e8974c50550..a4f0cc4e0270 100644
--- a/net/smc/smc_core.h
+++ b/net/smc/smc_core.h
@@ -84,14 +84,15 @@ struct smc_link {
 	wait_queue_head_t	wr_reg_wait;	/* wait for wr_reg result */
 	enum smc_wr_reg_state	wr_reg_state;	/* state of wr_reg request */
 
-	union ib_gid		gid;		/* gid matching used vlan id */
+	u8			gid[SMC_GID_SIZE];/* gid matching used vlan id*/
+	u8			sgid_index;	/* gid index for vlan id      */
 	u32			peer_qpn;	/* QP number of peer */
 	enum ib_mtu		path_mtu;	/* used mtu */
 	enum ib_mtu		peer_mtu;	/* mtu size of peer */
 	u32			psn_initial;	/* QP tx initial packet seqno */
 	u32			peer_psn;	/* QP rx initial packet seqno */
 	u8			peer_mac[ETH_ALEN];	/* = gid[8:10||13:15] */
-	u8			peer_gid[sizeof(union ib_gid)];	/* gid of peer*/
+	u8			peer_gid[SMC_GID_SIZE];	/* gid of peer*/
 	u8			link_id;	/* unique # within link group */
 
 	enum smc_link_state	state;		/* state of link */
diff --git a/net/smc/smc_diag.c b/net/smc/smc_diag.c
index d772cd10297e..a3cf7313a2d3 100644
--- a/net/smc/smc_diag.c
+++ b/net/smc/smc_diag.c
@@ -154,7 +154,7 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb,
 		       smc->conn.lgr->lnk[0].smcibdev->ibdev->name,
 		       sizeof(smc->conn.lgr->lnk[0].smcibdev->ibdev->name));
 		smc_gid_be16_convert(linfo.lnk[0].gid,
-				     smc->conn.lgr->lnk[0].gid.raw);
+				     smc->conn.lgr->lnk[0].gid);
 		smc_gid_be16_convert(linfo.lnk[0].peer_gid,
 				     smc->conn.lgr->lnk[0].peer_gid);
 
diff --git a/net/smc/smc_ib.c b/net/smc/smc_ib.c
index 4706ab7092a9..2cc64bc8ae20 100644
--- a/net/smc/smc_ib.c
+++ b/net/smc/smc_ib.c
@@ -68,7 +68,7 @@ static int smc_ib_modify_qp_rtr(struct smc_link *lnk)
 	qp_attr.path_mtu = min(lnk->path_mtu, lnk->peer_mtu);
 	qp_attr.ah_attr.type = RDMA_AH_ATTR_TYPE_ROCE;
 	rdma_ah_set_port_num(&qp_attr.ah_attr, lnk->ibport);
-	rdma_ah_set_grh(&qp_attr.ah_attr, NULL, 0, 0, 1, 0);
+	rdma_ah_set_grh(&qp_attr.ah_attr, NULL, 0, lnk->sgid_index, 1, 0);
 	rdma_ah_set_dgid_raw(&qp_attr.ah_attr, lnk->peer_gid);
 	memcpy(&qp_attr.ah_attr.roce.dmac, lnk->peer_mac,
 	       sizeof(lnk->peer_mac));
@@ -142,13 +142,13 @@ int smc_ib_ready_link(struct smc_link *lnk)
 	return rc;
 }
 
-static int smc_ib_fill_gid_and_mac(struct smc_ib_device *smcibdev, u8 ibport)
+static int smc_ib_fill_mac(struct smc_ib_device *smcibdev, u8 ibport)
 {
 	struct ib_gid_attr gattr;
+	union ib_gid gid;
 	int rc;
 
-	rc = ib_query_gid(smcibdev->ibdev, ibport, 0,
-			  &smcibdev->gid[ibport - 1], &gattr);
+	rc = ib_query_gid(smcibdev->ibdev, ibport, 0, &gid, &gattr);
 	if (rc || !gattr.ndev)
 		return -ENODEV;
 
@@ -175,6 +175,37 @@ bool smc_ib_port_active(struct smc_ib_device *smcibdev, u8 ibport)
 	return smcibdev->pattr[ibport - 1].state == IB_PORT_ACTIVE;
 }
 
+/* determine the gid for an ib-device port and vlan id */
+int smc_ib_determine_gid(struct smc_ib_device *smcibdev, u8 ibport,
+			 unsigned short vlan_id, u8 gid[], u8 *sgid_index)
+{
+	struct ib_gid_attr gattr;
+	union ib_gid _gid;
+	int i;
+
+	for (i = 0; i < smcibdev->pattr[ibport - 1].gid_tbl_len; i++) {
+		memset(&_gid, 0, SMC_GID_SIZE);
+		memset(&gattr, 0, sizeof(gattr));
+		if (ib_query_gid(smcibdev->ibdev, ibport, i, &_gid, &gattr))
+			continue;
+		if (!gattr.ndev)
+			continue;
+		if (((!vlan_id && !is_vlan_dev(gattr.ndev)) ||
+		     (vlan_id && is_vlan_dev(gattr.ndev) &&
+		      vlan_dev_vlan_id(gattr.ndev) == vlan_id)) &&
+		    gattr.gid_type == IB_GID_TYPE_IB) {
+			if (gid)
+				memcpy(gid, &_gid, SMC_GID_SIZE);
+			if (sgid_index)
+				*sgid_index = i;
+			dev_put(gattr.ndev);
+			return 0;
+		}
+		dev_put(gattr.ndev);
+	}
+	return -ENODEV;
+}
+
 static int smc_ib_remember_port_attr(struct smc_ib_device *smcibdev, u8 ibport)
 {
 	int rc;
@@ -186,7 +217,7 @@ static int smc_ib_remember_port_attr(struct smc_ib_device *smcibdev, u8 ibport)
 	if (rc)
 		goto out;
 	/* the SMC protocol requires specification of the RoCE MAC address */
-	rc = smc_ib_fill_gid_and_mac(smcibdev, ibport);
+	rc = smc_ib_fill_mac(smcibdev, ibport);
 	if (rc)
 		goto out;
 	if (!strncmp(local_systemid, SMC_LOCAL_SYSTEMID_RESET,
diff --git a/net/smc/smc_ib.h b/net/smc/smc_ib.h
index 7c1223c91229..bac7fd65a4c0 100644
--- a/net/smc/smc_ib.h
+++ b/net/smc/smc_ib.h
@@ -40,7 +40,6 @@ struct smc_ib_device {				/* ib-device infos for smc */
 	struct tasklet_struct	recv_tasklet;	/* called by recv cq handler */
 	char			mac[SMC_MAX_PORTS][ETH_ALEN];
 						/* mac address per port*/
-	union ib_gid		gid[SMC_MAX_PORTS]; /* gid per port */
 	u8			pnetid[SMC_MAX_PORTS][SMC_MAX_PNETID_LEN];
 						/* pnetid per port */
 	u8			initialized : 1; /* ib dev CQ, evthdl done */
@@ -77,4 +76,6 @@ void smc_ib_sync_sg_for_cpu(struct smc_ib_device *smcibdev,
 void smc_ib_sync_sg_for_device(struct smc_ib_device *smcibdev,
 			       struct smc_buf_desc *buf_slot,
 			       enum dma_data_direction data_direction);
+int smc_ib_determine_gid(struct smc_ib_device *smcibdev, u8 ibport,
+			 unsigned short vlan_id, u8 gid[], u8 *sgid_index);
 #endif
diff --git a/net/smc/smc_llc.c b/net/smc/smc_llc.c
index f2ba99c2e69a..a88c01029fa6 100644
--- a/net/smc/smc_llc.c
+++ b/net/smc/smc_llc.c
@@ -203,8 +203,7 @@ int smc_llc_send_confirm_link(struct smc_link *link,
 		confllc->hd.flags |= SMC_LLC_FLAG_RESP;
 	memcpy(confllc->sender_mac, link->smcibdev->mac[link->ibport - 1],
 	       ETH_ALEN);
-	memcpy(confllc->sender_gid, &link->smcibdev->gid[link->ibport - 1],
-	       SMC_GID_SIZE);
+	memcpy(confllc->sender_gid, link->gid, SMC_GID_SIZE);
 	hton24(confllc->sender_qp_num, link->roce_qp->qp_num);
 	confllc->link_num = link->link_id;
 	memcpy(confllc->link_uid, lgr->id, SMC_LGR_ID_SIZE);
@@ -241,8 +240,7 @@ static int smc_llc_send_confirm_rkey(struct smc_link *link,
 
 /* prepare an add link message */
 static void smc_llc_prep_add_link(struct smc_llc_msg_add_link *addllc,
-				  struct smc_link *link, u8 mac[],
-				  union ib_gid *gid,
+				  struct smc_link *link, u8 mac[], u8 gid[],
 				  enum smc_llc_reqresp reqresp)
 {
 	memset(addllc, 0, sizeof(*addllc));
@@ -259,8 +257,7 @@ static void smc_llc_prep_add_link(struct smc_llc_msg_add_link *addllc,
 }
 
 /* send ADD LINK request or response */
-int smc_llc_send_add_link(struct smc_link *link, u8 mac[],
-			  union ib_gid *gid,
+int smc_llc_send_add_link(struct smc_link *link, u8 mac[], u8 gid[],
 			  enum smc_llc_reqresp reqresp)
 {
 	struct smc_llc_msg_add_link *addllc;
@@ -423,14 +420,12 @@ static void smc_llc_rx_add_link(struct smc_link *link,
 		if (lgr->role == SMC_SERV) {
 			smc_llc_prep_add_link(llc, link,
 					link->smcibdev->mac[link->ibport - 1],
-					&link->smcibdev->gid[link->ibport - 1],
-					SMC_LLC_REQ);
+					link->gid, SMC_LLC_REQ);
 
 		} else {
 			smc_llc_prep_add_link(llc, link,
 					link->smcibdev->mac[link->ibport - 1],
-					&link->smcibdev->gid[link->ibport - 1],
-					SMC_LLC_RESP);
+					link->gid, SMC_LLC_RESP);
 		}
 		smc_llc_send_message(link, llc, sizeof(*llc));
 	}
diff --git a/net/smc/smc_llc.h b/net/smc/smc_llc.h
index 9a29fcbbcea8..95a7f3662e59 100644
--- a/net/smc/smc_llc.h
+++ b/net/smc/smc_llc.h
@@ -38,7 +38,7 @@ enum smc_llc_msg_type {
 /* transmit */
 int smc_llc_send_confirm_link(struct smc_link *lnk,
 			      enum smc_llc_reqresp reqresp);
-int smc_llc_send_add_link(struct smc_link *link, u8 mac[], union ib_gid *gid,
+int smc_llc_send_add_link(struct smc_link *link, u8 mac[], u8 gid[],
 			  enum smc_llc_reqresp reqresp);
 int smc_llc_send_delete_link(struct smc_link *link,
 			     enum smc_llc_reqresp reqresp);
diff --git a/net/smc/smc_pnet.c b/net/smc/smc_pnet.c
index 1b6c066d3495..01c6ce042a1c 100644
--- a/net/smc/smc_pnet.c
+++ b/net/smc/smc_pnet.c
@@ -535,11 +535,13 @@ static struct net_device *pnet_find_base_ndev(struct net_device *ndev)
 }
 
 /* Determine the corresponding IB device port based on the hardware PNETID.
- * Searching stops at the first matching active IB device port.
+ * Searching stops at the first matching active IB device port with vlan_id
+ * configured.
  */
 static void smc_pnet_find_roce_by_pnetid(struct net_device *ndev,
 					 struct smc_ib_device **smcibdev,
-					 u8 *ibport)
+					 u8 *ibport, unsigned short vlan_id,
+					 u8 gid[])
 {
 	u8 ndev_pnetid[SMC_MAX_PNETID_LEN];
 	struct smc_ib_device *ibdev;
@@ -553,15 +555,20 @@ static void smc_pnet_find_roce_by_pnetid(struct net_device *ndev,
 	spin_lock(&smc_ib_devices.lock);
 	list_for_each_entry(ibdev, &smc_ib_devices.list, list) {
 		for (i = 1; i <= SMC_MAX_PORTS; i++) {
+			if (!rdma_is_port_valid(ibdev->ibdev, i))
+				continue;
 			if (!memcmp(ibdev->pnetid[i - 1], ndev_pnetid,
 				    SMC_MAX_PNETID_LEN) &&
-			    smc_ib_port_active(ibdev, i)) {
+			    smc_ib_port_active(ibdev, i) &&
+			    !smc_ib_determine_gid(ibdev, i, vlan_id, gid,
+						  NULL))  {
 				*smcibdev = ibdev;
 				*ibport = i;
-				break;
+				goto out;
 			}
 		}
 	}
+out:
 	spin_unlock(&smc_ib_devices.lock);
 }
 
@@ -589,7 +596,8 @@ static void smc_pnet_find_ism_by_pnetid(struct net_device *ndev,
 /* Lookup of coupled ib_device via SMC pnet table */
 static void smc_pnet_find_roce_by_table(struct net_device *netdev,
 					struct smc_ib_device **smcibdev,
-					u8 *ibport)
+					u8 *ibport, unsigned short vlan_id,
+					u8 gid[])
 {
 	struct smc_pnetentry *pnetelem;
 
@@ -597,7 +605,10 @@ static void smc_pnet_find_roce_by_table(struct net_device *netdev,
 	list_for_each_entry(pnetelem, &smc_pnettable.pnetlist, list) {
 		if (netdev == pnetelem->ndev) {
 			if (smc_ib_port_active(pnetelem->smcibdev,
-					       pnetelem->ib_port)) {
+					       pnetelem->ib_port) &&
+			    !smc_ib_determine_gid(pnetelem->smcibdev,
+						  pnetelem->ib_port, vlan_id,
+						  gid, NULL)) {
 				*smcibdev = pnetelem->smcibdev;
 				*ibport = pnetelem->ib_port;
 			}
@@ -612,7 +623,8 @@ static void smc_pnet_find_roce_by_table(struct net_device *netdev,
  * ethernet interface.
  */
 void smc_pnet_find_roce_resource(struct sock *sk,
-				 struct smc_ib_device **smcibdev, u8 *ibport)
+				 struct smc_ib_device **smcibdev, u8 *ibport,
+				 unsigned short vlan_id, u8 gid[])
 {
 	struct dst_entry *dst = sk_dst_get(sk);
 
@@ -625,12 +637,12 @@ void smc_pnet_find_roce_resource(struct sock *sk,
 		goto out_rel;
 
 	/* if possible, lookup via hardware-defined pnetid */
-	smc_pnet_find_roce_by_pnetid(dst->dev, smcibdev, ibport);
+	smc_pnet_find_roce_by_pnetid(dst->dev, smcibdev, ibport, vlan_id, gid);
 	if (*smcibdev)
 		goto out_rel;
 
 	/* lookup via SMC PNET table */
-	smc_pnet_find_roce_by_table(dst->dev, smcibdev, ibport);
+	smc_pnet_find_roce_by_table(dst->dev, smcibdev, ibport, vlan_id, gid);
 
 out_rel:
 	dst_release(dst);
diff --git a/net/smc/smc_pnet.h b/net/smc/smc_pnet.h
index 1e94fd4df7bc..8ff777636e32 100644
--- a/net/smc/smc_pnet.h
+++ b/net/smc/smc_pnet.h
@@ -33,7 +33,8 @@ int smc_pnet_init(void) __init;
 void smc_pnet_exit(void);
 int smc_pnet_remove_by_ibdev(struct smc_ib_device *ibdev);
 void smc_pnet_find_roce_resource(struct sock *sk,
-				 struct smc_ib_device **smcibdev, u8 *ibport);
+				 struct smc_ib_device **smcibdev, u8 *ibport,
+				 unsigned short vlan_id, u8 gid[]);
 void smc_pnet_find_ism_resource(struct sock *sk, struct smcd_dev **smcismdev);
 
 #endif
-- 
2.16.4

^ permalink raw reply related

* [PATCH net-next 1/4] net/smc: fewer parameters for smc_llc_send_confirm_link()
From: Ursula Braun @ 2018-07-25 14:35 UTC (permalink / raw)
  To: davem; +Cc: netdev, linux-s390, schwidefsky, heiko.carstens, raspl,
	linux-kernel
In-Reply-To: <20180725143533.7082-1-ubraun@linux.ibm.com>

Link confirmation will always be sent across the new link being
confirmed. This allows to shrink the parameter list.
No functional change.

Signed-off-by: Ursula Braun <ubraun@linux.ibm.com>
---
 net/smc/af_smc.c  | 10 ++--------
 net/smc/smc_llc.c |  9 +++++----
 net/smc/smc_llc.h |  2 +-
 3 files changed, 8 insertions(+), 13 deletions(-)

diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c
index 7fc810ec31c5..7883f70f7c6d 100644
--- a/net/smc/af_smc.c
+++ b/net/smc/af_smc.c
@@ -352,10 +352,7 @@ static int smc_clnt_conf_first_link(struct smc_sock *smc)
 		return SMC_CLC_DECL_INTERR;
 
 	/* send CONFIRM LINK response over RoCE fabric */
-	rc = smc_llc_send_confirm_link(link,
-				       link->smcibdev->mac[link->ibport - 1],
-				       &link->smcibdev->gid[link->ibport - 1],
-				       SMC_LLC_RESP);
+	rc = smc_llc_send_confirm_link(link, SMC_LLC_RESP);
 	if (rc < 0)
 		return SMC_CLC_DECL_TCL;
 
@@ -951,10 +948,7 @@ static int smc_serv_conf_first_link(struct smc_sock *smc)
 		return SMC_CLC_DECL_INTERR;
 
 	/* send CONFIRM LINK request to client over the RoCE fabric */
-	rc = smc_llc_send_confirm_link(link,
-				       link->smcibdev->mac[link->ibport - 1],
-				       &link->smcibdev->gid[link->ibport - 1],
-				       SMC_LLC_REQ);
+	rc = smc_llc_send_confirm_link(link, SMC_LLC_REQ);
 	if (rc < 0)
 		return SMC_CLC_DECL_TCL;
 
diff --git a/net/smc/smc_llc.c b/net/smc/smc_llc.c
index b7944aa1ffc3..f2ba99c2e69a 100644
--- a/net/smc/smc_llc.c
+++ b/net/smc/smc_llc.c
@@ -182,8 +182,7 @@ static int smc_llc_add_pending_send(struct smc_link *link,
 }
 
 /* high-level API to send LLC confirm link */
-int smc_llc_send_confirm_link(struct smc_link *link, u8 mac[],
-			      union ib_gid *gid,
+int smc_llc_send_confirm_link(struct smc_link *link,
 			      enum smc_llc_reqresp reqresp)
 {
 	struct smc_link_group *lgr = smc_get_lgr(link);
@@ -202,8 +201,10 @@ int smc_llc_send_confirm_link(struct smc_link *link, u8 mac[],
 	confllc->hd.flags |= SMC_LLC_FLAG_NO_RMBE_EYEC;
 	if (reqresp == SMC_LLC_RESP)
 		confllc->hd.flags |= SMC_LLC_FLAG_RESP;
-	memcpy(confllc->sender_mac, mac, ETH_ALEN);
-	memcpy(confllc->sender_gid, gid, SMC_GID_SIZE);
+	memcpy(confllc->sender_mac, link->smcibdev->mac[link->ibport - 1],
+	       ETH_ALEN);
+	memcpy(confllc->sender_gid, &link->smcibdev->gid[link->ibport - 1],
+	       SMC_GID_SIZE);
 	hton24(confllc->sender_qp_num, link->roce_qp->qp_num);
 	confllc->link_num = link->link_id;
 	memcpy(confllc->link_uid, lgr->id, SMC_LGR_ID_SIZE);
diff --git a/net/smc/smc_llc.h b/net/smc/smc_llc.h
index 65c8645e96a1..9a29fcbbcea8 100644
--- a/net/smc/smc_llc.h
+++ b/net/smc/smc_llc.h
@@ -36,7 +36,7 @@ enum smc_llc_msg_type {
 };
 
 /* transmit */
-int smc_llc_send_confirm_link(struct smc_link *lnk, u8 mac[], union ib_gid *gid,
+int smc_llc_send_confirm_link(struct smc_link *lnk,
 			      enum smc_llc_reqresp reqresp);
 int smc_llc_send_add_link(struct smc_link *link, u8 mac[], union ib_gid *gid,
 			  enum smc_llc_reqresp reqresp);
-- 
2.16.4

^ permalink raw reply related

* Re: [PATCH rdma-next v2 0/8] Support mlx5 flow steering with RAW data
From: Jason Gunthorpe @ 2018-07-25 14:35 UTC (permalink / raw)
  To: Leon Romanovsky
  Cc: Doug Ledford, RDMA mailing list, Yishai Hadas, Saeed Mahameed,
	linux-netdev
In-Reply-To: <20180725053703.GK3189@mtr-leonro.mtl.com>

On Wed, Jul 25, 2018 at 08:37:03AM +0300, Leon Romanovsky wrote:

> > Also, I would like to keep the specs consistently formatted according
> > to clang-format with 'BinPackParameters: true', so I reflowed them as
> > well.
> 
> I'm using default VIM clang-format.py without anything in .clang-format.
> Do you have an extra definitions there, except BinPackParameters?

These days Linux includes a top level .clang-format that does a
pretty good job.

I have to manually switch BinPackParameters on when working with these
specs to get the right indenting.. A pain, but maybe there is a better
way someday..

Jason

^ permalink raw reply

* Re: [PATCH net-next v3 5/5] act_mirred: use TC_ACT_REINJECT when possible
From: Jiri Pirko @ 2018-07-25 14:30 UTC (permalink / raw)
  To: Paolo Abeni
  Cc: netdev, Jamal Hadi Salim, Cong Wang, Daniel Borkmann,
	Marcelo Ricardo Leitner, Eyal Birger, David S. Miller
In-Reply-To: <721f42fb778546dfc1f9e2ae1309fcf4476d3c84.camel@redhat.com>

Wed, Jul 25, 2018 at 04:04:03PM CEST, pabeni@redhat.com wrote:
>On Wed, 2018-07-25 at 15:52 +0200, Jiri Pirko wrote:
>> Tue, Jul 24, 2018 at 10:06:43PM CEST, pabeni@redhat.com wrote:
>> > When mirred is invoked from the ingress path, and it wants to redirect
>> > the processed packet, it can now use the TC_ACT_REINJECT action,
>> > filling the tcf_result accordingly, and avoiding a per packet
>> > skb_clone().
>> > 
>> > Overall this gives a ~10% improvement in forwarding performance for the
>> > TC S/W data path and TC S/W performances are now comparable to the
>> > kernel openvswitch datapath.
>> > 
>> > v1 -> v2: use ACT_MIRRED instead of ACT_REDIRECT
>> > v2 -> v3: updated after action rename, fixed a typo into the commit
>> > 	message
>> > 
>> > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>> > ---
>> > net/sched/act_mirred.c | 34 ++++++++++++++++++++++++----------
>> > 1 file changed, 24 insertions(+), 10 deletions(-)
>> > 
>> > diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c
>> > index eeb335f03102..368187312136 100644
>> > --- a/net/sched/act_mirred.c
>> > +++ b/net/sched/act_mirred.c
>> > @@ -25,6 +25,7 @@
>> > #include <net/net_namespace.h>
>> > #include <net/netlink.h>
>> > #include <net/pkt_sched.h>
>> > +#include <net/pkt_cls.h>
>> > #include <linux/tc_act/tc_mirred.h>
>> > #include <net/tc_act/tc_mirred.h>
>> > 
>> > @@ -171,10 +172,12 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
>> > 		      struct tcf_result *res)
>> > {
>> > 	struct tcf_mirred *m = to_mirred(a);
>> > +	struct sk_buff *skb2 = skb;
>> > 	bool m_mac_header_xmit;
>> > 	struct net_device *dev;
>> > -	struct sk_buff *skb2;
>> > 	int retval, err = 0;
>> > +	bool want_ingress;
>> > +	bool is_redirect;
>> > 	int m_eaction;
>> > 	int mac_len;
>> > 
>> > @@ -196,16 +199,19 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
>> > 		goto out;
>> > 	}
>> > 
>> > -	skb2 = skb_clone(skb, GFP_ATOMIC);
>> > -	if (!skb2)
>> > -		goto out;
>> > +	is_redirect = tcf_mirred_is_act_redirect(m_eaction);
>> > +	if (!skb_at_tc_ingress(skb) || !is_redirect) {
>> > +		skb2 = skb_clone(skb, GFP_ATOMIC);
>> > +		if (!skb2)
>> > +			goto out;
>> > +	}
>> > 
>> > 	/* If action's target direction differs than filter's direction,
>> > 	 * and devices expect a mac header on xmit, then mac push/pull is
>> > 	 * needed.
>> > 	 */
>> > -	if (skb_at_tc_ingress(skb) != tcf_mirred_act_wants_ingress(m_eaction) &&
>> > -	    m_mac_header_xmit) {
>> > +	want_ingress = tcf_mirred_act_wants_ingress(m_eaction);
>> > +	if (skb_at_tc_ingress(skb) != want_ingress && m_mac_header_xmit) {
>> > 		if (!skb_at_tc_ingress(skb)) {
>> > 			/* caught at egress, act ingress: pull mac */
>> > 			mac_len = skb_network_header(skb) - skb_mac_header(skb);
>> > @@ -216,15 +222,23 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
>> > 		}
>> > 	}
>> > 
>> > +	skb2->skb_iif = skb->dev->ifindex;
>> > +	skb2->dev = dev;
>> > +
>> > 	/* mirror is always swallowed */
>> > -	if (tcf_mirred_is_act_redirect(m_eaction)) {
>> > +	if (is_redirect) {
>> > 		skb2->tc_redirected = 1;
>> > 		skb2->tc_from_ingress = skb2->tc_at_ingress;
>> > +
>> > +		/* let's the caller reinject the packet, if possible */
>> > +		if (skb_at_tc_ingress(skb)) {
>> 
>> I probably missed something. Why only on ingress?
>
>To keep the implementation as simple as possible: if I read correctly,
>it is impossible for a filter detect if called by the clsact or the dev
>root qdisc, and I think we could safely avoid the skb clone with a not
>invasive patch, only if called from the clsact.
>
>[please let me know if the above is somewhat clear ;)]
>
>Also this covers nicely the relevant use case (TC S/W datapath).

Sure. I was just curious. Perhaps put a comment to this optimisation
describing why it is not possible for egress. It might help future
readers.

Thanks!

>
>Thanks,
>
>Paolo
>

^ permalink raw reply

* Re: [PATCH v2 net-next] net: phy: add helper phy_polling_mode
From: Andrew Lunn @ 2018-07-25 14:26 UTC (permalink / raw)
  To: Heiner Kallweit; +Cc: Florian Fainelli, David Miller, netdev@vger.kernel.org
In-Reply-To: <e27ffb1d-4c9f-60ee-fb18-a19fc8937b62@gmail.com>

On Mon, Jul 23, 2018 at 09:40:07PM +0200, Heiner Kallweit wrote:
> Add a helper for checking whether polling is used to detect PHY status
> changes.
> 
> Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>

Reviewed-by: Andrew Lunn <andrew@lunn.ch>

    Andrew

^ permalink raw reply

* Re: [PATCH net-next v3 4/5] net/tc: introduce TC_ACT_REINJECT.
From: Paolo Abeni @ 2018-07-25 14:24 UTC (permalink / raw)
  To: Jamal Hadi Salim, Cong Wang
  Cc: Linux Kernel Network Developers, Jiri Pirko, Daniel Borkmann,
	Marcelo Ricardo Leitner, Eyal Birger, David Miller
In-Reply-To: <f2856833-d69c-7cef-1bf9-0d31c7cf7343@mojatatu.com>

On Wed, 2018-07-25 at 08:27 -0400, Jamal Hadi Salim wrote:
> On 25/07/18 04:29 AM, Paolo Abeni wrote:
> > On Tue, 2018-07-24 at 13:50 -0700, Cong Wang wrote:
> 
> [..]
> > > > I fail to understand why overlimit is increased in your case
> > > > here. I guess you want to increase 'drops' instead.
> > > 
> > > Hmm, actually the current mirred code increases overlimit too.
> > > But I still don't think it makes sense.
> > 
> > Yep, I chose to increment 'overlimits' to preserve the current mirred
> > semantic.
> > 
> > AFAICS, that was first introduced with:
> > 
> > commit 8919bc13e8d92c5b082c5c0321567383a071f5bc
> > Author: Jamal Hadi Salim <jhs@mojatatu.com>
> > Date:   Mon Aug 15 05:25:40 2011 +0000
> > 
> >      net_sched: fix port mirror/redirect stats reporting
> > 
> > Likely increasing 'drops' would be "better", but I'm unsure we can
> > change this established behavior without affecting some user.
> > 
> 
> Those changes were there from the beginning (above patch did
> not introduce them).
> IIRC, the reason was to distinguish between policy intended
> drops and drops because of errors.

Double-checking to avoid misinterepration on my side: you are ok with
keeping the 'overlimits' increment, right?

Thanks,

Paolo 

^ permalink raw reply

* Re: [PATCH 1/4] treewide: convert ISO_8859-1 text comments to utf-8
From: Joe Perches @ 2018-07-25 15:33 UTC (permalink / raw)
  To: Arnd Bergmann, Andrew Morton
  Cc: Samuel Ortiz, David S. Miller, Rob Herring, Michael Ellerman,
	Jonathan Cameron, linux-wireless, Networking, DTML,
	Linux Kernel Mailing List, Linux ARM,
	open list:HARDWARE RANDOM NUMBER GENERATOR CORE, linuxppc-dev,
	linux-iio-u79uwXL29TY76Z2rM5mHXA, Linux PM list,
	lvs-devel-u79uwXL29TY76Z2rM5mHXA,
	netfilter-devel-u79uwXL29TY76Z2rM5mHXA,
	coreteam-Cap9r6Oaw4JrovVCs/uTlw
In-Reply-To: <CAK8P3a3tOuP1FVS7oD1UhO5s4C+fLkL8VT3eCpRnSxBZxKzf6A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

On Wed, 2018-07-25 at 15:12 +0200, Arnd Bergmann wrote:
> tools/perf/tests/.gitignore:
>                             LLVM byte-codes, uncompressed
> On Wed, Jul 25, 2018 at 2:55 AM, Andrew Morton
> <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org> wrote:
> > On Tue, 24 Jul 2018 17:13:20 -0700 Joe Perches <joe-6d6DIl74uiNBDgjK7y7TUQ@public.gmane.org> wrote:
> > 
> > > On Tue, 2018-07-24 at 14:00 -0700, Andrew Morton wrote:
> > > > On Tue, 24 Jul 2018 13:13:25 +0200 Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org> wrote:
> > > > > Almost all files in the kernel are either plain text or UTF-8
> > > > > encoded. A couple however are ISO_8859-1, usually just a few
> > > > > characters in a C comments, for historic reasons.
> > > > > This converts them all to UTF-8 for consistency.
> > > 
> > > []
> > > > Will we be getting a checkpatch rule to keep things this way?
> > > 
> > > How would that be done?
> > 
> > I'm using this, seems to work.
> > 
> >         if ! file $p | grep -q -P ", ASCII text|, UTF-8 Unicode text"
> >         then
> >                 echo $p: weird charset
> >         fi
> 
> There are a couple of files that my version of 'find' incorrectly identified as
> something completely different, like:
> 
> Documentation/devicetree/bindings/pinctrl/pinctrl-sx150x.txt:
>             SemOne archive data
> Documentation/devicetree/bindings/rtc/epson,rtc7301.txt:
>             Microsoft Document Imaging Format
> Documentation/filesystems/nfs/pnfs-block-server.txt:
>             PPMN archive data
> arch/arm/boot/dts/bcm283x-rpi-usb-host.dtsi:
>         Sendmail frozen configuration  - version = "host";
> Documentation/networking/segmentation-offloads.txt:
>         StuffIt Deluxe Segment (data) : gmentation Offloads in the
> Linux Networking Stack
> arch/sparc/include/asm/visasm.h:                              SAS 7+
> arch/xtensa/kernel/setup.c:                                         ,
> init=0x454c, stat=0x090a, dev=0x2009, bas=0x2020
> drivers/cpufreq/powernow-k8.c:
> TI-XX Graphing Calculator (FLASH)
> tools/testing/selftests/net/forwarding/tc_shblocks.sh:
>                             Minix filesystem, V2 (big endian)
> tools/perf/tests/.gitignore:
>                             LLVM byte-codes, uncompressed
> 
> All of the above seem to be valid ASCII or UTF-8 files, so the check
> above will lead
> to false-positives, but it may be good enough as they are the
> exception, and may be
> bugs in 'file'.
> 
> Not sure if we need to worry about 'file' not being installed.

checkpatch works on patches so I think the test isn't
really relevant.  It has to use the appropriate email
header that sets the charset.

perhaps:
---
 scripts/checkpatch.pl | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 34e4683de7a3..57355fbd2d28 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -2765,9 +2765,13 @@ sub process {
 # Check if there is UTF-8 in a commit log when a mail header has explicitly
 # declined it, i.e defined some charset where it is missing.
 		if ($in_header_lines &&
-		    $rawline =~ /^Content-Type:.+charset="(.+)".*$/ &&
-		    $1 !~ /utf-8/i) {
-			$non_utf8_charset = 1;
+		    $rawline =~ /^Content-Type:.+charset="?([^\s;"]+)/) {
+			my $charset = $1;
+			$non_utf8_charset = 1 if ($charset !~ /^utf-8$/i);
+			if ($charset !~ /^(?:us-ascii|utf-8|iso-8859-1)$/) {
+				WARN("PATCH_CHARSET",
+				     "Unpreferred email header charset '$charset'\n" . $herecurr);
+			}
 		}
 
 		if ($in_commit_log && $non_utf8_charset && $realfile =~ /^$/ &&

^ permalink raw reply related

* Re: [PATCH 2/4] net: dsa: Add Lantiq / Intel GSWIP tag support
From: Andrew Lunn @ 2018-07-25 14:20 UTC (permalink / raw)
  To: Hauke Mehrtens
  Cc: davem, netdev, vivien.didelot, f.fainelli, john, linux-mips, dev,
	hauke.mehrtens
In-Reply-To: <20180721191358.13952-3-hauke@hauke-m.de>

On Sat, Jul 21, 2018 at 09:13:56PM +0200, Hauke Mehrtens wrote:
> This handles the tag added by the PMAC on the VRX200 SoC line.
> 
> The GSWIP uses internally a GSWIP special tag which is located after the
> Ethernet header. The PMAC which connects the GSWIP to the CPU converts
> this special tag used by the GSWIP into the PMAC special tag which is
> added in front of the Ethernet header.
> 
> This was tested with GSWIP 2.0 found in the VRX200 SoCs, other GSWIP
> versions use slightly different PMAC special tags
> 
> Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>

Hi Hauke

This looks good. A new minor nitpicks below.

> +#include <linux/bitops.h>
> +#include <linux/etherdevice.h>
> +#include <linux/skbuff.h>
> +#include <net/dsa.h>
> +
> +#include "dsa_priv.h"
> +
> +
> +#define GSWIP_TX_HEADER_LEN		4

Single newline is sufficient.

> +/* Byte 3 */
> +#define GSWIP_TX_CRCGEN_DIS		BIT(23)

BIT(23) in a byte is a bit odd.

> +#define GSWIP_TX_SLPID_SHIFT		0	/* source port ID */
> +#define  GSWIP_TX_SLPID_CPU		2
> +#define  GSWIP_TX_SLPID_APP1		3
> +#define  GSWIP_TX_SLPID_APP2		4
> +#define  GSWIP_TX_SLPID_APP3		5
> +#define  GSWIP_TX_SLPID_APP4		6
> +#define  GSWIP_TX_SLPID_APP5		7
> +
> +
> +#define GSWIP_RX_HEADER_LEN	8

Single newline is sufficient. Please fix them all, if there are more
of them.

> +
> +/* special tag in RX path header */
> +/* Byte 7 */
> +#define GSWIP_RX_SPPID_SHIFT		4
> +#define GSWIP_RX_SPPID_MASK		GENMASK(6, 4)
> +
> +static struct sk_buff *gswip_tag_rcv(struct sk_buff *skb,
> +				     struct net_device *dev,
> +				     struct packet_type *pt)
> +{
> +	int port;
> +	u8 *gswip_tag;
> +
> +	if (unlikely(!pskb_may_pull(skb, GSWIP_RX_HEADER_LEN)))
> +		return NULL;
> +
> +	gswip_tag = ((u8 *)skb->data) - ETH_HLEN;

The cast should not be needed, data already is an unsigned char.

> +	skb_pull_rcsum(skb, GSWIP_RX_HEADER_LEN);
> +
> +	/* Get source port information */
> +	port = (gswip_tag[7] & GSWIP_RX_SPPID_MASK) >> GSWIP_RX_SPPID_SHIFT;
> +	skb->dev = dsa_master_find_slave(dev, 0, port);
> +	if (!skb->dev)
> +		return NULL;
> +
> +	return skb;
> +}

  Andrew

^ permalink raw reply

* Re: [PATCH] rds: send: Fix dead code in rds_sendmsg
From: Sowmini Varadhan @ 2018-07-25 15:27 UTC (permalink / raw)
  To: Gustavo A. R. Silva
  Cc: Ka-Cheong Poon, Santosh Shilimkar, David S. Miller, netdev,
	linux-rdma, rds-devel, linux-kernel
In-Reply-To: <20180725152227.GA26303@embeddedor.com>

On (07/25/18 10:22), Gustavo A. R. Silva wrote:
> Currently, code at label *out* is unreachable. Fix this by updating
> variable *ret* with -EINVAL, so the jump to *out* can be properly
> executed instead of directly returning from function.
> 
> Addresses-Coverity-ID: 1472059 ("Structurally dead code")
> Fixes: 1e2b44e78eea ("rds: Enable RDS IPv6 support")
> Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>

Acked-by: Sowmini Varadhan <sowmini.varadhan@oracle.com>

^ permalink raw reply

* [PATCH] rds: send: Fix dead code in rds_sendmsg
From: Gustavo A. R. Silva @ 2018-07-25 15:22 UTC (permalink / raw)
  To: Ka-Cheong Poon, Santosh Shilimkar, David S. Miller
  Cc: netdev, linux-rdma, rds-devel, linux-kernel, Gustavo A. R. Silva

Currently, code at label *out* is unreachable. Fix this by updating
variable *ret* with -EINVAL, so the jump to *out* can be properly
executed instead of directly returning from function.

Addresses-Coverity-ID: 1472059 ("Structurally dead code")
Fixes: 1e2b44e78eea ("rds: Enable RDS IPv6 support")
Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
---
 net/rds/send.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/rds/send.c b/net/rds/send.c
index 9604e1f..18e2b4d 100644
--- a/net/rds/send.c
+++ b/net/rds/send.c
@@ -1126,7 +1126,7 @@ int rds_sendmsg(struct socket *sock, struct msghdr *msg, size_t payload_len)
 				if (addr4 == htonl(INADDR_ANY) ||
 				    addr4 == htonl(INADDR_BROADCAST) ||
 				    IN_MULTICAST(ntohl(addr4))) {
-					return -EINVAL;
+					ret = -EINVAL;
 					goto out;
 				}
 			}
-- 
2.7.4

^ permalink raw reply related

* [PATCH] net: sched: cls_api: fix dead code in switch
From: Gustavo A. R. Silva @ 2018-07-25 14:07 UTC (permalink / raw)
  To: Jiri Pirko, Jamal Hadi Salim, Cong Wang, David S. Miller
  Cc: netdev, linux-kernel, Gustavo A. R. Silva

Code at line 1850 is unreachable. Fix this by removing the break
statement above it, so the code for case RTM_GETCHAIN can be
properly executed.

Addresses-Coverity-ID: 1472050 ("Structurally dead code")
Fixes: 32a4f5ecd738 ("net: sched: introduce chain object to uapi")
Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
---
 net/sched/cls_api.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
index 5f7098b..f3d78c2 100644
--- a/net/sched/cls_api.c
+++ b/net/sched/cls_api.c
@@ -1846,7 +1846,6 @@ static int tc_ctl_chain(struct sk_buff *skb, struct nlmsghdr *n,
 		tcf_chain_put_explicitly_created(chain);
 		break;
 	case RTM_GETCHAIN:
-		break;
 		err = tc_chain_notify(chain, skb, n->nlmsg_seq,
 				      n->nlmsg_seq, n->nlmsg_type, true);
 		if (err < 0)
-- 
2.7.4

^ permalink raw reply related

* Re: [PATCH net-next v3 5/5] act_mirred: use TC_ACT_REINJECT when possible
From: Paolo Abeni @ 2018-07-25 14:04 UTC (permalink / raw)
  To: Jiri Pirko
  Cc: netdev, Jamal Hadi Salim, Cong Wang, Daniel Borkmann,
	Marcelo Ricardo Leitner, Eyal Birger, David S. Miller
In-Reply-To: <20180725135215.GM2164@nanopsycho>

On Wed, 2018-07-25 at 15:52 +0200, Jiri Pirko wrote:
> Tue, Jul 24, 2018 at 10:06:43PM CEST, pabeni@redhat.com wrote:
> > When mirred is invoked from the ingress path, and it wants to redirect
> > the processed packet, it can now use the TC_ACT_REINJECT action,
> > filling the tcf_result accordingly, and avoiding a per packet
> > skb_clone().
> > 
> > Overall this gives a ~10% improvement in forwarding performance for the
> > TC S/W data path and TC S/W performances are now comparable to the
> > kernel openvswitch datapath.
> > 
> > v1 -> v2: use ACT_MIRRED instead of ACT_REDIRECT
> > v2 -> v3: updated after action rename, fixed a typo into the commit
> > 	message
> > 
> > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> > ---
> > net/sched/act_mirred.c | 34 ++++++++++++++++++++++++----------
> > 1 file changed, 24 insertions(+), 10 deletions(-)
> > 
> > diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c
> > index eeb335f03102..368187312136 100644
> > --- a/net/sched/act_mirred.c
> > +++ b/net/sched/act_mirred.c
> > @@ -25,6 +25,7 @@
> > #include <net/net_namespace.h>
> > #include <net/netlink.h>
> > #include <net/pkt_sched.h>
> > +#include <net/pkt_cls.h>
> > #include <linux/tc_act/tc_mirred.h>
> > #include <net/tc_act/tc_mirred.h>
> > 
> > @@ -171,10 +172,12 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
> > 		      struct tcf_result *res)
> > {
> > 	struct tcf_mirred *m = to_mirred(a);
> > +	struct sk_buff *skb2 = skb;
> > 	bool m_mac_header_xmit;
> > 	struct net_device *dev;
> > -	struct sk_buff *skb2;
> > 	int retval, err = 0;
> > +	bool want_ingress;
> > +	bool is_redirect;
> > 	int m_eaction;
> > 	int mac_len;
> > 
> > @@ -196,16 +199,19 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
> > 		goto out;
> > 	}
> > 
> > -	skb2 = skb_clone(skb, GFP_ATOMIC);
> > -	if (!skb2)
> > -		goto out;
> > +	is_redirect = tcf_mirred_is_act_redirect(m_eaction);
> > +	if (!skb_at_tc_ingress(skb) || !is_redirect) {
> > +		skb2 = skb_clone(skb, GFP_ATOMIC);
> > +		if (!skb2)
> > +			goto out;
> > +	}
> > 
> > 	/* If action's target direction differs than filter's direction,
> > 	 * and devices expect a mac header on xmit, then mac push/pull is
> > 	 * needed.
> > 	 */
> > -	if (skb_at_tc_ingress(skb) != tcf_mirred_act_wants_ingress(m_eaction) &&
> > -	    m_mac_header_xmit) {
> > +	want_ingress = tcf_mirred_act_wants_ingress(m_eaction);
> > +	if (skb_at_tc_ingress(skb) != want_ingress && m_mac_header_xmit) {
> > 		if (!skb_at_tc_ingress(skb)) {
> > 			/* caught at egress, act ingress: pull mac */
> > 			mac_len = skb_network_header(skb) - skb_mac_header(skb);
> > @@ -216,15 +222,23 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
> > 		}
> > 	}
> > 
> > +	skb2->skb_iif = skb->dev->ifindex;
> > +	skb2->dev = dev;
> > +
> > 	/* mirror is always swallowed */
> > -	if (tcf_mirred_is_act_redirect(m_eaction)) {
> > +	if (is_redirect) {
> > 		skb2->tc_redirected = 1;
> > 		skb2->tc_from_ingress = skb2->tc_at_ingress;
> > +
> > +		/* let's the caller reinject the packet, if possible */
> > +		if (skb_at_tc_ingress(skb)) {
> 
> I probably missed something. Why only on ingress?

To keep the implementation as simple as possible: if I read correctly,
it is impossible for a filter detect if called by the clsact or the dev
root qdisc, and I think we could safely avoid the skb clone with a not
invasive patch, only if called from the clsact.

[please let me know if the above is somewhat clear ;)]

Also this covers nicely the relevant use case (TC S/W datapath).

Thanks,

Paolo

^ permalink raw reply

* Re: [PATCH net-next v3 4/5] net/tc: introduce TC_ACT_REINJECT.
From: Paolo Abeni @ 2018-07-25 13:55 UTC (permalink / raw)
  To: Jamal Hadi Salim, netdev
  Cc: Cong Wang, Jiri Pirko, Daniel Borkmann, Marcelo Ricardo Leitner,
	Eyal Birger, David S. Miller, Shmulik Ladkani
In-Reply-To: <2bf4fbfd-abe2-7d3b-a4f8-42805b7760c5@mojatatu.com>

Hi,

On Wed, 2018-07-25 at 08:16 -0400, Jamal Hadi Salim wrote:
> +Cc Shmulik
> 
> Paolo - please also run the tdc tests (and add anymore if you
> feel they dont do coverage to your changes)

I run successfully tdc tests on a patched before posting. I plan to
rerun them before posting the v4.

> On 24/07/18 04:06 PM, Paolo Abeni wrote:
> > This is similar TC_ACT_REDIRECT, but with a slightly different
> > semantic:
> > - on ingress the mirred skbs are passed to the target device
> > network stack without any additional check not scrubbing.
> > - the rcu-protected stats provided via the tcf_result struct
> >    are updated on error conditions.
> > 
> > This new tcfa_action value is not exposed to the user-space
> > and can be used only internally by clsact.
> > 
> > v1 -> v2: do not touch TC_ACT_REDIRECT code path, introduce
> >   a new action type instead
> > 
> > v2 -> v3:
> >   - rename the new action value TC_ACT_REINJECT, update the
> >     helper accordingly
> >   - take care of uncloned reinjected packets in XDP generic
> >     hook
> > 
> > Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> > ---
> >   include/net/pkt_cls.h     |  3 +++
> >   include/net/sch_generic.h | 19 +++++++++++++++++++
> >   net/core/dev.c            |  6 +++++-
> >   3 files changed, 27 insertions(+), 1 deletion(-)
> > 
> > diff --git a/include/net/pkt_cls.h b/include/net/pkt_cls.h
> > index 2081e4219f81..36ccfe2a303a 100644
> > --- a/include/net/pkt_cls.h
> > +++ b/include/net/pkt_cls.h
> > @@ -7,6 +7,9 @@
> >   #include <net/sch_generic.h>
> >   #include <net/act_api.h>
> >   
> > +/* TC action not accessible from user space */
> > +#define TC_ACT_REINJECT		(TC_ACT_VALUE_MAX + 1)
> 
> Lets say in the future we add a new opcode.
> Will old kernel, new iproute2 (new value) work?

It will works as it currently does in similar situation: opcode unknown
to the kernel are treated as TC_ACT_UNSPEC even now. 

Cheers,

Paolo

^ permalink raw reply

* Re: [PATCH net-next v3 5/5] act_mirred: use TC_ACT_REINJECT when possible
From: Jiri Pirko @ 2018-07-25 13:52 UTC (permalink / raw)
  To: Paolo Abeni
  Cc: netdev, Jamal Hadi Salim, Cong Wang, Daniel Borkmann,
	Marcelo Ricardo Leitner, Eyal Birger, David S. Miller
In-Reply-To: <612f6fb5b0cdddcbbc2d99f65b148b7f3b8b9a27.1532437050.git.pabeni@redhat.com>

Tue, Jul 24, 2018 at 10:06:43PM CEST, pabeni@redhat.com wrote:
>When mirred is invoked from the ingress path, and it wants to redirect
>the processed packet, it can now use the TC_ACT_REINJECT action,
>filling the tcf_result accordingly, and avoiding a per packet
>skb_clone().
>
>Overall this gives a ~10% improvement in forwarding performance for the
>TC S/W data path and TC S/W performances are now comparable to the
>kernel openvswitch datapath.
>
>v1 -> v2: use ACT_MIRRED instead of ACT_REDIRECT
>v2 -> v3: updated after action rename, fixed a typo into the commit
>	message
>
>Signed-off-by: Paolo Abeni <pabeni@redhat.com>
>---
> net/sched/act_mirred.c | 34 ++++++++++++++++++++++++----------
> 1 file changed, 24 insertions(+), 10 deletions(-)
>
>diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c
>index eeb335f03102..368187312136 100644
>--- a/net/sched/act_mirred.c
>+++ b/net/sched/act_mirred.c
>@@ -25,6 +25,7 @@
> #include <net/net_namespace.h>
> #include <net/netlink.h>
> #include <net/pkt_sched.h>
>+#include <net/pkt_cls.h>
> #include <linux/tc_act/tc_mirred.h>
> #include <net/tc_act/tc_mirred.h>
> 
>@@ -171,10 +172,12 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
> 		      struct tcf_result *res)
> {
> 	struct tcf_mirred *m = to_mirred(a);
>+	struct sk_buff *skb2 = skb;
> 	bool m_mac_header_xmit;
> 	struct net_device *dev;
>-	struct sk_buff *skb2;
> 	int retval, err = 0;
>+	bool want_ingress;
>+	bool is_redirect;
> 	int m_eaction;
> 	int mac_len;
> 
>@@ -196,16 +199,19 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
> 		goto out;
> 	}
> 
>-	skb2 = skb_clone(skb, GFP_ATOMIC);
>-	if (!skb2)
>-		goto out;
>+	is_redirect = tcf_mirred_is_act_redirect(m_eaction);
>+	if (!skb_at_tc_ingress(skb) || !is_redirect) {
>+		skb2 = skb_clone(skb, GFP_ATOMIC);
>+		if (!skb2)
>+			goto out;
>+	}
> 
> 	/* If action's target direction differs than filter's direction,
> 	 * and devices expect a mac header on xmit, then mac push/pull is
> 	 * needed.
> 	 */
>-	if (skb_at_tc_ingress(skb) != tcf_mirred_act_wants_ingress(m_eaction) &&
>-	    m_mac_header_xmit) {
>+	want_ingress = tcf_mirred_act_wants_ingress(m_eaction);
>+	if (skb_at_tc_ingress(skb) != want_ingress && m_mac_header_xmit) {
> 		if (!skb_at_tc_ingress(skb)) {
> 			/* caught at egress, act ingress: pull mac */
> 			mac_len = skb_network_header(skb) - skb_mac_header(skb);
>@@ -216,15 +222,23 @@ static int tcf_mirred(struct sk_buff *skb, const struct tc_action *a,
> 		}
> 	}
> 
>+	skb2->skb_iif = skb->dev->ifindex;
>+	skb2->dev = dev;
>+
> 	/* mirror is always swallowed */
>-	if (tcf_mirred_is_act_redirect(m_eaction)) {
>+	if (is_redirect) {
> 		skb2->tc_redirected = 1;
> 		skb2->tc_from_ingress = skb2->tc_at_ingress;
>+
>+		/* let's the caller reinject the packet, if possible */
>+		if (skb_at_tc_ingress(skb)) {

I probably missed something. Why only on ingress?

The patch looks good to me.


>+			res->ingress = want_ingress;
>+			res->qstats = this_cpu_ptr(m->common.cpu_qstats);
>+			return TC_ACT_REINJECT;
>+		}
> 	}
> 
>-	skb2->skb_iif = skb->dev->ifindex;
>-	skb2->dev = dev;
>-	if (!tcf_mirred_act_wants_ingress(m_eaction))
>+	if (!want_ingress)
> 		err = dev_queue_xmit(skb2);
> 	else
> 		err = netif_receive_skb(skb2);
>-- 
>2.17.1
>

^ permalink raw reply

* [PATCH net] cxgb4: Added missing break in ndo_udp_tunnel_{add/del}
From: Arjun Vynipadath @ 2018-07-25 14:09 UTC (permalink / raw)
  To: netdev, davem; +Cc: nirranjan, indranil, Arjun Vynipadath, Ganesh Goudar

Break statements were missing for Geneve case in
ndo_udp_tunnel_{add/del}, thereby raw mac matchall
entries were not getting added.

Fixes: c746fc0e8b2d("cxgb4: add geneve offload support for T6")
Signed-off-by: Arjun Vynipadath <arjun@chelsio.com>
Signed-off-by: Ganesh Goudar <ganeshgr@chelsio.com>
---
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
index 40cf8dc..a46a010 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
@@ -3074,6 +3074,7 @@ static void cxgb_del_udp_tunnel(struct net_device *netdev,
 
 		adapter->geneve_port = 0;
 		t4_write_reg(adapter, MPS_RX_GENEVE_TYPE_A, 0);
+		break;
 	default:
 		return;
 	}
@@ -3159,6 +3160,7 @@ static void cxgb_add_udp_tunnel(struct net_device *netdev,
 
 		t4_write_reg(adapter, MPS_RX_GENEVE_TYPE_A,
 			     GENEVE_V(be16_to_cpu(ti->port)) | GENEVE_EN_F);
+		break;
 	default:
 		return;
 	}
-- 
2.3.5

^ permalink raw reply related

* 袁谷歌“2018年6月”获奖
From:  Luis  Bolaños Esquivel @ 2018-07-25 13:01 UTC (permalink / raw)


[-- Attachment #1: Type: text/plain, Size: 2663 bytes --]

从桌子上:约翰·卢博斯先生付款中心服务台订户海外代理EUR Millions彩票推广区域办事处。伦敦市老格洛斯特街101号。
获奖号码:GUK / 877/798/2018票号:GUK / 699/33/2018
 袁谷歌“2018年6月”获奖
我们希望在此笔记中向您表示祝贺,因为我们在本月刚刚结束的内部促销活动中成为我们选定的获奖者的一部分,此次促销活动的目的是鼓励活跃的用户
谷歌搜索引擎和谷歌的辅助服务。
因此,我们相信您的获奖奖品,您将继续积极参与Google搜索引擎和服务。谷歌现在是全球最大的搜索引擎
为了确保它仍然是最广泛使用的搜索引擎,我们运行了一个在线电子邮件测试版,您的电子邮件地址赢得了六十五万元
(¥650,000.00)。我们希望正式通知您,您已成功通过了要求,法定义务,验证,验证和令人满意的报告测试
所有在线获奖者。
获奖支票将以Google推广奖的名义以您的名义发行,总额为六十五万元(¥650,000.00),还有一张奖品
索赔将与您在任何银行兑现的获奖支票一起发送。
建议您与指定的Google计划管理员/协调员联系,并提供以下详细信息,以避免不必要的延迟和复杂情况:
验证和资金发布表格
(1)您的联系地址/私人电子邮件地址(2)您的电话/传真号码(3)您的国籍/国家(4)你的全名(5)职业/公司(6)年龄/性别(7)曾经赢过在线彩票吗?(8)关于谷歌的评论
Jeffrey Dean - 谷歌高级研究员(项目管理员/协调员)电子邮件:johnlubos042@gmail.com联系电话:447451245513
Google重视您的隐私权!您的信息是100%安全的,仅用于此奖励的目的。
由于获奖者通知亲密朋友和第三方他们的获胜以及分享他们的针脚,Google推广奖励团队发现了大量双重索赔
数字。结果,这些朋友试图代表真正的赢家宣布彩票。 Google推广奖励小组已经从其总部做出任何双重决定
彩票委员会发现的声明将导致取消该特定的胜利,导致双重请求者和真正的赢家失败,因为它被认为是真正的赢家
是关于彩票的双重请求者的告密者。因此,强烈建议您在获得奖品之前,确保您的奖金严格保密。
Google互动彩票委员会的工作人员和成员恭喜您。
您忠诚的,
桑达皮采谷歌首席执行官

 请不要回复这封邮件
如果您对自己的陈述有任何疑问或抱怨您的中标价格 请点击发送电子邮件至johnlubos042@gmail.com与会员代言人联系,或致电+447451245513。

[-- Attachment #2: Type: text/html, Size: 108168 bytes --]

^ permalink raw reply

* 袁谷歌“2018年6月”获奖
From:  Luis  Bolaños Esquivel @ 2018-07-25 13:03 UTC (permalink / raw)


[-- Attachment #1: Type: text/plain, Size: 2663 bytes --]

从桌子上:约翰·卢博斯先生付款中心服务台订户海外代理EUR Millions彩票推广区域办事处。伦敦市老格洛斯特街101号。
获奖号码:GUK / 877/798/2018票号:GUK / 699/33/2018
 袁谷歌“2018年6月”获奖
我们希望在此笔记中向您表示祝贺,因为我们在本月刚刚结束的内部促销活动中成为我们选定的获奖者的一部分,此次促销活动的目的是鼓励活跃的用户
谷歌搜索引擎和谷歌的辅助服务。
因此,我们相信您的获奖奖品,您将继续积极参与Google搜索引擎和服务。谷歌现在是全球最大的搜索引擎
为了确保它仍然是最广泛使用的搜索引擎,我们运行了一个在线电子邮件测试版,您的电子邮件地址赢得了六十五万元
(¥650,000.00)。我们希望正式通知您,您已成功通过了要求,法定义务,验证,验证和令人满意的报告测试
所有在线获奖者。
获奖支票将以Google推广奖的名义以您的名义发行,总额为六十五万元(¥650,000.00),还有一张奖品
索赔将与您在任何银行兑现的获奖支票一起发送。
建议您与指定的Google计划管理员/协调员联系,并提供以下详细信息,以避免不必要的延迟和复杂情况:
验证和资金发布表格
(1)您的联系地址/私人电子邮件地址(2)您的电话/传真号码(3)您的国籍/国家(4)你的全名(5)职业/公司(6)年龄/性别(7)曾经赢过在线彩票吗?(8)关于谷歌的评论
Jeffrey Dean - 谷歌高级研究员(项目管理员/协调员)电子邮件:johnlubos042@gmail.com联系电话:447451245513
Google重视您的隐私权!您的信息是100%安全的,仅用于此奖励的目的。
由于获奖者通知亲密朋友和第三方他们的获胜以及分享他们的针脚,Google推广奖励团队发现了大量双重索赔
数字。结果,这些朋友试图代表真正的赢家宣布彩票。 Google推广奖励小组已经从其总部做出任何双重决定
彩票委员会发现的声明将导致取消该特定的胜利,导致双重请求者和真正的赢家失败,因为它被认为是真正的赢家
是关于彩票的双重请求者的告密者。因此,强烈建议您在获得奖品之前,确保您的奖金严格保密。
Google互动彩票委员会的工作人员和成员恭喜您。
您忠诚的,
桑达皮采谷歌首席执行官

 请不要回复这封邮件
如果您对自己的陈述有任何疑问或抱怨您的中标价格 请点击发送电子邮件至johnlubos042@gmail.com与会员代言人联系,或致电+447451245513。

[-- Attachment #2: Type: text/html, Size: 108168 bytes --]

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox