Re: [PATCH for-rc] IB/cma: Fix false P_Key mismatch messages

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, May 10, 2021 at 06:52:54PM +0000, Haakon Bugge wrote:
> 
> 
> > On 10 May 2021, at 19:04, Jason Gunthorpe <jgg@nvidia.com> wrote:
> > 
> > On Wed, May 05, 2021 at 02:54:01PM +0200, Håkon Bugge wrote:
> >> There are three conditions that must be fulfilled in order to consider
> >> a partition match. Those are:
> >> 
> >>      1. Both P_Keys must valid
> >>      2. At least one must be a full member
> >>      3. The partitions (lower 15 bits) must match
> >> 
> >> In system employing both limited and full membership ports, we see
> >> these false warning messages:
> >> 
> >> RDMA CMA: got different BTH P_Key (0x2a00) and primary path P_Key (0xaa00)
> >> RDMA CMA: in the future this may cause the request to be dropped
> >> 
> >> even though the partition is the same.
> >> 
> >> See IBTA 10.9.1.2 Special P_Keys and 10.9.3 Partition Key Matching for
> >> a reference.
> >> 
> >> Fixes: 84424a7fc793 ("IB/cma: Print warning on different inner and header P_Keys")
> >> Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
> >> drivers/infiniband/core/cma.c | 22 ++++++++++++++++++++--
> >> 1 file changed, 20 insertions(+), 2 deletions(-)
> > 
> > What is this trying to fix?
> 
> The false warning messages. The wrong way though:-)
> 
> > IMHO it is a bug on the sender side to send GMPs to use a pkey that
> > doesn't exactly match the data path pkey.
> 
> The active connector calls ib_addr_get_pkey(). This function
> extracts the pkey from byte 8/9 in the device's bcast
> address. However, RFC 4391 explicitly states:

pkeys in CM come only from path records that the SM returns, the above
should only be used to feed into a path record query which could then
return back a limited pkey.

Everything thereafter should use the SM's version of the pkey.

Jason