Re: [PATCH for-next V2 02/22] IB/core: change pkey table lookups to support full and partial membership for the same pkey

[Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 4298 bytes --]

On 8/3/2012 4:40 AM, Jack Morgenstein wrote:
> Enhance the cached and non-cached pkey table lookups to enable limited and full
> members of the same pkey to co-exist in the pkey table.
> 
> This is necessary for SRIOV to allow for a scheme where some guests would have the full
> membership pkey in their virtual pkey table, where other guests on the same hypervisor
> would have the limited one. In that sense, its an extension of the IBTA model for
> non virtualized nodes.

OK, maybe I'm not getting something, but I'm curious why we always pick
the full pkey in preference to the partial pkey.  Shouldn't we pick the
pkey that's appropriate for the vHCA sending the message?

Also, given the rule of least surprise, don't you think it would be best
to rename this function ib_find_cached_full_or_parital_pkey and in your
next patch instead of naming it ib_find_exact_pkey just call that one
ib_find_cached_pkey?

> To accomplish this, we need both the limited and full membership pkeys to be present
> in the master's (hypervisor physical port) pkey table.
> 
> The algorithm for supporting pkey tables which contain both the limited and the full
> membership versions of the same pkey works as follows:
> 
> When scanning the pkey table for a 15 bit pkey:
> 
> A. If there is a full member version of that pkey anywhere
>     in the table, return its index (even if a limited-member
>     version of the pkey exists earlier in the table).
> 
> B. If the full member version is not in the table,
>     but the limited-member version is in the table,
>     return the index of the limited pkey.
> 
> Signed-off-by: Liran Liss <liranl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Signed-off-by: Jack Morgenstein <jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
> Signed-off-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
>  drivers/infiniband/core/cache.c  |   14 +++++++++++---
>  drivers/infiniband/core/device.c |   17 +++++++++++++----
>  2 files changed, 24 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
> index 9353992..0f2f2b7 100644
> --- a/drivers/infiniband/core/cache.c
> +++ b/drivers/infiniband/core/cache.c
> @@ -167,6 +167,7 @@ int ib_find_cached_pkey(struct ib_device *device,
>  	unsigned long flags;
>  	int i;
>  	int ret = -ENOENT;
> +	int partial_ix = -1;
>  
>  	if (port_num < start_port(device) || port_num > end_port(device))
>  		return -EINVAL;
> @@ -179,10 +180,17 @@ int ib_find_cached_pkey(struct ib_device *device,
>  
>  	for (i = 0; i < cache->table_len; ++i)
>  		if ((cache->table[i] & 0x7fff) == (pkey & 0x7fff)) {
> -			*index = i;
> -			ret = 0;
> -			break;
> +			if (cache->table[i] & 0x8000) {
> +				*index = i;
> +				ret = 0;
> +				break;
> +			} else
> +				partial_ix = i;
>  		}
> +	if (ret && partial_ix >= 0) {
> +		*index = partial_ix;
> +		ret = 0;
> +	}
>  
>  	read_unlock_irqrestore(&device->cache.lock, flags);
>  
> diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
> index e711de4..a645c68 100644
> --- a/drivers/infiniband/core/device.c
> +++ b/drivers/infiniband/core/device.c
> @@ -707,18 +707,27 @@ int ib_find_pkey(struct ib_device *device,
>  {
>  	int ret, i;
>  	u16 tmp_pkey;
> +	int partial_ix = -1;
>  
>  	for (i = 0; i < device->pkey_tbl_len[port_num - start_port(device)]; ++i) {
>  		ret = ib_query_pkey(device, port_num, i, &tmp_pkey);
>  		if (ret)
>  			return ret;
> -
>  		if ((pkey & 0x7fff) == (tmp_pkey & 0x7fff)) {
> -			*index = i;
> -			return 0;
> +			/* if there is full-member pkey take it.*/
> +			if (tmp_pkey & 0x8000) {
> +				*index = i;
> +				return 0;
> +			}
> +			if (partial_ix < 0)
> +				partial_ix = i;
>  		}
>  	}
> -
> +	/*no full-member, if exists take the limited*/
> +	if (partial_ix >= 0) {
> +		*index = partial_ix;
> +		return 0;
> +	}
>  	return -ENOENT;
>  }
>  EXPORT_SYMBOL(ib_find_pkey);
> 


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD
	      http://people.redhat.com/dledford

Infiniband specific RPMs available at
	      http://people.redhat.com/dledford/Infiniband


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 895 bytes --]