RE: [PATCH v2 1/1] IPoIB: Improve parent interface p_key handling.

Estrin, Alex
2014-06-10T11:32:20Z

> -----Original Message-----
> From: Or Gerlitz [mailto:ogerlitz@mellanox.com]
> Sent: Tuesday, June 10, 2014 2:17 AM
> To: Estrin, Alex
> Cc: Roland Dreier; linux-rdma@vger.kernel.org
> Subject: Re: [PATCH v2 1/1] IPoIB: Improve parent interface p_key handling.
>
> On 10/06/2014 00:55, Alex Estrin wrote:
> > With reference to commit c2904141696ee19551f1553944446f23cdd5d95e.
> > It was noticed that parent interface keeps sending broadcast group
> > join requests if p_key index 0 is invalid. That creates unnecessary
> > noise on a fabric:
> >
> > ib0: multicast join failed for ff12:401b:8000:0000:0000:0000:ffff:ffff,
> > status -22
> >
> > Proposed patch re-init resources, then brings interface
> > up only if p_key idx 0 is valid either on bootup or on PKEY_CHANGE event.
> > Original code could run multicast task regardless of p_key value,
> > which we want to avoid.
> >
> > Modified event handler will utilize following strategy:
> > if interface is not initialized and event is not PKEY_CHANGE related - return.
> > call update_parent_pkey() -> if pkey hasn't changed - return.
> > if interface is initialized
> > flush it -> call ipoib_ib_dev_stop() - de-initialized.
> > Then start multicast task only if ipoib_ib_dev_open() has succeeded, reinitialized,
> > i.e. p_key is valid.
> >
> > Changes from v1:
> > p_key check for 'Invalid' value was moved to
> > ipoib_pkey_dev_check_presence() that is used now in ipoib_ib_dev_open()
> > for p_key validation.
> > Whitespace and format adjusted.
>
> move the last section of changes from prev version/s to be below the ---
> line that following so it doesn't get into the actual change-log
>
Yep, my bad. Will do.
Thanks.

>
> add here
>
> Fixes: c29041416 ('IPoIB: Fix pkey change flow for virtualization
> environments ')
> > Reviewed-by: Ira Weiny <ira.weiny@intel.com>
> > Signed-off-by: Alex Estrin <alex.estrin@intel.com>
> > ---
> > drivers/infiniband/ulp/ipoib/ipoib_ib.c | 31 +++++++++++++++++--------------
> > 1 files changed, 17 insertions(+), 14 deletions(-)
> >
> > diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> > index 6a7003d..627f74f 100644
> > --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> > +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> > @@ -52,6 +52,7 @@ MODULE_PARM_DESC(data_debug_level,
> > #endif
> >
> > static DEFINE_MUTEX(pkey_mutex);
> > +static void ipoib_pkey_dev_check_presence(struct net_device *dev);
> >
> > struct ipoib_ah *ipoib_create_ah(struct net_device *dev,
> > struct ib_pd *pd, struct ib_ah_attr *attr)
> > @@ -669,12 +670,13 @@ int ipoib_ib_dev_open(struct net_device *dev)
> > struct ipoib_dev_priv *priv = netdev_priv(dev);
> > int ret;
> >
> > - if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &priv->pkey_index)) {
> > - ipoib_warn(priv, "P_Key 0x%04x not found\n", priv->pkey);
> > - clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags);
> > + ipoib_pkey_dev_check_presence(dev);
> > +
> > + if (!test_bit(IPOIB_PKEY_ASSIGNED, &priv->flags)) {
> > + ipoib_warn(priv, "P_Key 0x%04x is %s\n", priv->pkey,
> > + !(priv->pkey & 0x7fff) ? "Invalid" : "not found");
> > return -1;
> > }
> > - set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags);
> >
> > ret = ipoib_init_qp(dev);
> > if (ret) {
> > @@ -712,9 +714,10 @@ dev_stop:
> > static void ipoib_pkey_dev_check_presence(struct net_device *dev)
> > {
> > struct ipoib_dev_priv *priv = netdev_priv(dev);
> > - u16 pkey_index = 0;
> >
> > - if (ib_find_pkey(priv->ca, priv->port, priv->pkey, &pkey_index))
> > + if (!(priv->pkey & 0x7fff) ||
> > + ib_find_pkey(priv->ca, priv->port, priv->pkey,
> > + &priv->pkey_index))
> > clear_bit(IPOIB_PKEY_ASSIGNED, &priv->flags);
> > else
> > set_bit(IPOIB_PKEY_ASSIGNED, &priv->flags);
> > @@ -987,12 +990,10 @@ static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv,
> > up_read(&priv->vlan_rwsem);
> >
> > if (!test_bit(IPOIB_FLAG_INITIALIZED, &priv->flags)) {
> > - /* for non-child devices must check/update the pkey value here */
> > - if (level == IPOIB_FLUSH_HEAVY &&
> > - !test_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags))
> > - update_parent_pkey(priv);
> > - ipoib_dbg(priv, "Not flushing - IPOIB_FLAG_INITIALIZED not set.\n");
> > - return;
> > + if (level < IPOIB_FLUSH_HEAVY) {
> > + ipoib_dbg(priv, "Not flushing - IPOIB_FLAG_INITIALIZED not
> set.\n");
> > + return;
> > + }
> > }
> >
> > if (!test_bit(IPOIB_FLAG_ADMIN_UP, &priv->flags)) {
> > @@ -1038,8 +1039,10 @@ static void __ipoib_ib_dev_flush(struct ipoib_dev_priv *priv,
> > ipoib_ib_dev_down(dev, 0);
> >
> > if (level == IPOIB_FLUSH_HEAVY) {
> > - ipoib_ib_dev_stop(dev, 0);
> > - ipoib_ib_dev_open(dev);
> > + if (test_bit(IPOIB_FLAG_INITIALIZED, &priv->flags))
> > + ipoib_ib_dev_stop(dev, 0);
> > + if (ipoib_ib_dev_open(dev) != 0)
> > + return;
> > }
> >
> > /*
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html