From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28D26C43461 for ; Mon, 7 Sep 2020 12:28:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C9D02206E6 for ; Mon, 7 Sep 2020 12:28:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1599481734; bh=aKuSZkFyViitvRAak1e5tHhoYFOMoWJp3/zODVGrRQI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=grAbPjDwCriPL7sn2MrfF/hVqHLFS1aPGMavhY9DHoAmrGy2/UV5Z5PQaaMzlK/4w gPsf3YArH/Fy6hKQ7ZQXxibG5wmfQOdwIoOUwC0CHGT+B5vCqRNECcAfnLH2PRrQOF R28H6pSfXmRjFb767MDJCjuDip7IV1CcYeK9PnPE= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728824AbgIGM1s (ORCPT ); Mon, 7 Sep 2020 08:27:48 -0400 Received: from mail.kernel.org ([198.145.29.99]:41448 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729277AbgIGMWd (ORCPT ); Mon, 7 Sep 2020 08:22:33 -0400 Received: from localhost (unknown [213.57.247.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2EBAB2137B; Mon, 7 Sep 2020 12:22:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1599481352; bh=aKuSZkFyViitvRAak1e5tHhoYFOMoWJp3/zODVGrRQI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pmCwQSkbMqTTYfd23DxoFWi3V1CXUM5ew8kKFV606WZdbjwRzHJH94sKZU0xHSVnw LuuDCpsjAWFVpn4CnP+CIztnWX79+Wfjvrf2hTpUyRittDzniO9/noaR3QSuMUGcDR B9uSuzqcBoIhpkIekBbPYyiIVDN7rdbAb5AV+xSk= From: Leon Romanovsky To: Doug Ledford , Jason Gunthorpe Cc: Leon Romanovsky , linux-rdma@vger.kernel.org Subject: [PATCH rdma-next v2 10/14] RDMA/restrack: Store all special QPs in restrack DB Date: Mon, 7 Sep 2020 15:21:52 +0300 Message-Id: <20200907122156.478360-11-leon@kernel.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200907122156.478360-1-leon@kernel.org> References: <20200907122156.478360-1-leon@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Leon Romanovsky Special QPs (SMI and GSI) have different rules in regards of their QP numbers. While all other QP numbers are unique per-device, the QP0 and QP1 are created per-port as requested by IBTA. In multiple port devices, the number of SMI and GSI QPs with be equal to the number ports. [leonro@vm ~]$ rdma dev 0: ibp0s9: node_type ca fw 4.4.9999 node_guid 5254:00c0:fe12:3455 sys_image_guid 5254:00c0:fe12:3455 [leonro@vm ~]$ rdma link 0/1: ibp0s9/1: subnet_prefix fe80:0000:0000:0000 lid 13397 sm_lid 49151 lmc 0 state ACTIVE physical_state LINK_UP 0/2: ibp0s9/2: subnet_prefix fe80:0000:0000:0000 lid 13397 sm_lid 49151 lmc 0 state UNKNOWN physical_state UNKNOWN Before: [leonro@mtl-leonro-l-vm ~]$ rdma res show qp type SMI,GSI link ibp0s9/1 lqpn 0 type SMI state RTS sq-psn 0 comm [ib_core] link ibp0s9/1 lqpn 1 type GSI state RTS sq-psn 0 comm [ib_core] After: [leonro@vm ~]$ rdma res show qp type SMI,GSI link ibp0s9/1 lqpn 0 type SMI state RTS sq-psn 0 comm [ib_core] link ibp0s9/1 lqpn 1 type GSI state RTS sq-psn 0 comm [ib_core] link ibp0s9/2 lqpn 0 type SMI state RTS sq-psn 0 comm [ib_core] link ibp0s9/2 lqpn 1 type GSI state RTS sq-psn 0 comm [ib_core] Signed-off-by: Leon Romanovsky --- drivers/infiniband/core/core_priv.h | 2 ++ drivers/infiniband/core/restrack.c | 11 +++++++++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h index e84b0fedaacb..7c4752c47f80 100644 --- a/drivers/infiniband/core/core_priv.h +++ b/drivers/infiniband/core/core_priv.h @@ -347,6 +347,8 @@ static inline struct ib_qp *_ib_create_qp(struct ib_device *dev, qp->srq = attr->srq; qp->rwq_ind_tbl = attr->rwq_ind_tbl; qp->event_handler = attr->event_handler; + qp->qp_type = attr->qp_type; + qp->port = attr->port_num; atomic_set(&qp->usecnt, 0); spin_lock_init(&qp->mr_lock); diff --git a/drivers/infiniband/core/restrack.c b/drivers/infiniband/core/restrack.c index 05ea7c61ae0b..2cc8d63e93ae 100644 --- a/drivers/infiniband/core/restrack.c +++ b/drivers/infiniband/core/restrack.c @@ -235,8 +235,15 @@ void rdma_restrack_add(struct rdma_restrack_entry *res) /* Special case to ensure that LQPN points to right QP */ struct ib_qp *qp = container_of(res, struct ib_qp, res); - ret = xa_insert(&rt->xa, qp->qp_num, res, GFP_KERNEL); - res->id = ret ? 0 : qp->qp_num; + WARN_ONCE(qp->qp_num >> 24 || qp->port >> 8, + "QP number 0x%0X and port 0x%0X", qp->qp_num, + qp->port); + res->id = qp->qp_num; + if (qp->qp_type == IB_QPT_SMI || qp->qp_type == IB_QPT_GSI) + res->id |= qp->port << 24; + ret = xa_insert(&rt->xa, res->id, res, GFP_KERNEL); + if (ret) + res->id = 0; } else if (res->type == RDMA_RESTRACK_COUNTER) { /* Special case to ensure that cntn points to right counter */ struct rdma_counter *counter; -- 2.26.2