From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-vk1-f227.google.com (mail-vk1-f227.google.com [209.85.221.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67D0729E110 for ; Fri, 29 Aug 2025 12:31:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.227 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756470685; cv=none; b=iaPb72do6SicY+6CJ+Z5rkEmRyfuSGlvh7BB31FV31y2o4VZAg1Vlo1zXBJ9hTuALGNCHIuAMtDNYjEkcQOwzWgcUOaq3T7JEOqM36EQStFlHo6SFmPyvp7jhGsT2F33l0KzrqaSlpvUdqYnkMzilrnboWauI55mV8VyE4XcBTA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756470685; c=relaxed/simple; bh=QQ5/d/ctfKXMji9r0IFstTlSPne+z4grdxzK8BCUkL8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KJOsgmw933jjPQ4zeyYEClTVN3wk2cLDr1xs8r1BvUY57N9XCktEN1qz9UeSUMtRLyXRnc7EZI+xrSbZE5EpHqDZFXN22PLFjUfxp0gLg4wACtXJ0J/bBm7jCr01MCX8DIUoPPInZ6qELAbQJzWXIPhh9RrxHmV7C/VRdX04aW0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=NFRhnKmy; arc=none smtp.client-ip=209.85.221.227 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="NFRhnKmy" Received: by mail-vk1-f227.google.com with SMTP id 71dfb90a1353d-5449432a9d7so158232e0c.3 for ; Fri, 29 Aug 2025 05:31:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756470681; x=1757075481; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6jNDerxCWY7R0RDSnHboaIKwbrYOEqJkI+IEOXiZnEQ=; b=vUfbfLNM7hH9AgtyAZMXI3ww8S6LKKdEDlUmlx0dJHwZRncHsk1jsxi4AgYwaaJo51 UpcNjxwI50Vcb1yaelNVdpBz6+B1K3BiCzCho5Ct0xvAc9hAJnuMFlwpDGit5TNOMJKJ SdPIOM117skdSfJq2VW2tT+eleCKEHuYNWyljrka6clUWu8NqWgpCnuN6glnlXRzqFN9 1riFWrxs8SJo2N+O2mbcjDqMkAdQ9zd43fSCqqmgF/JBs5AB1jiOc6g9qwS5gOW5h2uC f8Z3LbdNQTHQSkypX0kfA1bZchATCNSVR4drmT3LqgrggQMMH56ScMG98vmLqWWTqVRX jcTw== X-Gm-Message-State: AOJu0Yw51uPOSh13ese+UKRNGDa1NNff8GIl6fXgNtt4866y5UJv0XsZ qtYhgDD2zWtF1v4szsonC74x2P5oTAAlqTMIIYEURlS61y6/XWUZFWAPcNJTXgDVeoGFQpm+HL6 1+nobEXOWPjENNQ+L6eD1zWnshSQGRRYUXF35PevMHSSNEB9VChkyjs0nWLtjJi5RB99496mVPD 5Ss37SjcNOO60t/El/1QQ9kM5wsRwPLtxA6SEIGiAmUphS4+okQ3bxXBNBQ5ND3Mir0y1ZEZMN+ UUdO6cnQoIxVdkStw== X-Gm-Gg: ASbGnct40WowbeFy2GERPeCBSQD50G1AQoui+yaduRx7Kh1w0z9qT/5+2AvuH4hmr3e DgZyIqlYoBSLvrn0YnjkgR8mJ6IvYCducbuy4tkakrcuGUGAodjDCQuwD6yPnAzSBcGVHx5Msvd Cg1gX1dbPaga9jCEbR1IL49vPuav0J43XChfUII1yHPTo+OuA8XQnpAQwS0iX/s/Z9CcZ2bpGUH j31dUK+GlkJesvytRlvLObtS0LIgYcfxEau64kHnOyRYUJ4kbhOYJ5mXFgYx3iMr57otOQ4TlmT aODy2O4z9F3LiCbnx+/TNFMdLe5sYl7U6XKuFMMn+ykcsxErV+BQfJA/v6Vr8n2wQPzkCUGOcov 1kn751Pn+x+cxn3cZJt+tVRNlJh7FO5E0D74Lpx+NYK2Y4iAWy1EBj0TokqFZtjLCosLJH7da4Q == X-Google-Smtp-Source: AGHT+IEitvha25t/GeqwLwgP3TIL3nFExtn3u7MUtvP3nbwpnDNG08Xzo/wxrSmkverhg8TfG6nLLNZd+Iez X-Received: by 2002:a05:6122:8285:b0:53c:6d68:1cd5 with SMTP id 71dfb90a1353d-53c8a45b6e8mr9011283e0c.15.1756470681081; Fri, 29 Aug 2025 05:31:21 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com (address-144-49-247-16.dlp.protect.broadcom.com. [144.49.247.16]) by smtp-relay.gmail.com with ESMTPS id 71dfb90a1353d-544914976c6sm146583e0c.8.2025.08.29.05.31.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Aug 2025 05:31:21 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-qk1-f197.google.com with SMTP id af79cd13be357-7e870646b11so466830685a.2 for ; Fri, 29 Aug 2025 05:31:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1756470680; x=1757075480; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6jNDerxCWY7R0RDSnHboaIKwbrYOEqJkI+IEOXiZnEQ=; b=NFRhnKmyhXu7vN5mRormeBiFxiQP+Nx0bu/IZKCZxHQtvvVk+/m62ztdd8/A2zMpaB M3wGr3jBB8YoRU5R85/0S2XcvbvbHGuBZeWNDYh5pLxYvjHdB13sXNng5LRSq/Vl/5De Ln1vXo3lG0jPXejVI+g8QNUff/vpZF2MwN/hc= X-Received: by 2002:a05:620a:4091:b0:7e8:44ac:8136 with SMTP id af79cd13be357-7ea10f86c69mr2947252585a.2.1756470679684; Fri, 29 Aug 2025 05:31:19 -0700 (PDT) X-Received: by 2002:a05:620a:4091:b0:7e8:44ac:8136 with SMTP id af79cd13be357-7ea10f86c69mr2947248985a.2.1756470679131; Fri, 29 Aug 2025 05:31:19 -0700 (PDT) Received: from sjs-csg-thor3-swe-29.lvn.broadcom.net044broadcom.net ([192.19.224.250]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7fc16536012sm162384585a.66.2025.08.29.05.31.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Aug 2025 05:31:18 -0700 (PDT) From: Siva Reddy Kallam To: leonro@nvidia.com, jgg@nvidia.com Cc: linux-rdma@vger.kernel.org, netdev@vger.kernel.org, vikas.gupta@broadcom.com, selvin.xavier@broadcom.com, anand.subramanian@broadcom.com, Siva Reddy Kallam , Usman Ansari Subject: [PATCH 3/8] RDMA/bng_re: Register and get the resources from bnge driver Date: Fri, 29 Aug 2025 12:30:37 +0000 Message-Id: <20250829123042.44459-4-siva.kallam@broadcom.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250829123042.44459-1-siva.kallam@broadcom.com> References: <20250829123042.44459-1-siva.kallam@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e Register and get the basic required resources from bnge driver. Signed-off-by: Siva Reddy Kallam Reviewed-by: Usman Ansari --- drivers/infiniband/hw/bng_re/bng_dev.c | 149 +++++++++++++++++++++++++ drivers/infiniband/hw/bng_re/bng_re.h | 5 + drivers/infiniband/hw/bng_re/bng_res.h | 16 +++ 3 files changed, 170 insertions(+) create mode 100644 drivers/infiniband/hw/bng_re/bng_res.h diff --git a/drivers/infiniband/hw/bng_re/bng_dev.c b/drivers/infiniband/hw/bng_re/bng_dev.c index 208844e98bd6..a9196b765a58 100644 --- a/drivers/infiniband/hw/bng_re/bng_dev.c +++ b/drivers/infiniband/hw/bng_re/bng_dev.c @@ -7,8 +7,10 @@ #include +#include "bng_res.h" #include "bng_re.h" #include "bnge.h" +#include "bnge_hwrm.h" #include "bnge_auxr.h" static char version[] = @@ -40,6 +42,144 @@ static struct bng_re_dev *bng_re_dev_add(struct auxiliary_device *adev, return rdev; } + +static int bng_re_register_netdev(struct bng_re_dev *rdev) +{ + struct bnge_auxr_dev *aux_dev; + + aux_dev = rdev->aux_dev; + return bnge_register_dev(aux_dev, rdev->adev); +} + +static void bng_re_destroy_chip_ctx(struct bng_re_dev *rdev) +{ + struct bng_re_chip_ctx *chip_ctx; + + if (!rdev->chip_ctx) + return; + + chip_ctx = rdev->chip_ctx; + rdev->chip_ctx = NULL; + kfree(chip_ctx); +} + +static int bng_re_setup_chip_ctx(struct bng_re_dev *rdev) +{ + struct bng_re_chip_ctx *chip_ctx; + struct bnge_auxr_dev *aux_dev; + + aux_dev = rdev->aux_dev; + + chip_ctx = kzalloc(sizeof(*chip_ctx), GFP_KERNEL); + if (!chip_ctx) + return -ENOMEM; + chip_ctx->chip_num = aux_dev->chip_num; + chip_ctx->hw_stats_size = aux_dev->hw_ring_stats_size; + + rdev->chip_ctx = chip_ctx; + + return 0; +} + +static void bng_re_init_hwrm_hdr(struct input *hdr, u16 opcd) +{ + hdr->req_type = cpu_to_le16(opcd); + hdr->cmpl_ring = cpu_to_le16(-1); + hdr->target_id = cpu_to_le16(-1); +} + +static void bng_re_fill_fw_msg(struct bnge_fw_msg *fw_msg, void *msg, + int msg_len, void *resp, int resp_max_len, + int timeout) +{ + fw_msg->msg = msg; + fw_msg->msg_len = msg_len; + fw_msg->resp = resp; + fw_msg->resp_max_len = resp_max_len; + fw_msg->timeout = timeout; +} + +static void bng_re_query_hwrm_version(struct bng_re_dev *rdev) +{ + struct bnge_auxr_dev *aux_dev = rdev->aux_dev; + struct hwrm_ver_get_output ver_get_resp = {}; + struct hwrm_ver_get_input ver_get_req = {}; + struct bng_re_chip_ctx *cctx; + struct bnge_fw_msg fw_msg = {}; + int rc; + + bng_re_init_hwrm_hdr((void *)&ver_get_req, HWRM_VER_GET); + ver_get_req.hwrm_intf_maj = HWRM_VERSION_MAJOR; + ver_get_req.hwrm_intf_min = HWRM_VERSION_MINOR; + ver_get_req.hwrm_intf_upd = HWRM_VERSION_UPDATE; + bng_re_fill_fw_msg(&fw_msg, (void *)&ver_get_req, sizeof(ver_get_req), + (void *)&ver_get_resp, sizeof(ver_get_resp), + BNGE_DFLT_HWRM_CMD_TIMEOUT); + rc = bnge_send_msg(aux_dev, &fw_msg); + if (rc) { + ibdev_err(&rdev->ibdev, "Failed to query HW version, rc = 0x%x", + rc); + return; + } + + cctx = rdev->chip_ctx; + cctx->hwrm_intf_ver = + (u64)le16_to_cpu(ver_get_resp.hwrm_intf_major) << 48 | + (u64)le16_to_cpu(ver_get_resp.hwrm_intf_minor) << 32 | + (u64)le16_to_cpu(ver_get_resp.hwrm_intf_build) << 16 | + le16_to_cpu(ver_get_resp.hwrm_intf_patch); + + cctx->hwrm_cmd_max_timeout = le16_to_cpu(ver_get_resp.max_req_timeout); + + if (!cctx->hwrm_cmd_max_timeout) + cctx->hwrm_cmd_max_timeout = BNG_ROCE_FW_MAX_TIMEOUT; +} + +static int bng_re_dev_init(struct bng_re_dev *rdev) +{ + int rc; + + /* Registered a new RoCE device instance to netdev */ + rc = bng_re_register_netdev(rdev); + if (rc) { + ibdev_err(&rdev->ibdev, + "Failed to register with netedev: %#x\n", rc); + return -EINVAL; + } + + set_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); + + if (rdev->aux_dev->auxr_info->msix_requested < BNG_RE_MIN_MSIX) { + ibdev_err(&rdev->ibdev, + "RoCE requires minimum 2 MSI-X vectors, but only %d reserved\n", + rdev->aux_dev->auxr_info->msix_requested); + bnge_unregister_dev(rdev->aux_dev); + clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); + return -EINVAL; + } + ibdev_dbg(&rdev->ibdev, "Got %d MSI-X vectors\n", + rdev->aux_dev->auxr_info->msix_requested); + + rc = bng_re_setup_chip_ctx(rdev); + if (rc) { + bnge_unregister_dev(rdev->aux_dev); + clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags); + ibdev_err(&rdev->ibdev, "Failed to get chip context\n"); + return -EINVAL; + } + + bng_re_query_hwrm_version(rdev); + + return 0; +} + +static void bng_re_dev_uninit(struct bng_re_dev *rdev) +{ + bng_re_destroy_chip_ctx(rdev); + if (test_and_clear_bit(BNG_RE_FLAG_NETDEV_REGISTERED, &rdev->flags)) + bnge_unregister_dev(rdev->aux_dev); +} + static int bng_re_add_device(struct auxiliary_device *adev) { struct bnge_auxr_priv *auxr_priv = @@ -58,7 +198,14 @@ static int bng_re_add_device(struct auxiliary_device *adev) dev_info->rdev = rdev; + rc = bng_re_dev_init(rdev); + if (rc) + goto re_dev_dealloc; + return 0; + +re_dev_dealloc: + ib_dealloc_device(&rdev->ibdev); exit: return rc; } @@ -67,6 +214,7 @@ static int bng_re_add_device(struct auxiliary_device *adev) static void bng_re_remove_device(struct bng_re_dev *rdev, struct auxiliary_device *aux_dev) { + bng_re_dev_uninit(rdev); ib_dealloc_device(&rdev->ibdev); } @@ -90,6 +238,7 @@ static int bng_re_probe(struct auxiliary_device *adev, rc = bng_re_add_device(adev); if (rc) kfree(en_info); + return rc; } diff --git a/drivers/infiniband/hw/bng_re/bng_re.h b/drivers/infiniband/hw/bng_re/bng_re.h index bd3aacdc05c4..db692ad8db0e 100644 --- a/drivers/infiniband/hw/bng_re/bng_re.h +++ b/drivers/infiniband/hw/bng_re/bng_re.h @@ -11,6 +11,8 @@ #define rdev_to_dev(rdev) ((rdev) ? (&(rdev)->ibdev.dev) : NULL) +#define BNG_RE_MIN_MSIX 2 + struct bng_re_en_dev_info { struct bng_re_dev *rdev; struct bnge_auxr_dev *auxr_dev; @@ -18,9 +20,12 @@ struct bng_re_en_dev_info { struct bng_re_dev { struct ib_device ibdev; + unsigned long flags; +#define BNG_RE_FLAG_NETDEV_REGISTERED 0 struct net_device *netdev; struct auxiliary_device *adev; struct bnge_auxr_dev *aux_dev; + struct bng_re_chip_ctx *chip_ctx; int fn_id; }; diff --git a/drivers/infiniband/hw/bng_re/bng_res.h b/drivers/infiniband/hw/bng_re/bng_res.h new file mode 100644 index 000000000000..d64833498e2a --- /dev/null +++ b/drivers/infiniband/hw/bng_re/bng_res.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +// Copyright (c) 2025 Broadcom. + +#ifndef __BNG_RES_H__ +#define __BNG_RES_H__ + +#define BNG_ROCE_FW_MAX_TIMEOUT 60 + +struct bng_re_chip_ctx { + u16 chip_num; + u16 hw_stats_size; + u64 hwrm_intf_ver; + u16 hwrm_cmd_max_timeout; +}; + +#endif -- 2.34.1