From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010015.outbound.protection.outlook.com [52.101.193.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7257332D7F3 for ; Wed, 4 Feb 2026 23:44:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.15 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770248683; cv=fail; b=DRHmySGiOEyKSnFHWOXUoOxkNK+/jfnris0Eo+EdHEN+3Bum3Ut74V2Qw4tPdu5alu2qKQ6ixpRzkWe2+AK1XiAMN136UfcKCormnk3oGPh8W3A0VEJqMZA90n4u7aQME/UO3SpnFrV/NMOnbp2fFy9qQfhfWVEN+N/XBasbc5Q= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770248683; c=relaxed/simple; bh=Yz6+v7ewhhz0Jm65I51sVZCIB3h/iu3zqtIN2FwJJos=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=e39NrRA94XgCsbuI3OV7fMhYZTRCaFMhbAwWVTXCAbxyD2QIVEwF7QgDXi8XrexPwvrMvwOMwyumYb16T7hDHq6y0lSpjULRo0GwqLDDfdG2mkc0sLR4CEHrYTKKGyc2wih/bD1dhAq4f3y1nyhzTFpZs+Wbak2uQl28zG00lPs= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=cqE8O+2G; arc=fail smtp.client-ip=52.101.193.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="cqE8O+2G" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=id7WPsi37ZyRAafSYpCDymX0s0BesZq4oUIv7d037X4wvwwIRNVxDH0Erfo/+tQHdVq8yiTdfFC7m8jXFMi9NSpOCYq64OR7PBp56rGlhWI4ZaOEyJd2QtfysE43+6PsmezXB1NqH0L299LGpWz4zF8oHJioj7qAbDy4ayqN0UI6HgSHwGwzwYNsssCTgCFo8/3FVCXPtaZg1UgqhICgFWH4yYFiZFAJNWc3vsVL+yjo1VVHAUxmiXthA1fekJJBqkUvArWpbDDnrdEdCKuCudSJtAAX7ye6QljRAbnBjQy5ImkQ7YjKUqtdgMkQWCKO0UIREduZLDNGSAGCE90x0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WsxnffXpaDFavlNU8KJ+SXSEv14EO4dMynfW+YMRnRI=; b=C9VAAJpa4D9OHfDBK/FE84YY6hCXWhfVue2BuBNR0d5ZKtXtbKgdBAg4dJSGNYDiA2D9gDY2+ds++dH50GJ+5KX+hOJ86Mk4JoUcuSwjndKsr6VWynGsZpEVUFWvaOXUDbCRwmtukbp0V81CJiuF/yuqdmM4GQRx90Xpnx7MenjaCHCdqMVqSZwPCzPZq22HHqxGDwtjaoTrCM1zH2GFyGiFHZ4to7iItbOIVb4+r31Qze4BcUjp/3NaYQXKSGlBYedfhNytTiPtMVUUYyhm387SrvakWlsZoHd4d/nOONc/i5bVF81KdRQkPsTU57wIDm00nZ96nGGRRWSCbpmcJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WsxnffXpaDFavlNU8KJ+SXSEv14EO4dMynfW+YMRnRI=; b=cqE8O+2GPFt2KLUKYzc4T2jQ6iKyyWAZ5VrGRQeycMBFlEqG228MMD0CAIvq0b+DfNUKunzodrc6F/E5R0tYeAJAmFQeg80E9su2/XkVxOC/2dzxN57lDxPExvjVpzbs+XIUFRw4SKw/FyiYq1rOvrG85t9EP0/XEH0tBT3SzZf+9pA8qUmHBKS3vHw09xicVv3vDhV5crqYj3XI9B7VofNwzlXvxKaPT5UXAmxUjNtjDjbpwfycqb1Vn6bjRh5JqjyNl6qroyQRND17X3z3OLn06d1fuD60CiYxei1jnAJrby8gTH7Ou3BJRb9msAdp4uDNY3B3/6vgfeefr7gnlw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS0PR12MB6414.namprd12.prod.outlook.com (2603:10b6:8:cd::12) by IA4PR12MB9812.namprd12.prod.outlook.com (2603:10b6:208:55b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9587.12; Wed, 4 Feb 2026 23:44:38 +0000 Received: from DS0PR12MB6414.namprd12.prod.outlook.com ([fe80::2459:590d:bcb:1f9e]) by DS0PR12MB6414.namprd12.prod.outlook.com ([fe80::2459:590d:bcb:1f9e%4]) with mapi id 15.20.9587.010; Wed, 4 Feb 2026 23:44:38 +0000 Date: Wed, 4 Feb 2026 15:44:36 -0800 From: Saeed Mahameed To: Pavan Chebbi Cc: jgg@ziepe.ca, michael.chan@broadcom.com, linux-kernel@vger.kernel.org, dave.jiang@intel.com, Jonathan.Cameron@huawei.com, gospo@broadcom.com, selvin.xavier@broadcom.com, leon@kernel.org, kalesh-anakkur.purayil@broadcom.com Subject: Re: [PATCH v3 fwctl 2/5] fwctl/bnxt_en: Refactor aux bus functions to be more generic Message-ID: References: <20260129155453.3626544-1-pavan.chebbi@broadcom.com> <20260129155453.3626544-3-pavan.chebbi@broadcom.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20260129155453.3626544-3-pavan.chebbi@broadcom.com> X-ClientProxiedBy: SJ0PR05CA0165.namprd05.prod.outlook.com (2603:10b6:a03:339::20) To DS0PR12MB6414.namprd12.prod.outlook.com (2603:10b6:8:cd::12) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB6414:EE_|IA4PR12MB9812:EE_ X-MS-Office365-Filtering-Correlation-Id: c30c9d5d-8471-4aca-1be8-08de64475bcf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|7416014|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?NLpoKnOFnMkPSMjQ3S/3qrcfaltyz9lVutjOwS7VfQC64rV1SAvwOu1Fd0DV?= =?us-ascii?Q?HYQDtrQ73JmsmjEB3RJo96iLpLOFPykQ69xGYYGdpf4LPtM4Aek7gQAb9Q3i?= =?us-ascii?Q?GHkZKK5/QcGQy7F/h41im7GMTmAjHta/eZGsiVsdUgTLDE67KFW8k1S2zu8t?= =?us-ascii?Q?WuJmhumcVhownVTZGYVfm4LjHl2ZgLqVF/iuYp3vHnyCrjafOULdAY9zQrkI?= =?us-ascii?Q?dJBYXkD2HsUH96p6iCNMzjg5pnqFzDIttpTKwHj9s/CiHiTens1qorsvi4jK?= =?us-ascii?Q?ivAeZPySkeWx7HG29x2cI0DOBQpBtJmt6jMsp1Pcr/0YjiUYPGxvB2k+5DuC?= =?us-ascii?Q?bBxNtf8P6P+HBm1iUFh5L/JroDi4A5e5AbdX+y1M2nMxvCJ22Db42n4SLQpw?= =?us-ascii?Q?Fk21nSzw9YGeA8CyYP6mMKFntdNm30sGgh35Kr6TD1XhElAMiNYEOBmDqwrw?= =?us-ascii?Q?zaYvNWCxXLR05wQUVtx6NS5mi/uokpENvTZCWKLAzUxTBSBUPVgxetZLL/69?= =?us-ascii?Q?tM00lzJLNWSdqkm1L2NCev0l0D+LvGrXX46t7KMj+Mfhf6qakcOo3wuuLEx/?= =?us-ascii?Q?lm2QhvSLuXCmvo/2NOHBCi6D1d0lelrvqFflx3luQjTK0Z4hCcIrW1CEkNpp?= =?us-ascii?Q?VDOn5bM53tDDOHY65sVdZ1YARymZxfneGcEUbjwhmfUD+gILZwS9qKMsG+5I?= =?us-ascii?Q?bPo7m7CAZUOlhspelZ13llqkKyY+NdZKW3dEqeNz+NkrxhSiwbRrIPViZE/V?= =?us-ascii?Q?nzaEjjSV4NJC1JYdrqGW0AVWtVXBdwYvtN6Gw/dMkob1IX5yJnBunwC7gTfK?= =?us-ascii?Q?8y/fF2AzgxUFfvHvfk9OJKkrEMp4TWL2qSrhNzoRc+4mf/43+hhDPye3+lJS?= =?us-ascii?Q?PucuE11mT5pHwY4P3ubXPi1ZPKWVREHCWKcBkm/kd2w1R5lEUAcbAiITBeUt?= =?us-ascii?Q?uvTw1YrRQTDo8Pt1zOzTVOubeLz/aKaqf3inzSTpeidQb8oQ8642FyFdNlLw?= =?us-ascii?Q?oThdp1rG1Q/YNCs9EqPK+HsnwN94RBy2t/pqBda8GMavgSkj+90rrsfZsAuV?= =?us-ascii?Q?A6qfIf/cLo6vKgWS4X7yJEVIrHPNdXtwylyEjoSCgL53FAHZg95a2X5XsmQr?= =?us-ascii?Q?IHtgXdvdrz/hQin1CCRyuFbircfPgSP51DeHs61Jm6P+yZXZxxYrSScaP19V?= =?us-ascii?Q?Ptw1DIwJoR5Nzqj8Iv5tiLoWuQlTKCj+NxP1UXS4dBjFL3tCm3LIOCfyrQUg?= =?us-ascii?Q?MH6hZcdRfpDRI39wKrKrzG3yg5Nqq93xn8/0/00c/TppQCNnWAezYYQDz7tH?= =?us-ascii?Q?cKmJh8j+yGSsB5JWnMkMq3UbAJZ61EshiH5y5EKCDilsHyPWc0RRb6sKn+xm?= =?us-ascii?Q?JE0cu2DxB5JV4gfFWrCSq9U0qpL0TsjniWIXP7la38HSZkaqFJ6fC2yJtb2U?= =?us-ascii?Q?JFYXQcirioMVcCNqFxV/SRpgXF2XvHUPpZYhkfwcVKutRGF4utd2PXhuiIg0?= =?us-ascii?Q?pEIOqjf//n1Tx0SNuBG4+1YXbGa1vcd20s4F7vfhj6fm0KnD8LZeBd1Z+v8G?= =?us-ascii?Q?pSINkxoHoHwLxZx2J6A=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB6414.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(7416014)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?lokXGgBsbCWuowWpthVCrrLsw2IX/qsD4HC343GmMsca6jR2LpwGr9hshnvR?= =?us-ascii?Q?uVjVRs/gh5XZYkhksEzQKifSQgYYQaz3GMllvpWHl7rhcuRut0BJO79K6haD?= =?us-ascii?Q?sHH/VGhmyH1QzoTej3BYH6l+kvDVgjm5hAnetBvIfqMu7gZvm3ZlBg64tn5Q?= =?us-ascii?Q?b1m27l9CHCPLzI5pVQVJkcJR+LLRjhd6qgy4wCG6n2Ldcc1dpMdQRTIEZyuH?= =?us-ascii?Q?WwiPQ2mXyB1g9tA65ggYbW5HrTZL55DKQy291NjoZsekqITIgRDLwj1Vbvl8?= =?us-ascii?Q?RRzOueBYQNBiIFvrog/jISH1d79n3HmqYqmJNb5xNJenyko+rAgard+QkwaO?= =?us-ascii?Q?69Q10RGkmV6Ua+0vbkbY6fVPoQ/Pahc/gjuzslgf+EQU/20KnZ86zxbJWkFG?= =?us-ascii?Q?dLyqFLQg0CVnjkbt3SV+MnYXUMugp9m3filsQIspnUPD37n90ZNVMhCWoE9f?= =?us-ascii?Q?5xoT6in4VI90kN4f66YOV7CcQSbzAoh/pMjUeqlMeWRxWKgfXNZnOJ7q24i7?= =?us-ascii?Q?Np3LmcauQJLy1HBw1FyNTa3GEpJ2B0j1J4U2KqPPaOTZrPIYoK37axNrPY5M?= =?us-ascii?Q?uIWJ5RmwuDUZHailitGJplH5FN1JLrn6CfVRpxgCUtxnIfmIEwa4RaJPGvNA?= =?us-ascii?Q?bL9M1fPJv2L4Z3TLcE7MFvBZcHrFo2tarjm9IeTK9SmNZorrGCz3Hwp1Rj0C?= =?us-ascii?Q?PU07KLV0M28c75TzTTwoDt9E6CFtAMtNFMFXnYzaWp+QMgmOxp4+3LGqCkqL?= =?us-ascii?Q?wXYQg0qCw4skzDeeKlfvH0N1Ej4QBVQmbv4G5TSA4Evx5wYy7Yru2jOVarfi?= =?us-ascii?Q?qO6T79b1C84Df78xOtSwrELHEOVdSWHdIwEWEmbNST1xiEsM9hOE4ZZxELSJ?= =?us-ascii?Q?06NpDPTOX1pp4XjZWZian79GmD95HDAMYqaHlu2p/tvEDVoUZUrzXkuNAGSF?= =?us-ascii?Q?J8Cl7NfpQXZNmx/5gq/kYcn1rtT6PKbs1aF6xZKZuMecthuFPQiAsG0jEMH1?= =?us-ascii?Q?NAU5RpBcwOVPS4I6IFvXksrw/untXFWxI/omX89Ko+7+EtmVIQQatdb08rqY?= =?us-ascii?Q?lFZ099IRcJHs4uSdQzTVFJ+GxfvZQn4YHOMphCpBUdDg9lS9tMwIt8wqSIjO?= =?us-ascii?Q?2WeKLGYKoVhewtAmvZV27RE2ILXCfWqw21fywal8w9Hbh/mhdHJDqFoBOoJI?= =?us-ascii?Q?o1bADDugE1x0Pgal5xpRwCl9dxJZ2QUoQKAbqjo7hCpUp9quqdYsqt7qXYTk?= =?us-ascii?Q?p9pM5EPjQOoY8DxaarB1RJ4a6fwt5YB5z15U3ATzCwgBP7swRB8Olp9TP4W2?= =?us-ascii?Q?la9q7PzZlW4iW9q+Lv0nPXLXqG+ATQEISus6AArNtBf+HKZR89vOSN1GkY/I?= =?us-ascii?Q?5JGyBGV6a+cAifxjbouzpULLONl5epJvsrkhbQ7YOrvJQhGkCYgL9uy2bLVH?= =?us-ascii?Q?OxVrPYjlFaD66/FDj9qrELURxZzEXX16fhxb6ABsF+JfkeGcWY0YiTHol1DG?= =?us-ascii?Q?pg5DcdekOFnsCrmtDbqxP5754NiVJHzzxEDIHaEGhNGNx1kdOUH5w0Fg/3ou?= =?us-ascii?Q?Rq87lyIC/GSCzeOOIqnIHhQDgzM0DNShzHw74gMAfxmMvF345sg7+E+UB4Oo?= =?us-ascii?Q?Rq9p3WlLPpm0eVisMYskoIl7ke74xiVxTOK432JslgOZo0hTA1/QwZMsrrn0?= =?us-ascii?Q?L0qifPRugtyPXKleHYfJO4I9VzmAFAkK2uNnielSH710hhwxF+u/N32X0jpx?= =?us-ascii?Q?kwmAQOmkWg=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: c30c9d5d-8471-4aca-1be8-08de64475bcf X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB6414.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Feb 2026 23:44:38.0990 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VmbefiHXB0tSjauXmFd1ZYSd5L3eLFuendqfLwBrBSmwPQ0RGCysMTtqmCsQ0yvj8rZ021zlaR272OZM+xE/UQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA4PR12MB9812 On 29 Jan 07:54, Pavan Chebbi wrote: >Up until now there was only one auxiliary device that bnxt >created and that was for RoCE driver. bnxt fwctl is also >going to use an aux bus device that bnxt should create. >This requires some nomenclature changes and refactoring of >the existing bnxt aux dev functions. > >Convert 'aux_priv' and 'edev' members of struct bnxt into >arrays where each element contains supported auxbus device's >data. Move struct bnxt_aux_priv from bnxt.h to ulp.h because >that is where it belongs. Make aux bus init/uninit/add/del >functions more generic which will loop through all the aux >device types. Make bnxt_ulp_start/stop functions (the only >other common functions applicable to any aux device) loop >through the aux devices to update their config and states. >Make callers of bnxt_ulp_start() call it only when there >are no errors. > >Also, as an improvement in code, bnxt_register_dev() can skip >unnecessary dereferencing of edev from bp, instead use the >edev pointer from the function parameter. > >Future patches will reuse these functions to add an aux bus >device for fwctl. > >Reviewed-by: Andy Gospodarek >Signed-off-by: Pavan Chebbi >--- > drivers/net/ethernet/broadcom/bnxt/bnxt.c | 47 ++- > drivers/net/ethernet/broadcom/bnxt/bnxt.h | 19 +- > .../net/ethernet/broadcom/bnxt/bnxt_devlink.c | 8 +- > .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 2 +- > drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 339 +++++++++++------- > include/linux/bnxt/ulp.h | 25 +- > 6 files changed, 273 insertions(+), 167 deletions(-) > >diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c >index 4481d80cdfc2..a9bfbfabf121 100644 >--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c >+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c >@@ -6859,7 +6859,8 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic) > #endif > if ((bp->flags & BNXT_FLAG_STRIP_VLAN) || def_vlan) > req->flags |= cpu_to_le32(VNIC_CFG_REQ_FLAGS_VLAN_STRIP_MODE); >- if (vnic->vnic_id == BNXT_VNIC_DEFAULT && bnxt_ulp_registered(bp->edev)) >+ if (vnic->vnic_id == BNXT_VNIC_DEFAULT && >+ bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) > req->flags |= cpu_to_le32(bnxt_get_roce_vnic_mode(bp)); > > return hwrm_req_send(bp, req); >@@ -7974,6 +7975,7 @@ static int bnxt_get_avail_msix(struct bnxt *bp, int num); > > static int __bnxt_reserve_rings(struct bnxt *bp) > { >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; > struct bnxt_hw_rings hwr = {0}; > int rx_rings, old_rx_rings, rc; > int cp = bp->cp_nr_rings; >@@ -7984,7 +7986,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) > if (!bnxt_need_reserve_rings(bp)) > return 0; > >- if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(bp->edev)) { >+ if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(edev)) { > ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want); > if (!ulp_msix) > bnxt_set_ulp_stat_ctxs(bp, 0); >@@ -8035,8 +8037,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) > } > rx_rings = min_t(int, rx_rings, hwr.grp); > hwr.cp = min_t(int, hwr.cp, bp->cp_nr_rings); >- if (bnxt_ulp_registered(bp->edev) && >- hwr.stat > bnxt_get_ulp_stat_ctxs(bp)) >+ if (bnxt_ulp_registered(edev) && hwr.stat > bnxt_get_ulp_stat_ctxs(bp)) > hwr.stat -= bnxt_get_ulp_stat_ctxs(bp); > hwr.cp = min_t(int, hwr.cp, hwr.stat); > rc = bnxt_trim_rings(bp, &rx_rings, &hwr.tx, hwr.cp, sh); >@@ -8075,7 +8076,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) > !netif_is_rxfh_configured(bp->dev)) > bnxt_set_dflt_rss_indir_tbl(bp, NULL); > >- if (!bnxt_ulp_registered(bp->edev) && BNXT_NEW_RM(bp)) { >+ if (!bnxt_ulp_registered(edev) && BNXT_NEW_RM(bp)) { > int resv_msix, resv_ctx, ulp_ctxs; > struct bnxt_hw_resc *hw_resc; > >@@ -11430,6 +11431,7 @@ static void bnxt_clear_int_mode(struct bnxt *bp) > > int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init) > { >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; > bool irq_cleared = false; > bool irq_change = false; > int tcs = bp->num_tc; >@@ -11439,7 +11441,7 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init) > if (!bnxt_need_reserve_rings(bp)) > return 0; > >- if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(bp->edev)) { >+ if (BNXT_NEW_RM(bp) && !bnxt_ulp_registered(edev)) { > int ulp_msix = bnxt_get_avail_msix(bp, bp->ulp_num_msix_want); > > if (ulp_msix > bp->ulp_num_msix_want) >@@ -14522,7 +14524,7 @@ static void bnxt_fw_echo_reply(struct bnxt *bp) > static void bnxt_ulp_restart(struct bnxt *bp) > { > bnxt_ulp_stop(bp); >- bnxt_ulp_start(bp, 0); >+ bnxt_ulp_start(bp); > } > > static void bnxt_sp_task(struct work_struct *work) >@@ -14679,7 +14681,7 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs, > hwr.cp_p5 = hwr.tx + rx; > rc = bnxt_hwrm_check_rings(bp, &hwr); > if (!rc && pci_msix_can_alloc_dyn(bp->pdev)) { >- if (!bnxt_ulp_registered(bp->edev)) { >+ if (!bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) { > hwr.cp += bnxt_get_ulp_msix_num(bp); > hwr.cp = min_t(int, hwr.cp, bnxt_get_max_func_irqs(bp)); > } >@@ -15199,7 +15201,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) > bnxt_dl_health_fw_status_update(bp, true); > } > netdev_unlock(bp->dev); >- bnxt_ulp_start(bp, 0); >+ bnxt_ulp_start(bp); > bnxt_reenable_sriov(bp); > netdev_lock(bp->dev); > bnxt_vf_reps_alloc(bp); >@@ -15221,7 +15223,8 @@ static void bnxt_fw_reset_task(struct work_struct *work) > bnxt_fw_reset_abort(bp, rc); > netdev_unlock(bp->dev); > ulp_start: >- bnxt_ulp_start(bp, rc); >+ if (!rc) >+ bnxt_ulp_start(bp); > } > > static int bnxt_init_board(struct pci_dev *pdev, struct net_device *dev) >@@ -16211,12 +16214,13 @@ static void bnxt_remove_one(struct pci_dev *pdev) > if (BNXT_PF(bp)) > __bnxt_sriov_disable(bp); > >- bnxt_rdma_aux_device_del(bp); >+ bnxt_aux_devices_del(bp); > > unregister_netdev(dev); > bnxt_ptp_clear(bp); > >- bnxt_rdma_aux_device_uninit(bp); >+ bnxt_aux_devices_uninit(bp); >+ bnxt_auxdev_id_free(bp, bp->auxdev_id); > > bnxt_free_l2_filters(bp, true); > bnxt_free_ntp_fltrs(bp, true); >@@ -16802,7 +16806,9 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) > bnxt_set_tpa_flags(bp); > bnxt_init_ring_params(bp); > bnxt_set_ring_params(bp); >- bnxt_rdma_aux_device_init(bp); >+ mutex_init(&bp->auxdev_lock); >+ if (!bnxt_auxdev_id_alloc(bp)) >+ bnxt_aux_devices_init(bp); > rc = bnxt_set_dflt_rings(bp, true); > if (rc) { > if (BNXT_VF(bp) && rc == -ENODEV) { >@@ -16866,7 +16872,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) > > bnxt_dl_fw_reporters_create(bp); > >- bnxt_rdma_aux_device_add(bp); >+ bnxt_aux_devices_add(bp); > > bnxt_print_device_info(bp); > >@@ -16874,7 +16880,8 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) > > return 0; > init_err_cleanup: >- bnxt_rdma_aux_device_uninit(bp); >+ bnxt_aux_devices_uninit(bp); >+ bnxt_auxdev_id_free(bp, bp->auxdev_id); > bnxt_dl_unregister(bp); > init_err_dl: > bnxt_shutdown_tc(bp); >@@ -17008,9 +17015,10 @@ static int bnxt_resume(struct device *device) > > resume_exit: > netdev_unlock(bp->dev); >- bnxt_ulp_start(bp, rc); >- if (!rc) >+ if (!rc) { >+ bnxt_ulp_start(bp); > bnxt_reenable_sriov(bp); >+ } > return rc; > } > >@@ -17190,9 +17198,10 @@ static void bnxt_io_resume(struct pci_dev *pdev) > netif_device_attach(netdev); > > netdev_unlock(netdev); >- bnxt_ulp_start(bp, err); >- if (!err) >+ if (!err) { >+ bnxt_ulp_start(bp); > bnxt_reenable_sriov(bp); >+ } > } > > static const struct pci_error_handlers bnxt_err_handler = { >diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h >index f5f07a7e6b29..a739e7d52bdc 100644 >--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h >+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h >@@ -24,12 +24,12 @@ > #include > #include > #include >-#include > #include > #include > #include > #include > #include >+#include > #ifdef CONFIG_TEE_BNXT_FW > #include > #endif >@@ -2076,12 +2076,6 @@ struct bnxt_fw_health { > #define BNXT_FW_IF_RETRY 10 > #define BNXT_FW_SLOT_RESET_RETRY 4 > >-struct bnxt_aux_priv { >- struct auxiliary_device aux_dev; >- struct bnxt_en_dev *edev; >- int id; >-}; >- > enum board_idx { > BCM57301, > BCM57302, >@@ -2341,8 +2335,8 @@ struct bnxt { > #define BNXT_CHIP_P5_AND_MINUS(bp) \ > (BNXT_CHIP_P3(bp) || BNXT_CHIP_P4(bp) || BNXT_CHIP_P5(bp)) > >- struct bnxt_aux_priv *aux_priv; >- struct bnxt_en_dev *edev; >+ struct bnxt_aux_priv *aux_priv[__BNXT_AUXDEV_MAX]; >+ struct bnxt_en_dev *edev[__BNXT_AUXDEV_MAX]; > > struct bnxt_napi **bnapi; > >@@ -2751,6 +2745,13 @@ struct bnxt { > struct bnxt_ctx_pg_info *fw_crash_mem; > u32 fw_crash_len; > struct bnxt_bs_trace_info bs_trace[BNXT_TRACE_MAX]; >+ int auxdev_id; >+ /* synchronize validity checks of available aux devices */ >+ struct mutex auxdev_lock; >+ u8 auxdev_state[__BNXT_AUXDEV_MAX]; >+#define BNXT_ADEV_STATE_NONE 0 >+#define BNXT_ADEV_STATE_INIT 1 >+#define BNXT_ADEV_STATE_ADD 2 > }; > > #define BNXT_NUM_RX_RING_STATS 8 >diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c >index 230cd95d30a2..835f2b413931 100644 >--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c >+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c >@@ -440,13 +440,13 @@ static int bnxt_dl_reload_down(struct devlink *dl, bool netns_change, > "reload is unsupported while VFs are allocated or being configured"); > netdev_unlock(bp->dev); > rtnl_unlock(); >- bnxt_ulp_start(bp, 0); >+ bnxt_ulp_start(bp); > return -EOPNOTSUPP; > } > if (bp->dev->reg_state == NETREG_UNREGISTERED) { > netdev_unlock(bp->dev); > rtnl_unlock(); >- bnxt_ulp_start(bp, 0); >+ bnxt_ulp_start(bp); > return -ENODEV; > } > if (netif_running(bp->dev)) >@@ -578,8 +578,8 @@ static int bnxt_dl_reload_up(struct devlink *dl, enum devlink_reload_action acti > } > netdev_unlock(bp->dev); > rtnl_unlock(); >- if (action == DEVLINK_RELOAD_ACTION_DRIVER_REINIT) >- bnxt_ulp_start(bp, rc); >+ if (!rc && action == DEVLINK_RELOAD_ACTION_DRIVER_REINIT) >+ bnxt_ulp_start(bp); > return rc; > } > >diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c >index 8cad7b982664..064d7bc4ce8d 100644 >--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c >+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c >@@ -5100,7 +5100,7 @@ static void bnxt_self_test(struct net_device *dev, struct ethtool_test *etest, > > memset(buf, 0, sizeof(u64) * bp->num_tests); > if (etest->flags & ETH_TEST_FL_OFFLINE && >- bnxt_ulp_registered(bp->edev)) { >+ bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) { > etest->flags |= ETH_TEST_FL_FAILED; > netdev_warn(dev, "Offline tests cannot be run with RoCE driver loaded\n"); > return; >diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c >index fa513892db50..3097fc5755e6 100644 >--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c >+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c >@@ -29,9 +29,32 @@ > > static DEFINE_IDA(bnxt_aux_dev_ids); > >+struct bnxt_aux_device { >+ const char *name; >+}; >+ >+static void bnxt_auxdev_set_state(struct bnxt *bp, int idx, int state) >+{ >+ bp->auxdev_state[idx] = state; >+} >+ >+static bool bnxt_auxdev_is_init(struct bnxt *bp, int idx) >+{ >+ return (bp->auxdev_state[idx] == BNXT_ADEV_STATE_INIT); >+} >+ >+static bool bnxt_auxdev_is_active(struct bnxt *bp, int idx) >+{ >+ return (bp->auxdev_state[idx] == BNXT_ADEV_STATE_ADD); >+} >+ >+static struct bnxt_aux_device bnxt_aux_devices[__BNXT_AUXDEV_MAX] = {{ >+ .name = "rdma", >+}}; >+ > static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent) > { >- struct bnxt_en_dev *edev = bp->edev; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; > int num_msix, i; > > if (!edev->ulp_tbl->msix_requested) { >@@ -51,61 +74,75 @@ static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent) > > int bnxt_get_ulp_msix_num(struct bnxt *bp) > { >- if (bp->edev) >- return bp->edev->ulp_num_msix_vec; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; >+ >+ if (edev) >+ return edev->ulp_num_msix_vec; > return 0; > } > > void bnxt_set_ulp_msix_num(struct bnxt *bp, int num) > { >- if (bp->edev) >- bp->edev->ulp_num_msix_vec = num; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; >+ >+ if (edev) >+ edev->ulp_num_msix_vec = num; > } > > int bnxt_get_ulp_msix_num_in_use(struct bnxt *bp) > { >- if (bnxt_ulp_registered(bp->edev)) >- return bp->edev->ulp_num_msix_vec; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; >+ >+ if (bnxt_ulp_registered(edev)) >+ return edev->ulp_num_msix_vec; > return 0; > } > > int bnxt_get_ulp_stat_ctxs(struct bnxt *bp) > { >- if (bp->edev) >- return bp->edev->ulp_num_ctxs; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; >+ >+ if (edev) >+ return edev->ulp_num_ctxs; > return 0; > } > > void bnxt_set_ulp_stat_ctxs(struct bnxt *bp, int num_ulp_ctx) > { >- if (bp->edev) >- bp->edev->ulp_num_ctxs = num_ulp_ctx; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; >+ >+ if (edev) >+ edev->ulp_num_ctxs = num_ulp_ctx; > } > > int bnxt_get_ulp_stat_ctxs_in_use(struct bnxt *bp) > { >- if (bnxt_ulp_registered(bp->edev)) >- return bp->edev->ulp_num_ctxs; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; >+ >+ if (bnxt_ulp_registered(edev)) >+ return edev->ulp_num_ctxs; > return 0; > } > > void bnxt_set_dflt_ulp_stat_ctxs(struct bnxt *bp) > { >- if (bp->edev) { >- bp->edev->ulp_num_ctxs = BNXT_MIN_ROCE_STAT_CTXS; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; >+ >+ if (edev) { >+ edev->ulp_num_ctxs = BNXT_MIN_ROCE_STAT_CTXS; > /* Reserve one additional stat_ctx for PF0 (except > * on 1-port NICs) as it also creates one stat_ctx > * for PF1 in case of RoCE bonding. > */ > if (BNXT_PF(bp) && !bp->pf.port_id && > bp->port_count > 1) >- bp->edev->ulp_num_ctxs++; >+ edev->ulp_num_ctxs++; > > /* Reserve one additional stat_ctx when the device is capable > * of supporting port mirroring on RDMA device. > */ > if (BNXT_MIRROR_ON_ROCE_CAP(bp)) >- bp->edev->ulp_num_ctxs++; >+ edev->ulp_num_ctxs++; > } > } > >@@ -141,7 +178,7 @@ int bnxt_register_dev(struct bnxt_en_dev *edev, > > edev->ulp_tbl->msix_requested = bnxt_get_ulp_msix_num(bp); > >- bnxt_fill_msix_vecs(bp, bp->edev->msix_entries); >+ bnxt_fill_msix_vecs(bp, edev->msix_entries); > exit: > mutex_unlock(&edev->en_dev_lock); > netdev_unlock(dev); >@@ -227,20 +264,27 @@ EXPORT_SYMBOL(bnxt_send_msg); > > void bnxt_ulp_stop(struct bnxt *bp) > { >- struct bnxt_aux_priv *aux_priv = bp->aux_priv; >- struct bnxt_en_dev *edev = bp->edev; >+ int i; > >- if (!edev) >- return; >- >- mutex_lock(&edev->en_dev_lock); >- if (!bnxt_ulp_registered(edev) || >- (edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) >- goto ulp_stop_exit; >- >- edev->flags |= BNXT_EN_FLAG_ULP_STOPPED; >- if (aux_priv) { >+ mutex_lock(&bp->auxdev_lock); >+ for (i = 0; i < __BNXT_AUXDEV_MAX; i++) { >+ struct bnxt_aux_priv *aux_priv; > struct auxiliary_device *adev; >+ struct bnxt_en_dev *edev; >+ >+ if (!bnxt_auxdev_is_active(bp, i)) >+ continue; >+ >+ aux_priv = bp->aux_priv[i]; >+ edev = bp->edev[i]; >+ mutex_lock(&edev->en_dev_lock); >+ if (!bnxt_ulp_registered(edev) || >+ (edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) { >+ mutex_unlock(&edev->en_dev_lock); >+ continue; >+ } >+ >+ edev->flags |= BNXT_EN_FLAG_ULP_STOPPED; > > adev = &aux_priv->aux_dev; > if (adev->dev.driver) { >@@ -251,29 +295,35 @@ void bnxt_ulp_stop(struct bnxt *bp) > edev->en_state = bp->state; > adrv->suspend(adev, pm); > } >+ mutex_unlock(&edev->en_dev_lock); > } >-ulp_stop_exit: >- mutex_unlock(&edev->en_dev_lock); >+ mutex_unlock(&bp->auxdev_lock); > } > >-void bnxt_ulp_start(struct bnxt *bp, int err) >+void bnxt_ulp_start(struct bnxt *bp) > { >- struct bnxt_aux_priv *aux_priv = bp->aux_priv; >- struct bnxt_en_dev *edev = bp->edev; >+ int i; > >- if (!edev || err) >- return; >+ mutex_lock(&bp->auxdev_lock); >+ for (i = 0; i < __BNXT_AUXDEV_MAX; i++) { >+ struct bnxt_aux_priv *aux_priv; >+ struct auxiliary_device *adev; >+ struct bnxt_en_dev *edev; > >- mutex_lock(&edev->en_dev_lock); >- if (!bnxt_ulp_registered(edev) || >- !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) >- goto ulp_start_exit; >+ if (!bnxt_auxdev_is_active(bp, i)) >+ continue; > >- if (edev->ulp_tbl->msix_requested) >- bnxt_fill_msix_vecs(bp, edev->msix_entries); >+ aux_priv = bp->aux_priv[i]; >+ edev = bp->edev[i]; >+ mutex_lock(&edev->en_dev_lock); >+ if (!bnxt_ulp_registered(edev) || >+ !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) { >+ goto clear_flag_continue; >+ } >+ >+ if (edev->ulp_tbl->msix_requested) >+ bnxt_fill_msix_vecs(bp, edev->msix_entries); > >- if (aux_priv) { >- struct auxiliary_device *adev; > > adev = &aux_priv->aux_dev; > if (adev->dev.driver) { >@@ -283,22 +333,23 @@ void bnxt_ulp_start(struct bnxt *bp, int err) > edev->en_state = bp->state; > adrv->resume(adev); > } >+clear_flag_continue: >+ edev->flags &= ~BNXT_EN_FLAG_ULP_STOPPED; >+ mutex_unlock(&edev->en_dev_lock); > } >-ulp_start_exit: >- edev->flags &= ~BNXT_EN_FLAG_ULP_STOPPED; >- mutex_unlock(&edev->en_dev_lock); >+ mutex_unlock(&bp->auxdev_lock); > } > > void bnxt_ulp_irq_stop(struct bnxt *bp) > { >- struct bnxt_en_dev *edev = bp->edev; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; > struct bnxt_ulp_ops *ops; > bool reset = false; > > if (!edev) > return; > >- if (bnxt_ulp_registered(bp->edev)) { >+ if (bnxt_ulp_registered(edev)) { > struct bnxt_ulp *ulp = edev->ulp_tbl; > > if (!ulp->msix_requested) >@@ -315,13 +366,13 @@ void bnxt_ulp_irq_stop(struct bnxt *bp) > > void bnxt_ulp_irq_restart(struct bnxt *bp, int err) > { >- struct bnxt_en_dev *edev = bp->edev; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; > struct bnxt_ulp_ops *ops; > > if (!edev) > return; > >- if (bnxt_ulp_registered(bp->edev)) { >+ if (bnxt_ulp_registered(edev)) { > struct bnxt_ulp *ulp = edev->ulp_tbl; > struct bnxt_msix_entry *ent = NULL; > >@@ -347,7 +398,7 @@ void bnxt_ulp_irq_restart(struct bnxt *bp, int err) > void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl) > { > u16 event_id = le16_to_cpu(cmpl->event_id); >- struct bnxt_en_dev *edev = bp->edev; >+ struct bnxt_en_dev *edev = bp->edev[BNXT_AUXDEV_RDMA]; > struct bnxt_ulp_ops *ops; > struct bnxt_ulp *ulp; > >@@ -388,18 +439,21 @@ void bnxt_register_async_events(struct bnxt_en_dev *edev, > } > EXPORT_SYMBOL(bnxt_register_async_events); > >-void bnxt_rdma_aux_device_uninit(struct bnxt *bp) >+void bnxt_aux_devices_uninit(struct bnxt *bp) > { > struct bnxt_aux_priv *aux_priv; > struct auxiliary_device *adev; >- >- /* Skip if no auxiliary device init was done. */ >- if (!bp->aux_priv) >- return; >- >- aux_priv = bp->aux_priv; >- adev = &aux_priv->aux_dev; >- auxiliary_device_uninit(adev); >+ int idx; >+ >+ mutex_lock(&bp->auxdev_lock); >+ for (idx = 0; idx < __BNXT_AUXDEV_MAX; idx++) { >+ if (bnxt_auxdev_is_init(bp, idx)) { >+ aux_priv = bp->aux_priv[idx]; >+ adev = &aux_priv->aux_dev; >+ auxiliary_device_uninit(adev); >+ } >+ } >+ mutex_unlock(&bp->auxdev_lock); > } > > static void bnxt_aux_dev_release(struct device *dev) >@@ -408,20 +462,25 @@ static void bnxt_aux_dev_release(struct device *dev) > container_of(dev, struct bnxt_aux_priv, aux_dev.dev); > struct bnxt *bp = netdev_priv(aux_priv->edev->net); > >- ida_free(&bnxt_aux_dev_ids, aux_priv->id); > kfree(aux_priv->edev->ulp_tbl); >- bp->edev = NULL; >+ bp->edev[aux_priv->id] = NULL; > kfree(aux_priv->edev); > kfree(aux_priv); >- bp->aux_priv = NULL; >+ bp->aux_priv[aux_priv->id] = NULL; > } > >-void bnxt_rdma_aux_device_del(struct bnxt *bp) >+void bnxt_aux_devices_del(struct bnxt *bp) > { >- if (!bp->edev) >- return; >+ int idx; > >- auxiliary_device_delete(&bp->aux_priv->aux_dev); >+ mutex_lock(&bp->auxdev_lock); >+ for (idx = 0; idx < __BNXT_AUXDEV_MAX; idx++) { >+ if (bnxt_auxdev_is_active(bp, idx)) { >+ auxiliary_device_delete(&bp->aux_priv[idx]->aux_dev); >+ bnxt_auxdev_set_state(bp, idx, BNXT_ADEV_STATE_INIT); I can't really follow why you need to maintain a dangling device state ! Why not completely remove and free an aux dev and then on probe/add init and add from scratch, simplified patch #3 which prompted me to come and look back here. >+ } >+ } >+ mutex_unlock(&bp->auxdev_lock); > } > > static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) >@@ -451,83 +510,105 @@ static void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) > edev->bar0 = bp->bar0; > } > >-void bnxt_rdma_aux_device_add(struct bnxt *bp) >+void bnxt_aux_devices_add(struct bnxt *bp) > { > struct auxiliary_device *aux_dev; >- int rc; >- >- if (!bp->edev) >- return; >- >- aux_dev = &bp->aux_priv->aux_dev; >- rc = auxiliary_device_add(aux_dev); >- if (rc) { >- netdev_warn(bp->dev, "Failed to add auxiliary device for ROCE\n"); >- auxiliary_device_uninit(aux_dev); >- bp->flags &= ~BNXT_FLAG_ROCE_CAP; >+ int rc, idx; >+ >+ mutex_lock(&bp->auxdev_lock); >+ for (idx = 0; idx < __BNXT_AUXDEV_MAX; idx++) { >+ if (bnxt_auxdev_is_init(bp, idx)) { >+ aux_dev = &bp->aux_priv[idx]->aux_dev; >+ rc = auxiliary_device_add(aux_dev); >+ if (rc) { >+ netdev_warn(bp->dev, "Failed to add auxiliary device for ROCE\n"); >+ auxiliary_device_uninit(aux_dev); >+ if (idx == BNXT_AUXDEV_RDMA) >+ bp->flags &= ~BNXT_FLAG_ROCE_CAP; >+ continue; >+ } >+ bnxt_auxdev_set_state(bp, idx, BNXT_ADEV_STATE_ADD); >+ } > } >+ mutex_unlock(&bp->auxdev_lock); > } > >-void bnxt_rdma_aux_device_init(struct bnxt *bp) >+void bnxt_aux_devices_init(struct bnxt *bp) > { > struct auxiliary_device *aux_dev; > struct bnxt_aux_priv *aux_priv; > struct bnxt_en_dev *edev; > struct bnxt_ulp *ulp; >- int rc; >+ int rc, idx; >+ >+ mutex_lock(&bp->auxdev_lock); >+ for (idx = 0; idx < __BNXT_AUXDEV_MAX; idx++) { >+ bnxt_auxdev_set_state(bp, idx, BNXT_ADEV_STATE_NONE); >+ >+ if (idx == BNXT_AUXDEV_RDMA && >+ !(bp->flags & BNXT_FLAG_ROCE_CAP)) >+ continue; >+ >+ aux_priv = kzalloc(sizeof(*aux_priv), GFP_KERNEL); >+ if (!aux_priv) >+ goto next_auxdev; >+ >+ aux_dev = &aux_priv->aux_dev; >+ aux_dev->id = bp->auxdev_id; >+ aux_dev->name = bnxt_aux_devices[idx].name; >+ aux_dev->dev.parent = &bp->pdev->dev; >+ aux_dev->dev.release = bnxt_aux_dev_release; >+ >+ rc = auxiliary_device_init(aux_dev); >+ if (rc) { >+ kfree(aux_priv); >+ goto next_auxdev; >+ } >+ bp->aux_priv[idx] = aux_priv; > >- if (!(bp->flags & BNXT_FLAG_ROCE_CAP)) >- return; >+ /* From this point, all cleanup will happen via the .release >+ * callback & any error unwinding will need to include a call >+ * to auxiliary_device_uninit. >+ */ >+ edev = kzalloc(sizeof(*edev), GFP_KERNEL); >+ if (!edev) >+ goto aux_dev_uninit; > >- aux_priv = kzalloc(sizeof(*bp->aux_priv), GFP_KERNEL); >- if (!aux_priv) >- goto exit; >+ aux_priv->edev = edev; >+ bnxt_set_edev_info(edev, bp); > >- aux_priv->id = ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL); >- if (aux_priv->id < 0) { >- netdev_warn(bp->dev, >- "ida alloc failed for ROCE auxiliary device\n"); >- kfree(aux_priv); >- goto exit; >- } >+ ulp = kzalloc(sizeof(*ulp), GFP_KERNEL); >+ if (!ulp) >+ goto aux_dev_uninit; > >- aux_dev = &aux_priv->aux_dev; >- aux_dev->id = aux_priv->id; >- aux_dev->name = "rdma"; >- aux_dev->dev.parent = &bp->pdev->dev; >- aux_dev->dev.release = bnxt_aux_dev_release; >+ edev->ulp_tbl = ulp; >+ bp->edev[idx] = edev; >+ if (idx == BNXT_AUXDEV_RDMA) >+ bp->ulp_num_msix_want = bnxt_set_dflt_ulp_msix(bp); >+ aux_priv->id = idx; >+ bnxt_auxdev_set_state(bp, idx, BNXT_ADEV_STATE_INIT); > >- rc = auxiliary_device_init(aux_dev); >- if (rc) { >- ida_free(&bnxt_aux_dev_ids, aux_priv->id); >- kfree(aux_priv); >- goto exit; >+ continue; >+aux_dev_uninit: >+ auxiliary_device_uninit(aux_dev); >+next_auxdev: >+ if (idx == BNXT_AUXDEV_RDMA) >+ bp->flags &= ~BNXT_FLAG_ROCE_CAP; > } >- bp->aux_priv = aux_priv; >- >- /* From this point, all cleanup will happen via the .release callback & >- * any error unwinding will need to include a call to >- * auxiliary_device_uninit. >- */ >- edev = kzalloc(sizeof(*edev), GFP_KERNEL); >- if (!edev) >- goto aux_dev_uninit; >- >- aux_priv->edev = edev; >- >- ulp = kzalloc(sizeof(*ulp), GFP_KERNEL); >- if (!ulp) >- goto aux_dev_uninit; >+ mutex_unlock(&bp->auxdev_lock); >+} > >- edev->ulp_tbl = ulp; >- bp->edev = edev; >- bnxt_set_edev_info(edev, bp); >- bp->ulp_num_msix_want = bnxt_set_dflt_ulp_msix(bp); >+int bnxt_auxdev_id_alloc(struct bnxt *bp) >+{ >+ bp->auxdev_id = ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL); >+ if (bp->auxdev_id < 0) >+ return bp->auxdev_id; > >- return; >+ return 0; >+} > >-aux_dev_uninit: >- auxiliary_device_uninit(aux_dev); >-exit: >- bp->flags &= ~BNXT_FLAG_ROCE_CAP; >+void bnxt_auxdev_id_free(struct bnxt *bp, int id) >+{ >+ if (bp->auxdev_id >= 0) >+ ida_free(&bnxt_aux_dev_ids, id); > } >diff --git a/include/linux/bnxt/ulp.h b/include/linux/bnxt/ulp.h >index 3c5b8a53f715..1a4643c46f86 100644 >--- a/include/linux/bnxt/ulp.h >+++ b/include/linux/bnxt/ulp.h >@@ -10,6 +10,8 @@ > #ifndef BNXT_ULP_H > #define BNXT_ULP_H > >+#include >+ > #define BNXT_MIN_ROCE_CP_RINGS 2 > #define BNXT_MIN_ROCE_STAT_CTXS 1 > >@@ -20,6 +22,17 @@ > struct hwrm_async_event_cmpl; > struct bnxt; > >+enum bnxt_auxdev_type { >+ BNXT_AUXDEV_RDMA = 0, >+ __BNXT_AUXDEV_MAX >+}; >+ >+struct bnxt_aux_priv { >+ struct auxiliary_device aux_dev; >+ struct bnxt_en_dev *edev; >+ int id; >+}; >+ > struct bnxt_msix_entry { > u32 vector; > u32 ring_idx; >@@ -110,19 +123,21 @@ void bnxt_set_ulp_stat_ctxs(struct bnxt *bp, int num_ctxs); > int bnxt_get_ulp_stat_ctxs_in_use(struct bnxt *bp); > void bnxt_set_dflt_ulp_stat_ctxs(struct bnxt *bp); > void bnxt_ulp_stop(struct bnxt *bp); >-void bnxt_ulp_start(struct bnxt *bp, int err); >+void bnxt_ulp_start(struct bnxt *bp); > void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs); > void bnxt_ulp_irq_stop(struct bnxt *bp); > void bnxt_ulp_irq_restart(struct bnxt *bp, int err); > void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl); >-void bnxt_rdma_aux_device_uninit(struct bnxt *bp); >-void bnxt_rdma_aux_device_del(struct bnxt *bp); >-void bnxt_rdma_aux_device_add(struct bnxt *bp); >-void bnxt_rdma_aux_device_init(struct bnxt *bp); >+void bnxt_aux_devices_uninit(struct bnxt *bp); >+void bnxt_aux_devices_del(struct bnxt *bp); >+void bnxt_aux_devices_add(struct bnxt *bp); >+void bnxt_aux_devices_init(struct bnxt *bp); > int bnxt_register_dev(struct bnxt_en_dev *edev, struct bnxt_ulp_ops *ulp_ops, > void *handle); > void bnxt_unregister_dev(struct bnxt_en_dev *edev); > int bnxt_send_msg(struct bnxt_en_dev *edev, struct bnxt_fw_msg *fw_msg); > void bnxt_register_async_events(struct bnxt_en_dev *edev, > unsigned long *events_bmap, u16 max_id); >+int bnxt_auxdev_id_alloc(struct bnxt *bp); >+void bnxt_auxdev_id_free(struct bnxt *bp, int id); > #endif >-- >2.39.1 >