From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5F5B0C021B2 for ; Tue, 25 Feb 2025 07:55:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 027C960851; Tue, 25 Feb 2025 07:55:40 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id TXpn2dI2L2cj; Tue, 25 Feb 2025 07:55:39 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.142; helo=lists1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6359C60852 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1740470139; bh=ClBVpTw1OU80fijBaxSv2SgzPEBdGdTlk/Ze5pRbK44=; h=Date:From:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From; b=nS6VwcjZfkC3Lzjp4U3X1nIwXe6f2ve7DqEAYHYAXPkM60mioUDVpfJhrFidZazzC mvD0r1fB76Gl0Qi/g7a8rZZRgiSDgA8btniinL7+Qil82ClyqykGGMWXtpOlJVsX2I qVlvaVPVTeEq4IET7wdHCxjTsiaorxwoCm/y6wvlPJkWyNpK33GfYm3jB3rwss7fNF BsB4GxOZhk/rqgC7gOv5OCTyIEBD7+YqqTIG3kvl40bI02AMBntEESB1Y/7JPJk/oZ /VwucC6HzElOOuD28UNA0/jDIEl0HMJhSEIABeYusAXzj4O44LDN3mfogqEvZF2GJ/ q4OvnzEIK1oyw== Received: from lists1.osuosl.org (lists1.osuosl.org [140.211.166.142]) by smtp3.osuosl.org (Postfix) with ESMTP id 6359C60852; Tue, 25 Feb 2025 07:55:39 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists1.osuosl.org (Postfix) with ESMTP id E0AC82292 for ; Tue, 25 Feb 2025 07:55:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BD2A960852 for ; Tue, 25 Feb 2025 07:55:37 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id nD3baJcIADOX for ; Tue, 25 Feb 2025 07:55:37 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2604:1380:4641:c500::1; helo=dfw.source.kernel.org; envelope-from=leon@kernel.org; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org EFCD360851 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org EFCD360851 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by smtp3.osuosl.org (Postfix) with ESMTPS id EFCD360851 for ; Tue, 25 Feb 2025 07:55:36 +0000 (UTC) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id A96DE5C077B; Tue, 25 Feb 2025 07:54:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D08CDC4CEDD; Tue, 25 Feb 2025 07:55:34 +0000 (UTC) Date: Tue, 25 Feb 2025 09:55:30 +0200 From: Leon Romanovsky To: Tatyana Nikolova Cc: jgg@nvidia.com, intel-wired-lan@lists.osuosl.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, david.m.ertman@intel.com Message-ID: <20250225075530.GD53094@unreal> References: <20250225050428.2166-1-tatyana.e.nikolova@intel.com> <20250225050428.2166-2-tatyana.e.nikolova@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250225050428.2166-2-tatyana.e.nikolova@intel.com> X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740470135; bh=26kEWrHsJUAlh0ZBlcW151O8ytU26+XAmiyBTfJHjuE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=YVut6nzuR9giRKPXY4xVu2aIIY+V1CAxMbDfW7/hqsw0oDsfXkzJrGryRePr7QF8T FdAi1+SczvwO2Y2SGetPuPR4Oe/vHRGnB0A3KwBmfMGX1wiETbKfey9ZA0tP51z/pi cghhL6C68iNC7wx6YNTAMzWwfa++Z4aFCvANUYWS1IJ8wP5sNony4xqdLOatm+ZkZP FAbSF5nK1BmvCNZfoN5MvTs/7z3PZqhn+AT51q3uWxPZhmHqHNbKPBlb8oPb0usZg+ YGiOGZP89gvw42MIyYHsfI/RIVFHOBX7sMmjIkIyoyHlB/VFryur9z/0Hzwauyje4H pGZcwr/2rybpw== X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=YVut6nzu Subject: Re: [Intel-wired-lan] [iwl-next v4 1/1] iidc/ice/irdma: Update IDC to support multiple consumers X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" On Mon, Feb 24, 2025 at 11:04:28PM -0600, Tatyana Nikolova wrote: > From: Dave Ertman > > To support RDMA for E2000 product, the idpf driver will use the IDC > interface with the irdma auxiliary driver, thus becoming a second > consumer of it. This requires the IDC be updated to support multiple > consumers. The use of exported symbols no longer makes sense because it > will require all core drivers (ice/idpf) that can interface with irdma > auxiliary driver to be loaded even if hardware is not present for those > drivers. In auxiliary bus world, the code drivers (ice/idpf) need to created auxiliary devices only if specific device present. That auxiliary device will trigger the load of specific module (irdma in our case). EXPORT_SYMBOL won't trigger load of irdma driver, but the opposite is true, load of irdma will trigger load of ice/idpf drivers (depends on their exported symbol). > > To address this, implement an ops struct that will be universal set of > naked function pointers that will be populated by each core driver for > the irdma auxiliary driver to call. No, we worked very hard to make proper HW discovery and driver autoload, let's not return back. For now, it is no-go. <...> > +/* Following APIs are implemented by core PCI driver */ > +struct idc_rdma_core_ops { > + int (*vc_send_sync)(struct idc_rdma_core_dev_info *cdev_info, u8 *msg, > + u16 len, u8 *recv_msg, u16 *recv_len); > + int (*vc_queue_vec_map_unmap)(struct idc_rdma_core_dev_info *cdev_info, > + struct idc_rdma_qvlist_info *qvl_info, > + bool map); > + /* vport_dev_ctrl is for RDMA CORE driver to indicate it is either ready > + * for individual vport aux devices, or it is leaving the state where it > + * can support vports and they need to be downed > + */ > + int (*vport_dev_ctrl)(struct idc_rdma_core_dev_info *cdev_info, > + bool up); > + int (*request_reset)(struct idc_rdma_core_dev_info *cdev_info, > + enum idc_rdma_reset_type reset_type); > +}; Core driver can call to callbacks in irdma, like you already have for irdma_iidc_event_handler(), but all calls from irdma to core driver must be through exported symbols. It gives us race-free world in whole driver except one very specific place (irdma_iidc_event_handler). Thanks From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6FDA025A32D; Tue, 25 Feb 2025 07:55:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740470135; cv=none; b=ti4gLWdh3cWz6vjq0/hRbqQZasHws0yKylFKfFWbLdsuFF8CLSFWQJyJAPSxo3+tJHewvcT+zy6hNVQwD//5g/G13eT+GvE8jhSX6MwF5ENNVy3vZU/jQalhPzYMTh9h/TiKNJicPypR3kSggTdZNrYH/F0vQ2PwExMLUXjNqKU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740470135; c=relaxed/simple; bh=26kEWrHsJUAlh0ZBlcW151O8ytU26+XAmiyBTfJHjuE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZuqY8+7esGQESPiDyfCgatWOY9yzoDR+R4Ec2dpdDlJYclnlc2cdvnX7aXcgc+62ih29tyjMkuqtyPmxwsJ7BZQSSCJlIsqXK1jGmYf4JmoLibHF+bh7DbS1I/sT7I0uTCw5T4n5uojoEBNO7Ups7nFfPCWHnQm48q7f9Cpdrjg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YVut6nzu; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YVut6nzu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D08CDC4CEDD; Tue, 25 Feb 2025 07:55:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740470135; bh=26kEWrHsJUAlh0ZBlcW151O8ytU26+XAmiyBTfJHjuE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=YVut6nzuR9giRKPXY4xVu2aIIY+V1CAxMbDfW7/hqsw0oDsfXkzJrGryRePr7QF8T FdAi1+SczvwO2Y2SGetPuPR4Oe/vHRGnB0A3KwBmfMGX1wiETbKfey9ZA0tP51z/pi cghhL6C68iNC7wx6YNTAMzWwfa++Z4aFCvANUYWS1IJ8wP5sNony4xqdLOatm+ZkZP FAbSF5nK1BmvCNZfoN5MvTs/7z3PZqhn+AT51q3uWxPZhmHqHNbKPBlb8oPb0usZg+ YGiOGZP89gvw42MIyYHsfI/RIVFHOBX7sMmjIkIyoyHlB/VFryur9z/0Hzwauyje4H pGZcwr/2rybpw== Date: Tue, 25 Feb 2025 09:55:30 +0200 From: Leon Romanovsky To: Tatyana Nikolova Cc: jgg@nvidia.com, intel-wired-lan@lists.osuosl.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, david.m.ertman@intel.com Subject: Re: [iwl-next v4 1/1] iidc/ice/irdma: Update IDC to support multiple consumers Message-ID: <20250225075530.GD53094@unreal> References: <20250225050428.2166-1-tatyana.e.nikolova@intel.com> <20250225050428.2166-2-tatyana.e.nikolova@intel.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250225050428.2166-2-tatyana.e.nikolova@intel.com> On Mon, Feb 24, 2025 at 11:04:28PM -0600, Tatyana Nikolova wrote: > From: Dave Ertman > > To support RDMA for E2000 product, the idpf driver will use the IDC > interface with the irdma auxiliary driver, thus becoming a second > consumer of it. This requires the IDC be updated to support multiple > consumers. The use of exported symbols no longer makes sense because it > will require all core drivers (ice/idpf) that can interface with irdma > auxiliary driver to be loaded even if hardware is not present for those > drivers. In auxiliary bus world, the code drivers (ice/idpf) need to created auxiliary devices only if specific device present. That auxiliary device will trigger the load of specific module (irdma in our case). EXPORT_SYMBOL won't trigger load of irdma driver, but the opposite is true, load of irdma will trigger load of ice/idpf drivers (depends on their exported symbol). > > To address this, implement an ops struct that will be universal set of > naked function pointers that will be populated by each core driver for > the irdma auxiliary driver to call. No, we worked very hard to make proper HW discovery and driver autoload, let's not return back. For now, it is no-go. <...> > +/* Following APIs are implemented by core PCI driver */ > +struct idc_rdma_core_ops { > + int (*vc_send_sync)(struct idc_rdma_core_dev_info *cdev_info, u8 *msg, > + u16 len, u8 *recv_msg, u16 *recv_len); > + int (*vc_queue_vec_map_unmap)(struct idc_rdma_core_dev_info *cdev_info, > + struct idc_rdma_qvlist_info *qvl_info, > + bool map); > + /* vport_dev_ctrl is for RDMA CORE driver to indicate it is either ready > + * for individual vport aux devices, or it is leaving the state where it > + * can support vports and they need to be downed > + */ > + int (*vport_dev_ctrl)(struct idc_rdma_core_dev_info *cdev_info, > + bool up); > + int (*request_reset)(struct idc_rdma_core_dev_info *cdev_info, > + enum idc_rdma_reset_type reset_type); > +}; Core driver can call to callbacks in irdma, like you already have for irdma_iidc_event_handler(), but all calls from irdma to core driver must be through exported symbols. It gives us race-free world in whole driver except one very specific place (irdma_iidc_event_handler). Thanks