From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58D53634 for ; Sat, 19 Aug 2023 01:34:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20314C433C7; Sat, 19 Aug 2023 01:34:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1692408873; bh=zMC51/B321AR7uz7BcPk/N4ZRLIynOk27Ofz2Fjx84Q=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=OB/Xi1SMMqF9NszRvd9f8dbmv/Ht2/kL3y1D0v4SGiyNka+GXzvhM+czcJUwhbujs Ii6A1ArUFfhRDbJ09DtDZpdpkMknycN0f9Rcq5cZA/DGqtp0VSO0I3xzDB+acWdMHI umG7Gso3ojkmu3z2vfgOPD6Iv827ZB7y5RDT4JjeiNQ1HT0gmG/vb/I63cPhfuN75r HZspwXeCz2B87pKRErOaGq0G7nXsiEUm9rq6TrY0ZER+/Uq2/tJNuuHQ3jMbkQHtYp qiodt7PxUZiJ+/KTtBdSUZ3ooFzkTCe9xuIlJMN56H8rge+MIq++duV6AqtwFaMwgY gg/A36fobgdWQ== Message-ID: <7cac1a2d-6184-7cd6-116c-e2d80c502db5@kernel.org> Date: Fri, 18 Aug 2023 19:34:32 -0600 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Subject: Re: [RFC PATCH v2 02/11] netdev: implement netlink api to bind dma-buf to netdevice Content-Language: en-US To: Mina Almasry , Jakub Kicinski , Praveen Kaligineedi Cc: Willem de Bruijn , netdev@vger.kernel.org, Eric Dumazet , Paolo Abeni , Jesper Dangaard Brouer , Ilias Apalodimas , Magnus Karlsson , sdf@google.com, Willem de Bruijn , Kaiyuan Zhang References: <20230810015751.3297321-1-almasrymina@google.com> <20230810015751.3297321-3-almasrymina@google.com> <7dd4f5b0-0edf-391b-c8b4-3fa82046ab7c@kernel.org> <20230815171638.4c057dcd@kernel.org> <64dcf5834c4c8_23f1f8294fa@willemb.c.googlers.com.notmuch> <20230817190957.571ab350@kernel.org> From: David Ahern In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 8/18/23 3:52 PM, Mina Almasry wrote: > The sticking points are: > 1. From David: this proposal doesn't give an application the ability > to flush an rx queue, which means that we have to rely on a driver > reset that affects all queues to refill the rx queue buffers. Generically, the design needs to be able to flush (or invalidate) all references to the dma-buf once the process no longer "owns" it. > 2. From Jakub: the uAPI and implementation here needs to be in line > with his general direction & extensible to apply to existing use cases > `ethtool -L/-G`, etc. I think this is a bit more open ended given the openness of the netdev netlink API. i.e., managing a H/W queue (create, delete, stop / flush, associate a page_pool) could be done through this API. > > AFAIU this is what I need to do in the next version: > > 1. The uAPI will be changed such that it will either re-configure an > existing queue to bind it to the dma-buf, or allocate a new queue > bound to the dma-buf (not sure which is better at the moment). Either 1. API to manage a page-pool (create, delete, update). 2. API to add and remove a dma-buf (or host memory buffer) with a page-pool. Remove may take time to flush references pushed to hardware so this would be asynchronous. 3. Create a queue or use an existing queue id and associate a page-pool with it. > way, the configuration will take place immediately, and not rely on an > entire driver reset to actuate the change. yes > > 2. The uAPI will be changed such that if the netlink socket is closed, > or the process dies, the rx queue will be unbound from the dma-buf or > the rx queue will be freed entirely (again, not sure which is better I think those are separate actions. But, if the queue was created by and referenced by a process, then closing an fd means it should be freed. > at the moment). The configuration will take place immediately without > relying on a driver reset. yes on the reset. > > 3. I will add 4 new net_device_ops that Jakub specified: > queue_mem_alloc/free(), and queue_start/stop(). > > 4. The uAPI mentioned in #1 will use the new net_device_ops to > allocate or reconfigure a queue attached to the provided dma-buf. > > Does this sound roughly reasonable here? > > AFAICT the only technical difficulty is that I'm not sure it's > feasible for a driver to start or stop 1 rx-queue without triggering a > full driver reset. The (2) drivers I looked at both do a full reset to > change any queue configuration. I'll investigate.