From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A42091C6AF for ; Mon, 20 Nov 2023 14:38:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=collabora.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=collabora.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=collabora.com header.i=@collabora.com header.b="JaJd9Gl4" Received: from localhost (cola.collaboradmins.com [195.201.22.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbrezillon) by madras.collabora.co.uk (Postfix) with ESMTPSA id 1D0566605638; Mon, 20 Nov 2023 14:38:42 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1700491122; bh=eMHNEN+Mufog8F3hIBLGTVt7miTA+TkkEXxArV+ovkE=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=JaJd9Gl4v0TcApHa3lreIU2xf53sao6+oKLUD7h+cT2krHa7QqHGvUz7noNNthXTx tYiIxT+jeNBma1/qV0L6hDmltVKs4nUgsN7r5XpAliy0xObkz7PVGP3ThyquYQLrED x0dbSdQwB6q1cfreP6fptfjeU8lXVX+JU0RmgblO1dq6qcDXWFgaz+LYaIdlzPBBHG 5OnSILufdZ+TrxWXKT/3emEuxD+6LBl7IKxOe2aQjNoI0bBZit3jeJ8qzfROzeFbhi 1vcjvdYhaAIEmmqo5FWUI4MpBWY/Qp90H4dEdTJ0nam2w+AAjl9iwxRc4hl3ZXbxia VrecUCT4PVtPA== Date: Mon, 20 Nov 2023 15:38:38 +0100 From: Boris Brezillon To: Jason Gunthorpe Cc: Joerg Roedel , iommu@lists.linux.dev, Will Deacon , Robin Murphy , linux-arm-kernel@lists.infradead.org, Rob Clark , Gaurav Kohli , Steven Price Subject: Re: [PATCH v2 0/2] iommu: Allow passing custom allocators to pgtable drivers Message-ID: <20231120153838.2166e7b8@collabora.com> In-Reply-To: <20231120140425.GA10140@ziepe.ca> References: <20231110094352.565347-1-boris.brezillon@collabora.com> <20231110151428.GJ4634@ziepe.ca> <20231110164809.270f82bc@collabora.com> <20231110161229.GA462657@nvidia.com> <20231110201652.629b7228@collabora.com> <20231110194215.GR4488@nvidia.com> <20231113101103.1cc05c8c@collabora.com> <20231120140425.GA10140@ziepe.ca> Organization: Collabora X-Mailer: Claws Mail 4.1.1 (GTK 3.24.38; x86_64-redhat-linux-gnu) Precedence: bulk X-Mailing-List: iommu@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Hi Jason, On Mon, 20 Nov 2023 10:04:25 -0400 Jason Gunthorpe wrote: > On Tue, Nov 14, 2023 at 12:27:48PM -0400, Jason Gunthorpe wrote: > > > > Anyway, given you already thought it through, can I ask you to provide > > > a preliminary implementation for this IOVA range mechanism so I can > > > play with it and adjust panthor accordingly. And if you don't have the > > > time, can you at least give me extra details about the implementation > > > you had in mind, so I don't have to guess and come back with something > > > that's not matching what you had in mind. > > > > Oh, I don't know if I can manage patches in any reasonable time frame, > > though I think it is pretty straightforward really: > > > > - Patch to introduce some 'struct iopte_page' (see struct slab) > > - Adjust io pagetable implementations to consume it > > - Do RCU freeing of iopte_page > > - Add a reserve/unreserve io page table ops > > - Implement reserve/unresereve in arm by manipulating a new refcount > > in iopte_page. Rely on RCU to protect the derefs > > - Modify iommufd to call reserve/unreserve around areas attachment > > to have an intree user. > > > > Some of this is a bit interesting, like reserving probably will > > ideally want to invoke the batch allocator for efficiency which means > > computing the number of radix levels required to fully populate the > > current empty level - that should be general code somehow > > At LPC there was quite a lot if interest in improving the io page > table stuff to work better. Based on that I'm even more against adding > an external allocator at this time. :\ I'm sure this has been discussed with the IOMMU maintainers, but I'd really like to hear it from them if you don't mind. Especially since, as I mentioned, the custom allocator idea comes from Robin, not me... > > I may try to write a sketch of something but I'm not sure when.. Yeah, that's the main problem here. Making panthor (the GPU driver for newer mali GPUs) depend on some new stuff with no clear estimation of when this work will actually be conducted is not something I want to do. We've already had to wait a few months for some DRM deps to land (and those were actively being worked on), and I was hoping the fact this custom allocator idea originated from Robin (who's maintaining the subsystem with others), plus the fact there was no real push back on v1, was a sign this was good to go. If the final call is to reject the custom allocator approach, and there's no updates on your generic solution in the next couple weeks, I'll probably copy the pgtable code in panthor so I can implement my own stuff, and later switch back to the generic io-pgtable implementation when things have settled down. Nothing against you or your solution (I actually commit to move to the new approach when it's ready), but we have different priorities, and I can't really delay the driver merging by several months unless there's a very good reason to do it. Best Regards, Boris From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98FDFC197A0 for ; Mon, 20 Nov 2023 14:39:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NKNzfjaOxy0jetRT3XHqsNDuF67j04cd1ZZ9SSxkWqs=; b=V9c6PdRDITJZXn mpL8v/BzDPpiP0D0AUPbRlFmUtqtJQKOPZjNnnzSZvJxQv61ou4rrhm1XfJO8jRmbxrd//pFaG2nV /LhrUq+4pEUMnMvwlRUiQw5gsHl71RNR7eBTPwXh1h4urE3fT3kjeHAO06Vz4QKSKQAGLrNgnzpSl H+skO7xjrQ3O4Pjr2d5wDOb0sue5ERqgcCo0Pl1hE/gl6OXgwjwgW/mgc16nN4NMW+L5jgGbfVt7Y oGKMQ0GpawLf6IZ2oYZhWlr4KpnqTi+xmYSpn+Qq4ZCHda7yoVHMTsjHIvjlDLIqnkqsNuMGmXnUu GZB2HmYpFKPx73Qu0MPw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r55QH-00CJUX-0k; Mon, 20 Nov 2023 14:38:49 +0000 Received: from madras.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e5ab]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r55QD-00CJQm-2Y for linux-arm-kernel@lists.infradead.org; Mon, 20 Nov 2023 14:38:47 +0000 Received: from localhost (cola.collaboradmins.com [195.201.22.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbrezillon) by madras.collabora.co.uk (Postfix) with ESMTPSA id 1D0566605638; Mon, 20 Nov 2023 14:38:42 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1700491122; bh=eMHNEN+Mufog8F3hIBLGTVt7miTA+TkkEXxArV+ovkE=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=JaJd9Gl4v0TcApHa3lreIU2xf53sao6+oKLUD7h+cT2krHa7QqHGvUz7noNNthXTx tYiIxT+jeNBma1/qV0L6hDmltVKs4nUgsN7r5XpAliy0xObkz7PVGP3ThyquYQLrED x0dbSdQwB6q1cfreP6fptfjeU8lXVX+JU0RmgblO1dq6qcDXWFgaz+LYaIdlzPBBHG 5OnSILufdZ+TrxWXKT/3emEuxD+6LBl7IKxOe2aQjNoI0bBZit3jeJ8qzfROzeFbhi 1vcjvdYhaAIEmmqo5FWUI4MpBWY/Qp90H4dEdTJ0nam2w+AAjl9iwxRc4hl3ZXbxia VrecUCT4PVtPA== Date: Mon, 20 Nov 2023 15:38:38 +0100 From: Boris Brezillon To: Jason Gunthorpe Cc: Joerg Roedel , iommu@lists.linux.dev, Will Deacon , Robin Murphy , linux-arm-kernel@lists.infradead.org, Rob Clark , Gaurav Kohli , Steven Price Subject: Re: [PATCH v2 0/2] iommu: Allow passing custom allocators to pgtable drivers Message-ID: <20231120153838.2166e7b8@collabora.com> In-Reply-To: <20231120140425.GA10140@ziepe.ca> References: <20231110094352.565347-1-boris.brezillon@collabora.com> <20231110151428.GJ4634@ziepe.ca> <20231110164809.270f82bc@collabora.com> <20231110161229.GA462657@nvidia.com> <20231110201652.629b7228@collabora.com> <20231110194215.GR4488@nvidia.com> <20231113101103.1cc05c8c@collabora.com> <20231120140425.GA10140@ziepe.ca> Organization: Collabora X-Mailer: Claws Mail 4.1.1 (GTK 3.24.38; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231120_063846_158198_068A564F X-CRM114-Status: GOOD ( 36.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Jason, On Mon, 20 Nov 2023 10:04:25 -0400 Jason Gunthorpe wrote: > On Tue, Nov 14, 2023 at 12:27:48PM -0400, Jason Gunthorpe wrote: > > > > Anyway, given you already thought it through, can I ask you to provide > > > a preliminary implementation for this IOVA range mechanism so I can > > > play with it and adjust panthor accordingly. And if you don't have the > > > time, can you at least give me extra details about the implementation > > > you had in mind, so I don't have to guess and come back with something > > > that's not matching what you had in mind. > > > > Oh, I don't know if I can manage patches in any reasonable time frame, > > though I think it is pretty straightforward really: > > > > - Patch to introduce some 'struct iopte_page' (see struct slab) > > - Adjust io pagetable implementations to consume it > > - Do RCU freeing of iopte_page > > - Add a reserve/unreserve io page table ops > > - Implement reserve/unresereve in arm by manipulating a new refcount > > in iopte_page. Rely on RCU to protect the derefs > > - Modify iommufd to call reserve/unreserve around areas attachment > > to have an intree user. > > > > Some of this is a bit interesting, like reserving probably will > > ideally want to invoke the batch allocator for efficiency which means > > computing the number of radix levels required to fully populate the > > current empty level - that should be general code somehow > > At LPC there was quite a lot if interest in improving the io page > table stuff to work better. Based on that I'm even more against adding > an external allocator at this time. :\ I'm sure this has been discussed with the IOMMU maintainers, but I'd really like to hear it from them if you don't mind. Especially since, as I mentioned, the custom allocator idea comes from Robin, not me... > > I may try to write a sketch of something but I'm not sure when.. Yeah, that's the main problem here. Making panthor (the GPU driver for newer mali GPUs) depend on some new stuff with no clear estimation of when this work will actually be conducted is not something I want to do. We've already had to wait a few months for some DRM deps to land (and those were actively being worked on), and I was hoping the fact this custom allocator idea originated from Robin (who's maintaining the subsystem with others), plus the fact there was no real push back on v1, was a sign this was good to go. If the final call is to reject the custom allocator approach, and there's no updates on your generic solution in the next couple weeks, I'll probably copy the pgtable code in panthor so I can implement my own stuff, and later switch back to the generic io-pgtable implementation when things have settled down. Nothing against you or your solution (I actually commit to move to the new approach when it's ready), but we have different priorities, and I can't really delay the driver merging by several months unless there's a very good reason to do it. Best Regards, Boris _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel