From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49DB9C38145 for ; Wed, 7 Sep 2022 15:23:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229601AbiIGPXn (ORCPT ); Wed, 7 Sep 2022 11:23:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230424AbiIGPXl (ORCPT ); Wed, 7 Sep 2022 11:23:41 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D11A7B69CA; Wed, 7 Sep 2022 08:23:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Transfer-Encoding :Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Sender:Reply-To:Content-ID:Content-Description; bh=BcNIY2A5Ly0NVEPOgKCjZNwABZLi+n/SzSXVZWXlSVk=; b=xIWn7jJIvte4o9iMUtkRREt3f0 BRgZUmNIzYgCa5tmtZ9nBKViJYapzQV1p6WxYLbAQaiZCiKu5JWRHGktRBxzU751H3L8t25lEqK8a F8QDMVWX984jCkXgYLIgl9uNv/2gEHsTwiYarCe5wlvIypBCIOjnQ7l+mp8cbz0MbRRJMDlo3gTsR qvNf77EoZcMP/Dl8fiq4Q8pueQQGY4xUgPlNWdfp/IO+AVtBvvW/Vi6H1NeI6IU0/J9D05KHtbJ7t 9UzSAffrFHiq5pTccuTCB3MFIBAm/GOUGQqHM6hp7zfPXb2j/JffgczYTL8XIJdLKBuE/QoeC1SXW BeJWkEoA==; Received: from hch by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oVwtk-007JlI-GO; Wed, 07 Sep 2022 15:23:28 +0000 Date: Wed, 7 Sep 2022 08:23:28 -0700 From: Christoph Hellwig To: Christian =?iso-8859-1?Q?K=F6nig?= Cc: Christoph Hellwig , Jason Gunthorpe , Alex Williamson , Cornelia Huck , dri-devel@lists.freedesktop.org, kvm@vger.kernel.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, Sumit Semwal , Daniel Vetter , Leon Romanovsky , linux-rdma@vger.kernel.org, Maor Gottlieb , Oded Gabbay Subject: Re: [PATCH v2 4/4] vfio/pci: Allow MMIO regions to be exported through dma-buf Message-ID: References: <0-v2-472615b3877e+28f7-vfio_dma_buf_jgg@nvidia.com> <4-v2-472615b3877e+28f7-vfio_dma_buf_jgg@nvidia.com> <58d6e892-82df-7aa7-4798-9e5da7c634ad@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <58d6e892-82df-7aa7-4798-9e5da7c634ad@amd.com> X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On Wed, Sep 07, 2022 at 05:08:37PM +0200, Christian König wrote: > The key point is that you can't do any CPU fallback with this as long as the > CPU wouldn't do exactly the same thing as the original hardware device. E.g. > not write combine nor do any fully page copies etc... That is true for MMIO in general. You need to use the mmio accessors, and even then it needs to match what the hardware expects > This is not even a memory write, but rather just some trigger event. That's > essentially the background why I think having struct pages and sg_tables > doesn't make any sense at all for this use case. Well, Jasons patch uses a sg_table, just in a very broken way. > > > Would you mind if I start to tackle this problem? > > Yes, I've been waiting forever for someone to tacke how the scatterlist > > is abused in dma-buf.. > > How about we separate the scatterlist into page and DMA address container? As said before that is fundamentally the right thing to do. It just takes a lot of work. I think on the dma mapping side we're finally getting ready for it now that arm uses the common dma-direct code, although getting rid of the arm iommu ops first would be even better. Note that this only solves the problem of needing the pages to hold the physical address. The other thing the pages are used for in p2p is that (through the pgmap) it points to the owning device of the p2p memory and thus allows figuring out how it needs to be accessed.