From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84B4AC169C4 for ; Thu, 31 Jan 2019 15:11:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6237A218AF for ; Thu, 31 Jan 2019 15:11:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726239AbfAaPLu (ORCPT ); Thu, 31 Jan 2019 10:11:50 -0500 Received: from mx1.redhat.com ([209.132.183.28]:5855 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725855AbfAaPLu (ORCPT ); Thu, 31 Jan 2019 10:11:50 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B4551A08F0; Thu, 31 Jan 2019 15:11:49 +0000 (UTC) Received: from redhat.com (unknown [10.20.6.236]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 38D0660152; Thu, 31 Jan 2019 15:11:47 +0000 (UTC) Date: Thu, 31 Jan 2019 10:11:45 -0500 From: Jerome Glisse To: Christoph Hellwig Cc: Jason Gunthorpe , Logan Gunthorpe , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Greg Kroah-Hartman , "Rafael J . Wysocki" , Bjorn Helgaas , Christian Koenig , Felix Kuehling , "linux-pci@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , Marek Szyprowski , Robin Murphy , Joerg Roedel , "iommu@lists.linux-foundation.org" Subject: Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma Message-ID: <20190131151145.GC4619@redhat.com> References: <20190129193250.GK10108@mellanox.com> <99c228c6-ef96-7594-cb43-78931966c75d@deltatee.com> <20190129205827.GM10108@mellanox.com> <20190130080208.GC29665@lst.de> <20190130174424.GA17080@mellanox.com> <20190130191946.GD17080@mellanox.com> <3793c115-2451-1479-29a9-04bed2831e4b@deltatee.com> <20190130204414.GH17080@mellanox.com> <20190131080501.GB26495@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20190131080501.GB26495@lst.de> User-Agent: Mutt/1.10.0 (2018-05-17) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 31 Jan 2019 15:11:50 +0000 (UTC) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Thu, Jan 31, 2019 at 09:05:01AM +0100, Christoph Hellwig wrote: > On Wed, Jan 30, 2019 at 08:44:20PM +0000, Jason Gunthorpe wrote: > > Not really, for MRs most drivers care about DMA addresses only. The > > only reason struct page ever gets involved is because it is part of > > the GUP, SGL and dma_map family of APIs. > > And the only way you get the DMA address is through the dma mapping > APIs. Which except for the little oddball dma_map_resource expect > a struct page in some form. And dma_map_resource isn't really up > to speed for full blown P2P. > > Now we could and maybe eventually should change all this. But that > is a pre-requisitive for doing anything more fancy, and not something > to be hacked around. > > > O_DIRECT seems to be the justification for struct page, but nobody is > > signing up to make O_DIRECT have the required special GUP/SGL/P2P flow > > that would be needed to *actually* make that work - so it really isn't > > a justification today. > > O_DIRECT is just the messenger. Anything using GUP will need a struct > page, which is all our interfaces that do I/O directly to user pages. I do not want to allow GUP to pin I/O space this would open a pandora box that we do not want to open at all. Many driver manage their IO space and if they get random pinning because some other kernel bits they never heard of starts to do GUP on their stuff it is gonna cause havoc. So far mmap of device file have always been special and it has been reflected to userspace in all the instance i know of (media and GPU). Pretending we can handle them like any other vma is a lie because they were never designed that way in the first place and it would be disruptive to all those driver. Minimum disruption with minimun changes is what we should aim for and is what i am trying to do with this patchset. Using struct page and allowing GUP would mean rewritting huge chunk of GPU drivers (pretty much rewritting their whole memory management) with no benefit at the end. When something is special it is better to leave it that way. Cheers, Jérôme