From mboxrd@z Thu Jan 1 00:00:00 1970 From: Serguei Sagalovitch Subject: Re: Enabling peer to peer device transactions for PCIe devices Date: Thu, 24 Nov 2016 12:00:14 -0500 Message-ID: <87e6cfce-c65d-2d9f-a3c6-e4224cb94b2c@amd.com> References: <20161123190515.GA12146@obsidianresearch.com> <7bc38037-b6ab-943f-59db-6280e16901ab@amd.com> <20161123193228.GC12146@obsidianresearch.com> <20161123203332.GA15062@obsidianresearch.com> <20161123215510.GA16311@obsidianresearch.com> <20161123232503.GA13965@obsidianresearch.com> <20161124162620.GC20818@obsidianresearch.com> Mime-Version: 1.0 Content-Type: text/plain; charset="windows-1252"; Format="flowed" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20161124162620.GC20818-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: Jason Gunthorpe , =?UTF-8?Q?Christian_K=c3=b6nig?= Cc: Haggai Eran , "Bridgman, John" , "linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org" , "Kuehling, Felix" , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org" , "Blinzer, Paul" , "Suthikulpanit, Suravee" , "linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "Deucher, Alexander" , "Sander, Ben" , "Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" List-Id: linux-rdma@vger.kernel.org On 2016-11-24 11:26 AM, Jason Gunthorpe wrote: > On Thu, Nov 24, 2016 at 10:45:18AM +0100, Christian K=F6nig wrote: >> Am 24.11.2016 um 00:25 schrieb Jason Gunthorpe: >>> There is certainly nothing about the hardware that cares >>> about ZONE_DEVICE vs System memory. >> Well that is clearly not so simple. When your ZONE_DEVICE pages describe= a >> PCI BAR and another PCI device initiates a DMA to this address the DMA >> subsystem must be able to check if the interconnection really works. > I said the hardware doesn't care.. You are right, we still have an > outstanding problem in Linux of how to generically DMA map a P2P > address - which is a different issue from getting the P2P address from > a __user pointer... > > Jason I agreed but the problem is that one issue immediately introduce another = one to solve and so on (if we do not want to cut corners). I would think that a lot of them interconnected because the way how one problem could be solved may impact solution for another. btw: about "DMA map a p2p address": Right now to enable p2p between = devices it is required/recommended to disable iommu support (e.g. intel iommu = driver has special logic for graphics and comment "Reserve all PCI MMIO to avoid peer-to-peer access").