From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Thu, 1 Mar 2018 16:18:18 -0500 From: Jerome Glisse To: Logan Gunthorpe Cc: benh@au1.ibm.com, Dan Williams , Linux Kernel Mailing List , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma , linux-nvdimm , linux-block@vger.kernel.org, Stephen Bates , Christoph Hellwig , Jens Axboe , Keith Busch , Sagi Grimberg , Bjorn Helgaas , Jason Gunthorpe , Max Gurtovoy , Alex Williamson , Oliver OHalloran Subject: Re: [PATCH v2 00/10] Copy Offload in NVMe Fabrics with P2P PCI Memory Message-ID: <20180301211817.GC6742@redhat.com> References: <20180228234006.21093-1-logang@deltatee.com> <1519876489.4592.3.camel@kernel.crashing.org> <1519876569.4592.4.camel@au1.ibm.com> <1519938210.4592.30.camel@au1.ibm.com> <1de70207-40ce-29f0-6093-337112852475@deltatee.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 In-Reply-To: <1de70207-40ce-29f0-6093-337112852475@deltatee.com> List-ID: On Thu, Mar 01, 2018 at 02:11:34PM -0700, Logan Gunthorpe wrote: > > > On 01/03/18 02:03 PM, Benjamin Herrenschmidt wrote: > > However, what happens if anything calls page_address() on them ? Some > > DMA ops do that for example, or some devices might ... > > Although we could probably work around it with some pain, we rely on > page_address() and virt_to_phys(), etc to work on these pages. So on x86, > yes, it makes it into the linear mapping. This is pretty easy to do with HMM: unsigned long hmm_page_to_phys_pfn(struct page *page) { struct hmm_devmem *devmem; unsigned long ppfn; /* Sanity test maybe BUG_ON() */ if (!is_device_private_page(page)) return -1UL; devmem = page->pgmap->data; ppfn = page_to_page(page) - devmem->pfn_first; return ppfn + devmem->device_phys_base_pfn; } Note that last field does not exist in today HMM because i did not need such helper so far but this can be added. Cheers, J�r�me From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E9E72224E691C for ; Thu, 1 Mar 2018 13:12:14 -0800 (PST) Date: Thu, 1 Mar 2018 16:18:18 -0500 From: Jerome Glisse Subject: Re: [PATCH v2 00/10] Copy Offload in NVMe Fabrics with P2P PCI Memory Message-ID: <20180301211817.GC6742@redhat.com> References: <20180228234006.21093-1-logang@deltatee.com> <1519876489.4592.3.camel@kernel.crashing.org> <1519876569.4592.4.camel@au1.ibm.com> <1519938210.4592.30.camel@au1.ibm.com> <1de70207-40ce-29f0-6093-337112852475@deltatee.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1de70207-40ce-29f0-6093-337112852475@deltatee.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Logan Gunthorpe Cc: Jens Axboe , Keith Busch , Oliver OHalloran , benh@au1.ibm.com, linux-nvdimm , linux-rdma , linux-pci@vger.kernel.org, Linux Kernel Mailing List , linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, Alex Williamson , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-ID: On Thu, Mar 01, 2018 at 02:11:34PM -0700, Logan Gunthorpe wrote: > = > = > On 01/03/18 02:03 PM, Benjamin Herrenschmidt wrote: > > However, what happens if anything calls page_address() on them ? Some > > DMA ops do that for example, or some devices might ... > = > Although we could probably work around it with some pain, we rely on > page_address() and virt_to_phys(), etc to work on these pages. So on x86, > yes, it makes it into the linear mapping. This is pretty easy to do with HMM: unsigned long hmm_page_to_phys_pfn(struct page *page) { struct hmm_devmem *devmem; unsigned long ppfn; /* Sanity test maybe BUG_ON() */ if (!is_device_private_page(page)) return -1UL; devmem =3D page->pgmap->data; ppfn =3D page_to_page(page) - devmem->pfn_first; return ppfn + devmem->device_phys_base_pfn; } Note that last field does not exist in today HMM because i did not need such helper so far but this can be added. Cheers, J=E9r=F4me _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 From: jglisse@redhat.com (Jerome Glisse) Date: Thu, 1 Mar 2018 16:18:18 -0500 Subject: [PATCH v2 00/10] Copy Offload in NVMe Fabrics with P2P PCI Memory In-Reply-To: <1de70207-40ce-29f0-6093-337112852475@deltatee.com> References: <20180228234006.21093-1-logang@deltatee.com> <1519876489.4592.3.camel@kernel.crashing.org> <1519876569.4592.4.camel@au1.ibm.com> <1519938210.4592.30.camel@au1.ibm.com> <1de70207-40ce-29f0-6093-337112852475@deltatee.com> Message-ID: <20180301211817.GC6742@redhat.com> On Thu, Mar 01, 2018@02:11:34PM -0700, Logan Gunthorpe wrote: > > > On 01/03/18 02:03 PM, Benjamin Herrenschmidt wrote: > > However, what happens if anything calls page_address() on them ? Some > > DMA ops do that for example, or some devices might ... > > Although we could probably work around it with some pain, we rely on > page_address() and virt_to_phys(), etc to work on these pages. So on x86, > yes, it makes it into the linear mapping. This is pretty easy to do with HMM: unsigned long hmm_page_to_phys_pfn(struct page *page) { struct hmm_devmem *devmem; unsigned long ppfn; /* Sanity test maybe BUG_ON() */ if (!is_device_private_page(page)) return -1UL; devmem = page->pgmap->data; ppfn = page_to_page(page) - devmem->pfn_first; return ppfn + devmem->device_phys_base_pfn; } Note that last field does not exist in today HMM because i did not need such helper so far but this can be added. Cheers, J?r?me From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx3-rdu2.redhat.com ([66.187.233.73]:48752 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1033189AbeCAVSX (ORCPT ); Thu, 1 Mar 2018 16:18:23 -0500 Date: Thu, 1 Mar 2018 16:18:18 -0500 From: Jerome Glisse To: Logan Gunthorpe Cc: benh@au1.ibm.com, Dan Williams , Linux Kernel Mailing List , linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma , linux-nvdimm , linux-block@vger.kernel.org, Stephen Bates , Christoph Hellwig , Jens Axboe , Keith Busch , Sagi Grimberg , Bjorn Helgaas , Jason Gunthorpe , Max Gurtovoy , Alex Williamson , Oliver OHalloran Subject: Re: [PATCH v2 00/10] Copy Offload in NVMe Fabrics with P2P PCI Memory Message-ID: <20180301211817.GC6742@redhat.com> References: <20180228234006.21093-1-logang@deltatee.com> <1519876489.4592.3.camel@kernel.crashing.org> <1519876569.4592.4.camel@au1.ibm.com> <1519938210.4592.30.camel@au1.ibm.com> <1de70207-40ce-29f0-6093-337112852475@deltatee.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 In-Reply-To: <1de70207-40ce-29f0-6093-337112852475@deltatee.com> Sender: linux-pci-owner@vger.kernel.org List-ID: On Thu, Mar 01, 2018 at 02:11:34PM -0700, Logan Gunthorpe wrote: > > > On 01/03/18 02:03 PM, Benjamin Herrenschmidt wrote: > > However, what happens if anything calls page_address() on them ? Some > > DMA ops do that for example, or some devices might ... > > Although we could probably work around it with some pain, we rely on > page_address() and virt_to_phys(), etc to work on these pages. So on x86, > yes, it makes it into the linear mapping. This is pretty easy to do with HMM: unsigned long hmm_page_to_phys_pfn(struct page *page) { struct hmm_devmem *devmem; unsigned long ppfn; /* Sanity test maybe BUG_ON() */ if (!is_device_private_page(page)) return -1UL; devmem = page->pgmap->data; ppfn = page_to_page(page) - devmem->pfn_first; return ppfn + devmem->device_phys_base_pfn; } Note that last field does not exist in today HMM because i did not need such helper so far but this can be added. Cheers, Jérôme From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [PATCH v2 00/10] Copy Offload in NVMe Fabrics with P2P PCI Memory Date: Thu, 1 Mar 2018 16:18:18 -0500 Message-ID: <20180301211817.GC6742@redhat.com> References: <20180228234006.21093-1-logang@deltatee.com> <1519876489.4592.3.camel@kernel.crashing.org> <1519876569.4592.4.camel@au1.ibm.com> <1519938210.4592.30.camel@au1.ibm.com> <1de70207-40ce-29f0-6093-337112852475@deltatee.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <1de70207-40ce-29f0-6093-337112852475-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: Logan Gunthorpe Cc: Jens Axboe , Keith Busch , Oliver OHalloran , benh-8fk3Idey6ehBDgjK7y7TUQ@public.gmane.org, linux-nvdimm , linux-rdma , linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Linux Kernel Mailing List , linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Alex Williamson , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-Id: linux-rdma@vger.kernel.org On Thu, Mar 01, 2018 at 02:11:34PM -0700, Logan Gunthorpe wrote: > = > = > On 01/03/18 02:03 PM, Benjamin Herrenschmidt wrote: > > However, what happens if anything calls page_address() on them ? Some > > DMA ops do that for example, or some devices might ... > = > Although we could probably work around it with some pain, we rely on > page_address() and virt_to_phys(), etc to work on these pages. So on x86, > yes, it makes it into the linear mapping. This is pretty easy to do with HMM: unsigned long hmm_page_to_phys_pfn(struct page *page) { struct hmm_devmem *devmem; unsigned long ppfn; /* Sanity test maybe BUG_ON() */ if (!is_device_private_page(page)) return -1UL; devmem =3D page->pgmap->data; ppfn =3D page_to_page(page) - devmem->pfn_first; return ppfn + devmem->device_phys_base_pfn; } Note that last field does not exist in today HMM because i did not need such helper so far but this can be added. Cheers, J=E9r=F4me