RE: [PATCH 0/6] IOMMU/DMA map_resource support for peer-to-peer

[William Davis <wdavis@nvidia.com>]

> -----Original Message-----
> From: Mark Hounschell [mailto:markh@compro.net]
> Sent: Friday, May 08, 2015 3:46 PM
> To: Konrad Rzeszutek Wilk; William Davis
> Cc: linux-pci@vger.kernel.org; iommu@lists.linux-foundation.org; jglisse@redhat.com; John Hubbard;
> Terence Ripperda
> Subject: Re: [PATCH 0/6] IOMMU/DMA map_resource support for peer-to-peer
> 
> On 05/08/2015 04:21 PM, Konrad Rzeszutek Wilk wrote:
> > On Fri, May 01, 2015 at 01:32:12PM -0500, wdavis@nvidia.com wrote:
> >> From: Will Davis <wdavis@nvidia.com>
> >>
> >> Hi,
> >>
> ...
> >>
> >> This solves a long-standing problem with the existing DMA-remapping interfaces,
> >> which require that a struct page be given for the region to be mapped into a
> >> device's IOVA domain. This requirement cannot support peer device BAR ranges,
> >> for which no struct pages exist.
> >>
> ...
> >>
> >> The Intel and nommu versions have been verified on a dual Intel Xeon E5405
> >> workstation.
> 
> PCIe peer2peer is borked on all motherboards I've tried. Only writes are
> possible. Reads are not supported. I suppose if you have a platform with
> only PCI and an IOMMU this would be very useful. Without both read and
> write PCIe peer2peer support, this seems unnecessary.
> 

PCIe peer-to-peer isn't inherently broken or useless itself, even if a lot of its implementations are; I've successfully tested these patches with existing hardware (dual NVIDIA GPUs + an old-ish workstation), and it solves a longstanding problem for us [1], so I disagree with the assessment that this would be unnecessary.

I guess I don't see why the existence of some, or even a lot of, poor implementations suffices as a reason to reject a generic mechanism to support the "good", standardized implementations.

[1] http://stackoverflow.com/questions/19841815/does-the-nvidia-rdma-gpudirect-always-operate-only-physical-addresses-in-physic

Thanks,
Will

--
nvpublic