[PATCH v2] nvme-pci: Use PCI bus address for data/queues in CMB

[keith.busch@intel.com (Keith Busch)]

On Sun, Oct 01, 2017@09:42:03AM +0200, Christoph Hellwig wrote:
> This looks very convoluted, mostly because the existing code is
> doing weird things.  For one thing what is sq_dma_addr currently
> is not a DMA adddress - we either need the resource address
> for the ioremap, but we don't need to stash that away, and second
> the one programmed into the controller should be a pci_bus_addr_t.
> 
> Second we already have a nice PCI-layer helper called pci_bus_address
> to get the bus address for us and we should use it.
> 
> Something like the patch below should solve the issue:

Yah, calling this a DMA address was a misnomer and confusing.

> ---
> From b78f4164881125c4fecfdb87878d0120b2177c53 Mon Sep 17 00:00:00 2001
> From: Christoph Hellwig <hch at lst.de>
> Date: Sun, 1 Oct 2017 09:37:35 +0200
> Subject: nvme-pci: Use PCI bus address for data/queues in CMB
> 
> Currently, NVMe PCI host driver is programming CMB dma address as
> I/O SQs addresses. This results in failures on systems where 1:1
> outbound mapping is not used (example Broadcom iProc SOCs) because
> CMB BAR will be progammed with PCI bus address but NVMe PCI EP will
> try to access CMB using dma address.
> 
> To have CMB working on systems without 1:1 outbound mapping, we
> program PCI bus address for I/O SQs instead of dma address. This
> approach will work on systems with/without 1:1 outbound mapping.
> 
> Based on a report and previous patch from Abhishek Shah.
> 
> Fixes: 8ffaadf7 ("NVMe: Use CMB for the IO SQes if available")
> Cc: stable at vger.kernel.org
> Reported-by: Abhishek Shah <abhishek.shah at broadcom.com>
> Signed-off-by: Christoph Hellwig <hch at lst.de>

This looks good.

Reviewed-by: Keith Busch <keith.busch at intel.com>