[PATCH v5 00/13] DMA-API/PCI map_peer_resource support for peer-to-peer

[PATCH v5 00/13] DMA-API/PCI map_peer_resource support for peer-to-peer

[Will Davis]

Hi,

This is the fifth version of a patchset to add the DMA APIs necessary to
map and unmap a PCI device's BAR to and from another PCI device's IOVA
domain. This enables PCI peer-to-peer traffic on x86 platforms where an
IOMMU is enabled (this requires that peer-to-peer traffic be supported by
the underlying chipset as well, of course).

Thanks,
Will

v5:
- moved topology and ACS checks into a separate
  pci_peer_traffic_supported() API, which determines whether PCI peer-
  to-peer traffic may work based on language in the PCI and PCI Express
  specifications (excerpts included in comments)
- added CONFIG_HAS_DMA_P2P guards to x86 PCI no-mmu implementation of
  map_peer_resource
- modified a few patch descriptions for style

v4: http://www.spinics.net/lists/linux-pci/msg43136.html
- added HAS_DMA_P2P Kconfig option to guard new APIs, which are only
  supported on x86 with this patch series
- added a new dma_peer_addr_t type to handle future platform support
  where the dma_peer_addr_t needs to have the same width as the bus
  address, which can be wider than the dma_addr_t returned by other
  DMA APIs
- added a new dma_peer_mapping_error() and associated PCI wrapper to
  detect errors returned via dma_map_peer_resource() API and its
  PCI wrapper
- modified the interface of the new API to explicitly take the peer
  device as an argument
- renamed the (un)map_resource APIs to (un)map_peer_resource, since
  they directly receive a 'peer' argument now
- modified the new PCI interface to take the BAR index instead of the
  actual struct resource
- modified the pci-nommu implementation to perform topology and ACS
  checks before allowing
- modified both the AMD and Intel IOMMU implementations to require
  that both devices be behind the same host bridge
- exposed a couple of preexisting PCI utility functions to aid in the
  topology checks for all of the x86 implementations

v3: http://www.spinics.net/lists/linux-pci/msg41703.html
- changed dma_map_resource() to BUG() if the map_resource DMA op is not
  provided, instead of returning 0
- updated documentation to remove requirement to check for 0 return value
  as an error
- remove const keyword from struct dma_map_ops in new DMA APIs for
  consistency with other APIs

v2: http://www.spinics.net/lists/linux-pci/msg41192.html
- added documentation for the new DMA APIs
- fixed physical-to-bus address conversion in the nommu implementation

v1: http://www.spinics.net/lists/linux-pci/msg40747.html


Will Davis (13):
  lib/Kconfig: add HAS_DMA_P2P for peer-to-peer support
  linux/types.h: Add dma_peer_addr_t type
  dma-debug: add checking for map/unmap_peer_resource
  DMA-API: Introduce dma_(un)map_peer_resource
  dma-mapping: pci: add pci_(un)map_peer_resource
  DMA-API: Add peer resource mapping documentation
  PCI: Export pci_find_host_bridge()
  PCI: Add pci_find_common_upstream_dev()
  PCI: Add pci_peer_traffic_supported()
  iommu/amd: Implement (un)map_peer_resource
  iommu/vt-d: implement (un)map_peer_resource
  x86: add pci-nommu implementation of map_peer_resource
  x86: declare support for DMA P2P

 Documentation/DMA-API-HOWTO.txt          | 69 ++++++++++++++++++++++
 Documentation/DMA-API.txt                | 38 ++++++++++--
 arch/x86/Kconfig                         |  1 +
 arch/x86/include/asm/dma-mapping.h       | 18 +++++-
 arch/x86/kernel/pci-nommu.c              | 36 ++++++++++++
 drivers/iommu/amd_iommu.c                | 99 +++++++++++++++++++++++++++-----
 drivers/iommu/intel-iommu.c              | 38 ++++++++++++
 drivers/pci/host-bridge.c                | 19 +-----
 drivers/pci/pci.c                        | 99 ++++++++++++++++++++++++++++++++
 drivers/pci/search.c                     | 25 ++++++++
 include/asm-generic/dma-mapping-common.h | 43 ++++++++++++++
 include/asm-generic/pci-dma-compat.h     | 23 ++++++++
 include/linux/dma-debug.h                | 39 ++++++++++++-
 include/linux/dma-mapping.h              | 12 ++++
 include/linux/pci.h                      | 12 ++++
 include/linux/types.h                    | 13 ++++-
 lib/Kconfig                              |  5 ++
 lib/dma-debug.c                          | 86 ++++++++++++++++++++++++++-
 18 files changed, 633 insertions(+), 42 deletions(-)

-- 
2.5.0

[PATCH v5 01/13] lib/Kconfig: add HAS_DMA_P2P for peer-to-peer support

[Will Davis]

This option is dependent on HAS_DMA, and indicates whether or not the DMA
API should include the peer resource (P2P) interfaces.

Signed-off-by: Will Davis <wdavis@nvidia.com>
---
 lib/Kconfig | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/Kconfig b/lib/Kconfig
index 3a2ef67..0b670ab 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -392,6 +392,11 @@ config HAS_DMA
 	depends on !NO_DMA
 	default y
 
+config HAS_DMA_P2P
+	bool
+	depends on HAS_DMA
+	default n
+
 config CHECK_SIGNATURE
 	bool
 
-- 
2.5.0

[PATCH v5 02/13] linux/types.h: Add dma_peer_addr_t type

[Will Davis]

On some platforms, a dma_addr_t is only 32 bits wide, but the bus addresses
can be 64 bits wide. For peer-to-peer DMA, a device needs to use the full
64-bit address. Therefore, introduce a DMA address type that is not
constrained by the CPU DMA address size.

This is only needed for peer-to-peer DMA, so guard this definition with a
CONFIG_HAS_DMA_P2P check.

Signed-off-by: Will Davis <wdavis@nvidia.com>
---
 include/linux/types.h | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/include/linux/types.h b/include/linux/types.h
index 8715287..dd85173 100644
--- a/include/linux/types.h
+++ b/include/linux/types.h
@@ -140,13 +140,13 @@ typedef unsigned long blkcnt_t;
 #define pgoff_t unsigned long
 
 /*
- * A dma_addr_t can hold any valid DMA address, i.e., any address returned
+ * A dma_addr_t can hold any valid DMA address, i.e., most addresses returned
  * by the DMA API.
  *
  * If the DMA API only uses 32-bit addresses, dma_addr_t need only be 32
  * bits wide.  Bus addresses, e.g., PCI BARs, may be wider than 32 bits,
  * but drivers do memory-mapped I/O to ioremapped kernel virtual addresses,
- * so they don't care about the size of the actual bus addresses.
+ * so they don't care about the size of the actual bus addresses in most cases.
  */
 #ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
 typedef u64 dma_addr_t;
@@ -154,6 +154,15 @@ typedef u64 dma_addr_t;
 typedef u32 dma_addr_t;
 #endif
 
+/*
+ * Unlike dma_addr_t, a dma_peer_addr_t can hold any bus address for the
+ * platform. Drivers should only need this for peer-to-peer use cases, where
+ * the DMA target is also on the bus.
+ */
+#ifdef CONFIG_HAS_DMA_P2P
+typedef u64 dma_peer_addr_t;
+#endif
+
 typedef unsigned __bitwise__ gfp_t;
 typedef unsigned __bitwise__ fmode_t;
 typedef unsigned __bitwise__ oom_flags_t;
-- 
2.5.0

[PATCH v5 03/13] dma-debug: add checking for map/unmap_peer_resource

[Will Davis]

Add debug callbacks for the new dma_map_peer_resource and
dma_unmap_peer_resource functions.

Also add a callback for dma_peer_mapping_error. Most of the time this will
be identical to dma_mapping_error, except where the dma_addr_t is not as
wide as dma_peer_addr_t.

Guard these new callbacks behind a CONFIG_HAS_DMA_P2P check.

Signed-off-by: Will Davis <wdavis@nvidia.com>
Reviewed-by: Terence Ripperda <tripperda@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
---
 include/linux/dma-debug.h | 39 ++++++++++++++++++++-
 lib/dma-debug.c           | 86 +++++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 122 insertions(+), 3 deletions(-)

diff --git a/include/linux/dma-debug.h b/include/linux/dma-debug.h
index fe8cb61..4ff8364 100644
--- a/include/linux/dma-debug.h
+++ b/include/linux/dma-debug.h
@@ -43,7 +43,21 @@ extern void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr);
 
 extern void debug_dma_unmap_page(struct device *dev, dma_addr_t addr,
 				 size_t size, int direction, bool map_single);
-
+#ifdef CONFIG_HAS_DMA_P2P
+extern void debug_dma_map_peer_resource(struct device *dev,
+					struct device *peer,
+					struct resource *resource,
+					size_t offset, size_t size,
+					int direction,
+					dma_peer_addr_t dma_addr);
+
+extern void debug_dma_peer_mapping_error(struct device *dev,
+					 dma_peer_addr_t dma_addr);
+
+extern void debug_dma_unmap_peer_resource(struct device *dev,
+					  dma_peer_addr_t addr,
+					  size_t size, int direction);
+#endif
 extern void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 			     int nents, int mapped_ents, int direction);
 
@@ -120,6 +134,29 @@ static inline void debug_dma_unmap_page(struct device *dev, dma_addr_t addr,
 {
 }
 
+#ifdef CONFIG_HAS_DMA_P2P
+static inline void debug_dma_map_peer_resource(struct device *dev,
+					       struct device *peer,
+					       struct resource *resource,
+					       size_t offset, size_t size,
+					       int direction,
+					       dma_peer_addr_t dma_addr)
+{
+}
+
+static inline void debug_dma_peer_mapping_error(struct device *dev,
+						dma_peer_addr_t dma_addr)
+{
+}
+
+static inline void debug_dma_unmap_peer_resource(struct device *dev,
+						 dma_peer_addr_t addr,
+						 size_t size,
+						 int direction)
+{
+}
+#endif
+
 static inline void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 				    int nents, int mapped_ents, int direction)
 {
diff --git a/lib/dma-debug.c b/lib/dma-debug.c
index ae4b65e..622f569 100644
--- a/lib/dma-debug.c
+++ b/lib/dma-debug.c
@@ -43,6 +43,7 @@ enum {
 	dma_debug_page,
 	dma_debug_sg,
 	dma_debug_coherent,
+	dma_debug_peer_resource,
 };
 
 enum map_err_types {
@@ -64,7 +65,8 @@ enum map_err_types {
  * @direction: enum dma_data_direction
  * @sg_call_ents: 'nents' from dma_map_sg
  * @sg_mapped_ents: 'mapped_ents' from dma_map_sg
- * @map_err_type: track whether dma_mapping_error() was checked
+ * @map_err_type: track whether dma_mapping_error() or dma_peer_mapping_error()
+ *		  was called
  * @stacktrace: support backtraces when a violation is detected
  */
 struct dma_debug_entry {
@@ -1250,6 +1252,23 @@ out:
 	put_hash_bucket(bucket, &flags);
 }
 
+static void check_mapping_error(struct dma_debug_entry *entry, bool peer)
+{
+	if (peer && !(entry->type == dma_debug_peer_resource))
+		err_printk(entry->dev, entry, "DMA-API: device driver calls "
+				"dma_peer_mapping_error() on dma_addr_t "
+				"[device address=0x%016llx] [size=%llu bytes]\n",
+				(unsigned long long)entry->dev_addr,
+				entry->size);
+
+	if (!peer && (entry->type == dma_debug_peer_resource))
+		err_printk(entry->dev, entry, "DMA-API: device driver calls "
+				"dma_mapping_error() on dma_peer_addr_t "
+				"[device address=0x%016llx] [size=%llu bytes]\n",
+				(unsigned long long)entry->dev_addr,
+				entry->size);
+}
+
 void debug_dma_map_page(struct device *dev, struct page *page, size_t offset,
 			size_t size, int direction, dma_addr_t dma_addr,
 			bool map_single)
@@ -1289,7 +1308,9 @@ void debug_dma_map_page(struct device *dev, struct page *page, size_t offset,
 }
 EXPORT_SYMBOL(debug_dma_map_page);
 
-void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
+static void debug_dma_mapping_error_common(struct device *dev,
+					   u64 dma_addr,
+					   bool peer)
 {
 	struct dma_debug_entry ref;
 	struct dma_debug_entry *entry;
@@ -1319,12 +1340,18 @@ void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
 		 */
 		if (entry->map_err_type == MAP_ERR_NOT_CHECKED) {
 			entry->map_err_type = MAP_ERR_CHECKED;
+			check_mapping_error(entry, peer);
 			break;
 		}
 	}
 
 	put_hash_bucket(bucket, &flags);
 }
+
+void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
+{
+	debug_dma_mapping_error_common(dev, (u64)dma_addr, false);
+}
 EXPORT_SYMBOL(debug_dma_mapping_error);
 
 void debug_dma_unmap_page(struct device *dev, dma_addr_t addr,
@@ -1348,6 +1375,61 @@ void debug_dma_unmap_page(struct device *dev, dma_addr_t addr,
 }
 EXPORT_SYMBOL(debug_dma_unmap_page);
 
+#ifdef CONFIG_HAS_DMA_P2P
+void debug_dma_map_peer_resource(struct device *dev, struct device *peer,
+				 struct resource *resource, size_t offset,
+				 size_t size, int direction,
+				 dma_peer_addr_t dma_addr)
+{
+	struct dma_debug_entry *entry;
+
+	if (unlikely(dma_debug_disabled()))
+		return;
+
+	if (dma_mapping_error(dev, dma_addr))
+		return;
+
+	entry = dma_entry_alloc();
+	if (!entry)
+		return;
+
+	entry->dev       = dev;
+	entry->type      = dma_debug_peer_resource;
+	entry->pfn       = resource->start >> PAGE_SHIFT;
+	entry->offset    = offset,
+	entry->dev_addr  = dma_addr;
+	entry->size      = size;
+	entry->direction = direction;
+
+	add_dma_entry(entry);
+}
+EXPORT_SYMBOL(debug_dma_map_peer_resource);
+
+void debug_dma_peer_mapping_error(struct device *dev, dma_peer_addr_t dma_addr)
+{
+	debug_dma_mapping_error_common(dev, (u64)dma_addr, true);
+}
+EXPORT_SYMBOL(debug_dma_peer_mapping_error);
+
+void debug_dma_unmap_peer_resource(struct device *dev, dma_peer_addr_t addr,
+				   size_t size, int direction)
+{
+	struct dma_debug_entry ref = {
+		.type           = dma_debug_peer_resource,
+		.dev            = dev,
+		.dev_addr       = addr,
+		.size           = size,
+		.direction      = direction,
+	};
+
+	if (unlikely(dma_debug_disabled()))
+		return;
+
+	check_unmap(&ref);
+}
+EXPORT_SYMBOL(debug_dma_unmap_peer_resource);
+#endif
+
 void debug_dma_map_sg(struct device *dev, struct scatterlist *sg,
 		      int nents, int mapped_ents, int direction)
 {
-- 
2.5.0

[PATCH v5 04/13] DMA-API: Introduce dma_(un)map_peer_resource

[Will Davis]

Add functions to DMA-map and -unmap a peer device's resource for a given
device. This will allow devices to DMA-map, for example, another device's
BAR region on PCI to enable peer-to-peer transactions.

Guard these new functions behind CONFIG_HAS_DMA_P2P.

Signed-off-by: Will Davis <wdavis@nvidia.com>
---
 include/asm-generic/dma-mapping-common.h | 43 ++++++++++++++++++++++++++++++++
 include/linux/dma-mapping.h              | 12 +++++++++
 2 files changed, 55 insertions(+)

diff --git a/include/asm-generic/dma-mapping-common.h b/include/asm-generic/dma-mapping-common.h
index 940d5ec..45eec17 100644
--- a/include/asm-generic/dma-mapping-common.h
+++ b/include/asm-generic/dma-mapping-common.h
@@ -73,6 +73,42 @@ static inline void dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg
 		ops->unmap_sg(dev, sg, nents, dir, attrs);
 }
 
+#ifdef CONFIG_HAS_DMA_P2P
+static inline dma_peer_addr_t dma_map_peer_resource_attrs(struct device *dev,
+							  struct device *peer,
+							  struct resource *res,
+							  size_t offset,
+							  size_t size,
+							  enum dma_data_direction dir,
+							  struct dma_attrs *attrs)
+{
+	struct dma_map_ops *ops = get_dma_ops(dev);
+	dma_peer_addr_t addr;
+
+	BUG_ON(!valid_dma_direction(dir));
+	BUG_ON(ops->map_peer_resource == NULL);
+	addr = ops->map_peer_resource(dev, peer, res, offset, size, dir,
+				      attrs);
+	debug_dma_map_peer_resource(dev, peer, res, offset, size, dir, addr);
+
+	return addr;
+}
+
+static inline void dma_unmap_peer_resource_attrs(struct device *dev,
+						 dma_peer_addr_t addr,
+						 size_t size,
+						 enum dma_data_direction dir,
+						 struct dma_attrs *attrs)
+{
+	struct dma_map_ops *ops = get_dma_ops(dev);
+
+	BUG_ON(!valid_dma_direction(dir));
+	if (ops->unmap_peer_resource)
+		ops->unmap_peer_resource(dev, addr, size, dir, attrs);
+	debug_dma_unmap_peer_resource(dev, addr, size, dir);
+}
+#endif
+
 static inline dma_addr_t dma_map_page(struct device *dev, struct page *page,
 				      size_t offset, size_t size,
 				      enum dma_data_direction dir)
@@ -181,6 +217,13 @@ dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
 #define dma_map_sg(d, s, n, r) dma_map_sg_attrs(d, s, n, r, NULL)
 #define dma_unmap_sg(d, s, n, r) dma_unmap_sg_attrs(d, s, n, r, NULL)
 
+#ifdef CONFIG_HAS_DMA_P2P
+#define dma_map_peer_resource(d, p, e, o, s, r) \
+	dma_map_peer_resource_attrs(d, p, e, o, s, r, NULL)
+#define dma_unmap_peer_resource(d, a, s, r) \
+	dma_unmap_peer_resource_attrs(d, a, s, r, NULL)
+#endif
+
 extern int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,
 			   void *cpu_addr, dma_addr_t dma_addr, size_t size);
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index ac07ff0..7b8fddc 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -34,6 +34,18 @@ struct dma_map_ops {
 	void (*unmap_page)(struct device *dev, dma_addr_t dma_handle,
 			   size_t size, enum dma_data_direction dir,
 			   struct dma_attrs *attrs);
+#ifdef CONFIG_HAS_DMA_P2P
+	dma_peer_addr_t (*map_peer_resource)(struct device *dev,
+					     struct device *peer,
+					     struct resource *res,
+					     unsigned long offset, size_t size,
+					     enum dma_data_direction dir,
+					     struct dma_attrs *attrs);
+	void (*unmap_peer_resource)(struct device *dev,
+				    dma_peer_addr_t dma_handle,
+				    size_t size, enum dma_data_direction dir,
+				    struct dma_attrs *attrs);
+#endif
 	/*
 	 * map_sg returns 0 on error and a value > 0 on success.
 	 * It should never return a value < 0.
-- 
2.5.0

[PATCH v5 05/13] dma-mapping: pci: add pci_(un)map_peer_resource

[Will Davis]

Simply route these through to the new dma_(un)map_peer_resource APIs.

Also route pci_peer_mapping_error through to the new dma_peer_mapping_error
API.

Guard these new functions behind CONFIG_HAS_DMA_P2P.

Signed-off-by: Will Davis <wdavis@nvidia.com>
Reviewed-by: Terence Ripperda <tripperda@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
---
 include/asm-generic/pci-dma-compat.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/include/asm-generic/pci-dma-compat.h b/include/asm-generic/pci-dma-compat.h
index c110843..7baf09e 100644
--- a/include/asm-generic/pci-dma-compat.h
+++ b/include/asm-generic/pci-dma-compat.h
@@ -61,6 +61,29 @@ pci_unmap_page(struct pci_dev *hwdev, dma_addr_t dma_address,
 	dma_unmap_page(hwdev == NULL ? NULL : &hwdev->dev, dma_address, size, (enum dma_data_direction)direction);
 }
 
+#ifdef CONFIG_HAS_DMA_P2P
+static inline dma_peer_addr_t
+pci_map_peer_resource(struct pci_dev *hwdev, struct pci_dev *peer,
+		      int res, unsigned long offset, size_t size,
+		      int direction)
+{
+	return dma_map_peer_resource(hwdev == NULL ? NULL : &hwdev->dev, peer == NULL ? NULL : &peer->dev, peer == NULL ? NULL : &peer->resource[res], offset, size, (enum dma_data_direction)direction);
+}
+
+static inline int
+pci_dma_peer_mapping_error(struct pci_dev *pdev, dma_peer_addr_t dma_addr)
+{
+	return dma_peer_mapping_error(&pdev->dev, dma_addr);
+}
+
+static inline void
+pci_unmap_peer_resource(struct pci_dev *hwdev, dma_peer_addr_t dma_address,
+			size_t size, int direction)
+{
+	dma_unmap_peer_resource(hwdev == NULL ? NULL : &hwdev->dev, dma_address, size, (enum dma_data_direction)direction);
+}
+#endif
+
 static inline int
 pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sg,
 	   int nents, int direction)
-- 
2.5.0

[PATCH v5 06/13] DMA-API: Add peer resource mapping documentation

[Will Davis]

Add references to both the general API documentation, as well as the HOWTO,
about the dma_(un)map_peer_resource() and dma_peer_mapping_error() APIs.

Signed-off-by: Will Davis <wdavis@nvidia.com>
---
 Documentation/DMA-API-HOWTO.txt | 69 +++++++++++++++++++++++++++++++++++++++++
 Documentation/DMA-API.txt       | 38 ++++++++++++++++++++---
 2 files changed, 102 insertions(+), 5 deletions(-)

diff --git a/Documentation/DMA-API-HOWTO.txt b/Documentation/DMA-API-HOWTO.txt
index 55b70b9..053a18f 100644
--- a/Documentation/DMA-API-HOWTO.txt
+++ b/Documentation/DMA-API-HOWTO.txt
@@ -143,6 +143,10 @@ What about block I/O and networking buffers?  The block I/O and
 networking subsystems make sure that the buffers they use are valid
 for you to DMA from/to.
 
+In some systems, it may also be possible to DMA to and/or from a peer
+device's MMIO region, as described by a 'struct resource'. This is
+referred to as a peer-to-peer mapping.
+
 			DMA addressing limitations
 
 Does your device have any DMA addressing limitations?  For example, is
@@ -653,6 +657,45 @@ Every dma_map_{single,sg}() call should have its dma_unmap_{single,sg}()
 counterpart, because the DMA address space is a shared resource and
 you could render the machine unusable by consuming all DMA addresses.
 
+Peer-to-peer DMA mappings can be obtained using dma_map_peer_resource() to
+map another device's MMIO region for the given device:
+
+	dma_peer_addr_t dma_handle = dma_map_peer_resource(dev, peer, res,
+							   offset, size,
+							   direction);
+	if (dma_peer_mapping_error(dev, dma_handle))
+	{
+		/*
+		 * reduce current DMA mapping usage,
+		 * delay and try again later or
+		 * reset driver. P2P DMA may not be
+		 * supported between the devices.
+		 */
+		goto map_error_handling;
+	}
+
+	...
+
+	dma_unmap_peer_resource(dev, dma_handle, size, direction);
+
+Here, "res" is the struct resource * of the MMIO region, and "offset" means
+byte offset within the given resource.
+
+You should call dma_peer_mapping_error() as dma_map_peer_resource() could fail
+and return error.
+
+Not all implementations support dma_map_peer_resource(), but those that do
+should also implement dma_peer_mapping_error(). dma_map_peer_resource() may
+fail for various reasons depending on the implementation, but generally
+may fail for the same reasons as those outlined under the dma_map_single()
+discussion. In addition, the success of dma_map_peer_resource() may depend
+on the physical topology between the two devices. For example, in the x86
+implementations, peer-to-peer mappings will fail if the devices are behind
+different host bridges.
+
+You should call dma_unmap_peer_resource() when DMA activity is finished,
+e.g., from the interrupt which told you that the DMA transfer is done.
+
 If you need to use the same streaming DMA region multiple times and touch
 the data in between the DMA transfers, the buffer needs to be synced
 properly in order for the CPU and device to see the most up-to-date and
@@ -785,6 +828,23 @@ failure can be determined by:
 		goto map_error_handling;
 	}
 
+- checking the dma_peer_addr_t returned from dma_map_peer_resource() by using
+  dma_peer_mapping_error():
+
+  	dma_peer_addr_t dma_handle;
+
+	dma_handle = dma_map_peer_resource(dev, peer, res, offset, size,
+					   direction);
+	if (dma_peer_mapping_error(dev, dma_handle)) {
+		/*
+		 * reduce current DMA mapping usage,
+		 * delay and try again later or
+		 * reset driver. P2P DMA may not be
+		 * supported between the devices.
+		 */
+		goto map_error_handling;
+	}
+
 - unmap pages that are already mapped, when mapping error occurs in the middle
   of a multiple page mapping attempt. These example are applicable to
   dma_map_page() as well.
@@ -956,6 +1016,15 @@ to "Closing".
    support multiple types of IOMMUs in a single system, the example of
    x86 or powerpc helps.
 
+4) Peer-to-peer requirements.
+
+   If your platform can support peer-to-peer DMA, enable HAS_DMA_P2P to
+   include the peer-resource mapping interfaces in the DMA API, and
+   provide an implementation of dma_peer_mapping_error(). The latter part
+   is necessary because on some platforms, the dma_addr_t is not as wide
+   as the full bus address, and therefore the existing dma_mapping_error()
+   API may not be sufficient.
+
 			   Closing
 
 This document, and the API itself, would not be in its current
diff --git a/Documentation/DMA-API.txt b/Documentation/DMA-API.txt
index 7eba542..c6f11a1 100644
--- a/Documentation/DMA-API.txt
+++ b/Documentation/DMA-API.txt
@@ -283,14 +283,42 @@ and <size> parameters are provided to do partial page mapping, it is
 recommended that you never use these unless you really know what the
 cache width is.
 
+dma_peer_addr_t
+dma_map_peer_resource(struct device *dev, struct device *peer,
+		      struct resource *res, unsigned long offset,
+		      size_t size, enum dma_data_direction_direction)
+
+API for mapping resources. This API allows a driver to map a peer
+device's resource for DMA. All the notes and warnings for the other
+APIs apply here. Also, the success of this API does not validate or
+guarantee that peer-to-peer transactions between the device and its
+peer will be functional. They only grant access so that if such
+transactions are possible, an IOMMU will not prevent them from
+succeeding. However, a failure of this API may indicate that peer-to-peer
+transactions are not expected to succeed across the underlying topology
+between the devices.
+
+void
+dma_unmap_peer_resource(struct device *dev, dma_peer_addr_t dma_address,
+			size_t size, enum dma_data_direction direction)
+
+Unmaps the resource previously mapped. All the parameters passed in
+must be identical to those passed in to (and returned by) the mapping
+API.
+
 int
 dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
 
-In some circumstances dma_map_single() and dma_map_page() will fail to create
-a mapping. A driver can check for these errors by testing the returned
-DMA address with dma_mapping_error(). A non-zero return value means the mapping
-could not be created and the driver should take appropriate action (e.g.
-reduce current DMA mapping usage or delay and try again later).
+int
+dma_peer_mapping_error(struct device *dev, dma_peer_addr_t dma_addr)
+
+In some circumstances dma_map_single(), dma_map_page() and
+dma_map_peer_resource() will fail to create a mapping. A driver can check for
+these errors by testing the returned DMA address with dma_mapping_error()
+(or dma_peer_mapping_error() for dma_map_peer_resource()). A non-zero return
+value means the mapping could not be created and the driver should take
+appropriate action (e.g. reduce current DMA mapping usage or delay and try
+again later).
 
 	int
 	dma_map_sg(struct device *dev, struct scatterlist *sg,
-- 
2.5.0

[PATCH v5 07/13] PCI: Export pci_find_host_bridge()

[Will Davis]

Return the struct pci_host_bridge corresponding to the given
struct pci_bus.

Signed-off-by: Will Davis <wdavis@nvidia.com>
---
 drivers/pci/host-bridge.c | 19 ++-----------------
 include/linux/pci.h       |  7 +++++++
 2 files changed, 9 insertions(+), 17 deletions(-)

diff --git a/drivers/pci/host-bridge.c b/drivers/pci/host-bridge.c
index 5f4a2e0..6c2c82a 100644
--- a/drivers/pci/host-bridge.c
+++ b/drivers/pci/host-bridge.c
@@ -8,25 +8,10 @@
 
 #include "pci.h"
 
-static struct pci_bus *find_pci_root_bus(struct pci_bus *bus)
-{
-	while (bus->parent)
-		bus = bus->parent;
-
-	return bus;
-}
-
-struct pci_host_bridge *pci_find_host_bridge(struct pci_bus *bus)
-{
-	struct pci_bus *root_bus = find_pci_root_bus(bus);
-
-	return to_pci_host_bridge(root_bus->bridge);
-}
-
 struct device *pci_get_host_bridge_device(struct pci_dev *dev)
 {
-	struct pci_bus *root_bus = find_pci_root_bus(dev->bus);
-	struct device *bridge = root_bus->bridge;
+	struct pci_host_bridge *host_bridge = pci_find_host_bridge(dev->bus);
+	struct device *bridge = &host_bridge->dev;
 
 	kobject_get(&bridge->kobj);
 	return bridge;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 8a0321a..d58063e 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -512,6 +512,13 @@ static inline struct pci_dev *pci_upstream_bridge(struct pci_dev *dev)
 	return dev->bus->self;
 }
 
+static inline struct pci_host_bridge *pci_find_host_bridge(struct pci_bus *bus)
+{
+	while (bus->parent)
+		bus = bus->parent;
+	return to_pci_host_bridge(bus->bridge);
+}
+
 struct device *pci_get_host_bridge_device(struct pci_dev *dev);
 void pci_put_host_bridge_device(struct device *dev);
 
-- 
2.5.0

[PATCH v5 08/13] PCI: Add pci_find_common_upstream_dev()

[Will Davis]

Add an interface to find the first device which is upstream of both
devices.

Signed-off-by: Will Davis <wdavis@nvidia.com>
---
 drivers/pci/search.c | 25 +++++++++++++++++++++++++
 include/linux/pci.h  |  2 ++
 2 files changed, 27 insertions(+)

diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index a20ce7d..b6ed554 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -384,3 +384,28 @@ int pci_dev_present(const struct pci_device_id *ids)
 	return 0;
 }
 EXPORT_SYMBOL(pci_dev_present);
+
+/**
+ * pci_find_common_upstream_dev - Returns the first common upstream device
+ * @dev: the PCI device from which the bus hierarchy walk will start
+ * @other: the PCI device whose bus number will be used for the search
+ *
+ * Walks up the bus hierarchy from the @dev device, looking for the first bus
+ * which the @other device is downstream of. Returns %NULL if the devices do
+ * not share any upstream devices.
+ */
+struct pci_dev *pci_find_common_upstream_dev(struct pci_dev *dev,
+					     struct pci_dev *other)
+{
+	struct pci_dev *pdev = dev;
+
+	while (pdev != NULL) {
+		if ((other->bus->number >= pdev->bus->busn_res.start) &&
+		    (other->bus->number <= pdev->bus->busn_res.end))
+			return pdev;
+		pdev = pci_upstream_bridge(pdev);
+	}
+
+	return NULL;
+}
+EXPORT_SYMBOL(pci_find_common_upstream_dev);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index d58063e..8262b9e 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -860,6 +860,8 @@ static inline struct pci_dev *pci_get_bus_and_slot(unsigned int bus,
 }
 struct pci_dev *pci_get_class(unsigned int class, struct pci_dev *from);
 int pci_dev_present(const struct pci_device_id *ids);
+struct pci_dev *pci_find_common_upstream_dev(struct pci_dev *from,
+					     struct pci_dev *to);
 
 int pci_bus_read_config_byte(struct pci_bus *bus, unsigned int devfn,
 			     int where, u8 *val);
-- 
2.5.0

[PATCH v5 10/13] iommu/amd: Implement (un)map_peer_resource

[Will Davis]

Implement 'map_peer_resource' for the AMD IOMMU driver. Generalize the
existing map_page implementation to operate on a physical address, and
make both map_page and map_resource wrappers around that helper (and
similarly, for unmap_page and unmap_resource).

This allows a device to map another's resource, to enable peer-to-peer
transactions.

Add behind CONFIG_HAS_DMA_P2P guards, since the dma_map_ops members are
behind them as well.

Signed-off-by: Will Davis <wdavis@nvidia.com>
Reviewed-by: Terence Ripperda <tripperda@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
---
 drivers/iommu/amd_iommu.c | 99 ++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 86 insertions(+), 13 deletions(-)

diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index a57e9b7..13a47f283 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -471,6 +471,10 @@ DECLARE_STATS_COUNTER(cnt_map_single);
 DECLARE_STATS_COUNTER(cnt_unmap_single);
 DECLARE_STATS_COUNTER(cnt_map_sg);
 DECLARE_STATS_COUNTER(cnt_unmap_sg);
+#ifdef CONFIG_HAS_DMA_P2P
+DECLARE_STATS_COUNTER(cnt_map_peer_resource);
+DECLARE_STATS_COUNTER(cnt_unmap_peer_resource);
+#endif
 DECLARE_STATS_COUNTER(cnt_alloc_coherent);
 DECLARE_STATS_COUNTER(cnt_free_coherent);
 DECLARE_STATS_COUNTER(cross_page);
@@ -509,6 +513,10 @@ static void amd_iommu_stats_init(void)
 	amd_iommu_stats_add(&cnt_unmap_single);
 	amd_iommu_stats_add(&cnt_map_sg);
 	amd_iommu_stats_add(&cnt_unmap_sg);
+#ifdef CONFIG_HAS_DMA_P2P
+	amd_iommu_stats_add(&cnt_map_peer_resource);
+	amd_iommu_stats_add(&cnt_unmap_peer_resource);
+#endif
 	amd_iommu_stats_add(&cnt_alloc_coherent);
 	amd_iommu_stats_add(&cnt_free_coherent);
 	amd_iommu_stats_add(&cross_page);
@@ -2585,20 +2593,16 @@ static void __unmap_single(struct dma_ops_domain *dma_dom,
 }
 
 /*
- * The exported map_single function for dma_ops.
+ * Wrapper function that contains code common to mapping a physical address
+ * range from a page or a resource.
  */
-static dma_addr_t map_page(struct device *dev, struct page *page,
-			   unsigned long offset, size_t size,
-			   enum dma_data_direction dir,
-			   struct dma_attrs *attrs)
+static dma_addr_t __map_phys(struct device *dev, phys_addr_t paddr,
+			     size_t size, enum dma_data_direction dir)
 {
 	unsigned long flags;
 	struct protection_domain *domain;
 	dma_addr_t addr;
 	u64 dma_mask;
-	phys_addr_t paddr = page_to_phys(page) + offset;
-
-	INC_STATS_COUNTER(cnt_map_single);
 
 	domain = get_domain(dev);
 	if (PTR_ERR(domain) == -EINVAL)
@@ -2624,16 +2628,15 @@ out:
 }
 
 /*
- * The exported unmap_single function for dma_ops.
+ * Wrapper function that contains code common to unmapping a physical address
+ * range from a page or a resource.
  */
-static void unmap_page(struct device *dev, dma_addr_t dma_addr, size_t size,
-		       enum dma_data_direction dir, struct dma_attrs *attrs)
+static void __unmap_phys(struct device *dev, dma_addr_t dma_addr, size_t size,
+			 enum dma_data_direction dir)
 {
 	unsigned long flags;
 	struct protection_domain *domain;
 
-	INC_STATS_COUNTER(cnt_unmap_single);
-
 	domain = get_domain(dev);
 	if (IS_ERR(domain))
 		return;
@@ -2707,6 +2710,72 @@ unmap:
 }
 
 /*
+ * The exported map_single function for dma_ops.
+ */
+static dma_addr_t map_page(struct device *dev, struct page *page,
+			   unsigned long offset, size_t size,
+			   enum dma_data_direction dir,
+			   struct dma_attrs *attrs)
+{
+	INC_STATS_COUNTER(cnt_map_single);
+
+	return __map_phys(dev, page_to_phys(page) + offset, size, dir);
+}
+
+/*
+ * The exported unmap_single function for dma_ops.
+ */
+static void unmap_page(struct device *dev, dma_addr_t dma_addr, size_t size,
+		       enum dma_data_direction dir, struct dma_attrs *attrs)
+{
+	INC_STATS_COUNTER(cnt_unmap_single);
+
+	__unmap_phys(dev, dma_addr, size, dir);
+}
+
+#ifdef CONFIG_HAS_DMA_P2P
+/*
+ * The exported map_peer_resource function for dma_ops.
+ */
+static dma_peer_addr_t map_peer_resource(struct device *dev,
+					 struct device *peer,
+					 struct resource *res,
+					 unsigned long offset,
+					 size_t size,
+					 enum dma_data_direction dir,
+					 struct dma_attrs *attrs)
+{
+	struct pci_dev *pdev;
+	struct pci_dev *ppeer;
+
+	INC_STATS_COUNTER(cnt_map_peer_resource);
+
+	if (!dev_is_pci(dev) || !dev_is_pci(peer))
+		return DMA_ERROR_CODE;
+
+	pdev = to_pci_dev(dev);
+	ppeer = to_pci_dev(peer);
+
+	if (!pci_peer_traffic_supported(pdev, ppeer))
+		return DMA_ERROR_CODE;
+
+	return __map_phys(dev, res->start + offset, size, dir);
+}
+
+/*
+ * The exported unmap_peer_resource function for dma_ops.
+ */
+static void unmap_peer_resource(struct device *dev, dma_peer_addr_t dma_addr,
+				size_t size, enum dma_data_direction dir,
+				struct dma_attrs *attrs)
+{
+	INC_STATS_COUNTER(cnt_unmap_peer_resource);
+
+	__unmap_phys(dev, dma_addr, size, dir);
+}
+#endif
+
+/*
  * The exported map_sg function for dma_ops (handles scatter-gather
  * lists).
  */
@@ -2852,6 +2921,10 @@ static struct dma_map_ops amd_iommu_dma_ops = {
 	.unmap_page = unmap_page,
 	.map_sg = map_sg,
 	.unmap_sg = unmap_sg,
+#ifdef CONFIG_HAS_DMA_P2P
+	.map_peer_resource = map_peer_resource,
+	.unmap_peer_resource = unmap_peer_resource,
+#endif
 	.dma_supported = amd_iommu_dma_supported,
 };
 
-- 
2.5.0

[PATCH v5 11/13] iommu/vt-d: implement (un)map_peer_resource

[Will Davis]

Implement 'map_peer_resource' for the Intel IOMMU driver. Simply translate
the resource to a physical address and route it to the same handlers used
by the 'map_page' API.

This allows a device to map another's resource, to enable peer-to-peer
transactions.

Add behind CONFIG_HAS_DMA_P2P guards, since the dma_map_ops members are
behind them as well.

Signed-off-by: Will Davis <wdavis@nvidia.com>
Reviewed-by: Terence Ripperda <tripperda@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
---
 drivers/iommu/intel-iommu.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index a98a7b2..0d3746f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3544,6 +3544,40 @@ static void intel_unmap_page(struct device *dev, dma_addr_t dev_addr,
 	intel_unmap(dev, dev_addr);
 }
 
+#ifdef CONFIG_HAS_DMA_P2P
+static dma_peer_addr_t intel_map_peer_resource(struct device *dev,
+					       struct device *peer,
+					       struct resource *res,
+					       unsigned long offset,
+					       size_t size,
+					       enum dma_data_direction dir,
+					       struct dma_attrs *attrs)
+{
+	struct pci_dev *pdev;
+	struct pci_dev *ppeer;
+
+	if (!dev_is_pci(dev) || !dev_is_pci(peer))
+		return 0;
+
+	pdev = to_pci_dev(dev);
+	ppeer = to_pci_dev(peer);
+
+	if (!pci_peer_traffic_supported(pdev, ppeer))
+		return 0;
+
+	return __intel_map_single(dev, res->start + offset, size,
+				  dir, *dev->dma_mask);
+}
+
+static void intel_unmap_peer_resource(struct device *dev,
+				      dma_peer_addr_t dev_addr,
+				      size_t size, enum dma_data_direction dir,
+				      struct dma_attrs *attrs)
+{
+	intel_unmap(dev, dev_addr);
+}
+#endif
+
 static void *intel_alloc_coherent(struct device *dev, size_t size,
 				  dma_addr_t *dma_handle, gfp_t flags,
 				  struct dma_attrs *attrs)
@@ -3700,6 +3734,10 @@ struct dma_map_ops intel_dma_ops = {
 	.unmap_sg = intel_unmap_sg,
 	.map_page = intel_map_page,
 	.unmap_page = intel_unmap_page,
+#ifdef CONFIG_HAS_DMA_P2P
+	.map_peer_resource = intel_map_peer_resource,
+	.unmap_peer_resource = intel_unmap_peer_resource,
+#endif
 	.mapping_error = intel_mapping_error,
 };
 
-- 
2.5.0

[PATCH v5 12/13] x86: add pci-nommu implementation of map_peer_resource

[Will Davis]

Add Intel nommu implementation of 'map_peer_resource', which simply checks
whether peer traffic should be allowed and converts the resource to a bus
address.

Add behind CONFIG_HAS_DMA_P2P guards, since the dma_map_ops members are
behind them as well.

Signed-off-by: Will Davis <wdavis@nvidia.com>
---
 arch/x86/kernel/pci-nommu.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index da15918..4b45cc6 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -38,6 +38,39 @@ static dma_addr_t nommu_map_page(struct device *dev, struct page *page,
 	return bus;
 }
 
+#ifdef CONFIG_HAS_DMA_P2P
+static dma_peer_addr_t nommu_map_peer_resource(struct device *dev,
+					       struct device *peer,
+					       struct resource *res,
+					       unsigned long offset,
+					       size_t size,
+					       enum dma_data_direction dir,
+					       struct dma_attrs *attrs)
+{
+	struct pci_dev *pdev;
+	struct pci_dev *ppeer;
+	struct pci_bus_region region;
+	dma_peer_addr_t dma_address;
+
+	if (!dev_is_pci(dev) || !dev_is_pci(peer))
+		return DMA_ERROR_CODE;
+
+	pdev = to_pci_dev(dev);
+	ppeer = to_pci_dev(peer);
+
+	if (!pci_peer_traffic_supported(pdev, ppeer))
+		return DMA_ERROR_CODE;
+
+	pcibios_resource_to_bus(pdev->bus, &region, res);
+	dma_address = region.start + offset;
+	WARN_ON(size == 0);
+	if (!check_addr("map_peer_resource", dev, dma_address, size))
+		return DMA_ERROR_CODE;
+	flush_write_buffers();
+	return dma_address;
+}
+#endif
+
 /* Map a set of buffers described by scatterlist in streaming
  * mode for DMA.  This is the scatter-gather version of the
  * above pci_map_single interface.  Here the scatter gather list
@@ -93,6 +126,9 @@ struct dma_map_ops nommu_dma_ops = {
 	.free			= dma_generic_free_coherent,
 	.map_sg			= nommu_map_sg,
 	.map_page		= nommu_map_page,
+#ifdef CONFIG_HAS_DMA_P2P
+	.map_peer_resource	= nommu_map_peer_resource,
+#endif
 	.sync_single_for_device = nommu_sync_single_for_device,
 	.sync_sg_for_device	= nommu_sync_sg_for_device,
 	.is_phys		= 1,
-- 
2.5.0

[PATCH v5 13/13] x86: declare support for DMA P2P

[Will Davis]

Use CONFIG_HAS_DMA_P2P to declare that we want the DMA peer resource APIs
enabled on x86, and provide the requisite dma_peer_mapping_error()
implementation.

Signed-off-by: Will Davis <wdavis@nvidia.com>
---
 arch/x86/Kconfig                   |  1 +
 arch/x86/include/asm/dma-mapping.h | 18 ++++++++++++++++--
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 3dbb7e7..581d1ad 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -149,6 +149,7 @@ config X86
 	select VIRT_TO_BUS
 	select X86_DEV_DMA_OPS			if X86_64
 	select X86_FEATURE_NAMES		if PROC_FS
+	select HAS_DMA_P2P
 
 config INSTRUCTION_DECODER
 	def_bool y
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 1f5b728..64472d8 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -44,16 +44,30 @@ static inline struct dma_map_ops *get_dma_ops(struct device *dev)
 #include <asm-generic/dma-mapping-common.h>
 
 /* Make sure we keep the same behaviour */
-static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
+static inline int dma_mapping_error_common(struct device *dev, u64 dma_addr)
 {
 	struct dma_map_ops *ops = get_dma_ops(dev);
-	debug_dma_mapping_error(dev, dma_addr);
 	if (ops->mapping_error)
 		return ops->mapping_error(dev, dma_addr);
 
 	return (dma_addr == DMA_ERROR_CODE);
 }
 
+static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
+{
+	debug_dma_mapping_error(dev, dma_addr);
+	return dma_mapping_error_common(dev, (u64)dma_addr);
+}
+
+#ifdef CONFIG_HAS_DMA_P2P
+static inline int dma_peer_mapping_error(struct device *dev,
+					 dma_peer_addr_t dma_addr)
+{
+	debug_dma_peer_mapping_error(dev, dma_addr);
+	return dma_mapping_error_common(dev, (u64)dma_addr);
+}
+#endif
+
 #define dma_alloc_noncoherent(d, s, h, f) dma_alloc_coherent(d, s, h, f)
 #define dma_free_noncoherent(d, s, v, h) dma_free_coherent(d, s, v, h)
 
-- 
2.5.0

Re: [PATCH v5 04/13] DMA-API: Introduce dma_(un)map_peer_resource

[Konrad Wilk]

> +#ifdef CONFIG_HAS_DMA_P2P
> +static inline dma_peer_addr_t dma_map_peer_resource_attrs(struct
> device *dev,
> +							  struct device *peer,
> +							  struct resource *res,
> +							  size_t offset,
> +							  size_t size,
> +							  enum dma_data_direction dir,
> +							  struct dma_attrs *attrs)
> +{
> +	struct dma_map_ops *ops = get_dma_ops(dev);
> +	dma_peer_addr_t addr;
> +
> +	BUG_ON(!valid_dma_direction(dir));
> +	BUG_ON(ops->map_peer_resource == NULL);
> +	addr = ops->map_peer_resource(dev, peer, res, offset, size, dir,
> +				      attrs);
> +	debug_dma_map_peer_resource(dev, peer, res, offset, size, dir,
> addr);
> +
> +	return addr;
> +

With those 'BUG_ON' you need to make sure that every dma_map_ops has
this implemented. That means the calgary_dma_ops, gart_dma_ops, swiotlb_dma_ops,
and xen_swiotlb_dma_ops MUST have something implemented.

Please keep in mind that on one can do 'iommu=soft swiotbl=force' on the
command line which will force the IOMMUs to be ignored and only the
SWIOTLB code will be used.

RE: [PATCH v5 04/13] DMA-API: Introduce dma_(un)map_peer_resource

[William Davis]

PiA+ICsjaWZkZWYgQ09ORklHX0hBU19ETUFfUDJQDQo+ID4gK3N0YXRpYyBpbmxpbmUgZG1hX3Bl
ZXJfYWRkcl90IGRtYV9tYXBfcGVlcl9yZXNvdXJjZV9hdHRycyhzdHJ1Y3QNCj4gPiBkZXZpY2Ug
KmRldiwNCj4gPiArCQkJCQkJCSAgc3RydWN0IGRldmljZSAqcGVlciwNCj4gPiArCQkJCQkJCSAg
c3RydWN0IHJlc291cmNlICpyZXMsDQo+ID4gKwkJCQkJCQkgIHNpemVfdCBvZmZzZXQsDQo+ID4g
KwkJCQkJCQkgIHNpemVfdCBzaXplLA0KPiA+ICsJCQkJCQkJICBlbnVtIGRtYV9kYXRhX2RpcmVj
dGlvbiBkaXIsDQo+ID4gKwkJCQkJCQkgIHN0cnVjdCBkbWFfYXR0cnMgKmF0dHJzKQ0KPiA+ICt7
DQo+ID4gKwlzdHJ1Y3QgZG1hX21hcF9vcHMgKm9wcyA9IGdldF9kbWFfb3BzKGRldik7DQo+ID4g
KwlkbWFfcGVlcl9hZGRyX3QgYWRkcjsNCj4gPiArDQo+ID4gKwlCVUdfT04oIXZhbGlkX2RtYV9k
aXJlY3Rpb24oZGlyKSk7DQo+ID4gKwlCVUdfT04ob3BzLT5tYXBfcGVlcl9yZXNvdXJjZSA9PSBO
VUxMKTsNCj4gPiArCWFkZHIgPSBvcHMtPm1hcF9wZWVyX3Jlc291cmNlKGRldiwgcGVlciwgcmVz
LCBvZmZzZXQsIHNpemUsIGRpciwNCj4gPiArCQkJCSAgICAgIGF0dHJzKTsNCj4gPiArCWRlYnVn
X2RtYV9tYXBfcGVlcl9yZXNvdXJjZShkZXYsIHBlZXIsIHJlcywgb2Zmc2V0LCBzaXplLCBkaXIs
DQo+ID4gYWRkcik7DQo+ID4gKw0KPiA+ICsJcmV0dXJuIGFkZHI7DQo+ID4gKw0KPiANCj4gV2l0
aCB0aG9zZSAnQlVHX09OJyB5b3UgbmVlZCB0byBtYWtlIHN1cmUgdGhhdCBldmVyeSBkbWFfbWFw
X29wcyBoYXMNCj4gdGhpcyBpbXBsZW1lbnRlZC4gVGhhdCBtZWFucyB0aGUgY2FsZ2FyeV9kbWFf
b3BzLCBnYXJ0X2RtYV9vcHMsIHN3aW90bGJfZG1hX29wcywNCj4gYW5kIHhlbl9zd2lvdGxiX2Rt
YV9vcHMgTVVTVCBoYXZlIHNvbWV0aGluZyBpbXBsZW1lbnRlZC4NCj4gDQo+IFBsZWFzZSBrZWVw
IGluIG1pbmQgdGhhdCBvbiBvbmUgY2FuIGRvICdpb21tdT1zb2Z0IHN3aW90Ymw9Zm9yY2UnIG9u
IHRoZQ0KPiBjb21tYW5kIGxpbmUgd2hpY2ggd2lsbCBmb3JjZSB0aGUgSU9NTVVzIHRvIGJlIGln
bm9yZWQgYW5kIG9ubHkgdGhlDQo+IFNXSU9UTEIgY29kZSB3aWxsIGJlIHVzZWQuDQoNClRoYW5r
cyBmb3IgdGhlIGZlZWRiYWNrLCBLb25yYWQuIEkgd2lsbCBhZGQgaW1wbGVtZW50YXRpb25zIGZv
ciB0aG9zZQ0KZG1hX21hcF9vcHMgaW4gdjYuDQoNCldpbGwNCg0KLS0NCm52cHVibGljDQo=