* [PATCH v2 01/26] treewide: Constify most dma_map_ops structures
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
@ 2017-01-12 19:06 ` Bart Van Assche
2017-01-12 19:06 ` [PATCH v2 02/26] treewide: Move dma_ops from struct dev_archdata into struct device Bart Van Assche
` (26 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:06 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Benjamin Herrenschmidt, Boris Ostrovsky, David Woodhouse,
Juergen Gross, H . Peter Anvin, Ingo Molnar, linux-arch,
Russell King, x86
Most dma_map_ops structures are never modified. Constify these
structures such that these can be write-protected. This patch
has been generated as follows:
git grep -l 'struct dma_map_ops' |
xargs -d\\n sed -i \
-e 's/struct dma_map_ops/const struct dma_map_ops/g' \
-e 's/const struct dma_map_ops {/struct dma_map_ops {/g' \
-e 's/^const struct dma_map_ops;$/struct dma_map_ops;/' \
-e 's/const const struct dma_map_ops /const struct dma_map_ops /g';
sed -i -e 's/const \(struct dma_map_ops intel_dma_ops\)/\1/' \
$(git grep -l 'struct dma_map_ops intel_dma_ops');
sed -i -e 's/const \(struct dma_map_ops dma_iommu_ops\)/\1/' \
$(git grep -l 'struct dma_map_ops' | grep ^arch/powerpc);
sed -i -e '/^struct vmd_dev {$/,/^};$/ s/const \(struct dma_map_ops[[:blank:]]dma_ops;\)/\1/' \
-e '/^static void vmd_setup_dma_ops/,/^}$/ s/const \(struct dma_map_ops \*dest\)/\1/' \
-e 's/const \(struct dma_map_ops \*dest = \&vmd->dma_ops\)/\1/' \
drivers/pci/host/*.c
sed -i -e '/^void __init pci_iommu_alloc(void)$/,/^}$/ s/dma_ops->/intel_dma_ops./' arch/ia64/kernel/pci-dma.c
sed -i -e 's/static const struct dma_map_ops sn_dma_ops/static struct dma_map_ops sn_dma_ops/' arch/ia64/sn/pci/pci_dma.c
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Juergen Gross <jgross@suse.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Russell King <linux@armlinux.org.uk>
Cc: x86@kernel.org
---
arch/alpha/include/asm/dma-mapping.h | 4 +--
arch/alpha/kernel/pci-noop.c | 4 +--
arch/alpha/kernel/pci_iommu.c | 4 +--
arch/arc/include/asm/dma-mapping.h | 4 +--
arch/arc/mm/dma.c | 2 +-
arch/arm/common/dmabounce.c | 2 +-
arch/arm/include/asm/device.h | 2 +-
arch/arm/include/asm/dma-mapping.h | 10 +++---
arch/arm/mm/dma-mapping.c | 22 ++++++------
arch/arm/xen/mm.c | 4 +--
arch/arm64/include/asm/device.h | 2 +-
arch/arm64/include/asm/dma-mapping.h | 6 ++--
arch/arm64/mm/dma-mapping.c | 6 ++--
arch/avr32/include/asm/dma-mapping.h | 4 +--
arch/avr32/mm/dma-coherent.c | 2 +-
arch/blackfin/include/asm/dma-mapping.h | 4 +--
arch/blackfin/kernel/dma-mapping.c | 2 +-
arch/c6x/include/asm/dma-mapping.h | 4 +--
arch/c6x/kernel/dma.c | 2 +-
arch/cris/arch-v32/drivers/pci/dma.c | 2 +-
arch/cris/include/asm/dma-mapping.h | 6 ++--
arch/frv/include/asm/dma-mapping.h | 4 +--
arch/frv/mb93090-mb00/pci-dma-nommu.c | 2 +-
arch/frv/mb93090-mb00/pci-dma.c | 2 +-
arch/h8300/include/asm/dma-mapping.h | 4 +--
arch/h8300/kernel/dma.c | 2 +-
arch/hexagon/include/asm/dma-mapping.h | 4 +--
arch/hexagon/kernel/dma.c | 4 +--
arch/ia64/hp/common/hwsw_iommu.c | 4 +--
arch/ia64/hp/common/sba_iommu.c | 4 +--
arch/ia64/include/asm/dma-mapping.h | 2 +-
arch/ia64/include/asm/machvec.h | 4 +--
arch/ia64/kernel/dma-mapping.c | 4 +--
arch/ia64/kernel/pci-dma.c | 10 +++---
arch/ia64/kernel/pci-swiotlb.c | 2 +-
arch/m32r/include/asm/device.h | 2 +-
arch/m32r/include/asm/dma-mapping.h | 2 +-
arch/m68k/include/asm/dma-mapping.h | 4 +--
arch/m68k/kernel/dma.c | 2 +-
arch/metag/include/asm/dma-mapping.h | 4 +--
arch/metag/kernel/dma.c | 2 +-
arch/microblaze/include/asm/dma-mapping.h | 4 +--
arch/microblaze/kernel/dma.c | 2 +-
arch/mips/cavium-octeon/dma-octeon.c | 4 +--
arch/mips/include/asm/device.h | 2 +-
arch/mips/include/asm/dma-mapping.h | 4 +--
.../include/asm/mach-cavium-octeon/dma-coherence.h | 2 +-
arch/mips/include/asm/netlogic/common.h | 2 +-
arch/mips/loongson64/common/dma-swiotlb.c | 2 +-
arch/mips/mm/dma-default.c | 4 +--
arch/mips/netlogic/common/nlm-dma.c | 2 +-
arch/mn10300/include/asm/dma-mapping.h | 4 +--
arch/mn10300/mm/dma-alloc.c | 2 +-
arch/nios2/include/asm/dma-mapping.h | 4 +--
arch/nios2/mm/dma-mapping.c | 2 +-
arch/openrisc/include/asm/dma-mapping.h | 4 +--
arch/openrisc/kernel/dma.c | 2 +-
arch/parisc/include/asm/dma-mapping.h | 8 ++---
arch/parisc/kernel/drivers.c | 2 +-
arch/parisc/kernel/pci-dma.c | 4 +--
arch/powerpc/include/asm/device.h | 2 +-
arch/powerpc/include/asm/dma-mapping.h | 6 ++--
arch/powerpc/include/asm/pci.h | 4 +--
arch/powerpc/include/asm/swiotlb.h | 2 +-
arch/powerpc/kernel/dma-swiotlb.c | 2 +-
arch/powerpc/kernel/dma.c | 6 ++--
arch/powerpc/kernel/pci-common.c | 6 ++--
arch/powerpc/platforms/cell/iommu.c | 4 +--
arch/powerpc/platforms/powernv/npu-dma.c | 2 +-
arch/powerpc/platforms/ps3/system-bus.c | 4 +--
arch/powerpc/platforms/pseries/ibmebus.c | 2 +-
arch/powerpc/platforms/pseries/vio.c | 2 +-
arch/s390/include/asm/device.h | 2 +-
arch/s390/include/asm/dma-mapping.h | 4 +--
arch/s390/pci/pci_dma.c | 2 +-
arch/sh/include/asm/dma-mapping.h | 4 +--
arch/sh/kernel/dma-nommu.c | 2 +-
arch/sh/mm/consistent.c | 2 +-
arch/sparc/include/asm/dma-mapping.h | 8 ++---
arch/sparc/kernel/iommu.c | 4 +--
arch/sparc/kernel/ioport.c | 8 ++---
arch/sparc/kernel/pci_sun4v.c | 2 +-
arch/tile/include/asm/device.h | 2 +-
arch/tile/include/asm/dma-mapping.h | 12 +++----
arch/tile/kernel/pci-dma.c | 24 ++++++-------
arch/unicore32/include/asm/dma-mapping.h | 4 +--
arch/unicore32/mm/dma-swiotlb.c | 2 +-
arch/x86/include/asm/device.h | 4 +--
arch/x86/include/asm/dma-mapping.h | 4 +--
arch/x86/include/asm/iommu.h | 2 +-
arch/x86/kernel/amd_gart_64.c | 2 +-
arch/x86/kernel/pci-calgary_64.c | 2 +-
arch/x86/kernel/pci-dma.c | 4 +--
arch/x86/kernel/pci-nommu.c | 2 +-
arch/x86/kernel/pci-swiotlb.c | 2 +-
arch/x86/pci/sta2x11-fixup.c | 2 +-
arch/x86/xen/pci-swiotlb-xen.c | 2 +-
arch/xtensa/include/asm/device.h | 2 +-
arch/xtensa/include/asm/dma-mapping.h | 4 +--
arch/xtensa/kernel/pci-dma.c | 2 +-
drivers/iommu/amd_iommu.c | 4 +--
drivers/misc/mic/bus/mic_bus.c | 2 +-
drivers/misc/mic/bus/scif_bus.c | 2 +-
drivers/misc/mic/bus/scif_bus.h | 2 +-
drivers/misc/mic/bus/vop_bus.c | 2 +-
drivers/misc/mic/host/mic_boot.c | 4 +--
drivers/parisc/ccio-dma.c | 2 +-
drivers/parisc/sba_iommu.c | 2 +-
drivers/pci/host/vmd.c | 2 +-
include/linux/dma-mapping.h | 42 +++++++++++-----------
include/linux/mic_bus.h | 2 +-
include/xen/arm/hypervisor.h | 2 +-
lib/dma-noop.c | 2 +-
113 files changed, 227 insertions(+), 227 deletions(-)
diff --git a/arch/alpha/include/asm/dma-mapping.h b/arch/alpha/include/asm/dma-mapping.h
index c63b6ac19ee5..d3480562411d 100644
--- a/arch/alpha/include/asm/dma-mapping.h
+++ b/arch/alpha/include/asm/dma-mapping.h
@@ -1,9 +1,9 @@
#ifndef _ALPHA_DMA_MAPPING_H
#define _ALPHA_DMA_MAPPING_H
-extern struct dma_map_ops *dma_ops;
+extern const struct dma_map_ops *dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return dma_ops;
}
diff --git a/arch/alpha/kernel/pci-noop.c b/arch/alpha/kernel/pci-noop.c
index bb152e21e5ae..ffbdb3fb672f 100644
--- a/arch/alpha/kernel/pci-noop.c
+++ b/arch/alpha/kernel/pci-noop.c
@@ -128,7 +128,7 @@ static int alpha_noop_supported(struct device *dev, u64 mask)
return mask < 0x00ffffffUL ? 0 : 1;
}
-struct dma_map_ops alpha_noop_ops = {
+const struct dma_map_ops alpha_noop_ops = {
.alloc = alpha_noop_alloc_coherent,
.free = dma_noop_free_coherent,
.map_page = dma_noop_map_page,
@@ -137,5 +137,5 @@ struct dma_map_ops alpha_noop_ops = {
.dma_supported = alpha_noop_supported,
};
-struct dma_map_ops *dma_ops = &alpha_noop_ops;
+const struct dma_map_ops *dma_ops = &alpha_noop_ops;
EXPORT_SYMBOL(dma_ops);
diff --git a/arch/alpha/kernel/pci_iommu.c b/arch/alpha/kernel/pci_iommu.c
index 451fc9cdd323..7fd2329038a3 100644
--- a/arch/alpha/kernel/pci_iommu.c
+++ b/arch/alpha/kernel/pci_iommu.c
@@ -939,7 +939,7 @@ static int alpha_pci_mapping_error(struct device *dev, dma_addr_t dma_addr)
return dma_addr == 0;
}
-struct dma_map_ops alpha_pci_ops = {
+const struct dma_map_ops alpha_pci_ops = {
.alloc = alpha_pci_alloc_coherent,
.free = alpha_pci_free_coherent,
.map_page = alpha_pci_map_page,
@@ -950,5 +950,5 @@ struct dma_map_ops alpha_pci_ops = {
.dma_supported = alpha_pci_supported,
};
-struct dma_map_ops *dma_ops = &alpha_pci_ops;
+const struct dma_map_ops *dma_ops = &alpha_pci_ops;
EXPORT_SYMBOL(dma_ops);
diff --git a/arch/arc/include/asm/dma-mapping.h b/arch/arc/include/asm/dma-mapping.h
index 266f11c9bd59..fdff3aa60052 100644
--- a/arch/arc/include/asm/dma-mapping.h
+++ b/arch/arc/include/asm/dma-mapping.h
@@ -18,9 +18,9 @@
#include <plat/dma.h>
#endif
-extern struct dma_map_ops arc_dma_ops;
+extern const struct dma_map_ops arc_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &arc_dma_ops;
}
diff --git a/arch/arc/mm/dma.c b/arch/arc/mm/dma.c
index 08450a1a5b5f..2a07e6ecafbd 100644
--- a/arch/arc/mm/dma.c
+++ b/arch/arc/mm/dma.c
@@ -218,7 +218,7 @@ static int arc_dma_supported(struct device *dev, u64 dma_mask)
return dma_mask == DMA_BIT_MASK(32);
}
-struct dma_map_ops arc_dma_ops = {
+const struct dma_map_ops arc_dma_ops = {
.alloc = arc_dma_alloc,
.free = arc_dma_free,
.mmap = arc_dma_mmap,
diff --git a/arch/arm/common/dmabounce.c b/arch/arm/common/dmabounce.c
index 75055df1cda3..9b1b7be2ec0e 100644
--- a/arch/arm/common/dmabounce.c
+++ b/arch/arm/common/dmabounce.c
@@ -452,7 +452,7 @@ static int dmabounce_set_mask(struct device *dev, u64 dma_mask)
return arm_dma_ops.set_dma_mask(dev, dma_mask);
}
-static struct dma_map_ops dmabounce_ops = {
+static const struct dma_map_ops dmabounce_ops = {
.alloc = arm_dma_alloc,
.free = arm_dma_free,
.mmap = arm_dma_mmap,
diff --git a/arch/arm/include/asm/device.h b/arch/arm/include/asm/device.h
index 4111592f0130..d8a572f9c187 100644
--- a/arch/arm/include/asm/device.h
+++ b/arch/arm/include/asm/device.h
@@ -7,7 +7,7 @@
#define ASMARM_DEVICE_H
struct dev_archdata {
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
#ifdef CONFIG_DMABOUNCE
struct dmabounce_device_info *dmabounce;
#endif
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index bf02dbd9ccda..1aabd781306f 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -13,17 +13,17 @@
#include <asm/xen/hypervisor.h>
#define DMA_ERROR_CODE (~(dma_addr_t)0x0)
-extern struct dma_map_ops arm_dma_ops;
-extern struct dma_map_ops arm_coherent_dma_ops;
+extern const struct dma_map_ops arm_dma_ops;
+extern const struct dma_map_ops arm_coherent_dma_ops;
-static inline struct dma_map_ops *__generic_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *__generic_dma_ops(struct device *dev)
{
if (dev && dev->archdata.dma_ops)
return dev->archdata.dma_ops;
return &arm_dma_ops;
}
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
if (xen_initial_domain())
return xen_dma_ops;
@@ -31,7 +31,7 @@ static inline struct dma_map_ops *get_dma_ops(struct device *dev)
return __generic_dma_ops(dev);
}
-static inline void set_dma_ops(struct device *dev, struct dma_map_ops *ops)
+static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
{
BUG_ON(!dev);
dev->archdata.dma_ops = ops;
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index ab7710002ba6..d26fe1a35687 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -180,7 +180,7 @@ static void arm_dma_sync_single_for_device(struct device *dev,
__dma_page_cpu_to_dev(page, offset, size, dir);
}
-struct dma_map_ops arm_dma_ops = {
+const struct dma_map_ops arm_dma_ops = {
.alloc = arm_dma_alloc,
.free = arm_dma_free,
.mmap = arm_dma_mmap,
@@ -204,7 +204,7 @@ static int arm_coherent_dma_mmap(struct device *dev, struct vm_area_struct *vma,
void *cpu_addr, dma_addr_t dma_addr, size_t size,
unsigned long attrs);
-struct dma_map_ops arm_coherent_dma_ops = {
+const struct dma_map_ops arm_coherent_dma_ops = {
.alloc = arm_coherent_dma_alloc,
.free = arm_coherent_dma_free,
.mmap = arm_coherent_dma_mmap,
@@ -1067,7 +1067,7 @@ static void __dma_page_dev_to_cpu(struct page *page, unsigned long off,
int arm_dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
enum dma_data_direction dir, unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
struct scatterlist *s;
int i, j;
@@ -1101,7 +1101,7 @@ int arm_dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
void arm_dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents,
enum dma_data_direction dir, unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
struct scatterlist *s;
int i;
@@ -1120,7 +1120,7 @@ void arm_dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents,
void arm_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
int nents, enum dma_data_direction dir)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
struct scatterlist *s;
int i;
@@ -1139,7 +1139,7 @@ void arm_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
void arm_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
int nents, enum dma_data_direction dir)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
struct scatterlist *s;
int i;
@@ -2099,7 +2099,7 @@ static void arm_iommu_sync_single_for_device(struct device *dev,
__dma_page_cpu_to_dev(page, offset, size, dir);
}
-struct dma_map_ops iommu_ops = {
+const struct dma_map_ops iommu_ops = {
.alloc = arm_iommu_alloc_attrs,
.free = arm_iommu_free_attrs,
.mmap = arm_iommu_mmap_attrs,
@@ -2119,7 +2119,7 @@ struct dma_map_ops iommu_ops = {
.unmap_resource = arm_iommu_unmap_resource,
};
-struct dma_map_ops iommu_coherent_ops = {
+const struct dma_map_ops iommu_coherent_ops = {
.alloc = arm_coherent_iommu_alloc_attrs,
.free = arm_coherent_iommu_free_attrs,
.mmap = arm_coherent_iommu_mmap_attrs,
@@ -2319,7 +2319,7 @@ void arm_iommu_detach_device(struct device *dev)
}
EXPORT_SYMBOL_GPL(arm_iommu_detach_device);
-static struct dma_map_ops *arm_get_iommu_dma_map_ops(bool coherent)
+static const struct dma_map_ops *arm_get_iommu_dma_map_ops(bool coherent)
{
return coherent ? &iommu_coherent_ops : &iommu_ops;
}
@@ -2374,7 +2374,7 @@ static void arm_teardown_iommu_dma_ops(struct device *dev) { }
#endif /* CONFIG_ARM_DMA_USE_IOMMU */
-static struct dma_map_ops *arm_get_dma_map_ops(bool coherent)
+static const struct dma_map_ops *arm_get_dma_map_ops(bool coherent)
{
return coherent ? &arm_coherent_dma_ops : &arm_dma_ops;
}
@@ -2382,7 +2382,7 @@ static struct dma_map_ops *arm_get_dma_map_ops(bool coherent)
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *iommu, bool coherent)
{
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
dev->archdata.dma_coherent = coherent;
if (arm_setup_iommu_dma_ops(dev, dma_base, size, iommu))
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index bd62d94f8ac5..ce18c91b50a1 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -182,10 +182,10 @@ void xen_destroy_contiguous_region(phys_addr_t pstart, unsigned int order)
}
EXPORT_SYMBOL_GPL(xen_destroy_contiguous_region);
-struct dma_map_ops *xen_dma_ops;
+const struct dma_map_ops *xen_dma_ops;
EXPORT_SYMBOL(xen_dma_ops);
-static struct dma_map_ops xen_swiotlb_dma_ops = {
+static const struct dma_map_ops xen_swiotlb_dma_ops = {
.alloc = xen_swiotlb_alloc_coherent,
.free = xen_swiotlb_free_coherent,
.sync_single_for_cpu = xen_swiotlb_sync_single_for_cpu,
diff --git a/arch/arm64/include/asm/device.h b/arch/arm64/include/asm/device.h
index 243ef256b8c9..00c678cc31e1 100644
--- a/arch/arm64/include/asm/device.h
+++ b/arch/arm64/include/asm/device.h
@@ -17,7 +17,7 @@
#define __ASM_DEVICE_H
struct dev_archdata {
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
#ifdef CONFIG_IOMMU_API
void *iommu; /* private IOMMU data */
#endif
diff --git a/arch/arm64/include/asm/dma-mapping.h b/arch/arm64/include/asm/dma-mapping.h
index ccea82c2b089..1fedb43be712 100644
--- a/arch/arm64/include/asm/dma-mapping.h
+++ b/arch/arm64/include/asm/dma-mapping.h
@@ -25,9 +25,9 @@
#include <asm/xen/hypervisor.h>
#define DMA_ERROR_CODE (~(dma_addr_t)0)
-extern struct dma_map_ops dummy_dma_ops;
+extern const struct dma_map_ops dummy_dma_ops;
-static inline struct dma_map_ops *__generic_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *__generic_dma_ops(struct device *dev)
{
if (dev && dev->archdata.dma_ops)
return dev->archdata.dma_ops;
@@ -39,7 +39,7 @@ static inline struct dma_map_ops *__generic_dma_ops(struct device *dev)
return &dummy_dma_ops;
}
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
if (xen_initial_domain())
return xen_dma_ops;
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index e04082700bb1..bcef6368d48f 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -352,7 +352,7 @@ static int __swiotlb_dma_supported(struct device *hwdev, u64 mask)
return 1;
}
-static struct dma_map_ops swiotlb_dma_ops = {
+static const struct dma_map_ops swiotlb_dma_ops = {
.alloc = __dma_alloc,
.free = __dma_free,
.mmap = __swiotlb_mmap,
@@ -505,7 +505,7 @@ static int __dummy_dma_supported(struct device *hwdev, u64 mask)
return 0;
}
-struct dma_map_ops dummy_dma_ops = {
+const struct dma_map_ops dummy_dma_ops = {
.alloc = __dummy_alloc,
.free = __dummy_free,
.mmap = __dummy_mmap,
@@ -784,7 +784,7 @@ static void __iommu_unmap_sg_attrs(struct device *dev,
iommu_dma_unmap_sg(dev, sgl, nelems, dir, attrs);
}
-static struct dma_map_ops iommu_dma_ops = {
+static const struct dma_map_ops iommu_dma_ops = {
.alloc = __iommu_alloc_attrs,
.free = __iommu_free_attrs,
.mmap = __iommu_mmap_attrs,
diff --git a/arch/avr32/include/asm/dma-mapping.h b/arch/avr32/include/asm/dma-mapping.h
index 1115f2a645d1..b2b43c0e0774 100644
--- a/arch/avr32/include/asm/dma-mapping.h
+++ b/arch/avr32/include/asm/dma-mapping.h
@@ -4,9 +4,9 @@
extern void dma_cache_sync(struct device *dev, void *vaddr, size_t size,
int direction);
-extern struct dma_map_ops avr32_dma_ops;
+extern const struct dma_map_ops avr32_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &avr32_dma_ops;
}
diff --git a/arch/avr32/mm/dma-coherent.c b/arch/avr32/mm/dma-coherent.c
index 54534e5d0781..555222d4f414 100644
--- a/arch/avr32/mm/dma-coherent.c
+++ b/arch/avr32/mm/dma-coherent.c
@@ -191,7 +191,7 @@ static void avr32_dma_sync_sg_for_device(struct device *dev,
dma_cache_sync(dev, sg_virt(sg), sg->length, direction);
}
-struct dma_map_ops avr32_dma_ops = {
+const struct dma_map_ops avr32_dma_ops = {
.alloc = avr32_dma_alloc,
.free = avr32_dma_free,
.map_page = avr32_dma_map_page,
diff --git a/arch/blackfin/include/asm/dma-mapping.h b/arch/blackfin/include/asm/dma-mapping.h
index 3490570aaa82..320fb50fbd41 100644
--- a/arch/blackfin/include/asm/dma-mapping.h
+++ b/arch/blackfin/include/asm/dma-mapping.h
@@ -36,9 +36,9 @@ _dma_sync(dma_addr_t addr, size_t size, enum dma_data_direction dir)
__dma_sync(addr, size, dir);
}
-extern struct dma_map_ops bfin_dma_ops;
+extern const struct dma_map_ops bfin_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &bfin_dma_ops;
}
diff --git a/arch/blackfin/kernel/dma-mapping.c b/arch/blackfin/kernel/dma-mapping.c
index a27a74a18fb0..477bb29a7987 100644
--- a/arch/blackfin/kernel/dma-mapping.c
+++ b/arch/blackfin/kernel/dma-mapping.c
@@ -159,7 +159,7 @@ static inline void bfin_dma_sync_single_for_device(struct device *dev,
_dma_sync(handle, size, dir);
}
-struct dma_map_ops bfin_dma_ops = {
+const struct dma_map_ops bfin_dma_ops = {
.alloc = bfin_dma_alloc,
.free = bfin_dma_free,
diff --git a/arch/c6x/include/asm/dma-mapping.h b/arch/c6x/include/asm/dma-mapping.h
index 5717b1e52d96..88258b9ebc8e 100644
--- a/arch/c6x/include/asm/dma-mapping.h
+++ b/arch/c6x/include/asm/dma-mapping.h
@@ -17,9 +17,9 @@
*/
#define DMA_ERROR_CODE ~0
-extern struct dma_map_ops c6x_dma_ops;
+extern const struct dma_map_ops c6x_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &c6x_dma_ops;
}
diff --git a/arch/c6x/kernel/dma.c b/arch/c6x/kernel/dma.c
index 6752df32ef06..9fff8be75f58 100644
--- a/arch/c6x/kernel/dma.c
+++ b/arch/c6x/kernel/dma.c
@@ -123,7 +123,7 @@ static void c6x_dma_sync_sg_for_device(struct device *dev,
}
-struct dma_map_ops c6x_dma_ops = {
+const struct dma_map_ops c6x_dma_ops = {
.alloc = c6x_dma_alloc,
.free = c6x_dma_free,
.map_page = c6x_dma_map_page,
diff --git a/arch/cris/arch-v32/drivers/pci/dma.c b/arch/cris/arch-v32/drivers/pci/dma.c
index 1f0636793f0c..7072341995ff 100644
--- a/arch/cris/arch-v32/drivers/pci/dma.c
+++ b/arch/cris/arch-v32/drivers/pci/dma.c
@@ -69,7 +69,7 @@ static inline int v32_dma_supported(struct device *dev, u64 mask)
return 1;
}
-struct dma_map_ops v32_dma_ops = {
+const struct dma_map_ops v32_dma_ops = {
.alloc = v32_dma_alloc,
.free = v32_dma_free,
.map_page = v32_dma_map_page,
diff --git a/arch/cris/include/asm/dma-mapping.h b/arch/cris/include/asm/dma-mapping.h
index 5a370178a0e9..aae4fbc0a656 100644
--- a/arch/cris/include/asm/dma-mapping.h
+++ b/arch/cris/include/asm/dma-mapping.h
@@ -2,14 +2,14 @@
#define _ASM_CRIS_DMA_MAPPING_H
#ifdef CONFIG_PCI
-extern struct dma_map_ops v32_dma_ops;
+extern const struct dma_map_ops v32_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &v32_dma_ops;
}
#else
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
BUG();
return NULL;
diff --git a/arch/frv/include/asm/dma-mapping.h b/arch/frv/include/asm/dma-mapping.h
index 9a82bfa4303b..150cc00544a8 100644
--- a/arch/frv/include/asm/dma-mapping.h
+++ b/arch/frv/include/asm/dma-mapping.h
@@ -7,9 +7,9 @@
extern unsigned long __nongprelbss dma_coherent_mem_start;
extern unsigned long __nongprelbss dma_coherent_mem_end;
-extern struct dma_map_ops frv_dma_ops;
+extern const struct dma_map_ops frv_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &frv_dma_ops;
}
diff --git a/arch/frv/mb93090-mb00/pci-dma-nommu.c b/arch/frv/mb93090-mb00/pci-dma-nommu.c
index 187688128c65..4a96de7f0af4 100644
--- a/arch/frv/mb93090-mb00/pci-dma-nommu.c
+++ b/arch/frv/mb93090-mb00/pci-dma-nommu.c
@@ -164,7 +164,7 @@ static int frv_dma_supported(struct device *dev, u64 mask)
return 1;
}
-struct dma_map_ops frv_dma_ops = {
+const struct dma_map_ops frv_dma_ops = {
.alloc = frv_dma_alloc,
.free = frv_dma_free,
.map_page = frv_dma_map_page,
diff --git a/arch/frv/mb93090-mb00/pci-dma.c b/arch/frv/mb93090-mb00/pci-dma.c
index dba7df918144..e7130abc0dae 100644
--- a/arch/frv/mb93090-mb00/pci-dma.c
+++ b/arch/frv/mb93090-mb00/pci-dma.c
@@ -106,7 +106,7 @@ static int frv_dma_supported(struct device *dev, u64 mask)
return 1;
}
-struct dma_map_ops frv_dma_ops = {
+const struct dma_map_ops frv_dma_ops = {
.alloc = frv_dma_alloc,
.free = frv_dma_free,
.map_page = frv_dma_map_page,
diff --git a/arch/h8300/include/asm/dma-mapping.h b/arch/h8300/include/asm/dma-mapping.h
index 7ac7fadffed0..f804bca4c13f 100644
--- a/arch/h8300/include/asm/dma-mapping.h
+++ b/arch/h8300/include/asm/dma-mapping.h
@@ -1,9 +1,9 @@
#ifndef _H8300_DMA_MAPPING_H
#define _H8300_DMA_MAPPING_H
-extern struct dma_map_ops h8300_dma_map_ops;
+extern const struct dma_map_ops h8300_dma_map_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &h8300_dma_map_ops;
}
diff --git a/arch/h8300/kernel/dma.c b/arch/h8300/kernel/dma.c
index 3651da045806..225dd0a188dc 100644
--- a/arch/h8300/kernel/dma.c
+++ b/arch/h8300/kernel/dma.c
@@ -60,7 +60,7 @@ static int map_sg(struct device *dev, struct scatterlist *sgl,
return nents;
}
-struct dma_map_ops h8300_dma_map_ops = {
+const struct dma_map_ops h8300_dma_map_ops = {
.alloc = dma_alloc,
.free = dma_free,
.map_page = map_page,
diff --git a/arch/hexagon/include/asm/dma-mapping.h b/arch/hexagon/include/asm/dma-mapping.h
index 7ef58df909fc..b812e917cd95 100644
--- a/arch/hexagon/include/asm/dma-mapping.h
+++ b/arch/hexagon/include/asm/dma-mapping.h
@@ -32,9 +32,9 @@ struct device;
extern int bad_dma_address;
#define DMA_ERROR_CODE bad_dma_address
-extern struct dma_map_ops *dma_ops;
+extern const struct dma_map_ops *dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
if (unlikely(dev == NULL))
return NULL;
diff --git a/arch/hexagon/kernel/dma.c b/arch/hexagon/kernel/dma.c
index dbc4f1003da4..e74b65009587 100644
--- a/arch/hexagon/kernel/dma.c
+++ b/arch/hexagon/kernel/dma.c
@@ -25,7 +25,7 @@
#include <linux/module.h>
#include <asm/page.h>
-struct dma_map_ops *dma_ops;
+const struct dma_map_ops *dma_ops;
EXPORT_SYMBOL(dma_ops);
int bad_dma_address; /* globals are automatically initialized to zero */
@@ -203,7 +203,7 @@ static void hexagon_sync_single_for_device(struct device *dev,
dma_sync(dma_addr_to_virt(dma_handle), size, dir);
}
-struct dma_map_ops hexagon_dma_ops = {
+const struct dma_map_ops hexagon_dma_ops = {
.alloc = hexagon_dma_alloc_coherent,
.free = hexagon_free_coherent,
.map_sg = hexagon_map_sg,
diff --git a/arch/ia64/hp/common/hwsw_iommu.c b/arch/ia64/hp/common/hwsw_iommu.c
index 1e4cae5ae053..0310078a95f8 100644
--- a/arch/ia64/hp/common/hwsw_iommu.c
+++ b/arch/ia64/hp/common/hwsw_iommu.c
@@ -18,7 +18,7 @@
#include <linux/export.h>
#include <asm/machvec.h>
-extern struct dma_map_ops sba_dma_ops, swiotlb_dma_ops;
+extern const struct dma_map_ops sba_dma_ops, swiotlb_dma_ops;
/* swiotlb declarations & definitions: */
extern int swiotlb_late_init_with_default_size (size_t size);
@@ -34,7 +34,7 @@ static inline int use_swiotlb(struct device *dev)
!sba_dma_ops.dma_supported(dev, *dev->dma_mask);
}
-struct dma_map_ops *hwsw_dma_get_ops(struct device *dev)
+const struct dma_map_ops *hwsw_dma_get_ops(struct device *dev)
{
if (use_swiotlb(dev))
return &swiotlb_dma_ops;
diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
index 630ee8073899..aec4a3354abe 100644
--- a/arch/ia64/hp/common/sba_iommu.c
+++ b/arch/ia64/hp/common/sba_iommu.c
@@ -2096,7 +2096,7 @@ static int __init acpi_sba_ioc_init_acpi(void)
/* This has to run before acpi_scan_init(). */
arch_initcall(acpi_sba_ioc_init_acpi);
-extern struct dma_map_ops swiotlb_dma_ops;
+extern const struct dma_map_ops swiotlb_dma_ops;
static int __init
sba_init(void)
@@ -2216,7 +2216,7 @@ sba_page_override(char *str)
__setup("sbapagesize=",sba_page_override);
-struct dma_map_ops sba_dma_ops = {
+const struct dma_map_ops sba_dma_ops = {
.alloc = sba_alloc_coherent,
.free = sba_free_coherent,
.map_page = sba_map_page,
diff --git a/arch/ia64/include/asm/dma-mapping.h b/arch/ia64/include/asm/dma-mapping.h
index d472805edfa9..05e467d56d86 100644
--- a/arch/ia64/include/asm/dma-mapping.h
+++ b/arch/ia64/include/asm/dma-mapping.h
@@ -14,7 +14,7 @@
#define DMA_ERROR_CODE 0
-extern struct dma_map_ops *dma_ops;
+extern const struct dma_map_ops *dma_ops;
extern struct ia64_machine_vector ia64_mv;
extern void set_iommu_machvec(void);
diff --git a/arch/ia64/include/asm/machvec.h b/arch/ia64/include/asm/machvec.h
index ed7f09089f12..af285c423e1e 100644
--- a/arch/ia64/include/asm/machvec.h
+++ b/arch/ia64/include/asm/machvec.h
@@ -44,7 +44,7 @@ typedef void ia64_mv_kernel_launch_event_t(void);
/* DMA-mapping interface: */
typedef void ia64_mv_dma_init (void);
typedef u64 ia64_mv_dma_get_required_mask (struct device *);
-typedef struct dma_map_ops *ia64_mv_dma_get_ops(struct device *);
+typedef const struct dma_map_ops *ia64_mv_dma_get_ops(struct device *);
/*
* WARNING: The legacy I/O space is _architected_. Platforms are
@@ -248,7 +248,7 @@ extern void machvec_init_from_cmdline(const char *cmdline);
# endif /* CONFIG_IA64_GENERIC */
extern void swiotlb_dma_init(void);
-extern struct dma_map_ops *dma_get_ops(struct device *);
+extern const struct dma_map_ops *dma_get_ops(struct device *);
/*
* Define default versions so we can extend machvec for new platforms without having
diff --git a/arch/ia64/kernel/dma-mapping.c b/arch/ia64/kernel/dma-mapping.c
index 7f7916238208..e0dd97f4eb69 100644
--- a/arch/ia64/kernel/dma-mapping.c
+++ b/arch/ia64/kernel/dma-mapping.c
@@ -4,7 +4,7 @@
/* Set this to 1 if there is a HW IOMMU in the system */
int iommu_detected __read_mostly;
-struct dma_map_ops *dma_ops;
+const struct dma_map_ops *dma_ops;
EXPORT_SYMBOL(dma_ops);
#define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16)
@@ -17,7 +17,7 @@ static int __init dma_init(void)
}
fs_initcall(dma_init);
-struct dma_map_ops *dma_get_ops(struct device *dev)
+const struct dma_map_ops *dma_get_ops(struct device *dev)
{
return dma_ops;
}
diff --git a/arch/ia64/kernel/pci-dma.c b/arch/ia64/kernel/pci-dma.c
index 992c1098c522..9094a73f996f 100644
--- a/arch/ia64/kernel/pci-dma.c
+++ b/arch/ia64/kernel/pci-dma.c
@@ -90,11 +90,11 @@ void __init pci_iommu_alloc(void)
{
dma_ops = &intel_dma_ops;
- dma_ops->sync_single_for_cpu = machvec_dma_sync_single;
- dma_ops->sync_sg_for_cpu = machvec_dma_sync_sg;
- dma_ops->sync_single_for_device = machvec_dma_sync_single;
- dma_ops->sync_sg_for_device = machvec_dma_sync_sg;
- dma_ops->dma_supported = iommu_dma_supported;
+ intel_dma_ops.sync_single_for_cpu = machvec_dma_sync_single;
+ intel_dma_ops.sync_sg_for_cpu = machvec_dma_sync_sg;
+ intel_dma_ops.sync_single_for_device = machvec_dma_sync_single;
+ intel_dma_ops.sync_sg_for_device = machvec_dma_sync_sg;
+ intel_dma_ops.dma_supported = iommu_dma_supported;
/*
* The order of these functions is important for
diff --git a/arch/ia64/kernel/pci-swiotlb.c b/arch/ia64/kernel/pci-swiotlb.c
index 2933208c0285..a14989dacded 100644
--- a/arch/ia64/kernel/pci-swiotlb.c
+++ b/arch/ia64/kernel/pci-swiotlb.c
@@ -30,7 +30,7 @@ static void ia64_swiotlb_free_coherent(struct device *dev, size_t size,
swiotlb_free_coherent(dev, size, vaddr, dma_addr);
}
-struct dma_map_ops swiotlb_dma_ops = {
+const struct dma_map_ops swiotlb_dma_ops = {
.alloc = ia64_swiotlb_alloc_coherent,
.free = ia64_swiotlb_free_coherent,
.map_page = swiotlb_map_page,
diff --git a/arch/m32r/include/asm/device.h b/arch/m32r/include/asm/device.h
index 4a9f35e0973f..7955a9799466 100644
--- a/arch/m32r/include/asm/device.h
+++ b/arch/m32r/include/asm/device.h
@@ -4,7 +4,7 @@
* This file is released under the GPLv2
*/
struct dev_archdata {
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
};
struct pdev_archdata {
diff --git a/arch/m32r/include/asm/dma-mapping.h b/arch/m32r/include/asm/dma-mapping.h
index 2c43a77fe942..99c43d2f05dc 100644
--- a/arch/m32r/include/asm/dma-mapping.h
+++ b/arch/m32r/include/asm/dma-mapping.h
@@ -10,7 +10,7 @@
#define DMA_ERROR_CODE (~(dma_addr_t)0x0)
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
if (dev && dev->archdata.dma_ops)
return dev->archdata.dma_ops;
diff --git a/arch/m68k/include/asm/dma-mapping.h b/arch/m68k/include/asm/dma-mapping.h
index 96c536194287..863509939d5a 100644
--- a/arch/m68k/include/asm/dma-mapping.h
+++ b/arch/m68k/include/asm/dma-mapping.h
@@ -1,9 +1,9 @@
#ifndef _M68K_DMA_MAPPING_H
#define _M68K_DMA_MAPPING_H
-extern struct dma_map_ops m68k_dma_ops;
+extern const struct dma_map_ops m68k_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &m68k_dma_ops;
}
diff --git a/arch/m68k/kernel/dma.c b/arch/m68k/kernel/dma.c
index 07070065a425..0fc5dabb4a42 100644
--- a/arch/m68k/kernel/dma.c
+++ b/arch/m68k/kernel/dma.c
@@ -158,7 +158,7 @@ static int m68k_dma_map_sg(struct device *dev, struct scatterlist *sglist,
return nents;
}
-struct dma_map_ops m68k_dma_ops = {
+const struct dma_map_ops m68k_dma_ops = {
.alloc = m68k_dma_alloc,
.free = m68k_dma_free,
.map_page = m68k_dma_map_page,
diff --git a/arch/metag/include/asm/dma-mapping.h b/arch/metag/include/asm/dma-mapping.h
index 27af5d479ce6..c156a7ac732f 100644
--- a/arch/metag/include/asm/dma-mapping.h
+++ b/arch/metag/include/asm/dma-mapping.h
@@ -1,9 +1,9 @@
#ifndef _ASM_METAG_DMA_MAPPING_H
#define _ASM_METAG_DMA_MAPPING_H
-extern struct dma_map_ops metag_dma_ops;
+extern const struct dma_map_ops metag_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &metag_dma_ops;
}
diff --git a/arch/metag/kernel/dma.c b/arch/metag/kernel/dma.c
index 91968d92652b..f0ab3a498328 100644
--- a/arch/metag/kernel/dma.c
+++ b/arch/metag/kernel/dma.c
@@ -575,7 +575,7 @@ static void metag_dma_sync_sg_for_device(struct device *dev,
dma_sync_for_device(sg_virt(sg), sg->length, direction);
}
-struct dma_map_ops metag_dma_ops = {
+const struct dma_map_ops metag_dma_ops = {
.alloc = metag_dma_alloc,
.free = metag_dma_free,
.map_page = metag_dma_map_page,
diff --git a/arch/microblaze/include/asm/dma-mapping.h b/arch/microblaze/include/asm/dma-mapping.h
index 1768d4bdc8d3..c7faf2fb51d6 100644
--- a/arch/microblaze/include/asm/dma-mapping.h
+++ b/arch/microblaze/include/asm/dma-mapping.h
@@ -36,9 +36,9 @@
/*
* Available generic sets of operations
*/
-extern struct dma_map_ops dma_direct_ops;
+extern const struct dma_map_ops dma_direct_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &dma_direct_ops;
}
diff --git a/arch/microblaze/kernel/dma.c b/arch/microblaze/kernel/dma.c
index 818daf230eb4..12e093a03e60 100644
--- a/arch/microblaze/kernel/dma.c
+++ b/arch/microblaze/kernel/dma.c
@@ -187,7 +187,7 @@ int dma_direct_mmap_coherent(struct device *dev, struct vm_area_struct *vma,
#endif
}
-struct dma_map_ops dma_direct_ops = {
+const struct dma_map_ops dma_direct_ops = {
.alloc = dma_direct_alloc_coherent,
.free = dma_direct_free_coherent,
.mmap = dma_direct_mmap_coherent,
diff --git a/arch/mips/cavium-octeon/dma-octeon.c b/arch/mips/cavium-octeon/dma-octeon.c
index fd69528b24fb..897d32c888ee 100644
--- a/arch/mips/cavium-octeon/dma-octeon.c
+++ b/arch/mips/cavium-octeon/dma-octeon.c
@@ -205,7 +205,7 @@ static phys_addr_t octeon_unity_dma_to_phys(struct device *dev, dma_addr_t daddr
}
struct octeon_dma_map_ops {
- struct dma_map_ops dma_map_ops;
+ const struct dma_map_ops dma_map_ops;
dma_addr_t (*phys_to_dma)(struct device *dev, phys_addr_t paddr);
phys_addr_t (*dma_to_phys)(struct device *dev, dma_addr_t daddr);
};
@@ -333,7 +333,7 @@ static struct octeon_dma_map_ops _octeon_pci_dma_map_ops = {
},
};
-struct dma_map_ops *octeon_pci_dma_map_ops;
+const struct dma_map_ops *octeon_pci_dma_map_ops;
void __init octeon_pci_dma_init(void)
{
diff --git a/arch/mips/include/asm/device.h b/arch/mips/include/asm/device.h
index 21c2082a0dfb..ebc5c1265473 100644
--- a/arch/mips/include/asm/device.h
+++ b/arch/mips/include/asm/device.h
@@ -10,7 +10,7 @@ struct dma_map_ops;
struct dev_archdata {
/* DMA operations on that device */
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
#ifdef CONFIG_DMA_PERDEV_COHERENT
/* Non-zero if DMA is coherent with CPU caches */
diff --git a/arch/mips/include/asm/dma-mapping.h b/arch/mips/include/asm/dma-mapping.h
index 7aa71b9b0258..b59b084a7569 100644
--- a/arch/mips/include/asm/dma-mapping.h
+++ b/arch/mips/include/asm/dma-mapping.h
@@ -9,9 +9,9 @@
#include <dma-coherence.h>
#endif
-extern struct dma_map_ops *mips_dma_map_ops;
+extern const struct dma_map_ops *mips_dma_map_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
if (dev && dev->archdata.dma_ops)
return dev->archdata.dma_ops;
diff --git a/arch/mips/include/asm/mach-cavium-octeon/dma-coherence.h b/arch/mips/include/asm/mach-cavium-octeon/dma-coherence.h
index 460042ee5d6f..9110988b92a1 100644
--- a/arch/mips/include/asm/mach-cavium-octeon/dma-coherence.h
+++ b/arch/mips/include/asm/mach-cavium-octeon/dma-coherence.h
@@ -65,7 +65,7 @@ dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr);
phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr);
struct dma_map_ops;
-extern struct dma_map_ops *octeon_pci_dma_map_ops;
+extern const struct dma_map_ops *octeon_pci_dma_map_ops;
extern char *octeon_swiotlb;
#endif /* __ASM_MACH_CAVIUM_OCTEON_DMA_COHERENCE_H */
diff --git a/arch/mips/include/asm/netlogic/common.h b/arch/mips/include/asm/netlogic/common.h
index be52c2125d71..e0717d10e650 100644
--- a/arch/mips/include/asm/netlogic/common.h
+++ b/arch/mips/include/asm/netlogic/common.h
@@ -88,7 +88,7 @@ extern struct plat_smp_ops nlm_smp_ops;
extern char nlm_reset_entry[], nlm_reset_entry_end[];
/* SWIOTLB */
-extern struct dma_map_ops nlm_swiotlb_dma_ops;
+extern const struct dma_map_ops nlm_swiotlb_dma_ops;
extern unsigned int nlm_threads_per_core;
extern cpumask_t nlm_cpumask;
diff --git a/arch/mips/loongson64/common/dma-swiotlb.c b/arch/mips/loongson64/common/dma-swiotlb.c
index aab4fd681e1f..7296df043d92 100644
--- a/arch/mips/loongson64/common/dma-swiotlb.c
+++ b/arch/mips/loongson64/common/dma-swiotlb.c
@@ -122,7 +122,7 @@ phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
return daddr;
}
-static struct dma_map_ops loongson_dma_map_ops = {
+static const struct dma_map_ops loongson_dma_map_ops = {
.alloc = loongson_dma_alloc_coherent,
.free = loongson_dma_free_coherent,
.map_page = loongson_dma_map_page,
diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c
index a39c36af97ad..1cb84472cb58 100644
--- a/arch/mips/mm/dma-default.c
+++ b/arch/mips/mm/dma-default.c
@@ -417,7 +417,7 @@ void dma_cache_sync(struct device *dev, void *vaddr, size_t size,
EXPORT_SYMBOL(dma_cache_sync);
-static struct dma_map_ops mips_default_dma_map_ops = {
+static const struct dma_map_ops mips_default_dma_map_ops = {
.alloc = mips_dma_alloc_coherent,
.free = mips_dma_free_coherent,
.mmap = mips_dma_mmap,
@@ -433,7 +433,7 @@ static struct dma_map_ops mips_default_dma_map_ops = {
.dma_supported = mips_dma_supported
};
-struct dma_map_ops *mips_dma_map_ops = &mips_default_dma_map_ops;
+const struct dma_map_ops *mips_dma_map_ops = &mips_default_dma_map_ops;
EXPORT_SYMBOL(mips_dma_map_ops);
#define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16)
diff --git a/arch/mips/netlogic/common/nlm-dma.c b/arch/mips/netlogic/common/nlm-dma.c
index 0630693bec2a..0ec9d9da6d51 100644
--- a/arch/mips/netlogic/common/nlm-dma.c
+++ b/arch/mips/netlogic/common/nlm-dma.c
@@ -67,7 +67,7 @@ static void nlm_dma_free_coherent(struct device *dev, size_t size,
swiotlb_free_coherent(dev, size, vaddr, dma_handle);
}
-struct dma_map_ops nlm_swiotlb_dma_ops = {
+const struct dma_map_ops nlm_swiotlb_dma_ops = {
.alloc = nlm_dma_alloc_coherent,
.free = nlm_dma_free_coherent,
.map_page = swiotlb_map_page,
diff --git a/arch/mn10300/include/asm/dma-mapping.h b/arch/mn10300/include/asm/dma-mapping.h
index 1dcd44757f32..564e3927e005 100644
--- a/arch/mn10300/include/asm/dma-mapping.h
+++ b/arch/mn10300/include/asm/dma-mapping.h
@@ -14,9 +14,9 @@
#include <asm/cache.h>
#include <asm/io.h>
-extern struct dma_map_ops mn10300_dma_ops;
+extern const struct dma_map_ops mn10300_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &mn10300_dma_ops;
}
diff --git a/arch/mn10300/mm/dma-alloc.c b/arch/mn10300/mm/dma-alloc.c
index 4f4b9029f0ea..86108d2496b3 100644
--- a/arch/mn10300/mm/dma-alloc.c
+++ b/arch/mn10300/mm/dma-alloc.c
@@ -121,7 +121,7 @@ static int mn10300_dma_supported(struct device *dev, u64 mask)
return 1;
}
-struct dma_map_ops mn10300_dma_ops = {
+const struct dma_map_ops mn10300_dma_ops = {
.alloc = mn10300_dma_alloc,
.free = mn10300_dma_free,
.map_page = mn10300_dma_map_page,
diff --git a/arch/nios2/include/asm/dma-mapping.h b/arch/nios2/include/asm/dma-mapping.h
index bec8ac8e6ad2..aa00d839a64b 100644
--- a/arch/nios2/include/asm/dma-mapping.h
+++ b/arch/nios2/include/asm/dma-mapping.h
@@ -10,9 +10,9 @@
#ifndef _ASM_NIOS2_DMA_MAPPING_H
#define _ASM_NIOS2_DMA_MAPPING_H
-extern struct dma_map_ops nios2_dma_ops;
+extern const struct dma_map_ops nios2_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &nios2_dma_ops;
}
diff --git a/arch/nios2/mm/dma-mapping.c b/arch/nios2/mm/dma-mapping.c
index f6a5dcf9d682..7040c1adbb5e 100644
--- a/arch/nios2/mm/dma-mapping.c
+++ b/arch/nios2/mm/dma-mapping.c
@@ -192,7 +192,7 @@ static void nios2_dma_sync_sg_for_device(struct device *dev,
}
-struct dma_map_ops nios2_dma_ops = {
+const struct dma_map_ops nios2_dma_ops = {
.alloc = nios2_dma_alloc,
.free = nios2_dma_free,
.map_page = nios2_dma_map_page,
diff --git a/arch/openrisc/include/asm/dma-mapping.h b/arch/openrisc/include/asm/dma-mapping.h
index 1f260bccb368..88acbedb4947 100644
--- a/arch/openrisc/include/asm/dma-mapping.h
+++ b/arch/openrisc/include/asm/dma-mapping.h
@@ -28,9 +28,9 @@
#define DMA_ERROR_CODE (~(dma_addr_t)0x0)
-extern struct dma_map_ops or1k_dma_map_ops;
+extern const struct dma_map_ops or1k_dma_map_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &or1k_dma_map_ops;
}
diff --git a/arch/openrisc/kernel/dma.c b/arch/openrisc/kernel/dma.c
index 906998bac957..b10369b7e31b 100644
--- a/arch/openrisc/kernel/dma.c
+++ b/arch/openrisc/kernel/dma.c
@@ -232,7 +232,7 @@ or1k_sync_single_for_device(struct device *dev,
mtspr(SPR_DCBFR, cl);
}
-struct dma_map_ops or1k_dma_map_ops = {
+const struct dma_map_ops or1k_dma_map_ops = {
.alloc = or1k_dma_alloc,
.free = or1k_dma_free,
.map_page = or1k_map_page,
diff --git a/arch/parisc/include/asm/dma-mapping.h b/arch/parisc/include/asm/dma-mapping.h
index 16e024602737..1749073e44fc 100644
--- a/arch/parisc/include/asm/dma-mapping.h
+++ b/arch/parisc/include/asm/dma-mapping.h
@@ -21,13 +21,13 @@
*/
#ifdef CONFIG_PA11
-extern struct dma_map_ops pcxl_dma_ops;
-extern struct dma_map_ops pcx_dma_ops;
+extern const struct dma_map_ops pcxl_dma_ops;
+extern const struct dma_map_ops pcx_dma_ops;
#endif
-extern struct dma_map_ops *hppa_dma_ops;
+extern const struct dma_map_ops *hppa_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return hppa_dma_ops;
}
diff --git a/arch/parisc/kernel/drivers.c b/arch/parisc/kernel/drivers.c
index 700e2d2da096..fa78419100c8 100644
--- a/arch/parisc/kernel/drivers.c
+++ b/arch/parisc/kernel/drivers.c
@@ -40,7 +40,7 @@
#include <asm/parisc-device.h>
/* See comments in include/asm-parisc/pci.h */
-struct dma_map_ops *hppa_dma_ops __read_mostly;
+const struct dma_map_ops *hppa_dma_ops __read_mostly;
EXPORT_SYMBOL(hppa_dma_ops);
static struct device root = {
diff --git a/arch/parisc/kernel/pci-dma.c b/arch/parisc/kernel/pci-dma.c
index 697c53543a4d..5f0067a62738 100644
--- a/arch/parisc/kernel/pci-dma.c
+++ b/arch/parisc/kernel/pci-dma.c
@@ -572,7 +572,7 @@ static void pa11_dma_sync_sg_for_device(struct device *dev, struct scatterlist *
flush_kernel_vmap_range(sg_virt(sg), sg->length);
}
-struct dma_map_ops pcxl_dma_ops = {
+const struct dma_map_ops pcxl_dma_ops = {
.dma_supported = pa11_dma_supported,
.alloc = pa11_dma_alloc,
.free = pa11_dma_free,
@@ -608,7 +608,7 @@ static void pcx_dma_free(struct device *dev, size_t size, void *vaddr,
return;
}
-struct dma_map_ops pcx_dma_ops = {
+const struct dma_map_ops pcx_dma_ops = {
.dma_supported = pa11_dma_supported,
.alloc = pcx_dma_alloc,
.free = pcx_dma_free,
diff --git a/arch/powerpc/include/asm/device.h b/arch/powerpc/include/asm/device.h
index 406c2b1ff82d..49cbb0fca233 100644
--- a/arch/powerpc/include/asm/device.h
+++ b/arch/powerpc/include/asm/device.h
@@ -21,7 +21,7 @@ struct iommu_table;
*/
struct dev_archdata {
/* DMA operations on that device */
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
/*
* These two used to be a union. However, with the hybrid ops we need
diff --git a/arch/powerpc/include/asm/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h
index 84e3f8dd5e4f..2ec3eadf336f 100644
--- a/arch/powerpc/include/asm/dma-mapping.h
+++ b/arch/powerpc/include/asm/dma-mapping.h
@@ -76,9 +76,9 @@ static inline unsigned long device_to_mask(struct device *dev)
#ifdef CONFIG_PPC64
extern struct dma_map_ops dma_iommu_ops;
#endif
-extern struct dma_map_ops dma_direct_ops;
+extern const struct dma_map_ops dma_direct_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
/* We don't handle the NULL dev case for ISA for now. We could
* do it via an out of line call but it is not needed for now. The
@@ -91,7 +91,7 @@ static inline struct dma_map_ops *get_dma_ops(struct device *dev)
return dev->archdata.dma_ops;
}
-static inline void set_dma_ops(struct device *dev, struct dma_map_ops *ops)
+static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
{
dev->archdata.dma_ops = ops;
}
diff --git a/arch/powerpc/include/asm/pci.h b/arch/powerpc/include/asm/pci.h
index e9bd6cf0212f..93eded8d3843 100644
--- a/arch/powerpc/include/asm/pci.h
+++ b/arch/powerpc/include/asm/pci.h
@@ -53,8 +53,8 @@ static inline int pci_get_legacy_ide_irq(struct pci_dev *dev, int channel)
}
#ifdef CONFIG_PCI
-extern void set_pci_dma_ops(struct dma_map_ops *dma_ops);
-extern struct dma_map_ops *get_pci_dma_ops(void);
+extern void set_pci_dma_ops(const struct dma_map_ops *dma_ops);
+extern const struct dma_map_ops *get_pci_dma_ops(void);
#else /* CONFIG_PCI */
#define set_pci_dma_ops(d)
#define get_pci_dma_ops() NULL
diff --git a/arch/powerpc/include/asm/swiotlb.h b/arch/powerpc/include/asm/swiotlb.h
index de99d6e29430..01d45a5fd00b 100644
--- a/arch/powerpc/include/asm/swiotlb.h
+++ b/arch/powerpc/include/asm/swiotlb.h
@@ -13,7 +13,7 @@
#include <linux/swiotlb.h>
-extern struct dma_map_ops swiotlb_dma_ops;
+extern const struct dma_map_ops swiotlb_dma_ops;
static inline void dma_mark_clean(void *addr, size_t size) {}
diff --git a/arch/powerpc/kernel/dma-swiotlb.c b/arch/powerpc/kernel/dma-swiotlb.c
index c6689f658b50..d0ea7860e02b 100644
--- a/arch/powerpc/kernel/dma-swiotlb.c
+++ b/arch/powerpc/kernel/dma-swiotlb.c
@@ -46,7 +46,7 @@ static u64 swiotlb_powerpc_get_required(struct device *dev)
* map_page, and unmap_page on highmem, use normal dma_ops
* for everything else.
*/
-struct dma_map_ops swiotlb_dma_ops = {
+const struct dma_map_ops swiotlb_dma_ops = {
.alloc = __dma_direct_alloc_coherent,
.free = __dma_direct_free_coherent,
.mmap = dma_direct_mmap_coherent,
diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
index 6877e3fa95bb..03b98f1f98ec 100644
--- a/arch/powerpc/kernel/dma.c
+++ b/arch/powerpc/kernel/dma.c
@@ -274,7 +274,7 @@ static inline void dma_direct_sync_single(struct device *dev,
}
#endif
-struct dma_map_ops dma_direct_ops = {
+const struct dma_map_ops dma_direct_ops = {
.alloc = dma_direct_alloc_coherent,
.free = dma_direct_free_coherent,
.mmap = dma_direct_mmap_coherent,
@@ -316,7 +316,7 @@ EXPORT_SYMBOL(dma_set_coherent_mask);
int __dma_set_mask(struct device *dev, u64 dma_mask)
{
- struct dma_map_ops *dma_ops = get_dma_ops(dev);
+ const struct dma_map_ops *dma_ops = get_dma_ops(dev);
if ((dma_ops != NULL) && (dma_ops->set_dma_mask != NULL))
return dma_ops->set_dma_mask(dev, dma_mask);
@@ -344,7 +344,7 @@ EXPORT_SYMBOL(dma_set_mask);
u64 __dma_get_required_mask(struct device *dev)
{
- struct dma_map_ops *dma_ops = get_dma_ops(dev);
+ const struct dma_map_ops *dma_ops = get_dma_ops(dev);
if (unlikely(dma_ops == NULL))
return 0;
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 74bec5498972..09db4778435c 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -59,14 +59,14 @@ resource_size_t isa_mem_base;
EXPORT_SYMBOL(isa_mem_base);
-static struct dma_map_ops *pci_dma_ops = &dma_direct_ops;
+static const struct dma_map_ops *pci_dma_ops = &dma_direct_ops;
-void set_pci_dma_ops(struct dma_map_ops *dma_ops)
+void set_pci_dma_ops(const struct dma_map_ops *dma_ops)
{
pci_dma_ops = dma_ops;
}
-struct dma_map_ops *get_pci_dma_ops(void)
+const struct dma_map_ops *get_pci_dma_ops(void)
{
return pci_dma_ops;
}
diff --git a/arch/powerpc/platforms/cell/iommu.c b/arch/powerpc/platforms/cell/iommu.c
index 7ff51f96a00e..e1413e69e5fe 100644
--- a/arch/powerpc/platforms/cell/iommu.c
+++ b/arch/powerpc/platforms/cell/iommu.c
@@ -651,7 +651,7 @@ static int dma_fixed_dma_supported(struct device *dev, u64 mask)
static int dma_set_mask_and_switch(struct device *dev, u64 dma_mask);
-static struct dma_map_ops dma_iommu_fixed_ops = {
+static const struct dma_map_ops dma_iommu_fixed_ops = {
.alloc = dma_fixed_alloc_coherent,
.free = dma_fixed_free_coherent,
.map_sg = dma_fixed_map_sg,
@@ -1172,7 +1172,7 @@ __setup("iommu_fixed=", setup_iommu_fixed);
static u64 cell_dma_get_required_mask(struct device *dev)
{
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
if (!dev->dma_mask)
return 0;
diff --git a/arch/powerpc/platforms/powernv/npu-dma.c b/arch/powerpc/platforms/powernv/npu-dma.c
index 73b155fd4481..1c383f38031d 100644
--- a/arch/powerpc/platforms/powernv/npu-dma.c
+++ b/arch/powerpc/platforms/powernv/npu-dma.c
@@ -115,7 +115,7 @@ static u64 dma_npu_get_required_mask(struct device *dev)
return 0;
}
-static struct dma_map_ops dma_npu_ops = {
+static const struct dma_map_ops dma_npu_ops = {
.map_page = dma_npu_map_page,
.map_sg = dma_npu_map_sg,
.alloc = dma_npu_alloc,
diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
index 8af1c15aef85..c81450d98794 100644
--- a/arch/powerpc/platforms/ps3/system-bus.c
+++ b/arch/powerpc/platforms/ps3/system-bus.c
@@ -701,7 +701,7 @@ static u64 ps3_dma_get_required_mask(struct device *_dev)
return DMA_BIT_MASK(32);
}
-static struct dma_map_ops ps3_sb_dma_ops = {
+static const struct dma_map_ops ps3_sb_dma_ops = {
.alloc = ps3_alloc_coherent,
.free = ps3_free_coherent,
.map_sg = ps3_sb_map_sg,
@@ -712,7 +712,7 @@ static struct dma_map_ops ps3_sb_dma_ops = {
.unmap_page = ps3_unmap_page,
};
-static struct dma_map_ops ps3_ioc0_dma_ops = {
+static const struct dma_map_ops ps3_ioc0_dma_ops = {
.alloc = ps3_alloc_coherent,
.free = ps3_free_coherent,
.map_sg = ps3_ioc0_map_sg,
diff --git a/arch/powerpc/platforms/pseries/ibmebus.c b/arch/powerpc/platforms/pseries/ibmebus.c
index 614c28537141..2e36a0b8944a 100644
--- a/arch/powerpc/platforms/pseries/ibmebus.c
+++ b/arch/powerpc/platforms/pseries/ibmebus.c
@@ -136,7 +136,7 @@ static u64 ibmebus_dma_get_required_mask(struct device *dev)
return DMA_BIT_MASK(64);
}
-static struct dma_map_ops ibmebus_dma_ops = {
+static const struct dma_map_ops ibmebus_dma_ops = {
.alloc = ibmebus_alloc_coherent,
.free = ibmebus_free_coherent,
.map_sg = ibmebus_map_sg,
diff --git a/arch/powerpc/platforms/pseries/vio.c b/arch/powerpc/platforms/pseries/vio.c
index 2c8fb3ec989e..720493932486 100644
--- a/arch/powerpc/platforms/pseries/vio.c
+++ b/arch/powerpc/platforms/pseries/vio.c
@@ -615,7 +615,7 @@ static u64 vio_dma_get_required_mask(struct device *dev)
return dma_iommu_ops.get_required_mask(dev);
}
-static struct dma_map_ops vio_dma_mapping_ops = {
+static const struct dma_map_ops vio_dma_mapping_ops = {
.alloc = vio_dma_iommu_alloc_coherent,
.free = vio_dma_iommu_free_coherent,
.mmap = dma_direct_mmap_coherent,
diff --git a/arch/s390/include/asm/device.h b/arch/s390/include/asm/device.h
index 4a9f35e0973f..7955a9799466 100644
--- a/arch/s390/include/asm/device.h
+++ b/arch/s390/include/asm/device.h
@@ -4,7 +4,7 @@
* This file is released under the GPLv2
*/
struct dev_archdata {
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
};
struct pdev_archdata {
diff --git a/arch/s390/include/asm/dma-mapping.h b/arch/s390/include/asm/dma-mapping.h
index ffaba07f50ab..2776d205b1ff 100644
--- a/arch/s390/include/asm/dma-mapping.h
+++ b/arch/s390/include/asm/dma-mapping.h
@@ -10,9 +10,9 @@
#define DMA_ERROR_CODE (~(dma_addr_t) 0x0)
-extern struct dma_map_ops s390_pci_dma_ops;
+extern const struct dma_map_ops s390_pci_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
if (dev && dev->archdata.dma_ops)
return dev->archdata.dma_ops;
diff --git a/arch/s390/pci/pci_dma.c b/arch/s390/pci/pci_dma.c
index 1d7a9c71944a..9081a57fa340 100644
--- a/arch/s390/pci/pci_dma.c
+++ b/arch/s390/pci/pci_dma.c
@@ -650,7 +650,7 @@ static int __init dma_debug_do_init(void)
}
fs_initcall(dma_debug_do_init);
-struct dma_map_ops s390_pci_dma_ops = {
+const struct dma_map_ops s390_pci_dma_ops = {
.alloc = s390_dma_alloc,
.free = s390_dma_free,
.map_sg = s390_dma_map_sg,
diff --git a/arch/sh/include/asm/dma-mapping.h b/arch/sh/include/asm/dma-mapping.h
index 0052ad40e86d..a7382c34c241 100644
--- a/arch/sh/include/asm/dma-mapping.h
+++ b/arch/sh/include/asm/dma-mapping.h
@@ -1,10 +1,10 @@
#ifndef __ASM_SH_DMA_MAPPING_H
#define __ASM_SH_DMA_MAPPING_H
-extern struct dma_map_ops *dma_ops;
+extern const struct dma_map_ops *dma_ops;
extern void no_iommu_init(void);
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return dma_ops;
}
diff --git a/arch/sh/kernel/dma-nommu.c b/arch/sh/kernel/dma-nommu.c
index 47fee3b6e29c..d24c707b2181 100644
--- a/arch/sh/kernel/dma-nommu.c
+++ b/arch/sh/kernel/dma-nommu.c
@@ -65,7 +65,7 @@ static void nommu_sync_sg(struct device *dev, struct scatterlist *sg,
}
#endif
-struct dma_map_ops nommu_dma_ops = {
+const struct dma_map_ops nommu_dma_ops = {
.alloc = dma_generic_alloc_coherent,
.free = dma_generic_free_coherent,
.map_page = nommu_map_page,
diff --git a/arch/sh/mm/consistent.c b/arch/sh/mm/consistent.c
index 92b6976fde59..d1275adfa0ef 100644
--- a/arch/sh/mm/consistent.c
+++ b/arch/sh/mm/consistent.c
@@ -22,7 +22,7 @@
#define PREALLOC_DMA_DEBUG_ENTRIES 4096
-struct dma_map_ops *dma_ops;
+const struct dma_map_ops *dma_ops;
EXPORT_SYMBOL(dma_ops);
static int __init dma_init(void)
diff --git a/arch/sparc/include/asm/dma-mapping.h b/arch/sparc/include/asm/dma-mapping.h
index 1180ae254154..3d2babc0c4c6 100644
--- a/arch/sparc/include/asm/dma-mapping.h
+++ b/arch/sparc/include/asm/dma-mapping.h
@@ -18,13 +18,13 @@ static inline void dma_cache_sync(struct device *dev, void *vaddr, size_t size,
*/
}
-extern struct dma_map_ops *dma_ops;
-extern struct dma_map_ops *leon_dma_ops;
-extern struct dma_map_ops pci32_dma_ops;
+extern const struct dma_map_ops *dma_ops;
+extern const struct dma_map_ops *leon_dma_ops;
+extern const struct dma_map_ops pci32_dma_ops;
extern struct bus_type pci_bus_type;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
#ifdef CONFIG_SPARC_LEON
if (sparc_cpu_model == sparc_leon)
diff --git a/arch/sparc/kernel/iommu.c b/arch/sparc/kernel/iommu.c
index 9df997995f6b..c63ba99ca551 100644
--- a/arch/sparc/kernel/iommu.c
+++ b/arch/sparc/kernel/iommu.c
@@ -741,7 +741,7 @@ static void dma_4u_sync_sg_for_cpu(struct device *dev,
spin_unlock_irqrestore(&iommu->lock, flags);
}
-static struct dma_map_ops sun4u_dma_ops = {
+static const struct dma_map_ops sun4u_dma_ops = {
.alloc = dma_4u_alloc_coherent,
.free = dma_4u_free_coherent,
.map_page = dma_4u_map_page,
@@ -752,7 +752,7 @@ static struct dma_map_ops sun4u_dma_ops = {
.sync_sg_for_cpu = dma_4u_sync_sg_for_cpu,
};
-struct dma_map_ops *dma_ops = &sun4u_dma_ops;
+const struct dma_map_ops *dma_ops = &sun4u_dma_ops;
EXPORT_SYMBOL(dma_ops);
int dma_supported(struct device *dev, u64 device_mask)
diff --git a/arch/sparc/kernel/ioport.c b/arch/sparc/kernel/ioport.c
index 6ffaec44931a..cf20033a1458 100644
--- a/arch/sparc/kernel/ioport.c
+++ b/arch/sparc/kernel/ioport.c
@@ -401,7 +401,7 @@ static void sbus_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
BUG();
}
-static struct dma_map_ops sbus_dma_ops = {
+static const struct dma_map_ops sbus_dma_ops = {
.alloc = sbus_alloc_coherent,
.free = sbus_free_coherent,
.map_page = sbus_map_page,
@@ -637,7 +637,7 @@ static void pci32_sync_sg_for_device(struct device *device, struct scatterlist *
}
}
-struct dma_map_ops pci32_dma_ops = {
+const struct dma_map_ops pci32_dma_ops = {
.alloc = pci32_alloc_coherent,
.free = pci32_free_coherent,
.map_page = pci32_map_page,
@@ -652,10 +652,10 @@ struct dma_map_ops pci32_dma_ops = {
EXPORT_SYMBOL(pci32_dma_ops);
/* leon re-uses pci32_dma_ops */
-struct dma_map_ops *leon_dma_ops = &pci32_dma_ops;
+const struct dma_map_ops *leon_dma_ops = &pci32_dma_ops;
EXPORT_SYMBOL(leon_dma_ops);
-struct dma_map_ops *dma_ops = &sbus_dma_ops;
+const struct dma_map_ops *dma_ops = &sbus_dma_ops;
EXPORT_SYMBOL(dma_ops);
diff --git a/arch/sparc/kernel/pci_sun4v.c b/arch/sparc/kernel/pci_sun4v.c
index f4daccd12bf5..68bec7c97cb8 100644
--- a/arch/sparc/kernel/pci_sun4v.c
+++ b/arch/sparc/kernel/pci_sun4v.c
@@ -669,7 +669,7 @@ static void dma_4v_unmap_sg(struct device *dev, struct scatterlist *sglist,
local_irq_restore(flags);
}
-static struct dma_map_ops sun4v_dma_ops = {
+static const struct dma_map_ops sun4v_dma_ops = {
.alloc = dma_4v_alloc_coherent,
.free = dma_4v_free_coherent,
.map_page = dma_4v_map_page,
diff --git a/arch/tile/include/asm/device.h b/arch/tile/include/asm/device.h
index 6ab8bf146d4c..25f23ac7d361 100644
--- a/arch/tile/include/asm/device.h
+++ b/arch/tile/include/asm/device.h
@@ -18,7 +18,7 @@
struct dev_archdata {
/* DMA operations on that device */
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
/* Offset of the DMA address from the PA. */
dma_addr_t dma_offset;
diff --git a/arch/tile/include/asm/dma-mapping.h b/arch/tile/include/asm/dma-mapping.h
index 01ceb4a895b0..4a06cc75b856 100644
--- a/arch/tile/include/asm/dma-mapping.h
+++ b/arch/tile/include/asm/dma-mapping.h
@@ -24,12 +24,12 @@
#define ARCH_HAS_DMA_GET_REQUIRED_MASK
#endif
-extern struct dma_map_ops *tile_dma_map_ops;
-extern struct dma_map_ops *gx_pci_dma_map_ops;
-extern struct dma_map_ops *gx_legacy_pci_dma_map_ops;
-extern struct dma_map_ops *gx_hybrid_pci_dma_map_ops;
+extern const struct dma_map_ops *tile_dma_map_ops;
+extern const struct dma_map_ops *gx_pci_dma_map_ops;
+extern const struct dma_map_ops *gx_legacy_pci_dma_map_ops;
+extern const struct dma_map_ops *gx_hybrid_pci_dma_map_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
if (dev && dev->archdata.dma_ops)
return dev->archdata.dma_ops;
@@ -59,7 +59,7 @@ static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
static inline void dma_mark_clean(void *addr, size_t size) {}
-static inline void set_dma_ops(struct device *dev, struct dma_map_ops *ops)
+static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
{
dev->archdata.dma_ops = ops;
}
diff --git a/arch/tile/kernel/pci-dma.c b/arch/tile/kernel/pci-dma.c
index 24e0f8c21f2f..569bb6dd154a 100644
--- a/arch/tile/kernel/pci-dma.c
+++ b/arch/tile/kernel/pci-dma.c
@@ -329,7 +329,7 @@ tile_dma_supported(struct device *dev, u64 mask)
return 1;
}
-static struct dma_map_ops tile_default_dma_map_ops = {
+static const struct dma_map_ops tile_default_dma_map_ops = {
.alloc = tile_dma_alloc_coherent,
.free = tile_dma_free_coherent,
.map_page = tile_dma_map_page,
@@ -344,7 +344,7 @@ static struct dma_map_ops tile_default_dma_map_ops = {
.dma_supported = tile_dma_supported
};
-struct dma_map_ops *tile_dma_map_ops = &tile_default_dma_map_ops;
+const struct dma_map_ops *tile_dma_map_ops = &tile_default_dma_map_ops;
EXPORT_SYMBOL(tile_dma_map_ops);
/* Generic PCI DMA mapping functions */
@@ -516,7 +516,7 @@ tile_pci_dma_supported(struct device *dev, u64 mask)
return 1;
}
-static struct dma_map_ops tile_pci_default_dma_map_ops = {
+static const struct dma_map_ops tile_pci_default_dma_map_ops = {
.alloc = tile_pci_dma_alloc_coherent,
.free = tile_pci_dma_free_coherent,
.map_page = tile_pci_dma_map_page,
@@ -531,7 +531,7 @@ static struct dma_map_ops tile_pci_default_dma_map_ops = {
.dma_supported = tile_pci_dma_supported
};
-struct dma_map_ops *gx_pci_dma_map_ops = &tile_pci_default_dma_map_ops;
+const struct dma_map_ops *gx_pci_dma_map_ops = &tile_pci_default_dma_map_ops;
EXPORT_SYMBOL(gx_pci_dma_map_ops);
/* PCI DMA mapping functions for legacy PCI devices */
@@ -552,7 +552,7 @@ static void tile_swiotlb_free_coherent(struct device *dev, size_t size,
swiotlb_free_coherent(dev, size, vaddr, dma_addr);
}
-static struct dma_map_ops pci_swiotlb_dma_ops = {
+static const struct dma_map_ops pci_swiotlb_dma_ops = {
.alloc = tile_swiotlb_alloc_coherent,
.free = tile_swiotlb_free_coherent,
.map_page = swiotlb_map_page,
@@ -567,7 +567,7 @@ static struct dma_map_ops pci_swiotlb_dma_ops = {
.mapping_error = swiotlb_dma_mapping_error,
};
-static struct dma_map_ops pci_hybrid_dma_ops = {
+static const struct dma_map_ops pci_hybrid_dma_ops = {
.alloc = tile_swiotlb_alloc_coherent,
.free = tile_swiotlb_free_coherent,
.map_page = tile_pci_dma_map_page,
@@ -582,18 +582,18 @@ static struct dma_map_ops pci_hybrid_dma_ops = {
.dma_supported = tile_pci_dma_supported
};
-struct dma_map_ops *gx_legacy_pci_dma_map_ops = &pci_swiotlb_dma_ops;
-struct dma_map_ops *gx_hybrid_pci_dma_map_ops = &pci_hybrid_dma_ops;
+const struct dma_map_ops *gx_legacy_pci_dma_map_ops = &pci_swiotlb_dma_ops;
+const struct dma_map_ops *gx_hybrid_pci_dma_map_ops = &pci_hybrid_dma_ops;
#else
-struct dma_map_ops *gx_legacy_pci_dma_map_ops;
-struct dma_map_ops *gx_hybrid_pci_dma_map_ops;
+const struct dma_map_ops *gx_legacy_pci_dma_map_ops;
+const struct dma_map_ops *gx_hybrid_pci_dma_map_ops;
#endif
EXPORT_SYMBOL(gx_legacy_pci_dma_map_ops);
EXPORT_SYMBOL(gx_hybrid_pci_dma_map_ops);
int dma_set_mask(struct device *dev, u64 mask)
{
- struct dma_map_ops *dma_ops = get_dma_ops(dev);
+ const struct dma_map_ops *dma_ops = get_dma_ops(dev);
/*
* For PCI devices with 64-bit DMA addressing capability, promote
@@ -623,7 +623,7 @@ EXPORT_SYMBOL(dma_set_mask);
#ifdef CONFIG_ARCH_HAS_DMA_SET_COHERENT_MASK
int dma_set_coherent_mask(struct device *dev, u64 mask)
{
- struct dma_map_ops *dma_ops = get_dma_ops(dev);
+ const struct dma_map_ops *dma_ops = get_dma_ops(dev);
/*
* For PCI devices with 64-bit DMA addressing capability, promote
diff --git a/arch/unicore32/include/asm/dma-mapping.h b/arch/unicore32/include/asm/dma-mapping.h
index 4749854afd03..14d7729c7b73 100644
--- a/arch/unicore32/include/asm/dma-mapping.h
+++ b/arch/unicore32/include/asm/dma-mapping.h
@@ -21,9 +21,9 @@
#include <asm/memory.h>
#include <asm/cacheflush.h>
-extern struct dma_map_ops swiotlb_dma_map_ops;
+extern const struct dma_map_ops swiotlb_dma_map_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &swiotlb_dma_map_ops;
}
diff --git a/arch/unicore32/mm/dma-swiotlb.c b/arch/unicore32/mm/dma-swiotlb.c
index 3e9f6489ba38..525413d6690e 100644
--- a/arch/unicore32/mm/dma-swiotlb.c
+++ b/arch/unicore32/mm/dma-swiotlb.c
@@ -31,7 +31,7 @@ static void unicore_swiotlb_free_coherent(struct device *dev, size_t size,
swiotlb_free_coherent(dev, size, vaddr, dma_addr);
}
-struct dma_map_ops swiotlb_dma_map_ops = {
+const struct dma_map_ops swiotlb_dma_map_ops = {
.alloc = unicore_swiotlb_alloc_coherent,
.free = unicore_swiotlb_free_coherent,
.map_sg = swiotlb_map_sg_attrs,
diff --git a/arch/x86/include/asm/device.h b/arch/x86/include/asm/device.h
index 684ed6c3aa67..b2d0b4ced7e3 100644
--- a/arch/x86/include/asm/device.h
+++ b/arch/x86/include/asm/device.h
@@ -3,7 +3,7 @@
struct dev_archdata {
#ifdef CONFIG_X86_DEV_DMA_OPS
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
#endif
#if defined(CONFIG_INTEL_IOMMU) || defined(CONFIG_AMD_IOMMU)
void *iommu; /* hook for IOMMU specific extension */
@@ -13,7 +13,7 @@ struct dev_archdata {
#if defined(CONFIG_X86_DEV_DMA_OPS) && defined(CONFIG_PCI_DOMAINS)
struct dma_domain {
struct list_head node;
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
int domain_nr;
};
void add_dma_domain(struct dma_domain *domain);
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 44461626830e..5e4772886a1e 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -25,9 +25,9 @@ extern int iommu_merge;
extern struct device x86_dma_fallback_dev;
extern int panic_on_overflow;
-extern struct dma_map_ops *dma_ops;
+extern const struct dma_map_ops *dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
#ifndef CONFIG_X86_DEV_DMA_OPS
return dma_ops;
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index 345c99cef152..793869879464 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -1,7 +1,7 @@
#ifndef _ASM_X86_IOMMU_H
#define _ASM_X86_IOMMU_H
-extern struct dma_map_ops nommu_dma_ops;
+extern const struct dma_map_ops nommu_dma_ops;
extern int force_iommu, no_iommu;
extern int iommu_detected;
extern int iommu_pass_through;
diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c
index 63ff468a7986..82dfe32faaf4 100644
--- a/arch/x86/kernel/amd_gart_64.c
+++ b/arch/x86/kernel/amd_gart_64.c
@@ -695,7 +695,7 @@ static __init int init_amd_gatt(struct agp_kern_info *info)
return -1;
}
-static struct dma_map_ops gart_dma_ops = {
+static const struct dma_map_ops gart_dma_ops = {
.map_sg = gart_map_sg,
.unmap_sg = gart_unmap_sg,
.map_page = gart_map_page,
diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 5d400ba1349d..17f180148c80 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -478,7 +478,7 @@ static void calgary_free_coherent(struct device *dev, size_t size,
free_pages((unsigned long)vaddr, get_order(size));
}
-static struct dma_map_ops calgary_dma_ops = {
+static const struct dma_map_ops calgary_dma_ops = {
.alloc = calgary_alloc_coherent,
.free = calgary_free_coherent,
.map_sg = calgary_map_sg,
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index d30c37750765..76f4c039baae 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -17,7 +17,7 @@
static int forbid_dac __read_mostly;
-struct dma_map_ops *dma_ops = &nommu_dma_ops;
+const struct dma_map_ops *dma_ops = &nommu_dma_ops;
EXPORT_SYMBOL(dma_ops);
static int iommu_sac_force __read_mostly;
@@ -214,7 +214,7 @@ early_param("iommu", iommu_setup);
int dma_supported(struct device *dev, u64 mask)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
#ifdef CONFIG_PCI
if (mask > 0xffffffff && forbid_dac > 0) {
diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index 00e71ce396a8..a88952ef371c 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -88,7 +88,7 @@ static void nommu_sync_sg_for_device(struct device *dev,
flush_write_buffers();
}
-struct dma_map_ops nommu_dma_ops = {
+const struct dma_map_ops nommu_dma_ops = {
.alloc = dma_generic_alloc_coherent,
.free = dma_generic_free_coherent,
.map_sg = nommu_map_sg,
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index 410efb2c7b80..1e23577e17cf 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -45,7 +45,7 @@ void x86_swiotlb_free_coherent(struct device *dev, size_t size,
dma_generic_free_coherent(dev, size, vaddr, dma_addr, attrs);
}
-static struct dma_map_ops swiotlb_dma_ops = {
+static const struct dma_map_ops swiotlb_dma_ops = {
.mapping_error = swiotlb_dma_mapping_error,
.alloc = x86_swiotlb_alloc_coherent,
.free = x86_swiotlb_free_coherent,
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index 052c1cb76305..aa3828823170 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -179,7 +179,7 @@ static void *sta2x11_swiotlb_alloc_coherent(struct device *dev,
}
/* We have our own dma_ops: the same as swiotlb but from alloc (above) */
-static struct dma_map_ops sta2x11_dma_ops = {
+static const struct dma_map_ops sta2x11_dma_ops = {
.alloc = sta2x11_swiotlb_alloc_coherent,
.free = x86_swiotlb_free_coherent,
.map_page = swiotlb_map_page,
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
index a0b36a9d5df1..42b08f8fc2ca 100644
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ b/arch/x86/xen/pci-swiotlb-xen.c
@@ -18,7 +18,7 @@
int xen_swiotlb __read_mostly;
-static struct dma_map_ops xen_swiotlb_dma_ops = {
+static const struct dma_map_ops xen_swiotlb_dma_ops = {
.alloc = xen_swiotlb_alloc_coherent,
.free = xen_swiotlb_free_coherent,
.sync_single_for_cpu = xen_swiotlb_sync_single_for_cpu,
diff --git a/arch/xtensa/include/asm/device.h b/arch/xtensa/include/asm/device.h
index fe1f5c878493..a77d45d39f35 100644
--- a/arch/xtensa/include/asm/device.h
+++ b/arch/xtensa/include/asm/device.h
@@ -10,7 +10,7 @@ struct dma_map_ops;
struct dev_archdata {
/* DMA operations on that device */
- struct dma_map_ops *dma_ops;
+ const struct dma_map_ops *dma_ops;
};
struct pdev_archdata {
diff --git a/arch/xtensa/include/asm/dma-mapping.h b/arch/xtensa/include/asm/dma-mapping.h
index 3fc1170a6488..50d23106cce0 100644
--- a/arch/xtensa/include/asm/dma-mapping.h
+++ b/arch/xtensa/include/asm/dma-mapping.h
@@ -18,9 +18,9 @@
#define DMA_ERROR_CODE (~(dma_addr_t)0x0)
-extern struct dma_map_ops xtensa_dma_map_ops;
+extern const struct dma_map_ops xtensa_dma_map_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
if (dev && dev->archdata.dma_ops)
return dev->archdata.dma_ops;
diff --git a/arch/xtensa/kernel/pci-dma.c b/arch/xtensa/kernel/pci-dma.c
index 70e362e6038e..ecec5265a66d 100644
--- a/arch/xtensa/kernel/pci-dma.c
+++ b/arch/xtensa/kernel/pci-dma.c
@@ -249,7 +249,7 @@ int xtensa_dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
return 0;
}
-struct dma_map_ops xtensa_dma_map_ops = {
+const struct dma_map_ops xtensa_dma_map_ops = {
.alloc = xtensa_dma_alloc,
.free = xtensa_dma_free,
.map_page = xtensa_map_page,
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 3ef0f42984f2..3703fb9db419 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -117,7 +117,7 @@ static const struct iommu_ops amd_iommu_ops;
static ATOMIC_NOTIFIER_HEAD(ppr_notifier);
int amd_iommu_max_glx_val = -1;
-static struct dma_map_ops amd_iommu_dma_ops;
+static const struct dma_map_ops amd_iommu_dma_ops;
/*
* This struct contains device specific data for the IOMMU
@@ -2728,7 +2728,7 @@ static int amd_iommu_dma_supported(struct device *dev, u64 mask)
return check_device(dev);
}
-static struct dma_map_ops amd_iommu_dma_ops = {
+static const struct dma_map_ops amd_iommu_dma_ops = {
.alloc = alloc_coherent,
.free = free_coherent,
.map_page = map_page,
diff --git a/drivers/misc/mic/bus/mic_bus.c b/drivers/misc/mic/bus/mic_bus.c
index be37890abb93..c4b27a25662a 100644
--- a/drivers/misc/mic/bus/mic_bus.c
+++ b/drivers/misc/mic/bus/mic_bus.c
@@ -143,7 +143,7 @@ static void mbus_release_dev(struct device *d)
}
struct mbus_device *
-mbus_register_device(struct device *pdev, int id, struct dma_map_ops *dma_ops,
+mbus_register_device(struct device *pdev, int id, const struct dma_map_ops *dma_ops,
struct mbus_hw_ops *hw_ops, int index,
void __iomem *mmio_va)
{
diff --git a/drivers/misc/mic/bus/scif_bus.c b/drivers/misc/mic/bus/scif_bus.c
index ff6e01c25810..e5d377e97c86 100644
--- a/drivers/misc/mic/bus/scif_bus.c
+++ b/drivers/misc/mic/bus/scif_bus.c
@@ -138,7 +138,7 @@ static void scif_release_dev(struct device *d)
}
struct scif_hw_dev *
-scif_register_device(struct device *pdev, int id, struct dma_map_ops *dma_ops,
+scif_register_device(struct device *pdev, int id, const struct dma_map_ops *dma_ops,
struct scif_hw_ops *hw_ops, u8 dnode, u8 snode,
struct mic_mw *mmio, struct mic_mw *aper, void *dp,
void __iomem *rdp, struct dma_chan **chan, int num_chan,
diff --git a/drivers/misc/mic/bus/scif_bus.h b/drivers/misc/mic/bus/scif_bus.h
index 94f29ac608b6..ff59568219ad 100644
--- a/drivers/misc/mic/bus/scif_bus.h
+++ b/drivers/misc/mic/bus/scif_bus.h
@@ -113,7 +113,7 @@ int scif_register_driver(struct scif_driver *driver);
void scif_unregister_driver(struct scif_driver *driver);
struct scif_hw_dev *
scif_register_device(struct device *pdev, int id,
- struct dma_map_ops *dma_ops,
+ const struct dma_map_ops *dma_ops,
struct scif_hw_ops *hw_ops, u8 dnode, u8 snode,
struct mic_mw *mmio, struct mic_mw *aper,
void *dp, void __iomem *rdp,
diff --git a/drivers/misc/mic/bus/vop_bus.c b/drivers/misc/mic/bus/vop_bus.c
index 303da222f5b6..e3caa6c53922 100644
--- a/drivers/misc/mic/bus/vop_bus.c
+++ b/drivers/misc/mic/bus/vop_bus.c
@@ -154,7 +154,7 @@ vop_register_device(struct device *pdev, int id,
vdev->dev.parent = pdev;
vdev->id.device = id;
vdev->id.vendor = VOP_DEV_ANY_ID;
- vdev->dev.archdata.dma_ops = (struct dma_map_ops *)dma_ops;
+ vdev->dev.archdata.dma_ops = dma_ops;
vdev->dev.dma_mask = &vdev->dev.coherent_dma_mask;
dma_set_mask(&vdev->dev, DMA_BIT_MASK(64));
vdev->dev.release = vop_release_dev;
diff --git a/drivers/misc/mic/host/mic_boot.c b/drivers/misc/mic/host/mic_boot.c
index 9599d732aff3..c327985c9523 100644
--- a/drivers/misc/mic/host/mic_boot.c
+++ b/drivers/misc/mic/host/mic_boot.c
@@ -245,7 +245,7 @@ static void __mic_dma_unmap_sg(struct device *dev,
dma_unmap_sg(&mdev->pdev->dev, sg, nents, dir);
}
-static struct dma_map_ops __mic_dma_ops = {
+static const struct dma_map_ops __mic_dma_ops = {
.alloc = __mic_dma_alloc,
.free = __mic_dma_free,
.map_page = __mic_dma_map_page,
@@ -344,7 +344,7 @@ mic_dma_unmap_page(struct device *dev, dma_addr_t dma_addr,
mic_unmap_single(mdev, dma_addr, size);
}
-static struct dma_map_ops mic_dma_ops = {
+static const struct dma_map_ops mic_dma_ops = {
.map_page = mic_dma_map_page,
.unmap_page = mic_dma_unmap_page,
};
diff --git a/drivers/parisc/ccio-dma.c b/drivers/parisc/ccio-dma.c
index 553ef8a5d588..aeb073b5fe16 100644
--- a/drivers/parisc/ccio-dma.c
+++ b/drivers/parisc/ccio-dma.c
@@ -1011,7 +1011,7 @@ ccio_unmap_sg(struct device *dev, struct scatterlist *sglist, int nents,
DBG_RUN_SG("%s() DONE (nents %d)\n", __func__, nents);
}
-static struct dma_map_ops ccio_ops = {
+static const struct dma_map_ops ccio_ops = {
.dma_supported = ccio_dma_supported,
.alloc = ccio_alloc,
.free = ccio_free,
diff --git a/drivers/parisc/sba_iommu.c b/drivers/parisc/sba_iommu.c
index 151b86b6d2e2..33385e574433 100644
--- a/drivers/parisc/sba_iommu.c
+++ b/drivers/parisc/sba_iommu.c
@@ -1069,7 +1069,7 @@ sba_unmap_sg(struct device *dev, struct scatterlist *sglist, int nents,
}
-static struct dma_map_ops sba_ops = {
+static const struct dma_map_ops sba_ops = {
.dma_supported = sba_dma_supported,
.alloc = sba_alloc,
.free = sba_free,
diff --git a/drivers/pci/host/vmd.c b/drivers/pci/host/vmd.c
index 18ef1a93c10a..e27ad2a3bd33 100644
--- a/drivers/pci/host/vmd.c
+++ b/drivers/pci/host/vmd.c
@@ -282,7 +282,7 @@ static struct device *to_vmd_dev(struct device *dev)
return &vmd->dev->dev;
}
-static struct dma_map_ops *vmd_dma_ops(struct device *dev)
+static const struct dma_map_ops *vmd_dma_ops(struct device *dev)
{
return get_dma_ops(to_vmd_dev(dev));
}
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 10c5a17b1f51..f1da68b82c63 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -127,7 +127,7 @@ struct dma_map_ops {
int is_phys;
};
-extern struct dma_map_ops dma_noop_ops;
+extern const struct dma_map_ops dma_noop_ops;
#define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
@@ -170,8 +170,8 @@ int dma_mmap_from_coherent(struct device *dev, struct vm_area_struct *vma,
* dma dependent code. Code that depends on the dma-mapping
* API needs to set 'depends on HAS_DMA' in its Kconfig
*/
-extern struct dma_map_ops bad_dma_ops;
-static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+extern const struct dma_map_ops bad_dma_ops;
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
return &bad_dma_ops;
}
@@ -182,7 +182,7 @@ static inline dma_addr_t dma_map_single_attrs(struct device *dev, void *ptr,
enum dma_data_direction dir,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
dma_addr_t addr;
kmemcheck_mark_initialized(ptr, size);
@@ -201,7 +201,7 @@ static inline void dma_unmap_single_attrs(struct device *dev, dma_addr_t addr,
enum dma_data_direction dir,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
if (ops->unmap_page)
@@ -217,7 +217,7 @@ static inline int dma_map_sg_attrs(struct device *dev, struct scatterlist *sg,
int nents, enum dma_data_direction dir,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
int i, ents;
struct scatterlist *s;
@@ -235,7 +235,7 @@ static inline void dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg
int nents, enum dma_data_direction dir,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
debug_dma_unmap_sg(dev, sg, nents, dir);
@@ -249,7 +249,7 @@ static inline dma_addr_t dma_map_page_attrs(struct device *dev,
enum dma_data_direction dir,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
dma_addr_t addr;
kmemcheck_mark_initialized(page_address(page) + offset, size);
@@ -265,7 +265,7 @@ static inline void dma_unmap_page_attrs(struct device *dev,
enum dma_data_direction dir,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
if (ops->unmap_page)
@@ -279,7 +279,7 @@ static inline dma_addr_t dma_map_resource(struct device *dev,
enum dma_data_direction dir,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
dma_addr_t addr;
BUG_ON(!valid_dma_direction(dir));
@@ -300,7 +300,7 @@ static inline void dma_unmap_resource(struct device *dev, dma_addr_t addr,
size_t size, enum dma_data_direction dir,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
if (ops->unmap_resource)
@@ -312,7 +312,7 @@ static inline void dma_sync_single_for_cpu(struct device *dev, dma_addr_t addr,
size_t size,
enum dma_data_direction dir)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
if (ops->sync_single_for_cpu)
@@ -324,7 +324,7 @@ static inline void dma_sync_single_for_device(struct device *dev,
dma_addr_t addr, size_t size,
enum dma_data_direction dir)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
if (ops->sync_single_for_device)
@@ -364,7 +364,7 @@ static inline void
dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
int nelems, enum dma_data_direction dir)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
if (ops->sync_sg_for_cpu)
@@ -376,7 +376,7 @@ static inline void
dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
int nelems, enum dma_data_direction dir)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!valid_dma_direction(dir));
if (ops->sync_sg_for_device)
@@ -421,7 +421,7 @@ static inline int
dma_mmap_attrs(struct device *dev, struct vm_area_struct *vma, void *cpu_addr,
dma_addr_t dma_addr, size_t size, unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!ops);
if (ops->mmap)
return ops->mmap(dev, vma, cpu_addr, dma_addr, size, attrs);
@@ -439,7 +439,7 @@ dma_get_sgtable_attrs(struct device *dev, struct sg_table *sgt, void *cpu_addr,
dma_addr_t dma_addr, size_t size,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!ops);
if (ops->get_sgtable)
return ops->get_sgtable(dev, sgt, cpu_addr, dma_addr, size,
@@ -457,7 +457,7 @@ static inline void *dma_alloc_attrs(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t flag,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
void *cpu_addr;
BUG_ON(!ops);
@@ -479,7 +479,7 @@ static inline void dma_free_attrs(struct device *dev, size_t size,
void *cpu_addr, dma_addr_t dma_handle,
unsigned long attrs)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
BUG_ON(!ops);
WARN_ON(irqs_disabled());
@@ -537,7 +537,7 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
#ifndef HAVE_ARCH_DMA_SUPPORTED
static inline int dma_supported(struct device *dev, u64 mask)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
if (!ops)
return 0;
@@ -550,7 +550,7 @@ static inline int dma_supported(struct device *dev, u64 mask)
#ifndef HAVE_ARCH_DMA_SET_MASK
static inline int dma_set_mask(struct device *dev, u64 mask)
{
- struct dma_map_ops *ops = get_dma_ops(dev);
+ const struct dma_map_ops *ops = get_dma_ops(dev);
if (ops->set_dma_mask)
return ops->set_dma_mask(dev, mask);
diff --git a/include/linux/mic_bus.h b/include/linux/mic_bus.h
index 27d7c95fd0da..504d54c71bdb 100644
--- a/include/linux/mic_bus.h
+++ b/include/linux/mic_bus.h
@@ -90,7 +90,7 @@ struct mbus_hw_ops {
};
struct mbus_device *
-mbus_register_device(struct device *pdev, int id, struct dma_map_ops *dma_ops,
+mbus_register_device(struct device *pdev, int id, const struct dma_map_ops *dma_ops,
struct mbus_hw_ops *hw_ops, int index,
void __iomem *mmio_va);
void mbus_unregister_device(struct mbus_device *mbdev);
diff --git a/include/xen/arm/hypervisor.h b/include/xen/arm/hypervisor.h
index 95251512e2c4..44b587b49904 100644
--- a/include/xen/arm/hypervisor.h
+++ b/include/xen/arm/hypervisor.h
@@ -18,7 +18,7 @@ static inline enum paravirt_lazy_mode paravirt_get_lazy_mode(void)
return PARAVIRT_LAZY_NONE;
}
-extern struct dma_map_ops *xen_dma_ops;
+extern const struct dma_map_ops *xen_dma_ops;
#ifdef CONFIG_XEN
void __init xen_early_init(void);
diff --git a/lib/dma-noop.c b/lib/dma-noop.c
index 3d766e78fbe2..65e49dd35b7b 100644
--- a/lib/dma-noop.c
+++ b/lib/dma-noop.c
@@ -64,7 +64,7 @@ static int dma_noop_supported(struct device *dev, u64 mask)
return 1;
}
-struct dma_map_ops dma_noop_ops = {
+const struct dma_map_ops dma_noop_ops = {
.alloc = dma_noop_alloc,
.free = dma_noop_free,
.map_page = dma_noop_map_page,
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 02/26] treewide: Move dma_ops from struct dev_archdata into struct device
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
2017-01-12 19:06 ` [PATCH v2 01/26] treewide: Constify most dma_map_ops structures Bart Van Assche
@ 2017-01-12 19:06 ` Bart Van Assche
2017-01-19 9:10 ` Greg Kroah-Hartman
2017-01-12 19:06 ` [PATCH v2 03/26] treewide: Consolidate set_dma_ops() implementations Bart Van Assche
` (25 subsequent siblings)
27 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:06 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Benjamin Herrenschmidt, Boris Ostrovsky, David Woodhouse,
Juergen Gross, H . Peter Anvin, Ingo Molnar, linux-arch,
Russell King, x86
Some but not all architectures provide set_dma_ops(). Move dma_ops
from struct dev_archdata into struct device such that it becomes
possible on all architectures to configure dma_ops per device.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Juergen Gross <jgross@suse.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Russell King <linux@armlinux.org.uk>
Cc: x86@kernel.org
---
arch/arm/include/asm/device.h | 1 -
arch/arm/include/asm/dma-mapping.h | 6 +++---
arch/arm64/include/asm/device.h | 1 -
arch/arm64/include/asm/dma-mapping.h | 4 ++--
arch/arm64/mm/dma-mapping.c | 8 ++++----
arch/m32r/include/asm/device.h | 1 -
arch/m32r/include/asm/dma-mapping.h | 4 ++--
arch/mips/include/asm/device.h | 5 -----
arch/mips/include/asm/dma-mapping.h | 4 ++--
arch/mips/pci/pci-octeon.c | 2 +-
arch/powerpc/include/asm/device.h | 4 ----
arch/powerpc/include/asm/dma-mapping.h | 4 ++--
arch/powerpc/kernel/dma.c | 2 +-
arch/powerpc/platforms/cell/iommu.c | 2 +-
arch/powerpc/platforms/pasemi/iommu.c | 2 +-
arch/powerpc/platforms/pasemi/setup.c | 2 +-
arch/powerpc/platforms/ps3/system-bus.c | 4 ++--
arch/powerpc/platforms/pseries/ibmebus.c | 2 +-
arch/s390/include/asm/device.h | 1 -
arch/s390/include/asm/dma-mapping.h | 4 ++--
arch/s390/pci/pci.c | 2 +-
arch/tile/include/asm/device.h | 3 ---
arch/tile/include/asm/dma-mapping.h | 6 +++---
arch/x86/include/asm/device.h | 3 ---
arch/x86/include/asm/dma-mapping.h | 4 ++--
arch/x86/kernel/pci-calgary_64.c | 4 ++--
arch/x86/pci/common.c | 2 +-
arch/x86/pci/sta2x11-fixup.c | 8 ++++----
arch/xtensa/include/asm/device.h | 4 ----
arch/xtensa/include/asm/dma-mapping.h | 4 ++--
drivers/infiniband/ulp/srpt/ib_srpt.c | 2 +-
drivers/iommu/amd_iommu.c | 6 +++---
drivers/misc/mic/bus/mic_bus.c | 2 +-
drivers/misc/mic/bus/scif_bus.c | 2 +-
drivers/misc/mic/bus/vop_bus.c | 2 +-
include/linux/device.h | 1 +
36 files changed, 48 insertions(+), 70 deletions(-)
diff --git a/arch/arm/include/asm/device.h b/arch/arm/include/asm/device.h
index d8a572f9c187..220ba207be91 100644
--- a/arch/arm/include/asm/device.h
+++ b/arch/arm/include/asm/device.h
@@ -7,7 +7,6 @@
#define ASMARM_DEVICE_H
struct dev_archdata {
- const struct dma_map_ops *dma_ops;
#ifdef CONFIG_DMABOUNCE
struct dmabounce_device_info *dmabounce;
#endif
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index 1aabd781306f..312f4d0564d6 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -18,8 +18,8 @@ extern const struct dma_map_ops arm_coherent_dma_ops;
static inline const struct dma_map_ops *__generic_dma_ops(struct device *dev)
{
- if (dev && dev->archdata.dma_ops)
- return dev->archdata.dma_ops;
+ if (dev && dev->dma_ops)
+ return dev->dma_ops;
return &arm_dma_ops;
}
@@ -34,7 +34,7 @@ static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
{
BUG_ON(!dev);
- dev->archdata.dma_ops = ops;
+ dev->dma_ops = ops;
}
#define HAVE_ARCH_DMA_SUPPORTED 1
diff --git a/arch/arm64/include/asm/device.h b/arch/arm64/include/asm/device.h
index 00c678cc31e1..73d5bab015eb 100644
--- a/arch/arm64/include/asm/device.h
+++ b/arch/arm64/include/asm/device.h
@@ -17,7 +17,6 @@
#define __ASM_DEVICE_H
struct dev_archdata {
- const struct dma_map_ops *dma_ops;
#ifdef CONFIG_IOMMU_API
void *iommu; /* private IOMMU data */
#endif
diff --git a/arch/arm64/include/asm/dma-mapping.h b/arch/arm64/include/asm/dma-mapping.h
index 1fedb43be712..58ae36cc3b60 100644
--- a/arch/arm64/include/asm/dma-mapping.h
+++ b/arch/arm64/include/asm/dma-mapping.h
@@ -29,8 +29,8 @@ extern const struct dma_map_ops dummy_dma_ops;
static inline const struct dma_map_ops *__generic_dma_ops(struct device *dev)
{
- if (dev && dev->archdata.dma_ops)
- return dev->archdata.dma_ops;
+ if (dev && dev->dma_ops)
+ return dev->dma_ops;
/*
* We expect no ISA devices, and all other DMA masters are expected to
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index bcef6368d48f..dbab4c6c084b 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -837,7 +837,7 @@ static bool do_iommu_attach(struct device *dev, const struct iommu_ops *ops,
return false;
}
- dev->archdata.dma_ops = &iommu_dma_ops;
+ dev->dma_ops = &iommu_dma_ops;
return true;
}
@@ -941,7 +941,7 @@ static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
void arch_teardown_dma_ops(struct device *dev)
{
- dev->archdata.dma_ops = NULL;
+ dev->dma_ops = NULL;
}
#else
@@ -955,8 +955,8 @@ static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *iommu, bool coherent)
{
- if (!dev->archdata.dma_ops)
- dev->archdata.dma_ops = &swiotlb_dma_ops;
+ if (!dev->dma_ops)
+ dev->dma_ops = &swiotlb_dma_ops;
dev->archdata.dma_coherent = coherent;
__iommu_setup_dma_ops(dev, dma_base, size, iommu);
diff --git a/arch/m32r/include/asm/device.h b/arch/m32r/include/asm/device.h
index 7955a9799466..5203fc87f080 100644
--- a/arch/m32r/include/asm/device.h
+++ b/arch/m32r/include/asm/device.h
@@ -4,7 +4,6 @@
* This file is released under the GPLv2
*/
struct dev_archdata {
- const struct dma_map_ops *dma_ops;
};
struct pdev_archdata {
diff --git a/arch/m32r/include/asm/dma-mapping.h b/arch/m32r/include/asm/dma-mapping.h
index 99c43d2f05dc..27b1597ac563 100644
--- a/arch/m32r/include/asm/dma-mapping.h
+++ b/arch/m32r/include/asm/dma-mapping.h
@@ -12,8 +12,8 @@
static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
- if (dev && dev->archdata.dma_ops)
- return dev->archdata.dma_ops;
+ if (dev && dev->dma_ops)
+ return dev->dma_ops;
return &dma_noop_ops;
}
diff --git a/arch/mips/include/asm/device.h b/arch/mips/include/asm/device.h
index ebc5c1265473..6aa796f1081a 100644
--- a/arch/mips/include/asm/device.h
+++ b/arch/mips/include/asm/device.h
@@ -6,12 +6,7 @@
#ifndef _ASM_MIPS_DEVICE_H
#define _ASM_MIPS_DEVICE_H
-struct dma_map_ops;
-
struct dev_archdata {
- /* DMA operations on that device */
- const struct dma_map_ops *dma_ops;
-
#ifdef CONFIG_DMA_PERDEV_COHERENT
/* Non-zero if DMA is coherent with CPU caches */
bool dma_coherent;
diff --git a/arch/mips/include/asm/dma-mapping.h b/arch/mips/include/asm/dma-mapping.h
index b59b084a7569..dad3b09fe993 100644
--- a/arch/mips/include/asm/dma-mapping.h
+++ b/arch/mips/include/asm/dma-mapping.h
@@ -13,8 +13,8 @@ extern const struct dma_map_ops *mips_dma_map_ops;
static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
- if (dev && dev->archdata.dma_ops)
- return dev->archdata.dma_ops;
+ if (dev && dev->dma_ops)
+ return dev->dma_ops;
else
return mips_dma_map_ops;
}
diff --git a/arch/mips/pci/pci-octeon.c b/arch/mips/pci/pci-octeon.c
index 308d051fc45c..9ee01936862e 100644
--- a/arch/mips/pci/pci-octeon.c
+++ b/arch/mips/pci/pci-octeon.c
@@ -167,7 +167,7 @@ int pcibios_plat_dev_init(struct pci_dev *dev)
pci_write_config_dword(dev, pos + PCI_ERR_ROOT_STATUS, dconfig);
}
- dev->dev.archdata.dma_ops = octeon_pci_dma_map_ops;
+ dev->dev.dma_ops = octeon_pci_dma_map_ops;
return 0;
}
diff --git a/arch/powerpc/include/asm/device.h b/arch/powerpc/include/asm/device.h
index 49cbb0fca233..0245bfcaac32 100644
--- a/arch/powerpc/include/asm/device.h
+++ b/arch/powerpc/include/asm/device.h
@@ -6,7 +6,6 @@
#ifndef _ASM_POWERPC_DEVICE_H
#define _ASM_POWERPC_DEVICE_H
-struct dma_map_ops;
struct device_node;
#ifdef CONFIG_PPC64
struct pci_dn;
@@ -20,9 +19,6 @@ struct iommu_table;
* drivers/macintosh/macio_asic.c
*/
struct dev_archdata {
- /* DMA operations on that device */
- const struct dma_map_ops *dma_ops;
-
/*
* These two used to be a union. However, with the hybrid ops we need
* both so here we store both a DMA offset for direct mappings and
diff --git a/arch/powerpc/include/asm/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h
index 2ec3eadf336f..59fbd4abcbf8 100644
--- a/arch/powerpc/include/asm/dma-mapping.h
+++ b/arch/powerpc/include/asm/dma-mapping.h
@@ -88,12 +88,12 @@ static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
if (unlikely(dev == NULL))
return NULL;
- return dev->archdata.dma_ops;
+ return dev->dma_ops;
}
static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
{
- dev->archdata.dma_ops = ops;
+ dev->dma_ops = ops;
}
/*
diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
index 03b98f1f98ec..41c749586bd2 100644
--- a/arch/powerpc/kernel/dma.c
+++ b/arch/powerpc/kernel/dma.c
@@ -33,7 +33,7 @@ static u64 __maybe_unused get_pfn_limit(struct device *dev)
struct dev_archdata __maybe_unused *sd = &dev->archdata;
#ifdef CONFIG_SWIOTLB
- if (sd->max_direct_dma_addr && sd->dma_ops == &swiotlb_dma_ops)
+ if (sd->max_direct_dma_addr && dev->dma_ops == &swiotlb_dma_ops)
pfn = min_t(u64, pfn, sd->max_direct_dma_addr >> PAGE_SHIFT);
#endif
diff --git a/arch/powerpc/platforms/cell/iommu.c b/arch/powerpc/platforms/cell/iommu.c
index e1413e69e5fe..71b995bbcae0 100644
--- a/arch/powerpc/platforms/cell/iommu.c
+++ b/arch/powerpc/platforms/cell/iommu.c
@@ -692,7 +692,7 @@ static int cell_of_bus_notify(struct notifier_block *nb, unsigned long action,
return 0;
/* We use the PCI DMA ops */
- dev->archdata.dma_ops = get_pci_dma_ops();
+ dev->dma_ops = get_pci_dma_ops();
cell_dma_dev_setup(dev);
diff --git a/arch/powerpc/platforms/pasemi/iommu.c b/arch/powerpc/platforms/pasemi/iommu.c
index e74adc4e7fd8..7fec04de27fc 100644
--- a/arch/powerpc/platforms/pasemi/iommu.c
+++ b/arch/powerpc/platforms/pasemi/iommu.c
@@ -186,7 +186,7 @@ static void pci_dma_dev_setup_pasemi(struct pci_dev *dev)
*/
if (dev->vendor == 0x1959 && dev->device == 0xa007 &&
!firmware_has_feature(FW_FEATURE_LPAR)) {
- dev->dev.archdata.dma_ops = &dma_direct_ops;
+ dev->dev.dma_ops = &dma_direct_ops;
/*
* Set the coherent DMA mask to prevent the iommu
* being used unnecessarily
diff --git a/arch/powerpc/platforms/pasemi/setup.c b/arch/powerpc/platforms/pasemi/setup.c
index 3182400cf48f..c4a3e93dc324 100644
--- a/arch/powerpc/platforms/pasemi/setup.c
+++ b/arch/powerpc/platforms/pasemi/setup.c
@@ -363,7 +363,7 @@ static int pcmcia_notify(struct notifier_block *nb, unsigned long action,
return 0;
/* We use the direct ops for localbus */
- dev->archdata.dma_ops = &dma_direct_ops;
+ dev->dma_ops = &dma_direct_ops;
return 0;
}
diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
index c81450d98794..2d2e5f80a3d3 100644
--- a/arch/powerpc/platforms/ps3/system-bus.c
+++ b/arch/powerpc/platforms/ps3/system-bus.c
@@ -756,11 +756,11 @@ int ps3_system_bus_device_register(struct ps3_system_bus_device *dev)
switch (dev->dev_type) {
case PS3_DEVICE_TYPE_IOC0:
- dev->core.archdata.dma_ops = &ps3_ioc0_dma_ops;
+ dev->core.dma_ops = &ps3_ioc0_dma_ops;
dev_set_name(&dev->core, "ioc0_%02x", ++dev_ioc0_count);
break;
case PS3_DEVICE_TYPE_SB:
- dev->core.archdata.dma_ops = &ps3_sb_dma_ops;
+ dev->core.dma_ops = &ps3_sb_dma_ops;
dev_set_name(&dev->core, "sb_%02x", ++dev_sb_count);
break;
diff --git a/arch/powerpc/platforms/pseries/ibmebus.c b/arch/powerpc/platforms/pseries/ibmebus.c
index 2e36a0b8944a..99a6bf7f3bcf 100644
--- a/arch/powerpc/platforms/pseries/ibmebus.c
+++ b/arch/powerpc/platforms/pseries/ibmebus.c
@@ -169,7 +169,7 @@ static int ibmebus_create_device(struct device_node *dn)
return -ENOMEM;
dev->dev.bus = &ibmebus_bus_type;
- dev->dev.archdata.dma_ops = &ibmebus_dma_ops;
+ dev->dev.dma_ops = &ibmebus_dma_ops;
ret = of_device_add(dev);
if (ret)
diff --git a/arch/s390/include/asm/device.h b/arch/s390/include/asm/device.h
index 7955a9799466..5203fc87f080 100644
--- a/arch/s390/include/asm/device.h
+++ b/arch/s390/include/asm/device.h
@@ -4,7 +4,6 @@
* This file is released under the GPLv2
*/
struct dev_archdata {
- const struct dma_map_ops *dma_ops;
};
struct pdev_archdata {
diff --git a/arch/s390/include/asm/dma-mapping.h b/arch/s390/include/asm/dma-mapping.h
index 2776d205b1ff..a872027d0c1b 100644
--- a/arch/s390/include/asm/dma-mapping.h
+++ b/arch/s390/include/asm/dma-mapping.h
@@ -14,8 +14,8 @@ extern const struct dma_map_ops s390_pci_dma_ops;
static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
- if (dev && dev->archdata.dma_ops)
- return dev->archdata.dma_ops;
+ if (dev && dev->dma_ops)
+ return dev->dma_ops;
return &dma_noop_ops;
}
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c
index 38e17d4d9884..82abef8b8574 100644
--- a/arch/s390/pci/pci.c
+++ b/arch/s390/pci/pci.c
@@ -641,7 +641,7 @@ int pcibios_add_device(struct pci_dev *pdev)
int i;
pdev->dev.groups = zpci_attr_groups;
- pdev->dev.archdata.dma_ops = &s390_pci_dma_ops;
+ pdev->dev.dma_ops = &s390_pci_dma_ops;
zpci_map_resources(pdev);
for (i = 0; i < PCI_BAR_COUNT; i++) {
diff --git a/arch/tile/include/asm/device.h b/arch/tile/include/asm/device.h
index 25f23ac7d361..1cf45422a0df 100644
--- a/arch/tile/include/asm/device.h
+++ b/arch/tile/include/asm/device.h
@@ -17,9 +17,6 @@
#define _ASM_TILE_DEVICE_H
struct dev_archdata {
- /* DMA operations on that device */
- const struct dma_map_ops *dma_ops;
-
/* Offset of the DMA address from the PA. */
dma_addr_t dma_offset;
diff --git a/arch/tile/include/asm/dma-mapping.h b/arch/tile/include/asm/dma-mapping.h
index 4a06cc75b856..c0620697eaad 100644
--- a/arch/tile/include/asm/dma-mapping.h
+++ b/arch/tile/include/asm/dma-mapping.h
@@ -31,8 +31,8 @@ extern const struct dma_map_ops *gx_hybrid_pci_dma_map_ops;
static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
- if (dev && dev->archdata.dma_ops)
- return dev->archdata.dma_ops;
+ if (dev && dev->dma_ops)
+ return dev->dma_ops;
else
return tile_dma_map_ops;
}
@@ -61,7 +61,7 @@ static inline void dma_mark_clean(void *addr, size_t size) {}
static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
{
- dev->archdata.dma_ops = ops;
+ dev->dma_ops = ops;
}
static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
diff --git a/arch/x86/include/asm/device.h b/arch/x86/include/asm/device.h
index b2d0b4ced7e3..1b3ef26e77df 100644
--- a/arch/x86/include/asm/device.h
+++ b/arch/x86/include/asm/device.h
@@ -2,9 +2,6 @@
#define _ASM_X86_DEVICE_H
struct dev_archdata {
-#ifdef CONFIG_X86_DEV_DMA_OPS
- const struct dma_map_ops *dma_ops;
-#endif
#if defined(CONFIG_INTEL_IOMMU) || defined(CONFIG_AMD_IOMMU)
void *iommu; /* hook for IOMMU specific extension */
#endif
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 5e4772886a1e..94b5b96966cb 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -32,10 +32,10 @@ static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
#ifndef CONFIG_X86_DEV_DMA_OPS
return dma_ops;
#else
- if (unlikely(!dev) || !dev->archdata.dma_ops)
+ if (unlikely(!dev) || !dev->dma_ops)
return dma_ops;
else
- return dev->archdata.dma_ops;
+ return dev->dma_ops;
#endif
}
diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 17f180148c80..5070320780c6 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -1177,7 +1177,7 @@ static int __init calgary_init(void)
tbl = find_iommu_table(&dev->dev);
if (translation_enabled(tbl))
- dev->dev.archdata.dma_ops = &calgary_dma_ops;
+ dev->dev.dma_ops = &calgary_dma_ops;
}
return ret;
@@ -1201,7 +1201,7 @@ static int __init calgary_init(void)
calgary_disable_translation(dev);
calgary_free_bus(dev);
pci_dev_put(dev); /* Undo calgary_init_one()'s pci_dev_get() */
- dev->dev.archdata.dma_ops = NULL;
+ dev->dev.dma_ops = NULL;
} while (1);
return ret;
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index a4fdfa7dcc1b..0cb52ae0a8f0 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -667,7 +667,7 @@ static void set_dma_domain_ops(struct pci_dev *pdev)
spin_lock(&dma_domain_list_lock);
list_for_each_entry(domain, &dma_domain_list, node) {
if (pci_domain_nr(pdev->bus) == domain->domain_nr) {
- pdev->dev.archdata.dma_ops = domain->dma_ops;
+ pdev->dev.dma_ops = domain->dma_ops;
break;
}
}
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index aa3828823170..ec008e800b45 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -203,7 +203,7 @@ static void sta2x11_setup_pdev(struct pci_dev *pdev)
return;
pci_set_consistent_dma_mask(pdev, STA2X11_AMBA_SIZE - 1);
pci_set_dma_mask(pdev, STA2X11_AMBA_SIZE - 1);
- pdev->dev.archdata.dma_ops = &sta2x11_dma_ops;
+ pdev->dev.dma_ops = &sta2x11_dma_ops;
/* We must enable all devices as master, for audio DMA to work */
pci_set_master(pdev);
@@ -223,7 +223,7 @@ bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
{
struct sta2x11_mapping *map;
- if (dev->archdata.dma_ops != &sta2x11_dma_ops) {
+ if (dev->dma_ops != &sta2x11_dma_ops) {
if (!dev->dma_mask)
return false;
return addr + size - 1 <= *dev->dma_mask;
@@ -247,7 +247,7 @@ bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
*/
dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
{
- if (dev->archdata.dma_ops != &sta2x11_dma_ops)
+ if (dev->dma_ops != &sta2x11_dma_ops)
return paddr;
return p2a(paddr, to_pci_dev(dev));
}
@@ -259,7 +259,7 @@ dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
*/
phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
{
- if (dev->archdata.dma_ops != &sta2x11_dma_ops)
+ if (dev->dma_ops != &sta2x11_dma_ops)
return daddr;
return a2p(daddr, to_pci_dev(dev));
}
diff --git a/arch/xtensa/include/asm/device.h b/arch/xtensa/include/asm/device.h
index a77d45d39f35..1deeb8ebbb1b 100644
--- a/arch/xtensa/include/asm/device.h
+++ b/arch/xtensa/include/asm/device.h
@@ -6,11 +6,7 @@
#ifndef _ASM_XTENSA_DEVICE_H
#define _ASM_XTENSA_DEVICE_H
-struct dma_map_ops;
-
struct dev_archdata {
- /* DMA operations on that device */
- const struct dma_map_ops *dma_ops;
};
struct pdev_archdata {
diff --git a/arch/xtensa/include/asm/dma-mapping.h b/arch/xtensa/include/asm/dma-mapping.h
index 50d23106cce0..9eecfc3c5dc4 100644
--- a/arch/xtensa/include/asm/dma-mapping.h
+++ b/arch/xtensa/include/asm/dma-mapping.h
@@ -22,8 +22,8 @@ extern const struct dma_map_ops xtensa_dma_map_ops;
static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
{
- if (dev && dev->archdata.dma_ops)
- return dev->archdata.dma_ops;
+ if (dev && dev->dma_ops)
+ return dev->dma_ops;
else
return &xtensa_dma_map_ops;
}
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 3fc3f8b5752b..7ff9de063c53 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -2524,7 +2524,7 @@ static void srpt_add_one(struct ib_device *device)
int i;
pr_debug("device = %p, device->dma_ops = %p\n", device,
- device->dma_ops);
+ device->dma_device->dma_ops);
sdev = kzalloc(sizeof(*sdev), GFP_KERNEL);
if (!sdev)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 3703fb9db419..f7b86679bafe 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -515,7 +515,7 @@ static void iommu_uninit_device(struct device *dev)
iommu_group_remove_device(dev);
/* Remove dma-ops */
- dev->archdata.dma_ops = NULL;
+ dev->dma_ops = NULL;
/*
* We keep dev_data around for unplugged devices and reuse it when the
@@ -2164,7 +2164,7 @@ static int amd_iommu_add_device(struct device *dev)
dev_name(dev));
iommu_ignore_device(dev);
- dev->archdata.dma_ops = &nommu_dma_ops;
+ dev->dma_ops = &nommu_dma_ops;
goto out;
}
init_iommu_group(dev);
@@ -2181,7 +2181,7 @@ static int amd_iommu_add_device(struct device *dev)
if (domain->type == IOMMU_DOMAIN_IDENTITY)
dev_data->passthrough = true;
else
- dev->archdata.dma_ops = &amd_iommu_dma_ops;
+ dev->dma_ops = &amd_iommu_dma_ops;
out:
iommu_completion_wait(iommu);
diff --git a/drivers/misc/mic/bus/mic_bus.c b/drivers/misc/mic/bus/mic_bus.c
index c4b27a25662a..77b16ca66846 100644
--- a/drivers/misc/mic/bus/mic_bus.c
+++ b/drivers/misc/mic/bus/mic_bus.c
@@ -158,7 +158,7 @@ mbus_register_device(struct device *pdev, int id, const struct dma_map_ops *dma_
mbdev->dev.parent = pdev;
mbdev->id.device = id;
mbdev->id.vendor = MBUS_DEV_ANY_ID;
- mbdev->dev.archdata.dma_ops = dma_ops;
+ mbdev->dev.dma_ops = dma_ops;
mbdev->dev.dma_mask = &mbdev->dev.coherent_dma_mask;
dma_set_mask(&mbdev->dev, DMA_BIT_MASK(64));
mbdev->dev.release = mbus_release_dev;
diff --git a/drivers/misc/mic/bus/scif_bus.c b/drivers/misc/mic/bus/scif_bus.c
index e5d377e97c86..a444db5f61fe 100644
--- a/drivers/misc/mic/bus/scif_bus.c
+++ b/drivers/misc/mic/bus/scif_bus.c
@@ -154,7 +154,7 @@ scif_register_device(struct device *pdev, int id, const struct dma_map_ops *dma_
sdev->dev.parent = pdev;
sdev->id.device = id;
sdev->id.vendor = SCIF_DEV_ANY_ID;
- sdev->dev.archdata.dma_ops = dma_ops;
+ sdev->dev.dma_ops = dma_ops;
sdev->dev.release = scif_release_dev;
sdev->hw_ops = hw_ops;
sdev->dnode = dnode;
diff --git a/drivers/misc/mic/bus/vop_bus.c b/drivers/misc/mic/bus/vop_bus.c
index e3caa6c53922..fd7f2a6049f8 100644
--- a/drivers/misc/mic/bus/vop_bus.c
+++ b/drivers/misc/mic/bus/vop_bus.c
@@ -154,7 +154,7 @@ vop_register_device(struct device *pdev, int id,
vdev->dev.parent = pdev;
vdev->id.device = id;
vdev->id.vendor = VOP_DEV_ANY_ID;
- vdev->dev.archdata.dma_ops = dma_ops;
+ vdev->dev.dma_ops = dma_ops;
vdev->dev.dma_mask = &vdev->dev.coherent_dma_mask;
dma_set_mask(&vdev->dev, DMA_BIT_MASK(64));
vdev->dev.release = vop_release_dev;
diff --git a/include/linux/device.h b/include/linux/device.h
index 491b4c0ca633..46a567261ccc 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -921,6 +921,7 @@ struct device {
#ifdef CONFIG_NUMA
int numa_node; /* NUMA node this device is close to */
#endif
+ const struct dma_map_ops *dma_ops;
u64 *dma_mask; /* dma mask (if dma'able device) */
u64 coherent_dma_mask;/* Like dma_mask, but for
alloc_coherent mappings as
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PATCH v2 02/26] treewide: Move dma_ops from struct dev_archdata into struct device
2017-01-12 19:06 ` [PATCH v2 02/26] treewide: Move dma_ops from struct dev_archdata into struct device Bart Van Assche
@ 2017-01-19 9:10 ` Greg Kroah-Hartman
0 siblings, 0 replies; 45+ messages in thread
From: Greg Kroah-Hartman @ 2017-01-19 9:10 UTC (permalink / raw)
To: Bart Van Assche
Cc: Doug Ledford, linux-rdma, linux-kernel, Benjamin Herrenschmidt,
Boris Ostrovsky, David Woodhouse, Juergen Gross, H . Peter Anvin,
Ingo Molnar, linux-arch, Russell King, x86
On Thu, Jan 12, 2017 at 11:06:54AM -0800, Bart Van Assche wrote:
> Some but not all architectures provide set_dma_ops(). Move dma_ops
> from struct dev_archdata into struct device such that it becomes
> possible on all architectures to configure dma_ops per device.
>
> Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: David Woodhouse <dwmw2@infradead.org>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: H. Peter Anvin <hpa@zytor.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: linux-arch@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: x86@kernel.org
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
^ permalink raw reply [flat|nested] 45+ messages in thread
* [PATCH v2 03/26] treewide: Consolidate set_dma_ops() implementations
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
2017-01-12 19:06 ` [PATCH v2 01/26] treewide: Constify most dma_map_ops structures Bart Van Assche
2017-01-12 19:06 ` [PATCH v2 02/26] treewide: Move dma_ops from struct dev_archdata into struct device Bart Van Assche
@ 2017-01-12 19:06 ` Bart Van Assche
2017-01-12 19:06 ` [PATCH v2 04/26] treewide: Consolidate get_dma_ops() implementations Bart Van Assche
` (24 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:06 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Benjamin Herrenschmidt, Chris Metcalf, David Woodhouse,
linux-arch, linux-arm-kernel, linuxppc-dev, Paul Mackerras,
Russell King
Now that all set_dma_ops() implementations are identical (ignoring
BUG_ON() statements), remove the architecture specific definitions
and add a definition in <linux/dma-mapping.h>.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: linux-arch@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: Paul Mackerras <paulus@samba.org>
Cc: Russell King <linux@armlinux.org.uk>
---
arch/arm/include/asm/dma-mapping.h | 6 ------
arch/powerpc/include/asm/dma-mapping.h | 5 -----
arch/tile/include/asm/dma-mapping.h | 5 -----
include/linux/dma-mapping.h | 5 +++++
4 files changed, 5 insertions(+), 16 deletions(-)
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index 312f4d0564d6..c7432d647e53 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -31,12 +31,6 @@ static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
return __generic_dma_ops(dev);
}
-static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
-{
- BUG_ON(!dev);
- dev->dma_ops = ops;
-}
-
#define HAVE_ARCH_DMA_SUPPORTED 1
extern int dma_supported(struct device *dev, u64 mask);
diff --git a/arch/powerpc/include/asm/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h
index 59fbd4abcbf8..8275603ba4d5 100644
--- a/arch/powerpc/include/asm/dma-mapping.h
+++ b/arch/powerpc/include/asm/dma-mapping.h
@@ -91,11 +91,6 @@ static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
return dev->dma_ops;
}
-static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
-{
- dev->dma_ops = ops;
-}
-
/*
* get_dma_offset()
*
diff --git a/arch/tile/include/asm/dma-mapping.h b/arch/tile/include/asm/dma-mapping.h
index c0620697eaad..2562995a6ac9 100644
--- a/arch/tile/include/asm/dma-mapping.h
+++ b/arch/tile/include/asm/dma-mapping.h
@@ -59,11 +59,6 @@ static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr)
static inline void dma_mark_clean(void *addr, size_t size) {}
-static inline void set_dma_ops(struct device *dev, const struct dma_map_ops *ops)
-{
- dev->dma_ops = ops;
-}
-
static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
{
if (!dev->dma_mask)
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index f1da68b82c63..e97f23e8b2d9 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -164,6 +164,11 @@ int dma_mmap_from_coherent(struct device *dev, struct vm_area_struct *vma,
#ifdef CONFIG_HAS_DMA
#include <asm/dma-mapping.h>
+static inline void set_dma_ops(struct device *dev,
+ const struct dma_map_ops *dma_ops)
+{
+ dev->dma_ops = dma_ops;
+}
#else
/*
* Define the dma api to allow compilation but not linking of
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 04/26] treewide: Consolidate get_dma_ops() implementations
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (2 preceding siblings ...)
2017-01-12 19:06 ` [PATCH v2 03/26] treewide: Consolidate set_dma_ops() implementations Bart Van Assche
@ 2017-01-12 19:06 ` Bart Van Assche
2017-01-12 19:06 ` [PATCH v2 05/26] lib/dma-noop: Clarify a comment Bart Van Assche
` (23 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:06 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Benjamin Herrenschmidt, Boris Ostrovsky, David Woodhouse,
Juergen Gross, H . Peter Anvin, Ingo Molnar, linux-arch,
Russell King, x86
Introduce a new architecture-specific get_arch_dma_ops() function
that takes a strut bus_type * argument. Add get_dma_ops() in
<linux/dma-mapping.h>.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Juergen Gross <jgross@suse.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: linux-arch@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Russell King <linux@armlinux.org.uk>
Cc: x86@kernel.org
---
arch/alpha/include/asm/dma-mapping.h | 2 +-
arch/arc/include/asm/dma-mapping.h | 2 +-
arch/arm/include/asm/dma-mapping.h | 4 ++--
arch/arm64/include/asm/dma-mapping.h | 4 ++--
arch/avr32/include/asm/dma-mapping.h | 2 +-
arch/blackfin/include/asm/dma-mapping.h | 2 +-
arch/c6x/include/asm/dma-mapping.h | 2 +-
arch/cris/include/asm/dma-mapping.h | 4 ++--
arch/frv/include/asm/dma-mapping.h | 2 +-
arch/h8300/include/asm/dma-mapping.h | 2 +-
arch/hexagon/include/asm/dma-mapping.h | 5 +----
arch/ia64/include/asm/dma-mapping.h | 5 ++++-
arch/m32r/include/asm/dma-mapping.h | 4 +---
arch/m68k/include/asm/dma-mapping.h | 2 +-
arch/metag/include/asm/dma-mapping.h | 2 +-
arch/microblaze/include/asm/dma-mapping.h | 2 +-
arch/mips/include/asm/dma-mapping.h | 7 ++-----
arch/mn10300/include/asm/dma-mapping.h | 2 +-
arch/nios2/include/asm/dma-mapping.h | 2 +-
arch/openrisc/include/asm/dma-mapping.h | 2 +-
arch/parisc/include/asm/dma-mapping.h | 2 +-
arch/powerpc/include/asm/dma-mapping.h | 7 ++-----
arch/powerpc/include/asm/ps3.h | 2 +-
arch/s390/include/asm/dma-mapping.h | 4 +---
arch/sh/include/asm/dma-mapping.h | 2 +-
arch/sparc/include/asm/dma-mapping.h | 4 ++--
arch/tile/include/asm/dma-mapping.h | 7 ++-----
arch/unicore32/include/asm/dma-mapping.h | 2 +-
arch/x86/include/asm/dma-mapping.h | 9 +--------
arch/xtensa/include/asm/dma-mapping.h | 7 ++-----
include/linux/dma-mapping.h | 7 +++++++
31 files changed, 48 insertions(+), 64 deletions(-)
diff --git a/arch/alpha/include/asm/dma-mapping.h b/arch/alpha/include/asm/dma-mapping.h
index d3480562411d..5d53666935e6 100644
--- a/arch/alpha/include/asm/dma-mapping.h
+++ b/arch/alpha/include/asm/dma-mapping.h
@@ -3,7 +3,7 @@
extern const struct dma_map_ops *dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return dma_ops;
}
diff --git a/arch/arc/include/asm/dma-mapping.h b/arch/arc/include/asm/dma-mapping.h
index fdff3aa60052..94285031c4fb 100644
--- a/arch/arc/include/asm/dma-mapping.h
+++ b/arch/arc/include/asm/dma-mapping.h
@@ -20,7 +20,7 @@
extern const struct dma_map_ops arc_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &arc_dma_ops;
}
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index c7432d647e53..716656925975 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -23,12 +23,12 @@ static inline const struct dma_map_ops *__generic_dma_ops(struct device *dev)
return &arm_dma_ops;
}
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
if (xen_initial_domain())
return xen_dma_ops;
else
- return __generic_dma_ops(dev);
+ return __generic_dma_ops(NULL);
}
#define HAVE_ARCH_DMA_SUPPORTED 1
diff --git a/arch/arm64/include/asm/dma-mapping.h b/arch/arm64/include/asm/dma-mapping.h
index 58ae36cc3b60..505756cdc67a 100644
--- a/arch/arm64/include/asm/dma-mapping.h
+++ b/arch/arm64/include/asm/dma-mapping.h
@@ -39,12 +39,12 @@ static inline const struct dma_map_ops *__generic_dma_ops(struct device *dev)
return &dummy_dma_ops;
}
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
if (xen_initial_domain())
return xen_dma_ops;
else
- return __generic_dma_ops(dev);
+ return __generic_dma_ops(NULL);
}
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
diff --git a/arch/avr32/include/asm/dma-mapping.h b/arch/avr32/include/asm/dma-mapping.h
index b2b43c0e0774..7388451f9905 100644
--- a/arch/avr32/include/asm/dma-mapping.h
+++ b/arch/avr32/include/asm/dma-mapping.h
@@ -6,7 +6,7 @@ extern void dma_cache_sync(struct device *dev, void *vaddr, size_t size,
extern const struct dma_map_ops avr32_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &avr32_dma_ops;
}
diff --git a/arch/blackfin/include/asm/dma-mapping.h b/arch/blackfin/include/asm/dma-mapping.h
index 320fb50fbd41..04254ac36bed 100644
--- a/arch/blackfin/include/asm/dma-mapping.h
+++ b/arch/blackfin/include/asm/dma-mapping.h
@@ -38,7 +38,7 @@ _dma_sync(dma_addr_t addr, size_t size, enum dma_data_direction dir)
extern const struct dma_map_ops bfin_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &bfin_dma_ops;
}
diff --git a/arch/c6x/include/asm/dma-mapping.h b/arch/c6x/include/asm/dma-mapping.h
index 88258b9ebc8e..aca9f755e4f8 100644
--- a/arch/c6x/include/asm/dma-mapping.h
+++ b/arch/c6x/include/asm/dma-mapping.h
@@ -19,7 +19,7 @@
extern const struct dma_map_ops c6x_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &c6x_dma_ops;
}
diff --git a/arch/cris/include/asm/dma-mapping.h b/arch/cris/include/asm/dma-mapping.h
index aae4fbc0a656..256169de3743 100644
--- a/arch/cris/include/asm/dma-mapping.h
+++ b/arch/cris/include/asm/dma-mapping.h
@@ -4,12 +4,12 @@
#ifdef CONFIG_PCI
extern const struct dma_map_ops v32_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &v32_dma_ops;
}
#else
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
BUG();
return NULL;
diff --git a/arch/frv/include/asm/dma-mapping.h b/arch/frv/include/asm/dma-mapping.h
index 150cc00544a8..354900917585 100644
--- a/arch/frv/include/asm/dma-mapping.h
+++ b/arch/frv/include/asm/dma-mapping.h
@@ -9,7 +9,7 @@ extern unsigned long __nongprelbss dma_coherent_mem_end;
extern const struct dma_map_ops frv_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &frv_dma_ops;
}
diff --git a/arch/h8300/include/asm/dma-mapping.h b/arch/h8300/include/asm/dma-mapping.h
index f804bca4c13f..847c7562e046 100644
--- a/arch/h8300/include/asm/dma-mapping.h
+++ b/arch/h8300/include/asm/dma-mapping.h
@@ -3,7 +3,7 @@
extern const struct dma_map_ops h8300_dma_map_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &h8300_dma_map_ops;
}
diff --git a/arch/hexagon/include/asm/dma-mapping.h b/arch/hexagon/include/asm/dma-mapping.h
index b812e917cd95..d3a87bd9b686 100644
--- a/arch/hexagon/include/asm/dma-mapping.h
+++ b/arch/hexagon/include/asm/dma-mapping.h
@@ -34,11 +34,8 @@ extern int bad_dma_address;
extern const struct dma_map_ops *dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
- if (unlikely(dev == NULL))
- return NULL;
-
return dma_ops;
}
diff --git a/arch/ia64/include/asm/dma-mapping.h b/arch/ia64/include/asm/dma-mapping.h
index 05e467d56d86..73ec3c6f4cfe 100644
--- a/arch/ia64/include/asm/dma-mapping.h
+++ b/arch/ia64/include/asm/dma-mapping.h
@@ -23,7 +23,10 @@ extern void machvec_dma_sync_single(struct device *, dma_addr_t, size_t,
extern void machvec_dma_sync_sg(struct device *, struct scatterlist *, int,
enum dma_data_direction);
-#define get_dma_ops(dev) platform_dma_get_ops(dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
+{
+ return platform_dma_get_ops(NULL);
+}
static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
{
diff --git a/arch/m32r/include/asm/dma-mapping.h b/arch/m32r/include/asm/dma-mapping.h
index 27b1597ac563..c01d9f52d228 100644
--- a/arch/m32r/include/asm/dma-mapping.h
+++ b/arch/m32r/include/asm/dma-mapping.h
@@ -10,10 +10,8 @@
#define DMA_ERROR_CODE (~(dma_addr_t)0x0)
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
- if (dev && dev->dma_ops)
- return dev->dma_ops;
return &dma_noop_ops;
}
diff --git a/arch/m68k/include/asm/dma-mapping.h b/arch/m68k/include/asm/dma-mapping.h
index 863509939d5a..9210e470771b 100644
--- a/arch/m68k/include/asm/dma-mapping.h
+++ b/arch/m68k/include/asm/dma-mapping.h
@@ -3,7 +3,7 @@
extern const struct dma_map_ops m68k_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &m68k_dma_ops;
}
diff --git a/arch/metag/include/asm/dma-mapping.h b/arch/metag/include/asm/dma-mapping.h
index c156a7ac732f..fad3dc3cb210 100644
--- a/arch/metag/include/asm/dma-mapping.h
+++ b/arch/metag/include/asm/dma-mapping.h
@@ -3,7 +3,7 @@
extern const struct dma_map_ops metag_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &metag_dma_ops;
}
diff --git a/arch/microblaze/include/asm/dma-mapping.h b/arch/microblaze/include/asm/dma-mapping.h
index c7faf2fb51d6..3fad5e722a66 100644
--- a/arch/microblaze/include/asm/dma-mapping.h
+++ b/arch/microblaze/include/asm/dma-mapping.h
@@ -38,7 +38,7 @@
*/
extern const struct dma_map_ops dma_direct_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &dma_direct_ops;
}
diff --git a/arch/mips/include/asm/dma-mapping.h b/arch/mips/include/asm/dma-mapping.h
index dad3b09fe993..aba71385f9d1 100644
--- a/arch/mips/include/asm/dma-mapping.h
+++ b/arch/mips/include/asm/dma-mapping.h
@@ -11,12 +11,9 @@
extern const struct dma_map_ops *mips_dma_map_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
- if (dev && dev->dma_ops)
- return dev->dma_ops;
- else
- return mips_dma_map_ops;
+ return mips_dma_map_ops;
}
static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
diff --git a/arch/mn10300/include/asm/dma-mapping.h b/arch/mn10300/include/asm/dma-mapping.h
index 564e3927e005..737ef574b3ea 100644
--- a/arch/mn10300/include/asm/dma-mapping.h
+++ b/arch/mn10300/include/asm/dma-mapping.h
@@ -16,7 +16,7 @@
extern const struct dma_map_ops mn10300_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &mn10300_dma_ops;
}
diff --git a/arch/nios2/include/asm/dma-mapping.h b/arch/nios2/include/asm/dma-mapping.h
index aa00d839a64b..7b3c6f280293 100644
--- a/arch/nios2/include/asm/dma-mapping.h
+++ b/arch/nios2/include/asm/dma-mapping.h
@@ -12,7 +12,7 @@
extern const struct dma_map_ops nios2_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &nios2_dma_ops;
}
diff --git a/arch/openrisc/include/asm/dma-mapping.h b/arch/openrisc/include/asm/dma-mapping.h
index 88acbedb4947..0c0075f17145 100644
--- a/arch/openrisc/include/asm/dma-mapping.h
+++ b/arch/openrisc/include/asm/dma-mapping.h
@@ -30,7 +30,7 @@
extern const struct dma_map_ops or1k_dma_map_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &or1k_dma_map_ops;
}
diff --git a/arch/parisc/include/asm/dma-mapping.h b/arch/parisc/include/asm/dma-mapping.h
index 1749073e44fc..5404c6a726b2 100644
--- a/arch/parisc/include/asm/dma-mapping.h
+++ b/arch/parisc/include/asm/dma-mapping.h
@@ -27,7 +27,7 @@ extern const struct dma_map_ops pcx_dma_ops;
extern const struct dma_map_ops *hppa_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return hppa_dma_ops;
}
diff --git a/arch/powerpc/include/asm/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h
index 8275603ba4d5..181a095468e4 100644
--- a/arch/powerpc/include/asm/dma-mapping.h
+++ b/arch/powerpc/include/asm/dma-mapping.h
@@ -78,17 +78,14 @@ extern struct dma_map_ops dma_iommu_ops;
#endif
extern const struct dma_map_ops dma_direct_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
/* We don't handle the NULL dev case for ISA for now. We could
* do it via an out of line call but it is not needed for now. The
* only ISA DMA device we support is the floppy and we have a hack
* in the floppy driver directly to get a device for us.
*/
- if (unlikely(dev == NULL))
- return NULL;
-
- return dev->dma_ops;
+ return NULL;
}
/*
diff --git a/arch/powerpc/include/asm/ps3.h b/arch/powerpc/include/asm/ps3.h
index a19f831a4cc9..17ee719e799f 100644
--- a/arch/powerpc/include/asm/ps3.h
+++ b/arch/powerpc/include/asm/ps3.h
@@ -435,7 +435,7 @@ static inline void *ps3_system_bus_get_drvdata(
return dev_get_drvdata(&dev->core);
}
-/* These two need global scope for get_dma_ops(). */
+/* These two need global scope for get_arch_dma_ops(). */
extern struct bus_type ps3_system_bus_type;
diff --git a/arch/s390/include/asm/dma-mapping.h b/arch/s390/include/asm/dma-mapping.h
index a872027d0c1b..3108b8dbe266 100644
--- a/arch/s390/include/asm/dma-mapping.h
+++ b/arch/s390/include/asm/dma-mapping.h
@@ -12,10 +12,8 @@
extern const struct dma_map_ops s390_pci_dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
- if (dev && dev->dma_ops)
- return dev->dma_ops;
return &dma_noop_ops;
}
diff --git a/arch/sh/include/asm/dma-mapping.h b/arch/sh/include/asm/dma-mapping.h
index a7382c34c241..d99008af5f73 100644
--- a/arch/sh/include/asm/dma-mapping.h
+++ b/arch/sh/include/asm/dma-mapping.h
@@ -4,7 +4,7 @@
extern const struct dma_map_ops *dma_ops;
extern void no_iommu_init(void);
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return dma_ops;
}
diff --git a/arch/sparc/include/asm/dma-mapping.h b/arch/sparc/include/asm/dma-mapping.h
index 3d2babc0c4c6..69cc627779f2 100644
--- a/arch/sparc/include/asm/dma-mapping.h
+++ b/arch/sparc/include/asm/dma-mapping.h
@@ -24,14 +24,14 @@ extern const struct dma_map_ops pci32_dma_ops;
extern struct bus_type pci_bus_type;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
#ifdef CONFIG_SPARC_LEON
if (sparc_cpu_model == sparc_leon)
return leon_dma_ops;
#endif
#if defined(CONFIG_SPARC32) && defined(CONFIG_PCI)
- if (dev->bus == &pci_bus_type)
+ if (bus == &pci_bus_type)
return &pci32_dma_ops;
#endif
return dma_ops;
diff --git a/arch/tile/include/asm/dma-mapping.h b/arch/tile/include/asm/dma-mapping.h
index 2562995a6ac9..bbc71a29b2c6 100644
--- a/arch/tile/include/asm/dma-mapping.h
+++ b/arch/tile/include/asm/dma-mapping.h
@@ -29,12 +29,9 @@ extern const struct dma_map_ops *gx_pci_dma_map_ops;
extern const struct dma_map_ops *gx_legacy_pci_dma_map_ops;
extern const struct dma_map_ops *gx_hybrid_pci_dma_map_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
- if (dev && dev->dma_ops)
- return dev->dma_ops;
- else
- return tile_dma_map_ops;
+ return tile_dma_map_ops;
}
static inline dma_addr_t get_dma_offset(struct device *dev)
diff --git a/arch/unicore32/include/asm/dma-mapping.h b/arch/unicore32/include/asm/dma-mapping.h
index 14d7729c7b73..518ba5848dd6 100644
--- a/arch/unicore32/include/asm/dma-mapping.h
+++ b/arch/unicore32/include/asm/dma-mapping.h
@@ -23,7 +23,7 @@
extern const struct dma_map_ops swiotlb_dma_map_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
return &swiotlb_dma_map_ops;
}
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 94b5b96966cb..08a0838b83fb 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -27,16 +27,9 @@ extern int panic_on_overflow;
extern const struct dma_map_ops *dma_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
-#ifndef CONFIG_X86_DEV_DMA_OPS
return dma_ops;
-#else
- if (unlikely(!dev) || !dev->dma_ops)
- return dma_ops;
- else
- return dev->dma_ops;
-#endif
}
bool arch_dma_alloc_attrs(struct device **dev, gfp_t *gfp);
diff --git a/arch/xtensa/include/asm/dma-mapping.h b/arch/xtensa/include/asm/dma-mapping.h
index 9eecfc3c5dc4..c6140fa8c0be 100644
--- a/arch/xtensa/include/asm/dma-mapping.h
+++ b/arch/xtensa/include/asm/dma-mapping.h
@@ -20,12 +20,9 @@
extern const struct dma_map_ops xtensa_dma_map_ops;
-static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
{
- if (dev && dev->dma_ops)
- return dev->dma_ops;
- else
- return &xtensa_dma_map_ops;
+ return &xtensa_dma_map_ops;
}
void dma_cache_sync(struct device *dev, void *vaddr, size_t size,
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index e97f23e8b2d9..ab8710888ddf 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -164,6 +164,13 @@ int dma_mmap_from_coherent(struct device *dev, struct vm_area_struct *vma,
#ifdef CONFIG_HAS_DMA
#include <asm/dma-mapping.h>
+static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
+{
+ if (dev && dev->dma_ops)
+ return dev->dma_ops;
+ return get_arch_dma_ops(dev ? dev->bus : NULL);
+}
+
static inline void set_dma_ops(struct device *dev,
const struct dma_map_ops *dma_ops)
{
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 05/26] lib/dma-noop: Clarify a comment
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (3 preceding siblings ...)
2017-01-12 19:06 ` [PATCH v2 04/26] treewide: Consolidate get_dma_ops() implementations Bart Van Assche
@ 2017-01-12 19:06 ` Bart Van Assche
2017-01-12 19:06 ` [PATCH v2 06/26] lib/dma-noop: Only build dma_noop_ops for m32r and s390 Bart Van Assche
` (22 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:06 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Christian Borntraeger, Joerg Roedel, Andy Lutomirski,
Michael S . Tsirkin
The next patch in this series will introduce another set of DMA
operations that map 1:1 with memory. Clarify that dma-noop maps
to physical addresses.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Joerg Roedel <jroedel@suse.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Michael S. Tsirkin <mst@redhat.com>
---
lib/dma-noop.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/dma-noop.c b/lib/dma-noop.c
index 65e49dd35b7b..de26c8b68f34 100644
--- a/lib/dma-noop.c
+++ b/lib/dma-noop.c
@@ -1,7 +1,7 @@
/*
* lib/dma-noop.c
*
- * Simple DMA noop-ops that map 1:1 with memory
+ * DMA operations that map to physical addresses without flushing memory.
*/
#include <linux/export.h>
#include <linux/mm.h>
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 06/26] lib/dma-noop: Only build dma_noop_ops for m32r and s390
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (4 preceding siblings ...)
2017-01-12 19:06 ` [PATCH v2 05/26] lib/dma-noop: Clarify a comment Bart Van Assche
@ 2017-01-12 19:06 ` Bart Van Assche
2017-01-13 8:38 ` Christian Borntraeger
2017-01-12 19:06 ` [PATCH v2 07/26] lib/dma-virt: Add dma_virt_ops Bart Van Assche
` (21 subsequent siblings)
27 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:06 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Christian Borntraeger, Joerg Roedel, Andy Lutomirski,
Michael S . Tsirkin, Christoph Hellwig
This was suggested by Christoph Hellwig.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Joerg Roedel <jroedel@suse.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
---
arch/m32r/Kconfig | 1 +
arch/s390/Kconfig | 1 +
lib/Kconfig | 5 +++++
lib/Makefile | 2 +-
4 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/m32r/Kconfig b/arch/m32r/Kconfig
index d227a6988d6b..95474460b367 100644
--- a/arch/m32r/Kconfig
+++ b/arch/m32r/Kconfig
@@ -18,6 +18,7 @@ config M32R
select MODULES_USE_ELF_RELA
select HAVE_DEBUG_STACKOVERFLOW
select CPU_NO_EFFICIENT_FFS
+ select DMA_NOOP_OPS
config SBUS
bool
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index c6722112527d..38ad42358c41 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -137,6 +137,7 @@ config S390
select HAVE_DEBUG_KMEMLEAK
select HAVE_DMA_API_DEBUG
select HAVE_DMA_CONTIGUOUS
+ select DMA_NOOP_OPS
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
select HAVE_EFFICIENT_UNALIGNED_ACCESS
diff --git a/lib/Kconfig b/lib/Kconfig
index 260a80e313b9..b6baf0609e8b 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -395,6 +395,11 @@ config HAS_DMA
depends on !NO_DMA
default y
+config DMA_NOOP_OPS
+ bool
+ depends on HAS_DMA && (!64BIT || ARCH_DMA_ADDR_T_64BIT)
+ default n
+
config CHECK_SIGNATURE
bool
diff --git a/lib/Makefile b/lib/Makefile
index bc4073a8cd08..ada06d5652e8 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -26,7 +26,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
lib-$(CONFIG_MMU) += ioremap.o
lib-$(CONFIG_SMP) += cpumask.o
-lib-$(CONFIG_HAS_DMA) += dma-noop.o
+lib-$(CONFIG_DMA_NOOP_OPS) += dma-noop.o
lib-y += kobject.o klist.o
obj-y += lockref.o
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PATCH v2 06/26] lib/dma-noop: Only build dma_noop_ops for m32r and s390
2017-01-12 19:06 ` [PATCH v2 06/26] lib/dma-noop: Only build dma_noop_ops for m32r and s390 Bart Van Assche
@ 2017-01-13 8:38 ` Christian Borntraeger
0 siblings, 0 replies; 45+ messages in thread
From: Christian Borntraeger @ 2017-01-13 8:38 UTC (permalink / raw)
To: Bart Van Assche, Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Joerg Roedel,
Andy Lutomirski, Michael S . Tsirkin, Christoph Hellwig,
linux-s390
On 01/12/2017 08:06 PM, Bart Van Assche wrote:
> This was suggested by Christoph Hellwig.
>
> Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Cc: Joerg Roedel <jroedel@suse.de>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Christoph Hellwig <hch@lst.de>
Should be ok for s390 (as we do have ARCH_DMA_ADDR_T_64BIT).
> ---
> arch/m32r/Kconfig | 1 +
> arch/s390/Kconfig | 1 +
> lib/Kconfig | 5 +++++
> lib/Makefile | 2 +-
> 4 files changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/arch/m32r/Kconfig b/arch/m32r/Kconfig
> index d227a6988d6b..95474460b367 100644
> --- a/arch/m32r/Kconfig
> +++ b/arch/m32r/Kconfig
> @@ -18,6 +18,7 @@ config M32R
> select MODULES_USE_ELF_RELA
> select HAVE_DEBUG_STACKOVERFLOW
> select CPU_NO_EFFICIENT_FFS
> + select DMA_NOOP_OPS
>
> config SBUS
> bool
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index c6722112527d..38ad42358c41 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -137,6 +137,7 @@ config S390
> select HAVE_DEBUG_KMEMLEAK
> select HAVE_DMA_API_DEBUG
> select HAVE_DMA_CONTIGUOUS
> + select DMA_NOOP_OPS
> select HAVE_DYNAMIC_FTRACE
> select HAVE_DYNAMIC_FTRACE_WITH_REGS
> select HAVE_EFFICIENT_UNALIGNED_ACCESS
> diff --git a/lib/Kconfig b/lib/Kconfig
> index 260a80e313b9..b6baf0609e8b 100644
> --- a/lib/Kconfig
> +++ b/lib/Kconfig
> @@ -395,6 +395,11 @@ config HAS_DMA
> depends on !NO_DMA
> default y
>
> +config DMA_NOOP_OPS
> + bool
> + depends on HAS_DMA && (!64BIT || ARCH_DMA_ADDR_T_64BIT)
> + default n
> +
> config CHECK_SIGNATURE
> bool
>
> diff --git a/lib/Makefile b/lib/Makefile
> index bc4073a8cd08..ada06d5652e8 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -26,7 +26,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
>
> lib-$(CONFIG_MMU) += ioremap.o
> lib-$(CONFIG_SMP) += cpumask.o
> -lib-$(CONFIG_HAS_DMA) += dma-noop.o
> +lib-$(CONFIG_DMA_NOOP_OPS) += dma-noop.o
>
> lib-y += kobject.o klist.o
> obj-y += lockref.o
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* [PATCH v2 07/26] lib/dma-virt: Add dma_virt_ops
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (5 preceding siblings ...)
2017-01-12 19:06 ` [PATCH v2 06/26] lib/dma-noop: Only build dma_noop_ops for m32r and s390 Bart Van Assche
@ 2017-01-12 19:06 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 08/26] IB/hf1: Remove DMA mapping code Bart Van Assche
` (20 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:06 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Christian Borntraeger, Joerg Roedel, Andy Lutomirski,
Michael S . Tsirkin
Several RDMA drivers need to provide a DMA mapping API but use the
CPU to transfer data. Provide DMA mapping operations that are
suitable for these drivers.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Joerg Roedel <jroedel@suse.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Michael S. Tsirkin <mst@redhat.com>
---
include/linux/dma-mapping.h | 1 +
lib/Kconfig | 5 ++++
lib/Makefile | 1 +
lib/dma-virt.c | 73 +++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 80 insertions(+)
create mode 100644 lib/dma-virt.c
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index ab8710888ddf..426c43d4fdbf 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -128,6 +128,7 @@ struct dma_map_ops {
};
extern const struct dma_map_ops dma_noop_ops;
+extern const struct dma_map_ops dma_virt_ops;
#define DMA_BIT_MASK(n) (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
diff --git a/lib/Kconfig b/lib/Kconfig
index b6baf0609e8b..97af23b26f33 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -400,6 +400,11 @@ config DMA_NOOP_OPS
depends on HAS_DMA && (!64BIT || ARCH_DMA_ADDR_T_64BIT)
default n
+config DMA_VIRT_OPS
+ bool
+ depends on HAS_DMA && (!64BIT || ARCH_DMA_ADDR_T_64BIT)
+ default n
+
config CHECK_SIGNATURE
bool
diff --git a/lib/Makefile b/lib/Makefile
index ada06d5652e8..b97e9a814854 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -27,6 +27,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
lib-$(CONFIG_MMU) += ioremap.o
lib-$(CONFIG_SMP) += cpumask.o
lib-$(CONFIG_DMA_NOOP_OPS) += dma-noop.o
+lib-$(CONFIG_DMA_VIRT_OPS) += dma-virt.o
lib-y += kobject.o klist.o
obj-y += lockref.o
diff --git a/lib/dma-virt.c b/lib/dma-virt.c
new file mode 100644
index 000000000000..b3573b6ad5b1
--- /dev/null
+++ b/lib/dma-virt.c
@@ -0,0 +1,73 @@
+/*
+ * lib/dma-virt.c
+ *
+ * DMA operations that map to virtual addresses without flushing memory.
+ */
+#include <linux/export.h>
+#include <linux/mm.h>
+#include <linux/dma-mapping.h>
+#include <linux/scatterlist.h>
+
+static void *dma_virt_alloc(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, gfp_t gfp,
+ unsigned long attrs)
+{
+ void *ret;
+
+ ret = (void *)__get_free_pages(gfp, get_order(size));
+ if (ret)
+ *dma_handle = (uintptr_t)ret;
+ return ret;
+}
+
+static void dma_virt_free(struct device *dev, size_t size,
+ void *cpu_addr, dma_addr_t dma_addr,
+ unsigned long attrs)
+{
+ free_pages((unsigned long)cpu_addr, get_order(size));
+}
+
+static dma_addr_t dma_virt_map_page(struct device *dev, struct page *page,
+ unsigned long offset, size_t size,
+ enum dma_data_direction dir,
+ unsigned long attrs)
+{
+ return (uintptr_t)(page_address(page) + offset);
+}
+
+static int dma_virt_map_sg(struct device *dev, struct scatterlist *sgl, int nents,
+ enum dma_data_direction dir,
+ unsigned long attrs)
+{
+ int i;
+ struct scatterlist *sg;
+
+ for_each_sg(sgl, sg, nents, i) {
+ BUG_ON(!sg_page(sg));
+ sg_dma_address(sg) = (uintptr_t)sg_virt(sg);
+ sg_dma_len(sg) = sg->length;
+ }
+
+ return nents;
+}
+
+static int dma_virt_mapping_error(struct device *dev, dma_addr_t dma_addr)
+{
+ return false;
+}
+
+static int dma_virt_supported(struct device *dev, u64 mask)
+{
+ return true;
+}
+
+const struct dma_map_ops dma_virt_ops = {
+ .alloc = dma_virt_alloc,
+ .free = dma_virt_free,
+ .map_page = dma_virt_map_page,
+ .map_sg = dma_virt_map_sg,
+ .mapping_error = dma_virt_mapping_error,
+ .dma_supported = dma_virt_supported,
+};
+
+EXPORT_SYMBOL(dma_virt_ops);
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 08/26] IB/hf1: Remove DMA mapping code
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (6 preceding siblings ...)
2017-01-12 19:06 ` [PATCH v2 07/26] lib/dma-virt: Add dma_virt_ops Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 09/26] IB/qib: " Bart Van Assche
` (19 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Dennis Dalessandro, Dean Luick
The hfi1 DMA mapping code has never been built in any upstream kernel.
Hence remove it.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Dennis Dalessandro <dennis.dalessandro@intel.com>
Cc: Dean Luick <dean.luick@intel.com>
---
drivers/infiniband/hw/hfi1/dma.c | 183 ---------------------------------------
1 file changed, 183 deletions(-)
delete mode 100644 drivers/infiniband/hw/hfi1/dma.c
diff --git a/drivers/infiniband/hw/hfi1/dma.c b/drivers/infiniband/hw/hfi1/dma.c
deleted file mode 100644
index 7e8dab892848..000000000000
--- a/drivers/infiniband/hw/hfi1/dma.c
+++ /dev/null
@@ -1,183 +0,0 @@
-/*
- * Copyright(c) 2015, 2016 Intel Corporation.
- *
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * BSD LICENSE
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * - Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * - Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * - Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- */
-#include <linux/types.h>
-#include <linux/scatterlist.h>
-
-#include "verbs.h"
-
-#define BAD_DMA_ADDRESS ((u64)0)
-
-/*
- * The following functions implement driver specific replacements
- * for the ib_dma_*() functions.
- *
- * These functions return kernel virtual addresses instead of
- * device bus addresses since the driver uses the CPU to copy
- * data instead of using hardware DMA.
- */
-
-static int hfi1_mapping_error(struct ib_device *dev, u64 dma_addr)
-{
- return dma_addr == BAD_DMA_ADDRESS;
-}
-
-static u64 hfi1_dma_map_single(struct ib_device *dev, void *cpu_addr,
- size_t size, enum dma_data_direction direction)
-{
- if (WARN_ON(!valid_dma_direction(direction)))
- return BAD_DMA_ADDRESS;
-
- return (u64)cpu_addr;
-}
-
-static void hfi1_dma_unmap_single(struct ib_device *dev, u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- /* This is a stub, nothing to be done here */
-}
-
-static u64 hfi1_dma_map_page(struct ib_device *dev, struct page *page,
- unsigned long offset, size_t size,
- enum dma_data_direction direction)
-{
- u64 addr;
-
- if (WARN_ON(!valid_dma_direction(direction)))
- return BAD_DMA_ADDRESS;
-
- if (offset + size > PAGE_SIZE)
- return BAD_DMA_ADDRESS;
-
- addr = (u64)page_address(page);
- if (addr)
- addr += offset;
-
- return addr;
-}
-
-static void hfi1_dma_unmap_page(struct ib_device *dev, u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- /* This is a stub, nothing to be done here */
-}
-
-static int hfi1_map_sg(struct ib_device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction direction)
-{
- struct scatterlist *sg;
- u64 addr;
- int i;
- int ret = nents;
-
- if (WARN_ON(!valid_dma_direction(direction)))
- return BAD_DMA_ADDRESS;
-
- for_each_sg(sgl, sg, nents, i) {
- addr = (u64)page_address(sg_page(sg));
- if (!addr) {
- ret = 0;
- break;
- }
- sg->dma_address = addr + sg->offset;
-#ifdef CONFIG_NEED_SG_DMA_LENGTH
- sg->dma_length = sg->length;
-#endif
- }
- return ret;
-}
-
-static void hfi1_unmap_sg(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction)
-{
- /* This is a stub, nothing to be done here */
-}
-
-static void hfi1_sync_single_for_cpu(struct ib_device *dev, u64 addr,
- size_t size, enum dma_data_direction dir)
-{
-}
-
-static void hfi1_sync_single_for_device(struct ib_device *dev, u64 addr,
- size_t size,
- enum dma_data_direction dir)
-{
-}
-
-static void *hfi1_dma_alloc_coherent(struct ib_device *dev, size_t size,
- u64 *dma_handle, gfp_t flag)
-{
- struct page *p;
- void *addr = NULL;
-
- p = alloc_pages(flag, get_order(size));
- if (p)
- addr = page_address(p);
- if (dma_handle)
- *dma_handle = (u64)addr;
- return addr;
-}
-
-static void hfi1_dma_free_coherent(struct ib_device *dev, size_t size,
- void *cpu_addr, u64 dma_handle)
-{
- free_pages((unsigned long)cpu_addr, get_order(size));
-}
-
-struct ib_dma_mapping_ops hfi1_dma_mapping_ops = {
- .mapping_error = hfi1_mapping_error,
- .map_single = hfi1_dma_map_single,
- .unmap_single = hfi1_dma_unmap_single,
- .map_page = hfi1_dma_map_page,
- .unmap_page = hfi1_dma_unmap_page,
- .map_sg = hfi1_map_sg,
- .unmap_sg = hfi1_unmap_sg,
- .sync_single_for_cpu = hfi1_sync_single_for_cpu,
- .sync_single_for_device = hfi1_sync_single_for_device,
- .alloc_coherent = hfi1_dma_alloc_coherent,
- .free_coherent = hfi1_dma_free_coherent
-};
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 09/26] IB/qib: Remove DMA mapping code
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (7 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 08/26] IB/hf1: Remove DMA mapping code Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 10/26] IB: Use dma_virt_ops instead of duplicating it Bart Van Assche
` (18 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Mike Marciniszyn, Dennis Dalessandro
The qib DMA mapping code is no longer built since commit eb636ac0e49e
("IB/qib: Remove dma.c and use rdmavt version of dma functions"). Hence
remove it.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Mike Marciniszyn <mike.marciniszyn@intel.com>
Cc: Dennis Dalessandro <dennis.dalessandro@intel.com>
---
drivers/infiniband/hw/qib/qib_dma.c | 169 -----------------------------------
drivers/infiniband/hw/qib/qib_keys.c | 5 +-
2 files changed, 1 insertion(+), 173 deletions(-)
delete mode 100644 drivers/infiniband/hw/qib/qib_dma.c
diff --git a/drivers/infiniband/hw/qib/qib_dma.c b/drivers/infiniband/hw/qib/qib_dma.c
deleted file mode 100644
index 59fe092b4b0f..000000000000
--- a/drivers/infiniband/hw/qib/qib_dma.c
+++ /dev/null
@@ -1,169 +0,0 @@
-/*
- * Copyright (c) 2006, 2009, 2010 QLogic, Corporation. All rights reserved.
- *
- * This software is available to you under a choice of one of two
- * licenses. You may choose to be licensed under the terms of the GNU
- * General Public License (GPL) Version 2, available from the file
- * COPYING in the main directory of this source tree, or the
- * OpenIB.org BSD license below:
- *
- * Redistribution and use in source and binary forms, with or
- * without modification, are permitted provided that the following
- * conditions are met:
- *
- * - Redistributions of source code must retain the above
- * copyright notice, this list of conditions and the following
- * disclaimer.
- *
- * - Redistributions in binary form must reproduce the above
- * copyright notice, this list of conditions and the following
- * disclaimer in the documentation and/or other materials
- * provided with the distribution.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- */
-#include <linux/types.h>
-#include <linux/scatterlist.h>
-
-#include "qib_verbs.h"
-
-#define BAD_DMA_ADDRESS ((u64) 0)
-
-/*
- * The following functions implement driver specific replacements
- * for the ib_dma_*() functions.
- *
- * These functions return kernel virtual addresses instead of
- * device bus addresses since the driver uses the CPU to copy
- * data instead of using hardware DMA.
- */
-
-static int qib_mapping_error(struct ib_device *dev, u64 dma_addr)
-{
- return dma_addr == BAD_DMA_ADDRESS;
-}
-
-static u64 qib_dma_map_single(struct ib_device *dev, void *cpu_addr,
- size_t size, enum dma_data_direction direction)
-{
- BUG_ON(!valid_dma_direction(direction));
- return (u64) cpu_addr;
-}
-
-static void qib_dma_unmap_single(struct ib_device *dev, u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- BUG_ON(!valid_dma_direction(direction));
-}
-
-static u64 qib_dma_map_page(struct ib_device *dev, struct page *page,
- unsigned long offset, size_t size,
- enum dma_data_direction direction)
-{
- u64 addr;
-
- BUG_ON(!valid_dma_direction(direction));
-
- if (offset + size > PAGE_SIZE) {
- addr = BAD_DMA_ADDRESS;
- goto done;
- }
-
- addr = (u64) page_address(page);
- if (addr)
- addr += offset;
- /* TODO: handle highmem pages */
-
-done:
- return addr;
-}
-
-static void qib_dma_unmap_page(struct ib_device *dev, u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- BUG_ON(!valid_dma_direction(direction));
-}
-
-static int qib_map_sg(struct ib_device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction direction)
-{
- struct scatterlist *sg;
- u64 addr;
- int i;
- int ret = nents;
-
- BUG_ON(!valid_dma_direction(direction));
-
- for_each_sg(sgl, sg, nents, i) {
- addr = (u64) page_address(sg_page(sg));
- /* TODO: handle highmem pages */
- if (!addr) {
- ret = 0;
- break;
- }
- sg->dma_address = addr + sg->offset;
-#ifdef CONFIG_NEED_SG_DMA_LENGTH
- sg->dma_length = sg->length;
-#endif
- }
- return ret;
-}
-
-static void qib_unmap_sg(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction)
-{
- BUG_ON(!valid_dma_direction(direction));
-}
-
-static void qib_sync_single_for_cpu(struct ib_device *dev, u64 addr,
- size_t size, enum dma_data_direction dir)
-{
-}
-
-static void qib_sync_single_for_device(struct ib_device *dev, u64 addr,
- size_t size,
- enum dma_data_direction dir)
-{
-}
-
-static void *qib_dma_alloc_coherent(struct ib_device *dev, size_t size,
- u64 *dma_handle, gfp_t flag)
-{
- struct page *p;
- void *addr = NULL;
-
- p = alloc_pages(flag, get_order(size));
- if (p)
- addr = page_address(p);
- if (dma_handle)
- *dma_handle = (u64) addr;
- return addr;
-}
-
-static void qib_dma_free_coherent(struct ib_device *dev, size_t size,
- void *cpu_addr, u64 dma_handle)
-{
- free_pages((unsigned long) cpu_addr, get_order(size));
-}
-
-struct ib_dma_mapping_ops qib_dma_mapping_ops = {
- .mapping_error = qib_mapping_error,
- .map_single = qib_dma_map_single,
- .unmap_single = qib_dma_unmap_single,
- .map_page = qib_dma_map_page,
- .unmap_page = qib_dma_unmap_page,
- .map_sg = qib_map_sg,
- .unmap_sg = qib_unmap_sg,
- .sync_single_for_cpu = qib_sync_single_for_cpu,
- .sync_single_for_device = qib_sync_single_for_device,
- .alloc_coherent = qib_dma_alloc_coherent,
- .free_coherent = qib_dma_free_coherent
-};
diff --git a/drivers/infiniband/hw/qib/qib_keys.c b/drivers/infiniband/hw/qib/qib_keys.c
index 2c3c93572c17..8fdf79f8d4e4 100644
--- a/drivers/infiniband/hw/qib/qib_keys.c
+++ b/drivers/infiniband/hw/qib/qib_keys.c
@@ -158,10 +158,7 @@ int qib_rkey_ok(struct rvt_qp *qp, struct rvt_sge *sge,
unsigned n, m;
size_t off;
- /*
- * We use RKEY == zero for kernel virtual addresses
- * (see qib_get_dma_mr and qib_dma.c).
- */
+ /* We use RKEY == zero for kernel virtual addresses */
rcu_read_lock();
if (rkey == 0) {
struct rvt_pd *pd = ibpd_to_rvtpd(qp->ibqp.pd);
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 10/26] IB: Use dma_virt_ops instead of duplicating it
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (8 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 09/26] IB/qib: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 11/26] RDS: IB: Remove an unused structure member Bart Van Assche
` (17 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Christoph Hellwig, Andrew Boyer, Dennis Dalessandro,
Jonathan Toppins, Alex Estrin
Additionally, switch from struct ib_dma_mapping_ops to struct
dma_mapping_ops. Update the comments that refer to the source
files removed by this patch.
This patch eliminates one branch from every ib_dma_map_*() call.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Andrew Boyer <andrew.boyer@dell.com>
Cc: Dennis Dalessandro <dennis.dalessandro@intel.com>
Cc: Jonathan Toppins <jtoppins@redhat.com>
Cc: Alex Estrin <alex.estrin@intel.com>
---
drivers/infiniband/sw/rdmavt/Kconfig | 1 +
drivers/infiniband/sw/rdmavt/Makefile | 2 +-
drivers/infiniband/sw/rdmavt/dma.c | 198 ----------------------------------
drivers/infiniband/sw/rdmavt/dma.h | 53 ---------
drivers/infiniband/sw/rdmavt/mr.c | 8 +-
drivers/infiniband/sw/rdmavt/vt.c | 5 +-
drivers/infiniband/sw/rdmavt/vt.h | 1 -
drivers/infiniband/sw/rxe/Kconfig | 1 +
drivers/infiniband/sw/rxe/Makefile | 1 -
drivers/infiniband/sw/rxe/rxe_dma.c | 183 -------------------------------
drivers/infiniband/sw/rxe/rxe_loc.h | 2 -
drivers/infiniband/sw/rxe/rxe_verbs.c | 3 +-
include/rdma/ib_verbs.h | 117 +++-----------------
13 files changed, 27 insertions(+), 548 deletions(-)
delete mode 100644 drivers/infiniband/sw/rdmavt/dma.c
delete mode 100644 drivers/infiniband/sw/rdmavt/dma.h
delete mode 100644 drivers/infiniband/sw/rxe/rxe_dma.c
diff --git a/drivers/infiniband/sw/rdmavt/Kconfig b/drivers/infiniband/sw/rdmavt/Kconfig
index 1da8d01a6855..fdd001ce13d8 100644
--- a/drivers/infiniband/sw/rdmavt/Kconfig
+++ b/drivers/infiniband/sw/rdmavt/Kconfig
@@ -1,5 +1,6 @@
config INFINIBAND_RDMAVT
tristate "RDMA verbs transport library"
depends on 64BIT
+ select DMA_VIRT_OPS
---help---
This is a common software verbs provider for RDMA networks.
diff --git a/drivers/infiniband/sw/rdmavt/Makefile b/drivers/infiniband/sw/rdmavt/Makefile
index ccaa7992ac97..2a821d2fb569 100644
--- a/drivers/infiniband/sw/rdmavt/Makefile
+++ b/drivers/infiniband/sw/rdmavt/Makefile
@@ -7,7 +7,7 @@
#
obj-$(CONFIG_INFINIBAND_RDMAVT) += rdmavt.o
-rdmavt-y := vt.o ah.o cq.o dma.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o \
+rdmavt-y := vt.o ah.o cq.o mad.o mcast.o mmap.o mr.o pd.o qp.o srq.o \
trace.o
CFLAGS_trace.o = -I$(src)
diff --git a/drivers/infiniband/sw/rdmavt/dma.c b/drivers/infiniband/sw/rdmavt/dma.c
deleted file mode 100644
index f2cefb0d9180..000000000000
--- a/drivers/infiniband/sw/rdmavt/dma.c
+++ /dev/null
@@ -1,198 +0,0 @@
-/*
- * Copyright(c) 2016 Intel Corporation.
- *
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * BSD LICENSE
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * - Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * - Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * - Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- */
-#include <linux/types.h>
-#include <linux/scatterlist.h>
-#include <rdma/ib_verbs.h>
-
-#include "dma.h"
-
-#define BAD_DMA_ADDRESS ((u64)0)
-
-/*
- * The following functions implement driver specific replacements
- * for the ib_dma_*() functions.
- *
- * These functions return kernel virtual addresses instead of
- * device bus addresses since the driver uses the CPU to copy
- * data instead of using hardware DMA.
- */
-
-static int rvt_mapping_error(struct ib_device *dev, u64 dma_addr)
-{
- return dma_addr == BAD_DMA_ADDRESS;
-}
-
-static u64 rvt_dma_map_single(struct ib_device *dev, void *cpu_addr,
- size_t size, enum dma_data_direction direction)
-{
- if (WARN_ON(!valid_dma_direction(direction)))
- return BAD_DMA_ADDRESS;
-
- return (u64)cpu_addr;
-}
-
-static void rvt_dma_unmap_single(struct ib_device *dev, u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- /* This is a stub, nothing to be done here */
-}
-
-static u64 rvt_dma_map_page(struct ib_device *dev, struct page *page,
- unsigned long offset, size_t size,
- enum dma_data_direction direction)
-{
- u64 addr;
-
- if (WARN_ON(!valid_dma_direction(direction)))
- return BAD_DMA_ADDRESS;
-
- addr = (u64)page_address(page);
- if (addr)
- addr += offset;
-
- return addr;
-}
-
-static void rvt_dma_unmap_page(struct ib_device *dev, u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- /* This is a stub, nothing to be done here */
-}
-
-static int rvt_map_sg(struct ib_device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction direction)
-{
- struct scatterlist *sg;
- u64 addr;
- int i;
- int ret = nents;
-
- if (WARN_ON(!valid_dma_direction(direction)))
- return 0;
-
- for_each_sg(sgl, sg, nents, i) {
- addr = (u64)page_address(sg_page(sg));
- if (!addr) {
- ret = 0;
- break;
- }
- sg->dma_address = addr + sg->offset;
-#ifdef CONFIG_NEED_SG_DMA_LENGTH
- sg->dma_length = sg->length;
-#endif
- }
- return ret;
-}
-
-static void rvt_unmap_sg(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction)
-{
- /* This is a stub, nothing to be done here */
-}
-
-static int rvt_map_sg_attrs(struct ib_device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction direction,
- unsigned long attrs)
-{
- return rvt_map_sg(dev, sgl, nents, direction);
-}
-
-static void rvt_unmap_sg_attrs(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction,
- unsigned long attrs)
-{
- return rvt_unmap_sg(dev, sg, nents, direction);
-}
-
-static void rvt_sync_single_for_cpu(struct ib_device *dev, u64 addr,
- size_t size, enum dma_data_direction dir)
-{
-}
-
-static void rvt_sync_single_for_device(struct ib_device *dev, u64 addr,
- size_t size,
- enum dma_data_direction dir)
-{
-}
-
-static void *rvt_dma_alloc_coherent(struct ib_device *dev, size_t size,
- u64 *dma_handle, gfp_t flag)
-{
- struct page *p;
- void *addr = NULL;
-
- p = alloc_pages(flag, get_order(size));
- if (p)
- addr = page_address(p);
- if (dma_handle)
- *dma_handle = (u64)addr;
- return addr;
-}
-
-static void rvt_dma_free_coherent(struct ib_device *dev, size_t size,
- void *cpu_addr, u64 dma_handle)
-{
- free_pages((unsigned long)cpu_addr, get_order(size));
-}
-
-struct ib_dma_mapping_ops rvt_default_dma_mapping_ops = {
- .mapping_error = rvt_mapping_error,
- .map_single = rvt_dma_map_single,
- .unmap_single = rvt_dma_unmap_single,
- .map_page = rvt_dma_map_page,
- .unmap_page = rvt_dma_unmap_page,
- .map_sg = rvt_map_sg,
- .unmap_sg = rvt_unmap_sg,
- .map_sg_attrs = rvt_map_sg_attrs,
- .unmap_sg_attrs = rvt_unmap_sg_attrs,
- .sync_single_for_cpu = rvt_sync_single_for_cpu,
- .sync_single_for_device = rvt_sync_single_for_device,
- .alloc_coherent = rvt_dma_alloc_coherent,
- .free_coherent = rvt_dma_free_coherent
-};
diff --git a/drivers/infiniband/sw/rdmavt/dma.h b/drivers/infiniband/sw/rdmavt/dma.h
deleted file mode 100644
index 979f07e09195..000000000000
--- a/drivers/infiniband/sw/rdmavt/dma.h
+++ /dev/null
@@ -1,53 +0,0 @@
-#ifndef DEF_RDMAVTDMA_H
-#define DEF_RDMAVTDMA_H
-
-/*
- * Copyright(c) 2016 Intel Corporation.
- *
- * This file is provided under a dual BSD/GPLv2 license. When using or
- * redistributing this file, you may do so under either license.
- *
- * GPL LICENSE SUMMARY
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * BSD LICENSE
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- *
- * - Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * - Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in
- * the documentation and/or other materials provided with the
- * distribution.
- * - Neither the name of Intel Corporation nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- */
-
-extern struct ib_dma_mapping_ops rvt_default_dma_mapping_ops;
-
-#endif /* DEF_RDMAVTDMA_H */
diff --git a/drivers/infiniband/sw/rdmavt/mr.c b/drivers/infiniband/sw/rdmavt/mr.c
index 52fd15276ee6..14d0ac6efd08 100644
--- a/drivers/infiniband/sw/rdmavt/mr.c
+++ b/drivers/infiniband/sw/rdmavt/mr.c
@@ -305,8 +305,8 @@ static void __rvt_free_mr(struct rvt_mr *mr)
* @acc: access flags
*
* Return: the memory region on success, otherwise returns an errno.
- * Note that all DMA addresses should be created via the
- * struct ib_dma_mapping_ops functions (see dma.c).
+ * Note that all DMA addresses should be created via the functions in
+ * struct dma_virt_ops.
*/
struct ib_mr *rvt_get_dma_mr(struct ib_pd *pd, int acc)
{
@@ -782,7 +782,7 @@ int rvt_lkey_ok(struct rvt_lkey_table *rkt, struct rvt_pd *pd,
/*
* We use LKEY == zero for kernel virtual addresses
- * (see rvt_get_dma_mr and dma.c).
+ * (see rvt_get_dma_mr() and dma_virt_ops).
*/
rcu_read_lock();
if (sge->lkey == 0) {
@@ -880,7 +880,7 @@ int rvt_rkey_ok(struct rvt_qp *qp, struct rvt_sge *sge,
/*
* We use RKEY == zero for kernel virtual addresses
- * (see rvt_get_dma_mr and dma.c).
+ * (see rvt_get_dma_mr() and dma_virt_ops).
*/
rcu_read_lock();
if (rkey == 0) {
diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c
index d430c2f7cec4..6a81b179f631 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -47,6 +47,7 @@
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/dma-mapping.h>
#include "vt.h"
#include "trace.h"
@@ -777,8 +778,8 @@ int rvt_register_device(struct rvt_dev_info *rdi)
}
/* DMA Operations */
- rdi->ibdev.dma_ops =
- rdi->ibdev.dma_ops ? : &rvt_default_dma_mapping_ops;
+ if (rdi->ibdev.dma_device->dma_ops == NULL)
+ set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
/* Protection Domain */
spin_lock_init(&rdi->n_pds_lock);
diff --git a/drivers/infiniband/sw/rdmavt/vt.h b/drivers/infiniband/sw/rdmavt/vt.h
index 6b01eaa4461b..f363505312be 100644
--- a/drivers/infiniband/sw/rdmavt/vt.h
+++ b/drivers/infiniband/sw/rdmavt/vt.h
@@ -50,7 +50,6 @@
#include <rdma/rdma_vt.h>
#include <linux/pci.h>
-#include "dma.h"
#include "pd.h"
#include "qp.h"
#include "ah.h"
diff --git a/drivers/infiniband/sw/rxe/Kconfig b/drivers/infiniband/sw/rxe/Kconfig
index 1e4e628fe7b0..7d1ac27ed251 100644
--- a/drivers/infiniband/sw/rxe/Kconfig
+++ b/drivers/infiniband/sw/rxe/Kconfig
@@ -2,6 +2,7 @@ config RDMA_RXE
tristate "Software RDMA over Ethernet (RoCE) driver"
depends on INET && PCI && INFINIBAND
depends on NET_UDP_TUNNEL
+ select DMA_VIRT_OPS
---help---
This driver implements the InfiniBand RDMA transport over
the Linux network stack. It enables a system with a
diff --git a/drivers/infiniband/sw/rxe/Makefile b/drivers/infiniband/sw/rxe/Makefile
index 3b3fb9d1c470..ec35ff022a42 100644
--- a/drivers/infiniband/sw/rxe/Makefile
+++ b/drivers/infiniband/sw/rxe/Makefile
@@ -14,7 +14,6 @@ rdma_rxe-y := \
rxe_qp.o \
rxe_cq.o \
rxe_mr.o \
- rxe_dma.o \
rxe_opcode.o \
rxe_mmap.o \
rxe_icrc.o \
diff --git a/drivers/infiniband/sw/rxe/rxe_dma.c b/drivers/infiniband/sw/rxe/rxe_dma.c
deleted file mode 100644
index a0f8af5851ae..000000000000
--- a/drivers/infiniband/sw/rxe/rxe_dma.c
+++ /dev/null
@@ -1,183 +0,0 @@
-/*
- * Copyright (c) 2016 Mellanox Technologies Ltd. All rights reserved.
- * Copyright (c) 2015 System Fabric Works, Inc. All rights reserved.
- *
- * This software is available to you under a choice of one of two
- * licenses. You may choose to be licensed under the terms of the GNU
- * General Public License (GPL) Version 2, available from the file
- * COPYING in the main directory of this source tree, or the
- * OpenIB.org BSD license below:
- *
- * Redistribution and use in source and binary forms, with or
- * without modification, are permitted provided that the following
- * conditions are met:
- *
- * - Redistributions of source code must retain the above
- * copyright notice, this list of conditions and the following
- * disclaimer.
- *
- * - Redistributions in binary form must reproduce the above
- * copyright notice, this list of conditions and the following
- * disclaimer in the documentation and/or other materials
- * provided with the distribution.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- */
-
-#include "rxe.h"
-#include "rxe_loc.h"
-
-#define DMA_BAD_ADDER ((u64)0)
-
-static int rxe_mapping_error(struct ib_device *dev, u64 dma_addr)
-{
- return dma_addr == DMA_BAD_ADDER;
-}
-
-static u64 rxe_dma_map_single(struct ib_device *dev,
- void *cpu_addr, size_t size,
- enum dma_data_direction direction)
-{
- WARN_ON(!valid_dma_direction(direction));
- return (uintptr_t)cpu_addr;
-}
-
-static void rxe_dma_unmap_single(struct ib_device *dev,
- u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- WARN_ON(!valid_dma_direction(direction));
-}
-
-static u64 rxe_dma_map_page(struct ib_device *dev,
- struct page *page,
- unsigned long offset,
- size_t size, enum dma_data_direction direction)
-{
- u64 addr;
-
- WARN_ON(!valid_dma_direction(direction));
-
- if (offset + size > PAGE_SIZE) {
- addr = DMA_BAD_ADDER;
- goto done;
- }
-
- addr = (uintptr_t)page_address(page);
- if (addr)
- addr += offset;
-
-done:
- return addr;
-}
-
-static void rxe_dma_unmap_page(struct ib_device *dev,
- u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- WARN_ON(!valid_dma_direction(direction));
-}
-
-static int rxe_map_sg(struct ib_device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction direction)
-{
- struct scatterlist *sg;
- u64 addr;
- int i;
- int ret = nents;
-
- WARN_ON(!valid_dma_direction(direction));
-
- for_each_sg(sgl, sg, nents, i) {
- addr = (uintptr_t)page_address(sg_page(sg));
- if (!addr) {
- ret = 0;
- break;
- }
- sg->dma_address = addr + sg->offset;
-#ifdef CONFIG_NEED_SG_DMA_LENGTH
- sg->dma_length = sg->length;
-#endif
- }
-
- return ret;
-}
-
-static void rxe_unmap_sg(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction)
-{
- WARN_ON(!valid_dma_direction(direction));
-}
-
-static int rxe_map_sg_attrs(struct ib_device *dev, struct scatterlist *sgl,
- int nents, enum dma_data_direction direction,
- unsigned long attrs)
-{
- return rxe_map_sg(dev, sgl, nents, direction);
-}
-
-static void rxe_unmap_sg_attrs(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction,
- unsigned long attrs)
-{
- rxe_unmap_sg(dev, sg, nents, direction);
-}
-
-static void rxe_sync_single_for_cpu(struct ib_device *dev,
- u64 addr,
- size_t size, enum dma_data_direction dir)
-{
-}
-
-static void rxe_sync_single_for_device(struct ib_device *dev,
- u64 addr,
- size_t size, enum dma_data_direction dir)
-{
-}
-
-static void *rxe_dma_alloc_coherent(struct ib_device *dev, size_t size,
- u64 *dma_handle, gfp_t flag)
-{
- struct page *p;
- void *addr = NULL;
-
- p = alloc_pages(flag, get_order(size));
- if (p)
- addr = page_address(p);
-
- if (dma_handle)
- *dma_handle = (uintptr_t)addr;
-
- return addr;
-}
-
-static void rxe_dma_free_coherent(struct ib_device *dev, size_t size,
- void *cpu_addr, u64 dma_handle)
-{
- free_pages((unsigned long)cpu_addr, get_order(size));
-}
-
-struct ib_dma_mapping_ops rxe_dma_mapping_ops = {
- .mapping_error = rxe_mapping_error,
- .map_single = rxe_dma_map_single,
- .unmap_single = rxe_dma_unmap_single,
- .map_page = rxe_dma_map_page,
- .unmap_page = rxe_dma_unmap_page,
- .map_sg = rxe_map_sg,
- .unmap_sg = rxe_unmap_sg,
- .map_sg_attrs = rxe_map_sg_attrs,
- .unmap_sg_attrs = rxe_unmap_sg_attrs,
- .sync_single_for_cpu = rxe_sync_single_for_cpu,
- .sync_single_for_device = rxe_sync_single_for_device,
- .alloc_coherent = rxe_dma_alloc_coherent,
- .free_coherent = rxe_dma_free_coherent
-};
diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
index efe4c6a35442..267f89d91da7 100644
--- a/drivers/infiniband/sw/rxe/rxe_loc.h
+++ b/drivers/infiniband/sw/rxe/rxe_loc.h
@@ -221,8 +221,6 @@ int rxe_srq_from_attr(struct rxe_dev *rxe, struct rxe_srq *srq,
struct ib_srq_attr *attr, enum ib_srq_attr_mask mask,
struct ib_udata *udata);
-extern struct ib_dma_mapping_ops rxe_dma_mapping_ops;
-
void rxe_release(struct kref *kref);
int rxe_completer(void *arg);
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index beb7021ff18a..b75c19bfcb6b 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -31,6 +31,7 @@
* SOFTWARE.
*/
+#include <linux/dma-mapping.h>
#include "rxe.h"
#include "rxe_loc.h"
#include "rxe_queue.h"
@@ -1237,7 +1238,7 @@ int rxe_register_device(struct rxe_dev *rxe)
dev->dma_device = rxe->ifc_ops->dma_device(rxe);
dev->local_dma_lkey = 0;
dev->node_guid = rxe->ifc_ops->node_guid(rxe);
- dev->dma_ops = &rxe_dma_mapping_ops;
+ set_dma_ops(dev->dma_device, &dma_virt_ops);
dev->uverbs_abi_ver = RXE_UVERBS_ABI_VERSION;
dev->uverbs_cmd_mask = BIT_ULL(IB_USER_VERBS_CMD_GET_CONTEXT)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 958a24d8fae7..89e80eb77e06 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1783,53 +1783,6 @@ struct ib_cache {
u8 *lmc_cache;
};
-struct ib_dma_mapping_ops {
- int (*mapping_error)(struct ib_device *dev,
- u64 dma_addr);
- u64 (*map_single)(struct ib_device *dev,
- void *ptr, size_t size,
- enum dma_data_direction direction);
- void (*unmap_single)(struct ib_device *dev,
- u64 addr, size_t size,
- enum dma_data_direction direction);
- u64 (*map_page)(struct ib_device *dev,
- struct page *page, unsigned long offset,
- size_t size,
- enum dma_data_direction direction);
- void (*unmap_page)(struct ib_device *dev,
- u64 addr, size_t size,
- enum dma_data_direction direction);
- int (*map_sg)(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction);
- void (*unmap_sg)(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction);
- int (*map_sg_attrs)(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction,
- unsigned long attrs);
- void (*unmap_sg_attrs)(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction,
- unsigned long attrs);
- void (*sync_single_for_cpu)(struct ib_device *dev,
- u64 dma_handle,
- size_t size,
- enum dma_data_direction dir);
- void (*sync_single_for_device)(struct ib_device *dev,
- u64 dma_handle,
- size_t size,
- enum dma_data_direction dir);
- void *(*alloc_coherent)(struct ib_device *dev,
- size_t size,
- u64 *dma_handle,
- gfp_t flag);
- void (*free_coherent)(struct ib_device *dev,
- size_t size, void *cpu_addr,
- u64 dma_handle);
-};
-
struct iw_cm_verbs;
struct ib_port_immutable {
@@ -2091,7 +2044,6 @@ struct ib_device {
struct ib_rwq_ind_table_init_attr *init_attr,
struct ib_udata *udata);
int (*destroy_rwq_ind_table)(struct ib_rwq_ind_table *wq_ind_table);
- struct ib_dma_mapping_ops *dma_ops;
struct module *owner;
struct device dev;
@@ -2966,8 +2918,6 @@ static inline int ib_req_ncomp_notif(struct ib_cq *cq, int wc_cnt)
*/
static inline int ib_dma_mapping_error(struct ib_device *dev, u64 dma_addr)
{
- if (dev->dma_ops)
- return dev->dma_ops->mapping_error(dev, dma_addr);
return dma_mapping_error(dev->dma_device, dma_addr);
}
@@ -2982,8 +2932,6 @@ static inline u64 ib_dma_map_single(struct ib_device *dev,
void *cpu_addr, size_t size,
enum dma_data_direction direction)
{
- if (dev->dma_ops)
- return dev->dma_ops->map_single(dev, cpu_addr, size, direction);
return dma_map_single(dev->dma_device, cpu_addr, size, direction);
}
@@ -2998,10 +2946,7 @@ static inline void ib_dma_unmap_single(struct ib_device *dev,
u64 addr, size_t size,
enum dma_data_direction direction)
{
- if (dev->dma_ops)
- dev->dma_ops->unmap_single(dev, addr, size, direction);
- else
- dma_unmap_single(dev->dma_device, addr, size, direction);
+ dma_unmap_single(dev->dma_device, addr, size, direction);
}
static inline u64 ib_dma_map_single_attrs(struct ib_device *dev,
@@ -3036,8 +2981,6 @@ static inline u64 ib_dma_map_page(struct ib_device *dev,
size_t size,
enum dma_data_direction direction)
{
- if (dev->dma_ops)
- return dev->dma_ops->map_page(dev, page, offset, size, direction);
return dma_map_page(dev->dma_device, page, offset, size, direction);
}
@@ -3052,10 +2995,7 @@ static inline void ib_dma_unmap_page(struct ib_device *dev,
u64 addr, size_t size,
enum dma_data_direction direction)
{
- if (dev->dma_ops)
- dev->dma_ops->unmap_page(dev, addr, size, direction);
- else
- dma_unmap_page(dev->dma_device, addr, size, direction);
+ dma_unmap_page(dev->dma_device, addr, size, direction);
}
/**
@@ -3069,8 +3009,6 @@ static inline int ib_dma_map_sg(struct ib_device *dev,
struct scatterlist *sg, int nents,
enum dma_data_direction direction)
{
- if (dev->dma_ops)
- return dev->dma_ops->map_sg(dev, sg, nents, direction);
return dma_map_sg(dev->dma_device, sg, nents, direction);
}
@@ -3085,10 +3023,7 @@ static inline void ib_dma_unmap_sg(struct ib_device *dev,
struct scatterlist *sg, int nents,
enum dma_data_direction direction)
{
- if (dev->dma_ops)
- dev->dma_ops->unmap_sg(dev, sg, nents, direction);
- else
- dma_unmap_sg(dev->dma_device, sg, nents, direction);
+ dma_unmap_sg(dev->dma_device, sg, nents, direction);
}
static inline int ib_dma_map_sg_attrs(struct ib_device *dev,
@@ -3096,12 +3031,8 @@ static inline int ib_dma_map_sg_attrs(struct ib_device *dev,
enum dma_data_direction direction,
unsigned long dma_attrs)
{
- if (dev->dma_ops)
- return dev->dma_ops->map_sg_attrs(dev, sg, nents, direction,
- dma_attrs);
- else
- return dma_map_sg_attrs(dev->dma_device, sg, nents, direction,
- dma_attrs);
+ return dma_map_sg_attrs(dev->dma_device, sg, nents, direction,
+ dma_attrs);
}
static inline void ib_dma_unmap_sg_attrs(struct ib_device *dev,
@@ -3109,12 +3040,7 @@ static inline void ib_dma_unmap_sg_attrs(struct ib_device *dev,
enum dma_data_direction direction,
unsigned long dma_attrs)
{
- if (dev->dma_ops)
- return dev->dma_ops->unmap_sg_attrs(dev, sg, nents, direction,
- dma_attrs);
- else
- dma_unmap_sg_attrs(dev->dma_device, sg, nents, direction,
- dma_attrs);
+ dma_unmap_sg_attrs(dev->dma_device, sg, nents, direction, dma_attrs);
}
/**
* ib_sg_dma_address - Return the DMA address from a scatter/gather entry
@@ -3156,10 +3082,7 @@ static inline void ib_dma_sync_single_for_cpu(struct ib_device *dev,
size_t size,
enum dma_data_direction dir)
{
- if (dev->dma_ops)
- dev->dma_ops->sync_single_for_cpu(dev, addr, size, dir);
- else
- dma_sync_single_for_cpu(dev->dma_device, addr, size, dir);
+ dma_sync_single_for_cpu(dev->dma_device, addr, size, dir);
}
/**
@@ -3174,10 +3097,7 @@ static inline void ib_dma_sync_single_for_device(struct ib_device *dev,
size_t size,
enum dma_data_direction dir)
{
- if (dev->dma_ops)
- dev->dma_ops->sync_single_for_device(dev, addr, size, dir);
- else
- dma_sync_single_for_device(dev->dma_device, addr, size, dir);
+ dma_sync_single_for_device(dev->dma_device, addr, size, dir);
}
/**
@@ -3192,16 +3112,12 @@ static inline void *ib_dma_alloc_coherent(struct ib_device *dev,
u64 *dma_handle,
gfp_t flag)
{
- if (dev->dma_ops)
- return dev->dma_ops->alloc_coherent(dev, size, dma_handle, flag);
- else {
- dma_addr_t handle;
- void *ret;
-
- ret = dma_alloc_coherent(dev->dma_device, size, &handle, flag);
- *dma_handle = handle;
- return ret;
- }
+ dma_addr_t handle;
+ void *ret;
+
+ ret = dma_alloc_coherent(dev->dma_device, size, &handle, flag);
+ *dma_handle = handle;
+ return ret;
}
/**
@@ -3215,10 +3131,7 @@ static inline void ib_dma_free_coherent(struct ib_device *dev,
size_t size, void *cpu_addr,
u64 dma_handle)
{
- if (dev->dma_ops)
- dev->dma_ops->free_coherent(dev, size, cpu_addr, dma_handle);
- else
- dma_free_coherent(dev->dma_device, size, cpu_addr, dma_handle);
+ dma_free_coherent(dev->dma_device, size, cpu_addr, dma_handle);
}
/**
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 11/26] RDS: IB: Remove an unused structure member
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (9 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 10/26] IB: Use dma_virt_ops instead of duplicating it Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 12/26] IB: Convert ib_dma_*_coherent() argument type from u64 into dma_addr_t Bart Van Assche
` (16 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
David S . Miller, netdev, rds-devel
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: linux-rdma@vger.kernel.org
Cc: netdev@vger.kernel.org
Cc: rds-devel@oss.oracle.com
---
net/rds/ib_mr.h | 1 -
1 file changed, 1 deletion(-)
diff --git a/net/rds/ib_mr.h b/net/rds/ib_mr.h
index 1c754f4acbe5..24c086db4511 100644
--- a/net/rds/ib_mr.h
+++ b/net/rds/ib_mr.h
@@ -45,7 +45,6 @@
struct rds_ib_fmr {
struct ib_fmr *fmr;
- u64 *dma;
};
enum rds_ib_fr_state {
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 12/26] IB: Convert ib_dma_*_coherent() argument type from u64 into dma_addr_t
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (10 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 11/26] RDS: IB: Remove an unused structure member Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 13/26] IB/core: Inline ib_dma_map_*() functions Bart Van Assche
` (15 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
David S . Miller, netdev, rds-devel
This patch does not change any functionality.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: linux-rdma@vger.kernel.org
Cc: netdev@vger.kernel.org
Cc: rds-devel@oss.oracle.com
---
include/rdma/ib_verbs.h | 11 +++--------
net/rds/ib.h | 6 +++---
2 files changed, 6 insertions(+), 11 deletions(-)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 89e80eb77e06..de8dfb61d2b6 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -3109,15 +3109,10 @@ static inline void ib_dma_sync_single_for_device(struct ib_device *dev,
*/
static inline void *ib_dma_alloc_coherent(struct ib_device *dev,
size_t size,
- u64 *dma_handle,
+ dma_addr_t *dma_handle,
gfp_t flag)
{
- dma_addr_t handle;
- void *ret;
-
- ret = dma_alloc_coherent(dev->dma_device, size, &handle, flag);
- *dma_handle = handle;
- return ret;
+ return dma_alloc_coherent(dev->dma_device, size, dma_handle, flag);
}
/**
@@ -3129,7 +3124,7 @@ static inline void *ib_dma_alloc_coherent(struct ib_device *dev,
*/
static inline void ib_dma_free_coherent(struct ib_device *dev,
size_t size, void *cpu_addr,
- u64 dma_handle)
+ dma_addr_t dma_handle)
{
dma_free_coherent(dev->dma_device, size, cpu_addr, dma_handle);
}
diff --git a/net/rds/ib.h b/net/rds/ib.h
index 45ac8e8e58f4..d21ca88ab628 100644
--- a/net/rds/ib.h
+++ b/net/rds/ib.h
@@ -134,7 +134,7 @@ struct rds_ib_connection {
struct rds_ib_work_ring i_send_ring;
struct rm_data_op *i_data_op;
struct rds_header *i_send_hdrs;
- u64 i_send_hdrs_dma;
+ dma_addr_t i_send_hdrs_dma;
struct rds_ib_send_work *i_sends;
atomic_t i_signaled_sends;
@@ -144,7 +144,7 @@ struct rds_ib_connection {
struct rds_ib_incoming *i_ibinc;
u32 i_recv_data_rem;
struct rds_header *i_recv_hdrs;
- u64 i_recv_hdrs_dma;
+ dma_addr_t i_recv_hdrs_dma;
struct rds_ib_recv_work *i_recvs;
u64 i_ack_recv; /* last ACK received */
struct rds_ib_refill_cache i_cache_incs;
@@ -161,7 +161,7 @@ struct rds_ib_connection {
struct rds_header *i_ack;
struct ib_send_wr i_ack_wr;
struct ib_sge i_ack_sge;
- u64 i_ack_dma;
+ dma_addr_t i_ack_dma;
unsigned long i_ack_queued;
/* Flow control related information
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 13/26] IB/core: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (11 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 12/26] IB: Convert ib_dma_*_coherent() argument type from u64 into dma_addr_t Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 14/26] IB/mlx4: " Bart Van Assche
` (14 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Most changes in this patch and the subsequent patches, except for the
removal of variables that became superfluous and indentation adjustments,
have been generated as follows:
git grep -lE 'ib_(sg_|)dma_' |
xargs -d\\n \
sed -i -e 's/\([^[:alnum:]_]\)ib_dma_\([^(]*\)(\&\([^,]\+\),/\1dma_\2(\3.dma_device,/g' \
-e 's/\([^[:alnum:]_]\)ib_dma_\([^(]*\)(\([^,]\+\),/\1dma_\2(\3->dma_device,/g' \
-e 's/ib_sg_dma_\(len\|address\)(\([^,]\+\), /sg_dma_\1(/g'
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/infiniband/core/mad.c | 79 ++++++++++++++++++--------------------
drivers/infiniband/core/rw.c | 30 +++++++--------
drivers/infiniband/core/umem.c | 13 +++----
drivers/infiniband/core/umem_odp.c | 12 +++---
4 files changed, 61 insertions(+), 73 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index a009f7132c73..79aeef810c2e 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -1152,23 +1152,21 @@ int ib_send_mad(struct ib_mad_send_wr_private *mad_send_wr)
mad_agent = mad_send_wr->send_buf.mad_agent;
sge = mad_send_wr->sg_list;
- sge[0].addr = ib_dma_map_single(mad_agent->device,
- mad_send_wr->send_buf.mad,
- sge[0].length,
- DMA_TO_DEVICE);
- if (unlikely(ib_dma_mapping_error(mad_agent->device, sge[0].addr)))
+ sge[0].addr = dma_map_single(mad_agent->device->dma_device,
+ mad_send_wr->send_buf.mad,
+ sge[0].length, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(mad_agent->device->dma_device, sge[0].addr)))
return -ENOMEM;
mad_send_wr->header_mapping = sge[0].addr;
- sge[1].addr = ib_dma_map_single(mad_agent->device,
- ib_get_payload(mad_send_wr),
- sge[1].length,
- DMA_TO_DEVICE);
- if (unlikely(ib_dma_mapping_error(mad_agent->device, sge[1].addr))) {
- ib_dma_unmap_single(mad_agent->device,
- mad_send_wr->header_mapping,
- sge[0].length, DMA_TO_DEVICE);
+ sge[1].addr = dma_map_single(mad_agent->device->dma_device,
+ ib_get_payload(mad_send_wr),
+ sge[1].length, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(mad_agent->device->dma_device, sge[1].addr))) {
+ dma_unmap_single(mad_agent->device->dma_device,
+ mad_send_wr->header_mapping,
+ sge[0].length, DMA_TO_DEVICE);
return -ENOMEM;
}
mad_send_wr->payload_mapping = sge[1].addr;
@@ -1189,12 +1187,12 @@ int ib_send_mad(struct ib_mad_send_wr_private *mad_send_wr)
}
spin_unlock_irqrestore(&qp_info->send_queue.lock, flags);
if (ret) {
- ib_dma_unmap_single(mad_agent->device,
- mad_send_wr->header_mapping,
- sge[0].length, DMA_TO_DEVICE);
- ib_dma_unmap_single(mad_agent->device,
- mad_send_wr->payload_mapping,
- sge[1].length, DMA_TO_DEVICE);
+ dma_unmap_single(mad_agent->device->dma_device,
+ mad_send_wr->header_mapping,
+ sge[0].length, DMA_TO_DEVICE);
+ dma_unmap_single(mad_agent->device->dma_device,
+ mad_send_wr->payload_mapping,
+ sge[1].length, DMA_TO_DEVICE);
}
return ret;
}
@@ -2191,10 +2189,8 @@ static void ib_mad_recv_done(struct ib_cq *cq, struct ib_wc *wc)
mad_priv_hdr = container_of(mad_list, struct ib_mad_private_header,
mad_list);
recv = container_of(mad_priv_hdr, struct ib_mad_private, header);
- ib_dma_unmap_single(port_priv->device,
- recv->header.mapping,
- mad_priv_dma_size(recv),
- DMA_FROM_DEVICE);
+ dma_unmap_single(port_priv->device->dma_device, recv->header.mapping,
+ mad_priv_dma_size(recv), DMA_FROM_DEVICE);
/* Setup MAD receive work completion from "normal" work completion */
recv->header.wc = *wc;
@@ -2432,12 +2428,12 @@ static void ib_mad_send_done(struct ib_cq *cq, struct ib_wc *wc)
qp_info = send_queue->qp_info;
retry:
- ib_dma_unmap_single(mad_send_wr->send_buf.mad_agent->device,
- mad_send_wr->header_mapping,
- mad_send_wr->sg_list[0].length, DMA_TO_DEVICE);
- ib_dma_unmap_single(mad_send_wr->send_buf.mad_agent->device,
- mad_send_wr->payload_mapping,
- mad_send_wr->sg_list[1].length, DMA_TO_DEVICE);
+ dma_unmap_single(mad_send_wr->send_buf.mad_agent->device->dma_device,
+ mad_send_wr->header_mapping,
+ mad_send_wr->sg_list[0].length, DMA_TO_DEVICE);
+ dma_unmap_single(mad_send_wr->send_buf.mad_agent->device->dma_device,
+ mad_send_wr->payload_mapping,
+ mad_send_wr->sg_list[1].length, DMA_TO_DEVICE);
queued_send_wr = NULL;
spin_lock_irqsave(&send_queue->lock, flags);
list_del(&mad_list->list);
@@ -2853,11 +2849,11 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info,
}
}
sg_list.length = mad_priv_dma_size(mad_priv);
- sg_list.addr = ib_dma_map_single(qp_info->port_priv->device,
- &mad_priv->grh,
- mad_priv_dma_size(mad_priv),
- DMA_FROM_DEVICE);
- if (unlikely(ib_dma_mapping_error(qp_info->port_priv->device,
+ sg_list.addr = dma_map_single(qp_info->port_priv->device->dma_device,
+ &mad_priv->grh,
+ mad_priv_dma_size(mad_priv),
+ DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(qp_info->port_priv->device->dma_device,
sg_list.addr))) {
ret = -ENOMEM;
break;
@@ -2878,10 +2874,10 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info,
list_del(&mad_priv->header.mad_list.list);
recv_queue->count--;
spin_unlock_irqrestore(&recv_queue->lock, flags);
- ib_dma_unmap_single(qp_info->port_priv->device,
- mad_priv->header.mapping,
- mad_priv_dma_size(mad_priv),
- DMA_FROM_DEVICE);
+ dma_unmap_single(qp_info->port_priv->device->dma_device,
+ mad_priv->header.mapping,
+ mad_priv_dma_size(mad_priv),
+ DMA_FROM_DEVICE);
kfree(mad_priv);
dev_err(&qp_info->port_priv->device->dev,
"ib_post_recv failed: %d\n", ret);
@@ -2917,10 +2913,9 @@ static void cleanup_recv_queue(struct ib_mad_qp_info *qp_info)
/* Remove from posted receive MAD list */
list_del(&mad_list->list);
- ib_dma_unmap_single(qp_info->port_priv->device,
- recv->header.mapping,
- mad_priv_dma_size(recv),
- DMA_FROM_DEVICE);
+ dma_unmap_single(qp_info->port_priv->device->dma_device,
+ recv->header.mapping, mad_priv_dma_size(recv),
+ DMA_FROM_DEVICE);
kfree(recv);
}
diff --git a/drivers/infiniband/core/rw.c b/drivers/infiniband/core/rw.c
index dbfd854c32c9..f8aef874f636 100644
--- a/drivers/infiniband/core/rw.c
+++ b/drivers/infiniband/core/rw.c
@@ -178,7 +178,6 @@ static int rdma_rw_init_map_wrs(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
struct scatterlist *sg, u32 sg_cnt, u32 offset,
u64 remote_addr, u32 rkey, enum dma_data_direction dir)
{
- struct ib_device *dev = qp->pd->device;
u32 max_sge = dir == DMA_TO_DEVICE ? qp->max_write_sge :
qp->max_read_sge;
struct ib_sge *sge;
@@ -208,8 +207,8 @@ static int rdma_rw_init_map_wrs(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
rdma_wr->wr.sg_list = sge;
for (j = 0; j < nr_sge; j++, sg = sg_next(sg)) {
- sge->addr = ib_sg_dma_address(dev, sg) + offset;
- sge->length = ib_sg_dma_len(dev, sg) - offset;
+ sge->addr = sg_dma_address(sg) + offset;
+ sge->length = sg_dma_len(sg) - offset;
sge->lkey = qp->pd->local_dma_lkey;
total_len += sge->length;
@@ -235,14 +234,13 @@ static int rdma_rw_init_single_wr(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
struct scatterlist *sg, u32 offset, u64 remote_addr, u32 rkey,
enum dma_data_direction dir)
{
- struct ib_device *dev = qp->pd->device;
struct ib_rdma_wr *rdma_wr = &ctx->single.wr;
ctx->nr_ops = 1;
ctx->single.sge.lkey = qp->pd->local_dma_lkey;
- ctx->single.sge.addr = ib_sg_dma_address(dev, sg) + offset;
- ctx->single.sge.length = ib_sg_dma_len(dev, sg) - offset;
+ ctx->single.sge.addr = sg_dma_address(sg) + offset;
+ ctx->single.sge.length = sg_dma_len(sg) - offset;
memset(rdma_wr, 0, sizeof(*rdma_wr));
if (dir == DMA_TO_DEVICE)
@@ -280,7 +278,7 @@ int rdma_rw_ctx_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
struct ib_device *dev = qp->pd->device;
int ret;
- ret = ib_dma_map_sg(dev, sg, sg_cnt, dir);
+ ret = dma_map_sg(dev->dma_device, sg, sg_cnt, dir);
if (!ret)
return -ENOMEM;
sg_cnt = ret;
@@ -289,7 +287,7 @@ int rdma_rw_ctx_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
* Skip to the S/G entry that sg_offset falls into:
*/
for (;;) {
- u32 len = ib_sg_dma_len(dev, sg);
+ u32 len = sg_dma_len(sg);
if (sg_offset < len)
break;
@@ -319,7 +317,7 @@ int rdma_rw_ctx_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
return ret;
out_unmap_sg:
- ib_dma_unmap_sg(dev, sg, sg_cnt, dir);
+ dma_unmap_sg(dev->dma_device, sg, sg_cnt, dir);
return ret;
}
EXPORT_SYMBOL(rdma_rw_ctx_init);
@@ -358,12 +356,12 @@ int rdma_rw_ctx_signature_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
return -EINVAL;
}
- ret = ib_dma_map_sg(dev, sg, sg_cnt, dir);
+ ret = dma_map_sg(dev->dma_device, sg, sg_cnt, dir);
if (!ret)
return -ENOMEM;
sg_cnt = ret;
- ret = ib_dma_map_sg(dev, prot_sg, prot_sg_cnt, dir);
+ ret = dma_map_sg(dev->dma_device, prot_sg, prot_sg_cnt, dir);
if (!ret) {
ret = -ENOMEM;
goto out_unmap_sg;
@@ -457,9 +455,9 @@ int rdma_rw_ctx_signature_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
out_free_ctx:
kfree(ctx->sig);
out_unmap_prot_sg:
- ib_dma_unmap_sg(dev, prot_sg, prot_sg_cnt, dir);
+ dma_unmap_sg(dev->dma_device, prot_sg, prot_sg_cnt, dir);
out_unmap_sg:
- ib_dma_unmap_sg(dev, sg, sg_cnt, dir);
+ dma_unmap_sg(dev->dma_device, sg, sg_cnt, dir);
return ret;
}
EXPORT_SYMBOL(rdma_rw_ctx_signature_init);
@@ -606,7 +604,7 @@ void rdma_rw_ctx_destroy(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
break;
}
- ib_dma_unmap_sg(qp->pd->device, sg, sg_cnt, dir);
+ dma_unmap_sg(qp->pd->device->dma_device, sg, sg_cnt, dir);
}
EXPORT_SYMBOL(rdma_rw_ctx_destroy);
@@ -631,11 +629,11 @@ void rdma_rw_ctx_destroy_signature(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
return;
ib_mr_pool_put(qp, &qp->rdma_mrs, ctx->sig->data.mr);
- ib_dma_unmap_sg(qp->pd->device, sg, sg_cnt, dir);
+ dma_unmap_sg(qp->pd->device->dma_device, sg, sg_cnt, dir);
if (ctx->sig->prot.mr) {
ib_mr_pool_put(qp, &qp->rdma_mrs, ctx->sig->prot.mr);
- ib_dma_unmap_sg(qp->pd->device, prot_sg, prot_sg_cnt, dir);
+ dma_unmap_sg(qp->pd->device->dma_device, prot_sg, prot_sg_cnt, dir);
}
ib_mr_pool_put(qp, &qp->sig_mrs, ctx->sig->sig_mr);
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 1e62a5f0cb28..d30803441f68 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -50,9 +50,8 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
int i;
if (umem->nmap > 0)
- ib_dma_unmap_sg(dev, umem->sg_head.sgl,
- umem->npages,
- DMA_BIDIRECTIONAL);
+ dma_unmap_sg(dev->dma_device, umem->sg_head.sgl, umem->npages,
+ DMA_BIDIRECTIONAL);
for_each_sg(umem->sg_head.sgl, sg, umem->npages, i) {
@@ -214,11 +213,9 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
sg_list_start = sg;
}
- umem->nmap = ib_dma_map_sg_attrs(context->device,
- umem->sg_head.sgl,
- umem->npages,
- DMA_BIDIRECTIONAL,
- dma_attrs);
+ umem->nmap = dma_map_sg_attrs(context->device->dma_device,
+ umem->sg_head.sgl, umem->npages,
+ DMA_BIDIRECTIONAL, dma_attrs);
if (umem->nmap <= 0) {
ret = -ENOMEM;
diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c
index 6b079a31dced..8ebaf97bd7be 100644
--- a/drivers/infiniband/core/umem_odp.c
+++ b/drivers/infiniband/core/umem_odp.c
@@ -456,11 +456,9 @@ static int ib_umem_odp_map_dma_single_page(
goto out;
}
if (!(umem->odp_data->dma_list[page_index])) {
- dma_addr = ib_dma_map_page(dev,
- page,
- 0, PAGE_SIZE,
- DMA_BIDIRECTIONAL);
- if (ib_dma_mapping_error(dev, dma_addr)) {
+ dma_addr = dma_map_page(dev->dma_device, page, 0, PAGE_SIZE,
+ DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev->dma_device, dma_addr)) {
ret = -EFAULT;
goto out;
}
@@ -645,8 +643,8 @@ void ib_umem_odp_unmap_dma_pages(struct ib_umem *umem, u64 virt,
WARN_ON(!dma_addr);
- ib_dma_unmap_page(dev, dma_addr, PAGE_SIZE,
- DMA_BIDIRECTIONAL);
+ dma_unmap_page(dev->dma_device, dma_addr, PAGE_SIZE,
+ DMA_BIDIRECTIONAL);
if (dma & ODP_WRITE_ALLOWED_BIT) {
struct page *head_page = compound_head(page);
/*
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 14/26] IB/mlx4: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (12 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 13/26] IB/core: Inline ib_dma_map_*() functions Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 15/26] IB/mlx5: " Bart Van Assche
` (13 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Acked-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/infiniband/hw/mlx4/cq.c | 8 ++--
drivers/infiniband/hw/mlx4/mad.c | 79 ++++++++++++++++++++--------------------
drivers/infiniband/hw/mlx4/mr.c | 8 ++--
drivers/infiniband/hw/mlx4/qp.c | 29 ++++++++-------
4 files changed, 63 insertions(+), 61 deletions(-)
diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index 6a0fec357dae..22b88c10e86d 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -584,10 +584,10 @@ static void use_tunnel_data(struct mlx4_ib_qp *qp, struct mlx4_ib_cq *cq, struct
{
struct mlx4_ib_proxy_sqp_hdr *hdr;
- ib_dma_sync_single_for_cpu(qp->ibqp.device,
- qp->sqp_proxy_rcv[tail].map,
- sizeof (struct mlx4_ib_proxy_sqp_hdr),
- DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(qp->ibqp.device->dma_device,
+ qp->sqp_proxy_rcv[tail].map,
+ sizeof(struct mlx4_ib_proxy_sqp_hdr),
+ DMA_FROM_DEVICE);
hdr = (struct mlx4_ib_proxy_sqp_hdr *) (qp->sqp_proxy_rcv[tail].addr);
wc->pkey_index = be16_to_cpu(hdr->tun.pkey_index);
wc->src_qp = be32_to_cpu(hdr->tun.flags_src_qp) & 0xFFFFFF;
diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c
index db564ccc0f92..b23a29bb29e9 100644
--- a/drivers/infiniband/hw/mlx4/mad.c
+++ b/drivers/infiniband/hw/mlx4/mad.c
@@ -582,10 +582,10 @@ int mlx4_ib_send_to_slave(struct mlx4_ib_dev *dev, int slave, u8 port,
if (tun_qp->tx_ring[tun_tx_ix].ah)
ib_destroy_ah(tun_qp->tx_ring[tun_tx_ix].ah);
tun_qp->tx_ring[tun_tx_ix].ah = ah;
- ib_dma_sync_single_for_cpu(&dev->ib_dev,
- tun_qp->tx_ring[tun_tx_ix].buf.map,
- sizeof (struct mlx4_rcv_tunnel_mad),
- DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(dev->ib_dev.dma_device,
+ tun_qp->tx_ring[tun_tx_ix].buf.map,
+ sizeof(struct mlx4_rcv_tunnel_mad),
+ DMA_TO_DEVICE);
/* copy over to tunnel buffer */
if (grh)
@@ -624,10 +624,10 @@ int mlx4_ib_send_to_slave(struct mlx4_ib_dev *dev, int slave, u8 port,
tun_mad->hdr.slid_mac_47_32 = cpu_to_be16(wc->slid);
}
- ib_dma_sync_single_for_device(&dev->ib_dev,
- tun_qp->tx_ring[tun_tx_ix].buf.map,
- sizeof (struct mlx4_rcv_tunnel_mad),
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(dev->ib_dev.dma_device,
+ tun_qp->tx_ring[tun_tx_ix].buf.map,
+ sizeof(struct mlx4_rcv_tunnel_mad),
+ DMA_TO_DEVICE);
list.addr = tun_qp->tx_ring[tun_tx_ix].buf.map;
list.length = sizeof (struct mlx4_rcv_tunnel_mad);
@@ -1321,8 +1321,8 @@ static int mlx4_ib_post_pv_qp_buf(struct mlx4_ib_demux_pv_ctx *ctx,
recv_wr.num_sge = 1;
recv_wr.wr_id = (u64) index | MLX4_TUN_WRID_RECV |
MLX4_TUN_SET_WRID_QPN(tun_qp->proxy_qpt);
- ib_dma_sync_single_for_device(ctx->ib_dev, tun_qp->ring[index].map,
- size, DMA_FROM_DEVICE);
+ dma_sync_single_for_device(ctx->ib_dev->dma_device, tun_qp->ring[index].map,
+ size, DMA_FROM_DEVICE);
return ib_post_recv(tun_qp->qp, &recv_wr, &bad_recv_wr);
}
@@ -1412,17 +1412,17 @@ int mlx4_ib_send_to_wire(struct mlx4_ib_dev *dev, int slave, u8 port,
if (sqp->tx_ring[wire_tx_ix].ah)
ib_destroy_ah(sqp->tx_ring[wire_tx_ix].ah);
sqp->tx_ring[wire_tx_ix].ah = ah;
- ib_dma_sync_single_for_cpu(&dev->ib_dev,
- sqp->tx_ring[wire_tx_ix].buf.map,
- sizeof (struct mlx4_mad_snd_buf),
- DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(dev->ib_dev.dma_device,
+ sqp->tx_ring[wire_tx_ix].buf.map,
+ sizeof(struct mlx4_mad_snd_buf),
+ DMA_TO_DEVICE);
memcpy(&sqp_mad->payload, mad, sizeof *mad);
- ib_dma_sync_single_for_device(&dev->ib_dev,
- sqp->tx_ring[wire_tx_ix].buf.map,
- sizeof (struct mlx4_mad_snd_buf),
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(dev->ib_dev.dma_device,
+ sqp->tx_ring[wire_tx_ix].buf.map,
+ sizeof(struct mlx4_mad_snd_buf),
+ DMA_TO_DEVICE);
list.addr = sqp->tx_ring[wire_tx_ix].buf.map;
list.length = sizeof (struct mlx4_mad_snd_buf);
@@ -1504,9 +1504,10 @@ static void mlx4_ib_multiplex_mad(struct mlx4_ib_demux_pv_ctx *ctx, struct ib_wc
}
/* Map transaction ID */
- ib_dma_sync_single_for_cpu(ctx->ib_dev, tun_qp->ring[wr_ix].map,
- sizeof (struct mlx4_tunnel_mad),
- DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(ctx->ib_dev->dma_device,
+ tun_qp->ring[wr_ix].map,
+ sizeof(struct mlx4_tunnel_mad),
+ DMA_FROM_DEVICE);
switch (tunnel->mad.mad_hdr.method) {
case IB_MGMT_METHOD_SET:
case IB_MGMT_METHOD_GET:
@@ -1627,11 +1628,11 @@ static int mlx4_ib_alloc_pv_bufs(struct mlx4_ib_demux_pv_ctx *ctx,
tun_qp->ring[i].addr = kmalloc(rx_buf_size, GFP_KERNEL);
if (!tun_qp->ring[i].addr)
goto err;
- tun_qp->ring[i].map = ib_dma_map_single(ctx->ib_dev,
- tun_qp->ring[i].addr,
- rx_buf_size,
- DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(ctx->ib_dev, tun_qp->ring[i].map)) {
+ tun_qp->ring[i].map = dma_map_single(ctx->ib_dev->dma_device,
+ tun_qp->ring[i].addr,
+ rx_buf_size,
+ DMA_FROM_DEVICE);
+ if (dma_mapping_error(ctx->ib_dev->dma_device, tun_qp->ring[i].map)) {
kfree(tun_qp->ring[i].addr);
goto err;
}
@@ -1643,11 +1644,10 @@ static int mlx4_ib_alloc_pv_bufs(struct mlx4_ib_demux_pv_ctx *ctx,
if (!tun_qp->tx_ring[i].buf.addr)
goto tx_err;
tun_qp->tx_ring[i].buf.map =
- ib_dma_map_single(ctx->ib_dev,
- tun_qp->tx_ring[i].buf.addr,
- tx_buf_size,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(ctx->ib_dev,
+ dma_map_single(ctx->ib_dev->dma_device,
+ tun_qp->tx_ring[i].buf.addr,
+ tx_buf_size, DMA_TO_DEVICE);
+ if (dma_mapping_error(ctx->ib_dev->dma_device,
tun_qp->tx_ring[i].buf.map)) {
kfree(tun_qp->tx_ring[i].buf.addr);
goto tx_err;
@@ -1664,8 +1664,9 @@ static int mlx4_ib_alloc_pv_bufs(struct mlx4_ib_demux_pv_ctx *ctx,
tx_err:
while (i > 0) {
--i;
- ib_dma_unmap_single(ctx->ib_dev, tun_qp->tx_ring[i].buf.map,
- tx_buf_size, DMA_TO_DEVICE);
+ dma_unmap_single(ctx->ib_dev->dma_device,
+ tun_qp->tx_ring[i].buf.map,
+ tx_buf_size, DMA_TO_DEVICE);
kfree(tun_qp->tx_ring[i].buf.addr);
}
kfree(tun_qp->tx_ring);
@@ -1674,8 +1675,8 @@ static int mlx4_ib_alloc_pv_bufs(struct mlx4_ib_demux_pv_ctx *ctx,
err:
while (i > 0) {
--i;
- ib_dma_unmap_single(ctx->ib_dev, tun_qp->ring[i].map,
- rx_buf_size, DMA_FROM_DEVICE);
+ dma_unmap_single(ctx->ib_dev->dma_device, tun_qp->ring[i].map,
+ rx_buf_size, DMA_FROM_DEVICE);
kfree(tun_qp->ring[i].addr);
}
kfree(tun_qp->ring);
@@ -1704,14 +1705,14 @@ static void mlx4_ib_free_pv_qp_bufs(struct mlx4_ib_demux_pv_ctx *ctx,
for (i = 0; i < MLX4_NUM_TUNNEL_BUFS; i++) {
- ib_dma_unmap_single(ctx->ib_dev, tun_qp->ring[i].map,
- rx_buf_size, DMA_FROM_DEVICE);
+ dma_unmap_single(ctx->ib_dev->dma_device, tun_qp->ring[i].map,
+ rx_buf_size, DMA_FROM_DEVICE);
kfree(tun_qp->ring[i].addr);
}
for (i = 0; i < MLX4_NUM_TUNNEL_BUFS; i++) {
- ib_dma_unmap_single(ctx->ib_dev, tun_qp->tx_ring[i].buf.map,
- tx_buf_size, DMA_TO_DEVICE);
+ dma_unmap_single(ctx->ib_dev->dma_device, tun_qp->tx_ring[i].buf.map,
+ tx_buf_size, DMA_TO_DEVICE);
kfree(tun_qp->tx_ring[i].buf.addr);
if (tun_qp->tx_ring[i].ah)
ib_destroy_ah(tun_qp->tx_ring[i].ah);
diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c
index 5d73989d9771..00d08e6b3b09 100644
--- a/drivers/infiniband/hw/mlx4/mr.c
+++ b/drivers/infiniband/hw/mlx4/mr.c
@@ -538,13 +538,13 @@ int mlx4_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
mr->npages = 0;
- ib_dma_sync_single_for_cpu(ibmr->device, mr->page_map,
- mr->page_map_size, DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(ibmr->device->dma_device, mr->page_map,
+ mr->page_map_size, DMA_TO_DEVICE);
rc = ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, mlx4_set_page);
- ib_dma_sync_single_for_device(ibmr->device, mr->page_map,
- mr->page_map_size, DMA_TO_DEVICE);
+ dma_sync_single_for_device(ibmr->device->dma_device, mr->page_map,
+ mr->page_map_size, DMA_TO_DEVICE);
return rc;
}
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index c068add8838b..a8f4e4d37e4f 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -570,10 +570,11 @@ static int alloc_proxy_bufs(struct ib_device *dev, struct mlx4_ib_qp *qp)
if (!qp->sqp_proxy_rcv[i].addr)
goto err;
qp->sqp_proxy_rcv[i].map =
- ib_dma_map_single(dev, qp->sqp_proxy_rcv[i].addr,
- sizeof (struct mlx4_ib_proxy_sqp_hdr),
- DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(dev, qp->sqp_proxy_rcv[i].map)) {
+ dma_map_single(dev->dma_device,
+ qp->sqp_proxy_rcv[i].addr,
+ sizeof (struct mlx4_ib_proxy_sqp_hdr),
+ DMA_FROM_DEVICE);
+ if (dma_mapping_error(dev->dma_device, qp->sqp_proxy_rcv[i].map)) {
kfree(qp->sqp_proxy_rcv[i].addr);
goto err;
}
@@ -583,9 +584,9 @@ static int alloc_proxy_bufs(struct ib_device *dev, struct mlx4_ib_qp *qp)
err:
while (i > 0) {
--i;
- ib_dma_unmap_single(dev, qp->sqp_proxy_rcv[i].map,
- sizeof (struct mlx4_ib_proxy_sqp_hdr),
- DMA_FROM_DEVICE);
+ dma_unmap_single(dev->dma_device, qp->sqp_proxy_rcv[i].map,
+ sizeof (struct mlx4_ib_proxy_sqp_hdr),
+ DMA_FROM_DEVICE);
kfree(qp->sqp_proxy_rcv[i].addr);
}
kfree(qp->sqp_proxy_rcv);
@@ -598,9 +599,9 @@ static void free_proxy_bufs(struct ib_device *dev, struct mlx4_ib_qp *qp)
int i;
for (i = 0; i < qp->rq.wqe_cnt; i++) {
- ib_dma_unmap_single(dev, qp->sqp_proxy_rcv[i].map,
- sizeof (struct mlx4_ib_proxy_sqp_hdr),
- DMA_FROM_DEVICE);
+ dma_unmap_single(dev->dma_device, qp->sqp_proxy_rcv[i].map,
+ sizeof (struct mlx4_ib_proxy_sqp_hdr),
+ DMA_FROM_DEVICE);
kfree(qp->sqp_proxy_rcv[i].addr);
}
kfree(qp->sqp_proxy_rcv);
@@ -3306,10 +3307,10 @@ int mlx4_ib_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr,
if (qp->mlx4_ib_qp_type & (MLX4_IB_QPT_PROXY_SMI_OWNER |
MLX4_IB_QPT_PROXY_SMI | MLX4_IB_QPT_PROXY_GSI)) {
- ib_dma_sync_single_for_device(ibqp->device,
- qp->sqp_proxy_rcv[ind].map,
- sizeof (struct mlx4_ib_proxy_sqp_hdr),
- DMA_FROM_DEVICE);
+ dma_sync_single_for_device(ibqp->device->dma_device,
+ qp->sqp_proxy_rcv[ind].map,
+ sizeof(struct mlx4_ib_proxy_sqp_hdr),
+ DMA_FROM_DEVICE);
scat->byte_count =
cpu_to_be32(sizeof (struct mlx4_ib_proxy_sqp_hdr));
/* use dma lkey from upper layer entry */
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 15/26] IB/mlx5: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (13 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 14/26] IB/mlx4: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 16/26] IB/IPoIB: " Bart Van Assche
` (12 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Acked-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/infiniband/hw/mlx5/mr.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 00175191fdc6..4d3daef6d859 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1865,9 +1865,9 @@ int mlx5_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
mr->ndescs = 0;
- ib_dma_sync_single_for_cpu(ibmr->device, mr->desc_map,
- mr->desc_size * mr->max_descs,
- DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(ibmr->device->dma_device, mr->desc_map,
+ mr->desc_size * mr->max_descs,
+ DMA_TO_DEVICE);
if (mr->access_mode == MLX5_MKC_ACCESS_MODE_KLMS)
n = mlx5_ib_sg_to_klms(mr, sg, sg_nents, sg_offset);
@@ -1875,9 +1875,9 @@ int mlx5_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
n = ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset,
mlx5_set_page);
- ib_dma_sync_single_for_device(ibmr->device, mr->desc_map,
- mr->desc_size * mr->max_descs,
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(ibmr->device->dma_device, mr->desc_map,
+ mr->desc_size * mr->max_descs,
+ DMA_TO_DEVICE);
return n;
}
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 16/26] IB/IPoIB: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (14 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 15/26] IB/mlx5: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-13 11:33 ` Yuval Shaia
2017-01-12 19:07 ` [PATCH v2 17/26] IB/iser: " Bart Van Assche
` (11 subsequent siblings)
27 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/infiniband/ulp/ipoib/ipoib_cm.c | 34 +++++++++++++++-----------
drivers/infiniband/ulp/ipoib/ipoib_ib.c | 42 +++++++++++++++++----------------
2 files changed, 42 insertions(+), 34 deletions(-)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index 096c4f6fbd65..6d91593852e9 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -83,10 +83,12 @@ static void ipoib_cm_dma_unmap_rx(struct ipoib_dev_priv *priv, int frags,
{
int i;
- ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_single(priv->ca->dma_device, mapping[0], IPOIB_CM_HEAD_SIZE,
+ DMA_FROM_DEVICE);
for (i = 0; i < frags; ++i)
- ib_dma_unmap_page(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_page(priv->ca->dma_device, mapping[i + 1], PAGE_SIZE,
+ DMA_FROM_DEVICE);
}
static int ipoib_cm_post_receive_srq(struct net_device *dev, int id)
@@ -158,9 +160,9 @@ static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev,
*/
skb_reserve(skb, IPOIB_CM_RX_RESERVE);
- mapping[0] = ib_dma_map_single(priv->ca, skb->data, IPOIB_CM_HEAD_SIZE,
- DMA_FROM_DEVICE);
- if (unlikely(ib_dma_mapping_error(priv->ca, mapping[0]))) {
+ mapping[0] = dma_map_single(priv->ca->dma_device, skb->data,
+ IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(priv->ca->dma_device, mapping[0]))) {
dev_kfree_skb_any(skb);
return NULL;
}
@@ -172,9 +174,9 @@ static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev,
goto partial_error;
skb_fill_page_desc(skb, i, page, 0, PAGE_SIZE);
- mapping[i + 1] = ib_dma_map_page(priv->ca, page,
- 0, PAGE_SIZE, DMA_FROM_DEVICE);
- if (unlikely(ib_dma_mapping_error(priv->ca, mapping[i + 1])))
+ mapping[i + 1] = dma_map_page(priv->ca->dma_device, page,
+ 0, PAGE_SIZE, DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(priv->ca->dma_device, mapping[i + 1])))
goto partial_error;
}
@@ -183,10 +185,12 @@ static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev,
partial_error:
- ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_single(priv->ca->dma_device, mapping[0], IPOIB_CM_HEAD_SIZE,
+ DMA_FROM_DEVICE);
for (; i > 0; --i)
- ib_dma_unmap_page(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_page(priv->ca->dma_device, mapping[i], PAGE_SIZE,
+ DMA_FROM_DEVICE);
dev_kfree_skb_any(skb);
return NULL;
@@ -626,11 +630,13 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
small_skb = dev_alloc_skb(dlen + IPOIB_CM_RX_RESERVE);
if (small_skb) {
skb_reserve(small_skb, IPOIB_CM_RX_RESERVE);
- ib_dma_sync_single_for_cpu(priv->ca, rx_ring[wr_id].mapping[0],
- dlen, DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(priv->ca->dma_device,
+ rx_ring[wr_id].mapping[0],
+ dlen, DMA_FROM_DEVICE);
skb_copy_from_linear_data(skb, small_skb->data, dlen);
- ib_dma_sync_single_for_device(priv->ca, rx_ring[wr_id].mapping[0],
- dlen, DMA_FROM_DEVICE);
+ dma_sync_single_for_device(priv->ca->dma_device,
+ rx_ring[wr_id].mapping[0],
+ dlen, DMA_FROM_DEVICE);
skb_put(small_skb, dlen);
skb = small_skb;
goto copied;
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
index 5038f9d2d753..79204bd966bd 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
@@ -92,9 +92,8 @@ void ipoib_free_ah(struct kref *kref)
static void ipoib_ud_dma_unmap_rx(struct ipoib_dev_priv *priv,
u64 mapping[IPOIB_UD_RX_SG])
{
- ib_dma_unmap_single(priv->ca, mapping[0],
- IPOIB_UD_BUF_SIZE(priv->max_ib_mtu),
- DMA_FROM_DEVICE);
+ dma_unmap_single(priv->ca->dma_device, mapping[0],
+ IPOIB_UD_BUF_SIZE(priv->max_ib_mtu), DMA_FROM_DEVICE);
}
static int ipoib_ib_post_receive(struct net_device *dev, int id)
@@ -139,9 +138,9 @@ static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id)
skb_reserve(skb, sizeof(struct ipoib_pseudo_header));
mapping = priv->rx_ring[id].mapping;
- mapping[0] = ib_dma_map_single(priv->ca, skb->data, buf_size,
- DMA_FROM_DEVICE);
- if (unlikely(ib_dma_mapping_error(priv->ca, mapping[0])))
+ mapping[0] = dma_map_single(priv->ca->dma_device, skb->data, buf_size,
+ DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(priv->ca->dma_device, mapping[0])))
goto error;
priv->rx_ring[id].skb = skb;
@@ -278,9 +277,9 @@ int ipoib_dma_map_tx(struct ib_device *ca, struct ipoib_tx_buf *tx_req)
int off;
if (skb_headlen(skb)) {
- mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb),
- DMA_TO_DEVICE);
- if (unlikely(ib_dma_mapping_error(ca, mapping[0])))
+ mapping[0] = dma_map_single(ca->dma_device, skb->data,
+ skb_headlen(skb), DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(ca->dma_device, mapping[0])))
return -EIO;
off = 1;
@@ -289,11 +288,12 @@ int ipoib_dma_map_tx(struct ib_device *ca, struct ipoib_tx_buf *tx_req)
for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) {
const skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
- mapping[i + off] = ib_dma_map_page(ca,
- skb_frag_page(frag),
- frag->page_offset, skb_frag_size(frag),
- DMA_TO_DEVICE);
- if (unlikely(ib_dma_mapping_error(ca, mapping[i + off])))
+ mapping[i + off] = dma_map_page(ca->dma_device,
+ skb_frag_page(frag),
+ frag->page_offset,
+ skb_frag_size(frag),
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(ca->dma_device, mapping[i + off])))
goto partial_error;
}
return 0;
@@ -302,11 +302,13 @@ int ipoib_dma_map_tx(struct ib_device *ca, struct ipoib_tx_buf *tx_req)
for (; i > 0; --i) {
const skb_frag_t *frag = &skb_shinfo(skb)->frags[i - 1];
- ib_dma_unmap_page(ca, mapping[i - !off], skb_frag_size(frag), DMA_TO_DEVICE);
+ dma_unmap_page(ca->dma_device, mapping[i - !off],
+ skb_frag_size(frag), DMA_TO_DEVICE);
}
if (off)
- ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE);
+ dma_unmap_single(ca->dma_device, mapping[0], skb_headlen(skb),
+ DMA_TO_DEVICE);
return -EIO;
}
@@ -320,8 +322,8 @@ void ipoib_dma_unmap_tx(struct ipoib_dev_priv *priv,
int off;
if (skb_headlen(skb)) {
- ib_dma_unmap_single(priv->ca, mapping[0], skb_headlen(skb),
- DMA_TO_DEVICE);
+ dma_unmap_single(priv->ca->dma_device, mapping[0],
+ skb_headlen(skb), DMA_TO_DEVICE);
off = 1;
} else
off = 0;
@@ -329,8 +331,8 @@ void ipoib_dma_unmap_tx(struct ipoib_dev_priv *priv,
for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) {
const skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
- ib_dma_unmap_page(priv->ca, mapping[i + off],
- skb_frag_size(frag), DMA_TO_DEVICE);
+ dma_unmap_page(priv->ca->dma_device, mapping[i + off],
+ skb_frag_size(frag), DMA_TO_DEVICE);
}
}
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PATCH v2 16/26] IB/IPoIB: Inline ib_dma_map_*() functions
2017-01-12 19:07 ` [PATCH v2 16/26] IB/IPoIB: " Bart Van Assche
@ 2017-01-13 11:33 ` Yuval Shaia
0 siblings, 0 replies; 45+ messages in thread
From: Yuval Shaia @ 2017-01-13 11:33 UTC (permalink / raw)
To: Bart Van Assche
Cc: Doug Ledford, linux-rdma, linux-kernel, Greg Kroah-Hartman
On Thu, Jan 12, 2017 at 11:07:08AM -0800, Bart Van Assche wrote:
> Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
> ---
> drivers/infiniband/ulp/ipoib/ipoib_cm.c | 34 +++++++++++++++-----------
> drivers/infiniband/ulp/ipoib/ipoib_ib.c | 42 +++++++++++++++++----------------
> 2 files changed, 42 insertions(+), 34 deletions(-)
>
> diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> index 096c4f6fbd65..6d91593852e9 100644
> --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
> @@ -83,10 +83,12 @@ static void ipoib_cm_dma_unmap_rx(struct ipoib_dev_priv *priv, int frags,
> {
> int i;
>
> - ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE);
> + dma_unmap_single(priv->ca->dma_device, mapping[0], IPOIB_CM_HEAD_SIZE,
> + DMA_FROM_DEVICE);
>
> for (i = 0; i < frags; ++i)
> - ib_dma_unmap_page(priv->ca, mapping[i + 1], PAGE_SIZE, DMA_FROM_DEVICE);
> + dma_unmap_page(priv->ca->dma_device, mapping[i + 1], PAGE_SIZE,
> + DMA_FROM_DEVICE);
> }
>
> static int ipoib_cm_post_receive_srq(struct net_device *dev, int id)
> @@ -158,9 +160,9 @@ static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev,
> */
> skb_reserve(skb, IPOIB_CM_RX_RESERVE);
>
> - mapping[0] = ib_dma_map_single(priv->ca, skb->data, IPOIB_CM_HEAD_SIZE,
> - DMA_FROM_DEVICE);
> - if (unlikely(ib_dma_mapping_error(priv->ca, mapping[0]))) {
> + mapping[0] = dma_map_single(priv->ca->dma_device, skb->data,
> + IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE);
> + if (unlikely(dma_mapping_error(priv->ca->dma_device, mapping[0]))) {
> dev_kfree_skb_any(skb);
> return NULL;
> }
> @@ -172,9 +174,9 @@ static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev,
> goto partial_error;
> skb_fill_page_desc(skb, i, page, 0, PAGE_SIZE);
>
> - mapping[i + 1] = ib_dma_map_page(priv->ca, page,
> - 0, PAGE_SIZE, DMA_FROM_DEVICE);
> - if (unlikely(ib_dma_mapping_error(priv->ca, mapping[i + 1])))
> + mapping[i + 1] = dma_map_page(priv->ca->dma_device, page,
> + 0, PAGE_SIZE, DMA_FROM_DEVICE);
> + if (unlikely(dma_mapping_error(priv->ca->dma_device, mapping[i + 1])))
> goto partial_error;
> }
>
> @@ -183,10 +185,12 @@ static struct sk_buff *ipoib_cm_alloc_rx_skb(struct net_device *dev,
>
> partial_error:
>
> - ib_dma_unmap_single(priv->ca, mapping[0], IPOIB_CM_HEAD_SIZE, DMA_FROM_DEVICE);
> + dma_unmap_single(priv->ca->dma_device, mapping[0], IPOIB_CM_HEAD_SIZE,
> + DMA_FROM_DEVICE);
>
> for (; i > 0; --i)
> - ib_dma_unmap_page(priv->ca, mapping[i], PAGE_SIZE, DMA_FROM_DEVICE);
> + dma_unmap_page(priv->ca->dma_device, mapping[i], PAGE_SIZE,
> + DMA_FROM_DEVICE);
>
> dev_kfree_skb_any(skb);
> return NULL;
> @@ -626,11 +630,13 @@ void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
> small_skb = dev_alloc_skb(dlen + IPOIB_CM_RX_RESERVE);
> if (small_skb) {
> skb_reserve(small_skb, IPOIB_CM_RX_RESERVE);
> - ib_dma_sync_single_for_cpu(priv->ca, rx_ring[wr_id].mapping[0],
> - dlen, DMA_FROM_DEVICE);
> + dma_sync_single_for_cpu(priv->ca->dma_device,
> + rx_ring[wr_id].mapping[0],
> + dlen, DMA_FROM_DEVICE);
> skb_copy_from_linear_data(skb, small_skb->data, dlen);
> - ib_dma_sync_single_for_device(priv->ca, rx_ring[wr_id].mapping[0],
> - dlen, DMA_FROM_DEVICE);
> + dma_sync_single_for_device(priv->ca->dma_device,
> + rx_ring[wr_id].mapping[0],
> + dlen, DMA_FROM_DEVICE);
> skb_put(small_skb, dlen);
> skb = small_skb;
> goto copied;
> diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> index 5038f9d2d753..79204bd966bd 100644
> --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
> @@ -92,9 +92,8 @@ void ipoib_free_ah(struct kref *kref)
> static void ipoib_ud_dma_unmap_rx(struct ipoib_dev_priv *priv,
> u64 mapping[IPOIB_UD_RX_SG])
> {
> - ib_dma_unmap_single(priv->ca, mapping[0],
> - IPOIB_UD_BUF_SIZE(priv->max_ib_mtu),
> - DMA_FROM_DEVICE);
> + dma_unmap_single(priv->ca->dma_device, mapping[0],
> + IPOIB_UD_BUF_SIZE(priv->max_ib_mtu), DMA_FROM_DEVICE);
> }
>
> static int ipoib_ib_post_receive(struct net_device *dev, int id)
> @@ -139,9 +138,9 @@ static struct sk_buff *ipoib_alloc_rx_skb(struct net_device *dev, int id)
> skb_reserve(skb, sizeof(struct ipoib_pseudo_header));
>
> mapping = priv->rx_ring[id].mapping;
> - mapping[0] = ib_dma_map_single(priv->ca, skb->data, buf_size,
> - DMA_FROM_DEVICE);
> - if (unlikely(ib_dma_mapping_error(priv->ca, mapping[0])))
> + mapping[0] = dma_map_single(priv->ca->dma_device, skb->data, buf_size,
> + DMA_FROM_DEVICE);
> + if (unlikely(dma_mapping_error(priv->ca->dma_device, mapping[0])))
> goto error;
>
> priv->rx_ring[id].skb = skb;
> @@ -278,9 +277,9 @@ int ipoib_dma_map_tx(struct ib_device *ca, struct ipoib_tx_buf *tx_req)
> int off;
>
> if (skb_headlen(skb)) {
> - mapping[0] = ib_dma_map_single(ca, skb->data, skb_headlen(skb),
> - DMA_TO_DEVICE);
> - if (unlikely(ib_dma_mapping_error(ca, mapping[0])))
> + mapping[0] = dma_map_single(ca->dma_device, skb->data,
> + skb_headlen(skb), DMA_TO_DEVICE);
> + if (unlikely(dma_mapping_error(ca->dma_device, mapping[0])))
> return -EIO;
>
> off = 1;
> @@ -289,11 +288,12 @@ int ipoib_dma_map_tx(struct ib_device *ca, struct ipoib_tx_buf *tx_req)
>
> for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) {
> const skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
> - mapping[i + off] = ib_dma_map_page(ca,
> - skb_frag_page(frag),
> - frag->page_offset, skb_frag_size(frag),
> - DMA_TO_DEVICE);
> - if (unlikely(ib_dma_mapping_error(ca, mapping[i + off])))
> + mapping[i + off] = dma_map_page(ca->dma_device,
> + skb_frag_page(frag),
> + frag->page_offset,
> + skb_frag_size(frag),
> + DMA_TO_DEVICE);
> + if (unlikely(dma_mapping_error(ca->dma_device, mapping[i + off])))
> goto partial_error;
> }
> return 0;
> @@ -302,11 +302,13 @@ int ipoib_dma_map_tx(struct ib_device *ca, struct ipoib_tx_buf *tx_req)
> for (; i > 0; --i) {
> const skb_frag_t *frag = &skb_shinfo(skb)->frags[i - 1];
>
> - ib_dma_unmap_page(ca, mapping[i - !off], skb_frag_size(frag), DMA_TO_DEVICE);
> + dma_unmap_page(ca->dma_device, mapping[i - !off],
> + skb_frag_size(frag), DMA_TO_DEVICE);
> }
>
> if (off)
> - ib_dma_unmap_single(ca, mapping[0], skb_headlen(skb), DMA_TO_DEVICE);
> + dma_unmap_single(ca->dma_device, mapping[0], skb_headlen(skb),
> + DMA_TO_DEVICE);
>
> return -EIO;
> }
> @@ -320,8 +322,8 @@ void ipoib_dma_unmap_tx(struct ipoib_dev_priv *priv,
> int off;
>
> if (skb_headlen(skb)) {
> - ib_dma_unmap_single(priv->ca, mapping[0], skb_headlen(skb),
> - DMA_TO_DEVICE);
> + dma_unmap_single(priv->ca->dma_device, mapping[0],
> + skb_headlen(skb), DMA_TO_DEVICE);
> off = 1;
> } else
> off = 0;
> @@ -329,8 +331,8 @@ void ipoib_dma_unmap_tx(struct ipoib_dev_priv *priv,
> for (i = 0; i < skb_shinfo(skb)->nr_frags; ++i) {
> const skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
>
> - ib_dma_unmap_page(priv->ca, mapping[i + off],
> - skb_frag_size(frag), DMA_TO_DEVICE);
> + dma_unmap_page(priv->ca->dma_device, mapping[i + off],
> + skb_frag_size(frag), DMA_TO_DEVICE);
> }
> }
>
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com>
> --
> 2.11.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 45+ messages in thread
* [PATCH v2 17/26] IB/iser: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (15 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 16/26] IB/IPoIB: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-13 21:26 ` Sagi Grimberg
2017-01-12 19:07 ` [PATCH v2 18/26] IB/isert: " Bart Van Assche
` (10 subsequent siblings)
27 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/infiniband/ulp/iser/iscsi_iser.c | 11 ++--
drivers/infiniband/ulp/iser/iser_initiator.c | 88 +++++++++++++++-------------
drivers/infiniband/ulp/iser/iser_memory.c | 13 ++--
drivers/infiniband/ulp/iser/iser_verbs.c | 6 +-
4 files changed, 62 insertions(+), 56 deletions(-)
diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.c b/drivers/infiniband/ulp/iser/iscsi_iser.c
index 9104e6b8cac9..08a6875711b2 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.c
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.c
@@ -198,9 +198,9 @@ iser_initialize_task_headers(struct iscsi_task *task,
goto out;
}
- dma_addr = ib_dma_map_single(device->ib_device, (void *)tx_desc,
- ISER_HEADERS_LEN, DMA_TO_DEVICE);
- if (ib_dma_mapping_error(device->ib_device, dma_addr)) {
+ dma_addr = dma_map_single(device->ib_device->dma_device, tx_desc,
+ ISER_HEADERS_LEN, DMA_TO_DEVICE);
+ if (dma_mapping_error(device->ib_device->dma_device, dma_addr)) {
ret = -ENOMEM;
goto out;
}
@@ -375,8 +375,9 @@ static void iscsi_iser_cleanup_task(struct iscsi_task *task)
return;
if (likely(tx_desc->mapped)) {
- ib_dma_unmap_single(device->ib_device, tx_desc->dma_addr,
- ISER_HEADERS_LEN, DMA_TO_DEVICE);
+ dma_unmap_single(device->ib_device->dma_device,
+ tx_desc->dma_addr, ISER_HEADERS_LEN,
+ DMA_TO_DEVICE);
tx_desc->mapped = false;
}
diff --git a/drivers/infiniband/ulp/iser/iser_initiator.c b/drivers/infiniband/ulp/iser/iser_initiator.c
index 81ae2e30dd12..b3f01bb9a846 100644
--- a/drivers/infiniband/ulp/iser/iser_initiator.c
+++ b/drivers/infiniband/ulp/iser/iser_initiator.c
@@ -164,8 +164,9 @@ static void iser_create_send_desc(struct iser_conn *iser_conn,
{
struct iser_device *device = iser_conn->ib_conn.device;
- ib_dma_sync_single_for_cpu(device->ib_device,
- tx_desc->dma_addr, ISER_HEADERS_LEN, DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(device->ib_device->dma_device,
+ tx_desc->dma_addr, ISER_HEADERS_LEN,
+ DMA_TO_DEVICE);
memset(&tx_desc->iser_header, 0, sizeof(struct iser_ctrl));
tx_desc->iser_header.flags = ISER_VER;
@@ -180,11 +181,11 @@ static void iser_free_login_buf(struct iser_conn *iser_conn)
if (!desc->req)
return;
- ib_dma_unmap_single(device->ib_device, desc->req_dma,
- ISCSI_DEF_MAX_RECV_SEG_LEN, DMA_TO_DEVICE);
+ dma_unmap_single(device->ib_device->dma_device, desc->req_dma,
+ ISCSI_DEF_MAX_RECV_SEG_LEN, DMA_TO_DEVICE);
- ib_dma_unmap_single(device->ib_device, desc->rsp_dma,
- ISER_RX_LOGIN_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_single(device->ib_device->dma_device, desc->rsp_dma,
+ ISER_RX_LOGIN_SIZE, DMA_FROM_DEVICE);
kfree(desc->req);
kfree(desc->rsp);
@@ -203,10 +204,10 @@ static int iser_alloc_login_buf(struct iser_conn *iser_conn)
if (!desc->req)
return -ENOMEM;
- desc->req_dma = ib_dma_map_single(device->ib_device, desc->req,
- ISCSI_DEF_MAX_RECV_SEG_LEN,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(device->ib_device,
+ desc->req_dma = dma_map_single(device->ib_device->dma_device, desc->req,
+ ISCSI_DEF_MAX_RECV_SEG_LEN,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(device->ib_device->dma_device,
desc->req_dma))
goto free_req;
@@ -214,10 +215,9 @@ static int iser_alloc_login_buf(struct iser_conn *iser_conn)
if (!desc->rsp)
goto unmap_req;
- desc->rsp_dma = ib_dma_map_single(device->ib_device, desc->rsp,
- ISER_RX_LOGIN_SIZE,
- DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(device->ib_device,
+ desc->rsp_dma = dma_map_single(device->ib_device->dma_device, desc->rsp,
+ ISER_RX_LOGIN_SIZE, DMA_FROM_DEVICE);
+ if (dma_mapping_error(device->ib_device->dma_device,
desc->rsp_dma))
goto free_rsp;
@@ -226,9 +226,8 @@ static int iser_alloc_login_buf(struct iser_conn *iser_conn)
free_rsp:
kfree(desc->rsp);
unmap_req:
- ib_dma_unmap_single(device->ib_device, desc->req_dma,
- ISCSI_DEF_MAX_RECV_SEG_LEN,
- DMA_TO_DEVICE);
+ dma_unmap_single(device->ib_device->dma_device, desc->req_dma,
+ ISCSI_DEF_MAX_RECV_SEG_LEN, DMA_TO_DEVICE);
free_req:
kfree(desc->req);
@@ -265,9 +264,10 @@ int iser_alloc_rx_descriptors(struct iser_conn *iser_conn,
rx_desc = iser_conn->rx_descs;
for (i = 0; i < iser_conn->qp_max_recv_dtos; i++, rx_desc++) {
- dma_addr = ib_dma_map_single(device->ib_device, (void *)rx_desc,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(device->ib_device, dma_addr))
+ dma_addr = dma_map_single(device->ib_device->dma_device,
+ rx_desc, ISER_RX_PAYLOAD_SIZE,
+ DMA_FROM_DEVICE);
+ if (dma_mapping_error(device->ib_device->dma_device, dma_addr))
goto rx_desc_dma_map_failed;
rx_desc->dma_addr = dma_addr;
@@ -284,8 +284,9 @@ int iser_alloc_rx_descriptors(struct iser_conn *iser_conn,
rx_desc_dma_map_failed:
rx_desc = iser_conn->rx_descs;
for (j = 0; j < i; j++, rx_desc++)
- ib_dma_unmap_single(device->ib_device, rx_desc->dma_addr,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_single(device->ib_device->dma_device,
+ rx_desc->dma_addr, ISER_RX_PAYLOAD_SIZE,
+ DMA_FROM_DEVICE);
kfree(iser_conn->rx_descs);
iser_conn->rx_descs = NULL;
rx_desc_alloc_fail:
@@ -309,8 +310,9 @@ void iser_free_rx_descriptors(struct iser_conn *iser_conn)
rx_desc = iser_conn->rx_descs;
for (i = 0; i < iser_conn->qp_max_recv_dtos; i++, rx_desc++)
- ib_dma_unmap_single(device->ib_device, rx_desc->dma_addr,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_single(device->ib_device->dma_device,
+ rx_desc->dma_addr, ISER_RX_PAYLOAD_SIZE,
+ DMA_FROM_DEVICE);
kfree(iser_conn->rx_descs);
/* make sure we never redo any unmapping */
iser_conn->rx_descs = NULL;
@@ -522,13 +524,15 @@ int iser_send_control(struct iscsi_conn *conn,
goto send_control_error;
}
- ib_dma_sync_single_for_cpu(device->ib_device, desc->req_dma,
- task->data_count, DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(device->ib_device->dma_device,
+ desc->req_dma, task->data_count,
+ DMA_TO_DEVICE);
memcpy(desc->req, task->data, task->data_count);
- ib_dma_sync_single_for_device(device->ib_device, desc->req_dma,
- task->data_count, DMA_TO_DEVICE);
+ dma_sync_single_for_device(device->ib_device->dma_device,
+ desc->req_dma, task->data_count,
+ DMA_TO_DEVICE);
tx_dsg->addr = desc->req_dma;
tx_dsg->length = task->data_count;
@@ -570,9 +574,9 @@ void iser_login_rsp(struct ib_cq *cq, struct ib_wc *wc)
return;
}
- ib_dma_sync_single_for_cpu(ib_conn->device->ib_device,
- desc->rsp_dma, ISER_RX_LOGIN_SIZE,
- DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(ib_conn->device->ib_device->dma_device,
+ desc->rsp_dma, ISER_RX_LOGIN_SIZE,
+ DMA_FROM_DEVICE);
hdr = desc->rsp + sizeof(struct iser_ctrl);
data = desc->rsp + ISER_HEADERS_LEN;
@@ -583,9 +587,9 @@ void iser_login_rsp(struct ib_cq *cq, struct ib_wc *wc)
iscsi_iser_recv(iser_conn->iscsi_conn, hdr, data, length);
- ib_dma_sync_single_for_device(ib_conn->device->ib_device,
- desc->rsp_dma, ISER_RX_LOGIN_SIZE,
- DMA_FROM_DEVICE);
+ dma_sync_single_for_device(ib_conn->device->ib_device->dma_device,
+ desc->rsp_dma, ISER_RX_LOGIN_SIZE,
+ DMA_FROM_DEVICE);
ib_conn->post_recv_buf_count--;
}
@@ -655,9 +659,9 @@ void iser_task_rsp(struct ib_cq *cq, struct ib_wc *wc)
return;
}
- ib_dma_sync_single_for_cpu(ib_conn->device->ib_device,
- desc->dma_addr, ISER_RX_PAYLOAD_SIZE,
- DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(ib_conn->device->ib_device->dma_device,
+ desc->dma_addr, ISER_RX_PAYLOAD_SIZE,
+ DMA_FROM_DEVICE);
hdr = &desc->iscsi_header;
length = wc->byte_len - ISER_HEADERS_LEN;
@@ -673,9 +677,9 @@ void iser_task_rsp(struct ib_cq *cq, struct ib_wc *wc)
iscsi_iser_recv(iser_conn->iscsi_conn, hdr, desc->data, length);
- ib_dma_sync_single_for_device(ib_conn->device->ib_device,
- desc->dma_addr, ISER_RX_PAYLOAD_SIZE,
- DMA_FROM_DEVICE);
+ dma_sync_single_for_device(ib_conn->device->ib_device->dma_device,
+ desc->dma_addr, ISER_RX_PAYLOAD_SIZE,
+ DMA_FROM_DEVICE);
/* decrementing conn->post_recv_buf_count only --after-- freeing the *
* task eliminates the need to worry on tasks which are completed in *
@@ -724,8 +728,8 @@ void iser_dataout_comp(struct ib_cq *cq, struct ib_wc *wc)
if (unlikely(wc->status != IB_WC_SUCCESS))
iser_err_comp(wc, "dataout");
- ib_dma_unmap_single(device->ib_device, desc->dma_addr,
- ISER_HEADERS_LEN, DMA_TO_DEVICE);
+ dma_unmap_single(device->ib_device->dma_device, desc->dma_addr,
+ ISER_HEADERS_LEN, DMA_TO_DEVICE);
kmem_cache_free(ig.desc_cache, desc);
}
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 9c3e9ab53a41..0633bb932216 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -145,9 +145,9 @@ static void iser_data_buf_dump(struct iser_data_buf *data,
for_each_sg(data->sg, sg, data->dma_nents, i)
iser_dbg("sg[%d] dma_addr:0x%lX page:0x%p "
"off:0x%x sz:0x%x dma_len:0x%x\n",
- i, (unsigned long)ib_sg_dma_address(ibdev, sg),
+ i, (unsigned long)sg_dma_address(sg),
sg_page(sg), sg->offset,
- sg->length, ib_sg_dma_len(ibdev, sg));
+ sg->length, sg_dma_len(sg));
}
static void iser_dump_page_vec(struct iser_page_vec *page_vec)
@@ -170,7 +170,8 @@ int iser_dma_map_task_data(struct iscsi_iser_task *iser_task,
iser_task->dir[iser_dir] = 1;
dev = iser_task->iser_conn->ib_conn.device->ib_device;
- data->dma_nents = ib_dma_map_sg(dev, data->sg, data->size, dma_dir);
+ data->dma_nents = dma_map_sg(dev->dma_device, data->sg, data->size,
+ dma_dir);
if (data->dma_nents == 0) {
iser_err("dma_map_sg failed!!!\n");
return -EINVAL;
@@ -185,7 +186,7 @@ void iser_dma_unmap_task_data(struct iscsi_iser_task *iser_task,
struct ib_device *dev;
dev = iser_task->iser_conn->ib_conn.device->ib_device;
- ib_dma_unmap_sg(dev, data->sg, data->size, dir);
+ dma_unmap_sg(dev->dma_device, data->sg, data->size, dir);
}
static int
@@ -204,8 +205,8 @@ iser_reg_dma(struct iser_device *device, struct iser_data_buf *mem,
reg->rkey = device->pd->unsafe_global_rkey;
else
reg->rkey = 0;
- reg->sge.addr = ib_sg_dma_address(device->ib_device, &sg[0]);
- reg->sge.length = ib_sg_dma_len(device->ib_device, &sg[0]);
+ reg->sge.addr = sg_dma_address(&sg[0]);
+ reg->sge.length = sg_dma_len(&sg[0]);
iser_dbg("Single DMA entry: lkey=0x%x, rkey=0x%x, addr=0x%llx,"
" length=0x%x\n", reg->sge.lkey, reg->rkey,
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index 8ae7a3beddb7..1f439ab917ae 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -1077,9 +1077,9 @@ int iser_post_send(struct ib_conn *ib_conn, struct iser_tx_desc *tx_desc,
struct ib_send_wr *bad_wr, *wr = iser_tx_next_wr(tx_desc);
int ib_ret;
- ib_dma_sync_single_for_device(ib_conn->device->ib_device,
- tx_desc->dma_addr, ISER_HEADERS_LEN,
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(ib_conn->device->ib_device->dma_device,
+ tx_desc->dma_addr, ISER_HEADERS_LEN,
+ DMA_TO_DEVICE);
wr->next = NULL;
wr->wr_cqe = &tx_desc->cqe;
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 18/26] IB/isert: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (16 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 17/26] IB/iser: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-13 21:26 ` Sagi Grimberg
2017-01-12 19:07 ` [PATCH v2 19/26] IB/srp: " Bart Van Assche
` (9 subsequent siblings)
27 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/infiniband/ulp/isert/ib_isert.c | 120 +++++++++++++++++---------------
1 file changed, 63 insertions(+), 57 deletions(-)
diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
index 314e95516068..ca919d472bd5 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.c
+++ b/drivers/infiniband/ulp/isert/ib_isert.c
@@ -189,9 +189,10 @@ isert_alloc_rx_descriptors(struct isert_conn *isert_conn)
rx_desc = isert_conn->rx_descs;
for (i = 0; i < ISERT_QP_MAX_RECV_DTOS; i++, rx_desc++) {
- dma_addr = ib_dma_map_single(ib_dev, (void *)rx_desc,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(ib_dev, dma_addr))
+ dma_addr = dma_map_single(ib_dev->dma_device, rx_desc,
+ ISER_RX_PAYLOAD_SIZE,
+ DMA_FROM_DEVICE);
+ if (dma_mapping_error(ib_dev->dma_device, dma_addr))
goto dma_map_fail;
rx_desc->dma_addr = dma_addr;
@@ -208,8 +209,8 @@ isert_alloc_rx_descriptors(struct isert_conn *isert_conn)
dma_map_fail:
rx_desc = isert_conn->rx_descs;
for (j = 0; j < i; j++, rx_desc++) {
- ib_dma_unmap_single(ib_dev, rx_desc->dma_addr,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_single(ib_dev->dma_device, rx_desc->dma_addr,
+ ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
}
kfree(isert_conn->rx_descs);
isert_conn->rx_descs = NULL;
@@ -229,8 +230,8 @@ isert_free_rx_descriptors(struct isert_conn *isert_conn)
rx_desc = isert_conn->rx_descs;
for (i = 0; i < ISERT_QP_MAX_RECV_DTOS; i++, rx_desc++) {
- ib_dma_unmap_single(ib_dev, rx_desc->dma_addr,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_single(ib_dev->dma_device, rx_desc->dma_addr,
+ ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
}
kfree(isert_conn->rx_descs);
@@ -410,13 +411,12 @@ isert_free_login_buf(struct isert_conn *isert_conn)
{
struct ib_device *ib_dev = isert_conn->device->ib_device;
- ib_dma_unmap_single(ib_dev, isert_conn->login_rsp_dma,
- ISER_RX_PAYLOAD_SIZE, DMA_TO_DEVICE);
+ dma_unmap_single(ib_dev->dma_device, isert_conn->login_rsp_dma,
+ ISER_RX_PAYLOAD_SIZE, DMA_TO_DEVICE);
kfree(isert_conn->login_rsp_buf);
- ib_dma_unmap_single(ib_dev, isert_conn->login_req_dma,
- ISER_RX_PAYLOAD_SIZE,
- DMA_FROM_DEVICE);
+ dma_unmap_single(ib_dev->dma_device, isert_conn->login_req_dma,
+ ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
kfree(isert_conn->login_req_buf);
}
@@ -431,10 +431,11 @@ isert_alloc_login_buf(struct isert_conn *isert_conn,
if (!isert_conn->login_req_buf)
return -ENOMEM;
- isert_conn->login_req_dma = ib_dma_map_single(ib_dev,
- isert_conn->login_req_buf,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
- ret = ib_dma_mapping_error(ib_dev, isert_conn->login_req_dma);
+ isert_conn->login_req_dma = dma_map_single(ib_dev->dma_device,
+ isert_conn->login_req_buf,
+ ISER_RX_PAYLOAD_SIZE,
+ DMA_FROM_DEVICE);
+ ret = dma_mapping_error(ib_dev->dma_device, isert_conn->login_req_dma);
if (ret) {
isert_err("login_req_dma mapping error: %d\n", ret);
isert_conn->login_req_dma = 0;
@@ -447,10 +448,11 @@ isert_alloc_login_buf(struct isert_conn *isert_conn,
goto out_unmap_login_req_buf;
}
- isert_conn->login_rsp_dma = ib_dma_map_single(ib_dev,
- isert_conn->login_rsp_buf,
- ISER_RX_PAYLOAD_SIZE, DMA_TO_DEVICE);
- ret = ib_dma_mapping_error(ib_dev, isert_conn->login_rsp_dma);
+ isert_conn->login_rsp_dma = dma_map_single(ib_dev->dma_device,
+ isert_conn->login_rsp_buf,
+ ISER_RX_PAYLOAD_SIZE,
+ DMA_TO_DEVICE);
+ ret = dma_mapping_error(ib_dev->dma_device, isert_conn->login_rsp_dma);
if (ret) {
isert_err("login_rsp_dma mapping error: %d\n", ret);
isert_conn->login_rsp_dma = 0;
@@ -462,8 +464,8 @@ isert_alloc_login_buf(struct isert_conn *isert_conn,
out_free_login_rsp_buf:
kfree(isert_conn->login_rsp_buf);
out_unmap_login_req_buf:
- ib_dma_unmap_single(ib_dev, isert_conn->login_req_dma,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_single(ib_dev->dma_device, isert_conn->login_req_dma,
+ ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
out_free_login_req_buf:
kfree(isert_conn->login_req_buf);
return ret;
@@ -854,8 +856,8 @@ isert_login_post_send(struct isert_conn *isert_conn, struct iser_tx_desc *tx_des
struct ib_send_wr send_wr, *send_wr_failed;
int ret;
- ib_dma_sync_single_for_device(ib_dev, tx_desc->dma_addr,
- ISER_HEADERS_LEN, DMA_TO_DEVICE);
+ dma_sync_single_for_device(ib_dev->dma_device, tx_desc->dma_addr,
+ ISER_HEADERS_LEN, DMA_TO_DEVICE);
tx_desc->tx_cqe.done = isert_login_send_done;
@@ -881,8 +883,8 @@ isert_create_send_desc(struct isert_conn *isert_conn,
struct isert_device *device = isert_conn->device;
struct ib_device *ib_dev = device->ib_device;
- ib_dma_sync_single_for_cpu(ib_dev, tx_desc->dma_addr,
- ISER_HEADERS_LEN, DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(ib_dev->dma_device, tx_desc->dma_addr,
+ ISER_HEADERS_LEN, DMA_TO_DEVICE);
memset(&tx_desc->iser_header, 0, sizeof(struct iser_ctrl));
tx_desc->iser_header.flags = ISCSI_CTRL;
@@ -903,10 +905,10 @@ isert_init_tx_hdrs(struct isert_conn *isert_conn,
struct ib_device *ib_dev = device->ib_device;
u64 dma_addr;
- dma_addr = ib_dma_map_single(ib_dev, (void *)tx_desc,
- ISER_HEADERS_LEN, DMA_TO_DEVICE);
- if (ib_dma_mapping_error(ib_dev, dma_addr)) {
- isert_err("ib_dma_mapping_error() failed\n");
+ dma_addr = dma_map_single(ib_dev->dma_device, (void *)tx_desc,
+ ISER_HEADERS_LEN, DMA_TO_DEVICE);
+ if (dma_mapping_error(ib_dev->dma_device, dma_addr)) {
+ isert_err("dma_mapping_error() failed\n");
return -ENOMEM;
}
@@ -992,13 +994,15 @@ isert_put_login_tx(struct iscsi_conn *conn, struct iscsi_login *login,
if (length > 0) {
struct ib_sge *tx_dsg = &tx_desc->tx_sg[1];
- ib_dma_sync_single_for_cpu(ib_dev, isert_conn->login_rsp_dma,
- length, DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(ib_dev->dma_device,
+ isert_conn->login_rsp_dma,
+ length, DMA_TO_DEVICE);
memcpy(isert_conn->login_rsp_buf, login->rsp_buf, length);
- ib_dma_sync_single_for_device(ib_dev, isert_conn->login_rsp_dma,
- length, DMA_TO_DEVICE);
+ dma_sync_single_for_device(ib_dev->dma_device,
+ isert_conn->login_rsp_dma,
+ length, DMA_TO_DEVICE);
tx_dsg->addr = isert_conn->login_rsp_dma;
tx_dsg->length = length;
@@ -1397,8 +1401,8 @@ isert_recv_done(struct ib_cq *cq, struct ib_wc *wc)
return;
}
- ib_dma_sync_single_for_cpu(ib_dev, rx_desc->dma_addr,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(ib_dev->dma_device, rx_desc->dma_addr,
+ ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
isert_dbg("DMA: 0x%llx, iSCSI opcode: 0x%02x, ITT: 0x%08x, flags: 0x%02x dlen: %d\n",
rx_desc->dma_addr, hdr->opcode, hdr->itt, hdr->flags,
@@ -1432,8 +1436,8 @@ isert_recv_done(struct ib_cq *cq, struct ib_wc *wc)
isert_rx_opcode(isert_conn, rx_desc,
read_stag, read_va, write_stag, write_va);
- ib_dma_sync_single_for_device(ib_dev, rx_desc->dma_addr,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_sync_single_for_device(ib_dev->dma_device, rx_desc->dma_addr,
+ ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
}
static void
@@ -1447,8 +1451,8 @@ isert_login_recv_done(struct ib_cq *cq, struct ib_wc *wc)
return;
}
- ib_dma_sync_single_for_cpu(ib_dev, isert_conn->login_req_dma,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(ib_dev->dma_device, isert_conn->login_req_dma,
+ ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
isert_conn->login_req_len = wc->byte_len - ISER_HEADERS_LEN;
@@ -1463,8 +1467,9 @@ isert_login_recv_done(struct ib_cq *cq, struct ib_wc *wc)
complete(&isert_conn->login_req_comp);
mutex_unlock(&isert_conn->mutex);
- ib_dma_sync_single_for_device(ib_dev, isert_conn->login_req_dma,
- ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
+ dma_sync_single_for_device(ib_dev->dma_device,
+ isert_conn->login_req_dma,
+ ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
}
static void
@@ -1571,8 +1576,8 @@ isert_unmap_tx_desc(struct iser_tx_desc *tx_desc, struct ib_device *ib_dev)
{
if (tx_desc->dma_addr != 0) {
isert_dbg("unmap single for tx_desc->dma_addr\n");
- ib_dma_unmap_single(ib_dev, tx_desc->dma_addr,
- ISER_HEADERS_LEN, DMA_TO_DEVICE);
+ dma_unmap_single(ib_dev->dma_device, tx_desc->dma_addr,
+ ISER_HEADERS_LEN, DMA_TO_DEVICE);
tx_desc->dma_addr = 0;
}
}
@@ -1583,8 +1588,8 @@ isert_completion_put(struct iser_tx_desc *tx_desc, struct isert_cmd *isert_cmd,
{
if (isert_cmd->pdu_buf_dma != 0) {
isert_dbg("unmap single for isert_cmd->pdu_buf_dma\n");
- ib_dma_unmap_single(ib_dev, isert_cmd->pdu_buf_dma,
- isert_cmd->pdu_buf_len, DMA_TO_DEVICE);
+ dma_unmap_single(ib_dev->dma_device, isert_cmd->pdu_buf_dma,
+ isert_cmd->pdu_buf_len, DMA_TO_DEVICE);
isert_cmd->pdu_buf_dma = 0;
}
@@ -1841,10 +1846,10 @@ isert_put_response(struct iscsi_conn *conn, struct iscsi_cmd *cmd)
hton24(hdr->dlength, (u32)cmd->se_cmd.scsi_sense_length);
pdu_len = cmd->se_cmd.scsi_sense_length + padding;
- isert_cmd->pdu_buf_dma = ib_dma_map_single(ib_dev,
- (void *)cmd->sense_buffer, pdu_len,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(ib_dev, isert_cmd->pdu_buf_dma))
+ isert_cmd->pdu_buf_dma = dma_map_single(ib_dev->dma_device,
+ cmd->sense_buffer,
+ pdu_len, DMA_TO_DEVICE);
+ if (dma_mapping_error(ib_dev->dma_device, isert_cmd->pdu_buf_dma))
return -ENOMEM;
isert_cmd->pdu_buf_len = pdu_len;
@@ -1970,10 +1975,10 @@ isert_put_reject(struct iscsi_cmd *cmd, struct iscsi_conn *conn)
isert_init_tx_hdrs(isert_conn, &isert_cmd->tx_desc);
hton24(hdr->dlength, ISCSI_HDR_LEN);
- isert_cmd->pdu_buf_dma = ib_dma_map_single(ib_dev,
- (void *)cmd->buf_ptr, ISCSI_HDR_LEN,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(ib_dev, isert_cmd->pdu_buf_dma))
+ isert_cmd->pdu_buf_dma = dma_map_single(ib_dev->dma_device,
+ cmd->buf_ptr, ISCSI_HDR_LEN,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(ib_dev->dma_device, isert_cmd->pdu_buf_dma))
return -ENOMEM;
isert_cmd->pdu_buf_len = ISCSI_HDR_LEN;
tx_dsg->addr = isert_cmd->pdu_buf_dma;
@@ -2013,9 +2018,10 @@ isert_put_text_rsp(struct iscsi_cmd *cmd, struct iscsi_conn *conn)
struct ib_sge *tx_dsg = &isert_cmd->tx_desc.tx_sg[1];
void *txt_rsp_buf = cmd->buf_ptr;
- isert_cmd->pdu_buf_dma = ib_dma_map_single(ib_dev,
- txt_rsp_buf, txt_rsp_len, DMA_TO_DEVICE);
- if (ib_dma_mapping_error(ib_dev, isert_cmd->pdu_buf_dma))
+ isert_cmd->pdu_buf_dma =
+ dma_map_single(ib_dev->dma_device, txt_rsp_buf,
+ txt_rsp_len, DMA_TO_DEVICE);
+ if (dma_mapping_error(ib_dev->dma_device, isert_cmd->pdu_buf_dma))
return -ENOMEM;
isert_cmd->pdu_buf_len = txt_rsp_len;
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 19/26] IB/srp: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (17 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 18/26] IB/isert: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 20/26] IB/srpt: " Bart Van Assche
` (8 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/infiniband/ulp/srp/ib_srp.c | 80 +++++++++++++++++--------------------
1 file changed, 37 insertions(+), 43 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index cd150c19d0d2..3b411c40a8ab 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -233,9 +233,9 @@ static struct srp_iu *srp_alloc_iu(struct srp_host *host, size_t size,
if (!iu->buf)
goto out_free_iu;
- iu->dma = ib_dma_map_single(host->srp_dev->dev, iu->buf, size,
- direction);
- if (ib_dma_mapping_error(host->srp_dev->dev, iu->dma))
+ iu->dma = dma_map_single(host->srp_dev->dev->dma_device, iu->buf, size,
+ direction);
+ if (dma_mapping_error(host->srp_dev->dev->dma_device, iu->dma))
goto out_free_buf;
iu->size = size;
@@ -256,8 +256,8 @@ static void srp_free_iu(struct srp_host *host, struct srp_iu *iu)
if (!iu)
return;
- ib_dma_unmap_single(host->srp_dev->dev, iu->dma, iu->size,
- iu->direction);
+ dma_unmap_single(host->srp_dev->dev->dma_device, iu->dma, iu->size,
+ iu->direction);
kfree(iu->buf);
kfree(iu);
}
@@ -843,9 +843,8 @@ static void srp_free_req_data(struct srp_target_port *target,
kfree(req->map_page);
}
if (req->indirect_dma_addr) {
- ib_dma_unmap_single(ibdev, req->indirect_dma_addr,
- target->indirect_size,
- DMA_TO_DEVICE);
+ dma_unmap_single(ibdev->dma_device, req->indirect_dma_addr,
+ target->indirect_size, DMA_TO_DEVICE);
}
kfree(req->indirect_desc);
}
@@ -888,10 +887,9 @@ static int srp_alloc_req_data(struct srp_rdma_ch *ch)
if (!req->indirect_desc)
goto out;
- dma_addr = ib_dma_map_single(ibdev, req->indirect_desc,
- target->indirect_size,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(ibdev, dma_addr))
+ dma_addr = dma_map_single(ibdev->dma_device, req->indirect_desc,
+ target->indirect_size, DMA_TO_DEVICE);
+ if (dma_mapping_error(ibdev->dma_device, dma_addr))
goto out;
req->indirect_dma_addr = dma_addr;
@@ -1096,8 +1094,8 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
ib_fmr_pool_unmap(*pfmr);
}
- ib_dma_unmap_sg(ibdev, scsi_sglist(scmnd), scsi_sg_count(scmnd),
- scmnd->sc_data_direction);
+ dma_unmap_sg(ibdev->dma_device, scsi_sglist(scmnd), scsi_sg_count(scmnd),
+ scmnd->sc_data_direction);
}
/**
@@ -1411,8 +1409,7 @@ static int srp_map_finish_fr(struct srp_map_state *state,
*state->fr.next++ = desc;
state->nmdesc++;
- srp_map_desc(state, desc->mr->iova,
- desc->mr->length, desc->mr->rkey);
+ srp_map_desc(state, desc->mr->iova, desc->mr->length, desc->mr->rkey);
err = ib_post_send(ch->qp, &wr.wr, &bad_wr);
if (unlikely(err)) {
@@ -1429,9 +1426,8 @@ static int srp_map_sg_entry(struct srp_map_state *state,
{
struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
- struct ib_device *ibdev = dev->dev;
- dma_addr_t dma_addr = ib_sg_dma_address(ibdev, sg);
- unsigned int dma_len = ib_sg_dma_len(ibdev, sg);
+ dma_addr_t dma_addr = sg_dma_address(sg);
+ unsigned int dma_len = sg_dma_len(sg);
unsigned int len = 0;
int ret;
@@ -1525,13 +1521,11 @@ static int srp_map_sg_dma(struct srp_map_state *state, struct srp_rdma_ch *ch,
int count)
{
struct srp_target_port *target = ch->target;
- struct srp_device *dev = target->srp_host->srp_dev;
struct scatterlist *sg;
int i;
for_each_sg(scat, sg, count, i) {
- srp_map_desc(state, ib_sg_dma_address(dev->dev, sg),
- ib_sg_dma_len(dev->dev, sg),
+ srp_map_desc(state, sg_dma_address(sg), sg_dma_len(sg),
target->pd->unsafe_global_rkey);
}
@@ -1659,7 +1653,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,
dev = target->srp_host->srp_dev;
ibdev = dev->dev;
- count = ib_dma_map_sg(ibdev, scat, nents, scmnd->sc_data_direction);
+ count = dma_map_sg(ibdev->dma_device, scat, nents, scmnd->sc_data_direction);
if (unlikely(count == 0))
return -EIO;
@@ -1691,9 +1685,9 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,
*/
struct srp_direct_buf *buf = (void *) cmd->add_data;
- buf->va = cpu_to_be64(ib_sg_dma_address(ibdev, scat));
+ buf->va = cpu_to_be64(sg_dma_address(scat));
buf->key = cpu_to_be32(pd->unsafe_global_rkey);
- buf->len = cpu_to_be32(ib_sg_dma_len(ibdev, scat));
+ buf->len = cpu_to_be32(sg_dma_len(scat));
req->nmdesc = 0;
/* Debugging help. */
@@ -1707,8 +1701,8 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,
*/
indirect_hdr = (void *) cmd->add_data;
- ib_dma_sync_single_for_cpu(ibdev, req->indirect_dma_addr,
- target->indirect_size, DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(ibdev->dma_device, req->indirect_dma_addr,
+ target->indirect_size, DMA_TO_DEVICE);
memset(&state, 0, sizeof(state));
state.desc = req->indirect_desc;
@@ -1789,8 +1783,8 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,
else
cmd->data_in_desc_cnt = count;
- ib_dma_sync_single_for_device(ibdev, req->indirect_dma_addr, table_len,
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(ibdev->dma_device, req->indirect_dma_addr,
+ table_len, DMA_TO_DEVICE);
map_complete:
/* Debugging help. */
@@ -2084,9 +2078,9 @@ static int srp_response_common(struct srp_rdma_ch *ch, s32 req_delta,
return 1;
}
- ib_dma_sync_single_for_cpu(dev, iu->dma, len, DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(dev->dma_device, iu->dma, len, DMA_TO_DEVICE);
memcpy(iu->buf, rsp, len);
- ib_dma_sync_single_for_device(dev, iu->dma, len, DMA_TO_DEVICE);
+ dma_sync_single_for_device(dev->dma_device, iu->dma, len, DMA_TO_DEVICE);
err = srp_post_send(ch, iu, len);
if (err) {
@@ -2144,8 +2138,8 @@ static void srp_recv_done(struct ib_cq *cq, struct ib_wc *wc)
return;
}
- ib_dma_sync_single_for_cpu(dev, iu->dma, ch->max_ti_iu_len,
- DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(dev->dma_device, iu->dma, ch->max_ti_iu_len,
+ DMA_FROM_DEVICE);
opcode = *(u8 *) iu->buf;
@@ -2181,8 +2175,8 @@ static void srp_recv_done(struct ib_cq *cq, struct ib_wc *wc)
break;
}
- ib_dma_sync_single_for_device(dev, iu->dma, ch->max_ti_iu_len,
- DMA_FROM_DEVICE);
+ dma_sync_single_for_device(dev->dma_device, iu->dma, ch->max_ti_iu_len,
+ DMA_FROM_DEVICE);
res = srp_post_recv(ch, iu);
if (res != 0)
@@ -2267,8 +2261,8 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
req = &ch->req_ring[idx];
dev = target->srp_host->srp_dev->dev;
- ib_dma_sync_single_for_cpu(dev, iu->dma, target->max_iu_len,
- DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(dev->dma_device, iu->dma, target->max_iu_len,
+ DMA_TO_DEVICE);
scmnd->host_scribble = (void *) req;
@@ -2302,8 +2296,8 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
goto err_iu;
}
- ib_dma_sync_single_for_device(dev, iu->dma, target->max_iu_len,
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(dev->dma_device, iu->dma, target->max_iu_len,
+ DMA_TO_DEVICE);
if (srp_post_send(ch, iu, len)) {
shost_printk(KERN_ERR, target->scsi_host, PFX "Send failed\n");
@@ -2689,8 +2683,8 @@ static int srp_send_tsk_mgmt(struct srp_rdma_ch *ch, u64 req_tag, u64 lun,
return -1;
}
- ib_dma_sync_single_for_cpu(dev, iu->dma, sizeof *tsk_mgmt,
- DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(dev->dma_device, iu->dma, sizeof *tsk_mgmt,
+ DMA_TO_DEVICE);
tsk_mgmt = iu->buf;
memset(tsk_mgmt, 0, sizeof *tsk_mgmt);
@@ -2700,8 +2694,8 @@ static int srp_send_tsk_mgmt(struct srp_rdma_ch *ch, u64 req_tag, u64 lun,
tsk_mgmt->tsk_mgmt_func = func;
tsk_mgmt->task_tag = req_tag;
- ib_dma_sync_single_for_device(dev, iu->dma, sizeof *tsk_mgmt,
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(dev->dma_device, iu->dma, sizeof *tsk_mgmt,
+ DMA_TO_DEVICE);
if (srp_post_send(ch, iu, sizeof(*tsk_mgmt))) {
srp_put_tx_iu(ch, iu, SRP_IU_TSK_MGMT);
mutex_unlock(&rport->mutex);
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 20/26] IB/srpt: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (18 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 19/26] IB/srp: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 21/26] staging/lustre: " Bart Van Assche
` (7 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/infiniband/ulp/srpt/ib_srpt.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 7ff9de063c53..df42ae43bce0 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -626,8 +626,8 @@ static struct srpt_ioctx *srpt_alloc_ioctx(struct srpt_device *sdev,
if (!ioctx->buf)
goto err_free_ioctx;
- ioctx->dma = ib_dma_map_single(sdev->device, ioctx->buf, dma_size, dir);
- if (ib_dma_mapping_error(sdev->device, ioctx->dma))
+ ioctx->dma = dma_map_single(sdev->device->dma_device, ioctx->buf, dma_size, dir);
+ if (dma_mapping_error(sdev->device->dma_device, ioctx->dma))
goto err_free_buf;
return ioctx;
@@ -649,7 +649,7 @@ static void srpt_free_ioctx(struct srpt_device *sdev, struct srpt_ioctx *ioctx,
if (!ioctx)
return;
- ib_dma_unmap_single(sdev->device, ioctx->dma, dma_size, dir);
+ dma_unmap_single(sdev->device->dma_device, ioctx->dma, dma_size, dir);
kfree(ioctx->buf);
kfree(ioctx);
}
@@ -1492,9 +1492,9 @@ static void srpt_handle_new_iu(struct srpt_rdma_ch *ch,
BUG_ON(!ch);
BUG_ON(!recv_ioctx);
- ib_dma_sync_single_for_cpu(ch->sport->sdev->device,
- recv_ioctx->ioctx.dma, srp_max_req_size,
- DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(ch->sport->sdev->device->dma_device,
+ recv_ioctx->ioctx.dma, srp_max_req_size,
+ DMA_FROM_DEVICE);
if (unlikely(ch->state == CH_CONNECTING))
goto out_wait;
@@ -2385,8 +2385,8 @@ static void srpt_queue_response(struct se_cmd *cmd)
goto out;
}
- ib_dma_sync_single_for_device(sdev->device, ioctx->ioctx.dma, resp_len,
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(sdev->device->dma_device, ioctx->ioctx.dma,
+ resp_len, DMA_TO_DEVICE);
sge.addr = ioctx->ioctx.dma;
sge.length = resp_len;
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 21/26] staging/lustre: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (19 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 20/26] IB/srpt: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 22/26] nvme-rdma: " Bart Van Assche
` (6 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Oleg Drokin, Andreas Dilger, James Simmons
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Cc: Oleg Drokin <oleg.drokin@intel.com>
Cc: Andreas Dilger <andreas.dilger@intel.com>
Cc: James Simmons <jsimmons@infradead.org>
---
drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
index 14576977200f..f2c0c60eee1d 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
@@ -925,21 +925,21 @@ kiblnd_rd_msg_size(struct kib_rdma_desc *rd, int msgtype, int n)
static inline __u64
kiblnd_dma_mapping_error(struct ib_device *dev, u64 dma_addr)
{
- return ib_dma_mapping_error(dev, dma_addr);
+ return dma_mapping_error(dev->dma_device, dma_addr);
}
static inline __u64 kiblnd_dma_map_single(struct ib_device *dev,
void *msg, size_t size,
enum dma_data_direction direction)
{
- return ib_dma_map_single(dev, msg, size, direction);
+ return dma_map_single(dev->dma_device, msg, size, direction);
}
static inline void kiblnd_dma_unmap_single(struct ib_device *dev,
__u64 addr, size_t size,
enum dma_data_direction direction)
{
- ib_dma_unmap_single(dev, addr, size, direction);
+ dma_unmap_single(dev->dma_device, addr, size, direction);
}
#define KIBLND_UNMAP_ADDR_SET(p, m, a) do {} while (0)
@@ -949,26 +949,26 @@ static inline int kiblnd_dma_map_sg(struct ib_device *dev,
struct scatterlist *sg, int nents,
enum dma_data_direction direction)
{
- return ib_dma_map_sg(dev, sg, nents, direction);
+ return dma_map_sg(dev->dma_device, sg, nents, direction);
}
static inline void kiblnd_dma_unmap_sg(struct ib_device *dev,
struct scatterlist *sg, int nents,
enum dma_data_direction direction)
{
- ib_dma_unmap_sg(dev, sg, nents, direction);
+ dma_unmap_sg(dev->dma_device, sg, nents, direction);
}
static inline __u64 kiblnd_sg_dma_address(struct ib_device *dev,
struct scatterlist *sg)
{
- return ib_sg_dma_address(dev, sg);
+ return sg_dma_address(sg);
}
static inline unsigned int kiblnd_sg_dma_len(struct ib_device *dev,
struct scatterlist *sg)
{
- return ib_sg_dma_len(dev, sg);
+ return sg_dma_len(sg);
}
/* XXX We use KIBLND_CONN_PARAM(e) as writable buffer, it's not strictly */
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 22/26] nvme-rdma: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (20 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 21/26] staging/lustre: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 23/26] net/9p: " Bart Van Assche
` (5 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Keith Busch
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Cc: Keith Busch <keith.busch@intel.com>
---
drivers/nvme/host/rdma.c | 35 +++++++++++++++++++----------------
drivers/nvme/target/rdma.c | 32 ++++++++++++++++----------------
2 files changed, 35 insertions(+), 32 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 208b6a08781c..877ff1982f38 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -207,7 +207,7 @@ static inline size_t nvme_rdma_inline_data_size(struct nvme_rdma_queue *queue)
static void nvme_rdma_free_qe(struct ib_device *ibdev, struct nvme_rdma_qe *qe,
size_t capsule_size, enum dma_data_direction dir)
{
- ib_dma_unmap_single(ibdev, qe->dma, capsule_size, dir);
+ dma_unmap_single(ibdev->dma_device, qe->dma, capsule_size, dir);
kfree(qe->data);
}
@@ -218,8 +218,9 @@ static int nvme_rdma_alloc_qe(struct ib_device *ibdev, struct nvme_rdma_qe *qe,
if (!qe->data)
return -ENOMEM;
- qe->dma = ib_dma_map_single(ibdev, qe->data, capsule_size, dir);
- if (ib_dma_mapping_error(ibdev, qe->dma)) {
+ qe->dma = dma_map_single(ibdev->dma_device, qe->data, capsule_size,
+ dir);
+ if (dma_mapping_error(ibdev->dma_device, qe->dma)) {
kfree(qe->data);
return -ENOMEM;
}
@@ -895,9 +896,8 @@ static void nvme_rdma_unmap_data(struct nvme_rdma_queue *queue,
}
}
- ib_dma_unmap_sg(ibdev, req->sg_table.sgl,
- req->nents, rq_data_dir(rq) ==
- WRITE ? DMA_TO_DEVICE : DMA_FROM_DEVICE);
+ dma_unmap_sg(ibdev->dma_device, req->sg_table.sgl, req->nents,
+ rq_data_dir(rq) == WRITE ? DMA_TO_DEVICE : DMA_FROM_DEVICE);
nvme_cleanup_cmd(rq);
sg_free_table_chained(&req->sg_table, true);
@@ -1008,7 +1008,7 @@ static int nvme_rdma_map_data(struct nvme_rdma_queue *queue,
req->nents = blk_rq_map_sg(rq->q, rq, req->sg_table.sgl);
- count = ib_dma_map_sg(ibdev, req->sg_table.sgl, req->nents,
+ count = dma_map_sg(ibdev->dma_device, req->sg_table.sgl, req->nents,
rq_data_dir(rq) == WRITE ? DMA_TO_DEVICE : DMA_FROM_DEVICE);
if (unlikely(count <= 0)) {
sg_free_table_chained(&req->sg_table, true);
@@ -1135,7 +1135,8 @@ static void nvme_rdma_submit_async_event(struct nvme_ctrl *arg, int aer_idx)
if (WARN_ON_ONCE(aer_idx != 0))
return;
- ib_dma_sync_single_for_cpu(dev, sqe->dma, sizeof(*cmd), DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(dev->dma_device, sqe->dma, sizeof(*cmd),
+ DMA_TO_DEVICE);
memset(cmd, 0, sizeof(*cmd));
cmd->common.opcode = nvme_admin_async_event;
@@ -1143,8 +1144,8 @@ static void nvme_rdma_submit_async_event(struct nvme_ctrl *arg, int aer_idx)
cmd->common.flags |= NVME_CMD_SGL_METABUF;
nvme_rdma_set_sg_null(cmd);
- ib_dma_sync_single_for_device(dev, sqe->dma, sizeof(*cmd),
- DMA_TO_DEVICE);
+ dma_sync_single_for_device(dev->dma_device, sqe->dma, sizeof(*cmd),
+ DMA_TO_DEVICE);
ret = nvme_rdma_post_send(queue, sqe, &sge, 1, NULL, false);
WARN_ON_ONCE(ret);
@@ -1194,7 +1195,8 @@ static int __nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc, int tag)
return 0;
}
- ib_dma_sync_single_for_cpu(ibdev, qe->dma, len, DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(ibdev->dma_device, qe->dma, len,
+ DMA_FROM_DEVICE);
/*
* AEN requests are special as they don't time out and can
* survive any kind of queue freeze and often don't respond to
@@ -1207,7 +1209,8 @@ static int __nvme_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc, int tag)
&cqe->result);
else
ret = nvme_rdma_process_nvme_rsp(queue, cqe, wc, tag);
- ib_dma_sync_single_for_device(ibdev, qe->dma, len, DMA_FROM_DEVICE);
+ dma_sync_single_for_device(ibdev->dma_device, qe->dma, len,
+ DMA_FROM_DEVICE);
nvme_rdma_post_recv(queue, qe);
return ret;
@@ -1455,8 +1458,8 @@ static int nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
return BLK_MQ_RQ_QUEUE_BUSY;
dev = queue->device->dev;
- ib_dma_sync_single_for_cpu(dev, sqe->dma,
- sizeof(struct nvme_command), DMA_TO_DEVICE);
+ dma_sync_single_for_cpu(dev->dma_device, sqe->dma,
+ sizeof(struct nvme_command), DMA_TO_DEVICE);
ret = nvme_setup_cmd(ns, rq, c);
if (ret != BLK_MQ_RQ_QUEUE_OK)
@@ -1473,8 +1476,8 @@ static int nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
goto err;
}
- ib_dma_sync_single_for_device(dev, sqe->dma,
- sizeof(struct nvme_command), DMA_TO_DEVICE);
+ dma_sync_single_for_device(dev->dma_device, sqe->dma,
+ sizeof(struct nvme_command), DMA_TO_DEVICE);
if (rq->cmd_type == REQ_TYPE_FS && req_op(rq) == REQ_OP_FLUSH)
flush = true;
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index 8c3760a78ac0..f8be76f9fb5c 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -246,9 +246,9 @@ static int nvmet_rdma_alloc_cmd(struct nvmet_rdma_device *ndev,
if (!c->nvme_cmd)
goto out;
- c->sge[0].addr = ib_dma_map_single(ndev->device, c->nvme_cmd,
- sizeof(*c->nvme_cmd), DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(ndev->device, c->sge[0].addr))
+ c->sge[0].addr = dma_map_single(ndev->device->dma_device, c->nvme_cmd,
+ sizeof(*c->nvme_cmd), DMA_FROM_DEVICE);
+ if (dma_mapping_error(ndev->device->dma_device, c->sge[0].addr))
goto out_free_cmd;
c->sge[0].length = sizeof(*c->nvme_cmd);
@@ -259,10 +259,10 @@ static int nvmet_rdma_alloc_cmd(struct nvmet_rdma_device *ndev,
get_order(NVMET_RDMA_INLINE_DATA_SIZE));
if (!c->inline_page)
goto out_unmap_cmd;
- c->sge[1].addr = ib_dma_map_page(ndev->device,
+ c->sge[1].addr = dma_map_page(ndev->device->dma_device,
c->inline_page, 0, NVMET_RDMA_INLINE_DATA_SIZE,
DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(ndev->device, c->sge[1].addr))
+ if (dma_mapping_error(ndev->device->dma_device, c->sge[1].addr))
goto out_free_inline_page;
c->sge[1].length = NVMET_RDMA_INLINE_DATA_SIZE;
c->sge[1].lkey = ndev->pd->local_dma_lkey;
@@ -282,8 +282,8 @@ static int nvmet_rdma_alloc_cmd(struct nvmet_rdma_device *ndev,
get_order(NVMET_RDMA_INLINE_DATA_SIZE));
}
out_unmap_cmd:
- ib_dma_unmap_single(ndev->device, c->sge[0].addr,
- sizeof(*c->nvme_cmd), DMA_FROM_DEVICE);
+ dma_unmap_single(ndev->device->dma_device, c->sge[0].addr,
+ sizeof(*c->nvme_cmd), DMA_FROM_DEVICE);
out_free_cmd:
kfree(c->nvme_cmd);
@@ -295,13 +295,13 @@ static void nvmet_rdma_free_cmd(struct nvmet_rdma_device *ndev,
struct nvmet_rdma_cmd *c, bool admin)
{
if (!admin) {
- ib_dma_unmap_page(ndev->device, c->sge[1].addr,
- NVMET_RDMA_INLINE_DATA_SIZE, DMA_FROM_DEVICE);
+ dma_unmap_page(ndev->device->dma_device, c->sge[1].addr,
+ NVMET_RDMA_INLINE_DATA_SIZE, DMA_FROM_DEVICE);
__free_pages(c->inline_page,
get_order(NVMET_RDMA_INLINE_DATA_SIZE));
}
- ib_dma_unmap_single(ndev->device, c->sge[0].addr,
- sizeof(*c->nvme_cmd), DMA_FROM_DEVICE);
+ dma_unmap_single(ndev->device->dma_device, c->sge[0].addr,
+ sizeof(*c->nvme_cmd), DMA_FROM_DEVICE);
kfree(c->nvme_cmd);
}
@@ -350,9 +350,9 @@ static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev,
if (!r->req.rsp)
goto out;
- r->send_sge.addr = ib_dma_map_single(ndev->device, r->req.rsp,
- sizeof(*r->req.rsp), DMA_TO_DEVICE);
- if (ib_dma_mapping_error(ndev->device, r->send_sge.addr))
+ r->send_sge.addr = dma_map_single(ndev->device->dma_device, r->req.rsp,
+ sizeof(*r->req.rsp), DMA_TO_DEVICE);
+ if (dma_mapping_error(ndev->device->dma_device, r->send_sge.addr))
goto out_free_rsp;
r->send_sge.length = sizeof(*r->req.rsp);
@@ -378,8 +378,8 @@ static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev,
static void nvmet_rdma_free_rsp(struct nvmet_rdma_device *ndev,
struct nvmet_rdma_rsp *r)
{
- ib_dma_unmap_single(ndev->device, r->send_sge.addr,
- sizeof(*r->req.rsp), DMA_TO_DEVICE);
+ dma_unmap_single(ndev->device->dma_device, r->send_sge.addr,
+ sizeof(*r->req.rsp), DMA_TO_DEVICE);
kfree(r->req.rsp);
}
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 23/26] net/9p: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (21 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 22/26] nvme-rdma: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:07 ` [PATCH v2 24/26] net/rds: " Bart Van Assche
` (4 subsequent siblings)
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Eric Van Hensbergen
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Cc: Eric Van Hensbergen <ericvh@gmail.com>
---
net/9p/trans_rdma.c | 24 +++++++++++-------------
1 file changed, 11 insertions(+), 13 deletions(-)
diff --git a/net/9p/trans_rdma.c b/net/9p/trans_rdma.c
index 553ed4ecb6a0..b49d5998289e 100644
--- a/net/9p/trans_rdma.c
+++ b/net/9p/trans_rdma.c
@@ -294,8 +294,8 @@ recv_done(struct ib_cq *cq, struct ib_wc *wc)
int16_t tag;
req = NULL;
- ib_dma_unmap_single(rdma->cm_id->device, c->busa, client->msize,
- DMA_FROM_DEVICE);
+ dma_unmap_single(rdma->cm_id->device->dma_device, c->busa,
+ client->msize, DMA_FROM_DEVICE);
if (wc->status != IB_WC_SUCCESS)
goto err_out;
@@ -339,9 +339,8 @@ send_done(struct ib_cq *cq, struct ib_wc *wc)
struct p9_rdma_context *c =
container_of(wc->wr_cqe, struct p9_rdma_context, cqe);
- ib_dma_unmap_single(rdma->cm_id->device,
- c->busa, c->req->tc->size,
- DMA_TO_DEVICE);
+ dma_unmap_single(rdma->cm_id->device->dma_device, c->busa,
+ c->req->tc->size, DMA_TO_DEVICE);
up(&rdma->sq_sem);
kfree(c);
}
@@ -379,10 +378,9 @@ post_recv(struct p9_client *client, struct p9_rdma_context *c)
struct ib_recv_wr wr, *bad_wr;
struct ib_sge sge;
- c->busa = ib_dma_map_single(rdma->cm_id->device,
- c->rc->sdata, client->msize,
- DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(rdma->cm_id->device, c->busa))
+ c->busa = dma_map_single(rdma->cm_id->device->dma_device, c->rc->sdata,
+ client->msize, DMA_FROM_DEVICE);
+ if (dma_mapping_error(rdma->cm_id->device->dma_device, c->busa))
goto error;
c->cqe.done = recv_done;
@@ -469,10 +467,10 @@ static int rdma_request(struct p9_client *client, struct p9_req_t *req)
}
c->req = req;
- c->busa = ib_dma_map_single(rdma->cm_id->device,
- c->req->tc->sdata, c->req->tc->size,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(rdma->cm_id->device, c->busa)) {
+ c->busa = dma_map_single(rdma->cm_id->device->dma_device,
+ c->req->tc->sdata, c->req->tc->size,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(rdma->cm_id->device->dma_device, c->busa)) {
err = -EIO;
goto send_error;
}
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 24/26] net/rds: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (22 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 23/26] net/9p: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:18 ` santosh.shilimkar
2017-01-12 19:07 ` [PATCH v2 25/26] xprtrdma: " Bart Van Assche
` (3 subsequent siblings)
27 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Santosh Shilimkar
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Cc: Santosh Shilimkar <santosh.shilimkar@oracle.com>
---
net/rds/ib.h | 39 ---------------------------------------
net/rds/ib_cm.c | 42 ++++++++++++++++++++++--------------------
net/rds/ib_fmr.c | 10 +++++-----
net/rds/ib_frmr.c | 12 ++++++------
net/rds/ib_rdma.c | 13 ++++++-------
net/rds/ib_recv.c | 19 +++++++++----------
net/rds/ib_send.c | 50 +++++++++++++++++++++++++-------------------------
7 files changed, 73 insertions(+), 112 deletions(-)
diff --git a/net/rds/ib.h b/net/rds/ib.h
index d21ca88ab628..02e5fe8d6af8 100644
--- a/net/rds/ib.h
+++ b/net/rds/ib.h
@@ -275,45 +275,6 @@ struct rds_ib_statistics {
extern struct workqueue_struct *rds_ib_wq;
-/*
- * Fake ib_dma_sync_sg_for_{cpu,device} as long as ib_verbs.h
- * doesn't define it.
- */
-static inline void rds_ib_dma_sync_sg_for_cpu(struct ib_device *dev,
- struct scatterlist *sglist,
- unsigned int sg_dma_len,
- int direction)
-{
- struct scatterlist *sg;
- unsigned int i;
-
- for_each_sg(sglist, sg, sg_dma_len, i) {
- ib_dma_sync_single_for_cpu(dev,
- ib_sg_dma_address(dev, sg),
- ib_sg_dma_len(dev, sg),
- direction);
- }
-}
-#define ib_dma_sync_sg_for_cpu rds_ib_dma_sync_sg_for_cpu
-
-static inline void rds_ib_dma_sync_sg_for_device(struct ib_device *dev,
- struct scatterlist *sglist,
- unsigned int sg_dma_len,
- int direction)
-{
- struct scatterlist *sg;
- unsigned int i;
-
- for_each_sg(sglist, sg, sg_dma_len, i) {
- ib_dma_sync_single_for_device(dev,
- ib_sg_dma_address(dev, sg),
- ib_sg_dma_len(dev, sg),
- direction);
- }
-}
-#define ib_dma_sync_sg_for_device rds_ib_dma_sync_sg_for_device
-
-
/* ib.c */
extern struct rds_transport rds_ib_transport;
struct rds_ib_device *rds_ib_get_client_data(struct ib_device *device);
diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
index 5b2ab95afa07..a6d3726ea27d 100644
--- a/net/rds/ib_cm.c
+++ b/net/rds/ib_cm.c
@@ -456,31 +456,32 @@ static int rds_ib_setup_qp(struct rds_connection *conn)
goto out;
}
- ic->i_send_hdrs = ib_dma_alloc_coherent(dev,
- ic->i_send_ring.w_nr *
+ ic->i_send_hdrs = dma_alloc_coherent(dev->dma_device,
+ ic->i_send_ring.w_nr *
sizeof(struct rds_header),
- &ic->i_send_hdrs_dma, GFP_KERNEL);
+ &ic->i_send_hdrs_dma, GFP_KERNEL);
if (!ic->i_send_hdrs) {
ret = -ENOMEM;
- rdsdebug("ib_dma_alloc_coherent send failed\n");
+ rdsdebug("dma_alloc_coherent send failed\n");
goto out;
}
- ic->i_recv_hdrs = ib_dma_alloc_coherent(dev,
- ic->i_recv_ring.w_nr *
+ ic->i_recv_hdrs = dma_alloc_coherent(dev->dma_device,
+ ic->i_recv_ring.w_nr *
sizeof(struct rds_header),
- &ic->i_recv_hdrs_dma, GFP_KERNEL);
+ &ic->i_recv_hdrs_dma, GFP_KERNEL);
if (!ic->i_recv_hdrs) {
ret = -ENOMEM;
- rdsdebug("ib_dma_alloc_coherent recv failed\n");
+ rdsdebug("dma_alloc_coherent recv failed\n");
goto out;
}
- ic->i_ack = ib_dma_alloc_coherent(dev, sizeof(struct rds_header),
+ ic->i_ack = dma_alloc_coherent(dev->dma_device,
+ sizeof(struct rds_header),
&ic->i_ack_dma, GFP_KERNEL);
if (!ic->i_ack) {
ret = -ENOMEM;
- rdsdebug("ib_dma_alloc_coherent ack failed\n");
+ rdsdebug("dma_alloc_coherent ack failed\n");
goto out;
}
@@ -781,22 +782,23 @@ void rds_ib_conn_path_shutdown(struct rds_conn_path *cp)
/* then free the resources that ib callbacks use */
if (ic->i_send_hdrs)
- ib_dma_free_coherent(dev,
- ic->i_send_ring.w_nr *
+ dma_free_coherent(dev->dma_device,
+ ic->i_send_ring.w_nr *
sizeof(struct rds_header),
- ic->i_send_hdrs,
- ic->i_send_hdrs_dma);
+ ic->i_send_hdrs,
+ ic->i_send_hdrs_dma);
if (ic->i_recv_hdrs)
- ib_dma_free_coherent(dev,
- ic->i_recv_ring.w_nr *
+ dma_free_coherent(dev->dma_device,
+ ic->i_recv_ring.w_nr *
sizeof(struct rds_header),
- ic->i_recv_hdrs,
- ic->i_recv_hdrs_dma);
+ ic->i_recv_hdrs,
+ ic->i_recv_hdrs_dma);
if (ic->i_ack)
- ib_dma_free_coherent(dev, sizeof(struct rds_header),
- ic->i_ack, ic->i_ack_dma);
+ dma_free_coherent(dev->dma_device,
+ sizeof(struct rds_header),
+ ic->i_ack, ic->i_ack_dma);
if (ic->i_sends)
rds_ib_send_clear_ring(ic);
diff --git a/net/rds/ib_fmr.c b/net/rds/ib_fmr.c
index 4fe8f4fec4ee..150e8f756bd9 100644
--- a/net/rds/ib_fmr.c
+++ b/net/rds/ib_fmr.c
@@ -100,7 +100,7 @@ int rds_ib_map_fmr(struct rds_ib_device *rds_ibdev, struct rds_ib_mr *ibmr,
int i, j;
int ret;
- sg_dma_len = ib_dma_map_sg(dev, sg, nents, DMA_BIDIRECTIONAL);
+ sg_dma_len = dma_map_sg(dev->dma_device, sg, nents, DMA_BIDIRECTIONAL);
if (unlikely(!sg_dma_len)) {
pr_warn("RDS/IB: %s failed!\n", __func__);
return -EBUSY;
@@ -110,8 +110,8 @@ int rds_ib_map_fmr(struct rds_ib_device *rds_ibdev, struct rds_ib_mr *ibmr,
page_cnt = 0;
for (i = 0; i < sg_dma_len; ++i) {
- unsigned int dma_len = ib_sg_dma_len(dev, &scat[i]);
- u64 dma_addr = ib_sg_dma_address(dev, &scat[i]);
+ unsigned int dma_len = sg_dma_len(&scat[i]);
+ u64 dma_addr = sg_dma_address(&scat[i]);
if (dma_addr & ~PAGE_MASK) {
if (i > 0)
@@ -140,8 +140,8 @@ int rds_ib_map_fmr(struct rds_ib_device *rds_ibdev, struct rds_ib_mr *ibmr,
page_cnt = 0;
for (i = 0; i < sg_dma_len; ++i) {
- unsigned int dma_len = ib_sg_dma_len(dev, &scat[i]);
- u64 dma_addr = ib_sg_dma_address(dev, &scat[i]);
+ unsigned int dma_len = sg_dma_len(&scat[i]);
+ u64 dma_addr = sg_dma_address(&scat[i]);
for (j = 0; j < dma_len; j += PAGE_SIZE)
dma_pages[page_cnt++] =
diff --git a/net/rds/ib_frmr.c b/net/rds/ib_frmr.c
index d921adc62765..2d44bf11d97f 100644
--- a/net/rds/ib_frmr.c
+++ b/net/rds/ib_frmr.c
@@ -169,8 +169,8 @@ static int rds_ib_map_frmr(struct rds_ib_device *rds_ibdev,
ibmr->sg_dma_len = 0;
frmr->sg_byte_len = 0;
WARN_ON(ibmr->sg_dma_len);
- ibmr->sg_dma_len = ib_dma_map_sg(dev, ibmr->sg, ibmr->sg_len,
- DMA_BIDIRECTIONAL);
+ ibmr->sg_dma_len = dma_map_sg(dev->dma_device, ibmr->sg, ibmr->sg_len,
+ DMA_BIDIRECTIONAL);
if (unlikely(!ibmr->sg_dma_len)) {
pr_warn("RDS/IB: %s failed!\n", __func__);
return -EBUSY;
@@ -182,8 +182,8 @@ static int rds_ib_map_frmr(struct rds_ib_device *rds_ibdev,
ret = -EINVAL;
for (i = 0; i < ibmr->sg_dma_len; ++i) {
- unsigned int dma_len = ib_sg_dma_len(dev, &ibmr->sg[i]);
- u64 dma_addr = ib_sg_dma_address(dev, &ibmr->sg[i]);
+ unsigned int dma_len = sg_dma_len(&ibmr->sg[i]);
+ u64 dma_addr = sg_dma_address(&ibmr->sg[i]);
frmr->sg_byte_len += dma_len;
if (dma_addr & ~PAGE_MASK) {
@@ -221,8 +221,8 @@ static int rds_ib_map_frmr(struct rds_ib_device *rds_ibdev,
return ret;
out_unmap:
- ib_dma_unmap_sg(rds_ibdev->dev, ibmr->sg, ibmr->sg_len,
- DMA_BIDIRECTIONAL);
+ dma_unmap_sg(rds_ibdev->dev->dma_device, ibmr->sg, ibmr->sg_len,
+ DMA_BIDIRECTIONAL);
ibmr->sg_dma_len = 0;
return ret;
}
diff --git a/net/rds/ib_rdma.c b/net/rds/ib_rdma.c
index 977f69886c00..9f5acba71a05 100644
--- a/net/rds/ib_rdma.c
+++ b/net/rds/ib_rdma.c
@@ -221,12 +221,12 @@ void rds_ib_sync_mr(void *trans_private, int direction)
switch (direction) {
case DMA_FROM_DEVICE:
- ib_dma_sync_sg_for_cpu(rds_ibdev->dev, ibmr->sg,
- ibmr->sg_dma_len, DMA_BIDIRECTIONAL);
+ dma_sync_sg_for_cpu(rds_ibdev->dev->dma_device, ibmr->sg,
+ ibmr->sg_dma_len, DMA_BIDIRECTIONAL);
break;
case DMA_TO_DEVICE:
- ib_dma_sync_sg_for_device(rds_ibdev->dev, ibmr->sg,
- ibmr->sg_dma_len, DMA_BIDIRECTIONAL);
+ dma_sync_sg_for_device(rds_ibdev->dev->dma_device, ibmr->sg,
+ ibmr->sg_dma_len, DMA_BIDIRECTIONAL);
break;
}
}
@@ -236,9 +236,8 @@ void __rds_ib_teardown_mr(struct rds_ib_mr *ibmr)
struct rds_ib_device *rds_ibdev = ibmr->device;
if (ibmr->sg_dma_len) {
- ib_dma_unmap_sg(rds_ibdev->dev,
- ibmr->sg, ibmr->sg_len,
- DMA_BIDIRECTIONAL);
+ dma_unmap_sg(rds_ibdev->dev->dma_device, ibmr->sg, ibmr->sg_len,
+ DMA_BIDIRECTIONAL);
ibmr->sg_dma_len = 0;
}
diff --git a/net/rds/ib_recv.c b/net/rds/ib_recv.c
index 606a11f681d2..32bc2df27e5a 100644
--- a/net/rds/ib_recv.c
+++ b/net/rds/ib_recv.c
@@ -225,7 +225,8 @@ static void rds_ib_recv_clear_one(struct rds_ib_connection *ic,
recv->r_ibinc = NULL;
}
if (recv->r_frag) {
- ib_dma_unmap_sg(ic->i_cm_id->device, &recv->r_frag->f_sg, 1, DMA_FROM_DEVICE);
+ dma_unmap_sg(ic->i_cm_id->device->dma_device,
+ &recv->r_frag->f_sg, 1, DMA_FROM_DEVICE);
rds_ib_frag_free(ic, recv->r_frag);
recv->r_frag = NULL;
}
@@ -331,8 +332,8 @@ static int rds_ib_recv_refill_one(struct rds_connection *conn,
if (!recv->r_frag)
goto out;
- ret = ib_dma_map_sg(ic->i_cm_id->device, &recv->r_frag->f_sg,
- 1, DMA_FROM_DEVICE);
+ ret = dma_map_sg(ic->i_cm_id->device->dma_device, &recv->r_frag->f_sg,
+ 1, DMA_FROM_DEVICE);
WARN_ON(ret != 1);
sge = &recv->r_sge[0];
@@ -340,8 +341,8 @@ static int rds_ib_recv_refill_one(struct rds_connection *conn,
sge->length = sizeof(struct rds_header);
sge = &recv->r_sge[1];
- sge->addr = ib_sg_dma_address(ic->i_cm_id->device, &recv->r_frag->f_sg);
- sge->length = ib_sg_dma_len(ic->i_cm_id->device, &recv->r_frag->f_sg);
+ sge->addr = sg_dma_address(&recv->r_frag->f_sg);
+ sge->length = sg_dma_len(&recv->r_frag->f_sg);
ret = 0;
out:
@@ -408,9 +409,7 @@ void rds_ib_recv_refill(struct rds_connection *conn, int prefill, gfp_t gfp)
ret = ib_post_recv(ic->i_cm_id->qp, &recv->r_wr, &failed_wr);
rdsdebug("recv %p ibinc %p page %p addr %lu ret %d\n", recv,
recv->r_ibinc, sg_page(&recv->r_frag->f_sg),
- (long) ib_sg_dma_address(
- ic->i_cm_id->device,
- &recv->r_frag->f_sg),
+ (long)sg_dma_address(&recv->r_frag->f_sg),
ret);
if (ret) {
rds_ib_conn_error(conn, "recv post on "
@@ -968,8 +967,8 @@ void rds_ib_recv_cqe_handler(struct rds_ib_connection *ic,
rds_ib_stats_inc(s_ib_rx_cq_event);
recv = &ic->i_recvs[rds_ib_ring_oldest(&ic->i_recv_ring)];
- ib_dma_unmap_sg(ic->i_cm_id->device, &recv->r_frag->f_sg, 1,
- DMA_FROM_DEVICE);
+ dma_unmap_sg(ic->i_cm_id->device->dma_device, &recv->r_frag->f_sg, 1,
+ DMA_FROM_DEVICE);
/* Also process recvs in connecting state because it is possible
* to get a recv completion _before_ the rdmacm ESTABLISHED
diff --git a/net/rds/ib_send.c b/net/rds/ib_send.c
index 84d90c97332f..aa79b6aa488e 100644
--- a/net/rds/ib_send.c
+++ b/net/rds/ib_send.c
@@ -74,9 +74,8 @@ static void rds_ib_send_unmap_data(struct rds_ib_connection *ic,
int wc_status)
{
if (op->op_nents)
- ib_dma_unmap_sg(ic->i_cm_id->device,
- op->op_sg, op->op_nents,
- DMA_TO_DEVICE);
+ dma_unmap_sg(ic->i_cm_id->device->dma_device, op->op_sg,
+ op->op_nents, DMA_TO_DEVICE);
}
static void rds_ib_send_unmap_rdma(struct rds_ib_connection *ic,
@@ -84,9 +83,9 @@ static void rds_ib_send_unmap_rdma(struct rds_ib_connection *ic,
int wc_status)
{
if (op->op_mapped) {
- ib_dma_unmap_sg(ic->i_cm_id->device,
- op->op_sg, op->op_nents,
- op->op_write ? DMA_TO_DEVICE : DMA_FROM_DEVICE);
+ dma_unmap_sg(ic->i_cm_id->device->dma_device, op->op_sg,
+ op->op_nents, op->op_write ? DMA_TO_DEVICE :
+ DMA_FROM_DEVICE);
op->op_mapped = 0;
}
@@ -106,7 +105,7 @@ static void rds_ib_send_unmap_rdma(struct rds_ib_connection *ic,
* handling in the ACK processing code.
*
* Note: There's no need to explicitly sync any RDMA buffers using
- * ib_dma_sync_sg_for_cpu - the completion for the RDMA
+ * dma_sync_sg_for_cpu - the completion for the RDMA
* operation itself unmapped the RDMA buffers, which takes care
* of synching.
*/
@@ -125,8 +124,8 @@ static void rds_ib_send_unmap_atomic(struct rds_ib_connection *ic,
{
/* unmap atomic recvbuf */
if (op->op_mapped) {
- ib_dma_unmap_sg(ic->i_cm_id->device, op->op_sg, 1,
- DMA_FROM_DEVICE);
+ dma_unmap_sg(ic->i_cm_id->device->dma_device, op->op_sg, 1,
+ DMA_FROM_DEVICE);
op->op_mapped = 0;
}
@@ -546,10 +545,10 @@ int rds_ib_xmit(struct rds_connection *conn, struct rds_message *rm,
/* map the message the first time we see it */
if (!ic->i_data_op) {
if (rm->data.op_nents) {
- rm->data.op_count = ib_dma_map_sg(dev,
- rm->data.op_sg,
- rm->data.op_nents,
- DMA_TO_DEVICE);
+ rm->data.op_count = dma_map_sg(dev->dma_device,
+ rm->data.op_sg,
+ rm->data.op_nents,
+ DMA_TO_DEVICE);
rdsdebug("ic %p mapping rm %p: %d\n", ic, rm, rm->data.op_count);
if (rm->data.op_count == 0) {
rds_ib_stats_inc(s_ib_tx_sg_mapping_failure);
@@ -640,16 +639,16 @@ int rds_ib_xmit(struct rds_connection *conn, struct rds_message *rm,
if (i < work_alloc
&& scat != &rm->data.op_sg[rm->data.op_count]) {
len = min(RDS_FRAG_SIZE,
- ib_sg_dma_len(dev, scat) - rm->data.op_dmaoff);
+ sg_dma_len(scat) - rm->data.op_dmaoff);
send->s_wr.num_sge = 2;
- send->s_sge[1].addr = ib_sg_dma_address(dev, scat);
+ send->s_sge[1].addr = sg_dma_address(scat);
send->s_sge[1].addr += rm->data.op_dmaoff;
send->s_sge[1].length = len;
bytes_sent += len;
rm->data.op_dmaoff += len;
- if (rm->data.op_dmaoff == ib_sg_dma_len(dev, scat)) {
+ if (rm->data.op_dmaoff == sg_dma_len(scat)) {
scat++;
rm->data.op_dmasg++;
rm->data.op_dmaoff = 0;
@@ -797,7 +796,8 @@ int rds_ib_xmit_atomic(struct rds_connection *conn, struct rm_atomic_op *op)
rds_message_addref(container_of(send->s_op, struct rds_message, atomic));
/* map 8 byte retval buffer to the device */
- ret = ib_dma_map_sg(ic->i_cm_id->device, op->op_sg, 1, DMA_FROM_DEVICE);
+ ret = dma_map_sg(ic->i_cm_id->device->dma_device, op->op_sg, 1,
+ DMA_FROM_DEVICE);
rdsdebug("ic %p mapping atomic op %p. mapped %d pg\n", ic, op, ret);
if (ret != 1) {
rds_ib_ring_unalloc(&ic->i_send_ring, work_alloc);
@@ -807,8 +807,8 @@ int rds_ib_xmit_atomic(struct rds_connection *conn, struct rm_atomic_op *op)
}
/* Convert our struct scatterlist to struct ib_sge */
- send->s_sge[0].addr = ib_sg_dma_address(ic->i_cm_id->device, op->op_sg);
- send->s_sge[0].length = ib_sg_dma_len(ic->i_cm_id->device, op->op_sg);
+ send->s_sge[0].addr = sg_dma_address(op->op_sg);
+ send->s_sge[0].length = sg_dma_len(op->op_sg);
send->s_sge[0].lkey = ic->i_pd->local_dma_lkey;
rdsdebug("rva %Lx rpa %Lx len %u\n", op->op_remote_addr,
@@ -861,9 +861,10 @@ int rds_ib_xmit_rdma(struct rds_connection *conn, struct rm_rdma_op *op)
/* map the op the first time we see it */
if (!op->op_mapped) {
- op->op_count = ib_dma_map_sg(ic->i_cm_id->device,
- op->op_sg, op->op_nents, (op->op_write) ?
- DMA_TO_DEVICE : DMA_FROM_DEVICE);
+ op->op_count = dma_map_sg(ic->i_cm_id->device->dma_device,
+ op->op_sg, op->op_nents,
+ op->op_write ? DMA_TO_DEVICE :
+ DMA_FROM_DEVICE);
rdsdebug("ic %p mapping op %p: %d\n", ic, op, op->op_count);
if (op->op_count == 0) {
rds_ib_stats_inc(s_ib_tx_sg_mapping_failure);
@@ -920,9 +921,8 @@ int rds_ib_xmit_rdma(struct rds_connection *conn, struct rm_rdma_op *op)
for (j = 0; j < send->s_rdma_wr.wr.num_sge &&
scat != &op->op_sg[op->op_count]; j++) {
- len = ib_sg_dma_len(ic->i_cm_id->device, scat);
- send->s_sge[j].addr =
- ib_sg_dma_address(ic->i_cm_id->device, scat);
+ len = sg_dma_len(scat);
+ send->s_sge[j].addr = sg_dma_address(scat);
send->s_sge[j].length = len;
send->s_sge[j].lkey = ic->i_pd->local_dma_lkey;
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* [PATCH v2 25/26] xprtrdma: Inline ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (23 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 24/26] net/rds: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-12 19:19 ` Chuck Lever
2017-01-12 19:07 ` [PATCH v2 26/26] IB/core: Remove " Bart Van Assche
` (2 subsequent siblings)
27 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche,
Trond Myklebust, Anna Schumaker
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Cc: Trond Myklebust <trond.myklebust@primarydata.com>
Cc: Anna Schumaker <anna.schumaker@netapp.com>
---
net/sunrpc/xprtrdma/fmr_ops.c | 12 ++++++------
net/sunrpc/xprtrdma/frwr_ops.c | 12 ++++++------
net/sunrpc/xprtrdma/rpc_rdma.c | 28 ++++++++++++++--------------
net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 7 ++++---
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 20 +++++++++-----------
net/sunrpc/xprtrdma/svc_rdma_sendto.c | 26 ++++++++++++++------------
net/sunrpc/xprtrdma/svc_rdma_transport.c | 17 +++++++----------
net/sunrpc/xprtrdma/verbs.c | 18 ++++++++----------
8 files changed, 68 insertions(+), 72 deletions(-)
diff --git a/net/sunrpc/xprtrdma/fmr_ops.c b/net/sunrpc/xprtrdma/fmr_ops.c
index 1ebb09e1ac4f..debc838e3a6a 100644
--- a/net/sunrpc/xprtrdma/fmr_ops.c
+++ b/net/sunrpc/xprtrdma/fmr_ops.c
@@ -136,8 +136,8 @@ fmr_op_recover_mr(struct rpcrdma_mw *mw)
rc = __fmr_unmap(mw);
/* ORDER: then DMA unmap */
- ib_dma_unmap_sg(r_xprt->rx_ia.ri_device,
- mw->mw_sg, mw->mw_nents, mw->mw_dir);
+ dma_unmap_sg(r_xprt->rx_ia.ri_device->dma_device, mw->mw_sg,
+ mw->mw_nents, mw->mw_dir);
if (rc)
goto out_release;
@@ -218,8 +218,8 @@ fmr_op_map(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mr_seg *seg,
if (i == 0)
goto out_dmamap_err;
- if (!ib_dma_map_sg(r_xprt->rx_ia.ri_device,
- mw->mw_sg, mw->mw_nents, mw->mw_dir))
+ if (!dma_map_sg(r_xprt->rx_ia.ri_device->dma_device, mw->mw_sg,
+ mw->mw_nents, mw->mw_dir))
goto out_dmamap_err;
for (i = 0, dma_pages = mw->fmr.fm_physaddrs; i < mw->mw_nents; i++)
@@ -284,8 +284,8 @@ fmr_op_unmap_sync(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req)
list_for_each_entry_safe(mw, tmp, &req->rl_registered, mw_list) {
list_del_init(&mw->mw_list);
list_del_init(&mw->fmr.fm_mr->list);
- ib_dma_unmap_sg(r_xprt->rx_ia.ri_device,
- mw->mw_sg, mw->mw_nents, mw->mw_dir);
+ dma_unmap_sg(r_xprt->rx_ia.ri_device->dma_device,
+ mw->mw_sg, mw->mw_nents, mw->mw_dir);
rpcrdma_put_mw(r_xprt, mw);
}
diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c
index 47bed5333c7f..9b7c604669f9 100644
--- a/net/sunrpc/xprtrdma/frwr_ops.c
+++ b/net/sunrpc/xprtrdma/frwr_ops.c
@@ -182,8 +182,8 @@ frwr_op_recover_mr(struct rpcrdma_mw *mw)
rc = __frwr_reset_mr(ia, mw);
if (state != FRMR_FLUSHED_LI)
- ib_dma_unmap_sg(ia->ri_device,
- mw->mw_sg, mw->mw_nents, mw->mw_dir);
+ dma_unmap_sg(ia->ri_device->dma_device, mw->mw_sg, mw->mw_nents,
+ mw->mw_dir);
if (rc)
goto out_release;
@@ -396,8 +396,8 @@ frwr_op_map(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mr_seg *seg,
if (i == 0)
goto out_dmamap_err;
- dma_nents = ib_dma_map_sg(ia->ri_device,
- mw->mw_sg, mw->mw_nents, mw->mw_dir);
+ dma_nents = dma_map_sg(ia->ri_device->dma_device, mw->mw_sg,
+ mw->mw_nents, mw->mw_dir);
if (!dma_nents)
goto out_dmamap_err;
@@ -538,8 +538,8 @@ frwr_op_unmap_sync(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req)
dprintk("RPC: %s: DMA unmapping frmr %p\n",
__func__, &mw->frmr);
list_del_init(&mw->mw_list);
- ib_dma_unmap_sg(ia->ri_device,
- mw->mw_sg, mw->mw_nents, mw->mw_dir);
+ dma_unmap_sg(ia->ri_device->dma_device, mw->mw_sg, mw->mw_nents,
+ mw->mw_dir);
rpcrdma_put_mw(r_xprt, mw);
}
return;
diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c
index c52e0f2ffe52..6b6e700a5a68 100644
--- a/net/sunrpc/xprtrdma/rpc_rdma.c
+++ b/net/sunrpc/xprtrdma/rpc_rdma.c
@@ -476,8 +476,8 @@ rpcrdma_prepare_hdr_sge(struct rpcrdma_ia *ia, struct rpcrdma_req *req,
}
sge->length = len;
- ib_dma_sync_single_for_device(ia->ri_device, sge->addr,
- sge->length, DMA_TO_DEVICE);
+ dma_sync_single_for_device(ia->ri_device->dma_device, sge->addr,
+ sge->length, DMA_TO_DEVICE);
req->rl_send_wr.num_sge++;
return true;
}
@@ -505,8 +505,8 @@ rpcrdma_prepare_msg_sges(struct rpcrdma_ia *ia, struct rpcrdma_req *req,
sge[sge_no].addr = rdmab_addr(rb);
sge[sge_no].length = xdr->head[0].iov_len;
sge[sge_no].lkey = rdmab_lkey(rb);
- ib_dma_sync_single_for_device(device, sge[sge_no].addr,
- sge[sge_no].length, DMA_TO_DEVICE);
+ dma_sync_single_for_device(device->dma_device, sge[sge_no].addr,
+ sge[sge_no].length, DMA_TO_DEVICE);
/* If there is a Read chunk, the page list is being handled
* via explicit RDMA, and thus is skipped here. However, the
@@ -547,10 +547,11 @@ rpcrdma_prepare_msg_sges(struct rpcrdma_ia *ia, struct rpcrdma_req *req,
goto out_mapping_overflow;
len = min_t(u32, PAGE_SIZE - page_base, remaining);
- sge[sge_no].addr = ib_dma_map_page(device, *ppages,
- page_base, len,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(device, sge[sge_no].addr))
+ sge[sge_no].addr = dma_map_page(device->dma_device,
+ *ppages, page_base, len,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(device->dma_device,
+ sge[sge_no].addr))
goto out_mapping_err;
sge[sge_no].length = len;
sge[sge_no].lkey = lkey;
@@ -574,10 +575,9 @@ rpcrdma_prepare_msg_sges(struct rpcrdma_ia *ia, struct rpcrdma_req *req,
map_tail:
sge_no++;
- sge[sge_no].addr = ib_dma_map_page(device, page,
- page_base, len,
- DMA_TO_DEVICE);
- if (ib_dma_mapping_error(device, sge[sge_no].addr))
+ sge[sge_no].addr = dma_map_page(device->dma_device, page,
+ page_base, len, DMA_TO_DEVICE);
+ if (dma_mapping_error(device->dma_device, sge[sge_no].addr))
goto out_mapping_err;
sge[sge_no].length = len;
sge[sge_no].lkey = lkey;
@@ -628,8 +628,8 @@ rpcrdma_unmap_sges(struct rpcrdma_ia *ia, struct rpcrdma_req *req)
sge = &req->rl_send_sge[2];
for (count = req->rl_mapped_sges; count--; sge++)
- ib_dma_unmap_page(device, sge->addr, sge->length,
- DMA_TO_DEVICE);
+ dma_unmap_page(device->dma_device, sge->addr, sge->length,
+ DMA_TO_DEVICE);
req->rl_mapped_sges = 0;
}
diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c
index 288e35c2d8f4..d65abb351f5b 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c
@@ -123,9 +123,10 @@ static int svc_rdma_bc_sendto(struct svcxprt_rdma *rdma,
ctxt->sge[0].lkey = rdma->sc_pd->local_dma_lkey;
ctxt->sge[0].length = sndbuf->len;
ctxt->sge[0].addr =
- ib_dma_map_page(rdma->sc_cm_id->device, ctxt->pages[0], 0,
- sndbuf->len, DMA_TO_DEVICE);
- if (ib_dma_mapping_error(rdma->sc_cm_id->device, ctxt->sge[0].addr)) {
+ dma_map_page(rdma->sc_cm_id->device->dma_device, ctxt->pages[0], 0,
+ sndbuf->len, DMA_TO_DEVICE);
+ if (dma_mapping_error(rdma->sc_cm_id->device->dma_device,
+ ctxt->sge[0].addr)) {
ret = -EIO;
goto out_unmap;
}
diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index 57d35fbb1c28..6ca1786c3493 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -151,12 +151,11 @@ int rdma_read_chunk_lcl(struct svcxprt_rdma *xprt,
rqstp->rq_respages = &rqstp->rq_arg.pages[pg_no+1];
rqstp->rq_next_page = rqstp->rq_respages + 1;
ctxt->sge[pno].addr =
- ib_dma_map_page(xprt->sc_cm_id->device,
- head->arg.pages[pg_no], pg_off,
- PAGE_SIZE - pg_off,
- DMA_FROM_DEVICE);
- ret = ib_dma_mapping_error(xprt->sc_cm_id->device,
- ctxt->sge[pno].addr);
+ dma_map_page(xprt->sc_cm_id->device->dma_device,
+ head->arg.pages[pg_no], pg_off,
+ PAGE_SIZE - pg_off, DMA_FROM_DEVICE);
+ ret = dma_mapping_error(xprt->sc_cm_id->device->dma_device,
+ ctxt->sge[pno].addr);
if (ret)
goto err;
svc_rdma_count_mappings(xprt, ctxt);
@@ -271,9 +270,8 @@ int rdma_read_chunk_frmr(struct svcxprt_rdma *xprt,
else
clear_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags);
- dma_nents = ib_dma_map_sg(xprt->sc_cm_id->device,
- frmr->sg, frmr->sg_nents,
- frmr->direction);
+ dma_nents = dma_map_sg(xprt->sc_cm_id->device->dma_device,
+ frmr->sg, frmr->sg_nents, frmr->direction);
if (!dma_nents) {
pr_err("svcrdma: failed to dma map sg %p\n",
frmr->sg);
@@ -347,8 +345,8 @@ int rdma_read_chunk_frmr(struct svcxprt_rdma *xprt,
atomic_inc(&rdma_stat_read);
return ret;
err:
- ib_dma_unmap_sg(xprt->sc_cm_id->device,
- frmr->sg, frmr->sg_nents, frmr->direction);
+ dma_unmap_sg(xprt->sc_cm_id->device->dma_device, frmr->sg,
+ frmr->sg_nents, frmr->direction);
svc_rdma_put_context(ctxt, 0);
svc_rdma_put_frmr(xprt, frmr);
return ret;
diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
index ad4d286a83c5..5fa33ad92484 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
@@ -148,8 +148,8 @@ static dma_addr_t dma_map_xdr(struct svcxprt_rdma *xprt,
page = virt_to_page(xdr->tail[0].iov_base);
}
}
- dma_addr = ib_dma_map_page(xprt->sc_cm_id->device, page, xdr_off,
- min_t(size_t, PAGE_SIZE, len), dir);
+ dma_addr = dma_map_page(xprt->sc_cm_id->device->dma_device, page,
+ xdr_off, min_t(size_t, PAGE_SIZE, len), dir);
return dma_addr;
}
@@ -269,8 +269,8 @@ static int send_write(struct svcxprt_rdma *xprt, struct svc_rqst *rqstp,
dma_map_xdr(xprt, &rqstp->rq_res, xdr_off,
sge_bytes, DMA_TO_DEVICE);
xdr_off += sge_bytes;
- if (ib_dma_mapping_error(xprt->sc_cm_id->device,
- sge[sge_no].addr))
+ if (dma_mapping_error(xprt->sc_cm_id->device->dma_device,
+ sge[sge_no].addr))
goto err;
svc_rdma_count_mappings(xprt, ctxt);
sge[sge_no].lkey = xprt->sc_pd->local_dma_lkey;
@@ -478,9 +478,10 @@ static int send_reply(struct svcxprt_rdma *rdma,
ctxt->sge[0].lkey = rdma->sc_pd->local_dma_lkey;
ctxt->sge[0].length = svc_rdma_xdr_get_reply_hdr_len(rdma_resp);
ctxt->sge[0].addr =
- ib_dma_map_page(rdma->sc_cm_id->device, page, 0,
- ctxt->sge[0].length, DMA_TO_DEVICE);
- if (ib_dma_mapping_error(rdma->sc_cm_id->device, ctxt->sge[0].addr))
+ dma_map_page(rdma->sc_cm_id->device->dma_device, page, 0,
+ ctxt->sge[0].length, DMA_TO_DEVICE);
+ if (dma_mapping_error(rdma->sc_cm_id->device->dma_device,
+ ctxt->sge[0].addr))
goto err;
svc_rdma_count_mappings(rdma, ctxt);
@@ -495,8 +496,8 @@ static int send_reply(struct svcxprt_rdma *rdma,
dma_map_xdr(rdma, &rqstp->rq_res, xdr_off,
sge_bytes, DMA_TO_DEVICE);
xdr_off += sge_bytes;
- if (ib_dma_mapping_error(rdma->sc_cm_id->device,
- ctxt->sge[sge_no].addr))
+ if (dma_mapping_error(rdma->sc_cm_id->device->dma_device,
+ ctxt->sge[sge_no].addr))
goto err;
svc_rdma_count_mappings(rdma, ctxt);
ctxt->sge[sge_no].lkey = rdma->sc_pd->local_dma_lkey;
@@ -677,9 +678,10 @@ void svc_rdma_send_error(struct svcxprt_rdma *xprt, struct rpcrdma_msg *rmsgp,
/* Prepare SGE for local address */
ctxt->sge[0].lkey = xprt->sc_pd->local_dma_lkey;
ctxt->sge[0].length = length;
- ctxt->sge[0].addr = ib_dma_map_page(xprt->sc_cm_id->device,
- p, 0, length, DMA_TO_DEVICE);
- if (ib_dma_mapping_error(xprt->sc_cm_id->device, ctxt->sge[0].addr)) {
+ ctxt->sge[0].addr = dma_map_page(xprt->sc_cm_id->device->dma_device,
+ p, 0, length, DMA_TO_DEVICE);
+ if (dma_mapping_error(xprt->sc_cm_id->device->dma_device,
+ ctxt->sge[0].addr)) {
dprintk("svcrdma: Error mapping buffer for protocol error\n");
svc_rdma_put_context(ctxt, 1);
return;
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index ca2799af05a6..ed19de5c9a34 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -237,10 +237,8 @@ void svc_rdma_unmap_dma(struct svc_rdma_op_ctxt *ctxt)
* last WR that uses it completes.
*/
if (ctxt->sge[i].lkey == lkey)
- ib_dma_unmap_page(device,
- ctxt->sge[i].addr,
- ctxt->sge[i].length,
- ctxt->direction);
+ dma_unmap_page(device->dma_device, ctxt->sge[i].addr,
+ ctxt->sge[i].length, ctxt->direction);
}
ctxt->mapped_sges = 0;
}
@@ -600,10 +598,9 @@ int svc_rdma_post_recv(struct svcxprt_rdma *xprt, gfp_t flags)
if (!page)
goto err_put_ctxt;
ctxt->pages[sge_no] = page;
- pa = ib_dma_map_page(xprt->sc_cm_id->device,
- page, 0, PAGE_SIZE,
- DMA_FROM_DEVICE);
- if (ib_dma_mapping_error(xprt->sc_cm_id->device, pa))
+ pa = dma_map_page(xprt->sc_cm_id->device->dma_device,
+ page, 0, PAGE_SIZE, DMA_FROM_DEVICE);
+ if (dma_mapping_error(xprt->sc_cm_id->device->dma_device, pa))
goto err_put_ctxt;
svc_rdma_count_mappings(xprt, ctxt);
ctxt->sge[sge_no].addr = pa;
@@ -941,8 +938,8 @@ void svc_rdma_put_frmr(struct svcxprt_rdma *rdma,
struct svc_rdma_fastreg_mr *frmr)
{
if (frmr) {
- ib_dma_unmap_sg(rdma->sc_cm_id->device,
- frmr->sg, frmr->sg_nents, frmr->direction);
+ dma_unmap_sg(rdma->sc_cm_id->device->dma_device, frmr->sg,
+ frmr->sg_nents, frmr->direction);
spin_lock_bh(&rdma->sc_frmr_q_lock);
WARN_ON_ONCE(!list_empty(&frmr->frmr_list));
list_add(&frmr->frmr_list, &rdma->sc_frmr_q);
diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
index 11d07748f699..ba15926173e5 100644
--- a/net/sunrpc/xprtrdma/verbs.c
+++ b/net/sunrpc/xprtrdma/verbs.c
@@ -179,9 +179,9 @@ rpcrdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc)
rep->rr_wc_flags = wc->wc_flags;
rep->rr_inv_rkey = wc->ex.invalidate_rkey;
- ib_dma_sync_single_for_cpu(rep->rr_device,
- rdmab_addr(rep->rr_rdmabuf),
- rep->rr_len, DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(rep->rr_device->dma_device,
+ rdmab_addr(rep->rr_rdmabuf),
+ rep->rr_len, DMA_FROM_DEVICE);
rpcrdma_update_granted_credits(rep);
@@ -1259,11 +1259,9 @@ __rpcrdma_dma_map_regbuf(struct rpcrdma_ia *ia, struct rpcrdma_regbuf *rb)
if (rb->rg_direction == DMA_NONE)
return false;
- rb->rg_iov.addr = ib_dma_map_single(ia->ri_device,
- (void *)rb->rg_base,
- rdmab_length(rb),
- rb->rg_direction);
- if (ib_dma_mapping_error(ia->ri_device, rdmab_addr(rb)))
+ rb->rg_iov.addr = dma_map_single(ia->ri_device->dma_device, rb->rg_base,
+ rdmab_length(rb), rb->rg_direction);
+ if (dma_mapping_error(ia->ri_device->dma_device, rdmab_addr(rb)))
return false;
rb->rg_device = ia->ri_device;
@@ -1277,8 +1275,8 @@ rpcrdma_dma_unmap_regbuf(struct rpcrdma_regbuf *rb)
if (!rpcrdma_regbuf_is_mapped(rb))
return;
- ib_dma_unmap_single(rb->rg_device, rdmab_addr(rb),
- rdmab_length(rb), rb->rg_direction);
+ dma_unmap_single(rb->rg_device->dma_device, rdmab_addr(rb),
+ rdmab_length(rb), rb->rg_direction);
rb->rg_device = NULL;
}
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PATCH v2 25/26] xprtrdma: Inline ib_dma_map_*() functions
2017-01-12 19:07 ` [PATCH v2 25/26] xprtrdma: " Bart Van Assche
@ 2017-01-12 19:19 ` Chuck Lever
0 siblings, 0 replies; 45+ messages in thread
From: Chuck Lever @ 2017-01-12 19:19 UTC (permalink / raw)
To: Bart Van Assche
Cc: Doug Ledford, linux-rdma, linux-kernel, Greg Kroah-Hartman,
Trond Myklebust, Anna Schumaker
> On Jan 12, 2017, at 2:07 PM, Bart Van Assche <bart.vanassche@sandisk.com> wrote:
>
> Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
> Cc: Trond Myklebust <trond.myklebust@primarydata.com>
> Cc: Anna Schumaker <anna.schumaker@netapp.com>
Thanks for taking care of this change, Bart.
Patch looks proper, but the svc_rdma* hunks go in a separate patch
to bfields@redhat.com, cc: linux-rdma and linux-nfs.
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
> ---
> net/sunrpc/xprtrdma/fmr_ops.c | 12 ++++++------
> net/sunrpc/xprtrdma/frwr_ops.c | 12 ++++++------
> net/sunrpc/xprtrdma/rpc_rdma.c | 28 ++++++++++++++--------------
> net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 7 ++++---
> net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 20 +++++++++-----------
> net/sunrpc/xprtrdma/svc_rdma_sendto.c | 26 ++++++++++++++------------
> net/sunrpc/xprtrdma/svc_rdma_transport.c | 17 +++++++----------
> net/sunrpc/xprtrdma/verbs.c | 18 ++++++++----------
> 8 files changed, 68 insertions(+), 72 deletions(-)
>
> diff --git a/net/sunrpc/xprtrdma/fmr_ops.c b/net/sunrpc/xprtrdma/fmr_ops.c
> index 1ebb09e1ac4f..debc838e3a6a 100644
> --- a/net/sunrpc/xprtrdma/fmr_ops.c
> +++ b/net/sunrpc/xprtrdma/fmr_ops.c
> @@ -136,8 +136,8 @@ fmr_op_recover_mr(struct rpcrdma_mw *mw)
> rc = __fmr_unmap(mw);
>
> /* ORDER: then DMA unmap */
> - ib_dma_unmap_sg(r_xprt->rx_ia.ri_device,
> - mw->mw_sg, mw->mw_nents, mw->mw_dir);
> + dma_unmap_sg(r_xprt->rx_ia.ri_device->dma_device, mw->mw_sg,
> + mw->mw_nents, mw->mw_dir);
> if (rc)
> goto out_release;
>
> @@ -218,8 +218,8 @@ fmr_op_map(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mr_seg *seg,
> if (i == 0)
> goto out_dmamap_err;
>
> - if (!ib_dma_map_sg(r_xprt->rx_ia.ri_device,
> - mw->mw_sg, mw->mw_nents, mw->mw_dir))
> + if (!dma_map_sg(r_xprt->rx_ia.ri_device->dma_device, mw->mw_sg,
> + mw->mw_nents, mw->mw_dir))
> goto out_dmamap_err;
>
> for (i = 0, dma_pages = mw->fmr.fm_physaddrs; i < mw->mw_nents; i++)
> @@ -284,8 +284,8 @@ fmr_op_unmap_sync(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req)
> list_for_each_entry_safe(mw, tmp, &req->rl_registered, mw_list) {
> list_del_init(&mw->mw_list);
> list_del_init(&mw->fmr.fm_mr->list);
> - ib_dma_unmap_sg(r_xprt->rx_ia.ri_device,
> - mw->mw_sg, mw->mw_nents, mw->mw_dir);
> + dma_unmap_sg(r_xprt->rx_ia.ri_device->dma_device,
> + mw->mw_sg, mw->mw_nents, mw->mw_dir);
> rpcrdma_put_mw(r_xprt, mw);
> }
>
> diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c
> index 47bed5333c7f..9b7c604669f9 100644
> --- a/net/sunrpc/xprtrdma/frwr_ops.c
> +++ b/net/sunrpc/xprtrdma/frwr_ops.c
> @@ -182,8 +182,8 @@ frwr_op_recover_mr(struct rpcrdma_mw *mw)
>
> rc = __frwr_reset_mr(ia, mw);
> if (state != FRMR_FLUSHED_LI)
> - ib_dma_unmap_sg(ia->ri_device,
> - mw->mw_sg, mw->mw_nents, mw->mw_dir);
> + dma_unmap_sg(ia->ri_device->dma_device, mw->mw_sg, mw->mw_nents,
> + mw->mw_dir);
> if (rc)
> goto out_release;
>
> @@ -396,8 +396,8 @@ frwr_op_map(struct rpcrdma_xprt *r_xprt, struct rpcrdma_mr_seg *seg,
> if (i == 0)
> goto out_dmamap_err;
>
> - dma_nents = ib_dma_map_sg(ia->ri_device,
> - mw->mw_sg, mw->mw_nents, mw->mw_dir);
> + dma_nents = dma_map_sg(ia->ri_device->dma_device, mw->mw_sg,
> + mw->mw_nents, mw->mw_dir);
> if (!dma_nents)
> goto out_dmamap_err;
>
> @@ -538,8 +538,8 @@ frwr_op_unmap_sync(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req)
> dprintk("RPC: %s: DMA unmapping frmr %p\n",
> __func__, &mw->frmr);
> list_del_init(&mw->mw_list);
> - ib_dma_unmap_sg(ia->ri_device,
> - mw->mw_sg, mw->mw_nents, mw->mw_dir);
> + dma_unmap_sg(ia->ri_device->dma_device, mw->mw_sg, mw->mw_nents,
> + mw->mw_dir);
> rpcrdma_put_mw(r_xprt, mw);
> }
> return;
> diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c
> index c52e0f2ffe52..6b6e700a5a68 100644
> --- a/net/sunrpc/xprtrdma/rpc_rdma.c
> +++ b/net/sunrpc/xprtrdma/rpc_rdma.c
> @@ -476,8 +476,8 @@ rpcrdma_prepare_hdr_sge(struct rpcrdma_ia *ia, struct rpcrdma_req *req,
> }
> sge->length = len;
>
> - ib_dma_sync_single_for_device(ia->ri_device, sge->addr,
> - sge->length, DMA_TO_DEVICE);
> + dma_sync_single_for_device(ia->ri_device->dma_device, sge->addr,
> + sge->length, DMA_TO_DEVICE);
> req->rl_send_wr.num_sge++;
> return true;
> }
> @@ -505,8 +505,8 @@ rpcrdma_prepare_msg_sges(struct rpcrdma_ia *ia, struct rpcrdma_req *req,
> sge[sge_no].addr = rdmab_addr(rb);
> sge[sge_no].length = xdr->head[0].iov_len;
> sge[sge_no].lkey = rdmab_lkey(rb);
> - ib_dma_sync_single_for_device(device, sge[sge_no].addr,
> - sge[sge_no].length, DMA_TO_DEVICE);
> + dma_sync_single_for_device(device->dma_device, sge[sge_no].addr,
> + sge[sge_no].length, DMA_TO_DEVICE);
>
> /* If there is a Read chunk, the page list is being handled
> * via explicit RDMA, and thus is skipped here. However, the
> @@ -547,10 +547,11 @@ rpcrdma_prepare_msg_sges(struct rpcrdma_ia *ia, struct rpcrdma_req *req,
> goto out_mapping_overflow;
>
> len = min_t(u32, PAGE_SIZE - page_base, remaining);
> - sge[sge_no].addr = ib_dma_map_page(device, *ppages,
> - page_base, len,
> - DMA_TO_DEVICE);
> - if (ib_dma_mapping_error(device, sge[sge_no].addr))
> + sge[sge_no].addr = dma_map_page(device->dma_device,
> + *ppages, page_base, len,
> + DMA_TO_DEVICE);
> + if (dma_mapping_error(device->dma_device,
> + sge[sge_no].addr))
> goto out_mapping_err;
> sge[sge_no].length = len;
> sge[sge_no].lkey = lkey;
> @@ -574,10 +575,9 @@ rpcrdma_prepare_msg_sges(struct rpcrdma_ia *ia, struct rpcrdma_req *req,
>
> map_tail:
> sge_no++;
> - sge[sge_no].addr = ib_dma_map_page(device, page,
> - page_base, len,
> - DMA_TO_DEVICE);
> - if (ib_dma_mapping_error(device, sge[sge_no].addr))
> + sge[sge_no].addr = dma_map_page(device->dma_device, page,
> + page_base, len, DMA_TO_DEVICE);
> + if (dma_mapping_error(device->dma_device, sge[sge_no].addr))
> goto out_mapping_err;
> sge[sge_no].length = len;
> sge[sge_no].lkey = lkey;
> @@ -628,8 +628,8 @@ rpcrdma_unmap_sges(struct rpcrdma_ia *ia, struct rpcrdma_req *req)
>
> sge = &req->rl_send_sge[2];
> for (count = req->rl_mapped_sges; count--; sge++)
> - ib_dma_unmap_page(device, sge->addr, sge->length,
> - DMA_TO_DEVICE);
> + dma_unmap_page(device->dma_device, sge->addr, sge->length,
> + DMA_TO_DEVICE);
> req->rl_mapped_sges = 0;
> }
>
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c
> index 288e35c2d8f4..d65abb351f5b 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c
> @@ -123,9 +123,10 @@ static int svc_rdma_bc_sendto(struct svcxprt_rdma *rdma,
> ctxt->sge[0].lkey = rdma->sc_pd->local_dma_lkey;
> ctxt->sge[0].length = sndbuf->len;
> ctxt->sge[0].addr =
> - ib_dma_map_page(rdma->sc_cm_id->device, ctxt->pages[0], 0,
> - sndbuf->len, DMA_TO_DEVICE);
> - if (ib_dma_mapping_error(rdma->sc_cm_id->device, ctxt->sge[0].addr)) {
> + dma_map_page(rdma->sc_cm_id->device->dma_device, ctxt->pages[0], 0,
> + sndbuf->len, DMA_TO_DEVICE);
> + if (dma_mapping_error(rdma->sc_cm_id->device->dma_device,
> + ctxt->sge[0].addr)) {
> ret = -EIO;
> goto out_unmap;
> }
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> index 57d35fbb1c28..6ca1786c3493 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
> @@ -151,12 +151,11 @@ int rdma_read_chunk_lcl(struct svcxprt_rdma *xprt,
> rqstp->rq_respages = &rqstp->rq_arg.pages[pg_no+1];
> rqstp->rq_next_page = rqstp->rq_respages + 1;
> ctxt->sge[pno].addr =
> - ib_dma_map_page(xprt->sc_cm_id->device,
> - head->arg.pages[pg_no], pg_off,
> - PAGE_SIZE - pg_off,
> - DMA_FROM_DEVICE);
> - ret = ib_dma_mapping_error(xprt->sc_cm_id->device,
> - ctxt->sge[pno].addr);
> + dma_map_page(xprt->sc_cm_id->device->dma_device,
> + head->arg.pages[pg_no], pg_off,
> + PAGE_SIZE - pg_off, DMA_FROM_DEVICE);
> + ret = dma_mapping_error(xprt->sc_cm_id->device->dma_device,
> + ctxt->sge[pno].addr);
> if (ret)
> goto err;
> svc_rdma_count_mappings(xprt, ctxt);
> @@ -271,9 +270,8 @@ int rdma_read_chunk_frmr(struct svcxprt_rdma *xprt,
> else
> clear_bit(RDMACTXT_F_LAST_CTXT, &ctxt->flags);
>
> - dma_nents = ib_dma_map_sg(xprt->sc_cm_id->device,
> - frmr->sg, frmr->sg_nents,
> - frmr->direction);
> + dma_nents = dma_map_sg(xprt->sc_cm_id->device->dma_device,
> + frmr->sg, frmr->sg_nents, frmr->direction);
> if (!dma_nents) {
> pr_err("svcrdma: failed to dma map sg %p\n",
> frmr->sg);
> @@ -347,8 +345,8 @@ int rdma_read_chunk_frmr(struct svcxprt_rdma *xprt,
> atomic_inc(&rdma_stat_read);
> return ret;
> err:
> - ib_dma_unmap_sg(xprt->sc_cm_id->device,
> - frmr->sg, frmr->sg_nents, frmr->direction);
> + dma_unmap_sg(xprt->sc_cm_id->device->dma_device, frmr->sg,
> + frmr->sg_nents, frmr->direction);
> svc_rdma_put_context(ctxt, 0);
> svc_rdma_put_frmr(xprt, frmr);
> return ret;
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> index ad4d286a83c5..5fa33ad92484 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> @@ -148,8 +148,8 @@ static dma_addr_t dma_map_xdr(struct svcxprt_rdma *xprt,
> page = virt_to_page(xdr->tail[0].iov_base);
> }
> }
> - dma_addr = ib_dma_map_page(xprt->sc_cm_id->device, page, xdr_off,
> - min_t(size_t, PAGE_SIZE, len), dir);
> + dma_addr = dma_map_page(xprt->sc_cm_id->device->dma_device, page,
> + xdr_off, min_t(size_t, PAGE_SIZE, len), dir);
> return dma_addr;
> }
>
> @@ -269,8 +269,8 @@ static int send_write(struct svcxprt_rdma *xprt, struct svc_rqst *rqstp,
> dma_map_xdr(xprt, &rqstp->rq_res, xdr_off,
> sge_bytes, DMA_TO_DEVICE);
> xdr_off += sge_bytes;
> - if (ib_dma_mapping_error(xprt->sc_cm_id->device,
> - sge[sge_no].addr))
> + if (dma_mapping_error(xprt->sc_cm_id->device->dma_device,
> + sge[sge_no].addr))
> goto err;
> svc_rdma_count_mappings(xprt, ctxt);
> sge[sge_no].lkey = xprt->sc_pd->local_dma_lkey;
> @@ -478,9 +478,10 @@ static int send_reply(struct svcxprt_rdma *rdma,
> ctxt->sge[0].lkey = rdma->sc_pd->local_dma_lkey;
> ctxt->sge[0].length = svc_rdma_xdr_get_reply_hdr_len(rdma_resp);
> ctxt->sge[0].addr =
> - ib_dma_map_page(rdma->sc_cm_id->device, page, 0,
> - ctxt->sge[0].length, DMA_TO_DEVICE);
> - if (ib_dma_mapping_error(rdma->sc_cm_id->device, ctxt->sge[0].addr))
> + dma_map_page(rdma->sc_cm_id->device->dma_device, page, 0,
> + ctxt->sge[0].length, DMA_TO_DEVICE);
> + if (dma_mapping_error(rdma->sc_cm_id->device->dma_device,
> + ctxt->sge[0].addr))
> goto err;
> svc_rdma_count_mappings(rdma, ctxt);
>
> @@ -495,8 +496,8 @@ static int send_reply(struct svcxprt_rdma *rdma,
> dma_map_xdr(rdma, &rqstp->rq_res, xdr_off,
> sge_bytes, DMA_TO_DEVICE);
> xdr_off += sge_bytes;
> - if (ib_dma_mapping_error(rdma->sc_cm_id->device,
> - ctxt->sge[sge_no].addr))
> + if (dma_mapping_error(rdma->sc_cm_id->device->dma_device,
> + ctxt->sge[sge_no].addr))
> goto err;
> svc_rdma_count_mappings(rdma, ctxt);
> ctxt->sge[sge_no].lkey = rdma->sc_pd->local_dma_lkey;
> @@ -677,9 +678,10 @@ void svc_rdma_send_error(struct svcxprt_rdma *xprt, struct rpcrdma_msg *rmsgp,
> /* Prepare SGE for local address */
> ctxt->sge[0].lkey = xprt->sc_pd->local_dma_lkey;
> ctxt->sge[0].length = length;
> - ctxt->sge[0].addr = ib_dma_map_page(xprt->sc_cm_id->device,
> - p, 0, length, DMA_TO_DEVICE);
> - if (ib_dma_mapping_error(xprt->sc_cm_id->device, ctxt->sge[0].addr)) {
> + ctxt->sge[0].addr = dma_map_page(xprt->sc_cm_id->device->dma_device,
> + p, 0, length, DMA_TO_DEVICE);
> + if (dma_mapping_error(xprt->sc_cm_id->device->dma_device,
> + ctxt->sge[0].addr)) {
> dprintk("svcrdma: Error mapping buffer for protocol error\n");
> svc_rdma_put_context(ctxt, 1);
> return;
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
> index ca2799af05a6..ed19de5c9a34 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
> @@ -237,10 +237,8 @@ void svc_rdma_unmap_dma(struct svc_rdma_op_ctxt *ctxt)
> * last WR that uses it completes.
> */
> if (ctxt->sge[i].lkey == lkey)
> - ib_dma_unmap_page(device,
> - ctxt->sge[i].addr,
> - ctxt->sge[i].length,
> - ctxt->direction);
> + dma_unmap_page(device->dma_device, ctxt->sge[i].addr,
> + ctxt->sge[i].length, ctxt->direction);
> }
> ctxt->mapped_sges = 0;
> }
> @@ -600,10 +598,9 @@ int svc_rdma_post_recv(struct svcxprt_rdma *xprt, gfp_t flags)
> if (!page)
> goto err_put_ctxt;
> ctxt->pages[sge_no] = page;
> - pa = ib_dma_map_page(xprt->sc_cm_id->device,
> - page, 0, PAGE_SIZE,
> - DMA_FROM_DEVICE);
> - if (ib_dma_mapping_error(xprt->sc_cm_id->device, pa))
> + pa = dma_map_page(xprt->sc_cm_id->device->dma_device,
> + page, 0, PAGE_SIZE, DMA_FROM_DEVICE);
> + if (dma_mapping_error(xprt->sc_cm_id->device->dma_device, pa))
> goto err_put_ctxt;
> svc_rdma_count_mappings(xprt, ctxt);
> ctxt->sge[sge_no].addr = pa;
> @@ -941,8 +938,8 @@ void svc_rdma_put_frmr(struct svcxprt_rdma *rdma,
> struct svc_rdma_fastreg_mr *frmr)
> {
> if (frmr) {
> - ib_dma_unmap_sg(rdma->sc_cm_id->device,
> - frmr->sg, frmr->sg_nents, frmr->direction);
> + dma_unmap_sg(rdma->sc_cm_id->device->dma_device, frmr->sg,
> + frmr->sg_nents, frmr->direction);
> spin_lock_bh(&rdma->sc_frmr_q_lock);
> WARN_ON_ONCE(!list_empty(&frmr->frmr_list));
> list_add(&frmr->frmr_list, &rdma->sc_frmr_q);
> diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
> index 11d07748f699..ba15926173e5 100644
> --- a/net/sunrpc/xprtrdma/verbs.c
> +++ b/net/sunrpc/xprtrdma/verbs.c
> @@ -179,9 +179,9 @@ rpcrdma_wc_receive(struct ib_cq *cq, struct ib_wc *wc)
> rep->rr_wc_flags = wc->wc_flags;
> rep->rr_inv_rkey = wc->ex.invalidate_rkey;
>
> - ib_dma_sync_single_for_cpu(rep->rr_device,
> - rdmab_addr(rep->rr_rdmabuf),
> - rep->rr_len, DMA_FROM_DEVICE);
> + dma_sync_single_for_cpu(rep->rr_device->dma_device,
> + rdmab_addr(rep->rr_rdmabuf),
> + rep->rr_len, DMA_FROM_DEVICE);
>
> rpcrdma_update_granted_credits(rep);
>
> @@ -1259,11 +1259,9 @@ __rpcrdma_dma_map_regbuf(struct rpcrdma_ia *ia, struct rpcrdma_regbuf *rb)
> if (rb->rg_direction == DMA_NONE)
> return false;
>
> - rb->rg_iov.addr = ib_dma_map_single(ia->ri_device,
> - (void *)rb->rg_base,
> - rdmab_length(rb),
> - rb->rg_direction);
> - if (ib_dma_mapping_error(ia->ri_device, rdmab_addr(rb)))
> + rb->rg_iov.addr = dma_map_single(ia->ri_device->dma_device, rb->rg_base,
> + rdmab_length(rb), rb->rg_direction);
> + if (dma_mapping_error(ia->ri_device->dma_device, rdmab_addr(rb)))
> return false;
>
> rb->rg_device = ia->ri_device;
> @@ -1277,8 +1275,8 @@ rpcrdma_dma_unmap_regbuf(struct rpcrdma_regbuf *rb)
> if (!rpcrdma_regbuf_is_mapped(rb))
> return;
>
> - ib_dma_unmap_single(rb->rg_device, rdmab_addr(rb),
> - rdmab_length(rb), rb->rg_direction);
> + dma_unmap_single(rb->rg_device->dma_device, rdmab_addr(rb),
> + rdmab_length(rb), rb->rg_direction);
> rb->rg_device = NULL;
> }
>
> --
> 2.11.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Chuck Lever
^ permalink raw reply [flat|nested] 45+ messages in thread
* [PATCH v2 26/26] IB/core: Remove ib_dma_map_*() functions
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (24 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 25/26] xprtrdma: " Bart Van Assche
@ 2017-01-12 19:07 ` Bart Van Assche
2017-01-13 18:27 ` [PATCH v2 00/26] IB: Optimize DMA mapping Estrin, Alex
2017-01-13 21:26 ` Sagi Grimberg
27 siblings, 0 replies; 45+ messages in thread
From: Bart Van Assche @ 2017-01-12 19:07 UTC (permalink / raw)
To: Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman, Bart Van Assche
Remove the ib_dma_map_*() functions because due to previous patches
these are no longer used.
Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
include/rdma/ib_verbs.h | 218 ------------------------------------------------
1 file changed, 218 deletions(-)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index de8dfb61d2b6..b4b83603b5df 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2912,224 +2912,6 @@ static inline int ib_req_ncomp_notif(struct ib_cq *cq, int wc_cnt)
}
/**
- * ib_dma_mapping_error - check a DMA addr for error
- * @dev: The device for which the dma_addr was created
- * @dma_addr: The DMA address to check
- */
-static inline int ib_dma_mapping_error(struct ib_device *dev, u64 dma_addr)
-{
- return dma_mapping_error(dev->dma_device, dma_addr);
-}
-
-/**
- * ib_dma_map_single - Map a kernel virtual address to DMA address
- * @dev: The device for which the dma_addr is to be created
- * @cpu_addr: The kernel virtual address
- * @size: The size of the region in bytes
- * @direction: The direction of the DMA
- */
-static inline u64 ib_dma_map_single(struct ib_device *dev,
- void *cpu_addr, size_t size,
- enum dma_data_direction direction)
-{
- return dma_map_single(dev->dma_device, cpu_addr, size, direction);
-}
-
-/**
- * ib_dma_unmap_single - Destroy a mapping created by ib_dma_map_single()
- * @dev: The device for which the DMA address was created
- * @addr: The DMA address
- * @size: The size of the region in bytes
- * @direction: The direction of the DMA
- */
-static inline void ib_dma_unmap_single(struct ib_device *dev,
- u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- dma_unmap_single(dev->dma_device, addr, size, direction);
-}
-
-static inline u64 ib_dma_map_single_attrs(struct ib_device *dev,
- void *cpu_addr, size_t size,
- enum dma_data_direction direction,
- unsigned long dma_attrs)
-{
- return dma_map_single_attrs(dev->dma_device, cpu_addr, size,
- direction, dma_attrs);
-}
-
-static inline void ib_dma_unmap_single_attrs(struct ib_device *dev,
- u64 addr, size_t size,
- enum dma_data_direction direction,
- unsigned long dma_attrs)
-{
- return dma_unmap_single_attrs(dev->dma_device, addr, size,
- direction, dma_attrs);
-}
-
-/**
- * ib_dma_map_page - Map a physical page to DMA address
- * @dev: The device for which the dma_addr is to be created
- * @page: The page to be mapped
- * @offset: The offset within the page
- * @size: The size of the region in bytes
- * @direction: The direction of the DMA
- */
-static inline u64 ib_dma_map_page(struct ib_device *dev,
- struct page *page,
- unsigned long offset,
- size_t size,
- enum dma_data_direction direction)
-{
- return dma_map_page(dev->dma_device, page, offset, size, direction);
-}
-
-/**
- * ib_dma_unmap_page - Destroy a mapping created by ib_dma_map_page()
- * @dev: The device for which the DMA address was created
- * @addr: The DMA address
- * @size: The size of the region in bytes
- * @direction: The direction of the DMA
- */
-static inline void ib_dma_unmap_page(struct ib_device *dev,
- u64 addr, size_t size,
- enum dma_data_direction direction)
-{
- dma_unmap_page(dev->dma_device, addr, size, direction);
-}
-
-/**
- * ib_dma_map_sg - Map a scatter/gather list to DMA addresses
- * @dev: The device for which the DMA addresses are to be created
- * @sg: The array of scatter/gather entries
- * @nents: The number of scatter/gather entries
- * @direction: The direction of the DMA
- */
-static inline int ib_dma_map_sg(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction)
-{
- return dma_map_sg(dev->dma_device, sg, nents, direction);
-}
-
-/**
- * ib_dma_unmap_sg - Unmap a scatter/gather list of DMA addresses
- * @dev: The device for which the DMA addresses were created
- * @sg: The array of scatter/gather entries
- * @nents: The number of scatter/gather entries
- * @direction: The direction of the DMA
- */
-static inline void ib_dma_unmap_sg(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction)
-{
- dma_unmap_sg(dev->dma_device, sg, nents, direction);
-}
-
-static inline int ib_dma_map_sg_attrs(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction,
- unsigned long dma_attrs)
-{
- return dma_map_sg_attrs(dev->dma_device, sg, nents, direction,
- dma_attrs);
-}
-
-static inline void ib_dma_unmap_sg_attrs(struct ib_device *dev,
- struct scatterlist *sg, int nents,
- enum dma_data_direction direction,
- unsigned long dma_attrs)
-{
- dma_unmap_sg_attrs(dev->dma_device, sg, nents, direction, dma_attrs);
-}
-/**
- * ib_sg_dma_address - Return the DMA address from a scatter/gather entry
- * @dev: The device for which the DMA addresses were created
- * @sg: The scatter/gather entry
- *
- * Note: this function is obsolete. To do: change all occurrences of
- * ib_sg_dma_address() into sg_dma_address().
- */
-static inline u64 ib_sg_dma_address(struct ib_device *dev,
- struct scatterlist *sg)
-{
- return sg_dma_address(sg);
-}
-
-/**
- * ib_sg_dma_len - Return the DMA length from a scatter/gather entry
- * @dev: The device for which the DMA addresses were created
- * @sg: The scatter/gather entry
- *
- * Note: this function is obsolete. To do: change all occurrences of
- * ib_sg_dma_len() into sg_dma_len().
- */
-static inline unsigned int ib_sg_dma_len(struct ib_device *dev,
- struct scatterlist *sg)
-{
- return sg_dma_len(sg);
-}
-
-/**
- * ib_dma_sync_single_for_cpu - Prepare DMA region to be accessed by CPU
- * @dev: The device for which the DMA address was created
- * @addr: The DMA address
- * @size: The size of the region in bytes
- * @dir: The direction of the DMA
- */
-static inline void ib_dma_sync_single_for_cpu(struct ib_device *dev,
- u64 addr,
- size_t size,
- enum dma_data_direction dir)
-{
- dma_sync_single_for_cpu(dev->dma_device, addr, size, dir);
-}
-
-/**
- * ib_dma_sync_single_for_device - Prepare DMA region to be accessed by device
- * @dev: The device for which the DMA address was created
- * @addr: The DMA address
- * @size: The size of the region in bytes
- * @dir: The direction of the DMA
- */
-static inline void ib_dma_sync_single_for_device(struct ib_device *dev,
- u64 addr,
- size_t size,
- enum dma_data_direction dir)
-{
- dma_sync_single_for_device(dev->dma_device, addr, size, dir);
-}
-
-/**
- * ib_dma_alloc_coherent - Allocate memory and map it for DMA
- * @dev: The device for which the DMA address is requested
- * @size: The size of the region to allocate in bytes
- * @dma_handle: A pointer for returning the DMA address of the region
- * @flag: memory allocator flags
- */
-static inline void *ib_dma_alloc_coherent(struct ib_device *dev,
- size_t size,
- dma_addr_t *dma_handle,
- gfp_t flag)
-{
- return dma_alloc_coherent(dev->dma_device, size, dma_handle, flag);
-}
-
-/**
- * ib_dma_free_coherent - Free memory allocated by ib_dma_alloc_coherent()
- * @dev: The device for which the DMA addresses were allocated
- * @size: The size of the region
- * @cpu_addr: the address returned by ib_dma_alloc_coherent()
- * @dma_handle: the DMA address returned by ib_dma_alloc_coherent()
- */
-static inline void ib_dma_free_coherent(struct ib_device *dev,
- size_t size, void *cpu_addr,
- dma_addr_t dma_handle)
-{
- dma_free_coherent(dev->dma_device, size, cpu_addr, dma_handle);
-}
-
-/**
* ib_dereg_mr - Deregisters a memory region and removes it from the
* HCA translation table.
* @mr: The memory region to deregister.
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* RE: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (25 preceding siblings ...)
2017-01-12 19:07 ` [PATCH v2 26/26] IB/core: Remove " Bart Van Assche
@ 2017-01-13 18:27 ` Estrin, Alex
2017-01-13 20:18 ` Bart Van Assche
2017-01-13 21:26 ` Sagi Grimberg
27 siblings, 1 reply; 45+ messages in thread
From: Estrin, Alex @ 2017-01-13 18:27 UTC (permalink / raw)
To: Bart Van Assche, Doug Ledford
Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
Greg Kroah-Hartman
Hi Bart,
This series applied to 4.10-rc3 breaks hfi1 sdma engines.
[ 34.712343] hfi1 0000:82:00.0: hfi1_0: SDMA (0) engine error: 0x21 state s50_HwHaltWait
[ 34.722752] hfi1 0000:82:00.0: hfi1_0: SDMA (0) descq_head: 0 descq_tail: 3 freecnt: 2044 FLE 0
[ 34.733933] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[0]: flags:--F- addr:0x00008808248c8000 gen:0 len:36 bytes
[ 34.746595] hfi1 0000:82:00.0: hfi1_0: desc0:0x80248808248c8000 desc1 0x0000000000000000
[ 34.757218] hfi1 0000:82:00.0: hfi1_0: aidx: 0 amode: 0 alen: 0
[ 34.765448] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[1]: flags:---- addr:0x0000880829fd5000 gen:0 len:24 bytes
[ 34.778184] hfi1 0000:82:00.0: hfi1_0: desc0:0x0018880829fd5000 desc1 0x0000000000000000
[ 34.788941] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[2]: flags:-H-L addr:0x0000880829fd5018 gen:0 len:440 bytes
[ 34.801849] hfi1 0000:82:00.0: hfi1_0: desc0:0x41b8880829fd5018 desc1 0x0000000000000001
...etc
BTW, what was the base for this series, as 19/26 did not apply cleanly to 4.10-rc3?
Thanks,
Alex.
> -----Original Message-----
> From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-
> owner@vger.kernel.org] On Behalf Of Bart Van Assche
> Sent: Thursday, January 12, 2017 2:07 PM
> To: Doug Ledford <dledford@redhat.com>
> Cc: linux-rdma@vger.kernel.org; linux-kernel@vger.kernel.org; Greg Kroah-Hartman
> <gregkh@linuxfoundation.org>; Bart Van Assche <bart.vanassche@sandisk.com>
> Subject: [PATCH v2 00/26] IB: Optimize DMA mapping
>
> Hello Doug,
>
> As you know there are two sets of DMA mapping operations in the Linux
> kernel:
> - One set of DMA mapping operations that is used by most drivers.
> - Another set of DMA mapping operations that is only used by the RDMA
> drivers.
> Having two sets of DMA mapping operations is not only a source of
> confusion but also a source of unnecessary overhead. The DMA mapping
> operations are in the hot path so it is important that the overhead
> of these operations is as low as possible. Hence this patch series
> that converts the RDMA code to the standard DMA mapping API and
> thereby eliminates the if (dev->dma_ops) test from the hot path. An
> additional benefit is that the size of HW and SW drivers that do not
> use DMA is reduced by switching to dma_virt_ops.
>
> The changes compared to version 1 of this patch series are:
> - Patch "Move dma_ops from archdata into struct device" has been
> split into three patches.
> - Patch "treewide: Inline ib_dma_map_*() functions" has been split
> into 15 patches (one per driver).
> - A patch has been added that builds dma_noop_ops only for the
> architectures that need it.
> - The new dma_virt_ops is only built if it is used by a driver.
> - In these last 15 patches indentation has been adjusted to keep
> the arguments aligned with the opening parenthesis.
>
> Bart Van Assche (26):
> treewide: Constify most dma_map_ops structures
> treewide: Move dma_ops from struct dev_archdata into struct device
> treewide: Consolidate set_dma_ops() implementations
> treewide: Consolidate get_dma_ops() implementations
> lib/dma-noop: Clarify a comment
> lib/dma-noop: Only build dma_noop_ops for m32r and s390
> lib/dma-virt: Add dma_virt_ops
> IB/hf1: Remove DMA mapping code
> IB/qib: Remove DMA mapping code
> IB: Use dma_virt_ops instead of duplicating it
> RDS: IB: Remove an unused structure member
> IB: Convert ib_dma_*_coherent() argument type from u64 into dma_addr_t
> IB/core: Inline ib_dma_map_*() functions
> IB/mlx4: Inline ib_dma_map_*() functions
> IB/mlx5: Inline ib_dma_map_*() functions
> IB/IPoIB: Inline ib_dma_map_*() functions
> IB/iser: Inline ib_dma_map_*() functions
> IB/isert: Inline ib_dma_map_*() functions
> IB/srp: Inline ib_dma_map_*() functions
> IB/srpt: Inline ib_dma_map_*() functions
> staging/lustre: Inline ib_dma_map_*() functions
> nvme-rdma: Inline ib_dma_map_*() functions
> net/9p: Inline ib_dma_map_*() functions
> net/rds: Inline ib_dma_map_*() functions
> xprtrdma: Inline ib_dma_map_*() functions
> IB/core: Remove ib_dma_map_*() functions
>
> arch/alpha/include/asm/dma-mapping.h | 4 +-
> arch/alpha/kernel/pci-noop.c | 4 +-
> arch/alpha/kernel/pci_iommu.c | 4 +-
> arch/arc/include/asm/dma-mapping.h | 4 +-
> arch/arc/mm/dma.c | 2 +-
> arch/arm/common/dmabounce.c | 2 +-
> arch/arm/include/asm/device.h | 1 -
> arch/arm/include/asm/dma-mapping.h | 20 +-
> arch/arm/mm/dma-mapping.c | 22 +-
> arch/arm/xen/mm.c | 4 +-
> arch/arm64/include/asm/device.h | 1 -
> arch/arm64/include/asm/dma-mapping.h | 12 +-
> arch/arm64/mm/dma-mapping.c | 14 +-
> arch/avr32/include/asm/dma-mapping.h | 4 +-
> arch/avr32/mm/dma-coherent.c | 2 +-
> arch/blackfin/include/asm/dma-mapping.h | 4 +-
> arch/blackfin/kernel/dma-mapping.c | 2 +-
> arch/c6x/include/asm/dma-mapping.h | 4 +-
> arch/c6x/kernel/dma.c | 2 +-
> arch/cris/arch-v32/drivers/pci/dma.c | 2 +-
> arch/cris/include/asm/dma-mapping.h | 6 +-
> arch/frv/include/asm/dma-mapping.h | 4 +-
> arch/frv/mb93090-mb00/pci-dma-nommu.c | 2 +-
> arch/frv/mb93090-mb00/pci-dma.c | 2 +-
> arch/h8300/include/asm/dma-mapping.h | 4 +-
> arch/h8300/kernel/dma.c | 2 +-
> arch/hexagon/include/asm/dma-mapping.h | 7 +-
> arch/hexagon/kernel/dma.c | 4 +-
> arch/ia64/hp/common/hwsw_iommu.c | 4 +-
> arch/ia64/hp/common/sba_iommu.c | 4 +-
> arch/ia64/include/asm/dma-mapping.h | 7 +-
> arch/ia64/include/asm/machvec.h | 4 +-
> arch/ia64/kernel/dma-mapping.c | 4 +-
> arch/ia64/kernel/pci-dma.c | 10 +-
> arch/ia64/kernel/pci-swiotlb.c | 2 +-
> arch/m32r/Kconfig | 1 +
> arch/m32r/include/asm/device.h | 1 -
> arch/m32r/include/asm/dma-mapping.h | 4 +-
> arch/m68k/include/asm/dma-mapping.h | 4 +-
> arch/m68k/kernel/dma.c | 2 +-
> arch/metag/include/asm/dma-mapping.h | 4 +-
> arch/metag/kernel/dma.c | 2 +-
> arch/microblaze/include/asm/dma-mapping.h | 4 +-
> arch/microblaze/kernel/dma.c | 2 +-
> arch/mips/cavium-octeon/dma-octeon.c | 4 +-
> arch/mips/include/asm/device.h | 5 -
> arch/mips/include/asm/dma-mapping.h | 9 +-
> .../include/asm/mach-cavium-octeon/dma-coherence.h | 2 +-
> arch/mips/include/asm/netlogic/common.h | 2 +-
> arch/mips/loongson64/common/dma-swiotlb.c | 2 +-
> arch/mips/mm/dma-default.c | 4 +-
> arch/mips/netlogic/common/nlm-dma.c | 2 +-
> arch/mips/pci/pci-octeon.c | 2 +-
> arch/mn10300/include/asm/dma-mapping.h | 4 +-
> arch/mn10300/mm/dma-alloc.c | 2 +-
> arch/nios2/include/asm/dma-mapping.h | 4 +-
> arch/nios2/mm/dma-mapping.c | 2 +-
> arch/openrisc/include/asm/dma-mapping.h | 4 +-
> arch/openrisc/kernel/dma.c | 2 +-
> arch/parisc/include/asm/dma-mapping.h | 8 +-
> arch/parisc/kernel/drivers.c | 2 +-
> arch/parisc/kernel/pci-dma.c | 4 +-
> arch/powerpc/include/asm/device.h | 4 -
> arch/powerpc/include/asm/dma-mapping.h | 14 +-
> arch/powerpc/include/asm/pci.h | 4 +-
> arch/powerpc/include/asm/ps3.h | 2 +-
> arch/powerpc/include/asm/swiotlb.h | 2 +-
> arch/powerpc/kernel/dma-swiotlb.c | 2 +-
> arch/powerpc/kernel/dma.c | 8 +-
> arch/powerpc/kernel/pci-common.c | 6 +-
> arch/powerpc/platforms/cell/iommu.c | 6 +-
> arch/powerpc/platforms/pasemi/iommu.c | 2 +-
> arch/powerpc/platforms/pasemi/setup.c | 2 +-
> arch/powerpc/platforms/powernv/npu-dma.c | 2 +-
> arch/powerpc/platforms/ps3/system-bus.c | 8 +-
> arch/powerpc/platforms/pseries/ibmebus.c | 4 +-
> arch/powerpc/platforms/pseries/vio.c | 2 +-
> arch/s390/Kconfig | 1 +
> arch/s390/include/asm/device.h | 1 -
> arch/s390/include/asm/dma-mapping.h | 6 +-
> arch/s390/pci/pci.c | 2 +-
> arch/s390/pci/pci_dma.c | 2 +-
> arch/sh/include/asm/dma-mapping.h | 4 +-
> arch/sh/kernel/dma-nommu.c | 2 +-
> arch/sh/mm/consistent.c | 2 +-
> arch/sparc/include/asm/dma-mapping.h | 10 +-
> arch/sparc/kernel/iommu.c | 4 +-
> arch/sparc/kernel/ioport.c | 8 +-
> arch/sparc/kernel/pci_sun4v.c | 2 +-
> arch/tile/include/asm/device.h | 3 -
> arch/tile/include/asm/dma-mapping.h | 20 +-
> arch/tile/kernel/pci-dma.c | 24 +-
> arch/unicore32/include/asm/dma-mapping.h | 4 +-
> arch/unicore32/mm/dma-swiotlb.c | 2 +-
> arch/x86/include/asm/device.h | 5 +-
> arch/x86/include/asm/dma-mapping.h | 11 +-
> arch/x86/include/asm/iommu.h | 2 +-
> arch/x86/kernel/amd_gart_64.c | 2 +-
> arch/x86/kernel/pci-calgary_64.c | 6 +-
> arch/x86/kernel/pci-dma.c | 4 +-
> arch/x86/kernel/pci-nommu.c | 2 +-
> arch/x86/kernel/pci-swiotlb.c | 2 +-
> arch/x86/pci/common.c | 2 +-
> arch/x86/pci/sta2x11-fixup.c | 10 +-
> arch/x86/xen/pci-swiotlb-xen.c | 2 +-
> arch/xtensa/include/asm/device.h | 4 -
> arch/xtensa/include/asm/dma-mapping.h | 9 +-
> arch/xtensa/kernel/pci-dma.c | 2 +-
> drivers/infiniband/core/mad.c | 79 +++---
> drivers/infiniband/core/rw.c | 30 +-
> drivers/infiniband/core/umem.c | 13 +-
> drivers/infiniband/core/umem_odp.c | 12 +-
> drivers/infiniband/hw/hfi1/dma.c | 183 ------------
> drivers/infiniband/hw/mlx4/cq.c | 8 +-
> drivers/infiniband/hw/mlx4/mad.c | 79 +++---
> drivers/infiniband/hw/mlx4/mr.c | 8 +-
> drivers/infiniband/hw/mlx4/qp.c | 29 +-
> drivers/infiniband/hw/mlx5/mr.c | 12 +-
> drivers/infiniband/hw/qib/qib_dma.c | 169 -----------
> drivers/infiniband/hw/qib/qib_keys.c | 5 +-
> drivers/infiniband/sw/rdmavt/Kconfig | 1 +
> drivers/infiniband/sw/rdmavt/Makefile | 2 +-
> drivers/infiniband/sw/rdmavt/dma.c | 198 -------------
> drivers/infiniband/sw/rdmavt/dma.h | 53 ----
> drivers/infiniband/sw/rdmavt/mr.c | 8 +-
> drivers/infiniband/sw/rdmavt/vt.c | 5 +-
> drivers/infiniband/sw/rdmavt/vt.h | 1 -
> drivers/infiniband/sw/rxe/Kconfig | 1 +
> drivers/infiniband/sw/rxe/Makefile | 1 -
> drivers/infiniband/sw/rxe/rxe_dma.c | 183 ------------
> drivers/infiniband/sw/rxe/rxe_loc.h | 2 -
> drivers/infiniband/sw/rxe/rxe_verbs.c | 3 +-
> drivers/infiniband/ulp/ipoib/ipoib_cm.c | 34 ++-
> drivers/infiniband/ulp/ipoib/ipoib_ib.c | 42 +--
> drivers/infiniband/ulp/iser/iscsi_iser.c | 11 +-
> drivers/infiniband/ulp/iser/iser_initiator.c | 88 +++---
> drivers/infiniband/ulp/iser/iser_memory.c | 13 +-
> drivers/infiniband/ulp/iser/iser_verbs.c | 6 +-
> drivers/infiniband/ulp/isert/ib_isert.c | 120 ++++----
> drivers/infiniband/ulp/srp/ib_srp.c | 80 +++---
> drivers/infiniband/ulp/srpt/ib_srpt.c | 18 +-
> drivers/iommu/amd_iommu.c | 10 +-
> drivers/misc/mic/bus/mic_bus.c | 4 +-
> drivers/misc/mic/bus/scif_bus.c | 4 +-
> drivers/misc/mic/bus/scif_bus.h | 2 +-
> drivers/misc/mic/bus/vop_bus.c | 2 +-
> drivers/misc/mic/host/mic_boot.c | 4 +-
> drivers/nvme/host/rdma.c | 35 +--
> drivers/nvme/target/rdma.c | 32 +--
> drivers/parisc/ccio-dma.c | 2 +-
> drivers/parisc/sba_iommu.c | 2 +-
> drivers/pci/host/vmd.c | 2 +-
> .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h | 14 +-
> include/linux/device.h | 1 +
> include/linux/dma-mapping.h | 55 ++--
> include/linux/mic_bus.h | 2 +-
> include/rdma/ib_verbs.h | 310 ---------------------
> include/xen/arm/hypervisor.h | 2 +-
> lib/Kconfig | 10 +
> lib/Makefile | 3 +-
> lib/dma-noop.c | 4 +-
> lib/dma-virt.c | 73 +++++
> net/9p/trans_rdma.c | 24 +-
> net/rds/ib.h | 45 +--
> net/rds/ib_cm.c | 42 +--
> net/rds/ib_fmr.c | 10 +-
> net/rds/ib_frmr.c | 12 +-
> net/rds/ib_mr.h | 1 -
> net/rds/ib_rdma.c | 13 +-
> net/rds/ib_recv.c | 19 +-
> net/rds/ib_send.c | 50 ++--
> net/sunrpc/xprtrdma/fmr_ops.c | 12 +-
> net/sunrpc/xprtrdma/frwr_ops.c | 12 +-
> net/sunrpc/xprtrdma/rpc_rdma.c | 28 +-
> net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 7 +-
> net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 20 +-
> net/sunrpc/xprtrdma/svc_rdma_sendto.c | 26 +-
> net/sunrpc/xprtrdma/svc_rdma_transport.c | 17 +-
> net/sunrpc/xprtrdma/verbs.c | 18 +-
> 179 files changed, 911 insertions(+), 2011 deletions(-)
> delete mode 100644 drivers/infiniband/hw/hfi1/dma.c
> delete mode 100644 drivers/infiniband/hw/qib/qib_dma.c
> delete mode 100644 drivers/infiniband/sw/rdmavt/dma.c
> delete mode 100644 drivers/infiniband/sw/rdmavt/dma.h
> delete mode 100644 drivers/infiniband/sw/rxe/rxe_dma.c
> create mode 100644 lib/dma-virt.c
>
> --
> 2.11.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-13 18:27 ` [PATCH v2 00/26] IB: Optimize DMA mapping Estrin, Alex
@ 2017-01-13 20:18 ` Bart Van Assche
2017-01-13 21:08 ` Estrin, Alex
0 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-13 20:18 UTC (permalink / raw)
To: alex.estrin@intel.com, dledford@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
gregkh@linuxfoundation.org
On Fri, 2017-01-13 at 18:27 +0000, Estrin, Alex wrote:
> This series applied to 4.10-rc3 breaks hfi1 sdma engines.
> [ 34.712343] hfi1 0000:82:00.0: hfi1_0: SDMA (0) engine error: 0x21
> state s50_HwHaltWait
> [ 34.722752] hfi1 0000:82:00.0: hfi1_0: SDMA (0) descq_head: 0
> descq_tail: 3 freecnt: 2044 FLE 0
> [ 34.733933] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[0]: flags:--F-
> addr:0x00008808248c8000 gen:0 len:36 bytes
> [ 34.746595] hfi1 0000:82:00.0: hfi1_0: desc0:0x80248808248c8000
> desc1 0x0000000000000000
> [ 34.757218] hfi1 0000:82:00.0: hfi1_0: aidx: 0 amode: 0 alen: 0
> [ 34.765448] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[1]: flags:----
> addr:0x0000880829fd5000 gen:0 len:24 bytes
> [ 34.778184] hfi1 0000:82:00.0: hfi1_0: desc0:0x0018880829fd5000
> desc1 0x0000000000000000
> [ 34.788941] hfi1 0000:82:00.0: hfi1_0: SDMA sdmadesc[2]: flags:-H-L
> addr:0x0000880829fd5018 gen:0 len:440 bytes
> [ 34.801849] hfi1 0000:82:00.0: hfi1_0: desc0:0x41b8880829fd5018
> desc1 0x0000000000000001
> ...etc
>
> BTW, what was the base for this series, as 19/26 did not apply cleanly to
> 4.10-rc3?
Hello Alex,
Sorry for this. Can you check whether the patch below fixes this regression?
diff --git a/drivers/infiniband/sw/rdmavt/vt.c
b/drivers/infiniband/sw/rdmavt/vt.c
index 6a81b179f631..f7fcd015ec09 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -778,8 +778,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
}
/* DMA Operations */
- if (rdi->ibdev.dma_device->dma_ops == NULL)
- set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
+ set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
/* Protection Domain */
spin_lock_init(&rdi->n_pds_lock);
--
2.11.0
I will make sure that this series applies cleanly on top of v4.10-rc3 when I
repost it.
Bart.
^ permalink raw reply related [flat|nested] 45+ messages in thread
* RE: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-13 20:18 ` Bart Van Assche
@ 2017-01-13 21:08 ` Estrin, Alex
2017-01-13 21:59 ` Bart Van Assche
0 siblings, 1 reply; 45+ messages in thread
From: Estrin, Alex @ 2017-01-13 21:08 UTC (permalink / raw)
To: Bart Van Assche, dledford@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
gregkh@linuxfoundation.org
>
> Hello Alex,
>
> Sorry for this. Can you check whether the patch below fixes this regression?
>
> diff --git a/drivers/infiniband/sw/rdmavt/vt.c
> b/drivers/infiniband/sw/rdmavt/vt.c
> index 6a81b179f631..f7fcd015ec09 100644
> --- a/drivers/infiniband/sw/rdmavt/vt.c
> +++ b/drivers/infiniband/sw/rdmavt/vt.c
> @@ -778,8 +778,7 @@ int rvt_register_device(struct rvt_dev_info *rdi)
> }
>
> /* DMA Operations */
> - if (rdi->ibdev.dma_device->dma_ops == NULL)
> - set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
> + set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
>
> /* Protection Domain */
> spin_lock_init(&rdi->n_pds_lock);
> --
> 2.11.0
>
> I will make sure that this series applies cleanly on top of v4.10-rc3 when I
> repost it.
>
> Bart.
Hi Bart,
It didn't fix the failure... Apparently there is an issue with generic mapping itself.
Thanks,
Alex.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-13 21:08 ` Estrin, Alex
@ 2017-01-13 21:59 ` Bart Van Assche
2017-01-14 2:05 ` Estrin, Alex
0 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-13 21:59 UTC (permalink / raw)
To: alex.estrin@intel.com, dledford@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
gregkh@linuxfoundation.org
On Fri, 2017-01-13 at 21:08 +0000, Estrin, Alex wrote:
> It didn't fix the failure... Apparently there is an issue with generic
> mapping itself.
Hello Alex,
The generic mapping code in lib/dma-virt.c works fine with at least the rxe
driver. Additionally, as far as I can see the mapping code in lib/dma-virt.c
is identical to the mapping in drivers/infiniband/sw/rdmavt/dma.c. What is
not clear to me however is how my patches can have caused the SDMA engine
error messages? As far as I can see the code in
drivers/infiniband/hw/hfi1/sdma.c uses PCI DMA mapping operations instead of
the generic dma_virt_ops. Is that correct?
Thanks,
Bart.
^ permalink raw reply [flat|nested] 45+ messages in thread
* RE: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-13 21:59 ` Bart Van Assche
@ 2017-01-14 2:05 ` Estrin, Alex
2017-01-17 21:48 ` Bart Van Assche
0 siblings, 1 reply; 45+ messages in thread
From: Estrin, Alex @ 2017-01-14 2:05 UTC (permalink / raw)
To: Bart Van Assche, dledford@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
gregkh@linuxfoundation.org
> -----Original Message-----
> From: Bart Van Assche [mailto:Bart.VanAssche@sandisk.com]
> Sent: Friday, January 13, 2017 5:00 PM
> To: Estrin, Alex <alex.estrin@intel.com>; dledford@redhat.com
> Cc: linux-kernel@vger.kernel.org; linux-rdma@vger.kernel.org;
> gregkh@linuxfoundation.org
> Subject: Re: [PATCH v2 00/26] IB: Optimize DMA mapping
>
> On Fri, 2017-01-13 at 21:08 +0000, Estrin, Alex wrote:
> > It didn't fix the failure... Apparently there is an issue with generic
> > mapping itself.
>
> Hello Alex,
>
> The generic mapping code in lib/dma-virt.c works fine with at least the rxe
> driver. Additionally, as far as I can see the mapping code in lib/dma-virt.c
> is identical to the mapping in drivers/infiniband/sw/rdmavt/dma.c. What is
> not clear to me however is how my patches can have caused the SDMA engine
> error messages? As far as I can see the code in
> drivers/infiniband/hw/hfi1/sdma.c uses PCI DMA mapping operations instead of
> the generic dma_virt_ops. Is that correct?
Hi Bart,
If you refer to code like this
from sdma.c:
/* dma map the coalesce buffer */
addr = dma_map_single(&dd->pcidev->dev,
.....
please see hfi1/verbs.c @ hfi1_register_ib_device()
'dma_device' field is initialized here
-> ibdev->dma_device = &dd->pcidev->dev;
followed by
ret = rvt_register_device(&dd->verbs_dev.rdi);
-> set_dma_ops(rdi->ibdev.dma_device, &dma_virt_ops);
Thanks,
Alex.
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-14 2:05 ` Estrin, Alex
@ 2017-01-17 21:48 ` Bart Van Assche
2017-01-17 22:00 ` Bart Van Assche
0 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-17 21:48 UTC (permalink / raw)
To: alex.estrin@intel.com, dledford@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
gregkh@linuxfoundation.org
[-- Attachment #1: Type: text/plain, Size: 638 bytes --]
On Sat, 2017-01-14 at 02:05 +0000, Estrin, Alex wrote:
> [ ... ]
> please see hfi1/verbs.c @ hfi1_register_ib_device()
> [ ... ]
Hello Alex,
I think I figured out what I did wrong: both the hfi1 and the qib drivers need two
sets of DMA mapping operations. ULPs have to use &dma_virt_ops and the SDMA code
has to use the PCIe DMA mapping operations. My patch series made the SDMA code use
dma_virt_ops and that's wrong. The attached patch should fix this. Unfortunately I
do not have access to a hfi1 or qib test setup. Can you help me by testing the
attached patch on top of the already posted patches?
Thanks,
Bart.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-hfi1-qib-rdmavt-Use-proper-DMA-mapping-operations.patch --]
[-- Type: text/x-patch; name="0001-hfi1-qib-rdmavt-Use-proper-DMA-mapping-operations.patch", Size: 3583 bytes --]
From 21ed4680a16b2281ea63caa9a3ef9e4c682a1d8b Mon Sep 17 00:00:00 2001
From: Bart Van Assche <bart.vanassche@sandisk.com>
Date: Tue, 17 Jan 2017 13:22:35 -0800
Subject: [PATCH] hfi1, qib, rdmavt: Use proper DMA mapping operations
Make users of the hfi1 and qib drivers use dma_virt_ops. Use the PCIe
mapping operations for SDMA. Make sure that the rdmavt code does not
change the PCIe DMA mapping operations pointer into &dma_virt_ops.
---
drivers/infiniband/hw/hfi1/mad.c | 2 +-
drivers/infiniband/hw/hfi1/verbs.c | 1 -
drivers/infiniband/hw/qib/qib_verbs.c | 1 -
drivers/infiniband/sw/rdmavt/vt.c | 2 ++
include/rdma/rdma_vt.h | 8 ++++++++
5 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/mad.c b/drivers/infiniband/hw/hfi1/mad.c
index 6e595afca24c..041d503c399b 100644
--- a/drivers/infiniband/hw/hfi1/mad.c
+++ b/drivers/infiniband/hw/hfi1/mad.c
@@ -4406,7 +4406,7 @@ int hfi1_process_mad(struct ib_device *ibdev, int mad_flags, u8 port,
switch (in_mad->base_version) {
case OPA_MGMT_BASE_VERSION:
if (unlikely(in_mad_size != sizeof(struct opa_mad))) {
- dev_err(ibdev->dma_device, "invalid in_mad_size\n");
+ dev_err(&ibdev->dev, "invalid in_mad_size\n");
return IB_MAD_RESULT_FAILURE;
}
return hfi1_process_opa_mad(ibdev, mad_flags, port,
diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c
index 95ed4d6da510..fa0ff3e11597 100644
--- a/drivers/infiniband/hw/hfi1/verbs.c
+++ b/drivers/infiniband/hw/hfi1/verbs.c
@@ -1784,7 +1784,6 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd)
strlcpy(ibdev->name + lcpysz, "_%d", IB_DEVICE_NAME_MAX - lcpysz);
ibdev->owner = THIS_MODULE;
ibdev->phys_port_cnt = dd->num_pports;
- ibdev->dma_device = &dd->pcidev->dev;
ibdev->modify_device = modify_device;
ibdev->alloc_hw_stats = alloc_hw_stats;
ibdev->get_hw_stats = get_hw_stats;
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c b/drivers/infiniband/hw/qib/qib_verbs.c
index 4b54c0ddd08a..daa28a7b7574 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -1632,7 +1632,6 @@ int qib_register_ib_device(struct qib_devdata *dd)
ibdev->owner = THIS_MODULE;
ibdev->node_guid = ppd->guid;
ibdev->phys_port_cnt = dd->num_pports;
- ibdev->dma_device = &dd->pcidev->dev;
ibdev->modify_device = qib_modify_device;
ibdev->process_mad = qib_process_mad;
diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c
index 6a81b179f631..af9dcd47dace 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -103,6 +103,8 @@ struct rvt_dev_info *rvt_alloc_device(size_t size, int nports)
if (!rdi->ports)
ib_dealloc_device(&rdi->ibdev);
+ rdi->ib_dev.dma_device = &rdi->dma_device;
+
return rdi;
}
EXPORT_SYMBOL(rvt_alloc_device);
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 861e23eaebda..b7259c9b056e 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -339,6 +339,14 @@ struct rvt_driver_provided {
struct rvt_dev_info {
struct ib_device ibdev; /* Keep this first. Nothing above here */
+ /*
+ * qib and hfi1 use two sets of DMA operations:
+ * - The DMA operations of the PCIe device for SDMA.
+ * - dma_virt_ops for users of the qib and hfi1 drivers.
+ * The only purpose of @dma_device is to provide a struct device that
+ * provides dma_virt_ops.
+ */
+ struct device dma_device;
/*
* Prior to calling for registration the driver will be responsible for
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-17 21:48 ` Bart Van Assche
@ 2017-01-17 22:00 ` Bart Van Assche
2017-01-17 22:27 ` Jason Gunthorpe
0 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-17 22:00 UTC (permalink / raw)
To: alex.estrin@intel.com, dledford@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
gregkh@linuxfoundation.org
[-- Attachment #1: Type: text/plain, Size: 867 bytes --]
On Tue, 2017-01-17 at 13:48 -0800, Bart Van Assche wrote:
> On Sat, 2017-01-14 at 02:05 +0000, Estrin, Alex wrote:
> > [ ... ]
> > please see hfi1/verbs.c @ hfi1_register_ib_device()
> > [ ... ]
>
> Hello Alex,
>
> I think I figured out what I did wrong: both the hfi1 and the qib drivers need two
> sets of DMA mapping operations. ULPs have to use &dma_virt_ops and the SDMA code
> has to use the PCIe DMA mapping operations. My patch series made the SDMA code use
> dma_virt_ops and that's wrong. The attached patch should fix this. Unfortunately I
> do not have access to a hfi1 or qib test setup. Can you help me by testing the
> attached patch on top of the already posted patches?
Hello Alex,
The wrong version of my patch was attached to previous e-mail. Sorry for this. Please
use the patch attached to this e-mail.
Thanks,
Bart.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-hfi1-qib-rdmavt-v2-Use-proper-DMA-mapping-operations.patch --]
[-- Type: text/x-patch; name="0001-hfi1-qib-rdmavt-v2-Use-proper-DMA-mapping-operations.patch", Size: 3587 bytes --]
From a462991fadfd1c350ecb41635f1157e227e133ef Mon Sep 17 00:00:00 2001
From: Bart Van Assche <bart.vanassche@sandisk.com>
Date: Tue, 17 Jan 2017 13:22:35 -0800
Subject: [PATCH] hfi1, qib, rdmavt [v2]: Use proper DMA mapping operations
Make users of the hfi1 and qib drivers use dma_virt_ops. Use the PCIe
mapping operations for SDMA. Make sure that the rdmavt code does not
change the PCIe DMA mapping operations pointer into &dma_virt_ops.
---
drivers/infiniband/hw/hfi1/mad.c | 2 +-
drivers/infiniband/hw/hfi1/verbs.c | 1 -
drivers/infiniband/hw/qib/qib_verbs.c | 1 -
drivers/infiniband/sw/rdmavt/vt.c | 2 ++
include/rdma/rdma_vt.h | 8 ++++++++
5 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/mad.c b/drivers/infiniband/hw/hfi1/mad.c
index 6e595afca24c..041d503c399b 100644
--- a/drivers/infiniband/hw/hfi1/mad.c
+++ b/drivers/infiniband/hw/hfi1/mad.c
@@ -4406,7 +4406,7 @@ int hfi1_process_mad(struct ib_device *ibdev, int mad_flags, u8 port,
switch (in_mad->base_version) {
case OPA_MGMT_BASE_VERSION:
if (unlikely(in_mad_size != sizeof(struct opa_mad))) {
- dev_err(ibdev->dma_device, "invalid in_mad_size\n");
+ dev_err(&ibdev->dev, "invalid in_mad_size\n");
return IB_MAD_RESULT_FAILURE;
}
return hfi1_process_opa_mad(ibdev, mad_flags, port,
diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c
index 95ed4d6da510..fa0ff3e11597 100644
--- a/drivers/infiniband/hw/hfi1/verbs.c
+++ b/drivers/infiniband/hw/hfi1/verbs.c
@@ -1784,7 +1784,6 @@ int hfi1_register_ib_device(struct hfi1_devdata *dd)
strlcpy(ibdev->name + lcpysz, "_%d", IB_DEVICE_NAME_MAX - lcpysz);
ibdev->owner = THIS_MODULE;
ibdev->phys_port_cnt = dd->num_pports;
- ibdev->dma_device = &dd->pcidev->dev;
ibdev->modify_device = modify_device;
ibdev->alloc_hw_stats = alloc_hw_stats;
ibdev->get_hw_stats = get_hw_stats;
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c b/drivers/infiniband/hw/qib/qib_verbs.c
index 4b54c0ddd08a..daa28a7b7574 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -1632,7 +1632,6 @@ int qib_register_ib_device(struct qib_devdata *dd)
ibdev->owner = THIS_MODULE;
ibdev->node_guid = ppd->guid;
ibdev->phys_port_cnt = dd->num_pports;
- ibdev->dma_device = &dd->pcidev->dev;
ibdev->modify_device = qib_modify_device;
ibdev->process_mad = qib_process_mad;
diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c
index 6a81b179f631..99b3cb39ed67 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -103,6 +103,8 @@ struct rvt_dev_info *rvt_alloc_device(size_t size, int nports)
if (!rdi->ports)
ib_dealloc_device(&rdi->ibdev);
+ rdi->ibdev.dma_device = &rdi->dma_device;
+
return rdi;
}
EXPORT_SYMBOL(rvt_alloc_device);
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 861e23eaebda..b7259c9b056e 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -339,6 +339,14 @@ struct rvt_driver_provided {
struct rvt_dev_info {
struct ib_device ibdev; /* Keep this first. Nothing above here */
+ /*
+ * qib and hfi1 use two sets of DMA operations:
+ * - The DMA operations of the PCIe device for SDMA.
+ * - dma_virt_ops for users of the qib and hfi1 drivers.
+ * The only purpose of @dma_device is to provide a struct device that
+ * provides dma_virt_ops.
+ */
+ struct device dma_device;
/*
* Prior to calling for registration the driver will be responsible for
--
2.11.0
^ permalink raw reply related [flat|nested] 45+ messages in thread* Re: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-17 22:00 ` Bart Van Assche
@ 2017-01-17 22:27 ` Jason Gunthorpe
2017-01-17 22:33 ` Bart Van Assche
0 siblings, 1 reply; 45+ messages in thread
From: Jason Gunthorpe @ 2017-01-17 22:27 UTC (permalink / raw)
To: Bart Van Assche
Cc: alex.estrin@intel.com, dledford@redhat.com,
linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
gregkh@linuxfoundation.org
On Tue, Jan 17, 2017 at 10:00:00PM +0000, Bart Van Assche wrote:
> + /*
> + * qib and hfi1 use two sets of DMA operations:
> + * - The DMA operations of the PCIe device for SDMA.
> + * - dma_virt_ops for users of the qib and hfi1 drivers.
> + * The only purpose of @dma_device is to provide a struct device that
> + * provides dma_virt_ops.
> + */
> + struct device dma_device;
Creating a struct device, not calling device_initialize() and then
passing it around it other places is pretty ugly and potentially
dangerous.
Is there some other way?
Can you setup ibdev to be the 'dma device' for the user side?
Jason
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-17 22:27 ` Jason Gunthorpe
@ 2017-01-17 22:33 ` Bart Van Assche
2017-01-17 22:50 ` Jason Gunthorpe
0 siblings, 1 reply; 45+ messages in thread
From: Bart Van Assche @ 2017-01-17 22:33 UTC (permalink / raw)
To: jgunthorpe@obsidianresearch.com
Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
gregkh@linuxfoundation.org, alex.estrin@intel.com,
dledford@redhat.com
On Tue, 2017-01-17 at 15:27 -0700, Jason Gunthorpe wrote:
> On Tue, Jan 17, 2017 at 10:00:00PM +0000, Bart Van Assche wrote:
> > + /*
> > + * qib and hfi1 use two sets of DMA operations:
> > + * - The DMA operations of the PCIe device for SDMA.
> > + * - dma_virt_ops for users of the qib and hfi1 drivers.
> > + * The only purpose of @dma_device is to provide a struct device that
> > + * provides dma_virt_ops.
> > + */
> > + struct device dma_device;
>
> Creating a struct device, not calling device_initialize() and then
> passing it around it other places is pretty ugly and potentially
> dangerous.
>
> Is there some other way?
>
> Can you setup ibdev to be the 'dma device' for the user side?
Hello Jason,
The only member of this struct device that is used is dma_ops so this approach
should be fine as an interim solution. My plan is, if this patch is sufficient
to unbreak the hfi1 driver, to remove the struct device cited in your e-mail
again and also the dma_device pointer from struct ib_device. Since the
dev.dma_ops pointer in struct ib_device is not yet used dev.dma_ops can be used
instead of the dma_device pointer.
Bart.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/26] IB: Optimize DMA mapping
2017-01-12 19:06 [PATCH v2 00/26] IB: Optimize DMA mapping Bart Van Assche
` (26 preceding siblings ...)
2017-01-13 18:27 ` [PATCH v2 00/26] IB: Optimize DMA mapping Estrin, Alex
@ 2017-01-13 21:26 ` Sagi Grimberg
27 siblings, 0 replies; 45+ messages in thread
From: Sagi Grimberg @ 2017-01-13 21:26 UTC (permalink / raw)
To: Bart Van Assche, Doug Ledford
Cc: linux-rdma, linux-kernel, Greg Kroah-Hartman
> Hello Doug,
>
> As you know there are two sets of DMA mapping operations in the Linux
> kernel:
> - One set of DMA mapping operations that is used by most drivers.
> - Another set of DMA mapping operations that is only used by the RDMA
> drivers.
> Having two sets of DMA mapping operations is not only a source of
> confusion but also a source of unnecessary overhead. The DMA mapping
> operations are in the hot path so it is important that the overhead
> of these operations is as low as possible. Hence this patch series
> that converts the RDMA code to the standard DMA mapping API and
> thereby eliminates the if (dev->dma_ops) test from the hot path. An
> additional benefit is that the size of HW and SW drivers that do not
> use DMA is reduced by switching to dma_virt_ops.
This looks really good!
Thanks Bart for doing this,
For the series:
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
^ permalink raw reply [flat|nested] 45+ messages in thread