[PATCH 00/11] swiotlb: Introduce architecture-specific APIs to replace __weak functions (v2)

[PATCH 00/11] swiotlb: Introduce architecture-specific APIs to replace __weak functions (v2)

[Ian Campbell]

This series:
* removes the swiotlb_(arch_)_phys_to_bus and bus_to_phys __weak
  hooks, replacing them with an architecture-specific phys_to_dma and
  dma_to_phys interface. These are used by both PowerPC and Xen to
  provide the correct mapping from physical to DMA addresses.
* removes the swiotlb_address_needs_mapping and
  swiotlb_range_needs_mapping __weak functions as well as
  is_buffer_dma_capable (which should never have been a generic
  function). All three are replaced by a single architecture-specific
  interface which meets the needs of both PowerPC and Xen.
* removes the swiotlb_virt_to_bus __weak function and replaces it with
  a CONFIG_HIGHMEM compatible version when high memory is in use. This
  is needed for 32 bit PowerPC swiotlb support.
* removes the swiotlb_alloc* __weak functions and replaces them with
  swiotlb_init_with_buffer which allows the use of a caller allocated
  buffer (and emergency pool).

I think these new interfaces are cleaner than the existing __weak
functions and isolate the swiotlb code from architecture internals.

This series does not contain any Xen or PowerPC specific changes, those
will follow in separate postings. The complete patchset has been boot
tested under Xen and native-x86 and compiled for IA64 and PowerPC

Changes since v1: 
- Fixed compile error in swiotlb_dma_to_virt highmem version. Moved
  #ifdef into function to avoid prototype drift.
- checkpatch fixes.
- missed a swiotlb_arch_range_needs_mapping in swiotlb.h and x86
  pci-swiotlb.c and swiotlb_bus_to_phys/phys_to_bus implementations in
  x86.
- additionally replaced __weak swiotlb_alloc* with
  swiotlb_init_with_buffer.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Becky Bruce <beckyb@kernel.crashing.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Greg KH <gregkh@suse.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: Olaf Kirch <okir@suse.de>
Cc: Tony Luck <tony.luck@intel.com>
Cc: linux-ia64@vger.kernel.org
Cc: linuxppc-dev@ozlabs.org
Cc: x86@kernel.org

[PATCH 06/11] swiotlb: use dma_to_phys and phys_to_dma

[Ian Campbell]

These new architecture-specific interfaces subsume the existing __weak
function hooks.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Olaf Kirch <okir@suse.de>
Cc: Greg KH <gregkh@suse.de>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Becky Bruce <beckyb@kernel.crashing.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: x86@kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linuxppc-dev@ozlabs.org
---
 arch/x86/kernel/pci-swiotlb.c |   10 ----------
 include/linux/swiotlb.h       |    5 -----
 lib/swiotlb.c                 |   18 ++++--------------
 3 files changed, 4 insertions(+), 29 deletions(-)

diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index a1712f2..e89cf99 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -23,16 +23,6 @@ void *swiotlb_alloc(unsigned order, unsigned long nslabs)
 	return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
 }
 
-dma_addr_t swiotlb_phys_to_bus(struct device *hwdev, phys_addr_t paddr)
-{
-	return paddr;
-}
-
-phys_addr_t swiotlb_bus_to_phys(struct device *hwdev, dma_addr_t baddr)
-{
-	return baddr;
-}
-
 int __weak swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size)
 {
 	return 0;
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index cb1a663..954feec 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -27,11 +27,6 @@ swiotlb_init(void);
 extern void *swiotlb_alloc_boot(size_t bytes, unsigned long nslabs);
 extern void *swiotlb_alloc(unsigned order, unsigned long nslabs);
 
-extern dma_addr_t swiotlb_phys_to_bus(struct device *hwdev,
-				      phys_addr_t address);
-extern phys_addr_t swiotlb_bus_to_phys(struct device *hwdev,
-				       dma_addr_t address);
-
 extern int swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size);
 
 extern void
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index bffe6d7..baa1991 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -124,25 +124,15 @@ void * __weak swiotlb_alloc(unsigned order, unsigned long nslabs)
 	return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
 }
 
-dma_addr_t __weak swiotlb_phys_to_bus(struct device *hwdev, phys_addr_t paddr)
-{
-	return paddr;
-}
-
-phys_addr_t __weak swiotlb_bus_to_phys(struct device *hwdev, dma_addr_t baddr)
-{
-	return baddr;
-}
-
 static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
 				      volatile void *address)
 {
-	return swiotlb_phys_to_bus(hwdev, virt_to_phys(address));
+	return phys_to_dma(hwdev, virt_to_phys(address));
 }
 
 void * __weak swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
 {
-	return phys_to_virt(swiotlb_bus_to_phys(hwdev, address));
+	return phys_to_virt(dma_to_phys(hwdev, address));
 }
 
 int __weak swiotlb_arch_address_needs_mapping(struct device *hwdev,
@@ -646,7 +636,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 			    struct dma_attrs *attrs)
 {
 	phys_addr_t phys = page_to_phys(page) + offset;
-	dma_addr_t dev_addr = swiotlb_phys_to_bus(dev, phys);
+	dma_addr_t dev_addr = phys_to_dma(dev, phys);
 	void *map;
 
 	BUG_ON(dir == DMA_NONE);
@@ -817,7 +807,7 @@ swiotlb_map_sg_attrs(struct device *hwdev, struct scatterlist *sgl, int nelems,
 
 	for_each_sg(sgl, sg, nelems, i) {
 		phys_addr_t paddr = sg_phys(sg);
-		dma_addr_t dev_addr = swiotlb_phys_to_bus(hwdev, paddr);
+		dma_addr_t dev_addr = phys_to_dma(hwdev, paddr);
 
 		if (range_needs_mapping(paddr, sg->length) ||
 		    address_needs_mapping(hwdev, dev_addr, sg->length)) {
-- 
1.5.6.5

[PATCH 07/11] swiotlb: use dma_map_range

[Ian Campbell]

This replaces usages of address_needs_mapping, range_needs_mapping and
is_buffer_dma_capable and the __weak architecture hooks to those
functions with a more flexible single function.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Olaf Kirch <okir@suse.de>
Cc: Greg KH <gregkh@suse.de>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Becky Bruce <beckyb@kernel.crashing.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: x86@kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linuxppc-dev@ozlabs.org
---
 arch/x86/kernel/pci-swiotlb.c |    5 ---
 include/linux/dma-mapping.h   |    5 ---
 include/linux/swiotlb.h       |    2 -
 lib/swiotlb.c                 |   59 +++++++++++++---------------------------
 4 files changed, 19 insertions(+), 52 deletions(-)

diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index e89cf99..fdcc0e2 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -23,11 +23,6 @@ void *swiotlb_alloc(unsigned order, unsigned long nslabs)
 	return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
 }
 
-int __weak swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size)
-{
-	return 0;
-}
-
 static void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 					dma_addr_t *dma_handle, gfp_t flags)
 {
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 8083b6a..85dafa1 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -96,11 +96,6 @@ static inline int is_device_dma_capable(struct device *dev)
 	return dev->dma_mask != NULL && *dev->dma_mask != DMA_MASK_NONE;
 }
 
-static inline int is_buffer_dma_capable(u64 mask, dma_addr_t addr, size_t size)
-{
-	return addr + size <= mask;
-}
-
 #ifdef CONFIG_HAS_DMA
 #include <asm/dma-mapping.h>
 #else
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 954feec..1b56dbf 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -27,8 +27,6 @@ swiotlb_init(void);
 extern void *swiotlb_alloc_boot(size_t bytes, unsigned long nslabs);
 extern void *swiotlb_alloc(unsigned order, unsigned long nslabs);
 
-extern int swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size);
-
 extern void
 *swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 			dma_addr_t *dma_handle, gfp_t flags);
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index baa1991..d37499b 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -135,17 +135,6 @@ void * __weak swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
 	return phys_to_virt(dma_to_phys(hwdev, address));
 }
 
-int __weak swiotlb_arch_address_needs_mapping(struct device *hwdev,
-					       dma_addr_t addr, size_t size)
-{
-	return !is_buffer_dma_capable(dma_get_mask(hwdev), addr, size);
-}
-
-int __weak swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size)
-{
-	return 0;
-}
-
 static void swiotlb_print_info(unsigned long bytes)
 {
 	phys_addr_t pstart, pend;
@@ -305,17 +294,6 @@ cleanup1:
 	return -ENOMEM;
 }
 
-static inline int
-address_needs_mapping(struct device *hwdev, dma_addr_t addr, size_t size)
-{
-	return swiotlb_arch_address_needs_mapping(hwdev, addr, size);
-}
-
-static inline int range_needs_mapping(phys_addr_t paddr, size_t size)
-{
-	return swiotlb_force || swiotlb_arch_range_needs_mapping(paddr, size);
-}
-
 static int is_swiotlb_buffer(char *addr)
 {
 	return addr >= io_tlb_start && addr < io_tlb_end;
@@ -542,7 +520,7 @@ void *
 swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 		       dma_addr_t *dma_handle, gfp_t flags)
 {
-	dma_addr_t dev_addr;
+	phys_addr_t phys;
 	void *ret;
 	int order = get_order(size);
 	u64 dma_mask = DMA_BIT_MASK(32);
@@ -551,9 +529,8 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 		dma_mask = hwdev->coherent_dma_mask;
 
 	ret = (void *)__get_free_pages(flags, order);
-	if (ret &&
-	    !is_buffer_dma_capable(dma_mask, swiotlb_virt_to_bus(hwdev, ret),
-				   size)) {
+	if (ret && !dma_map_range(hwdev, dma_mask, virt_to_phys(ret),
+				  size, dma_handle)) {
 		/*
 		 * The allocated memory isn't reachable by the device.
 		 */
@@ -572,19 +549,19 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
 	}
 
 	memset(ret, 0, size);
-	dev_addr = swiotlb_virt_to_bus(hwdev, ret);
+	phys = virt_to_phys(ret);
 
 	/* Confirm address can be DMA'd by device */
-	if (!is_buffer_dma_capable(dma_mask, dev_addr, size)) {
-		printk("hwdev DMA mask = 0x%016Lx, dev_addr = 0x%016Lx\n",
+	if (!dma_map_range(hwdev, dma_mask, phys, size, dma_handle)) {
+		printk(KERN_WARNING "hwdev DMA mask = 0x%016Lx, "
+				    "physical addr = 0x%016Lx\n",
 		       (unsigned long long)dma_mask,
-		       (unsigned long long)dev_addr);
+		       (unsigned long long)phys);
 
 		/* DMA_TO_DEVICE to avoid memcpy in unmap_single */
 		do_unmap_single(hwdev, ret, size, DMA_TO_DEVICE);
 		return NULL;
 	}
-	*dma_handle = dev_addr;
 	return ret;
 }
 EXPORT_SYMBOL(swiotlb_alloc_coherent);
@@ -636,7 +613,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 			    struct dma_attrs *attrs)
 {
 	phys_addr_t phys = page_to_phys(page) + offset;
-	dma_addr_t dev_addr = phys_to_dma(dev, phys);
+	dma_addr_t dev_addr;
 	void *map;
 
 	BUG_ON(dir == DMA_NONE);
@@ -645,8 +622,8 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 	 * we can safely return the device addr and not worry about bounce
 	 * buffering it.
 	 */
-	if (!address_needs_mapping(dev, dev_addr, size) &&
-	    !range_needs_mapping(phys, size))
+	if (dma_map_range(dev, dma_get_mask(dev), phys, size, &dev_addr) &&
+	    !swiotlb_force)
 		return dev_addr;
 
 	/*
@@ -658,12 +635,12 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
 		map = io_tlb_overflow_buffer;
 	}
 
-	dev_addr = swiotlb_virt_to_bus(dev, map);
+	phys = virt_to_phys(map);
 
 	/*
 	 * Ensure that the address returned is DMA'ble
 	 */
-	if (address_needs_mapping(dev, dev_addr, size))
+	if (!dma_map_range(dev, dma_get_mask(dev), phys, size, &dev_addr))
 		panic("map_single: bounce buffer is not DMA'ble");
 
 	return dev_addr;
@@ -807,10 +784,11 @@ swiotlb_map_sg_attrs(struct device *hwdev, struct scatterlist *sgl, int nelems,
 
 	for_each_sg(sgl, sg, nelems, i) {
 		phys_addr_t paddr = sg_phys(sg);
-		dma_addr_t dev_addr = phys_to_dma(hwdev, paddr);
+		dma_addr_t uninitialized_var(dev_addr);
 
-		if (range_needs_mapping(paddr, sg->length) ||
-		    address_needs_mapping(hwdev, dev_addr, sg->length)) {
+		if (!dma_map_range(hwdev, dma_get_mask(hwdev), dev_addr,
+				   sg->length, &dev_addr) ||
+		    swiotlb_force) {
 			void *map = map_single(hwdev, sg_phys(sg),
 					       sg->length, dir);
 			if (!map) {
@@ -822,7 +800,8 @@ swiotlb_map_sg_attrs(struct device *hwdev, struct scatterlist *sgl, int nelems,
 				sgl[0].dma_length = 0;
 				return 0;
 			}
-			sg->dma_address = swiotlb_virt_to_bus(hwdev, map);
+			paddr = virt_to_phys(map);
+			sg->dma_address = phys_to_dma(hwdev, paddr);
 		} else
 			sg->dma_address = dev_addr;
 		sg->dma_length = sg->length;
-- 
1.5.6.5

[PATCH 08/11] swiotlb: support HIGHMEM in swiotlb_bus_to_virt

[Ian Campbell]

Rather than supplying a __weak hook which architectures which support
highmem can overide simply provide a version of swiotlb_bus_to_virt
which works with high memory. Make it conditional since it is a more
expensive variant than the non-highmem version.

Acutal function contents taken from the PowerPC swiotlb patchset by
Becky Bruce.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Becky Bruce <beckyb@kernel.crashing.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: linuxppc-dev@ozlabs.org
---
 lib/swiotlb.c |   11 ++++++++++-
 1 files changed, 10 insertions(+), 1 deletions(-)

diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index d37499b..d2b296a 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -130,9 +130,18 @@ static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
 	return phys_to_dma(hwdev, virt_to_phys(address));
 }
 
-void * __weak swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
+static void *swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
 {
+#ifdef CONFIG_HIGHMEM
+	unsigned long pfn = PFN_DOWN(dma_to_phys(hwdev, address));
+	void *pageaddr = page_address(pfn_to_page(pfn));
+
+	if (pageaddr != NULL)
+		return pageaddr + (address % PAGE_SIZE);
+	return NULL;
+#else
 	return phys_to_virt(dma_to_phys(hwdev, address));
+#endif
 }
 
 static void swiotlb_print_info(unsigned long bytes)
-- 
1.5.6.5

Re: [PATCH 00/11] swiotlb: Introduce architecture-specific APIs to replace __weak functions (v2)

[Benjamin Herrenschmidt]

On Mon, 2009-06-01 at 16:32 +0100, Ian Campbell wrote:
> This series:
> * removes the swiotlb_(arch_)_phys_to_bus and bus_to_phys __weak
>   hooks, replacing them with an architecture-specific phys_to_dma and
>   dma_to_phys interface. These are used by both PowerPC and Xen to
>   provide the correct mapping from physical to DMA addresses.
> * removes the swiotlb_address_needs_mapping and
>   swiotlb_range_needs_mapping __weak functions as well as
>   is_buffer_dma_capable (which should never have been a generic
>   function). All three are replaced by a single architecture-specific
>   interface which meets the needs of both PowerPC and Xen.
> * removes the swiotlb_virt_to_bus __weak function and replaces it with
>   a CONFIG_HIGHMEM compatible version when high memory is in use. This
>   is needed for 32 bit PowerPC swiotlb support.
> * removes the swiotlb_alloc* __weak functions and replaces them with
>   swiotlb_init_with_buffer which allows the use of a caller allocated
>   buffer (and emergency pool).
> 
> I think these new interfaces are cleaner than the existing __weak
> functions and isolate the swiotlb code from architecture internals.
> 
> This series does not contain any Xen or PowerPC specific changes, those
> will follow in separate postings. The complete patchset has been boot
> tested under Xen and native-x86 and compiled for IA64 and PowerPC
> 
> Changes since v1: 
> - Fixed compile error in swiotlb_dma_to_virt highmem version. Moved
>   #ifdef into function to avoid prototype drift.
> - checkpatch fixes.
> - missed a swiotlb_arch_range_needs_mapping in swiotlb.h and x86
>   pci-swiotlb.c and swiotlb_bus_to_phys/phys_to_bus implementations in
>   x86.
> - additionally replaced __weak swiotlb_alloc* with
>   swiotlb_init_with_buffer.

Looks like I was only CCed on part of them... it's not very handy for me
as I end up having some of the patches in one folder and some
elsewhere :-)

I don't have a firm objection but they will have to go through Becky
and Kumar since they are the one who need swiotlb for their embedded
platforms.

Cheers,
Ben.

Re: [PATCH 00/11] swiotlb: Introduce architecture-specific APIs to replace __weak functions (v2)

[Ian Campbell]

On Fri, 2009-07-10 at 15:55 +1000, Benjamin Herrenschmidt wrote:
> On Mon, 2009-06-01 at 16:32 +0100, Ian Campbell wrote:
> > This series:[...]

> Looks like I was only CCed on part of them... it's not very handy for me
> as I end up having some of the patches in one folder and some
> elsewhere :-)

Sorry about that -- I was concerned about spamming the world and his dog
with patches which didn't directly interest them. For any future series
of this type I'll CC everyone on everything.

Ian.