[RFC 0/2] ARM: DMA-mapping & IOMMU integration

[RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Marek Szyprowski]

Hello,

Folloing the discussion about the driver for IOMMU controller for
Samsung Exynos4 platform and Arnd's suggestions I've decided to start
working on redesign of dma-mapping implementation for ARM architecture.
The goal is to add support for IOMMU in the way preffered by the
community :)

Some of the ideas about merging dma-mapping api and iommu api comes from
the following threads:
http://www.spinics.net/lists/linux-media/msg31453.html
http://www.spinics.net/lists/arm-kernel/msg122552.html
http://www.spinics.net/lists/arm-kernel/msg124416.html

They were also discussed on Linaro memory management meeting at UDS
(Budapest 9-12 May).

I've finaly managed to clean up a bit my works and present the initial,
very proof-of-concept version of patches that were ready just before
Linaro meeting.

What have been implemented: 

1. Introduced arm_dma_ops

dma_map_ops from include/linux/dma-mapping.h suffers from the following
limitations:
- lack of start address for sync operations
- lack of write-combine methods
- lack of mmap to user-space methods
- lack of map_single method

For the initial version I've decided to use custom arm_dma_ops.
Extending common interface will take time, until that I wanted to have
something already working.

dma_{alloc,free,mmap}_{coherent,writecombine} have been consolidated
into dma_{alloc,free,mmap}_attrib what have been suggested on Linaro
meeting. New attribute for WRITE_COMBINE memory have been introduced.

2. moved all inline ARM dma-mapping related operations to
arch/arm/mm/dma-mapping.c and put them as methods in generic arm_dma_ops
structure. The dma-mapping.c code deinitely needs cleanup, but this is
just a first step.

3. Added very initial IOMMU support. Right now it is limited only to
dma_alloc_attrib, dma_free_attrib and dma_mmap_attrib. It have been
tested with s5p-fimc driver on Samsung Exynos4 platform.

4. Adapted Samsung Exynos4 IOMUU driver to make use of the introduced
iommu_dma proposal.

This patch series contains only patches for common dma-mapping part.
There is also a patch that adds driver for Samsung IOMMU controller on
Exynos4 platform. All required patches are available on:

git://git.infradead.org/users/kmpark/linux-2.6-samsung dma-mapping branch

Git web interface:
http://git.infradead.org/users/kmpark/linux-2.6-samsung/shortlog/refs/heads/dma-mapping


Future:

1. Add all missing operations for IOMMU mappings (map_single/page/sg,
sync_*)

2. Move sync_* operations into separate function for better code sharing
between iommu and non-iommu dma-mapping code

3. Splitting out dma bounce code from non-bounce into separate set of
dma methods. Right now dma-bounce code is compiled conditionally and
spread over arch/arm/mm/dma-mapping.c and arch/arm/common/dmabounce.c.

4. Merging dma_map_single with dma_map_page. I haven't investigated
deeply why they have separate implementation on ARM. If this is a
requirement then dma_map_ops need to be extended with another method.

5. Fix dma_alloc to unmap from linear mapping.

6. Convert IO address space management code from gen-alloc to some
simpler bitmap based solution.

7. resolve issues that might araise during discussion & comments

Please note that this is very early version of patches, definitely NOT
intended for merging. I just wanted to make sure that the direction is
right and share the code with others that might want to cooperate on
dma-mapping improvements.

Best regards
-- 
Marek Szyprowski
Samsung Poland R&D Center



Patch summary:

Marek Szyprowski (2):
  ARM: Move dma related inlines into arm_dma_ops methods
  ARM: initial proof-of-concept IOMMU mapper for DMA-mapping

 arch/arm/Kconfig                   |    1 +
 arch/arm/include/asm/device.h      |    3 +
 arch/arm/include/asm/dma-iommu.h   |   30 ++
 arch/arm/include/asm/dma-mapping.h |  653 +++++++++++------------------
 arch/arm/mm/dma-mapping.c          |  817 +++++++++++++++++++++++++++++++++---
 arch/arm/mm/vmregion.h             |    2 +-
 include/linux/dma-attrs.h          |    1 +
 7 files changed, 1033 insertions(+), 474 deletions(-)
 create mode 100644 arch/arm/include/asm/dma-iommu.h

-- 
1.7.1.569.g6f426

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC 1/2] ARM: Move dma related inlines into arm_dma_ops methods

[Marek Szyprowski]

This is initial step in ARM DMA-mapping redesign. All calls have
been moved into separate methods from arm_dma_ops structure that
can be set separately for particular device's.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
---
 arch/arm/include/asm/device.h      |    1 +
 arch/arm/include/asm/dma-mapping.h |  653 +++++++++++++-----------------------
 arch/arm/mm/dma-mapping.c          |  491 ++++++++++++++++++++++++----
 arch/arm/mm/vmregion.h             |    2 +-
 include/linux/dma-attrs.h          |    1 +
 5 files changed, 674 insertions(+), 474 deletions(-)

diff --git a/arch/arm/include/asm/device.h b/arch/arm/include/asm/device.h
index 9f390ce..005791a 100644
--- a/arch/arm/include/asm/device.h
+++ b/arch/arm/include/asm/device.h
@@ -7,6 +7,7 @@
 #define ASMARM_DEVICE_H
 
 struct dev_archdata {
+	struct arm_dma_map_ops	*dma_ops;
 #ifdef CONFIG_DMABOUNCE
 	struct dmabounce_device_info *dmabounce;
 #endif
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index 4fff837..42f4625 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -6,155 +6,219 @@
 #include <linux/mm_types.h>
 #include <linux/scatterlist.h>
 #include <linux/dma-debug.h>
+#include <linux/kmemcheck.h>
 
 #include <asm-generic/dma-coherent.h>
 #include <asm/memory.h>
 
-#ifdef __arch_page_to_dma
-#error Please update to __arch_pfn_to_dma
-#endif
-
-/*
- * dma_to_pfn/pfn_to_dma/dma_to_virt/virt_to_dma are architecture private
- * functions used internally by the DMA-mapping API to provide DMA
- * addresses. They must not be used by drivers.
- */
-#ifndef __arch_pfn_to_dma
-static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
+#define DMA_ERROR_CODE		(~(dma_addr_t)0x0)
+
+struct arm_dma_map_ops {
+	void *(*alloc_attrs)(struct device *, size_t, dma_addr_t *, gfp_t,
+		struct dma_attrs *attrs);
+	void (*free_attrs)(struct device *, size_t, void *, dma_addr_t,
+		struct dma_attrs *attrs);
+	int (*mmap_attrs)(struct device *, struct vm_area_struct *,
+                void *, dma_addr_t, size_t, struct dma_attrs *attrs);
+
+	/* map single and map page might be merged together */
+	dma_addr_t (*map_single)(struct device *dev, void *cpu_addr,
+                size_t size, enum dma_data_direction dir);
+	void (*unmap_single)(struct device *dev, dma_addr_t handle,
+                size_t size, enum dma_data_direction dir);
+	dma_addr_t (*map_page)(struct device *dev, struct page *page,
+             unsigned long offset, size_t size, enum dma_data_direction dir);
+	void (*unmap_page)(struct device *dev, dma_addr_t handle,
+                size_t size, enum dma_data_direction dir);
+
+	int (*map_sg)(struct device *, struct scatterlist *, int,
+                enum dma_data_direction);
+	void (*unmap_sg)(struct device *, struct scatterlist *, int,
+                enum dma_data_direction);
+
+	void (*sync_single_for_device)(struct device *dev,
+                dma_addr_t handle, unsigned long offset, size_t size,
+		enum dma_data_direction dir);
+	void (*sync_single_for_cpu)(struct device *dev,
+                dma_addr_t handle, unsigned long offset, size_t size,
+		enum dma_data_direction dir);
+
+	void (*sync_sg_for_cpu)(struct device *, struct scatterlist *, int,
+                enum dma_data_direction);
+	void (*sync_sg_for_device)(struct device *, struct scatterlist *, int,
+                enum dma_data_direction);
+
+};
+
+extern struct arm_dma_map_ops dma_ops;
+
+static inline struct arm_dma_map_ops *get_dma_ops(struct device *dev)
 {
-	return (dma_addr_t)__pfn_to_bus(pfn);
+	if (dev->archdata.dma_ops)
+		return dev->archdata.dma_ops;
+	return &dma_ops;
 }
 
-static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
+static inline void set_dma_ops(struct device *dev, struct arm_dma_map_ops *ops)
 {
-	return __bus_to_pfn(addr);
+	dev->archdata.dma_ops = ops;
 }
 
-static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
-{
-	return (void *)__bus_to_virt(addr);
-}
+/********************************/
 
-static inline dma_addr_t virt_to_dma(struct device *dev, void *addr)
+static inline void *dma_alloc_coherent(struct device *dev, size_t size,
+				       dma_addr_t *dma_handle, gfp_t flag)
 {
-	return (dma_addr_t)__virt_to_bus((unsigned long)(addr));
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+	void *cpu_addr;
+
+	BUG_ON(!ops);
+
+	cpu_addr = ops->alloc_attrs(dev, size, dma_handle, flag, NULL);
+	debug_dma_alloc_coherent(dev, size, *dma_handle, cpu_addr);
+	return cpu_addr;
 }
-#else
-static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
+
+static inline void dma_free_coherent(struct device *dev, size_t size,
+				     void *cpu_addr, dma_addr_t dma_handle)
 {
-	return __arch_pfn_to_dma(dev, pfn);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+
+	BUG_ON(!ops);
+
+	debug_dma_free_coherent(dev, size, cpu_addr, dma_handle);
+	ops->free_attrs(dev, size, cpu_addr, dma_handle, NULL);
 }
 
-static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
+static inline void *dma_alloc_writecombine(struct device *dev, size_t size,
+				       dma_addr_t *dma_handle, gfp_t flag)
 {
-	return __arch_dma_to_pfn(dev, addr);
+	DEFINE_DMA_ATTRS(attrs);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+	void *cpu_addr;
+	dma_set_attr(DMA_ATTR_WRITE_COMBINE, &attrs);
+	BUG_ON(!ops);
+
+	cpu_addr = ops->alloc_attrs(dev, size, dma_handle, flag, &attrs);
+	debug_dma_alloc_coherent(dev, size, *dma_handle, cpu_addr);
+	return cpu_addr;
 }
 
-static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
+static inline void dma_free_writecombine(struct device *dev, size_t size,
+				     void *cpu_addr, dma_addr_t dma_handle)
 {
-	return __arch_dma_to_virt(dev, addr);
+	DEFINE_DMA_ATTRS(attrs);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+	dma_set_attr(DMA_ATTR_WRITE_COMBINE, &attrs);
+
+	BUG_ON(!ops);
+
+
+	debug_dma_free_coherent(dev, size, cpu_addr, dma_handle);
+	ops->free_attrs(dev, size, cpu_addr, dma_handle, &attrs);
 }
 
-static inline dma_addr_t virt_to_dma(struct device *dev, void *addr)
+static inline dma_addr_t dma_map_single(struct device *dev, void *ptr,
+					      size_t size,
+					      enum dma_data_direction dir)
 {
-	return __arch_virt_to_dma(dev, addr);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+	dma_addr_t addr;
+
+	kmemcheck_mark_initialized(ptr, size);
+	BUG_ON(!valid_dma_direction(dir));
+	addr = ops->map_single(dev, ptr, size, dir);
+	debug_dma_map_page(dev, virt_to_page(ptr),
+			   (unsigned long)ptr & ~PAGE_MASK, size,
+			   dir, addr, true);
+	return addr;
 }
-#endif
 
-/*
- * The DMA API is built upon the notion of "buffer ownership".  A buffer
- * is either exclusively owned by the CPU (and therefore may be accessed
- * by it) or exclusively owned by the DMA device.  These helper functions
- * represent the transitions between these two ownership states.
- *
- * Note, however, that on later ARMs, this notion does not work due to
- * speculative prefetches.  We model our approach on the assumption that
- * the CPU does do speculative prefetches, which means we clean caches
- * before transfers and delay cache invalidation until transfer completion.
- *
- * Private support functions: these are not part of the API and are
- * liable to change.  Drivers must not use these.
- */
-static inline void __dma_single_cpu_to_dev(const void *kaddr, size_t size,
-	enum dma_data_direction dir)
+static inline void dma_unmap_single(struct device *dev, dma_addr_t addr,
+					  size_t size,
+					  enum dma_data_direction dir)
 {
-	extern void ___dma_single_cpu_to_dev(const void *, size_t,
-		enum dma_data_direction);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
 
-	if (!arch_is_coherent())
-		___dma_single_cpu_to_dev(kaddr, size, dir);
+	BUG_ON(!valid_dma_direction(dir));
+	if (ops->unmap_single)
+		ops->unmap_single(dev, addr, size, dir);
+	debug_dma_unmap_page(dev, addr, size, dir, true);
 }
 
-static inline void __dma_single_dev_to_cpu(const void *kaddr, size_t size,
-	enum dma_data_direction dir)
+static inline dma_addr_t dma_map_page(struct device *dev, struct page *page,
+				      size_t offset, size_t size,
+				      enum dma_data_direction dir)
 {
-	extern void ___dma_single_dev_to_cpu(const void *, size_t,
-		enum dma_data_direction);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+	dma_addr_t addr;
+
+	kmemcheck_mark_initialized(page_address(page) + offset, size);
+	BUG_ON(!valid_dma_direction(dir));
+	addr = ops->map_page(dev, page, offset, size, dir);
+	debug_dma_map_page(dev, page, offset, size, dir, addr, false);
 
-	if (!arch_is_coherent())
-		___dma_single_dev_to_cpu(kaddr, size, dir);
+	return addr;
 }
 
-static inline void __dma_page_cpu_to_dev(struct page *page, unsigned long off,
-	size_t size, enum dma_data_direction dir)
+static inline void dma_unmap_page(struct device *dev, dma_addr_t addr,
+				  size_t size, enum dma_data_direction dir)
 {
-	extern void ___dma_page_cpu_to_dev(struct page *, unsigned long,
-		size_t, enum dma_data_direction);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
 
-	if (!arch_is_coherent())
-		___dma_page_cpu_to_dev(page, off, size, dir);
+	BUG_ON(!valid_dma_direction(dir));
+	if (ops->unmap_page)
+		ops->unmap_page(dev, addr, size, dir);
+	debug_dma_unmap_page(dev, addr, size, dir, false);
 }
 
-static inline void __dma_page_dev_to_cpu(struct page *page, unsigned long off,
-	size_t size, enum dma_data_direction dir)
+static inline void dma_sync_single_for_cpu(struct device *dev, dma_addr_t addr,
+					   size_t size,
+					   enum dma_data_direction dir)
 {
-	extern void ___dma_page_dev_to_cpu(struct page *, unsigned long,
-		size_t, enum dma_data_direction);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
 
-	if (!arch_is_coherent())
-		___dma_page_dev_to_cpu(page, off, size, dir);
+	BUG_ON(!valid_dma_direction(dir));
+	if (ops->sync_single_for_cpu)
+		ops->sync_single_for_cpu(dev, addr, 0, size, dir);
+	debug_dma_sync_single_for_cpu(dev, addr, size, dir);
 }
 
-/*
- * Return whether the given device DMA address mask can be supported
- * properly.  For example, if your device can only drive the low 24-bits
- * during bus mastering, then you would pass 0x00ffffff as the mask
- * to this function.
- *
- * FIXME: This should really be a platform specific issue - we should
- * return false if GFP_DMA allocations may not satisfy the supplied 'mask'.
- */
-static inline int dma_supported(struct device *dev, u64 mask)
+static inline void dma_sync_single_for_device(struct device *dev,
+					      dma_addr_t addr, size_t size,
+					      enum dma_data_direction dir)
 {
-	if (mask < ISA_DMA_THRESHOLD)
-		return 0;
-	return 1;
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+
+	BUG_ON(!valid_dma_direction(dir));
+	if (ops->sync_single_for_device)
+		ops->sync_single_for_device(dev, addr, 0, size, dir);
+	debug_dma_sync_single_for_device(dev, addr, size, dir);
 }
 
-static inline int dma_set_mask(struct device *dev, u64 dma_mask)
+static inline void dma_sync_single_range_for_cpu(struct device *dev, dma_addr_t addr,
+					   unsigned long offset, size_t size,
+					   enum dma_data_direction dir)
 {
-#ifdef CONFIG_DMABOUNCE
-	if (dev->archdata.dmabounce) {
-		if (dma_mask >= ISA_DMA_THRESHOLD)
-			return 0;
-		else
-			return -EIO;
-	}
-#endif
-	if (!dev->dma_mask || !dma_supported(dev, dma_mask))
-		return -EIO;
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
 
-	*dev->dma_mask = dma_mask;
-
-	return 0;
+	BUG_ON(!valid_dma_direction(dir));
+	if (ops->sync_single_for_cpu)
+		ops->sync_single_for_cpu(dev, addr, offset, size, dir);
+	debug_dma_sync_single_for_cpu(dev, addr, size, dir);
 }
 
-/*
- * DMA errors are defined by all-bits-set in the DMA address.
- */
-static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
+static inline void dma_sync_single_range_for_device(struct device *dev,
+					      dma_addr_t addr,
+					      unsigned long offset, size_t size,
+					      enum dma_data_direction dir)
 {
-	return dma_addr == ~0;
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+
+	BUG_ON(!valid_dma_direction(dir));
+	if (ops->sync_single_for_device)
+		ops->sync_single_for_device(dev, addr, offset, size, dir);
+	debug_dma_sync_single_for_device(dev, addr, size, dir);
 }
 
 /*
@@ -172,358 +236,117 @@ static inline void dma_free_noncoherent(struct device *dev, size_t size,
 {
 }
 
-/**
- * dma_alloc_coherent - allocate consistent memory for DMA
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @size: required memory size
- * @handle: bus-specific DMA address
- *
- * Allocate some uncached, unbuffered memory for a device for
- * performing DMA.  This function allocates pages, and will
- * return the CPU-viewed address, and sets @handle to be the
- * device-viewed address.
- */
-extern void *dma_alloc_coherent(struct device *, size_t, dma_addr_t *, gfp_t);
-
-/**
- * dma_free_coherent - free memory allocated by dma_alloc_coherent
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @size: size of memory originally requested in dma_alloc_coherent
- * @cpu_addr: CPU-view address returned from dma_alloc_coherent
- * @handle: device-view address returned from dma_alloc_coherent
- *
- * Free (and unmap) a DMA buffer previously allocated by
- * dma_alloc_coherent().
- *
- * References to memory and mappings associated with cpu_addr/handle
- * during and after this call executing are illegal.
- */
-extern void dma_free_coherent(struct device *, size_t, void *, dma_addr_t);
-
-/**
- * dma_mmap_coherent - map a coherent DMA allocation into user space
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @vma: vm_area_struct describing requested user mapping
- * @cpu_addr: kernel CPU-view address returned from dma_alloc_coherent
- * @handle: device-view address returned from dma_alloc_coherent
- * @size: size of memory originally requested in dma_alloc_coherent
- *
- * Map a coherent DMA buffer previously allocated by dma_alloc_coherent
- * into user space.  The coherent DMA buffer must not be freed by the
- * driver until the user space mapping has been released.
- */
-int dma_mmap_coherent(struct device *, struct vm_area_struct *,
-		void *, dma_addr_t, size_t);
-
-
-/**
- * dma_alloc_writecombine - allocate writecombining memory for DMA
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @size: required memory size
- * @handle: bus-specific DMA address
- *
- * Allocate some uncached, buffered memory for a device for
- * performing DMA.  This function allocates pages, and will
- * return the CPU-viewed address, and sets @handle to be the
- * device-viewed address.
- */
-extern void *dma_alloc_writecombine(struct device *, size_t, dma_addr_t *,
-		gfp_t);
-
-#define dma_free_writecombine(dev,size,cpu_addr,handle) \
-	dma_free_coherent(dev,size,cpu_addr,handle)
-
-int dma_mmap_writecombine(struct device *, struct vm_area_struct *,
-		void *, dma_addr_t, size_t);
-
-
-#ifdef CONFIG_DMABOUNCE
-/*
- * For SA-1111, IXP425, and ADI systems  the dma-mapping functions are "magic"
- * and utilize bounce buffers as needed to work around limited DMA windows.
- *
- * On the SA-1111, a bug limits DMA to only certain regions of RAM.
- * On the IXP425, the PCI inbound window is 64MB (256MB total RAM)
- * On some ADI engineering systems, PCI inbound window is 32MB (12MB total RAM)
- *
- * The following are helper functions used by the dmabounce subystem
- *
- */
-
-/**
- * dmabounce_register_dev
- *
- * @dev: valid struct device pointer
- * @small_buf_size: size of buffers to use with small buffer pool
- * @large_buf_size: size of buffers to use with large buffer pool (can be 0)
- *
- * This function should be called by low-level platform code to register
- * a device as requireing DMA buffer bouncing. The function will allocate
- * appropriate DMA pools for the device.
- *
- */
-extern int dmabounce_register_dev(struct device *, unsigned long,
-		unsigned long);
-
-/**
- * dmabounce_unregister_dev
- *
- * @dev: valid struct device pointer
- *
- * This function should be called by low-level platform code when device
- * that was previously registered with dmabounce_register_dev is removed
- * from the system.
- *
- */
-extern void dmabounce_unregister_dev(struct device *);
-
-/**
- * dma_needs_bounce
- *
- * @dev: valid struct device pointer
- * @dma_handle: dma_handle of unbounced buffer
- * @size: size of region being mapped
- *
- * Platforms that utilize the dmabounce mechanism must implement
- * this function.
- *
- * The dmabounce routines call this function whenever a dma-mapping
- * is requested to determine whether a given buffer needs to be bounced
- * or not. The function must return 0 if the buffer is OK for
- * DMA access and 1 if the buffer needs to be bounced.
- *
- */
-extern int dma_needs_bounce(struct device*, dma_addr_t, size_t);
-
-/*
- * The DMA API, implemented by dmabounce.c.  See below for descriptions.
- */
-extern dma_addr_t __dma_map_single(struct device *, void *, size_t,
-		enum dma_data_direction);
-extern void __dma_unmap_single(struct device *, dma_addr_t, size_t,
-		enum dma_data_direction);
-extern dma_addr_t __dma_map_page(struct device *, struct page *,
-		unsigned long, size_t, enum dma_data_direction);
-extern void __dma_unmap_page(struct device *, dma_addr_t, size_t,
-		enum dma_data_direction);
-
-/*
- * Private functions
- */
-int dmabounce_sync_for_cpu(struct device *, dma_addr_t, unsigned long,
-		size_t, enum dma_data_direction);
-int dmabounce_sync_for_device(struct device *, dma_addr_t, unsigned long,
-		size_t, enum dma_data_direction);
-#else
-static inline int dmabounce_sync_for_cpu(struct device *d, dma_addr_t addr,
-	unsigned long offset, size_t size, enum dma_data_direction dir)
+static inline int dma_mmap_coherent(struct device *dev, struct vm_area_struct *vma,
+		      void *cpu_addr, dma_addr_t dma_addr, size_t size)
 {
-	return 1;
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+	return ops->mmap_attrs(dev, vma, cpu_addr, dma_addr, size, NULL);
 }
 
-static inline int dmabounce_sync_for_device(struct device *d, dma_addr_t addr,
-	unsigned long offset, size_t size, enum dma_data_direction dir)
+static inline int dma_mmap_writecombine(struct device *dev, struct vm_area_struct *vma,
+		      void *cpu_addr, dma_addr_t dma_addr, size_t size)
 {
-	return 1;
+	DEFINE_DMA_ATTRS(attrs);
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+	dma_set_attr(DMA_ATTR_WRITE_COMBINE, &attrs);
+	return ops->mmap_attrs(dev, vma, cpu_addr, dma_addr, size, &attrs);
 }
 
-
-static inline dma_addr_t __dma_map_single(struct device *dev, void *cpu_addr,
-		size_t size, enum dma_data_direction dir)
+static inline int dma_map_sg(struct device *dev, struct scatterlist *sg,
+				   int nents, enum dma_data_direction dir)
 {
-	__dma_single_cpu_to_dev(cpu_addr, size, dir);
-	return virt_to_dma(dev, cpu_addr);
-}
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+	int i, ents;
+	struct scatterlist *s;
 
-static inline dma_addr_t __dma_map_page(struct device *dev, struct page *page,
-	     unsigned long offset, size_t size, enum dma_data_direction dir)
-{
-	__dma_page_cpu_to_dev(page, offset, size, dir);
-	return pfn_to_dma(dev, page_to_pfn(page)) + offset;
-}
+	for_each_sg(sg, s, nents, i)
+		kmemcheck_mark_initialized(sg_virt(s), s->length);
+	BUG_ON(!valid_dma_direction(dir));
+	ents = ops->map_sg(dev, sg, nents, dir);
+	debug_dma_map_sg(dev, sg, nents, ents, dir);
 
-static inline void __dma_unmap_single(struct device *dev, dma_addr_t handle,
-		size_t size, enum dma_data_direction dir)
-{
-	__dma_single_dev_to_cpu(dma_to_virt(dev, handle), size, dir);
+	return ents;
 }
 
-static inline void __dma_unmap_page(struct device *dev, dma_addr_t handle,
-		size_t size, enum dma_data_direction dir)
+static inline void dma_unmap_sg(struct device *dev, struct scatterlist *sg,
+				      int nents, enum dma_data_direction dir)
 {
-	__dma_page_dev_to_cpu(pfn_to_page(dma_to_pfn(dev, handle)),
-		handle & ~PAGE_MASK, size, dir);
-}
-#endif /* CONFIG_DMABOUNCE */
-
-/**
- * dma_map_single - map a single buffer for streaming DMA
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @cpu_addr: CPU direct mapped address of buffer
- * @size: size of buffer to map
- * @dir: DMA transfer direction
- *
- * Ensure that any data held in the cache is appropriately discarded
- * or written back.
- *
- * The device owns this memory once this call has completed.  The CPU
- * can regain ownership by calling dma_unmap_single() or
- * dma_sync_single_for_cpu().
- */
-static inline dma_addr_t dma_map_single(struct device *dev, void *cpu_addr,
-		size_t size, enum dma_data_direction dir)
-{
-	dma_addr_t addr;
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
 
 	BUG_ON(!valid_dma_direction(dir));
-
-	addr = __dma_map_single(dev, cpu_addr, size, dir);
-	debug_dma_map_page(dev, virt_to_page(cpu_addr),
-			(unsigned long)cpu_addr & ~PAGE_MASK, size,
-			dir, addr, true);
-
-	return addr;
+	debug_dma_unmap_sg(dev, sg, nents, dir);
+	if (ops->unmap_sg)
+		ops->unmap_sg(dev, sg, nents, dir);
 }
 
-/**
- * dma_map_page - map a portion of a page for streaming DMA
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @page: page that buffer resides in
- * @offset: offset into page for start of buffer
- * @size: size of buffer to map
- * @dir: DMA transfer direction
- *
- * Ensure that any data held in the cache is appropriately discarded
- * or written back.
- *
- * The device owns this memory once this call has completed.  The CPU
- * can regain ownership by calling dma_unmap_page().
- */
-static inline dma_addr_t dma_map_page(struct device *dev, struct page *page,
-	     unsigned long offset, size_t size, enum dma_data_direction dir)
+static inline void
+dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
+		    int nelems, enum dma_data_direction dir)
 {
-	dma_addr_t addr;
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
 
 	BUG_ON(!valid_dma_direction(dir));
-
-	addr = __dma_map_page(dev, page, offset, size, dir);
-	debug_dma_map_page(dev, page, offset, size, dir, addr, false);
-
-	return addr;
+	if (ops->sync_sg_for_cpu)
+		ops->sync_sg_for_cpu(dev, sg, nelems, dir);
+	debug_dma_sync_sg_for_cpu(dev, sg, nelems, dir);
 }
 
-/**
- * dma_unmap_single - unmap a single buffer previously mapped
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @handle: DMA address of buffer
- * @size: size of buffer (same as passed to dma_map_single)
- * @dir: DMA transfer direction (same as passed to dma_map_single)
- *
- * Unmap a single streaming mode DMA translation.  The handle and size
- * must match what was provided in the previous dma_map_single() call.
- * All other usages are undefined.
- *
- * After this call, reads by the CPU to the buffer are guaranteed to see
- * whatever the device wrote there.
- */
-static inline void dma_unmap_single(struct device *dev, dma_addr_t handle,
-		size_t size, enum dma_data_direction dir)
+static inline void
+dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
+		       int nelems, enum dma_data_direction dir)
 {
-	debug_dma_unmap_page(dev, handle, size, dir, true);
-	__dma_unmap_single(dev, handle, size, dir);
-}
+	struct arm_dma_map_ops *ops = get_dma_ops(dev);
+
+	BUG_ON(!valid_dma_direction(dir));
+	if (ops->sync_sg_for_device)
+		ops->sync_sg_for_device(dev, sg, nelems, dir);
+	debug_dma_sync_sg_for_device(dev, sg, nelems, dir);
 
-/**
- * dma_unmap_page - unmap a buffer previously mapped through dma_map_page()
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @handle: DMA address of buffer
- * @size: size of buffer (same as passed to dma_map_page)
- * @dir: DMA transfer direction (same as passed to dma_map_page)
- *
- * Unmap a page streaming mode DMA translation.  The handle and size
- * must match what was provided in the previous dma_map_page() call.
- * All other usages are undefined.
- *
- * After this call, reads by the CPU to the buffer are guaranteed to see
- * whatever the device wrote there.
- */
-static inline void dma_unmap_page(struct device *dev, dma_addr_t handle,
-		size_t size, enum dma_data_direction dir)
-{
-	debug_dma_unmap_page(dev, handle, size, dir, false);
-	__dma_unmap_page(dev, handle, size, dir);
 }
 
-/**
- * dma_sync_single_range_for_cpu
- * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
- * @handle: DMA address of buffer
- * @offset: offset of region to start sync
- * @size: size of region to sync
- * @dir: DMA transfer direction (same as passed to dma_map_single)
- *
- * Make physical memory consistent for a single streaming mode DMA
- * translation after a transfer.
+/********************************/
+
+/*
+ * Return whether the given device DMA address mask can be supported
+ * properly.  For example, if your device can only drive the low 24-bits
+ * during bus mastering, then you would pass 0x00ffffff as the mask
+ * to this function.
  *
- * If you perform a dma_map_single() but wish to interrogate the
- * buffer using the cpu, yet do not wish to teardown the PCI dma
- * mapping, you must call this function before doing so.  At the
- * next point you give the PCI dma address back to the card, you
- * must first the perform a dma_sync_for_device, and then the
- * device again owns the buffer.
+ * FIXME: This should really be a platform specific issue - we should
+ * return false if GFP_DMA allocations may not satisfy the supplied 'mask'.
  */
-static inline void dma_sync_single_range_for_cpu(struct device *dev,
-		dma_addr_t handle, unsigned long offset, size_t size,
-		enum dma_data_direction dir)
+static inline int dma_supported(struct device *dev, u64 mask)
 {
-	BUG_ON(!valid_dma_direction(dir));
-
-	debug_dma_sync_single_for_cpu(dev, handle + offset, size, dir);
-
-	if (!dmabounce_sync_for_cpu(dev, handle, offset, size, dir))
-		return;
-
-	__dma_single_dev_to_cpu(dma_to_virt(dev, handle) + offset, size, dir);
+	if (mask < ISA_DMA_THRESHOLD)
+		return 0;
+	return 1;
 }
 
-static inline void dma_sync_single_range_for_device(struct device *dev,
-		dma_addr_t handle, unsigned long offset, size_t size,
-		enum dma_data_direction dir)
+static inline int dma_set_mask(struct device *dev, u64 dma_mask)
 {
-	BUG_ON(!valid_dma_direction(dir));
-
-	debug_dma_sync_single_for_device(dev, handle + offset, size, dir);
-
-	if (!dmabounce_sync_for_device(dev, handle, offset, size, dir))
-		return;
-
-	__dma_single_cpu_to_dev(dma_to_virt(dev, handle) + offset, size, dir);
-}
+#ifdef CONFIG_DMABOUNCE
+	if (dev->archdata.dmabounce) {
+		if (dma_mask >= ISA_DMA_THRESHOLD)
+			return 0;
+		else
+			return -EIO;
+	}
+#endif
+	if (!dev->dma_mask || !dma_supported(dev, dma_mask))
+		return -EIO;
 
-static inline void dma_sync_single_for_cpu(struct device *dev,
-		dma_addr_t handle, size_t size, enum dma_data_direction dir)
-{
-	dma_sync_single_range_for_cpu(dev, handle, 0, size, dir);
-}
+	*dev->dma_mask = dma_mask;
 
-static inline void dma_sync_single_for_device(struct device *dev,
-		dma_addr_t handle, size_t size, enum dma_data_direction dir)
-{
-	dma_sync_single_range_for_device(dev, handle, 0, size, dir);
+	return 0;
 }
 
 /*
- * The scatter list versions of the above methods.
+ * DMA errors are defined by all-bits-set in the DMA address.
  */
-extern int dma_map_sg(struct device *, struct scatterlist *, int,
-		enum dma_data_direction);
-extern void dma_unmap_sg(struct device *, struct scatterlist *, int,
-		enum dma_data_direction);
-extern void dma_sync_sg_for_cpu(struct device *, struct scatterlist *, int,
-		enum dma_data_direction);
-extern void dma_sync_sg_for_device(struct device *, struct scatterlist *, int,
-		enum dma_data_direction);
-
+static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
+{
+	return dma_addr == ~0;
+}
 
 #endif /* __KERNEL__ */
 #endif
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 82a093c..f8c6972 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -18,6 +18,7 @@
 #include <linux/device.h>
 #include <linux/dma-mapping.h>
 #include <linux/highmem.h>
+#include <linux/slab.h>
 
 #include <asm/memory.h>
 #include <asm/highmem.h>
@@ -25,6 +26,389 @@
 #include <asm/tlbflush.h>
 #include <asm/sizes.h>
 
+#ifdef __arch_page_to_dma
+#error Please update to __arch_pfn_to_dma
+#endif
+
+/*
+ * dma_to_pfn/pfn_to_dma/dma_to_virt/virt_to_dma are architecture private
+ * functions used internally by the DMA-mapping API to provide DMA
+ * addresses. They must not be used by drivers.
+ */
+#ifndef __arch_pfn_to_dma
+static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
+{
+	return (dma_addr_t)__pfn_to_bus(pfn);
+}
+
+static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
+{
+	return __bus_to_pfn(addr);
+}
+
+static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
+{
+	return (void *)__bus_to_virt(addr);
+}
+
+static inline dma_addr_t virt_to_dma(struct device *dev, void *addr)
+{
+	return (dma_addr_t)__virt_to_bus((unsigned long)(addr));
+}
+#else
+static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
+{
+	return __arch_pfn_to_dma(dev, pfn);
+}
+
+static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
+{
+	return __arch_dma_to_pfn(dev, addr);
+}
+
+static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
+{
+	return __arch_dma_to_virt(dev, addr);
+}
+
+static inline dma_addr_t virt_to_dma(struct device *dev, void *addr)
+{
+	return __arch_virt_to_dma(dev, addr);
+}
+#endif
+
+/*
+ * The DMA API is built upon the notion of "buffer ownership".  A buffer
+ * is either exclusively owned by the CPU (and therefore may be accessed
+ * by it) or exclusively owned by the DMA device.  These helper functions
+ * represent the transitions between these two ownership states.
+ *
+ * Note, however, that on later ARMs, this notion does not work due to
+ * speculative prefetches.  We model our approach on the assumption that
+ * the CPU does do speculative prefetches, which means we clean caches
+ * before transfers and delay cache invalidation until transfer completion.
+ *
+ * Private support functions: these are not part of the API and are
+ * liable to change.  Drivers must not use these.
+ */
+static inline void __dma_single_cpu_to_dev(const void *kaddr, size_t size,
+	enum dma_data_direction dir)
+{
+	extern void ___dma_single_cpu_to_dev(const void *, size_t,
+		enum dma_data_direction);
+
+	if (!arch_is_coherent())
+		___dma_single_cpu_to_dev(kaddr, size, dir);
+}
+
+static inline void __dma_single_dev_to_cpu(const void *kaddr, size_t size,
+	enum dma_data_direction dir)
+{
+	extern void ___dma_single_dev_to_cpu(const void *, size_t,
+		enum dma_data_direction);
+
+	if (!arch_is_coherent())
+		___dma_single_dev_to_cpu(kaddr, size, dir);
+}
+
+static inline void __dma_page_cpu_to_dev(struct page *page, unsigned long off,
+	size_t size, enum dma_data_direction dir)
+{
+	extern void ___dma_page_cpu_to_dev(struct page *, unsigned long,
+		size_t, enum dma_data_direction);
+
+	if (!arch_is_coherent())
+		___dma_page_cpu_to_dev(page, off, size, dir);
+}
+
+static inline void __dma_page_dev_to_cpu(struct page *page, unsigned long off,
+	size_t size, enum dma_data_direction dir)
+{
+	extern void ___dma_page_dev_to_cpu(struct page *, unsigned long,
+		size_t, enum dma_data_direction);
+
+	if (!arch_is_coherent())
+		___dma_page_dev_to_cpu(page, off, size, dir);
+}
+
+
+#ifdef CONFIG_DMABOUNCE
+/*
+ * For SA-1111, IXP425, and ADI systems  the dma-mapping functions are "magic"
+ * and utilize bounce buffers as needed to work around limited DMA windows.
+ *
+ * On the SA-1111, a bug limits DMA to only certain regions of RAM.
+ * On the IXP425, the PCI inbound window is 64MB (256MB total RAM)
+ * On some ADI engineering systems, PCI inbound window is 32MB (12MB total RAM)
+ *
+ * The following are helper functions used by the dmabounce subystem
+ *
+ */
+
+/**
+ * dmabounce_register_dev
+ *
+ * @dev: valid struct device pointer
+ * @small_buf_size: size of buffers to use with small buffer pool
+ * @large_buf_size: size of buffers to use with large buffer pool (can be 0)
+ *
+ * This function should be called by low-level platform code to register
+ * a device as requireing DMA buffer bouncing. The function will allocate
+ * appropriate DMA pools for the device.
+ *
+ */
+extern int dmabounce_register_dev(struct device *, unsigned long,
+		unsigned long);
+
+/**
+ * dmabounce_unregister_dev
+ *
+ * @dev: valid struct device pointer
+ *
+ * This function should be called by low-level platform code when device
+ * that was previously registered with dmabounce_register_dev is removed
+ * from the system.
+ *
+ */
+extern void dmabounce_unregister_dev(struct device *);
+
+/**
+ * dma_needs_bounce
+ *
+ * @dev: valid struct device pointer
+ * @dma_handle: dma_handle of unbounced buffer
+ * @size: size of region being mapped
+ *
+ * Platforms that utilize the dmabounce mechanism must implement
+ * this function.
+ *
+ * The dmabounce routines call this function whenever a dma-mapping
+ * is requested to determine whether a given buffer needs to be bounced
+ * or not. The function must return 0 if the buffer is OK for
+ * DMA access and 1 if the buffer needs to be bounced.
+ *
+ */
+extern int dma_needs_bounce(struct device*, dma_addr_t, size_t);
+
+/*
+ * The DMA API, implemented by dmabounce.c.  See below for descriptions.
+ */
+extern dma_addr_t __dma_map_single(struct device *, void *, size_t,
+		enum dma_data_direction);
+extern void __dma_unmap_single(struct device *, dma_addr_t, size_t,
+		enum dma_data_direction);
+extern dma_addr_t __dma_map_page(struct device *, struct page *,
+		unsigned long, size_t, enum dma_data_direction);
+extern void __dma_unmap_page(struct device *, dma_addr_t, size_t,
+		enum dma_data_direction);
+
+/*
+ * Private functions
+ */
+int dmabounce_sync_for_cpu(struct device *, dma_addr_t, unsigned long,
+		size_t, enum dma_data_direction);
+int dmabounce_sync_for_device(struct device *, dma_addr_t, unsigned long,
+		size_t, enum dma_data_direction);
+#else
+static inline int dmabounce_sync_for_cpu(struct device *d, dma_addr_t addr,
+	unsigned long offset, size_t size, enum dma_data_direction dir)
+{
+	return 1;
+}
+
+static inline int dmabounce_sync_for_device(struct device *d, dma_addr_t addr,
+	unsigned long offset, size_t size, enum dma_data_direction dir)
+{
+	return 1;
+}
+
+
+static inline dma_addr_t __dma_map_single(struct device *dev, void *cpu_addr,
+		size_t size, enum dma_data_direction dir)
+{
+	__dma_single_cpu_to_dev(cpu_addr, size, dir);
+	return virt_to_dma(dev, cpu_addr);
+}
+
+static inline dma_addr_t __dma_map_page(struct device *dev, struct page *page,
+	     unsigned long offset, size_t size, enum dma_data_direction dir)
+{
+	__dma_page_cpu_to_dev(page, offset, size, dir);
+	return pfn_to_dma(dev, page_to_pfn(page)) + offset;
+}
+
+static inline void __dma_unmap_single(struct device *dev, dma_addr_t handle,
+		size_t size, enum dma_data_direction dir)
+{
+	__dma_single_dev_to_cpu(dma_to_virt(dev, handle), size, dir);
+}
+
+static inline void __dma_unmap_page(struct device *dev, dma_addr_t handle,
+		size_t size, enum dma_data_direction dir)
+{
+	__dma_page_dev_to_cpu(pfn_to_page(dma_to_pfn(dev, handle)),
+		handle & ~PAGE_MASK, size, dir);
+}
+#endif /* CONFIG_DMABOUNCE */
+
+/**
+ * dma_map_single - map a single buffer for streaming DMA
+ * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
+ * @cpu_addr: CPU direct mapped address of buffer
+ * @size: size of buffer to map
+ * @dir: DMA transfer direction
+ *
+ * Ensure that any data held in the cache is appropriately discarded
+ * or written back.
+ *
+ * The device owns this memory once this call has completed.  The CPU
+ * can regain ownership by calling dma_unmap_single() or
+ * dma_sync_single_for_cpu().
+ */
+static inline dma_addr_t arm_dma_map_single(struct device *dev, void *cpu_addr,
+		size_t size, enum dma_data_direction dir)
+{
+	dma_addr_t addr;
+
+	BUG_ON(!valid_dma_direction(dir));
+
+	addr = __dma_map_single(dev, cpu_addr, size, dir);
+	debug_dma_map_page(dev, virt_to_page(cpu_addr),
+			(unsigned long)cpu_addr & ~PAGE_MASK, size,
+			dir, addr, true);
+
+	return addr;
+}
+
+/**
+ * dma_map_page - map a portion of a page for streaming DMA
+ * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
+ * @page: page that buffer resides in
+ * @offset: offset into page for start of buffer
+ * @size: size of buffer to map
+ * @dir: DMA transfer direction
+ *
+ * Ensure that any data held in the cache is appropriately discarded
+ * or written back.
+ *
+ * The device owns this memory once this call has completed.  The CPU
+ * can regain ownership by calling dma_unmap_page().
+ */
+static inline dma_addr_t arm_dma_map_page(struct device *dev, struct page *page,
+	     unsigned long offset, size_t size, enum dma_data_direction dir)
+{
+	dma_addr_t addr;
+
+	BUG_ON(!valid_dma_direction(dir));
+
+	addr = __dma_map_page(dev, page, offset, size, dir);
+	debug_dma_map_page(dev, page, offset, size, dir, addr, false);
+
+	return addr;
+}
+
+/**
+ * dma_unmap_single - unmap a single buffer previously mapped
+ * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
+ * @handle: DMA address of buffer
+ * @size: size of buffer (same as passed to dma_map_single)
+ * @dir: DMA transfer direction (same as passed to dma_map_single)
+ *
+ * Unmap a single streaming mode DMA translation.  The handle and size
+ * must match what was provided in the previous dma_map_single() call.
+ * All other usages are undefined.
+ *
+ * After this call, reads by the CPU to the buffer are guaranteed to see
+ * whatever the device wrote there.
+ */
+static inline void arm_dma_unmap_single(struct device *dev, dma_addr_t handle,
+		size_t size, enum dma_data_direction dir)
+{
+	debug_dma_unmap_page(dev, handle, size, dir, true);
+	__dma_unmap_single(dev, handle, size, dir);
+}
+
+/**
+ * dma_unmap_page - unmap a buffer previously mapped through dma_map_page()
+ * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
+ * @handle: DMA address of buffer
+ * @size: size of buffer (same as passed to dma_map_page)
+ * @dir: DMA transfer direction (same as passed to dma_map_page)
+ *
+ * Unmap a page streaming mode DMA translation.  The handle and size
+ * must match what was provided in the previous dma_map_page() call.
+ * All other usages are undefined.
+ *
+ * After this call, reads by the CPU to the buffer are guaranteed to see
+ * whatever the device wrote there.
+ */
+static inline void arm_dma_unmap_page(struct device *dev, dma_addr_t handle,
+		size_t size, enum dma_data_direction dir)
+{
+	debug_dma_unmap_page(dev, handle, size, dir, false);
+	__dma_unmap_page(dev, handle, size, dir);
+}
+
+/**
+ * dma_sync_single_range_for_cpu
+ * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
+ * @handle: DMA address of buffer
+ * @offset: offset of region to start sync
+ * @size: size of region to sync
+ * @dir: DMA transfer direction (same as passed to dma_map_single)
+ *
+ * Make physical memory consistent for a single streaming mode DMA
+ * translation after a transfer.
+ *
+ * If you perform a dma_map_single() but wish to interrogate the
+ * buffer using the cpu, yet do not wish to teardown the PCI dma
+ * mapping, you must call this function before doing so.  At the
+ * next point you give the PCI dma address back to the card, you
+ * must first the perform a dma_sync_for_device, and then the
+ * device again owns the buffer.
+ */
+static inline void arm_dma_sync_single_range_for_cpu(struct device *dev,
+		dma_addr_t handle, unsigned long offset, size_t size,
+		enum dma_data_direction dir)
+{
+	BUG_ON(!valid_dma_direction(dir));
+
+	debug_dma_sync_single_for_cpu(dev, handle + offset, size, dir);
+
+	if (!dmabounce_sync_for_cpu(dev, handle, offset, size, dir))
+		return;
+
+	__dma_single_dev_to_cpu(dma_to_virt(dev, handle) + offset, size, dir);
+}
+
+static inline void arm_dma_sync_single_range_for_device(struct device *dev,
+		dma_addr_t handle, unsigned long offset, size_t size,
+		enum dma_data_direction dir)
+{
+	BUG_ON(!valid_dma_direction(dir));
+
+	debug_dma_sync_single_for_device(dev, handle + offset, size, dir);
+
+	if (!dmabounce_sync_for_device(dev, handle, offset, size, dir))
+		return;
+
+	__dma_single_cpu_to_dev(dma_to_virt(dev, handle) + offset, size, dir);
+}
+
+static inline void arm_dma_sync_single_for_cpu(struct device *dev,
+		dma_addr_t handle, unsigned long offset, size_t size,
+		enum dma_data_direction dir)
+{
+	dma_sync_single_range_for_cpu(dev, handle, offset, size, dir);
+}
+
+static inline void arm_dma_sync_single_for_device(struct device *dev,
+		dma_addr_t handle, unsigned long offset, size_t size,
+		enum dma_data_direction dir)
+{
+	dma_sync_single_range_for_device(dev, handle, offset, size, dir);
+}
+
 static u64 get_coherent_dma_mask(struct device *dev)
 {
 	u64 mask = ISA_DMA_THRESHOLD;
@@ -224,7 +608,7 @@ __dma_alloc_remap(struct page *page, size_t size, gfp_t gfp, pgprot_t prot)
 		u32 off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1);
 
 		pte = consistent_pte[idx] + off;
-		c->vm_pages = page;
+		c->priv = page;
 
 		do {
 			BUG_ON(!pte_none(*pte));
@@ -330,39 +714,36 @@ __dma_alloc(struct device *dev, size_t size, dma_addr_t *handle, gfp_t gfp,
  * Allocate DMA-coherent memory space and return both the kernel remapped
  * virtual and bus address for that space.
  */
-void *
-dma_alloc_coherent(struct device *dev, size_t size, dma_addr_t *handle, gfp_t gfp)
+void *arm_dma_alloc_attrs(struct device *dev, size_t size, dma_addr_t *handle,
+			  gfp_t gfp, struct dma_attrs *attrs)
 {
 	void *memory;
 
-	if (dma_alloc_from_coherent(dev, size, handle, &memory))
-		return memory;
-
-	return __dma_alloc(dev, size, handle, gfp,
-			   pgprot_dmacoherent(pgprot_kernel));
-}
-EXPORT_SYMBOL(dma_alloc_coherent);
-
-/*
- * Allocate a writecombining region, in much the same way as
- * dma_alloc_coherent above.
- */
-void *
-dma_alloc_writecombine(struct device *dev, size_t size, dma_addr_t *handle, gfp_t gfp)
-{
-	return __dma_alloc(dev, size, handle, gfp,
+	if (dma_get_attr(DMA_ATTR_WRITE_COMBINE, attrs)) {
+		return __dma_alloc(dev, size, handle, gfp,
 			   pgprot_writecombine(pgprot_kernel));
+	} else {
+		if (dma_alloc_from_coherent(dev, size, handle, &memory))
+			return memory;
+		return __dma_alloc(dev, size, handle, gfp,
+			   pgprot_dmacoherent(pgprot_kernel));
+	}
 }
-EXPORT_SYMBOL(dma_alloc_writecombine);
 
-static int dma_mmap(struct device *dev, struct vm_area_struct *vma,
-		    void *cpu_addr, dma_addr_t dma_addr, size_t size)
+static int arm_dma_mmap_attrs(struct device *dev, struct vm_area_struct *vma,
+		    void *cpu_addr, dma_addr_t dma_addr, size_t size,
+		    struct dma_attrs *attrs)
 {
-	int ret = -ENXIO;
-#ifdef CONFIG_MMU
 	unsigned long user_size, kern_size;
 	struct arm_vmregion *c;
+	int ret = -ENXIO;
+
+	if (dma_get_attr(DMA_ATTR_WRITE_COMBINE, attrs))
+		vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+	else
+		vma->vm_page_prot = pgprot_dmacoherent(vma->vm_page_prot);
 
+#ifdef CONFIG_MMU
 	user_size = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
 
 	c = arm_vmregion_find(&consistent_head, (unsigned long)cpu_addr);
@@ -373,8 +754,9 @@ static int dma_mmap(struct device *dev, struct vm_area_struct *vma,
 
 		if (off < kern_size &&
 		    user_size <= (kern_size - off)) {
+			struct page *vm_pages = c->priv;
 			ret = remap_pfn_range(vma, vma->vm_start,
-					      page_to_pfn(c->vm_pages) + off,
+					      page_to_pfn(vm_pages) + off,
 					      user_size << PAGE_SHIFT,
 					      vma->vm_page_prot);
 		}
@@ -384,27 +766,12 @@ static int dma_mmap(struct device *dev, struct vm_area_struct *vma,
 	return ret;
 }
 
-int dma_mmap_coherent(struct device *dev, struct vm_area_struct *vma,
-		      void *cpu_addr, dma_addr_t dma_addr, size_t size)
-{
-	vma->vm_page_prot = pgprot_dmacoherent(vma->vm_page_prot);
-	return dma_mmap(dev, vma, cpu_addr, dma_addr, size);
-}
-EXPORT_SYMBOL(dma_mmap_coherent);
-
-int dma_mmap_writecombine(struct device *dev, struct vm_area_struct *vma,
-			  void *cpu_addr, dma_addr_t dma_addr, size_t size)
-{
-	vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
-	return dma_mmap(dev, vma, cpu_addr, dma_addr, size);
-}
-EXPORT_SYMBOL(dma_mmap_writecombine);
-
 /*
  * free a page as defined by the above mapping.
  * Must not be called with IRQs disabled.
  */
-void dma_free_coherent(struct device *dev, size_t size, void *cpu_addr, dma_addr_t handle)
+void arm_dma_free_attrs(struct device *dev, size_t size, void *cpu_addr,
+			dma_addr_t handle, struct dma_attrs *attrs)
 {
 	WARN_ON(irqs_disabled());
 
@@ -418,7 +785,6 @@ void dma_free_coherent(struct device *dev, size_t size, void *cpu_addr, dma_addr
 
 	__dma_free_buffer(pfn_to_page(dma_to_pfn(dev, handle)), size);
 }
-EXPORT_SYMBOL(dma_free_coherent);
 
 /*
  * Make an area consistent for devices.
@@ -443,7 +809,6 @@ void ___dma_single_cpu_to_dev(const void *kaddr, size_t size,
 	}
 	/* FIXME: non-speculating: flush on bidirectional mappings? */
 }
-EXPORT_SYMBOL(___dma_single_cpu_to_dev);
 
 void ___dma_single_dev_to_cpu(const void *kaddr, size_t size,
 	enum dma_data_direction dir)
@@ -459,7 +824,6 @@ void ___dma_single_dev_to_cpu(const void *kaddr, size_t size,
 
 	dmac_unmap_area(kaddr, size, dir);
 }
-EXPORT_SYMBOL(___dma_single_dev_to_cpu);
 
 static void dma_cache_maint_page(struct page *page, unsigned long offset,
 	size_t size, enum dma_data_direction dir,
@@ -520,7 +884,6 @@ void ___dma_page_cpu_to_dev(struct page *page, unsigned long off,
 	}
 	/* FIXME: non-speculating: flush on bidirectional mappings? */
 }
-EXPORT_SYMBOL(___dma_page_cpu_to_dev);
 
 void ___dma_page_dev_to_cpu(struct page *page, unsigned long off,
 	size_t size, enum dma_data_direction dir)
@@ -540,10 +903,9 @@ void ___dma_page_dev_to_cpu(struct page *page, unsigned long off,
 	if (dir != DMA_TO_DEVICE && off == 0 && size >= PAGE_SIZE)
 		set_bit(PG_dcache_clean, &page->flags);
 }
-EXPORT_SYMBOL(___dma_page_dev_to_cpu);
 
 /**
- * dma_map_sg - map a set of SG buffers for streaming mode DMA
+ * arm_dma_map_sg - map a set of SG buffers for streaming mode DMA
  * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
  * @sg: list of buffers
  * @nents: number of buffers to map
@@ -558,7 +920,7 @@ EXPORT_SYMBOL(___dma_page_dev_to_cpu);
  * Device ownership issues as mentioned for dma_map_single are the same
  * here.
  */
-int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
+static int arm_dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
 		enum dma_data_direction dir)
 {
 	struct scatterlist *s;
@@ -580,10 +942,9 @@ int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
 		__dma_unmap_page(dev, sg_dma_address(s), sg_dma_len(s), dir);
 	return 0;
 }
-EXPORT_SYMBOL(dma_map_sg);
 
 /**
- * dma_unmap_sg - unmap a set of SG buffers mapped by dma_map_sg
+ * arm_dma_unmap_sg - unmap a set of SG buffers mapped by dma_map_sg
  * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
  * @sg: list of buffers
  * @nents: number of buffers to unmap (same as was passed to dma_map_sg)
@@ -592,7 +953,7 @@ EXPORT_SYMBOL(dma_map_sg);
  * Unmap a set of streaming mode DMA translations.  Again, CPU access
  * rules concerning calls here are the same as for dma_unmap_single().
  */
-void dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents,
+static void arm_dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents,
 		enum dma_data_direction dir)
 {
 	struct scatterlist *s;
@@ -603,7 +964,6 @@ void dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents,
 	for_each_sg(sg, s, nents, i)
 		__dma_unmap_page(dev, sg_dma_address(s), sg_dma_len(s), dir);
 }
-EXPORT_SYMBOL(dma_unmap_sg);
 
 /**
  * dma_sync_sg_for_cpu
@@ -612,7 +972,7 @@ EXPORT_SYMBOL(dma_unmap_sg);
  * @nents: number of buffers to map (returned from dma_map_sg)
  * @dir: DMA transfer direction (same as was passed to dma_map_sg)
  */
-void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
+static void arm_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
 			int nents, enum dma_data_direction dir)
 {
 	struct scatterlist *s;
@@ -629,16 +989,15 @@ void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg,
 
 	debug_dma_sync_sg_for_cpu(dev, sg, nents, dir);
 }
-EXPORT_SYMBOL(dma_sync_sg_for_cpu);
 
 /**
- * dma_sync_sg_for_device
+ * arm_dma_sync_sg_for_device
  * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
  * @sg: list of buffers
  * @nents: number of buffers to map (returned from dma_map_sg)
  * @dir: DMA transfer direction (same as was passed to dma_map_sg)
  */
-void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
+static void arm_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
 			int nents, enum dma_data_direction dir)
 {
 	struct scatterlist *s;
@@ -655,7 +1014,23 @@ void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg,
 
 	debug_dma_sync_sg_for_device(dev, sg, nents, dir);
 }
-EXPORT_SYMBOL(dma_sync_sg_for_device);
+
+struct arm_dma_map_ops dma_ops = {
+	.alloc_attrs = arm_dma_alloc_attrs,
+	.free_attrs = arm_dma_free_attrs,
+	.map_single = arm_dma_map_single,
+	.unmap_single = arm_dma_unmap_single,
+	.map_page = arm_dma_map_page,
+	.unmap_page = arm_dma_unmap_page,
+	.map_sg = arm_dma_map_sg,
+	.unmap_sg = arm_dma_unmap_sg,
+	.sync_single_for_device = arm_dma_sync_single_for_device,
+	.sync_single_for_cpu = arm_dma_sync_single_for_cpu,
+	.sync_sg_for_cpu = arm_dma_sync_sg_for_cpu,
+	.sync_sg_for_device = arm_dma_sync_sg_for_device,
+	.mmap_attrs = arm_dma_mmap_attrs,
+};
+EXPORT_SYMBOL_GPL(dma_ops);
 
 #define PREALLOC_DMA_DEBUG_ENTRIES	4096
 
diff --git a/arch/arm/mm/vmregion.h b/arch/arm/mm/vmregion.h
index 15e9f04..6bbc402 100644
--- a/arch/arm/mm/vmregion.h
+++ b/arch/arm/mm/vmregion.h
@@ -17,7 +17,7 @@ struct arm_vmregion {
 	struct list_head	vm_list;
 	unsigned long		vm_start;
 	unsigned long		vm_end;
-	struct page		*vm_pages;
+	void			*priv;
 	int			vm_active;
 };
 
diff --git a/include/linux/dma-attrs.h b/include/linux/dma-attrs.h
index 71ad34e..ada61e1 100644
--- a/include/linux/dma-attrs.h
+++ b/include/linux/dma-attrs.h
@@ -13,6 +13,7 @@
 enum dma_attr {
 	DMA_ATTR_WRITE_BARRIER,
 	DMA_ATTR_WEAK_ORDERING,
+	DMA_ATTR_WRITE_COMBINE,
 	DMA_ATTR_MAX,
 };
 
-- 
1.7.1.569.g6f426

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC 2/2] ARM: initial proof-of-concept IOMMU mapper for DMA-mapping

[Marek Szyprowski]

Add initial proof of concept implementation of DMA-mapping API for
devices that have IOMMU support. Right now only dma_alloc_coherent,
dma_free_coherent and dma_mmap_coherent functions are supported.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
---
 arch/arm/Kconfig                 |    1 +
 arch/arm/include/asm/device.h    |    2 +
 arch/arm/include/asm/dma-iommu.h |   30 ++++
 arch/arm/mm/dma-mapping.c        |  326 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 359 insertions(+), 0 deletions(-)
 create mode 100644 arch/arm/include/asm/dma-iommu.h

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 377a7a5..61900f1 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -8,6 +8,8 @@ config ARM
 	select RTC_LIB
 	select SYS_SUPPORTS_APM_EMULATION
 	select GENERIC_ATOMIC64 if (CPU_V6 || !CPU_32v6K || !AEABI)
+	select GENERIC_ALLOCATOR
+	select HAVE_DMA_ATTRS
 	select HAVE_OPROFILE if (HAVE_PERF_EVENTS)
 	select HAVE_ARCH_KGDB
 	select HAVE_KPROBES if (!XIP_KERNEL && !THUMB2_KERNEL)
diff --git a/arch/arm/include/asm/device.h b/arch/arm/include/asm/device.h
index 005791a..d3ec1e9 100644
--- a/arch/arm/include/asm/device.h
+++ b/arch/arm/include/asm/device.h
@@ -11,6 +11,8 @@ struct dev_archdata {
 #ifdef CONFIG_DMABOUNCE
 	struct dmabounce_device_info *dmabounce;
 #endif
+	void				*iommu_priv;
+	struct dma_iommu_mapping	*mapping;
 };
 
 struct pdev_archdata {
diff --git a/arch/arm/include/asm/dma-iommu.h b/arch/arm/include/asm/dma-iommu.h
new file mode 100644
index 0000000..c246ff3
--- /dev/null
+++ b/arch/arm/include/asm/dma-iommu.h
@@ -0,0 +1,30 @@
+#ifndef ASMARM_DMA_IOMMU_H
+#define ASMARM_DMA_IOMMU_H
+
+#ifdef __KERNEL__
+
+#include <linux/mm_types.h>
+#include <linux/scatterlist.h>
+#include <linux/dma-debug.h>
+#include <linux/kmemcheck.h>
+
+#include <asm/memory.h>
+
+struct dma_iommu_mapping {
+	/* iommu specific data */
+	struct iommu_domain	*domain;
+
+	/* address space data */
+	struct gen_pool		*pool;
+
+	dma_addr_t		base;
+	size_t			size;
+
+	atomic_t		ref_count;
+	struct mutex		lock;
+};
+
+int __init arm_iommu_assign_device(struct device *dev, dma_addr_t base, dma_addr_t size);
+
+#endif /* __KERNEL__ */
+#endif
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index f8c6972..b6397c1 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -19,6 +19,7 @@
 #include <linux/dma-mapping.h>
 #include <linux/highmem.h>
 #include <linux/slab.h>
+#include <linux/genalloc.h>
 
 #include <asm/memory.h>
 #include <asm/highmem.h>
@@ -26,6 +27,9 @@
 #include <asm/tlbflush.h>
 #include <asm/sizes.h>
 
+#include <linux/iommu.h>
+#include <asm/dma-iommu.h>
+
 #ifdef __arch_page_to_dma
 #error Please update to __arch_pfn_to_dma
 #endif
@@ -1040,3 +1044,325 @@ static int __init dma_debug_do_init(void)
 	return 0;
 }
 fs_initcall(dma_debug_do_init);
+
+
+/* IOMMU */
+
+/*
+ * Allocate a DMA buffer for 'dev' of size 'size' using the
+ * specified gfp mask.  Note that 'size' must be page aligned.
+ */
+static struct page **__iommu_alloc_buffer(struct device *dev, size_t size, gfp_t gfp)
+{
+	struct page **pages;
+	int count = size >> PAGE_SHIFT;
+	void *ptr;
+	int i;
+
+	pages = kzalloc(count * sizeof(struct page*), gfp);
+	if (!pages)
+		return NULL;
+
+	printk("IOMMU: page table allocated\n");
+
+	for (i=0; i<count; i++) {
+		pages[i] = alloc_page(gfp); //alloc_pages(gfp, 0);
+
+		if (!pages[i])
+			goto error;
+
+		/*
+		 * Ensure that the allocated pages are zeroed, and that any data
+		 * lurking in the kernel direct-mapped region is invalidated.
+		 */
+		ptr = page_address(pages[i]);
+		memset(ptr, 0, PAGE_SIZE);
+		dmac_flush_range(ptr, ptr + PAGE_SIZE);
+		outer_flush_range(__pa(ptr), __pa(ptr) + PAGE_SIZE);
+	}
+	printk("IOMMU: pages allocated\n");
+	return pages;
+error:
+	printk("IOMMU: error allocating pages\n");
+	while (--i)
+		if (pages[i])
+			__free_pages(pages[i], 0);
+	kfree(pages);
+	return NULL;
+}
+
+static int __iommu_free_buffer(struct device *dev, struct page **pages, size_t size)
+{
+	int count = size >> PAGE_SHIFT;
+	int i;
+	for (i=0; i< count; i++)
+		if (pages[i])
+			__free_pages(pages[i], 0);
+	kfree(pages);
+	return 0;
+}
+
+static void *
+__iommu_alloc_remap(struct page **pages, size_t size, gfp_t gfp, pgprot_t prot)
+{
+	struct arm_vmregion *c;
+	size_t align;
+	size_t count = size >> PAGE_SHIFT;
+	int bit;
+
+	if (!consistent_pte[0]) {
+		printk(KERN_ERR "%s: not initialised\n", __func__);
+		dump_stack();
+		return NULL;
+	}
+
+	/*
+	 * Align the virtual region allocation - maximum alignment is
+	 * a section size, minimum is a page size.  This helps reduce
+	 * fragmentation of the DMA space, and also prevents allocations
+	 * smaller than a section from crossing a section boundary.
+	 */
+	bit = fls(size - 1);
+	if (bit > SECTION_SHIFT)
+		bit = SECTION_SHIFT;
+	align = 1 << bit;
+
+	/*
+	 * Allocate a virtual address in the consistent mapping region.
+	 */
+	c = arm_vmregion_alloc(&consistent_head, align, size,
+			    gfp & ~(__GFP_DMA | __GFP_HIGHMEM));
+	if (c) {
+		pte_t *pte;
+		int idx = CONSISTENT_PTE_INDEX(c->vm_start);
+		int i = 0;
+		u32 off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1);
+
+		pte = consistent_pte[idx] + off;
+		c->priv = pages;
+
+		do {
+			BUG_ON(!pte_none(*pte));
+
+			set_pte_ext(pte, mk_pte(pages[i], prot), 0);
+			pte++;
+			off++;
+			i++;
+			if (off >= PTRS_PER_PTE) {
+				off = 0;
+				pte = consistent_pte[++idx];
+			}
+		} while (i < count);
+
+		dsb();
+
+		return (void *)c->vm_start;
+	}
+	return NULL;
+}
+
+static dma_addr_t __iommu_create_mapping(struct device *dev, struct page **pages, size_t size)
+{
+	struct dma_iommu_mapping *mapping = dev->archdata.mapping;
+	unsigned int count = size >> PAGE_SHIFT;
+	dma_addr_t dma_addr, iova;
+	int i, ret = 0;
+
+	printk("IOMMU: mapping %p\n", mapping);
+
+	iova = gen_pool_alloc(mapping->pool, size);
+
+	printk("IOMMU: gen_alloc res %x\n", iova);
+
+	if (iova == 0)
+		goto fail;
+
+	dma_addr = iova;
+
+	for (i=0; i<count; i++) {
+		unsigned int phys = page_to_phys(pages[i]);
+		ret = iommu_map(mapping->domain, iova, phys, 0, 0);
+		if (ret < 0)
+			goto fail;
+		iova += PAGE_SIZE;
+	}
+
+	return dma_addr;
+fail:
+	return 0;
+}
+
+static int __iommu_remove_mapping(struct device *dev, dma_addr_t iova, size_t size)
+{
+	struct dma_iommu_mapping *mapping = dev->archdata.mapping;
+	unsigned int count = size >> PAGE_SHIFT;
+	int i;
+
+	gen_pool_free(mapping->pool, iova, size);
+
+	for (i=0; i<count; i++) {
+		iommu_unmap(mapping->domain, iova, 0);
+		iova += PAGE_SIZE;
+	}
+	return 0;
+}
+
+int arm_iommu_init(struct device *dev);
+
+static void *arm_iommu_alloc_attrs(struct device *dev, size_t size,
+	    dma_addr_t *handle, gfp_t gfp, struct dma_attrs *attrs)
+{
+	struct dma_iommu_mapping *mapping = dev->archdata.mapping;
+	struct page **pages;
+	void *addr = NULL;
+	pgprot_t prot;
+
+	if (dma_get_attr(DMA_ATTR_WRITE_COMBINE, attrs))
+		prot = pgprot_writecombine(pgprot_kernel);
+	else
+		prot = pgprot_dmacoherent(pgprot_kernel);
+
+	arm_iommu_init(dev);
+
+	mutex_lock(&mapping->lock);
+
+	*handle = ~0;
+	size = PAGE_ALIGN(size);
+
+	printk("IOMMU: requested size %d\n", size);
+
+	pages = __iommu_alloc_buffer(dev, size, gfp);
+	if (!pages)
+		return NULL;
+
+	printk("IOMMU: allocated pages: %p\n", pages);
+
+	*handle = __iommu_create_mapping(dev, pages, size);
+
+	printk("IOMMU: created iova: %08x\n", *handle);
+
+	if (!*handle)
+		goto err_buffer;
+
+	addr = __iommu_alloc_remap(pages, size, gfp, prot);
+	if (!addr)
+		goto err_iommu;
+
+	printk("IOMMU: allocated iova %08x, virt %p\n", *handle, addr);
+
+err_iommu:
+err_buffer:
+	mutex_unlock(&mapping->lock);
+	return addr;
+}
+
+static int arm_iommu_mmap_attrs(struct device *dev, struct vm_area_struct *vma,
+		    void *cpu_addr, dma_addr_t dma_addr, size_t size,
+		    struct dma_attrs *attrs)
+{
+	unsigned long user_size;
+	struct arm_vmregion *c;
+
+	if (dma_get_attr(DMA_ATTR_WRITE_COMBINE, attrs))
+		vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+	else
+		vma->vm_page_prot = pgprot_dmacoherent(vma->vm_page_prot);
+
+	printk("IOMMU: mmap virt %p, dma %08x, size %d\n", cpu_addr, dma_addr, size);
+
+	user_size = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
+
+	c = arm_vmregion_find(&consistent_head, (unsigned long)cpu_addr);
+	if (c) {
+		struct page **pages = c->priv;
+
+		unsigned long uaddr = vma->vm_start;
+		unsigned long usize = vma->vm_end - vma->vm_start;
+		int i = 0;
+
+		do {
+			int ret;
+
+			ret = vm_insert_page(vma, uaddr, pages[i++]);
+			if (ret) {
+				printk(KERN_ERR "Remapping memory, error: %d\n", ret);
+				return ret;
+			}
+
+			uaddr += PAGE_SIZE;
+			usize -= PAGE_SIZE;
+		} while (usize > 0);
+	}
+	return 0;
+}
+
+/*
+ * free a page as defined by the above mapping.
+ * Must not be called with IRQs disabled.
+ */
+void arm_iommu_free_attrs(struct device *dev, size_t size, void *cpu_addr,
+			  dma_addr_t handle, struct dma_attrs *attrs)
+{
+	struct dma_iommu_mapping *mapping = dev->archdata.mapping;
+	struct arm_vmregion *c;
+	size = PAGE_ALIGN(size);
+
+	mutex_lock(&mapping->lock);
+	c = arm_vmregion_find(&consistent_head, (unsigned long)cpu_addr);
+	if (c) {
+		struct page **pages = c->priv;
+		__dma_free_remap(cpu_addr, size);
+		__iommu_remove_mapping(dev, handle, size);
+		__iommu_free_buffer(dev, pages, size);
+	}
+	mutex_unlock(&mapping->lock);
+}
+
+struct arm_dma_map_ops iommu_ops = {
+	.alloc_attrs = arm_iommu_alloc_attrs,
+	.free_attrs = arm_iommu_free_attrs,
+	.mmap_attrs = arm_iommu_mmap_attrs,
+};
+EXPORT_SYMBOL_GPL(iommu_ops);
+
+int arm_iommu_init(struct device *dev)
+{
+	struct dma_iommu_mapping *mapping = dev->archdata.mapping;
+
+	if (mapping->pool)
+		return 0;
+
+	mutex_init(&mapping->lock);
+
+	mapping->pool = gen_pool_create(16, -1);
+	if (!mapping->pool)
+		return -ENOMEM;
+
+	if (gen_pool_add(mapping->pool, mapping->base, mapping->size, -1) != 0)
+		return -ENOMEM;
+
+	mapping->domain = iommu_domain_alloc();
+	if (!mapping->domain)
+		return -ENOMEM;
+
+	if (iommu_attach_device(mapping->domain, dev) != 0)
+		return -ENOMEM;
+
+	return 0;
+}
+
+int __init arm_iommu_assign_device(struct device *dev, dma_addr_t base, dma_addr_t size)
+{
+	struct dma_iommu_mapping *mapping;
+	mapping = kzalloc(sizeof(struct dma_iommu_mapping), GFP_KERNEL);
+	if (!mapping)
+		return -ENOMEM;
+	mapping->base = base;
+	mapping->size = size;
+
+	dev->archdata.mapping = mapping;
+	set_dma_ops(dev, &iommu_ops);
+	printk(KERN_INFO "Assigned IOMMU device to %s\n", dev_name(dev));
+
+	return 0;
+}
-- 
1.7.1.569.g6f426

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[KyongHo Cho]

I don't think dma_alloc_writecombine() is useful
because it is actually not different from dma_alloc_coherent().
Moreover, no architecture implements it except ARM and AVR32
and 'struct dma_map_ops' in <linux/dma-mapping.h> does not cover it.

The only difference of dma_alloc_writecombine() from dma_alloc_coherent() is
whether a caller needs to decide to use memory barrier after call
dma_alloc_writecombine().

Of course, the mapping created by by dma_alloc_writecombine()
may be more efficient for CPU to update the DMA buffer.
But I think mapping with dma_alloc_coherent() is not such a
performance bottleneck.

I think it is better to remove dma_alloc_writecombine() and replace
all of it with dma_alloc_coherent().

In addition, IMHO, mapping to user's address is not a duty of dma_map_ops.
dma_mmap_*() is not suitable for a system that has IOMMU
because a DMA address does not equal to its correspondent physical
address semantically.

I think DMA APIs of ARM must be changed drastically to support IOMMU
because IOMMU API does not manage virtual address space.

I've also concerned about IOMMU implementation in ARM architecture for
several months.
But i found that there are some obstacles to overcome.

Best regards.

On Wed, May 25, 2011 at 4:35 PM, Marek Szyprowski
<m.szyprowski@samsung.com> wrote:
> Hello,
>
> Folloing the discussion about the driver for IOMMU controller for
> Samsung Exynos4 platform and Arnd's suggestions I've decided to start
> working on redesign of dma-mapping implementation for ARM architecture.
> The goal is to add support for IOMMU in the way preffered by the
> community :)
>
> Some of the ideas about merging dma-mapping api and iommu api comes from
> the following threads:
> http://www.spinics.net/lists/linux-media/msg31453.html
> http://www.spinics.net/lists/arm-kernel/msg122552.html
> http://www.spinics.net/lists/arm-kernel/msg124416.html
>
> They were also discussed on Linaro memory management meeting at UDS
> (Budapest 9-12 May).
>
> I've finaly managed to clean up a bit my works and present the initial,
> very proof-of-concept version of patches that were ready just before
> Linaro meeting.
>
> What have been implemented:
>
> 1. Introduced arm_dma_ops
>
> dma_map_ops from include/linux/dma-mapping.h suffers from the following
> limitations:
> - lack of start address for sync operations
> - lack of write-combine methods
> - lack of mmap to user-space methods
> - lack of map_single method
>
> For the initial version I've decided to use custom arm_dma_ops.
> Extending common interface will take time, until that I wanted to have
> something already working.
>
> dma_{alloc,free,mmap}_{coherent,writecombine} have been consolidated
> into dma_{alloc,free,mmap}_attrib what have been suggested on Linaro
> meeting. New attribute for WRITE_COMBINE memory have been introduced.
>
> 2. moved all inline ARM dma-mapping related operations to
> arch/arm/mm/dma-mapping.c and put them as methods in generic arm_dma_ops
> structure. The dma-mapping.c code deinitely needs cleanup, but this is
> just a first step.
>
> 3. Added very initial IOMMU support. Right now it is limited only to
> dma_alloc_attrib, dma_free_attrib and dma_mmap_attrib. It have been
> tested with s5p-fimc driver on Samsung Exynos4 platform.
>
> 4. Adapted Samsung Exynos4 IOMUU driver to make use of the introduced
> iommu_dma proposal.
>
> This patch series contains only patches for common dma-mapping part.
> There is also a patch that adds driver for Samsung IOMMU controller on
> Exynos4 platform. All required patches are available on:
>
> git://git.infradead.org/users/kmpark/linux-2.6-samsung dma-mapping branch
>
> Git web interface:
> http://git.infradead.org/users/kmpark/linux-2.6-samsung/shortlog/refs/heads/dma-mapping
>
>
> Future:
>
> 1. Add all missing operations for IOMMU mappings (map_single/page/sg,
> sync_*)
>
> 2. Move sync_* operations into separate function for better code sharing
> between iommu and non-iommu dma-mapping code
>
> 3. Splitting out dma bounce code from non-bounce into separate set of
> dma methods. Right now dma-bounce code is compiled conditionally and
> spread over arch/arm/mm/dma-mapping.c and arch/arm/common/dmabounce.c.
>
> 4. Merging dma_map_single with dma_map_page. I haven't investigated
> deeply why they have separate implementation on ARM. If this is a
> requirement then dma_map_ops need to be extended with another method.
>
> 5. Fix dma_alloc to unmap from linear mapping.
>
> 6. Convert IO address space management code from gen-alloc to some
> simpler bitmap based solution.
>
> 7. resolve issues that might araise during discussion & comments
>
> Please note that this is very early version of patches, definitely NOT
> intended for merging. I just wanted to make sure that the direction is
> right and share the code with others that might want to cooperate on
> dma-mapping improvements.
>
> Best regards
> --
> Marek Szyprowski
> Samsung Poland R&D Center
>
>
>
> Patch summary:
>
> Marek Szyprowski (2):
>  ARM: Move dma related inlines into arm_dma_ops methods
>  ARM: initial proof-of-concept IOMMU mapper for DMA-mapping
>
>  arch/arm/Kconfig                   |    1 +
>  arch/arm/include/asm/device.h      |    3 +
>  arch/arm/include/asm/dma-iommu.h   |   30 ++
>  arch/arm/include/asm/dma-mapping.h |  653 +++++++++++------------------
>  arch/arm/mm/dma-mapping.c          |  817 +++++++++++++++++++++++++++++++++---
>  arch/arm/mm/vmregion.h             |    2 +-
>  include/linux/dma-attrs.h          |    1 +
>  7 files changed, 1033 insertions(+), 474 deletions(-)
>  create mode 100644 arch/arm/include/asm/dma-iommu.h
>
> --
> 1.7.1.569.g6f426
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Arnd Bergmann]

On Monday 13 June 2011 16:12:05 KyongHo Cho wrote:
> Of course, the mapping created by by dma_alloc_writecombine()
> may be more efficient for CPU to update the DMA buffer.
> But I think mapping with dma_alloc_coherent() is not such a
> performance bottleneck.
> 
> I think it is better to remove dma_alloc_writecombine() and replace
> all of it with dma_alloc_coherent().

I'm sure that the graphics people will disagree with you on that.
Having the frame buffer mapped in write-combine mode is rather
important when you want to efficiently output videos from your
CPU.

> In addition, IMHO, mapping to user's address is not a duty of dma_map_ops.
> dma_mmap_*() is not suitable for a system that has IOMMU
> because a DMA address does not equal to its correspondent physical
> address semantically.
> 
> I think DMA APIs of ARM must be changed drastically to support IOMMU
> because IOMMU API does not manage virtual address space.

I can understand that there are arguments why mapping a DMA buffer into
user space doesn't belong into dma_map_ops, but I don't see how the
presence of an IOMMU is one of them.

The entire purpose of dma_map_ops is to hide from the user whether
you have an IOMMU or not, so that would be the main argument for
putting it in there, not against doing so.

	Arnd

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[KyongHo Cho]

Hi.

On Tue, Jun 14, 2011 at 12:07 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> I'm sure that the graphics people will disagree with you on that.
> Having the frame buffer mapped in write-combine mode is rather
> important when you want to efficiently output videos from your
> CPU.
>
I agree with you.
But I am discussing about dma_alloc_writecombine() in ARM.
You can see that only ARM and AVR32 implement it and there are few
drivers which use it.
No function in dma_map_ops corresponds to dma_alloc_writecombine().
That's why Marek tried to add 'alloc_writecombine' to dma_map_ops.

> I can understand that there are arguments why mapping a DMA buffer into
> user space doesn't belong into dma_map_ops, but I don't see how the
> presence of an IOMMU is one of them.
>
> The entire purpose of dma_map_ops is to hide from the user whether
> you have an IOMMU or not, so that would be the main argument for
> putting it in there, not against doing so.
>
I also understand the reasons why dma_map_ops maps a buffer into user space.
Mapping in device and user space at the same time or in a simple
approach may look good.
But I think mapping to user must be and driver-specific.
Moreover, kernel already provides various ways to map physical memory
to user space.
And I think that remapping DMA address that is in device address space
to user space is not a good idea
because DMA address is not same to physical address semantically if
features of IOMMU are implemented.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Catalin Marinas]

On Tue, Jun 14, 2011 at 12:30:44AM +0900, KyongHo Cho wrote:
> On Tue, Jun 14, 2011 at 12:07 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> > I'm sure that the graphics people will disagree with you on that.
> > Having the frame buffer mapped in write-combine mode is rather
> > important when you want to efficiently output videos from your
> > CPU.
>
> I agree with you.
> But I am discussing about dma_alloc_writecombine() in ARM.
> You can see that only ARM and AVR32 implement it and there are few
> drivers which use it.
> No function in dma_map_ops corresponds to dma_alloc_writecombine().
> That's why Marek tried to add 'alloc_writecombine' to dma_map_ops.

FWIW, on ARMv6 and later hardware, the dma_alloc_coherent() provides
writecombine memory (i.e. Normal Noncacheable), so no need for
dma_alloc_writecombine(). On earlier architectures it is creating
Strongly Ordered mappings (no writecombine).

-- 
Catalin

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Arnd Bergmann]

On Monday 13 June 2011 17:30:44 KyongHo Cho wrote:
> On Tue, Jun 14, 2011 at 12:07 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> > I'm sure that the graphics people will disagree with you on that.
> > Having the frame buffer mapped in write-combine mode is rather
> > important when you want to efficiently output videos from your
> > CPU.
> >
> I agree with you.
> But I am discussing about dma_alloc_writecombine() in ARM.
> You can see that only ARM and AVR32 implement it and there are few
> drivers which use it.
> No function in dma_map_ops corresponds to dma_alloc_writecombine().
> That's why Marek tried to add 'alloc_writecombine' to dma_map_ops.

Yes, and I think Marek's patch is really necessary. The reason we
need dma_alloc_writecombine on ARM is because the page attributes in
the kernel need to match the ones in user space, while other
architectures either handle the writecombine flag outside of the
page table or can have multiple conflicting mappings.

The reason that I suspect AVR32 needs it is to share device drivers
with ARM.

> > I can understand that there are arguments why mapping a DMA buffer into
> > user space doesn't belong into dma_map_ops, but I don't see how the
> > presence of an IOMMU is one of them.
> >
> > The entire purpose of dma_map_ops is to hide from the user whether
> > you have an IOMMU or not, so that would be the main argument for
> > putting it in there, not against doing so.
> >
> I also understand the reasons why dma_map_ops maps a buffer into user space.
> Mapping in device and user space at the same time or in a simple
> approach may look good.
> But I think mapping to user must be and driver-specific.
> Moreover, kernel already provides various ways to map physical memory
> to user space.

I believe the idea of providing dma_mmap_... is to ensure that the
page attributes are not conflicting and the DMA code is the place
that decides on the page attributes for the kernel mapping, so no
other place in the kernel can really know what it should be in user
space.

> And I think that remapping DMA address that is in device address space
> to user space is not a good idea
> because DMA address is not same to physical address semantically if
> features of IOMMU are implemented.

I'm totally not following this argument. This has nothing to do with IOMMU
or not. If you have an IOMMU, the dma code will know where the pages are
anyway, so it can always map them into user space. The dma code might
have an easier way to do it other than follwoing the page tables.

	Arnd

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[KyongHo Cho]

> I'm totally not following this argument. This has nothing to do with IOMMU
> or not. If you have an IOMMU, the dma code will know where the pages are
> anyway, so it can always map them into user space. The dma code might
> have an easier way to do it other than follwoing the page tables.
>
Ah. Sorry for that. I mixed dma_alloc_* up with dma_map_*.
I identified the reason why mmap_* in dma_map_ops is required.
You mean that nothing but DMA API knows what pages will be mapped to user space.
Thanks anyway.

KyongHo.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[KyongHo Cho]

> FWIW, on ARMv6 and later hardware, the dma_alloc_coherent() provides
> writecombine memory (i.e. Normal Noncacheable), so no need for
> dma_alloc_writecombine(). On earlier architectures it is creating
> Strongly Ordered mappings (no writecombine).
>
Thanks.

Do you mean that dma_alloc_coherent() and dma_alloc_writecombine() are
not different
except some additional features of dma_alloc_coherent() in ARM?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Michael K. Edwards]

[-- Attachment #1: Type: text/plain, Size: 3138 bytes --]

The need to allocate pages for "write combining" access goes deeper
than anything to do with DMA or IOMMUs.  Please keep "write combine"
distinct from "coherent" in the allocation/mapping APIs.

Write-combining is a special case because it's an end-to-end
requirement, usually architecturally invisible, and getting it to
happen requires a very specific combination of mappings and code.
There's a good explanation here of the requirements on some Intel
implementations of the x86 architecture:
http://software.intel.com/en-us/articles/copying-accelerated-video-decode-frame-buffers/
.  As I understand it, similar considerations apply on at least some
ARMv7 implementations, with NEON multi-register load/store operations
taking the place of MOVNTDQ.  (See
http://www.arm.com/files/pdf/A8_Paper.pdf for instance; although I
don't think there's enough detail about the conditions under which "if
the full cache line is written, the Level-2 line is simply marked
dirty and no external memory requests are required.")

As far as I can tell, there is not yet any way to get real
cache-bypassing write-combining from userland in a mainline kernel,
for x86/x86_64 or ARM.  I have been able to do it from inside a driver
on x86, including in an ISR with some fixes to the kernel's FPU
context save/restore code (patch attached, if you're curious);
otherwise I haven't yet seen write-combining in operation on Linux.
The code that needs to bypass the cache is part of a SoC silicon
erratum workaround supplied by Intel.  It didn't work as delivered --
it oopsed the kernel -- but is now shipping inside our product, and no
problems have been reported from QA or the field.  So I'm fairly sure
that the changes I made are effective.

I am not expert in this area; I was just forced to learn something
about it in order to make a product work.  My assertion that "there's
no way to do it yet" is almost certainly wrong.  I am hoping and
expecting to be immediately contradicted, with a working code example
and benchmarks that show that cache lines are not being fetched,
clobbered, and stored again, with the latencies hidden inside the
cache architecture.  :-)  (Seriously: there are four bits in the
Cortex-A8's "L2 Cache Auxiliary Control Register" that control various
aspects of this mechanism, and if you don't have a fairly good
explanation of which bits do and don't affect your benchmark, then I
contend that the job isn't done.  I don't begin to understand the
equivalent for the multi-core A9 I'm targeting next.)

If some kind person doesn't help me see the error of my ways, I'm
going to have to figure it out for myself on ARM in the next couple of
months, this time for performance reasons rather than to work around
silicon errata.  Unfortunately, I do not expect it to be particularly
low-hanging fruit.  I expect to switch to the hard-float ABI first
(the only remaining obstacle being a couple of TI-supplied binary-only
libraries).  That might provide enough of a system-level performance
win (by allowing the compiler to reorder fetches to NEON registers
across function/method calls) to obviate the need.

Cheers,
- Michael

[-- Attachment #2: 0011-Clean-up-task-FPU-state-thoroughly-during-exec-and-p.patch --]
[-- Type: application/octet-stream, Size: 2040 bytes --]

From ffb3feb73f89a34459b84f229a9ac699a589ed8f Mon Sep 17 00:00:00 2001
From: Michael Edwards <michaedw@cisco.com>
Date: Sat, 24 Apr 2010 20:53:38 -0700
Subject: [PATCH] x86,fpu: Protect FPU context cleanup against SSE2 in ISRs

 Clean up task FPU state thoroughly during exec() and process tear-down,
 and lock out local IRQs while doing it, so that SSE2 instructions in
 ISRs don't cause fxsave/fxrstor to/from a null pointer.
 (They still need to be guarded with kernel_fpu_(begin|end), of course.)

Signed-off-by: Michael Edwards <michaedw@cisco.com>
---
 arch/x86/kernel/process.c    |   10 ++++++++++
 arch/x86/kernel/process_32.c |    4 +---
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 876e918..bde7d09 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -8,6 +8,7 @@
 #include <linux/pm.h>
 #include <linux/clockchips.h>
 #include <asm/system.h>
+#include <asm/i387.h>
 
 unsigned long idle_halt;
 EXPORT_SYMBOL(idle_halt);
@@ -31,12 +32,21 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
 	return 0;
 }
 
+/*
+ * Locks out local IRQs while clearing FPU state and
+ * related task properties, so that ISRs can use SSE2.
+ */
 void free_thread_xstate(struct task_struct *tsk)
 {
+	local_irq_disable();
 	if (tsk->thread.xstate) {
+		tsk->fpu_counter = 0;
+		clear_stopped_child_used_math(tsk);
+		__clear_fpu(tsk);
 		kmem_cache_free(task_xstate_cachep, tsk->thread.xstate);
 		tsk->thread.xstate = NULL;
 	}
+	local_irq_enable();
 }
 
 void free_thread_info(struct thread_info *ti)
diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c
index 31f40b2..7122160 100644
--- a/arch/x86/kernel/process_32.c
+++ b/arch/x86/kernel/process_32.c
@@ -294,9 +294,7 @@ void flush_thread(void)
 	/*
 	 * Forget coprocessor state..
 	 */
-	tsk->fpu_counter = 0;
-	clear_fpu(tsk);
-	clear_used_math();
+	free_thread_xstate(tsk);
 }
 
 void release_thread(struct task_struct *dead_task)
-- 
1.7.0

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Catalin Marinas]

On Mon, Jun 13, 2011 at 05:00:16PM +0100, KyongHo Cho wrote:
> > FWIW, on ARMv6 and later hardware, the dma_alloc_coherent() provides
> > writecombine memory (i.e. Normal Noncacheable), so no need for
> > dma_alloc_writecombine(). On earlier architectures it is creating
> > Strongly Ordered mappings (no writecombine).
>
> Do you mean that dma_alloc_coherent() and dma_alloc_writecombine() are
> not different
> except some additional features of dma_alloc_coherent() in ARM?
 
When CONFIG_DMA_MEM_BUFFERABLE is enabled (by default on ARMv7 and ARMv6
with some exceptions because of hardware issues), the resulting mapping
for both coherent and writecombine is the same. In both cases the
mapping is done as L_PTE_MT_BUFFERABLE which is what you want with
writecombine. You can check the pgprot_writecombine() and
pgprot_dmacoherent() macros in asm/pgtable.h

-- 
Catalin

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Jesse Barnes]

On Mon, 13 Jun 2011 10:55:59 -0700
"Michael K. Edwards" <m.k.edwards@gmail.com> wrote:

> As far as I can tell, there is not yet any way to get real
> cache-bypassing write-combining from userland in a mainline kernel,
> for x86/x86_64 or ARM. 

Well only if things are really broken.  sysfs exposes _wc resource
files to allow userland drivers to map a given PCI BAR using write
combining, if the underlying platform supports it.

Similarly, userland mapping of GEM objects through the GTT are supposed
to be write combined, though I need to verify this (we've had trouble
with it in the past).

-- 
Jesse Barnes, Intel Open Source Technology Center

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Marek Szyprowski]

Hello,

On Monday, June 13, 2011 5:31 PM KyongHo Cho wrote:

> On Tue, Jun 14, 2011 at 12:07 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> > I'm sure that the graphics people will disagree with you on that.
> > Having the frame buffer mapped in write-combine mode is rather
> > important when you want to efficiently output videos from your
> > CPU.
> >
> I agree with you.
> But I am discussing about dma_alloc_writecombine() in ARM.
> You can see that only ARM and AVR32 implement it and there are few
> drivers which use it.
> No function in dma_map_ops corresponds to dma_alloc_writecombine().
> That's why Marek tried to add 'alloc_writecombine' to dma_map_ops.

I also introduced dma_alloc_attrs() to allow other types of 
memory&mappings combinations in the future. For example in case of
IOMMU the driver might like to call a function that will allocate 
a buffer that will 'work best with hardware'. This means that the
buffer might be build from pages larger than 4KiB, aligned to 
particular IOMMU requirements. Handling such requirements are
definitely not a part of the driver, only particular implementation
of dma-mapping will know them. The driver may just provide a some
hints how the memory will be used. The one that I'm particularly
thinking of are different types of caching.

> > I can understand that there are arguments why mapping a DMA buffer into
> > user space doesn't belong into dma_map_ops, but I don't see how the
> > presence of an IOMMU is one of them.
> >
> > The entire purpose of dma_map_ops is to hide from the user whether
> > you have an IOMMU or not, so that would be the main argument for
> > putting it in there, not against doing so.
>
> I also understand the reasons why dma_map_ops maps a buffer into user space.
> Mapping in device and user space at the same time or in a simple
> approach may look good.
> But I think mapping to user must be and driver-specific.
> Moreover, kernel already provides various ways to map physical memory
> to user space.
> And I think that remapping DMA address that is in device address space
> to user space is not a good idea
> because DMA address is not same to physical address semantically if
> features of IOMMU are implemented.

Mapping DMA address to user-space is one of the common feature of various
APIs (framebuffer, v4l2, alsa). In most cases the kernel virtual address 
in not even required for such drivers, because they just want to expose 
the buffer content to userspace. It would be great if dma-mapping will allow
allocating a coherent buffer without the need of mapping it to kernel space
at all. Kernel virtual space is really limited. For some multimedia
processing (like capturing & encoding HD movie from camera sensor) we
might need buffers of total size over 128MB or even more).

Best regards
-- 
Marek Szyprowski
Samsung Poland R&D Center



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Michael K. Edwards]

On Mon, Jun 13, 2011 at 11:54 AM, Jesse Barnes <jbarnes@virtuousgeek.org> wrote:
> Well only if things are really broken.  sysfs exposes _wc resource
> files to allow userland drivers to map a given PCI BAR using write
> combining, if the underlying platform supports it.

Mmm, I hadn't spotted that; that is useful, at least as sample code.
Doesn't do me any good directly, though; I'm not on a PCI device, I'm
on a SoC.  And what I need to do is to allocate normal memory through
an uncacheable write-combining page table entry (with certainty that
it is not aliased by a cacheable entry for the same physical memory),
and use it for interchange of data (GPU assets, compressed video) with
other on-chip cores.  (Or with off-chip PCI devices which use DMA to
transfer data to/from these buffers and then interrupt the CPU to
notify it to rotate them.)

What doesn't seem to be straightforward to do from userland is to
allocate pages that are locked to physical memory and mapped for
write-combining.  The device driver shouldn't have to mediate their
allocation, just map to a physical address (or set up an IOMMU entry,
I suppose) and pass that to the hardware that needs it.  Typical
userland code that could use such a mechanism would be the Qt/OpenGL
back end (which needs to store decompressed images and other
pre-rendered assets in GPU-ready buffers) and media pipelines.

> Similarly, userland mapping of GEM objects through the GTT are supposed
> to be write combined, though I need to verify this (we've had trouble
> with it in the past).

Also a nice source of sample code; though, again, I don't want this to
be driver-specific.  I might want a stage in my media pipeline that
uses the GPU to perform, say, lens distortion correction.  I shouldn't
have to go through contortions to use the same buffers from the GPU
and the video capture device.  The two devices are likely to have
their own variants on scatter-gather DMA, with a circularly linked
list of block descriptors with ownership bits and all that jazz; but
the actual data buffers should be generic, and the userland pipeline
setup code should just allocate them (presumably as contiguous regions
in a write-combining hugepage) and feed them to the plumbing.

Cheers,
- Michael

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Jesse Barnes]

On Tue, 14 Jun 2011 11:15:38 -0700
"Michael K. Edwards" <m.k.edwards@gmail.com> wrote:
> What doesn't seem to be straightforward to do from userland is to
> allocate pages that are locked to physical memory and mapped for
> write-combining.  The device driver shouldn't have to mediate their
> allocation, just map to a physical address (or set up an IOMMU entry,
> I suppose) and pass that to the hardware that needs it.  Typical
> userland code that could use such a mechanism would be the Qt/OpenGL
> back end (which needs to store decompressed images and other
> pre-rendered assets in GPU-ready buffers) and media pipelines.

We try to avoid allowing userspace to pin arbitrary buffers though.  So
on the gfx side, userspace can allocate buffers, but they're only
actually pinned when some operation is performed on them (e.g. they're
referenced in a command buffer or used for a mode set operation).

Something like ION or GEM can provide the basic alloc & map API, but
the platform code still has to deal with grabbing hunks of memory,
making them uncached or write combine, and mapping them to app space
without conflicts.

> Also a nice source of sample code; though, again, I don't want this to
> be driver-specific.  I might want a stage in my media pipeline that
> uses the GPU to perform, say, lens distortion correction.  I shouldn't
> have to go through contortions to use the same buffers from the GPU
> and the video capture device.  The two devices are likely to have
> their own variants on scatter-gather DMA, with a circularly linked
> list of block descriptors with ownership bits and all that jazz; but
> the actual data buffers should be generic, and the userland pipeline
> setup code should just allocate them (presumably as contiguous regions
> in a write-combining hugepage) and feed them to the plumbing.

Totally agree.  That's one reason I don't think enhancing the DMA
mapping API in the kernel is a complete solution.  Sure, the platform
code needs to be able to map buffers to devices and use any available
IOMMUs, but we still need a userspace API for all of that, with its
associated changes to the CPU MMU handling.

-- 
Jesse Barnes, Intel Open Source Technology Center

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Zach Pfeffer]

On 14 June 2011 13:21, Jesse Barnes <jbarnes@virtuousgeek.org> wrote:
> On Tue, 14 Jun 2011 11:15:38 -0700
> "Michael K. Edwards" <m.k.edwards@gmail.com> wrote:
>> What doesn't seem to be straightforward to do from userland is to
>> allocate pages that are locked to physical memory and mapped for
>> write-combining.  The device driver shouldn't have to mediate their
>> allocation, just map to a physical address (or set up an IOMMU entry,
>> I suppose) and pass that to the hardware that needs it.  Typical
>> userland code that could use such a mechanism would be the Qt/OpenGL
>> back end (which needs to store decompressed images and other
>> pre-rendered assets in GPU-ready buffers) and media pipelines.
>
> We try to avoid allowing userspace to pin arbitrary buffers though.  So
> on the gfx side, userspace can allocate buffers, but they're only
> actually pinned when some operation is performed on them (e.g. they're
> referenced in a command buffer or used for a mode set operation).
>
> Something like ION or GEM can provide the basic alloc & map API, but
> the platform code still has to deal with grabbing hunks of memory,
> making them uncached or write combine, and mapping them to app space
> without conflicts.
>
>> Also a nice source of sample code; though, again, I don't want this to
>> be driver-specific.  I might want a stage in my media pipeline that
>> uses the GPU to perform, say, lens distortion correction.  I shouldn't
>> have to go through contortions to use the same buffers from the GPU
>> and the video capture device.  The two devices are likely to have
>> their own variants on scatter-gather DMA, with a circularly linked
>> list of block descriptors with ownership bits and all that jazz; but
>> the actual data buffers should be generic, and the userland pipeline
>> setup code should just allocate them (presumably as contiguous regions
>> in a write-combining hugepage) and feed them to the plumbing.
>
> Totally agree.  That's one reason I don't think enhancing the DMA
> mapping API in the kernel is a complete solution.  Sure, the platform
> code needs to be able to map buffers to devices and use any available
> IOMMUs, but we still need a userspace API for all of that, with its
> associated changes to the CPU MMU handling.

I haven't seen all the discussions but it sounds like creating the
correct userspace abstraction and then looking at how the kernel needs
to change (instead of the other way around) may add some clarity to
things.

> --
> Jesse Barnes, Intel Open Source Technology Center
>
> _______________________________________________
> Linaro-mm-sig mailing list
> Linaro-mm-sig@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/linaro-mm-sig
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [Linaro-mm-sig] [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Michael K. Edwards]

On Tue, Jun 14, 2011 at 11:21 AM, Jesse Barnes <jbarnes@virtuousgeek.org> wrote:
> We try to avoid allowing userspace to pin arbitrary buffers though.  So
> on the gfx side, userspace can allocate buffers, but they're only
> actually pinned when some operation is performed on them (e.g. they're
> referenced in a command buffer or used for a mode set operation).

The issue isn't so much pinning; I don't really care if the physical
memory moves out from under me as long as the mappings are properly
updated in all the process page tables that share it and all the
hardware units that care.  But the mapping has to have the right cache
policy from the beginning, so that I get the important part of write
combining (the fill buffer allocation -- without bothering to load
contents from DRAM that are likely to be completely clobbered -- and
the cache-line-sized flush once it's filled).  In any case, supposedly
there are weird aliasing issues if you try to take a page that is
already mapped cacheable and remap it write-combine; and in the case
of shared pages, you'd need to look up all processes that have the
page mapped and alter their page tables, even if they're currently
running on other SMP cores.  Nasty.

Besides, I don't want little 4K pages; I want a hugepage with the
right cache policy, in which I can build a malloc pool (tcmalloc,
jemalloc, something like that) and allocate buffers for a variety of
purposes.  (I also want to use this to pass whole data structures,
like priority search trees built using offset pointers, among cores
that don't share a cache hierarchy or a cache coherency protocol.)

Presumably the privilege of write-combine buffer allocation would be
limited to processes that have been granted the appropriate
capability; but then that process should be able to share it with
others.  I would think the natural thing would be for the special-page
allocation API to return a file descriptor, which can then be passed
over local domain sockets and mmap()ed by as many processes as
necessary.  For many usage patterns, there will be no need for a
kernel virtual mapping; hardware wants physical addresses (or IOMMU
mappings) anyway.

> Something like ION or GEM can provide the basic alloc & map API, but
> the platform code still has to deal with grabbing hunks of memory,
> making them uncached or write combine, and mapping them to app space
> without conflicts.

Absolutely.  Much like any other hugepage allocation, right?  Not
really something ION or GEM or any other device driver needs to be
involved in.  Except for alignment issues, I suppose; I haven't given
that much thought.

The part about setting up corresponding mappings to the same physical
addresses in the device's DMA mechanics is not buffer *allocation*,
it's buffer *registration*.  That's sort of like V4L2's "user pointer
I/O" mode, in which the userspace app allocates the buffers and uses
the QBUF ioctl to register them.  I see no reason why the enforcement
of minimum alignment and cache policy couldn't be done at buffer
registration time rather than region allocation time.

Cheers,
- Michael

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Subash Patel]

Hi Marek,

In function: 
dma_alloc_coherent()->arm_iommu_alloc_attrs()->__iommu_alloc_buffer()

I have following questions:

a) Before we come to this point, we would have enabled SYSMMU in a call 
to arm_iommu_init(). Shouldnt the SYSMMU be enabled after call to 
__iommu_alloc_buffer(), but before __iommu_create_mapping()? If in case 
the __iommu_alloc_buffer() fails, we dont disable the SYSMMU.

b) For huge buffer sizes, the pressure on SYSMMU would be very high. 
Cant we have option to dictate the page size for the IOMMU from driver 
in such cases? Should it always be the size of system pages?

Regards,
Subash
SISO-SLG

On 05/25/2011 01:05 PM, Marek Szyprowski wrote:
> Hello,
>
> Folloing the discussion about the driver for IOMMU controller for
> Samsung Exynos4 platform and Arnd's suggestions I've decided to start
> working on redesign of dma-mapping implementation for ARM architecture.
> The goal is to add support for IOMMU in the way preffered by the
> community :)
>
> Some of the ideas about merging dma-mapping api and iommu api comes from
> the following threads:
> http://www.spinics.net/lists/linux-media/msg31453.html
> http://www.spinics.net/lists/arm-kernel/msg122552.html
> http://www.spinics.net/lists/arm-kernel/msg124416.html
>
> They were also discussed on Linaro memory management meeting at UDS
> (Budapest 9-12 May).
>
> I've finaly managed to clean up a bit my works and present the initial,
> very proof-of-concept version of patches that were ready just before
> Linaro meeting.
>
> What have been implemented:
>
> 1. Introduced arm_dma_ops
>
> dma_map_ops from include/linux/dma-mapping.h suffers from the following
> limitations:
> - lack of start address for sync operations
> - lack of write-combine methods
> - lack of mmap to user-space methods
> - lack of map_single method
>
> For the initial version I've decided to use custom arm_dma_ops.
> Extending common interface will take time, until that I wanted to have
> something already working.
>
> dma_{alloc,free,mmap}_{coherent,writecombine} have been consolidated
> into dma_{alloc,free,mmap}_attrib what have been suggested on Linaro
> meeting. New attribute for WRITE_COMBINE memory have been introduced.
>
> 2. moved all inline ARM dma-mapping related operations to
> arch/arm/mm/dma-mapping.c and put them as methods in generic arm_dma_ops
> structure. The dma-mapping.c code deinitely needs cleanup, but this is
> just a first step.
>
> 3. Added very initial IOMMU support. Right now it is limited only to
> dma_alloc_attrib, dma_free_attrib and dma_mmap_attrib. It have been
> tested with s5p-fimc driver on Samsung Exynos4 platform.
>
> 4. Adapted Samsung Exynos4 IOMUU driver to make use of the introduced
> iommu_dma proposal.
>
> This patch series contains only patches for common dma-mapping part.
> There is also a patch that adds driver for Samsung IOMMU controller on
> Exynos4 platform. All required patches are available on:
>
> git://git.infradead.org/users/kmpark/linux-2.6-samsung dma-mapping branch
>
> Git web interface:
> http://git.infradead.org/users/kmpark/linux-2.6-samsung/shortlog/refs/heads/dma-mapping
>
>
> Future:
>
> 1. Add all missing operations for IOMMU mappings (map_single/page/sg,
> sync_*)
>
> 2. Move sync_* operations into separate function for better code sharing
> between iommu and non-iommu dma-mapping code
>
> 3. Splitting out dma bounce code from non-bounce into separate set of
> dma methods. Right now dma-bounce code is compiled conditionally and
> spread over arch/arm/mm/dma-mapping.c and arch/arm/common/dmabounce.c.
>
> 4. Merging dma_map_single with dma_map_page. I haven't investigated
> deeply why they have separate implementation on ARM. If this is a
> requirement then dma_map_ops need to be extended with another method.
>
> 5. Fix dma_alloc to unmap from linear mapping.
>
> 6. Convert IO address space management code from gen-alloc to some
> simpler bitmap based solution.
>
> 7. resolve issues that might araise during discussion&  comments
>
> Please note that this is very early version of patches, definitely NOT
> intended for merging. I just wanted to make sure that the direction is
> right and share the code with others that might want to cooperate on
> dma-mapping improvements.
>
> Best regards

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [RFC 0/2] ARM: DMA-mapping & IOMMU integration

[Marek Szyprowski]

Hello,

On Monday, June 20, 2011 4:31 PM Subash Patel wrote:

> In function:
> dma_alloc_coherent()->arm_iommu_alloc_attrs()->__iommu_alloc_buffer()
> 
> I have following questions:
> 
> a) Before we come to this point, we would have enabled SYSMMU in a call
> to arm_iommu_init(). Shouldnt the SYSMMU be enabled after call to
> __iommu_alloc_buffer(), but before __iommu_create_mapping()? If in case
> the __iommu_alloc_buffer() fails, we dont disable the SYSMMU.

I want to move enabling and disabling SYSMMU completely to the runtime_pm
framework. As You can notice, the updated SYSMMU driver automatically
becomes a parent of respective multimedia device and a child of the power
domain to which both belongs. This means that sysmmu will operate only
when multimedia device is enabled, what really makes sense. The sysmmu
driver will need to be updated not to poke into the registers if it is
disabled, but this should be really trivial change.

> b) For huge buffer sizes, the pressure on SYSMMU would be very high.
> Cant we have option to dictate the page size for the IOMMU from driver
> in such cases? Should it always be the size of system pages?

This was just a first version of dma-mapping and IOMMU integration, just
to show the development road and start the discussion. Of course in the
final version support for pages larger than 4KiB is highly expected. We
can even reuse the recently posted CMA to allocate large pages for IOMMU
to improve the performance and make sure that the framework will be able
to allocate such pages even if the device is running for long time and 
memory got fragmented by typically movable pages.

Best regards
-- 
Marek Szyprowski
Samsung Poland R&D Center



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>