[PATCH 0 of 3] MMIO 32-bit copy routine, the final frontier

[PATCH 0 of 3] MMIO 32-bit copy routine, the final frontier

[Bryan O'Sullivan]

[-- Attachment #1: Type: text/plain, Size: 722 bytes --]

After yet more review comments from several people, here is a reworked
set of 32-bit MMIO copy patches.  This may even be the final set.

These define the generic __raw_memcpy_toio32 as a weak symbol, which
arches are free to override.  We provide a specialised implementation
for x86_64.

These patches should apply cleanly against current -git, and have been
tested on i386 and x86_64.

The patch series is as follows:

raw_memcpy_io.patch
  Introduce the generic MMIO 32-bit copy routine.

x86_64-memcpy32.patch
  Add memcpy32 routine to x86_64.

arch-specific-raw_memcpy_io.patch
  Get each arch to use generic memcpy_io code, except x86_64, which
  uses memcpy32.

Signed-off-by: Bryan O'Sullivan <bos@pathscale.com>

[PATCH 1 of 3] Introduce __raw_memcpy_toio32

[Bryan O'Sullivan]

This arch-independent routine copies data to a memory-mapped I/O region,
using 32-bit accesses.  It does not guarantee access ordering, nor does
it perform a memory barrier afterwards.  This style of access is required
by some devices.

Signed-off-by: Bryan O'Sullivan <bos@pathscale.com>

diff -r c90267e4a29b -r 05b3d1af27eb lib/Makefile
--- a/lib/Makefile	Wed Jan 11 13:31:24 2006 +0800
+++ b/lib/Makefile	Wed Jan 11 14:35:45 2006 -0800
@@ -5,7 +5,7 @@
 lib-y := errno.o ctype.o string.o vsprintf.o cmdline.o \
 	 bust_spinlocks.o rbtree.o radix-tree.o dump_stack.o \
 	 idr.o div64.o int_sqrt.o bitmap.o extable.o prio_tree.o \
-	 sha1.o
+	 sha1.o raw_memcpy_io.o
 
 lib-y	+= kobject.o kref.o kobject_uevent.o klist.o
 
diff -r c90267e4a29b -r 05b3d1af27eb lib/raw_memcpy_io.c
--- /dev/null	Thu Jan  1 00:00:00 1970 +0000
+++ b/lib/raw_memcpy_io.c	Wed Jan 11 14:35:45 2006 -0800
@@ -0,0 +1,41 @@
+/*
+ * Copyright 2006 PathScale, Inc.  All Rights Reserved.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+#include <linux/module.h>
+#include <asm/io.h>
+
+/**
+ * __raw_memcpy_toio32 - copy data to MMIO space, in 32-bit units
+ * @to: destination, in MMIO space (must be 32-bit aligned)
+ * @from: source (must be 32-bit aligned)
+ * @count: number of 32-bit quantities to copy
+ *
+ * Copy data from kernel space to MMIO space, in units of 32 bits at a
+ * time.  Order of access is not guaranteed, nor is a memory barrier
+ * performed afterwards.
+ */
+void __attribute__((weak)) __raw_memcpy_toio32(void __iomem *to,
+					       const void *from, size_t count)
+{
+	u32 __iomem *dst = to;
+	const u32 *src = from;
+	const u32 *end = src + count;
+
+	while (src < end)
+		__raw_writel(*src++, dst++);
+}
+EXPORT_SYMBOL_GPL(__raw_memcpy_toio32);

[PATCH 2 of 3] memcpy32 for x86_64

[Bryan O'Sullivan]

Introduce an x86_64-specific memcpy32 routine.  The routine is similar
to memcpy, but is guaranteed to work in units of 32 bits at a time.

Signed-off-by: Bryan O'Sullivan <bos@pathscale.com>

diff -r 05b3d1af27eb -r 1052904816d7 arch/x86_64/kernel/x8664_ksyms.c
--- a/arch/x86_64/kernel/x8664_ksyms.c	Wed Jan 11 14:35:45 2006 -0800
+++ b/arch/x86_64/kernel/x8664_ksyms.c	Wed Jan 11 14:35:45 2006 -0800
@@ -163,6 +163,8 @@
 EXPORT_SYMBOL(memcpy);
 EXPORT_SYMBOL(__memcpy);
 
+EXPORT_SYMBOL_GPL(memcpy32);
+
 #ifdef CONFIG_RWSEM_XCHGADD_ALGORITHM
 /* prototypes are wrong, these are assembly with custom calling functions */
 extern void rwsem_down_read_failed_thunk(void);
diff -r 05b3d1af27eb -r 1052904816d7 arch/x86_64/lib/Makefile
--- a/arch/x86_64/lib/Makefile	Wed Jan 11 14:35:45 2006 -0800
+++ b/arch/x86_64/lib/Makefile	Wed Jan 11 14:35:45 2006 -0800
@@ -9,4 +9,4 @@
 lib-y := csum-partial.o csum-copy.o csum-wrappers.o delay.o \
 	usercopy.o getuser.o putuser.o  \
 	thunk.o clear_page.o copy_page.o bitstr.o bitops.o
-lib-y += memcpy.o memmove.o memset.o copy_user.o
+lib-y += memcpy.o memcpy32.o memmove.o memset.o copy_user.o
diff -r 05b3d1af27eb -r 1052904816d7 include/asm-x86_64/string.h
--- a/include/asm-x86_64/string.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-x86_64/string.h	Wed Jan 11 14:35:45 2006 -0800
@@ -45,6 +45,9 @@
 #define __HAVE_ARCH_MEMMOVE
 void * memmove(void * dest,const void *src,size_t count);
 
+/* copy data, 32 bits at a time */
+void memcpy32(void *dst, const void *src, size_t count);
+
 /* Use C out of line version for memcmp */ 
 #define memcmp __builtin_memcmp
 int memcmp(const void * cs,const void * ct,size_t count);
diff -r 05b3d1af27eb -r 1052904816d7 arch/x86_64/lib/memcpy32.S
--- /dev/null	Thu Jan  1 00:00:00 1970 +0000
+++ b/arch/x86_64/lib/memcpy32.S	Wed Jan 11 14:35:45 2006 -0800
@@ -0,0 +1,32 @@
+/*
+ * Copyright 2006 PathScale, Inc.  All Rights Reserved.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+/**
+ * memcpy32 - copy data, in units of 32 bits at a time
+ * @dst: destination (must be 32-bit aligned)
+ * @src: source (must be 32-bit aligned)
+ * @count: number of 32-bit quantities to copy
+ */
+ 	.globl memcpy32
+memcpy32:
+	movl %edx,%ecx
+	shrl $1,%ecx
+	andl $1,%edx
+	rep movsq
+	movl %edx,%ecx
+	rep movsd
+	ret

[PATCH 3 of 3] Add __raw_memcpy_toio32 to each arch

[Bryan O'Sullivan]

Most arches use the generic routine.  x86_64 uses memcpy32 instead;
this is substantially faster, even over a bus that is much slower than
the CPU.

Signed-off-by: Bryan O'Sullivan <bos@pathscale.com>

diff -r 1052904816d7 -r ee6ce7e55dc7 arch/x86_64/lib/io.c
--- a/arch/x86_64/lib/io.c	Wed Jan 11 14:35:45 2006 -0800
+++ b/arch/x86_64/lib/io.c	Wed Jan 11 14:35:45 2006 -0800
@@ -21,3 +21,9 @@
 	memset((void *)a,b,c);
 }
 EXPORT_SYMBOL(memset_io);
+
+/* override generic definition in lib/raw_memcpy_io.c */
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count)
+{
+	memcpy32((void __force *) to, from, count);
+}
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-alpha/io.h
--- a/include/asm-alpha/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-alpha/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -504,6 +504,8 @@
 extern void memcpy_toio(volatile void __iomem *, const void *, long);
 extern void _memset_c_io(volatile void __iomem *, unsigned long, long);
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 static inline void memset_io(volatile void __iomem *addr, u8 c, long len)
 {
 	_memset_c_io(addr, 0x0101010101010101UL * c, len);
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-arm/io.h
--- a/include/asm-arm/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-arm/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -189,6 +189,8 @@
 #define memset_io(c,v,l)	_memset_io(__mem_pci(c),(v),(l))
 #define memcpy_fromio(a,c,l)	_memcpy_fromio((a),__mem_pci(c),(l))
 #define memcpy_toio(c,a,l)	_memcpy_toio(__mem_pci(c),(a),(l))
+
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
 
 #define eth_io_copy_and_sum(s,c,l,b) \
 				eth_copy_and_sum((s),__mem_pci(c),(l),(b))
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-cris/io.h
--- a/include/asm-cris/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-cris/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -121,6 +121,8 @@
 #define memcpy_fromio(a,b,c)	memcpy((a),(void *)(b),(c))
 #define memcpy_toio(a,b,c)	memcpy((void *)(a),(b),(c))
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 /*
  * Again, CRIS does not require mem IO specific function.
  */
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-frv/io.h
--- a/include/asm-frv/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-frv/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -127,6 +127,8 @@
 	memcpy((void __force *) dst, src, count);
 }
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 static inline uint8_t inb(unsigned long addr)
 {
 	return __builtin_read8((void *)addr);
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-h8300/io.h
--- a/include/asm-h8300/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-h8300/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -209,6 +209,8 @@
 #define memcpy_fromio(a,b,c)	memcpy((a),(void *)(b),(c))
 #define memcpy_toio(a,b,c)	memcpy((void *)(a),(b),(c))
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 #define mmiowb()
 
 #define inb(addr)    ((h8300_buswidth(addr))?readw((addr) & ~1) & 0xff:readb(addr))
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-i386/io.h
--- a/include/asm-i386/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-i386/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -203,6 +203,8 @@
 {
 	__memcpy((void __force *) dst, src, count);
 }
+
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
 
 /*
  * ISA space is 'always mapped' on a typical x86 system, no need to
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-ia64/io.h
--- a/include/asm-ia64/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-ia64/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -444,6 +444,8 @@
 extern void memcpy_toio(volatile void __iomem *dst, const void *src, long n);
 extern void memset_io(volatile void __iomem *s, int c, long n);
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 #define dma_cache_inv(_start,_size)             do { } while (0)
 #define dma_cache_wback(_start,_size)           do { } while (0)
 #define dma_cache_wback_inv(_start,_size)       do { } while (0)
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-m32r/io.h
--- a/include/asm-m32r/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-m32r/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -216,6 +216,8 @@
 	memcpy((void __force *) dst, src, count);
 }
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 /*
  * Convert a physical pointer to a virtual kernel pointer for /dev/mem
  * access
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-m68knommu/io.h
--- a/include/asm-m68knommu/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-m68knommu/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -113,6 +113,8 @@
 #define memcpy_fromio(a,b,c)	memcpy((a),(void *)(b),(c))
 #define memcpy_toio(a,b,c)	memcpy((void *)(a),(b),(c))
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 #define inb(addr)    readb(addr)
 #define inw(addr)    readw(addr)
 #define inl(addr)    readl(addr)
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-mips/io.h
--- a/include/asm-mips/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-mips/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -534,6 +534,8 @@
 	memcpy((void __force *) dst, src, count);
 }
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 /*
  * Memory Mapped I/O
  */
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-parisc/io.h
--- a/include/asm-parisc/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-parisc/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -294,6 +294,8 @@
 void memcpy_fromio(void *dst, const volatile void __iomem *src, int count);
 void memcpy_toio(volatile void __iomem *dst, const void *src, int count);
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 /* Support old drivers which don't ioremap.
  * NB this interface is scheduled to disappear in 2.5
  */
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-powerpc/io.h
--- a/include/asm-powerpc/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-powerpc/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -64,6 +64,8 @@
 #define memcpy_fromio(a,b,c)	iSeries_memcpy_fromio((a), (b), (c))
 #define memcpy_toio(a,b,c)	iSeries_memcpy_toio((a), (b), (c))
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 #define inb(addr)		readb(((void __iomem *)(long)(addr)))
 #define inw(addr)		readw(((void __iomem *)(long)(addr)))
 #define inl(addr)		readl(((void __iomem *)(long)(addr)))
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-ppc/io.h
--- a/include/asm-ppc/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-ppc/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -369,6 +369,8 @@
 }
 #endif
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 #define eth_io_copy_and_sum(a,b,c,d)		eth_copy_and_sum((a),(void __force *)(void __iomem *)(b),(c),(d))
 
 /*
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-s390/io.h
--- a/include/asm-s390/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-s390/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -99,6 +99,8 @@
 #define memcpy_fromio(a,b,c)    memcpy((a),__io_virt(b),(c))
 #define memcpy_toio(a,b,c)      memcpy(__io_virt(a),(b),(c))
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 #define inb_p(addr) readb(addr)
 #define inb(addr) readb(addr)
 
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-sh/io.h
--- a/include/asm-sh/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-sh/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -177,6 +177,8 @@
 extern void memcpy_toio(unsigned long, const void *, unsigned long);
 extern void memset_io(unsigned long, int, unsigned long);
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 /* SuperH on-chip I/O functions */
 static __inline__ unsigned char ctrl_inb(unsigned long addr)
 {
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-sh64/io.h
--- a/include/asm-sh64/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-sh64/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -125,6 +125,8 @@
 
 void memcpy_toio(void __iomem *to, const void *from, long count);
 void memcpy_fromio(void *to, void __iomem *from, long count);
+
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
 
 #define mmiowb()
 
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-sparc/io.h
--- a/include/asm-sparc/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-sparc/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -239,6 +239,8 @@
 
 #define memcpy_toio(d,s,sz)	_memcpy_toio(d,s,sz)
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 #ifdef __KERNEL__
 
 /*
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-sparc64/io.h
--- a/include/asm-sparc64/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-sparc64/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -440,6 +440,8 @@
 
 #define memcpy_toio(d,s,sz)	_memcpy_toio(d,s,sz)
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 static inline int check_signature(void __iomem *io_addr,
 				  const unsigned char *signature,
 				  int length)
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-v850/io.h
--- a/include/asm-v850/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-v850/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -130,6 +130,8 @@
 #define memcpy_fromio(dst, src, len) memcpy (dst, (void *)src, len)
 #define memcpy_toio(dst, src, len) memcpy ((void *)dst, src, len)
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 /*
  * Convert a physical pointer to a virtual kernel pointer for /dev/mem
  * access
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-x86_64/io.h
--- a/include/asm-x86_64/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-x86_64/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -252,6 +252,8 @@
 	__memcpy_toio((unsigned long)to,from,len);
 }
 
+void __raw_memcpy_toio32(void __iomem *dst, const void *src, size_t count);
+
 void memset_io(volatile void __iomem *a, int b, size_t c);
 
 /*
diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-xtensa/io.h
--- a/include/asm-xtensa/io.h	Wed Jan 11 14:35:45 2006 -0800
+++ b/include/asm-xtensa/io.h	Wed Jan 11 14:35:45 2006 -0800
@@ -159,6 +159,8 @@
 #define memcpy_fromio(a,b,c)   memcpy((a),(void *)(b),(c))
 #define memcpy_toio(a,b,c)      memcpy((void *)(a),(b),(c))
 
+void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
+
 /* At this point the Xtensa doesn't provide byte swap instructions */
 
 #ifdef __XTENSA_EB__

Re: [PATCH 1 of 3] Introduce __raw_memcpy_toio32

[Andrew Morton]

"Bryan O'Sullivan" <bos@pathscale.com> wrote:
>
>   lib-y := errno.o ctype.o string.o vsprintf.o cmdline.o \
>   	 bust_spinlocks.o rbtree.o radix-tree.o dump_stack.o \
>   	 idr.o div64.o int_sqrt.o bitmap.o extable.o prio_tree.o \
>  -	 sha1.o
>  +	 sha1.o raw_memcpy_io.o

You'll find that if nothing in vmlinux references __raw_memcpy_toio32 then
this file won't be included in vmlinux and __raw_memcpy_toio32 won't be
available to modules.

I'll move this to obj-y, which does the right thing.

Re: [PATCH 2 of 3] memcpy32 for x86_64

[Roland Dreier]

 > +/**
 > + * memcpy32 - copy data, in units of 32 bits at a time
 > + * @dst: destination (must be 32-bit aligned)
 > + * @src: source (must be 32-bit aligned)
 > + * @count: number of 32-bit quantities to copy
 > + */
 > + 	.globl memcpy32
 > +memcpy32:
 > +	movl %edx,%ecx
 > +	shrl $1,%ecx
 > +	andl $1,%edx
 > +	rep movsq
 > +	movl %edx,%ecx
 > +	rep movsd
 > +	ret

Sorry to keep this going still further, but I'm still confused.  Why
can't this assembly just define __raw_memcpy_toio32() directly?  In
other words, Why do we need to introduce the indirection of having a
stub in C that calls the memcpy32 assembly routine?  Is there some
reason having to do with linker magic and weak symbols?  Could it be
solved by using gcc inline assembly rather than putting the assembly
in a .S file?

Also why does memcpy32() need to be exported?  There are no users
other than the x86_64 version of __raw_memcpy_toio32(), and memcpy32()
doesn't seem like an API we want to add to every arch anyway.

 - R.

Re: [PATCH 3 of 3] Add __raw_memcpy_toio32 to each arch

[Andrew Morton]

"Bryan O'Sullivan" <bos@pathscale.com> wrote:
>
> Most arches use the generic routine.  x86_64 uses memcpy32 instead;
>  this is substantially faster, even over a bus that is much slower than
>  the CPU.
> 
>  Signed-off-by: Bryan O'Sullivan <bos@pathscale.com>
> 
>  diff -r 1052904816d7 -r ee6ce7e55dc7 arch/x86_64/lib/io.c
>  --- a/arch/x86_64/lib/io.c	Wed Jan 11 14:35:45 2006 -0800
>  +++ b/arch/x86_64/lib/io.c	Wed Jan 11 14:35:45 2006 -0800
>  @@ -21,3 +21,9 @@
>   	memset((void *)a,b,c);
>   }
>   EXPORT_SYMBOL(memset_io);
>  +
>  +/* override generic definition in lib/raw_memcpy_io.c */
>  +void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count)
>  +{
>  +	memcpy32((void __force *) to, from, count);
>  +}
>  diff -r 1052904816d7 -r ee6ce7e55dc7 include/asm-alpha/io.h
>  --- a/include/asm-alpha/io.h	Wed Jan 11 14:35:45 2006 -0800
>  +++ b/include/asm-alpha/io.h	Wed Jan 11 14:35:45 2006 -0800
>  @@ -504,6 +504,8 @@
>   extern void memcpy_toio(volatile void __iomem *, const void *, long);
>   extern void _memset_c_io(volatile void __iomem *, unsigned long, long);
>   
>  +void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
>  +

<etc>

How's about we add a new linux/io.h which does:

#include <asm/io.h>
void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);

?

Re: [PATCH 2 of 3] memcpy32 for x86_64

[Bryan O'Sullivan]

On Wed, 2006-01-11 at 15:45 -0800, Roland Dreier wrote:

> Sorry to keep this going still further, but I'm still confused.  Why
> can't this assembly just define __raw_memcpy_toio32() directly?

It certainly can.  I've just been buried in this bloody thing for a
little too long.

	<b

Re: [PATCH 3 of 3] Add __raw_memcpy_toio32 to each arch

[Bryan O'Sullivan]

On Wed, 2006-01-11 at 15:46 -0800, Andrew Morton wrote:

> How's about we add a new linux/io.h which does:
> 
> #include <asm/io.h>
> void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);

I thought about this, and about moving other duplicated definitions from
asm-*/io.h in here, but I couldn't find any other obvious candidates, so
I wasn't anxious to introduce a new file.

If you think that's OK, though, it obviously makes the patch a lot
smaller, and gives a common place to put future cross-arch definitions.

I'll run another spin of the patch with your and Roland's suggestions.

	<b

Re: [PATCH 3 of 3] Add __raw_memcpy_toio32 to each arch

[Andrew Morton]

"Bryan O'Sullivan" <bos@pathscale.com> wrote:
>
> On Wed, 2006-01-11 at 15:46 -0800, Andrew Morton wrote:
> 
> > How's about we add a new linux/io.h which does:
> > 
> > #include <asm/io.h>
> > void __raw_memcpy_toio32(void __iomem *to, const void *from, size_t count);
> 
> I thought about this, and about moving other duplicated definitions from
> asm-*/io.h in here, but I couldn't find any other obvious candidates, so
> I wasn't anxious to introduce a new file.
> 

Well it's obviously better than duplicating the thing.

There are other common things which can be hoisted to linux/io.h, but if we
do that then zillions of .c files need to be changed to include linux/io.h
rather than asm/io.h.  That's a good janitorial thing to do, but I doubt if
you want to do it ;)

Re: [PATCH 3 of 3] Add __raw_memcpy_toio32 to each arch

[Bryan O'Sullivan]

On Wed, 2006-01-11 at 16:13 -0800, Andrew Morton wrote:

> There are other common things which can be hoisted to linux/io.h, but if we
> do that then zillions of .c files need to be changed to include linux/io.h
> rather than asm/io.h.

Right.

> That's a good janitorial thing to do, but I doubt if
> you want to do it ;)

Not as part of these patches, anyway.  They've left me a dried-up husk.

	<b