[PATCH 0 of 2] MMIO copy patches, the next generation

[PATCH 0 of 2] MMIO copy patches, the next generation

[Bryan O'Sullivan]

[-- Attachment #1: Type: text/plain, Size: 1209 bytes --]

After another round of review, here is a revised set of MMIO copy patches.

These have switched the name of the copy routine to __iowrite32_copy,
to match the naming convention in include/asm-generic/iomap.h, and at
Andi's request.  The name is double-underscored to make it clear that
the routine does not guarantee the order of writes or perform a memory
barrier; the kernel doc also explicitly states this.

These define the generic __iowrite32_copy as a weak symbol, which
arches are free to override.  We provide a specialised implementation
for x86_64.  In a change from prior versions of these patches, the x86_64
version no longer makes any assumptions about the bus breaking up 64-bit
transactions, and uses 32-bit copies directly.

We also introduce include/linux/io.h, which is tiny now, but a candidate
for later cleanups of all the per-arch asm-*/{io,iomap}.h files.

These patches should apply cleanly against current -git, and have been
tested on i386 and x86_64.

The patch series is as follows:

iomap_copy.patch
  Introduce the generic MMIO 32-bit copy routine.

x86_64-iomap_copy.patch
  Add a faster __iowrite32_copy routine to x86_64.

Signed-off-by: Bryan O'Sullivan <bos@pathscale.com>

[PATCH 1 of 2] Introduce __iowrite32_copy

[Bryan O'Sullivan]

This arch-independent routine copies data to a memory-mapped I/O region,
using 32-bit accesses.  The naming is double-underscored to make it clear
that it does not guarantee write ordering, nor does it perform a
memory barrier afterwards; the kernel doc also explicitly states this.
This style of access is required by some devices.

This change also introduces include/linux/io.h, at Andrew's suggestion.
It only has one occupant at the moment, but is a logical destination
for oft-replicated contents of include/asm-*/{io,iomap}.h to migrate to.

Signed-off-by: Bryan O'Sullivan <bos@pathscale.com>

diff -r c90267e4a29b -r ec2b3675168a lib/Makefile
--- a/lib/Makefile	Wed Jan 11 13:31:24 2006 +0800
+++ b/lib/Makefile	Thu Jan 12 09:03:37 2006 -0800
@@ -9,7 +9,7 @@
 
 lib-y	+= kobject.o kref.o kobject_uevent.o klist.o
 
-obj-y += sort.o parser.o halfmd4.o
+obj-y += sort.o parser.o halfmd4.o iomap_copy.o
 
 ifeq ($(CONFIG_DEBUG_KOBJECT),y)
 CFLAGS_kobject.o += -DDEBUG
diff -r c90267e4a29b -r ec2b3675168a include/linux/io.h
--- /dev/null	Thu Jan  1 00:00:00 1970 +0000
+++ b/include/linux/io.h	Thu Jan 12 09:03:37 2006 -0800
@@ -0,0 +1,25 @@
+/*
+ * Copyright 2006 PathScale, Inc.  All Rights Reserved.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+#ifndef _LINUX_IO_H
+#define _LINUX_IO_H
+
+#include <asm/io.h>
+
+void __iowrite32_copy(void __iomem *to, const void *from, size_t count);
+
+#endif /* _LINUX_IO_H */
diff -r c90267e4a29b -r ec2b3675168a lib/iomap_copy.c
--- /dev/null	Thu Jan  1 00:00:00 1970 +0000
+++ b/lib/iomap_copy.c	Thu Jan 12 09:03:37 2006 -0800
@@ -0,0 +1,42 @@
+/*
+ * Copyright 2006 PathScale, Inc.  All Rights Reserved.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+#include <linux/io.h>
+#include <linux/module.h>
+
+/**
+ * __iowrite32_copy - copy data to MMIO space, in 32-bit units
+ * @to: destination, in MMIO space (must be 32-bit aligned)
+ * @from: source (must be 32-bit aligned)
+ * @count: number of 32-bit quantities to copy
+ *
+ * Copy data from kernel space to MMIO space, in units of 32 bits at a
+ * time.  Order of access is not guaranteed, nor is a memory barrier
+ * performed afterwards.
+ */
+void __attribute__((weak)) __iowrite32_copy(void __iomem *to,
+					    const void *from,
+					    size_t count)
+{
+	u32 __iomem *dst = to;
+	const u32 *src = from;
+	const u32 *end = src + count;
+
+	while (src < end)
+		__raw_writel(*src++, dst++);
+}
+EXPORT_SYMBOL_GPL(__iowrite32_copy);

[PATCH 2 of 2] Add faster __iowrite32_copy routine for x86_64

[Bryan O'Sullivan]

This assembly version is measurably faster than the generic version
in lib/iomap_copy.c.

Signed-off-by: Bryan O'Sullivan <bos@pathscale.com>

diff -r ec2b3675168a -r 30c7112c6e81 arch/x86_64/lib/Makefile
--- a/arch/x86_64/lib/Makefile	Thu Jan 12 09:03:37 2006 -0800
+++ b/arch/x86_64/lib/Makefile	Thu Jan 12 09:03:37 2006 -0800
@@ -4,7 +4,7 @@
 
 CFLAGS_csum-partial.o := -funroll-loops
 
-obj-y := io.o
+obj-y := io.o iomap_copy.o
 
 lib-y := csum-partial.o csum-copy.o csum-wrappers.o delay.o \
 	usercopy.o getuser.o putuser.o  \
diff -r ec2b3675168a -r 30c7112c6e81 arch/x86_64/lib/iomap_copy.S
--- /dev/null	Thu Jan  1 00:00:00 1970 +0000
+++ b/arch/x86_64/lib/iomap_copy.S	Thu Jan 12 09:03:37 2006 -0800
@@ -0,0 +1,26 @@
+/*
+ * Copyright 2006 PathScale, Inc.  All Rights Reserved.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+/*
+ * override generic version in lib/iomap_copy.c
+ */
+ 	.globl __iowrite32_copy
+	.p2align 4
+__iowrite32_copy:
+	movl %edx,%ecx
+	rep movsd
+	ret