[patch 0/8] Immediates Values (real variables)

[patch 0/8] Immediates Values (real variables)

[Mathieu Desnoyers]

Hi Andrew,

Here is the new version of the immediate values. It should be _must_ more useful
like this: it now provides full variables encoded as load immediates in the
instruction stream. It has been thoroughly tested too.

It applies to 2.6.22-rc6-mm1 (just like the text edit lock)
It depends on the text edit lock patches.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 1/8] Immediate values - Global modules list and module mutex

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-global-modules-list-and-mutex.patch --]
[-- Type: text/plain, Size: 1876 bytes --]

Remove "static" from module_mutex and the modules list so it can be used by
other builtin objects in the kernel. Otherwise, every code depending on the
module list would have to be put in kernel/module.c. Since the immediate values
depends on the module list but can be considered as logically different, it
makes sense to implement them in their own file.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 include/linux/module.h |    4 ++++
 kernel/module.c        |    4 ++--
 2 files changed, 6 insertions(+), 2 deletions(-)

Index: linux-2.6-lttng/kernel/module.c
===================================================================
--- linux-2.6-lttng.orig/kernel/module.c	2007-07-13 19:30:44.000000000 -0400
+++ linux-2.6-lttng/kernel/module.c	2007-07-13 20:23:34.000000000 -0400
@@ -65,8 +65,8 @@
 static DEFINE_SPINLOCK(modlist_lock);
 
 /* List of modules, protected by module_mutex AND modlist_lock */
-static DEFINE_MUTEX(module_mutex);
-static LIST_HEAD(modules);
+DEFINE_MUTEX(module_mutex);
+LIST_HEAD(modules);
 static DECLARE_MUTEX(notify_mutex);
 
 static BLOCKING_NOTIFIER_HEAD(module_notify_list);
Index: linux-2.6-lttng/include/linux/module.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/module.h	2007-07-13 20:24:22.000000000 -0400
+++ linux-2.6-lttng/include/linux/module.h	2007-07-13 20:24:26.000000000 -0400
@@ -60,6 +60,10 @@
 	struct kobject *drivers_dir;
 };
 
+/* Protects the list of modules. */
+extern struct mutex module_mutex;
+extern struct list_head modules;
+
 /* These are either module local, or the kernel's dummy ones. */
 extern int init_module(void);
 extern void cleanup_module(void);

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 2/8] Immediate Value - Architecture Independent Code

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-architecture-independent-code.patch --]
[-- Type: text/plain, Size: 13002 bytes --]

Immediate values are used as read mostly variables that are rarely updated. They
use code patching to modify the values inscribed in the instruction stream. It
provides a way to save precious cache lines that would otherwise have to be used
by these variables.

There is a generic _immediate_read() version, which uses standard global
variables, and optimized per architecture immediate_read() implementations,
which use a load immediate to remove a data cache hit. When the immediate values
functionnality is disabled in the kernel, it falls back to global variables.

It adds a new rodata section "__immediate" to place the pointers to the enable
value. Immediate values activation functions sits in kernel/immediate.c.

Immediate values refer to the memory address of a previously declared integer.
This integer holds the information about the state of the immediate values
associated, and must be accessed through the API found in linux/immediate.h.

At module load time, each immediate value is checked to see if it must be
enabled. It would be the case if the variable they refer to is exported from
another module and already enabled.

In the early stages of start_kernel(), the immediate values are updated to
reflect the state of the variable they refer to.

* Why should this be merged *

It improves performances on heavy memory I/O workloads.

An interesting result shows the potential this infrastructure has by
showing the slowdown a simple system call such as getppid() suffers when it is
used under heavy user-space cache trashing:

Random walk L1 and L2 trashing surrounding a getppid() call:
(note: in this test, do_syscal_trace was taken at each system call, see
Documentation/immediate.txt in these patches for details)
- No memory pressure :   getppid() takes  1573 cycles
- With memory pressure : getppid() takes 15589 cycles

We therefore have a slowdown of 10 times just to get the kernel variables from
memory. Another test on the same architecture (Intel P4) measured the memory
latency to be 559 cycles. Therefore, each cache line removed from the hot path
would improve the syscall time of 3.5% in these conditions.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 include/asm-generic/vmlinux.lds.h |    7 ++
 include/linux/immediate.h         |   53 ++++++++++++++++
 include/linux/module.h            |    6 +
 init/main.c                       |    2 
 kernel/Makefile                   |    1 
 kernel/immediate.c                |  119 ++++++++++++++++++++++++++++++++++++++
 kernel/module.c                   |   16 +++++
 7 files changed, 204 insertions(+)

Index: linux-2.6-lttng/include/linux/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/linux/immediate.h	2007-07-13 20:40:05.000000000 -0400
@@ -0,0 +1,53 @@
+#ifndef _LINUX_IMMEDIATE_H
+#define _LINUX_IMMEDIATE_H
+
+/*
+ * Immediate values, can be updated at runtime and save cache lines.
+ *
+ * (C) Copyright 2007 Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
+ *
+ * This file is released under the GPLv2.
+ * See the file COPYING for more details.
+ */
+
+#ifdef __KERNEL__
+
+#ifdef CONFIG_IMMEDIATE
+#include <asm/immediate.h>
+#else
+#include <asm-generic/immediate.h>
+#endif
+
+/*
+ * Always access this type with the provided functions.
+ */
+#define DEFINE_IMMEDIATE_TYPE(type, name) \
+	typedef struct { type value; } name
+
+/*
+ * Standard pre-defined immediate types.
+ */
+DEFINE_IMMEDIATE_TYPE(char, immediate_char_t);
+DEFINE_IMMEDIATE_TYPE(short, immediate_short_t);
+DEFINE_IMMEDIATE_TYPE(int, immediate_int_t);
+DEFINE_IMMEDIATE_TYPE(long, immediate_long_t);
+DEFINE_IMMEDIATE_TYPE(void*, immediate_void_ptr_t);
+
+/*
+ * Use this macro to initialize an immediate value to an initial static value.
+ */
+#define IMMEDIATE_INIT(i)		{ (i) }
+
+/*
+ * Force a data read of the immediate value instead of the immediate value
+ * based mechanism. Useful for __init and __exit section data read.
+ */
+#define _immediate_read(var)		(var)->value
+
+/*
+ * Force the use of a normal if () statement depending on an immediate value.
+ */
+#define _immediate_if(var)		if (_immediate_read(var))
+
+#endif /* __KERNEL__ */
+#endif
Index: linux-2.6-lttng/include/asm-generic/vmlinux.lds.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-generic/vmlinux.lds.h	2007-07-13 20:24:47.000000000 -0400
+++ linux-2.6-lttng/include/asm-generic/vmlinux.lds.h	2007-07-13 20:40:10.000000000 -0400
@@ -122,6 +122,13 @@
 		VMLINUX_SYMBOL(__stop___kcrctab_gpl_future) = .;	\
 	}								\
 									\
+	/* Immediate values: pointers */				\
+	__immediate : AT(ADDR(__immediate) - LOAD_OFFSET) {		\
+		VMLINUX_SYMBOL(__start___immediate) = .;		\
+		*(__immediate)						\
+		VMLINUX_SYMBOL(__stop___immediate) = .;			\
+	}								\
+									\
 	/* Kernel symbol table: strings */				\
         __ksymtab_strings : AT(ADDR(__ksymtab_strings) - LOAD_OFFSET) {	\
 		*(__ksymtab_strings)					\
Index: linux-2.6-lttng/include/linux/module.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/module.h	2007-07-13 20:25:12.000000000 -0400
+++ linux-2.6-lttng/include/linux/module.h	2007-07-13 20:40:10.000000000 -0400
@@ -15,6 +15,7 @@
 #include <linux/stringify.h>
 #include <linux/kobject.h>
 #include <linux/moduleparam.h>
+#include <linux/immediate.h>
 #include <asm/local.h>
 
 #include <asm/module.h>
@@ -374,6 +375,11 @@
 	/* The command line arguments (may be mangled).  People like
 	   keeping pointers to this stuff */
 	char *args;
+
+#ifdef CONFIG_IMMEDIATE
+	const struct __immediate *immediate;
+	unsigned int num_immediate;
+#endif
 };
 #ifndef MODULE_ARCH_INIT
 #define MODULE_ARCH_INIT {}
Index: linux-2.6-lttng/kernel/module.c
===================================================================
--- linux-2.6-lttng.orig/kernel/module.c	2007-07-13 20:25:12.000000000 -0400
+++ linux-2.6-lttng/kernel/module.c	2007-07-13 20:40:10.000000000 -0400
@@ -32,6 +32,7 @@
 #include <linux/cpu.h>
 #include <linux/moduleparam.h>
 #include <linux/errno.h>
+#include <linux/immediate.h>
 #include <linux/err.h>
 #include <linux/vermagic.h>
 #include <linux/notifier.h>
@@ -1623,6 +1624,7 @@
 	unsigned int unusedcrcindex;
 	unsigned int unusedgplindex;
 	unsigned int unusedgplcrcindex;
+	unsigned int immediateindex = 0;
 	struct module *mod;
 	long err = 0;
 	void *percpu = NULL, *ptr = NULL; /* Stops spurious gcc warning */
@@ -1719,6 +1721,9 @@
 #ifdef ARCH_UNWIND_SECTION_NAME
 	unwindex = find_sec(hdr, sechdrs, secstrings, ARCH_UNWIND_SECTION_NAME);
 #endif
+#ifdef CONFIG_IMMEDIATE
+	immediateindex = find_sec(hdr, sechdrs, secstrings, "__immediate");
+#endif
 
 	/* Don't keep modinfo section */
 	sechdrs[infoindex].sh_flags &= ~(unsigned long)SHF_ALLOC;
@@ -1729,6 +1734,8 @@
 #endif
 	if (unwindex)
 		sechdrs[unwindex].sh_flags |= SHF_ALLOC;
+	if (immediateindex)
+		sechdrs[immediateindex].sh_flags |= SHF_ALLOC;
 
 	/* Check module struct version now, before we try to use module. */
 	if (!check_modstruct_version(sechdrs, versindex, mod)) {
@@ -1869,6 +1876,13 @@
 	mod->gpl_future_syms = (void *)sechdrs[gplfutureindex].sh_addr;
 	if (gplfuturecrcindex)
 		mod->gpl_future_crcs = (void *)sechdrs[gplfuturecrcindex].sh_addr;
+#ifdef CONFIG_IMMEDIATE
+	if (immediateindex) {
+		mod->immediate = (void *)sechdrs[immediateindex].sh_addr;
+		mod->num_immediate =
+			sechdrs[immediateindex].sh_size / sizeof(*mod->immediate);
+	}
+#endif
 
 	mod->unused_syms = (void *)sechdrs[unusedindex].sh_addr;
 	if (unusedcrcindex)
@@ -1934,6 +1948,8 @@
         }
 #endif
 
+	module_immediate_setup(mod);
+
 	err = module_finalize(hdr, sechdrs, mod);
 	if (err < 0)
 		goto cleanup;
Index: linux-2.6-lttng/kernel/Makefile
===================================================================
--- linux-2.6-lttng.orig/kernel/Makefile	2007-07-13 20:24:47.000000000 -0400
+++ linux-2.6-lttng/kernel/Makefile	2007-07-13 20:40:10.000000000 -0400
@@ -58,6 +58,7 @@
 obj-$(CONFIG_SYSCTL) += utsname_sysctl.o
 obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o
 obj-$(CONFIG_TASKSTATS) += taskstats.o tsacct.o
+obj-$(CONFIG_IMMEDIATE) += immediate.o
 
 ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y)
 # According to Alan Modra <alan@linuxcare.com.au>, the -fno-omit-frame-pointer is
Index: linux-2.6-lttng/kernel/immediate.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/kernel/immediate.c	2007-07-13 20:25:12.000000000 -0400
@@ -0,0 +1,119 @@
+/*
+ * Copyright (C) 2007 Mathieu Desnoyers
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ */
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/immediate.h>
+#include <linux/memory.h>
+
+extern const struct __immediate __start___immediate[];
+extern const struct __immediate __stop___immediate[];
+
+/*
+ * modules_mutex nests inside immediate_mutex. immediate_mutex protects builtin
+ * immediates and module immediates.
+ */
+DEFINE_MUTEX(immediate_mutex);
+
+/*
+ * Sets a range of immediates to a enabled state : set the enable bit.
+ */
+static void _immediate_update_range(
+	const struct __immediate *begin, const struct __immediate *end)
+{
+	const struct __immediate *iter;
+	int ret;
+
+	for (iter = begin; iter < end; iter++) {
+		mutex_lock(&immediate_mutex);
+		kernel_text_lock();
+		ret = arch_immediate_update(iter);
+		kernel_text_unlock();
+		if (ret)
+			printk(KERN_WARNING "Invalid immediate value. "
+					    "Variable at %p, "
+					    "instruction at %p, size %lu\n",
+					    (void*)iter->immediate,
+					    (void*)iter->var, iter->size);
+		mutex_unlock(&immediate_mutex);
+	}
+}
+
+#ifdef CONFIG_MODULES
+/*
+ * Setup the immediate according to the variable upon which it depends.  Called
+ * by load_module with module_mutex held. This mutex protects against concurrent
+ * modifications to modules'immediates. Therefore, since
+ * module_immediate_setup() does not modify builtin immediates, it does not need
+ * to take the immediate_mutex.
+ */
+void module_immediate_setup(struct module *mod)
+{
+	_immediate_update_range(mod->immediate, mod->immediate+mod->num_immediate);
+}
+#endif
+
+#ifdef CONFIG_MODULES
+/*
+ * immediate mutex nests inside the modules mutex.
+ */
+static inline void immediate_update_modules(int lock)
+{
+	struct module *mod;
+
+	if (lock)
+		mutex_lock(&module_mutex);
+	list_for_each_entry(mod, &modules, list) {
+		if (mod->taints)
+			continue;
+		_immediate_update_range(mod->immediate,
+			mod->immediate + mod->num_immediate);
+	}
+	if (lock)
+		mutex_unlock(&module_mutex);
+}
+#else
+static inline void immediate_update_modules(int lock) { }
+#endif
+
+void immediate_update(int lock)
+{
+	/* Core kernel immediates */
+	_immediate_update_range(__start___immediate, __stop___immediate);
+	/* immediates in modules. */
+	immediate_update_modules(lock);
+}
+EXPORT_SYMBOL_GPL(immediate_update);
+
+/*
+ * Update the immediate values to the state of the variables they refer to. It
+ * is done before SMP is active, at the very beginning of start_kernel().
+ */
+void __init immediate_update_early_range(
+	const struct __immediate *begin, const struct __immediate *end)
+{
+	const struct __immediate *iter;
+
+	for (iter = begin; iter < end; iter++)
+		arch_immediate_update_early(iter);
+}
+
+
+void __init immediate_update_early(void)
+{
+	immediate_update_early_range(__start___immediate, __stop___immediate);
+}
Index: linux-2.6-lttng/init/main.c
===================================================================
--- linux-2.6-lttng.orig/init/main.c	2007-07-13 20:24:47.000000000 -0400
+++ linux-2.6-lttng/init/main.c	2007-07-13 20:25:12.000000000 -0400
@@ -56,6 +56,7 @@
 #include <linux/pid_namespace.h>
 #include <linux/device.h>
 #include <linux/kthread.h>
+#include <linux/immediate.h>
 
 #include <asm/io.h>
 #include <asm/bugs.h>
@@ -521,6 +522,7 @@
 	unwind_init();
 	lockdep_init();
 	container_init_early();
+	immediate_update_early();
 
 	local_irq_disable();
 	early_boot_irqs_off();

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 3/8] Immediate Values - Non Optimized Architectures

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-non-optimized-architectures.patch --]
[-- Type: text/plain, Size: 10092 bytes --]

Architecture agnostic, generic, version of the immediate values. It uses a
global variable to mimic the immediate values.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 include/asm-alpha/immediate.h     |    1 +
 include/asm-arm/immediate.h       |    1 +
 include/asm-arm26/immediate.h     |    1 +
 include/asm-blackfin/immediate.h  |    1 +
 include/asm-cris/immediate.h      |    1 +
 include/asm-frv/immediate.h       |    1 +
 include/asm-generic/immediate.h   |   19 +++++++++++++++++++
 include/asm-h8300/immediate.h     |    1 +
 include/asm-i386/immediate.h      |    1 +
 include/asm-ia64/immediate.h      |    1 +
 include/asm-m32r/immediate.h      |    1 +
 include/asm-m68k/immediate.h      |    1 +
 include/asm-m68knommu/immediate.h |    1 +
 include/asm-mips/immediate.h      |    1 +
 include/asm-parisc/immediate.h    |    1 +
 include/asm-powerpc/immediate.h   |    1 +
 include/asm-ppc/immediate.h       |    1 +
 include/asm-s390/immediate.h      |    1 +
 include/asm-sh/immediate.h        |    1 +
 include/asm-sh64/immediate.h      |    1 +
 include/asm-sparc/immediate.h     |    1 +
 include/asm-sparc64/immediate.h   |    1 +
 include/asm-um/immediate.h        |    1 +
 include/asm-v850/immediate.h      |    1 +
 include/asm-x86_64/immediate.h    |    1 +
 include/asm-xtensa/immediate.h    |    1 +
 26 files changed, 44 insertions(+)

Index: linux-2.6-lttng/include/asm-generic/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-generic/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1,19 @@
+#ifndef _ASM_GENERIC_IMMEDIATE_H
+#define _ASM_GENERIC_IMMEDIATE_H
+
+/*
+ * Generic immediate values: a simple, standard, memory load.
+ */
+
+struct module;
+
+#define immediate_read(var)		_immediate_read(var)
+#define immediate_set(var, i)		((var)->value = (i))
+#define _immediate_set(var, i)		immediate_set(var, i)
+#define immediate_set_early(var, i)	immediate_set(var, i)
+#define immediate_if(var)		if (immediate_read(var))
+
+static inline void module_immediate_setup(struct module *mod) { }
+static inline void immediate_update_early(void) { }
+
+#endif /* _ASM_GENERIC_IMMEDIATE_H */
Index: linux-2.6-lttng/include/asm-alpha/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-alpha/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-arm/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-arm/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-arm26/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-arm26/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-cris/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-cris/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-frv/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-frv/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-h8300/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-h8300/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-ia64/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-ia64/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-m32r/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-m32r/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-m68k/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-m68k/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-m68knommu/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-m68knommu/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-mips/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-mips/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-parisc/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-parisc/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-ppc/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-ppc/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-s390/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-s390/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-sh/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-sh/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-sh64/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-sh64/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-sparc/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-sparc/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-sparc64/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-sparc64/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-um/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-um/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-v850/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-v850/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-x86_64/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-x86_64/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-xtensa/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-xtensa/immediate.h	2007-07-13 19:30:52.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-i386/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-i386/immediate.h	2007-07-13 20:02:08.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-powerpc/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-powerpc/immediate.h	2007-07-13 20:02:08.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
Index: linux-2.6-lttng/include/asm-blackfin/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-blackfin/immediate.h	2007-07-13 20:02:35.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 4/8] Immediate Value - Add kconfig menus

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-kconfig-menus.patch --]
[-- Type: text/plain, Size: 14286 bytes --]

Immediate values provide a way to use dynamic code patching to update variables
sitting within the instruction stream. It saves caches lines normally used by
static read mostly variables.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Adrian Bunk <bunk@stusta.de>
CC: Andi Kleen <andi@firstfloor.org>
---

 arch/alpha/Kconfig       |    6 ++++++
 arch/arm/Kconfig         |    6 ++++++
 arch/arm26/Kconfig       |    6 ++++++
 arch/avr32/Kconfig.debug |    7 +++++++
 arch/cris/Kconfig        |    6 ++++++
 arch/frv/Kconfig         |    6 ++++++
 arch/h8300/Kconfig       |    6 ++++++
 arch/i386/Kconfig        |    2 ++
 arch/ia64/Kconfig        |    3 +++
 arch/m32r/Kconfig        |    6 ++++++
 arch/m68k/Kconfig        |    6 ++++++
 arch/m68knommu/Kconfig   |    6 ++++++
 arch/mips/Kconfig        |    6 ++++++
 arch/parisc/Kconfig      |    6 ++++++
 arch/powerpc/Kconfig     |    3 +++
 arch/ppc/Kconfig         |    6 ++++++
 arch/s390/Kconfig        |    2 ++
 arch/sh/Kconfig          |    6 ++++++
 arch/sh64/Kconfig        |    6 ++++++
 arch/sparc/Kconfig       |    2 ++
 arch/sparc64/Kconfig     |    3 +++
 arch/um/Kconfig          |    6 ++++++
 arch/v850/Kconfig        |    6 ++++++
 arch/x86_64/Kconfig      |    3 +++
 arch/xtensa/Kconfig      |    6 ++++++
 kernel/Kconfig.immediate |   11 +++++++++++
 26 files changed, 138 insertions(+)

Index: linux-2.6-lttng/arch/alpha/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/alpha/Kconfig	2007-07-13 14:17:59.000000000 -0400
+++ linux-2.6-lttng/arch/alpha/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -653,6 +653,12 @@
 
 source "arch/alpha/oprofile/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/alpha/Kconfig.debug"
 
 # DUMMY_CONSOLE may be defined in drivers/video/console/Kconfig
Index: linux-2.6-lttng/arch/arm/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/arm/Kconfig	2007-07-13 14:17:59.000000000 -0400
+++ linux-2.6-lttng/arch/arm/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -1046,6 +1046,12 @@
 
 source "arch/arm/oprofile/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/arm/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/arm26/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/arm26/Kconfig	2007-07-13 14:17:59.000000000 -0400
+++ linux-2.6-lttng/arch/arm26/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -244,6 +244,12 @@
 
 source "drivers/usb/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/arm26/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/cris/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/cris/Kconfig	2007-07-13 14:17:59.000000000 -0400
+++ linux-2.6-lttng/arch/cris/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -198,6 +198,12 @@
 
 source "drivers/usb/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/cris/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/frv/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/frv/Kconfig	2007-07-13 14:17:59.000000000 -0400
+++ linux-2.6-lttng/arch/frv/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -375,6 +375,12 @@
 
 source "fs/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/frv/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/h8300/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/h8300/Kconfig	2007-07-13 14:17:59.000000000 -0400
+++ linux-2.6-lttng/arch/h8300/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -223,6 +223,12 @@
 
 source "fs/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/h8300/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/i386/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/i386/Kconfig	2007-07-13 14:17:59.000000000 -0400
+++ linux-2.6-lttng/arch/i386/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -1249,6 +1249,8 @@
 	  for kernel debugging, non-intrusive instrumentation and testing.
 	  If in doubt, say "N".
 
+source "kernel/Kconfig.immediate"
+
 endif # INSTRUMENTATION
 
 source "arch/i386/Kconfig.debug"
Index: linux-2.6-lttng/arch/ia64/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/ia64/Kconfig	2007-07-13 14:17:59.000000000 -0400
+++ linux-2.6-lttng/arch/ia64/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -593,6 +593,9 @@
 	  a probepoint and specifies the callback.  Kprobes is useful
 	  for kernel debugging, non-intrusive instrumentation and testing.
 	  If in doubt, say "N".
+
+source "kernel/Kconfig.immediate"
+
 endmenu
 
 source "arch/ia64/Kconfig.debug"
Index: linux-2.6-lttng/arch/m32r/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/m32r/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/m32r/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -408,6 +408,12 @@
 
 source "arch/m32r/oprofile/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/m32r/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/m68k/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/m68k/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/m68k/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -676,6 +676,12 @@
 
 source "fs/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/m68k/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/m68knommu/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/m68knommu/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/m68knommu/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -668,6 +668,12 @@
 
 source "fs/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/m68knommu/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/mips/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/mips/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/mips/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -1946,6 +1946,12 @@
 
 source "arch/mips/oprofile/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/mips/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/parisc/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/parisc/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/parisc/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -269,6 +269,12 @@
 
 source "arch/parisc/oprofile/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/parisc/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/powerpc/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/powerpc/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/powerpc/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -655,6 +655,9 @@
 	  a probepoint and specifies the callback.  Kprobes is useful
 	  for kernel debugging, non-intrusive instrumentation and testing.
 	  If in doubt, say "N".
+
+source "kernel/Kconfig.immediate"
+
 endmenu
 
 source "arch/powerpc/Kconfig.debug"
Index: linux-2.6-lttng/arch/ppc/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/ppc/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/ppc/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -1451,8 +1451,14 @@
 
 source "lib/Kconfig"
 
+menu "Instrumentation Support"
+
 source "arch/powerpc/oprofile/Kconfig"
 
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/ppc/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/s390/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/s390/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/s390/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -547,6 +547,8 @@
 	  for kernel debugging, non-intrusive instrumentation and testing.
 	  If in doubt, say "N".
 
+source "kernel/Kconfig.immediate"
+
 endmenu
 
 source "arch/s390/Kconfig.debug"
Index: linux-2.6-lttng/arch/sh/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/sh/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/sh/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -721,6 +721,12 @@
 
 source "arch/sh/oprofile/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/sh/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/sh64/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/sh64/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/sh64/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -284,6 +284,12 @@
 
 source "arch/sh64/oprofile/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/sh64/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/sparc/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/sparc/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/sparc/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -313,6 +313,8 @@
 
 source "arch/sparc/oprofile/Kconfig"
 
+source "kernel/Kconfig.immediate"
+
 endmenu
 
 source "arch/sparc/Kconfig.debug"
Index: linux-2.6-lttng/arch/sparc64/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/sparc64/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/sparc64/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -451,6 +451,9 @@
 	  a probepoint and specifies the callback.  Kprobes is useful
 	  for kernel debugging, non-intrusive instrumentation and testing.
 	  If in doubt, say "N".
+
+source "kernel/Kconfig.immediate"
+
 endmenu
 
 source "arch/sparc64/Kconfig.debug"
Index: linux-2.6-lttng/arch/um/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/um/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/um/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -335,4 +335,10 @@
 	bool
 	default n
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/um/Kconfig.debug"
Index: linux-2.6-lttng/arch/v850/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/v850/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/v850/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -331,6 +331,12 @@
 
 source "drivers/usb/Kconfig"
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/v850/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/arch/x86_64/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/x86_64/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/x86_64/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -842,6 +842,9 @@
 	  a probepoint and specifies the callback.  Kprobes is useful
 	  for kernel debugging, non-intrusive instrumentation and testing.
 	  If in doubt, say "N".
+
+source "kernel/Kconfig.immediate"
+
 endmenu
 
 source "arch/x86_64/Kconfig.debug"
Index: linux-2.6-lttng/arch/xtensa/Kconfig
===================================================================
--- linux-2.6-lttng.orig/arch/xtensa/Kconfig	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/xtensa/Kconfig	2007-07-13 14:33:51.000000000 -0400
@@ -251,6 +251,12 @@
 	  provide one yourself.
 endmenu
 
+menu "Instrumentation Support"
+
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 source "arch/xtensa/Kconfig.debug"
 
 source "security/Kconfig"
Index: linux-2.6-lttng/kernel/Kconfig.immediate
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/kernel/Kconfig.immediate	2007-07-13 14:33:47.000000000 -0400
@@ -0,0 +1,11 @@
+# Immediate values configuration
+
+config IMMEDIATE
+	bool "Enable immediate values"
+	depends on X86_32 || PPC
+	default y if !EMBEDDED
+	help
+	  Immediate values are used as read mostly variables that are rarely
+	  updated. They use code patching to modify the values inscribed in the
+	  instruction stream. It provides a way to save precious cache lines
+	  that would otherwise have to be used by these variables.
Index: linux-2.6-lttng/arch/avr32/Kconfig.debug
===================================================================
--- linux-2.6-lttng.orig/arch/avr32/Kconfig.debug	2007-07-13 14:18:00.000000000 -0400
+++ linux-2.6-lttng/arch/avr32/Kconfig.debug	2007-07-13 14:33:51.000000000 -0400
@@ -6,6 +6,9 @@
 
 source "lib/Kconfig.debug"
 
+menu "Instrumentation Support"
+	depends on EXPERIMENTAL
+
 config KPROBES
 	bool "Kprobes"
 	depends on DEBUG_KERNEL
@@ -16,4 +19,8 @@
           for kernel debugging, non-intrusive instrumentation and testing.
           If in doubt, say "N".
 
+source "kernel/Kconfig.immediate"
+
+endmenu
+
 endmenu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 5/8] Immediate Values - kprobe header fix

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-kprobes-headers.patch --]
[-- Type: text/plain, Size: 1299 bytes --]

Since the immediate values depend on the same int3 handler as kprobes implements
for i386, we have to get architecture specific defines available for the kprobes
trap handler (especially restore_interrupts()) wven when CONFIG_KPROBES is not
selected.

That kind of ifdef around a whole header does not make sense in the first place
anyway.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: prasanna@in.ibm.com
CC: ananth@in.ibm.com
CC: anil.s.keshavamurthy@intel.com
CC: davem@davemloft.net
---
 include/linux/kprobes.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-2.6-lttng/include/linux/kprobes.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/kprobes.h	2007-07-13 18:47:57.000000000 -0400
+++ linux-2.6-lttng/include/linux/kprobes.h	2007-07-13 18:48:45.000000000 -0400
@@ -36,9 +36,9 @@
 #include <linux/spinlock.h>
 #include <linux/rcupdate.h>
 
-#ifdef CONFIG_KPROBES
 #include <asm/kprobes.h>
 
+#ifdef CONFIG_KPROBES
 /* kprobe_status settings */
 #define KPROBE_HIT_ACTIVE	0x00000001
 #define KPROBE_HIT_SS		0x00000002

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 6/8] Immediate Value - i386 Optimization

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-i386-optimization.patch --]
[-- Type: text/plain, Size: 16410 bytes --]

i386 optimization of the immediate values which uses a movl with code patching
to set/unset the value used to populate the register used as variable source.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 arch/i386/kernel/Makefile    |    1 
 arch/i386/kernel/immediate.c |  295 +++++++++++++++++++++++++++++++++++++++++++
 arch/i386/kernel/traps.c     |    8 -
 include/asm-i386/immediate.h |  118 +++++++++++++++++
 4 files changed, 417 insertions(+), 5 deletions(-)

Index: linux-2.6-lttng/include/asm-i386/immediate.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-i386/immediate.h	2007-07-13 18:48:43.000000000 -0400
+++ linux-2.6-lttng/include/asm-i386/immediate.h	2007-07-13 19:28:17.000000000 -0400
@@ -1 +1,117 @@
-#include <asm-generic/immediate.h>
+#ifndef _ASM_I386_IMMEDIATE_H
+#define _ASM_I386_IMMEDIATE_H
+
+/*
+ * Immediate values. i386 architecture optimizations.
+ *
+ * (C) Copyright 2006 Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
+ *
+ * This file is released under the GPLv2.
+ * See the file COPYING for more details.
+ */
+
+struct module;
+
+struct __immediate {
+	long var;		/* Pointer to the identifier variable of the
+				 * immediate value
+				 */
+	long immediate;		/*
+				 * Pointer to the memory location of the
+				 * immediate value within the instruction.
+				 */
+	long size;		/* Type size. */
+};
+
+/*
+ * Optimized version of the immediate.
+ * Make sure the 2 and 4 bytes update will be atomic by aligning the immediate
+ * value.
+ * 2 bytes (short) uses a 66H prefix.
+ * If size is bigger than 4 bytes, fall back on a memory read.
+ * Do not use in __init and __exit functions. Use _immediate_read() instead.
+ */
+#define immediate_read(var)						\
+	({								\
+		__typeof__((var)->value) value;				\
+		switch (sizeof(value)) {				\
+		case 1:							\
+			asm (	".section __immediate, \"a\", @progbits;\n\t" \
+					".long %1, (0f)+1, 1;\n\t"	\
+					".previous;\n\t"		\
+					"0:\n\t"			\
+					"mov %2,%0;\n\t"		\
+				: "=r" (value)				\
+				: "m" ((var)->value),			\
+				  "i" (0));				\
+			break;						\
+		case 2:							\
+			asm (	".section __immediate, \"a\", @progbits;\n\t" \
+					".long %1, (0f)+2, 2;\n\t"	\
+					".previous;\n\t"		\
+					"1:\n\t"			\
+					".align 2;\n\t"			\
+					"0:\n\t"			\
+					"mov %2,%0;\n\t"		\
+				: "=r" (value)				\
+				: "m" ((var)->value),			\
+				  "i" (0));				\
+			break;						\
+		case 4:							\
+			asm (	".section __immediate, \"a\", @progbits;\n\t" \
+					".long %1, (0f)+1, 4;\n\t"	\
+					".previous;\n\t"		\
+					"1:\n\t"			\
+					".org (1b)+(3-((1b)%%4)), 0x90;\n\t" \
+					"0:\n\t"			\
+					"mov %2,%0;\n\t"		\
+				: "=r" (value)				\
+				: "m" ((var)->value),			\
+				  "i" (0));				\
+			break;						\
+		default:value = (var)->value;				\
+			break;						\
+		};							\
+		value;							\
+	})
+
+/*
+ * Update immediate value, can take module mutex.
+ */
+#define immediate_set(var, i) \
+	(var)->value = (i); \
+	immediate_update(1);
+
+/*
+ * Update immediate value. Module mutex must already be taken.
+ */
+#define _immediate_set(var, i) \
+	(var)->value = (i); \
+	immediate_update(0);
+
+/*
+ * Update immediate value at early boot.
+ */
+#define immediate_set_early(var, i) \
+	(var)->value = (i); \
+	immediate_update_early();
+
+/*
+ * Branch depending on an immediate value. Could eventually be optimized further
+ * by improving gcc to give the ability to patch a jump instruction instead of
+ * the value it depends on.
+ * Do not use in __init and __exit functions. Use _immediate_if() instead.
+ */
+#define immediate_if(var)	if (unlikely(immediate_read(var)))
+
+/*
+ * Used internally.
+ */
+
+extern void immediate_update(int lock);
+extern void module_immediate_setup(struct module *mod);
+extern void immediate_update_early(void);
+extern int arch_immediate_update(const struct __immediate *immediate);
+extern void arch_immediate_update_early(const struct __immediate *immediate);
+
+#endif /* _ASM_I386_IMMEDIATE_H */
Index: linux-2.6-lttng/arch/i386/kernel/Makefile
===================================================================
--- linux-2.6-lttng.orig/arch/i386/kernel/Makefile	2007-07-13 18:35:15.000000000 -0400
+++ linux-2.6-lttng/arch/i386/kernel/Makefile	2007-07-13 18:49:22.000000000 -0400
@@ -35,6 +35,7 @@
 obj-y				+= sysenter.o vsyscall.o
 obj-$(CONFIG_ACPI_SRAT) 	+= srat.o
 obj-$(CONFIG_EFI) 		+= efi.o efi_stub.o
+obj-$(CONFIG_IMMEDIATE)		+= immediate.o
 obj-$(CONFIG_DOUBLEFAULT) 	+= doublefault.o
 obj-$(CONFIG_SERIAL_8250)	+= legacy_serial.o
 obj-$(CONFIG_VM86)		+= vm86.o
Index: linux-2.6-lttng/arch/i386/kernel/immediate.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/arch/i386/kernel/immediate.c	2007-07-13 19:21:21.000000000 -0400
@@ -0,0 +1,295 @@
+/*
+ * Immediate Value - i386 architecture specific code.
+ *
+ * Rationale
+ *
+ * Required because of :
+ * - Erratum 49 fix for Intel PIII.
+ * - Still present on newer processors : Intel Core 2 Duo Processor for Intel
+ *   Centrino Duo Processor Technology Specification Update, AH33.
+ *   Unsynchronized Cross-Modifying Code Operations Can Cause Unexpected
+ *   Instruction Execution Results.
+ *
+ * Permits immediate value modification by XMC with correct serialization.
+ *
+ * Reentrant for NMI and trap handler instrumentation. Permits XMC to a
+ * location that has preemption enabled because it involves no temporary or
+ * reused data structure.
+ *
+ * Quoting Richard J Moore, source of the information motivating this
+ * implementation which differs from the one proposed by Intel which is not
+ * suitable for kernel context (does not support NMI and would require disabling
+ * interrupts on every CPU for a long period) :
+ *
+ * "There is another issue to consider when looking into using probes other
+ * then int3:
+ *
+ * Intel erratum 54 - Unsynchronized Cross-modifying code - refers to the
+ * practice of modifying code on one processor where another has prefetched
+ * the unmodified version of the code. Intel states that unpredictable general
+ * protection faults may result if a synchronizing instruction (iret, int,
+ * int3, cpuid, etc ) is not executed on the second processor before it
+ * executes the pre-fetched out-of-date copy of the instruction.
+ *
+ * When we became aware of this I had a long discussion with Intel's
+ * microarchitecture guys. It turns out that the reason for this erratum
+ * (which incidentally Intel does not intend to fix) is because the trace
+ * cache - the stream of micorops resulting from instruction interpretation -
+ * cannot guaranteed to be valid. Reading between the lines I assume this
+ * issue arises because of optimization done in the trace cache, where it is
+ * no longer possible to identify the original instruction boundaries. If the
+ * CPU discoverers that the trace cache has been invalidated because of
+ * unsynchronized cross-modification then instruction execution will be
+ * aborted with a GPF. Further discussion with Intel revealed that replacing
+ * the first opcode byte with an int3 would not be subject to this erratum.
+ *
+ * So, is cmpxchg reliable? One has to guarantee more than mere atomicity."
+ *
+ * Overall design
+ *
+ * The algorithm proposed by Intel applies not so well in kernel context: it
+ * would imply disabling interrupts and looping on every CPUs while modifying
+ * the code and would not support instrumentation of code called from interrupt
+ * sources that cannot be disabled.
+ *
+ * Therefore, we use a different algorithm to respect Intel's erratum (see the
+ * quoted discussion above). We make sure that no CPU sees an out-of-date copy
+ * of a pre-fetched instruction by 1 - using a breakpoint, which skips the
+ * instruction that is going to be modified, 2 - issuing an IPI to every CPU to
+ * execute a sync_core(), to make sure that even when the breakpoint is removed,
+ * no cpu could possibly still have the out-of-date copy of the instruction,
+ * modify the now unused 2nd byte of the instruction, and then put back the
+ * original 1st byte of the instruction.
+ *
+ * It has exactly the same intent as the algorithm proposed by Intel, but
+ * it has less side-effects, scales better and supports NMI, SMI and MCE.
+ *
+ * Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
+ */
+
+#include <linux/notifier.h>
+#include <linux/preempt.h>
+#include <linux/smp.h>
+#include <linux/notifier.h>
+#include <linux/module.h>
+#include <linux/immediate.h>
+#include <linux/kdebug.h>
+#include <linux/rcupdate.h>
+#include <linux/kprobes.h>
+
+#include <asm/cacheflush.h>
+
+#define BREAKPOINT_INSTRUCTION  0xcc
+#define BREAKPOINT_INS_LEN	1
+#define NOP_INSTRUCTION		0x90
+#define NR_NOPS			8
+
+static long target_after_int3;		/* EIP of the target after the int3 */
+static long bypass_eip;			/* EIP of the bypass. */
+static long bypass_after_int3;		/* EIP after the end-of-bypass int3 */
+static long after_immediate;		/*
+					 * EIP where to resume after the
+					 * single-stepping.
+					 */
+
+/*
+ * Size of the movl instruction (without the immediate value) in bytes.
+ * The 2 bytes load immediate has a 66H prefix, which makes the opcode 2 bytes
+ * wide.
+ */
+static inline size_t _immediate_get_insn_size(long size)
+{
+	switch (size) {
+		case 1: return 1;
+		case 2: return 2;
+		case 4: return 1;
+		default: BUG();
+	};
+}
+
+/*
+ * Internal bypass used during value update. The bypass is skipped by the
+ * function in which it is inserted.
+ * No need to be aligned because we exclude readers from the site during
+ * update.
+ * Layout is:
+ * nop nop nop nop nop nop nop nop int3
+ * The nops are the target replaced by the instruction to single-step.
+ */
+static inline void _immediate_bypass(long *bypassaddr, long *breaknextaddr)
+{
+		asm volatile (	"jmp 2f;\n\t"
+				"0:\n\t"
+				".space 8, 0x90;\n\t"
+				"1:\n\t"
+				"int3;\n\t"
+				"2:\n\t"
+				"movl $(0b),%0;\n\t"
+				"movl $((1b)+1),%1;\n\t"
+				: "=r" (*bypassaddr),
+				  "=r" (*breaknextaddr) : );
+}
+
+static void immediate_synchronize_core(void *info)
+{
+	sync_core();	/* use cpuid to stop speculative execution */
+}
+
+/*
+ * The eip value points right after the breakpoint instruction, in the second
+ * byte of the movl.
+ * Disable preemption in the bypass to make sure no thread will be preempted in
+ * it. We can then use synchronize_sched() to make sure every bypass users have
+ * ended.
+ */
+static int immediate_notifier(struct notifier_block *nb,
+	unsigned long val, void *data)
+{
+	enum die_val die_val = (enum die_val) val;
+	struct die_args *args = data;
+
+	if (!args->regs || user_mode_vm(args->regs))
+		return NOTIFY_DONE;
+
+	if (die_val == DIE_INT3) {
+		if (args->regs->eip == target_after_int3) {
+			preempt_disable();
+			args->regs->eip = bypass_eip;
+			return NOTIFY_STOP;
+		} else if (args->regs->eip == bypass_after_int3) {
+			args->regs->eip = after_immediate;
+			preempt_enable();
+			return NOTIFY_STOP;
+		}
+	}
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block immediate_notify = {
+	.notifier_call = immediate_notifier,
+	.priority = 0x7fffffff,	/* we need to be notified first */
+};
+
+/*
+ * Must be called with immediate_mutex held.
+ */
+__kprobes int arch_immediate_update(const struct __immediate *immediate)
+{
+	int ret;
+	size_t insn_size = _immediate_get_insn_size(immediate->size);
+	long insn = immediate->immediate - insn_size;
+	long bypass_immediate;
+
+#ifdef CONFIG_KPROBES
+	/*
+	 * Fail if a kprobe has been set on this instruction.
+	 * (TODO: we could eventually do better and modify all the (possibly
+	 * nested) kprobes for this site if kprobes had an API for this.
+	 */
+	if (unlikely(*(unsigned char*)insn == BREAKPOINT_INSTRUCTION)) {
+		printk(KERN_WARNING "Immediate value in conflict with kprobe. "
+				    "Variable at %p, "
+				    "instruction at %p, size %lu\n",
+				    (void*)immediate->immediate,
+				    (void*)immediate->var, immediate->size);
+		return -EBUSY;
+	}
+#endif
+
+	/*
+	 * If the variable and the instruction have the same value, there is
+	 * nothing to do.
+	 */
+	switch (immediate->size) {
+		case 1:	if (*(uint8_t*)immediate->immediate
+					== *(uint8_t*)immediate->var)
+				return 0;
+			break;
+		case 2:	if (*(uint16_t*)immediate->immediate
+					== *(uint16_t*)immediate->var)
+				return 0;
+			break;
+		case 4:	if (*(uint32_t*)immediate->immediate
+					== *(uint32_t*)immediate->var)
+				return 0;
+			break;
+		default:return -EINVAL;
+	}
+
+	_immediate_bypass(&bypass_eip, &bypass_after_int3);
+
+	after_immediate = immediate->immediate + immediate->size;
+	bypass_immediate = bypass_eip + insn_size;
+
+	kernel_text_mark_rw((unsigned long)bypass_eip, NR_NOPS);
+	memcpy((void*)bypass_eip, (void*)insn, insn_size + immediate->size);
+	/*
+	 * Fill the rest with nops.
+	 */
+	memset((void*)(bypass_immediate + immediate->size),
+		NOP_INSTRUCTION,
+		NR_NOPS - immediate->size - insn_size);
+	kernel_text_unmark((unsigned long)bypass_eip, NR_NOPS);
+
+	kernel_text_mark_rw((unsigned long)insn, insn_size + immediate->size);
+	target_after_int3 = insn + BREAKPOINT_INS_LEN;
+	/* register_die_notifier has memory barriers */
+	register_die_notifier(&immediate_notify);
+	/* The breakpoint will single-step the bypass */
+	*(char*)insn = BREAKPOINT_INSTRUCTION;
+	wmb();
+	/*
+	 * Execute serializing instruction on each CPU.
+	 * Acts as a memory barrier.
+	 */
+	ret = on_each_cpu(immediate_synchronize_core, NULL, 1, 1);
+	BUG_ON(ret != 0);
+
+	memcpy((void*)immediate->immediate, (void*)immediate->var,
+			immediate->size);
+	wmb();
+	*(char*)insn = *(char*)bypass_eip;
+	kernel_text_unmark((unsigned long)insn, insn_size + immediate->size);
+		/*
+		 * Wait for all int3 handlers to end
+		 * (interrupts are disabled in int3).
+		 * This CPU is clearly not in a int3 handler,
+		 * because int3 handler is not preemptible and
+		 * there cannot be any more int3 handler called
+		 * for this site, because we placed the original
+		 * instruction back.
+		 * synchronize_sched has memory barriers.
+		 */
+	synchronize_sched();
+	unregister_die_notifier(&immediate_notify);
+	/* unregister_die_notifier has memory barriers */
+	return 0;
+}
+
+/*
+ * Very early initialization of the in-core immediate values.
+ */
+void __init arch_immediate_update_early(const struct __immediate *immediate)
+{
+	/*
+	 * If the variable and the instruction have the same value, there is
+	 * nothing to do.
+	 */
+	switch (immediate->size) {
+		case 1:	if (*(uint8_t*)immediate->immediate
+					== *(uint8_t*)immediate->var)
+				return;
+			break;
+		case 2:	if (*(uint16_t*)immediate->immediate
+					== *(uint16_t*)immediate->var)
+				return;
+			break;
+		case 4:	if (*(uint32_t*)immediate->immediate
+					== *(uint32_t*)immediate->var)
+				return;
+			break;
+		default:return;
+	}
+	memcpy((void*)immediate->immediate, (void*)immediate->var,
+			immediate->size);
+}
Index: linux-2.6-lttng/arch/i386/kernel/traps.c
===================================================================
--- linux-2.6-lttng.orig/arch/i386/kernel/traps.c	2007-07-13 18:35:15.000000000 -0400
+++ linux-2.6-lttng/arch/i386/kernel/traps.c	2007-07-13 19:25:38.000000000 -0400
@@ -629,7 +629,7 @@
 }
 
 DO_VM86_ERROR_INFO( 0, SIGFPE,  "divide error", divide_error, FPE_INTDIV, regs->eip)
-#ifndef CONFIG_KPROBES
+#if !defined(CONFIG_KPROBES) && !defined(CONFIG_IMMEDIATE)
 DO_VM86_ERROR( 3, SIGTRAP, "int3", int3)
 #endif
 DO_VM86_ERROR( 4, SIGSEGV, "overflow", overflow)
@@ -856,14 +856,14 @@
 	nmi_exit();
 }
 
-#ifdef CONFIG_KPROBES
+#if defined(CONFIG_KPROBES) || defined(CONFIG_IMMEDIATE)
 fastcall void __kprobes do_int3(struct pt_regs *regs, long error_code)
 {
 	if (notify_die(DIE_INT3, "int3", regs, error_code, 3, SIGTRAP)
 			== NOTIFY_STOP)
 		return;
-	/* This is an interrupt gate, because kprobes wants interrupts
-	disabled.  Normal trap handlers don't. */
+	/* This is an interrupt gate, because kprobes and immediate values wants
+	 * interrupts disabled. Normal trap handlers don't. */
 	restore_interrupts(regs);
 	do_trap(3, SIGTRAP, "int3", 1, regs, error_code, NULL);
 }

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 7/8] Immediate Value - PowerPC Optimization

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-powerpc-optimization.patch --]
[-- Type: text/plain, Size: 7359 bytes --]

PowerPC optimization of the immediate values which uses a li instruction,
patched with an immediate value.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 arch/powerpc/kernel/Makefile    |    1 
 arch/powerpc/kernel/immediate.c |   98 ++++++++++++++++++++++++++++++++++++
 include/asm-powerpc/immediate.h |  107 +++++++++++++++++++++++++++++++++++++++-
 3 files changed, 205 insertions(+), 1 deletion(-)

Index: linux-2.6-lttng/include/asm-powerpc/immediate.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-powerpc/immediate.h	2007-07-13 19:26:18.000000000 -0400
+++ linux-2.6-lttng/include/asm-powerpc/immediate.h	2007-07-13 19:28:49.000000000 -0400
@@ -1 +1,106 @@
-#include <asm-generic/immediate.h>
+#ifndef _ASM_POWERPC_IMMEDIATE_H
+#define _ASM_POWERPC_IMMEDIATE_H
+
+/*
+ * Immediate values. PowerPC architecture optimizations.
+ *
+ * (C) Copyright 2006 Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
+ *
+ * This file is released under the GPLv2.
+ * See the file COPYING for more details.
+ */
+
+#include <asm/asm-compat.h>
+
+struct module;
+
+struct __immediate {
+	long var;		/* Identifier variable of the immediate value */
+	long immediate;		/*
+				 * Pointer to the memory location that holds
+				 * the immediate value within the load immediate
+				 * instruction.
+				 */
+	long size;		/* Type size. */
+};
+
+/*
+ * Optimized version of the immediate.
+ * Make sure the 2 bytes update will be atomic by aligning the immediate
+ * value.
+ * Use a normal memory read for the 4 bytes immediate because there is no way to
+ * atomically update it without using a seqlock read side, which would cost more
+ * in term of total i-cache and d-cache space than a simple memory read.
+ * Do not use in __init and __exit functions. Use _immediate_read() instead.
+ */
+#define immediate_read(var)						\
+	({								\
+		__typeof__((var)->value) value;				\
+		switch (sizeof(value)) {				\
+		case 1:							\
+			asm (	".section __immediate, \"a\", @progbits;\n\t" \
+					PPC_LONG "%1, ((0f)+3), 1;\n\t"	\
+					".previous;\n\t"		\
+					"0:\n\t"			\
+					"li %0,%2;\n\t"			\
+				: "=r" (value)				\
+				: "i" (&(var)->value),			\
+				  "i" (0));				\
+			break;						\
+		case 2:							\
+			asm (	".section __immediate, \"a\", @progbits;\n\t" \
+					PPC_LONG "%1, ((0f)+2), 2;\n\t"	\
+					".previous;\n\t"		\
+					".align 2\n\t"			\
+					"0:\n\t"			\
+					"li %0,%2;\n\t"			\
+				: "=r" (value)				\
+				: "i" (&(var)->value),			\
+				  "i" (0));				\
+			break;						\
+		default:						\
+			value = (var)->value;				\
+			break;						\
+		};							\
+		value;							\
+	})
+
+/*
+ * Update immediate value, can take module mutex.
+ */
+#define immediate_set(var, i) \
+	(var)->value = (i); \
+	immediate_update(1);
+
+/*
+ * Update immediate value. Module mutex must already be taken.
+ */
+#define _immediate_set(var, i) \
+	(var)->value = (i); \
+	immediate_update(0);
+
+/*
+ * Update immediate value at early boot.
+ */
+#define immediate_set_early(var, i) \
+	(var)->value = (i); \
+	immediate_update_early();
+
+/*
+ * Branch depending on an immediate value. Could eventually be optimized further
+ * by improving gcc to give the ability to patch a jump instruction instead of
+ * the value it depends on.
+ * Do not use in __init and __exit functions. Use _immediate_if() instead.
+ */
+#define immediate_if(var)	if (unlikely(immediate_read(var)))
+
+/*
+ * Used internally.
+ */
+extern void immediate_update(int lock);
+extern void module_immediate_setup(struct module *mod);
+extern void immediate_update_early(void);
+extern int arch_immediate_update(const struct __immediate *immediate);
+extern void arch_immediate_update_early(const struct __immediate *immediate);
+
+#endif /* _ASM_POWERPC_IMMEDIATE_H */
Index: linux-2.6-lttng/arch/powerpc/kernel/Makefile
===================================================================
--- linux-2.6-lttng.orig/arch/powerpc/kernel/Makefile	2007-07-13 19:26:18.000000000 -0400
+++ linux-2.6-lttng/arch/powerpc/kernel/Makefile	2007-07-13 19:28:29.000000000 -0400
@@ -103,3 +103,4 @@
 
 extra-$(CONFIG_PPC_FPU)		+= fpu.o
 extra-$(CONFIG_PPC64)		+= entry_64.o
+obj-$(CONFIG_IMMEDIATE)		+= immediate.o
Index: linux-2.6-lttng/arch/powerpc/kernel/immediate.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/arch/powerpc/kernel/immediate.c	2007-07-13 19:28:29.000000000 -0400
@@ -0,0 +1,98 @@
+/*
+ * Powerpc optimized immediate values enabling/disabling.
+ *
+ * Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
+ */
+
+#include <linux/module.h>
+#include <linux/immediate.h>
+#include <linux/string.h>
+#include <linux/kprobes.h>
+#include <asm/cacheflush.h>
+#include <asm/page.h>
+
+#define LI_OPCODE_LEN	2
+
+/*
+ * The immediate value are aligned.
+ */
+int arch_immediate_update(const struct __immediate *immediate)
+{
+#ifdef CONFIG_KPROBES
+	kprobe_opcode_t *insn;
+	/*
+	 * Fail if a kprobe has been set on this instruction.
+	 * (TODO: we could eventually do better and modify all the (possibly
+	 * nested) kprobes for this site if kprobes had an API for this.
+	 */
+	switch (immediate->size) {
+		case 1:	/* The uint8_t points to the 3rd byte of the
+			 * instruction */
+			insn = (void*)(immediate->immediate - 1 - LI_OPCODE_LEN);
+			break;
+		case 2:	insn = (void*)(immediate->immediate - LI_OPCODE_LEN);
+			break;
+		default:
+		return -EINVAL;
+	}
+
+	if (unlikely(*insn == BREAKPOINT_INSTRUCTION)) {
+		printk(KERN_WARNING "Immediate value in conflict with kprobe. "
+				    "Variable at %p, "
+				    "instruction at %p, size %lu\n",
+				    (void*)immediate->immediate,
+				    (void*)immediate->var, immediate->size);
+		return -EBUSY;
+	}
+#endif
+
+	/*
+	 * If the variable and the instruction have the same value, there is
+	 * nothing to do.
+	 */
+	switch (immediate->size) {
+		case 1:	if (*(uint8_t*)immediate->immediate
+					== *(uint8_t*)immediate->var)
+				return 0;
+			break;
+		case 2:	if (*(uint16_t*)immediate->immediate
+					== *(uint16_t*)immediate->var)
+				return 0;
+			break;
+		default:return -EINVAL;
+	}
+	memcpy((void*)immediate->immediate, (void*)immediate->var,
+			immediate->size);
+	flush_icache_range((unsigned long)immediate->immediate,
+				immediate->size);
+	return 0;
+}
+
+/*
+ * Very early initialization of the in-core immediate values.
+ * We can use flush_icache_range, since the cpu identification has been done in
+ * the early_init stage.
+ */
+void __init arch_immediate_update_early(const struct __immediate *immediate)
+{
+	/*
+	 * If the variable and the instruction have the same value, there is
+	 * nothing to do.
+	 */
+	switch (immediate->size) {
+		case 1:	if (*(uint8_t*)immediate->immediate
+					== *(uint8_t*)immediate->var)
+				return;
+			break;
+		case 2:	if (*(uint16_t*)immediate->immediate
+					== *(uint16_t*)immediate->var)
+				return;
+			break;
+		default:return;
+	}
+	memcpy((void*)immediate->immediate, (void*)immediate->var,
+			immediate->size);
+	flush_icache_range((unsigned long)immediate->immediate,
+				immediate->size);
+}
+

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 8/8] Immediate Value - Documentation

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-documentation.patch --]
[-- Type: text/plain, Size: 7283 bytes --]

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 Documentation/immediate.txt |  180 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 180 insertions(+)

Index: linux-2.6-lttng/Documentation/immediate.txt
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/Documentation/immediate.txt	2007-07-13 20:39:51.000000000 -0400
@@ -0,0 +1,180 @@
+		        Using the Immediate Values
+
+			    Mathieu Desnoyers
+
+
+This document introduces Immediate Values and their use.
+
+* Purpose of immediate values
+
+An immediate value is used to compile into the kernel variables that sits within
+the instruction stream. They are meant to be rarely updated but read often.
+Using immediate values for these variables will save cache lines.
+
+This infrastructure is specialized in supporting dynamic patching of the values
+in the instruction stream when multiple CPUs are running without disturbing the
+normal system behavior.
+
+Compiling code meant to be rarely enabled at runtime can be done using
+immediate_if() as condition surrounding the code.
+
+* Usage
+
+In order to use the macro immediate, you should include linux/immediate.h.
+
+#include <linux/immediate.h>
+
+immediate_char_t this_immediate;
+EXPORT_SYMBOL(this_immediate);
+
+
+Add, in your code :
+
+Use immediate_set(&this_immediate) to set the immediate value.
+
+Use immediate_read(&this_immediate) to read the immediate value.
+
+The immediate mechanism supports inserting multiple instances of the same
+immediate. Immediate values can be put in inline functions, inlined static
+functions, and unrolled loops.
+
+If you have to read the immediate values from a function declared as __init or
+__exit, you should explicitly use _immediate_read(), which will fall back on a
+global variable read. Failing to do so will leave a reference to the __init
+section after it is freed (it would generate a modpost warning).
+
+The prefered idiom to dynamically enable compiled-in code is to use
+immediate_if (&this_immediate), which may eventually use gcc improvements to
+provide a jump instruction patching based condition instead of a immediate value
+feeding a conditional jump. You should use _immediate_if () instead of
+immediate_if () in functions marked __init or __exit.
+
+immediate_set_early() should be used only at early kernel boot time, before SMP
+is activated.
+
+If you need to declare your own immediate types (for instance, a pointer to
+struct task_struct), use:
+
+DEFINE_IMMEDIATE_TYPE(struct task_struct*, immediate_task_struct_ptr_t);
+
+and declare your variable with:
+immediate_task_struct_ptr_t myptr;
+
+You can choose to set an initial static value to the immediate by using, for
+instance:
+
+immediate_task_struct_ptr_t myptr = IMMEDIATE_INIT(10);
+
+
+* Optimization for a given architecture
+
+One can implement optimized immediate values for a given architecture by
+replacing asm-$ARCH/immediate.h.
+
+* Performance improvement
+
+* Memory hit for a data-based branch
+
+Here are the results on a 3GHz Pentium 4:
+
+number of tests : 100
+number of branches per test : 100000
+memory hit cycles per iteration (mean) : 636.611
+L1 cache hit cycles per iteration (mean) : 89.6413
+instruction stream based test, cycles per iteration (mean) : 85.3438
+Just getting the pointer from a modulo on a pseudo-random value, doing
+  noting with it, cycles per iteration (mean) : 77.5044
+
+So:
+Base case:                      77.50 cycles
+instruction stream based test:  +7.8394 cycles
+L1 cache hit based test:        +12.1369 cycles
+Memory load based test:         +559.1066 cycles
+
+So let's say we have a ping flood coming at
+(14014 packets transmitted, 14014 received, 0% packet loss, time 1826ms)
+7674 packets per second. If we put 2 markers for irq entry/exit, it
+brings us to 15348 markers sites executed per second.
+
+(15348 exec/s) * (559 cycles/exec) / (3G cycles/s) = 0.0029
+We therefore have a 0.29% slowdown just on this case.
+
+Compared to this, the instruction stream based test will cause a
+slowdown of:
+
+(15348 exec/s) * (7.84 cycles/exec) / (3G cycles/s) = 0.00004
+For a 0.004% slowdown.
+
+If we plan to use this for memory allocation, spinlock, and all sort of
+very high event rate tracing, we can assume it will execute 10 to 100
+times more sites per second, which brings us to 0.4% slowdown with the
+instruction stream based test compared to 29% slowdown with the memory
+load based test on a system with high memory pressure.
+
+
+
+* Markers impact under heavy memory load
+
+Running a kernel with my LTTng instrumentation set, in a test that
+generates memory pressure (from userspace) by trashing L1 and L2 caches
+between calls to getppid() (note: syscall_trace is active and calls
+a marker upon syscall entry and syscall exit; markers are disarmed).
+This test is done in user-space, so there are some delays due to IRQs
+coming and to the scheduler. (UP 2.6.22-rc6-mm1 kernel, task with -20
+nice level)
+
+My first set of results : Linear cache trashing, turned out not to be
+very interesting, because it seems like the linearity of the memset on a
+full array is somehow detected and it does not "really" trash the
+caches.
+
+Now the most interesting result : Random walk L1 and L2 trashing
+surrounding a getppid() call.
+
+- Markers compiled out (but syscall_trace execution forced)
+number of tests : 10000
+No memory pressure
+Reading timestamps takes 108.033 cycles
+getppid : 1681.4 cycles
+With memory pressure
+Reading timestamps takes 102.938 cycles
+getppid : 15691.6 cycles
+
+
+- With the immediate values based markers:
+number of tests : 10000
+No memory pressure
+Reading timestamps takes 108.006 cycles
+getppid : 1681.84 cycles
+With memory pressure
+Reading timestamps takes 100.291 cycles
+getppid : 11793 cycles
+
+
+- With global variables based markers:
+number of tests : 10000
+No memory pressure
+Reading timestamps takes 107.999 cycles
+getppid : 1669.06 cycles
+With memory pressure
+Reading timestamps takes 102.839 cycles
+getppid : 12535 cycles
+
+
+The result is quite interesting in that the kernel is slower without
+markers than with markers. I explain it by the fact that the data
+accessed is not layed out in the same manner in the cache lines when the
+markers are compiled in or out. It seems that it aligns the function's
+data better to compile-in the markers in this case.
+
+But since the interesting comparison is between the immediate values and
+global variables based markers, and because they share the same memory
+layout, except for the movl being replaced by a movz, we see that the
+global variable based markers (2 markers) adds 742 cycles to each system
+call (syscall entry and exit are traced and memory locations for both
+global variables lie on the same cache line).
+
+Therefore, not only is it interesting to use the immediate values to
+dynamically activate dormant code such as the markers, but I think it
+should also be considered as a replacement for many of the "read mostly"
+static variables.

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 4/8] Immediate Value - Add kconfig menus

[Alexey Dobriyan]

On Fri, Jul 13, 2007 at 09:24:43PM -0400, Mathieu Desnoyers wrote:
> Immediate values provide a way to use dynamic code patching to update variables
> sitting within the instruction stream. It saves caches lines normally used by
> static read mostly variables.

>  kernel/Kconfig.immediate |   11 +++++++++++

NAK, if this immediate stuff is so super-cool¹ it should be default on.

¹ it isn't

Re: [patch 4/8] Immediate Value - Add kconfig menus

[Mathieu Desnoyers]

* Alexey Dobriyan (adobriyan@gmail.com) wrote:
> On Fri, Jul 13, 2007 at 09:24:43PM -0400, Mathieu Desnoyers wrote:
> > Immediate values provide a way to use dynamic code patching to update variables
> > sitting within the instruction stream. It saves caches lines normally used by
> > static read mostly variables.
> 
> >  kernel/Kconfig.immediate |   11 +++++++++++
> 
> NAK, if this immediate stuff is so super-cool¹ it should be default on.
> 
> ¹ it isn't
> 

Hi Alexey,

On embedded systems, the tradeoff is not the same. The immediate values
trade a little bit of system memory (to keep the pointers to the
variable and instruction as well as the size of the variable, only used
when the variable is updated) in order to remove cache line hot paths.

Also, embedded systems with physically read-only memory clearly does not
want to enable this.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 2/8] Immediate Value - Architecture Independent Code

[Christoph Hellwig]

On Fri, Jul 13, 2007 at 09:24:41PM -0400, Mathieu Desnoyers wrote:
> +#ifdef __KERNEL__

no need for this.  unless you add the header to header-y in the Kbuild
file it's not exported to userspace at all.

> +#ifdef CONFIG_IMMEDIATE
> +#include <asm/immediate.h>
> +#else
> +#include <asm-generic/immediate.h>
> +#endif

Nack on this one. linux/*.h should never include asm-generic headers.

Either put the !CONFIG_IMMEDIATE code directly into this file or let
every arch have a one-liner immediate.h that includes the asm-generic
one.  The first method is probably a lot nicer.

> +/*
> + * modules_mutex nests inside immediate_mutex. immediate_mutex protects builtin
> + * immediates and module immediates.
> + */
> +DEFINE_MUTEX(immediate_mutex);

Why is this non-static?

> +
> +/*
> + * Sets a range of immediates to a enabled state : set the enable bit.
> + */
> +static void _immediate_update_range(
> +	const struct __immediate *begin, const struct __immediate *end)

static void _immediate_update_range(const struct __immediate *begin,
		const struct __immediate *end)

same for a few more functions here.

> +#ifdef CONFIG_MODULES

...

> +#endif
> +
> +#ifdef CONFIG_MODULES

please put all this code into a single ifdef block.


note that the exported functions probably want some kerneldoc documentation.

Re: [patch 5/8] Immediate Values - kprobe header fix

[Christoph Hellwig]

On Fri, Jul 13, 2007 at 09:24:44PM -0400, Mathieu Desnoyers wrote:
> Since the immediate values depend on the same int3 handler as kprobes implements
> for i386, we have to get architecture specific defines available for the kprobes
> trap handler (especially restore_interrupts()) wven when CONFIG_KPROBES is not
> selected.
> 
> That kind of ifdef around a whole header does not make sense in the first place
> anyway.
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> CC: prasanna@in.ibm.com
> CC: ananth@in.ibm.com
> CC: anil.s.keshavamurthy@intel.com
> CC: davem@davemloft.net
> ---
>  include/linux/kprobes.h |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> Index: linux-2.6-lttng/include/linux/kprobes.h
> ===================================================================
> --- linux-2.6-lttng.orig/include/linux/kprobes.h	2007-07-13 18:47:57.000000000 -0400
> +++ linux-2.6-lttng/include/linux/kprobes.h	2007-07-13 18:48:45.000000000 -0400
> @@ -36,9 +36,9 @@
>  #include <linux/spinlock.h>
>  #include <linux/rcupdate.h>
>  
> -#ifdef CONFIG_KPROBES
>  #include <asm/kprobes.h>

but this one isn't available on architectures that don't have kprobes,
so you can't include linux/kprobes.h in generic code without ifdef
protection anymore.

I'd say move the code to asm/kdebug.h if it's need by more than just
kprobes.

[PATCH] Immediate Values - Architecture Independent Code - Fixes following HCH comments

[Mathieu Desnoyers]

Immediate Values - Architecture Independent Code - Fixes following HCH comments

Various coding style fixes following Christoph's comments
Put asm-generic/immediate.h content into linux/immediate.h.
Remove ifdef __KERNEL__ in .h never exported to user space.
Make immediate_mutex static.

It applies after "Immediate Values - Architecture Independent Code".

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Christoph Hellwig <hch@infradead.org>
---
 include/linux/immediate.h |   18 ++++++++++++++----
 kernel/immediate.c        |   15 +++++++--------
 2 files changed, 21 insertions(+), 12 deletions(-)

Index: linux-2.6-lttng/include/linux/immediate.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/immediate.h	2007-07-14 18:13:58.000000000 -0400
+++ linux-2.6-lttng/include/linux/immediate.h	2007-07-14 18:16:26.000000000 -0400
@@ -10,12 +10,23 @@
  * See the file COPYING for more details.
  */
 
-#ifdef __KERNEL__
-
 #ifdef CONFIG_IMMEDIATE
 #include <asm/immediate.h>
 #else
-#include <asm-generic/immediate.h>
+/*
+ * Generic immediate values: a simple, standard, memory load.
+ */
+
+struct module;
+
+#define immediate_read(var)		_immediate_read(var)
+#define immediate_set(var, i)		((var)->value = (i))
+#define _immediate_set(var, i)		immediate_set(var, i)
+#define immediate_set_early(var, i)	immediate_set(var, i)
+#define immediate_if(var)		if (immediate_read(var))
+
+static inline void module_immediate_setup(struct module *mod) { }
+static inline void immediate_update_early(void) { }
 #endif
 
 /*
@@ -49,5 +60,4 @@ DEFINE_IMMEDIATE_TYPE(void*, immediate_v
  */
 #define _immediate_if(var)		if (_immediate_read(var))
 
-#endif /* __KERNEL__ */
 #endif
Index: linux-2.6-lttng/kernel/immediate.c
===================================================================
--- linux-2.6-lttng.orig/kernel/immediate.c	2007-07-14 18:13:58.000000000 -0400
+++ linux-2.6-lttng/kernel/immediate.c	2007-07-14 18:27:03.000000000 -0400
@@ -27,13 +27,13 @@ extern const struct __immediate __stop__
  * modules_mutex nests inside immediate_mutex. immediate_mutex protects builtin
  * immediates and module immediates.
  */
-DEFINE_MUTEX(immediate_mutex);
+static DEFINE_MUTEX(immediate_mutex);
 
 /*
  * Sets a range of immediates to a enabled state : set the enable bit.
  */
-static void _immediate_update_range(
-	const struct __immediate *begin, const struct __immediate *end)
+static inline void _immediate_update_range(const struct __immediate *begin,
+		const struct __immediate *end)
 {
 	const struct __immediate *iter;
 	int ret;
@@ -63,11 +63,10 @@ static void _immediate_update_range(
  */
 void module_immediate_setup(struct module *mod)
 {
-	_immediate_update_range(mod->immediate, mod->immediate+mod->num_immediate);
+	_immediate_update_range(mod->immediate,
+				mod->immediate+mod->num_immediate);
 }
-#endif
 
-#ifdef CONFIG_MODULES
 /*
  * immediate mutex nests inside the modules mutex.
  */
@@ -103,8 +102,8 @@ EXPORT_SYMBOL_GPL(immediate_update);
  * Update the immediate values to the state of the variables they refer to. It
  * is done before SMP is active, at the very beginning of start_kernel().
  */
-void __init immediate_update_early_range(
-	const struct __immediate *begin, const struct __immediate *end)
+void __init immediate_update_early_range(const struct __immediate *begin,
+		const struct __immediate *end)
 {
 	const struct __immediate *iter;
 
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 3/8] Immediate Values - Non Optimized Architectures

[Mathieu Desnoyers]

Should be dropped, following Christoph's advice.

* Mathieu Desnoyers (mathieu.desnoyers@polymtl.ca) wrote:
> Architecture agnostic, generic, version of the immediate values. It uses a
> global variable to mimic the immediate values.
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> ---
>  include/asm-alpha/immediate.h     |    1 +
>  include/asm-arm/immediate.h       |    1 +
>  include/asm-arm26/immediate.h     |    1 +
>  include/asm-blackfin/immediate.h  |    1 +
>  include/asm-cris/immediate.h      |    1 +
>  include/asm-frv/immediate.h       |    1 +
>  include/asm-generic/immediate.h   |   19 +++++++++++++++++++
>  include/asm-h8300/immediate.h     |    1 +
>  include/asm-i386/immediate.h      |    1 +
>  include/asm-ia64/immediate.h      |    1 +
>  include/asm-m32r/immediate.h      |    1 +
>  include/asm-m68k/immediate.h      |    1 +
>  include/asm-m68knommu/immediate.h |    1 +
>  include/asm-mips/immediate.h      |    1 +
>  include/asm-parisc/immediate.h    |    1 +
>  include/asm-powerpc/immediate.h   |    1 +
>  include/asm-ppc/immediate.h       |    1 +
>  include/asm-s390/immediate.h      |    1 +
>  include/asm-sh/immediate.h        |    1 +
>  include/asm-sh64/immediate.h      |    1 +
>  include/asm-sparc/immediate.h     |    1 +
>  include/asm-sparc64/immediate.h   |    1 +
>  include/asm-um/immediate.h        |    1 +
>  include/asm-v850/immediate.h      |    1 +
>  include/asm-x86_64/immediate.h    |    1 +
>  include/asm-xtensa/immediate.h    |    1 +
>  26 files changed, 44 insertions(+)
> 
> Index: linux-2.6-lttng/include/asm-generic/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-generic/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1,19 @@
> +#ifndef _ASM_GENERIC_IMMEDIATE_H
> +#define _ASM_GENERIC_IMMEDIATE_H
> +
> +/*
> + * Generic immediate values: a simple, standard, memory load.
> + */
> +
> +struct module;
> +
> +#define immediate_read(var)		_immediate_read(var)
> +#define immediate_set(var, i)		((var)->value = (i))
> +#define _immediate_set(var, i)		immediate_set(var, i)
> +#define immediate_set_early(var, i)	immediate_set(var, i)
> +#define immediate_if(var)		if (immediate_read(var))
> +
> +static inline void module_immediate_setup(struct module *mod) { }
> +static inline void immediate_update_early(void) { }
> +
> +#endif /* _ASM_GENERIC_IMMEDIATE_H */
> Index: linux-2.6-lttng/include/asm-alpha/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-alpha/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-arm/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-arm/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-arm26/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-arm26/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-cris/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-cris/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-frv/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-frv/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-h8300/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-h8300/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-ia64/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-ia64/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-m32r/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-m32r/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-m68k/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-m68k/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-m68knommu/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-m68knommu/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-mips/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-mips/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-parisc/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-parisc/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-ppc/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-ppc/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-s390/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-s390/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-sh/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-sh/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-sh64/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-sh64/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-sparc/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-sparc/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-sparc64/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-sparc64/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-um/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-um/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-v850/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-v850/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-x86_64/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-x86_64/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-xtensa/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-xtensa/immediate.h	2007-07-13 19:30:52.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-i386/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-i386/immediate.h	2007-07-13 20:02:08.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-powerpc/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-powerpc/immediate.h	2007-07-13 20:02:08.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> Index: linux-2.6-lttng/include/asm-blackfin/immediate.h
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/include/asm-blackfin/immediate.h	2007-07-13 20:02:35.000000000 -0400
> @@ -0,0 +1 @@
> +#include <asm-generic/immediate.h>
> 
> -- 
> Mathieu Desnoyers
> Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 5/8] Immediate Values - kprobe header fix

[Mathieu Desnoyers]

Should be dropped, following Christoph's advice.

* Mathieu Desnoyers (mathieu.desnoyers@polymtl.ca) wrote:
> Since the immediate values depend on the same int3 handler as kprobes implements
> for i386, we have to get architecture specific defines available for the kprobes
> trap handler (especially restore_interrupts()) wven when CONFIG_KPROBES is not
> selected.
> 
> That kind of ifdef around a whole header does not make sense in the first place
> anyway.
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> CC: prasanna@in.ibm.com
> CC: ananth@in.ibm.com
> CC: anil.s.keshavamurthy@intel.com
> CC: davem@davemloft.net
> ---
>  include/linux/kprobes.h |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> Index: linux-2.6-lttng/include/linux/kprobes.h
> ===================================================================
> --- linux-2.6-lttng.orig/include/linux/kprobes.h	2007-07-13 18:47:57.000000000 -0400
> +++ linux-2.6-lttng/include/linux/kprobes.h	2007-07-13 18:48:45.000000000 -0400
> @@ -36,9 +36,9 @@
>  #include <linux/spinlock.h>
>  #include <linux/rcupdate.h>
>  
> -#ifdef CONFIG_KPROBES
>  #include <asm/kprobes.h>
>  
> +#ifdef CONFIG_KPROBES
>  /* kprobe_status settings */
>  #define KPROBE_HIT_ACTIVE	0x00000001
>  #define KPROBE_HIT_SS		0x00000002
> 
> -- 
> Mathieu Desnoyers
> Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - Move Kprobes i386 restore_interrupt to kdebug.h

[Mathieu Desnoyers]

Immediate Values - Move Kprobes i386 restore_interrupt to kdebug.h

Since the breakpoint handler is useful both to kprobes and immediate values, it
makes sense to make the required restore_interrupt() available through
asm-i386/kdebug.h.

This patch replaces "immediate-values-kprobes-headers.patch".

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Christoph Hellwig <hch@infradead.org>
CC: prasanna@in.ibm.com
CC: ananth@in.ibm.com
CC: anil.s.keshavamurthy@intel.com
CC: davem@davemloft.net
---
 include/asm-i386/kdebug.h  |   10 ++++++++++
 include/asm-i386/kprobes.h |    9 ---------
 2 files changed, 10 insertions(+), 9 deletions(-)

Index: linux-2.6-lttng/include/asm-i386/kdebug.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-i386/kdebug.h	2007-07-14 18:32:56.000000000 -0400
+++ linux-2.6-lttng/include/asm-i386/kdebug.h	2007-07-14 18:37:32.000000000 -0400
@@ -6,6 +6,7 @@
  * from x86_64 architecture.
  */
 #include <linux/notifier.h>
+#include <asm/ptrace.h>
 
 struct pt_regs;
 
@@ -31,4 +32,13 @@ enum die_val {
 	DIE_PAGE_FAULT_NO_CONTEXT,
 };
 
+/* trap3/1 are intr gates for kprobes.  So, restore the status of IF,
+ * if necessary, before executing the original int3/1 (trap) handler.
+ */
+static inline void restore_interrupts(struct pt_regs *regs)
+{
+	if (regs->eflags & IF_MASK)
+		local_irq_enable();
+}
+
 #endif
Index: linux-2.6-lttng/include/asm-i386/kprobes.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-i386/kprobes.h	2007-07-14 18:32:52.000000000 -0400
+++ linux-2.6-lttng/include/asm-i386/kprobes.h	2007-07-14 18:33:59.000000000 -0400
@@ -80,15 +80,6 @@ struct kprobe_ctlblk {
 	struct prev_kprobe prev_kprobe;
 };
 
-/* trap3/1 are intr gates for kprobes.  So, restore the status of IF,
- * if necessary, before executing the original int3/1 (trap) handler.
- */
-static inline void restore_interrupts(struct pt_regs *regs)
-{
-	if (regs->eflags & IF_MASK)
-		local_irq_enable();
-}
-
 extern int kprobe_exceptions_notify(struct notifier_block *self,
 				    unsigned long val, void *data);
 extern int kernel_text_is_ro;
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - Pre Fix i386 Optimization

[Mathieu Desnoyers]

Immediate Values - Pre Fix i386 Optimization

Since I removed the immediate-values-non-optimized-architectures.patch, we need
to redo it to apply immediate-values-i386-optimization.patch correctly.

It should be applied _before_ the
immediate-values-i386-optimization.patch and folded with it.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Christoph Hellwig <hch@infradead.org>
---
 include/asm-i386/immediate.h |    1 +
 1 file changed, 1 insertion(+)

Index: linux-2.6-lttng/include/asm-i386/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-i386/immediate.h	2007-07-14 18:20:49.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - Pre fix powerpc Optimization

[Mathieu Desnoyers]

Immediate Values - Pre fix powerpc Optimization

Since I removed the immediate-values-non-optimized-architectures.patch, we need
to redo it to apply immediate-values-powerpc-optimization.patch correctly.

It applies _before_ immediate-values-powerpc-optimization.patch and can
be folded with it.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Christoph Hellwig <hch@infradead.org>
---
 include/asm-powerpc/immediate.h |    1 +
 1 file changed, 1 insertion(+)

Index: linux-2.6-lttng/include/asm-powerpc/immediate.h
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-lttng/include/asm-powerpc/immediate.h	2007-07-14 18:21:07.000000000 -0400
@@ -0,0 +1 @@
+#include <asm-generic/immediate.h>
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Value - Architecture Independent Code Deferred Sync

[Mathieu Desnoyers]

Linux Kernel Markers - Architecture Independent Code Deferred Sync

Upon marker probe_unregister, we delay call to synchronize_sched() to
accelerate mass unregistration (only when there is no more reference to a
give module do we call synchronize_sched()). However, we need to make sure
every critical region have ended before we re-arm a marker that has been
unregistered and then registered back with a different probe data.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Christoph Hellwig <hch@infradead.org>
---
 kernel/marker.c |   22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

Index: linux-2.6-lttng/kernel/marker.c
===================================================================
--- linux-2.6-lttng.orig/kernel/marker.c	2007-07-14 19:38:11.000000000 -0400
+++ linux-2.6-lttng/kernel/marker.c	2007-07-14 19:46:25.000000000 -0400
@@ -30,11 +30,21 @@ extern struct __mark_marker __stop___mar
 
 /*
  * module_mutex nests inside markers_mutex. Markers mutex protects the builtin
- * and module markers, and the hash table.
+ * and module markers, the hash table and deferred_sync.
  */
 DEFINE_MUTEX(markers_mutex);
 
 /*
+ * Marker deferred synchronization.
+ * Upon marker probe_unregister, we delay call to synchronize_sched() to
+ * accelerate mass unregistration (only when there is no more reference to a
+ * give module do we call synchronize_sched()). However, we need to make sure
+ * every critical region have ended before we re-arm a marker that has been
+ * unregistered and then registered back with a different probe data.
+ */
+static int deferred_sync;
+
+/*
  * Marker hash table, containing the active markers.
  * Protected by module_mutex.
  */
@@ -303,8 +313,10 @@ static inline void __marker_update_probe
 			__stop___markers, probe_module, &refcount);
 	/* Markers in modules. */
 	__marker_update_probes_modules(probe_module, &refcount);
-	if (probe_module && refcount == 0)
+	if (probe_module && refcount == 0) {
 		synchronize_sched();
+		deferred_sync = 0;
+	}
 }
 
 #ifdef CONFIG_MODULES
@@ -353,6 +365,10 @@ int marker_probe_register(const char *na
 		ret = -EBUSY;
 		goto end;
 	}
+	if (deferred_sync) {
+		synchronize_sched();
+		deferred_sync = 0;
+	}
 	ret = _add_marker(name, format, probe, pdata);
 	if (ret)
 		goto end;
@@ -384,6 +400,7 @@ void *marker_probe_unregister(const char
 	/* In what module is the probe handler ? */
 	probe_module = __module_text_address((unsigned long)entry->probe);
 	pdata = _remove_marker(name);
+	deferred_sync = 1;
 	_marker_update_probes(probe_module);
 end:
 	mutex_unlock(&markers_mutex);
@@ -425,6 +442,7 @@ iter_end:
 	/* In what module is the probe handler ? */
 	probe_module = __module_text_address((unsigned long)entry->probe);
 	pdata = _remove_marker(entry->name);
+	deferred_sync = 1;
 	_marker_update_probes(probe_module);
 end:
 	mutex_unlock(&markers_mutex);
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [PATCH] Immediate Value - Architecture Independent Code Deferred Sync

[Mathieu Desnoyers]

Please drop, wrong thread. Will repost in the right one.

* Mathieu Desnoyers (mathieu.desnoyers@polymtl.ca) wrote:
> Linux Kernel Markers - Architecture Independent Code Deferred Sync
> 
> Upon marker probe_unregister, we delay call to synchronize_sched() to
> accelerate mass unregistration (only when there is no more reference to a
> give module do we call synchronize_sched()). However, we need to make sure
> every critical region have ended before we re-arm a marker that has been
> unregistered and then registered back with a different probe data.
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> CC: Christoph Hellwig <hch@infradead.org>
> ---
>  kernel/marker.c |   22 ++++++++++++++++++++--
>  1 file changed, 20 insertions(+), 2 deletions(-)
> 
> Index: linux-2.6-lttng/kernel/marker.c
> ===================================================================
> --- linux-2.6-lttng.orig/kernel/marker.c	2007-07-14 19:38:11.000000000 -0400
> +++ linux-2.6-lttng/kernel/marker.c	2007-07-14 19:46:25.000000000 -0400
> @@ -30,11 +30,21 @@ extern struct __mark_marker __stop___mar
>  
>  /*
>   * module_mutex nests inside markers_mutex. Markers mutex protects the builtin
> - * and module markers, and the hash table.
> + * and module markers, the hash table and deferred_sync.
>   */
>  DEFINE_MUTEX(markers_mutex);
>  
>  /*
> + * Marker deferred synchronization.
> + * Upon marker probe_unregister, we delay call to synchronize_sched() to
> + * accelerate mass unregistration (only when there is no more reference to a
> + * give module do we call synchronize_sched()). However, we need to make sure
> + * every critical region have ended before we re-arm a marker that has been
> + * unregistered and then registered back with a different probe data.
> + */
> +static int deferred_sync;
> +
> +/*
>   * Marker hash table, containing the active markers.
>   * Protected by module_mutex.
>   */
> @@ -303,8 +313,10 @@ static inline void __marker_update_probe
>  			__stop___markers, probe_module, &refcount);
>  	/* Markers in modules. */
>  	__marker_update_probes_modules(probe_module, &refcount);
> -	if (probe_module && refcount == 0)
> +	if (probe_module && refcount == 0) {
>  		synchronize_sched();
> +		deferred_sync = 0;
> +	}
>  }
>  
>  #ifdef CONFIG_MODULES
> @@ -353,6 +365,10 @@ int marker_probe_register(const char *na
>  		ret = -EBUSY;
>  		goto end;
>  	}
> +	if (deferred_sync) {
> +		synchronize_sched();
> +		deferred_sync = 0;
> +	}
>  	ret = _add_marker(name, format, probe, pdata);
>  	if (ret)
>  		goto end;
> @@ -384,6 +400,7 @@ void *marker_probe_unregister(const char
>  	/* In what module is the probe handler ? */
>  	probe_module = __module_text_address((unsigned long)entry->probe);
>  	pdata = _remove_marker(name);
> +	deferred_sync = 1;
>  	_marker_update_probes(probe_module);
>  end:
>  	mutex_unlock(&markers_mutex);
> @@ -425,6 +442,7 @@ iter_end:
>  	/* In what module is the probe handler ? */
>  	probe_module = __module_text_address((unsigned long)entry->probe);
>  	pdata = _remove_marker(entry->name);
> +	deferred_sync = 1;
>  	_marker_update_probes(probe_module);
>  end:
>  	mutex_unlock(&markers_mutex);
> -- 
> Mathieu Desnoyers
> Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - Architecture Independent Code - kerneldoc

[Mathieu Desnoyers]

Immediate Values - Architecture Independent Code - kerneldoc

Add kerneldoc to Immediate Values API.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: hch@infradead.org
---
 include/linux/immediate.h |   66 ++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 61 insertions(+), 5 deletions(-)

Index: linux-2.6-lttng/include/linux/immediate.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/immediate.h	2007-07-14 20:27:40.000000000 -0400
+++ linux-2.6-lttng/include/linux/immediate.h	2007-07-14 20:40:40.000000000 -0400
@@ -19,18 +19,64 @@
 
 struct module;
 
+/**
+ * immediate_read - read immediate variable
+ * @var: pointer of type immediate_*_t
+ *
+ * Reads the value of @var.
+ */
 #define immediate_read(var)		_immediate_read(var)
+
+/**
+ * immediate_set - set immediate variable (with locking)
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var, taking the module_mutex if required by
+ * the architecture.
+ */
 #define immediate_set(var, i)		((var)->value = (i))
+
+/**
+ * _immediate_set - set immediate variable (without locking)
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var. Must be called with module_mutex held.
+ */
 #define _immediate_set(var, i)		immediate_set(var, i)
+
+/**
+ * immediate_set_early - set immediate variable at early boot
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var. Should be used for early boot updates.
+ */
 #define immediate_set_early(var, i)	immediate_set(var, i)
+
+/**
+ * immediate_if - if () statement depending on an immediate value
+ * @var: pointer of type immediate_*_t
+ *
+ * Use as an if () statement depending on an immediate value.
+ */
 #define immediate_if(var)		if (immediate_read(var))
 
+/*
+ * Internal update functions.
+ */
 static inline void module_immediate_setup(struct module *mod) { }
 static inline void immediate_update_early(void) { }
 #endif
 
-/*
- * Always access this type with the provided functions.
+/**
+ * DEFINE_IMMEDIATE_TYPE - Define an immediate type
+ * @type: type that the immediate should hold
+ * @name: name of the immediate type
+ *
+ * Define new immediate types. Naming scheme is immediate_*_t.
+ * Always access these types with the provided functions.
  */
 #define DEFINE_IMMEDIATE_TYPE(type, name) \
 	typedef struct { type value; } name
@@ -44,18 +90,28 @@ DEFINE_IMMEDIATE_TYPE(int, immediate_int
 DEFINE_IMMEDIATE_TYPE(long, immediate_long_t);
 DEFINE_IMMEDIATE_TYPE(void*, immediate_void_ptr_t);
 
-/*
- * Use this macro to initialize an immediate value to an initial static value.
+/**
+ * IMMEDIATE_INIT - Static initialization of an immediate variable
+ * @i: required value
+ *
+ * Use this macro to initialize an immediate value to an initial static
+ * value.
  */
 #define IMMEDIATE_INIT(i)		{ (i) }
 
-/*
+/**
+ * _immediate_read - Read immediate value with standard memory load.
+ * @var: pointer of type immediate_*_t
+ *
  * Force a data read of the immediate value instead of the immediate value
  * based mechanism. Useful for __init and __exit section data read.
  */
 #define _immediate_read(var)		(var)->value
 
 /*
+ * _immediate_if - if () statement depending on immediate value (memory load)
+ * @var: pointer of type immediate_*_t
+ *
  * Force the use of a normal if () statement depending on an immediate value.
  */
 #define _immediate_if(var)		if (_immediate_read(var))
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - i386 Optimization - kerneldoc

[Mathieu Desnoyers]

Immediate Values - i386 Optimization - kerneldoc

Add kerneldoc to Immediate Values (i386 Optimization) API.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: hch@infradead.org
---
 include/asm-i386/immediate.h |   50 ++++++++++++++++++++++++++++++-------------
 1 file changed, 35 insertions(+), 15 deletions(-)

Index: linux-2.6-lttng/include/asm-i386/immediate.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-i386/immediate.h	2007-07-14 20:54:23.000000000 -0400
+++ linux-2.6-lttng/include/asm-i386/immediate.h	2007-07-14 20:54:25.000000000 -0400
@@ -23,13 +23,16 @@ struct __immediate {
 	long size;		/* Type size. */
 };
 
-/*
+/**
+ * immediate_read - read immediate variable
+ * @var: pointer of type immediate_*_t
+ *
+ * Reads the value of @var.
  * Optimized version of the immediate.
- * Make sure the 2 and 4 bytes update will be atomic by aligning the immediate
- * value.
- * 2 bytes (short) uses a 66H prefix.
- * If size is bigger than 4 bytes, fall back on a memory read.
  * Do not use in __init and __exit functions. Use _immediate_read() instead.
+ * Makes sure the 2 and 4 bytes update will be atomic by aligning the immediate
+ * value. 2 bytes (short) uses a 66H prefix. If size is bigger than 4 bytes,
+ * fall back on a memory read.
  */
 #define immediate_read(var)						\
 	({								\
@@ -75,39 +78,56 @@ struct __immediate {
 		value;							\
 	})
 
-/*
- * Update immediate value, can take module mutex.
+
+/**
+ * immediate_set - set immediate variable (with locking)
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var, taking the module_mutex if required by
+ * the architecture.
  */
 #define immediate_set(var, i) \
 	(var)->value = (i); \
 	immediate_update(1);
 
-/*
- * Update immediate value. Module mutex must already be taken.
+/**
+ * _immediate_set - set immediate variable (without locking)
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var. Must be called with module_mutex held.
  */
 #define _immediate_set(var, i) \
 	(var)->value = (i); \
 	immediate_update(0);
 
-/*
- * Update immediate value at early boot.
+/**
+ * immediate_set_early - set immediate variable at early boot
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var. Should be used for early boot updates.
  */
 #define immediate_set_early(var, i) \
 	(var)->value = (i); \
 	immediate_update_early();
 
-/*
+/**
+ * immediate_if - if () statement depending on an immediate value
+ * @var: pointer of type immediate_*_t
+ *
+ * Use as an if () statement depending on an immediate value.
+ * Do not use in __init and __exit functions. Use _immediate_if() instead.
  * Branch depending on an immediate value. Could eventually be optimized further
  * by improving gcc to give the ability to patch a jump instruction instead of
  * the value it depends on.
- * Do not use in __init and __exit functions. Use _immediate_if() instead.
  */
 #define immediate_if(var)	if (unlikely(immediate_read(var)))
 
 /*
- * Used internally.
+ * Internal update functions.
  */
-
 extern void immediate_update(int lock);
 extern void module_immediate_setup(struct module *mod);
 extern void immediate_update_early(void);
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - PowerPC Optimization - kerneldoc

[Mathieu Desnoyers]

Immediate Values - PowerPC Optimization - kerneldoc

Add kerneldoc to Immediate Values (PowerPC Optimization) API.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: hch@infradead.org
---
 include/asm-powerpc/immediate.h |   51 ++++++++++++++++++++++++++++------------
 1 file changed, 36 insertions(+), 15 deletions(-)

Index: linux-2.6-lttng/include/asm-powerpc/immediate.h
===================================================================
--- linux-2.6-lttng.orig/include/asm-powerpc/immediate.h	2007-07-14 20:54:47.000000000 -0400
+++ linux-2.6-lttng/include/asm-powerpc/immediate.h	2007-07-14 20:56:50.000000000 -0400
@@ -24,14 +24,18 @@ struct __immediate {
 	long size;		/* Type size. */
 };
 
-/*
+/**
+ * immediate_read - read immediate variable
+ * @var: pointer of type immediate_*_t
+ *
+ * Reads the value of @var.
  * Optimized version of the immediate.
- * Make sure the 2 bytes update will be atomic by aligning the immediate
- * value.
- * Use a normal memory read for the 4 bytes immediate because there is no way to
- * atomically update it without using a seqlock read side, which would cost more
- * in term of total i-cache and d-cache space than a simple memory read.
  * Do not use in __init and __exit functions. Use _immediate_read() instead.
+ * Makes sure the 2 bytes update will be atomic by aligning the immediate
+ * value. Use a normal memory read for the 4 bytes immediate because there is no
+ * way to atomically update it without using a seqlock read side, which would
+ * cost more in term of total i-cache and d-cache space than a simple memory
+ * read.
  */
 #define immediate_read(var)						\
 	({								\
@@ -65,37 +69,54 @@ struct __immediate {
 		value;							\
 	})
 
-/*
- * Update immediate value, can take module mutex.
+/**
+ * immediate_set - set immediate variable (with locking)
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var, taking the module_mutex if required by
+ * the architecture.
  */
 #define immediate_set(var, i) \
 	(var)->value = (i); \
 	immediate_update(1);
 
-/*
- * Update immediate value. Module mutex must already be taken.
+/**
+ * _immediate_set - set immediate variable (without locking)
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var. Must be called with module_mutex held.
  */
 #define _immediate_set(var, i) \
 	(var)->value = (i); \
 	immediate_update(0);
 
-/*
- * Update immediate value at early boot.
+/**
+ * immediate_set_early - set immediate variable at early boot
+ * @var: pointer of type immediate_*_t
+ * @i: required value
+ *
+ * Sets the value of @var. Should be used for early boot updates.
  */
 #define immediate_set_early(var, i) \
 	(var)->value = (i); \
 	immediate_update_early();
 
-/*
+/**
+ * immediate_if - if () statement depending on an immediate value
+ * @var: pointer of type immediate_*_t
+ *
+ * Use as an if () statement depending on an immediate value.
+ * Do not use in __init and __exit functions. Use _immediate_if() instead.
  * Branch depending on an immediate value. Could eventually be optimized further
  * by improving gcc to give the ability to patch a jump instruction instead of
  * the value it depends on.
- * Do not use in __init and __exit functions. Use _immediate_if() instead.
  */
 #define immediate_if(var)	if (unlikely(immediate_read(var)))
 
 /*
- * Used internally.
+ * Internal update functions.
  */
 extern void immediate_update(int lock);
 extern void module_immediate_setup(struct module *mod);
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - Architecture Independent Code - kerneldoc for implementation

[Mathieu Desnoyers]

Immediate Values - Architecture Independent Code - kerneldoc for implementation

Add kerneldoc to Immediate Values API.
It applies after
"Immediate Values - Architecture Independent Code - kerneldoc"

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: hch@infradead.org
---
 kernel/immediate.c |   24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

Index: linux-2.6-lttng/kernel/immediate.c
===================================================================
--- linux-2.6-lttng.orig/kernel/immediate.c	2007-07-15 19:04:08.000000000 -0400
+++ linux-2.6-lttng/kernel/immediate.c	2007-07-15 19:07:14.000000000 -0400
@@ -54,7 +54,10 @@ static inline void _immediate_update_ran
 }
 
 #ifdef CONFIG_MODULES
-/*
+/**
+ * module_immediate_setup - Update immediate values in a module
+ * @mod: pointer to the struct module
+ *
  * Setup the immediate according to the variable upon which it depends.  Called
  * by load_module with module_mutex held. This mutex protects against concurrent
  * modifications to modules'immediates. Therefore, since
@@ -89,6 +92,12 @@ static inline void immediate_update_modu
 static inline void immediate_update_modules(int lock) { }
 #endif
 
+/**
+ * immediate_update - update all immediate values in the kernel
+ * @lock: should a module_mutex be taken ?
+ *
+ * Iterate on the kernel core and modules to update the immediate values.
+ */
 void immediate_update(int lock)
 {
 	/* Core kernel immediates */
@@ -98,11 +107,7 @@ void immediate_update(int lock)
 }
 EXPORT_SYMBOL_GPL(immediate_update);
 
-/*
- * Update the immediate values to the state of the variables they refer to. It
- * is done before SMP is active, at the very beginning of start_kernel().
- */
-void __init immediate_update_early_range(const struct __immediate *begin,
+static void __init immediate_update_early_range(const struct __immediate *begin,
 		const struct __immediate *end)
 {
 	const struct __immediate *iter;
@@ -111,7 +116,12 @@ void __init immediate_update_early_range
 		arch_immediate_update_early(iter);
 }
 
-
+/**
+ * immediate_update_early - Update immediate values at boot time
+ *
+ * Update the immediate values to the state of the variables they refer to. It
+ * is done before SMP is active, at the very beginning of start_kernel().
+ */
 void __init immediate_update_early(void)
 {
 	immediate_update_early_range(__start___immediate, __stop___immediate);
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - i386 Optimization - kerneldoc for implementation

[Mathieu Desnoyers]

Immediate Values - i386 Optimization - kerneldoc for implementation

Add kerneldoc to Immediate Values (i386 Optimization) API.
It applies after
"Immediate Values - i386 Optimization - kerneldoc"

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: hch@infradead.org
---
 arch/i386/kernel/immediate.c |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

Index: linux-2.6-lttng/arch/i386/kernel/immediate.c
===================================================================
--- linux-2.6-lttng.orig/arch/i386/kernel/immediate.c	2007-07-15 19:08:03.000000000 -0400
+++ linux-2.6-lttng/arch/i386/kernel/immediate.c	2007-07-15 19:09:56.000000000 -0400
@@ -170,8 +170,12 @@ static struct notifier_block immediate_n
 	.priority = 0x7fffffff,	/* we need to be notified first */
 };
 
-/*
- * Must be called with immediate_mutex held.
+
+/**
+ * arch_immediate_update - update one immediate value
+ * @immediate: pointer of type const struct __immediate to update
+ *
+ * Update one immediate value. Must be called with immediate_mutex held.
  */
 __kprobes int arch_immediate_update(const struct __immediate *immediate)
 {
@@ -266,8 +270,11 @@ __kprobes int arch_immediate_update(cons
 	return 0;
 }
 
-/*
- * Very early initialization of the in-core immediate values.
+/**
+ * arch_immediate_update_early - update one immediate value at boot time
+ * @immediate: pointer of type const struct __immediate to update
+ *
+ * Update one immediate value at boot time.
  */
 void __init arch_immediate_update_early(const struct __immediate *immediate)
 {
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[PATCH] Immediate Values - PowerPC Optimization - kerneldoc for implementation

[Mathieu Desnoyers]

Immediate Values - PowerPC Optimization - kerneldoc for implementation

Add kerneldoc to Immediate Values (PowerPC Optimization) API.
It applies after
"Immediate Values - PowerPC Optimization - kerneldoc"

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: hch@infradead.org
---
 arch/powerpc/kernel/immediate.c |   15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

Index: linux-2.6-lttng/arch/powerpc/kernel/immediate.c
===================================================================
--- linux-2.6-lttng.orig/arch/powerpc/kernel/immediate.c	2007-07-15 19:10:32.000000000 -0400
+++ linux-2.6-lttng/arch/powerpc/kernel/immediate.c	2007-07-15 19:11:50.000000000 -0400
@@ -13,8 +13,11 @@
 
 #define LI_OPCODE_LEN	2
 
-/*
- * The immediate value are aligned.
+/**
+ * arch_immediate_update - update one immediate value
+ * @immediate: pointer of type const struct __immediate to update
+ *
+ * Update one immediate value. Must be called with immediate_mutex held.
  */
 int arch_immediate_update(const struct __immediate *immediate)
 {
@@ -68,8 +71,11 @@ int arch_immediate_update(const struct _
 	return 0;
 }
 
-/*
- * Very early initialization of the in-core immediate values.
+/**
+ * arch_immediate_update_early - update one immediate value at boot time
+ * @immediate: pointer of type const struct __immediate to update
+ *
+ * Update one immediate value at boot time.
  * We can use flush_icache_range, since the cpu identification has been done in
  * the early_init stage.
  */
@@ -95,4 +101,3 @@ void __init arch_immediate_update_early(
 	flush_icache_range((unsigned long)immediate->immediate,
 				immediate->size);
 }
-
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 4/8] Immediate Value - Add kconfig menus

[Mathieu Desnoyers]

Should be dropped,
replaced by :
immediate-values-kconfig-embedded.patch

* Mathieu Desnoyers (mathieu.desnoyers@polymtl.ca) wrote:
> Immediate values provide a way to use dynamic code patching to update variables
> sitting within the instruction stream. It saves caches lines normally used by
> static read mostly variables.
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
> CC: Adrian Bunk <bunk@stusta.de>
> CC: Andi Kleen <andi@firstfloor.org>
> ---
> 
>  arch/alpha/Kconfig       |    6 ++++++
>  arch/arm/Kconfig         |    6 ++++++
>  arch/arm26/Kconfig       |    6 ++++++
>  arch/avr32/Kconfig.debug |    7 +++++++
>  arch/cris/Kconfig        |    6 ++++++
>  arch/frv/Kconfig         |    6 ++++++
>  arch/h8300/Kconfig       |    6 ++++++
>  arch/i386/Kconfig        |    2 ++
>  arch/ia64/Kconfig        |    3 +++
>  arch/m32r/Kconfig        |    6 ++++++
>  arch/m68k/Kconfig        |    6 ++++++
>  arch/m68knommu/Kconfig   |    6 ++++++
>  arch/mips/Kconfig        |    6 ++++++
>  arch/parisc/Kconfig      |    6 ++++++
>  arch/powerpc/Kconfig     |    3 +++
>  arch/ppc/Kconfig         |    6 ++++++
>  arch/s390/Kconfig        |    2 ++
>  arch/sh/Kconfig          |    6 ++++++
>  arch/sh64/Kconfig        |    6 ++++++
>  arch/sparc/Kconfig       |    2 ++
>  arch/sparc64/Kconfig     |    3 +++
>  arch/um/Kconfig          |    6 ++++++
>  arch/v850/Kconfig        |    6 ++++++
>  arch/x86_64/Kconfig      |    3 +++
>  arch/xtensa/Kconfig      |    6 ++++++
>  kernel/Kconfig.immediate |   11 +++++++++++
>  26 files changed, 138 insertions(+)
> 
> Index: linux-2.6-lttng/arch/alpha/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/alpha/Kconfig	2007-07-13 14:17:59.000000000 -0400
> +++ linux-2.6-lttng/arch/alpha/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -653,6 +653,12 @@
>  
>  source "arch/alpha/oprofile/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/alpha/Kconfig.debug"
>  
>  # DUMMY_CONSOLE may be defined in drivers/video/console/Kconfig
> Index: linux-2.6-lttng/arch/arm/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/arm/Kconfig	2007-07-13 14:17:59.000000000 -0400
> +++ linux-2.6-lttng/arch/arm/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -1046,6 +1046,12 @@
>  
>  source "arch/arm/oprofile/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/arm/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/arm26/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/arm26/Kconfig	2007-07-13 14:17:59.000000000 -0400
> +++ linux-2.6-lttng/arch/arm26/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -244,6 +244,12 @@
>  
>  source "drivers/usb/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/arm26/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/cris/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/cris/Kconfig	2007-07-13 14:17:59.000000000 -0400
> +++ linux-2.6-lttng/arch/cris/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -198,6 +198,12 @@
>  
>  source "drivers/usb/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/cris/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/frv/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/frv/Kconfig	2007-07-13 14:17:59.000000000 -0400
> +++ linux-2.6-lttng/arch/frv/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -375,6 +375,12 @@
>  
>  source "fs/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/frv/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/h8300/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/h8300/Kconfig	2007-07-13 14:17:59.000000000 -0400
> +++ linux-2.6-lttng/arch/h8300/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -223,6 +223,12 @@
>  
>  source "fs/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/h8300/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/i386/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/i386/Kconfig	2007-07-13 14:17:59.000000000 -0400
> +++ linux-2.6-lttng/arch/i386/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -1249,6 +1249,8 @@
>  	  for kernel debugging, non-intrusive instrumentation and testing.
>  	  If in doubt, say "N".
>  
> +source "kernel/Kconfig.immediate"
> +
>  endif # INSTRUMENTATION
>  
>  source "arch/i386/Kconfig.debug"
> Index: linux-2.6-lttng/arch/ia64/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/ia64/Kconfig	2007-07-13 14:17:59.000000000 -0400
> +++ linux-2.6-lttng/arch/ia64/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -593,6 +593,9 @@
>  	  a probepoint and specifies the callback.  Kprobes is useful
>  	  for kernel debugging, non-intrusive instrumentation and testing.
>  	  If in doubt, say "N".
> +
> +source "kernel/Kconfig.immediate"
> +
>  endmenu
>  
>  source "arch/ia64/Kconfig.debug"
> Index: linux-2.6-lttng/arch/m32r/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/m32r/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/m32r/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -408,6 +408,12 @@
>  
>  source "arch/m32r/oprofile/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/m32r/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/m68k/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/m68k/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/m68k/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -676,6 +676,12 @@
>  
>  source "fs/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/m68k/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/m68knommu/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/m68knommu/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/m68knommu/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -668,6 +668,12 @@
>  
>  source "fs/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/m68knommu/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/mips/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/mips/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/mips/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -1946,6 +1946,12 @@
>  
>  source "arch/mips/oprofile/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/mips/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/parisc/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/parisc/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/parisc/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -269,6 +269,12 @@
>  
>  source "arch/parisc/oprofile/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/parisc/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/powerpc/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/powerpc/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/powerpc/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -655,6 +655,9 @@
>  	  a probepoint and specifies the callback.  Kprobes is useful
>  	  for kernel debugging, non-intrusive instrumentation and testing.
>  	  If in doubt, say "N".
> +
> +source "kernel/Kconfig.immediate"
> +
>  endmenu
>  
>  source "arch/powerpc/Kconfig.debug"
> Index: linux-2.6-lttng/arch/ppc/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/ppc/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/ppc/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -1451,8 +1451,14 @@
>  
>  source "lib/Kconfig"
>  
> +menu "Instrumentation Support"
> +
>  source "arch/powerpc/oprofile/Kconfig"
>  
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/ppc/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/s390/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/s390/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/s390/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -547,6 +547,8 @@
>  	  for kernel debugging, non-intrusive instrumentation and testing.
>  	  If in doubt, say "N".
>  
> +source "kernel/Kconfig.immediate"
> +
>  endmenu
>  
>  source "arch/s390/Kconfig.debug"
> Index: linux-2.6-lttng/arch/sh/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/sh/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/sh/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -721,6 +721,12 @@
>  
>  source "arch/sh/oprofile/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/sh/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/sh64/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/sh64/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/sh64/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -284,6 +284,12 @@
>  
>  source "arch/sh64/oprofile/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/sh64/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/sparc/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/sparc/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/sparc/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -313,6 +313,8 @@
>  
>  source "arch/sparc/oprofile/Kconfig"
>  
> +source "kernel/Kconfig.immediate"
> +
>  endmenu
>  
>  source "arch/sparc/Kconfig.debug"
> Index: linux-2.6-lttng/arch/sparc64/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/sparc64/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/sparc64/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -451,6 +451,9 @@
>  	  a probepoint and specifies the callback.  Kprobes is useful
>  	  for kernel debugging, non-intrusive instrumentation and testing.
>  	  If in doubt, say "N".
> +
> +source "kernel/Kconfig.immediate"
> +
>  endmenu
>  
>  source "arch/sparc64/Kconfig.debug"
> Index: linux-2.6-lttng/arch/um/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/um/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/um/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -335,4 +335,10 @@
>  	bool
>  	default n
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/um/Kconfig.debug"
> Index: linux-2.6-lttng/arch/v850/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/v850/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/v850/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -331,6 +331,12 @@
>  
>  source "drivers/usb/Kconfig"
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/v850/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/arch/x86_64/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/x86_64/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/x86_64/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -842,6 +842,9 @@
>  	  a probepoint and specifies the callback.  Kprobes is useful
>  	  for kernel debugging, non-intrusive instrumentation and testing.
>  	  If in doubt, say "N".
> +
> +source "kernel/Kconfig.immediate"
> +
>  endmenu
>  
>  source "arch/x86_64/Kconfig.debug"
> Index: linux-2.6-lttng/arch/xtensa/Kconfig
> ===================================================================
> --- linux-2.6-lttng.orig/arch/xtensa/Kconfig	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/xtensa/Kconfig	2007-07-13 14:33:51.000000000 -0400
> @@ -251,6 +251,12 @@
>  	  provide one yourself.
>  endmenu
>  
> +menu "Instrumentation Support"
> +
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  source "arch/xtensa/Kconfig.debug"
>  
>  source "security/Kconfig"
> Index: linux-2.6-lttng/kernel/Kconfig.immediate
> ===================================================================
> --- /dev/null	1970-01-01 00:00:00.000000000 +0000
> +++ linux-2.6-lttng/kernel/Kconfig.immediate	2007-07-13 14:33:47.000000000 -0400
> @@ -0,0 +1,11 @@
> +# Immediate values configuration
> +
> +config IMMEDIATE
> +	bool "Enable immediate values"
> +	depends on X86_32 || PPC
> +	default y if !EMBEDDED
> +	help
> +	  Immediate values are used as read mostly variables that are rarely
> +	  updated. They use code patching to modify the values inscribed in the
> +	  instruction stream. It provides a way to save precious cache lines
> +	  that would otherwise have to be used by these variables.
> Index: linux-2.6-lttng/arch/avr32/Kconfig.debug
> ===================================================================
> --- linux-2.6-lttng.orig/arch/avr32/Kconfig.debug	2007-07-13 14:18:00.000000000 -0400
> +++ linux-2.6-lttng/arch/avr32/Kconfig.debug	2007-07-13 14:33:51.000000000 -0400
> @@ -6,6 +6,9 @@
>  
>  source "lib/Kconfig.debug"
>  
> +menu "Instrumentation Support"
> +	depends on EXPERIMENTAL
> +
>  config KPROBES
>  	bool "Kprobes"
>  	depends on DEBUG_KERNEL
> @@ -16,4 +19,8 @@
>            for kernel debugging, non-intrusive instrumentation and testing.
>            If in doubt, say "N".
>  
> +source "kernel/Kconfig.immediate"
> +
> +endmenu
> +
>  endmenu
> 
> -- 
> Mathieu Desnoyers
> Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 4/8] Immediate Value - Add kconfig menus

[Andi Kleen]

> On embedded systems, the tradeoff is not the same. The immediate values
> trade a little bit of system memory (to keep the pointers to the
> variable and instruction as well as the size of the variable, only used
> when the variable is updated) in order to remove cache line hot paths.

Please remove the Kconfig. I don't think it makes sense. Such optimizations
should be always enabled. We don't have CONFIG_GO_FASTER configs normally.
Don't introduce them now.

> 
> Also, embedded systems with physically read-only memory clearly does not
> want to enable this.

We always patch the x86 kernel, so they have to deal with it anyways.
The x86 port doesn't support XIP.

-Andi

[PATCH] Immediate Value - Kconfig menu in EMBEDDED

[Mathieu Desnoyers]

Immediate Values - Kconfig menu in EMBEDDED

Immediate values provide a way to use dynamic code patching to update variables
sitting within the instruction stream. It saves caches lines normally used by
static read mostly variables. Enable it by default, but let users disable it
through the EMBEDDED menu with the "Disable immediate values" submenu entry.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Adrian Bunk <bunk@stusta.de>
CC: Andi Kleen <andi@firstfloor.org>
CC: Alexey Dobriyan <adobriyan@gmail.com>
CC: Christoph Hellwig <hch@infradead.org>
---
 init/Kconfig |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

Index: linux-2.6-lttng/init/Kconfig
===================================================================
--- linux-2.6-lttng.orig/init/Kconfig	2007-07-15 20:09:53.000000000 -0400
+++ linux-2.6-lttng/init/Kconfig	2007-07-15 20:23:16.000000000 -0400
@@ -410,6 +410,17 @@ config CC_OPTIMIZE_FOR_SIZE
 config SYSCTL
 	bool
 
+config IMMEDIATE
+	default y if !DISABLE_IMMEDIATE
+	depends on X86_32 || PPC
+	bool
+	help
+	  Immediate values are used as read mostly variables that are rarely
+	  updated. They use code patching to modify the values inscribed in the
+	  instruction stream. It provides a way to save precious cache lines
+	  that would otherwise have to be used by these variables. Can be
+	  disabled through the EMBEDDED menu.
+
 menuconfig EMBEDDED
 	bool "Configure standard kernel features (for small systems)"
 	help
@@ -672,6 +683,16 @@ config PROC_KPAGEMAP
           information on page-level memory usage. Disabling this interface
           will reduce the size of the kernel for small machines.
 
+config DISABLE_IMMEDIATE
+	default y if EMBEDDED
+	bool "Disable immediate values" if EMBEDDED
+	depends on X86_32 || PPC
+	help
+	  Disable code patching based immediate values for embedded systems. It
+	  consumes slightly more memory and requires to modify the instruction
+	  stream each time a variable is updated. Should really be disabled for
+	  embedded systems with read-only text.
+
 endmenu		# General setup
 
 config RT_MUTEXES
-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 4/8] Immediate Value - Add kconfig menus

[Mathieu Desnoyers]

* Andi Kleen (andi@firstfloor.org) wrote:
> > On embedded systems, the tradeoff is not the same. The immediate values
> > trade a little bit of system memory (to keep the pointers to the
> > variable and instruction as well as the size of the variable, only used
> > when the variable is updated) in order to remove cache line hot paths.
> 
> Please remove the Kconfig. I don't think it makes sense. Such optimizations
> should be always enabled. We don't have CONFIG_GO_FASTER configs normally.
> Don't introduce them now.
> 
> > 
> > Also, embedded systems with physically read-only memory clearly does not
> > want to enable this.
> 
> We always patch the x86 kernel, so they have to deal with it anyways.
> The x86 port doesn't support XIP.
> 
> -Andi

Hehe, good timing, please see the
immediate-values-kconfig-embedded.patch I just sent. ;) Well, the idea
is to give the option to every architecture in the embedded menu, which
is not limited to X86.

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-global-modules-list-and-mutex.patch --]
[-- Type: text/plain, Size: 2184 bytes --]

Remove "static" from module_mutex and the modules list so it can be used by
other builtin objects in the kernel. Otherwise, every code depending on the
module list would have to be put in kernel/module.c. Since the immediate values
depends on the module list but can be considered as logically different, it
makes sense to implement them in their own file.

The alternative to this would be to disable preemption in code path that need
such synchronization, so they can be protected against module unload by
stop_machine(), but not being able to sleep within while needing such
synchronization is limiting.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 include/linux/module.h |    4 ++++
 kernel/module.c        |    4 ++--
 2 files changed, 6 insertions(+), 2 deletions(-)

Index: linux-2.6-lttng/kernel/module.c
===================================================================
--- linux-2.6-lttng.orig/kernel/module.c	2007-08-07 11:03:56.000000000 -0400
+++ linux-2.6-lttng/kernel/module.c	2007-08-07 11:40:22.000000000 -0400
@@ -64,8 +64,8 @@ extern int module_sysfs_initialized;
 
 /* List of modules, protected by module_mutex or preempt_disable
  * (add/delete uses stop_machine). */
-static DEFINE_MUTEX(module_mutex);
-static LIST_HEAD(modules);
+DEFINE_MUTEX(module_mutex);
+LIST_HEAD(modules);
 static DECLARE_MUTEX(notify_mutex);
 
 static BLOCKING_NOTIFIER_HEAD(module_notify_list);
Index: linux-2.6-lttng/include/linux/module.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/module.h	2007-08-07 11:03:48.000000000 -0400
+++ linux-2.6-lttng/include/linux/module.h	2007-08-07 11:39:55.000000000 -0400
@@ -60,6 +60,10 @@ struct module_kobject
 	struct kobject *drivers_dir;
 };
 
+/* Protects the list of modules. */
+extern struct mutex module_mutex;
+extern struct list_head modules;
+
 /* These are either module local, or the kernel's dummy ones. */
 extern int init_module(void);
 extern void cleanup_module(void);

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-global-modules-list-and-mutex.patch --]
[-- Type: text/plain, Size: 2184 bytes --]

Remove "static" from module_mutex and the modules list so it can be used by
other builtin objects in the kernel. Otherwise, every code depending on the
module list would have to be put in kernel/module.c. Since the immediate values
depends on the module list but can be considered as logically different, it
makes sense to implement them in their own file.

The alternative to this would be to disable preemption in code path that need
such synchronization, so they can be protected against module unload by
stop_machine(), but not being able to sleep within while needing such
synchronization is limiting.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 include/linux/module.h |    4 ++++
 kernel/module.c        |    4 ++--
 2 files changed, 6 insertions(+), 2 deletions(-)

Index: linux-2.6-lttng/kernel/module.c
===================================================================
--- linux-2.6-lttng.orig/kernel/module.c	2007-08-07 11:03:56.000000000 -0400
+++ linux-2.6-lttng/kernel/module.c	2007-08-07 11:40:22.000000000 -0400
@@ -64,8 +64,8 @@ extern int module_sysfs_initialized;
 
 /* List of modules, protected by module_mutex or preempt_disable
  * (add/delete uses stop_machine). */
-static DEFINE_MUTEX(module_mutex);
-static LIST_HEAD(modules);
+DEFINE_MUTEX(module_mutex);
+LIST_HEAD(modules);
 static DECLARE_MUTEX(notify_mutex);
 
 static BLOCKING_NOTIFIER_HEAD(module_notify_list);
Index: linux-2.6-lttng/include/linux/module.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/module.h	2007-08-07 11:03:48.000000000 -0400
+++ linux-2.6-lttng/include/linux/module.h	2007-08-07 11:39:55.000000000 -0400
@@ -60,6 +60,10 @@ struct module_kobject
 	struct kobject *drivers_dir;
 };
 
+/* Protects the list of modules. */
+extern struct mutex module_mutex;
+extern struct list_head modules;
+
 /* These are either module local, or the kernel's dummy ones. */
 extern int init_module(void);
 extern void cleanup_module(void);

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-global-modules-list-and-mutex.patch --]
[-- Type: text/plain, Size: 2184 bytes --]

Remove "static" from module_mutex and the modules list so it can be used by
other builtin objects in the kernel. Otherwise, every code depending on the
module list would have to be put in kernel/module.c. Since the immediate values
depends on the module list but can be considered as logically different, it
makes sense to implement them in their own file.

The alternative to this would be to disable preemption in code path that need
such synchronization, so they can be protected against module unload by
stop_machine(), but not being able to sleep within while needing such
synchronization is limiting.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 include/linux/module.h |    4 ++++
 kernel/module.c        |    4 ++--
 2 files changed, 6 insertions(+), 2 deletions(-)

Index: linux-2.6-lttng/kernel/module.c
===================================================================
--- linux-2.6-lttng.orig/kernel/module.c	2007-08-07 11:03:56.000000000 -0400
+++ linux-2.6-lttng/kernel/module.c	2007-08-07 11:40:22.000000000 -0400
@@ -64,8 +64,8 @@ extern int module_sysfs_initialized;
 
 /* List of modules, protected by module_mutex or preempt_disable
  * (add/delete uses stop_machine). */
-static DEFINE_MUTEX(module_mutex);
-static LIST_HEAD(modules);
+DEFINE_MUTEX(module_mutex);
+LIST_HEAD(modules);
 static DECLARE_MUTEX(notify_mutex);
 
 static BLOCKING_NOTIFIER_HEAD(module_notify_list);
Index: linux-2.6-lttng/include/linux/module.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/module.h	2007-08-07 11:03:48.000000000 -0400
+++ linux-2.6-lttng/include/linux/module.h	2007-08-07 11:39:55.000000000 -0400
@@ -60,6 +60,10 @@ struct module_kobject
 	struct kobject *drivers_dir;
 };
 
+/* Protects the list of modules. */
+extern struct mutex module_mutex;
+extern struct list_head modules;
+
 /* These are either module local, or the kernel's dummy ones. */
 extern int init_module(void);
 extern void cleanup_module(void);

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

[patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

[-- Attachment #1: immediate-values-global-modules-list-and-mutex.patch --]
[-- Type: text/plain, Size: 2184 bytes --]

Remove "static" from module_mutex and the modules list so it can be used by
other builtin objects in the kernel. Otherwise, every code depending on the
module list would have to be put in kernel/module.c. Since the immediate values
depends on the module list but can be considered as logically different, it
makes sense to implement them in their own file.

The alternative to this would be to disable preemption in code path that need
such synchronization, so they can be protected against module unload by
stop_machine(), but not being able to sleep within while needing such
synchronization is limiting.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
---
 include/linux/module.h |    4 ++++
 kernel/module.c        |    4 ++--
 2 files changed, 6 insertions(+), 2 deletions(-)

Index: linux-2.6-lttng/kernel/module.c
===================================================================
--- linux-2.6-lttng.orig/kernel/module.c	2007-08-07 11:03:56.000000000 -0400
+++ linux-2.6-lttng/kernel/module.c	2007-08-07 11:40:22.000000000 -0400
@@ -64,8 +64,8 @@ extern int module_sysfs_initialized;
 
 /* List of modules, protected by module_mutex or preempt_disable
  * (add/delete uses stop_machine). */
-static DEFINE_MUTEX(module_mutex);
-static LIST_HEAD(modules);
+DEFINE_MUTEX(module_mutex);
+LIST_HEAD(modules);
 static DECLARE_MUTEX(notify_mutex);
 
 static BLOCKING_NOTIFIER_HEAD(module_notify_list);
Index: linux-2.6-lttng/include/linux/module.h
===================================================================
--- linux-2.6-lttng.orig/include/linux/module.h	2007-08-07 11:03:48.000000000 -0400
+++ linux-2.6-lttng/include/linux/module.h	2007-08-07 11:39:55.000000000 -0400
@@ -60,6 +60,10 @@ struct module_kobject
 	struct kobject *drivers_dir;
 };
 
+/* Protects the list of modules. */
+extern struct mutex module_mutex;
+extern struct list_head modules;
+
 /* These are either module local, or the kernel's dummy ones. */
 extern int init_module(void);
 extern void cleanup_module(void);

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Alexey Dobriyan]

On Thu, Sep 06, 2007 at 04:02:29PM -0400, Mathieu Desnoyers wrote:
> Remove "static" from module_mutex and the modules list so it can be used by
> other builtin objects in the kernel. Otherwise, every code depending on the
> module list would have to be put in kernel/module.c. Since the immediate values
> depends on the module list but can be considered as logically different, it
> makes sense to implement them in their own file.
> 
> The alternative to this would be to disable preemption in code path that need
> such synchronization, so they can be protected against module unload by
> stop_machine(), but not being able to sleep within while needing such
> synchronization is limiting.

> --- linux-2.6-lttng.orig/kernel/module.c
> +++ linux-2.6-lttng/kernel/module.c
> @@ -64,8 +64,8 @@ extern int module_sysfs_initialized;
>  
>  /* List of modules, protected by module_mutex or preempt_disable
>   * (add/delete uses stop_machine). */
> -static DEFINE_MUTEX(module_mutex);
> -static LIST_HEAD(modules);
> +DEFINE_MUTEX(module_mutex);
> +LIST_HEAD(modules);
>  static DECLARE_MUTEX(notify_mutex);
>  
>  static BLOCKING_NOTIFIER_HEAD(module_notify_list);
> --- linux-2.6-lttng.orig/include/linux/module.h
> +++ linux-2.6-lttng/include/linux/module.h
> @@ -60,6 +60,10 @@ struct module_kobject
>  	struct kobject *drivers_dir;
>  };
>  
> +/* Protects the list of modules. */
> +extern struct mutex module_mutex;
> +extern struct list_head modules;

Rusty, do you still want to keep module_mutex virgin? If not, I can
backout /proc/*/wchan vs rmmod race fix et al and use muuuch simpler version.

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Rusty Russell]

On Sat, 2007-09-08 at 11:28 +0400, Alexey Dobriyan wrote:
> On Thu, Sep 06, 2007 at 04:02:29PM -0400, Mathieu Desnoyers wrote:
> > Remove "static" from module_mutex and the modules list so it can be used by
> > other builtin objects in the kernel. Otherwise, every code depending on the
> > module list would have to be put in kernel/module.c. Since the immediate values
> > depends on the module list but can be considered as logically different, it
> > makes sense to implement them in their own file.

If I understand this code correctly, then changing immediate values
needs some exclusion to avoid patching live code.  You leave this to the
user with some very unclear rules.

The result is a real mess that has nothing to do with the module mutex
and list.  These patches need a lot more work 8(

1) The immediate types are just kind of silly.  See per-cpu for how it
handles this already.  DECLARE_IMMEDIATE(type, var) is probably enough.

2) immediate_if() needs an implementation before you introduce it.  Your
assumption that it's always unlikely seems non-orthogonal.

3) immediate_set(), _immediate_set() and immediate_set_early()?  No
thanks!  AFAICT you really want an "init_immediate(var, val)".  This
means "you can patch all the references now, they're not executing".
Later on we could possibly have a super-stop-machine version which
ensures noone's preempted and handles the concurrent case.  Maybe.

4) With an "init" interface not a "set" interface, you don't need
locking.  Simpler.

Hope that helps,
Rusty.

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

* Rusty Russell (rusty@rustcorp.com.au) wrote:
> On Sat, 2007-09-08 at 11:28 +0400, Alexey Dobriyan wrote:
> > On Thu, Sep 06, 2007 at 04:02:29PM -0400, Mathieu Desnoyers wrote:
> > > Remove "static" from module_mutex and the modules list so it can be used by
> > > other builtin objects in the kernel. Otherwise, every code depending on the
> > > module list would have to be put in kernel/module.c. Since the immediate values
> > > depends on the module list but can be considered as logically different, it
> > > makes sense to implement them in their own file.
> 
> If I understand this code correctly, then changing immediate values
> needs some exclusion to avoid patching live code.  You leave this to the
> user with some very unclear rules.
> 

I think you really misunderstood some major features of these patches
then. Doing an immediate_set on a variable has the same semantic as
setting a normal static variable. If the write to the equivalent
variable would be performed atomically (e.g. 32 bits or less variable on
a 32 bits architecture), it will also be done atomically on an
immediate value reference.

Code patching of _live_ SMP code is allowed. This is why I went through
all this trouble on i386.

So the locking is the same for an immediate value that it would be for a
static variable. If you are concerned about the fact that we have to
iterate on multiple load immediate sites to enable the variable (which
is done non atomically when there are multiple references), we end up in
a moment where references may be in a different state at a given time.
But this is no different from out of order read/writes of/from a static
variable and how older copies may be seen by another CPU : if you hope
that multiple CPUs will see a coherent copy of a static variable at a
given point in time, you clearly have to use the proper locking that
will order read/writes.

Immediate values are mostly meant to be used in contexts where we can
allow this time window where, during the update, some references will
have one value and others won't. Therefore, doing :

immediate_int_t var;
in function:
BUG_ON(immediate_read(&var) != immediate_read(&var));

might fail if it is executed while an immediate_set is done on var. But
if we would have:

volatile int var;
in function:
BUG_ON(var != var);

and, in another thread:
var++;

We _might_ race and see a different value. What is do different about
it?

When we activate a feature that is protected by a boolean, we currently
have to accept that we will, at some point, be in a state where code
paths will see the enabled value and others will see the disabled one.
(this is the choice we do in tracing, and the choice that has to be done
when using atomic updates without locking).

The only guarantee you get is that once the immediate_set is over, all
further references to the variable in memory will see the new variable.

> The result is a real mess that has nothing to do with the module mutex
> and list.  These patches need a lot more work 8(
> 

module mutex is only there to allow coherent iteration on the module
list so we can patch each module's code.

> 1) The immediate types are just kind of silly.  See per-cpu for how it
> handles this already.  DECLARE_IMMEDIATE(type, var) is probably enough.
> 

I would be ok with that.

> 2) immediate_if() needs an implementation before you introduce it.  Your
> assumption that it's always unlikely seems non-orthogonal.
> 

I could remove the unlikely(), no problem with that. Your point about
this is valid. However, I would like to leave the immediate_if() there
because it may become very useful if someday gcc permits to extract the
address of a branch instruction (and to generate assembly that would not
be reached without doing code patching).

Quoting my discussion with H. Peter Anvin:

<quote>
Mathieu Desnoyers wrote:
>
> If we can change the compiler, here is what we could do:
>
> Tell GCC to put NOPs that could be altered by a branch alternative to
> some specified code. We should be able to get the instruction pointers
> (think of inlines) to these nop/branch instructions so we can change
> them dynamically.
>

Changing the compiler should be perfectly feasible, *BUT* I think we
need a transitional solution that works on existing compilers.

> I suspect this would be inherently tricky. If someone is ready to do
> this and tells me "yes, it will be there in 1 month", I am more than
> ready to switch my markers to this and help, but since the core of my
> work is kernel tracing, I don't have the time nor the ressources to
> tackle this problem.
>
> In the event that someone answers "we'll do this in the following 3
> years", I might consider to change the if (immediate(var)) into an
> immediate_if (var) so we can later proceed to the change with simple
> ifdefs without rewriting all the kernel code that would use it.

This is much more of "we'll do that in the following 1-2 years", since
we have to deal with a full gcc development cycle.  However, I really
want to see this being implemented in a way that would let us DTRT in
the long run."
</quote>



> 3) immediate_set(), _immediate_set() and immediate_set_early()?  No
> thanks!  AFAICT you really want an "init_immediate(var, val)".  This
> means "you can patch all the references now, they're not executing".
> Later on we could possibly have a super-stop-machine version which
> ensures noone's preempted and handles the concurrent case.  Maybe.
> 
> 4) With an "init" interface not a "set" interface, you don't need
> locking.  Simpler.
> 

We don't need to stop execution of references at all. Not needed. (as
explained above). I wouldn't want that anyway, since I want to use this
for tracing which must be as non-intrusive as possible.

No locking, live update, is imho much simpler :)


> Hope that helps,
> Rusty.
> 

Thanks for the review,

Mathieu

> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Rusty Russell]

On Mon, 2007-09-10 at 20:45 -0400, Mathieu Desnoyers wrote:
> Code patching of _live_ SMP code is allowed. This is why I went through
> all this trouble on i386.

Oh, I was pretty sure it wasn't.  OK.

So now why three versions of immediate_set()?  And why are you using my
lock for exclusion?  Against what?

Why not just have one immediate_set() which iterates through and fixes
up all the references?  It can use an internal lock if you want to avoid
concurrent immediate_set() calls.

I understand the need for a "module_immediate_fixup()" but that can also
use your internal lock.

> > 2) immediate_if() needs an implementation before you introduce it.  Your
> > assumption that it's always unlikely seems non-orthogonal.
> 
> I could remove the unlikely(), no problem with that. Your point about
> this is valid. However, I would like to leave the immediate_if() there
> because it may become very useful if someday gcc permits to extract the
> address of a branch instruction (and to generate assembly that would not
> be reached without doing code patching).

Why is it easier to patch the sites now than later?  Currently it's just
churn.  You could go back and find them when this mythical patch gets
merged into this mythical future gcc version.  It could well need a
completely different macro style, like "cond_imm(var, code)".

Rusty.

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

* Rusty Russell (rusty@rustcorp.com.au) wrote:
> On Mon, 2007-09-10 at 20:45 -0400, Mathieu Desnoyers wrote:
> > Code patching of _live_ SMP code is allowed. This is why I went through
> > all this trouble on i386.
> 
> Oh, I was pretty sure it wasn't.  OK.
> 
> So now why three versions of immediate_set()?  And why are you using my
> lock for exclusion?  Against what?
> 

If we need to patch code at boot time, when interrupts are still
disabled (it happens when we parse the kernel arguments for instance),
we cannot afford to use IPIs to call sync_core() on each cpu, using
breakpoints/notifier chains could be tricky (because we are very early
at boot and alternatives or paravirt may not have been applied yet).

So, for early boot code patching, immediate_set_early() is used. It
presumes that variables are not used when the code is patched, that we
are in UP context and it is really minimalistic.

On the other hand, immediate_set() updates kernel and module variables
while the (potentially smp) system is running. It provides coherent
variable updates even if the code referencing then is executing.

_immediate_set() has been introduced because of the way immediate values
are used by markers: the linux kernel markers already hold the module
mutex when they need to update the immediate values. Taking the mutex
twice makes no sence, so _immediate_set() is used when the caller
already holds the module mutex.


> Why not just have one immediate_set() which iterates through and fixes
> up all the references?

(reasons explained above)

> It can use an internal lock if you want to avoid
> concurrent immediate_set() calls.
> 

An internal lock won't protect against modules load/unload race. We have
to iterate on the module list.

> I understand the need for a "module_immediate_fixup()" but that can also
> use your internal lock.

It looks interesting.. can you elaborate a little bit more on this idea?
If we can find a way to encapsulate in module.c everything that needs to
touch the module list, I am all for it.

> 
> > > 2) immediate_if() needs an implementation before you introduce it.  Your
> > > assumption that it's always unlikely seems non-orthogonal.
> > 
> > I could remove the unlikely(), no problem with that. Your point about
> > this is valid. However, I would like to leave the immediate_if() there
> > because it may become very useful if someday gcc permits to extract the
> > address of a branch instruction (and to generate assembly that would not
> > be reached without doing code patching).
> 
> Why is it easier to patch the sites now than later?  Currently it's just
> churn.  You could go back and find them when this mythical patch gets
> merged into this mythical future gcc version.  It could well need a
> completely different macro style, like "cond_imm(var, code)".
> 

Maybe you're right. My though was that if we have a way to express a
strictly boolean if() statement that can later be optimized further by
gcc using a jump rather than a conditionnal branch and currently emulate
it by using a load immediate/test/branch, we might want to do so right
now so we don't have to do a second code transition from
if (immediate_read(&var)) to immediate_if (&var) later. But you might be
right in that the form could potentially change anyway when the
implementation would come, although I don't see how.

Mathieu

> Rusty.
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Rusty Russell]

On Tue, 2007-09-11 at 10:27 -0400, Mathieu Desnoyers wrote: 
> * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > On Mon, 2007-09-10 at 20:45 -0400, Mathieu Desnoyers wrote:
> > > Code patching of _live_ SMP code is allowed. This is why I went through
> > > all this trouble on i386.
> > 
> > Oh, I was pretty sure it wasn't.  OK.
> > 
> > So now why three versions of immediate_set()?  And why are you using my
> > lock for exclusion?  Against what?
> > 
> 
> If we need to patch code at boot time, when interrupts are still
> disabled (it happens when we parse the kernel arguments for instance),
> we cannot afford to use IPIs to call sync_core() on each cpu, using
> breakpoints/notifier chains could be tricky (because we are very early
> at boot and alternatives or paravirt may not have been applied yet).

Hi Mathieu,

Sure, but why is that the caller's problem?  immediate_set() isn't
fastpath, so why not make it do an "if (early_boot)" internally?

> _immediate_set() has been introduced because of the way immediate values
> are used by markers: the linux kernel markers already hold the module
> mutex when they need to update the immediate values. Taking the mutex
> twice makes no sence, so _immediate_set() is used when the caller
> already holds the module mutex.

> Why not just have one immediate_set() which iterates through and fixes
> > up all the references?
> 
> (reasons explained above)
> 
> > It can use an internal lock if you want to avoid
> > concurrent immediate_set() calls.
> > 
> 
> An internal lock won't protect against modules load/unload race. We have
> to iterate on the module list.

Sure, but it seems like that's fairly easy to do within module.c:

/* This updates all the immediates even though only one might have
* changed.  But it's so rare it's not worth optimizing. */
void module_update_immediates(void)
{
mutex_lock(&module_mutex);
list_for_each_entry(mod, &modules, list)
update_immediates(mod->immediate, mod->num_immediate);
mutex_unlock(&module_mutex);
}

Then during module load you do:

update_immediates(mod->immediate, mod->num_immediate);

Your immediate_update() just becomes:

update_immediates(__start___immediate,
		  __stop___immediate - __start___immediate);
module_update_immediates();

update_immediates() can grab the immediate_mutex if you want.

> > Why is it easier to patch the sites now than later?  Currently it's just
> > churn.  You could go back and find them when this mythical patch gets
> > merged into this mythical future gcc version.  It could well need a
> > completely different macro style, like "cond_imm(var, code)".
> 
> Maybe you're right. My though was that if we have a way to express a
> strictly boolean if() statement that can later be optimized further by
> gcc using a jump rather than a conditionnal branch and currently emulate
> it by using a load immediate/test/branch, we might want to do so right
> now so we don't have to do a second code transition from
> if (immediate_read(&var)) to immediate_if (&var) later. But you might be
> right in that the form could potentially change anyway when the
> implementation would come, although I don't see how.

I was thinking that we might find useful specific cases before we get
GCC support, which archs can override with tricky asm if they wish.

Cheers,
Rusty.

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

* Rusty Russell (rusty@rustcorp.com.au) wrote:
> On Tue, 2007-09-11 at 10:27 -0400, Mathieu Desnoyers wrote: 
> > * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > > On Mon, 2007-09-10 at 20:45 -0400, Mathieu Desnoyers wrote:
> > > > Code patching of _live_ SMP code is allowed. This is why I went through
> > > > all this trouble on i386.
> > > 
> > > Oh, I was pretty sure it wasn't.  OK.
> > > 
> > > So now why three versions of immediate_set()?  And why are you using my
> > > lock for exclusion?  Against what?
> > > 
> > 
> > If we need to patch code at boot time, when interrupts are still
> > disabled (it happens when we parse the kernel arguments for instance),
> > we cannot afford to use IPIs to call sync_core() on each cpu, using
> > breakpoints/notifier chains could be tricky (because we are very early
> > at boot and alternatives or paravirt may not have been applied yet).
> 
> Hi Mathieu,
> 
> Sure, but why is that the caller's problem?  immediate_set() isn't
> fastpath, so why not make it do an "if (early_boot)" internally?
> 

I see two reasons:
1 - early_boot, or anything that looks like this, does not exist
currently (and the following reason might show why).
2 - If we use this, we cannot declare the early code with __init, so it
will have to stay there forever insteaf of being removable once boot is
over.

Therefore, I think it's better to stick to an immediate_set_early
version.

> > _immediate_set() has been introduced because of the way immediate values
> > are used by markers: the linux kernel markers already hold the module
> > mutex when they need to update the immediate values. Taking the mutex
> > twice makes no sence, so _immediate_set() is used when the caller
> > already holds the module mutex.
> 
> > Why not just have one immediate_set() which iterates through and fixes
> > > up all the references?
> > 
> > (reasons explained above)
> > 
> > > It can use an internal lock if you want to avoid
> > > concurrent immediate_set() calls.
> > > 
> > 
> > An internal lock won't protect against modules load/unload race. We have
> > to iterate on the module list.
> 
> Sure, but it seems like that's fairly easy to do within module.c:
> 
> /* This updates all the immediates even though only one might have
> * changed.  But it's so rare it's not worth optimizing. */
> void module_update_immediates(void)
> {
> mutex_lock(&module_mutex);
> list_for_each_entry(mod, &modules, list)
> update_immediates(mod->immediate, mod->num_immediate);
> mutex_unlock(&module_mutex);
> }
> 
> Then during module load you do:
> 
> update_immediates(mod->immediate, mod->num_immediate);
> 
> Your immediate_update() just becomes:
> 
> update_immediates(__start___immediate,
> 		  __stop___immediate - __start___immediate);
> module_update_immediates();
> 
> update_immediates() can grab the immediate_mutex if you want.
> 

Yup, excellent idea. I just changed the linux kernel markers too.


> > > Why is it easier to patch the sites now than later?  Currently it's just
> > > churn.  You could go back and find them when this mythical patch gets
> > > merged into this mythical future gcc version.  It could well need a
> > > completely different macro style, like "cond_imm(var, code)".
> > 
> > Maybe you're right. My though was that if we have a way to express a
> > strictly boolean if() statement that can later be optimized further by
> > gcc using a jump rather than a conditionnal branch and currently emulate
> > it by using a load immediate/test/branch, we might want to do so right
> > now so we don't have to do a second code transition from
> > if (immediate_read(&var)) to immediate_if (&var) later. But you might be
> > right in that the form could potentially change anyway when the
> > implementation would come, although I don't see how.
> 
> I was thinking that we might find useful specific cases before we get
> GCC support, which archs can override with tricky asm if they wish.
> 

The first useful case is the Linux Kernel Markers, which really needs a
completely boolean if: active or inactive. That would be a good test
case to get gcc support.

Mathieu

> Cheers,
> Rusty.
> 
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Rusty Russell]

On Thu, 2007-09-13 at 17:21 -0400, Mathieu Desnoyers wrote:
> * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > Sure, but why is that the caller's problem?  immediate_set() isn't
> > fastpath, so why not make it do an "if (early_boot)" internally?
> 
> I see two reasons:
> 1 - early_boot, or anything that looks like this, does not exist
> currently (and the following reason might show why).
> 2 - If we use this, we cannot declare the early code with __init, so it
> will have to stay there forever insteaf of being removable once boot is
> over.
> 
> Therefore, I think it's better to stick to an immediate_set_early
> version.

My problem is that I don't know when to use immediate_set_early!  You
say "early boot": what does that mean?  Is it arch-specific?  And why is
it my problem anyway?

Since arch_immediate_update_early() is a memcpy, I don't really buy the
bloat argument (it has some silly-looking optimization, but that should
be removed anyway).  I'd really rather see arch_immediate_update()
examine an internal flag and do the memcpy if it's too early for the
full algorithm.  That's the only place which really knows, after all.
You may need some external hook to update that internal "too early"
flag, of course.

Alternatively, if you called it "immediate_init" then the semantics
change slightly, but are more obvious (ie. only use this when the value
isn't being accessed yet).  But it can't be __init then anyway.

On an unrelated note, did you consider simply IPI-ing and doing the
substitution with all CPUs stopped?  If you only updated the immediate
references to this particular var, it should be fast enough not to upset
the RT guys, even.

> > I was thinking that we might find useful specific cases before we get
> > GCC support, which archs can override with tricky asm if they wish.
> > 
> 
> The first useful case is the Linux Kernel Markers, which really needs a
> completely boolean if: active or inactive. That would be a good test
> case to get gcc support.

Well, you can do that in asm without gcc support.  It's a little nasty:
since gcc will know nothing about the function call, it can't have side
effects which are visible in this function, and you'll have to save and
restore *all* regs if you decide to do the function call.  But it's
possible (a 5-byte nop gets changed to a call, the call does the pushes
and sets the args regs, calls the function, then pops everything and
rets).

Cheers,
Rusty.

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

* Rusty Russell (rusty@rustcorp.com.au) wrote:
> On Thu, 2007-09-13 at 17:21 -0400, Mathieu Desnoyers wrote:
> > * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > > Sure, but why is that the caller's problem?  immediate_set() isn't
> > > fastpath, so why not make it do an "if (early_boot)" internally?
> > 
> > I see two reasons:
> > 1 - early_boot, or anything that looks like this, does not exist
> > currently (and the following reason might show why).
> > 2 - If we use this, we cannot declare the early code with __init, so it
> > will have to stay there forever insteaf of being removable once boot is
> > over.
> > 
> > Therefore, I think it's better to stick to an immediate_set_early
> > version.
> 
> My problem is that I don't know when to use immediate_set_early!  You
> say "early boot": what does that mean?  Is it arch-specific?  And why is
> it my problem anyway?
> 
> Since arch_immediate_update_early() is a memcpy, I don't really buy the
> bloat argument (it has some silly-looking optimization, but that should
> be removed anyway).  I'd really rather see arch_immediate_update()
> examine an internal flag and do the memcpy if it's too early for the
> full algorithm.  That's the only place which really knows, after all.
> You may need some external hook to update that internal "too early"
> flag, of course.
> 
> Alternatively, if you called it "immediate_init" then the semantics
> change slightly, but are more obvious (ie. only use this when the value
> isn't being accessed yet).  But it can't be __init then anyway.
> 

I think your idea is good. immediate_init() could be used to update the
immediate values at boot time _and_ at module load time, and we could
use an architecture specific arch_immediate_update_init() to support it.

As for "when" to use this, it should be used at boot time when
interrupts are still disabled, still running in UP. It can also be used
at module load time before any of the module code is executed, as long
as the module code pages are writable (which they always are, for
now..). Therefore, the flag seems inappropriate for module load
arch_immediate_update_init. It cannot be put in __init section neither
though if we use it like this.


> On an unrelated note, did you consider simply IPI-ing and doing the
> substitution with all CPUs stopped?  If you only updated the immediate
> references to this particular var, it should be fast enough not to upset
> the RT guys, even.
> 

Yes, I thought about this, but since I use immediate values in the
kernel markers, which can be put in exception handlers (including nmi,
mce handler), which cannot be disabled without important side-effects, I
don't think trying to stop the CPUs is a workable solution.


> > > I was thinking that we might find useful specific cases before we get
> > > GCC support, which archs can override with tricky asm if they wish.
> > > 
> > 
> > The first useful case is the Linux Kernel Markers, which really needs a
> > completely boolean if: active or inactive. That would be a good test
> > case to get gcc support.
> 
> Well, you can do that in asm without gcc support.  It's a little nasty:
> since gcc will know nothing about the function call, it can't have side
> effects which are visible in this function, and you'll have to save and
> restore *all* regs if you decide to do the function call.  But it's
> possible (a 5-byte nop gets changed to a call, the call does the pushes
> and sets the args regs, calls the function, then pops everything and
> rets).
> 

GCC support is required if we want to embed inline functions inside
unlikely branches depending on immediate values (no function call
there). It also permits passing local variables as arguments to the
function call (stack setup), which would be tricky, instrumentation site
specific and non portable if done in assembly.

Mathieu

> Cheers,
> Rusty.
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Rusty Russell]

On Fri, 2007-09-14 at 11:32 -0400, Mathieu Desnoyers wrote:
> * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > Alternatively, if you called it "immediate_init" then the semantics
> > change slightly, but are more obvious (ie. only use this when the value
> > isn't being accessed yet).  But it can't be __init then anyway.
> > 
> 
> I think your idea is good. immediate_init() could be used to update the
> immediate values at boot time _and_ at module load time, and we could
> use an architecture specific arch_immediate_update_init() to support it.

Right.

> As for "when" to use this, it should be used at boot time when
> interrupts are still disabled, still running in UP. It can also be used
> at module load time before any of the module code is executed, as long
> as the module code pages are writable (which they always are, for
> now..). Therefore, the flag seems inappropriate for module load
> arch_immediate_update_init. It cannot be put in __init section neither
> though if we use it like this.

I think from a user's POV it would be nice to have a 1:1 mapping with
normal initialization semantics (ie. it will work as long as you don't
access this value until initialized).  And I think this would be the
case.  eg:

        int foo_func(void)
        {
        	if (immediate_read(&some_immediate))
        		return 0;
        	...
        }
        
        int some_init(void)
        {
        	immediate_init(some_immediate, 0);
        	register_foo(foo_func);
        	...
        }


> > On an unrelated note, did you consider simply IPI-ing and doing the
> > substitution with all CPUs stopped?  If you only updated the immediate
> > references to this particular var, it should be fast enough not to upset
> > the RT guys, even.
> > 
> 
> Yes, I thought about this, but since I use immediate values in the
> kernel markers, which can be put in exception handlers (including nmi,
> mce handler), which cannot be disabled without important side-effects, I
> don't think trying to stop the CPUs is a workable solution.

OK, but can you justify the use of immediates within the nmi or mce
handlers?  They don't strike me as useful candidates for optimization.

> > Well, you can do that in asm without gcc support.  It's a little nasty:
> > since gcc will know nothing about the function call, it can't have side
> > effects which are visible in this function, and you'll have to save and
> > restore *all* regs if you decide to do the function call.  But it's
> > possible (a 5-byte nop gets changed to a call, the call does the pushes
> > and sets the args regs, calls the function, then pops everything and
> > rets).
> 
> GCC support is required if we want to embed inline functions inside
> unlikely branches depending on immediate values (no function call
> there). It also permits passing local variables as arguments to the
> function call (stack setup), which would be tricky, instrumentation site
> specific and non portable if done in assembly.

Well if this is the slow path, you don't want inline anyway.  But it
would be horribly, horribly arch-specific, yes.

Rusty.

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

* Rusty Russell (rusty@rustcorp.com.au) wrote:
> On Fri, 2007-09-14 at 11:32 -0400, Mathieu Desnoyers wrote:
> > * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > > Alternatively, if you called it "immediate_init" then the semantics
> > > change slightly, but are more obvious (ie. only use this when the value
> > > isn't being accessed yet).  But it can't be __init then anyway.
> > > 
> > 
> > I think your idea is good. immediate_init() could be used to update the
> > immediate values at boot time _and_ at module load time, and we could
> > use an architecture specific arch_immediate_update_init() to support it.
> 
> Right.
> 
> > As for "when" to use this, it should be used at boot time when
> > interrupts are still disabled, still running in UP. It can also be used
> > at module load time before any of the module code is executed, as long
> > as the module code pages are writable (which they always are, for
> > now..). Therefore, the flag seems inappropriate for module load
> > arch_immediate_update_init. It cannot be put in __init section neither
> > though if we use it like this.
> 
> I think from a user's POV it would be nice to have a 1:1 mapping with
> normal initialization semantics (ie. it will work as long as you don't
> access this value until initialized).  And I think this would be the
> case.  eg:
> 
>         int foo_func(void)
>         {
>         	if (immediate_read(&some_immediate))
>         		return 0;
>         	...
>         }
>         
>         int some_init(void)
>         {
>         	immediate_init(some_immediate, 0);
>         	register_foo(foo_func);
>         	...
>         }
> 

There are other considerations that differs between the boot-time case
and the general "init" case: the write-protection flag must be
cleared-saved/restored when the kernel is running to patch read-only
text, but we don't want to modify cr0 at early boot on i386 because
paravirt is not executed yet (at boot time, pages are not
write-protected yet).

And I am not sure that it buys us anything to create an immediate_init()
when we can do exactly the same job with immediate_set. Yes, it might be
a bit slower, but we are not on a fast path.


> 
> > > On an unrelated note, did you consider simply IPI-ing and doing the
> > > substitution with all CPUs stopped?  If you only updated the immediate
> > > references to this particular var, it should be fast enough not to upset
> > > the RT guys, even.
> > > 
> > 
> > Yes, I thought about this, but since I use immediate values in the
> > kernel markers, which can be put in exception handlers (including nmi,
> > mce handler), which cannot be disabled without important side-effects, I
> > don't think trying to stop the CPUs is a workable solution.
> 
> OK, but can you justify the use of immediates within the nmi or mce
> handlers?  They don't strike me as useful candidates for optimization.
> 

Yes, immediate values are used by the Linux Kernel Markers, which
instrument many code paths, including functions called from nmi and mce
contexts (including printk).

> > > Well, you can do that in asm without gcc support.  It's a little nasty:
> > > since gcc will know nothing about the function call, it can't have side
> > > effects which are visible in this function, and you'll have to save and
> > > restore *all* regs if you decide to do the function call.  But it's
> > > possible (a 5-byte nop gets changed to a call, the call does the pushes
> > > and sets the args regs, calls the function, then pops everything and
> > > rets).
> > 
> > GCC support is required if we want to embed inline functions inside
> > unlikely branches depending on immediate values (no function call
> > there). It also permits passing local variables as arguments to the
> > function call (stack setup), which would be tricky, instrumentation site
> > specific and non portable if done in assembly.
> 
> Well if this is the slow path, you don't want inline anyway.  But it
> would be horribly, horribly arch-specific, yes.
> 

Yes, doing arch specific calls without gcc support seems to be unlikely
to give us a neat portable solution.

Mathieu

> Rusty.
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Rusty Russell]

On Tue, 2007-09-18 at 09:41 -0400, Mathieu Desnoyers wrote:
> * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > On Fri, 2007-09-14 at 11:32 -0400, Mathieu Desnoyers wrote:
> > > * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > > > Alternatively, if you called it "immediate_init" then the semantics
> > > > change slightly, but are more obvious (ie. only use this when the value
> > > > isn't being accessed yet).  But it can't be __init then anyway.
> > > > 
> > > 
> > > I think your idea is good. immediate_init() could be used to update the
> > > immediate values at boot time _and_ at module load time, and we could
> > > use an architecture specific arch_immediate_update_init() to support it.
> > 
> > Right.
> > 
> > > As for "when" to use this, it should be used at boot time when
> > > interrupts are still disabled, still running in UP. It can also be used
> > > at module load time before any of the module code is executed, as long
> > > as the module code pages are writable (which they always are, for
> > > now..). Therefore, the flag seems inappropriate for module load
> > > arch_immediate_update_init. It cannot be put in __init section neither
> > > though if we use it like this.
> > 
> > I think from a user's POV it would be nice to have a 1:1 mapping with
> > normal initialization semantics (ie. it will work as long as you don't
> > access this value until initialized).  And I think this would be the
> > case.  eg:
> > 
> >         int foo_func(void)
> >         {
> >         	if (immediate_read(&some_immediate))
> >         		return 0;
> >         	...
> >         }
> >         
> >         int some_init(void)
> >         {
> >         	immediate_init(some_immediate, 0);
> >         	register_foo(foo_func);
> >         	...
> >         }
> > 
> 
> There are other considerations that differs between the boot-time case
> and the general "init" case: the write-protection flag must be
> cleared-saved/restored when the kernel is running to patch read-only
> text, but we don't want to modify cr0 at early boot on i386 because
> paravirt is not executed yet (at boot time, pages are not
> write-protected yet).
> 
> And I am not sure that it buys us anything to create an immediate_init()
> when we can do exactly the same job with immediate_set. Yes, it might be
> a bit slower, but we are not on a fast path.

Good points.  Well I'd say hiding it all behind a friendly
"immediate_set()" interface is the best option then.

> > OK, but can you justify the use of immediates within the nmi or mce
> > handlers?  They don't strike me as useful candidates for optimization.
> 
> Yes, immediate values are used by the Linux Kernel Markers, which
> instrument many code paths, including functions called from nmi and mce
> contexts (including printk).

Is this really worth worrying about?  Isn't there already a problem with
printk() in nmi?

Cheers,
Rusty.

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Mathieu Desnoyers]

* Rusty Russell (rusty@rustcorp.com.au) wrote:
> On Tue, 2007-09-18 at 09:41 -0400, Mathieu Desnoyers wrote:
> > * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > > On Fri, 2007-09-14 at 11:32 -0400, Mathieu Desnoyers wrote:
> > > > * Rusty Russell (rusty@rustcorp.com.au) wrote:
> > > > > Alternatively, if you called it "immediate_init" then the semantics
> > > > > change slightly, but are more obvious (ie. only use this when the value
> > > > > isn't being accessed yet).  But it can't be __init then anyway.
> > > > > 
> > > > 
> > > > I think your idea is good. immediate_init() could be used to update the
> > > > immediate values at boot time _and_ at module load time, and we could
> > > > use an architecture specific arch_immediate_update_init() to support it.
> > > 
> > > Right.
> > > 
> > > > As for "when" to use this, it should be used at boot time when
> > > > interrupts are still disabled, still running in UP. It can also be used
> > > > at module load time before any of the module code is executed, as long
> > > > as the module code pages are writable (which they always are, for
> > > > now..). Therefore, the flag seems inappropriate for module load
> > > > arch_immediate_update_init. It cannot be put in __init section neither
> > > > though if we use it like this.
> > > 
> > > I think from a user's POV it would be nice to have a 1:1 mapping with
> > > normal initialization semantics (ie. it will work as long as you don't
> > > access this value until initialized).  And I think this would be the
> > > case.  eg:
> > > 
> > >         int foo_func(void)
> > >         {
> > >         	if (immediate_read(&some_immediate))
> > >         		return 0;
> > >         	...
> > >         }
> > >         
> > >         int some_init(void)
> > >         {
> > >         	immediate_init(some_immediate, 0);
> > >         	register_foo(foo_func);
> > >         	...
> > >         }
> > > 
> > 
> > There are other considerations that differs between the boot-time case
> > and the general "init" case: the write-protection flag must be
> > cleared-saved/restored when the kernel is running to patch read-only
> > text, but we don't want to modify cr0 at early boot on i386 because
> > paravirt is not executed yet (at boot time, pages are not
> > write-protected yet).
> > 
> > And I am not sure that it buys us anything to create an immediate_init()
> > when we can do exactly the same job with immediate_set. Yes, it might be
> > a bit slower, but we are not on a fast path.
> 
> Good points.  Well I'd say hiding it all behind a friendly
> "immediate_set()" interface is the best option then.
> 

Then we can't benefit of the __init section to have the code removed
after boot. I don't see the point in doing so.

> > > OK, but can you justify the use of immediates within the nmi or mce
> > > handlers?  They don't strike me as useful candidates for optimization.
> > 
> > Yes, immediate values are used by the Linux Kernel Markers, which
> > instrument many code paths, including functions called from nmi and mce
> > contexts (including printk).
> 
> Is this really worth worrying about?  Isn't there already a problem with
> printk() in nmi?
> 

printk() is just an example taken from my current instrumentation. Let's
say I plan to instrument kmalloc() (which will happen someday): it's
used in every context, including NMI. And it's not because printk is
broken that its instrumentation can afford to be broken even further,
that logic seems wrong to me. Somebody go fix printk then ;)

Mathieu

> Cheers,
> Rusty.
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Re: [patch 1/8] Immediate Values - Global Modules List and Module Mutex

[Rusty Russell]

On Fri, 2007-09-21 at 09:37 -0400, Mathieu Desnoyers wrote:
> > Good points.  Well I'd say hiding it all behind a friendly
> > "immediate_set()" interface is the best option then.
> 
> Then we can't benefit of the __init section to have the code removed
> after boot. I don't see the point in doing so.

Because having a magic early version is a bad burden to place on
programmers.  It's an admission of failure that we cannot create a
simpler interface.

AFAICT it's a handful of bytes which would be freed.

> > Is this really worth worrying about?  Isn't there already a problem with
> > printk() in nmi?
> 
> printk() is just an example taken from my current instrumentation. Let's
> say I plan to instrument kmalloc() (which will happen someday): it's
> used in every context, including NMI.

Again, I don't think calling kmalloc in NMI is valid.  Unless I'm
missing something, any code which uses locks is susceptible to deadlock
if used from an NMI handler.   So you really can't do much.

It's nice that you have the perfect solution.  But I'd really rather see
a sufficient solution which is 1/4 of the code and 1/10 the complexity.

Rusty.