* [PATCH 1/4] ftrace: allow archs to preform pre and post process for code modification
2009-02-20 19:32 [PATCH 0/4] [git pull v2 @num@/@total@] updates for tip Steven Rostedt
@ 2009-02-20 19:32 ` Steven Rostedt
2009-02-20 19:32 ` [PATCH 2/4] ftrace, x86: make kernel text writable only for conversions Steven Rostedt
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Steven Rostedt @ 2009-02-20 19:32 UTC (permalink / raw)
To: linux-kernel
Cc: Ingo Molnar, Andrew Morton, Thomas Gleixner, Peter Zijlstra,
Frederic Weisbecker, Steven Rostedt
[-- Attachment #1: 0001-ftrace-allow-archs-to-preform-pre-and-post-process.patch --]
[-- Type: text/plain, Size: 2246 bytes --]
From: Steven Rostedt <srostedt@redhat.com>
This patch creates the weak functions: ftrace_arch_code_modify_prepare
and ftrace_arch_code_modify_post_process that are called before and
after the stop machine is called to modify the kernel text.
If the arch needs to do pre or post processing, it only needs to define
these functions.
[ Update: Ingo Molnar suggested using the name ftrace_arch_code_modify_*
over using ftrace_arch_modify_* ]
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
---
include/linux/ftrace.h | 3 +++
kernel/trace/ftrace.c | 28 ++++++++++++++++++++++++++++
2 files changed, 31 insertions(+), 0 deletions(-)
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 677432b..fdb2a89 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -99,6 +99,9 @@ stack_trace_sysctl(struct ctl_table *table, int write,
/* asm/ftrace.h must be defined for archs supporting dynamic ftrace */
#include <asm/ftrace.h>
+int ftrace_arch_code_modify_prepare(void);
+int ftrace_arch_code_modify_post_process(void);
+
enum {
FTRACE_FL_FREE = (1 << 0),
FTRACE_FL_FAILED = (1 << 1),
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index fdf913d..72316d9 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -585,6 +585,24 @@ ftrace_code_disable(struct module *mod, struct dyn_ftrace *rec)
return 1;
}
+/*
+ * archs can override this function if they must do something
+ * before the modifying code is performed.
+ */
+int __weak ftrace_arch_code_modify_prepare(void)
+{
+ return 0;
+}
+
+/*
+ * archs can override this function if they must do something
+ * after the modifying code is performed.
+ */
+int __weak ftrace_arch_code_modify_post_process(void)
+{
+ return 0;
+}
+
static int __ftrace_modify_code(void *data)
{
int *command = data;
@@ -607,7 +625,17 @@ static int __ftrace_modify_code(void *data)
static void ftrace_run_update_code(int command)
{
+ int ret;
+
+ ret = ftrace_arch_code_modify_prepare();
+ FTRACE_WARN_ON(ret);
+ if (ret)
+ return;
+
stop_machine(__ftrace_modify_code, &command, NULL);
+
+ ret = ftrace_arch_code_modify_post_process();
+ FTRACE_WARN_ON(ret);
}
static ftrace_func_t saved_ftrace_func;
--
1.5.6.5
--
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/4] ftrace, x86: make kernel text writable only for conversions
2009-02-20 19:32 [PATCH 0/4] [git pull v2 @num@/@total@] updates for tip Steven Rostedt
2009-02-20 19:32 ` [PATCH 1/4] ftrace: allow archs to preform pre and post process for code modification Steven Rostedt
@ 2009-02-20 19:32 ` Steven Rostedt
2009-02-20 19:32 ` [PATCH 3/4] ftrace: immediately stop code modification if failure is detected Steven Rostedt
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Steven Rostedt @ 2009-02-20 19:32 UTC (permalink / raw)
To: linux-kernel
Cc: Ingo Molnar, Andrew Morton, Thomas Gleixner, Peter Zijlstra,
Frederic Weisbecker, Steven Rostedt
[-- Attachment #1: 0002-ftrace-x86-make-kernel-text-writable-only-for-conv.patch --]
[-- Type: text/plain, Size: 5905 bytes --]
From: Steven Rostedt <srostedt@redhat.com>
Impact: keep kernel text read only
Because dynamic ftrace converts the calls to mcount into and out of
nops at run time, we needed to always keep the kernel text writable.
But this defeats the point of CONFIG_DEBUG_RODATA. This patch converts
the kernel code to writable before ftrace modifies the text, and converts
it back to read only afterward.
The kernel text is converted to read/write, stop_machine is called to
modify the code, then the kernel text is converted back to read only.
The original version used SYSTEM_STATE to determine when it was OK
or not to change the code to rw or ro. Andrew Morton pointed out that
using SYSTEM_STATE is a bad idea since there is no guarantee to what
its state will actually be.
Instead, I moved the check into the set_kernel_text_* functions
themselves, and use a local variable to determine when it is
OK to change the kernel text RW permissions.
[ Update: Ingo Molnar suggested moving the prototypes to cacheflush.h ]
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
---
arch/x86/include/asm/cacheflush.h | 5 +++++
arch/x86/kernel/ftrace.c | 13 +++++++++++++
arch/x86/mm/init_32.c | 35 ++++++++++++++++++++++++++++++++---
arch/x86/mm/init_64.c | 37 ++++++++++++++++++++++++++++++++-----
4 files changed, 82 insertions(+), 8 deletions(-)
diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h
index 2f84665..6145063 100644
--- a/arch/x86/include/asm/cacheflush.h
+++ b/arch/x86/include/asm/cacheflush.h
@@ -104,6 +104,11 @@ void clflush_cache_range(void *addr, unsigned int size);
#ifdef CONFIG_DEBUG_RODATA
void mark_rodata_ro(void);
extern const int rodata_test_data;
+void set_kernel_text_rw(void);
+void set_kernel_text_ro(void);
+#else
+static inline void set_kernel_text_rw(void) { }
+static inline void set_kernel_text_ro(void) { }
#endif
#ifdef CONFIG_DEBUG_RODATA_TEST
diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index 231bdd3..77857d4 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -18,6 +18,7 @@
#include <linux/init.h>
#include <linux/list.h>
+#include <asm/cacheflush.h>
#include <asm/ftrace.h>
#include <linux/ftrace.h>
#include <asm/nops.h>
@@ -26,6 +27,18 @@
#ifdef CONFIG_DYNAMIC_FTRACE
+int ftrace_arch_code_modify_prepare(void)
+{
+ set_kernel_text_rw();
+ return 0;
+}
+
+int ftrace_arch_code_modify_post_process(void)
+{
+ set_kernel_text_ro();
+ return 0;
+}
+
union ftrace_code_union {
char code[MCOUNT_INSN_SIZE];
struct {
diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c
index 2cef050..3eb2ed1 100644
--- a/arch/x86/mm/init_32.c
+++ b/arch/x86/mm/init_32.c
@@ -1155,17 +1155,47 @@ static noinline int do_test_wp_bit(void)
const int rodata_test_data = 0xC3;
EXPORT_SYMBOL_GPL(rodata_test_data);
+static int kernel_set_to_readonly;
+
+void set_kernel_text_rw(void)
+{
+ unsigned long start = PFN_ALIGN(_text);
+ unsigned long size = PFN_ALIGN(_etext) - start;
+
+ if (!kernel_set_to_readonly)
+ return;
+
+ pr_debug("Set kernel text: %lx - %lx for read write\n",
+ start, start+size);
+
+ set_pages_rw(virt_to_page(start), size >> PAGE_SHIFT);
+}
+
+void set_kernel_text_ro(void)
+{
+ unsigned long start = PFN_ALIGN(_text);
+ unsigned long size = PFN_ALIGN(_etext) - start;
+
+ if (!kernel_set_to_readonly)
+ return;
+
+ pr_debug("Set kernel text: %lx - %lx for read only\n",
+ start, start+size);
+
+ set_pages_ro(virt_to_page(start), size >> PAGE_SHIFT);
+}
+
void mark_rodata_ro(void)
{
unsigned long start = PFN_ALIGN(_text);
unsigned long size = PFN_ALIGN(_etext) - start;
-#ifndef CONFIG_DYNAMIC_FTRACE
- /* Dynamic tracing modifies the kernel text section */
set_pages_ro(virt_to_page(start), size >> PAGE_SHIFT);
printk(KERN_INFO "Write protecting the kernel text: %luk\n",
size >> 10);
+ kernel_set_to_readonly = 1;
+
#ifdef CONFIG_CPA_DEBUG
printk(KERN_INFO "Testing CPA: Reverting %lx-%lx\n",
start, start+size);
@@ -1174,7 +1204,6 @@ void mark_rodata_ro(void)
printk(KERN_INFO "Testing CPA: write protecting again\n");
set_pages_ro(virt_to_page(start), size>>PAGE_SHIFT);
#endif
-#endif /* CONFIG_DYNAMIC_FTRACE */
start += size;
size = (unsigned long)__end_rodata - start;
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index e6d36b4..63fdc53 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -986,21 +986,48 @@ void free_initmem(void)
const int rodata_test_data = 0xC3;
EXPORT_SYMBOL_GPL(rodata_test_data);
+static int kernel_set_to_readonly;
+
+void set_kernel_text_rw(void)
+{
+ unsigned long start = PFN_ALIGN(_stext);
+ unsigned long end = PFN_ALIGN(__start_rodata);
+
+ if (!kernel_set_to_readonly)
+ return;
+
+ pr_debug("Set kernel text: %lx - %lx for read write\n",
+ start, end);
+
+ set_memory_rw(start, (end - start) >> PAGE_SHIFT);
+}
+
+void set_kernel_text_ro(void)
+{
+ unsigned long start = PFN_ALIGN(_stext);
+ unsigned long end = PFN_ALIGN(__start_rodata);
+
+ if (!kernel_set_to_readonly)
+ return;
+
+ pr_debug("Set kernel text: %lx - %lx for read only\n",
+ start, end);
+
+ set_memory_ro(start, (end - start) >> PAGE_SHIFT);
+}
+
void mark_rodata_ro(void)
{
unsigned long start = PFN_ALIGN(_stext), end = PFN_ALIGN(__end_rodata);
unsigned long rodata_start =
((unsigned long)__start_rodata + PAGE_SIZE - 1) & PAGE_MASK;
-#ifdef CONFIG_DYNAMIC_FTRACE
- /* Dynamic tracing modifies the kernel text section */
- start = rodata_start;
-#endif
-
printk(KERN_INFO "Write protecting the kernel read-only data: %luk\n",
(end - start) >> 10);
set_memory_ro(start, (end - start) >> PAGE_SHIFT);
+ kernel_set_to_readonly = 1;
+
/*
* The rodata section (but not the kernel text!) should also be
* not-executable.
--
1.5.6.5
--
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/4] ftrace: immediately stop code modification if failure is detected
2009-02-20 19:32 [PATCH 0/4] [git pull v2 @num@/@total@] updates for tip Steven Rostedt
2009-02-20 19:32 ` [PATCH 1/4] ftrace: allow archs to preform pre and post process for code modification Steven Rostedt
2009-02-20 19:32 ` [PATCH 2/4] ftrace, x86: make kernel text writable only for conversions Steven Rostedt
@ 2009-02-20 19:32 ` Steven Rostedt
2009-02-20 19:32 ` [PATCH 4/4] ftrace: break out modify loop immediately on detection of error Steven Rostedt
2009-02-22 17:10 ` [PATCH 0/4] [git pull v2 @num@/@total@] updates for tip Ingo Molnar
4 siblings, 0 replies; 6+ messages in thread
From: Steven Rostedt @ 2009-02-20 19:32 UTC (permalink / raw)
To: linux-kernel
Cc: Ingo Molnar, Andrew Morton, Thomas Gleixner, Peter Zijlstra,
Frederic Weisbecker, Steven Rostedt
[-- Attachment #1: 0003-ftrace-immediately-stop-code-modification-if-failur.patch --]
[-- Type: text/plain, Size: 1634 bytes --]
From: Steven Rostedt <srostedt@redhat.com>
Impact: fix to prevent NMI lockup
If the page fault handler produces a WARN_ON in the modifying of
text, and the system is setup to have a high frequency of NMIs,
we can lock up the system on a failure to modify code.
The modifying of code with NMIs allows all NMIs to modify the code
if it is about to run. This prevents a modifier on one CPU from
modifying code running in NMI context on another CPU. The modifying
is done through stop_machine, so only NMIs must be considered.
But if the write causes the page fault handler to produce a warning,
the print can slow it down enough that as soon as it is done
it will take another NMI before going back to the process context.
The new NMI will perform the write again causing another print and
this will hang the box.
This patch turns off the writing as soon as a failure is detected
and does not wait for it to be turned off by the process context.
This will keep NMIs from getting stuck in this back and forth
of print outs.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
---
arch/x86/kernel/ftrace.c | 4 ++++
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index 77857d4..c56d738 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -124,6 +124,10 @@ static void ftrace_mod_code(void)
*/
mod_code_status = probe_kernel_write(mod_code_ip, mod_code_newcode,
MCOUNT_INSN_SIZE);
+
+ /* if we fail, then kill any new writers */
+ if (mod_code_status)
+ mod_code_write = 0;
}
void ftrace_nmi_enter(void)
--
1.5.6.5
--
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 4/4] ftrace: break out modify loop immediately on detection of error
2009-02-20 19:32 [PATCH 0/4] [git pull v2 @num@/@total@] updates for tip Steven Rostedt
` (2 preceding siblings ...)
2009-02-20 19:32 ` [PATCH 3/4] ftrace: immediately stop code modification if failure is detected Steven Rostedt
@ 2009-02-20 19:32 ` Steven Rostedt
2009-02-22 17:10 ` [PATCH 0/4] [git pull v2 @num@/@total@] updates for tip Ingo Molnar
4 siblings, 0 replies; 6+ messages in thread
From: Steven Rostedt @ 2009-02-20 19:32 UTC (permalink / raw)
To: linux-kernel
Cc: Ingo Molnar, Andrew Morton, Thomas Gleixner, Peter Zijlstra,
Frederic Weisbecker, Steven Rostedt
[-- Attachment #1: 0004-ftrace-break-out-modify-loop-immediately-on-detecti.patch --]
[-- Type: text/plain, Size: 857 bytes --]
From: Steven Rostedt <srostedt@redhat.com>
Impact: added precaution on failure detection
Break out of the modifying loop as soon as a failure is detected.
This is just an added precaution found by code review and was not
found by any bug chasing.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
---
kernel/trace/ftrace.c | 5 ++++-
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 72316d9..11ad796 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -561,8 +561,11 @@ static void ftrace_replace_code(int enable)
if ((system_state == SYSTEM_BOOTING) ||
!core_kernel_text(rec->ip)) {
ftrace_free_rec(rec);
- } else
+ } else {
ftrace_bug(failed, rec->ip);
+ /* Stop processing */
+ return;
+ }
}
}
}
--
1.5.6.5
--
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 0/4] [git pull v2 @num@/@total@] updates for tip
2009-02-20 19:32 [PATCH 0/4] [git pull v2 @num@/@total@] updates for tip Steven Rostedt
` (3 preceding siblings ...)
2009-02-20 19:32 ` [PATCH 4/4] ftrace: break out modify loop immediately on detection of error Steven Rostedt
@ 2009-02-22 17:10 ` Ingo Molnar
4 siblings, 0 replies; 6+ messages in thread
From: Ingo Molnar @ 2009-02-22 17:10 UTC (permalink / raw)
To: Steven Rostedt
Cc: linux-kernel, Andrew Morton, Thomas Gleixner, Peter Zijlstra,
Frederic Weisbecker
* Steven Rostedt <rostedt@goodmis.org> wrote:
>
> Ingo,
>
> I incorporated your comments, and rebased against your tip/x86/urgent
> branch.
>
> Updates are:
> - rename ftrace_arch_modify_* to ftrace_arch_code_modify_*
> - use pr_debug over KERN_INFO in set_kernel_text_*
> - removed comment about ftrace on set_kernel_text_*
> - use return instead of goto to break out of double loop
> - merged akpm's suggestion into 2cd patch
>
> My last commit is:
>
> 4377245aa93b65b6597e4b7bb460fb9abc48b56b
> ftrace: break out modify loop immediately on detection of error
>
> Because I did recently rebase, you might want to confirm that
> the SHA1 matches and is not stale data.
>
> Please pull the latest tip/x86/ftrace tree, which can be found at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
> tip/x86/ftrace
>
>
> Steven Rostedt (4):
> ftrace: allow archs to preform pre and post process for code modification
> ftrace, x86: make kernel text writable only for conversions
> ftrace: immediately stop code modification if failure is detected
> ftrace: break out modify loop immediately on detection of error
>
> ----
> arch/x86/include/asm/cacheflush.h | 5 +++++
> arch/x86/kernel/ftrace.c | 17 +++++++++++++++++
> arch/x86/mm/init_32.c | 35 ++++++++++++++++++++++++++++++++---
> arch/x86/mm/init_64.c | 37 ++++++++++++++++++++++++++++++++-----
> include/linux/ftrace.h | 3 +++
> kernel/trace/ftrace.c | 33 ++++++++++++++++++++++++++++++++-
> 6 files changed, 121 insertions(+), 9 deletions(-)
Pulled into tip:tracing/ftrace, thanks Steve!
Ingo
^ permalink raw reply [flat|nested] 6+ messages in thread