Re: [RFC][PATCH] x86/mm: Sync all vmalloc mappings before text_poke()

Joerg Roedel
2020-04-29T10:59:47Z

Hi Steven,

On Wed, Apr 29, 2020 at 05:48:57AM -0400, Steven Rostedt wrote:
> From: Steven Rostedt (VMware) <rostedt@goodmis.org>
>
> Tzvetomir was adding a feature to trace-cmd that would allow the user
> to specify filtering on process IDs within a tracing instance (or
> buffer). When he added this feature and tested it on tracing PIDs 1 and
> 2, it caused his kernel to hang.
>
> He sent me his code and I was able to reproduce the hang as well. I
> bisected it down to this commit 763802b53a42 ("x86/mm: split
> vmalloc_sync_all()"). It was 100% reproducible. With the commit it
> would hang, and reverting the commit, it would work.
>
> Adding a bunch of printk()s, I found where it locked up. It was after
> the recording was finished, and a write of "0" to
> tracefs/instance/foo/events/enable. And in the code, it was:
>
> (you may skip to the end of the chain)
>
> system_enable_write() {
> __ftrace_set_clr_event() {
> __ftrace_set_clr_event_nolock() {
> ftrace_event_enable_disable() {
> __ftrace_event_enable_disable() {
> call->class->reg() <trace_event_reg()> {
> trace_point_probe_unregister() {
> tracepoint_remove_func() {
> static_key_slow_dec() {
> __static_key_slow_dec() {
>
> <continued>
>
> __static_key_slow_dec_cpus_locked() {
> jump_label_update() {
> __jump_label_update()
> arch_jump_label_transform() {
> jump_label_transform() {
> __jump_label_transform() {
> text_poke_bp() {
> text_poke_bp_batch() {
> text_poke() {
> __text_poke() {
>
> <continued> (This is where you want to see)
>
> use_temporary_mm() {
> switch_mm_irqs_off() {
> load_new_mm_cr3() {
> write_cr3() <<--- Lock up!

I don't see how it could lock up in write_cr3(), at least on bare-metal.
What is the environment this happens, 32 or 64 bit, in a VM or
bare-metal?

I think it is more likely that your lockup is actually a page-fault
loop, where the #PF handler does not map the faulting address correctly.

But I have to look closer into how text_poke() works before I can say
more.

Btw, in case it happens on x86-64, does it also happen without
vmalloc-stacks?

Regards,

Joerg