Re: [RFC PATCH 2/6] jump label v3 - x86: Introduce generic jump patching without stop_machine

[Masami Hiramatsu <mhiramat@redhat.com>]

Mathieu Desnoyers wrote:
> * Jason Baron (jbaron@redhat.com) wrote:
>> Add text_poke_fixup() which takes a fixup address to where a processor
>> jumps if it hits the modifying address while code modifying.
>> text_poke_fixup() does following steps for this purpose.
>>
>>   1. Setup int3 handler for fixup.
>>   2. Put a breakpoint (int3) on the first byte of modifying region,
>>      and synchronize code on all CPUs.
>>   3. Modify other bytes of modifying region, and synchronize code on all CPUs.
>>   4. Modify the first byte of modifying region, and synchronize code
>>      on all CPUs.
>>   5. Clear int3 handler.
>>
>
> Hi Masami,
>
> I like the approach and the API is clean. I have intersped comments
> below.
>
> Ingo: I raise a question about text_mutex vs on_each_cpu hangs I
> experienced recently in the message below. Might be worth having a look,
> I suspect this might have caused the hangs Paul McKenney had with his
> past TREE RCU callback migration. I think he did take a mutex in the cpu
> hotplug callbacks and might have used IPIs within that same lock.

Hi Mathieu,

I guess that the hang might happen as below;

----
lock text_mutex
modify code
on_each_cpu(do_something)
                               cpu-hotplug (down)
                               lock cpu-hotplug mutex
                               online_cpus is changed
                               native_cpu_die()
                               ->alternatives_smp_switch(0)
                                 ->lock text_mutex -> sleep
(wait for offlined cpu...)
----

If this is correct, I think we can fix it as below.

----
lock cpu-hotplug mutex
lock text_mutex
modify code
on_each_cpu(do_something)
unlock text_mutex
unlock cpu-hotplug mutex
                               cpu-hotplug (down)
                               lock cpu-hotplug mutex
                               online_cpus is changed
                               native_cpu_die()
                               ->alternatives_smp_switch(0)
                                 ->lock text_mutex
                                   modify code
                                   unlock text_mutex
                               ...
                               unlock cpu-hotplug mutex
----
Or,
----
lock text_mutex
modify code
unlock text_mutex
on_each_cpu(do_something)
                               cpu-hotplug (down)
                               lock cpu-hotplug mutex
                               online_cpus is changed
                               native_cpu_die()
                               ->alternatives_smp_switch(0)
                                 ->lock text_mutex
                                   modify code
                                   unlock text_mutex
                               ...
                               unlock cpu-hotplug mutex
----
The latter needs another mutex for int3 handler and
frequently mutex_lock/unlock in this patch.

Hmm?

Thank you,

-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhiramat@redhat.com