Re: [RFC PATCH 2/6] jump label v3 - x86: Introduce generic jump patching without stop_machine

[Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>]

* Masami Hiramatsu (mhiramat@redhat.com) wrote:
> Mathieu Desnoyers wrote:
>> * Jason Baron (jbaron@redhat.com) wrote:
>>> Add text_poke_fixup() which takes a fixup address to where a processor
>>> jumps if it hits the modifying address while code modifying.
>>> text_poke_fixup() does following steps for this purpose.
>>>
>>>   1. Setup int3 handler for fixup.
>>>   2. Put a breakpoint (int3) on the first byte of modifying region,
>>>      and synchronize code on all CPUs.
>>>   3. Modify other bytes of modifying region, and synchronize code on all CPUs.
>>>   4. Modify the first byte of modifying region, and synchronize code
>>>      on all CPUs.
>>>   5. Clear int3 handler.
>>>
>>
>> Hi Masami,
>>
>> I like the approach and the API is clean. I have intersped comments
>> below.
>>
>> Ingo: I raise a question about text_mutex vs on_each_cpu hangs I
>> experienced recently in the message below. Might be worth having a look,
>> I suspect this might have caused the hangs Paul McKenney had with his
>> past TREE RCU callback migration. I think he did take a mutex in the cpu
>> hotplug callbacks and might have used IPIs within that same lock.
>
> Hi Mathieu,
>
> I guess that the hang might happen as below;
>
> ----
> lock text_mutex
> modify code
> on_each_cpu(do_something)
>                               cpu-hotplug (down)
>                               lock cpu-hotplug mutex
>                               online_cpus is changed
>                               native_cpu_die()
>                               ->alternatives_smp_switch(0)
>                                 ->lock text_mutex -> sleep
> (wait for offlined cpu...)
> ----
>
> If this is correct, I think we can fix it as below.
>
> ----
> lock cpu-hotplug mutex

Yes, this is the solution I used in my own immediate values code too.

> lock text_mutex
> modify code
> on_each_cpu(do_something)
> unlock text_mutex
> unlock cpu-hotplug mutex
>                               cpu-hotplug (down)
>                               lock cpu-hotplug mutex
>                               online_cpus is changed
>                               native_cpu_die()
>                               ->alternatives_smp_switch(0)
>                                 ->lock text_mutex
>                                   modify code
>                                   unlock text_mutex
>                               ...
>                               unlock cpu-hotplug mutex
> ----
> Or,
> ----
> lock text_mutex
> modify code
> unlock text_mutex
> on_each_cpu(do_something)
>                               cpu-hotplug (down)
>                               lock cpu-hotplug mutex
>                               online_cpus is changed
>                               native_cpu_die()
>                               ->alternatives_smp_switch(0)
>                                 ->lock text_mutex
>                                   modify code
>                                   unlock text_mutex
>                               ...
>                               unlock cpu-hotplug mutex
> ----
> The latter needs another mutex for int3 handler and
> frequently mutex_lock/unlock in this patch.
>
> Hmm?

The simplest solution seems to be the best one IMHO. But have you been
able to reproduce the lockup ?

Mathieu

>
> Thank you,
>
> -- 
> Masami Hiramatsu
>
> Software Engineer
> Hitachi Computer Products (America), Inc.
> Software Solutions Division
>
> e-mail: mhiramat@redhat.com
>

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68