Re: [PATCH v12 09/11] pvqspinlock, x86: Add para-virtualization support

[Waiman Long <waiman.long@hp.com>]

On 10/24/2014 04:54 AM, Peter Zijlstra wrote:
> On Thu, Oct 16, 2014 at 02:10:38PM -0400, Waiman Long wrote:
>
>> Since enabling paravirt spinlock will disable unlock function inlining,
>> a jump label can be added to the unlock function without adding patch
>> sites all over the kernel.
> But you don't have to. My patches allowed for the inline to remain,
> again reducing the overhead of enabling PV spinlocks while running on a
> real machine.
>
> Look at:
>
>    http://lkml.kernel.org/r/20140615130154.213923590@chello.nl
>
> In particular this hunk:
>
> Index: linux-2.6/arch/x86/kernel/paravirt_patch_64.c
> ===================================================================
> --- linux-2.6.orig/arch/x86/kernel/paravirt_patch_64.c
> +++ linux-2.6/arch/x86/kernel/paravirt_patch_64.c
> @@ -22,6 +22,10 @@ DEF_NATIVE(pv_cpu_ops, swapgs, "swapgs")
>   DEF_NATIVE(, mov32, "mov %edi, %eax");
>   DEF_NATIVE(, mov64, "mov %rdi, %rax");
>
> +#if defined(CONFIG_PARAVIRT_SPINLOCKS)&&  defined(CONFIG_QUEUE_SPINLOCK)
> +DEF_NATIVE(pv_lock_ops, queue_unlock, "movb $0, (%rdi)");
> +#endif
> +
>   unsigned paravirt_patch_ident_32(void *insnbuf, unsigned len)
>   {
>          return paravirt_patch_insns(insnbuf, len,
> @@ -61,6 +65,9 @@ unsigned native_patch(u8 type, u16 clobb
>                  PATCH_SITE(pv_cpu_ops, clts);
>                  PATCH_SITE(pv_mmu_ops, flush_tlb_single);
>                  PATCH_SITE(pv_cpu_ops, wbinvd);
> +#if defined(CONFIG_PARAVIRT_SPINLOCKS)&&  defined(CONFIG_QUEUE_SPINLOCK)
> +               PATCH_SITE(pv_lock_ops, queue_unlock);
> +#endif
>
>          patch_site:
>                  ret = paravirt_patch_insns(ibuf, len, start, end);
>
>
> That makes sure to overwrite the callee-saved call to the
> pv_lock_ops::queue_unlock with the immediate asm "movb $0, (%rdi)".
>
>
> Therefore you can retain the inlined unlock with hardly (there might be
> some NOP padding) any overhead at all. On PV it reverts to a callee
> saved function call.

My concern is that spin_unlock() can be called in many places, including 
loadable kernel modules. Can the paravirt_patch_ident_32() function able 
to patch all of them in reasonable time? How about a kernel module 
loaded later at run time?

So I think we may still need to disable unlock function inlining even if 
we used your way kernel site patching.

Regards,
Longman