[PATCH] x86/mm/fault: Put pgd_lock in its own cacheline

[PATCH] x86/mm/fault: Put pgd_lock in its own cacheline

[Waiman Long]

On a large system with many cores, massive creation or destruction
of processes/threads can sometime cause a fair amount of spinlock
contention in the pgd_lock used by the pgd_alloc() and pgd_free()
functions. This patch tries to reduce false cacheline sharing by
putting the pgd_lock in its own cacheline which help to reduce
contention on the lock.

Signed-off-by: Waiman Long <Waiman.Long@hp.com>
---
 arch/x86/mm/fault.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 181c53b..fae48df 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -185,7 +185,7 @@ force_sig_info_fault(int si_signo, int si_code, unsigned long address,
 	force_sig_info(si_signo, &info, tsk);
 }
 
-DEFINE_SPINLOCK(pgd_lock);
+__cacheline_aligned_in_smp DEFINE_SPINLOCK(pgd_lock);
 LIST_HEAD(pgd_list);
 
 #ifdef CONFIG_X86_32
-- 
1.7.1

Re: [PATCH] x86/mm/fault: Put pgd_lock in its own cacheline

[Ingo Molnar]

* Waiman Long <Waiman.Long@hp.com> wrote:

> On a large system with many cores, massive creation or destruction
> of processes/threads can sometime cause a fair amount of spinlock
> contention in the pgd_lock used by the pgd_alloc() and pgd_free()
> functions. This patch tries to reduce false cacheline sharing by
> putting the pgd_lock in its own cacheline which help to reduce
> contention on the lock.
> 
> Signed-off-by: Waiman Long <Waiman.Long@hp.com>
> ---
>  arch/x86/mm/fault.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> index 181c53b..fae48df 100644
> --- a/arch/x86/mm/fault.c
> +++ b/arch/x86/mm/fault.c
> @@ -185,7 +185,7 @@ force_sig_info_fault(int si_signo, int si_code, unsigned long address,
>  	force_sig_info(si_signo, &info, tsk);
>  }
>  
> -DEFINE_SPINLOCK(pgd_lock);
> +__cacheline_aligned_in_smp DEFINE_SPINLOCK(pgd_lock);
>  LIST_HEAD(pgd_list);

Yeah, so if you are seeing pgd_lock contention we could do a lot more to improve 
the performance of pgd allocation than just putting it on a separate cacheline: we 
can actually eliminate pgd_list and turn all pgd_list users into RCU walkers of 
the tasklist lock, and make pgd_alloc()/pgd_free() lockless on x86.

I've done a series that implements this, I'll post it soon, please stay tuned!

Thanks,

	Ingo