Re: [profile]: [0/23] mmap() support for /proc/profile

[Anton Blanchard <anton@samba.org>]

> I was trying to profile a mostly-idle workload to get an idea of what
> area of the kernel things were diving into and falling asleep in during
> an OAST run. Without these patches, kerneltop et al showed heavy /proc/
> activity along with copy_to_user() at the top of the profiles.

Interesting stuff. FYI we did some analysis of the hottest addresses in
the kernel and profile_lock featured very high up:

void profile_hook(struct pt_regs * regs)
{
        read_lock(&profile_lock);
        notifier_call_chain(&profile_listeners, 0, regs);
        read_unlock(&profile_lock);
}

Thats 2 atomic operations to the same cacheline per timer interrupt per
cpu. Considering how rarely timer based profiling is used, perhaps RCU
or even just a profiling_enabled sysctl flag would help here. Id prefer
not to compile it out in distro kernels if possible, its a very useful
feature when required.

In the mean time, how about this quick fix?

Anton

--

Cacheline align profile_lock, analysis shows it to be one of the hottest
memory locations on large SMP boxes.

Signed-off-by: Anton Blanchard <anton@samba.org>

===== kernel/profile.c 1.5 vs edited =====
--- 1.5/kernel/profile.c	Wed Jul 16 18:09:04 2003
+++ edited/kernel/profile.c	Wed Jun 23 09:13:28 2004
@@ -8,6 +8,7 @@
 #include <linux/bootmem.h>
 #include <linux/notifier.h>
 #include <linux/mm.h>
+#include <linux/cache.h>
 #include <asm/sections.h>
 
 unsigned int * prof_buffer;
@@ -119,7 +120,7 @@
 }
 
 static struct notifier_block * profile_listeners;
-static rwlock_t profile_lock = RW_LOCK_UNLOCKED;
+static rwlock_t profile_lock __cacheline_aligned_in_smp = RW_LOCK_UNLOCKED;
  
 int register_profile_notifier(struct notifier_block * nb)
 {