Re: Clear performance regression on reaim7 in 2.6.15-git6

Martin J. Bligh
2006-04-15T22:30:21Z

Andrew Morton wrote:
> "Martin J. Bligh" <mbligh@google.com> wrote:
>
>>drilling down into the results directories can get you the command line,
>> looks like "reaim -f workfile.short -s 1 -e 10 -i 2" to me. Buggered if
>> I can recall what that did though.
>>
>> (http://test.kernel.org/abat/20229/004.reaim.test/results/cmdline)
>>
>> I *think* it's only ia32 NUMA boxes, at least as far as I can see from
>> a quick poke around. Which would make me guess at scheduler code. Gitweb
>> would be nice to use, but it doesn't tag the -git snapshots, AFAICS,
>> which is a real shame. Nor does the git snapshot include the git tag,
>> as far as I know. Grrrr. I guess I'll download the snapshots and diff
>> them, and try to pull out the sched changes by hand. Much suckage.
>
>
> The diffstat for 2.6.15-git5 -> 2.6.15-git6 is at
> http://www.zip.com.au/~akpm/linux/patches/stuff/2 - only a single line
> changed in sched.c:
>
> diff -uNr 2.6.15-git5/kernel/sched.c 2.6.15-git6/kernel/sched.c
> --- 2.6.15-git5/kernel/sched.c 2006-04-15 14:10:43.000000000 -0700
> +++ 2.6.15-git6/kernel/sched.c 2006-04-15 14:10:52.000000000 -0700
> @@ -4386,6 +4386,7 @@
> } while_each_thread(g, p);
>
> read_unlock(&tasklist_lock);
> + mutex_debug_show_all_locks();
> }

Hmm. whilst it's probably not that call, it does look like mutex
debugging. Look at the profiles from reaim between -git5 and -git6:

before:
http://test.kernel.org/abat/20115/004.reaim.test/profiling/profile.text

after:
http://test.kernel.org/abat/20229/004.reaim.test/profiling/profile.text

1262 kfree 3.5056
820 mutex_lock_interruptible 164.0000
752 __mutex_lock_slowpath 0.8754
43 schedule 0.0284
35 _spin_lock 2.3333
25 debug_mutex_set_owner 0.1613
23 debug_mutex_add_waiter 0.1586

/me hugs his huge stacks of data.

--------------------------------

config DEBUG_MUTEXES
bool "Mutex debugging, deadlock detection"
default y
depends on DEBUG_KERNEL
help
This allows mutex semantics violations and mutex related deadlocks
(lockups) to be detected and reported automatically.

--------------------------------

Hmmm. Who the hell thought defaulting that to 'y' was a good plan????
It's still broken in 17-rc1 ... will send you a patch in a sec.

M.

PS. Jeff Garzik pointed out there *are* git tags for each -git snapshot
http://www.kernel.org/pub/linux/kernel/v2.6/snapshots/patch-2.6.17-rc1-git11.id
which is really helpful.