Re: [Bug #11308] tbench regression on each kernel release from 2.6.22 -> 2.6.28

Eric Dumazet
2008-09-17T14:47:13Z

Mike Galbraith a écrit :
> On Wed, 2008-09-17 at 14:49 +0200, Ingo Molnar wrote:
>> * Mike Galbraith <efault-Mmb7MZpHnFY@public.gmane.org> wrote:
>>
>>>> It looks like a potentially bogus bisection result, but _maybe_ it
>>>> has relevance: changes the size of "struct security_operations",
>>>> which could have alignment and layout effects on all sorts of kernel
>>>> variables, kmalloc sizes, etc.
>>> This may well be a mythical creature infestation for all I know ;-),
>>> but it's address is somewhere in the 2069f45..847106f block, 316
>>> commits, none of which look like they should be the least bit
>>> interesting to netperf. I reverted this particular commit in 27.git,
>>> got the expected result. Looks like I'll keep poking at it, can't
>>> seem to resist. Grr.
>> are you sure it's 2069f45..847106f? Filtering out the
>> likely-uninteresting commits:
>
> Yeah, as sure as I can be. I've built both (et al) kernels several
> times, and it has repeated every time. Would be nice if someone would
> try to confirm/deny though. For my little quad, I do..
>
> #!/bin/sh
>
> echo 0 > /proc/sys/kernel/sched_wakeup_granularity_ns
>
> netserver -p 12865
> netserver -p 12866
> netserver -p 12867
> netserver -p 12868
>
> netperf -p 12865 -t TCP_RR -l 60 -H 127.0.0.1 -T 0,0 -- -r 1,1&
> netperf -p 12866 -t TCP_RR -l 60 -H 127.0.0.1 -T 1,1 -- -r 1,1&
> netperf -p 12867 -t TCP_RR -l 60 -H 127.0.0.1 -T 2,2 -- -r 1,1&
> netperf -p 12868 -t TCP_RR -l 60 -H 127.0.0.1 -T 3,3 -- -r 1,1&
>
> wait
> killall netserver
>
>
>> git log --pretty=format:"%h: %s" 2069f45..847106f | grep -viE \
>> 'block|alsa|pcmcia|sound|Merge|iosched|blk|DAC960|scsi|s390|paride|pktcdvd|filter|cdrom|drm'
>>
>> gives us:
>>
>> 7daf705: Start using the new '%pS' infrastructure to print symbols
>> 6f0f0fd: security: remove register_security hook
>> 93cbace: security: remove dummy module fix
>> 5915eb5: security: remove dummy module
>> b478a9f: security: remove unused sb_get_mnt_opts hook
>> 32502b8: splice: fix generic_file_splice_read() race with page invalidation
>> 8b3d356: ramfs: enable splice write
>> a144ff0: xen: Avoid allocations causing swap activity on the resume path
>>
>> which really only leaves that security commit your bisection fingered.
>> Which _slightly_ raises its likelyhood of being implicated. Structure
>> size changes can move two formerly far-apart netperf-relevant symbols on
>> the same cacheline, which can start cache ping-pong-ing badly.
>
> I sure hope it's something like ping-pong, it's driving me NUTS.

Could you please try following patch ?

[PATCH] security_ops moved to read_mostly section

"struct security_operations *security_ops" should be moved to read_mostly
section in order to NOT let it share a cache line with higly modified variables.

Signed-off-by: Eric Dumazet <dada1-fPLkHRcR87vqlBn2x/YWAg@public.gmane.org>

diff --git a/security/security.c b/security/security.c
index 3a4b4f5..0b13d65 100644
--- a/security/security.c
+++ b/security/security.c
@@ -24,7 +24,7 @@ static __initdata char chosen_lsm[SECURITY_NAME_MAX + 1];
extern struct security_operations default_security_ops;
extern void security_fixup_ops(struct security_operations *ops);

-struct security_operations *security_ops; /* Initialized to NULL */
+struct security_operations *security_ops __read_mostly;e

/* amount of vm to protect from userspace access */
unsigned long mmap_min_addr = CONFIG_SECURITY_DEFAULT_MMAP_MIN_ADDR;