Re: [PATCH] Fix WARN_ON / WARN_ON_ONCE regression

Tim Chen
2006-10-04T17:11:12Z

On Wed, 2006-10-04 at 09:30 -0700, Andrew Morton wrote:

> > I have measured the cache miss with tool. So it is not just my theory.
> >
>
> And what did that tool tell you?

I am using emon. Measuring a 20 second stretch of tbench run saw the L2
cache miss go from 14 million to 25 million on each of the cpu core.
>
> Please don't just ignore my questions. *why* are we getting a cache miss
> rate on that integer which is causing measurable performance changes? If
> we're reading it that frequently then the variable should be in cache(!).
>

The point is valid, __warn_once should be in cache, unless something
evicts it. What I have found so far is with patch by Andrew and Leonid
that avoid looking up the __warn_once integer, the cache miss rate is
reduced to the level before.

> Again: do you know which callsite is causing the problem? I assume one of
> the ones in softirq.c? Do you know what the cache miss frequency is? etc.
>
Unfortunately emon does not directly give the callsite. Oprofile data
shows a marked increase in time spent in do_softirq and local_bh_enable.
What I could do is to individually turn off WARN_ON_ONCE at these sites
and see if they are responsible for the cache miss. Will let you know
what I found.

Oprofile data --
Before WARN_ON_ONCE patch:

117767 thread_return
106651 local_bh_enable
83767 tcp_v4_rcv
72266 copy_user_generic_unrolled
47136 do_softirq

41100 tcp_recvmsg
39394 tcp_sendmsg
118383 thread_return
88171 copy_user_generic

..
8281 local_bh_enable
6790 do_softirq

After WARN_ON_ONCE patch:

117767 thread_return
106651 local_bh_enable
83767 tcp_v4_rcv
72266 copy_user_generic_unrolled
47136 do_softirq

Tim