Re: [PATCH] Fix WARN_ON / WARN_ON_ONCE regression

[Steven Rostedt <rostedt@goodmis.org>]

Tim,

Great work on finding this!

On Mon, 2006-10-09 at 18:09 -0700, Tim Chen wrote:

> 
> 
> The original "C" code looks very innocent:
> 
>     if (WARN_ON(__ret_warn_once));
>         __warn_once = 0;
> 
> The equivalent asm code generated by gcc looks like:
> 
>     temp = 0;
>     if (!WARN_ON(__ret_warn_once))
>         temp = __warn_once;
>     __warn_once = temp;
> 
> 
> As a result, a global variable is being written from all CPUs 
> everywhere and caused excessive cache line bouncing on SMP.  
> We measured that HITM event increased by 75% and 
> read-for-ownership event increased by 50%. Adding a
> __read_mostly directive to __warn_once didn't help 
> because gcc still generate assembly code that write to 
> that global variable.

Holy crap!  I wonder where else in the kernel gcc is doing this. (of
course I'm using gcc4 so I don't know).  Is there another gcc attribute
to actually tell gcc that a variable is really mostly read only (besides
placing it in a mostly read only elf section)?

What was wrong with the original WARN_ON_ONCE with

  if (unlikely(condition) && __warn_once)

This didn't have the cache crash problem too, did it?
I don't have a gcc3 around to test.

-- Steve