Re: [PATCH] Fix WARN_ON / WARN_ON_ONCE regression

Steven Rostedt
2006-10-10T13:05:08Z

Tim,

Great work on finding this!

On Mon, 2006-10-09 at 18:09 -0700, Tim Chen wrote:

>
>
> The original "C" code looks very innocent:
>
> if (WARN_ON(__ret_warn_once));
> __warn_once = 0;
>
> The equivalent asm code generated by gcc looks like:
>
> temp = 0;
> if (!WARN_ON(__ret_warn_once))
> temp = __warn_once;
> __warn_once = temp;
>
>
> As a result, a global variable is being written from all CPUs
> everywhere and caused excessive cache line bouncing on SMP.
> We measured that HITM event increased by 75% and
> read-for-ownership event increased by 50%. Adding a
> __read_mostly directive to __warn_once didn't help
> because gcc still generate assembly code that write to
> that global variable.

Holy crap! I wonder where else in the kernel gcc is doing this. (of
course I'm using gcc4 so I don't know). Is there another gcc attribute
to actually tell gcc that a variable is really mostly read only (besides
placing it in a mostly read only elf section)?

What was wrong with the original WARN_ON_ONCE with

if (unlikely(condition) && __warn_once)

This didn't have the cache crash problem too, did it?
I don't have a gcc3 around to test.

-- Steve