Re: [bug, netconsole, SLUB] BUG skbuff_head_cache: Poison overwritten

Ingo Molnar
2008-07-21T10:52:27Z

* Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:

> Hi.
>
> On Mon, Jul 21, 2008 at 12:52:45PM +0300, Pekka Enberg (penberg@cs.helsinki.fi) wrote:
> > On Mon, Jul 21, 2008 at 12:41 PM, Ingo Molnar <mingo@elte.hu> wrote:
> > > update about this problem: just triggered another colorful crash, see
> > > below. This was with the 4K object dump patch already, maybe the dump
> > > gives a clue?
> >
> > ...to point out the obvious:
> >
> > > =============================================================================
> > > BUG skbuff_head_cache: Poison overwritten
> > > -----------------------------------------------------------------------------
> > >
> > > INFO: 0xf7ccc100-0xf7ccc103. First byte 0x0 instead of 0x6b
> > > INFO: Allocated in __alloc_skb+0x30/0x10e age=1 cpu=1 pid=1
> > > INFO: Freed in __kfree_skb+0x63/0x66 age=1 cpu=0 pid=0
> > > INFO: Slab 0xc1c34ca0 objects=16 used=1 fp=0xf7ccc100 flags=0x400000c3
> > > INFO: Object 0xf7ccc100 @offset=256 fp=0xf7ccc200
> > >
> > > Bytes b4 0xf7ccc0f0: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
> > > Object 0xf7ccc100: 00 00 00 00 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b ....kkkkkkkkkkkk
> >
> > Use after free where first four bytes are zeroed.
>
> Not that obvious...
> skb->next is cleared in lots of places, in xmit network helper
> for example, but since rest of the packet was not modified, it
> means given skb was not freed, so it will not help.
>
> Ingo do you see other similar dumps with last byte modified? That's
> the one which can help to determine the reason.

the problem is, most of the crashes dont come with any usable dump. This
is a laptop so netconsole is the only reliable route out - and if
something in networking crashes chances are that it hoses netconsole
before it can get anything out.

Another thing is that i'm activating netconsole on this box via a kernel
boot line and from within a bzImage (to get it activated as early as
possible) - maybe that's a tad too early for certain initialization
sequences?

I could try run tests with netconsole deactivated, if you think that's a
worthwile line of probing this problem. (although that would make me do
blind tests in essence - having kernel log output is really essential.)

Ingo