Re: Bug, kernel panic, NULL dereference , cleanup_once / icmp_route_lookup.clone.19.clone / nat , 2.6.39-rc7-git11

Denys Fedoryshchenko
2011-05-18T09:53:37Z

On Wed, 18 May 2011 11:37:51 +0200, Eric Dumazet wrote:
> Le mercredi 18 mai 2011 à 12:27 +0300, Denys Fedoryshchenko a écrit :
>> On Wed, 18 May 2011 01:16:29 +0300, Denys Fedoryshchenko wrote:
>> > Just got recently. 32Bit, PPPoE NAS, shapers, firewall, NAT
>> > Kernel i mention in subject, 2.6.39-rc7-git11
>> > If required i can give more information
>> >
>> > sharanal (sorry for ugly name) is libpcap based traffic analyser,
>> > sure userspace
>> >
>> Here is some info, i hope it will be a little useful
>>
>> (gdb) l *(cleanup_once + 0x49)
>> 0xc02e85cc is in cleanup_once (include/linux/list.h:88).
>> 83 * This is only for internal list manipulation where we
>> know
>> 84 * the prev/next entries already!
>> 85 */
>> 86 static inline void __list_del(struct list_head * prev,
>> struct
>> list_head * next)
>> 87 {
>> 88 next->prev = prev;
>> 89 prev->next = next;
>> 90 }
>> 91
>> 92 /**
>>
>> (gdb) l *(inet_getpeer + 0x2ab)
>> 0xc02e8ae8 is in inet_getpeer (net/ipv4/inetpeer.c:530).
>> 525 if (base->total >= inet_peer_threshold)
>> 526 /* Remove one less-recently-used entry. */
>> 527 cleanup_once(0, stack);
>> 528
>> 529 return p;
>> 530 }
>> 531
>> 532 static int compute_total(void)
>> 533 {
>> 534 return v4_peers.total + v6_peers.total;
>>
>
> I really begin to think we have a bug here...
>
> In previous reports, I suggested to use slub_nomerge because I
> thought
> one corruption from another kernel layer was going on.
>
> (inetpeer was using 64 bytes objects). But now that inetpeer objects
> are
> bigger and sit in another kmemcache, its bad news.
>
> Could you try this, and eventually add some SLUB debugging stuff as
> well ?

Yes, i will try. I should enable SLUB debugging only, or also anything
else?

But possible it will take time to reproduce bug, seems it is occuring
rare. With 2.6.39 release i will rollout update to few hundreds PPPoE's,
maybe it will increase chances to get information.