Re: Errors and later panics in 2.6.0-test11.

[Jens Axboe <axboe@suse.de>]

On Wed, Dec 03 2003, Linus Torvalds wrote:
> 
> 
> On Wed, 3 Dec 2003, David Martínez Moreno wrote:
> >
> > 	I've just rebooted about six hours ago, and it's giving panics elsewhere:
> >
> > [...]
> > Ending XFS recovery on filesystem: md0 (dev: md0)
> > b44: eth0: Link is down.
> > b44: eth0: Link is up at 100 Mbps, full duplex.
> > b44: eth0: Flow control is on for TX and on for RX.
> > eth0: no IPv6 routers present
> > Unable to handle kernel paging request at virtual address 00100104
> 
> That's the LIST_POISON stuff: 00100100 is the "bad list pointer". Somebody
> tried to remove a page twice.
> 
> Doesn't mean a lot - if your "struct page" got corrupted, anything can
> happen. Quite possibly it's a double free.
> 
> > 	I can rebuild the Debian mirror for not using the RAID and using the SATA
> > disks separately, but will be tomorrow, it's a lot of space to move, and I
> > need remote intervention.
> >
> > 	Anyway I'd love to know before doing if it will be useful, looking at what
> > Jens has said just ten minutes ago about RAIDs 0/5. Will it help to you? Say
> > so and I'll go for it.
> 
> It might be more useful to leave it as RAID0, if you're willing to try out
> patches to try to debug this. The slab-debugging thing I sent out earlier
> is one such patch (but may well cause out-of-memory problems under load),
> and possibly the atomic-decrement checker patch (appended). And maybe Jens
> and Neil can come up with something..

I can reproduce on raid5 with linear dm on top (using XFS). I need to
kill the slab and memory debugging, I've put some bio debugging in there
instead (the memory debugging interferes with it). It's definitely a bio
use after free case, clone_endio() ends up with a freed bio.

Program received signal SIGEMT, Emulation trap.
0xc02dd454 in handle_stripe (sh=0xc17cf630) at drivers/md/raid5.c:1009
1009                                        wbi = wbi2;
(gdb) bt
#0  0xc02dd454 in handle_stripe (sh=0xc17cf630) at drivers/md/raid5.c:1009
#1  0xc02de31f in raid5d (mddev=0xdfd2d200) at drivers/md/raid5.c:1436
#2  0xc02e675a in md_thread (arg=0xdffdc1a0) at drivers/md/md.c:2692
#3  0xc010752d in kernel_thread_helper () at arch/i386/kernel/process.c:226

wbi (dev->written) has already been freed by someone else.

My puny 512MB test box cannot use your slab-debug patch :). The
atomic-checker didn't catch anything.

-- 
Jens Axboe