* Crash flood fills syslog
@ 2015-06-24 14:52 John Bridges
2015-06-24 21:53 ` NeilBrown
0 siblings, 1 reply; 2+ messages in thread
From: John Bridges @ 2015-06-24 14:52 UTC (permalink / raw)
To: linux-raid
Messages like this flooded my syslog ballooning it up to 90GB, and
causing other failures when the system drive had no free space left
(bad).
Distributor ID: Ubuntu
Description: Ubuntu 14.04.2 LTS
Release: 14.04
Codename: trusty
Jun 23 08:25:08 monster kernel: [762785.217575] ------------[ cut here
]------------
Jun 23 08:25:08 monster kernel: [762785.217580] WARNING: CPU: 0 PID:
28173 at /build/buildd/linux-3.13.0/drivers/md/raid5.c:3856
handle_stripe+0x24e4/0x2500 [raid456]()
Jun 23 08:25:08 monster kernel: [762785.217581] Modules linked in:
pci_stub vboxpci(OF) vboxnetadp(OF) vboxnetflt(OF) vboxdrv(OF)
gpio_ich coretemp kvm_intel kvm serio_raw bnep rfcomm bluetooth
nvidia(POF) i3000_edac parport_pc drm lpc_ich ppdev lp shpchp mac_hid
edac_core parport xfs btrfs libcrc32c raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor async_tx xor usb_storage pata_acpi
ahci psmouse libahci e1000e raid6_pq ptp pps_core raid1 raid0
multipath sata_mv floppy linear
Jun 23 08:25:08 monster kernel: [762785.217612] CPU: 0 PID: 28173
Comm: md2_resync Tainted: PF W O 3.13.0-29-generic #53-Ubuntu
Jun 23 08:25:08 monster kernel: [762785.217613] Hardware name:
Supermicro PDSM4+/PDSM4+, BIOS 6.00 11/04/2008
Jun 23 08:25:08 monster kernel: [762785.217615] 0000000000000009
ffff88017e997af0 ffffffff8171a214 0000000000000000
Jun 23 08:25:08 monster kernel: [762785.217618] ffff88017e997b28
ffffffff810676bd ffff88010d90a6b0 0000000000000009
Jun 23 08:25:08 monster kernel: [762785.217621] 0000000000000009
ffff8800369e8c00 000000000000000e ffff88017e997b38
Jun 23 08:25:08 monster kernel: [762785.217624] Call Trace:
Jun 23 08:25:08 monster kernel: [762785.217627] [<ffffffff8171a214>]
dump_stack+0x45/0x56
Jun 23 08:25:08 monster kernel: [762785.217630] [<ffffffff810676bd>]
warn_slowpath_common+0x7d/0xa0
Jun 23 08:25:08 monster kernel: [762785.217633] [<ffffffff8106779a>]
warn_slowpath_null+0x1a/0x20
Jun 23 08:25:08 monster kernel: [762785.217637] [<ffffffffa0108cd4>]
handle_stripe+0x24e4/0x2500 [raid456]
Jun 23 08:25:08 monster kernel: [762785.217640] [<ffffffff8109a7b2>]
? default_wake_function+0x12/0x20
Jun 23 08:25:08 monster kernel: [762785.217644] [<ffffffff810aaeb2>]
? autoremove_wake_function+0x12/0x40
Jun 23 08:25:08 monster kernel: [762785.217646] [<ffffffff810aa7b8>]
? __wake_up_common+0x58/0x90
Jun 23 08:25:08 monster kernel: [762785.217651] [<ffffffffa0104c1e>]
? get_active_stripe+0x58e/0x7c0 [raid456]
Jun 23 08:25:08 monster kernel: [762785.217656] [<ffffffffa0109b9c>]
sync_request+0x17c/0x3e0 [raid456]
Jun 23 08:25:08 monster kernel: [762785.217658] [<ffffffff815a40b1>]
? is_mddev_idle+0xd1/0x140
Jun 23 08:25:08 monster kernel: [762785.217661] [<ffffffff815a7393>]
md_do_sync+0x993/0xdc0
Jun 23 08:25:08 monster kernel: [762785.217664] [<ffffffff810aaea0>]
? prepare_to_wait_event+0x100/0x100
Jun 23 08:25:08 monster kernel: [762785.217667] [<ffffffff815a3f48>]
md_thread+0x118/0x130
Jun 23 08:25:08 monster kernel: [762785.217670] [<ffffffff815a3e30>]
? mddev_unlock+0xe0/0xe0
Jun 23 08:25:08 monster kernel: [762785.217673] [<ffffffff8108b322>]
kthread+0xd2/0xf0
Jun 23 08:25:08 monster kernel: [762785.217676] [<ffffffff8108b250>]
? kthread_create_on_node+0x1d0/0x1d0
Jun 23 08:25:08 monster kernel: [762785.217678] [<ffffffff8172ab3c>]
ret_from_fork+0x7c/0xb0
Jun 23 08:25:08 monster kernel: [762785.217681] [<ffffffff8108b250>]
? kthread_create_on_node+0x1d0/0x1d0
Jun 23 08:25:08 monster kernel: [762785.217683] ---[ end trace
518bf06a77e3600a ]---
Jun 23 08:25:08 monster kernel: [762785.217694] ------------[ cut here
]------------
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: Crash flood fills syslog
2015-06-24 14:52 Crash flood fills syslog John Bridges
@ 2015-06-24 21:53 ` NeilBrown
0 siblings, 0 replies; 2+ messages in thread
From: NeilBrown @ 2015-06-24 21:53 UTC (permalink / raw)
To: John Bridges; +Cc: linux-raid
On Wed, 24 Jun 2015 10:52:50 -0400 John Bridges
<john.bridges@gmail.com> wrote:
> Messages like this flooded my syslog ballooning it up to 90GB, and
> causing other failures when the system drive had no free space left
> (bad).
>
> Distributor ID: Ubuntu
> Description: Ubuntu 14.04.2 LTS
> Release: 14.04
> Codename: trusty
A bit more context always helps...
Did you do something unusual, or did something unusual happen, shortly
before these started?
It looks like a faulty device was being "replaced" - e.g. via
mdadm --replace
is that correct?
How many devices in the array? RAID5 or RAID6? Was it degraded?
3.13.0 is a little bit old, the 3.13 series is up to 3.13.11 which has
at least one raid5 bugfix in it - it may affect this issue, but I'm not
certain. Is there a kernel upgrade available from ubuntu?
NeilBrown
>
>
> Jun 23 08:25:08 monster kernel: [762785.217575] ------------[ cut here
> ]------------
> Jun 23 08:25:08 monster kernel: [762785.217580] WARNING: CPU: 0 PID:
> 28173 at /build/buildd/linux-3.13.0/drivers/md/raid5.c:3856
> handle_stripe+0x24e4/0x2500 [raid456]()
> Jun 23 08:25:08 monster kernel: [762785.217581] Modules linked in:
> pci_stub vboxpci(OF) vboxnetadp(OF) vboxnetflt(OF) vboxdrv(OF)
> gpio_ich coretemp kvm_intel kvm serio_raw bnep rfcomm bluetooth
> nvidia(POF) i3000_edac parport_pc drm lpc_ich ppdev lp shpchp mac_hid
> edac_core parport xfs btrfs libcrc32c raid10 raid456 async_raid6_recov
> async_memcpy async_pq async_xor async_tx xor usb_storage pata_acpi
> ahci psmouse libahci e1000e raid6_pq ptp pps_core raid1 raid0
> multipath sata_mv floppy linear
> Jun 23 08:25:08 monster kernel: [762785.217612] CPU: 0 PID: 28173
> Comm: md2_resync Tainted: PF W O 3.13.0-29-generic #53-Ubuntu
> Jun 23 08:25:08 monster kernel: [762785.217613] Hardware name:
> Supermicro PDSM4+/PDSM4+, BIOS 6.00 11/04/2008
> Jun 23 08:25:08 monster kernel: [762785.217615] 0000000000000009
> ffff88017e997af0 ffffffff8171a214 0000000000000000
> Jun 23 08:25:08 monster kernel: [762785.217618] ffff88017e997b28
> ffffffff810676bd ffff88010d90a6b0 0000000000000009
> Jun 23 08:25:08 monster kernel: [762785.217621] 0000000000000009
> ffff8800369e8c00 000000000000000e ffff88017e997b38
> Jun 23 08:25:08 monster kernel: [762785.217624] Call Trace:
> Jun 23 08:25:08 monster kernel: [762785.217627] [<ffffffff8171a214>]
> dump_stack+0x45/0x56
> Jun 23 08:25:08 monster kernel: [762785.217630] [<ffffffff810676bd>]
> warn_slowpath_common+0x7d/0xa0
> Jun 23 08:25:08 monster kernel: [762785.217633] [<ffffffff8106779a>]
> warn_slowpath_null+0x1a/0x20
> Jun 23 08:25:08 monster kernel: [762785.217637] [<ffffffffa0108cd4>]
> handle_stripe+0x24e4/0x2500 [raid456]
> Jun 23 08:25:08 monster kernel: [762785.217640] [<ffffffff8109a7b2>]
> ? default_wake_function+0x12/0x20
> Jun 23 08:25:08 monster kernel: [762785.217644] [<ffffffff810aaeb2>]
> ? autoremove_wake_function+0x12/0x40
> Jun 23 08:25:08 monster kernel: [762785.217646] [<ffffffff810aa7b8>]
> ? __wake_up_common+0x58/0x90
> Jun 23 08:25:08 monster kernel: [762785.217651] [<ffffffffa0104c1e>]
> ? get_active_stripe+0x58e/0x7c0 [raid456]
> Jun 23 08:25:08 monster kernel: [762785.217656] [<ffffffffa0109b9c>]
> sync_request+0x17c/0x3e0 [raid456]
> Jun 23 08:25:08 monster kernel: [762785.217658] [<ffffffff815a40b1>]
> ? is_mddev_idle+0xd1/0x140
> Jun 23 08:25:08 monster kernel: [762785.217661] [<ffffffff815a7393>]
> md_do_sync+0x993/0xdc0
> Jun 23 08:25:08 monster kernel: [762785.217664] [<ffffffff810aaea0>]
> ? prepare_to_wait_event+0x100/0x100
> Jun 23 08:25:08 monster kernel: [762785.217667] [<ffffffff815a3f48>]
> md_thread+0x118/0x130
> Jun 23 08:25:08 monster kernel: [762785.217670] [<ffffffff815a3e30>]
> ? mddev_unlock+0xe0/0xe0
> Jun 23 08:25:08 monster kernel: [762785.217673] [<ffffffff8108b322>]
> kthread+0xd2/0xf0
> Jun 23 08:25:08 monster kernel: [762785.217676] [<ffffffff8108b250>]
> ? kthread_create_on_node+0x1d0/0x1d0
> Jun 23 08:25:08 monster kernel: [762785.217678] [<ffffffff8172ab3c>]
> ret_from_fork+0x7c/0xb0
> Jun 23 08:25:08 monster kernel: [762785.217681] [<ffffffff8108b250>]
> ? kthread_create_on_node+0x1d0/0x1d0
> Jun 23 08:25:08 monster kernel: [762785.217683] ---[ end trace
> 518bf06a77e3600a ]---
> Jun 23 08:25:08 monster kernel: [762785.217694] ------------[ cut here
> ]------------
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2015-06-24 21:53 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-06-24 14:52 Crash flood fills syslog John Bridges
2015-06-24 21:53 ` NeilBrown
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox