* Re: Deadlock with nilfs on 2.6.31.4
2009-10-22 17:51 ` Ryusuke Konishi
@ 2009-10-22 20:19 ` Bruno Prémont
2009-11-02 17:05 ` Ryusuke Konishi
0 siblings, 1 reply; 5+ messages in thread
From: Bruno Prémont @ 2009-10-22 20:19 UTC (permalink / raw)
To: Ryusuke Konishi; +Cc: users, linux-fsdevel, ryusuke
On Fri, 23 October 2009 Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp> wrote:
> Thank you for reporting the issue.
>
> According to the log, the log-writer of nilfs looks to be idle even
> though it has some requests waiting.
>
> Could you try the following patch to narrow down the issue ?
>
> I'll dig into this issue next week since I'm now away from my office
> to attend the Linux symposium in Tokyo.
>
> Thank you,
> Ryusuke Konishi
>
>
> diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c
> index 51ff3d0..0932571 100644
> --- a/fs/nilfs2/segment.c
> +++ b/fs/nilfs2/segment.c
I tried the patch, below is full dmesg output from system start-up to
frozen syslog-ng (and collectd thread). (with echo t > /proc/sysrq-trigger)
Hard to tell at what time syslog-ng did freeze, but chances are big it's
somewhere between 435.x and 591.x when nilfs stops sending/getting events.
The collectd instance in D-state is most probably the one that wants to
write data to RRD file.
At least it looks very easy to reproduce! Just restarting collectd a few
times and enabling its rrdtool plugin. (syslog-ng writing to one nilfs
partition, collectd to another one, both on the same SD card)
Bruno
[ 0.000000] Linux version 2.6.31.4 (kbuild@neptune) (gcc version 4.3.2 (Gentoo 4.3.2-r4 p1.7, pie-10.1.5) ) #4 PREEMPT Fri Oct 16 11:38:54 CEST 2009
[ 0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053177
[ 0.000000] CPU: VIVT data cache, VIVT instruction cache
[ 0.000000] Machine: Marvell SheevaPlug Reference Board
[ 0.000000] Memory policy: ECC disabled, Data cache writeback
[ 0.000000] On node 0 totalpages: 131072
[ 0.000000] free_area_init_node: node 0, pgdat c03cbe70, node_mem_map c0473000
[ 0.000000] Normal zone: 1024 pages used for memmap
[ 0.000000] Normal zone: 0 pages reserved
[ 0.000000] Normal zone: 130048 pages, LIFO batch:31
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048
[ 0.000000] Kernel command line: console=ttyS0,115200 rw root=/dev/mtdblock2 rootfstype=jffs2
[ 0.000000] PID hash table entries: 2048 (order: 11, 8192 bytes)
[ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[ 0.000000] Memory: 256MB 256MB = 512MB total
[ 0.000000] Memory: 515072KB available (3516K code, 786K data, 104K init, 0K highmem)
[ 0.000000] SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS:114
[ 21.474936] Console: colour dummy device 80x30
[ 21.474962] Calibrating delay loop... 1192.75 BogoMIPS (lpj=5963776)
[ 21.714926] Mount-cache hash table entries: 512
[ 21.715205] CPU: Testing write buffer coherency: ok
[ 21.717409] NET: Registered protocol family 16
[ 21.719394] Kirkwood: MV88F6281-A0, TCLK=200000000.
[ 21.719406] Feroceon L2: Enabling L2
[ 21.719439] Feroceon L2: Cache support initialised.
[ 21.720481] initial MPP regs: 01111111 11113322 00001111 00100000 00000000 00000000 00000000
[ 21.720502] final MPP regs: 01111111 11113322 00001111 00000000 00000000 00000000 00000000
[ 21.728206] bio: create slab <bio-0> at 0
[ 21.728983] SCSI subsystem initialized
[ 21.729578] usbcore: registered new interface driver usbfs
[ 21.729698] usbcore: registered new interface driver hub
[ 21.729857] usbcore: registered new device driver usb
[ 21.732153] NET: Registered protocol family 2
[ 21.732389] IP route cache hash table entries: 16384 (order: 4, 65536 bytes)
[ 21.733107] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[ 21.734472] TCP bind hash table entries: 65536 (order: 6, 262144 bytes)
[ 21.734880] Switched to high resolution mode on CPU 0
[ 21.735205] TCP: Hash tables configured (established 65536 bind 65536)
[ 21.735214] TCP reno registered
[ 21.735357] NET: Registered protocol family 1
[ 21.747264] JFFS2 version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
[ 21.748189] msgmni has been set to 1006
[ 21.748297] io scheduler noop registered
[ 21.748305] io scheduler deadline registered
[ 21.748558] io scheduler cfq registered (default)
[ 21.758617] Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
[ 21.759320] serial8250.0: ttyS0 at MMIO 0xf1012000 (irq = 33) is a 16550A
[ 21.759341] console [ttyS0] enabled
[ 21.999285] loop: module loaded
[ 22.003405] MV-643xx 10/100/1000 ethernet driver version 1.4
[ 22.009717] mv643xx_eth smi: probed
[ 22.015138] net eth0: port 0 with MAC address 00:50:43:01:c8:6d
[ 22.021630] NAND device: Manufacturer ID: 0xad, Chip ID: 0xdc (Hynix NAND 512MiB 3,3V 8-bit)
[ 22.030136] Scanning device for bad blocks
[ 22.045763] Bad eraseblock 309 at 0x0000026a0000
[ 22.050436] Bad eraseblock 310 at 0x0000026c0000
[ 22.095214] Bad eraseblock 1392 at 0x00000ae00000
[ 22.127588] Bad eraseblock 2137 at 0x000010b20000
[ 22.169929] Bad eraseblock 3151 at 0x0000189e0000
[ 22.209673] Creating 3 MTD partitions on "orion_nand":
[ 22.214835] 0x000000000000-0x000000100000 : "u-boot"
[ 22.220580] 0x000000100000-0x000000500000 : "uImage"
[ 22.226172] 0x000000500000-0x000020000000 : "root"
[ 22.233100] aoe: AoE v47 initialised.
[ 22.236835] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 22.243422] orion-ehci orion-ehci.0: Marvell Orion EHCI
[ 22.248704] orion-ehci orion-ehci.0: new USB bus registered, assigned bus number 1
[ 22.284887] orion-ehci orion-ehci.0: irq 19, io mem 0xf1050000
[ 22.304878] orion-ehci orion-ehci.0: USB 2.0 started, EHCI 1.00
[ 22.311162] usb usb1: configuration #1 chosen from 1 choice
[ 22.316965] hub 1-0:1.0: USB hub found
[ 22.320751] hub 1-0:1.0: 1 port detected
[ 22.325069] Initializing USB Mass Storage driver...
[ 22.330114] usbcore: registered new interface driver usb-storage
[ 22.336172] USB Mass Storage support registered.
[ 22.340919] usbcore: registered new interface driver ums-datafab
[ 22.347095] usbcore: registered new interface driver ums-freecom
[ 22.353240] usbcore: registered new interface driver ums-jumpshot
[ 22.359490] usbcore: registered new interface driver ums-sddr09
[ 22.365566] usbcore: registered new interface driver ums-sddr55
[ 22.371813] mice: PS/2 mouse device common for all mice
[ 22.377289] rtc-mv rtc-mv: rtc core: registered rtc-mv as rtc0
[ 22.383250] i2c /dev entries driver
[ 22.387276] Orion Watchdog Timer: Initial timeout 21 sec
[ 22.392939] cpuidle: using governor ladder
[ 22.397473] cpuidle: using governor menu
[ 22.401630] sdhci: Secure Digital Host Controller Interface driver
[ 22.407872] sdhci: Copyright(c) Pierre Ossman
[ 22.412459] mmc0: mvsdio driver initialized, lacking card detect (fall back to polling)
[ 22.420858] Registered led device: plug:green:health
[ 22.426027] mv_xor_shared mv_xor_shared.0: Marvell shared XOR driver
[ 22.432428] mv_xor_shared mv_xor_shared.1: Marvell shared XOR driver
[ 22.474908] mv_xor mv_xor.0: Marvell XOR: ( xor cpy )
[ 22.486307] mmc0: host does not support reading read-only switch. assuming write-enable.
[ 22.494451] mmc0: new high speed SDHC card at address bc20
[ 22.500478] mmcblk0: mmc0:bc20 SD08G 7.60 GiB
[ 22.505090] mmcblk0:
[ 22.514915] mv_xor mv_xor.1: Marvell XOR: ( xor fill cpy )
[ 22.520902] p1 p2 p3
[ 22.554899] mv_xor mv_xor.2: Marvell XOR: ( xor cpy )
[ 22.594897] mv_xor mv_xor.3: Marvell XOR: ( xor fill cpy )
[ 22.603945] usbcore: registered new interface driver usbhid
[ 22.609578] usbhid: v2.6:USB HID core driver
[ 22.614782] TCP cubic registered
[ 22.619129] NET: Registered protocol family 10
[ 22.626052] NET: Registered protocol family 17
[ 22.631033] RPC: Registered udp transport module.
[ 22.635841] RPC: Registered tcp transport module.
[ 22.640569] Gating clock of unused units
[ 22.640576] before: 0x00df03dd
[ 22.640582] after: 0x00c701d9
[ 22.641264] rtc-mv rtc-mv: setting system clock to 2009-10-22 21:54:31 UTC (1256248471)
[ 36.375182] JFFS2 notice: (1) jffs2_build_xattr_subsystem: complete building xattr subsystem, 4 of xdatum (0 unchecked, 0 orphan) and 14 of xref (0 dead, 0 orphan) found.
[ 36.396916] VFS: Mounted root (jffs2 filesystem) on device 31:2.
[ 36.403038] Freeing init memory: 104K
[ 39.348950] udev: starting version 141
[ 42.664233] segctord starting. Construction interval = 5 seconds, CP frequency < 30 seconds
[ 42.664249] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 44.117775] segctord starting. Construction interval = 5 seconds, CP frequency < 30 seconds
[ 44.117791] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 44.152167] segctord starting. Construction interval = 5 seconds, CP frequency < 30 seconds
[ 44.152183] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 46.384346] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 48.047176] eth0: link up, 100 Mb/s, full duplex, flow control disabled
[ 48.047670] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 49.764909] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 49.773540] nilfs_segctor_notify: completed request from=0 to=0
[ 49.773610] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=0
[ 54.764886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 54.778199] nilfs_segctor_notify: completed request from=0 to=0
[ 54.778219] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 59.774888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 59.780404] nilfs_segctor_notify: completed request from=0 to=0
[ 59.780418] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 64.784887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 64.792656] nilfs_segctor_notify: completed request from=0 to=0
[ 64.792672] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 69.794886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 69.806849] nilfs_segctor_notify: completed request from=0 to=0
[ 69.806868] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 74.804887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 74.815413] nilfs_segctor_notify: completed request from=0 to=0
[ 74.815431] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 79.814881] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 79.819755] nilfs_segctor_notify: completed request from=0 to=0
[ 79.819769] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 84.824887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 84.831692] nilfs_segctor_notify: completed request from=0 to=0
[ 84.831708] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 89.834881] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 89.839619] nilfs_segctor_notify: completed request from=0 to=0
[ 89.839634] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 94.844884] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 94.851209] nilfs_segctor_notify: completed request from=0 to=0
[ 94.851225] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 99.854879] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 99.861902] nilfs_segctor_notify: completed request from=0 to=0
[ 99.861917] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 104.864886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 104.872528] nilfs_segctor_notify: completed request from=0 to=0
[ 104.872544] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 109.874882] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 109.880275] nilfs_segctor_notify: completed request from=0 to=0
[ 109.880290] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 114.884890] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 114.893995] nilfs_segctor_notify: completed request from=0 to=0
[ 114.894011] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 119.894888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 119.899362] nilfs_segctor_notify: completed request from=0 to=0
[ 119.899377] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 124.904889] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 124.911686] nilfs_segctor_notify: completed request from=0 to=0
[ 124.911701] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 129.914887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 129.920117] nilfs_segctor_notify: completed request from=0 to=0
[ 129.920132] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 134.924886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 134.932741] nilfs_segctor_notify: completed request from=0 to=0
[ 134.932756] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 139.934887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 139.940469] nilfs_segctor_notify: completed request from=0 to=0
[ 139.940484] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 144.944888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 144.952811] nilfs_segctor_notify: completed request from=0 to=0
[ 144.952827] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 149.954882] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 149.960334] nilfs_segctor_notify: completed request from=0 to=0
[ 149.960350] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 154.964883] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 154.972784] nilfs_segctor_notify: completed request from=0 to=0
[ 154.972800] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 159.974886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 159.979847] nilfs_segctor_notify: completed request from=0 to=0
[ 159.979862] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 164.984890] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 164.991647] nilfs_segctor_notify: completed request from=0 to=0
[ 164.991662] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 169.994885] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 170.001930] nilfs_segctor_notify: completed request from=0 to=0
[ 170.001946] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 175.004888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 175.009267] nilfs_segctor_notify: completed request from=0 to=0
[ 175.009282] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 180.014887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 180.019487] nilfs_segctor_notify: completed request from=0 to=0
[ 180.019502] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 185.024882] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 185.029440] nilfs_segctor_notify: completed request from=0 to=0
[ 185.029455] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 190.034889] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 190.040785] nilfs_segctor_notify: completed request from=0 to=0
[ 190.040801] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 195.044887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 195.049440] nilfs_segctor_notify: completed request from=0 to=0
[ 195.049456] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 200.054881] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 200.059330] nilfs_segctor_notify: completed request from=0 to=0
[ 200.059344] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 205.064886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 205.069514] nilfs_segctor_notify: completed request from=0 to=0
[ 205.069528] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 210.074881] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 210.079706] nilfs_segctor_notify: completed request from=0 to=0
[ 210.079721] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 215.084880] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 215.090995] nilfs_segctor_notify: completed request from=0 to=0
[ 215.091010] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 220.094886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 220.103336] nilfs_segctor_notify: completed request from=0 to=0
[ 220.103352] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 225.104886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 225.113401] nilfs_segctor_notify: completed request from=0 to=0
[ 225.113416] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 230.114885] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 230.119557] nilfs_segctor_notify: completed request from=0 to=0
[ 230.119572] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 235.124886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 235.129868] nilfs_segctor_notify: completed request from=0 to=0
[ 235.129883] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 240.134887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 240.139681] nilfs_segctor_notify: completed request from=0 to=0
[ 240.139697] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 245.144887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 245.149965] nilfs_segctor_notify: completed request from=0 to=0
[ 245.149980] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 250.154882] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 250.159893] nilfs_segctor_notify: completed request from=0 to=0
[ 250.159908] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 255.164881] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 255.170127] nilfs_segctor_notify: completed request from=0 to=0
[ 255.170142] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 260.174886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 260.180109] nilfs_segctor_notify: completed request from=0 to=0
[ 260.180124] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 265.184891] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 265.190234] nilfs_segctor_notify: completed request from=0 to=0
[ 265.190250] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 270.194887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 270.200385] nilfs_segctor_notify: completed request from=0 to=0
[ 270.200400] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 275.204883] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 275.210321] nilfs_segctor_notify: completed request from=0 to=0
[ 275.210336] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 280.214890] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 280.220521] nilfs_segctor_notify: completed request from=0 to=0
[ 280.220537] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 285.224880] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 285.229841] nilfs_segctor_notify: completed request from=0 to=0
[ 285.229855] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 290.234885] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 290.239923] nilfs_segctor_notify: completed request from=0 to=0
[ 290.239939] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 295.244887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 295.250140] nilfs_segctor_notify: completed request from=0 to=0
[ 295.250155] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 300.254887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 300.260324] nilfs_segctor_notify: completed request from=0 to=0
[ 300.260340] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 305.264882] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 305.270189] nilfs_segctor_notify: completed request from=0 to=0
[ 305.270204] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 310.274880] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 310.279312] nilfs_segctor_notify: completed request from=0 to=0
[ 310.279327] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 315.284890] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 315.289479] nilfs_segctor_notify: completed request from=0 to=0
[ 315.289495] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 320.294887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 320.299408] nilfs_segctor_notify: completed request from=0 to=0
[ 320.299423] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 325.304889] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 325.309601] nilfs_segctor_notify: completed request from=0 to=0
[ 325.309616] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 330.314888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 330.319765] nilfs_segctor_notify: completed request from=0 to=0
[ 330.319780] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 335.324887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 335.329891] nilfs_segctor_notify: completed request from=0 to=0
[ 335.329906] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 340.334887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 340.339880] nilfs_segctor_notify: completed request from=0 to=0
[ 340.339895] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 345.344888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 345.350064] nilfs_segctor_notify: completed request from=0 to=0
[ 345.350079] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 350.354881] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 350.360269] nilfs_segctor_notify: completed request from=0 to=0
[ 350.360285] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 355.364887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 355.370297] nilfs_segctor_notify: completed request from=0 to=0
[ 355.370313] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 360.374882] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 360.380317] nilfs_segctor_notify: completed request from=0 to=0
[ 360.380332] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 365.384887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 365.390516] nilfs_segctor_notify: completed request from=0 to=0
[ 365.390532] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 370.394881] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 370.400472] nilfs_segctor_notify: completed request from=0 to=0
[ 370.400488] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 375.404887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 375.409780] nilfs_segctor_notify: completed request from=0 to=0
[ 375.409795] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 380.414888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 380.419791] nilfs_segctor_notify: completed request from=0 to=0
[ 380.419806] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 385.424887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 385.430078] nilfs_segctor_notify: completed request from=0 to=0
[ 385.430093] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 390.434888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 390.444359] nilfs_segctor_notify: completed request from=0 to=0
[ 390.444374] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 395.444886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 395.449035] nilfs_segctor_notify: completed request from=0 to=0
[ 395.449050] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 400.454887] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 400.459393] nilfs_segctor_notify: completed request from=0 to=0
[ 400.459409] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 405.464881] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 405.469459] nilfs_segctor_notify: completed request from=0 to=0
[ 405.469474] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 410.474896] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 410.479610] nilfs_segctor_notify: completed request from=0 to=0
[ 410.479625] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 415.484885] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 415.489786] nilfs_segctor_notify: completed request from=0 to=0
[ 415.489802] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 420.494883] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 420.499985] nilfs_segctor_notify: completed request from=0 to=0
[ 420.500000] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 425.504885] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 425.509909] nilfs_segctor_notify: completed request from=0 to=0
[ 425.509924] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 430.514886] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 430.520159] nilfs_segctor_notify: completed request from=0 to=0
[ 430.520175] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 435.524888] nilfs_segctor_thread: sequence: req=0, done=0, state=4, timeout=1
[ 435.530061] nilfs_segctor_notify: completed request from=0 to=0
[ 435.530075] nilfs_segctor_thread: sequence: req=0, done=0, state=0, timeout=0
[ 591.025297] SysRq : Show State
[ 591.028379] task PC stack pid father
[ 591.028387] init S c02c8610 0 1 0 0x00000000
[ 591.028404] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c98c4>] (schedule_hrtimeout_range+0x114/0x188)
[ 591.028436] [<c02c98c4>] (schedule_hrtimeout_range+0x114/0x188) from [<c00a65b8>] (poll_schedule_timeout+0x38/0x58)
[ 591.028462] [<c00a65b8>] (poll_schedule_timeout+0x38/0x58) from [<c00a7110>] (do_select+0x52c/0x568)
[ 591.028482] [<c00a7110>] (do_select+0x52c/0x568) from [<c00a72ac>] (core_sys_select+0x160/0x318)
[ 591.028501] [<c00a72ac>] (core_sys_select+0x160/0x318) from [<c00a7558>] (sys_select+0xf4/0x1e0)
[ 591.028521] [<c00a7558>] (sys_select+0xf4/0x1e0) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.028543] kthreadd S c02c8610 0 2 0 0x00000000
[ 591.028556] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c004aa28>] (kthreadd+0x150/0x16c)
[ 591.028577] [<c004aa28>] (kthreadd+0x150/0x16c) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.028597] ksoftirqd/0 S c02c8610 0 3 2 0x00000000
[ 591.028609] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c003a120>] (ksoftirqd+0xf4/0x170)
[ 591.028632] [<c003a120>] (ksoftirqd+0xf4/0x170) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.028650] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.028668] watchdog/0 S c02c8610 0 4 2 0x00000000
[ 591.028681] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c00639a8>] (watchdog+0x68/0xac)
[ 591.028701] [<c00639a8>] (watchdog+0x68/0xac) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.028718] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.028736] events/0 S c02c8610 0 5 2 0x00000000
[ 591.028749] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.028772] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.028790] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.028808] khelper S c02c8610 0 6 2 0x00000000
[ 591.028821] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.028840] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.028858] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.028877] netns S c02c8610 0 9 2 0x00000000
[ 591.028889] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.028909] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.028927] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.028945] async/mgr S c02c8610 0 10 2 0x00000000
[ 591.028958] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c00517d0>] (async_manager_thread+0xb8/0x11c)
[ 591.028984] [<c00517d0>] (async_manager_thread+0xb8/0x11c) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029003] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029022] kblockd/0 S c02c8610 0 109 2 0x00000000
[ 591.029035] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.029056] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029074] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029092] ksuspend_usbd S c02c8610 0 117 2 0x00000000
[ 591.029105] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.029125] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029143] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029161] khubd S c02c8610 0 121 2 0x00000000
[ 591.029174] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c01be3b0>] (hub_thread+0x1050/0x11a0)
[ 591.029199] [<c01be3b0>] (hub_thread+0x1050/0x11a0) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029217] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029236] kmmcd S c02c8610 0 125 2 0x00000000
[ 591.029248] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.029269] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029287] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029305] khungtaskd S c02c8610 0 149 2 0x00000000
[ 591.029318] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c8e60>] (schedule_timeout+0x14c/0x1e8)
[ 591.029339] [<c02c8e60>] (schedule_timeout+0x14c/0x1e8) from [<c0063c08>] (watchdog+0x38/0x284)
[ 591.029359] [<c0063c08>] (watchdog+0x38/0x284) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029376] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029394] pdflush S c02c8610 0 150 2 0x00000000
[ 591.029406] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c007179c>] (pdflush+0xec/0x2cc)
[ 591.029426] [<c007179c>] (pdflush+0xec/0x2cc) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029443] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029461] pdflush S c02c8610 0 151 2 0x00000000
[ 591.029473] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c007179c>] (pdflush+0xec/0x2cc)
[ 591.029492] [<c007179c>] (pdflush+0xec/0x2cc) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029509] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029527] kswapd0 S c02c8610 0 152 2 0x00000000
[ 591.029539] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c00770c4>] (kswapd+0x534/0x55c)
[ 591.029561] [<c00770c4>] (kswapd+0x534/0x55c) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029577] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029596] aio/0 S c02c8610 0 198 2 0x00000000
[ 591.029608] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.029629] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029647] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029665] nfsiod S c02c8610 0 201 2 0x00000000
[ 591.029678] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.029698] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029716] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029734] mtdblockd S c02c8610 0 319 2 0x00000000
[ 591.029747] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c01accc8>] (mtd_blktrans_thread+0x20c/0x37c)
[ 591.029772] [<c01accc8>] (mtd_blktrans_thread+0x20c/0x37c) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029791] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029810] orion_spi S c02c8610 0 332 2 0x00000000
[ 591.029822] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.029843] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029862] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029880] mmcqd S c02c8610 0 375 2 0x00000000
[ 591.029893] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c01f09fc>] (mmc_queue_thread+0xe4/0x118)
[ 591.029915] [<c01f09fc>] (mmc_queue_thread+0xe4/0x118) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.029932] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.029950] usbhid_resume S c02c8610 0 412 2 0x00000000
[ 591.029963] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.029984] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.030002] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.030020] rpciod/0 S c02c8610 0 421 2 0x00000000
[ 591.030033] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0046dd0>] (worker_thread+0x200/0x218)
[ 591.030053] [<c0046dd0>] (worker_thread+0x200/0x218) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.030070] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.030088] jffs2_gcd_mtd S c02c8610 0 425 2 0x00000000
[ 591.030101] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c8e60>] (schedule_timeout+0x14c/0x1e8)
[ 591.030122] [<c02c8e60>] (schedule_timeout+0x14c/0x1e8) from [<c0127830>] (jffs2_garbage_collect_thread+0xb8/0x1d0)
[ 591.030146] [<c0127830>] (jffs2_garbage_collect_thread+0xb8/0x1d0) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.030167] udevd S c02c8610 0 535 1 0x00000000
[ 591.030180] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c9914>] (schedule_hrtimeout_range+0x164/0x188)
[ 591.030202] [<c02c9914>] (schedule_hrtimeout_range+0x164/0x188) from [<c00a65b8>] (poll_schedule_timeout+0x38/0x58)
[ 591.030225] [<c00a65b8>] (poll_schedule_timeout+0x38/0x58) from [<c00a69b0>] (do_sys_poll+0x334/0x440)
[ 591.030245] [<c00a69b0>] (do_sys_poll+0x334/0x440) from [<c00a6b1c>] (sys_poll+0x60/0xcc)
[ 591.030263] [<c00a6b1c>] (sys_poll+0x60/0xcc) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.030283] segctord S c02c8610 0 858 2 0x00000000
[ 591.030296] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<bf015518>] (nilfs_segctor_thread+0x2f4/0x330 [nilfs2])
[ 591.030400] [<bf015518>] (nilfs_segctor_thread+0x2f4/0x330 [nilfs2]) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.030481] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.030501] nilfs_cleaner S c02c8610 0 859 1 0x00000000
[ 591.030514] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c99f4>] (do_nanosleep+0xb0/0x110)
[ 591.030537] [<c02c99f4>] (do_nanosleep+0xb0/0x110) from [<c004eed0>] (hrtimer_nanosleep+0xa4/0x12c)
[ 591.030560] [<c004eed0>] (hrtimer_nanosleep+0xa4/0x12c) from [<c004eff4>] (sys_nanosleep+0x9c/0xa4)
[ 591.030580] [<c004eff4>] (sys_nanosleep+0x9c/0xa4) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.030599] segctord S c02c8610 0 861 2 0x00000000
[ 591.030612] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<bf015518>] (nilfs_segctor_thread+0x2f4/0x330 [nilfs2])
[ 591.030696] [<bf015518>] (nilfs_segctor_thread+0x2f4/0x330 [nilfs2]) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.030775] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.030795] nilfs_cleaner S c02c8610 0 862 1 0x00000000
[ 591.030808] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c99f4>] (do_nanosleep+0xb0/0x110)
[ 591.030831] [<c02c99f4>] (do_nanosleep+0xb0/0x110) from [<c004eed0>] (hrtimer_nanosleep+0xa4/0x12c)
[ 591.030853] [<c004eed0>] (hrtimer_nanosleep+0xa4/0x12c) from [<c004eff4>] (sys_nanosleep+0x9c/0xa4)
[ 591.030872] [<c004eff4>] (sys_nanosleep+0x9c/0xa4) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.030893] segctord S c02c8610 0 864 2 0x00000000
[ 591.030905] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<bf015518>] (nilfs_segctor_thread+0x2f4/0x330 [nilfs2])
[ 591.030988] [<bf015518>] (nilfs_segctor_thread+0x2f4/0x330 [nilfs2]) from [<c004aac0>] (kthread+0x7c/0x84)
[ 591.031067] [<c004aac0>] (kthread+0x7c/0x84) from [<c0023940>] (kernel_thread_exit+0x0/0x8)
[ 591.031087] nilfs_cleaner S c02c8610 0 865 1 0x00000000
[ 591.031100] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c99f4>] (do_nanosleep+0xb0/0x110)
[ 591.031122] [<c02c99f4>] (do_nanosleep+0xb0/0x110) from [<c004eed0>] (hrtimer_nanosleep+0xa4/0x12c)
[ 591.031144] [<c004eed0>] (hrtimer_nanosleep+0xa4/0x12c) from [<c004eff4>] (sys_nanosleep+0x9c/0xa4)
[ 591.031163] [<c004eff4>] (sys_nanosleep+0x9c/0xa4) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.031184] syslog-ng D c02c8610 0 1956 1 0x00000000
[ 591.031196] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c8e60>] (schedule_timeout+0x14c/0x1e8)
[ 591.031217] [<c02c8e60>] (schedule_timeout+0x14c/0x1e8) from [<c02c8344>] (io_schedule_timeout+0x34/0x58)
[ 591.031238] [<c02c8344>] (io_schedule_timeout+0x34/0x58) from [<c007caf0>] (congestion_wait+0x5c/0x80)
[ 591.031261] [<c007caf0>] (congestion_wait+0x5c/0x80) from [<c0070fb0>] (balance_dirty_pages_ratelimited_nr+0xe8/0x290)
[ 591.031283] [<c0070fb0>] (balance_dirty_pages_ratelimited_nr+0xe8/0x290) from [<c0069f04>] (generic_file_buffered_write+0x10c/0x348)
[ 591.031308] [<c0069f04>] (generic_file_buffered_write+0x10c/0x348) from [<c006a64c>] (__generic_file_aio_write_nolock+0x264/0x4f4)
[ 591.031331] [<c006a64c>] (__generic_file_aio_write_nolock+0x264/0x4f4) from [<c006b3b8>] (generic_file_aio_write+0x74/0xe8)
[ 591.031353] [<c006b3b8>] (generic_file_aio_write+0x74/0xe8) from [<c0097228>] (do_sync_write+0xbc/0x100)
[ 591.031376] [<c0097228>] (do_sync_write+0xbc/0x100) from [<c0097d3c>] (vfs_write+0xb0/0x164)
[ 591.031394] [<c0097d3c>] (vfs_write+0xb0/0x164) from [<c0097ec0>] (sys_write+0x40/0x70)
[ 591.031412] [<c0097ec0>] (sys_write+0x40/0x70) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.031432] ntpd S c02c8610 0 1986 1 0x00000000
[ 591.031445] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c98c4>] (schedule_hrtimeout_range+0x114/0x188)
[ 591.031467] [<c02c98c4>] (schedule_hrtimeout_range+0x114/0x188) from [<c00a65b8>] (poll_schedule_timeout+0x38/0x58)
[ 591.031491] [<c00a65b8>] (poll_schedule_timeout+0x38/0x58) from [<c00a69b0>] (do_sys_poll+0x334/0x440)
[ 591.031511] [<c00a69b0>] (do_sys_poll+0x334/0x440) from [<c00a6b1c>] (sys_poll+0x60/0xcc)
[ 591.031529] [<c00a6b1c>] (sys_poll+0x60/0xcc) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.031548] sshd S c02c8610 0 1990 1 0x00000000
[ 591.031561] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c9914>] (schedule_hrtimeout_range+0x164/0x188)
[ 591.031582] [<c02c9914>] (schedule_hrtimeout_range+0x164/0x188) from [<c00a65b8>] (poll_schedule_timeout+0x38/0x58)
[ 591.031604] [<c00a65b8>] (poll_schedule_timeout+0x38/0x58) from [<c00a7110>] (do_select+0x52c/0x568)
[ 591.031624] [<c00a7110>] (do_select+0x52c/0x568) from [<c00a72ac>] (core_sys_select+0x160/0x318)
[ 591.031643] [<c00a72ac>] (core_sys_select+0x160/0x318) from [<c00a7490>] (sys_select+0x2c/0x1e0)
[ 591.031662] [<c00a7490>] (sys_select+0x2c/0x1e0) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.031681] ntpd S c02c8610 0 1993 1 0x00000000
[ 591.031694] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c98c4>] (schedule_hrtimeout_range+0x114/0x188)
[ 591.031715] [<c02c98c4>] (schedule_hrtimeout_range+0x114/0x188) from [<c00a65b8>] (poll_schedule_timeout+0x38/0x58)
[ 591.031738] [<c00a65b8>] (poll_schedule_timeout+0x38/0x58) from [<c00a69b0>] (do_sys_poll+0x334/0x440)
[ 591.031757] [<c00a69b0>] (do_sys_poll+0x334/0x440) from [<c00a6b1c>] (sys_poll+0x60/0xcc)
[ 591.031776] [<c00a6b1c>] (sys_poll+0x60/0xcc) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.031795] agetty S c02c8610 0 2005 1 0x00000000
[ 591.031807] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c8e98>] (schedule_timeout+0x184/0x1e8)
[ 591.031827] [<c02c8e98>] (schedule_timeout+0x184/0x1e8) from [<c016dcdc>] (n_tty_read+0x4f0/0x748)
[ 591.031848] [<c016dcdc>] (n_tty_read+0x4f0/0x748) from [<c0169044>] (tty_read+0x8c/0xd0)
[ 591.031869] [<c0169044>] (tty_read+0x8c/0xd0) from [<c0097fa0>] (vfs_read+0xb0/0x164)
[ 591.031888] [<c0097fa0>] (vfs_read+0xb0/0x164) from [<c0098124>] (sys_read+0x40/0x70)
[ 591.031907] [<c0098124>] (sys_read+0x40/0x70) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.031926] sshd S c02c8610 0 2006 1990 0x00000000
[ 591.031938] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c9914>] (schedule_hrtimeout_range+0x164/0x188)
[ 591.031960] [<c02c9914>] (schedule_hrtimeout_range+0x164/0x188) from [<c00a65b8>] (poll_schedule_timeout+0x38/0x58)
[ 591.031983] [<c00a65b8>] (poll_schedule_timeout+0x38/0x58) from [<c00a7110>] (do_select+0x52c/0x568)
[ 591.032002] [<c00a7110>] (do_select+0x52c/0x568) from [<c00a72ac>] (core_sys_select+0x160/0x318)
[ 591.032021] [<c00a72ac>] (core_sys_select+0x160/0x318) from [<c00a7490>] (sys_select+0x2c/0x1e0)
[ 591.032040] [<c00a7490>] (sys_select+0x2c/0x1e0) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.032059] bash R running 0 2008 2006 0x00000000
[ 591.032071] [<c00280f8>] (unwind_backtrace+0x0/0xe4) from [<c0026b8c>] (show_stack+0x14/0x1c)
[ 591.032094] [<c0026b8c>] (show_stack+0x14/0x1c) from [<c002fc00>] (show_state_filter+0x68/0xc4)
[ 591.032118] [<c002fc00>] (show_state_filter+0x68/0xc4) from [<c01802ec>] (__handle_sysrq+0xbc/0x1b8)
[ 591.032139] [<c01802ec>] (__handle_sysrq+0xbc/0x1b8) from [<c0180420>] (write_sysrq_trigger+0x38/0x3c)
[ 591.032158] [<c0180420>] (write_sysrq_trigger+0x38/0x3c) from [<c00da040>] (proc_reg_write+0x88/0xcc)
[ 591.032183] [<c00da040>] (proc_reg_write+0x88/0xcc) from [<c0097d3c>] (vfs_write+0xb0/0x164)
[ 591.032203] [<c0097d3c>] (vfs_write+0xb0/0x164) from [<c0097ec0>] (sys_write+0x40/0x70)
[ 591.032221] [<c0097ec0>] (sys_write+0x40/0x70) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.032240] sshd S c02c8610 0 2013 1990 0x00000000
[ 591.032252] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c9914>] (schedule_hrtimeout_range+0x164/0x188)
[ 591.032275] [<c02c9914>] (schedule_hrtimeout_range+0x164/0x188) from [<c00a65b8>] (poll_schedule_timeout+0x38/0x58)
[ 591.032298] [<c00a65b8>] (poll_schedule_timeout+0x38/0x58) from [<c00a7110>] (do_select+0x52c/0x568)
[ 591.032318] [<c00a7110>] (do_select+0x52c/0x568) from [<c00a72ac>] (core_sys_select+0x160/0x318)
[ 591.032337] [<c00a72ac>] (core_sys_select+0x160/0x318) from [<c00a7490>] (sys_select+0x2c/0x1e0)
[ 591.032356] [<c00a7490>] (sys_select+0x2c/0x1e0) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.032376] bash S c02c8610 0 2015 2013 0x00000000
[ 591.032388] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c8e98>] (schedule_timeout+0x184/0x1e8)
[ 591.032409] [<c02c8e98>] (schedule_timeout+0x184/0x1e8) from [<c016dcdc>] (n_tty_read+0x4f0/0x748)
[ 591.032429] [<c016dcdc>] (n_tty_read+0x4f0/0x748) from [<c0169044>] (tty_read+0x8c/0xd0)
[ 591.032448] [<c0169044>] (tty_read+0x8c/0xd0) from [<c0097fa0>] (vfs_read+0xb0/0x164)
[ 591.032467] [<c0097fa0>] (vfs_read+0xb0/0x164) from [<c0098124>] (sys_read+0x40/0x70)
[ 591.032485] [<c0098124>] (sys_read+0x40/0x70) from [<c0022f00>] (ret_fast_syscall+0x0/0x2c)
[ 591.032504] collectd ? c02c8610 0 2114 1 0x00000001
[ 591.032517] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c0038028>] (do_exit+0x494/0x708)
[ 591.032539] [<c0038028>] (do_exit+0x494/0x708) from [<c00382d4>] (do_group_exit+0x38/0xe8)
[ 591.032556] [<c00382d4>] (do_group_exit+0x38/0xe8) from [<c0043bf0>] (get_signal_to_deliver+0x1dc/0x464)
[ 591.032578] [<c0043bf0>] (get_signal_to_deliver+0x1dc/0x464) from [<c00250cc>] (do_signal+0x50/0x50c)
[ 591.032599] [<c00250cc>] (do_signal+0x50/0x50c) from [<c0022f4c>] (work_pending+0x1c/0x20)
[ 591.032617] collectd D c02c8610 0 2115 1 0x00000001
[ 591.032630] [<c02c8610>] (schedule+0x2a8/0x3b0) from [<c02c8e60>] (schedule_timeout+0x14c/0x1e8)
[ 591.032651] [<c02c8e60>] (schedule_timeout+0x14c/0x1e8) from [<c02c8344>] (io_schedule_timeout+0x34/0x58)
[ 591.032672] [<c02c8344>] (io_schedule_timeout+0x34/0x58) from [<c007caf0>] (congestion_wait+0x5c/0x80)
[ 591.032694] [<c007caf0>] (congestion_wait+0x5c/0x80) from [<c0070fb0>] (balance_dirty_pages_ratelimited_nr+0xe8/0x290)
[ 591.032715] [<c0070fb0>] (balance_dirty_pages_ratelimited_nr+0xe8/0x290) from [<c007e6bc>] (do_wp_page+0x294/0x868)
[ 591.032737] [<c007e6bc>] (do_wp_page+0x294/0x868) from [<c007f2ec>] (handle_mm_fault+0x1e8/0x6e8)
[ 591.032758] [<c007f2ec>] (handle_mm_fault+0x1e8/0x6e8) from [<c0029310>] (do_page_fault+0x1a4/0x24c)
[ 591.032778] [<c0029310>] (do_page_fault+0x1a4/0x24c) from [<c0022294>] (do_DataAbort+0x30/0x94)
[ 591.032797] [<c0022294>] (do_DataAbort+0x30/0x94) from [<c0022e9c>] (ret_from_exception+0x0/0x10)
[ 591.032816] Exception stack(0xdf2c9fb0 to 0xdf2c9ff8)
[ 591.032822] 9fa0: 437cb788 401c5378 00000007 00000055
[ 591.032839] 9fc0: 0000001d 437cb788 00000000 00000000 40fc9d40 00000000 00000000 00000000
[ 591.032857] 9fe0: 437cb788 40fc9b38 401a7718 400c1f64 20000010 ffffffff
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread