From: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
To: Marcel Ritter <ritter.marcel@gmail.com>, <linux-btrfs@vger.kernel.org>
Subject: Re: Kernel bug in 3.19-rc4
Date: Fri, 16 Jan 2015 16:43:55 +0900 [thread overview]
Message-ID: <54B8C13B.7010204@jp.fujitsu.com> (raw)
In-Reply-To: <CADurGxwv9gyiA49dY12c5sZsZFc57BvfPbsXE5KRo-OU_J-iAg@mail.gmail.com>
Hi Marcel,
On 2015/01/16 4:46, Marcel Ritter wrote:
> Hi,
>
> I just started some btrfs stress testing on latest linux kernel 3.19-rc4:
> A few hours later, filesystem stopped working - the kernel bug report
> can be found below.
>
> The test consists of one massive IO thread (writing 100GB files with dd),
> and 2 tar instances extracting kernel sources and deleting them afterwards
> (I can provide the simple bash script doing this, if needed).
Could you give me this script?
Thanks,
Satoru
>
> System information (Ubuntu 14.04.1, latest kernel):
>
> root@thunder # uname -a
> Linux thunder 3.19.0-rc4-custom #1 SMP Mon Jan 12 16:13:44 CET 2015
> x86_64 x86_64 x86_64 GNU/Linux
>
> root@thunder # /root/btrfs-progs/btrfs --version
> Btrfs v3.18-36-g0173148
>
> Tests are done on 14 SCSI disks, using raid6 for data and metadata:
>
> root@thunder # /root/btrfs-progs/btrfs fi show
> Label: 'raid6' uuid: cbe34d2b-5f75-46cf-9263-9813028ebc19
> Total devices 14 FS bytes used 674.62GiB
> devid 1 size 279.39GiB used 59.24GiB path /dev/cciss/c1d0
> devid 2 size 279.39GiB used 59.22GiB path /dev/cciss/c1d1
> devid 3 size 279.39GiB used 59.22GiB path /dev/cciss/c1d10
> devid 4 size 279.39GiB used 59.22GiB path /dev/cciss/c1d11
> devid 5 size 279.39GiB used 59.22GiB path /dev/cciss/c1d12
> devid 6 size 279.39GiB used 59.22GiB path /dev/cciss/c1d13
> devid 7 size 279.39GiB used 59.22GiB path /dev/cciss/c1d2
> devid 8 size 279.39GiB used 59.22GiB path /dev/cciss/c1d3
> devid 9 size 279.39GiB used 59.22GiB path /dev/cciss/c1d4
> devid 10 size 279.39GiB used 59.22GiB path /dev/cciss/c1d5
> devid 11 size 279.39GiB used 59.22GiB path /dev/cciss/c1d6
> devid 12 size 279.39GiB used 59.22GiB path /dev/cciss/c1d7
> devid 13 size 279.39GiB used 59.22GiB path /dev/cciss/c1d8
> devid 14 size 279.39GiB used 59.22GiB path /dev/cciss/c1d9
>
> Btrfs v3.18-36-g0173148
>
> # This is provided for completeness only, and is taken
> # somewhen *before* the kernel crash occured, so basic
> # setup is the same, but allocated/free sizes won't match
> root@thunder # /root/btrfs-progs/btrfs fi df /tmp/m
> Data, single: total=8.00MiB, used=0.00B
> Data, RAID6: total=727.45GiB, used=697.84GiB
> System, single: total=4.00MiB, used=0.00B
> System, RAID6: total=13.50MiB, used=64.00KiB
> Metadata, single: total=8.00MiB, used=0.00B
> Metadata, RAID6: total=3.43GiB, used=805.91MiB
> GlobalReserve, single: total=272.00MiB, used=0.00B
>
>
> Here's what happens after some hours of stress testing:
>
> [85162.472989] ------------[ cut here ]------------
> [85162.473071] kernel BUG at fs/btrfs/inode.c:3142!
> [85162.473139] invalid opcode: 0000 [#1] SMP
> [85162.473212] Modules linked in: btrfs(E) xor(E) raid6_pq(E)
> radeon(E) ttm(E) drm_kms_helper(E) drm(E) hpwdt(E) amd64_edac_mod(E)
> kvm(E) edac_core(E) shpchp(E) k8temp(E) serio_raw(E) hpilo(E)
> edac_mce_amd(E) mac_hid(E) i2c_algo_bit(E) ipmi_si(E) nfsd(E)
> auth_rpcgss(E) nfs_acl(E) nfs(E) lockd(E) grace(E) sunrpc(E) lp(E)
> fscache(E) parport(E) hid_generic(E) usbhid(E) hid(E) hpsa(E)
> psmouse(E) bnx2(E) cciss(E) pata_acpi(E) pata_amd(E)
> [85162.473911] CPU: 4 PID: 3039 Comm: btrfs-cleaner Tainted: G
> E 3.19.0-rc4-custom #1
> [85162.474028] Hardware name: HP ProLiant DL585 G2 , BIOS A07 05/02/2011
> [85162.474122] task: ffff88085b054aa0 ti: ffff88205ad4c000 task.ti:
> ffff88205ad4c000
> [85162.474230] RIP: 0010:[<ffffffffa06a8182>] [<ffffffffa06a8182>]
> btrfs_orphan_add+0x1d2/0x1e0 [btrfs]
> [85162.474422] RSP: 0018:ffff88205ad4fc48 EFLAGS: 00010286
> [85162.474497] RAX: 00000000ffffffe4 RBX: ffff8810a35d42f8 RCX: ffff88185b896000
> [85162.474595] RDX: 0000000000006a54 RSI: 0000000000040000 RDI: ffff88185b896138
> [85162.474694] RBP: ffff88205ad4fc88 R08: 000000000001e670 R09: ffff88016194b240
> [85162.474793] R10: ffffffffa06bd797 R11: ffffea0004f71800 R12: ffff88185baa2000
> [85162.474892] R13: ffff88085f6d7630 R14: ffff88185baa2458 R15: 0000000000000001
> [85162.474992] FS: 00007fb3f27fb740(0000) GS:ffff88085fd00000(0000)
> knlGS:0000000000000000
> [85162.475105] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [85162.475184] CR2: 00007f896c02c220 CR3: 000000085b328000 CR4: 00000000000007e0
> [85162.475286] Stack:
> [85162.475318] ffff88205ad4fc88 ffffffffa06e6a14 ffff88185b896b04
> ffff88105b03e800
> [85162.475442] ffff88016194b240 ffff8810a35d42f8 ffff881e8ffe9a00
> ffff88133dc48ea0
> [85162.475561] ffff88205ad4fd18 ffffffffa0691a57 ffff88016194b244
> ffff88016194b240
> [85162.475680] Call Trace:
> [85162.475738] [<ffffffffa06e6a14>] ?
> lookup_free_space_inode+0x44/0x100 [btrfs]
> [85162.475849] [<ffffffffa0691a57>]
> btrfs_remove_block_group+0x137/0x740 [btrfs]
> [85162.475964] [<ffffffffa06ca8d2>] btrfs_remove_chunk+0x672/0x780 [btrfs]
> [85162.476065] [<ffffffffa06922bf>] btrfs_delete_unused_bgs+0x25f/0x280 [btrfs]
> [85162.476172] [<ffffffffa0699e0c>] cleaner_kthread+0x12c/0x190 [btrfs]
> [85162.476269] [<ffffffffa0699ce0>] ? check_leaf+0x350/0x350 [btrfs]
> [85162.476355] [<ffffffff8108f8d2>] kthread+0xd2/0xf0
> [85162.476424] [<ffffffff8108f800>] ? kthread_create_on_node+0x180/0x180
> [85162.476519] [<ffffffff8177bcbc>] ret_from_fork+0x7c/0xb0
> [85162.476592] [<ffffffff8108f800>] ? kthread_create_on_node+0x180/0x180
> [85162.476648] Code: ff ff 0f 1f 80 00 00 00 00 89 45 c8 f0 80 63 80
> fd 48 89 df e8 d0 23 fe ff 8b 45 c8 e9 14 ff ff ff b8 f4 ff ff ff e9
> 12 ff ff ff <0f> 0b 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90
> 55 48
> [85162.476648] RIP [<ffffffffa06a8182>] btrfs_orphan_add+0x1d2/0x1e0 [btrfs]
> [85162.476648] RSP <ffff88205ad4fc48>
> [85162.640076] ---[ end trace 396c6a6abc5a7fce ]---
>
> One reboot later, creating a new, clean filesystem and running the
> same tests again:
>
> [30204.556282] ------------[ cut here ]------------
> [30204.556358] kernel BUG at fs/btrfs/inode.c:3142!
> [30204.556422] invalid opcode: 0000 [#1] SMP
> [30204.556492] Modules linked in: btrfs(E) xor(E) radeon(E) ttm(E)
> drm_kms_helper(E) raid6_pq(E) drm(E) kvm(E) amd64_edac_mod(E)
> edac_core(E) i2c_algo_bit(E) edac_mce_amd(E) mac_hid(E) shpchp(E)
> serio_raw(E) k8temp(E) hpwdt(E) ipmi_si(E) hpilo(E) nfsd(E)
> auth_rpcgss(E) nfs_acl(E) nfs(E) lockd(E) grace(E) sunrpc(E)
> fscache(E) lp(E) parport(E) hpsa(E) hid_generic(E) usbhid(E) hid(E)
> pata_acpi(E) psmouse(E) bnx2(E) cciss(E) pata_amd(E)
> [30204.557194] CPU: 2 PID: 2186 Comm: btrfs-cleaner Tainted: G
> E 3.19.0-rc4-custom #1
> [30204.557313] Hardware name: HP ProLiant DL585 G2 , BIOS A07 05/02/2011
> [30204.557407] task: ffff88105b644aa0 ti: ffff88185c2b8000 task.ti:
> ffff88185c2b8000
> [30204.557510] RIP: 0010:[<ffffffffa0dab182>] [<ffffffffa0dab182>]
> btrfs_orphan_add+0x1d2/0x1e0 [btrfs]
> [30204.557687] RSP: 0018:ffff88185c2bbc48 EFLAGS: 00010286
> [30204.557762] RAX: 00000000ffffffe4 RBX: ffff881091e9fca0 RCX: ffff88205bb15000
> [30204.557860] RDX: 000000000000bd74 RSI: 0000000000040000 RDI: ffff88205bb15138
> [30204.557959] RBP: ffff88185c2bbc88 R08: 000000000001e670 R09: ffff8810a3c963f0
> [30204.558058] R10: ffffffffa0dc0797 R11: ffffea004255ba00 R12: ffff882059bc5000
> [30204.558157] R13: ffff8818588526e0 R14: ffff882059bc5458 R15: 0000000000000001
> [30204.558256] FS: 00007f34ad4b3840(0000) GS:ffff88185fc00000(0000)
> knlGS:0000000000000000
> [30204.558374] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [30204.558453] CR2: 00007fab99cbe700 CR3: 0000000fbd16c000 CR4: 00000000000007e0
> [30204.558552] Stack:
> [30204.558582] ffff88185c2bbc88 ffffffffa0de9a14 ffff88205bb15b04
> ffff88205c62e000
> [30204.558707] ffff8810a3c963f0 ffff881091e9fca0 ffff88105b02c000
> ffff8808b9130480
> [30204.558826] ffff88185c2bbd18 ffffffffa0d94a57 ffff8810a3c963f4
> ffff8810a3c963f0
> [30204.558945] Call Trace:
> [30204.558996] [<ffffffffa0de9a14>] ?
> lookup_free_space_inode+0x44/0x100 [btrfs]
> [30204.559102] [<ffffffffa0d94a57>]
> btrfs_remove_block_group+0x137/0x740 [btrfs]
> [30204.559210] [<ffffffffa0dcd8d2>] btrfs_remove_chunk+0x672/0x780 [btrfs]
> [30204.559306] [<ffffffffa0d952bf>] btrfs_delete_unused_bgs+0x25f/0x280 [btrfs]
> [30204.559408] [<ffffffffa0d9ce0c>] cleaner_kthread+0x12c/0x190 [btrfs]
> [30204.559501] [<ffffffffa0d9cce0>] ? check_leaf+0x350/0x350 [btrfs]
> [30204.559583] [<ffffffff8108f8d2>] kthread+0xd2/0xf0
> [30204.559649] [<ffffffff8108f800>] ? kthread_create_on_node+0x180/0x180
> [30204.559743] [<ffffffff8177bcbc>] ret_from_fork+0x7c/0xb0
> [30204.559816] [<ffffffff8108f800>] ? kthread_create_on_node+0x180/0x180
> [30204.559907] Code: ff ff 0f 1f 80 00 00 00 00 89 45 c8 f0 80 63 80
> fd 48 89 df e8 d0 23 fe ff 8b 45 c8 e9 14 ff ff ff b8 f4 ff ff ff e9
> 12 ff ff ff <0f> 0b 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90
> 55 48
> [30204.560138] RIP [<ffffffffa0dab182>] btrfs_orphan_add+0x1d2/0x1e0 [btrfs]
> [30204.560138] RSP <ffff88185c2bbc48>
> [30204.719832] ---[ end trace bbc20b459964e0ed ]---
>
> Maybe this helps to locate the error. If I can do more tests, or
> provide more necessary information to diagnose this, please let me
> know.
>
> Bye,
> Marcel
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
next prev parent reply other threads:[~2015-01-16 7:44 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-01-15 19:46 Kernel bug in 3.19-rc4 Marcel Ritter
2015-01-16 7:43 ` Satoru Takeuchi [this message]
-- strict thread matches above, loose matches on Subject: below --
2015-01-16 1:05 Tomasz Chmielewski
2015-01-16 7:33 ` Satoru Takeuchi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=54B8C13B.7010204@jp.fujitsu.com \
--to=takeuchi_satoru@jp.fujitsu.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=ritter.marcel@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.