From: Dave Chinner <david@fromorbit.com>
To: linux-btrfs@vger.kernel.org
Subject: [3.2-rc7] slowdown, warning + oops creating lots of files
Date: Thu, 5 Jan 2012 08:44:45 +1100 [thread overview]
Message-ID: <20120104214445.GE17026@dastard> (raw)
[-- Attachment #1: Type: text/plain, Size: 5739 bytes --]
Hi there buttery folks,
I just hit this warning and oops running a parallel fs_mark create
workload on a test VM using a 17TB btrfs filesystem (12 disk dm
RAID0) using default mkfs and mount parmeters, mounted on
/mnt/scratch. The VM has 8p and 4GB RAM, and the fs_mark command
line was:
$ ./fs_mark -D 10000 -S0 -n 100000 -s 0 -L 250 \
-d /mnt/scratch/0 -d /mnt/scratch/1 \
-d /mnt/scratch/2 -d /mnt/scratch/3 \
-d /mnt/scratch/4 -d /mnt/scratch/5 \
-d /mnt/scratch/6 -d /mnt/scratch/7
The attached image should give you a better idea of the performance
drop-off that was well under way when the crash occurred at about 96
million files created.
I'm rerunning the test on a fresh filesystem, so I guess I'll see if
this a one-off in the next couple of hours....
Cheers,
Dave.
[70972.788725] device fsid 5c8b0884-4e94-41ac-9a7e-8628a03a6f6d devid 1 transid 4 /dev/vdc
[70972.795441] btrfs: disk space caching is enabled
[121727.817810] ------------[ cut here ]------------
[121727.819430] WARNING: at fs/btrfs/extent-tree.c:4771 __btrfs_free_extent+0x630/0x6d0()
[121727.821729] Hardware name: Bochs
[121727.823417] Modules linked in:
[121727.824601] Pid: 2984, comm: btrfs-transacti Not tainted 3.2.0-rc7-dgc+ #167
[121727.826784] Call Trace:
[121727.827508] [<ffffffff8108a69f>] warn_slowpath_common+0x7f/0xc0
[121727.829290] [<ffffffff8108a6fa>] warn_slowpath_null+0x1a/0x20
[121727.830785] [<ffffffff815b0680>] __btrfs_free_extent+0x630/0x6d0
[121727.832486] [<ffffffff815ac110>] ? block_group_cache_tree_search+0x90/0xc0
[121727.833907] [<ffffffff815b42f1>] run_clustered_refs+0x381/0x800
[121727.834627] [<ffffffff815b483a>] btrfs_run_delayed_refs+0xca/0x220
[121727.836118] [<ffffffff815c5469>] btrfs_commit_transaction+0x359/0x840
[121727.837042] [<ffffffff810ac420>] ? add_wait_queue+0x60/0x60
[121727.838053] [<ffffffff815c5da4>] ? start_transaction+0x94/0x2b0
[121727.839133] [<ffffffff810986d0>] ? usleep_range+0x50/0x50
[121727.840642] [<ffffffff815be0d5>] transaction_kthread+0x265/0x280
[121727.842918] [<ffffffff815bde70>] ? btrfs_congested_fn+0xb0/0xb0
[121727.844480] [<ffffffff810ab97c>] kthread+0x8c/0xa0
[121727.845772] [<ffffffff81abb5f4>] kernel_thread_helper+0x4/0x10
[121727.847204] [<ffffffff810ab8f0>] ? flush_kthread_worker+0xa0/0xa0
[121727.848803] [<ffffffff81abb5f0>] ? gs_change+0x13/0x13
[121727.850073] ---[ end trace 804317a2ff1cf200 ]---
[121727.851207] btrfs unable to find ref byte nr 14368411648 parent 0 root 2 owner 0 offset 0
[121727.853324] BUG: unable to handle kernel NULL pointer dereference at (null)
[121727.855261] IP: [<ffffffff815e60f2>] map_private_extent_buffer+0x12/0x150
[121727.856730] PGD 0
[121727.857105] Oops: 0000 [#1] SMP
[121727.857243] CPU 1
[121727.857243] Modules linked in:
[121727.857243]
[121727.857243] Pid: 2984, comm: btrfs-transacti Tainted: G W 3.2.0-rc7-dgc+ #167 Bochs Bochs
[121727.857243] RIP: 0010:[<ffffffff815e60f2>] [<ffffffff815e60f2>] map_private_extent_buffer+0x12/0x150
[121727.857243] RSP: 0018:ffff880117b49b00 EFLAGS: 00010286
[121727.857243] RAX: 0000000000000000 RBX: 0000000000000065 RCX: ffff880117b49b30
[121727.857243] RDX: 0000000000000004 RSI: 000000000000007a RDI: 0000000000000000
[121727.857243] RBP: ffff880117b49b20 R08: ffff880117b49b38 R09: ffff880117b49b40
[121727.857243] R10: 0000000000000000 R11: 0000000000000001 R12: 000000000000007a
[121727.857243] R13: 0000000000000000 R14: 00000000ffffffe4 R15: 0000000000001000
[121727.857243] FS: 0000000000000000(0000) GS:ffff88011fc40000(0000) knlGS:0000000000000000
[121727.857243] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[121727.857243] CR2: 0000000000000000 CR3: 0000000001ec1000 CR4: 00000000000006e0
[121727.857243] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[121727.857243] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[121727.857243] Process btrfs-transacti (pid: 2984, threadinfo ffff880117b48000, task ffff88010f38e280)
[121727.857243] Stack:
[121727.857243] 00000000ffffffe4 0000000000000065 000000000000007a 0000000000000000
[121727.857243] ffff880117b49b70 ffffffff815db96c ffff880117b49b80 ffff880117b49b40
[121727.857243] 0000000000000002 00000003586c9000 0000000000000000 ffff8801042d97f0
[121727.857243] Call Trace:
[121727.857243] [<ffffffff815db96c>] btrfs_item_size+0x3c/0x90
[121727.857243] [<ffffffff815b0282>] __btrfs_free_extent+0x232/0x6d0
[121727.857243] [<ffffffff815ac110>] ? block_group_cache_tree_search+0x90/0xc0
[121727.857243] [<ffffffff815b42f1>] run_clustered_refs+0x381/0x800
[121727.857243] [<ffffffff815b483a>] btrfs_run_delayed_refs+0xca/0x220
[121727.857243] [<ffffffff815c5469>] btrfs_commit_transaction+0x359/0x840
[121727.857243] [<ffffffff810ac420>] ? add_wait_queue+0x60/0x60
[121727.857243] [<ffffffff815c5da4>] ? start_transaction+0x94/0x2b0
[121727.857243] [<ffffffff810986d0>] ? usleep_range+0x50/0x50
[121727.857243] [<ffffffff815be0d5>] transaction_kthread+0x265/0x280
[121727.857243] [<ffffffff815bde70>] ? btrfs_congested_fn+0xb0/0xb0
[121727.857243] [<ffffffff810ab97c>] kthread+0x8c/0xa0
[121727.857243] [<ffffffff81abb5f4>] kernel_thread_helper+0x4/0x10
[121727.857243] [<ffffffff810ab8f0>] ? flush_kthread_worker+0xa0/0xa0
[121727.857243] [<ffffffff81abb5f0>] ? gs_change+0x13/0x13
[121727.857243] Code: 83 c0 01 48 89 85 78 ff ff ff e9 c0 fc ff ff 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 55 41 54 53 48 83 ec 08 66 66 66 66 90 <4c> 8b 27 48 89
[121727.857243] RIP [<ffffffff815e60f2>] map_private_extent_buffer+0x12/0x150
[121727.857243] RSP <ffff880117b49b00>
[121727.857243] CR2: 0000000000000000
[121727.904461] ---[ end trace 804317a2ff1cf201 ]---
--
Dave Chinner
david@fromorbit.com
[-- Attachment #2: xfs-btrfs-ext4.png --]
[-- Type: image/png, Size: 44436 bytes --]
next reply other threads:[~2012-01-04 21:44 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-01-04 21:44 Dave Chinner [this message]
2012-01-04 22:11 ` [3.2-rc7] slowdown, warning + oops creating lots of files Dave Chinner
2012-01-04 22:23 ` Chris Samuel
2012-01-04 23:01 ` Dave Chinner
2012-01-05 2:23 ` Liu Bo
2012-01-05 2:26 ` Dave Chinner
2012-01-05 19:11 ` Liu Bo
2012-01-05 11:43 ` Dave Chinner
2012-01-05 18:46 ` Chris Mason
2012-01-05 19:45 ` Chris Mason
2012-01-05 20:12 ` Dave Chinner
2012-01-05 21:02 ` Chris Mason
2012-01-05 21:24 ` Chris Samuel
2012-01-06 1:22 ` Chris Mason
2012-01-07 21:34 ` Christian Brunner
2012-01-12 16:18 ` Christian Brunner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120104214445.GE17026@dastard \
--to=david@fromorbit.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).