* quota rescan hangs
@ 2015-12-31 10:09 Xavier Romero
2015-12-31 11:27 ` Xavier Romero
2015-12-31 13:10 ` Duncan
0 siblings, 2 replies; 3+ messages in thread
From: Xavier Romero @ 2015-12-31 10:09 UTC (permalink / raw)
To: linux-btrfs@vger.kernel.org
Hello,
Using BTRFS on CentOS 7, I get the filesystem hung by running
btrfs quota rescan /mnt/btrfs/
After that I could not access to the filesystem anymore until system restart.
I did the scan because BTRFS suggested it:
[root@nex-dstrg-ctrl-1 btrfs]# btrfs qgroup show -F /mnt/btrfs/
WARNING: Qgroup data inconsistent, rescan recommended
qgroupid rfer excl
-------- ---- ----
0/5 27.91GiB 27.91GiB
[root@nex-dstrg-ctrl-1 ~]# btrfs qgroup show -pcre /mnt/btrfs/
WARNING: Qgroup data inconsistent, rescan recommended
qgroupid rfer excl max_rfer max_excl parent child
-------- ---- ---- -------- -------- ------ -----
0/5 27.91GiB 27.91GiB 0.00B 0.00B --- ---
0/389 9.77GiB 9.77GiB 0.00B 0.00B --- ---
0/391 16.00KiB 16.00KiB 10.00GiB 0.00B --- ---
0/525 9.99GiB 0.00B 0.00B 0.00B --- ---
0/599 16.00KiB 16.00KiB 2.00GiB 0.00B --- ---
dmesg output:
[154385.628385] INFO: task btrfs:22324 blocked for more than 120 seconds.
[154385.629044] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[154385.629648] btrfs D ffff881fc3bdef20 0 22324 3677 0x00000080
[154385.629652] ffff88114d01bc60 0000000000000082 ffff880e2fecae00 ffff88114d01bfd8
[154385.629656] ffff88114d01bfd8 ffff88114d01bfd8 ffff880e2fecae00 ffff881fc3be1100
[154385.629658] ffff883fc826c9f0 ffff883fc826c9f0 0000000000000001 ffff881fc3bdef20
[154385.629661] Call Trace:
[154385.629673] [<ffffffff8163a889>] schedule+0x29/0x70
[154385.629690] [<ffffffffa0707117>] wait_current_trans.isra.20+0xe7/0x130 [btrfs]
[154385.629696] [<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30
[154385.629704] [<ffffffffa07089b8>] start_transaction+0x2b8/0x5a0 [btrfs]
[154385.629711] [<ffffffffa0708cf7>] btrfs_join_transaction+0x17/0x20 [btrfs]
[154385.629722] [<ffffffffa0771569>] btrfs_qgroup_rescan+0x39/0x90 [btrfs]
[154385.629731] [<ffffffffa0741d22>] btrfs_ioctl+0x20b2/0x2b70 [btrfs]
[154385.629736] [<ffffffff811d1552>] ? __mem_cgroup_commit_charge+0x152/0x390
[154385.629740] [<ffffffff81177d5e>] ? lru_cache_add+0xe/0x10
[154385.629745] [<ffffffff811a1af1>] ? page_add_new_anon_rmap+0x91/0x130
[154385.629750] [<ffffffff81197290>] ? handle_mm_fault+0x7c0/0xf50
[154385.629752] [<ffffffff8119bb98>] ? __vma_link_rb+0xb8/0xe0
[154385.629759] [<ffffffff811f1e85>] do_vfs_ioctl+0x2e5/0x4c0
[154385.629765] [<ffffffff8128bbfe>] ? file_has_perm+0xae/0xc0
[154385.629769] [<ffffffff81640d01>] ? __do_page_fault+0xb1/0x450
[154385.629771] [<ffffffff811f2101>] SyS_ioctl+0xa1/0xc0
[154385.629776] [<ffffffff816458c9>] system_call_fastpath+0x16/0x1b
Additional Info:
[root@nex-dstrg-ctrl-1 ~]# modinfo btrfs
filename: /lib/modules/3.10.0-327.3.1.el7.x86_64/kernel/fs/btrfs/btrfs.ko
license: GPL
alias: devname:btrfs-control
alias: char-major-10-234
alias: fs-btrfs
rhelversion: 7.2
srcversion: B92059408E7CB90AE2D9A2F
depends: raid6_pq,xor,zlib_deflate
intree: Y
vermagic: 3.10.0-327.3.1.el7.x86_64 SMP mod_unload modversions
signer: CentOS Linux kernel signing key
sig_key: 3D:4E:71:B0:42:9A:39:8B:8B:78:3B:6F:8B:ED:3B:AF:09:9E:E9:A7
sig_hashalgo: sha256
[root@nex-dstrg-ctrl-1 ~]# btrfs filesystem show /mnt/btrfs/
Label: none uuid: 6289b7b3-ba25-4c9e-af95-4a5fb18eeea1
Total devices 12 FS bytes used 29.86GiB
devid 1 size 894.13GiB used 4.52GiB path /dev/md101
devid 2 size 894.13GiB used 4.52GiB path /dev/md102
devid 3 size 894.13GiB used 4.52GiB path /dev/md103
devid 4 size 894.13GiB used 4.52GiB path /dev/md104
devid 5 size 894.13GiB used 4.52GiB path /dev/md105
devid 6 size 894.13GiB used 4.52GiB path /dev/md106
devid 7 size 894.13GiB used 4.52GiB path /dev/md107
devid 8 size 894.13GiB used 4.52GiB path /dev/md108
devid 9 size 894.13GiB used 4.52GiB path /dev/md109
devid 10 size 894.13GiB used 4.52GiB path /dev/md110
devid 11 size 894.13GiB used 4.52GiB path /dev/md111
devid 12 size 894.13GiB used 4.52GiB path /dev/md112
btrfs-progs v3.19.1
[root@nex-dstrg-ctrl-1 ~]# btrfs quota rescan -s /mnt/btrfs/
rescan operation running (current key 0)
[root@nex-dstrg-ctrl-1 ~]# btrfs subvolume list /mnt/btrfs/
ID 389 gen 10663 top level 5 path vol0
ID 391 gen 10514 top level 5 path vol1
ID 599 gen 10664 top level 5 path CLOUD_SSD_02
I'm just starting with BTRFS so I could be doing something wrong! Any ideas?
Best regards,
Xavier Romero
^ permalink raw reply [flat|nested] 3+ messages in thread
* RE: quota rescan hangs
2015-12-31 10:09 quota rescan hangs Xavier Romero
@ 2015-12-31 11:27 ` Xavier Romero
2015-12-31 13:10 ` Duncan
1 sibling, 0 replies; 3+ messages in thread
From: Xavier Romero @ 2015-12-31 11:27 UTC (permalink / raw)
To: linux-btrfs@vger.kernel.org
After restarting and removing all data, "qgroup show " tells me that a rescan is in progress, but "quota rescan -s" tells me the opposite.
[root@nex-dstrg-ctrl-1 btrfs]# btrfs quota rescan -w /mnt/btrfs/
quota rescan started
[root@nex-dstrg-ctrl-1 btrfs]# btrfs quota rescan -s /mnt/btrfs/
no rescan operation in progress
[root@nex-dstrg-ctrl-1 btrfs]# btrfs qgroup show -pcre /mnt/btrfs/
WARNING: Rescan is running, qgroup data may be incorrect
qgroupid rfer excl max_rfer max_excl parent child
-------- ---- ---- -------- -------- ------ -----
0/5 16.00KiB 16.00KiB 0.00B 0.00B --- ---
0/389 0.00B 0.00B 0.00B 0.00B --- ---
0/391 0.00B 0.00B 10.00GiB 0.00B --- ---
0/525 9.99GiB 0.00B 0.00B 0.00B --- ---
0/599 16.00KiB 16.00KiB 2.00GiB 0.00B --- ---
[root@nex-dstrg-ctrl-1 btrfs]# btrfs sub list /mnt/btrfs/
ID 599 gen 10754 top level 5 path CLOUD_SSD_02
Not sure how to proceed !
-----Mensaje original-----
De: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vger.kernel.org] En nombre de Xavier Romero
Enviado el: dijous, 31 de desembre de 2015 11:09
Para: linux-btrfs@vger.kernel.org
Asunto: quota rescan hangs
Hello,
Using BTRFS on CentOS 7, I get the filesystem hung by running btrfs quota rescan /mnt/btrfs/
After that I could not access to the filesystem anymore until system restart.
I did the scan because BTRFS suggested it:
[root@nex-dstrg-ctrl-1 btrfs]# btrfs qgroup show -F /mnt/btrfs/
WARNING: Qgroup data inconsistent, rescan recommended
qgroupid rfer excl
-------- ---- ----
0/5 27.91GiB 27.91GiB
[root@nex-dstrg-ctrl-1 ~]# btrfs qgroup show -pcre /mnt/btrfs/
WARNING: Qgroup data inconsistent, rescan recommended
qgroupid rfer excl max_rfer max_excl parent child
-------- ---- ---- -------- -------- ------ -----
0/5 27.91GiB 27.91GiB 0.00B 0.00B --- ---
0/389 9.77GiB 9.77GiB 0.00B 0.00B --- ---
0/391 16.00KiB 16.00KiB 10.00GiB 0.00B --- ---
0/525 9.99GiB 0.00B 0.00B 0.00B --- ---
0/599 16.00KiB 16.00KiB 2.00GiB 0.00B --- ---
dmesg output:
[154385.628385] INFO: task btrfs:22324 blocked for more than 120 seconds.
[154385.629044] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[154385.629648] btrfs D ffff881fc3bdef20 0 22324 3677 0x00000080
[154385.629652] ffff88114d01bc60 0000000000000082 ffff880e2fecae00 ffff88114d01bfd8 [154385.629656] ffff88114d01bfd8 ffff88114d01bfd8 ffff880e2fecae00 ffff881fc3be1100 [154385.629658] ffff883fc826c9f0 ffff883fc826c9f0 0000000000000001 ffff881fc3bdef20 [154385.629661] Call Trace:
[154385.629673] [<ffffffff8163a889>] schedule+0x29/0x70 [154385.629690] [<ffffffffa0707117>] wait_current_trans.isra.20+0xe7/0x130 [btrfs] [154385.629696] [<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30 [154385.629704] [<ffffffffa07089b8>] start_transaction+0x2b8/0x5a0 [btrfs] [154385.629711] [<ffffffffa0708cf7>] btrfs_join_transaction+0x17/0x20 [btrfs] [154385.629722] [<ffffffffa0771569>] btrfs_qgroup_rescan+0x39/0x90 [btrfs] [154385.629731] [<ffffffffa0741d22>] btrfs_ioctl+0x20b2/0x2b70 [btrfs] [154385.629736] [<ffffffff811d1552>] ? __mem_cgroup_commit_charge+0x152/0x390
[154385.629740] [<ffffffff81177d5e>] ? lru_cache_add+0xe/0x10 [154385.629745] [<ffffffff811a1af1>] ? page_add_new_anon_rmap+0x91/0x130 [154385.629750] [<ffffffff81197290>] ? handle_mm_fault+0x7c0/0xf50 [154385.629752] [<ffffffff8119bb98>] ? __vma_link_rb+0xb8/0xe0 [154385.629759] [<ffffffff811f1e85>] do_vfs_ioctl+0x2e5/0x4c0 [154385.629765] [<ffffffff8128bbfe>] ? file_has_perm+0xae/0xc0 [154385.629769] [<ffffffff81640d01>] ? __do_page_fault+0xb1/0x450 [154385.629771] [<ffffffff811f2101>] SyS_ioctl+0xa1/0xc0 [154385.629776] [<ffffffff816458c9>] system_call_fastpath+0x16/0x1b
Additional Info:
[root@nex-dstrg-ctrl-1 ~]# modinfo btrfs
filename: /lib/modules/3.10.0-327.3.1.el7.x86_64/kernel/fs/btrfs/btrfs.ko
license: GPL
alias: devname:btrfs-control
alias: char-major-10-234
alias: fs-btrfs
rhelversion: 7.2
srcversion: B92059408E7CB90AE2D9A2F
depends: raid6_pq,xor,zlib_deflate
intree: Y
vermagic: 3.10.0-327.3.1.el7.x86_64 SMP mod_unload modversions
signer: CentOS Linux kernel signing key
sig_key: 3D:4E:71:B0:42:9A:39:8B:8B:78:3B:6F:8B:ED:3B:AF:09:9E:E9:A7
sig_hashalgo: sha256
[root@nex-dstrg-ctrl-1 ~]# btrfs filesystem show /mnt/btrfs/
Label: none uuid: 6289b7b3-ba25-4c9e-af95-4a5fb18eeea1
Total devices 12 FS bytes used 29.86GiB
devid 1 size 894.13GiB used 4.52GiB path /dev/md101
devid 2 size 894.13GiB used 4.52GiB path /dev/md102
devid 3 size 894.13GiB used 4.52GiB path /dev/md103
devid 4 size 894.13GiB used 4.52GiB path /dev/md104
devid 5 size 894.13GiB used 4.52GiB path /dev/md105
devid 6 size 894.13GiB used 4.52GiB path /dev/md106
devid 7 size 894.13GiB used 4.52GiB path /dev/md107
devid 8 size 894.13GiB used 4.52GiB path /dev/md108
devid 9 size 894.13GiB used 4.52GiB path /dev/md109
devid 10 size 894.13GiB used 4.52GiB path /dev/md110
devid 11 size 894.13GiB used 4.52GiB path /dev/md111
devid 12 size 894.13GiB used 4.52GiB path /dev/md112
btrfs-progs v3.19.1
[root@nex-dstrg-ctrl-1 ~]# btrfs quota rescan -s /mnt/btrfs/ rescan operation running (current key 0)
[root@nex-dstrg-ctrl-1 ~]# btrfs subvolume list /mnt/btrfs/ ID 389 gen 10663 top level 5 path vol0 ID 391 gen 10514 top level 5 path vol1 ID 599 gen 10664 top level 5 path CLOUD_SSD_02
I'm just starting with BTRFS so I could be doing something wrong! Any ideas?
Best regards,
Xavier Romero
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: quota rescan hangs
2015-12-31 10:09 quota rescan hangs Xavier Romero
2015-12-31 11:27 ` Xavier Romero
@ 2015-12-31 13:10 ` Duncan
1 sibling, 0 replies; 3+ messages in thread
From: Duncan @ 2015-12-31 13:10 UTC (permalink / raw)
To: linux-btrfs
Xavier Romero posted on Thu, 31 Dec 2015 10:09:22 +0000 as excerpted:
> Using BTRFS on CentOS 7, I get the filesystem hung by running btrfs
> quota rescan /mnt/btrfs/
>
> After that I could not access to the filesystem anymore until system
> restart.
> Additional Info:
>
> [root@nex-dstrg-ctrl-1 ~]# modinfo btrfs
> filename:
> /lib/modules/3.10.0-327.3.1.el7.x86_64/kernel/fs/btrfs/btrfs.ko
> btrfs-progs v3.19.1
> I'm just starting with BTRFS so I could be doing something wrong! Any
> ideas?
[OK, the below lays it on kinda thick. I'm not saying your choice of
centos /or/ of btrfs is bad, only that if your interest is in something
so old and stal^hble as centos with its 3.10 kernel, then you really
should reconsider whether btrfs is an appropriate choice for you, because
it seems to be a seriously bad mismatch. The answer to your actual
question is below that discussion.]
First thing wrong, here's a quote from the Stability Status section right
on the front page of the btrfs wiki:
https://btrfs.wiki.kernel.org/index.php/Main_Page
>>>>>
The Btrfs code base is under heavy development. Every effort is being
made to keep it stable and fast. Due to the fast development speed, the
state of development of the filesystem improves noticeably with every new
Linux version, so it's recommended to run the most modern kernel possible.
<<<<<
And here's what the getting started page says:
https://btrfs.wiki.kernel.org/index.php/Getting_started
>>>>>
btrfs is a fast-moving target. There are typically a great many bug fixes
and enhancements between one kernel release and the next. Therefore:
If you have btrfs filesystems, run the latest kernel.
If you are running a kernel two or more versions behind the latest one
available from kernel.org, the first thing you will be asked to when you
report a problem is to upgrade to the latest kernel. Some distributions
keep backports of recent kernels to earlier releases -- see the page
below for details.
Having the latest user-space tools are also useful, as they contain
additional features and tools which may be of use in debugging or
recovering your filesystem if something goes wrong.
<<<<<
Centos, running kernel 3.10, is anything *BUT* "the latest kernel". With
five release cycles a year, 4.0 being 10 release cycles beyond 3.10, and
4.4 very near release, 3.10 is now nearing three years old!
Further, btrfs didn't even have the experimental sticker peeled off until
IIRC 3.12 or so, so that btrfs 3.10 isn't just nearly three years
outdated, it's also still experimental!
OK, so we know that the enterprise distros support btrfs and backport
stuff, but only they know what they backported, while we're focused on
the mainline kernel here on this list. So while the upstream btrfs and
list recommendation is keep current, you're running what for all we know
is a three year old experimental btrfs, with who knows what backports?
If you want support for that, you really should be asking the distro that
says they support it, not the upstream that says it's now ancient history
from when the filesystem was still experimental.
Meanwhile, from here, running the still under heavy development
"stabilizing but not yet entirely stable or mature" btrfs, on an
enterprise distro that runs years old versions... seems there's some sort
of bad-match incompatibility there. If your emphasis is that old and
stable, you really should reconsider whether the still under heavy
development btrfs is an appropriate choice for you, or if a filesystem
more suitably stable is more in keeping with your stability needs. One
or the other would seem to be the wrong choice, as they're at rather
opposite ends of the spectrum and don't work well together.
OK, on to the specific question. Tho the devs have been and are working
very hard on quotas, to date (4.3 release kernel) they've never worked
entirely correctly or reliably in btrfs, and my recommendation has always
been, if you're not working with the devs on the latest version to help
test, find and fix problems, which if you are, thanks, then you either
need quota functionality or you don't. Since quotas have never worked
reliably in btrfs, if you need that functionality, you really need to be
on a filesystem where it's much more stable and reliable than that
function has been on btrfs. OTOH, if you don't need quota functionality,
then I strongly recommend turning it off and leaving it off until at
least two kernel cycles have gone by with it working with no stability-
level issues.
Tho I'm not a dev, only a btrfs user and list regular, and my own use-
case doesn't need quotas, so given their problems I've kept them off, and
I'm not actually sure what the 4.4 status is. However, even if there's
no known problems with btrf quotas in 4.4, given the history, as I said
above, I strongly recommend not enabling them until at least two complete
kernel cycles have completed with no quota issues, which would mean even
if 4.4 and 4.5 are clear, I'd not actually trust the feature until 4.6
time. At that point, since 4.4 is to be an LTS kernel, you could
potentially enable them on 4.4 as an LTS kernel, as well as 4.6. But
that of course is assuming there's no known or new quota issues in 4.4 or
4.5, and we don't know that yet, so it could be well beyond 4.6 before
they're actually stable enough to depend on.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2015-12-31 13:10 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-12-31 10:09 quota rescan hangs Xavier Romero
2015-12-31 11:27 ` Xavier Romero
2015-12-31 13:10 ` Duncan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).