Re: system hangs due to qgroups

[Marc Joliet <marcec@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 4148 bytes --]

On Saturday 03 December 2016 13:42:42 Chris Murphy wrote:
> On Sat, Dec 3, 2016 at 11:40 AM, Marc Joliet <marcec@gmx.de> wrote:
> > Hello all,
> > 
> > I'm having some trouble with btrfs on a laptop, possibly due to qgroups.
> > Specifically, some file system activities (e.g., snapshot creation,
> > baloo_file_extractor from KDE Plasma) cause the system to hang for up to
> > about 40 minutes, maybe more.
> 
> Do you get any blocked tasks kernel messages? If so, issue sysrq+w
> during the hang, and then check the system log (dmesg may not contain
> everything if the command fills the message buffer). If it's a hang
> without any kernel messages, then issue sysrq+t.
> 
> https://www.kernel.org/doc/Documentation/sysrq.txt

As it's a rescue shell, I have only the one shell AFAIK, and it's occupied by 
mount.  So I can't tell if there are dmesg entries, however, when this happens 
during a normal running system, I never saw any dmesg entries.  Anyway, I ran 
both.

The output of sysrq+w mentions two tasks: "btrfs-transaction" with 
btrfs_scrub_pause+0xbe/0xd0 as the top-most entry in the call trace, and 
"mount" with its top-most entry at schedule+0x33/0x90 (it looks like it's 
still in the "early" processing, since there's also 
"btrfs_parse_early_options+0190/0x190" in the call trace).

The output of sysrq+t is too big to capture all of it (i.e., I can't scroll 
back to the beginning), but just looking at the task names that I *can* see, 
there are: btrfs-fixup, various btrfs-endio*, btrfs-rmw, btrfs-freespace, 
btrfs-delayed-m (cut off), btrfs-readahead, btrfs-qgroup-re (cut off), btrfs-
extent-re (cut off), btrfs-cleaner, and btrfs-transaction.  Oh, and a bunch of 
kworkers.

Should I take photos?  That'll be annoying to do with all the scrolling, but I 
can do that if need be.

> > After I next turned on the laptop, the balance resumed, causing bootup to
> > fail, after which I remembered about the skip_balance mount option, which
> > I
> > tried in a rescue shell from an initramfs.
> 
> The file system is the root filesystem? If so, skip_balance may not be
> happening soon enough. Use kernel parameter rootflags=skip_balance
> which will apply this mount option at the very first moment the file
> system is mounted during boot.

Yes, it's the root file system (there's that plus a swap partition).  I 
believe I tried rootflags, but I think it also failed, which is why I'm using 
a rescue shell now.  I can try it again, though, if anybody thinks that 
there's no point in waiting, especially if btrfs_scrub_pause in the btrfs-
transaction call trace is significant.

> > Since I couldn't use skip_balance, and logically can't destroy qgroups on
> > a
> > read-only file system, I decided to wait for a regular mount to finish. 
> > That has been running since Tuesday, and I am slowly growing impatient.
> Haha, no kidding! I think that's very patient.

Heh :) . I've still got my main desktop (as ancient as it may be), so I'm 
content with waiting for now, but I don't want to wait forever, especially if 
there might not even be a point.

> > Thus I arrive at my question(s): is there anything else I can try, short
> > of
> > reformatting and restoring from backup?  Can I use btrfs-check here, or
> > any
> > other tool?  Or...?
> 
> Yes, btrfs-progs 4.8.5 has the latest qgroup checks, so if there's
> something wrong it should find it and if not that's a bug of its own.

The initramfs has 4.8.4, but it looks like 4.8.5 was "only" an urgent bug fix, 
with no changes to qgroups handling, so I can use that, too.  Can it repair 
qgroups problems, too?

> > Also, should I be able to avoid reformatting: how do I properly disable
> > quota support?
> 
> 'btrfs quota disable' is the only command that applies to this and it
> requires rw mount; there's no 'noquota' mount option.

OK, thanks.

So what should I try next?  I'm sick at home, so I can spend more time on this 
than usual.

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 801 bytes --]