Re: system hangs when running btrfs balance

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2270 bytes --]



On 2020/8/10 下午5:22, Johannes Rohr wrote:
> Dear devs,
> 
> since I upgraded our system from Ubuntu 18.04 LTS to 20.04 LTS, the file
> system completely freezes when I run a btrfs balance on it. The only way
> to get a usable system for the time being is with the mount option
> "skip_balance".
> 
> The server has a raid1 with 4 SSDs with 500 GB each. 
> [Sun Aug  9 12:21:35 2020] CPU: 1 PID: 4537 Comm: btrfs-balance Tainted: G           O      5.4.47 #1

A quick git log glance shows that, some reloc tree related fixes haven't
landed in v5.4.47.

E.g. (commits are upstream commits, not stable tree commits)\

1dae7e0e58b484eaa43d530f211098fdeeb0f404 btrfs: reloc: clear
DEAD_RELOC_TREE bit for orphan roots to prevent runaway balance
51415b6c1b117e223bc083e30af675cb5c5498f3 btrfs: reloc: fix reloc root
leak and NULL pointer dereference.

And above fixes only landed in v5.4.54, so I guess you have to update
your kernel anyway.

> There has been a related bug report at kernel.org for a year,
> https://bugzilla.kernel.org/show_bug.cgi?id=203405 and I have found
> similar reports here and there, some pertaining to quite old kernel
> versions, but we have only been hit with kernel 5.4. After this first
> occurred, I had no better luck though, with older kernels (4 something
> from Debian buster, also 4 something from Ubuntu 18.04).

Nope, the mentioned one is another bug, we had some clue on this, but
need some time to solve it.
(It's mostly related to some special timing in canceling, leading to
parted dropped trees).

> 
> Apart from fixing the underlying issue, would there be any wordaround
> for it?

Update your kernel to at least v5.4.54, then mount with skip_balance and
finally  "btrfs balance cancel <mnt>".
After that, doing whatever you like should be fine.

I prefer to do a btrfs check on the unmounted or at least ro mounted fs
to ensure your fs is sane in the first place.

Thanks,
Qu

> Currently the balance for the fs is in suspended status. Since
> there is quite a few people who depend on this server, I can't just play
> around with it at random. That's why I am asking for advice here...
> 
> Thanks so much for any suggestions you might have!
> 
> Johannes
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]