Re: FIDEDUPERANGE woes may continue (or unrelated issue?)

[Zygo Blaxell <ce3g8jdj@umail.furryterror.org>]

On Tue, Mar 31, 2020 at 08:13:30AM +0000, halfdog wrote:
> halfdog writes:
> > Zygo Blaxell writes:
> >> ...
> >> I would try a mainline kernel just to make sure Debian didn't
> >> backport something they shouldn't have.
> >
> > OK, so let's go for that... If I got you right, you mentioned
> > two scenarios, that might yield relevant information:
> >
> > * Try a mainline kernel prior to "reloc_root" to see if the bug
> >   could already be reproduced with that one.
> > * Try a new 5.5.3 or later to see if the bug still can be reproduced.
> >
> > Which of both would be or higher value to you for the first test?
> >
> > Could you please share a kernel.org link to the exact tarball
> > that should be tested? If there is a specific kernel configuration
> > you deem superior for tests, that would be useful too. Otherwise
> > I would use one from a Debian package with a kernel version quite
> > close and adapt it to the given kernel.
> 
> Yesterday I started preparing test infrastructure and to see
> if my old test documentation still works with current software.
> I ran a modified trinity test on a 128MB btrfs loop mount. The
> test started at 12:02, at 14:30 trinity was OOM killed. As I
> did not monitor the virtual machine, over the next hours without
> trinity running any more also other processes were killed one
> after another until at 21:13 finally also init was killed.
> 
> As I run similar tests for many days on ext4 filesystems, could
> this be related to a btrfs memory leak even leaking just due
> to the btrfs kernel workers? 

How big is the test VM?  I run btrfs on machines as small as 512M,
but no smaller--and I don't try to do memory-hungry things like dedupe
or balance on such machines.  Some kernel debug options use a lot of
memory too, or break it up into pieces too small to use (e.g. KASAN and
the btrfs ref verifier).

> If so, when compiling a test kernel,
> is there any option you recommend setting to verify/rule out/
> pin-point btrfs leakage with trinity?

There is kmemleak.

You can also run 'slabtop' and just watch the numbers grow.
'slabtop' usually names the thing that is leaking.

If the thing you've got too much of is btrfs_delayed_ref_head,
you should definitely go back to 4.19 for now.

> > ...
>