Re: fstests btrfs/042 triggle 'qgroup reserved space leaked'

[Wang Yugui <wangyugui@e16-tech.com>]

Hi,

> 
> On 2022/9/24 12:07, Wang Yugui wrote:
> > Hi,
> >
> >> On 2022/9/24 10:11, Wang Yugui wrote:
> >>> Hi,
> >>>
> >>>>
> >>>> On 2022/9/24 07:43, Wang Yugui wrote:
> >>>>> Hi,
> >>>>>
> >>>>> fstests btrfs/042 triggle 'qgroup reserved space leaked'
> >>>>>
> >>>>> kernel source: btrfs misc-next
> >>>>
> >>>> Which commit HEAD?
> >>>>
> >>>> As I can not reproduce using a somewhat older misc-next.
> >>>>
> >>>> The HEAD I'm on is 2d1aef6504bf8bdd7b6ca9fa4c0c5ab32f4da2a8 ("btrfs: stop tracking failed reads in the I/O tree").
> >>>>
> >>>> If it's a regression it can be much easier to pin down.
> >>>>
> >>>>> kernel config:
> >>>>> 	memory debug: CONFIG_KASAN/CONFIG_DEBUG_KMEMLEAK/...
> >>>>> 	lock debug: CONFIG_PROVE_LOCKING/...
> >>>>
> >>>> And any reproducibility? 16 runs no reproduce.
> >>>
> >>> btrfs source version: misc-next: bf940dd88f48,
> >>> 	plus some minor local patch(no qgroup related)
> >>> kernel: 6.0-rc6
> >>>
> >>> reproduce rate:
> >>> 1) 100%(3/3) when local debug config **1
> >>> 2)  0% (0/3) when local release config
> >>>
> >>> **1:local debug config, about 100x slow than release config
> >>> a) memory debug
> >>> 	CONFIG_KASAN/CONFIG_DEBUG_KMEMLEAK/...
> >>> b) lockdep debug
> >>> 	CONFIG_PROVE_LOCKING/...
> >>> c) btrfs debug
> >>> CONFIG_BTRFS_FS_CHECK_INTEGRITY=y
> >>> CONFIG_BTRFS_FS_RUN_SANITY_TESTS=y
> >>> CONFIG_BTRFS_DEBUG=y
> >>> CONFIG_BTRFS_ASSERT=y
> >>> CONFIG_BTRFS_FS_REF_VERIFY=y
> >>
> >> I always run with all btrfs features enabled.
> >>
> >> So is the lockdep.
> >>
> >> KASAN is known to be slow, thus that is only enabled when there is suspision on memory corruption caused by some wild pointer.
> >>
> >>>
> >>>
> >>>   From source:
> >>> fs/btrfs/disk-io.c:4668
> >>>       if (btrfs_check_quota_leak(fs_info)) {
> >>> L4668        WARN_ON(IS_ENABLED(CONFIG_BTRFS_DEBUG));
> >>>           btrfs_err(fs_info, "qgroup reserved space leaked");
> >>>       }
> >>>
> >>> This problem will triggle fstests btrfs/042 to failure only when
> >>> CONFIG_BTRFS_DEBUG=y ?
> >>>
> >>>
> >>> maybe related issue:
> >>> when lockdep debug is enabled, the following issue become very easy to
> >>> reproduce too.
> >>> https://lore.kernel.org/linux-nfs/3E21DFEA-8DF7-484B-8122-D578BFF7F9E0@oracle.com/
> >>> so there maybe some lockdep debug related , but not btrfs related
> >>> problem in kernel 6.0.
> >>>
> >>>
> >>> more test(remove some minor local patch(no qgroup related)) will be done,
> >>> and then I will report the result.
> >>
> >> Better to provide the patches, as I just finished a 16 runs of btrfs/042, no reproduce.
> >>
> >> Thus I'm starting to suspect the off-tree patches.
> >
> > This problem happen on linux 6.0-rc6+ (master a63f2e7cb110, without
> > btrfs misc-next patch, without local off-tree patch)
> 
> Same base, still nope.
> 
> > so this problem is not related to the patches still in btrfs misc-next.
> >
> > reproduce rate:
> > 100%(3/3) when local debug config
> > and the whole config file is attached.
> >
> 
> I don't think the config makes much difference, as the main difference
> is in KASAN and KMEMLEAK, which should not impact the test result.
> 
> And are you running just that test, or with the full auto group?

For 6.0-rc6 with btrfs misc-next, I tried to run  full auto group.
btrfs/042 failed, others(btrfs/001 ~ btrfs/157) are OK, and then I
rebooted the test machine.

for 6.0-rc6 without btrfs misc-next, I tested btrfs/042 and btrfs/001 on
the same machine.
then I tested 6.0-rc6 without btrfs misc-next on another 2 servers.

reproduce rate:
server1	3/3
server2	2/3
server3	3/3
total rate: 8/9

all 3 servers are in good status(ECC memory status and SSD status).

Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2022/09/24