Re: [PATCH 0/3] Btrfs: fix free space tree bitmaps+tests on big-endian systems

[Chandan Rajendra <chandan@linux.vnet.ibm.com>]

On Monday, July 18, 2016 03:31:04 PM Omar Sandoval wrote:
> On Mon, Jul 18, 2016 at 02:43:26PM -0400, Chris Mason wrote:
> > 
> > 
> > On 07/17/2016 08:19 AM, Chandan Rajendra wrote:
> > > On Friday, July 15, 2016 12:15:15 PM Omar Sandoval wrote:
> > > > On Fri, Jul 15, 2016 at 12:34:10PM +0530, Chandan Rajendra wrote:
> > > > > On Thursday, July 14, 2016 07:47:04 PM Chris Mason wrote:
> > > > > > On 07/14/2016 07:31 PM, Omar Sandoval wrote:
> > > > > > > From: Omar Sandoval <osandov@fb.com>
> > > > > > > 
> > > > > > > So it turns out that the free space tree bitmap handling has always been
> > > > > > > broken on big-endian systems. Totally my bad.
> > > > > > > 
> > > > > > > Patch 1 fixes this. Technically, it's a disk format change for
> > > > > > > big-endian systems, but it never could have worked before, so I won't go
> > > > > > > through the trouble of any incompat bits. If you've somehow been using
> > > > > > > space_cache=v2 on a big-endian system (I doubt anyone is), you're going
> > > > > > > to want to mount with nospace_cache to clear it and wait for this to go
> > > > > > > in.
> > > > > > > 
> > > > > > > Patch 2 fixes a similar error in the sanity tests (it's the same as the
> > > > > > > v2 I posted here [1]) and patch 3 expands the sanity tests to catch the
> > > > > > > oversight that patch 1 fixes.
> > > > > > > 
> > > > > > > Applies to v4.7-rc7. No regressions in xfstests, and the sanity tests
> > > > > > > pass on x86_64 and MIPS.
> > > > > > 
> > > > > > Thanks for fixing this up Omar.  Any big endian friends want to try this
> > > > > > out in extended testing and make sure we've nailed it down?
> > > > > > 
> > > > > 
> > > > > Hi Omar & Chris,
> > > > > 
> > > > > I will run fstests with this patchset applied on ppc64 BE and inform you about
> > > > > the results.
> > > > > 
> > > > 
> > > > Thanks, Chandan! I set up my xfstests for space_cache=v2 by doing:
> > > > 
> > > >     mkfs.btrfs "$TEST_DEV"
> > > >     mount -o space_cache=v2 "$TEST_DEV" "$TEST_DIR"
> > > >     umount "$TEST_DEV"
> > > > 
> > > > and adding
> > > > 
> > > >     export MOUNT_OPTIONS="-o space_cache=v2"
> > > > 
> > > > to local.config. btrfsck also needs the patch here [1].
> > > > 
> > > > 
> > > 
> > > Hi,
> > > 
> > > I did execute the fstests tests suite on ppc64 BE as per above configuration
> > > and there were no new regressions. Also, I did execute fsx (via generic/127)
> > > thrice on the same filesystem instance,
> > > 1. With the unpatched kernel and later
> > > 2. With the patched kernel and again
> > > 3. With the unpatched kernel
> > > ... there were no new regressions when executing the above steps.
> > 
> > Thanks Chandan!  But I'm a little confused.  If the patch is helping, we
> > should be storing bitmaps wrong on disk unpatched.  There should be problems
> > going back and forth.
> > 
> > -chris
> 
> Yeah, this should definitely not work. It's possible that things are
> just silently failing and getting corrupted if the module isn't built
> with CONFIG_BTRFS_ASSERT, but btrfsck v4.6.1 + my patch should catch
> that.
> 
> Chandan, is fsx creating enough fragmentation to trigger the switch to
> bitmaps? You can check with `btrfs inspect dump-tree`; there should be
> FREE_SPACE_BITMAP items. If there are only FREE_SPACE_EXTENT items, then
> it's not testing the right code path.
> 
> I have a script here [1] that I've been using to test the free space
> tree. When I ran it with `--check` on MIPS, it failed on the old kernel
> and passed with this series. If you stick a return after the call to
> `unlink_every_other_file()`, you'll get a nice, fragmented filesystem to
> feed to xfstests, as well.

You are right, There were only FREE_SPACE_EXTENT items in the filesystem that
was operated on by fsx. I executed fragment_free_space_tree.py to create a
filesystem with FREE_SPACE_BITMAP items. When such a filesystem is created
with the unpatched kernel, later mounted on a patched kernel and fsx executed
on it, I see that we fail assertion statements in free-space-tree.c. For e.g.

BTRFS error (device loop0): incorrect extent count for 289406976; counted 8186, expected 8192
BTRFS: assertion failed: 0, file: /root/repos/linux/fs/btrfs/free-space-tree.c, line: 1485

-- 
chandan