Btrfs progs release 6.16.1

Btrfs progs release 6.16.1

[David Sterba]

Hi,

btrfs-progs version 6.16.1 have been released. This is a minor feature update
(mkfs.btrfs) and bugfix release.

Changelog:

* mkfs:
   * add option --reflink, when used with --rootdir clone file extents instead
     of copying (requires source and target image on the same filesystem)
   * improved tracking of inodes and subvolumes for option --inode-flags
   * fix initializing raid-stripe-tree
   * extend what is trimmed/discarded during initialization (temporary chunks,
     free space)
* check: detect duplicate file names in directory items
* inspect tree-stats: accept string names for option -t
* receive: allow to dump stream from different user
* other:
   * updated documentation
   * new and updated tests

Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
Release: https://github.com/kdave/btrfs-progs/releases/tag/v6.16.1

Re: Btrfs progs release 6.16.1

[Filipe Manana]

On Wed, Sep 10, 2025 at 7:15 PM David Sterba <dsterba@suse.com> wrote:
>
> Hi,
>
> btrfs-progs version 6.16.1 have been released. This is a minor feature update
> (mkfs.btrfs) and bugfix release.
>
> Changelog:
>
> * mkfs:
>    * add option --reflink, when used with --rootdir clone file extents instead
>      of copying (requires source and target image on the same filesystem)
>    * improved tracking of inodes and subvolumes for option --inode-flags
>    * fix initializing raid-stripe-tree
>    * extend what is trimmed/discarded during initialization (temporary chunks,
>      free space)
> * check: detect duplicate file names in directory items
> * inspect tree-stats: accept string names for option -t
> * receive: allow to dump stream from different user
> * other:
>    * updated documentation
>    * new and updated tests

With this btrfs-progs release, running 'btrfs check' fails on a
filesystem created by an older mkfs.btrfs.

For example, create an empty fs with 6.14:

$ mkfs.btrfs -f /dev/sdc
btrfs-progs v6.14
See https://btrfs.readthedocs.io for more information.

Performing full device TRIM /dev/sdc (100.00GiB) ...
NOTE: several default settings have changed in version 5.15, please make sure
      this does not affect your deployments:
      - DUP for metadata (-m dup)
      - enabled no-holes (-O no-holes)
      - enabled free-space-tree (-R free-space-tree)

Label:              (null)
UUID:               86a7103b-88eb-4a6a-b542-05523067d6f9
Node size:          16384
Sector size:        4096 (CPU page size: 4096)
Filesystem size:    100.00GiB
Block group profiles:
  Data:             single            8.00MiB
  Metadata:         DUP               1.00GiB
  System:           DUP               8.00MiB
SSD detected:       no
Zoned device:       no
Features:           extref, skinny-metadata, no-holes, free-space-tree
Checksum:           crc32c
Number of devices:  1
Devices:
   ID        SIZE  PATH
    1   100.00GiB  /dev/sdc

Now run 'btrfs check' from this release and it fails:

$ btrfs check /dev/sdc
Opening filesystem to check...
Checking filesystem on /dev/sdc
UUID: 86a7103b-88eb-4a6a-b542-05523067d6f9
[1/8] checking log skipped (none written)
[2/8] checking root items
[3/8] checking extents
[4/8] checking free space tree
We have a space info key for a block group that doesn't exist
[5/8] checking fs roots
[6/8] checking only csums items (without verifying data)
[7/8] checking root refs
[8/8] checking quota groups skipped (not enabled on this FS)
found 147456 bytes used, error(s) found
total csum bytes: 0
total tree bytes: 147456
total fs tree bytes: 32768
total extent tree bytes: 16384
btree space waste bytes: 140530
file data blocks allocated: 0
 referenced 0

A bisection points to:

commit e2cf6a03796b73d446b086022c0dfcf6a6552928
Author: Mark Harmstone <maharmstone@fb.com>
Date:   Fri Jul 18 15:26:27 2025 +0100

    btrfs-progs: use btrfs_lookup_block_group() in check_free_space_tree()




>
> Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
> Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
> Release: https://github.com/kdave/btrfs-progs/releases/tag/v6.16.1
>

Re: Btrfs progs release 6.16.1

[David Sterba]

On Wed, Sep 17, 2025 at 04:41:43PM +0100, Filipe Manana wrote:
> With this btrfs-progs release, running 'btrfs check' fails on a
> filesystem created by an older mkfs.btrfs.

> A bisection points to:
> 
> commit e2cf6a03796b73d446b086022c0dfcf6a6552928
> Author: Mark Harmstone <maharmstone@fb.com>
> Date:   Fri Jul 18 15:26:27 2025 +0100
> 
>     btrfs-progs: use btrfs_lookup_block_group() in check_free_space_tree()

Thanks for the report, I'll do a release with this patch reverted unless
there's an updated fix.

Re: Btrfs progs release 6.16.1

[Mark Harmstone]

On 18/09/2025 1.33 am, David Sterba wrote:
> On Wed, Sep 17, 2025 at 04:41:43PM +0100, Filipe Manana wrote:
>> With this btrfs-progs release, running 'btrfs check' fails on a
>> filesystem created by an older mkfs.btrfs.
> 
>> A bisection points to:
>>
>> commit e2cf6a03796b73d446b086022c0dfcf6a6552928
>> Author: Mark Harmstone <maharmstone@fb.com>
>> Date:   Fri Jul 18 15:26:27 2025 +0100
>>
>>      btrfs-progs: use btrfs_lookup_block_group() in check_free_space_tree()
> 
> Thanks for the report, I'll do a release with this patch reverted unless
> there's an updated fix.

The patch is correct: older versions of mkfs were creating spurious
free-space entries. btrfs-check was supposed to diagnose this but there
was a bug that meant it only triggered for entries at the end of the tree.

Re: Btrfs progs release 6.16.1

[Filipe Manana]

On Thu, Sep 18, 2025 at 6:10 AM Mark Harmstone <mark@harmstone.com> wrote:
>
> On 18/09/2025 1.33 am, David Sterba wrote:
> > On Wed, Sep 17, 2025 at 04:41:43PM +0100, Filipe Manana wrote:
> >> With this btrfs-progs release, running 'btrfs check' fails on a
> >> filesystem created by an older mkfs.btrfs.
> >
> >> A bisection points to:
> >>
> >> commit e2cf6a03796b73d446b086022c0dfcf6a6552928
> >> Author: Mark Harmstone <maharmstone@fb.com>
> >> Date:   Fri Jul 18 15:26:27 2025 +0100
> >>
> >>      btrfs-progs: use btrfs_lookup_block_group() in check_free_space_tree()
> >
> > Thanks for the report, I'll do a release with this patch reverted unless
> > there's an updated fix.
>
> The patch is correct: older versions of mkfs were creating spurious
> free-space entries. btrfs-check was supposed to diagnose this but there
> was a bug that meant it only triggered for entries at the end of the tree.

Sure, but now we have btrfs check failing for every fs created up to
btrfs-progs v16.0 since we have the free space tree feature.
That makes scripts fail since btrfs check returns a non-zero status
now for such fs.
Do we have any code to remove those unnecessary and harmless free space entries?

This will confuse and potentially scare some users, besides breaking
scripts that verify the result from btrfs check exit status.

Re: Btrfs progs release 6.16.1

[Mark Harmstone]

On 18/09/2025 1.17 pm, Filipe Manana wrote:
> On Thu, Sep 18, 2025 at 6:10 AM Mark Harmstone <mark@harmstone.com> wrote:
>>
>> On 18/09/2025 1.33 am, David Sterba wrote:
>>> On Wed, Sep 17, 2025 at 04:41:43PM +0100, Filipe Manana wrote:
>>>> With this btrfs-progs release, running 'btrfs check' fails on a
>>>> filesystem created by an older mkfs.btrfs.
>>>
>>>> A bisection points to:
>>>>
>>>> commit e2cf6a03796b73d446b086022c0dfcf6a6552928
>>>> Author: Mark Harmstone <maharmstone@fb.com>
>>>> Date:   Fri Jul 18 15:26:27 2025 +0100
>>>>
>>>>       btrfs-progs: use btrfs_lookup_block_group() in check_free_space_tree()
>>>
>>> Thanks for the report, I'll do a release with this patch reverted unless
>>> there's an updated fix.
>>
>> The patch is correct: older versions of mkfs were creating spurious
>> free-space entries. btrfs-check was supposed to diagnose this but there
>> was a bug that meant it only triggered for entries at the end of the tree.
> 
> Sure, but now we have btrfs check failing for every fs created up to
> btrfs-progs v16.0 since we have the free space tree feature.
> That makes scripts fail since btrfs check returns a non-zero status
> now for such fs.
> Do we have any code to remove those unnecessary and harmless free space entries?
> 
> This will confuse and potentially scare some users, besides breaking
> scripts that verify the result from btrfs check exit status.

To be clear, this patch does address an actual issue: it caused me problems
while developing btrfs-discard-check. Leo independently ran into it last year
too.

btrfs rescue clear-space-cache will get rid of the error, though it's
obviously a little heavy-handed. I don't know if btrfs check --repair fixes
the FST.

Maybe the thing to do is to add a compat_flag to say that the FST doesn't
contain spurious entries, and to clean these up and set the flag on mount
if it's not there - what do you think? Like we have for
BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID already, though compat rather
than compat_ro because the kernel won't create these entries itself.

Re: Btrfs progs release 6.16.1

[Chris Murphy]

I resolved it with a single command: 
mount -o clear_cache,space-cache=v2 /dev/dm-1 /mnt

clear_cache alone doesn't resolve it because a subsequent mount creates a v1 cache by default (kernel 6.17.0-0.rc6.49.fc43.x86_64)

My question is if it makes sense for the kernel to be capable of, by default, fixing problems like this? Future feature?

For now though, the `btrfs check` message lacks an indication of the (un)seriousness of the problem, or what the user should do about it. Maybe this message could be prefaced with INFO: ? to suggest it's just an informational level concern? 

The problem with the message and documentation is users are being advised to ask a btrfs developer or expert what to do about this message. So I kinda expect there will be messages on this list from users asking about it.

---
Chris Murphy

Re: Btrfs progs release 6.16.1

[Chris Murphy]

On Sun, Sep 21, 2025, at 2:25 PM, Chris Murphy wrote:

> clear_cache alone doesn't resolve it because a subsequent mount creates 
> a v1 cache by default (kernel 6.17.0-0.rc6.49.fc43.x86_64)

OK that is wrong. 

clear_cache clears the tree and recreates it. A v1 cache is not created.

If `btrfs rescue` is used to clear v1 and v2 caches, a mount without options does create a v1 cache.


-- 
Chris Murphy