Btrfs progs release 6.1

Btrfs progs release 6.1

[David Sterba]

Hi,

btrfs-progs version 6.1 have been released.

There are new commands that are in the experimental mode until the interface
stabilizes, which means that it's a good time to give feedback or point out
deficiencies. There's no ETA set to remove the experimental status, I think and
hope that the feedback loop could work better if there's something tangible and
ready to test. The code is not the hard part unlike the interfaces and
formatting, this starts at the use case and user input is the important part.

The json output is also kind of experimental but there are tools that want it
now and can give feedback before there's some common format for all commands.

Relevant issues

* reflink group and commands: https://github.com/kdave/btrfs-progs/issues/396
* inspect list-chunks: https://github.com/kdave/btrfs-progs/issues/559
* all current experimental features: https://github.com/kdave/btrfs-progs/issues?q=is%3Aissue+is%3Aopen+label%3Aexperimental

Changelog:

   * filesystem df: add json output
   * qgroup show: add json output
   * new command: 'inspect-internal map-swapfile' to check swapfile and its
     swapfile_offset value used for hibernation
   * corrupt-block: fix parsing of option --root argument
   * experimental (interfaces not finalized):
      * new command 'inspect-internal list-chunks'
      * new group reflink, command clone
   * other:
      * synchronize some files with kernel versions
      * docs updates
      * build: use gnu11

Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git

Shortlog:

David Sterba (21):
      btrfs-progs: inspect: new subcommand to list chunks
      btrfs-progs: string-table: cleanup and enhance formatting capabilities
      btrfs-progs: string-table: check bounds before writing to a cell
      btrfs-progs: string-table: add ranged API for printing and clearing
      btrfs-progs: tests: add string-table test framework
      btrfs-progs: inspect: new command map-swapfile
      btrfs-progs: fi df: implement json output
      btrfs-progs: docs: swapfile and hibernation
      btrfs-progs: fi mkswapfile: update help text
      btrfs-progs: add new group reflink and command
      btrfs-progs: docs: fix typos
      btrfs-progs: docs: add 6.1 development statistics
      btrfs-progs: docs: updates, clarifications
      btrfs-progs: qgroup show: add json output
      btrfs-progs: add json formatter for escaped string
      btrfs-progs: use escaped json format for paths
      btrfs-progs: device stats: fix json formatter type for devid
      btrfs-progs: docs: typo fixups and formatting updates
      btrfs-progs: docs: add some kernel 6.1 release notes
      btrfs-progs: update CHANGES for 6.1
      Btrfs progs v6.1

Josef Bacik (21):
      btrfs-progs: build: turn on more compiler warnings and use -Wall
      btrfs-progs: fix make clean to clean convert properly
      btrfs-progs: build: use -std=gnu11
      btrfs-progs: ioctl: move btrfs_err_str into common/utils.h
      btrfs-progs: rename qgroup items to match the kernel naming scheme
      btrfs-progs: make btrfs_qgroup_level helper match the kernel
      btrfs-progs: ioctl: move dev-replace NO_RESULT definition into replace.c
      btrfs-progs: image: rename BLOCK_* to IMAGE_BLOCK_* for metadump
      btrfs-progs: rename btrfs_item_end to btrfs_item_data_end
      btrfs-progs: copy ioctl.h into libbtrfs
      btrfs-progs: stop using btrfs_root_item_v0
      btrfs-progs: make the find extent buffer helpers take fs_info
      btrfs-progs: move dirty eb tracking to it's own io_tree
      btrfs-progs: do not pass io_tree into verify_parent_transid
      btrfs-progs: move extent cache code directly into btrfs_fs_info
      btrfs-progs: delete state_private code
      btrfs-progs: rename extent buffer flags to EXTENT_BUFFER_*
      btrfs-progs: sync compression.h from the kernel
      btrfs-progs: replace btrfs_leaf_data with btrfs_item_nr_offset
      btrfs-progs: don't use btrfs_header_csum helper
      btrfs-progs: make write_extent_buffer take a const eb

Qu Wenruo (1):
      btrfs-progs: corrupt-block: fix the mismatch in --root and -r options

Re: Btrfs progs release 6.1

[waxhead]

I have nagged about this before, but here goes again.
Can we as users get some kind of deduplicaiton support for btrfs-progs?

As a conservative Debian user I tend to stick with what is in the 
package repos, and I also believe that most people will feel more 
comfortable with a "official" implementation rather than some random 
program for this kind of stuff.

something as "simple" as "btrfs filesystem deduplicate -R /mnt" would be 
wonderful.

So pretty please with sugar coated syrup with on top of nectar or 
something ...and since it is Christmas - please do consider adding some 
kind of deduplicaiton feature to btrfs progs in the near future.

PS! Merry Christmas to everyone! :)

David Sterba wrote:
> Hi,
> 
> btrfs-progs version 6.1 have been released.
> 
> There are new commands that are in the experimental mode until the interface
> stabilizes, which means that it's a good time to give feedback or point out
> deficiencies. There's no ETA set to remove the experimental status, I think and
> hope that the feedback loop could work better if there's something tangible and
> ready to test. The code is not the hard part unlike the interfaces and
> formatting, this starts at the use case and user input is the important part.
> 
> The json output is also kind of experimental but there are tools that want it
> now and can give feedback before there's some common format for all commands.
> 
> Relevant issues
> 
> * reflink group and commands: https://github.com/kdave/btrfs-progs/issues/396
> * inspect list-chunks: https://github.com/kdave/btrfs-progs/issues/559
> * all current experimental features: https://github.com/kdave/btrfs-progs/issues?q=is%3Aissue+is%3Aopen+label%3Aexperimental
> 
> Changelog:
> 
>     * filesystem df: add json output
>     * qgroup show: add json output
>     * new command: 'inspect-internal map-swapfile' to check swapfile and its
>       swapfile_offset value used for hibernation
>     * corrupt-block: fix parsing of option --root argument
>     * experimental (interfaces not finalized):
>        * new command 'inspect-internal list-chunks'
>        * new group reflink, command clone
>     * other:
>        * synchronize some files with kernel versions
>        * docs updates
>        * build: use gnu11
> 
> Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
> Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
> 
> Shortlog:
> 
> David Sterba (21):
>        btrfs-progs: inspect: new subcommand to list chunks
>        btrfs-progs: string-table: cleanup and enhance formatting capabilities
>        btrfs-progs: string-table: check bounds before writing to a cell
>        btrfs-progs: string-table: add ranged API for printing and clearing
>        btrfs-progs: tests: add string-table test framework
>        btrfs-progs: inspect: new command map-swapfile
>        btrfs-progs: fi df: implement json output
>        btrfs-progs: docs: swapfile and hibernation
>        btrfs-progs: fi mkswapfile: update help text
>        btrfs-progs: add new group reflink and command
>        btrfs-progs: docs: fix typos
>        btrfs-progs: docs: add 6.1 development statistics
>        btrfs-progs: docs: updates, clarifications
>        btrfs-progs: qgroup show: add json output
>        btrfs-progs: add json formatter for escaped string
>        btrfs-progs: use escaped json format for paths
>        btrfs-progs: device stats: fix json formatter type for devid
>        btrfs-progs: docs: typo fixups and formatting updates
>        btrfs-progs: docs: add some kernel 6.1 release notes
>        btrfs-progs: update CHANGES for 6.1
>        Btrfs progs v6.1
> 
> Josef Bacik (21):
>        btrfs-progs: build: turn on more compiler warnings and use -Wall
>        btrfs-progs: fix make clean to clean convert properly
>        btrfs-progs: build: use -std=gnu11
>        btrfs-progs: ioctl: move btrfs_err_str into common/utils.h
>        btrfs-progs: rename qgroup items to match the kernel naming scheme
>        btrfs-progs: make btrfs_qgroup_level helper match the kernel
>        btrfs-progs: ioctl: move dev-replace NO_RESULT definition into replace.c
>        btrfs-progs: image: rename BLOCK_* to IMAGE_BLOCK_* for metadump
>        btrfs-progs: rename btrfs_item_end to btrfs_item_data_end
>        btrfs-progs: copy ioctl.h into libbtrfs
>        btrfs-progs: stop using btrfs_root_item_v0
>        btrfs-progs: make the find extent buffer helpers take fs_info
>        btrfs-progs: move dirty eb tracking to it's own io_tree
>        btrfs-progs: do not pass io_tree into verify_parent_transid
>        btrfs-progs: move extent cache code directly into btrfs_fs_info
>        btrfs-progs: delete state_private code
>        btrfs-progs: rename extent buffer flags to EXTENT_BUFFER_*
>        btrfs-progs: sync compression.h from the kernel
>        btrfs-progs: replace btrfs_leaf_data with btrfs_item_nr_offset
>        btrfs-progs: don't use btrfs_header_csum helper
>        btrfs-progs: make write_extent_buffer take a const eb
> 
> Qu Wenruo (1):
>        btrfs-progs: corrupt-block: fix the mismatch in --root and -r options
> 

Re: Btrfs progs release 6.1

[David Sterba]

On Sat, Dec 24, 2022 at 11:20:20AM +0100, waxhead wrote:
> I have nagged about this before, but here goes again.
> Can we as users get some kind of deduplicaiton support for btrfs-progs?
> 
> As a conservative Debian user I tend to stick with what is in the 
> package repos, and I also believe that most people will feel more 
> comfortable with a "official" implementation rather than some random 
> program for this kind of stuff.
> 
> something as "simple" as "btrfs filesystem deduplicate -R /mnt" would be 
> wonderful.

Yes, this is planned, it would be convenient to quickly deduplicate a
few files or let it work on a directory. This would probably copy what
duperemove does and most likely also steal the implementation as it's in
C.

I'm not sure if btrfs command should provide exactly the same set of
features and options and duplicate the project, I think dupremove is not
currently maintained. I don't want to reimplement everything from
scratch so some level of sharing will make things easier, either direct
copy or as git submodule. It'll start as an experimental command.