Re: when does btrfs create sparse extents?

[Filipe Manana <fdmanana@gmail.com>]

On Thu, Apr 23, 2020 at 12:42 PM Marek Behun <marek.behun@nic.cz> wrote:
>
> On Thu, 23 Apr 2020 11:49:16 +0100
> Filipe Manana <fdmanana@gmail.com> wrote:
>
> > On Wed, Apr 22, 2020 at 10:00 PM Marek Behun <marek.behun@nic.cz> wrote:
> > >
> > > On Wed, 22 Apr 2020 14:44:46 -0600
> > > Chris Murphy <lists@colorremedies.com> wrote:
> > >
> > > > e.g. from a 10m file created with truncate on two Btrfs file systems
> > > >
> > > > original holes format (default)
> > > >
> > > >     item 6 key (257 EXTENT_DATA 0) itemoff 15768 itemsize 53
> > > >         generation 7412 type 1 (regular)
> > > >         extent data disk byte 0 nr 0
> > > >         extent data offset 0 nr 10485760 ram 10485760
> > > >         extent compression 0 (none)
> > > >
> > > > On a file system with no-holes feature set, this item simply doesn't
> > > > exist. I think basically it works by inference. Both kinds of files
> > > > have size in the INODE_ITEM, e.g.
> > > >
> > > >     item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160
> > > >         generation 889509 transid 889509 size 10485760 nbytes 0
> > > >
> > > > Sparse extents are explicitly stated in the original format with disk
> > > > byte 0 in an EXTENT_DATA item; whereas in the newer format, sparse
> > > > extents exist whenever EXTENT_DATA items don't completely describe the
> > > > file's size.
> > >
> > > Ok this means that U-Boot currently gained support for the original
> > > sparse extents.
> >
> > To clear any confusion, what you mean by sparse extents is actually holes.
> > The concept of sparse files exists (files with holes, regions of a
> > file for which there is no allocated extent), but not sparse extents.
> >
> > >
> > > I fear that current u-boot does not handle the new no-holes feature.
> >
> > The no-holes feature has been around since 2013, not exactly new, but
> > it's not the default yet when creating a new filesystem.
> >
> > As it has been mentioned earlier by Chris, it just removes the need
> > for explicitly having metadata representing holes.
> > When not using the no-holes feature, there is an explicit file extent
> > item pointing to a disk location of 0 (disk_bytenr field has a value
> > of 0) for each file hole.
> > When using no-holes, there's no such file extent item - btrfs knows
> > about the hole by checking that there is a gap between two consecutive
> > file extent items (both having a disk_bytenr > 0).
>
> This I already understand. My main question though is: does kernel or
> btrfs do checking (at least sometimes) when writing a block of data onto
> disk if this block is all zero, and if yes, then this block is written
> as a hole (either by writing hole item or not writing anything)?
>
> Or does this happen ONLY when requested by userspace?

There's nothing in btrfs that converts a sequence of zeroes
automatically to a hole.

It always has to be done by user space, either by writes that leave
holes intentionally (e.g. create file, write 64K to offset 0, write 4K
to offset 128, leaves a hole from range 64K to 128K) or by hole
punching through fallocate().

>
> Because for the love of god I cannot find why our kernel is being
> written this way onto disk - the installer doesn't explicitly request
> for PUNCH_HOLES nor anything, as far as I looked.
>
> Marek
>
> Marek



-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”