Re: FIDEDUPERANGE and compression

[Zygo Blaxell <ce3g8jdj@umail.furryterror.org>]

On Wed, Mar 09, 2022 at 03:04:40PM -0500, ov2k wrote:
> On Sat, Mar 5, 2022 at 11:44 PM Zygo Blaxell
> <ce3g8jdj@umail.furryterror.org> wrote:
> >
> > On Mon, Feb 21, 2022 at 05:31:13PM -0500, ov2k wrote:
> > > It looks like btrfs coalesces adjacent uncompressed extents.  I'm not
> > > sure whether this is done by FIDEDUPERANGE or FS_IOC_FIEMAP.  I think
> > > the problem is that adjacent decompressed ranges (defined by #3 and
> > > #4) within the same compressed block are not coalesced in a similar
> > > manner.  Is there a particular reason why this isn't done, or is this
> > > simply a case of nobody having done it?
> >
> > It hasn't been done because FIEMAP can't produce results for compressed
> > extents that aren't nonsense.  The interface can't cope with compressed
> > data.
> >
> 
> I think there's a misunderstanding here.  The issue isn't making FS_IOC_FIEMAP
> represent compressed data sensibly.  The goal is for btrfs_fiemap() to handle
> adjacent subranges of a compressed extent in much the same way as it handles
> adjacent uncompressed extents.  The result should be no more or less
> nonsensical than it already is.
[...]
> I'm talking about emitting a single struct fiemap_extent that corresponds to
> two adjacent subranges of the same compressed btrfs extent.  The two btrfs
> extents would simply have to satisfy:
> 
>         extent 1 #2 (bytenr) == extent2 #2 (bytenr)
> 
>         extent 1 #1 (seek offset) + extent 1 #3 (decompressed subrange length)
>         == extent 2 #1 (seek offset)
> 
>         extent 1 #4 (decompressed subrange offset) + extent 1 #3 (decompressed
>         subrange length) == extent 2 #4 (decompressed subrange offset)
> 
> The resulting struct fiemap_extent would have:
> 
>         fe_logical: extent 1 #1 (seek offset)
> 
>         fe_physical: extent 1 #2 (bytenr)
> 
>         fe_length: extent 1 #3 (decompressed subrange length) + extent 2 #3
>         (decompressed subrange length)

OK, FIEMAP could handle that one special case.  And it is a frequently
requested feature--filefrag's physically-contiguous-extent counter report
doesn't work at all on compressed files, and it could work in the common
case of a simple sequential write (or reflink thereof).

On the other hand, if you're trying to do dedupe on btrfs, you'll need
access to all the other extent fields to avoid bookending issues.