From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3578C433F5 for ; Sat, 12 Mar 2022 02:47:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230119AbiCLCss (ORCPT ); Fri, 11 Mar 2022 21:48:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229505AbiCLCss (ORCPT ); Fri, 11 Mar 2022 21:48:48 -0500 Received: from drax.kayaks.hungrycats.org (drax.kayaks.hungrycats.org [174.142.148.226]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2CD9118A78F for ; Fri, 11 Mar 2022 18:47:43 -0800 (PST) Received: by drax.kayaks.hungrycats.org (Postfix, from userid 1002) id 6F4962502AA; Fri, 11 Mar 2022 21:47:41 -0500 (EST) Date: Fri, 11 Mar 2022 21:47:40 -0500 From: Zygo Blaxell To: ov2k Cc: linux-btrfs@vger.kernel.org Subject: Re: FIDEDUPERANGE and compression Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On Wed, Mar 09, 2022 at 03:04:40PM -0500, ov2k wrote: > On Sat, Mar 5, 2022 at 11:44 PM Zygo Blaxell > wrote: > > > > On Mon, Feb 21, 2022 at 05:31:13PM -0500, ov2k wrote: > > > It looks like btrfs coalesces adjacent uncompressed extents. I'm not > > > sure whether this is done by FIDEDUPERANGE or FS_IOC_FIEMAP. I think > > > the problem is that adjacent decompressed ranges (defined by #3 and > > > #4) within the same compressed block are not coalesced in a similar > > > manner. Is there a particular reason why this isn't done, or is this > > > simply a case of nobody having done it? > > > > It hasn't been done because FIEMAP can't produce results for compressed > > extents that aren't nonsense. The interface can't cope with compressed > > data. > > > > I think there's a misunderstanding here. The issue isn't making FS_IOC_FIEMAP > represent compressed data sensibly. The goal is for btrfs_fiemap() to handle > adjacent subranges of a compressed extent in much the same way as it handles > adjacent uncompressed extents. The result should be no more or less > nonsensical than it already is. [...] > I'm talking about emitting a single struct fiemap_extent that corresponds to > two adjacent subranges of the same compressed btrfs extent. The two btrfs > extents would simply have to satisfy: > > extent 1 #2 (bytenr) == extent2 #2 (bytenr) > > extent 1 #1 (seek offset) + extent 1 #3 (decompressed subrange length) > == extent 2 #1 (seek offset) > > extent 1 #4 (decompressed subrange offset) + extent 1 #3 (decompressed > subrange length) == extent 2 #4 (decompressed subrange offset) > > The resulting struct fiemap_extent would have: > > fe_logical: extent 1 #1 (seek offset) > > fe_physical: extent 1 #2 (bytenr) > > fe_length: extent 1 #3 (decompressed subrange length) + extent 2 #3 > (decompressed subrange length) OK, FIEMAP could handle that one special case. And it is a frequently requested feature--filefrag's physically-contiguous-extent counter report doesn't work at all on compressed files, and it could work in the common case of a simple sequential write (or reflink thereof). On the other hand, if you're trying to do dedupe on btrfs, you'll need access to all the other extent fields to avoid bookending issues.