From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dave Chinner Date: Thu, 6 Apr 2023 08:51:25 +1000 Subject: [Cluster-devel] [PATCH v2 21/23] xfs: handle merkle tree block size != fs blocksize != PAGE_SIZE In-Reply-To: <20230405151234.sgkuasb7lwmgetzz@aalbersh.remote.csb> References: <20230404145319.2057051-1-aalbersh@redhat.com> <20230404145319.2057051-22-aalbersh@redhat.com> <20230404233224.GE1893@sol.localdomain> <20230405151234.sgkuasb7lwmgetzz@aalbersh.remote.csb> Message-ID: <20230405225125.GS3223426@dread.disaster.area> List-Id: To: cluster-devel.redhat.com MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Wed, Apr 05, 2023 at 05:12:34PM +0200, Andrey Albershteyn wrote: > Hi Eric, > > On Tue, Apr 04, 2023 at 04:32:24PM -0700, Eric Biggers wrote: > > Hi Andrey, > > > > On Tue, Apr 04, 2023 at 04:53:17PM +0200, Andrey Albershteyn wrote: > > > In case of different Merkle tree block size fs-verity expects > > > ->read_merkle_tree_page() to return Merkle tree page filled with > > > Merkle tree blocks. The XFS stores each merkle tree block under > > > extended attribute. Those attributes are addressed by block offset > > > into Merkle tree. > > > > > > This patch make ->read_merkle_tree_page() to fetch multiple merkle > > > tree blocks based on size ratio. Also the reference to each xfs_buf > > > is passed with page->private to ->drop_page(). > > > > > > Signed-off-by: Andrey Albershteyn > > > --- > > > fs/xfs/xfs_verity.c | 74 +++++++++++++++++++++++++++++++++++---------- > > > fs/xfs/xfs_verity.h | 8 +++++ > > > 2 files changed, 66 insertions(+), 16 deletions(-) > > > > > > diff --git a/fs/xfs/xfs_verity.c b/fs/xfs/xfs_verity.c > > > index a9874ff4efcd..ef0aff216f06 100644 > > > --- a/fs/xfs/xfs_verity.c > > > +++ b/fs/xfs/xfs_verity.c > > > @@ -134,6 +134,10 @@ xfs_read_merkle_tree_page( > > > struct page *page = NULL; > > > __be64 name = cpu_to_be64(index << PAGE_SHIFT); > > > uint32_t bs = 1 << log_blocksize; > > > + int blocks_per_page = > > > + (1 << (PAGE_SHIFT - log_blocksize)); > > > + int n = 0; > > > + int offset = 0; > > > struct xfs_da_args args = { > > > .dp = ip, > > > .attr_filter = XFS_ATTR_VERITY, > > > @@ -143,26 +147,59 @@ xfs_read_merkle_tree_page( > > > .valuelen = bs, > > > }; > > > int error = 0; > > > + bool is_checked = true; > > > + struct xfs_verity_buf_list *buf_list; > > > > > > page = alloc_page(GFP_KERNEL); > > > if (!page) > > > return ERR_PTR(-ENOMEM); > > > > > > - error = xfs_attr_get(&args); > > > - if (error) { > > > - kmem_free(args.value); > > > - xfs_buf_rele(args.bp); > > > + buf_list = kzalloc(sizeof(struct xfs_verity_buf_list), GFP_KERNEL); > > > + if (!buf_list) { > > > put_page(page); > > > - return ERR_PTR(-EFAULT); > > > + return ERR_PTR(-ENOMEM); > > > } > > > > > > - if (args.bp->b_flags & XBF_VERITY_CHECKED) > > > + /* > > > + * Fill the page with Merkle tree blocks. The blcoks_per_page is higher > > > + * than 1 when fs block size != PAGE_SIZE or Merkle tree block size != > > > + * PAGE SIZE > > > + */ > > > + for (n = 0; n < blocks_per_page; n++) { > > > + offset = bs * n; > > > + name = cpu_to_be64(((index << PAGE_SHIFT) + offset)); > > > + args.name = (const uint8_t *)&name; > > > + > > > + error = xfs_attr_get(&args); > > > + if (error) { > > > + kmem_free(args.value); > > > + /* > > > + * No more Merkle tree blocks (e.g. this was the last > > > + * block of the tree) > > > + */ > > > + if (error == -ENOATTR) > > > + break; > > > + xfs_buf_rele(args.bp); > > > + put_page(page); > > > + kmem_free(buf_list); > > > + return ERR_PTR(-EFAULT); > > > + } > > > + > > > + buf_list->bufs[buf_list->buf_count++] = args.bp; > > > + > > > + /* One of the buffers was dropped */ > > > + if (!(args.bp->b_flags & XBF_VERITY_CHECKED)) > > > + is_checked = false; > > > + > > > + memcpy(page_address(page) + offset, args.value, args.valuelen); > > > + kmem_free(args.value); > > > + args.value = NULL; > > > + } > > > > I was really hoping for a solution where the cached data can be used directly, > > instead allocating a temporary page and copying the cached data into it every > > time the cache is accessed. The problem with what you have now is that every > > time a single 32-byte hash is accessed, a full page (potentially 64KB!) will be > > allocated and filled. That's not very efficient. The need to allocate the > > temporary page can also cause ENOMEM (which will get reported as EIO). > > > > Did you consider alternatives that would work more efficiently? I think it > > would be worth designing something that works properly with how XFS is planned > > to cache the Merkle tree, instead of designing a workaround. > > ->read_merkle_tree_page was not really designed for what you are doing here. > > > > How about replacing ->read_merkle_tree_page with a function that takes in a > > Merkle tree block index (not a page index!) and hands back a (page, offset) pair > > that identifies where the Merkle tree block's data is located? Or (folio, > > offset), I suppose. {kaddr, len}, please. > > > > With that, would it be possible to directly return the XFS cache? > > > > - Eric > > > > Yeah, I also don't like it, I didn't want to change fs-verity much > so went with this workaround. But as it's ok, I will look into trying > to pass xfs buffers to fs-verity without direct use of > ->read_merkle_tree_page(). I think it's possible with (folio, > offset), the xfs buffers aren't xattr value align so the 4k merkle > tree block is stored in two pages. I don't think this is necessary to actually merge the code. We want it to work correctly as the primary concern, performance is a secondary concern. Regardless, as you mention, the xfs_buf is not made up of contiguous pages so the merkle tree block data will be split across two (or more) pages. AFAICT, the fsverity code doesn't work with data structures that span multiple disjoint pages... Another problem is that the xfs-buf might be backed by heap memory (e.g. 4kB fs block size on 64kB PAGE_SIZE) and so it cannot be treated like a page cache page by the fsverity merkle tree code. We most definitely do not want to be passing pages containing heap memory to functions expecting to be passed lru-resident page cache pages.... That said, xfs-bufs do have a stable method of addressing the data in the buffers, and all the XFS code uses this to access and manipulate data directly in the buffers. That is, xfs_buf_offset() returns a mapped kaddr that points to the contiguous memory region containing the metadata in the buffer. If the xfs_buf spans multiple pages, it will return a kaddr pointing into the contiguous vmapped memory address that maps the entire buffer data range. If it is heap memory, it simply returns a pointer into that heap memory. If it's a single page, then it returns the kaddr for the data within the page. If you move all the assumptions about how the merkle tree data is managed out of fsverity and require the fielsystems to do the mapping to kaddrs and reference counting to guarantee life times, then the need for multiple different methods for reading merkle tree data go away... Cheers, Dave. -- Dave Chinner david at fromorbit.com From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BD24322A for ; Wed, 5 Apr 2023 22:51:29 +0000 (UTC) Received: by mail-pf1-f176.google.com with SMTP id z11so24700741pfh.4 for ; Wed, 05 Apr 2023 15:51:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20210112.gappssmtp.com; s=20210112; t=1680735089; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=dLSxfaMFCVS8468KN/z0VwXJmbpcmKd5R8VPq1b2igA=; b=DKNaEMAvLoxgxcW7C/DG2hGwlybNwuFNi7Q8qY482/jzM9d4xL2UjV2S9BGfeJcIDr MA5ql8Ba3dLnaVVaPhzE2O1VDUcT7iJ/QxiGopWq24vyCi9gjKBCmMqTVc80ZzvfS+zT y1i56dDgdCCa+DALwBNBfN3/jA9Mdt2k/DUFQJQwT8rTqbxQlLXKbG94azFShwTCHrRC j2vDIirPUgjQi/BpGinlJPvMNSyzo5n0IBLLvknGncJH3Tm1MZO+xIiGDixxqlk3i4Bo R1xKFWnLl5NxzMVulcM3Ebj7nNcXnWCd0yodUgm1mhmBHJF1ocZct58oemW+AHAqXqk9 /VPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680735089; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=dLSxfaMFCVS8468KN/z0VwXJmbpcmKd5R8VPq1b2igA=; b=xie4MlBFPwpXKzJFtI/5aqEl0DG1KPxvUjsT6KqYZtjEfTgITUE3QVPVZkyS+xoXCf m6JXEMJ3BYvSLjYL0BeCpQBnoXDl7MVZl2Z59Hf07xWHap9mwU5aH4XjlynA6XqykMnl VzNAYLXWY9941KBXXnU9AXv4m2AEzuN9CRGQ70RMm511sSZpz3iyqxF3nKa8MVafsRvG aGuDlHR8i4fo5rm2LivtSuXZQWASIjTzwSBo8n9WsqCq2OpiZ5vCxTtI+RcEgojJydQQ 57olrgihBUdBn9d7fT+zWC4tH0v5ifst1tAyBMAkYooknbkK6/mjh+vCeUMqlvxFpwTK rQTA== X-Gm-Message-State: AAQBX9fhMbhsZMY92CrFK8WmYkcwx6KcTUcjSfcxZriVstEFziDY6r9z 3RrEtHkXI+xDlyxM80NhjovMfw== X-Google-Smtp-Source: AKy350byI2dOXAx2UUhieoBDXlt6kMcfX++CjjnLqbSOozpqmQEUOmvJ7mOnsddzMAKRV6q6VXMDHg== X-Received: by 2002:a62:7b8b:0:b0:625:cc03:df33 with SMTP id w133-20020a627b8b000000b00625cc03df33mr7050385pfc.31.1680735088792; Wed, 05 Apr 2023 15:51:28 -0700 (PDT) Received: from dread.disaster.area (pa49-181-91-157.pa.nsw.optusnet.com.au. [49.181.91.157]) by smtp.gmail.com with ESMTPSA id m3-20020aa79003000000b006260645f2a7sm11619498pfo.17.2023.04.05.15.51.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 15:51:28 -0700 (PDT) Received: from dave by dread.disaster.area with local (Exim 4.92.3) (envelope-from ) id 1pkByP-00HV2h-IA; Thu, 06 Apr 2023 08:51:25 +1000 Date: Thu, 6 Apr 2023 08:51:25 +1000 From: Dave Chinner To: Andrey Albershteyn Cc: Eric Biggers , djwong@kernel.org, dchinner@redhat.com, hch@infradead.org, linux-xfs@vger.kernel.org, fsverity@lists.linux.dev, rpeterso@redhat.com, agruenba@redhat.com, xiang@kernel.org, chao@kernel.org, damien.lemoal@opensource.wdc.com, jth@kernel.org, linux-erofs@lists.ozlabs.org, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com Subject: Re: [PATCH v2 21/23] xfs: handle merkle tree block size != fs blocksize != PAGE_SIZE Message-ID: <20230405225125.GS3223426@dread.disaster.area> References: <20230404145319.2057051-1-aalbersh@redhat.com> <20230404145319.2057051-22-aalbersh@redhat.com> <20230404233224.GE1893@sol.localdomain> <20230405151234.sgkuasb7lwmgetzz@aalbersh.remote.csb> Precedence: bulk X-Mailing-List: fsverity@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230405151234.sgkuasb7lwmgetzz@aalbersh.remote.csb> On Wed, Apr 05, 2023 at 05:12:34PM +0200, Andrey Albershteyn wrote: > Hi Eric, > > On Tue, Apr 04, 2023 at 04:32:24PM -0700, Eric Biggers wrote: > > Hi Andrey, > > > > On Tue, Apr 04, 2023 at 04:53:17PM +0200, Andrey Albershteyn wrote: > > > In case of different Merkle tree block size fs-verity expects > > > ->read_merkle_tree_page() to return Merkle tree page filled with > > > Merkle tree blocks. The XFS stores each merkle tree block under > > > extended attribute. Those attributes are addressed by block offset > > > into Merkle tree. > > > > > > This patch make ->read_merkle_tree_page() to fetch multiple merkle > > > tree blocks based on size ratio. Also the reference to each xfs_buf > > > is passed with page->private to ->drop_page(). > > > > > > Signed-off-by: Andrey Albershteyn > > > --- > > > fs/xfs/xfs_verity.c | 74 +++++++++++++++++++++++++++++++++++---------- > > > fs/xfs/xfs_verity.h | 8 +++++ > > > 2 files changed, 66 insertions(+), 16 deletions(-) > > > > > > diff --git a/fs/xfs/xfs_verity.c b/fs/xfs/xfs_verity.c > > > index a9874ff4efcd..ef0aff216f06 100644 > > > --- a/fs/xfs/xfs_verity.c > > > +++ b/fs/xfs/xfs_verity.c > > > @@ -134,6 +134,10 @@ xfs_read_merkle_tree_page( > > > struct page *page = NULL; > > > __be64 name = cpu_to_be64(index << PAGE_SHIFT); > > > uint32_t bs = 1 << log_blocksize; > > > + int blocks_per_page = > > > + (1 << (PAGE_SHIFT - log_blocksize)); > > > + int n = 0; > > > + int offset = 0; > > > struct xfs_da_args args = { > > > .dp = ip, > > > .attr_filter = XFS_ATTR_VERITY, > > > @@ -143,26 +147,59 @@ xfs_read_merkle_tree_page( > > > .valuelen = bs, > > > }; > > > int error = 0; > > > + bool is_checked = true; > > > + struct xfs_verity_buf_list *buf_list; > > > > > > page = alloc_page(GFP_KERNEL); > > > if (!page) > > > return ERR_PTR(-ENOMEM); > > > > > > - error = xfs_attr_get(&args); > > > - if (error) { > > > - kmem_free(args.value); > > > - xfs_buf_rele(args.bp); > > > + buf_list = kzalloc(sizeof(struct xfs_verity_buf_list), GFP_KERNEL); > > > + if (!buf_list) { > > > put_page(page); > > > - return ERR_PTR(-EFAULT); > > > + return ERR_PTR(-ENOMEM); > > > } > > > > > > - if (args.bp->b_flags & XBF_VERITY_CHECKED) > > > + /* > > > + * Fill the page with Merkle tree blocks. The blcoks_per_page is higher > > > + * than 1 when fs block size != PAGE_SIZE or Merkle tree block size != > > > + * PAGE SIZE > > > + */ > > > + for (n = 0; n < blocks_per_page; n++) { > > > + offset = bs * n; > > > + name = cpu_to_be64(((index << PAGE_SHIFT) + offset)); > > > + args.name = (const uint8_t *)&name; > > > + > > > + error = xfs_attr_get(&args); > > > + if (error) { > > > + kmem_free(args.value); > > > + /* > > > + * No more Merkle tree blocks (e.g. this was the last > > > + * block of the tree) > > > + */ > > > + if (error == -ENOATTR) > > > + break; > > > + xfs_buf_rele(args.bp); > > > + put_page(page); > > > + kmem_free(buf_list); > > > + return ERR_PTR(-EFAULT); > > > + } > > > + > > > + buf_list->bufs[buf_list->buf_count++] = args.bp; > > > + > > > + /* One of the buffers was dropped */ > > > + if (!(args.bp->b_flags & XBF_VERITY_CHECKED)) > > > + is_checked = false; > > > + > > > + memcpy(page_address(page) + offset, args.value, args.valuelen); > > > + kmem_free(args.value); > > > + args.value = NULL; > > > + } > > > > I was really hoping for a solution where the cached data can be used directly, > > instead allocating a temporary page and copying the cached data into it every > > time the cache is accessed. The problem with what you have now is that every > > time a single 32-byte hash is accessed, a full page (potentially 64KB!) will be > > allocated and filled. That's not very efficient. The need to allocate the > > temporary page can also cause ENOMEM (which will get reported as EIO). > > > > Did you consider alternatives that would work more efficiently? I think it > > would be worth designing something that works properly with how XFS is planned > > to cache the Merkle tree, instead of designing a workaround. > > ->read_merkle_tree_page was not really designed for what you are doing here. > > > > How about replacing ->read_merkle_tree_page with a function that takes in a > > Merkle tree block index (not a page index!) and hands back a (page, offset) pair > > that identifies where the Merkle tree block's data is located? Or (folio, > > offset), I suppose. {kaddr, len}, please. > > > > With that, would it be possible to directly return the XFS cache? > > > > - Eric > > > > Yeah, I also don't like it, I didn't want to change fs-verity much > so went with this workaround. But as it's ok, I will look into trying > to pass xfs buffers to fs-verity without direct use of > ->read_merkle_tree_page(). I think it's possible with (folio, > offset), the xfs buffers aren't xattr value align so the 4k merkle > tree block is stored in two pages. I don't think this is necessary to actually merge the code. We want it to work correctly as the primary concern, performance is a secondary concern. Regardless, as you mention, the xfs_buf is not made up of contiguous pages so the merkle tree block data will be split across two (or more) pages. AFAICT, the fsverity code doesn't work with data structures that span multiple disjoint pages... Another problem is that the xfs-buf might be backed by heap memory (e.g. 4kB fs block size on 64kB PAGE_SIZE) and so it cannot be treated like a page cache page by the fsverity merkle tree code. We most definitely do not want to be passing pages containing heap memory to functions expecting to be passed lru-resident page cache pages.... That said, xfs-bufs do have a stable method of addressing the data in the buffers, and all the XFS code uses this to access and manipulate data directly in the buffers. That is, xfs_buf_offset() returns a mapped kaddr that points to the contiguous memory region containing the metadata in the buffer. If the xfs_buf spans multiple pages, it will return a kaddr pointing into the contiguous vmapped memory address that maps the entire buffer data range. If it is heap memory, it simply returns a pointer into that heap memory. If it's a single page, then it returns the kaddr for the data within the page. If you move all the assumptions about how the merkle tree data is managed out of fsverity and require the fielsystems to do the mapping to kaddrs and reference counting to guarantee life times, then the need for multiple different methods for reading merkle tree data go away... Cheers, Dave. -- Dave Chinner david@fromorbit.com From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 252F1C7619A for ; Wed, 5 Apr 2023 22:51:39 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4PsKcp12bGz3f9s for ; Thu, 6 Apr 2023 08:51:38 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lists.ozlabs.org; s=201707; t=1680735098; bh=dLSxfaMFCVS8468KN/z0VwXJmbpcmKd5R8VPq1b2igA=; h=Date:To:Subject:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=WxDP7UnszEsECnJ6XVaiT4cF9N9YN6s77Xz4X3IaCjHOiYJTgxvPnYSh1VAPeXnUb oAf8moTWAoA+Tl1IFCm297ab44jJOHFHBCuVZoQTLBVWloLwc/2+j2W18H/5b0s53a eaoODTSUF1tSasvtenPLJAABFtqXix1UTylincGwR3cDo/MfU71H5HZnKx5YCvt3O0 XGRFXC/mz0BJs5pYFyZjVSOTid8QuALVo8BRVmKpUsg8t0OMqS63O4+AQ4WzrzLcur asZ/LxnOEzTYaksj8BkhEKBAGqyuZnQEukF7erbyqKA9usOVAjohXTWSrC+qP+PuKL yxWfVbzcL5YpA== Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=fromorbit.com (client-ip=2607:f8b0:4864:20::433; helo=mail-pf1-x433.google.com; envelope-from=david@fromorbit.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=fromorbit-com.20210112.gappssmtp.com header.i=@fromorbit-com.20210112.gappssmtp.com header.a=rsa-sha256 header.s=20210112 header.b=DKNaEMAv; dkim-atps=neutral Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4PsKcj5k4Xz3chZ for ; Thu, 6 Apr 2023 08:51:31 +1000 (AEST) Received: by mail-pf1-x433.google.com with SMTP id bm13so10557916pfb.5 for ; Wed, 05 Apr 2023 15:51:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680735089; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=dLSxfaMFCVS8468KN/z0VwXJmbpcmKd5R8VPq1b2igA=; b=TpPo3SUBMXt7VVrP99xIADVMgfVx89p7U3iwoEwFZN5IJq7Bh9YyfDnwhmSMwvogPu saw/aBbY4SMbCzC0kv0rM4mqAFbw/08HXejTg+du+bof6AHn4rnybq7llmaYOYyApR5x LcX45HN7Fr3qvOdE5YuR/BuO9YMI8QmnevgC/3KARmh7ySncOXjJQtHNMfnpUVlokgCQ mfslDimpKLsuC/W6qFjHAoTTjlv46hNA3RtWPPwglhw05u7dYRfjL6VD3XHfLOUhi6p/ J03DlH81DCvGXdnAFj5sjYOp/42C80Va25plhac0KVzDW6doPSsq1PhC9h7MDuxSi7D7 xM1Q== X-Gm-Message-State: AAQBX9cHfDPi0wEy94Toyqm1RATxd/G+cAmtq0oDE0QG1fs7n1rBOQRd 6U6dlgIKotodSrgRlOOYeSAr8Q== X-Google-Smtp-Source: AKy350byI2dOXAx2UUhieoBDXlt6kMcfX++CjjnLqbSOozpqmQEUOmvJ7mOnsddzMAKRV6q6VXMDHg== X-Received: by 2002:a62:7b8b:0:b0:625:cc03:df33 with SMTP id w133-20020a627b8b000000b00625cc03df33mr7050385pfc.31.1680735088792; Wed, 05 Apr 2023 15:51:28 -0700 (PDT) Received: from dread.disaster.area (pa49-181-91-157.pa.nsw.optusnet.com.au. [49.181.91.157]) by smtp.gmail.com with ESMTPSA id m3-20020aa79003000000b006260645f2a7sm11619498pfo.17.2023.04.05.15.51.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 15:51:28 -0700 (PDT) Received: from dave by dread.disaster.area with local (Exim 4.92.3) (envelope-from ) id 1pkByP-00HV2h-IA; Thu, 06 Apr 2023 08:51:25 +1000 Date: Thu, 6 Apr 2023 08:51:25 +1000 To: Andrey Albershteyn Subject: Re: [PATCH v2 21/23] xfs: handle merkle tree block size != fs blocksize != PAGE_SIZE Message-ID: <20230405225125.GS3223426@dread.disaster.area> References: <20230404145319.2057051-1-aalbersh@redhat.com> <20230404145319.2057051-22-aalbersh@redhat.com> <20230404233224.GE1893@sol.localdomain> <20230405151234.sgkuasb7lwmgetzz@aalbersh.remote.csb> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230405151234.sgkuasb7lwmgetzz@aalbersh.remote.csb> X-BeenThere: linux-erofs@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development of Linux EROFS file system List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Dave Chinner via Linux-erofs Reply-To: Dave Chinner Cc: fsverity@lists.linux.dev, linux-xfs@vger.kernel.org, linux-ext4@vger.kernel.org, agruenba@redhat.com, hch@infradead.org, djwong@kernel.org, damien.lemoal@opensource.wdc.com, linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com, dchinner@redhat.com, rpeterso@redhat.com, jth@kernel.org, linux-erofs@lists.ozlabs.org, linux-btrfs@vger.kernel.org Errors-To: linux-erofs-bounces+linux-erofs=archiver.kernel.org@lists.ozlabs.org Sender: "Linux-erofs" On Wed, Apr 05, 2023 at 05:12:34PM +0200, Andrey Albershteyn wrote: > Hi Eric, > > On Tue, Apr 04, 2023 at 04:32:24PM -0700, Eric Biggers wrote: > > Hi Andrey, > > > > On Tue, Apr 04, 2023 at 04:53:17PM +0200, Andrey Albershteyn wrote: > > > In case of different Merkle tree block size fs-verity expects > > > ->read_merkle_tree_page() to return Merkle tree page filled with > > > Merkle tree blocks. The XFS stores each merkle tree block under > > > extended attribute. Those attributes are addressed by block offset > > > into Merkle tree. > > > > > > This patch make ->read_merkle_tree_page() to fetch multiple merkle > > > tree blocks based on size ratio. Also the reference to each xfs_buf > > > is passed with page->private to ->drop_page(). > > > > > > Signed-off-by: Andrey Albershteyn > > > --- > > > fs/xfs/xfs_verity.c | 74 +++++++++++++++++++++++++++++++++++---------- > > > fs/xfs/xfs_verity.h | 8 +++++ > > > 2 files changed, 66 insertions(+), 16 deletions(-) > > > > > > diff --git a/fs/xfs/xfs_verity.c b/fs/xfs/xfs_verity.c > > > index a9874ff4efcd..ef0aff216f06 100644 > > > --- a/fs/xfs/xfs_verity.c > > > +++ b/fs/xfs/xfs_verity.c > > > @@ -134,6 +134,10 @@ xfs_read_merkle_tree_page( > > > struct page *page = NULL; > > > __be64 name = cpu_to_be64(index << PAGE_SHIFT); > > > uint32_t bs = 1 << log_blocksize; > > > + int blocks_per_page = > > > + (1 << (PAGE_SHIFT - log_blocksize)); > > > + int n = 0; > > > + int offset = 0; > > > struct xfs_da_args args = { > > > .dp = ip, > > > .attr_filter = XFS_ATTR_VERITY, > > > @@ -143,26 +147,59 @@ xfs_read_merkle_tree_page( > > > .valuelen = bs, > > > }; > > > int error = 0; > > > + bool is_checked = true; > > > + struct xfs_verity_buf_list *buf_list; > > > > > > page = alloc_page(GFP_KERNEL); > > > if (!page) > > > return ERR_PTR(-ENOMEM); > > > > > > - error = xfs_attr_get(&args); > > > - if (error) { > > > - kmem_free(args.value); > > > - xfs_buf_rele(args.bp); > > > + buf_list = kzalloc(sizeof(struct xfs_verity_buf_list), GFP_KERNEL); > > > + if (!buf_list) { > > > put_page(page); > > > - return ERR_PTR(-EFAULT); > > > + return ERR_PTR(-ENOMEM); > > > } > > > > > > - if (args.bp->b_flags & XBF_VERITY_CHECKED) > > > + /* > > > + * Fill the page with Merkle tree blocks. The blcoks_per_page is higher > > > + * than 1 when fs block size != PAGE_SIZE or Merkle tree block size != > > > + * PAGE SIZE > > > + */ > > > + for (n = 0; n < blocks_per_page; n++) { > > > + offset = bs * n; > > > + name = cpu_to_be64(((index << PAGE_SHIFT) + offset)); > > > + args.name = (const uint8_t *)&name; > > > + > > > + error = xfs_attr_get(&args); > > > + if (error) { > > > + kmem_free(args.value); > > > + /* > > > + * No more Merkle tree blocks (e.g. this was the last > > > + * block of the tree) > > > + */ > > > + if (error == -ENOATTR) > > > + break; > > > + xfs_buf_rele(args.bp); > > > + put_page(page); > > > + kmem_free(buf_list); > > > + return ERR_PTR(-EFAULT); > > > + } > > > + > > > + buf_list->bufs[buf_list->buf_count++] = args.bp; > > > + > > > + /* One of the buffers was dropped */ > > > + if (!(args.bp->b_flags & XBF_VERITY_CHECKED)) > > > + is_checked = false; > > > + > > > + memcpy(page_address(page) + offset, args.value, args.valuelen); > > > + kmem_free(args.value); > > > + args.value = NULL; > > > + } > > > > I was really hoping for a solution where the cached data can be used directly, > > instead allocating a temporary page and copying the cached data into it every > > time the cache is accessed. The problem with what you have now is that every > > time a single 32-byte hash is accessed, a full page (potentially 64KB!) will be > > allocated and filled. That's not very efficient. The need to allocate the > > temporary page can also cause ENOMEM (which will get reported as EIO). > > > > Did you consider alternatives that would work more efficiently? I think it > > would be worth designing something that works properly with how XFS is planned > > to cache the Merkle tree, instead of designing a workaround. > > ->read_merkle_tree_page was not really designed for what you are doing here. > > > > How about replacing ->read_merkle_tree_page with a function that takes in a > > Merkle tree block index (not a page index!) and hands back a (page, offset) pair > > that identifies where the Merkle tree block's data is located? Or (folio, > > offset), I suppose. {kaddr, len}, please. > > > > With that, would it be possible to directly return the XFS cache? > > > > - Eric > > > > Yeah, I also don't like it, I didn't want to change fs-verity much > so went with this workaround. But as it's ok, I will look into trying > to pass xfs buffers to fs-verity without direct use of > ->read_merkle_tree_page(). I think it's possible with (folio, > offset), the xfs buffers aren't xattr value align so the 4k merkle > tree block is stored in two pages. I don't think this is necessary to actually merge the code. We want it to work correctly as the primary concern, performance is a secondary concern. Regardless, as you mention, the xfs_buf is not made up of contiguous pages so the merkle tree block data will be split across two (or more) pages. AFAICT, the fsverity code doesn't work with data structures that span multiple disjoint pages... Another problem is that the xfs-buf might be backed by heap memory (e.g. 4kB fs block size on 64kB PAGE_SIZE) and so it cannot be treated like a page cache page by the fsverity merkle tree code. We most definitely do not want to be passing pages containing heap memory to functions expecting to be passed lru-resident page cache pages.... That said, xfs-bufs do have a stable method of addressing the data in the buffers, and all the XFS code uses this to access and manipulate data directly in the buffers. That is, xfs_buf_offset() returns a mapped kaddr that points to the contiguous memory region containing the metadata in the buffer. If the xfs_buf spans multiple pages, it will return a kaddr pointing into the contiguous vmapped memory address that maps the entire buffer data range. If it is heap memory, it simply returns a pointer into that heap memory. If it's a single page, then it returns the kaddr for the data within the page. If you move all the assumptions about how the merkle tree data is managed out of fsverity and require the fielsystems to do the mapping to kaddrs and reference counting to guarantee life times, then the need for multiple different methods for reading merkle tree data go away... Cheers, Dave. -- Dave Chinner david@fromorbit.com From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.sourceforge.net (lists.sourceforge.net [216.105.38.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 499CDC7618D for ; Wed, 5 Apr 2023 22:51:39 +0000 (UTC) Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1pkByZ-00058A-Qe; Wed, 05 Apr 2023 22:51:36 +0000 Received: from [172.30.20.202] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pkByY-000584-Ek for linux-f2fs-devel@lists.sourceforge.net; Wed, 05 Apr 2023 22:51:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=In-Reply-To:Content-Type:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=dLSxfaMFCVS8468KN/z0VwXJmbpcmKd5R8VPq1b2igA=; b=eCQf+iLtD3MS6jw0RfLVI3rF7L IwFsubH6zvetwOQdiGYRk7sHsSYabQlnULyKdnUgHyjgpWsgA8PY1k93iBPBKlwR016NpNQS3dcHb T3gNoOh8GBblSppbuvKwdmilZi+jCbhZffb0J8NY8Sd6UgvcE8gysLZ/zaND8t8BPhDw=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To :From:Date:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=dLSxfaMFCVS8468KN/z0VwXJmbpcmKd5R8VPq1b2igA=; b=XIrdPKZqQS39huzJr2GIseGvi2 i/s32pUjjeorxtGlzoUTmq+YGSMuFMDRm/t7PgGZneTrwGsrUIXYVWWWqLNvPyd7h8WA31lnDFmdp A/0fy0Z/2ouhPwEpVFgsmmgNJ23fxkd1yI4OrpuzA+Tr27VqWX/nuLoWGKZZXG29ECak=; Received: from mail-pg1-f181.google.com ([209.85.215.181]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.95) id 1pkByY-00GLuv-F9 for linux-f2fs-devel@lists.sourceforge.net; Wed, 05 Apr 2023 22:51:35 +0000 Received: by mail-pg1-f181.google.com with SMTP id d8so22718067pgm.3 for ; Wed, 05 Apr 2023 15:51:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20210112.gappssmtp.com; s=20210112; t=1680735089; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=dLSxfaMFCVS8468KN/z0VwXJmbpcmKd5R8VPq1b2igA=; b=DKNaEMAvLoxgxcW7C/DG2hGwlybNwuFNi7Q8qY482/jzM9d4xL2UjV2S9BGfeJcIDr MA5ql8Ba3dLnaVVaPhzE2O1VDUcT7iJ/QxiGopWq24vyCi9gjKBCmMqTVc80ZzvfS+zT y1i56dDgdCCa+DALwBNBfN3/jA9Mdt2k/DUFQJQwT8rTqbxQlLXKbG94azFShwTCHrRC j2vDIirPUgjQi/BpGinlJPvMNSyzo5n0IBLLvknGncJH3Tm1MZO+xIiGDixxqlk3i4Bo R1xKFWnLl5NxzMVulcM3Ebj7nNcXnWCd0yodUgm1mhmBHJF1ocZct58oemW+AHAqXqk9 /VPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680735089; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=dLSxfaMFCVS8468KN/z0VwXJmbpcmKd5R8VPq1b2igA=; b=NQuA5umlbLAWq/ZJXcW+FRiYBvF1PoDZEO6Tl2qmnISBOTO0EiW8hMYevDGu5V+YEn Uh7IHjYO0DTGj2A2O4mGXUrMguDWgYG4MGK/O4D2SZxjrF6/g/s9noMuVjawSXyDmSgH wQVmYnbT2KrtyewinJY+L0FmPTIdaXGB4YoDEfhPOQjd3oJX/yOdmBOLafi5ltgi6WHq Yeq0/ZLqGiedRGg1O8QAZuouN6wKdIY8eqze0WoTrr4u1yJhWVSMy2U4NG7iwfrXXGI3 KXmsv/8Y8RDqU38526bsGG2wtM16vv0oi9Msx1OLhdPUiAEOdUYS/XUEJvBeH79vk9HI t+fw== X-Gm-Message-State: AAQBX9fciCoGPZyt5WY5W9bLeTaE3L1rfJH8VIc1gnjQrNeuWN9y1iPX DwHmJ3i8q2TTgk5khQx0XwzXAQ== X-Google-Smtp-Source: AKy350byI2dOXAx2UUhieoBDXlt6kMcfX++CjjnLqbSOozpqmQEUOmvJ7mOnsddzMAKRV6q6VXMDHg== X-Received: by 2002:a62:7b8b:0:b0:625:cc03:df33 with SMTP id w133-20020a627b8b000000b00625cc03df33mr7050385pfc.31.1680735088792; Wed, 05 Apr 2023 15:51:28 -0700 (PDT) Received: from dread.disaster.area (pa49-181-91-157.pa.nsw.optusnet.com.au. [49.181.91.157]) by smtp.gmail.com with ESMTPSA id m3-20020aa79003000000b006260645f2a7sm11619498pfo.17.2023.04.05.15.51.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 15:51:28 -0700 (PDT) Received: from dave by dread.disaster.area with local (Exim 4.92.3) (envelope-from ) id 1pkByP-00HV2h-IA; Thu, 06 Apr 2023 08:51:25 +1000 Date: Thu, 6 Apr 2023 08:51:25 +1000 To: Andrey Albershteyn Message-ID: <20230405225125.GS3223426@dread.disaster.area> References: <20230404145319.2057051-1-aalbersh@redhat.com> <20230404145319.2057051-22-aalbersh@redhat.com> <20230404233224.GE1893@sol.localdomain> <20230405151234.sgkuasb7lwmgetzz@aalbersh.remote.csb> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230405151234.sgkuasb7lwmgetzz@aalbersh.remote.csb> X-Headers-End: 1pkByY-00GLuv-F9 Subject: Re: [f2fs-dev] [PATCH v2 21/23] xfs: handle merkle tree block size != fs blocksize != PAGE_SIZE X-BeenThere: linux-f2fs-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Dave Chinner via Linux-f2fs-devel Reply-To: Dave Chinner Cc: fsverity@lists.linux.dev, linux-xfs@vger.kernel.org, linux-ext4@vger.kernel.org, agruenba@redhat.com, hch@infradead.org, djwong@kernel.org, damien.lemoal@opensource.wdc.com, linux-f2fs-devel@lists.sourceforge.net, Eric Biggers , cluster-devel@redhat.com, dchinner@redhat.com, rpeterso@redhat.com, xiang@kernel.org, jth@kernel.org, linux-erofs@lists.ozlabs.org, linux-btrfs@vger.kernel.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-f2fs-devel-bounces@lists.sourceforge.net On Wed, Apr 05, 2023 at 05:12:34PM +0200, Andrey Albershteyn wrote: > Hi Eric, > > On Tue, Apr 04, 2023 at 04:32:24PM -0700, Eric Biggers wrote: > > Hi Andrey, > > > > On Tue, Apr 04, 2023 at 04:53:17PM +0200, Andrey Albershteyn wrote: > > > In case of different Merkle tree block size fs-verity expects > > > ->read_merkle_tree_page() to return Merkle tree page filled with > > > Merkle tree blocks. The XFS stores each merkle tree block under > > > extended attribute. Those attributes are addressed by block offset > > > into Merkle tree. > > > > > > This patch make ->read_merkle_tree_page() to fetch multiple merkle > > > tree blocks based on size ratio. Also the reference to each xfs_buf > > > is passed with page->private to ->drop_page(). > > > > > > Signed-off-by: Andrey Albershteyn > > > --- > > > fs/xfs/xfs_verity.c | 74 +++++++++++++++++++++++++++++++++++---------- > > > fs/xfs/xfs_verity.h | 8 +++++ > > > 2 files changed, 66 insertions(+), 16 deletions(-) > > > > > > diff --git a/fs/xfs/xfs_verity.c b/fs/xfs/xfs_verity.c > > > index a9874ff4efcd..ef0aff216f06 100644 > > > --- a/fs/xfs/xfs_verity.c > > > +++ b/fs/xfs/xfs_verity.c > > > @@ -134,6 +134,10 @@ xfs_read_merkle_tree_page( > > > struct page *page = NULL; > > > __be64 name = cpu_to_be64(index << PAGE_SHIFT); > > > uint32_t bs = 1 << log_blocksize; > > > + int blocks_per_page = > > > + (1 << (PAGE_SHIFT - log_blocksize)); > > > + int n = 0; > > > + int offset = 0; > > > struct xfs_da_args args = { > > > .dp = ip, > > > .attr_filter = XFS_ATTR_VERITY, > > > @@ -143,26 +147,59 @@ xfs_read_merkle_tree_page( > > > .valuelen = bs, > > > }; > > > int error = 0; > > > + bool is_checked = true; > > > + struct xfs_verity_buf_list *buf_list; > > > > > > page = alloc_page(GFP_KERNEL); > > > if (!page) > > > return ERR_PTR(-ENOMEM); > > > > > > - error = xfs_attr_get(&args); > > > - if (error) { > > > - kmem_free(args.value); > > > - xfs_buf_rele(args.bp); > > > + buf_list = kzalloc(sizeof(struct xfs_verity_buf_list), GFP_KERNEL); > > > + if (!buf_list) { > > > put_page(page); > > > - return ERR_PTR(-EFAULT); > > > + return ERR_PTR(-ENOMEM); > > > } > > > > > > - if (args.bp->b_flags & XBF_VERITY_CHECKED) > > > + /* > > > + * Fill the page with Merkle tree blocks. The blcoks_per_page is higher > > > + * than 1 when fs block size != PAGE_SIZE or Merkle tree block size != > > > + * PAGE SIZE > > > + */ > > > + for (n = 0; n < blocks_per_page; n++) { > > > + offset = bs * n; > > > + name = cpu_to_be64(((index << PAGE_SHIFT) + offset)); > > > + args.name = (const uint8_t *)&name; > > > + > > > + error = xfs_attr_get(&args); > > > + if (error) { > > > + kmem_free(args.value); > > > + /* > > > + * No more Merkle tree blocks (e.g. this was the last > > > + * block of the tree) > > > + */ > > > + if (error == -ENOATTR) > > > + break; > > > + xfs_buf_rele(args.bp); > > > + put_page(page); > > > + kmem_free(buf_list); > > > + return ERR_PTR(-EFAULT); > > > + } > > > + > > > + buf_list->bufs[buf_list->buf_count++] = args.bp; > > > + > > > + /* One of the buffers was dropped */ > > > + if (!(args.bp->b_flags & XBF_VERITY_CHECKED)) > > > + is_checked = false; > > > + > > > + memcpy(page_address(page) + offset, args.value, args.valuelen); > > > + kmem_free(args.value); > > > + args.value = NULL; > > > + } > > > > I was really hoping for a solution where the cached data can be used directly, > > instead allocating a temporary page and copying the cached data into it every > > time the cache is accessed. The problem with what you have now is that every > > time a single 32-byte hash is accessed, a full page (potentially 64KB!) will be > > allocated and filled. That's not very efficient. The need to allocate the > > temporary page can also cause ENOMEM (which will get reported as EIO). > > > > Did you consider alternatives that would work more efficiently? I think it > > would be worth designing something that works properly with how XFS is planned > > to cache the Merkle tree, instead of designing a workaround. > > ->read_merkle_tree_page was not really designed for what you are doing here. > > > > How about replacing ->read_merkle_tree_page with a function that takes in a > > Merkle tree block index (not a page index!) and hands back a (page, offset) pair > > that identifies where the Merkle tree block's data is located? Or (folio, > > offset), I suppose. {kaddr, len}, please. > > > > With that, would it be possible to directly return the XFS cache? > > > > - Eric > > > > Yeah, I also don't like it, I didn't want to change fs-verity much > so went with this workaround. But as it's ok, I will look into trying > to pass xfs buffers to fs-verity without direct use of > ->read_merkle_tree_page(). I think it's possible with (folio, > offset), the xfs buffers aren't xattr value align so the 4k merkle > tree block is stored in two pages. I don't think this is necessary to actually merge the code. We want it to work correctly as the primary concern, performance is a secondary concern. Regardless, as you mention, the xfs_buf is not made up of contiguous pages so the merkle tree block data will be split across two (or more) pages. AFAICT, the fsverity code doesn't work with data structures that span multiple disjoint pages... Another problem is that the xfs-buf might be backed by heap memory (e.g. 4kB fs block size on 64kB PAGE_SIZE) and so it cannot be treated like a page cache page by the fsverity merkle tree code. We most definitely do not want to be passing pages containing heap memory to functions expecting to be passed lru-resident page cache pages.... That said, xfs-bufs do have a stable method of addressing the data in the buffers, and all the XFS code uses this to access and manipulate data directly in the buffers. That is, xfs_buf_offset() returns a mapped kaddr that points to the contiguous memory region containing the metadata in the buffer. If the xfs_buf spans multiple pages, it will return a kaddr pointing into the contiguous vmapped memory address that maps the entire buffer data range. If it is heap memory, it simply returns a pointer into that heap memory. If it's a single page, then it returns the kaddr for the data within the page. If you move all the assumptions about how the merkle tree data is managed out of fsverity and require the fielsystems to do the mapping to kaddrs and reference counting to guarantee life times, then the need for multiple different methods for reading merkle tree data go away... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel