From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C59AC25B75 for ; Mon, 3 Jun 2024 18:50:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZLfCa6orrsZgQ33EkVtQY43sNa1mBfcGSia21wK6Rws=; b=304uGkc+fMSPdW 1siccNP72Be6qDvBwFOAlWjuZ7YAuD9syo8jTxz7Rc9RRhXzdIRe11IGTPXiRykp0OE371DHjid7Q 2c6ta8d2FqE/IStF+W2U+VVbrnWeTWvMIK+/U5r2L4UcqqwempqTjrc4RWDGWAqI+ykjNtu6RAtnf X+OchfIt82O6DYYxql9rprfA3hbJwwCrheWxfJYXDuifgPE0Ht8fr1/h/uPH1kUzlXSrNp7wGFQPw viJNbr0iNuJZrZva+EP7kd9VC0GgTXfBjHFggchdeKTFD9pi5fp3ClIeyySIMLAYtd744zR49sztS HNwU1jZTc/T2vLdAWm+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sECl1-000000004ox-0HQ6; Mon, 03 Jun 2024 18:50:11 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sECkw-000000004nU-42aC for linux-arm-kernel@lists.infradead.org; Mon, 03 Jun 2024 18:50:08 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id EE4AACE0EC4; Mon, 3 Jun 2024 18:50:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DF99DC2BD10; Mon, 3 Jun 2024 18:50:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717440604; bh=tAoh6E2HyT2I7zRPieUiEXpLsGE3VAG7l4WoETX8/KA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=GEf/jxEHtJGXB7kScrOu7h0DjjNeWIg9lmhillOkUjsewEoDtkkXOBYTxIubLXC0D AF4raCk7yLDQsjaEiVt9L4jATQZdkgU4JttVpSxom5jc+RhW+ziwtinDv6R8RdlHfV 2aq5gfLjxWyk1YHIbAwIGdaZo9nVrrLnghqYrrhZV1LMVmHyB1umUBQI+ZgnubF8Kd 3KpP1RkPnIGdxnhu8CnsDUw7RZmxAFd6pHxvh6XERMEtX/eCfJDWXm7RlHjS4oYoai JflohoHAJ9g5V2MQ1cx1bAp9uDtUvT/iiJL/ne32mD6T06oKQfIC+1n85wD8kDzzHu 1ndEUQNWLyvTA== Date: Mon, 3 Jun 2024 11:50:02 -0700 From: Eric Biggers To: Herbert Xu Cc: linux-crypto@vger.kernel.org, fsverity@lists.linux.dev, dm-devel@lists.linux.dev, x86@kernel.org, linux-arm-kernel@lists.infradead.org, ardb@kernel.org, samitolvanen@google.com, bvanassche@acm.org Subject: Re: [PATCH v3 6/8] fsverity: improve performance by using multibuffer hashing Message-ID: <20240603185002.GA35358@sol.localdomain> References: <20240507002343.239552-7-ebiggers@kernel.org> <20240531061348.GG6505@sol.localdomain> <20240531065258.GH6505@sol.localdomain> <20240531185126.GA1153@sol.localdomain> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240531185126.GA1153@sol.localdomain> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240603_115007_389472_559627DD X-CRM114-Status: GOOD ( 36.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, May 31, 2024 at 11:51:26AM -0700, Eric Biggers wrote: > On Fri, May 31, 2024 at 04:49:28PM +0800, Herbert Xu wrote: > > On Thu, May 30, 2024 at 11:52:58PM -0700, Eric Biggers wrote: > > > > > > Looking at it again a bit more closely, both fsverity and dm-verity have > > > per-block information that they need to keep track of in the queue in addition > > > to the data buffers and hashes: the block number, and in dm-verity's case also a > > > bvec_iter pointing to that block. > > > > Again I'm not asking you to make this API asynchronous at all. > > What exactly are you suggesting, then? It seems that you want multibuffer > hashing to be supported by the existing ahash API. However, that only works if > it's made asynchronous, given how the messages would have to be queued up on a > global queue. That has a huge number of issues which I've already explained. > (And it was even tried before, and it failed.) > > > I was just commenting on the added complexity in fsverify due to > > the use of the linear shash API instead of the page-based ahash API. > > It's the other way around. The shash version is much simpler. Just look at the > diff of commit 8fcd94add6c5 that changed from ahash to shash: > > 4 files changed, 71 insertions(+), 200 deletions(-) > > > This complexity was then compounded by the multi-buffer support. > > fsverity and dm-verity will have to be updated to use multibuffer hashing > anyway, given that transparently supporting it in the existing API is not > viable. If your concern is that in my current patchset fsverity and dm-verity > have separate code paths for multibuffer vs. single-buffer, as I mentioned I can > address that by restructuring them to operate on arrays (similar to what I > already did with the crypto API part). > > > I think this would look a lot simpler if it moved back to ahash. > > > > The original commit mentioned that ahash was bad for fsverify > > because of vmalloc. But the only use of linear pointers in fsverify > > seems to be from kmalloc. Where is the vmalloc coming from? > > XFS is working on adding support for fsverity, and XFS was planning to provide > their Merkle tree blocks in vmalloced buffers. Their plans have shifted several > times, and I think their latest plan no longer uses vmalloced buffers. But in > any case it's still very convenient and much simpler to be able to just use > virtual addresses without having to worry about what type of memory it is. > I've now unified the code paths for single-block and multi-block processing in fsverity and dm-verity, which I think addresses your remaining concern that's feasible to address. Your feedback has been conflating different issues, and it only comes in once every few weeks, so it's a bit hard to follow. But I think the remaining thing that you're asking for is to make the API to be part of "ahash" and use the existing async completion callback. That's not actually feasible, for the reasons that I've explained in detail. See in particular https://lore.kernel.org/linux-crypto/20240503152810.GA1132@sol.localdomain/. Thanks, - Eric _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel