From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88568231A32; Tue, 18 Nov 2025 06:22:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763446940; cv=none; b=Cz1P/O6imU3CEwwIpnqdTzrJd1CxIpN920FD6nWXrZk5aKVnNLFoyQxYYROg9Rs8ZqrD44xZ32s8hdX1abkirlmkZ8n+8EPQBipjE8+zVaQx/9CLT+DMHrz6DCzW1PrsbpzjI+pVveb6JsQPJpQ0vYj6J1MPIPW0sEWe2v622UA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763446940; c=relaxed/simple; bh=kem2m2eUDE4k+KhL2rN3cSp7QaY3fXSsl80MWZLtFpY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XAtop1h7iL/7ROfATB82R+cUPfZlSTcsHhh5FD0NbnQ3RSwiWP8XDNCv7keZSG2zsTkp2fifehDvPqPAHLu2bHamtJ8UGd1rJ4WoxpjLLHYso88e7zq2zrUNF8X8ALk0J/C+yNz6oS0aRlPN2liROekRX8RcqmUvXNRWXR/IS+o= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=lst.de; spf=none smtp.mailfrom=bombadil.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=UUfiS0Af; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=lst.de Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=bombadil.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="UUfiS0Af" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=NwDc3eWt44SMvPQrlSWqoaKOT+KSCiuB5doX9517C78=; b=UUfiS0Af52Wxs2aeb7F/+GjoSB htV3fXAgUH1hx8VQZ06Ti5KeyHvuTIixu2+w0IUculzvNL0NC50Y+E2LJBGrIB6DC4GvSLfCntG10 dsejzY84wMsy62LBlD0fLiQ31r9gYokEav4khZ3mug4UUiSin5HVyEXWgRugDkg1WwCCGr+1DHLRW RNh6ssRhSJjxoSrI7gN+IwkN5E9278WhqKMnCMpujJcsMG+v/YtMR8OPUBPuFcFlmLl1qyrmqTSrv MCMUskbkhyAfYYXulMSb4gRD95Gpo3rxGdATdG7+wfL9dR0HL4pDdjpXyLRnDqiKsIgNEXND6sMep IRDrVcJQ==; Received: from 2a02-8389-2341-5b80-d601-7564-c2e0-491c.cable.dynamic.v6.surfer.at ([2a02:8389:2341:5b80:d601:7564:c2e0:491c] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux)) id 1vLF6Y-0000000HUOf-2rrk; Tue, 18 Nov 2025 06:22:19 +0000 From: Christoph Hellwig To: Eric Biggers Cc: "Theodore Y. Ts'o" , Jaegeuk Kim , Andreas Dilger , Chao Yu , Christian Brauner , "Darrick J. Wong" , linux-fscrypt@vger.kernel.org, linux-ext4@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-fsdevel@vger.kernel.org Subject: [PATCH 02/11] fscrypt: keep multiple bios in flight in fscrypt_zeroout_range_inline_crypt Date: Tue, 18 Nov 2025 07:21:45 +0100 Message-ID: <20251118062159.2358085-3-hch@lst.de> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251118062159.2358085-1-hch@lst.de> References: <20251118062159.2358085-1-hch@lst.de> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html This should slightly improve performance for large zeroing operations, but more importantly prepares for blk-crypto refactoring that requires all fscrypt users to call submit_bio directly. Signed-off-by: Christoph Hellwig Reviewed-by: Eric Biggers --- fs/crypto/bio.c | 86 +++++++++++++++++++++++++++++++------------------ 1 file changed, 54 insertions(+), 32 deletions(-) diff --git a/fs/crypto/bio.c b/fs/crypto/bio.c index 68b0424d879a..c2b3ca100f8d 100644 --- a/fs/crypto/bio.c +++ b/fs/crypto/bio.c @@ -47,49 +47,71 @@ bool fscrypt_decrypt_bio(struct bio *bio) } EXPORT_SYMBOL(fscrypt_decrypt_bio); +struct fscrypt_zero_done { + atomic_t pending; + blk_status_t status; + struct completion done; +}; + +static void fscrypt_zeroout_range_done(struct fscrypt_zero_done *done) +{ + if (atomic_dec_and_test(&done->pending)) + complete(&done->done); +} + +static void fscrypt_zeroout_range_end_io(struct bio *bio) +{ + struct fscrypt_zero_done *done = bio->bi_private; + + if (bio->bi_status) + cmpxchg(&done->status, 0, bio->bi_status); + fscrypt_zeroout_range_done(done); + bio_put(bio); +} + static int fscrypt_zeroout_range_inline_crypt(const struct inode *inode, pgoff_t lblk, sector_t sector, unsigned int len) { const unsigned int blockbits = inode->i_blkbits; const unsigned int blocks_per_page = 1 << (PAGE_SHIFT - blockbits); - struct bio *bio; - int ret, err = 0; - int num_pages = 0; - - /* This always succeeds since __GFP_DIRECT_RECLAIM is set. */ - bio = bio_alloc(inode->i_sb->s_bdev, BIO_MAX_VECS, REQ_OP_WRITE, - GFP_NOFS); + struct fscrypt_zero_done done = { + .pending = ATOMIC_INIT(1), + .done = COMPLETION_INITIALIZER_ONSTACK(done.done), + }; while (len) { - unsigned int blocks_this_page = min(len, blocks_per_page); - unsigned int bytes_this_page = blocks_this_page << blockbits; + struct bio *bio; + unsigned int n; - if (num_pages == 0) { - fscrypt_set_bio_crypt_ctx(bio, inode, lblk, GFP_NOFS); - bio->bi_iter.bi_sector = sector; - } - ret = bio_add_page(bio, ZERO_PAGE(0), bytes_this_page, 0); - if (WARN_ON_ONCE(ret != bytes_this_page)) { - err = -EIO; - goto out; - } - num_pages++; - len -= blocks_this_page; - lblk += blocks_this_page; - sector += (bytes_this_page >> SECTOR_SHIFT); - if (num_pages == BIO_MAX_VECS || !len || - !fscrypt_mergeable_bio(bio, inode, lblk)) { - err = submit_bio_wait(bio); - if (err) - goto out; - bio_reset(bio, inode->i_sb->s_bdev, REQ_OP_WRITE); - num_pages = 0; + bio = bio_alloc(inode->i_sb->s_bdev, BIO_MAX_VECS, REQ_OP_WRITE, + GFP_NOFS); + bio->bi_iter.bi_sector = sector; + bio->bi_private = &done; + bio->bi_end_io = fscrypt_zeroout_range_end_io; + fscrypt_set_bio_crypt_ctx(bio, inode, lblk, GFP_NOFS); + + for (n = 0; n < BIO_MAX_VECS; n++) { + unsigned int blocks_this_page = + min(len, blocks_per_page); + unsigned int bytes_this_page = blocks_this_page << blockbits; + + __bio_add_page(bio, ZERO_PAGE(0), bytes_this_page, 0); + len -= blocks_this_page; + lblk += blocks_this_page; + sector += (bytes_this_page >> SECTOR_SHIFT); + if (!len || !fscrypt_mergeable_bio(bio, inode, lblk)) + break; } + + atomic_inc(&done.pending); + submit_bio(bio); } -out: - bio_put(bio); - return err; + + fscrypt_zeroout_range_done(&done); + + wait_for_completion(&done.done); + return blk_status_to_errno(done.status); } /** -- 2.47.3