From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39E73C83F1B for ; Thu, 10 Jul 2025 17:23:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C62A36B009A; Thu, 10 Jul 2025 13:23:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C3A5F6B009B; Thu, 10 Jul 2025 13:23:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B501E6B009C; Thu, 10 Jul 2025 13:23:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A60906B009A for ; Thu, 10 Jul 2025 13:23:04 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5A92910C85C for ; Thu, 10 Jul 2025 17:23:04 +0000 (UTC) X-FDA: 83649025488.20.13AF0BA Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id 7FA7E1C000D for ; Thu, 10 Jul 2025 17:23:02 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QKJhFope; spf=pass (imf20.hostedemail.com: domain of snitzer@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=snitzer@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752168182; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=slRprlHKdt7fJehMbGom0ZlVAAX9ns1PGb3476vDgBQ=; b=qZ7DSkWsmnMT01K0Y5vc5WoGBMLqK9ldLAp+oDZLR7y6+d9F73yo9fAQz9nTkxphKxSaMV RjMaBxLE0cN23euor7nC+Znjfq0tS4iKxx0X+EzzgsBgGXnt2EBRdxvoAULDjjU/no7beQ rJBqCgzwxJZ/mVzdh/O2XsgjTcBrOYo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QKJhFope; spf=pass (imf20.hostedemail.com: domain of snitzer@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=snitzer@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752168182; a=rsa-sha256; cv=none; b=FNRDzrg91X/INv1HjkvQGq0fN3ZnCWfWP1W0NDn/rep1KolYZITph0xJOfWL9NMP6c/39C GgKpHdxDyVEf5v7x314RDQD0zwQofym/ZSI/LnmQHes0yJxZbx/XMbzVDoigcVQeP0NA/f Qs5lGOXBrbhmffaIcVasgJOSXy6Ku5M= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 59E5E5C6F36; Thu, 10 Jul 2025 17:23:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C552AC4CEE3; Thu, 10 Jul 2025 17:23:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752168181; bh=V255YLFB1n8qZG3lZpwtNT15v0OgXI6hDpqjbbF7/9w=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=QKJhFopeKDrpzY+2KvoVXeTP5orLVtemBroS8JmMLRSGXW/E82wWf06sAU3FFfZvL 2Ch64of+GDN5EQC8R8FgRG/oUI0MPV+mCiAwh5RmEVcdwwvCA3479oF5xFEZbx0D/K Vlp60eRAesixtko2acE1HVniyLOxaTPkJvOUmcnUxaRZRhahKsv+lDw85+ugVmaGec 4bhCPpdl4dX6lw3dTHVzKMEj1gXEIaMRp6yEUaGOS2FTu044ccQwCdHCdq5KSM2dOj 2uR3eMK3kBW+mFU9KD6XB2ZfvxFnYZE45umNq2694KBwvo7U22IdiNqjZBA/zZPQVu vm+VbjiRWNuOA== Date: Thu, 10 Jul 2025 13:22:59 -0400 From: Mike Snitzer To: Keith Busch Cc: Ming Lei , Jens Axboe , Jeff Layton , Chuck Lever , NeilBrown , Olga Kornievskaia , Dai Ngo , Tom Talpey , Trond Myklebust , Anna Schumaker , linux-nfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, hch@infradead.org, linux-block@vger.kernel.org Subject: Re: [RFC PATCH v2 4/8] lib/iov_iter: remove piecewise bvec length checking in iov_iter_aligned_bvec Message-ID: References: <20250708160619.64800-1-snitzer@kernel.org> <20250708160619.64800-5-snitzer@kernel.org> <5819d6c5bb194613a14d2dcf05605e701683ba49.camel@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 7FA7E1C000D X-Stat-Signature: r6bfdh41qjmjh44jddr7t3n4xk4cq84b X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1752168182-276891 X-HE-Meta: U2FsdGVkX1+XtqJtOR356ICC0Lr4WJ2aCVs3tdN9J3Tyl0VkZOmzS3hzCyYTeMXrNq5b9IUVLwqQlkonjkReNS3Zcl3Nlpqstns2wFN6E4tR6VGPMuPnS5V2wWE+VVdGNmvo42q2glgT6HIhZ5vCn43ymytJhJGjiETzsiLsmkyw34tHpKkxmck/Cf0g4qnAIUBl+m/X8TXSX4wcP8a/L6ZWzRm1BQjNe5cFq0q8BCvAQHAZ9WsAWPZEsN1hMB5ag5lEOATF+6YtkUKecr/Nl8Bt85irabBj3MS01Zb4wlERkY8oQPHMGweRhqZkAhUUoIfu16J71cxAp1dXqYsunbFJbCpXWfmpLZWg7rCEbIr5FUDeFzrBExxuIiCvFJZyosWgHIXxW+wpxshwSmrcEg0EinKfXOMFRCUgNS92sAULJDzTCiIUUez8QWLrE/3rLU9Fkum2dLvoIUKhsZmIEN4dxw+OcHgmHLyBTmTI5suu28EV1oqbIuSFRotNTIZo8Tco30v2K+z+ulWgh1opEINR0TnlHGHeQz7uM3tMe+fS82P/tTbvJF83aR4VcGpKNCTkx2XTfYj049ksVOyiVspcbRq81qhNN/w6lC601p+RSPhR4PJTxrESz1A0o0TUy0B4xDQj+Nn+iCsE91PbmFJblmAjGZaopvr1+F+G61HzmZTiHUrp9X55u2O9aM+GY2fQMdZjaqFpoodlR2VyZNJH1xHBMzRroAMfNTmPSWzQQCVKtfsw5u1V/WQb2z1rlqwl0YWgM8zgiiE9+u53M1WGkJo31W4xZeUoywzUC2pZ13zzkvEFfvyl40z3uyFdCcIY0VxdnCth926Np5xQocwyK+0vz9OYN6dXAXFEn1l7qaq7ATmHFuGBhQ69YsxzY0fBtKxe6fd61nRBPn6TMmeohzLdtOF7eQkB/Z+UAJyI1m93dwDyi7ROsk6/PeT4auPSsouIVqmgXXA6Ocs mj44EGfP yCeIb1e/S21IY9/0xzXfy94t22jzLbjsZvprF+THSPLIkBk9B2rdDR8EXQ9Y8AaDpULbrjmFA/7XxWFJ+LUcZaCV51jUA8YZRFoFG0Terl2ps9bO0TW2k+32pVpbF9gmHr2f+d55RBnlIIYgGPAT30n9ZgFqLWHDTIhl/2OF1iA12kv5dFmq9dgUUh39FTkuhtcrE4QUydqJusmBQUbL1M8sCHgVZYRiGYJaShr3mXbqzMR8JoUQu891ZvvhT/qKU2K0iQd4CvTeDUZuse4XgZJvkercUVKaw48h6DGvu8QMVjon3bER/aOx1DB2QKEvcQglF2mZi/Fi0LvQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jul 10, 2025 at 10:29:22AM -0600, Keith Busch wrote: > On Thu, Jul 10, 2025 at 12:12:29PM -0400, Mike Snitzer wrote: > > On Thu, Jul 10, 2025 at 08:48:04AM -0600, Keith Busch wrote: > > > On Thu, Jul 10, 2025 at 09:52:53AM -0400, Jeff Layton wrote: > > > > On Tue, 2025-07-08 at 12:06 -0400, Mike Snitzer wrote: > > > > > iov_iter_aligned_bvec() is strictly checking alignment of each element > > > > > of the bvec to arrive at whether the bvec is aligned relative to > > > > > dma_alignment and on-disk alignment. Checking each element > > > > > individually results in disallowing a bvec that in aggregate is > > > > > perfectly aligned relative to the provided @len_mask. > > > > > > > > > > Relax the on-disk alignment checking such that it is done on the full > > > > > extent described by the bvec but still do piecewise checking of the > > > > > dma_alignment for each bvec's bv_offset. > > > > > > > > > > This allows for NFS's WRITE payload to be issued using O_DIRECT as > > > > > long as the bvec created with xdr_buf_to_bvec() is composed of pages > > > > > that respect the underlying device's dma_alignment (@addr_mask) and > > > > > the overall contiguous on-disk extent is aligned relative to the > > > > > logical_block_size (@len_mask). > > > > > > > > > > Signed-off-by: Mike Snitzer > > > > > --- > > > > > lib/iov_iter.c | 5 +++-- > > > > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/lib/iov_iter.c b/lib/iov_iter.c > > > > > index bdb37d572e97..b2ae482b8a1d 100644 > > > > > --- a/lib/iov_iter.c > > > > > +++ b/lib/iov_iter.c > > > > > @@ -819,13 +819,14 @@ static bool iov_iter_aligned_bvec(const struct iov_iter *i, unsigned addr_mask, > > > > > unsigned skip = i->iov_offset; > > > > > size_t size = i->count; > > > > > > > > > > + if (size & len_mask) > > > > > + return false; > > > > > + > > > > > do { > > > > > size_t len = bvec->bv_len; > > > > > > > > > > if (len > size) > > > > > len = size; > > > > > - if (len & len_mask) > > > > > - return false; > > > > > if ((unsigned long)(bvec->bv_offset + skip) & addr_mask) > > > > > return false; > > > > > > > > > > > > > cc'ing Keith too since he wrote this helper originally. > > > > > > Thanks. > > > > > > There's a comment in __bio_iov_iter_get_pages that says it expects each > > > vector to be a multiple of the block size. That makes it easier to > > > slit when needed, and this patch would allow vectors that break the > > > current assumption when calculating the "trim" value. > > > > Thanks for the pointer, that high-level bio code is being too > > restrictive. > > > > But not seeing any issues with the trim calculation itself, 'trim' is > > the number of bytes that are past the last logical_block_size aligned > > boundary. And then iov_iter_revert() will rollback the iov such that > > it doesn't include those. Then size is reduced by trim bytes. > > The trim calculation assumes the current bi_size is already a block size > multiple, but it may not be with your propsal. So the trim bytes needs > to take into account the existing bi_size to know how much to trim off > to arrive at a proper total bi_size instead of assuming we can append a > block sized multiple carved out the current iov. The trim "calculation" doesn't assume anything, it just lops off whatever is past the end of the last logical_block_size aligned boundary of the requested pages (which is meant to be bi_size). The fact that the trim ever gets anything implies bi_size is *not* always logical_block_size aligned. No? But sure, with my change it opens the door for bvecs with vectors that aren't all logical_block_size aligned. I'll revisit this code, but if you see a way forward to fix __bio_iov_iter_get_pages to cope with my desired iov_iter_aligned_bvec change please don't be shy with a patch ;) > > All said, in practice I haven't had any issues with this patch. But > > it could just be I don't have the stars aligned to test the case that > > might have problems. If you know of such a case I'd welcome > > suggestions. > > It might be a little harder with iter_bvec, but you also mentioned doing > the same for iter_iovec too, which I think should be pretty easy to > cause a problem for nvme: just submit an O_DIRECT read or write with > individual iovec sizes that are not block size granularities. I made the iter_iovec change yesterday (before I realized I don't actually need it for my NFSD case) and all was fine issuing O_DIRECT IO (via NFSD, so needing the relaxed checking) through to 16 XFS-on-NVMe devices. SO I think the devil will be in the details if NVMe actually cares. Mike