From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 208EAC77B7C for ; Tue, 24 Jun 2025 12:39:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B40108D0006; Tue, 24 Jun 2025 08:39:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B18358D0001; Tue, 24 Jun 2025 08:39:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2E928D0006; Tue, 24 Jun 2025 08:39:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 90A128D0001 for ; Tue, 24 Jun 2025 08:39:09 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 42E2E103F48 for ; Tue, 24 Jun 2025 12:39:09 +0000 (UTC) X-FDA: 83590249218.06.3E0E2E0 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf22.hostedemail.com (Postfix) with ESMTP id 2DFA7C0010 for ; Tue, 24 Jun 2025 12:39:06 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=W6GHJ9nD; spf=none (imf22.hostedemail.com: domain of BATV+9d4a3ef6e4b9936c6115+7975+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+9d4a3ef6e4b9936c6115+7975+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750768747; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0hakl0/kYPdggumUkyNzKLxp5ngjPrYeJTMOFwUQIRA=; b=GglsARHsLHdzcA6lZoTGDHKbaPJcCEMLlEhcOSSCl5kgR6Vsz7iHeVpPCuGNiv+EAto9Mj fVsdjjQsB5d19ok3JzetIthCULsyzGsDKqE51jMvcyK5qyBqO0NS3MeY9U4MbAIzMQ7ADy 4lyAegE4rk7m0ELVn93icxXd46Js+Cg= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=W6GHJ9nD; spf=none (imf22.hostedemail.com: domain of BATV+9d4a3ef6e4b9936c6115+7975+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+9d4a3ef6e4b9936c6115+7975+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750768747; a=rsa-sha256; cv=none; b=nu1jZb3dwpH9LaXli54bcgRh9SAnO5AZLJofrpGOBxYn24Ipd7Fwj1EWCkE6XBwyiSXMlc PAvREiXf+cnHJvzNx7Wj4VAnX4xC9QXCqyGPHUjuh/HnU8xB72JBQMqXmCKTaFwmpAUWrC RBfGP8XLTD28FI2k2v4fWHI4S86hhHk= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0hakl0/kYPdggumUkyNzKLxp5ngjPrYeJTMOFwUQIRA=; b=W6GHJ9nDe8Pjyc8lP/+jiNtUot tEV2YfGRhCnLYBES0b6YVkr8LBiUF8sj81PF+eHHOH1G3NPn5GhDshzrr8iZec7DNgrnHoHIgfSG8 mYmlXBZl7lUIdQOM2ADjdzD8dbkUx8tszlW3N1dar2Zy0fyxFeHZS12t5fZeqO5eB2UkDd0+i2LQe CVMMgiq3wc/JqmPHJU3WZgdiw5jCGKKKximG6PYpCDZLy4Ldbx1q0hWJIQY3rljTUpDloQIypJBOF rn4Ulq7/ZtTByPIQ+Wy3e4+mD8O4K3gVvFFnzCcVR/VwzsMkQWMdSNvbnor6GQbfYMgBKqbW/87Mq hM4pGj5Q==; Received: from hch by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1uU2vU-00000005bfF-49O8; Tue, 24 Jun 2025 12:39:00 +0000 Date: Tue, 24 Jun 2025 05:39:00 -0700 From: Christoph Hellwig To: David Howells Cc: Christoph Hellwig , Andrew Lunn , Eric Dumazet , "David S. Miller" , Jakub Kicinski , David Hildenbrand , John Hubbard , Mina Almasry , willy@infradead.org, Christian Brauner , Al Viro , netdev@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Leon Romanovsky , Logan Gunthorpe , Jason Gunthorpe Subject: Re: How to handle P2P DMA with only {physaddr,len} in bio_vec? Message-ID: References: <2135907.1747061490@warthog.procyon.org.uk> <1069540.1746202908@warthog.procyon.org.uk> <165f5d5b-34f2-40de-b0ec-8c1ca36babe8@lunn.ch> <0aa1b4a2-47b2-40a4-ae14-ce2dd457a1f7@lunn.ch> <1015189.1746187621@warthog.procyon.org.uk> <1021352.1746193306@warthog.procyon.org.uk> <1098395.1750675858@warthog.procyon.org.uk> <1143687.1750755725@warthog.procyon.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1143687.1750755725@warthog.procyon.org.uk> X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 2DFA7C0010 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: 5h6db19f495e7tzokzg1ewne6pyikgf3 X-HE-Tag: 1750768746-555989 X-HE-Meta: U2FsdGVkX19xG6ABiejnKlsTjy7bn5rw//wmHpNWExMlaXEHwG+7V2mZAsE+I3idy7F7U2t+ZByYsuvU5wvPPklG62HKGkMvS3788dmW25zGMgr1qj+LmmzNrk0zTGEin4sZ9EFMZMxotedBGGwdy0uW6jUCfg0c1NbzbhvhwpY5fLNYM41oaVtnh0P7/627lwy/FwwChBcJx8Fz22MGFSB6oCiLQo7gXqjtFDkRIz7qhhJ0uuGE8rHsZXfvIE41qX4NEWffamjAvkqgVyM7Jl90kpEzZ2Ka4ZPcG3pnDQ60SqX6pNfk/WLXa5IXlg90DSAogW10IAYmjBs5xgSuJdCaOgmpay/D4QGS6XHu0PD/pY5owct886FdjGPyd65LRDImTDGmbmBaLpyGJ3ZE+ME9S4O1kcxWhjrv/0NvnP6Q5hHjh+RxQx3Px66EQ3kv2kGpRpl6p6vbnYL9IjP7usB380CwaPWCyRXhLdnuR6kADxLZknNmxBrJ+oBy6dGsUWzGemj7xKMzppjfJmCDHhdVHuIImn0xdvjdTt7JuHy2E9MCOsHi+4eSKhllsCgv/Bqv/0cvkkOotPrABVoqUVlelIPfO71UkUDcgekDdE8cyaw/pcnTA2PDlpYLnroTEK9/OcU7gFgLE04nK8oWmFl/1DZIlpZukg/aAoWLRxFzZosCty/7NlKD7x2IW+fzG3za7osHTcfLqMlKQXt3P5jV9u/g8rskfN3giIFhi8kK+YNXY4BprpNeZpBoVf0AEToCYPsjoSuR1Yg9a6Ynm8OAVYpPn5eDgpEVZqM/p30EbK1kSXCwHQIxUESvYCYB0JA/v5+W09kJM0iaBvoT31iXxVaTOl/BEGDep58/o2YC8ZbW/8ET/3r4CX/OeNaUK/LdLwBp+yT5OJDiYMcmaUxW0EOQQhJ/n+yosPMU99htDdP/3gGIYLaN0ygJ3vR04m2rBMT9qXT/qZbwMnh qI0Xyy8p /WfTPd/SR+cgFGH9gYHC+T7DkP5iEEST8WsSfFFxyjsCwZ/i22dgl3+Zy9tKHcqCCvWQnZx00UIAUOVRuBJ4ZfCl+XgQdADKDYwvugUuvvUG1qe1GBv52GUyV+slZa6O8iagnCO7IVf5HIvVqa9NCY1YqIS4MDEdyrtKp2HvxFFK05FRhmXeuV3YhyoTjwrih5u7t30u3Hh/9OuPhbBYwbnk13KcUonysFSp/NmFzAvexEtVHCxuodiU/hXPSo32P5q+r9JAMm/Z/wAneI165JU18uyonUrWwbwGWJHSv0oXy8g2UBz2LA74EP9n/ReQaT/aedS7qLxrjTqwFcuAtR4GuG4jXFRytraU9Eovihx9fEdqPo5e83RXjS+inYaZdCeNVd8/wgstecqztZiVHMngxrXFDk39ooxI10b747Fbn3yuL02NLToE/i/5qAWhzQTCs1xDx2GcL6KO1fxSFvKXspsAdko8C1puN7JPhVjPL14alUWcJ0orswjWQMFpHlFfAANO18wJYG60s5iq+6/5KnL3HugPJm5Sc8XhYaVxWXXmRgbVf5WjoEoCFiWdPK2vV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jun 24, 2025 at 10:02:05AM +0100, David Howells wrote: > > There isn't a very easy way. Also because if you actually need to do > > peer to peer transfers, you right now absolutely need the page to find > > the pgmap that has the information on how to perform the peer to peer > > transfer. > > Are you expecting P2P to become particularly common? What do you mean with 'particularly common'? In general it's a very niche thing. But in certain niches it gets used more and more. > Because page struct > lookups will become more expensive because we'll have to do type checking and > Willy may eventually move them from a fixed array into a maple tree - so if we > can record the P2P flag in the bio_vec, it would help speed up the "not P2P" > case. As said before, the best place for that is a higher level structure than the bio_vec. > Do we actually need 32 bits for bv_len, especially given that MAX_RW_COUNT is > capped at a bit less than 2GiB? Could we, say, do: > > struct bio_vec { > phys_addr_t bv_phys; > u32 bv_len:31; > u32 bv_use_p2p:1; > } __packed; I've already heard people complain 32-bit might not be enough :) > And rather than storing the how-to-do-P2P info in the page struct, does it > make sense to hold it separately, keyed on bv_phys? Maybe. But then you need to invent your own new refcounting for the section representing the hot pluggable p2p memory. > Also, is it possible for the networking stack, say, to trivially map the P2P > memory in order to checksum it? I presume bv_phys in that case would point to > a mapping of device memory? P2P is always to MMIO regions. So you can access it using the usual MMIO helpers.