From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06AC7C77B7C for ; Tue, 24 Jun 2025 01:17:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C7C38D000A; Mon, 23 Jun 2025 21:17:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 99DBE8D0003; Mon, 23 Jun 2025 21:17:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DAB08D000A; Mon, 23 Jun 2025 21:17:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 80AD98D0003 for ; Mon, 23 Jun 2025 21:17:56 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 46AB1141279 for ; Tue, 24 Jun 2025 01:17:56 +0000 (UTC) X-FDA: 83588532552.17.4373D00 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf03.hostedemail.com (Postfix) with ESMTP id BC66420008 for ; Tue, 24 Jun 2025 01:17:53 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; spf=pass (imf03.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750727874; a=rsa-sha256; cv=none; b=van2He18Q3wT8Mf4lN1qPaBRaC6JDbWH4i+sQeH2lw7zTDaeApsn9Q/uTDUkDZGvmewj/6 I5/pRhLzAAfQzRiRzvPgOUjWWwqchJok/tS29wffyK9PADMX+w9TcHpq+u6s7Kdoof52kA uT26pJHackLaftt1b0ySMWbX9gyB7wo= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf03.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750727874; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zNf/+h1Squ8U70Pqn6xEG15Pb1yJWsBqS52pnrDeBGs=; b=ON8jjinXDsDUsLHN1DAYo3unGT9WNzKFqGWpMsdyrR5os8hCRKZywjUrS5gVlxOU9ysn/F v45pBUXhzWsZRCXRJ9cE1rFdPyNvbWWKmjPP/+yppV5RoBEu67aNbu4Hrke+I5417SME6p 8E1fLiX1WtI+fn9AeBnDCzI3wvwSDIg= X-AuditID: a67dfc5b-669ff7000002311f-66-6859fcbd1974 Date: Tue, 24 Jun 2025 10:17:44 +0900 From: Byungchul Park To: Mina Almasry Cc: Harry Yoo , David Hildenbrand , willy@infradead.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, kuba@kernel.org, ilias.apalodimas@linaro.org, hawk@kernel.org, akpm@linux-foundation.org, davem@davemloft.net, john.fastabend@gmail.com, andrew+netdev@lunn.ch, asml.silence@gmail.com, toke@redhat.com, tariqt@nvidia.com, edumazet@google.com, pabeni@redhat.com, saeedm@nvidia.com, leon@kernel.org, ast@kernel.org, daniel@iogearbox.net, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, horms@kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, vishal.moola@gmail.com, hannes@cmpxchg.org, ziy@nvidia.com, jackmanb@google.com Subject: Re: [PATCH net-next v6 1/9] netmem: introduce struct netmem_desc mirroring struct page Message-ID: <20250624011744.GA5820@system.software.com> References: <20250620041224.46646-1-byungchul@sk.com> <20250620041224.46646-2-byungchul@sk.com> <8eaf52bf-4c3c-4007-afe5-a22da9f228f9@redhat.com> <20250623102821.GC3199@system.software.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA02Se0hTcRTH+917d+91Obguq18JJjOJDHtKnkpKivASFEUQlfa45a0t54NN TYNgpihZWz6CarOYWWpqjZbpCrWaZkZR5ov10PnIIiolLXEqlZtE/vfhnO85n/PHYUn5XclC VhWfJGriBbWCllLS797XQ+om9ylXWtr8odBSSUPFWCqU9tgkUFhejeCn6z0DI43PaCguGiWh 8HUmBb8s4yQMNPUxUGHdDs6STxTUZteQ0HehmQZ95gQJda5BBs7YyghoqTZI4OL4TRJqdD0M tD0spKG78o8EPtn1FDw33qLAaYiAJvM8GH3xDUGjpYaA0fNXaShoNdPQn+lE0NrQR4Ep3YDA Uu+QwMTY1A7T024mYjHf8G2I5KtuvSX4B8Yuhjdbk/l7ZcF8jqOV5K3lZ2neOpzP8B86a2m+ +fIExT+wjRC8PmOQ5n8MvKP4ofoOmrdUdVD8S3Mjs9NnvzQ8RlSrUkTNio2Hpcr2YReR+EqR qhvOI3Uo1y8HebGYC8WGtl6Ug1gPv/86112muCD8uVNPuJnmlmCHw0W62Zdbim/U50ncTHIP aZzR5OE5nIAv5fd78jIuDBcXfZ/KS1k5V0LgrKoGZrrhg59f+UhNDy/Bk9daSbeX5Pxw6W92 urwIZ9w3eVxe3C7c/i7bMzqXC8SPq58R7p2Ye8nivvR8cvr+BfhJmYPKRT7GGQrjDIXxv8I4 Q2FGVDmSq+JT4gSVOnS5Mi1elbr8aEKcFU09UsnpySgbGm7ZbUccixTeMtv6fUq5REjRpsXZ EWZJha/MvnmPUi6LEdJOiZqEQ5pktai1Iz+WUsyXrR49GSPnjgtJYqwoJoqaf12C9VqoQ4HN oVtdQY/qf7Qkznqz7cSWkoPjW5NPLAvxVtSdrd7QH/slfO3Xvbo5AQF2bdgL3/RjkRZZnm3z tov0uuiw6Eg946c2Zfuu6WJL75zbkX7gp9apVR+RDgX2P/U3ZdSOaHf1DobcVhcPCCPOrhW9 VFBWgeFy1LmKcE3oJstAwuzuBQpKqxRWBZMarfAXkil51kQDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA02Sa0hTcRjG+59zdnZcTU7L6p9Cl1WUlqbQ5cUuWB/yX2CUQrcvterQRjpl U9Eg0iZG0kzNqOYSxdK8jpbptJSaZUlpNi8s72iakGlZinN28ULkt4f3eZ7f8+XlaNlNkTun UkcKGrUiVM5KGMmhnTrvqqkTSt+6765gNBWxUDgRA3k9FhEYC8oQ/HS0i+HHy9cs5GSP02B8 n8DAmGmShv7aXjEUmoOgO3eAgWdXy2novfGGBX2Ck4Yqx7AYrlgeUlBzr04EjWXJIkiffEBD eVyPGJoqjSx0Ff0RwYBVz0CdIZ+B7uQAqM1aBuNvhxC8NJVTMH79Hgs3bVks9CV0I7DV9DKQ EZ+MwFRtF4FzYpqR8apLHLCe1AyN0KQ0/yNFKgydYpJljiKPH3qRJLuNJuaCaywxj6aJSUfr M5a8ueNkSIXlB0X0umGWfO9vY8hIdQtLcga/UcRU2sIclp2U7DonhKqiBc2WPaclyuZRBxXR II+JG02l41CKRxLiOMxvxe1fliYhF47h1+PPrXpqRrP8Bmy3O+gZ7cZ74vvVqaIZTfOVLNbV zuolvALfTuubzUv5HTgn++t0XsLJ+FwKJ5bWiOeMxbju7idmrrwBT2Xa6JldmvfAeb+5ufMq rHuSMbvlwh/BzW1XZ6tL+bX4edlrKgW5GuaRDPNIhv8kwzxSFmIKkJtKHR2mUIVu89FeUMaq VTE+Z8PDzGj6VXIvTaVa0M+mQCviOSRfJLX4n1DKRIpobWyYFWGOlrtJrfuOKmXSc4rYi4Im /JQmKlTQWpEHx8iXSw8eE07L+POKSOGCIEQImn8uxbm4xyFXxzdbyPvMA9KxdeqMt0Rfy6Sd kRU7KzJ9elcvvOsZsWL3i41O7+Jr9TnBfn6msBImsXKwxxI/PFT0a7//yBrb8aeFHTdCqi6V +uoKA4P8Qx6t7PyQ3tj/LmBBZ1PmJvs4fbx+ODIxv6RRMoYb7Ns9Y9bd8greW3xGdrmloXFz ilHOaJUKPy9ao1X8BSc5QhUmAwAA X-CFilter-Loop: Reflected X-Stat-Signature: 1f17oxafwr5jthaqmf58ki9cmftx7ijx X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: BC66420008 X-Rspam-User: X-HE-Tag: 1750727873-167024 X-HE-Meta: U2FsdGVkX19J3fM4OlOY2iDvlMw/rNTP0umQ5XEcCb177GqIZMJkRzWGxq2g9UR/a+pAOmSuW3hPCWgwMHgriVRpr8O9aksURs1y5LzIZzh2HOLUQWyjhMtB+i+SI5m1o0k00FzPZT+lkFUAKFE9nOaXeUfaNkSsMztBiAzeD/yE3tUVY6+QoYLCogA7msvSHZjDTpK+feNu2u5/FhAh2tHXk2WV3iuE9EHj3qAk7x9M3YjrPY9yTGqgoQt5lqr3yObn+XxYk4fCIr1AUS2uuoCcF0kVh2ym54si2pUEgZwMwkg8M3TiNaQRWOn9bWWmOoSy73GpT4kyL6+JC3L1t8i0yTtp+ZRwbyLScRcM8PEm2af5IsaIhjwuygaOcgRODRNbhp/cwEBhonFFLYOWZhjuLNDhmKi14pXAtTnlUwPYQDerhnlLoXvdm74YARnYazN0L7+9kXks6u42iVmTDW5J8SK8H3fLIJ9H7Ae9Ztw4FqiFkDAL2csShdTGbPqhWvGFfyIgkC9VdstCusLWZmQBPnJzhAg0wj24vkt/5lWXLO1z/0pIUueaKMqWDBj1vgfH5AMvt9xP0ePp9VdBY34zA+gwbABputbM/2GQIuFpAtRbjZt8XOFY8849gpCg/ifQaHAhPiclm9lsCsHK1Eu+qu3UybwM++xyF95jGhpb1mxbHCd/n8OluC+732XSCPTaq4BgFnO6eLr6wpX5WtmqFVAG10yfLoOUvNp02j/IXT6m6aNJczdFPk1rEnefKZGOa0SBfC+Zc/abcCpYNicfMh5ZCh79tdHWPNa2+MD3Fqkp6ccgADmSzO1EwUb2WNO61imkknW5abWDxTKCbUAUvrAciyzRSxKfQv4n1f+7USHAYJyCLuOrvolYZSsL0HBSDlLC2I4GyINbaKSg1PM+m1pfsA8+FVr80AZb24QeZHAZtvihhUaOjBt5ljSFjUPr3r8tL6UuFP12fEU 9dA7iSV2 /+eF6CuEt/skk5SyvnJeNyQds2s1nTaC9Xx9sePTkCrJr82fmdxrSSov60BxF3Wgfxj5Wngz1PoS0WqdN7R7CwyurLg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jun 23, 2025 at 12:09:09PM -0700, Mina Almasry wrote: > On Mon, Jun 23, 2025 at 5:18 AM Harry Yoo wrote: > > On Mon, Jun 23, 2025 at 07:28:21PM +0900, Byungchul Park wrote: > > > On Mon, Jun 23, 2025 at 11:32:16AM +0200, David Hildenbrand wrote: > > > > On 20.06.25 06:12, Byungchul Park wrote: > > > > > To simplify struct page, the page pool members of struct page should be > > > > > moved to other, allowing these members to be removed from struct page. > > > > > > > > > > Introduce a network memory descriptor to store the members, struct > > > > > netmem_desc, and make it union'ed with the existing fields in struct > > > > > net_iov, allowing to organize the fields of struct net_iov. > > > > > > > > It would be great adding some result from the previous discussions in > > > > here, such as that the layout of "struct net_iov" can be changed because > > > > it is not a "struct page" overlay, what the next steps based on this > > > > > > I think the network folks already know how to use and interpret their > > > data struct, struct net_iov for sure.. but I will add the comment if it > > > you think is needed. Thanks for the comment. > > > > I agree with David - it's not immediately obvious at first glance. > > That was my feedback on the previous version as well :) > > > > I think it'd be great to add that explanation, since this is where MM and > > networking intersect. > > > > I think a lot of people are now saying the same thing: (1) lets keep > net_iov and page/netmem_desc separate, and (2) lets add comments > explaining their relation so this intersection between MM and > networking is not confused in the long term . > > For #1, concretely I would recommend removing the union inside struct > net_iov? And also revert all the changes to net_iov for that matter. It seems like many got it wrong. I didn't change the layout of net_iov much. I did nothing but replaced the existing pp fields in net_iov with a wrapper, netmem_desc that still has the same fields. Even with net_iov reverted, net_iov still has the fields of netmemdesc. Just to clarify, netmem_desc is the intersection but net_iov is not. net_iov is a network's thing. > They are all to bring netmem_desc and net_iov closer together, but the It was already closer together even before this series, since netmem_ref is used to unify the related usages. > feedback is that we should keep them separate, and I kinda agree with > that. The fact that net_iov includes a netmem_desc in your patch makes > readers think they're very closely related. Again, they were already very closely related before this series. Of course I agree with that it should be kept separated but it's another issue. It can be done on top of this series by e.g. Pavel as he said. > For #2, add this comment (roughly) on top of struct net_iov? Untested > with kdoc and spell checker: > > diff --git a/include/net/netmem.h b/include/net/netmem.h > index 7a1dafa3f080..8fb2b294e5f2 100644 > --- a/include/net/netmem.h > +++ b/include/net/netmem.h > @@ -30,6 +30,25 @@ enum net_iov_type { > NET_IOV_MAX = ULONG_MAX > }; > > +/* A memory descriptor representing abstract networking I/O vectors. > + * > + * net_iovs are allocated by networking code, and generally represent some > + * abstract form of non-paged memory used by the networking stack. The size > + * of the chunk is PAGE_SIZE. > + * > + * This memory can be any form of non-struct paged memory. Examples include > + * imported dmabuf memory and imported io_uring memory. See > net_iov_type for all > + * the supported types. The description should be changed depending on the result of discussion. However, I will basically add this doc with some adjustment. Thanks Mina. Byungchul > + * > + * @type: the type of the memory. Different types of net_iovs are supported. > + * @pp_magic: pp field, similar to the one in struct page/struct netmem_desc. > + * @pp: the pp this net_iov belongs to, if any. > + * @owner: the net_iov_area this net_iov belongs to, if any. > + * @dma_addr: the dma addrs of the net_iov. Needed for the network card to > + * send/receive this net_iov. > + * @pp_ref_count: the pp ref count of this net_iov, exactly the same usage as > + * struct page/struct netmem_desc. > + */ > > > > > > -- > Thanks, > Mina