From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D625EB64DA for ; Thu, 22 Jun 2023 20:28:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8192A8D0002; Thu, 22 Jun 2023 16:28:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C73E8D0001; Thu, 22 Jun 2023 16:28:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B66A8D0002; Thu, 22 Jun 2023 16:28:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 597458D0001 for ; Thu, 22 Jun 2023 16:28:40 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2400C16049B for ; Thu, 22 Jun 2023 20:28:40 +0000 (UTC) X-FDA: 80931522000.06.609A68A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf06.hostedemail.com (Postfix) with ESMTP id 4463018000C for ; Thu, 22 Jun 2023 20:28:37 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WxMN7ynz; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf06.hostedemail.com: domain of kuba@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=kuba@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687465718; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LtPcEn1t+rEPBjpna07BNFcfggJNoraOF0kjQbsuiHQ=; b=KhuT56WuMkIdwvtnH7PulVXI3tF+6Ihk5hz2J/2tLqHW/aLzGQOPj03t2Mskl3KYHtchL5 sCl15oN7RhTUdi24MvW9wOiSXffm6xCtGqv43Gsxn3OsovDAVQsDVKVeLUv0OnA6CGIi+u RVgzzgzHKQuHo9PXA/YsEONalsclcIQ= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WxMN7ynz; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf06.hostedemail.com: domain of kuba@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=kuba@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687465718; a=rsa-sha256; cv=none; b=c3CSrcHsC5W/xmbZQY9zdjwViMsAQ0UaLvshwC9TzQP9NbO2ZgFG7OsVUF7G5GneyEGQ0A Ntkxrsrr7LbgSPBBU6mn5u6skdElf8XkK+KPME+VJgWtJgytyCV3mcu9ql1AUxATg8Xo7d Z/7dUE20jrwjbXnSBnli8V881E9nmI4= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 35F7B61710; Thu, 22 Jun 2023 20:28:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 28BA5C433C0; Thu, 22 Jun 2023 20:28:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1687465716; bh=y/yWd5uVMRCaLK5FhZxHraUGBdV/mEzDKca+3yl4kns=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=WxMN7ynzoeDIgKp9qigd+U7A+dKCvNZNhyw9863fmtMX3FKqJWn+0zNnaxc0UMlYo pg5eLKEaUwtXcJ846Q6rTVD058OFrAkHSmXXIY/MNsZe7wZL6ivWyl2ySsoPT9Qrs9 aLBR6Ev6OzLVxLEn/6tLTRGJwuoZq+dbZlkp8MSvpfXXAEAtVn5FV5LXc8rhnwQo/k IC+sfBPVYRsCJ8bJJEmFrUo4j8cAFOSk9Y3IFdxDok1bUE1kDnAekvwAnlBgAqxyQu RvfVhnW7aIfE3tyGlGf6R3lJKP7Zfo48Sgw1UFO4Ma9VUlFJ5ZQSqdFE5I9t4IxrE9 O7BtuG0QmHmqA== Date: Thu, 22 Jun 2023 13:28:35 -0700 From: Jakub Kicinski To: David Howells , Eric Dumazet Cc: netdev@vger.kernel.org, Alexander Duyck , "David S. Miller" , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Menglong Dong Subject: Re: [PATCH net-next v3 01/18] net: Copy slab data for sendmsg(MSG_SPLICE_PAGES) Message-ID: <20230622132835.3c4e38ea@kernel.org> In-Reply-To: <1952674.1687462843@warthog.procyon.org.uk> References: <20230622111234.23aadd87@kernel.org> <20230620145338.1300897-1-dhowells@redhat.com> <20230620145338.1300897-2-dhowells@redhat.com> <1952674.1687462843@warthog.procyon.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 4463018000C X-Stat-Signature: kasypd663hkwz1m6hig5md1ipghrna5e X-Rspam-User: X-HE-Tag: 1687465717-870303 X-HE-Meta: U2FsdGVkX19xiIUxKAUWErfg3ANlH8woeZJcGQKUjSCs/0elVHmq3v4jXlthDat+td5o+BvUCbcnYkgrAPM/V5s/7sUAcxsQvH7YBYgww8XmhMHCWBlPOmVzSuL7Lbs+tUebJ+jl/yIuxWI7b64jZNcGxciDcng3Gv/7W5lXT3mN8v22NfgaGxZBqlZm6ID8667AOqguI5GeoOU8eazxwveI8RLAAE+Qj+1B+RVfUjIynbtNsLkPlXErvrmOjDP3bMoNMtaTbtheFxgwaqwioX9QNbwFzQeKxYJytDVDiv7xmUx3gLCABkeDVvmPLPwAFzrSUmbL0O6e5AX4S3vbYni1aXxrcfmpBqsal0c25cPrZBfJwgd8kkDCfR058WNEVaep7xolUXwrZSmg6oreuo2jvAaV3MkGNkP03yd9tUZGAYD6TCx6S46UXfDgXS9wANkbFsbCmPaooXIdZ8JS+OLd21+KBtabd6Sjwr/afuMhXui46QS8v/PdAWX/Bd7EfvobhQ5jiDqqF10jXNbHfPPPCYZLr+3/80e5bo+iAy+Krb0y0MqNA1jCbs7D2tSlrK0EN37/iE8RzvMaoTJrmVXwFdFXhsUA1xJ3hGpqp/Ubu+KpJnOilaejcRht5lC6cwtGJba9/7IOf4ywlKB924Fpqsl4zy95B2PSh9aZMK3WyiFS8ZOXicPwqGNev4werfeNsNyE2Dh2nMkYFJSpq0lGR1CqnGOu4oiv0uumpopKa2Z8nMeZZpuJHTMXuIwj5MMcFHLHV3z9HYL15gUztJYpju/o8m86uI6wsqCxSf+URXmMU1A8kt9TuIwpLmtu69vyjdlfg2eEP5r8ZrbnB7XpmVqf/IyPZwXwOltHAZZYiV7FgI9U7PImMnDVplF/NpnPHx+FfONmBFWKFFeI8jfNUsUt3YJOQH2IrVNSkFVrt92Q1crtxaYCiEosiWwKxngGO/P5T/DC8VieGX9 o8IDVOuX La8XNRmabFcrJqTSzGehJXXW01zpRcMMiNgwSqNPgh7ZtSV85rfaRN5VFUCfp0Ys0gr7M4MAqgjRrC36nxbVB32f7jiwR6iJpKTE4SfTFwHc/wb/8HvoAz85DlvkQzAHWTgHymmQkoQOOxa/SavBDkWF4awbCG/1kezUnF9LtyZxCjYeWgl/178z14rpIeUy9ilt3rAVhExd9KOU18tehrNRnPCoEBRf1uKgWRVSVWtwYifSCbsJU7vE/djC4+C/PjeCsxaN3f1eiabKUrC+7hNXY6RONIy3k76iTnvjYdA/wuRRBTJ1VF+MTjJ5bwZvweHSzDa/wg0K1XWT/IpJPIItDYEjoNtISPuBdvxmfZRqtGbBeyeUZ1AoDchqdUAZ4ITIWzwTMbpOOfI91MxlaVgqHV6mz18HdMlPl X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, 22 Jun 2023 20:40:43 +0100 David Howells wrote: > > How did that happen? I thought MSG_SPLICE_PAGES comes from former > > sendpage users and sendpage can't operate on slab pages. > > Some of my patches, take the siw one for example, now aggregate all the bits > that make up a message into a single sendmsg() call, including any protocol > header and trailer in the same bio_vec[] as the payload where before it would > have to do, say, sendmsg+sendpage+sendpage+...+sendpage+sendmsg. Maybe it's just me but I'd prefer to keep the clear rule that splice operates on pages not slab objects. SIW is the software / fake implementation of RDMA, right? You couldn't have picked a less important user :( Paolo indicated that he'll take a look tomorrow, we'll see what he thinks. > I'm trying to make it so that I make the minimum number of sendmsg calls > (ie. 1 where possible) and the loop that processes the data is inside of that. The in-kernel users can be fixed to not use slab, and user space can't feed us slab objects. > This offers the opportunity, at least in the future, to append slab data to an > already-existing private fragment in the skbuff. Maybe we can get Eric to comment. The ability to identify "frag type" seems cool indeed, but I haven't thought about using it to attach slab objects. > > The locking is to local_bh_disable(). Does the milliont^w new frag > > allocator have any additional benefits? > > It is shareable because it does locking. Multiple sockets of multiple > protocols can share the pages it has reserved. It drops the lock around calls > to the page allocator so that GFP_KERNEL/GFP_NOFS can be used with it. > > Without this, the page fragment allocator would need to be per-socket, I > think, or be done further up the stack where the higher level drivers would > have to have a fragment bucket per whatever unit they use to deal with the > lack of locking. There's also the per task frag which can be used under normal conditions (sk_use_task_frag). > Doing it here makes cleanup simpler since I just transfer my ref on the > fragment to the skbuff frag list and it will automatically be cleaned up with > the skbuff. > > Willy suggested that I just allocate a page for each thing I want to copy, but > I would rather not do that for, say, an 8-byte bit of protocol data. TBH my intuition would also be get a full page and let the callers who care about performance fix themselves. Assuming we want to let slab objects in in the first place.