From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A5851EB5FD for ; Fri, 21 Mar 2025 20:35:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.170 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742589357; cv=none; b=R3fOnTbVu1QVMW0/hZ4XoFNQ+yVafCydxrBOZRqPL+wPE9Za/B4nJ7+lPEbwZWASiDV+e41NT/T5VpIlVDxySRGEQG4NF9bbmZQ3PhLalRR1dFyJf3pTQPINsbIPyPbcYD0B310+EGPr9ggN2sfOsgjFlz3viSWu3/ofNHSiPpI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742589357; c=relaxed/simple; bh=Q4yJA3jsyp0Q05YHLWgkGPj0BYZnr/626JQJoRaup9Y=; h=Message-ID:Date:MIME-Version:Subject:To:References:From: In-Reply-To:Content-Type; b=uexQbz4aAgkA2NV4FV6RTs0nfn2BI2vLRBXVoOG74ARt9rpu/h7CqBd1vR6we+TqwY2BYO/EMPdCfcK8p8CTYmTjIxhHsWTOkS2QLQU78eawPJ1t6aspDWHOmIZ88dGlUNKwQlCYVaB034r55+HO7NC61Rap/JrXhY0K9DHb+nY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=MZQKcIkO; arc=none smtp.client-ip=209.85.166.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="MZQKcIkO" Received: by mail-il1-f170.google.com with SMTP id e9e14a558f8ab-3d439f01698so11375755ab.1 for ; Fri, 21 Mar 2025 13:35:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1742589355; x=1743194155; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=qvMwDu0BMSrh7CaSCaQYm2+c+HftuBgbOun19NXMMCQ=; b=MZQKcIkODtlB8IzsiOOakT6x76IB4Yq1lgJYIKJSTEfoaBWSl+wJL0k7IvBWJnDZ/C sRMiO2aGs0q36yGgrmHcpDsyuol45b/Ixhfn+eePULsLQdR7fznLuGjBvz+rBjB2iTSR WomG4LprAGXddDy2JYJ46fcToSzGFiyeSw/NtomiMHWW9jBES0LQdHRbC+JZLluB5P3a JPQREz4U8K1fJcmFBnSttbB2QZE+/eCdUX48UmYC/uE11GT0ieI9pSOi5alU/a243cmA YJp5ZuKC+PmiS4NXhoLZiPMvz5ysXo3aUc5vqqv8jU8DpqxVGBS0DwctRuIWU/RsVSds 9RaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742589355; x=1743194155; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qvMwDu0BMSrh7CaSCaQYm2+c+HftuBgbOun19NXMMCQ=; b=fND7pxXTVshfi5JAPenSK2UJE5q2L39feBxiBeLUiD0Lhj4WUg3R6FZ4ds8u4MIoQM 5vY1Q/eEJTUqQNqL3tR9DgXl++TRpOgmUNd0TbYqySwfsyhPFMCORytkaeNgnHeKsW6C 39VjznXTw0ujnognZ4tmNLVtgtoJafpWdpgolwwyzh2n4VcE3XjVEctfXGTctg0hl5RA hTxwO22eMD0opUpIfBS2q8dN9Vx4aQabXworPaGGn6M0V2zqAO+DKmFol6oLtXToq1qm jaKup0PThJTjRfBORoxJHoEs9tieJncdxdC0YpeJ1rV72Y8Sy8B6AiXiZ4FrHwvgR8NX tFeg== X-Forwarded-Encrypted: i=1; AJvYcCWUljnG42Yo2gPnAWkRDKuoU5U2Zdf8l3zk3gxAHx4eJnDwottfqU/UsqZMtKb7h7Qd7Oj4OiKqegk6Ze8=@vger.kernel.org X-Gm-Message-State: AOJu0YynGtLMNCZeYfz+rRHra/GFJ//Lzqwdv6wEt6QTAfktikBTcYtg Z65RhSb8FJmiURpAc4LhHg/aM4rPl+K+LJQLvAFrQprgv0TDAvCN+HYozHdOGTc= X-Gm-Gg: ASbGncvZCkmB3+3/6RqoV3IIVCNBYLrw+ie5dEdJy8rGaR8ZrW+nHz8rFxNvw5IDTJT FJlxaYY7mn4rTkMXcHbkUmVAZviy6DN2Twdh3G2nu8Awm8fOjdsFOok2e4JLPK3XV8yd0yRUZzP tMcbnh/VzEfEX+GEocZXrdpg8tvEb6WY71pDWh1wUotEOTZkpUVl0/3zgaJZjx8lYorN172vXXc akFzRo1LMKzwCUHvRpYzl6BKNdGtf9LngNAkQZ8fXToJOJHVU/qSZTbDLaa5EJUmC2WqUQbTiHM 7uxvWRvLK2DVsZWwa8iPd2btND5uQ9V94h6wotJCaw== X-Google-Smtp-Source: AGHT+IEPWzk5ry4zHWNFdi8KTg5KcDVgbTXsCl1DzYqeTt8WkJX5AIeUX7DfgyW/W2CDL0vQ6sVZlg== X-Received: by 2002:a05:6e02:1f01:b0:3d3:d1a8:8e82 with SMTP id e9e14a558f8ab-3d595f91c7emr52500135ab.9.1742589354635; Fri, 21 Mar 2025 13:35:54 -0700 (PDT) Received: from [192.168.1.150] ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3d59607ea6fsm6112915ab.24.2025.03.21.13.35.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 21 Mar 2025 13:35:53 -0700 (PDT) Message-ID: <4e582b92-2f4d-4a0d-b479-3cf4f054bb5f@kernel.dk> Date: Fri, 21 Mar 2025 14:35:51 -0600 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC -next 00/10] Add ZC notifications to splice and sendfile To: Joe Damato , Christoph Hellwig , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, asml.silence@gmail.com, linux-fsdevel@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, horms@kernel.org, linux-api@vger.kernel.org, linux-arch@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, kuba@kernel.org, shuah@kernel.org, sdf@fomichev.me, mingo@redhat.com, arnd@arndb.de, brauner@kernel.org, akpm@linux-foundation.org, tglx@linutronix.de, jolsa@kernel.org, linux-kselftest@vger.kernel.org References: <2d68bc91-c22c-4b48-a06d-fa9ec06dfb25@kernel.dk> <19e3056c-2f7b-4f41-9c40-98955c4a9ed3@kernel.dk> Content-Language: en-US From: Jens Axboe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 3/21/25 10:36 AM, Joe Damato wrote: > On Fri, Mar 21, 2025 at 05:14:59AM -0600, Jens Axboe wrote: >> On 3/20/25 11:56 PM, Christoph Hellwig wrote: >>>> I don't know the entire historical context, but I presume sendmsg >>>> did that because there was no other mechanism at the time. >>> >>> At least aio had been around for about 15 years at the point, but >>> networking folks tend to be pretty insular and reinvent things. >> >> Yep... >> >>>> It seems like Jens suggested that plumbing this through for splice >>>> was a possibility, but sounds like you disagree. >>> >>> Yes, very strongly. >> >> And that is very much not what I suggested, fwiw. > > Your earlier message said: > > If the answer is "because splice", then it would seem saner to > plumb up those bits only. Would be much simpler too... > > wherein I interpreted "plumb those bits" to mean plumbing the error > queue notifications on TX completions. > > My sincere apologies that I misunderstood your prior message and/or > misconstrued what you said -- it was not clear to me what you meant. > > It is clear to me now, though, that adding a flag to splice as > previously proposed and extending sendfile based on the SO_ZEROCOPY > sock flag being set are both unacceptable solutions. > > If you happen to have a suggestion of some piece of code that I > should read (other than the iouring implementation) to inform how I > might build an RFCv2, I would appreciate the pointer. I don't know what to point you at - you need an API that can deliver notifications, and I'm obviously going to say that io_uring would be one way to do that. Nothing else exists on the networking side, as far as I'm aware. Like Christoph said, struct kiocb is generally how the kernel passes around async or sync IO, which comes with a completion callback for IO that initially returns -EIOCBQUEUED, meaning the operation is started but not yet complete. Outside of that, yeah need some delivery mechanism, as you're generating two events per zero copy send here. You could obviously roll your own, good luck with that, or use the existing one. -- Jens Axboe