From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1516291E for ; Wed, 9 Oct 2024 16:57:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728493075; cv=none; b=TFjQz318FGHpcetMEgN4jAX2ShG3DnSJZynGY5Mppld6Ato5vcpK/VcYe173gyyyIDmlKlI5ARBKObX0ruLtlLujD8t0sPli57MWalnmFs2KXjxqYPDLrcBolfY70wI0IwC021ZgKaXi6tawIqWLJZa/SMu2ViyN2iywsbS5Hus= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728493075; c=relaxed/simple; bh=Ju4hZ5bZoH8UntlFeshzeL1LyX5BnYG+8f73zLzxi+0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=BTRp7cTDPHiCMooGopBpI0QqI797/8XfMycPMrM0WfGQJVRQNrCGEf8Xon8fXusKWY7kdmHeIY3ntTr7+0iM0j0DyDjIL/vdD5FFaGEa9+73VsuuGHeFVEHaWLM7DxNa817oNpjTudrqxBiLe/fFEjSNUmwGOko/T3uiuNIBsU0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=cbuZk7r6; arc=none smtp.client-ip=209.85.166.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="cbuZk7r6" Received: by mail-io1-f53.google.com with SMTP id ca18e2360f4ac-8354851fcabso4851539f.2 for ; Wed, 09 Oct 2024 09:57:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1728493073; x=1729097873; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=Bw7VwXAIznwpICo3kV2k/LCYf2ehWSVoVVH7gRs38NI=; b=cbuZk7r6GxWb2GHlDvO5shJ8HxBuJRT25J74OQMBH5AN1mBwIxntiummN6nyjPIwkj arxHVGHNgBK5+5uq22YB39ubgcc+1qta5xHUODCZbv2GP8X9tbiepbI5evTYcr3GSyYN nbkmmosDmfIimUfhfNtoaDFUseFLFybKqm1S14/rigrX7DXDZtIM1hdBaj82FO6BLARb a2jZN55cnDUeeuIQ3Tkyclry3UAoBU2qDkIiSQnwjeQlWHYWJYFpc7iNqxZDkjMfzzzX AsoiXu+7M1jKPKALlBDYd7tcUogjEqOOAi/WHizE2MwfC6zmukZuLm7X2XY0cbm+5EuQ nPwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728493073; x=1729097873; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Bw7VwXAIznwpICo3kV2k/LCYf2ehWSVoVVH7gRs38NI=; b=S9XSE53E7yrHbHR7UcxTs9wDmngvnFylwHGMxdgcnfjpD+KK2hMz7eRjIneBXyJo7W PNQST3A1NOjERG5/ezoCZ8+QHTm9juDsNURaXql7Hhk/XgmBIh8CgUztUmpkpvtvudG1 O+LQKdwaIRDRCmuUWEouCuBchAw/eXWsyueZePG1gwx13ZdkaNYwnWkt06TOOuRV1e7D XADUjHyW3CT2aKtXxjPda8e6Cjrfoh0F9ztKDtYtUO8JwUoxAhuKNxTdGUz+SEjPrz// Yt8R2SIrhoUkPGuiNUipNwKiM+K06gaWJ5eZQdDaeUM9S9iUTFG8LDLuqQdiRGil2d35 9MZA== X-Forwarded-Encrypted: i=1; AJvYcCVCfwSY9Kd5JVvNSImuMM5EJL5DVsZMl2Xx3PsiZqWahxHRwYejh0JGLZBjTsvF7QSHBZLK3JU=@vger.kernel.org X-Gm-Message-State: AOJu0YwjRE1+7NbwipNcYt8cWF6tFHPeiv3HcFzNUbMNPNk/ogRIbDmp NB9zK8EkjJiozgP2tny3xUd+B3LlscTNWTvVN5KO6vKkF2n/RF3KWQ/WSaZB1YU= X-Google-Smtp-Source: AGHT+IGdh7EgQyozYxNrOGrr9QuWmEDNHnNyIda/hil5TWKH9O1j55aApx0++gBFJWR3wL4VphHMfQ== X-Received: by 2002:a05:6602:2c07:b0:82c:eeaa:b1e0 with SMTP id ca18e2360f4ac-8353d5263e5mr413582139f.11.1728493073056; Wed, 09 Oct 2024 09:57:53 -0700 (PDT) Received: from [192.168.1.116] ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4db6ec72819sm2098146173.170.2024.10.09.09.57.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 09 Oct 2024 09:57:52 -0700 (PDT) Message-ID: <016059c6-b84d-4b55-937c-e56edbedc53a@kernel.dk> Date: Wed, 9 Oct 2024 10:57:51 -0600 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 00/15] io_uring zero copy rx To: Mina Almasry , David Wei Cc: io-uring@vger.kernel.org, netdev@vger.kernel.org, Pavel Begunkov , Jakub Kicinski , Paolo Abeni , "David S. Miller" , Eric Dumazet , Jesper Dangaard Brouer , David Ahern References: <20241007221603.1703699-1-dw@davidwei.uk> Content-Language: en-US From: Jens Axboe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 10/9/24 10:55 AM, Mina Almasry wrote: > On Mon, Oct 7, 2024 at 3:16?PM David Wei wrote: >> >> This patchset adds support for zero copy rx into userspace pages using >> io_uring, eliminating a kernel to user copy. >> >> We configure a page pool that a driver uses to fill a hw rx queue to >> hand out user pages instead of kernel pages. Any data that ends up >> hitting this hw rx queue will thus be dma'd into userspace memory >> directly, without needing to be bounced through kernel memory. 'Reading' >> data out of a socket instead becomes a _notification_ mechanism, where >> the kernel tells userspace where the data is. The overall approach is >> similar to the devmem TCP proposal. >> >> This relies on hw header/data split, flow steering and RSS to ensure >> packet headers remain in kernel memory and only desired flows hit a hw >> rx queue configured for zero copy. Configuring this is outside of the >> scope of this patchset. >> >> We share netdev core infra with devmem TCP. The main difference is that >> io_uring is used for the uAPI and the lifetime of all objects are bound >> to an io_uring instance. > > I've been thinking about this a bit, and I hope this feedback isn't > too late, but I think your work may be useful for users not using > io_uring. I.e. zero copy to host memory that is not dependent on page > aligned MSS sizing. I.e. AF_XDP zerocopy but using the TCP stack. Not David, but come on, let's please get this moving forward. It's been stuck behind dependencies for seemingly forever, which are finally resolved. I don't think this is a reasonable ask at all for this patchset. If you want to work on that after the fact, then that's certainly an option. But gating this on now new requirements for something that isn't even a goal of this patchset, that's getting pretty silly imho. -- Jens Axboe