From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f65.google.com (mail-ej1-f65.google.com [209.85.218.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3515A28C85C; Fri, 27 Jun 2025 22:09:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.65 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751062186; cv=none; b=cERefnwT7Cp97q2CFOgKSf9Ud08URexCw4xte+yF7wGJpzz2pGMZTK6rQQKD4hrxullnfTIH8SrFL2qPvOmMcyqI1HpjX3JjBkLG1SfwYM0mnjl6W6OAgIODLfR1X2M+I5XXfZdm2YCnRLTjOGniiZJDNI5mnuGyZADBfRGucMA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751062186; c=relaxed/simple; bh=lrNG4yzBx69D+gLKmLXwiFU1bu51syjy0/aoOe2CnwU=; h=Message-ID:Date:MIME-Version:Cc:Subject:To:References:From: In-Reply-To:Content-Type; b=XSWQnmuIivLVG/H5WLfghEXLBPp3WudSz+i43EBcVd13Wc+bPkDwa21jGCaxW54/UBe13T/azNjK8UhmAvPGs5jsukE1xeubHTLMa/1pWgegzP67ZCc7NRsJyjbRFRYnY9E4t2SknxZEOgCX6HpyjS4uDD8oE3iD04I9170OUuA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ovn.org; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.218.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ovn.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-ej1-f65.google.com with SMTP id a640c23a62f3a-ae0e0271d82so382277866b.3; Fri, 27 Jun 2025 15:09:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751062182; x=1751666982; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:to:subject:cc:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=zfSvvvGsrIO3AAylc8GW5D03bS+v1TQ3CSOYR17lOkE=; b=gW1K7KqhNhGD6NijsMEtE0ZCGrN3EnOBUZ0ZQnmUU7JrN9AU6JK0/d/9KsDR3VOi47 ePIBlKFA1EWIYdvk73ohQ0ncyAXDyfz2OtqtnOYTalY7HdEK5qSLii44OoV64qNVNbep q9uA14D6Pu/I/BO90YlRg4WBcwCsktmuDZuohXY+/BttXi3mJYILOlgiDyMMFf4FfCRW KuB11CABnDY6O350Nx9wYtRXAE4W1Vif7byaAXFmYUlQNKfH9R+hR8ggdYZzr8GKfypi VciqIOXFqca6Rf3RxEuNmtl+InDP7cIIwYr+5wZmk16N/1MCTVirya8Q1UXh3nfVsqld LfZQ== X-Forwarded-Encrypted: i=1; AJvYcCVJlVzE860eKC3ELgYToW/iUZ0tBIYDwKptoBYN6LQ7pkCHEpph5pVsfwcWNCs/1EpoK/Kpbn0Lhdq8q6c=@vger.kernel.org X-Gm-Message-State: AOJu0YyGAXj++5jb7iCEb/vuWud4bJNtiVtHSpoZzGtT6F/iQPNqylWC c6MvODATqLbNaoJrFNQ+veQO+XqmkaHJchX0AAwx9nBeDPXWRDbCvoISFhu0RziX X-Gm-Gg: ASbGncsNDYzEh2lDl8nR9N5VkUOc52omI0PKqQyFG7QpITrOgFtqm5xLFRxhapjOB4W 26rVHF1Bkpz1a9LWh2HVL3f9Wzkfq/S4HIYd7RuTby75k7srfnjm3nqm/k5l4TEk6G9By2EFVgi 99wIFzio0iy+om/SmXY91hSlb5supKBdPqTVOn8UPZ88kulcdzAZcA93DpCt0h4WxQOXD/p6XYz vOOAIuYLxukgHmkREY/nQ/AIJOKhcD8ZlYLNogPgFKMiIvc1peiAIF3Ly5tq5E8PNBW0Um8zlIg 5mwpCnVHboS+xoJuH+Nybok54y/6CQh3S7rBZp231viNTU3FcGwO2wRMRO44j07dWTs/C4jNS4b 4s9N8HiV8DV5nWxTepA3Um8RVYQyJQXfhHg== X-Google-Smtp-Source: AGHT+IHbohDQdHd7TX0s9+3ydPmnBsODPNRLXXo/o9/6RnkE2kl2hjdUReb+7Qcz1dyM/u1mWIJxkA== X-Received: by 2002:a17:907:94cc:b0:ae0:e18b:e92f with SMTP id a640c23a62f3a-ae34fdbe13dmr501046666b.23.1751062182371; Fri, 27 Jun 2025 15:09:42 -0700 (PDT) Received: from [192.168.88.252] (78-80-106-150.customers.tmcz.cz. [78.80.106.150]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ae35e2dafcbsm164449866b.116.2025.06.27.15.09.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 27 Jun 2025 15:09:42 -0700 (PDT) Message-ID: <969ae437-398a-4201-8ea6-92ec4db87cdc@ovn.org> Date: Sat, 28 Jun 2025 00:09:41 +0200 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Cc: i.maximets@ovn.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , dev@openvswitch.org, linux-kernel@vger.kernel.org, Eelco Chaudron , Aaron Conole Subject: Re: [PATCH net-next] net: openvswitch: allow providing upcall pid for the 'execute' command To: netdev@vger.kernel.org References: <20250627220219.1504221-1-i.maximets@ovn.org> Content-Language: en-US From: Ilya Maximets Autocrypt: addr=i.maximets@ovn.org; keydata= xsFNBF77bOMBEADVZQ4iajIECGfH3hpQMQjhIQlyKX4hIB3OccKl5XvB/JqVPJWuZQRuqNQG /B70MP6km95KnWLZ4H1/5YOJK2l7VN7nO+tyF+I+srcKq8Ai6S3vyiP9zPCrZkYvhqChNOCF pNqdWBEmTvLZeVPmfdrjmzCLXVLi5De9HpIZQFg/Ztgj1AZENNQjYjtDdObMHuJQNJ6ubPIW cvOOn4WBr8NsP4a2OuHSTdVyAJwcDhu+WrS/Bj3KlQXIdPv3Zm5x9u/56NmCn1tSkLrEgi0i /nJNeH5QhPdYGtNzPixKgPmCKz54/LDxU61AmBvyRve+U80ukS+5vWk8zvnCGvL0ms7kx5sA tETpbKEV3d7CB3sQEym8B8gl0Ux9KzGp5lbhxxO995KWzZWWokVUcevGBKsAx4a/C0wTVOpP FbQsq6xEpTKBZwlCpxyJi3/PbZQJ95T8Uw6tlJkPmNx8CasiqNy2872gD1nN/WOP8m+cIQNu o6NOiz6VzNcowhEihE8Nkw9V+zfCxC8SzSBuYCiVX6FpgKzY/Tx+v2uO4f/8FoZj2trzXdLk BaIiyqnE0mtmTQE8jRa29qdh+s5DNArYAchJdeKuLQYnxy+9U1SMMzJoNUX5uRy6/3KrMoC/ 7zhn44x77gSoe7XVM6mr/mK+ViVB7v9JfqlZuiHDkJnS3yxKPwARAQABzSJJbHlhIE1heGlt ZXRzIDxpLm1heGltZXRzQG92bi5vcmc+wsGUBBMBCAA+AhsDBQsJCAcCBhUKCQgLAgQWAgMB Ah4BAheAFiEEh+ma1RKWrHCY821auffsd8gpv5YFAmfB9JAFCQyI7q0ACgkQuffsd8gpv5YQ og/8DXt1UOznvjdXRHVydbU6Ws+1iUrxlwnFH4WckoFgH4jAabt25yTa1Z4YX8Vz0mbRhTPX M/j1uORyObLem3of4YCd4ymh7nSu++KdKnNsZVHxMcoiic9ILPIaWYa8kTvyIDT2AEVfn9M+ vskM0yDbKa6TAHgr/0jCxbS+mvN0ZzDuR/LHTgy3e58097SWJohj0h3Dpu+XfuNiZCLCZ1/G AbBCPMw+r7baH/0evkX33RCBZwvh6tKu+rCatVGk72qRYNLCwF0YcGuNBsJiN9Aa/7ipkrA7 Xp7YvY3Y1OrKnQfdjp3mSXmknqPtwqnWzXvdfkWkZKShu0xSk+AjdFWCV3NOzQaH3CJ67NXm aPjJCIykoTOoQ7eEP6+m3WcgpRVkn9bGK9ng03MLSymTPmdINhC5pjOqBP7hLqYi89GN0MIT Ly2zD4m/8T8wPV9yo7GRk4kkwD0yN05PV2IzJECdOXSSStsf5JWObTwzhKyXJxQE+Kb67Wwa LYJgltFjpByF5GEO4Xe7iYTjwEoSSOfaR0kokUVM9pxIkZlzG1mwiytPadBt+VcmPQWcO5pi WxUI7biRYt4aLriuKeRpk94ai9+52KAk7Lz3KUWoyRwdZINqkI/aDZL6meWmcrOJWCUMW73e 4cMqK5XFnGqolhK4RQu+8IHkSXtmWui7LUeEvO/OwU0EXvts4wEQANCXyDOic0j2QKeyj/ga OD1oKl44JQfOgcyLVDZGYyEnyl6b/tV1mNb57y/YQYr33fwMS1hMj9eqY6tlMTNz+ciGZZWV YkPNHA+aFuPTzCLrapLiz829M5LctB2448bsgxFq0TPrr5KYx6AkuWzOVq/X5wYEM6djbWLc VWgJ3o0QBOI4/uB89xTf7mgcIcbwEf6yb/86Cs+jaHcUtJcLsVuzW5RVMVf9F+Sf/b98Lzrr 2/mIB7clOXZJSgtV79Alxym4H0cEZabwiXnigjjsLsp4ojhGgakgCwftLkhAnQT3oBLH/6ix 87ahawG3qlyIB8ZZKHsvTxbWte6c6xE5dmmLIDN44SajAdmjt1i7SbAwFIFjuFJGpsnfdQv1 OiIVzJ44kdRJG8kQWPPua/k+AtwJt/gjCxv5p8sKVXTNtIP/sd3EMs2xwbF8McebLE9JCDQ1 RXVHceAmPWVCq3WrFuX9dSlgf3RWTqNiWZC0a8Hn6fNDp26TzLbdo9mnxbU4I/3BbcAJZI9p 9ELaE9rw3LU8esKqRIfaZqPtrdm1C+e5gZa2gkmEzG+WEsS0MKtJyOFnuglGl1ZBxR1uFvbU VXhewCNoviXxkkPk/DanIgYB1nUtkPC+BHkJJYCyf9Kfl33s/bai34aaxkGXqpKv+CInARg3 fCikcHzYYWKaXS6HABEBAAHCwXwEGAEIACYCGwwWIQSH6ZrVEpascJjzbVq59+x3yCm/lgUC Z8H0qQUJDIjuxgAKCRC59+x3yCm/loAdD/wJCOhPp9711J18B9c4f+eNAk5vrC9Cj3RyOusH Hebb9HtSFm155Zz3xiizw70MSyOVikjbTocFAJo5VhkyuN0QJIP678SWzriwym+EG0B5P97h FSLBlRsTi4KD8f1Ll3OT03lD3o/5Qt37zFgD4mCD6OxAShPxhI3gkVHBuA0GxF01MadJEjMu jWgZoj75rCLG9sC6L4r28GEGqUFlTKjseYehLw0s3iR53LxS7HfJVHcFBX3rUcKFJBhuO6Ha /GggRvTbn3PXxR5UIgiBMjUlqxzYH4fe7pYR7z1m4nQcaFWW+JhY/BYHJyMGLfnqTn1FsIwP dbhEjYbFnJE9Vzvf+RJcRQVyLDn/TfWbETf0bLGHeF2GUPvNXYEu7oKddvnUvJK5U/BuwQXy TRFbae4Ie96QMcPBL9ZLX8M2K4XUydZBeHw+9lP1J6NJrQiX7MzexpkKNy4ukDzPrRE/ruui yWOKeCw9bCZX4a/uFw77TZMEq3upjeq21oi6NMTwvvWWMYuEKNi0340yZRrBdcDhbXkl9x/o skB2IbnvSB8iikbPng1ihCTXpA2yxioUQ96Akb+WEGopPWzlxTTK+T03G2ljOtspjZXKuywV Wu/eHyqHMyTu8UVcMRR44ki8wam0LMs+fH4dRxw5ck69AkV+JsYQVfI7tdOu7+r465LUfg== In-Reply-To: <20250627220219.1504221-1-i.maximets@ovn.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 6/28/25 12:01 AM, Ilya Maximets wrote: > When a packet enters OVS datapath and there is no flow to handle it, > packet goes to userspace through a MISS upcall. With per-CPU upcall > dispatch mechanism, we're using the current CPU id to select the > Netlink PID on which to send this packet. This allows us to send > packets from the same traffic flow through the same handler. > > The handler will process the packet, install required flow into the > kernel and re-inject the original packet via OVS_PACKET_CMD_EXECUTE. > > While handling OVS_PACKET_CMD_EXECUTE, however, we may hit a > recirculation action that will pass the (likely modified) packet > through the flow lookup again. And if the flow is not found, the > packet will be sent to userspace again through another MISS upcall. > > However, the handler thread in userspace is likely running on a > different CPU core, and the OVS_PACKET_CMD_EXECUTE request is handled > in the syscall context of that thread. So, when the time comes to > send the packet through another upcall, the per-CPU dispatch will > choose a different Netlink PID, and this packet will end up processed > by a different handler thread on a different CPU. > > The process continues as long as there are new recirculations, each > time the packet goes to a different handler thread before it is sent > out of the OVS datapath to the destination port. In real setups the > number of recirculations can go up to 4 or 5, sometimes more. > > There is always a chance to re-order packets while processing upcalls, > because userspace will first install the flow and then re-inject the > original packet. So, there is a race window when the flow is already > installed and the second packet can match it and be forwarded to the > destination before the first packet is re-injected. But the fact that > packets are going through multiple upcalls handled by different > userspace threads makes the reordering noticeably more likely, because > we not only have a race between the kernel and a userspace handler > (which is hard to avoid), but also between multiple userspace handlers. > > For example, let's assume that 10 packets got enqueued through a MISS > upcall for handler-1, it will start processing them, will install the > flow into the kernel and start re-injecting packets back, from where > they will go through another MISS to handler-2. Handler-2 will install > the flow into the kernel and start re-injecting the packets, while > handler-1 continues to re-inject the last of the 10 packets, they will > hit the flow installed by handler-2 and be forwarded without going to > the handler-2, while handler-2 still re-injects the first of these 10 > packets. Given multiple recirculations and misses, these 10 packets > may end up completely mixed up on the output from the datapath. > > Let's allow userspace to specify on which Netlink PID the packets > should be upcalled while processing OVS_PACKET_CMD_EXECUTE. > This makes it possible to ensure that all the packets are processed > by the same handler thread in the userspace even with them being > upcalled multiple times in the process. Packets will remain in order > since they will be enqueued to the same socket and re-injected in the > same order. This doesn't eliminate re-ordering as stated above, since > we still have a race between kernel and the userspace thread, but it > allows to eliminate races between multiple userspace threads. > > Userspace knows the PID of the socket on which the original upcall is > received, so there is no need to send it up from the kernel. > > Solution requires storing the value somewhere for the duration of the > packet processing. There are two potential places for this: our skb > extension or the per-CPU storage. It's not clear which is better, > so just following currently used scheme of storing this kind of things > along the skb. > > Signed-off-by: Ilya Maximets > --- The userspace counterpart that makes use of this fucntionality is available here: https://patchwork.ozlabs.org/project/openvswitch/patch/20250627220752.1505153-1-i.maximets@ovn.org/ Best regards, Ilya Maximets.