From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 961F0188580 for ; Sun, 24 Aug 2025 21:54:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756072463; cv=none; b=WoW1pVMoBToMTnyVDiUn54jdZ16dC7uvmv6o5CnZS7qOJ1xSAg4MRU0yb1f8k5yU/YGnFcXHeLTdVB51kBz3iNeDhjA+INg8/kmviVR4NOlyNpkAUyiFqWfxAtAivUzyMTKpl1KXYfS+arrqYuLiSRhV/iQjn6l/bikGW4UxY9w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756072463; c=relaxed/simple; bh=dMrGo6QiK49JNr8WKvhliAJMned7TSy+xW2ywFZxGK8=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=kx20P+cTB7YBCPbd+x9hWKDtRal8LL8fhbu06pK0sRkix6IE9zUAc5jTIODFJKaawaYDSlj/3VZCKmXtLpEnm6xHFuHcbwPS7DssJq4s7nMgKO0bM019oPjk0vQ5NMg9+/31mUZAfmlINncJZV5Wa4ago7hHTQmYxfrmqINQLdg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2f1/zR0T; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2f1/zR0T" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-24457f59889so38265095ad.0 for ; Sun, 24 Aug 2025 14:54:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1756072460; x=1756677260; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=CXvXXToAG4aU4mSeQyRDJSdFn0iV+aTCHrQl/V0W970=; b=2f1/zR0TbvlnJCYeBoB15SWJWbjqyLZEcMTB8h5Q14edvTchC/qwnU0PSF7cW9e5DN KkwP3cGP2qiC5rCtkXjbPMzvTPFML78fp3iFU6BR+dhrdpSEUhcRuDIX8ZotmfvnkKxS EYCd3EC6yWFbihslZApzi6MsPreugZmY/WGgEbh8sTtynVnE1TygqgE46F/f341J2oAJ XdsJdzwaRwB75/Aga+bxYoB/N/TbXECuO0wGGP5WesOgEX8FCbZFGsuezbdiFd/vBICy lH6XR0BwrT+mav8Fbnll1TIfVqpRgM8akvC06CE6NyvRzORd/hu6/NoK1J62nqu6K1oS EdVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756072460; x=1756677260; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=CXvXXToAG4aU4mSeQyRDJSdFn0iV+aTCHrQl/V0W970=; b=DHkd3gIKBYWCBTwsyw3SnnYPCSAhSuuALaOo/O2IoKxWfv7m1oVT1WYrPEGzKFcF67 81plf2Qi32LwP/G3I308FvDVc6s2z+V0oVSHxMc+xUKOQ7B5ofBhvTSW61uMUHfl331U 2Zxdfq4OXv0gP5vetHahzLApxY0NV3Ron0G2dT/UfR/RjPAF/nIK+w6SzHDujG13oF3M hLn0kI/h963erg27LHQ3E3iDEJjGphM6MB4OZTxvtAeda09pipUjS6JvLj/V2MRvF8DB GTJKAulKBGcaRER3qV3P1Mu9r+7Tr26MFMPGU1eWl8hCxUekpi1XDWhOBiZiwjmm5iZI 4toQ== X-Forwarded-Encrypted: i=1; AJvYcCXrp7Z/dLkaLDQRNPMdXZj7tdPlbSpOBHzwNLyt0k7biiBV+r1ktBnBwGwJnQRdcNRoXq9gQ1c=@vger.kernel.org X-Gm-Message-State: AOJu0YyZhhSLMEp9QR0mHjdOAH4ATQbR6R+Ir9FXQ0BheO0mB+UUTsSO f5Qjb4gEIMHpm6Oi5oBuF1KWxw0Ccn8QHb/ty7CPdrkmhuzRk/skipdyz4Wrwr166UglZJXswrb Ar4mwuokzP8AJ3A== X-Google-Smtp-Source: AGHT+IGuvbHY8hHC4Hrd3g9hx1GjLnQiu2+EL1vn/ND9iRqrIvioPdEhDJDVa/zoZD4eDW8InIseeHK4xT7dVw== X-Received: from plzt4.prod.google.com ([2002:a17:902:bc44:b0:246:22da:b381]) (user=skhawaja job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:246:b0:240:49d1:6347 with SMTP id d9443c01a7336-2462ef1f2e0mr128686445ad.35.1756072459838; Sun, 24 Aug 2025 14:54:19 -0700 (PDT) Date: Sun, 24 Aug 2025 21:54:16 +0000 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.51.0.rc2.233.g662b1ed5c5-goog Message-ID: <20250824215418.257588-1-skhawaja@google.com> Subject: [PATCH net-next v7 0/2] Add support to do threaded napi busy poll From: Samiullah Khawaja To: Jakub Kicinski , "David S . Miller " , Eric Dumazet , Paolo Abeni , almasrymina@google.com, willemb@google.com, mkarsten@uwaterloo.ca Cc: Joe Damato , netdev@vger.kernel.org, skhawaja@google.com Content-Type: text/plain; charset="UTF-8" Extend the already existing support of threaded napi poll to do continuous busy polling. This is used for doing continuous polling of napi to fetch descriptors from backing RX/TX queues for low latency applications. Allow enabling of threaded busypoll using netlink so this can be enabled on a set of dedicated napis for low latency applications. Once enabled user can fetch the PID of the kthread doing NAPI polling and set affinity, priority and scheduler for it depending on the low-latency requirements. Currently threaded napi is only enabled at device level using sysfs. Add support to enable/disable threaded mode for a napi individually. This can be done using the netlink interface. Extend `napi-set` op in netlink spec that allows setting the `threaded` attribute of a napi. Extend the threaded attribute in napi struct to add an option to enable continuous busy polling. Extend the netlink and sysfs interface to allow enabling/disabling threaded busypolling at device or individual napi level. We use this for our AF_XDP based hard low-latency usecase with usecs level latency requirement. For our usecase we want low jitter and stable latency at P99. Following is an analysis and comparison of available (and compatible) busy poll interfaces for a low latency usecase with stable P99. Please note that the throughput and cpu efficiency is a non-goal. For analysis we use an AF_XDP based benchmarking tool `xdp_rr`. The description of the tool and how it tries to simulate the real workload is following, - It sends UDP packets between 2 machines. - The client machine sends packets at a fixed frequency. To maintain the frequency of the packet being sent, we use open-loop sampling. That is the packets are sent in a separate thread. - The server replies to the packet inline by reading the pkt from the recv ring and replies using the tx ring. - To simulate the application processing time, we use a configurable delay in usecs on the client side after a reply is received from the server. The xdp_rr tool is posted separately as an RFC for tools/testing/selftest. We use this tool with following napi polling configurations, - Interrupts only - SO_BUSYPOLL (inline in the same thread where the client receives the packet). - SO_BUSYPOLL (separate thread and separate core) - Threaded NAPI busypoll System is configured using following script in all 4 cases, ``` echo 0 | sudo tee /sys/class/net/eth0/threaded echo 0 | sudo tee /proc/sys/kernel/timer_migration echo off | sudo tee /sys/devices/system/cpu/smt/control sudo ethtool -L eth0 rx 1 tx 1 sudo ethtool -G eth0 rx 1024 echo 0 | sudo tee /proc/sys/net/core/rps_sock_flow_entries echo 0 | sudo tee /sys/class/net/eth0/queues/rx-0/rps_cpus # pin IRQs on CPU 2 IRQS="$(gawk '/eth0-(TxRx-)?1/ {match($1, /([0-9]+)/, arr); \ print arr[0]}' < /proc/interrupts)" for irq in "${IRQS}"; \ do echo 2 | sudo tee /proc/irq/$irq/smp_affinity_list; done echo -1 | sudo tee /proc/sys/kernel/sched_rt_runtime_us for i in /sys/devices/virtual/workqueue/*/cpumask; \ do echo $i; echo 1,2,3,4,5,6 > $i; done if [[ -z "$1" ]]; then echo 400 | sudo tee /proc/sys/net/core/busy_read echo 100 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs echo 15000 | sudo tee /sys/class/net/eth0/gro_flush_timeout fi sudo ethtool -C eth0 adaptive-rx off adaptive-tx off rx-usecs 0 tx-usecs 0 if [[ "$1" == "enable_threaded" ]]; then echo 0 | sudo tee /proc/sys/net/core/busy_poll echo 0 | sudo tee /proc/sys/net/core/busy_read echo 100 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs echo 15000 | sudo tee /sys/class/net/eth0/gro_flush_timeout echo 2 | sudo tee /sys/class/net/eth0/threaded NAPI_T=$(ps -ef | grep napi | grep -v grep | awk '{ print $2 }') sudo chrt -f -p 50 $NAPI_T # pin threaded poll thread to CPU 2 sudo taskset -pc 2 $NAPI_T fi if [[ "$1" == "enable_interrupt" ]]; then echo 0 | sudo tee /proc/sys/net/core/busy_read echo 0 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs echo 15000 | sudo tee /sys/class/net/eth0/gro_flush_timeout fi ``` To enable various configurations, script can be run as following, - Interrupt Only ```