From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BF6E4908AB for ; Tue, 25 Aug 2026 22:55:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787698524; cv=none; b=TUN9jXNZsSMfwYTzNV6mNeeS9CeD9fTIZwpGtfL14J65DoAxzfiFofKTlkVOXbK3QEDK5e7WmjkYan03I/PFF/KNrcUelS0O2pJkEKRi58J5r04Em233dERXaBNJR6LCS25fSPLTyBEc/FMMocrjilWDRwkzRqn5dWE9eg2Y+Zg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787698524; c=relaxed/simple; bh=g8RGxE7giMMr46tTI9lcGOZyzcS9JQ8ajj7RuZ4LyDU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=B6T4DewiplipOaH2VgCwFP8Q1vcgFdHZxHzOxPm6SE5I8YDDRzmMMFBt2sSjS2zVRuQheeq1sMOvbL+0pgakh7Ccby9beSc9/Tn00272oYdgk7J7/8ebRfzn2QM5DxRyuoqyAgUmiHdCNGacZmydDsKm9WPYi/0a0Yxheso5Lyw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Z0tqrGkK; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Z0tqrGkK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1787698519; x=1819234519; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=g8RGxE7giMMr46tTI9lcGOZyzcS9JQ8ajj7RuZ4LyDU=; b=Z0tqrGkKdm2RxXp6caeeMKGCWuEX6W9/fV0PWkcLQeC0meLmzBs8Z3Xg QI8dwejc1/JT1UP86xSBDVFlHF6M1Pi1sHxqiWqf/9GtXPRPgnlYd+XjE AbOjPFL9lnmjx6/unoCO8J45sT5TVewkUd/8ABgIEqrK5VQU9Au/R9TQ2 pV9NVXLazpwRMZ9UG0Y3/dkW74TUuS7lpC85YR1aZBr6RjjEwLPZw365T 3dI7+4j4ivSBpyu7/pvLXvLXFtiIdK3Mq+p72tj6+ZubdEQ0cx2+8lj/o 4MiMyYbvZJnqwLQQ+uJMnLQWV2WZQ8mXw8VKUrwNp0KCzAVhUdnnCRKOE g==; X-CSE-ConnectionGUID: O7ha2VneRUuEecOrtZCFsg== X-CSE-MsgGUID: hoxIkpotSDKRGxASCJkQiQ== X-IronPort-AV: E=McAfee;i="6800,10657,11886"; a="87120454" X-IronPort-AV: E=Sophos;i="6.25,243,1779174000"; d="scan'208";a="87120454" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Aug 2026 15:55:08 -0700 X-CSE-ConnectionGUID: qlKXyJrESmeq4oyYWE/PQw== X-CSE-MsgGUID: dnY+pwqCQKCHdZVvhev10Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,243,1779174000"; d="scan'208";a="264789707" Received: from orcnseosdtjek.jf.intel.com (HELO [10.166.28.109]) ([10.166.28.109]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Aug 2026 15:55:07 -0700 From: Jacob Keller Date: Tue, 25 Aug 2026 15:53:38 -0700 Subject: [PATCH iwl-net v2 14/14] ice: don't clear in_use until HW clears ready bitmap Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260825-jk-e825c-minimized-fixes-v2-14-8223f95d26e3@intel.com> References: <20260825-jk-e825c-minimized-fixes-v2-0-8223f95d26e3@intel.com> In-Reply-To: <20260825-jk-e825c-minimized-fixes-v2-0-8223f95d26e3@intel.com> To: Intel Wired LAN Cc: netdev@vger.kernel.org, Maciej Machnikowski , Anthony Nguyen , Przemyslaw Korba , Grzegorz Nitka , Petr Oros , alexander.nowlin@intel.com, kevin.bross@intel.com, ranjit.cavatur@intel.com, Jacob Keller X-Mailer: b4 0.17-dev-c276d X-Developer-Signature: v=1; a=openpgp-sha256; l=7381; i=jacob.e.keller@intel.com; h=from:subject:message-id; bh=g8RGxE7giMMr46tTI9lcGOZyzcS9JQ8ajj7RuZ4LyDU=; b=owGbwMvMwCWWNS3WLp9f4wXjabUkhqw+WTt1/x0eJkGpf9a/KCq798Fo9frwXf9t9XUu1b19X JIvunxGRykLgxgXg6yYIouCQ8jK68YTwrTeOMvBzGFlAhnCwMUpABPZMJeR4eGaxc7fPkqvqWuQ 575bscXzB7u2+b05M155J/xlOlZrqMfIcFBy5h+Hng8ZJczaGgcc7xZ/j+f3SzobrSIR0By3ZtF 7bgA= X-Developer-Key: i=jacob.e.keller@intel.com; a=openpgp; fpr=204054A9D73390562AEC431E6A965D3E6F0F28E8 During a link down transition, the E825 PHY has a small window where it does not properly respond to reading the PHY timestamp registers. When this occurs, the PHY does not automatically clear the ready bitmap or the valid bit for the timestamp. This begins happening slightly before a link transition even before the firmware has notified the driver of the state change. The driver happily completes the timestamp, releasing the in_use bit. This allows another request to reuse the bit potentially reporting an invalid stale timestamp. Additionally, with the ready bit still set high the driver continues to re-trigger the IRQ and check for timestamps in a tight loop, wasting CPU cycles. To fix this, re-read the PHY timestamp memory status after each read of a PHY index. Double check if the hardware cleared the index properly. If it hasn't, mark the timestamp index as stale and skip processing it. Stale timestamps are already ignored by the ice_any_port_has_timestamps() function. However, the ice_ptp_tx_tstamps_pending() function also checks the ready bitmap. Instead, modify it to only check the software tracker. Additionally, stop re-triggering the interrupt from the IRQ if the timestamp tracker is calibrating or has the link marked as down. Continue to check the hardware ready bitmap from the watchdog to catch cases of unexpected timestamps. With these changes, the timestamp processing no longer triggers a repeated spamming of the IRQ during link down events where timestamps get stuck as the PHY transitions to link down. Once link is restored, the PHY will be reset and the stuck timestamps are cleared. Measuring CPU utilization of the miscellaneous IRQ thread function during timestamp storms near a link reset shows that this prevents the spikes caused by the "stuck" ready bit. Without this fix, the CPU handling the IRQ becomes slammed due to the IRQ re-triggering logic. Measuring latency using the ice Tx timestamp traces does show that this fix comes at a latency cost. Latency is measured using the ice Tx timestamp traces for the request to completion time. I measured a couple of different workloads both before and after this fix: * ptp4l using a profile with ~16 SYNC messages per second before: 159.40 microseconds mean, stdev 45.28 after: 182.07 microseconds mean, stdev 43.43 * a C program generating 16 timestamp requests every 10 milliseconds on two different ports: before: 604.35 microseconds mean, stdev 345.32 after: 990.13 microseconds mean, stdev 625.64 In the normal work flows this comes with about a 20 microsecond penalty on the average, and the standard deviation remains approximately the same. For heavy workloads with many more timestamps than expected for typical applications this comes at a significant cost. This is because we handle all timestamps in a single thread. If there are many concurrent timestamps being requested at once, any which use the later slots on ports later in the port list will take much longer to be processed once the interrupt is fired. Since each timestamp now requires an additional PHY register access, this cost is much higher in the case where the device is under unusually heavy load. However, *correctness* is more important than speed here. Additionally, we still remain well below the default limit of 10 milliseconds that ptp4l will wait before complaining about missing timestamps. Fixes: 7cab44f1c35f ("ice: Introduce ETH56G PHY model for E825C products") Signed-off-by: Jacob Keller --- drivers/net/ethernet/intel/ice/ice_ptp.c | 59 +++++++++++++++++--------------- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c index 96e96ba1731d..d96a44ec53a7 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp.c +++ b/drivers/net/ethernet/intel/ice/ice_ptp.c @@ -620,6 +620,19 @@ static void ice_ptp_process_tx_tstamp(struct ice_ptp_tx *tx) if (err && !drop_ts) continue; + /* verify ready bit cleared */ + if (tx->has_ready_bitmap) { + err = ice_get_phy_tx_tstamp_ready(hw, tx->block, &tstamp_ready); + if (err || tstamp_ready & BIT_ULL(phy_idx)) { + spin_lock_irqsave(&tx->lock, flags); + if (!test_and_set_bit(idx, tx->stale)) + dev_dbg(ice_pf_to_dev(pf), "PHY port %u failed to clear ready bit for idx %u\n", + ptp_port->port_num, phy_idx); + spin_unlock_irqrestore(&tx->lock, flags); + continue; + } + } + ice_trace(tx_tstamp_fw_done, tx->tstamps[idx].skb, idx); /* For PHYs which don't implement a proper timestamp ready @@ -2768,10 +2781,14 @@ static bool ice_port_has_timestamps(struct ice_ptp_tx *tx, bool in_irq) if (!tx->init) return false; - if (in_irq) + if (in_irq) { + if (!ice_ptp_is_tx_tracker_up(tx)) + return false; + return bitmap_andnot(tstamps, tx->in_use, tx->stale, tx->len); - else + } else { return !bitmap_empty(tx->in_use, tx->len); + } } } @@ -2794,41 +2811,18 @@ static bool ice_any_port_has_timestamps(struct ice_pf *pf, bool in_irq) bool ice_ptp_tx_tstamps_pending(struct ice_pf *pf, bool in_irq) { - struct ice_hw *hw = &pf->hw; - int ret; - - /* Check software indicator */ switch (pf->ptp.tx_interrupt_mode) { case ICE_PTP_TX_INTERRUPT_NONE: return false; case ICE_PTP_TX_INTERRUPT_SELF: - if (ice_port_has_timestamps(&pf->ptp.port.tx, in_irq)) - return true; - break; + return ice_port_has_timestamps(&pf->ptp.port.tx, in_irq); case ICE_PTP_TX_INTERRUPT_ALL: - if (ice_any_port_has_timestamps(pf, in_irq)) - return true; - break; + return ice_any_port_has_timestamps(pf, in_irq); default: WARN_ONCE(1, "Unexpected Tx timestamp interrupt mode %u\n", pf->ptp.tx_interrupt_mode); - break; - } - - /* Check hardware indicator */ - ret = ice_check_phy_tx_tstamp_ready(hw); - if (ret < 0) { - dev_dbg(ice_pf_to_dev(pf), "Unable to read PHY Tx timestamp ready bitmap, err %d\n", - ret); - /* Stop triggering IRQs if we're unable to read PHY */ return false; } - - /* ice_check_phy_tx_tstamp_ready() returns 1 if there are timestamps - * available, 0 if there are no waiting timestamps, and a negative - * value if there was an error (which we checked for above). - */ - return ret > 0; } /** @@ -2912,6 +2906,7 @@ static void ice_ptp_maybe_trigger_tx_interrupt(struct ice_pf *pf) { struct device *dev = ice_pf_to_dev(pf); struct ice_hw *hw = &pf->hw; + int ret; if (!pf->ptp.port.tx.has_ready_bitmap) return; @@ -2919,7 +2914,15 @@ static void ice_ptp_maybe_trigger_tx_interrupt(struct ice_pf *pf) if (!ice_pf_src_tmr_owned(pf)) return; - if (ice_ptp_tx_tstamps_pending(pf, false)) { + ret = ice_check_phy_tx_tstamp_ready(hw); + if (ret < 0) { + dev_dbg(dev, "Unable to read PHY Tx timestamp ready bitmap, err %pe\n", + ERR_PTR(ret)); + /* Don't trigger an IRQ if we are unable to access the PHY */ + return; + } + + if (ret > 0 || ice_ptp_tx_tstamps_pending(pf, false)) { dev_dbg(dev, "PTP periodic task detected waiting timestamps. Triggering Tx timestamp interrupt now.\n"); wr32(hw, PFINT_OICR, PFINT_OICR_TSYN_TX_M); -- 2.55.0.814.gc42f45431d0f