All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba@kernel.org>
To: Nimrod Oren <noren@nvidia.com>
Cc: Dragos Tatulea <dtatulea@nvidia.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: [TEST] CX7 timeouts on reconfig w/ page pool failure injection
Date: Tue, 11 Aug 2026 10:38:32 -0700	[thread overview]
Message-ID: <20260811103832.28e5c8d4@kernel.org> (raw)
In-Reply-To: <8aa8c49b-736c-42b3-92d3-26bdfeb90a22@nvidia.com>

On Tue, 11 Aug 2026 19:57:31 +0300 Nimrod Oren wrote:
> On 11/08/2026 18:04, Jakub Kicinski wrote:
> > On Tue, 11 Aug 2026 10:20:52 +0300 Nimrod Oren wrote:  
> >> On 04/08/2026 19:45, Jakub Kicinski wrote:  
>  [...]  
> >>
> >> Seems like the bigger culprit is the earlier xdp test, which leaves MTU
> >> at 9000:
> >>
> >> # # Defer Exception| net.lib.py.utils.CmdExitFailure: Command failed
> >> # # Defer Exception| CMD: ip link set dev ens25f1np1 mtu 1500 xdpdrv off
> >> # # Defer Exception|   EXIT: -15
> >> # # Defer Exception|
> >> # not ok 17 xdp.test_xdp_native_adjst_head_grow_data.ipv4
> >> # # Totals: pass:14 fail:3 xfail:0 xpass:0 skip:0 error:0
> >> not ok 1 selftests: drivers/net: xdp.py # TIMEOUT 360 seconds
> >>
> >> https://netdev.bots.linux.dev/logview.html?f=/logs/hwksft/CX7-dbg/results/763842/test-outputs/13-xdp-py/stdout  
> > 
> > Ah, well spotted. That's the test timeout, I'll bump the timeout from
> > the current 6min to 10min. Once it passes we'll know the true time
> > needed, and we shall set the timeout to 150% of the expected case..
> 
> Sounds good, thanks.

10min wasn't enough, bumping to 15min :(

> >> I'll try to reproduce this internally and investigate what happened.
> >>
> >> Also, I noticed that the pp-alloc test isn't properly restoring the rx
> >> ring size due to the timeout. In the subsequent retry, it's attempting
> >> to double it again, this time from 2K to 4K:
> >>
> >> # # Exception| subprocess.TimeoutExpired: Command '['ethtool', '-G',
> >> 'ens25f1np1', 'rx', '4096']' timed out after 20 seconds
> >> # # Exception|
> >> # not ok 1 pp_alloc_fail.test_pp_alloc
> >>
> >> https://netdev.bots.linux.dev/logs/hwksft/CX7-dbg/results/763842/test-outputs/36-pp-alloc-fail-py-retry/stdout
> >>
> >> The high MTU and ring size along with the heavy debug kernel likely make
> >> the pp-alloc test significantly slower. I'll also prepare a patch to fix
> >> this pp-alloc cleanup issue.  
> > 
> > To be clear - if the commands time out - how can we fix the cleanup?
> > Or do you mean some other bug?  
> 
> In the pp-alloc test, the ring size command timed out, but the change
> was still applied in the end.
> 
> Currently that test only attempts to restore the original ring size if
> the set command returns successfully. This case makes me think that it
> should attempt the restoration regardless...

Would the defer() not time out as well if the command to apply 
the config timed out? Any idea what the main time consumer is 
for the reconfig? It can't be anything device related since 
the device isn't really impacted by kernel config options...

      reply	other threads:[~2026-08-11 17:38 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-04 16:45 [TEST] CX7 timeouts on reconfig w/ page pool failure injection Jakub Kicinski
2026-08-04 17:16 ` Dragos Tatulea
2026-08-04 19:44   ` Jakub Kicinski
2026-08-11  7:20 ` Nimrod Oren
2026-08-11 15:04   ` Jakub Kicinski
2026-08-11 16:57     ` Nimrod Oren
2026-08-11 17:38       ` Jakub Kicinski [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260811103832.28e5c8d4@kernel.org \
    --to=kuba@kernel.org \
    --cc=dtatulea@nvidia.com \
    --cc=netdev@vger.kernel.org \
    --cc=noren@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.