From: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
To: Seth Forshee <sforshee@kernel.org>
Cc: Paul Menzel <pmenzel@molgen.mpg.de>,
netdev@vger.kernel.org,
Jesse Brandeburg <jesse.brandeburg@intel.com>,
intel-wired-lan@lists.osuosl.org,
Tony Nguyen <anthony.l.nguyen@intel.com>,
magnus.karlsson@intel.com
Subject: Re: [Intel-wired-lan] i40e XDP program stops transmitting after link down/up
Date: Wed, 31 Jan 2024 17:38:53 +0100 [thread overview]
Message-ID: <Zbp3ndhiNGw0uhll@boxer> (raw)
In-Reply-To: <Zbp0n4HTPw/xMtB5@do-x1extreme>
On Wed, Jan 31, 2024 at 10:26:07AM -0600, Seth Forshee wrote:
> On Tue, Jan 30, 2024 at 11:06:07PM +0100, Maciej Fijalkowski wrote:
> > On Tue, Jan 30, 2024 at 08:28:24PM +0100, Maciej Fijalkowski wrote:
> > > On Tue, Jan 30, 2024 at 10:59:13AM -0600, Seth Forshee wrote:
> > > > On Tue, Jan 30, 2024 at 05:14:23PM +0100, Paul Menzel wrote:
> > > > > Dear Seth,
> > > > >
> > > > >
> > > > > Thank you for bring this up.
> > > > >
> > > > > Am 30.01.24 um 15:17 schrieb Seth Forshee:
> > > > > > I got a inquiry from a colleague about a behavior he's seeing with i40e
> > > > > > but not with other NICs. The interfaces are bonded with a XDP
> > > > > > load-balancer program attached to them. After 'ip link set ethX down; ip
> > > > > > link set ethX up' on one of the interfaces the XDP program on that
> > > > > > interface is no longer transmitting packets. He found that tx starts
> > > > > > again after running 'sudo ethtool -t ethX'.
> > > > > >
> > > > > > There's a 'i40e 0000:d8:00.1: VSI seid 391 XDP Tx ring 0 disable
> > > > > > timeout' message in dmesg when disabling the interface. I've included
> > > > > > the relevant portions from dmesg below.
> > > > > >
> > > > > > This was first observed with a 6.1 kernel, but we've confirmed that the
> > > > > > behavior is the same in 6.7. I realize the firmware is pretty old, so
> > > > > > far our attempts to update the NVM have failed.
> > > > >
> > > > > Does that mean, the problem didn’t happen before Linux 6.1? If so, if you
> > > > > have the reproducer and the time, bisecting the issue is normally the
> > > > > fastest way to solve the issue.
> > > >
> > > > No, sorry, I should have worded that better. I meant that they were
> > > > using 6.1 when they noticed the issue, not that kernels before 6.1 did
> > > > not have that issue. We've also tried a 5.15 kernel build now and still
> > > > see the issue there, we haven't tested anything older than that.
> > >
> > > Hey Seth,
> > >
> > > I am observing same thing on my side with xdpsock in txonly mode, so I'll
> > > take a look at this and will keep you updated.
> >
> > Can you try this diff on your side?
>
> That fixes the problem!
Awesome. I'll send a proper fix then and will keep you in loop.
>
> Thanks,
> Seth
WARNING: multiple messages have this Message-ID (diff)
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
To: Seth Forshee <sforshee@kernel.org>
Cc: Paul Menzel <pmenzel@molgen.mpg.de>,
Jesse Brandeburg <jesse.brandeburg@intel.com>,
Tony Nguyen <anthony.l.nguyen@intel.com>,
<netdev@vger.kernel.org>, <intel-wired-lan@lists.osuosl.org>,
<magnus.karlsson@intel.com>
Subject: Re: [Intel-wired-lan] i40e XDP program stops transmitting after link down/up
Date: Wed, 31 Jan 2024 17:38:53 +0100 [thread overview]
Message-ID: <Zbp3ndhiNGw0uhll@boxer> (raw)
In-Reply-To: <Zbp0n4HTPw/xMtB5@do-x1extreme>
On Wed, Jan 31, 2024 at 10:26:07AM -0600, Seth Forshee wrote:
> On Tue, Jan 30, 2024 at 11:06:07PM +0100, Maciej Fijalkowski wrote:
> > On Tue, Jan 30, 2024 at 08:28:24PM +0100, Maciej Fijalkowski wrote:
> > > On Tue, Jan 30, 2024 at 10:59:13AM -0600, Seth Forshee wrote:
> > > > On Tue, Jan 30, 2024 at 05:14:23PM +0100, Paul Menzel wrote:
> > > > > Dear Seth,
> > > > >
> > > > >
> > > > > Thank you for bring this up.
> > > > >
> > > > > Am 30.01.24 um 15:17 schrieb Seth Forshee:
> > > > > > I got a inquiry from a colleague about a behavior he's seeing with i40e
> > > > > > but not with other NICs. The interfaces are bonded with a XDP
> > > > > > load-balancer program attached to them. After 'ip link set ethX down; ip
> > > > > > link set ethX up' on one of the interfaces the XDP program on that
> > > > > > interface is no longer transmitting packets. He found that tx starts
> > > > > > again after running 'sudo ethtool -t ethX'.
> > > > > >
> > > > > > There's a 'i40e 0000:d8:00.1: VSI seid 391 XDP Tx ring 0 disable
> > > > > > timeout' message in dmesg when disabling the interface. I've included
> > > > > > the relevant portions from dmesg below.
> > > > > >
> > > > > > This was first observed with a 6.1 kernel, but we've confirmed that the
> > > > > > behavior is the same in 6.7. I realize the firmware is pretty old, so
> > > > > > far our attempts to update the NVM have failed.
> > > > >
> > > > > Does that mean, the problem didn’t happen before Linux 6.1? If so, if you
> > > > > have the reproducer and the time, bisecting the issue is normally the
> > > > > fastest way to solve the issue.
> > > >
> > > > No, sorry, I should have worded that better. I meant that they were
> > > > using 6.1 when they noticed the issue, not that kernels before 6.1 did
> > > > not have that issue. We've also tried a 5.15 kernel build now and still
> > > > see the issue there, we haven't tested anything older than that.
> > >
> > > Hey Seth,
> > >
> > > I am observing same thing on my side with xdpsock in txonly mode, so I'll
> > > take a look at this and will keep you updated.
> >
> > Can you try this diff on your side?
>
> That fixes the problem!
Awesome. I'll send a proper fix then and will keep you in loop.
>
> Thanks,
> Seth
next prev parent reply other threads:[~2024-01-31 16:39 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-30 14:17 [Intel-wired-lan] i40e XDP program stops transmitting after link down/up Seth Forshee
2024-01-30 14:17 ` Seth Forshee
2024-01-30 16:14 ` [Intel-wired-lan] " Paul Menzel
2024-01-30 16:14 ` Paul Menzel
2024-01-30 16:59 ` Seth Forshee
2024-01-30 16:59 ` Seth Forshee
2024-01-30 19:28 ` Maciej Fijalkowski
2024-01-30 19:28 ` Maciej Fijalkowski
2024-01-30 22:06 ` Maciej Fijalkowski
2024-01-30 22:06 ` Maciej Fijalkowski
2024-01-31 16:26 ` Seth Forshee
2024-01-31 16:26 ` Seth Forshee
2024-01-31 16:38 ` Maciej Fijalkowski [this message]
2024-01-31 16:38 ` Maciej Fijalkowski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Zbp3ndhiNGw0uhll@boxer \
--to=maciej.fijalkowski@intel.com \
--cc=anthony.l.nguyen@intel.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jesse.brandeburg@intel.com \
--cc=magnus.karlsson@intel.com \
--cc=netdev@vger.kernel.org \
--cc=pmenzel@molgen.mpg.de \
--cc=sforshee@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.