From: David Laight <David.Laight@ACULAB.COM>
To: 'Jakub Kicinski' <kuba@kernel.org>
Cc: "netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: RE: rawip: delayed and mis-sequenced transmits
Date: Thu, 7 Jul 2022 08:02:05 +0000 [thread overview]
Message-ID: <fd92e34c41b94cd1ac9bfcadd4a94ee6@AcuMS.aculab.com> (raw)
In-Reply-To: <20220706185417.2fcbcdf0@kernel.org>
From: Jakub Kicinski
> Sent: 07 July 2022 02:54
>
> On Wed, 6 Jul 2022 15:54:18 +0000 David Laight wrote:
> > Anyone any ideas before I start digging through the kernel code?
>
> If the qdisc is pfifo_fast and kernel is old there could be races.
> But I don't think that's likely given you probably run something
> recent and next packet tx would usually flush the stuck packet.
> In any case - switching qdisc could be a useful test, also bpftrace
> is your friend for catching patckets with long sojourn time.
Yes, I forgot to mention the system is running a 5.18-rc6 kernel.
(I've already got some diagnostics in the receive path.)
I'd expect bugs in the qdisc layer to (mostly) keep packets
in order.
In this case I'm sending several packets at 20ms intervals that
overtake the 'stuck' packet.
So it really must be on a per socket queue.
Also the 'stuck' packet is sent after the packet that unblocks it.
I'm not sure normal tracing will help.
I'm seeing one mis-sequence every couple of million packets.
I can add code to the ethernet tx code to call ftrace_stop()
when the delayed packet gets sent.
(I've got the same check in the rx path to detect lost packets.)
That should show where it came from and probably why it got queued.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
next prev parent reply other threads:[~2022-07-07 8:02 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-06 15:54 rawip: delayed and mis-sequenced transmits David Laight
2022-07-07 1:54 ` Jakub Kicinski
2022-07-07 8:02 ` David Laight [this message]
2022-07-07 9:34 ` David Laight
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fd92e34c41b94cd1ac9bfcadd4a94ee6@AcuMS.aculab.com \
--to=david.laight@aculab.com \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).