From: Andres Freund <andres@anarazel.de>
To: Jarek Poplawski <jarkao2@gmail.com>
Cc: LKML <linux-kernel@vger.kernel.org>,
netdev@vger.kernel.org, Stephen Hemminger <shemminger@vyatta.com>,
Patrick McHardy <kaber@trash.net>
Subject: Re: Soft-Lockup/Race in networking in 2.6.31-rc1+195 (possibly caused by netem)
Date: Thu, 02 Jul 2009 02:37:24 +0200 [thread overview]
Message-ID: <4A4C0144.5070203@anarazel.de> (raw)
In-Reply-To: <4A4BD384.3090407@anarazel.de>
On 07/01/2009 11:22 PM, Andres Freund wrote:
> On 07/01/2009 08:39 PM, Jarek Poplawski wrote:
>> Andres Freund wrote, On 07/01/2009 01:20 AM:
>>> While playing around with netem (time, not packet count based loss-
>>> bursts) I experienced soft lockups several times - to exclude it was my
>>> modifications causing this I recompiled with the original and it is
>>> still locking up.
>>> I captured several of those traces via the thankfully
>>> still working netconsole.
>>> The simplest policy I could reproduce the error with was:
>>> tc qdisc add dev eth0 root handle 1: netem delay 10ms loss 0
>>>
>>> I could not reproduce the error without delay - but that may only be a
>>> timing issue, as the host I was mainly transferring data to was on a
>>> local network.
>>> I could not reproduce the issue on lo.
>>>
>>> The time to reproduce the error varied from seconds after executing tc
>>> to several minutes.
>>>
>>> Traces 5+6 are made with vanilla
>>> 52989765629e7d182b4f146050ebba0abf2cb0b7
>>>
>>> The earlier traces are made with parts of my patches applied, and only
>>> included for completeness as I don't believe my modifications were
>>> causing this and all traces are different, so it may give some clues.
>>>
>>> Lockdep was enabled but did not diagnose anything relevant (one dvb
>>> warning during bootup).
>>>
>>> Any ideas for debugging?
>>
>> Maybe these traces will be enough, but lockdep report could save time.
>> If dvb warning triggers every time then lockdep probably turns off
>> just after (it works this way, unless something was changed). So,
>> could you try to repeat this without dvb? Btw., did you try this on
>> some earlier kernel?
> Yes. Today I could not manage to reproduce it on 2.6.30 but could on
> current git...
> Will try without dvb.
So I tried - and I did not catch any lockdep output before the crash.
Unfortunately I do not have another machine on the same local network to
catch any messages after the crash... So I could be missing some warning
(I did synchronous logging though).
Will check with netconsole tomorrow.
Andres
next prev parent reply other threads:[~2009-07-02 0:37 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-06-30 23:20 Soft-Lockup/Race in networking in 2.6.31-rc1+195 (possibly caused by netem) Andres Freund
2009-07-01 18:39 ` Jarek Poplawski
2009-07-01 21:22 ` Andres Freund
2009-07-02 0:37 ` Andres Freund [this message]
2009-07-02 9:30 ` Jarek Poplawski
2009-07-02 10:12 ` Jarek Poplawski
2009-07-02 10:51 ` Joao Correia
2009-07-02 11:09 ` Jarek Poplawski
2009-07-02 11:11 ` Andres Freund
2009-07-02 11:43 ` Jarek Poplawski
2009-07-02 11:43 ` Andres Freund
2009-07-02 11:54 ` Jarek Poplawski
2009-07-02 11:59 ` Andres Freund
-- strict thread matches above, loose matches on Subject: below --
2009-07-03 1:31 Soft-Lockup/Race in networking in 2.6.31-rc1+195 ( possibly " Andres Freund
2009-07-03 6:12 ` Soft-Lockup/Race in networking in 2.6.31-rc1+195 ( possibly?caused " Jarek Poplawski
2009-07-03 11:26 ` Andres Freund
2009-07-03 12:03 ` Jarek Poplawski
2009-07-03 12:30 ` Andres Freund
2009-07-03 20:22 ` David Miller
2009-07-03 22:56 ` Jarek Poplawski
2009-07-04 1:55 ` David Miller
2009-07-04 6:36 ` Jarek Poplawski
2009-07-04 15:18 ` Jarek Poplawski
2009-07-06 4:53 ` Joao Correia
2009-07-06 8:14 ` Jarek Poplawski
2009-07-06 11:28 ` Joao Correia
2009-07-06 14:19 ` Jarek Poplawski
2009-07-06 16:13 ` Andres Freund
2009-07-06 16:31 ` Jarek Poplawski
2009-07-06 17:23 ` Joao Correia
2009-07-06 17:26 ` Andres Freund
2009-07-07 6:50 ` Jarek Poplawski
2009-07-07 10:40 ` Joao Correia
2009-07-07 10:47 ` Andres Freund
2009-07-07 13:18 ` Jarek Poplawski
2009-07-07 13:22 ` Andres Freund
2009-07-07 13:29 ` Jarek Poplawski
2009-07-07 13:34 ` Andres Freund
2009-07-07 13:57 ` Jarek Poplawski
2009-07-07 16:11 ` Andres Freund
2009-07-08 8:08 ` Jarek Poplawski
2009-07-08 8:29 ` Andres Freund
2009-07-08 9:13 ` Jarek Poplawski
2009-07-08 21:44 ` Joao Correia
2009-07-08 22:07 ` Jarek Poplawski
2009-07-08 22:27 ` Joao Correia
2009-07-08 22:42 ` Jarek Poplawski
2009-07-08 22:48 ` Joao Correia
2009-07-08 22:23 ` Andres Freund
2009-07-08 22:48 ` Jarek Poplawski
2009-07-09 10:31 ` Thomas Gleixner
2009-07-09 10:44 ` Jarek Poplawski
2009-07-09 12:03 ` Thomas Gleixner
2009-07-09 13:22 ` Jarek Poplawski
2009-07-09 14:15 ` Thomas Gleixner
2009-07-09 14:24 ` Jarek Poplawski
2009-07-09 14:25 ` Joao Correia
2009-07-09 14:28 ` Thomas Gleixner
2009-07-09 15:28 ` Andres Freund
2009-07-09 16:01 ` Thomas Gleixner
2009-07-09 16:46 ` Andres Freund
2009-07-09 17:44 ` Thomas Gleixner
2009-07-09 21:19 ` Joao Correia
2009-07-07 13:20 ` Jarek Poplawski
2009-07-06 17:24 ` Andres Freund
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4A4C0144.5070203@anarazel.de \
--to=andres@anarazel.de \
--cc=jarkao2@gmail.com \
--cc=kaber@trash.net \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=shemminger@vyatta.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).