public inbox for netdev@vger.kernel.org
 help / color / mirror / Atom feed
From: Eric Dumazet <eric.dumazet@gmail.com>
To: Joe Buehler <aspam@cox.net>
Cc: netdev@vger.kernel.org
Subject: Re: kernel panic in fib_rules_lookup [2.6.27.7 vendor-patched]
Date: Sat, 23 Oct 2010 18:35:45 +0200	[thread overview]
Message-ID: <1287851745.2658.364.camel@edumazet-laptop> (raw)
In-Reply-To: <4CC30055.5040509@cox.net>

Le samedi 23 octobre 2010 à 11:33 -0400, Joe Buehler a écrit :

> It is always possible that there is some issue with the Octeon memory
> barrier stuff, but I would think that the system would be much more
> unstable than it is -- we're really beating on a dual CPU LINUX instance
> that has Java and C++ apps running and also doing some network I/O.
> 
> My strategy at this point is logging events to memory and dumping the
> log to the console at the time of the panic.  I might be able to figure
> out the sequence of events causing the crash.
> 
> The load test that causes the panic is using several dozen TAP
> interfaces, ifconfig'd up/down every 10 seconds or so, with
> source-routes, DNAT and SNAT being set up and taken down also.

With a normal workload, on a dual cpu machine, a missing memory barrier
can stay un-noticed for quite a long time. The race window is so small
that probability for the bug might be 0.0000001 % or something like
that :(

You could try to run a test dual threaded program to reproduce the
problem in user land, faster...




  reply	other threads:[~2010-10-23 16:35 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-10-12 17:14 kernel panic in fib_rules_lookup [2.6.27.7 vendor-patched] Joe Buehler
2010-10-12 17:40 ` Eric Dumazet
2010-10-13 11:18   ` Joe Buehler
2010-10-20 17:43   ` Joe Buehler
2010-10-20 22:05     ` Eric Dumazet
2010-10-22 20:30       ` Joe Buehler
2010-10-23  3:44         ` Eric Dumazet
2010-10-23 15:11           ` Eric Dumazet
2010-10-23 15:24             ` Eric Dumazet
2010-10-23 15:40               ` Joe Buehler
2010-10-23 16:07                 ` Eric Dumazet
2010-10-23 15:33             ` Joe Buehler
2010-10-23 16:35               ` Eric Dumazet [this message]
2010-10-23 17:37                 ` "Oleg A. Arkhangelsky"
2010-10-23 19:44                   ` Eric Dumazet
2010-10-26 18:43                     ` David Miller
2010-10-26 13:29                 ` Joe Buehler
2010-10-26 13:36                   ` Eric Dumazet
2010-10-26 13:53                     ` Joe Buehler
2010-10-26 13:58                       ` Eric Dumazet
2010-10-26 14:33                       ` Eric Dumazet
2010-10-26 14:41                         ` Joe Buehler
2010-10-27 21:01                         ` Joe Buehler
2010-10-27 21:05                           ` Eric Dumazet
2010-12-02 13:36                             ` Joe Buehler
2010-12-02 14:04                               ` Eric Dumazet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1287851745.2658.364.camel@edumazet-laptop \
    --to=eric.dumazet@gmail.com \
    --cc=aspam@cox.net \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox