All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ben Greear <greearb@candelatech.com>
To: Joe Lawrence <joe.lawrence@stratus.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	stable@vger.kernel.org
Subject: Re: Please add to stable:  module: don't unlink the module until we've removed all exposure.
Date: Tue, 04 Jun 2013 09:53:21 -0700	[thread overview]
Message-ID: <51AE1B81.20900@candelatech.com> (raw)
In-Reply-To: <20130604100744.7cdf8777@jlaw-desktop.mno.stratus.com>

On 06/04/2013 07:07 AM, Joe Lawrence wrote:
> On Tue, 04 Jun 2013 15:26:28 +0930
> Rusty Russell <rusty@rustcorp.com.au> wrote:
>
>> Do you have a backtrace of the 3.9.4 crash?  You can add "CFLAGS_module.o
>> = -O0" to get a clearer backtrace if you want...
>
> Hi Rusty,
>
> See my 3.9 stack traces below, which may or may not be what Ben had
> been seeing.  If you like, I can try a similar loop as the one you were
> testing in the other email.

My stack traces are similar.  I had better luck reproducing the problem
once I enabled lots of debugging (slub memory poisoning, lockdep,
object debugging, etc).

I'm using Fedora 17 on 2-core core-i7 (4 CPU threads total) for most of this
testing.  We reproduced on dual-core Atom system as well
(32-bit Fedora 14 and Fedora 17).  Relatively standard hardware as far
as I know.

I'll run the insmod/rmmod stress test on my patched systems
and see if I can reproduce with the patch in the title applied.

Rusty:  I'm also seeing lockups related to migration on stock 3.9.4+
(with and without the 'don't unlink the module...' patch.  Much harder
to reproduce.  But, that code appears to be mostly called during
module load/unload, so it's possible it is related.  The first
traces are from a system with local patches, applied, but a later
post by me has traces from clean upstream kernel.

Further debugging showed that this could be a race, because it seems
that all migration/ threads think they are done with their state machine,
but the atomic thread counter sits at 1, so no progress is ever made.

http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg443471.html

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com


  parent reply	other threads:[~2013-06-04 16:53 UTC|newest]

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-05-31 18:14 Please add to stable: module: don't unlink the module until we've removed all exposure Ben Greear
2013-06-02  5:09 ` Rusty Russell
2013-06-03  3:46   ` Joe Lawrence
2013-06-03 11:25     ` Joe Lawrence
2013-06-03 14:17       ` Joe Lawrence
2013-06-03 15:59         ` Ben Greear
2013-06-03 16:36           ` Ben Greear
2013-06-04  4:37             ` Rusty Russell
2013-06-04  5:56             ` Rusty Russell
2013-06-04 14:07               ` Joe Lawrence
2013-06-04 16:50                 ` Joe Lawrence
2013-06-04 16:53                 ` Ben Greear [this message]
2013-06-04 17:45                   ` Ben Greear
2013-06-05  4:17                     ` Rusty Russell
2013-06-05  7:15                       ` Tejun Heo
2013-06-05 16:59                         ` Ben Greear
2013-06-05 18:48                           ` Tejun Heo
2013-06-05 19:11                             ` Ben Greear
2013-06-05 19:31                               ` stop_machine lockup issue in 3.9.y Ben Greear
2013-06-05 20:58                                 ` Ben Greear
2013-06-05 21:11                                   ` [ath9k-devel] " Tejun Heo
2013-06-05 21:11                                     ` Tejun Heo
2013-06-05 21:11                                     ` Tejun Heo
2013-06-05 21:33                                     ` [ath9k-devel] " Ben Greear
2013-06-05 21:33                                       ` Ben Greear
2013-06-06  1:34                                     ` [ath9k-devel] " Eric Dumazet
2013-06-06  1:34                                       ` Eric Dumazet
2013-06-06  1:34                                       ` Eric Dumazet
2013-06-06  3:14                                       ` [ath9k-devel] " Tejun Heo
2013-06-06  3:14                                         ` Tejun Heo
2013-06-06  3:14                                         ` Tejun Heo
2013-06-06  3:26                                         ` [ath9k-devel] " Eric Dumazet
2013-06-06  3:26                                           ` Eric Dumazet
2013-06-06  3:26                                           ` Eric Dumazet
2013-06-06  3:41                                           ` [ath9k-devel] " Ben Greear
2013-06-06  3:41                                             ` Ben Greear
2013-06-06  3:46                                             ` [ath9k-devel] " Eric Dumazet
2013-06-06  3:46                                               ` Eric Dumazet
2013-06-06  3:50                                               ` [ath9k-devel] " Ben Greear
2013-06-06  3:50                                                 ` Ben Greear
2013-06-06  4:08                                                 ` [ath9k-devel] " Eric Dumazet
2013-06-06  4:08                                                   ` Eric Dumazet
2013-06-06 20:55                                             ` [ath9k-devel] " Tejun Heo
2013-06-06 20:55                                               ` Tejun Heo
2013-06-06 21:15                                               ` [ath9k-devel] " Ben Greear
2013-06-06 21:15                                                 ` Ben Greear
2013-06-06 21:17                                                 ` [ath9k-devel] " Tejun Heo
2013-06-06 21:17                                                   ` Tejun Heo
2013-06-05  3:29                 ` Please add to stable: module: don't unlink the module until we've removed all exposure Rusty Russell
2013-06-05  5:07         ` Greg KH
2013-06-05  7:13           ` Rusty Russell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=51AE1B81.20900@candelatech.com \
    --to=greearb@candelatech.com \
    --cc=joe.lawrence@stratus.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=rusty@rustcorp.com.au \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.