linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Dave Jones <davej@redhat.com>
To: Stephen Smalley <sds@tycho.nsa.gov>,
	Linux Kernel <linux-kernel@vger.kernel.org>,
	eparis@redhat.com, james.l.morris@oracle.com
Subject: Re: [PATCH] conditionally reschedule while loading selinux policy.
Date: Mon, 30 Sep 2013 13:24:47 -0400	[thread overview]
Message-ID: <20130930172447.GA10562@redhat.com> (raw)
In-Reply-To: <20130916184030.GA15800@redhat.com>

On Mon, Sep 16, 2013 at 02:40:30PM -0400, Dave Jones wrote:
 > On a slow machine (with debugging enabled), upgrading selinux policy may take
 > a considerable amount of time. Long enough that the softlockup detector
 > gets triggered.
 > 
 > The backtrace looks like this..
 > 
 >  > BUG: soft lockup - CPU#2 stuck for 23s! [load_policy:19045]
 >  > Call Trace:
 >  >  [<ffffffff81221ddf>] symcmp+0xf/0x20
 >  >  [<ffffffff81221c27>] hashtab_search+0x47/0x80
 >  >  [<ffffffff8122e96c>] mls_convert_context+0xdc/0x1c0
 >  >  [<ffffffff812294e8>] convert_context+0x378/0x460
 >  >  [<ffffffff81229170>] ? security_context_to_sid_core+0x240/0x240
 >  >  [<ffffffff812221b5>] sidtab_map+0x45/0x80
 >  >  [<ffffffff8122bb9f>] security_load_policy+0x3ff/0x580
 
With that patch applied, the problem seems to have moved elsewhere..

BUG: soft lockup - CPU#3 stuck for 22s! [load_policy:8119]
 irq event stamp: 1590886
 hardirqs last  enabled at (1590885): [<ffffffff8152c3dd>] __slab_alloc.constprop.78+0x4c0/0x4d7
 hardirqs last disabled at (1590886): [<ffffffff8153e06a>] apic_timer_interrupt+0x6a/0x80
 softirqs last  enabled at (1590336): [<ffffffff810480d9>] __do_softirq+0x169/0x200
 softirqs last disabled at (1590331): [<ffffffff8104839d>] irq_exit+0x11d/0x140
 RIP: 0010:[<ffffffff81223182>]  [<ffffffff81223182>] hashtab_insert+0x62/0x110

Call Trace:
 [<ffffffff812283b5>] policydb_read+0xc25/0x1200
 [<ffffffff810b1639>] ? is_module_text_address+0x19/0x40
 [<ffffffff8122cefe>] security_load_policy+0x10e/0x580
 [<ffffffff81078878>] ? sched_clock_cpu+0xa8/0x100
 [<ffffffff810786ad>] ? sched_clock_local+0x1d/0x80
 [<ffffffff81078878>] ? sched_clock_cpu+0xa8/0x100
 [<ffffffff810786ad>] ? sched_clock_local+0x1d/0x80
 [<ffffffff81078878>] ? sched_clock_cpu+0xa8/0x100
 [<ffffffff8103098a>] ? __change_page_attr_set_clr+0x82a/0xa50
 [<ffffffff81078878>] ? sched_clock_cpu+0xa8/0x100
 [<ffffffff8153669c>] ? retint_restore_args+0xe/0xe
 [<ffffffff810a196d>] ? trace_hardirqs_on_caller+0xfd/0x1c0
 [<ffffffff8127b07e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
 [<ffffffff810d2958>] ? rcu_irq_exit+0x68/0xb0
 [<ffffffff8153669c>] ? retint_restore_args+0xe/0xe
 [<ffffffff8121ff97>] sel_write_load+0xa7/0x770
 [<ffffffff811394c3>] ? vfs_write+0x1c3/0x200
 [<ffffffff812124de>] ? security_file_permission+0x1e/0xa0
 [<ffffffff811393bb>] vfs_write+0xbb/0x200
 [<ffffffff81158337>] ? fget_light+0x397/0x4b0
 [<ffffffff81139ab7>] SyS_write+0x47/0xa0
 [<ffffffff8153d634>] tracesys+0xdd/0xe2

We're holding a bunch of locks here, so we can't just cond_resched.  Thoughts ?

	Dave


  parent reply	other threads:[~2013-09-30 17:25 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-09-16 17:30 lockup during selinux policy load Dave Jones
2013-09-16 17:43 ` Stephen Smalley
2013-09-16 18:40   ` [PATCH] conditionally reschedule while loading selinux policy Dave Jones
2013-09-16 18:51     ` Stephen Smalley
2013-09-30 17:24     ` Dave Jones [this message]
2013-09-30 17:37       ` Stephen Smalley
2013-09-30 21:13         ` Dave Jones
2013-10-03 20:36           ` Paul Moore
2013-10-03 20:40             ` Dave Jones
2013-10-03 20:49               ` Paul Moore

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130930172447.GA10562@redhat.com \
    --to=davej@redhat.com \
    --cc=eparis@redhat.com \
    --cc=james.l.morris@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=sds@tycho.nsa.gov \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).