Re: 2.6.33.1-rt11 BUG: sleeping function called from invalid context at kernel/rtmutex.c:684

Thomas Gleixner
2010-04-06T08:07:31Z

On Sun, 4 Apr 2010, Udo van den Heuvel wrote:

> Hello,
>
> I see a load of these after booting into 2.6.33.1-rt11:
>
> BUG: sleeping function called from invalid context at kernel/rtmutex.c:684
> pcnt: 1 0 in_atomic(): 1, irqs_disabled(): 0, pid: 1507, name: md1_raid5
> Pid: 1507, comm: md1_raid5 Not tainted 2.6.33.1-rt11 #1
> Call Trace:
> [<ffffffff8138fb8c>] ? rt_spin_lock+0x2c/0x70
> [<ffffffff812becc4>] ? __raid_run_ops+0x304/0xc60
> [<ffffffff812c0ccd>] ? handle_stripe+0x6bd/0x1a70
> [<ffffffff8104b460>] ? mod_timer+0x150/0x200
> [<ffffffff812c23f6>] ? raid5d+0x376/0x4f0
> [<ffffffff8138e5bd>] ? schedule_timeout+0x22d/0x2b0
> [<ffffffff8138fb8c>] ? rt_spin_lock+0x2c/0x70
> [<ffffffff812cd0f3>] ? md_thread+0x53/0x120
> [<ffffffff810573a0>] ? autoremove_wake_function+0x0/0x30
> [<ffffffff812cd0a0>] ? md_thread+0x0/0x120
> [<ffffffff81057016>] ? kthread+0x96/0xa0
> [<ffffffff81037908>] ? finish_task_switch+0x58/0xd0
> [<ffffffff810032d4>] ? kernel_thread_helper+0x4/0x10
> [<ffffffff81056f80>] ? kthread+0x0/0xa0
> [<ffffffff810032d0>] ? kernel_thread_helper+0x0/0x10
>
> As these appear to be touching my raid array I am quite eager to learn
> how I can fix the BUGs.
>
> Please have a look and explain.

That's caused by the get_cpu()/put_cpu() preempt disabled region. Can
you try the following (untested) patch ?

Thanks,

tglx
---
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index ceb24af..b61eaa6 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -1149,8 +1149,9 @@ static void __raid_run_ops(struct stripe_head *sh, unsigned long ops_request)
struct raid5_percpu *percpu;
unsigned long cpu;

- cpu = get_cpu();
+ cpu = raw_smp_processor_id();
percpu = per_cpu_ptr(conf->percpu, cpu);
+ spin_lock(&percpu->lock);
if (test_bit(STRIPE_OP_BIOFILL, &ops_request)) {
ops_run_biofill(sh);
overlap_clear++;
@@ -1202,7 +1203,7 @@ static void __raid_run_ops(struct stripe_head *sh, unsigned long ops_request)
if (test_and_clear_bit(R5_Overlap, &dev->flags))
wake_up(&sh->raid_conf->wait_for_overlap);
}
- put_cpu();
+ spin_unlock(&percpu->lock);
}

#ifdef CONFIG_MULTICORE_RAID456
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index dd70835..2db71cd 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -400,6 +400,7 @@ struct raid5_private_data {
*/
/* per cpu variables */
struct raid5_percpu {
+ spinlock_t lock; /* Protection for -RT */
struct page *spare_page; /* Used when checking P/Q in raid6 */
void *scribble; /* space for constructing buffer
* lists and performing address