Re: next-20081215: kernel BUG at block/cfq-iosched.c:1243

[Jens Axboe <jens.axboe@oracle.com>]

On Mon, Dec 15 2008, Alexander Beregalov wrote:
> 2008/12/15 Jens Axboe <jens.axboe@oracle.com>:
> >> [   23.080733] kernel BUG at block/cfq-iosched.c:1243!
> >> [   23.080733] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> 
> > Odd, is it reproducible? What is your .config?
> Yes, it is
> 
> Config is in attach

It's weird, because the only change for CFQ in that area is the one
below. Does the oops go away if you back this out?


>From 030a3f02bb42fad91ae84c99924fe3ee162a634d Mon Sep 17 00:00:00 2001
From: Jens Axboe <jens.axboe@oracle.com>
Date: Fri, 12 Dec 2008 15:56:31 +0100
Subject: [PATCH] cfq-iosched: fix race between exiting queue and exiting task

Original patch from Nikanth Karthikesan <knikanth@suse.de>

When a queue exits the queue lock is taken and cfq_exit_queue() would free all
the cic's associated with the queue.

But when a task exits, cfq_exit_io_context() gets cic one by one and then
locks the associated queue to call __cfq_exit_single_io_context. It looks like
between getting a cic from the ioc and locking the queue, the queue might have
exited on another cpu.

Fix this by rechecking the cfq_io_context queue key inside the queue lock
again, and not calling into __cfq_exit_single_io_context() if somebody
beat us to it.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
---
 block/cfq-iosched.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index ee8a90c..8a2485d 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -1314,7 +1314,16 @@ static void cfq_exit_single_io_context(struct io_context *ioc,
 		unsigned long flags;
 
 		spin_lock_irqsave(q->queue_lock, flags);
+
+		/*
+		 * Ensure we get a fresh copy of the ->key to prevent
+		 * race between exiting task and queue
+		 */
+		smp_read_barrier_depends();
+		if (cic->key)
+			__cfq_exit_single_io_context(cfqd, cic);
 		__cfq_exit_single_io_context(cfqd, cic);
+
 		spin_unlock_irqrestore(q->queue_lock, flags);
 	}
 }
-- 
1.6.0.4.755.g6e139


-- 
Jens Axboe