WARNING: at kernel/timer.c:1012 del_timer_sync+0x39/0x4e()

WARNING: at kernel/timer.c:1012 del_timer_sync+0x39/0x4e()

[Jeremy Fitzhardinge]

In today's linux-2.6.38+ I'm seeing this in my Xen domains:

------------[ cut here ]------------
WARNING: at /home/jeremy/git/upstream/kernel/timer.c:1012 del_timer_sync+0x39/0x4e()
Modules linked in: sunrpc microcode [last unloaded: scsi_wait_scan]
Pid: 0, comm: swapper Tainted: G        W   2.6.38+ #349
Call Trace:
 <IRQ>  [<ffffffff8104c283>] ? del_timer_sync+0x39/0x4e
 [<ffffffff8103f080>] warn_slowpath_common+0x85/0x9d
 [<ffffffff8103f0b2>] warn_slowpath_null+0x1a/0x1c
 [<ffffffff8104c283>] del_timer_sync+0x39/0x4e
 [<ffffffff81206d4e>] blk_stop_queue+0x1d/0x6d
 [<ffffffff812e82f6>] do_blkif_request+0x35d/0x3ba
 [<ffffffff8120560f>] __blk_run_queue+0x7d/0xd7
 [<ffffffff81205880>] blk_start_queue+0x78/0x7d
 [<ffffffff812e837b>] kick_pending_request_queues+0x28/0x38
 [<ffffffff812e85d5>] blkif_interrupt+0x1f8/0x21c
 [<ffffffff8108e866>] handle_irq_event_percpu+0x61/0x1a5
 [<ffffffff8108e9fa>] handle_irq_event+0x50/0x74
 [<ffffffff81090c87>] handle_fasteoi_irq+0x7e/0xaa
 [<ffffffff812853aa>] __xen_evtchn_do_upcall+0x166/0x206
 [<ffffffff81286d32>] xen_evtchn_do_upcall+0x2f/0x42
 [<ffffffff8150148e>] xen_do_hypervisor_callback+0x1e/0x30
 <EOI>  [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
 [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
 [<ffffffff81006c34>] ? xen_safe_halt+0x10/0x18
 [<ffffffff81010251>] ? default_idle+0x6a/0xc3
 [<ffffffff810091af>] ? cpu_idle+0x6b/0x8a
 [<ffffffff814e1b2e>] ? rest_init+0x72/0x74
 [<ffffffff81b04b70>] ? start_kernel+0x39b/0x3a6
 [<ffffffff81b042cb>] ? x86_64_start_reservations+0xb6/0xba
 [<ffffffff81b07d6f>] ? xen_start_kernel+0x5dc/0x5e3
---[ end trace 1605d7ec4cdfe9eb ]---

Which is the result of this call to blk_stop_queue() in drivers/block/xen-blkfront.c:

static void do_blkif_request(struct request_queue *rq)
{
[...]
		if (blkif_queue_request(req)) {
			blk_requeue_request(rq, req);
wait:
			/* Avoid pointless unplugs. */
			blk_stop_queue(rq);
			break;
		}



How should this be fixed? Is this call to blk_stop_queue() bogus, or is
it OK to call it in a handler like this?  If so, is the warning itself
wrong?

Does this relate to  7eaceaccab5f4 "block: remove per-queue plugging"?

Thanks,
    J

Re: WARNING: at kernel/timer.c:1012 del_timer_sync+0x39/0x4e()

[Jens Axboe]

On 2011-03-25 13:28, Jeremy Fitzhardinge wrote:
> In today's linux-2.6.38+ I'm seeing this in my Xen domains:
> 
> ------------[ cut here ]------------
> WARNING: at /home/jeremy/git/upstream/kernel/timer.c:1012 del_timer_sync+0x39/0x4e()
> Modules linked in: sunrpc microcode [last unloaded: scsi_wait_scan]
> Pid: 0, comm: swapper Tainted: G        W   2.6.38+ #349
> Call Trace:
>  <IRQ>  [<ffffffff8104c283>] ? del_timer_sync+0x39/0x4e
>  [<ffffffff8103f080>] warn_slowpath_common+0x85/0x9d
>  [<ffffffff8103f0b2>] warn_slowpath_null+0x1a/0x1c
>  [<ffffffff8104c283>] del_timer_sync+0x39/0x4e
>  [<ffffffff81206d4e>] blk_stop_queue+0x1d/0x6d
>  [<ffffffff812e82f6>] do_blkif_request+0x35d/0x3ba
>  [<ffffffff8120560f>] __blk_run_queue+0x7d/0xd7
>  [<ffffffff81205880>] blk_start_queue+0x78/0x7d
>  [<ffffffff812e837b>] kick_pending_request_queues+0x28/0x38
>  [<ffffffff812e85d5>] blkif_interrupt+0x1f8/0x21c
>  [<ffffffff8108e866>] handle_irq_event_percpu+0x61/0x1a5
>  [<ffffffff8108e9fa>] handle_irq_event+0x50/0x74
>  [<ffffffff81090c87>] handle_fasteoi_irq+0x7e/0xaa
>  [<ffffffff812853aa>] __xen_evtchn_do_upcall+0x166/0x206
>  [<ffffffff81286d32>] xen_evtchn_do_upcall+0x2f/0x42
>  [<ffffffff8150148e>] xen_do_hypervisor_callback+0x1e/0x30
>  <EOI>  [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
>  [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
>  [<ffffffff81006c34>] ? xen_safe_halt+0x10/0x18
>  [<ffffffff81010251>] ? default_idle+0x6a/0xc3
>  [<ffffffff810091af>] ? cpu_idle+0x6b/0x8a
>  [<ffffffff814e1b2e>] ? rest_init+0x72/0x74
>  [<ffffffff81b04b70>] ? start_kernel+0x39b/0x3a6
>  [<ffffffff81b042cb>] ? x86_64_start_reservations+0xb6/0xba
>  [<ffffffff81b07d6f>] ? xen_start_kernel+0x5dc/0x5e3
> ---[ end trace 1605d7ec4cdfe9eb ]---
> 
> Which is the result of this call to blk_stop_queue() in drivers/block/xen-blkfront.c:
> 
> static void do_blkif_request(struct request_queue *rq)
> {
> [...]
> 		if (blkif_queue_request(req)) {
> 			blk_requeue_request(rq, req);
> wait:
> 			/* Avoid pointless unplugs. */
> 			blk_stop_queue(rq);
> 			break;
> 		}
> 
> 
> 
> How should this be fixed? Is this call to blk_stop_queue() bogus, or is
> it OK to call it in a handler like this?  If so, is the warning itself
> wrong?

Calling it in there should be fine, the problem is that the work cancel
turns into a sync timer delete. That is not needed. Can you try the
below?

> Does this relate to  7eaceaccab5f4 "block: remove per-queue plugging"?

No

diff --git a/block/blk-core.c b/block/blk-core.c
index 59b5c00..f97c088 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -271,7 +271,7 @@ EXPORT_SYMBOL(blk_start_queue);
  **/
 void blk_stop_queue(struct request_queue *q)
 {
-	cancel_delayed_work(&q->delay_work);
+	__cancel_delayed_work(&q->delay_work);
 	queue_flag_set(QUEUE_FLAG_STOPPED, q);
 }
 EXPORT_SYMBOL(blk_stop_queue);


-- 
Jens Axboe

Re: WARNING: at kernel/timer.c:1012 del_timer_sync+0x39/0x4e()

[Jeremy Fitzhardinge]

On 03/25/2011 12:52 PM, Jens Axboe wrote:
>> How should this be fixed? Is this call to blk_stop_queue() bogus, or is
>> it OK to call it in a handler like this?  If so, is the warning itself
>> wrong?
> Calling it in there should be fine, the problem is that the work cancel
> turns into a sync timer delete. That is not needed. Can you try the
> below?

Yep, that works.  Tested-by: Jeremy Fitzhardinge
<jeremy.fitzhardinge@citrix.com>

>> Does this relate to  7eaceaccab5f4 "block: remove per-queue plugging"?
> No

OK.  Is the correct fix for that change to just remove all the
plug/unplug code?  We have some still out-of-tree code that stopped
compiling.

Thanks,
    J