From: Jeremy Fitzhardinge <jeremy@goop.org>
To: Daniel Stodden <daniel.stodden@citrix.com>
Cc: Xen-devel <xen-devel@lists.xensource.com>,
Jake Wires <Jake.Wires@citrix.com>
Subject: Re: Crash on blktap shutdown
Date: Wed, 24 Feb 2010 15:26:05 -0800 [thread overview]
Message-ID: <4B85B58D.20204@goop.org> (raw)
In-Reply-To: <1267053644.5962.409.camel@agari.van.xensource.com>
On 02/24/2010 03:20 PM, Daniel Stodden wrote:
> Jake, any immediate ideas?
>
Just got another one on domain shutdown. The crashing instruction is:
0xffffffff8104a3f2 <lock_timer_base+17>: mov 0x28(%r12),%r14
r12 = 6b6b6b6b6b6b6c8b
0x6b is the use-after-free poison value. So I think a use-after-free.
0xffffffff8104a3f2 is in lock_timer_base (/home/jeremy/git/linux/kernel/timer.c:620).
615 __acquires(timer->base->lock)
616 {
617 struct tvec_base *base;
618
619 for (;;) {
620 struct tvec_base *prelock_base = timer->base;
621 base = tbase_get_base(prelock_base);
622 if (likely(base != NULL)) {
623 spin_lock_irqsave(&base->lock, *flags);
624 if (likely(prelock_base == timer->base))
general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
last sysfs file: /sys/devices/virtual/blktap2/blktap0/remove
CPU 1
Modules linked in: e1000 evdev ahci dm_mod sd_mod mptspi mptscsih mptbase scsi_]
Pid: 6533, comm: xend Not tainted 2.6.32.9 #356 PowerEdge 1850
RIP: e030:[<ffffffff8104a3f2>] [<ffffffff8104a3f2>] lock_timer_base+0x11/0x4d
RSP: e02b:ffff880021a73ce8 EFLAGS: 00010286
RAX: ffff88001d858f40 RBX: 6b6b6b6b6b6b6c8b RCX: 0000000000000000
RDX: ffffffff8104abda RSI: ffff880021a73d20 RDI: 6b6b6b6b6b6b6c8b
RBP: ffff880021a73d08 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff8104abda R11: ffff880003cd1810 R12: 6b6b6b6b6b6b6c8b
R13: ffff880021a73d20 R14: 000000000000011e R15: ffff880021a73e20
FS: 00007f164dffb910(0000) GS:ffff8800028fb000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000001d62140 CR3: 000000002eac8000 CR4: 0000000000000660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process xend (pid: 6533, threadinfo ffff880021a72000, task ffff88001d858f40)
Stack:
6b6b6b6b6b6b6c8b 00000000ffffffff ffff88002f2802e8 000000000000011e
<0> ffff880021a73d38 ffffffff8104a7b5 0000000000000001 ffffffff8104abda
<0> 6b6b6b6b6b6b6c8b 6b6b6b6b6b6b6cbb ffff880021a73d78 ffffffff8104ac68
Call Trace:
[<ffffffff8104a7b5>] try_to_del_timer_sync+0x1b/0x81
[<ffffffff8104abda>] ? del_timer_sync+0x0/0xa1
[<ffffffff8104ac68>] del_timer_sync+0x8e/0xa1
[<ffffffff8104abda>] ? del_timer_sync+0x0/0xa1
[<ffffffff811e79b7>] ? kobject_release+0x0/0x66
[<ffffffff811d842c>] blk_sync_queue+0x18/0x34
[<ffffffff811d8457>] blk_cleanup_queue+0xf/0x4b
[<ffffffff81254039>] blktap_device_destroy+0xad/0xd7
[<ffffffff812512a5>] blktap_control_destroy_device+0x55/0x154
[<ffffffff81390438>] ? mutex_lock_nested+0x2a5/0x2b4
[<ffffffff81254de5>] blktap_sysfs_remove_device+0x39/0x49
[<ffffffff81294170>] dev_attr_store+0x1b/0x1d
[<ffffffff810fa9d4>] sysfs_write_file+0xf6/0x132
[<ffffffff810b03f0>] vfs_write+0xad/0x14e
[<ffffffff810b0c1b>] ? fget_light+0x52/0xeb
[<ffffffff811eab92>] ? __up_read+0x1c/0xa2
[<ffffffff810b054a>] sys_write+0x45/0x6c
[<ffffffff81011c82>] system_call_fastpath+0x16/0x1b
Code: 55 31 d2 48 89 e5 31 f6 65 48 8b 3c 25 c0 cb 00 00 e8 95 77 00 00 c9 48 9
RIP [<ffffffff8104a3f2>] lock_timer_base+0x11/0x4d
RSP<ffff880021a73ce8>
---[ end trace 767ddf28dd1b4a3e ]---
> Daniel
>
> On Wed, 2010-02-24 at 17:55 -0500, Jeremy Fitzhardinge wrote:
>
>> When rebooting the machine, I got this crash from blktap. The rip maps to line 262 in
>> 0xffffffff812548a1 is in blktap_request_pool_free (/home/jeremy/git/linux/drivers/xen/blktap/request.c:262).
>> 257 spin_lock_irqsave(&pool.lock, flags);
>> 258
>> 259 pool.status = BLKTAP_POOL_CLOSING;
>> 260 while (atomic_read(&pool.reqs_in_use)) {
>> 261 spin_unlock_irqrestore(&pool.lock, flags);
>> 262 wait_event(pool.wait_queue, !atomic_read(&pool.reqs_in_use));
>> 263 spin_lock_irqsave(&pool.lock, flags);
>> 264 }
>> 265
>> 266 for (i = 0; i< MAX_BUCKETS; i++) {
>>
>>
>> blktap_ring_vm_close: unmapping ring 6
>> blktap_ring_release: freeing device 6
>> general protection fault: 0000 [#2] SMP DEBUG_PAGEALLOC
>> last sysfs file: /sys/devices/virtual/net/eth0/address
>> CPU 1
>> Modules linked in: e1000 evdev ahci dm_mod sd_mod mptspi mptscsih mptbase scsi_]
>> Pid: 993, comm: tapdisk2 Tainted: G D 2.6.32.8 #355 PowerEdge 1850
>> RIP: e030:[<ffffffff8125413b>] [<ffffffff8125413b>] blktap_device_restart+0x7a8
>> RSP: e02b:ffff88002d767be8 EFLAGS: 00010092
>> RAX: ffff88002ea06b08 RBX: ffff88002f319090 RCX: 0000000000000000
>> RDX: 0000000000000000 RSI: 0000000000000000 RDI: 6b6b6b6b6b6b6b6b
>> RBP: ffff88002d767bf8 R08: 0000000000000002 R09: 0000000000000001
>> R10: ffffffff8125412d R11: ffffffff811eaa4a R12: ffff88002f319330
>> R13: ffff88002f3191b8 R14: ffff8800242a3a50 R15: 0000000000000001
>> FS: 00007f7e3234a740(0000) GS:ffff8800028fb000(0000) knlGS:0000000000000000
>> CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> CR2: 00000036a05a8d84 CR3: 000000002d364000 CR4: 0000000000000660
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> Process tapdisk2 (pid: 993, threadinfo ffff88002d766000, task ffff8800242c3d00)
>> Stack:
>> ffff88002f319090 ffff88002f319238 ffff88002d767c28 ffffffff81251b3b
>> <0> ffff8800242a3a50 ffff88002f2c2870 ffff880002909820 ffff88002400ad60
>> <0> ffff88002d767c48 ffffffff8109aead ffff8800242a3a50 ffff88002400ad00
>> Call Trace:
>> [<ffffffff81251b3b>] blktap_ring_vm_close+0x39/0x12d
>> [<ffffffff8109aead>] remove_vma+0x3b/0x71
>> [<ffffffff8109b036>] exit_mmap+0x153/0x175
>> [<ffffffff8103eef6>] mmput+0x3e/0xd9
>> [<ffffffff81042b83>] exit_mm+0x100/0x10b
>> [<ffffffff81044416>] do_exit+0x1b9/0x638
>> [<ffffffff8104d797>] ? get_signal_to_deliver+0x2dd/0x36e
>> [<ffffffff8100efef>] ? xen_restore_fl_direct_end+0x0/0x1
>> [<ffffffff81044908>] do_group_exit+0x73/0x9c
>> [<ffffffff8104d809>] get_signal_to_deliver+0x34f/0x36e
>> [<ffffffff810111c4>] do_signal+0x6d/0x6b0
>> [<ffffffff8104ef1f>] ? sys_getsid+0x88/0xaf
>> [<ffffffff810bd680>] ? poll_select_copy_remaining+0xc9/0xee
>> [<ffffffff8101182e>] do_notify_resume+0x27/0x47
>> [<ffffffff81390f80>] ? trace_hardirqs_on_thunk+0x3a/0x3f
>> [<ffffffff810549ef>] ? remove_wait_queue+0x12/0x45
>> [<ffffffff81011f56>] int_signal+0x12/0x17
>> Code: a8 01 74 0a 48 89 df e8 24 e6 ff ff eb 46 4c 8d a3 a0 02 00 00 4c 89 e7 e
>> RIP [<ffffffff8125413b>] blktap_device_restart+0x7a/0xa8
>> RSP<ffff88002d767be8>
>> ---[ end trace 1b88501e9b8effb5 ]---
>>
>> J
>>
>>
>
>
next prev parent reply other threads:[~2010-02-24 23:26 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-02-24 22:55 Crash on blktap shutdown Jeremy Fitzhardinge
2010-02-24 23:20 ` Daniel Stodden
2010-02-24 23:26 ` Jeremy Fitzhardinge [this message]
2010-02-25 0:12 ` Daniel Stodden
2010-02-25 0:16 ` Daniel Stodden
2010-02-25 0:24 ` [PATCH] Fix wild ptr deref during device destruction Daniel Stodden
2010-02-25 8:28 ` Jan Beulich
2010-02-25 9:57 ` Daniel Stodden
2010-02-25 10:02 ` Daniel Stodden
2010-02-25 22:54 ` Yet another [PATCH] blkfront: " Daniel Stodden
2010-02-24 23:49 ` Crash on blktap shutdown Daniel Stodden
2010-02-24 23:52 ` Jeremy Fitzhardinge
2010-02-25 0:29 ` Daniel Stodden
2010-02-25 0:37 ` Jeremy Fitzhardinge
2010-02-25 1:47 ` Daniel Stodden
2010-02-25 3:03 ` [PATCH] " Daniel Stodden
2010-02-25 23:18 ` Jeremy Fitzhardinge
2010-02-26 15:38 ` Daniel Stodden
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4B85B58D.20204@goop.org \
--to=jeremy@goop.org \
--cc=Jake.Wires@citrix.com \
--cc=daniel.stodden@citrix.com \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).