From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Gerd Jakobovitsch <gerd@alog.com.br>, daniel.stodden@citrix.com
Cc: xen-devel@lists.xensource.com
Subject: Re: Tapdisk failures / kernel general protection fault at xen 4.0.2rc3 / kernel pvops 2.6.32.36
Date: Thu, 14 Apr 2011 09:15:43 -0400 [thread overview]
Message-ID: <20110414131543.GE5548@dumpdata.com> (raw)
In-Reply-To: <4DA60F55.4000604@alog.com.br>
On Wed, Apr 13, 2011 at 06:02:13PM -0300, Gerd Jakobovitsch wrote:
> I'm trying to run several VMs (linux hvm, with tapdisk:aio disks at
> a storage over nfs) on a CentOS system, using the up-to-date version
> of xen 4.0 / kernel pvops 2.6.32.x stable. With a configuration
> without (most of) debug activated, I can start several instances -
> I'm running 7 of them - but shortly afterwards the system stops
> responding. I can't find any information on this.
First time I see it.
>
> Activating several debug configuration items, among them
> DEBUG_PAGEALLOC, I get an exception as soon as I try to start up a
> VM. The system reboots.
Oooh, and is the log below from that situation?
Daniel, any thoughs?
>
> Below the log from /var/log/messages:
>
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: Created
> /dev/xen/blktap-2/control device
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: Created
> /dev/xen/blktap-2/blktap0 device
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: Created
> /dev/xen/blktap-2/tapdev0 device
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: new interface: ring:
> 251, device: 253, minor: 0
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: I/O queue driver: lio
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: block-aio open('/storage5_nfs/3/CD996633-linux-centos-5-64b-base-rip-sx-7253/hda')
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: open(/storage5_nfs/3/CD996633-linux-centos-5-64b-base-rip-sx-7253/hda)
> with O_DIRECT
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: Image size: pre
> sector_shift [134217728] post sector_shift [262144]
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: opened image
> /storage5_nfs/3/CD996633-linux-centos-5-64b-base-rip-sx-7253/hda (1
> users, state: 0x00000001, type: 0)
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]: VBD CHAIN:
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[4988]:
> /storage5_nfs/3/CD996633-linux-centos-5-64b-base-rip-sx-7253/hda: 0
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.158549] block tda:
> sector-size: 512 capacity: 262144
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.200514] general
> protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.200703] last sysfs
> file: /sys/block/tda/removable
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.200761] CPU 0
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.200847] Modules linked
> in: bridge stp bonding bnx2i libiscsi scsi_transport_iscsi cnic uio
> bnx2 megaraid_sas
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201363] Pid: 4988,
> comm: tapdisk2 Not tainted 2.6.32.36 #3 PowerEdge M610
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201424] RIP:
> e030:[<ffffffff812b9c24>] [<ffffffff812b9c24>]
> blktap_device_end_request+0x49/0x5e
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201543] RSP:
> e02b:ffff88006a7f7cd8 EFLAGS: 00010046
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201600] RAX:
> 6b6b6b6b6b6b6b6b RBX: ffff88006a6fc000 RCX: ffff88006a7f7c38
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201662] RDX:
> 0000000000000000 RSI: 0000000000000000 RDI: ffff88006a5c3500
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201723] RBP:
> ffff88006a7f7cf8 R08: ffffffff818383c0 R09: ffff88006a7f7c38
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201784] R10:
> 0000000000000000 R11: ffff88007b697b18 R12: ffff88007b697b18
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201845] R13:
> ffff88006a5c3360 R14: 0000000000000000 R15: ffff88006a5c3370
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201910] FS:
> 00007f50a9445730(0000) GS:ffff8800280c7000(0000)
> knlGS:0000000000000000
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.201974] CS: e033 DS:
> 0000 ES: 0000 CR0: 000000008005003b
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.202032] CR2:
> 00007fb35d12e6e8 CR3: 000000006a4ce000 CR4: 0000000000002660
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.202093] DR0:
> 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.202154] DR3:
> 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.202436] Process
> tapdisk2 (pid: 4988, threadinfo ffff88006a7f6000, task
> ffff88006b5a0000)
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.202941] Stack:
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.203206]
> ffff88006b5a0000 0000000000000000 0000000000000000 0000000000000000
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.203609] <0>
> ffff88006a7f7e88 ffffffff812b9416 ffff88006a6c80f8 0000000100000000
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.204310] <0>
> 00000000ffffffff ffff88006a5c3360 000000017edd7ab0 0000000000000000
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.205284] Call Trace:
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.205553]
> [<ffffffff812b9416>] blktap_ring_ioctl+0x183/0x2d8
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.205838]
> [<ffffffff81209a64>] ? inode_has_perm+0xa1/0xb3
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.206120]
> [<ffffffff8157641f>] ? _spin_unlock+0x26/0x2a
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.206400]
> [<ffffffff81126ff9>] ? aio_read_evt+0x56/0xe0
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.206678]
> [<ffffffff81127071>] ? aio_read_evt+0xce/0xe0
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.206957]
> [<ffffffff8124f5c1>] ? _raw_spin_lock+0x77/0x12d
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.207236]
> [<ffffffff81209bf8>] ? file_has_perm+0xb4/0xc6
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.207516]
> [<ffffffff8110464e>] vfs_ioctl+0x5e/0x77
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.207793]
> [<ffffffff81104b63>] do_vfs_ioctl+0x484/0x4d5
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.208069]
> [<ffffffff81104c0b>] sys_ioctl+0x57/0x7a
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.208346]
> [<ffffffff81012cc2>] system_call_fastpath+0x16/0x1b
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.208621] Code: 89 de 4c
> 89 ef e8 60 f4 ff ff 49 8b 44 24 40 48 8b b8 90 04 00 00 e8 41 c9 2b
> 00 44 89 f6 4c 89 e7 e8 39 fc ff ff 49 8b 44 24 40 <48> 8b b8 90 04
> 00 00 e8 66 c7 2b 00 5b 41 5c 41 5d 41 5e c9 c3
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.211986] RIP
> [<ffffffff812b9c24>] blktap_device_end_request+0x49/0x5e
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.212306] RSP <ffff88006a7f7cd8>
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.212579] ---[ end trace
> b97070122f44735d ]---
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: Created
> /dev/xen/blktap-2/blktap1 device
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: Created
> /dev/xen/blktap-2/tapdev1 device
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: new interface: ring:
> 251, device: 253, minor: 1
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: I/O queue driver: lio
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: block-aio open('/storage5_nfs/3/CD996633-linux-centos-5-64b-base-rip-sx-7253/xvda')
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: open(/storage5_nfs/3/CD996633-linux-centos-5-64b-base-rip-sx-7253/xvda)
> with O_DIRECT
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: Image size: pre
> sector_shift [10737418240] post sector_shift [20971520]
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: opened image
> /storage5_nfs/3/CD996633-linux-centos-5-64b-base-rip-sx-7253/xvda (1
> users, state: 0x00000001, type: 0)
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]: VBD CHAIN:
> Apr 13 17:47:23 r2b16ch2x28p2 tapdisk2[5009]:
> /storage5_nfs/3/CD996633-linux-centos-5-64b-base-rip-sx-7253/xvda: 0
> Apr 13 17:47:23 r2b16ch2x28p2 kernel: [ 179.317931] block tdb:
> sector-size: 512 capacity: 20971520
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
next prev parent reply other threads:[~2011-04-14 13:15 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-04-13 21:02 Tapdisk failures / kernel general protection fault at xen 4.0.2rc3 / kernel pvops 2.6.32.36 Gerd Jakobovitsch
2011-04-14 13:15 ` Konrad Rzeszutek Wilk [this message]
2011-04-14 16:38 ` Daniel Stodden
2011-04-14 17:32 ` Daniel Stodden
2011-04-14 17:43 ` Daniel Stodden
2011-04-14 18:05 ` Gerd Jakobovitsch
2011-04-14 23:42 ` Daniel Stodden
2011-04-15 14:21 ` xen 4.0.2rc3/kernel 2.6.32.36: BUG: unable to handle kernel paging request Gerd Jakobovitsch
2011-04-15 15:44 ` Pasi Kärkkäinen
2011-04-15 15:47 ` Gerd Jakobovitsch
2011-04-18 21:05 ` Konrad Rzeszutek Wilk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110414131543.GE5548@dumpdata.com \
--to=konrad.wilk@oracle.com \
--cc=daniel.stodden@citrix.com \
--cc=gerd@alog.com.br \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.