* Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-10 22:30 ` Bruce Edge
@ 2010-11-10 23:03 ` Bruce Edge
2010-11-11 15:56 ` Konrad Rzeszutek Wilk
2010-11-11 6:13 ` Bruce Edge
` (2 subsequent siblings)
3 siblings, 1 reply; 45+ messages in thread
From: Bruce Edge @ 2010-11-10 23:03 UTC (permalink / raw)
To: xen-devel, Jeremy Fitzhardinge, Konrad Rzeszutek Wilk
On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
>>
>> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
>>
>> The domU console output hung up prior to spitting out a prompt post root login. I logged into the domU via ssh and ran an 'ls' of an NFS mounted dir and
>> this immediately came out on the domU console:
>>
>> [ 480.467525] BUG: unable to handle kernel paging request at ffff880077e9f030
>> [ 480.467547] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
>> [ 480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
>> [ 480.467587] Oops: 0003 [#1] SMP
>> [ 480.467601] last sysfs file: /sys/kernel/uevent_seqnum
>> [ 480.467611] CPU 3
>> [ 480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront xen_blkfront
>> [ 480.467676]
>> [ 480.467684] Pid: 3957, comm: zsh Not tainted 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
>> [ 480.467695] RIP: e030:[<ffffffff810355f3>] [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
>> [ 480.467710] RSP: e02b:ffff880077ebfbd8 EFLAGS: 00010202
>> [ 480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX: 800000004d686167
>> [ 480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI: ffff8800763ddb80
>> [ 480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09: ffffea0001a3b2d8
>> [ 480.467748] R10: 0000000000000100 R11: 0000000000000000 R12: 00007f5cb26069d0
>> [ 480.467759] R13: 0000000000000001 R14: 0000000000000000 R15: ffff8800763ddb80
>> [ 480.467772] FS: 00007f5cb2606700(0000) GS:ffff88007ff94000(0000) knlGS:0000000000000000
>> [ 480.467783] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4: 0000000000002660
>> [ 480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [ 480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task ffff880078b80000)
>> [ 480.467832] Stack:
>> [ 480.467838] 0000000000000100 ffffea0001a3b2d8 7000000000000000 0000000000000000
>> [ 480.467862] ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8 ffffffff810e2200
>> [ 480.467886] ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8 0000000000000001
>> [ 480.467910] Call Trace:
>> [ 480.467921] [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
>> [ 480.467934] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
>> [ 480.467946] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
>> [ 480.467958] [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
>> [ 480.467972] [<ffffffff814463fe>] do_page_fault+0x374/0x396
>> [ 480.467984] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
>> [ 480.467996] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
>> [ 480.468009] [<ffffffff810072a2>] ? check_events+0x12/0x20
>> [ 480.468020] [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
>> [ 480.468034] [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
>> [ 480.468045] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
>> [ 480.468059] [<ffffffff810072a2>] ? check_events+0x12/0x20
>> [ 480.468072] [<ffffffff81443615>] page_fault+0x25/0x30
>> [ 480.468085] [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
>> [ 480.468096] [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
>> [ 480.468108] [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
>> [ 480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
>> [ 480.468323] RIP [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
>> [ 480.468323] RSP <ffff880077ebfbd8>
>> [ 480.468323] CR2: ffff880077e9f030
>> [ 480.468323] ---[ end trace 6d1623321ecd50ac ]---
>>
>> The domU config is attached as config.2.6.37_domU_debug
>>
>>
>> -Bruce
>
> Here's another attempt from the same config with the full domU boot logs:
>
> Started domain dpm (id=4)
> [ 0.000000] Initializing cgroup subsys cpuset
> [ 0.000000] Initializing cgroup subsys cpu
> [ 0.000000] Linux version
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
> 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
> [ 0.000000] Command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [ 0.000000] ACPI in unprivileged domain disabled
> [ 0.000000] released 0 pages of unused memory
> [ 0.000000] BIOS-provided physical RAM map:
> [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
> [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
> [ 0.000000] Xen: 0000000000100000 - 0000000080800000 (usable)
> [ 0.000000] NX (Execute Disable) protection: active
> [ 0.000000] DMI not present or invalid.
> [ 0.000000] No AGP bridge found
> [ 0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
> [ 0.000000] Scanning 0 areas for low memory corruption
> [ 0.000000] init_memory_mapping: 0000000000000000-0000000080800000
> [ 0.000000] RAMDISK: 01d95000 - 02fa9000
> [ 0.000000] No NUMA configuration found
> [ 0.000000] Faking a node at 0000000000000000-0000000080800000
> [ 0.000000] Initmem setup node 0 0000000000000000-0000000080800000
> [ 0.000000] NODE_DATA [000000007fffb000 - 000000007fffffff]
> [ 0.000000] Zone PFN ranges:
> [ 0.000000] DMA 0x00000010 -> 0x00001000
> [ 0.000000] DMA32 0x00001000 -> 0x00100000
> [ 0.000000] Normal empty
> [ 0.000000] Movable zone start PFN for each node
> [ 0.000000] early_node_map[2] active PFN ranges
> [ 0.000000] 0: 0x00000010 -> 0x000000a0
> [ 0.000000] 0: 0x00000100 -> 0x00080800
> [ 0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
> [ 0.000000] No local APIC present
> [ 0.000000] APIC: disable apic facility
> [ 0.000000] APIC: switched to apic NOOP
> [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
> [ 0.000000] Allocating PCI resources starting at 80800000 (gap:
> 80800000:7f800000)
> [ 0.000000] Booting paravirtualized kernel on Xen
> [ 0.000000] Xen version: 4.1-unstable (preserve-AD)
> [ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
> nr_cpu_ids:6 nr_node_ids:1
> [ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
> r8192 d23744 u114688
> [ 0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
> [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
> [ 0.000000] Built 1 zonelists in Node order, mobility grouping on.
> Total pages: 519022
> [ 0.000000] Policy zone: DMA32
> [ 0.000000] Kernel command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [ 0.000000] Placing 64MB software IO TLB between ffff880079400000 -
> ffff88007d400000
> [ 0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
> [ 0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
> 448k absent, 145000k reserved, 6327k data, 756k init)
> [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
> CPUs=6, Nodes=1
> [ 0.000000] Hierarchical RCU implementation.
> [ 0.000000] RCU-based detection of stalled CPUs is disabled.
> [ 0.000000] NR_IRQS:4352 nr_irqs:320 16
> [ 0.000000] Console: colour dummy device 80x25
> [ 0.000000] console [tty0] enabled
> [ 0.000000] console [hvc0] enabled
> [ 0.000000] installing Xen timer for CPU 0
> [ 0.000000] Detected 2660.064 MHz processor.
> [ 0.000999] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
> [ 0.000999] pid_max: default: 32768 minimum: 301
> [ 0.000999] Security Framework initialized
> [ 0.000999] SELinux: Initializing.
> [ 0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
> [ 0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
> [ 0.001635] Mount-cache hash table entries: 256
> [ 0.002195] Initializing cgroup subsys ns
> [ 0.002208] ns_cgroup deprecated: consider using the
> 'clone_children' flag without the ns_cgroup.
> [ 0.002220] Initializing cgroup subsys cpuacct
> [ 0.002231] Initializing cgroup subsys freezer
> [ 0.002356] CPU: Unsupported number of siblings 16
> [ 0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
> driver, software events only.
> [ 0.002483] SMP alternatives: switching to UP code
> [ 0.004119] cpu 0 spinlock event irq 318
> [ 0.004765] NMI watchdog failed to create perf event on cpu0:
> fffffffffffffffe
> [ 0.005050] installing Xen timer for CPU 1
> [ 0.005157] cpu 1 spinlock event irq 312
> [ 0.005194] SMP alternatives: switching to SMP code
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.006158] NMI watchdog failed to create perf event on cpu1:
> fffffffffffffffe
> [ 0.006474] installing Xen timer for CPU 2
> [ 0.006585] cpu 2 spinlock event irq 306
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.006816] NMI watchdog failed to create perf event on cpu2:
> fffffffffffffffe
> [ 0.006999] installing Xen timer for CPU 3
> [ 0.007037] cpu 3 spinlock event irq 300
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.007300] NMI watchdog failed to create perf event on cpu3:
> fffffffffffffffe
> [ 0.007534] installing Xen timer for CPU 4
> [ 0.007636] cpu 4 spinlock event irq 294
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.007874] NMI watchdog failed to create perf event on cpu4:
> fffffffffffffffe
> [ 0.008041] installing Xen timer for CPU 5
> [ 0.008083] cpu 5 spinlock event irq 288
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.008344] NMI watchdog failed to create perf event on cpu5:
> fffffffffffffffe
> [ 0.008356] Brought up 6 CPUs
> [ 0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
> [ 0.012051] Grant table initialized
> [ 0.030978] Time: 165:165:165 Date: 165/165/65
> [ 0.031229] NET: Registered protocol family 16
> [ 0.036672] PCI: setting up Xen PCI frontend stub
> [ 0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
> [ 0.074302] bio: create slab <bio-0> at 0
> [ 0.075385] ACPI: Interpreter disabled.
> [ 0.077056] xen_balloon: Initialising balloon driver.
> [ 0.079323] vgaarb: loaded
> [ 0.081270] usbcore: registered new interface driver usbfs
> [ 0.081283] usbcore: registered new interface driver hub
> [ 0.081283] usbcore: registered new device driver usb
> [ 0.083183] PCI: System does not support PCI
> [ 0.084007] PCI: System does not support PCI
> [ 0.085236] NetLabel: Initializing
> [ 0.085236] NetLabel: domain hash size = 128
> [ 0.085236] NetLabel: protocols = UNLABELED CIPSOv4
> [ 0.085313] NetLabel: unlabeled traffic allowed by default
> [ 0.086544] Switching to clocksource xen
> [ 0.106256] pnp: PnP ACPI: disabled
> [ 0.129799] NET: Registered protocol family 2
> [ 0.130174] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
> [ 0.131882] TCP established hash table entries: 262144 (order: 10,
> 4194304 bytes)
> [ 0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [ 0.133179] TCP: Hash tables configured (established 262144 bind 65536)
> [ 0.133191] TCP reno registered
> [ 0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
> [ 0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
> [ 0.133914] NET: Registered protocol family 1
> [ 0.134582] RPC: Registered udp transport module.
> [ 0.134590] RPC: Registered tcp transport module.
> [ 0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [ 0.134892] Trying to unpack rootfs image as initramfs...
> [ 0.178824] Freeing initrd memory: 18512k freed
> [ 0.268798] DMA-API: preallocated 32768 debug entries
> [ 0.268810] DMA-API: debugging enabled by kernel config
> [ 0.271181] platform rtc_cmos: registered platform RTC device (no
> PNP device found)
> [ 0.285410] Machine check injector initialized
> [ 0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.295590] microcode: Microcode Update Driver: v2.00
> <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> [ 0.295607] Scanning for low memory corruption every 60 seconds
> [ 0.298813] audit: initializing netlink socket (disabled)
> [ 0.298940] type=2000 audit(1289403082.795:1): initialized
> [ 0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [ 0.376985] VFS: Disk quotas dquot_6.5.2
> [ 0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [ 0.383121] NTFS driver 2.1.29 [Flags: R/W].
> [ 0.385014] msgmni has been set to 3864
> [ 0.389953] Block layer SCSI generic (bsg) driver version 0.4
> loaded (major 253)
> [ 0.389970] io scheduler noop registered
> [ 0.389979] io scheduler deadline registered
> [ 0.390666] io scheduler cfq registered (default)
> [ 0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [ 0.406238] pcifront pci-0: Installing PCI frontend
> [ 0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
> [ 0.731775] Non-volatile memory driver v1.3
> [ 0.731790] Linux agpgart interface v0.103
> [ 0.733731] [drm] Initialized drm 1.1.0 20060810
> [ 0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [ 0.827202] brd: module loaded
> [ 0.840788] loop: module loaded
> [ 0.841965] Fixed MDIO Bus: probed
> [ 0.844025] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> [ 0.844651] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> [ 0.845194] uhci_hcd: USB Universal Host Controller Interface driver
> [ 0.846985] usbcore: registered new interface driver usblp
> [ 0.847763] usbcore: registered new interface driver libusual
> [ 0.849433] PNP: No PS/2 controller found. Probing ports directly.
> [ 0.850279] i8042.c: No controller found.
> [ 0.850883] mice: PS/2 mouse device common for all mice
> [ 0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [ 0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
> [ 0.855928] cpuidle: using governor ladder
> [ 0.855941] cpuidle: using governor menu
> [ 0.856029] Netfilter messages via NETLINK v0.30.
> [ 0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
> [ 0.857888] ctnetlink v0.93: registering with nfnetlink.
> [ 0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
> [ 0.860677] TCP cubic registered
> [ 0.860687] Initializing XFRM netlink socket
> [ 0.863286] NET: Registered protocol family 10
> [ 0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
> [ 0.867612] IPv6 over IPv4 tunneling driver
> [ 0.871326] NET: Registered protocol family 17
> [ 0.871566] Registering the dns_resolver key type
> [ 0.873668] registered taskstats version 1
> [ 0.873912] kmemleak: Kernel memory leak detector initialized
> [ 0.873928] kmemleak: Automatic memory scanning thread started
> [ 0.875057] XENBUS: Device with no driver: device/vbd/51712
> [ 0.875057] XENBUS: Device with no driver: device/vif/0
> [ 0.875057] XENBUS: Device with no driver: device/console/0
> [ 0.875057] Magic number: 1:252:3141
> [ 13.388958] Freeing unused kernel memory: 756k freed
> [ 13.389190] Write protecting the kernel read-only data: 10240k
> [ 13.398774] Freeing unused kernel memory: 1724k freed
> [ 13.400663] Freeing unused kernel memory: 1992k freed
> [ 13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
> [ 13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
> use /proc/1111/oom_score_adj instead.
> Begin: Loading essential drivers... ...
> [ 13.559435] blkid used greatest stack depth: 5408 bytes left
> [ 13.568391] blkfront: xvda: barriers disabled
> [ 13.574762] blkid used greatest stack depth: 5264 bytes left
> [ 13.580813] xvda: xvda1 xvda2 < xvda5 >
> [ 13.645562] modprobe used greatest stack depth: 5088 bytes left
> [ 13.653160] Initialising Xen virtual ethernet driver.
> [ 13.659540] blkid used greatest stack depth: 4960 bytes left
> [ 13.735467] blkid used greatest stack depth: 4752 bytes left
> Done.
> Begin: Running /scripts/init-premount ...
> Done.
> Begin: Mounting root file system... ...
> Begin: Running /scripts/local-top ...
> Done.
> Begin: Running /scripts/local-premount ...
> Done.
> [ 13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
> [ 13.903799] EXT4-fs (xvda1): write access will be enabled during recovery
> [ 14.033719] EXT4-fs (xvda1): recovery complete
> [ 14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
> mode. Opts: (null)
> [ 14.035510] exe used greatest stack depth: 4112 bytes left
> Begin: Running /scripts/local-bottom ...
> Done.
> Done.
> Begin: Running /scripts/init-bottom ...
> Done.
> init: ureadahead main process (1214) terminated with status 5
> fsck from util-linux-ng 2.17.2
> e2fsck 1.41.11 (14-Mar-2010)
> /dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
> * Enable login on console [ OK ]
> * Check for hostname changes... [ OK ]
> ...done.
> * Setting sensors limits
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> [ OK ]
> * Initializing random number generator... [ OK ]
> * Setting up X server socket directory /tmp/.X11-unix... [ OK ]
> * Setting up ICE socket directory /tmp/.ICE-unix... [ OK ]
> * Not starting fancontrol; run pwmconfig first.
> * Starting internet superserver inetd [ OK ]
> Starting openntpd: ntpd.
> * Starting network management services:
> * Setting core file location... [ OK ]
> * Restoring resolver state... [ OK ]
> * Checking tomcat configuration
> ...done.
> * Starting Tomcat servlet engine tomcat6
> Using CATALINA_BASE: /var/lib/tomcat6
> Using CATALINA_HOME: /usr/share/tomcat6
> Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
> Using JRE_HOME: /usr/lib/jvm/java-6-openjdk
> Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar
> [ OK ]
> collectd has been disabled - see /etc/default/collectd.
> * Recording successful boot for GRUB
> ...done.
> * Running local boot scripts (/etc/rc.local) [ OK ]
> mountall: Plymouth command failed
> mountall: Disconnected from Plymouth
>
> 6.0.16-01
> kaan-04-dpm login: root
> Password:
> [ 42.646538] ------------[ cut here ]------------
> [ 42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
> [ 42.646566] invalid opcode: 0000 [#1] SMP
> [ 42.646582] last sysfs file: /sys/kernel/uevent_seqnum
> [ 42.646592] CPU 0
> [ 42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
> xen_netfront xen_blkfront
> [ 42.646665]
> [ 42.646674] Pid: 3916, comm: lsb_release Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [ 42.646688] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
> pin_pagetable_pfn+0x31/0x37
> [ 42.646711] RSP: e02b:ffff88007595bce8 EFLAGS: 00010282
> [ 42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX: 0000000000000001
> [ 42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
> [ 42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09: ffff880000000000
> [ 42.646756] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
> [ 42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15: ffff880075161d40
> [ 42.646784] FS: 00007fee63b01700(0000) GS:ffff88007ff40000(0000)
> knlGS:0000000000000000
> [ 42.646797] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4: 0000000000002660
> [ 42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 42.646844] Process lsb_release (pid: 3916, threadinfo
> ffff88007595a000, task ffff880075161d40)
> [ 42.646857] Stack:
> [ 42.646864] ffff880000000000 00000000000484f0 ffff8800777c6d80
> 000000000007751e
> [ 42.646892] ffff88007595bd28 ffffffff810066a8 000000000007751e
> ffff8800777c6d80
> [ 42.646919] ffff88007595bd38 ffffffff810066db ffff88007595bd78
> ffffffff810e048f
> [ 42.646946] Call Trace:
> [ 42.646959] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
> [ 42.646973] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
> [ 42.646989] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
> [ 42.647003] [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
> [ 42.647017] [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
> [ 42.647033] [<ffffffff8100eb84>] ? arch_get_unmapped_area_topdown+0x4a/0x28a
> [ 42.647052] [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [ 42.647059] [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
> [ 42.647059] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [ 42.647059] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [ 42.647059] [<ffffffff81443615>] page_fault+0x25/0x30
> [ 42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
> e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
> 85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48
> 8b 3d
> [ 42.647059] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
> [ 42.647059] RSP <ffff88007595bce8>
> [ 42.647059] ---[ end trace 3fbbffee414a5166 ]---
>
Once more, this time with CONFIG_XEN_PVHVM=n just in case it was
related to Stefano's PVonHVM work.
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version
2.6.37-rc1-kernel.org-domu-6.0.nopvhvm.bedge-debug (root@opal) (gcc
version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Wed Nov 10 12:01:55 PST
2010
[ 0.000000] Command line:
root=UUID=e4e223aa-080e-421b-98ac-ab0b3e0e57e5 ro swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] released 0 pages of unused memory
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
[ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
[ 0.000000] Xen: 0000000000100000 - 0000000080800000 (usable)
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] DMI not present or invalid.
[ 0.000000] No AGP bridge found
[ 0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[ 0.000000] Scanning 0 areas for low memory corruption
[ 0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[ 0.000000] RAMDISK: 01d95000 - 02faa000
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at 0000000000000000-0000000080800000
[ 0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[ 0.000000] NODE_DATA [000000007fffb000 - 000000007fffffff]
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal empty
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[2] active PFN ranges
[ 0.000000] 0: 0x00000010 -> 0x000000a0
[ 0.000000] 0: 0x00000100 -> 0x00080800
[ 0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[ 0.000000] No local APIC present
[ 0.000000] APIC: disable apic facility
[ 0.000000] APIC: switched to apic NOOP
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.1-unstable (preserve-AD)
[ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[ 0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[ 0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line:
root=UUID=e4e223aa-080e-421b-98ac-ab0b3e0e57e5 ro swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[ 0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[ 0.000000] Memory: 1959892k/2105344k available (4398k kernel code,
448k absent, 145004k reserved, 6327k data, 756k init)
[ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] RCU-based detection of stalled CPUs is disabled.
[ 0.000000] NR_IRQS:4352 nr_irqs:320 16
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] console [hvc0] enabled
[ 0.000000] installing Xen timer for CPU 0
[ 0.000000] Detected 2660.064 MHz processor.
[ 0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
[ 0.000999] pid_max: default: 32768 minimum: 301
[ 0.000999] Security Framework initialized
[ 0.000999] SELinux: Initializing.
[ 0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[ 0.001365] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[ 0.001631] Mount-cache hash table entries: 256
[ 0.002177] Initializing cgroup subsys ns
[ 0.002191] ns_cgroup deprecated: consider using the
'clone_children' flag without the ns_cgroup.
[ 0.002203] Initializing cgroup subsys cpuacct
[ 0.002214] Initializing cgroup subsys freezer
[ 0.002342] CPU: Unsupported number of siblings 16
[ 0.002348] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[ 0.004120] cpu 0 spinlock event irq 318
[ 0.004755] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[ 0.004999] installing Xen timer for CPU 1
[ 0.005056] cpu 1 spinlock event irq 312
[ 0.005093] SMP alternatives: switching to SMP code
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.006155] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[ 0.006471] installing Xen timer for CPU 2
[ 0.006582] cpu 2 spinlock event irq 306
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.006810] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[ 0.006999] installing Xen timer for CPU 3
[ 0.007033] cpu 3 spinlock event irq 300
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.007353] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[ 0.007587] installing Xen timer for CPU 4
[ 0.007626] cpu 4 spinlock event irq 294
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.007854] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[ 0.008032] installing Xen timer for CPU 5
[ 0.008072] cpu 5 spinlock event irq 288
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.008391] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[ 0.008402] Brought up 6 CPUs
[ 0.009156] kworker/u:0 used greatest stack depth: 5696 bytes left
[ 0.011444] Grant table initialized
[ 0.030860] Time: 165:165:165 Date: 165/165/65
[ 0.031124] NET: Registered protocol family 16
[ 0.036023] PCI: setting up Xen PCI frontend stub
[ 0.037149] kworker/u:0 used greatest stack depth: 5528 bytes left
[ 0.063224] bio: create slab <bio-0> at 0
[ 0.064258] ACPI: Interpreter disabled.
[ 0.065066] xen_balloon: Initialising balloon driver.
[ 0.066380] vgaarb: loaded
[ 0.067219] usbcore: registered new interface driver usbfs
[ 0.068076] usbcore: registered new interface driver hub
[ 0.069160] PCI: System does not support PCI
[ 0.070062] PCI: System does not support PCI
[ 0.071229] NetLabel: Initializing
[ 0.071229] NetLabel: domain hash size = 128
[ 0.071229] NetLabel: protocols = UNLABELED CIPSOv4
[ 0.071271] NetLabel: unlabeled traffic allowed by default
[ 0.072589] Switching to clocksource xen
[ 0.086952] pnp: PnP ACPI: disabled
[ 0.110928] NET: Registered protocol family 2
[ 0.111286] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.112870] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[ 0.113959] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 0.114223] TCP: Hash tables configured (established 262144 bind 65536)
[ 0.114235] TCP reno registered
[ 0.114348] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[ 0.114454] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[ 0.115033] NET: Registered protocol family 1
[ 0.139648] RPC: Registered udp transport module.
[ 0.139656] RPC: Registered tcp transport module.
[ 0.139662] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.139898] Trying to unpack rootfs image as initramfs...
[ 0.179608] Freeing initrd memory: 18516k freed
[ 0.267847] DMA-API: preallocated 32768 debug entries
[ 0.267860] DMA-API: debugging enabled by kernel config
[ 0.270280] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[ 0.284949] Machine check injector initialized
[ 0.294390] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294463] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294516] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294584] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294680] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294768] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[ 0.295370] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[ 0.295387] Scanning for low memory corruption every 60 seconds
[ 0.297945] audit: initializing netlink socket (disabled)
[ 0.298034] type=2000 audit(1289404539.466:1): initialized
[ 0.371706] VFS: Disk quotas dquot_6.5.2
[ 0.372818] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.378043] NTFS driver 2.1.29 [Flags: R/W].
[ 0.379946] msgmni has been set to 3864
[ 0.384845] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[ 0.384862] io scheduler noop registered
[ 0.384871] io scheduler deadline registered
[ 0.385499] io scheduler cfq registered (default)
[ 0.387281] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.401306] pcifront pci-0: Installing PCI frontend
[ 0.402048] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[ 0.411423] pcifront pci-0: claiming resource 0000:00:00.0/0
[ 0.411423] pcifront pci-0: claiming resource 0000:00:00.0/2
[ 0.411423] pcifront pci-0: claiming resource 0000:00:00.0/3
[ 0.411423] pcifront pci-0: claiming resource 0000:00:00.0/4
[ 0.411423] pcifront pci-0: claiming resource 0000:00:01.0/0
[ 0.411423] pcifront pci-0: claiming resource 0000:00:01.0/2
[ 0.411423] pcifront pci-0: claiming resource 0000:00:01.0/3
[ 0.411423] pcifront pci-0: claiming resource 0000:00:01.0/4
[ 0.411423] pcifront pci-0: claiming resource 0000:00:02.0/0
[ 0.411423] pcifront pci-0: claiming resource 0000:00:02.0/2
[ 0.411423] pcifront pci-0: claiming resource 0000:00:02.0/3
[ 0.411423] pcifront pci-0: claiming resource 0000:00:02.0/4
[ 0.411423] pcifront pci-0: claiming resource 0000:00:03.0/0
[ 0.411423] pcifront pci-0: claiming resource 0000:00:03.0/2
[ 0.411423] pcifront pci-0: claiming resource 0000:00:03.0/3
[ 0.411423] pcifront pci-0: claiming resource 0000:00:03.0/4
[ 0.718875] Non-volatile memory driver v1.3
[ 0.718890] Linux agpgart interface v0.103
[ 0.720286] [drm] Initialized drm 1.1.0 20060810
[ 0.720307] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.816097] brd: module loaded
[ 0.829618] loop: module loaded
[ 0.830819] Fixed MDIO Bus: probed
[ 0.833373] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.833977] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.836341] usbcore: registered new interface driver usblp
[ 0.836850] usbcore: registered new interface driver libusual
[ 0.838519] PNP: No PS/2 controller found. Probing ports directly.
[ 0.839365] i8042.c: No controller found.
[ 0.840021] mice: PS/2 mouse device common for all mice
[ 0.842821] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[ 0.843646] rtc_cmos: probe of rtc_cmos failed with error -38
[ 0.845545] cpuidle: using governor ladder
[ 0.845558] cpuidle: using governor menu
[ 0.845645] Netfilter messages via NETLINK v0.30.
[ 0.845732] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 0.847527] ctnetlink v0.93: registering with nfnetlink.
[ 0.849858] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 0.850277] TCP cubic registered
[ 0.850288] Initializing XFRM netlink socket
[ 0.852995] NET: Registered protocol family 10
[ 0.856904] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 0.857303] IPv6 over IPv4 tunneling driver
[ 0.861148] NET: Registered protocol family 17
[ 0.861376] Registering the dns_resolver key type
[ 0.863749] registered taskstats version 1
[ 0.864020] kmemleak: Kernel memory leak detector initialized
[ 0.864035] kmemleak: Automatic memory scanning thread started
[ 0.866079] XENBUS: Device with no driver: device/vbd/51712
[ 0.866079] XENBUS: Device with no driver: device/vif/0
[ 0.866079] XENBUS: Device with no driver: device/console/0
[ 0.866079] Magic number: 1:252:3141
[ 13.380040] Freeing unused kernel memory: 756k freed
[ 13.380152] Write protecting the kernel read-only data: 10240k
[ 13.390107] Freeing unused kernel memory: 1724k freed
[ 13.391580] Freeing unused kernel memory: 1992k freed
Loading, please wait...
[ 13.484058] exe used greatest stack depth: 5360 bytes left
[ 13.510024] udevd (1111): /proc/1111/oom_adj is deprecated, please
use /proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[ 13.533300] blkid used greatest stack depth: 5088 bytes left
[ 13.564959] blkfront: xvda: barriers disabled
[ 13.576723] xvda: xvda1 xvda2 < xvda5 >
[ 13.639471] blkid used greatest stack depth: 4752 bytes left
[ 13.662716] Initialising Xen virtual ethernet driver.
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[ 13.927847] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
[ 14.608840] hostname used greatest stack depth: 4464 bytes left
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
Superblock last write time is in the future.
(by less than a day, probably due to the hardware clock being
incorrectly set). Fix? yes
/dev/xvda1: clean, 55961/288576 files, 289520/1153280 blocks
* Enable login on console [ OK ]
* Check for hostname changes... [ OK ]
...done.
* Setting sensors limits
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
[ OK ]
* Initializing random number generator... [ OK ]
* Setting up X server socket directory /tmp/.X11-unix... [ OK ]
* Not starting fancontrol; run pwmconfig first.
* Starting internet superserver inetd [ OK ]
Starting openntpd: ntpd.
* Starting network management services:
* Starting the system activity data collector sadc
Cannot open /var/log/sysstat/sa10: No such file or directory
[fail]
* Setting core file location... [ OK ]
* Restoring resolver state... [ OK ]
* Checking tomcat configuration
...done.
* Starting Tomcat servlet engine tomcat6
Using CATALINA_BASE: /var/lib/tomcat6
Using CATALINA_HOME: /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME: /usr/lib/jvm/java-6-openjdk
Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar
[ OK ]
collectd has been disabled - see /etc/default/collectd.
* Recording successful boot for GRUB
...done.
* Running local boot scripts (/etc/rc.local) [ OK ]
mountall: Plymouth command failed
mountall: Disconnected from Plymouth
6.0.16-01
kaan-04-dpm login: [ 144.548118] BUG: unable to handle kernel paging
request at ffff88007733d508
[ 144.548145] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[ 144.548167] PGD 1a04067 PUD 3dd7067 PMD 3f91067 PTE 801000007733d065
[ 144.548194] Oops: 0003 [#1] SMP
[ 144.548211] last sysfs file: /sys/kernel/uevent_seqnum
[ 144.548222] CPU 3
[ 144.548228] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[ 144.548345]
[ 144.548354] Pid: 3938, comm: sh Not tainted
2.6.37-rc1-kernel.org-domu-6.0.nopvhvm.bedge-debug #1 /
[ 144.548367] RIP: e030:[<ffffffff810355f3>] [<ffffffff810355f3>]
ptep_set_access_flags+0x27/0x4d
[ 144.548395] RAX: 800000005adc6167 RBX: ffff880077403b80 RCX: 800000005adc6167
[ 144.548408] RDX: ffff88007733d508 RSI: 00007f52324a19d0 RDI: ffff880077403b80
[ 144.548420] RBP: ffff880077cabc08 R08: 0000000000000001 R09: ffffea0001a13568
[ 144.548431] R10: 0000000000000100 R11: ffffffff00000001 R12: 00007f52324a19d0
[ 144.548443] R13: 0000000000000001 R14: 0000000000000000 R15: ffff880077403b80
[ 144.548495] FS: 00007f52324a1700(0000) GS:ffff88007ff94000(0000)
knlGS:0000000000000000
[ 144.548509] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 144.548520] CR2: ffff88007733d508 CR3: 00000000783ad000 CR4: 0000000000002660
[ 144.548531] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 144.548543] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 144.548556] Process sh (pid: 3938, threadinfo ffff880077caa000,
task ffff880077a615f0)
[ 144.548568] Stack:
[ 144.548575] 0000000000000100 ffffea0001a13568 f000000000000000
0000000000000000
[ 144.548602] ffff88007733d508 ffffea0001a13568 ffff880077cabca8
ffffffff810e2200
[ 144.548630] ffff880077cabc38 ffffffff81007c09 ffffea0001a13568
0000000000000001
[ 144.548657] Call Trace:
[ 144.548672] [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
[ 144.548688] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[ 144.548702] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[ 144.548716] [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
[ 144.548732] [<ffffffff814463fe>] do_page_fault+0x374/0x396
[ 144.548747] [<ffffffff8100ba0b>] ? xen_hypervisor_callback+0x1b/0x20
[ 144.548762] [<ffffffff8144384a>] ? error_exit+0x2a/0x60
[ 144.548776] [<ffffffff8144339d>] ? retint_restore_args+0x5/0x6
[ 144.548792] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1006
[ 144.548806] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1006
[ 144.548821] [<ffffffff81443615>] page_fault+0x25/0x30
[ 144.548837] [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
[ 144.548851] [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
[ 144.548865] [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
[ 144.548875] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49
89 f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45
85 ed 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df
e8 b2
[ 144.549091] RIP [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
[ 144.549091] RSP <ffff880077cabbd8>
[ 144.549091] CR2: ffff88007733d508
[ 144.549091] ---[ end trace e2156ffb7bd19b0d ]---
This time it also generated a stack dump on dom0
BUG: scheduling while atomic: xenwatch/12/0x00000002
Modules linked in: xt_physdev ipmi_msghandler ipv6 osa_mfgdom0
xen_evtchn tun bridge stp serio_raw joydev llc ioatdma dca usb_storage
e1000e
Pid: 12, comm: xenwatch Not tainted
2.6.32.24-pvops-stable-dom0-5.3.bedge-debug #1
Call Trace:
[<ffffffff810424d7>] __schedule_bug+0x5c/0x61
[<ffffffff814964ac>] schedule+0xea/0x84d
[<ffffffff8100f5dd>] ? xen_force_evtchn_callback+0xd/0xf
[<ffffffff8100fdff>] ? xen_restore_fl_direct_end+0x0/0x1
[<ffffffff814989e5>] ? _spin_unlock_irqrestore+0x1d/0x40
[<ffffffff812b76bb>] read_reply+0x9c/0x13a
[<ffffffff8105ea57>] ? autoremove_wake_function+0x0/0x34
[<ffffffff812b78bf>] xs_talkv+0xab/0x16d
[<ffffffff81010be2>] ? xen_spin_lock+0xc7/0xda
[<ffffffff812b7c9a>] unregister_xenbus_watch+0xdb/0x1e7
[<ffffffff812bd303>] pciback_xenbus_remove+0x33/0x75
[<ffffffff812b8bba>] xenbus_dev_remove+0x44/0x5b
[<ffffffff812fead4>] __device_release_driver+0x70/0xc3
[<ffffffff812febf5>] device_release_driver+0x1e/0x2b
[<ffffffff812fe17c>] bus_remove_device+0x96/0xa8
[<ffffffff812fc8ca>] device_del+0x135/0x1a2
[<ffffffff812fc948>] device_unregister+0x11/0x1e
[<ffffffff812b89c8>] xenbus_dev_changed+0x92/0x186
[<ffffffff812b8f0a>] backend_changed+0x16/0x18
[<ffffffff812b75e4>] xenwatch_thread+0x123/0x15e
[<ffffffff8105ea57>] ? autoremove_wake_function+0x0/0x34
[<ffffffff812b74c1>] ? xenwatch_thread+0x0/0x15e
[<ffffffff8105e7c5>] kthread+0x7a/0x82
[<ffffffff81014bda>] child_rip+0xa/0x20
[<ffffffff81013d61>] ? int_ret_from_sys_call+0x7/0x1b
[<ffffffff81014521>] ? retint_restore_args+0x5/0x6
[<ffffffff81014bd0>] ? child_rip+0x0/0x20
pciback 0000:01:00.0: free-ing dynamically allocated virtual
configuration space fields
pciback 0000:01:00.0: resetting virtual configuration space
pciback 0000:01:00.1: free-ing dynamically allocated virtual
configuration space fields
pciback 0000:01:00.1: resetting virtual configuration space
pciback 0000:04:00.0: free-ing dynamically allocated virtual
configuration space fields
pciback 0000:04:00.0: resetting virtual configuration space
pciback 0000:04:00.1: free-ing dynamically allocated virtual
configuration space fields
pciback 0000:04:00.1: resetting virtual configuration space
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-10 23:03 ` Bruce Edge
@ 2010-11-11 15:56 ` Konrad Rzeszutek Wilk
2010-11-11 16:32 ` Boris Derzhavets
0 siblings, 1 reply; 45+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-11-11 15:56 UTC (permalink / raw)
To: Bruce Edge; +Cc: Jeremy Fitzhardinge, xen-devel
On Wed, Nov 10, 2010 at 03:03:34PM -0800, Bruce Edge wrote:
> On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> > On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> >>
> >> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> >>
> >> The domU console output hung up prior to spitting out a prompt post root login. I logged into the domU via ssh and ran an 'ls' of an NFS mounted dir and
> >> this immediately came out on the domU console:
So... does this show if you don't do PCI passthrough? Just as a normal PV guests with
NFS mount?
I am not going to look at this yet - debugging a bootup issue with the 2.6.37-rc1 having
a fit on AMD CPUs.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 15:56 ` Konrad Rzeszutek Wilk
@ 2010-11-11 16:32 ` Boris Derzhavets
0 siblings, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-11 16:32 UTC (permalink / raw)
To: Bruce Edge, Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 1426 bytes --]
> So... does this show if you don't do PCI passthrough?
> Just as a normal PV guests with NFS mount?
Yes.
Boris.
--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Bruce Edge" <bruce.edge@gmail.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com
Date: Thursday, November 11, 2010, 10:56 AM
On Wed, Nov 10, 2010 at 03:03:34PM -0800, Bruce Edge wrote:
> On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> > On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> >>
> >> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> >>
> >> The domU console output hung up prior to spitting out a prompt post root login. I logged into the domU via ssh and ran an 'ls' of an NFS mounted dir and
> >> this immediately came out on the domU console:
So... does this show if you don't do PCI passthrough? Just as a normal PV guests with
NFS mount?
I am not going to look at this yet - debugging a bootup issue with the 2.6.37-rc1 having
a fit on AMD CPUs.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 2252 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-10 22:30 ` Bruce Edge
2010-11-10 23:03 ` Bruce Edge
@ 2010-11-11 6:13 ` Bruce Edge
2010-11-11 12:01 ` Boris Derzhavets
2010-11-11 15:08 ` Bruce Edge
3 siblings, 0 replies; 45+ messages in thread
From: Bruce Edge @ 2010-11-11 6:13 UTC (permalink / raw)
To: xen-devel, Jeremy Fitzhardinge, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 24403 bytes --]
On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> >
> > This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
> 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> >
> > The domU console output hung up prior to spitting out a prompt post root
> login. I logged into the domU via ssh and ran an 'ls' of an NFS mounted dir
> and
> > this immediately came out on the domU console:
> >
> > [ 480.467525] BUG: unable to handle kernel paging request at
> ffff880077e9f030
> > [ 480.467547] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> > [ 480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> > [ 480.467587] Oops: 0003 [#1] SMP
> > [ 480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> > [ 480.467611] CPU 3
> > [ 480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
> xen_blkfront
> > [ 480.467676]
> > [ 480.467684] Pid: 3957, comm: zsh Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> > [ 480.467695] RIP: e030:[<ffffffff810355f3>] [<ffffffff810355f3>]
> ptep_set_access_flags+0x27/0x4d
> > [ 480.467710] RSP: e02b:ffff880077ebfbd8 EFLAGS: 00010202
> > [ 480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
> 800000004d686167
> > [ 480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
> ffff8800763ddb80
> > [ 480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
> ffffea0001a3b2d8
> > [ 480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
> 00007f5cb26069d0
> > [ 480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
> ffff8800763ddb80
> > [ 480.467772] FS: 00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
> knlGS:0000000000000000
> > [ 480.467783] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> > [ 480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
> 0000000000002660
> > [ 480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> > [ 480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> > [ 480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
> ffff880078b80000)
> > [ 480.467832] Stack:
> > [ 480.467838] 0000000000000100 ffffea0001a3b2d8 7000000000000000
> 0000000000000000
> > [ 480.467862] ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
> ffffffff810e2200
> > [ 480.467886] ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
> 0000000000000001
> > [ 480.467910] Call Trace:
> > [ 480.467921] [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> > [ 480.467934] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> > [ 480.467946] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> > [ 480.467958] [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> > [ 480.467972] [<ffffffff814463fe>] do_page_fault+0x374/0x396
> > [ 480.467984] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> > [ 480.467996] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> > [ 480.468009] [<ffffffff810072a2>] ? check_events+0x12/0x20
> > [ 480.468020] [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
> > [ 480.468034] [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> > [ 480.468045] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> > [ 480.468059] [<ffffffff810072a2>] ? check_events+0x12/0x20
> > [ 480.468072] [<ffffffff81443615>] page_fault+0x25/0x30
> > [ 480.468085] [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> > [ 480.468096] [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> > [ 480.468108] [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> > [ 480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89
> f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74
> 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> > [ 480.468323] RIP [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> > [ 480.468323] RSP <ffff880077ebfbd8>
> > [ 480.468323] CR2: ffff880077e9f030
> > [ 480.468323] ---[ end trace 6d1623321ecd50ac ]---
> >
> > The domU config is attached as config.2.6.37_domU_debug
> >
> >
> > -Bruce
>
> Here's another attempt from the same config with the full domU boot logs:
>
> Started domain dpm (id=4)
> [ 0.000000] Initializing cgroup subsys cpuset
> [ 0.000000] Initializing cgroup subsys cpu
> [ 0.000000] Linux version
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
> 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
> [ 0.000000] Command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [ 0.000000] ACPI in unprivileged domain disabled
> [ 0.000000] released 0 pages of unused memory
> [ 0.000000] BIOS-provided physical RAM map:
> [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
> [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
> [ 0.000000] Xen: 0000000000100000 - 0000000080800000 (usable)
> [ 0.000000] NX (Execute Disable) protection: active
> [ 0.000000] DMI not present or invalid.
> [ 0.000000] No AGP bridge found
> [ 0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
> [ 0.000000] Scanning 0 areas for low memory corruption
> [ 0.000000] init_memory_mapping: 0000000000000000-0000000080800000
> [ 0.000000] RAMDISK: 01d95000 - 02fa9000
> [ 0.000000] No NUMA configuration found
> [ 0.000000] Faking a node at 0000000000000000-0000000080800000
> [ 0.000000] Initmem setup node 0 0000000000000000-0000000080800000
> [ 0.000000] NODE_DATA [000000007fffb000 - 000000007fffffff]
> [ 0.000000] Zone PFN ranges:
> [ 0.000000] DMA 0x00000010 -> 0x00001000
> [ 0.000000] DMA32 0x00001000 -> 0x00100000
> [ 0.000000] Normal empty
> [ 0.000000] Movable zone start PFN for each node
> [ 0.000000] early_node_map[2] active PFN ranges
> [ 0.000000] 0: 0x00000010 -> 0x000000a0
> [ 0.000000] 0: 0x00000100 -> 0x00080800
> [ 0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
> [ 0.000000] No local APIC present
> [ 0.000000] APIC: disable apic facility
> [ 0.000000] APIC: switched to apic NOOP
> [ 0.000000] PM: Registered nosave memory: 00000000000a0000 -
> 0000000000100000
> [ 0.000000] Allocating PCI resources starting at 80800000 (gap:
> 80800000:7f800000)
> [ 0.000000] Booting paravirtualized kernel on Xen
> [ 0.000000] Xen version: 4.1-unstable (preserve-AD)
> [ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
> nr_cpu_ids:6 nr_node_ids:1
> [ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
> r8192 d23744 u114688
> [ 0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
> [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
> [ 0.000000] Built 1 zonelists in Node order, mobility grouping on.
> Total pages: 519022
> [ 0.000000] Policy zone: DMA32
> [ 0.000000] Kernel command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [ 0.000000] Placing 64MB software IO TLB between ffff880079400000 -
> ffff88007d400000
> [ 0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
> [ 0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
> 448k absent, 145000k reserved, 6327k data, 756k init)
> [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
> CPUs=6, Nodes=1
> [ 0.000000] Hierarchical RCU implementation.
> [ 0.000000] RCU-based detection of stalled CPUs is disabled.
> [ 0.000000] NR_IRQS:4352 nr_irqs:320 16
> [ 0.000000] Console: colour dummy device 80x25
> [ 0.000000] console [tty0] enabled
> [ 0.000000] console [hvc0] enabled
> [ 0.000000] installing Xen timer for CPU 0
> [ 0.000000] Detected 2660.064 MHz processor.
> [ 0.000999] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
> [ 0.000999] pid_max: default: 32768 minimum: 301
> [ 0.000999] Security Framework initialized
> [ 0.000999] SELinux: Initializing.
> [ 0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152
> bytes)
> [ 0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576
> bytes)
> [ 0.001635] Mount-cache hash table entries: 256
> [ 0.002195] Initializing cgroup subsys ns
> [ 0.002208] ns_cgroup deprecated: consider using the
> 'clone_children' flag without the ns_cgroup.
> [ 0.002220] Initializing cgroup subsys cpuacct
> [ 0.002231] Initializing cgroup subsys freezer
> [ 0.002356] CPU: Unsupported number of siblings 16
> [ 0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
> driver, software events only.
> [ 0.002483] SMP alternatives: switching to UP code
> [ 0.004119] cpu 0 spinlock event irq 318
> [ 0.004765] NMI watchdog failed to create perf event on cpu0:
> fffffffffffffffe
> [ 0.005050] installing Xen timer for CPU 1
> [ 0.005157] cpu 1 spinlock event irq 312
> [ 0.005194] SMP alternatives: switching to SMP code
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.006158] NMI watchdog failed to create perf event on cpu1:
> fffffffffffffffe
> [ 0.006474] installing Xen timer for CPU 2
> [ 0.006585] cpu 2 spinlock event irq 306
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.006816] NMI watchdog failed to create perf event on cpu2:
> fffffffffffffffe
> [ 0.006999] installing Xen timer for CPU 3
> [ 0.007037] cpu 3 spinlock event irq 300
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.007300] NMI watchdog failed to create perf event on cpu3:
> fffffffffffffffe
> [ 0.007534] installing Xen timer for CPU 4
> [ 0.007636] cpu 4 spinlock event irq 294
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.007874] NMI watchdog failed to create perf event on cpu4:
> fffffffffffffffe
> [ 0.008041] installing Xen timer for CPU 5
> [ 0.008083] cpu 5 spinlock event irq 288
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.008344] NMI watchdog failed to create perf event on cpu5:
> fffffffffffffffe
> [ 0.008356] Brought up 6 CPUs
> [ 0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
> [ 0.012051] Grant table initialized
> [ 0.030978] Time: 165:165:165 Date: 165/165/65
> [ 0.031229] NET: Registered protocol family 16
> [ 0.036672] PCI: setting up Xen PCI frontend stub
> [ 0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
> [ 0.074302] bio: create slab <bio-0> at 0
> [ 0.075385] ACPI: Interpreter disabled.
> [ 0.077056] xen_balloon: Initialising balloon driver.
> [ 0.079323] vgaarb: loaded
> [ 0.081270] usbcore: registered new interface driver usbfs
> [ 0.081283] usbcore: registered new interface driver hub
> [ 0.081283] usbcore: registered new device driver usb
> [ 0.083183] PCI: System does not support PCI
> [ 0.084007] PCI: System does not support PCI
> [ 0.085236] NetLabel: Initializing
> [ 0.085236] NetLabel: domain hash size = 128
> [ 0.085236] NetLabel: protocols = UNLABELED CIPSOv4
> [ 0.085313] NetLabel: unlabeled traffic allowed by default
> [ 0.086544] Switching to clocksource xen
> [ 0.106256] pnp: PnP ACPI: disabled
> [ 0.129799] NET: Registered protocol family 2
> [ 0.130174] IP route cache hash table entries: 65536 (order: 7, 524288
> bytes)
> [ 0.131882] TCP established hash table entries: 262144 (order: 10,
> 4194304 bytes)
> [ 0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [ 0.133179] TCP: Hash tables configured (established 262144 bind 65536)
> [ 0.133191] TCP reno registered
> [ 0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
> [ 0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
> [ 0.133914] NET: Registered protocol family 1
> [ 0.134582] RPC: Registered udp transport module.
> [ 0.134590] RPC: Registered tcp transport module.
> [ 0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [ 0.134892] Trying to unpack rootfs image as initramfs...
> [ 0.178824] Freeing initrd memory: 18512k freed
> [ 0.268798] DMA-API: preallocated 32768 debug entries
> [ 0.268810] DMA-API: debugging enabled by kernel config
> [ 0.271181] platform rtc_cmos: registered platform RTC device (no
> PNP device found)
> [ 0.285410] Machine check injector initialized
> [ 0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.295590] microcode: Microcode Update Driver: v2.00
> <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> [ 0.295607] Scanning for low memory corruption every 60 seconds
> [ 0.298813] audit: initializing netlink socket (disabled)
> [ 0.298940] type=2000 audit(1289403082.795:1): initialized
> [ 0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [ 0.376985] VFS: Disk quotas dquot_6.5.2
> [ 0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [ 0.383121] NTFS driver 2.1.29 [Flags: R/W].
> [ 0.385014] msgmni has been set to 3864
> [ 0.389953] Block layer SCSI generic (bsg) driver version 0.4
> loaded (major 253)
> [ 0.389970] io scheduler noop registered
> [ 0.389979] io scheduler deadline registered
> [ 0.390666] io scheduler cfq registered (default)
> [ 0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [ 0.406238] pcifront pci-0: Installing PCI frontend
> [ 0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
> [ 0.731775] Non-volatile memory driver v1.3
> [ 0.731790] Linux agpgart interface v0.103
> [ 0.733731] [drm] Initialized drm 1.1.0 20060810
> [ 0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [ 0.827202] brd: module loaded
> [ 0.840788] loop: module loaded
> [ 0.841965] Fixed MDIO Bus: probed
> [ 0.844025] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> [ 0.844651] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> [ 0.845194] uhci_hcd: USB Universal Host Controller Interface driver
> [ 0.846985] usbcore: registered new interface driver usblp
> [ 0.847763] usbcore: registered new interface driver libusual
> [ 0.849433] PNP: No PS/2 controller found. Probing ports directly.
> [ 0.850279] i8042.c: No controller found.
> [ 0.850883] mice: PS/2 mouse device common for all mice
> [ 0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [ 0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
> [ 0.855928] cpuidle: using governor ladder
> [ 0.855941] cpuidle: using governor menu
> [ 0.856029] Netfilter messages via NETLINK v0.30.
> [ 0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
> [ 0.857888] ctnetlink v0.93: registering with nfnetlink.
> [ 0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
> [ 0.860677] TCP cubic registered
> [ 0.860687] Initializing XFRM netlink socket
> [ 0.863286] NET: Registered protocol family 10
> [ 0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
> [ 0.867612] IPv6 over IPv4 tunneling driver
> [ 0.871326] NET: Registered protocol family 17
> [ 0.871566] Registering the dns_resolver key type
> [ 0.873668] registered taskstats version 1
> [ 0.873912] kmemleak: Kernel memory leak detector initialized
> [ 0.873928] kmemleak: Automatic memory scanning thread started
> [ 0.875057] XENBUS: Device with no driver: device/vbd/51712
> [ 0.875057] XENBUS: Device with no driver: device/vif/0
> [ 0.875057] XENBUS: Device with no driver: device/console/0
> [ 0.875057] Magic number: 1:252:3141
> [ 13.388958] Freeing unused kernel memory: 756k freed
> [ 13.389190] Write protecting the kernel read-only data: 10240k
> [ 13.398774] Freeing unused kernel memory: 1724k freed
> [ 13.400663] Freeing unused kernel memory: 1992k freed
> [ 13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
> [ 13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
> use /proc/1111/oom_score_adj instead.
> Begin: Loading essential drivers... ...
> [ 13.559435] blkid used greatest stack depth: 5408 bytes left
> [ 13.568391] blkfront: xvda: barriers disabled
> [ 13.574762] blkid used greatest stack depth: 5264 bytes left
> [ 13.580813] xvda: xvda1 xvda2 < xvda5 >
> [ 13.645562] modprobe used greatest stack depth: 5088 bytes left
> [ 13.653160] Initialising Xen virtual ethernet driver.
> [ 13.659540] blkid used greatest stack depth: 4960 bytes left
> [ 13.735467] blkid used greatest stack depth: 4752 bytes left
> Done.
> Begin: Running /scripts/init-premount ...
> Done.
> Begin: Mounting root file system... ...
> Begin: Running /scripts/local-top ...
> Done.
> Begin: Running /scripts/local-premount ...
> Done.
> [ 13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly
> filesystem
> [ 13.903799] EXT4-fs (xvda1): write access will be enabled during
> recovery
> [ 14.033719] EXT4-fs (xvda1): recovery complete
> [ 14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
> mode. Opts: (null)
> [ 14.035510] exe used greatest stack depth: 4112 bytes left
> Begin: Running /scripts/local-bottom ...
> Done.
> Done.
> Begin: Running /scripts/init-bottom ...
> Done.
> init: ureadahead main process (1214) terminated with status 5
> fsck from util-linux-ng 2.17.2
> e2fsck 1.41.11 (14-Mar-2010)
> /dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
> * Enable login on console [
> OK ]
> * Check for hostname changes... [
> OK ]
> ...done.
> * Setting sensors limits
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> [
> OK ]
> * Initializing random number generator... [
> OK ]
> * Setting up X server socket directory /tmp/.X11-unix... [
> OK ]
> * Setting up ICE socket directory /tmp/.ICE-unix... [
> OK ]
> * Not starting fancontrol; run pwmconfig first.
> * Starting internet superserver inetd [
> OK ]
> Starting openntpd: ntpd.
> * Starting network management services:
> * Setting core file location... [
> OK ]
> * Restoring resolver state... [
> OK ]
> * Checking tomcat configuration
> ...done.
> * Starting Tomcat servlet engine tomcat6
> Using CATALINA_BASE: /var/lib/tomcat6
> Using CATALINA_HOME: /usr/share/tomcat6
> Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
> Using JRE_HOME: /usr/lib/jvm/java-6-openjdk
> Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar
> [
> OK ]
> collectd has been disabled - see /etc/default/collectd.
> * Recording successful boot for GRUB
> ...done.
> * Running local boot scripts (/etc/rc.local) [
> OK ]
> mountall: Plymouth command failed
> mountall: Disconnected from Plymouth
>
> 6.0.16-01
> kaan-04-dpm login: root
> Password:
>
This same mmu.c:1831 is also generated by the stable/xen-pcifront-fixes
branch.
Identical symptoms.
-Bruce
> [ 42.646538] ------------[ cut here ]------------
> [ 42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
> [ 42.646566] invalid opcode: 0000 [#1] SMP
> [ 42.646582] last sysfs file: /sys/kernel/uevent_seqnum
> [ 42.646592] CPU 0
> [ 42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
> xen_netfront xen_blkfront
> [ 42.646665]
> [ 42.646674] Pid: 3916, comm: lsb_release Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [ 42.646688] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
> pin_pagetable_pfn+0x31/0x37
> [ 42.646711] RSP: e02b:ffff88007595bce8 EFLAGS: 00010282
> [ 42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX:
> 0000000000000001
> [ 42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
> 00000000deadbeef
> [ 42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09:
> ffff880000000000
> [ 42.646756] R10: 00000000deadbeef R11: dead000000200200 R12:
> 0000000000000003
> [ 42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15:
> ffff880075161d40
> [ 42.646784] FS: 00007fee63b01700(0000) GS:ffff88007ff40000(0000)
> knlGS:0000000000000000
> [ 42.646797] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4:
> 0000000000002660
> [ 42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [ 42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [ 42.646844] Process lsb_release (pid: 3916, threadinfo
> ffff88007595a000, task ffff880075161d40)
> [ 42.646857] Stack:
> [ 42.646864] ffff880000000000 00000000000484f0 ffff8800777c6d80
> 000000000007751e
> [ 42.646892] ffff88007595bd28 ffffffff810066a8 000000000007751e
> ffff8800777c6d80
> [ 42.646919] ffff88007595bd38 ffffffff810066db ffff88007595bd78
> ffffffff810e048f
> [ 42.646946] Call Trace:
> [ 42.646959] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
> [ 42.646973] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
> [ 42.646989] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
> [ 42.647003] [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
> [ 42.647017] [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
> [ 42.647033] [<ffffffff8100eb84>] ?
> arch_get_unmapped_area_topdown+0x4a/0x28a
> [ 42.647052] [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [ 42.647059] [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
> [ 42.647059] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [ 42.647059] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [ 42.647059] [<ffffffff81443615>] page_fault+0x25/0x30
> [ 42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
> e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
> 85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48
> 8b 3d
> [ 42.647059] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
> [ 42.647059] RSP <ffff88007595bce8>
> [ 42.647059] ---[ end trace 3fbbffee414a5166 ]---
>
[-- Attachment #1.2: Type: text/html, Size: 26699 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-10 22:30 ` Bruce Edge
2010-11-10 23:03 ` Bruce Edge
2010-11-11 6:13 ` Bruce Edge
@ 2010-11-11 12:01 ` Boris Derzhavets
2010-11-11 12:08 ` Boris Derzhavets
2010-11-11 12:26 ` Boris Derzhavets
2010-11-11 15:08 ` Bruce Edge
3 siblings, 2 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-11 12:01 UTC (permalink / raw)
To: xen-devel, Jeremy Fitzhardinge, Konrad Rzeszutek Wilk, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 36568 bytes --]
This is a mainline 2.6.37-rc1 pvops domU on top of a xen 4.0.1 and xen/stable-2.6.32.x 2.6.32.25 pvops dom0.
Now prepare NFS remote share , mount it at DomU on /mnt/tmp
after normal login and run `ls -l /mnt/tmp`
***********
Report
***********
root@ServerLLX:/home/boris/ubuntu# xm create -c u10.cfg
Using config file "./u10.cfg".
Started domain Ubuntu10PV (id=9)
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.37-rc1 (root@ServerLLX) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Fri Nov 5 19:29:09 MSK 2010
Command line: root=/dev/xvda1 ro console=hvc0
ACPI in unprivileged domain disabled
released 0 pages of unused memory
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 00000000000a0000 (usable)
Xen: 00000000000a0000 - 0000000000100000 (reserved)
Xen: 0000000000100000 - 0000000080800000 (usable)
NX (Execute Disable) protection: active
DMI not present or invalid.
No AGP bridge found
last_pfn = 0x80800 max_arch_pfn = 0x400000000
init_memory_mapping: 0000000000000000-0000000080800000
RAMDISK: 01e98000 - 0e9fa000
No NUMA configuration found
Faking a node at 0000000000000000-0000000080800000
Initmem setup node 0 0000000000000000-0000000080800000
NODE_DATA [000000007ffec000 - 000000007fffffff]
Zone PFN ranges:
DMA 0x00000010 -> 0x00001000
DMA32 0x00001000 -> 0x00100000
Normal empty
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
0: 0x00000010 -> 0x000000a0
0: 0x00000100 -> 0x00080800
SMP: Allowing 1 CPUs, 0 hotplug CPUs
No local APIC present
APIC: disable apic facility
APIC: switched to apic NOOP
PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
Allocating PCI resources starting at 80800000 (gap: 80800000:7f800000)
Booting paravirtualized kernel on Xen
Xen version: 4.0.1 (preserve-AD)
setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
PERCPU: Embedded 28 pages/cpu @ffff88007ffbd000 s82752 r8192 d23744 u114688
pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
pcpu-alloc: [0] 0
Built 1 zonelists in Node order, mobility grouping on. Total pages: 519022
Policy zone: DMA32
Kernel command line: root=/dev/xvda1 ro console=hvc0
PID hash table entries: 4096 (order: 3, 32768 bytes)
Checking aperture...
No AGP bridge found
Memory: 1835560k/2105344k available (4462k kernel code, 448k absent, 269336k reserved, 6442k data, 1564k init)
Hierarchical RCU implementation.
RCU-based detection of stalled CPUs is disabled.
NR_IRQS:16640 nr_irqs:256 16
Console: colour dummy device 80x25
console [tty0] enabled
console [hvc0] enabled
allocated 22282240 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
installing Xen timer for CPU 0
Detected 2833.068 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 5666.13 BogoMIPS (lpj=2833068)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux: Initializing.
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys debug
Initializing cgroup subsys ns
ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup.
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
CPU: Unsupported number of siblings 4
Performance Events: unsupported p6 CPU model 23 no PMU driver, software events only.
SMP alternatives: switching to UP code
Freeing SMP alternatives: 20k freed
cpu 0 spinlock event irq 254
Brought up 1 CPUs
Grant table initialized
regulator: core version 0.5
regulator: dummy:
Time: 165:165:165 Date: 165/165/65
NET: Registered protocol family 16
PCI: setting up Xen PCI frontend stub
bio: create slab <bio-0> at 0
ACPI: Interpreter disabled.
xen_balloon: Initialising balloon driver.
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel: domain hash size = 128
NetLabel: protocols = UNLABELED CIPSOv4
NetLabel: unlabeled traffic allowed by default
Switching to clocksource xen
pnp: PnP ACPI: disabled
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 208264k freed
platform rtc_cmos: registered platform RTC device (no PNP device found)
audit: initializing netlink socket (disabled)
type=2000 audit(1289476232.768:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 3991
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
pci-stub: invalid id string ""
Event-channel device installed.
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
Initialising Xen virtual ethernet driver.
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
rtc_cmos: probe of rtc_cmos failed with error -38
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
IPVS: Registered protocols (TCP, UDP, AH, ESP)
IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
IPVS: ipvs loaded.
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Bridge firewalling registered
Ebtables v2.0 registered
Registering the dns_resolver key type
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!
blkfront: xvda: barriers disabled
xvda: xvda1 xvda2
XENBUS: Device with no driver: device/console/0
Magic number: 1:252:3141
drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Initalizing network drop monitor service
Freeing unused kernel memory: 1564k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 1664k freed
Freeing unused kernel memory: 1904k freed
Loading, please wait...
mount: mounting none on /dev failed: No such device
udevd (63): /proc/63/oom_adj is deprecated, please use /proc/63/oom_score_adj instead.
Begin: Loading essential drivers... ...
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
EXT4-fs (xvda1): write access will be enabled during recovery
EXT4-fs (xvda1): orphan cleanup on readonly fs
EXT4-fs (xvda1): 5 orphan inodes deleted
EXT4-fs (xvda1): recovery complete
EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (157) terminated with status 5
fsck from util-linux-ng 2.17.2
/dev/xvda1: clean, 188758/760368 files, 1083747/3041280 blocks
* Setting sensors limits [ OK ]
* Exporting directories for NFS kernel daemon... [ OK ]
* Starting NFS kernel daemon [ OK ]
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
* PulseAudio configured for per-user sessions
saned disabled; edit /etc/default/saned
* Enabling additional executable binary formats binfmt-support [ OK ]
fsck from util-linux-ng 2.17.2
/dev/xvda1: clean, 188758/760368 files, 1083747/3041280 blocks
* Setting sensors limits [ OK ]
* Exporting directories for NFS kernel daemon... [ OK ]
* Starting NFS kernel daemon [ OK ]
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
* PulseAudio configured for per-user sessions
saned disabled; edit /etc/default/saned
* Enabling additional executable binary formats binfmt-support [ OK ]
* Checking battery state... [ OK ]
Ubuntu 10.10 boris-HVM-domU hvc0
boris-HVM-domU login: root
Password:
Last login: Thu Nov 11 12:29:06 CET 2010 on hvc0
Linux boris-HVM-domU 2.6.37-rc1 #1 SMP Fri Nov 5 19:29:09 MSK 2010 x86_64 GNU/Linux
Ubuntu 10.10
Welcome to Ubuntu!
* Documentation: https://help.ubuntu.com/
root@boris-HVM-domU:~# mount 192.168.1.11:/home/boris /mnt/tmp
root@boris-HVM-domU:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 12G 4.0G 6.9G 37% /
none 1001M 116K 1001M 1% /dev
none 1001M 0 1001M 0% /dev/shm
none 1001M 100K 1001M 1% /var/run
none 1001M 0 1001M 0% /var/lock
192.168.1.11:/home/boris
44G 21G 21G 50% /mnt/tmp
*************************************************
RUNNING 'ls -l' against NFS mounted share
*************************************************
root@boris-HVM-domU:~# ls -l /mnt/tmp
total 693924
-rw-r--r-- 1 root root 709871616 2009-06-07 18:51 osol-0906-x86.iso
drwxr-xr-x 2 root root 4096 2009-06-07 19:06 solaris
root@boris-HVM-domU:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3e:48:b3:3c
inet addr:192.168.1.9 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fe48:b33c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:300 errors:0 dropped:0 overruns:0 frame:0
TX packets:94 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:30391 (30.3 KB) TX bytes:13562 (13.5 KB)
Interrupt:246
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:96 errors:0 dropped:0 overruns:0 frame:0
TX packets:96 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8996 (8.9 KB) TX bytes:8996 (8.9 KB)
root@boris-HVM-domU:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 12G 4.0G 6.9G 37% /
none 1001M 116K 1001M 1% /dev
none 1001M 0 1001M 0% /dev/shm
none 1001M 100K 1001M 1% /var/run
none 1001M 0 1001M 0% /var/lock
192.168.1.11:/home/boris
44G 21G 21G 51% /mnt/tmp
Boris.
--- On Wed, 11/10/10, Bruce Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Wednesday, November 10, 2010, 5:30 PM
On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
>
> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
>
> The domU console output hung up prior to spitting out a prompt post root login. I logged into the domU via ssh and ran an 'ls' of an NFS mounted dir and
> this immediately came out on the domU console:
>
> [ 480.467525] BUG: unable to handle kernel paging request at ffff880077e9f030
> [ 480.467547] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> [ 480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> [ 480.467587] Oops: 0003 [#1] SMP
> [ 480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> [ 480.467611] CPU 3
> [ 480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront xen_blkfront
> [ 480.467676]
> [ 480.467684] Pid: 3957, comm: zsh Not tainted 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [ 480.467695] RIP: e030:[<ffffffff810355f3>] [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> [ 480.467710] RSP: e02b:ffff880077ebfbd8 EFLAGS: 00010202
> [ 480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX: 800000004d686167
> [ 480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI: ffff8800763ddb80
> [ 480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09: ffffea0001a3b2d8
> [ 480.467748] R10: 0000000000000100 R11: 0000000000000000 R12: 00007f5cb26069d0
> [ 480.467759] R13: 0000000000000001 R14: 0000000000000000 R15: ffff8800763ddb80
> [ 480.467772] FS: 00007f5cb2606700(0000) GS:ffff88007ff94000(0000) knlGS:0000000000000000
> [ 480.467783] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4: 0000000000002660
> [ 480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task ffff880078b80000)
> [ 480.467832] Stack:
> [ 480.467838] 0000000000000100 ffffea0001a3b2d8 7000000000000000 0000000000000000
> [ 480.467862] ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8 ffffffff810e2200
> [ 480.467886] ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8 0000000000000001
> [ 480.467910] Call Trace:
> [ 480.467921] [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> [ 480.467934] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [ 480.467946] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [ 480.467958] [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> [ 480.467972] [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [ 480.467984] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> [ 480.467996] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> [ 480.468009] [<ffffffff810072a2>] ? check_events+0x12/0x20
> [ 480.468020] [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
> [ 480.468034] [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> [ 480.468045] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> [ 480.468059] [<ffffffff810072a2>] ? check_events+0x12/0x20
> [ 480.468072] [<ffffffff81443615>] page_fault+0x25/0x30
> [ 480.468085] [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> [ 480.468096] [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> [ 480.468108] [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> [ 480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> [ 480.468323] RIP [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> [ 480.468323] RSP <ffff880077ebfbd8>
> [ 480.468323] CR2: ffff880077e9f030
> [ 480.468323] ---[ end trace 6d1623321ecd50ac ]---
>
> The domU config is attached as config.2.6.37_domU_debug
>
>
> -Bruce
Here's another attempt from the same config with the full domU boot logs:
Started domain dpm (id=4)
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
[ 0.000000] Command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] released 0 pages of unused memory
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
[ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
[ 0.000000] Xen: 0000000000100000 - 0000000080800000 (usable)
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] DMI not present or invalid.
[ 0.000000] No AGP bridge found
[ 0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[ 0.000000] Scanning 0 areas for low memory corruption
[ 0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[ 0.000000] RAMDISK: 01d95000 - 02fa9000
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at 0000000000000000-0000000080800000
[ 0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[ 0.000000] NODE_DATA [000000007fffb000 - 000000007fffffff]
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal empty
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[2] active PFN ranges
[ 0.000000] 0: 0x00000010 -> 0x000000a0
[ 0.000000] 0: 0x00000100 -> 0x00080800
[ 0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[ 0.000000] No local APIC present
[ 0.000000] APIC: disable apic facility
[ 0.000000] APIC: switched to apic NOOP
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[ 0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.1-unstable (preserve-AD)
[ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[ 0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[ 0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[ 0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[ 0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
448k absent, 145000k reserved, 6327k data, 756k init)
[ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] RCU-based detection of stalled CPUs is disabled.
[ 0.000000] NR_IRQS:4352 nr_irqs:320 16
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] console [hvc0] enabled
[ 0.000000] installing Xen timer for CPU 0
[ 0.000000] Detected 2660.064 MHz processor.
[ 0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
[ 0.000999] pid_max: default: 32768 minimum: 301
[ 0.000999] Security Framework initialized
[ 0.000999] SELinux: Initializing.
[ 0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[ 0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[ 0.001635] Mount-cache hash table entries: 256
[ 0.002195] Initializing cgroup subsys ns
[ 0.002208] ns_cgroup deprecated: consider using the
'clone_children' flag without the ns_cgroup.
[ 0.002220] Initializing cgroup subsys cpuacct
[ 0.002231] Initializing cgroup subsys freezer
[ 0.002356] CPU: Unsupported number of siblings 16
[ 0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[ 0.002483] SMP alternatives: switching to UP code
[ 0.004119] cpu 0 spinlock event irq 318
[ 0.004765] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[ 0.005050] installing Xen timer for CPU 1
[ 0.005157] cpu 1 spinlock event irq 312
[ 0.005194] SMP alternatives: switching to SMP code
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.006158] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[ 0.006474] installing Xen timer for CPU 2
[ 0.006585] cpu 2 spinlock event irq 306
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.006816] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[ 0.006999] installing Xen timer for CPU 3
[ 0.007037] cpu 3 spinlock event irq 300
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.007300] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[ 0.007534] installing Xen timer for CPU 4
[ 0.007636] cpu 4 spinlock event irq 294
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.007874] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[ 0.008041] installing Xen timer for CPU 5
[ 0.008083] cpu 5 spinlock event irq 288
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.008344] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[ 0.008356] Brought up 6 CPUs
[ 0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
[ 0.012051] Grant table initialized
[ 0.030978] Time: 165:165:165 Date: 165/165/65
[ 0.031229] NET: Registered protocol family 16
[ 0.036672] PCI: setting up Xen PCI frontend stub
[ 0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
[ 0.074302] bio: create slab <bio-0> at 0
[ 0.075385] ACPI: Interpreter disabled.
[ 0.077056] xen_balloon: Initialising balloon driver.
[ 0.079323] vgaarb: loaded
[ 0.081270] usbcore: registered new interface driver usbfs
[ 0.081283] usbcore: registered new interface driver hub
[ 0.081283] usbcore: registered new device driver usb
[ 0.083183] PCI: System does not support PCI
[ 0.084007] PCI: System does not support PCI
[ 0.085236] NetLabel: Initializing
[ 0.085236] NetLabel: domain hash size = 128
[ 0.085236] NetLabel: protocols = UNLABELED CIPSOv4
[ 0.085313] NetLabel: unlabeled traffic allowed by default
[ 0.086544] Switching to clocksource xen
[ 0.106256] pnp: PnP ACPI: disabled
[ 0.129799] NET: Registered protocol family 2
[ 0.130174] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.131882] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[ 0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 0.133179] TCP: Hash tables configured (established 262144 bind 65536)
[ 0.133191] TCP reno registered
[ 0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[ 0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[ 0.133914] NET: Registered protocol family 1
[ 0.134582] RPC: Registered udp transport module.
[ 0.134590] RPC: Registered tcp transport module.
[ 0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.134892] Trying to unpack rootfs image as initramfs...
[ 0.178824] Freeing initrd memory: 18512k freed
[ 0.268798] DMA-API: preallocated 32768 debug entries
[ 0.268810] DMA-API: debugging enabled by kernel config
[ 0.271181] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[ 0.285410] Machine check injector initialized
[ 0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[ 0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[ 0.295590] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[ 0.295607] Scanning for low memory corruption every 60 seconds
[ 0.298813] audit: initializing netlink socket (disabled)
[ 0.298940] type=2000 audit(1289403082.795:1): initialized
[ 0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.376985] VFS: Disk quotas dquot_6.5.2
[ 0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.383121] NTFS driver 2.1.29 [Flags: R/W].
[ 0.385014] msgmni has been set to 3864
[ 0.389953] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[ 0.389970] io scheduler noop registered
[ 0.389979] io scheduler deadline registered
[ 0.390666] io scheduler cfq registered (default)
[ 0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.406238] pcifront pci-0: Installing PCI frontend
[ 0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
[ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
[ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
[ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
[ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
[ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
[ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
[ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
[ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
[ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
[ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
[ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
[ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
[ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
[ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
[ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
[ 0.731775] Non-volatile memory driver v1.3
[ 0.731790] Linux agpgart interface v0.103
[ 0.733731] [drm] Initialized drm 1.1.0 20060810
[ 0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.827202] brd: module loaded
[ 0.840788] loop: module loaded
[ 0.841965] Fixed MDIO Bus: probed
[ 0.844025] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.844651] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.845194] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.846985] usbcore: registered new interface driver usblp
[ 0.847763] usbcore: registered new interface driver libusual
[ 0.849433] PNP: No PS/2 controller found. Probing ports directly.
[ 0.850279] i8042.c: No controller found.
[ 0.850883] mice: PS/2 mouse device common for all mice
[ 0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[ 0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
[ 0.855928] cpuidle: using governor ladder
[ 0.855941] cpuidle: using governor menu
[ 0.856029] Netfilter messages via NETLINK v0.30.
[ 0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 0.857888] ctnetlink v0.93: registering with nfnetlink.
[ 0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 0.860677] TCP cubic registered
[ 0.860687] Initializing XFRM netlink socket
[ 0.863286] NET: Registered protocol family 10
[ 0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 0.867612] IPv6 over IPv4 tunneling driver
[ 0.871326] NET: Registered protocol family 17
[ 0.871566] Registering the dns_resolver key type
[ 0.873668] registered taskstats version 1
[ 0.873912] kmemleak: Kernel memory leak detector initialized
[ 0.873928] kmemleak: Automatic memory scanning thread started
[ 0.875057] XENBUS: Device with no driver: device/vbd/51712
[ 0.875057] XENBUS: Device with no driver: device/vif/0
[ 0.875057] XENBUS: Device with no driver: device/console/0
[ 0.875057] Magic number: 1:252:3141
[ 13.388958] Freeing unused kernel memory: 756k freed
[ 13.389190] Write protecting the kernel read-only data: 10240k
[ 13.398774] Freeing unused kernel memory: 1724k freed
[ 13.400663] Freeing unused kernel memory: 1992k freed
[ 13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
[ 13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
use /proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[ 13.559435] blkid used greatest stack depth: 5408 bytes left
[ 13.568391] blkfront: xvda: barriers disabled
[ 13.574762] blkid used greatest stack depth: 5264 bytes left
[ 13.580813] xvda: xvda1 xvda2 < xvda5 >
[ 13.645562] modprobe used greatest stack depth: 5088 bytes left
[ 13.653160] Initialising Xen virtual ethernet driver.
[ 13.659540] blkid used greatest stack depth: 4960 bytes left
[ 13.735467] blkid used greatest stack depth: 4752 bytes left
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[ 13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
[ 13.903799] EXT4-fs (xvda1): write access will be enabled during recovery
[ 14.033719] EXT4-fs (xvda1): recovery complete
[ 14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
[ 14.035510] exe used greatest stack depth: 4112 bytes left
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (1214) terminated with status 5
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
* Enable login on console [ OK ]
* Check for hostname changes... [ OK ]
...done.
* Setting sensors limits
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
[ OK ]
* Initializing random number generator... [ OK ]
* Setting up X server socket directory /tmp/.X11-unix... [ OK ]
* Setting up ICE socket directory /tmp/.ICE-unix... [ OK ]
* Not starting fancontrol; run pwmconfig first.
* Starting internet superserver inetd [ OK ]
Starting openntpd: ntpd.
* Starting network management services:
* Setting core file location... [ OK ]
* Restoring resolver state... [ OK ]
* Checking tomcat configuration
...done.
* Starting Tomcat servlet engine tomcat6
Using CATALINA_BASE: /var/lib/tomcat6
Using CATALINA_HOME: /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME: /usr/lib/jvm/java-6-openjdk
Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar
[ OK ]
collectd has been disabled - see /etc/default/collectd.
* Recording successful boot for GRUB
...done.
* Running local boot scripts (/etc/rc.local) [ OK ]
mountall: Plymouth command failed
mountall: Disconnected from Plymouth
6.0.16-01
kaan-04-dpm login: root
Password:
[ 42.646538] ------------[ cut here ]------------
[ 42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
[ 42.646566] invalid opcode: 0000 [#1] SMP
[ 42.646582] last sysfs file: /sys/kernel/uevent_seqnum
[ 42.646592] CPU 0
[ 42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[ 42.646665]
[ 42.646674] Pid: 3916, comm: lsb_release Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
[ 42.646688] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[ 42.646711] RSP: e02b:ffff88007595bce8 EFLAGS: 00010282
[ 42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX: 0000000000000001
[ 42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
[ 42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09: ffff880000000000
[ 42.646756] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
[ 42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15: ffff880075161d40
[ 42.646784] FS: 00007fee63b01700(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[ 42.646797] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4: 0000000000002660
[ 42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 42.646844] Process lsb_release (pid: 3916, threadinfo
ffff88007595a000, task ffff880075161d40)
[ 42.646857] Stack:
[ 42.646864] ffff880000000000 00000000000484f0 ffff8800777c6d80
000000000007751e
[ 42.646892] ffff88007595bd28 ffffffff810066a8 000000000007751e
ffff8800777c6d80
[ 42.646919] ffff88007595bd38 ffffffff810066db ffff88007595bd78
ffffffff810e048f
[ 42.646946] Call Trace:
[ 42.646959] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[ 42.646973] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[ 42.646989] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[ 42.647003] [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[ 42.647017] [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
[ 42.647033] [<ffffffff8100eb84>] ? arch_get_unmapped_area_topdown+0x4a/0x28a
[ 42.647052] [<ffffffff814463fe>] do_page_fault+0x374/0x396
[ 42.647059] [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[ 42.647059] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[ 42.647059] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[ 42.647059] [<ffffffff81443615>] page_fault+0x25/0x30
[ 42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48
8b 3d
[ 42.647059] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[ 42.647059] RSP <ffff88007595bce8>
[ 42.647059] ---[ end trace 3fbbffee414a5166 ]---
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 48397 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 12:01 ` Boris Derzhavets
@ 2010-11-11 12:08 ` Boris Derzhavets
2010-11-11 16:09 ` Konrad Rzeszutek Wilk
2010-11-11 12:26 ` Boris Derzhavets
1 sibling, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-11 12:08 UTC (permalink / raw)
To: xen-devel, Jeremy Fitzhardinge, Konrad Rzeszutek Wilk, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 39939 bytes --]
Now reproduced :-
root@boris-HVM-domU:~# scp 1.log 192.168.1.12:/home/boris/ubuntu
BUG: unable to handle kernel paging request at ffff88000e676930
IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
Oops: 0003 [#1] SMP
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0
Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ipv6 lp parport
Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
RIP: e030:[<ffffffff810069c3>] [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
RSP: e02b:ffff88006e697b18 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
FS: 00007fa525896700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
Stack:
00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
Call Trace:
[<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
[<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
[<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
[<ffffffff8104a7c5>] mmput+0x5e/0xd1
[<ffffffff8110694b>] flush_old_exec+0x526/0x600
[<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
[<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
[<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
[<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
[<ffffffff81106195>] search_binary_handler+0xbb/0x24d
[<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
[<ffffffff8110785d>] do_execve+0x1c1/0x2ae
[<ffffffff810119e5>] sys_execve+0x3e/0x55
[<ffffffff8100af9c>] stub_execve+0x6c/0xc0
Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4 ff ff 48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb 16 e8 99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
RIP [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
RSP <ffff88006e697b18>
CR2: ffff88000e676930
---[ end trace 073da74f4ce02315 ]---
lost connection
--- On Thu, 11/11/10, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, "Bruce Edge" <bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 7:01 AM
This is a mainline 2.6.37-rc1 pvops domU on top of a xen 4.0.1 and xen/stable-2.6.32.x 2.6.32.25 pvops dom0.
Now prepare NFS remote share , mount it at DomU on /mnt/tmp
after normal login and run `ls -l /mnt/tmp`
***********
Report
***********
root@ServerLLX:/home/boris/ubuntu# xm create -c u10.cfg
Using config file "./u10.cfg".
Started domain Ubuntu10PV (id=9)
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.37-rc1 (root@ServerLLX) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Fri Nov 5 19:29:09 MSK 2010
Command line: root=/dev/xvda1 ro console=hvc0
ACPI in unprivileged domain
disabled
released 0 pages of unused memory
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 00000000000a0000 (usable)
Xen: 00000000000a0000 - 0000000000100000 (reserved)
Xen: 0000000000100000 - 0000000080800000 (usable)
NX (Execute Disable) protection: active
DMI not present or invalid.
No AGP bridge found
last_pfn = 0x80800 max_arch_pfn = 0x400000000
init_memory_mapping: 0000000000000000-0000000080800000
RAMDISK: 01e98000 - 0e9fa000
No NUMA configuration found
Faking a node at 0000000000000000-0000000080800000
Initmem setup node 0 0000000000000000-0000000080800000
NODE_DATA [000000007ffec000 - 000000007fffffff]
Zone PFN ranges:
DMA 0x00000010 -> 0x00001000
DMA32 0x00001000 -> 0x00100000
Normal empty
Movable zone start PFN for each node
early_node_map[2] active PFN
ranges
0: 0x00000010 -> 0x000000a0
0: 0x00000100 -> 0x00080800
SMP: Allowing 1 CPUs, 0 hotplug CPUs
No local APIC present
APIC: disable apic facility
APIC: switched to apic NOOP
PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
Allocating PCI resources starting at 80800000 (gap: 80800000:7f800000)
Booting paravirtualized kernel on Xen
Xen version: 4.0.1 (preserve-AD)
setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
PERCPU: Embedded 28 pages/cpu @ffff88007ffbd000 s82752 r8192 d23744 u114688
pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
pcpu-alloc: [0] 0
Built 1 zonelists in Node order, mobility grouping on. Total pages: 519022
Policy zone: DMA32
Kernel command line: root=/dev/xvda1 ro console=hvc0
PID hash table entries: 4096 (order: 3, 32768 bytes)
Checking aperture...
No AGP bridge
found
Memory: 1835560k/2105344k available (4462k kernel code, 448k absent, 269336k reserved, 6442k data, 1564k init)
Hierarchical RCU implementation.
RCU-based detection of stalled CPUs is disabled.
NR_IRQS:16640 nr_irqs:256 16
Console: colour dummy device 80x25
console [tty0] enabled
console [hvc0] enabled
allocated 22282240 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
installing Xen timer for CPU 0
Detected 2833.068 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 5666.13 BogoMIPS (lpj=2833068)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux: Initializing.
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys
debug
Initializing cgroup subsys ns
ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup.
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
CPU: Unsupported number of siblings 4
Performance Events: unsupported p6 CPU model 23 no PMU driver, software events only.
SMP alternatives: switching to UP code
Freeing SMP alternatives: 20k freed
cpu 0 spinlock event irq 254
Brought up 1 CPUs
Grant table initialized
regulator: core version 0.5
regulator: dummy:
Time: 165:165:165 Date: 165/165/65
NET: Registered protocol family 16
PCI: setting up Xen PCI frontend stub
bio: create slab <bio-0> at 0
ACPI: Interpreter disabled.
xen_balloon: Initialising balloon driver.
vgaarb: loaded
SCSI subsystem initialized
usbcore:
registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel: domain hash size = 128
NetLabel: protocols = UNLABELED CIPSOv4
NetLabel: unlabeled traffic allowed by default
Switching to clocksource xen
pnp: PnP ACPI: disabled
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
NET: Registered protocol family 1
Trying to unpack rootfs image as
initramfs...
Freeing initrd memory: 208264k freed
platform rtc_cmos: registered platform RTC device (no PNP device found)
audit: initializing netlink socket (disabled)
type=2000 audit(1289476232.768:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 3991
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
pci-stub: invalid id string ""
Event-channel device installed.
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing
enabled
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
Initialising Xen virtual ethernet driver.
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
rtc_cmos: probe of rtc_cmos failed with error -38
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
IPVS: Registered protocols (TCP, UDP, AH, ESP)
IPVS: Connection hash table
configured (size=4096, memory=64Kbytes)
IPVS: ipvs loaded.
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Bridge firewalling registered
Ebtables v2.0 registered
Registering the dns_resolver key type
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!
blkfront: xvda: barriers disabled
xvda: xvda1 xvda2
XENBUS: Device with no driver: device/console/0
Magic number: 1:252:3141
drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Initalizing network drop monitor service
Freeing unused kernel memory: 1564k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 1664k freed
Freeing unused kernel memory: 1904k freed
Loading, please wait...
mount: mounting none on /dev failed: No such device
udevd (63): /proc/63/oom_adj is deprecated,
please use /proc/63/oom_score_adj instead.
Begin: Loading essential drivers... ...
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
EXT4-fs (xvda1): write access will be enabled during recovery
EXT4-fs (xvda1): orphan cleanup on readonly fs
EXT4-fs (xvda1): 5 orphan inodes deleted
EXT4-fs (xvda1): recovery complete
EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (157) terminated with status 5
fsck from util-linux-ng 2.17.2
/dev/xvda1: clean, 188758/760368 files, 1083747/3041280 blocks
* Setting sensors
limits [ OK ]
* Exporting directories for NFS kernel daemon... [ OK ]
* Starting NFS kernel daemon [ OK ]
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
* PulseAudio configured for per-user sessions
saned disabled; edit
/etc/default/saned
* Enabling additional executable binary formats binfmt-support [ OK ]
fsck from util-linux-ng 2.17.2
/dev/xvda1: clean, 188758/760368 files, 1083747/3041280 blocks
* Setting sensors limits [ OK ]
* Exporting directories for NFS kernel daemon... [ OK ]
* Starting NFS kernel
daemon [ OK ]
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
* PulseAudio configured for per-user sessions
saned disabled; edit /etc/default/saned
* Enabling additional executable binary formats binfmt-support [ OK ]
* Checking battery state... [ OK ]
Ubuntu 10.10 boris-HVM-domU hvc0
boris-HVM-domU login: root
Password:
Last login: Thu Nov 11 12:29:06 CET 2010 on hvc0
Linux boris-HVM-domU 2.6.37-rc1 #1 SMP Fri Nov 5 19:29:09 MSK 2010 x86_64 GNU/Linux
Ubuntu 10.10
Welcome to Ubuntu!
* Documentation: https://help.ubuntu.com/
root@boris-HVM-domU:~# mount 192.168.1.11:/home/boris /mnt/tmp
root@boris-HVM-domU:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 12G 4.0G 6.9G 37% /
none 1001M 116K 1001M 1% /dev
none 1001M 0 1001M 0%
/dev/shm
none 1001M 100K 1001M 1% /var/run
none 1001M 0 1001M 0% /var/lock
192.168.1.11:/home/boris
44G 21G 21G 50% /mnt/tmp
*************************************************
RUNNING 'ls -l' against NFS mounted share
*************************************************
root@boris-HVM-domU:~# ls -l /mnt/tmp
total 693924
-rw-r--r-- 1 root root 709871616 2009-06-07 18:51 osol-0906-x86.iso
drwxr-xr-x 2 root root 4096 2009-06-07 19:06 solaris
root@boris-HVM-domU:~#
ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3e:48:b3:3c
inet addr:192.168.1.9 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fe48:b33c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:300 errors:0 dropped:0 overruns:0 frame:0
TX packets:94 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:30391 (30.3 KB) TX bytes:13562 (13.5 KB)
Interrupt:246
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:96 errors:0 dropped:0 overruns:0 frame:0
TX packets:96 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8996 (8.9 KB) TX bytes:8996 (8.9 KB)
root@boris-HVM-domU:~# df -h
Filesystem Size Used Avail Use% Mounted
on
/dev/xvda1 12G 4.0G 6.9G 37% /
none 1001M 116K 1001M 1% /dev
none 1001M 0 1001M 0% /dev/shm
none 1001M 100K 1001M 1% /var/run
none 1001M 0 1001M 0% /var/lock
192.168.1.11:/home/boris
44G 21G 21G 51%
/mnt/tmp
Boris.
--- On Wed, 11/10/10, Bruce Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Wednesday, November 10, 2010, 5:30 PM
On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
>
> This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
>
> The domU console output hung up prior to spitting out a
prompt post root login. I logged into the domU via ssh and ran an 'ls' of an NFS mounted dir and
> this immediately came out on the domU console:
>
> [ 480.467525] BUG: unable to handle kernel paging request at ffff880077e9f030
> [ 480.467547] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> [ 480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> [ 480.467587] Oops: 0003 [#1] SMP
> [ 480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> [ 480.467611] CPU 3
> [ 480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront xen_blkfront
> [ 480.467676]
> [ 480.467684] Pid: 3957, comm: zsh Not tainted 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [ 480.467695] RIP: e030:[<ffffffff810355f3>]
[<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> [ 480.467710] RSP: e02b:ffff880077ebfbd8 EFLAGS: 00010202
> [ 480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX: 800000004d686167
> [ 480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI: ffff8800763ddb80
> [ 480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09: ffffea0001a3b2d8
> [ 480.467748] R10: 0000000000000100 R11: 0000000000000000 R12: 00007f5cb26069d0
> [ 480.467759] R13: 0000000000000001 R14: 0000000000000000 R15: ffff8800763ddb80
> [ 480.467772] FS: 00007f5cb2606700(0000) GS:ffff88007ff94000(0000) knlGS:0000000000000000
> [ 480.467783] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4: 0000000000002660
> [ 480.467802] DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
> [ 480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task ffff880078b80000)
> [ 480.467832] Stack:
> [ 480.467838] 0000000000000100 ffffea0001a3b2d8 7000000000000000 0000000000000000
> [ 480.467862] ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8 ffffffff810e2200
> [ 480.467886] ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8 0000000000000001
> [ 480.467910] Call Trace:
> [ 480.467921] [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> [ 480.467934] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [ 480.467946] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [ 480.467958] [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
>
[ 480.467972] [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [ 480.467984] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> [ 480.467996] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> [ 480.468009] [<ffffffff810072a2>] ? check_events+0x12/0x20
> [ 480.468020] [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
> [ 480.468034] [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> [ 480.468045] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> [ 480.468059] [<ffffffff810072a2>] ? check_events+0x12/0x20
> [ 480.468072] [<ffffffff81443615>] page_fault+0x25/0x30
> [ 480.468085] [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> [ 480.468096] [<ffffffff810498a6>] ?
schedule_tail+0x5c/0x60
> [ 480.468108] [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> [ 480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89 f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> [ 480.468323] RIP [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> [ 480.468323] RSP <ffff880077ebfbd8>
> [ 480.468323] CR2: ffff880077e9f030
> [ 480.468323] ---[ end trace 6d1623321ecd50ac ]---
>
> The domU config is attached as config.2.6.37_domU_debug
>
>
> -Bruce
Here's another attempt from the same config with the full domU boot logs:
Started domain dpm (id=4)
[ 0.000000]
Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
[ 0.000000] Command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] released 0 pages of unused memory
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
[ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
[ 0.000000] Xen: 0000000000100000 - 0000000080800000 (usable)
[
0.000000] NX (Execute Disable) protection: active
[ 0.000000] DMI not present or invalid.
[ 0.000000] No AGP bridge found
[ 0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[ 0.000000] Scanning 0 areas for low memory corruption
[ 0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[ 0.000000] RAMDISK: 01d95000 - 02fa9000
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at 0000000000000000-0000000080800000
[ 0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[ 0.000000] NODE_DATA [000000007fffb000 - 000000007fffffff]
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] DMA32 0x00001000
-> 0x00100000
[ 0.000000] Normal empty
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[2] active PFN ranges
[ 0.000000] 0: 0x00000010 -> 0x000000a0
[ 0.000000] 0: 0x00000100 -> 0x00080800
[ 0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[ 0.000000] No local APIC present
[ 0.000000] APIC: disable apic facility
[ 0.000000] APIC: switched to apic NOOP
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[ 0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.1-unstable (preserve-AD)
[
0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
nr_cpu_ids:6 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
r8192 d23744 u114688
[ 0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[ 0.000000] Built 1 zonelists in Node order, mobility grouping on.
Total pages: 519022
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line:
root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[ 0.000000] software IO TLB
at phys 0x79400000 - 0x7d400000
[ 0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
448k absent, 145000k reserved, 6327k data, 756k init)
[ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] RCU-based detection of stalled CPUs is disabled.
[ 0.000000] NR_IRQS:4352 nr_irqs:320 16
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] console [hvc0] enabled
[ 0.000000] installing Xen timer for CPU 0
[ 0.000000] Detected 2660.064 MHz processor.
[ 0.000999] Calibrating delay loop (skipped), value calculated
using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
[ 0.000999] pid_max: default: 32768 minimum:
301
[ 0.000999] Security Framework initialized
[ 0.000999] SELinux: Initializing.
[ 0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[ 0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[ 0.001635] Mount-cache hash table entries: 256
[ 0.002195] Initializing cgroup subsys ns
[ 0.002208] ns_cgroup deprecated: consider using the
'clone_children' flag without the ns_cgroup.
[ 0.002220] Initializing cgroup subsys cpuacct
[ 0.002231] Initializing cgroup subsys freezer
[ 0.002356] CPU: Unsupported number of siblings 16
[ 0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[ 0.002483] SMP alternatives: switching to UP code
[ 0.004119] cpu 0 spinlock
event irq 318
[ 0.004765] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[ 0.005050] installing Xen timer for CPU 1
[ 0.005157] cpu 1 spinlock event irq 312
[ 0.005194] SMP alternatives: switching to SMP code
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.006158] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[ 0.006474] installing Xen timer for CPU 2
[ 0.006585] cpu 2 spinlock event irq 306
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.006816] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[ 0.006999] installing Xen timer for CPU 3
[ 0.007037] cpu 3 spinlock event irq 300
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.007300] NMI watchdog
failed to create perf event on cpu3:
fffffffffffffffe
[ 0.007534] installing Xen timer for CPU 4
[ 0.007636] cpu 4 spinlock event irq 294
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.007874] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[ 0.008041] installing Xen timer for CPU 5
[ 0.008083] cpu 5 spinlock event irq 288
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.008344] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[ 0.008356] Brought up 6 CPUs
[ 0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
[ 0.012051] Grant table initialized
[ 0.030978] Time: 165:165:165 Date: 165/165/65
[ 0.031229] NET: Registered protocol family 16
[ 0.036672] PCI:
setting up Xen PCI frontend stub
[ 0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
[ 0.074302] bio: create slab <bio-0> at 0
[ 0.075385] ACPI: Interpreter disabled.
[ 0.077056] xen_balloon: Initialising balloon driver.
[ 0.079323] vgaarb: loaded
[ 0.081270] usbcore: registered new interface driver usbfs
[ 0.081283] usbcore: registered new interface driver hub
[ 0.081283] usbcore: registered new device driver usb
[ 0.083183] PCI: System does not support PCI
[ 0.084007] PCI: System does not support PCI
[ 0.085236] NetLabel: Initializing
[ 0.085236] NetLabel: domain hash size = 128
[ 0.085236] NetLabel: protocols = UNLABELED CIPSOv4
[ 0.085313] NetLabel: unlabeled traffic allowed by
default
[ 0.086544] Switching to clocksource xen
[ 0.106256] pnp: PnP ACPI: disabled
[ 0.129799] NET: Registered protocol family 2
[ 0.130174] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.131882] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[ 0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 0.133179] TCP: Hash tables configured (established 262144 bind 65536)
[ 0.133191] TCP reno registered
[ 0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[ 0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[ 0.133914] NET: Registered protocol family 1
[ 0.134582] RPC: Registered udp transport module.
[ 0.134590] RPC: Registered tcp transport
module.
[ 0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.134892] Trying to unpack rootfs image as initramfs...
[ 0.178824] Freeing initrd memory: 18512k freed
[ 0.268798] DMA-API: preallocated 32768 debug entries
[ 0.268810] DMA-API: debugging enabled by kernel config
[ 0.271181] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[ 0.285410] Machine check injector initialized
[ 0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[ 0.295043]
microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[ 0.295590] microcode: Microcode Update Driver: v2.00
<tigran@aivazian.fsnet.co.uk>, Peter Oruba
[ 0.295607] Scanning for low memory corruption every 60 seconds
[ 0.298813] audit: initializing netlink socket (disabled)
[ 0.298940] type=2000 audit(1289403082.795:1): initialized
[ 0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.376985] VFS: Disk quotas dquot_6.5.2
[ 0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.383121] NTFS driver 2.1.29 [Flags: R/W].
[ 0.385014] msgmni has been set to 3864
[ 0.389953] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[
0.389970] io scheduler noop registered
[ 0.389979] io scheduler deadline registered
[ 0.390666] io scheduler cfq registered (default)
[ 0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.406238] pcifront pci-0: Installing PCI frontend
[ 0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
[ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
[ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
[ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
[ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
[ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
[ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
[ 0.417634] pcifront
pci-0: claiming resource 0000:00:01.0/4
[ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
[ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
[ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
[ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
[ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
[ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
[ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
[ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
[ 0.731775] Non-volatile memory driver v1.3
[ 0.731790] Linux agpgart interface v0.103
[ 0.733731] [drm] Initialized drm 1.1.0 20060810
[ 0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.827202] brd: module
loaded
[ 0.840788] loop: module loaded
[ 0.841965] Fixed MDIO Bus: probed
[ 0.844025] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.844651] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.845194] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.846985] usbcore: registered new interface driver usblp
[ 0.847763] usbcore: registered new interface driver libusual
[ 0.849433] PNP: No PS/2 controller found. Probing ports directly.
[ 0.850279] i8042.c: No controller found.
[ 0.850883] mice: PS/2 mouse device common for all mice
[ 0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[ 0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
[ 0.855928] cpuidle: using governor ladder
[
0.855941] cpuidle: using governor menu
[ 0.856029] Netfilter messages via NETLINK v0.30.
[ 0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 0.857888] ctnetlink v0.93: registering with nfnetlink.
[ 0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 0.860677] TCP cubic registered
[ 0.860687] Initializing XFRM netlink socket
[ 0.863286] NET: Registered protocol family 10
[ 0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 0.867612] IPv6 over IPv4 tunneling driver
[ 0.871326] NET: Registered protocol family 17
[ 0.871566] Registering the dns_resolver key type
[ 0.873668] registered taskstats version 1
[ 0.873912] kmemleak: Kernel memory leak detector initialized
[ 0.873928] kmemleak: Automatic memory
scanning thread started
[ 0.875057] XENBUS: Device with no driver: device/vbd/51712
[ 0.875057] XENBUS: Device with no driver: device/vif/0
[ 0.875057] XENBUS: Device with no driver: device/console/0
[ 0.875057] Magic number: 1:252:3141
[ 13.388958] Freeing unused kernel memory: 756k freed
[ 13.389190] Write protecting the kernel read-only data: 10240k
[ 13.398774] Freeing unused kernel memory: 1724k freed
[ 13.400663] Freeing unused kernel memory: 1992k freed
[ 13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
[ 13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
use /proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[ 13.559435] blkid used greatest stack depth: 5408 bytes
left
[ 13.568391] blkfront: xvda: barriers disabled
[ 13.574762] blkid used greatest stack depth: 5264 bytes left
[ 13.580813] xvda: xvda1 xvda2 < xvda5 >
[ 13.645562] modprobe used greatest stack depth: 5088 bytes left
[ 13.653160] Initialising Xen virtual ethernet driver.
[ 13.659540] blkid used greatest stack depth: 4960 bytes left
[ 13.735467] blkid used greatest stack depth: 4752 bytes left
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[ 13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly filesystem
[ 13.903799] EXT4-fs (xvda1): write access will be enabled during
recovery
[ 14.033719] EXT4-fs (xvda1): recovery complete
[ 14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
mode. Opts: (null)
[ 14.035510] exe used greatest stack depth: 4112 bytes left
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (1214) terminated with status 5
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
* Enable login on console [ OK ]
* Check for hostname changes...
[ OK ]
...done.
* Setting sensors limits
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
[ OK ]
* Initializing random number generator... [ OK ]
* Setting up X server socket directory /tmp/.X11-unix... [ OK ]
* Setting up ICE
socket directory /tmp/.ICE-unix... [ OK ]
* Not starting fancontrol; run pwmconfig first.
* Starting internet superserver inetd [ OK ]
Starting openntpd: ntpd.
* Starting network management services:
* Setting core file location... [ OK ]
* Restoring resolver state... [ OK ]
* Checking tomcat configuration
...done.
* Starting Tomcat servlet engine tomcat6
Using CATALINA_BASE: /var/lib/tomcat6
Using CATALINA_HOME: /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME: /usr/lib/jvm/java-6-openjdk
Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar
[ OK ]
collectd has been disabled - see /etc/default/collectd.
* Recording successful boot for GRUB
...done.
* Running local boot scripts (/etc/rc.local) [ OK ]
mountall: Plymouth command failed
mountall: Disconnected from
Plymouth
6.0.16-01
kaan-04-dpm login: root
Password:
[ 42.646538] ------------[ cut here ]------------
[ 42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
[ 42.646566] invalid opcode: 0000 [#1] SMP
[ 42.646582] last sysfs file: /sys/kernel/uevent_seqnum
[ 42.646592] CPU 0
[ 42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
xen_netfront xen_blkfront
[ 42.646665]
[ 42.646674] Pid: 3916, comm: lsb_release Not tainted
2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
[ 42.646688] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[ 42.646711] RSP: e02b:ffff88007595bce8 EFLAGS:
00010282
[ 42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX: 0000000000000001
[ 42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
[ 42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09: ffff880000000000
[ 42.646756] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003
[ 42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15: ffff880075161d40
[ 42.646784] FS: 00007fee63b01700(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[ 42.646797] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4: 0000000000002660
[ 42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 42.646832] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
[ 42.646844] Process lsb_release (pid: 3916, threadinfo
ffff88007595a000, task ffff880075161d40)
[ 42.646857] Stack:
[ 42.646864] ffff880000000000 00000000000484f0 ffff8800777c6d80
000000000007751e
[ 42.646892] ffff88007595bd28 ffffffff810066a8 000000000007751e
ffff8800777c6d80
[ 42.646919] ffff88007595bd38 ffffffff810066db ffff88007595bd78
ffffffff810e048f
[ 42.646946] Call Trace:
[ 42.646959] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[ 42.646973] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[ 42.646989] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[ 42.647003] [<ffffffff810e2717>]
handle_mm_fault+0x131/0x8c1
[ 42.647017] [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
[ 42.647033] [<ffffffff8100eb84>] ? arch_get_unmapped_area_topdown+0x4a/0x28a
[ 42.647052] [<ffffffff814463fe>] do_page_fault+0x374/0x396
[ 42.647059] [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[ 42.647059] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[ 42.647059] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[ 42.647059] [<ffffffff81443615>] page_fault+0x25/0x30
[ 42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48
8b
3d
[ 42.647059] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[ 42.647059] RSP <ffff88007595bce8>
[ 42.647059] ---[ end trace 3fbbffee414a5166 ]---
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
-----Inline Attachment Follows-----
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 52362 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 12:08 ` Boris Derzhavets
@ 2010-11-11 16:09 ` Konrad Rzeszutek Wilk
2010-11-11 16:29 ` Boris Derzhavets
2010-11-11 16:46 ` Boris Derzhavets
0 siblings, 2 replies; 45+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-11-11 16:09 UTC (permalink / raw)
To: Boris Derzhavets; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
On Thu, Nov 11, 2010 at 04:08:03AM -0800, Boris Derzhavets wrote:
> Now reproduced :-
>
> root@boris-HVM-domU:~# scp 1.log 192.168.1.12:/home/boris/ubuntu
> BUG: unable to handle kernel paging request at ffff88000e676930
> IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
> Oops: 0003 [#1] SMP
> last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
> CPU 0
> Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ipv6 lp parport
>
> Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
Yikes. So no PCI passthrough, just a normal PV DomU.
Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
> RIP: e030:[<ffffffff810069c3>] [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> RSP: e02b:ffff88006e697b18 EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
> RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
> RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
> R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
> R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
> FS: 00007fa525896700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
> CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
> Stack:
> 00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
> ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
> 00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
> Call Trace:
> [<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
> [<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
> [<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
> [<ffffffff8104a7c5>] mmput+0x5e/0xd1
> [<ffffffff8110694b>] flush_old_exec+0x526/0x600
> [<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
> [<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
> [<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
> [<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
> [<ffffffff81106195>] search_binary_handler+0xbb/0x24d
> [<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
> [<ffffffff8110785d>] do_execve+0x1c1/0x2ae
> [<ffffffff810119e5>] sys_execve+0x3e/0x55
> [<ffffffff8100af9c>] stub_execve+0x6c/0xc0
> Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4 ff ff 48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb 16 e8 99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
> RIP [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> RSP <ffff88006e697b18>
> CR2: ffff88000e676930
> ---[ end trace 073da74f4ce02315 ]---
> lost connection
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 16:09 ` Konrad Rzeszutek Wilk
@ 2010-11-11 16:29 ` Boris Derzhavets
2010-11-11 16:46 ` Konrad Rzeszutek Wilk
2010-11-11 16:46 ` Boris Derzhavets
1 sibling, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-11 16:29 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 3715 bytes --]
> Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
Konrad,
The version is 2.6.37-rc1, been built from upstream source.
Boris.
--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 11:09 AM
On Thu, Nov 11, 2010 at 04:08:03AM -0800, Boris Derzhavets wrote:
> Now reproduced :-
>
> root@boris-HVM-domU:~# scp 1.log 192.168.1.12:/home/boris/ubuntu
> BUG: unable to handle kernel paging request at ffff88000e676930
> IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
> Oops: 0003 [#1] SMP
> last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
> CPU 0
> Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ipv6 lp parport
>
> Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
Yikes. So no PCI passthrough, just a normal PV DomU.
Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
> RIP: e030:[<ffffffff810069c3>] [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> RSP: e02b:ffff88006e697b18 EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
> RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
> RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
> R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
> R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
> FS: 00007fa525896700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
> CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
> Stack:
> 00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
> ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
> 00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
> Call Trace:
> [<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
> [<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
> [<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
> [<ffffffff8104a7c5>] mmput+0x5e/0xd1
> [<ffffffff8110694b>] flush_old_exec+0x526/0x600
> [<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
> [<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
> [<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
> [<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
> [<ffffffff81106195>] search_binary_handler+0xbb/0x24d
> [<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
> [<ffffffff8110785d>] do_execve+0x1c1/0x2ae
> [<ffffffff810119e5>] sys_execve+0x3e/0x55
> [<ffffffff8100af9c>] stub_execve+0x6c/0xc0
> Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4 ff ff 48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb 16 e8 99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
> RIP [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> RSP <ffff88006e697b18>
> CR2: ffff88000e676930
> ---[ end trace 073da74f4ce02315 ]---
> lost connection
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 4747 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 16:29 ` Boris Derzhavets
@ 2010-11-11 16:46 ` Konrad Rzeszutek Wilk
2010-11-11 16:53 ` Boris Derzhavets
0 siblings, 1 reply; 45+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-11-11 16:46 UTC (permalink / raw)
To: Boris Derzhavets; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
On Thu, Nov 11, 2010 at 08:29:38AM -0800, Boris Derzhavets wrote:
> > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
>
> Konrad,
>
> The version is 2.6.37-rc1, been built from upstream source.
I know that. I was asking what version worked for you in the past. As in, did you do
the same tests with older kernels (2.6.36 for example?)
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 16:46 ` Konrad Rzeszutek Wilk
@ 2010-11-11 16:53 ` Boris Derzhavets
2010-11-12 14:40 ` Bruce Edge
0 siblings, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-11 16:53 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 1010 bytes --]
Ubuntu's 10.10 2.6.35-22-generic (or server) works fine for PV DomUs sharing
same image device. I never tested vanilla 2.6.35 (36) itself for PV DomU support.
Boris.
--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 11:46 AM
On Thu, Nov 11, 2010 at 08:29:38AM -0800, Boris Derzhavets wrote:
> > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
>
> Konrad,
>
> The version is 2.6.37-rc1, been built from upstream source.
I know that. I was asking what version worked for you in the past. As in, did you do
the same tests with older kernels (2.6.36 for example?)
[-- Attachment #1.2: Type: text/html, Size: 1410 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 16:53 ` Boris Derzhavets
@ 2010-11-12 14:40 ` Bruce Edge
2010-11-12 16:06 ` Boris Derzhavets
2010-11-12 16:27 ` Sander Eikelenboom
0 siblings, 2 replies; 45+ messages in thread
From: Bruce Edge @ 2010-11-12 14:40 UTC (permalink / raw)
To: Boris Derzhavets; +Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 1529 bytes --]
The git bisecting is slow going. I've never tried that before and I'm a git
rookie.
I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
panic at boot so I'm obviously doing something wrong.
I'll RTFM a bit more and keep at it.
-Bruce
On Thu, Nov 11, 2010 at 8:53 AM, Boris Derzhavets <bderzhavets@yahoo.com>wrote:
> Ubuntu's 10.10 2.6.35-22-generic (or server) works fine for PV DomUs
> sharing
> same image device. I never tested vanilla 2.6.35 (36) itself for PV DomU
> support.
>
>
> Boris.
>
> --- On *Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>*wrote:
>
>
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com,
> "Bruce Edge" <bruce.edge@gmail.com>
> Date: Thursday, November 11, 2010, 11:46 AM
>
>
> On Thu, Nov 11, 2010 at 08:29:38AM -0800, Boris Derzhavets wrote:
> > > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36?
> 2.6.35?
> >
> > Konrad,
> >
> > The version is 2.6.37-rc1, been built from upstream source.
>
> I know that. I was asking what version worked for you in the past. As in,
> did you do
> the same tests with older kernels (2.6.36 for example?)
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
[-- Attachment #1.2: Type: text/html, Size: 2791 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-12 14:40 ` Bruce Edge
@ 2010-11-12 16:06 ` Boris Derzhavets
2010-11-13 8:37 ` Boris Derzhavets
2010-11-12 16:27 ` Sander Eikelenboom
1 sibling, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-12 16:06 UTC (permalink / raw)
To: Bruce Edge; +Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 2563 bytes --]
Bruce,
I just tried 2.6.36 + patch-2.6.36-git13 via patch-2.6.36-git13.bz2 ( i picked it up from MY's first kernel-2.6.37.fc15.rc0.git13.src.rpm ). Kernel 2.6.36-git13 already has a BUG .
So , problem is inside 11 MB patch-2.6.36-git13.bz2 . Might be it gives 37-rc1,
i am not sure. If you want , i can try to send it to you.
Boris.
P.S. Bisecting 36 - 37-rc1 is too much anyway.
--- On Fri, 11/12/10, Bruce Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Friday, November 12, 2010, 9:40 AM
The git bisecting is slow going. I've never tried that before and I'm a git rookie.
I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all panic at boot so I'm obviously doing something wrong.
I'll RTFM a bit more and keep at it.
-Bruce
On Thu, Nov 11, 2010 at 8:53 AM, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
Ubuntu's 10.10 2.6.35-22-generic (or server) works fine for PV DomUs sharing
same image device. I never tested vanilla 2.6.35 (36) itself for PV DomU support.
Boris.
--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 11:46 AM
On Thu, Nov 11, 2010 at 08:29:38AM -0800,
Boris Derzhavets wrote:
> > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
>
> Konrad,
>
> The version is 2.6.37-rc1, been built from upstream source.
I know that. I was asking what version worked for you in the past. As in, did you do
the same tests with older kernels (2.6.36 for example?)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
-----Inline Attachment Follows-----
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 4962 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-12 16:06 ` Boris Derzhavets
@ 2010-11-13 8:37 ` Boris Derzhavets
0 siblings, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-13 8:37 UTC (permalink / raw)
To: Jeremy Fitzhardinge; +Cc: xen-devel, Bruce Edge, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 6697 bytes --]
Same stack trace comes up when i attempted tp "cp" files from NFS folder (at Dom0)
mounted at DomU ( 2.6.37-rc1-git9 kernel )
root@boris-HVM-domU:~# mount 192.168.1.6:/home/boris /mnt/tmp
root@boris-HVM-domU:~# ls -l /mnt/tmp
total 140300
drwxr-xr-x 2 boris boris 4096 2010-10-14 18:29 Desktop
drwxr-xr-x 2 boris boris 4096 2010-10-14 18:29 Documents
drwxr-xr-x 2 boris boris 4096 2010-10-14 18:29 Downloads
-rw-r--r-- 1 boris boris 179 2010-10-14 18:06 examples.desktop
-rw-r--r-- 1 root root 70277083 2010-11-12 21:40 linux-2.6.36.tar.bz2
-rw-r--r-- 1 root root 73335884 2010-11-12 21:42 linux-2.6.37-rc1.tar.bz2
drwxr-xr-x 2 boris boris 4096 2010-10-14 18:29 Music
drwxr-xr-x 2 boris boris 4096 2010-10-14 18:29 Pictures
drwxr-xr-x 2 boris boris 4096 2010-10-14 18:29 Public
drwxr-xr-x 2 root root 4096 2010-11-05 15:12 solaris
drwxr-xr-x 2 boris boris 4096 2010-10-14 18:29 Templates
drwxr-xr-x 2 root root 4096 2010-11-13 11:19 ubuntu
drwxr-xr-x 2 boris boris 4096 2010-10-14 18:29 Videos
root@boris-HVM-domU:~# cp /mnt/tmp/linux-2.6.36.tar.bz2 .
root@boris-HVM-domU:~# cp /mnt/tmp/linux-2.6.37-rc1.tar.bz2 .
BUG: unable to handle kernel paging request at ffff880079b96cc0
IP: [<ffffffff810daa31>] do_wp_page+0x597/0x6fa
PGD 1a04067 PUD 41d5067 PMD 43a3067 PTE 8010000079b96065
Oops: 0003 [#1] SMP
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0
Modules linked in: parport_pc ppdev ipv6 lp parport
Pid: 3509, comm: bash Not tainted 2.6.37-rc1-git9 #1 /
RIP: e030:[<ffffffff810daa31>] [<ffffffff810daa31>] do_wp_page+0x597/0x6fa
RSP: e02b:ffff88007a305cd8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffea0001a0bcd0 RCX: 0000000000000001
RDX: 8000000136d77065 RSI: ffffffff81c47050 RDI: ffff880078adf040
RBP: ffff88007a305d68 R08: 0000000000000100 R09: 0000000000000041
R10: ffff8800798869c0 R11: ffff880078adf040 R12: ffffea0001a1ada8
R13: ffff880079b96cc0 R14: ffff880078adf040 R15: ffff88007a3c8700
FS: 00007f3a44186700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff880079b96cc0 CR3: 00000000798b8000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process bash (pid: 3509, threadinfo ffff88007a304000, task ffff880078f66680)
Stack:
ffff88007a305cf8 ffffffff81007c31 ffffea0001aa08e0 0000000000000000
01ff88007a305d48 ffffea0001aa08e0 8000000136d77067 00007f3a441989a8
00007f3a441869d0 ffff88007a3c8700 ffff88007988d100 ffff880078adf040
Call Trace:
[<ffffffff81007c31>] ? __spin_time_accum+0x21/0x37
[<ffffffff810db54e>] handle_mm_fault+0x9ba/0xa0e
[<ffffffff814cd5dc>] do_page_fault+0x387/0x3a9
[<ffffffff810072af>] ? xen_restore_fl_direct_end+0x0/0x1
[<ffffffff81004274>] ? xen_mc_flush+0x1cd/0x1f2
[<ffffffff81003e5f>] ? xen_clts+0x33/0x59
[<ffffffff81007bf1>] ? xen_spin_unlock+0x11/0x30
[<ffffffff81042d24>] ? finish_task_switch+0x4f/0xc5
[<ffffffff8103c580>] ? __might_sleep+0x23/0xea
[<ffffffff814ca695>] page_fault+0x25/0x30
Code: 48 c1 e7 0c 48 09 c7 ff 14 25 e8 27 a1 81 48 89 c2 48 83 c8 42 49 8b 3f 48 83 ca 40 41 f6 47 30 02 48 0f 44 c2 48 89 45 a0 31 c0 <49> 87 45 00 48 8b 75 a8 4c 89 ea 0f 1f 80 00 00 00 00 48 8b 75
RIP [<ffffffff810daa31>] do_wp_page+0x597/0x6fa
RSP <ffff88007a305cd8>
CR2: ffff880079b96cc0
---[ end trace a83a1add1cfb7333 ]---
--- On Fri, 11/12/10, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Bruce Edge" <bruce.edge@gmail.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Friday, November 12, 2010, 11:06 AM
Bruce,
I just tried 2.6.36 + patch-2.6.36-git13 via patch-2.6.36-git13.bz2 ( i picked it up from MY's first kernel-2.6.37.fc15.rc0.git13.src.rpm ). Kernel 2.6.36-git13 already has a BUG .
So , problem is inside 11 MB patch-2.6.36-git13.bz2 . Might be it gives 37-rc1,
i am not sure. If you want , i can try to send it to you.
Boris.
P.S. Bisecting 36 - 37-rc1 is too much anyway.
--- On Fri, 11/12/10, Bruce Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Friday, November 12, 2010, 9:40 AM
The git bisecting is slow going. I've never tried that before and I'm a git rookie.
I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all panic at boot so I'm obviously doing something wrong.
I'll RTFM a bit more and keep at it.
-Bruce
On Thu, Nov 11, 2010 at 8:53 AM, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
Ubuntu's 10.10 2.6.35-22-generic (or server) works fine for PV DomUs sharing
same image device. I never tested vanilla 2.6.35 (36) itself for PV DomU support.
Boris.
--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 11:46 AM
On Thu, Nov 11, 2010 at 08:29:38AM -0800,
Boris Derzhavets wrote:
> > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
>
> Konrad,
>
> The version is 2.6.37-rc1, been built from upstream source.
I know that. I was asking what version worked for you in the past. As in, did you do
the same tests with older kernels (2.6.36 for example?)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
-----Inline Attachment Follows-----
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
-----Inline Attachment Follows-----
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 9445 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-12 14:40 ` Bruce Edge
2010-11-12 16:06 ` Boris Derzhavets
@ 2010-11-12 16:27 ` Sander Eikelenboom
2010-11-12 17:01 ` Konrad Rzeszutek Wilk
1 sibling, 1 reply; 45+ messages in thread
From: Sander Eikelenboom @ 2010-11-12 16:27 UTC (permalink / raw)
To: Bruce Edge
Cc: Boris Derzhavets, xen-devel, Jeremy Fitzhardinge,
Konrad Rzeszutek Wilk
Hi Bruce,
Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
--
Sander
Friday, November 12, 2010, 3:40:11 PM, you wrote:
> The git bisecting is slow going. I've never tried that before and I'm a git
> rookie.
> I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
> panic at boot so I'm obviously doing something wrong.
> I'll RTFM a bit more and keep at it.
> -Bruce
> On Thu, Nov 11, 2010 at 8:53 AM, Boris Derzhavets <bderzhavets@yahoo.com>wrote:
>> Ubuntu's 10.10 2.6.35-22-generic (or server) works fine for PV DomUs
>> sharing
>> same image device. I never tested vanilla 2.6.35 (36) itself for PV DomU
>> support.
>>
>>
>> Boris.
>>
>> --- On *Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>*wrote:
>>
>>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
>> handle kernel paging request
>> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
>> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com,
>> "Bruce Edge" <bruce.edge@gmail.com>
>> Date: Thursday, November 11, 2010, 11:46 AM
>>
>>
>> On Thu, Nov 11, 2010 at 08:29:38AM -0800, Boris Derzhavets wrote:
>> > > Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36?
>> 2.6.35?
>> >
>> > Konrad,
>> >
>> > The version is 2.6.37-rc1, been built from upstream source.
>>
>> I know that. I was asking what version worked for you in the past. As in,
>> did you do
>> the same tests with older kernels (2.6.36 for example?)
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
--
Best regards,
Sander mailto:linux@eikelenboom.it
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-12 16:27 ` Sander Eikelenboom
@ 2010-11-12 17:01 ` Konrad Rzeszutek Wilk
2010-11-14 16:37 ` Boris Derzhavets
0 siblings, 1 reply; 45+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-11-12 17:01 UTC (permalink / raw)
To: Sander Eikelenboom
Cc: Boris Derzhavets, xen-devel, Bruce Edge, Jeremy Fitzhardinge
On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
> Hi Bruce,
>
> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
But that is the theory and ..
> > The git bisecting is slow going. I've never tried that before and I'm a git
> > rookie.
> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
> > panic at boot so I'm obviously doing something wrong.
> > I'll RTFM a bit more and keep at it.
.. as Bruce experiences this is not the case. Hmm..
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-12 17:01 ` Konrad Rzeszutek Wilk
@ 2010-11-14 16:37 ` Boris Derzhavets
2010-11-14 16:56 ` Sander Eikelenboom
0 siblings, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-14 16:37 UTC (permalink / raw)
To: Sander Eikelenboom, Konrad Rzeszutek Wilk
Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 2015 bytes --]
I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
Boris.
--- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
Date: Friday, November 12, 2010, 12:01 PM
On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
> Hi Bruce,
>
> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
But that is the theory and ..
> > The git bisecting is slow going. I've never tried that before and I'm a git
> > rookie.
> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
> > panic at boot so I'm obviously doing something wrong.
> > I'll RTFM a bit more and keep at it.
.. as Bruce experiences this is not the case. Hmm..
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 2641 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-14 16:37 ` Boris Derzhavets
@ 2010-11-14 16:56 ` Sander Eikelenboom
2010-11-14 17:09 ` Boris Derzhavets
` (2 more replies)
0 siblings, 3 replies; 45+ messages in thread
From: Sander Eikelenboom @ 2010-11-14 16:56 UTC (permalink / raw)
To: Boris Derzhavets
Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge, Konrad Rzeszutek Wilk
Hmmm have you tried do do a lot of I/O with something else as NFS ?
That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
--
Sander
Sunday, November 14, 2010, 5:37:59 PM, you wrote:
> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
> Boris.
> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
> To: "Sander Eikelenboom" <linux@eikelenboom.it>
> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
> Date: Friday, November 12, 2010, 12:01 PM
> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>> Hi Bruce,
>>
>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
> But that is the theory and ..
>> > The git bisecting is slow going. I've never tried that before and I'm a git
>> > rookie.
>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>> > panic at boot so I'm obviously doing something wrong.
>> > I'll RTFM a bit more and keep at it.
> .. as Bruce experiences this is not the case. Hmm..
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
--
Best regards,
Sander mailto:linux@eikelenboom.it
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-14 16:56 ` Sander Eikelenboom
@ 2010-11-14 17:09 ` Boris Derzhavets
2010-11-14 17:19 ` Sander Eikelenboom
2010-11-14 21:35 ` Bruce Edge
2010-11-15 20:21 ` Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request Bruce Edge
2 siblings, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-14 17:09 UTC (permalink / raw)
To: Sander Eikelenboom
Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 3967 bytes --]
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
I've tried scp a lot from DomU to Dom0 on F14 no problems, on Ubuntu 10.04 Server
DomU as NFS client cannot survive more then a 2-3 min with I/O generated by scp.
If i unmount NFS share everything goes fine ( Ubuntu 10.04)
> That would perhaps pinpoint it to NFS doing something not completely compatible with
Xen.
If DomU ( Ubuntu 10.04 ) is running by 2.6.36 kernel there is no any NFS or other network related problems . Issue is specific for 2.6.37-rc1 kernel as kernel running DomU at Xen 4.0.1 Dom0 ( 2.6.32.25 pvops) on top of Ubuntu Lucid Server.
Boris.
-- On Sun, 11/14/10, Sander Eikelenboom <linux@eikelenboom.it> wrote:
From: Sander Eikelenboom <linux@eikelenboom.it>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
Date: Sunday, November 14, 2010, 11:56 AM
Hmmm have you tried do do a lot of I/O with something else as NFS ?
That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
--
Sander
Sunday, November 14, 2010, 5:37:59 PM, you wrote:
> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
> Boris.
> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
> To: "Sander Eikelenboom" <linux@eikelenboom.it>
> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
> Date: Friday, November 12, 2010, 12:01 PM
> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>> Hi Bruce,
>>
>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
> But that is the theory and ..
>> > The git bisecting is slow going. I've never tried that before and I'm a git
>> > rookie.
>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>> > panic at boot so I'm obviously doing something wrong.
>> > I'll RTFM a bit more and keep at it.
> .. as Bruce experiences this is not the case. Hmm..
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
--
Best regards,
Sander mailto:linux@eikelenboom.it
[-- Attachment #1.2: Type: text/html, Size: 5613 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-14 17:09 ` Boris Derzhavets
@ 2010-11-14 17:19 ` Sander Eikelenboom
0 siblings, 0 replies; 45+ messages in thread
From: Sander Eikelenboom @ 2010-11-14 17:19 UTC (permalink / raw)
To: Boris Derzhavets
Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge, Konrad Rzeszutek Wilk
Perhaps these patches (especially Stefano's) could make a difference ?
http://lkml.indiana.edu/hypermail/linux/kernel/1011.1/02348.html
I don't think these are applied to your domU kernel since it's git8, it worth the shot i guess...
--
Sander
Sunday, November 14, 2010, 6:09:14 PM, you wrote:
>> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> I've tried scp a lot from DomU to Dom0 on F14 no problems, on Ubuntu 10.04 Server
> DomU as NFS client cannot survive more then a 2-3 min with I/O generated by scp.
> If i unmount NFS share everything goes fine ( Ubuntu 10.04)
>> That would perhaps pinpoint it to NFS doing something not completely compatible with
> Xen.
> If DomU ( Ubuntu 10.04 ) is running by 2.6.36 kernel there is no any NFS or other network related problems . Issue is specific for 2.6.37-rc1 kernel as kernel running DomU at Xen 4.0.1 Dom0 ( 2.6.32.25 pvops) on top of Ubuntu Lucid Server.
> Boris.
> -- On Sun, 11/14/10, Sander Eikelenboom <linux@eikelenboom.it> wrote:
> From: Sander Eikelenboom <linux@eikelenboom.it>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
> Date: Sunday, November 14, 2010, 11:56 AM
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
> --
> Sander
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>> Boris.
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>> But that is the theory and ..
>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>> > panic at boot so I'm obviously doing something wrong.
>>> > I'll RTFM a bit more and keep at it.
>> .. as Bruce experiences this is not the case. Hmm..
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
--
Best regards,
Sander mailto:linux@eikelenboom.it
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-14 16:56 ` Sander Eikelenboom
2010-11-14 17:09 ` Boris Derzhavets
@ 2010-11-14 21:35 ` Bruce Edge
2010-11-15 8:06 ` Boris Derzhavets
` (2 more replies)
2010-11-15 20:21 ` Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request Bruce Edge
2 siblings, 3 replies; 45+ messages in thread
From: Bruce Edge @ 2010-11-14 21:35 UTC (permalink / raw)
To: Sander Eikelenboom
Cc: Boris Derzhavets, xen-devel, Jeremy Fitzhardinge,
Konrad Rzeszutek Wilk
On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting 'task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I'll see if I get the
same problems with glusterfs.
-Bruce
>
> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
>
> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>> > panic at boot so I'm obviously doing something wrong.
>>> > I'll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
> Sander mailto:linux@eikelenboom.it
>
>
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-14 21:35 ` Bruce Edge
@ 2010-11-15 8:06 ` Boris Derzhavets
2010-11-15 11:05 ` Boris Derzhavets
2010-11-15 14:48 ` Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request, nfsd changes for 2.6.37 Boris Derzhavets
2010-11-15 17:14 ` Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request as Dom0 kernel Boris Derzhavets
2 siblings, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-15 8:06 UTC (permalink / raw)
To: Sander Eikelenboom, Bruce Edge
Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 13206 bytes --]
Stack trace on f14 when working with NFS mount
[ 218.984818] ------------[ cut here ]------------
[ 218.984834] kernel BUG at mm/mmap.c:2399!
[ 218.984844] invalid opcode: 0000 [#1] SMP
[ 218.984857] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[ 218.984872] CPU 1
[ 218.984879] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[ 218.985011]
[ 218.985011] Pid: 1566, comm: ls Not tainted 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[ 218.985011] RIP: e030:[<ffffffff8110ada1>] [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[ 218.985011] RSP: e02b:ffff8800774a9e18 EFLAGS: 00010202
[ 218.985011] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0020000000000000
[ 218.985011] RDX: 0000000000100004 RSI: ffff8800770ea1b8 RDI: ffffea0001a00230
[ 218.985011] RBP: ffff8800774a9e48 R08: ffff88007d045108 R09: 000000000000005a
[ 218.985011] R10: ffffffff8100750f R11: ffffea000182b7b0 R12: ffff880077dc6300
[ 218.985011] R13: ffff88007fa1b1e0 R14: ffff880077dc6368 R15: 0000000000000001
[ 218.985011] FS: 00007f4a38dd17c0(0000) GS:ffff88007fa0d000(0000) knlGS:0000000000000000
[ 218.985011] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 218.985011] CR2: 00007f4a380a1940 CR3: 0000000001a03000 CR4: 0000000000002660
[ 218.985011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 218.985011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 218.985011] Process ls (pid: 1566, threadinfo ffff8800774a8000, task ffff880003ca47c0)
[ 218.985011] Stack:
[ 218.985011] 000000000000006b ffff88007fa1b1e0 ffff8800774a9e38 ffff880077dc6300
[ 218.985011] ffff880077dc6440 ffff880003ca4db0 ffff8800774a9e68 ffffffff810505fc
[ 218.985011] ffff880003ca47c0 ffff880077dc6300 ffff8800774a9eb8 ffffffff81056747
[ 218.985011] Call Trace:
[ 218.985011] [<ffffffff810505fc>] mmput+0x65/0xd8
[ 218.985011] [<ffffffff81056747>] exit_mm+0x13e/0x14b
[ 218.985011] [<ffffffff81056976>] do_exit+0x222/0x7c6
[ 218.985011] [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[ 218.985011] [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[ 218.985011] [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67
[ 218.985011] [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[ 218.985011] [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[ 218.985011] [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[ 218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[ 218.985011] RIP [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[ 218.985011] RSP <ffff8800774a9e18>
[ 218.985011] ---[ end trace 99b09fa378e85262 ]---
[ 218.985011] Fixing recursive fault but reboot is needed!
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.984818] ------------[ cut here ]------------
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.984844] invalid opcode: 0000 [#1] SMP
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.984857] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.985011] Stack:
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.985011] Call Trace:
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[ 259.093423] BUG: unable to handle kernel paging request at ffff880077d352a8
[ 259.093441] IP: [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[ 259.093456] PGD 1a04067 PUD 59c9067 PMD 5b88067 PTE 8010000077d35065
[ 259.093472] Oops: 0003 [#2] SMP
[ 259.093481] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[ 259.093493] CPU 1
[ 259.093498] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[ 259.093652]
[ 259.093658] Pid: 1567, comm: abrtd Tainted: G D 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[ 259.093669] RIP: e030:[<ffffffff81037648>] [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[ 259.093683] RSP: e02b:ffff8800770e7bf8 EFLAGS: 00010202
[ 259.093690] RAX: 80000001bf75f101 RBX: ffff880077521400 RCX: 80000001bf75f167
[ 259.093699] RDX: ffff880077d352a8 RSI: 00007fb9b9255ad0 RDI: ffff880077521400
[ 259.093708] RBP: ffff8800770e7c28 R08: 0000000000000001 R09: 1580000000000000
[ 259.093717] R10: ffffffff8100750f R11: ffff880077dc5800 R12: 00007fb9b9255ad0
[ 259.093726] R13: 0000000000000001 R14: ffff880003f2f9f8 R15: ffff880077d352a8
[ 259.093737] FS: 00007fb9b9255800(0000) GS:ffff88007fa0d000(0000) knlGS:0000000000000000
[ 259.093747] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 259.093755] CR2: ffff880077d352a8 CR3: 00000000043c8000 CR4: 0000000000002660
[ 259.093764] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 259.093773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 259.093783] Process abrtd (pid: 1567, threadinfo ffff8800770e6000, task ffff880003d2c7c0)
[ 259.093800] Stack:
[ 259.093807] ffffea00018382b0 0000000000000000 0000000000000034 0000000000000000
[ 259.093829] ffff880077521400 0000000000000000 ffff8800770e7cb8 ffffffff81104a57
[ 259.093851] ffffffff810050a3 ffffffff00000001 ffff880004307e48 ffff8800770e7ca8
[ 259.093873] Call Trace:
[ 259.093885] [<ffffffff81104a57>] do_wp_page+0x241/0x53d
[ 259.093899] [<ffffffff810050a3>] ? xen_pte_val+0x6a/0x6c
[ 259.093911] [<ffffffff81004635>] ? __raw_callee_save_xen_pte_val+0x11/0x1e
[ 259.093926] [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[ 259.093941] [<ffffffff81106491>] ? handle_mm_fault+0x6ea/0x7af
[ 259.093954] [<ffffffff811064e2>] handle_mm_fault+0x73b/0x7af
[ 259.093969] [<ffffffff81073597>] ? down_read_trylock+0x44/0x4e
[ 259.093983] [<ffffffff814b7aa4>] do_page_fault+0x363/0x385
[ 259.093996] [<ffffffff81006f59>] ? xen_force_evtchn_callback+0xd/0xf
[ 259.094011] [<ffffffff81007522>] ? check_events+0x12/0x20
[ 259.094025] [<ffffffff814b3912>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[ 259.094039] [<ffffffff814b4ad5>] page_fault+0x25/0x30
[ 259.094053] [<ffffffff8125403d>] ? __put_user_4+0x1d/0x30
[ 259.094066] [<ffffffff8104bf66>] ? schedule_tail+0x61/0x65
[ 259.094079] [<ffffffff8100abf3>] ret_from_fork+0x13/0x80
[ 259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 48 39 0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb
[ 259.094149] RIP [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[ 259.094149] RSP <ffff8800770e7bf8>
[ 259.094149] CR2: ffff880077d352a8
[ 259.094149] ---[ end trace 99b09fa378e85263 ]---
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.093472] Oops: 0003 [#2] SMP
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.093481] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.093800] Stack:
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.093873] Call Trace:
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 48 39 0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.094149] CR2: ffff880077d352a8
--- On Sun, 11/14/10, Bruce Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Sunday, November 14, 2010, 4:35 PM
On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting 'task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I'll see if I get the
same problems with glusterfs.
-Bruce
>
> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
>
> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>> > panic at boot so I'm obviously doing something wrong.
>>> > I'll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
> Sander mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 16593 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-15 8:06 ` Boris Derzhavets
@ 2010-11-15 11:05 ` Boris Derzhavets
0 siblings, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-15 11:05 UTC (permalink / raw)
To: Sander Eikelenboom, Bruce Edge
Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 15257 bytes --]
Mount with stock kernel in PV DomU - No problems
---------------------------------------------------------------------
Started domain F14PV (id=4)
[ 0.030070] PCI: Fatal: No config space access function found
[ 0.142207] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Fedora release 14 (Laughlin)
Kernel 2.6.35.6-45.fc14.x86_64 on an x86_64 (/dev/hvc0)
fedora14 login: root
Password:
Last login: Mon Nov 15 13:42:03 on hvc0
[root@fedora14 ~]# mount 192.168.1.9:/home/boris /mnt/nfs
[root@fedora14 ~]#
----------------------------------------------------------------------
|
vs
|
Mount with the most recent Michael Young kernel - Crashing kernel
----------------------------------------------------------------------
Fedora release 14 (Laughlin)
Kernel 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 on an x86_64 (/dev/hvc0)
fedora14 login: root
Password: [ 25.825048] eth0: no IPv6 routers present
Last login: Mon Nov 15 13:48:31 on hvc0
[root@fedora14 ~]# mount 192.168.1.9:/home/boris /mnt/nfs
[ 44.240979] FS-Cache: Loaded
[ 44.275659] FS-Cache: Netfs 'nfs' registered for caching
[root@fedora14 ~]#
------------------------------------------------------------------------------
Boris.
--- On Mon, 11/15/10, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>, "Bruce Edge" <bruce.edge@gmail.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Monday, November 15, 2010, 3:06 AM
Stack trace on f14 when working with NFS mount
[ 218.984818] ------------[ cut here ]------------
[ 218.984834] kernel BUG at mm/mmap.c:2399!
[ 218.984844] invalid opcode: 0000 [#1] SMP
[ 218.984857] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[ 218.984872] CPU 1
[ 218.984879] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs
sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[ 218.985011]
[ 218.985011] Pid: 1566, comm: ls Not tainted 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[ 218.985011] RIP: e030:[<ffffffff8110ada1>] [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[ 218.985011] RSP: e02b:ffff8800774a9e18 EFLAGS: 00010202
[ 218.985011] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0020000000000000
[ 218.985011] RDX: 0000000000100004 RSI: ffff8800770ea1b8 RDI: ffffea0001a00230
[ 218.985011] RBP: ffff8800774a9e48 R08: ffff88007d045108 R09: 000000000000005a
[ 218.985011] R10: ffffffff8100750f R11: ffffea000182b7b0 R12: ffff880077dc6300
[ 218.985011] R13: ffff88007fa1b1e0 R14: ffff880077dc6368 R15: 0000000000000001
[ 218.985011] FS:
00007f4a38dd17c0(0000) GS:ffff88007fa0d000(0000) knlGS:0000000000000000
[ 218.985011] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 218.985011] CR2: 00007f4a380a1940 CR3: 0000000001a03000 CR4: 0000000000002660
[ 218.985011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 218.985011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 218.985011] Process ls (pid: 1566, threadinfo ffff8800774a8000, task ffff880003ca47c0)
[ 218.985011] Stack:
[ 218.985011] 000000000000006b ffff88007fa1b1e0 ffff8800774a9e38 ffff880077dc6300
[ 218.985011] ffff880077dc6440 ffff880003ca4db0 ffff8800774a9e68 ffffffff810505fc
[ 218.985011] ffff880003ca47c0 ffff880077dc6300 ffff8800774a9eb8 ffffffff81056747
[ 218.985011] Call Trace:
[ 218.985011] [<ffffffff810505fc>] mmput+0x65/0xd8
[
218.985011] [<ffffffff81056747>] exit_mm+0x13e/0x14b
[ 218.985011] [<ffffffff81056976>] do_exit+0x222/0x7c6
[ 218.985011] [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[ 218.985011] [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[ 218.985011] [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67
[ 218.985011] [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[ 218.985011] [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[ 218.985011] [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[ 218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[ 218.985011] RIP [<ffffffff8110ada1>]
exit_mmap+0x10c/0x119
[ 218.985011] RSP <ffff8800774a9e18>
[ 218.985011] ---[ end trace 99b09fa378e85262 ]---
[ 218.985011] Fixing recursive fault but reboot is needed!
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.984818] ------------[ cut here ]------------
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.984844] invalid opcode: 0000 [#1] SMP
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.984857] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.985011] Stack:
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.985011] Call Trace:
Message from syslogd@fedora14 at Nov 15 11:03:20 ...
kernel:[ 218.985011] Code: 8d 7d 18 e8 c3
8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[ 259.093423] BUG: unable to handle kernel paging request at ffff880077d352a8
[ 259.093441] IP: [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[ 259.093456] PGD 1a04067 PUD 59c9067 PMD 5b88067 PTE 8010000077d35065
[ 259.093472] Oops: 0003 [#2] SMP
[ 259.093481] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[ 259.093493] CPU 1
[ 259.093498] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel
xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[ 259.093652]
[ 259.093658] Pid: 1567, comm: abrtd Tainted: G D 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[ 259.093669] RIP: e030:[<ffffffff81037648>] [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[ 259.093683] RSP: e02b:ffff8800770e7bf8 EFLAGS: 00010202
[ 259.093690] RAX: 80000001bf75f101 RBX: ffff880077521400 RCX: 80000001bf75f167
[ 259.093699] RDX: ffff880077d352a8 RSI: 00007fb9b9255ad0 RDI: ffff880077521400
[ 259.093708] RBP: ffff8800770e7c28 R08:
0000000000000001 R09: 1580000000000000
[ 259.093717] R10: ffffffff8100750f R11: ffff880077dc5800 R12: 00007fb9b9255ad0
[ 259.093726] R13: 0000000000000001 R14: ffff880003f2f9f8 R15: ffff880077d352a8
[ 259.093737] FS: 00007fb9b9255800(0000) GS:ffff88007fa0d000(0000) knlGS:0000000000000000
[ 259.093747] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 259.093755] CR2: ffff880077d352a8 CR3: 00000000043c8000 CR4: 0000000000002660
[ 259.093764] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 259.093773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 259.093783] Process abrtd (pid: 1567, threadinfo ffff8800770e6000, task ffff880003d2c7c0)
[ 259.093800] Stack:
[ 259.093807] ffffea00018382b0 0000000000000000 0000000000000034 0000000000000000
[ 259.093829] ffff880077521400 0000000000000000
ffff8800770e7cb8 ffffffff81104a57
[ 259.093851] ffffffff810050a3 ffffffff00000001 ffff880004307e48 ffff8800770e7ca8
[ 259.093873] Call Trace:
[ 259.093885] [<ffffffff81104a57>] do_wp_page+0x241/0x53d
[ 259.093899] [<ffffffff810050a3>] ? xen_pte_val+0x6a/0x6c
[ 259.093911] [<ffffffff81004635>] ? __raw_callee_save_xen_pte_val+0x11/0x1e
[ 259.093926] [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[ 259.093941] [<ffffffff81106491>] ? handle_mm_fault+0x6ea/0x7af
[ 259.093954] [<ffffffff811064e2>] handle_mm_fault+0x73b/0x7af
[ 259.093969] [<ffffffff81073597>] ? down_read_trylock+0x44/0x4e
[ 259.093983] [<ffffffff814b7aa4>] do_page_fault+0x363/0x385
[ 259.093996] [<ffffffff81006f59>] ? xen_force_evtchn_callback+0xd/0xf
[
259.094011] [<ffffffff81007522>] ? check_events+0x12/0x20
[ 259.094025] [<ffffffff814b3912>] ? trace_hardirqs_off_thunk+0x3a/0x3c
[ 259.094039] [<ffffffff814b4ad5>] page_fault+0x25/0x30
[ 259.094053] [<ffffffff8125403d>] ? __put_user_4+0x1d/0x30
[ 259.094066] [<ffffffff8104bf66>] ? schedule_tail+0x61/0x65
[ 259.094079] [<ffffffff8100abf3>] ret_from_fork+0x13/0x80
[ 259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 48 39 0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb
[ 259.094149] RIP [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[ 259.094149] RSP <ffff8800770e7bf8>
[ 259.094149] CR2: ffff880077d352a8
[ 259.094149] ---[ end trace 99b09fa378e85263
]---
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.093472] Oops: 0003 [#2] SMP
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.093481] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.093800] Stack:
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.093873] Call Trace:
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 48 39 0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb
Message from syslogd@fedora14 at Nov 15 11:04:00 ...
kernel:[ 259.094149] CR2: ffff880077d352a8
--- On Sun, 11/14/10, Bruce
Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Sunday, November 14, 2010, 4:35 PM
On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely
compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting 'task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I'll see if I get the
same problems with glusterfs.
-Bruce
>
> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
>
> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1
kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge"
<jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>> > panic at boot so I'm
obviously doing something wrong.
>>> > I'll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
> Sander mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
-----Inline Attachment Follows-----
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 19043 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request, nfsd changes for 2.6.37
2010-11-14 21:35 ` Bruce Edge
2010-11-15 8:06 ` Boris Derzhavets
@ 2010-11-15 14:48 ` Boris Derzhavets
2010-11-15 17:14 ` Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request as Dom0 kernel Boris Derzhavets
2 siblings, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-15 14:48 UTC (permalink / raw)
To: Sander Eikelenboom, Bruce Edge
Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 4407 bytes --]
My skills are not good enough to decide might it affect Xen or not.
Please, view :-
http://www.spinics.net/lists/linux-nfs/msg16893.html
http://www.spinics.net/lists/linux-nfs/msg17071.html
Boris.
P.S. Finally, i got F14 PV DomU crashing , when running NFS client, as well.
Just get crash on F14 takes longer then on Ubuntu 10.04.1. As soon as it
crashed once it started crashing on regular basis ( looks like a kind of locking
mechanism).
--- On Sun, 11/14/10, Bruce Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>
Date: Sunday, November 14, 2010, 4:35 PM
On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting 'task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I'll see if I get the
same problems with glusterfs.
-Bruce
>
> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
>
> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>> > panic at boot so I'm obviously doing something wrong.
>>> > I'll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
> Sander mailto:linux@eikelenboom.it
>
>
[-- Attachment #1.2: Type: text/html, Size: 6351 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request as Dom0 kernel
2010-11-14 21:35 ` Bruce Edge
2010-11-15 8:06 ` Boris Derzhavets
2010-11-15 14:48 ` Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request, nfsd changes for 2.6.37 Boris Derzhavets
@ 2010-11-15 17:14 ` Boris Derzhavets
2 siblings, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-15 17:14 UTC (permalink / raw)
To: Sander Eikelenboom, Bruce Edge
Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 4514 bytes --]
I've also tested the most recent kernel from Michael Young
http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434]
kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64(rc1.git8 + xen-next.patch)
as Dom0 kernel under Xen 4.0.1 on top of F14.
Attempt to work with NFS remote folder results kernel crash again
- unable to handle paging request and stack trace.
Kernel works OK only as vanilla.
Boris.
--- On Sun, 11/14/10, Bruce Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Date: Sunday, November 14, 2010, 4:35 PM
On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
I have my own suspicions regarding the more recent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS transfers even without
Xen. Any more than a few 100 Megs and you start getting 'task blocked
for more than 120 sec..." messages along with stack traces showing
part of the NFS call stack.
Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with
something other than NFS for remote filesystems. I'll see if I get the
same problems with glusterfs.
-Bruce
>
> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
>
> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>> > panic at boot so I'm obviously doing something wrong.
>>> > I'll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
> Sander mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 6625 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-14 16:56 ` Sander Eikelenboom
2010-11-14 17:09 ` Boris Derzhavets
2010-11-14 21:35 ` Bruce Edge
@ 2010-11-15 20:21 ` Bruce Edge
2010-11-15 20:32 ` Sander Eikelenboom
2010-11-16 18:43 ` Boris Derzhavets
2 siblings, 2 replies; 45+ messages in thread
From: Bruce Edge @ 2010-11-15 20:21 UTC (permalink / raw)
To: Sander Eikelenboom
Cc: Boris Derzhavets, xen-devel, Jeremy Fitzhardinge,
Konrad Rzeszutek Wilk
On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
>
> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
Sander,
I took a quick look at glusterfs and it uses the same nfs client:
http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes_Using_NFS
I'm assuming that this would cause the same problem on the VM as we're
seeing with NFS, unless it's really an Xen/dom0 NFS server problem
that triggering the domU nfs client crash.
Perhaps the context is different. I'm exporting filesystems to domU
from dom0 via NFS. Is that how you're using glusterfs, or are you
using it to host your file backed VM's storage? In the latter, that
may explain why you're not seeing these problems because you're not
using the nfs client on domU.
> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
What kind of NFS problems? It was working very well for us until this
problem cropped up.
-Bruce
>
> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>> > panic at boot so I'm obviously doing something wrong.
>>> > I'll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
> Sander mailto:linux@eikelenboom.it
>
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-15 20:21 ` Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request Bruce Edge
@ 2010-11-15 20:32 ` Sander Eikelenboom
2010-11-16 18:43 ` Boris Derzhavets
1 sibling, 0 replies; 45+ messages in thread
From: Sander Eikelenboom @ 2010-11-15 20:32 UTC (permalink / raw)
To: Bruce Edge
Cc: Boris Derzhavets, xen-devel, Jeremy Fitzhardinge,
Konrad Rzeszutek Wilk
Monday, November 15, 2010, 9:21:28 PM, you wrote:
> On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
> <linux@eikelenboom.it> wrote:
>> Hmmm have you tried do do a lot of I/O with something else as NFS ?
>> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
>>
>> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
> Sander,
> I took a quick look at glusterfs and it uses the same nfs client:
> http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes_Using_NFS
> I'm assuming that this would cause the same problem on the VM as we're
> seeing with NFS, unless it's really an Xen/dom0 NFS server problem
> that triggering the domU nfs client crash.
> Perhaps the context is different. I'm exporting filesystems to domU
> from dom0 via NFS. Is that how you're using glusterfs, or are you
> using it to host your file backed VM's storage? In the latter, that
> may explain why you're not seeing these problems because you're not
> using the nfs client on domU.
Nope it CAN export volumes as NFS, but that's the non native way.
I don't know what distribution you use, but for debian there are packages available http://packages.debian.org/search?keywords=glusterfs&searchon=names&suite=all§ion=all
in the backports repository.
And indeed i don't use NFS anywhere. (I have tried, but had configuration and other problems. So i looked for an alternative.)
--
Sander
>> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
> What kind of NFS problems? It was working very well for us until this
> problem cropped up.
> -Bruce
>>
>> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>>
>> --
>>
>> Sander
>>
>>
>>
>>
>>
>> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>>
>>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>>
>>> Boris.
>>
>>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>>
>>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
>>> Date: Friday, November 12, 2010, 12:01 PM
>>
>>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>>> Hi Bruce,
>>>>
>>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>>
>>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>>
>>> But that is the theory and ..
>>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>>> > rookie.
>>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>>> > panic at boot so I'm obviously doing something wrong.
>>>> > I'll RTFM a bit more and keep at it.
>>
>>> .. as Bruce experiences this is not the case. Hmm..
>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>
>>
>>
>>>
>>
>>
>>
>> --
>> Best regards,
>> Sander mailto:linux@eikelenboom.it
>>
>>
--
Best regards,
Sander mailto:linux@eikelenboom.it
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-15 20:21 ` Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request Bruce Edge
2010-11-15 20:32 ` Sander Eikelenboom
@ 2010-11-16 18:43 ` Boris Derzhavets
2010-11-16 19:00 ` Konrad Rzeszutek Wilk
1 sibling, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-16 18:43 UTC (permalink / raw)
To: Bruce Edge, Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 5264 bytes --]
Today i've built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
via Michael's http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.
# Xen patches
ApplyPatch xen.next-2.6.37.patch
# ApplyPatch xen.upstream.core.patch
ApplyPatch xen.pcifront.fixes.patch
# ApplyPatch xen.pvhvm.fixes.patch
as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1
F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
I was able several times copied from NFS folder F14's ISO image (3.2 GB)
to DomU and scp'ed it back and didn't get any kernel crashing on DomU.
On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched via
xen.next-2.6.37.patch
xen.pcifront.fixes.patch
All required upstream patches may be taken (as well as 2 above)
from http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
I believe as soon as xen.pcifront.fixes.patch will be accepted by upstream
NFS client issue on F14 will be gone
Boris
--- On Mon, 11/15/10, Bruce Edge <bruce.edge@gmail.com> wrote:
From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>, xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>
Date: Monday, November 15, 2010, 3:21 PM
On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom
<linux@eikelenboom.it> wrote:
> Hmmm have you tried do do a lot of I/O with something else as NFS ?
> That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.
>
> I'm not using NFS (I still use file: based guests, and i use glusterfs (fuse based userspace cluster fs) to share diskspace to domU's via ethernet).
Sander,
I took a quick look at glusterfs and it uses the same nfs client:
http://www.gluster.com/community/documentation/index.php/Gluster_3.1:_Manually_Mounting_Volumes_Using_NFS
I'm assuming that this would cause the same problem on the VM as we're
seeing with NFS, unless it's really an Xen/dom0 NFS server problem
that triggering the domU nfs client crash.
Perhaps the context is different. I'm exporting filesystems to domU
from dom0 via NFS. Is that how you're using glusterfs, or are you
using it to host your file backed VM's storage? In the latter, that
may explain why you're not seeing these problems because you're not
using the nfs client on domU.
> I tried NFS in the past, but had some troubles setting it up, and even more problems with disconnects.
What kind of NFS problems? It was working very well for us until this
problem cropped up.
-Bruce
>
> I haven't seen any "unable to handle page request" problems with my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:
>
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable to handle page request" was reported once by F14 DomU, but it didn't actually crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash in about several minutes. Dom0's instances dual boot on same development box ( Q9500,ASUS P5Q3,8GB)
>
>> Boris.
>
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
>> To: "Sander Eikelenboom" <linux@eikelenboom.it>
>> Cc: "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>, "Jeremy Fitzhardinge" <jeremy@goop.org>
>> Date: Friday, November 12, 2010, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>>> Perhaps handpick some kernels before and after the pulls of the xen patches (pv-on-hvm etc) to begin with ?
>>> When you let git choose, especially with rc-1 kernels, you will end up with kernels in between patch series, resulting in panics.
>
>> Well, just the bare-bone boot of PV guests with nothing fancy ought to work.
>
>> But that is the theory and ..
>>> > The git bisecting is slow going. I've never tried that before and I'm a git
>>> > rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisects all
>>> > panic at boot so I'm obviously doing something wrong.
>>> > I'll RTFM a bit more and keep at it.
>
>> .. as Bruce experiences this is not the case. Hmm..
>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>>
>
>
>
> --
> Best regards,
> Sander mailto:linux@eikelenboom.it
>
>
[-- Attachment #1.2: Type: text/html, Size: 7411 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-16 18:43 ` Boris Derzhavets
@ 2010-11-16 19:00 ` Konrad Rzeszutek Wilk
2010-11-16 20:43 ` Boris Derzhavets
2010-11-16 20:50 ` Boris Derzhavets
0 siblings, 2 replies; 45+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-11-16 19:00 UTC (permalink / raw)
To: Boris Derzhavets; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
On Tue, Nov 16, 2010 at 10:43:49AM -0800, Boris Derzhavets wrote:
> Today i've built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
> via Michael's http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.
>
> # Xen patches
> ApplyPatch xen.next-2.6.37.patch
> # ApplyPatch xen.upstream.core.patch
> ApplyPatch xen.pcifront.fixes.patch
> # ApplyPatch xen.pvhvm.fixes.patch
>
> as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1
> F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
>
> I was able several times copied from NFS folder F14's ISO image (3.2 GB)
> to DomU and scp'ed it back and didn't get any kernel crashing on DomU.
Huh. I .. what? I am confused. I thought we established that the issue
was not related to Xen PCI front? You also seem to uncomment the
upstream.core.patches and the xen.pvhvm.patch - why?
>
> On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched via
> xen.next-2.6.37.patch
> xen.pcifront.fixes.patch
> All required upstream patches may be taken (as well as 2 above)
> from http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> I believe as soon as xen.pcifront.fixes.patch will be accepted by upstream
> NFS client issue on F14 will be gone
Ok, they are.. v2.6.37-rc2 which came out today has the fixes.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-16 19:00 ` Konrad Rzeszutek Wilk
@ 2010-11-16 20:43 ` Boris Derzhavets
2010-11-16 21:15 ` Konrad Rzeszutek Wilk
2010-11-16 20:50 ` Boris Derzhavets
1 sibling, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-16 20:43 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 2712 bytes --]
> Huh. I .. what? I am confused. I thought we established that the issue
> was not related to Xen PCI front? You also seem to uncomment the
> upstream.core.patches and the xen.pvhvm.patch - why?
I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
it gives failed HUNKs
> Ok, they are.. v2.6.37-rc2 which came out today has the fixes
I am pretty sure rc2 doesn't contain everything from xen.next-2.6.37.patch,
gntdev's stuff for sure. I've built 2.6.37-rc2 kernel rpms and loaded
kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
Device /dev/xen/gntdev has not been created. I understand that it's
unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i cannot
get 3.2 GB copied over to DomU from NFS share at Dom0.
The best results for 2.6.37-rc1 + xen.next-2.6.37.patch + xen.pcifront.fixes.patch
Boris.
--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Bruce Edge" <bruce.edge@gmail.com>, xen-devel@lists.xensource.com, "Jeremy Fitzhardinge" <jeremy@goop.org>
Date: Tuesday, November 16, 2010, 2:00 PM
On Tue, Nov 16, 2010 at 10:43:49AM -0800, Boris Derzhavets wrote:
> Today i've built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
> via Michael's http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.
>
> # Xen patches
> ApplyPatch xen.next-2.6.37.patch
> # ApplyPatch xen.upstream.core.patch
> ApplyPatch xen.pcifront.fixes.patch
> # ApplyPatch xen.pvhvm.fixes.patch
>
> as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1
> F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
>
> I was able several times copied from NFS folder F14's ISO image (3.2 GB)
> to DomU and scp'ed it back and didn't get any kernel crashing on DomU.
Huh. I .. what? I am confused. I thought we established that the issue
was not related to Xen PCI front? You also seem to uncomment the
upstream.core.patches and the xen.pvhvm.patch - why?
>
> On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched via
> xen.next-2.6.37.patch
> xen.pcifront.fixes.patch
> All required upstream patches may be taken (as well as 2 above)
> from http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> I believe as soon as xen.pcifront.fixes.patch will be accepted by upstream
> NFS client issue on F14 will be gone
Ok, they are.. v2.6.37-rc2 which came out today has the fixes.
[-- Attachment #1.2: Type: text/html, Size: 3425 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-16 20:43 ` Boris Derzhavets
@ 2010-11-16 21:15 ` Konrad Rzeszutek Wilk
2010-11-16 21:42 ` Boris Derzhavets
2010-11-16 21:49 ` Boris Derzhavets
0 siblings, 2 replies; 45+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-11-16 21:15 UTC (permalink / raw)
To: Boris Derzhavets; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch - why?
>
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.
>
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
>
> I am pretty sure rc2 doesn't contain everything from xen.next-2.6.37.patch,
> gntdev's stuff for sure. I've built 2.6.37-rc2 kernel rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> Device /dev/xen/gntdev has not been created. I understand that it's
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.
So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-16 21:15 ` Konrad Rzeszutek Wilk
@ 2010-11-16 21:42 ` Boris Derzhavets
2010-11-16 21:49 ` Boris Derzhavets
1 sibling, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-16 21:42 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 1776 bytes --]
> So what I think you are saying is that you keep on getting the bug in DomU?
> Is the stack-trace the same as in rc1?
Yes.
When i want to get 1-2 hr of stable work :-
# service network restart
# service nfs restart
at Dom0.
I also believe that presence of xen-pcifront.fix.patch is making things much more stable
on F14.
Boris.
--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Tuesday, November 16, 2010, 4:15 PM
On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch - why?
>
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.
>
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
>
> I am pretty sure rc2 doesn't contain everything from xen.next-2.6.37.patch,
> gntdev's stuff for sure. I've built 2.6.37-rc2 kernel rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> Device /dev/xen/gntdev has not been created. I understand that it's
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.
So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?
[-- Attachment #1.2: Type: text/html, Size: 2283 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-16 21:15 ` Konrad Rzeszutek Wilk
2010-11-16 21:42 ` Boris Derzhavets
@ 2010-11-16 21:49 ` Boris Derzhavets
2010-11-17 21:28 ` Bruce Edge
1 sibling, 1 reply; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-16 21:49 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 5691 bytes --]
Yes, here we are
[ 186.975228] ------------[ cut here ]------------
[ 186.975245] kernel BUG at mm/mmap.c:2399!
[ 186.975254] invalid opcode: 0000 [#1] SMP
[ 186.975269] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[ 186.975284] CPU 0
[ 186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan]
[ 186.975507]
[ 186.975515] Pid: 1562, comm: ls Not tainted 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[ 186.975529] RIP: e030:[<ffffffff8110ada1>] [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[ 186.975550] RSP: e02b:ffff8800781bde18 EFLAGS: 00010202
[ 186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[ 186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI: ffffea00000c0280
[ 186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09: 0000000000000001
[ 186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12: ffff880076c68b00
[ 186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15: 0000000000000001
[ 186.975625] FS: 00007f8e471d97c0(0000) GS:ffff88007f831000(0000) knlGS:0000000000000000
[ 186.975639] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4: 0000000000002660
[ 186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task ffff8800788223e0)
[ 186.976012] Stack:
[ 186.976012] 000000000000006b ffff88007f83f1e0 ffff8800781bde38 ffff880076c68b00
[ 186.976012] ffff880076c68c40 ffff8800788229d0 ffff8800781bde68 ffffffff810505fc
[ 186.976012] ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8 ffffffff81056747
[ 186.976012] Call Trace:
[ 186.976012] [<ffffffff810505fc>] mmput+0x65/0xd8
[ 186.976012] [<ffffffff81056747>] exit_mm+0x13e/0x14b
[ 186.976012] [<ffffffff81056976>] do_exit+0x222/0x7c6
[ 186.976012] [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[ 186.976012] [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
[ 186.976012] [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67
[ 186.976012] [<ffffffff810571b0>] do_group_exit+0x88/0xb6
[ 186.976012] [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
[ 186.976012] [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[ 186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[ 186.976012] RIP [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[ 186.976012] RSP <ffff8800781bde18>
[ 186.976012] ---[ end trace c0f4eff4054a67e4 ]---
[ 186.976012] Fixing recursive fault but reboot is needed!
Message from syslogd@fedora14 at Nov 17 00:47:40 ...
kernel:[ 186.975228] ------------[ cut here ]------------
Message from syslogd@fedora14 at Nov 17 00:47:40 ...
kernel:[ 186.975254] invalid opcode: 0000 [#1] SMP
Message from syslogd@fedora14 at Nov 17 00:47:40 ...
kernel:[ 186.975269] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
Message from syslogd@fedora14 at Nov 17 00:47:40 ...
kernel:[ 186.976012] Stack:
Message from syslogd@fedora14 at Nov 17 00:47:40 ...
kernel:[ 186.976012] Call Trace:
Message from syslogd@fedora14 at Nov 17 00:47:40 ...
kernel:[ 186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Tuesday, November 16, 2010, 4:15 PM
On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > Huh. I .. what? I am confused. I thought we established that the issue
> > was not related to Xen PCI front? You also seem to uncomment the
> > upstream.core.patches and the xen.pvhvm.patch - why?
>
> I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> it gives failed HUNKs
Uhh.. I am even more confused.
>
> > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
>
> I am pretty sure rc2 doesn't contain everything from xen.next-2.6.37.patch,
> gntdev's stuff for sure. I've built 2.6.37-rc2 kernel rpms and loaded
> kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> Device /dev/xen/gntdev has not been created. I understand that it's
> unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i cannot
> get 3.2 GB copied over to DomU from NFS share at Dom0.
So what I think you are saying is that you keep on getting the bug in DomU?
Is the stack-trace the same as in rc1?
[-- Attachment #1.2: Type: text/html, Size: 6681 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-16 21:49 ` Boris Derzhavets
@ 2010-11-17 21:28 ` Bruce Edge
0 siblings, 0 replies; 45+ messages in thread
From: Bruce Edge @ 2010-11-17 21:28 UTC (permalink / raw)
To: Boris Derzhavets; +Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 6184 bytes --]
On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets <bderzhavets@yahoo.com>wrote:
> Yes, here we are
>
> [ 186.975228] ------------[ cut here ]------------
> [ 186.975245] kernel BUG at mm/mmap.c:2399!
> [ 186.975254] invalid opcode: 0000 [#1] SMP
> [ 186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
> [ 186.975284] CPU 0
> [ 186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr
> camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic
> twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic
> des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet
> xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport
> xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp
> ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl
> auth_rpcgss exportfs sunrpc ipv6 uinput xen_netfront microcode xen_blkfront
> [last unloaded: scsi_wait_scan]
> [ 186.975507]
> [ 186.975515] Pid: 1562, comm: ls Not tainted
> 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
> [ 186.975529] RIP: e030:[<ffffffff8110ada1>] [<ffffffff8110ada1>]
> exit_mmap+0x10c/0x119
> [ 186.975550] RSP: e02b:ffff8800781bde18 EFLAGS: 00010202
> [ 186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000000
> [ 186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI:
> ffffea00000c0280
> [ 186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09:
> 0000000000000001
> [ 186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12:
> ffff880076c68b00
> [ 186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15:
> 0000000000000001
> [ 186.975625] FS: 00007f8e471d97c0(0000) GS:ffff88007f831000(0000)
> knlGS:0000000000000000
> [ 186.975639] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4:
> 0000000000002660
> [ 186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [ 186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [ 186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task
> ffff8800788223e0)
> [ 186.976012] Stack:
> [ 186.976012] 000000000000006b ffff88007f83f1e0 ffff8800781bde38
> ffff880076c68b00
> [ 186.976012] ffff880076c68c40 ffff8800788229d0 ffff8800781bde68
> ffffffff810505fc
> [ 186.976012] ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8
> ffffffff81056747
> [ 186.976012] Call Trace:
> [ 186.976012] [<ffffffff810505fc>] mmput+0x65/0xd8
> [ 186.976012] [<ffffffff81056747>] exit_mm+0x13e/0x14b
> [ 186.976012] [<ffffffff81056976>] do_exit+0x222/0x7c6
> [ 186.976012] [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
> [ 186.976012] [<ffffffff8107ea7c>] ? arch_local_irq_restore+0xb/0xd
> [ 186.976012] [<ffffffff814b3949>] ? lockdep_sys_exit_thunk+0x35/0x67
> [ 186.976012] [<ffffffff810571b0>] do_group_exit+0x88/0xb6
> [ 186.976012] [<ffffffff810571f5>] sys_exit_group+0x17/0x1b
> [ 186.976012] [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
> [ 186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89
> df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02
> <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
> [ 186.976012] RIP [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
> [ 186.976012] RSP <ffff8800781bde18>
> [ 186.976012] ---[ end trace c0f4eff4054a67e4 ]---
> [ 186.976012] Fixing recursive fault but reboot is needed!
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
> kernel:[ 186.975228] ------------[ cut here ]------------
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
> kernel:[ 186.975254] invalid opcode: 0000 [#1] SMP
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
> kernel:[ 186.975269] last sysfs file:
> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
> kernel:[ 186.976012] Stack:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
> kernel:[ 186.976012] Call Trace:
>
> Message from syslogd@fedora14 at Nov 17 00:47:40 ...
> kernel:[ 186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00
> 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00
> 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
>
> --- On *Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>*wrote:
>
>
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to
> handle kernel paging request
> To: "Boris Derzhavets" <bderzhavets@yahoo.com>
> Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com,
> "Bruce Edge" <bruce.edge@gmail.com>
> Date: Tuesday, November 16, 2010, 4:15 PM
>
> On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote:
> > > Huh. I .. what? I am confused. I thought we established that the issue
> > > was not related to Xen PCI front? You also seem to uncomment the
> > > upstream.core.patches and the xen.pvhvm.patch - why?
> >
> > I cannot uncomment upstream.core.patches and the xen.pvhvm.patch
> > it gives failed HUNKs
>
> Uhh.. I am even more confused.
> >
> > > Ok, they are.. v2.6.37-rc2 which came out today has the fixes
> >
> > I am pretty sure rc2 doesn't contain everything from
> xen.next-2.6.37.patch,
> > gntdev's stuff for sure. I've built 2.6.37-rc2 kernel rpms and loaded
> > kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1.
> > Device /dev/xen/gntdev has not been created. I understand that it's
> > unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i
> cannot
> > get 3.2 GB copied over to DomU from NFS share at Dom0.
>
> So what I think you are saying is that you keep on getting the bug in DomU?
> Is the stack-trace the same as in rc1?
>
>
>
I haven't had much time to look into the broken/working version issues here,
but I did confirm a couple of points:
1) The 2.6.37-rc2 has the same problem still
2) This problem goes away of one is not using NFS.
Not staggeringly helpful I know, but it's one small data point.
-Bruce
[-- Attachment #1.2: Type: text/html, Size: 7510 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-16 19:00 ` Konrad Rzeszutek Wilk
2010-11-16 20:43 ` Boris Derzhavets
@ 2010-11-16 20:50 ` Boris Derzhavets
1 sibling, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-16 20:50 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 2116 bytes --]
Sorry, typo. Correct :-
The best results for 2.6.37-rc1-git8 + xen.next-2.6.37.patch +
xen.pcifront.fixes.patch
Boris.
--- On Tue, 11/16/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Tuesday, November 16, 2010, 2:00 PM
On Tue, Nov 16, 2010 at 10:43:49AM -0800, Boris Derzhavets wrote:
> Today i've built vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64
> via Michael's http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> and uncommented xen.pcifront.fixes.patch in kernel.spec, i.e.
>
> # Xen patches
> ApplyPatch xen.next-2.6.37.patch
> # ApplyPatch xen.upstream.core.patch
> ApplyPatch xen.pcifront.fixes.patch
> # ApplyPatch xen.pvhvm.fixes.patch
>
> as a fesult a got kernel wich runs pretty stable NFS client at Xen 4.0.1
> F14 Dom0 (2.6.32.25-172.xendom0.fc14.x86_64).
>
> I was able several times copied from NFS folder F14's ISO image (3.2 GB)
> to DomU and scp'ed it back and didn't get any kernel crashing on DomU.
Huh. I .. what? I am confused. I thought we established that the issue
was not related to Xen PCI front? You also seem to uncomment the
upstream.core.patches and the xen.pvhvm.patch - why?
>
> On Ubuntu 10.04 this kernel may be built as 2.6.37-rc1&git8 patched via
> xen.next-2.6.37.patch
> xen.pcifront.fixes.patch
> All required upstream patches may be taken (as well as 2 above)
> from http://koji.fedoraproject.org/koji/taskinfo?taskID=2598434
> I believe as soon as xen.pcifront.fixes.patch will be accepted by upstream
> NFS client issue on F14 will be gone
Ok, they are.. v2.6.37-rc2 which came out today has the fixes.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 2968 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 16:09 ` Konrad Rzeszutek Wilk
2010-11-11 16:29 ` Boris Derzhavets
@ 2010-11-11 16:46 ` Boris Derzhavets
1 sibling, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-11 16:46 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Jeremy Fitzhardinge, xen-devel, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 6168 bytes --]
This time to reproduce crash `ls -l` was not enough , but similar stack trace comes up sooner or later :
root@boris-HVM-domU:~# mount 192.168.1.9:/home/boris /mnt/tmp
. . . . . .
CRASH :-
root@boris-HVM-domU:~# cp /mnt/tmp/*.iso .
BUG: unable to handle kernel paging request at ffff88006f3824f0
IP: [<ffffffff810dd0c3>] do_wp_page+0x597/0x6fa
PGD 1a04067 PUD f57b067 PMD f6f5067 PTE 801000006f382065
Oops: 0003 [#1] SMP
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0
Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ipv6 lp parport
Pid: 3533, comm: bash Not tainted 2.6.37-rc1 #1 /
RIP: e030:[<ffffffff810dd0c3>] [<ffffffff810dd0c3>] do_wp_page+0x597/0x6fa
RSP: e02b:ffff880077b25cd8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffea000034dcf8 RCX: 0000000000000001
RDX: 8000000162e88065 RSI: ffffffff81c3d050 RDI: ffff88006d3e6100
RBP: ffff880077b25d68 R08: 0000000000000100 R09: 0000000000000041
R10: ffff880077b25a78 R11: ffff88006d3e6100 R12: ffffea0000277030
R13: ffff88006f3824f0 R14: ffff88006d3e6100 R15: ffff88006f0b34d8
FS: 00007fc18508b700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff88006f3824f0 CR3: 000000007a103000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process bash (pid: 3533, threadinfo ffff880077b24000, task ffff88006f0ee2c0)
Stack:
ffff880077b25cf8 ffffffff81007c31 ffffea0001854480 0000000000000000
01ff880077b25d48 ffffea0001854480 8000000162e88067 00007fc18509e9a8
00007fc18508b9d0 ffff88006f0b34d8 ffff88006f3a2140 ffff88006d3e6100
Call Trace:
[<ffffffff81007c31>] ? __spin_time_accum+0x21/0x37
[<ffffffff810ddbd7>] handle_mm_fault+0x9b1/0xa05
[<ffffffff8145665a>] do_page_fault+0x387/0x3a9
[<ffffffff810072af>] ? xen_restore_fl_direct_end+0x0/0x1
[<ffffffff81004274>] ? xen_mc_flush+0x1cd/0x1f2
[<ffffffff81003e5f>] ? xen_clts+0x33/0x59
[<ffffffff81007bf1>] ? xen_spin_unlock+0x11/0x30
[<ffffffff81045a6f>] ? finish_task_switch+0x4f/0xc5
[<ffffffff8103d0a9>] ? __might_sleep+0x23/0xea
[<ffffffff81453715>] page_fault+0x25/0x30
Code: 48 c1 e7 0c 48 09 c7 ff 14 25 28 28 a1 81 48 89 c2 48 83 c8 42 49 8b 3f 48 83 ca 40 41 f6 47 30 02 48 0f 44 c2 48 89 45 a0 31 c0 <49> 87 45 00 48 8b 75 a8 4c 89 ea 0f 1f 80 00 00 00 00 48 8b 75
RIP [<ffffffff810dd0c3>] do_wp_page+0x597/0x6fa
RSP <ffff880077b25cd8>
CR2: ffff88006f3824f0
---[ end trace 69a76296779698ff ]---
Boris.
--- On Thu, 11/11/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>, xen-devel@lists.xensource.com, "Bruce Edge" <bruce.edge@gmail.com>
Date: Thursday, November 11, 2010, 11:09 AM
On Thu, Nov 11, 2010 at 04:08:03AM -0800, Boris Derzhavets wrote:
> Now reproduced :-
>
> root@boris-HVM-domU:~# scp 1.log 192.168.1.12:/home/boris/ubuntu
> BUG: unable to handle kernel paging request at ffff88000e676930
> IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
> Oops: 0003 [#1] SMP
> last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
> CPU 0
> Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ipv6 lp parport
>
> Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
Yikes. So no PCI passthrough, just a normal PV DomU.
Boris, what version of Linux kernel worked for you as PV DomU? 2.6.36? 2.6.35?
> RIP: e030:[<ffffffff810069c3>] [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> RSP: e02b:ffff88006e697b18 EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
> RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
> RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
> R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
> R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
> FS: 00007fa525896700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
> CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
> Stack:
> 00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
> ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
> 00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
> Call Trace:
> [<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
> [<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
> [<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
> [<ffffffff8104a7c5>] mmput+0x5e/0xd1
> [<ffffffff8110694b>] flush_old_exec+0x526/0x600
> [<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
> [<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
> [<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
> [<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
> [<ffffffff81106195>] search_binary_handler+0xbb/0x24d
> [<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
> [<ffffffff8110785d>] do_execve+0x1c1/0x2ae
> [<ffffffff810119e5>] sys_execve+0x3e/0x55
> [<ffffffff8100af9c>] stub_execve+0x6c/0xc0
> Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4 ff ff 48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb 16 e8 99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
> RIP [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
> RSP <ffff88006e697b18>
> CR2: ffff88000e676930
> ---[ end trace 073da74f4ce02315 ]---
> lost connection
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
[-- Attachment #1.2: Type: text/html, Size: 7521 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-11 12:01 ` Boris Derzhavets
2010-11-11 12:08 ` Boris Derzhavets
@ 2010-11-11 12:26 ` Boris Derzhavets
1 sibling, 0 replies; 45+ messages in thread
From: Boris Derzhavets @ 2010-11-11 12:26 UTC (permalink / raw)
To: xen-devel, Jeremy Fitzhardinge, Konrad Rzeszutek Wilk, Bruce Edge
[-- Attachment #1.1: Type: text/plain, Size: 5055 bytes --]
Full stack trace obtained in text console of 2.6.37-rc1 PV guest :-
Attempt to scp generates output ( or maybe just time delay expired)
root@boris-HVM-domU:~# scp 1.log 192.168.1.12:/home/boris/ubuntu
BUG: unable to handle kernel paging request at ffff88000e676930
IP: [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
PGD 1a04067 PUD 1a08067 PMD ee73067 PTE 801000000e676065
Oops: 0003 [#1] SMP
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0
Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ipv6 lp parport
Pid: 3534, comm: scp Not tainted 2.6.37-rc1 #1 /
RIP: e030:[<ffffffff810069c3>] [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
RSP: e02b:ffff88006e697b18 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88000e676930 RCX: ffffc00000000f98
RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff88000e676930
RBP: ffff88006e697b28 R08: 00007fff9d120000 R09: ffff88007ffc80a0
R10: 0000000000000000 R11: ffff88006e697ae8 R12: 0000000000000000
R13: ffff88007ffcb140 R14: ffff88000e676930 R15: ffff88006f08d7f8
FS: 00007fa525896700(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffff88000e676930 CR3: 0000000077b0f000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process scp (pid: 3534, threadinfo ffff88006e696000, task ffff88007a32c2c0)
Stack:
00007fa524e00000 00007fa525b37000 ffff88006e697be8 ffffffff810dc4a4
ffff88007a26ac88 ffff88006e690298 ffff88007a26ac78 00007fa524a00000
00007f8000000000 00007fff80000000 00007fa525b36fff 00007fa525b36fff
Call Trace:
[<ffffffff810dc4a4>] free_pgd_range+0x213/0x397
[<ffffffff810dc6cc>] free_pgtables+0xa4/0xc7
[<ffffffff810dfd82>] exit_mmap+0xf2/0x12a
[<ffffffff8104a7c5>] mmput+0x5e/0xd1
[<ffffffff8110694b>] flush_old_exec+0x526/0x600
[<ffffffff8113e006>] load_elf_binary+0x2dc/0x1752
[<ffffffff8113b687>] ? load_misc_binary+0x6e/0x367
[<ffffffff811dcee4>] ? ima_must_measure+0x21/0x2b
[<ffffffff811dc7d6>] ? process_measurement+0x56/0xe4
[<ffffffff81106195>] search_binary_handler+0xbb/0x24d
[<ffffffff8113dd2a>] ? load_elf_binary+0x0/0x1752
[<ffffffff8110785d>] do_execve+0x1c1/0x2ae
[<ffffffff810119e5>] sys_execve+0x3e/0x55
[<ffffffff8100af9c>] stub_execve+0x6c/0xc0
Code: 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48 89 fb e8 b5 e4 ff ff 48 89 df ff 05 3e 55 c3 00 e8 d5 dd ff ff 84 c0 75 05 <4c> 89 23 eb 16 e8 99 e4 ff ff 4c 89 e6 48 89 df ff 05 23 55 c3
RIP [<ffffffff810069c3>] xen_set_pmd+0x24/0x44
RSP <ffff88006e697b18>
CR2: ffff88000e676930
---[ end trace 073da74f4ce02315 ]---
lost connection
root@boris-HVM-domU:~# ------------[ cut here ]------------
kernel BUG at arch/x86/xen/mmu.c:1831!
invalid opcode: 0000 [#2] SMP
last sysfs file: /sys/devices/virtual/bdi/0:22/uevent
CPU 0
Modules linked in: nfs fscache parport_pc ppdev nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ipv6 lp parport
Pid: 3536, comm: cron Tainted: G D 2.6.37-rc1 #1 /
RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
RSP: e02b:ffff88006d391cd8 EFLAGS: 00010282
RAX: 00000000ffffffea RBX: 000000000006d0f5 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88006d391cd8
RBP: ffff88006d391cf8 R08: 00003ffffffff000 R09: ffff880000000000
R10: 0000000000007ff0 R11: ffff880000000041 R12: 0000000000000003
R13: ffff88006f082010 R14: ffff88006e7c1420 R15: ffff88007a266340
FS: 00007f3e0f0f87a0(0000) GS:ffff88007ffbd000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000406039 CR3: 000000006d2ad000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process cron (pid: 3536, threadinfo ffff88006d390000, task ffff88007a266340)
Stack:
ffff880000000000 00000000001411dd ffff88000e598b00 000000000006d0f5
ffff88006d391d18 ffffffff810066ce 000000000006d0f5 ffff88000e598b00
ffff88006d391d28 ffffffff81006701 ffff88006d391d68 ffffffff810dafc6
Call Trace:
[<ffffffff810066ce>] xen_alloc_ptpage+0x68/0x6d
[<ffffffff81006701>] xen_alloc_pte+0xe/0x10
[<ffffffff810dafc6>] __pte_alloc+0x72/0xd7
[<ffffffff810dd355>] handle_mm_fault+0x12f/0xa05
[<ffffffff8145665a>] do_page_fault+0x387/0x3a9
[<ffffffff81453715>] ? page_fault+0x25/0x30
[<ffffffff8145394a>] ? error_exit+0x2a/0x60
[<ffffffff8145349d>] ? retint_restore_args+0x5/0x6
[<ffffffff8120510d>] ? __put_user_4+0x1d/0x30
[<ffffffff81453715>] page_fault+0x25/0x30
Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
RSP <ffff88006d391cd8>
---[ end trace 073da74f4ce02316 ]---
Boris.
[-- Attachment #1.2: Type: text/html, Size: 5840 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request
2010-11-10 22:30 ` Bruce Edge
` (2 preceding siblings ...)
2010-11-11 12:01 ` Boris Derzhavets
@ 2010-11-11 15:08 ` Bruce Edge
3 siblings, 0 replies; 45+ messages in thread
From: Bruce Edge @ 2010-11-11 15:08 UTC (permalink / raw)
To: xen-devel, Jeremy Fitzhardinge, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 56437 bytes --]
On Wed, Nov 10, 2010 at 2:30 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> On Wed, Nov 10, 2010 at 2:15 PM, Bruce Edge <bruce.edge@gmail.com> wrote:
> >
> > This is a mainline 2.6.37-rc1 pvops domU on top of a xen-unstable build
> 22241 and xen/stable-2.6.32.x 2.6.32.24 pvops dom0.
> >
> > The domU console output hung up prior to spitting out a prompt post root
> login. I logged into the domU via ssh and ran an 'ls' of an NFS mounted dir
> and
> > this immediately came out on the domU console:
> >
> > [ 480.467525] BUG: unable to handle kernel paging request at
> ffff880077e9f030
> > [ 480.467547] IP: [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> > [ 480.467564] PGD 1a04067 PUD 3dd7067 PMD 3f97067 PTE 8010000077e9f065
> > [ 480.467587] Oops: 0003 [#1] SMP
> > [ 480.467601] last sysfs file: /sys/kernel/uevent_seqnum
> > [ 480.467611] CPU 3
> > [ 480.467616] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
> xen_blkfront
> > [ 480.467676]
> > [ 480.467684] Pid: 3957, comm: zsh Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> > [ 480.467695] RIP: e030:[<ffffffff810355f3>] [<ffffffff810355f3>]
> ptep_set_access_flags+0x27/0x4d
> > [ 480.467710] RSP: e02b:ffff880077ebfbd8 EFLAGS: 00010202
> > [ 480.467719] RAX: 800000004d686167 RBX: ffff8800763ddb80 RCX:
> 800000004d686167
> > [ 480.467729] RDX: ffff880077e9f030 RSI: 00007f5cb26069d0 RDI:
> ffff8800763ddb80
> > [ 480.467739] RBP: ffff880077ebfc08 R08: 0000000000000001 R09:
> ffffea0001a3b2d8
> > [ 480.467748] R10: 0000000000000100 R11: 0000000000000000 R12:
> 00007f5cb26069d0
> > [ 480.467759] R13: 0000000000000001 R14: 0000000000000000 R15:
> ffff8800763ddb80
> > [ 480.467772] FS: 00007f5cb2606700(0000) GS:ffff88007ff94000(0000)
> knlGS:0000000000000000
> > [ 480.467783] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> > [ 480.467792] CR2: ffff880077e9f030 CR3: 0000000077dbc000 CR4:
> 0000000000002660
> > [ 480.467802] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> > [ 480.467812] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> > [ 480.467822] Process zsh (pid: 3957, threadinfo ffff880077ebe000, task
> ffff880078b80000)
> > [ 480.467832] Stack:
> > [ 480.467838] 0000000000000100 ffffea0001a3b2d8 7000000000000000
> 0000000000000000
> > [ 480.467862] ffff880077e9f030 ffffea0001a3b2d8 ffff880077ebfca8
> ffffffff810e2200
> > [ 480.467886] ffff880077ebfc38 ffffffff81007c09 ffffea0001a3b2d8
> 0000000000000001
> > [ 480.467910] Call Trace:
> > [ 480.467921] [<ffffffff810e2200>] do_wp_page+0x2d8/0x6be
> > [ 480.467934] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> > [ 480.467946] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> > [ 480.467958] [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
> > [ 480.467972] [<ffffffff814463fe>] do_page_fault+0x374/0x396
> > [ 480.467984] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> > [ 480.467996] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> > [ 480.468009] [<ffffffff810072a2>] ? check_events+0x12/0x20
> > [ 480.468020] [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
> > [ 480.468034] [<ffffffff810a6fa4>] ? __call_rcu+0x123/0x12c
> > [ 480.468045] [<ffffffff81006c3d>] ? xen_force_evtchn_callback+0xd/0xf
> > [ 480.468059] [<ffffffff810072a2>] ? check_events+0x12/0x20
> > [ 480.468072] [<ffffffff81443615>] page_fault+0x25/0x30
> > [ 480.468085] [<ffffffff81206add>] ? __put_user_4+0x1d/0x30
> > [ 480.468096] [<ffffffff810498a6>] ? schedule_tail+0x5c/0x60
> > [ 480.468108] [<ffffffff8100a9c3>] ret_from_fork+0x13/0x80
> > [ 480.468116] Code: 41 5d c9 c3 55 48 89 e5 41 55 45 31 ed 41 54 49 89
> f4 53 48 89 fb 48 83 ec 18 48 39 0a 41 0f 95 c5 45 85 c0 74 1d 45 85 ed 74
> 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b2
> > [ 480.468323] RIP [<ffffffff810355f3>] ptep_set_access_flags+0x27/0x4d
> > [ 480.468323] RSP <ffff880077ebfbd8>
> > [ 480.468323] CR2: ffff880077e9f030
> > [ 480.468323] ---[ end trace 6d1623321ecd50ac ]---
> >
> > The domU config is attached as config.2.6.37_domU_debug
> >
> >
> > -Bruce
>
> Here's another attempt from the same config with the full domU boot logs:
>
> Started domain dpm (id=4)
> [ 0.000000] Initializing cgroup subsys cpuset
> [ 0.000000] Initializing cgroup subsys cpu
> [ 0.000000] Linux version
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug (root@opal) (gcc version
> 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Tue Nov 9 20:26:03 PST 2010
> [ 0.000000] Command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [ 0.000000] ACPI in unprivileged domain disabled
> [ 0.000000] released 0 pages of unused memory
> [ 0.000000] BIOS-provided physical RAM map:
> [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
> [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
> [ 0.000000] Xen: 0000000000100000 - 0000000080800000 (usable)
> [ 0.000000] NX (Execute Disable) protection: active
> [ 0.000000] DMI not present or invalid.
> [ 0.000000] No AGP bridge found
> [ 0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
> [ 0.000000] Scanning 0 areas for low memory corruption
> [ 0.000000] init_memory_mapping: 0000000000000000-0000000080800000
> [ 0.000000] RAMDISK: 01d95000 - 02fa9000
> [ 0.000000] No NUMA configuration found
> [ 0.000000] Faking a node at 0000000000000000-0000000080800000
> [ 0.000000] Initmem setup node 0 0000000000000000-0000000080800000
> [ 0.000000] NODE_DATA [000000007fffb000 - 000000007fffffff]
> [ 0.000000] Zone PFN ranges:
> [ 0.000000] DMA 0x00000010 -> 0x00001000
> [ 0.000000] DMA32 0x00001000 -> 0x00100000
> [ 0.000000] Normal empty
> [ 0.000000] Movable zone start PFN for each node
> [ 0.000000] early_node_map[2] active PFN ranges
> [ 0.000000] 0: 0x00000010 -> 0x000000a0
> [ 0.000000] 0: 0x00000100 -> 0x00080800
> [ 0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
> [ 0.000000] No local APIC present
> [ 0.000000] APIC: disable apic facility
> [ 0.000000] APIC: switched to apic NOOP
> [ 0.000000] PM: Registered nosave memory: 00000000000a0000 -
> 0000000000100000
> [ 0.000000] Allocating PCI resources starting at 80800000 (gap:
> 80800000:7f800000)
> [ 0.000000] Booting paravirtualized kernel on Xen
> [ 0.000000] Xen version: 4.1-unstable (preserve-AD)
> [ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32
> nr_cpu_ids:6 nr_node_ids:1
> [ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752
> r8192 d23744 u114688
> [ 0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
> [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
> [ 0.000000] Built 1 zonelists in Node order, mobility grouping on.
> Total pages: 519022
> [ 0.000000] Policy zone: DMA32
> [ 0.000000] Kernel command line:
> root=UUID=4d87e4cb-8317-41d4-aea2-ac2ebee699f7 ro swiotlb=force
> noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
> ip=:127.0.255.255::::eth0:dhcp
> [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [ 0.000000] Placing 64MB software IO TLB between ffff880079400000 -
> ffff88007d400000
> [ 0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
> [ 0.000000] Memory: 1959896k/2105344k available (4398k kernel code,
> 448k absent, 145000k reserved, 6327k data, 756k init)
> [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
> CPUs=6, Nodes=1
> [ 0.000000] Hierarchical RCU implementation.
> [ 0.000000] RCU-based detection of stalled CPUs is disabled.
> [ 0.000000] NR_IRQS:4352 nr_irqs:320 16
> [ 0.000000] Console: colour dummy device 80x25
> [ 0.000000] console [tty0] enabled
> [ 0.000000] console [hvc0] enabled
> [ 0.000000] installing Xen timer for CPU 0
> [ 0.000000] Detected 2660.064 MHz processor.
> [ 0.000999] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 5320.12 BogoMIPS (lpj=2660064)
> [ 0.000999] pid_max: default: 32768 minimum: 301
> [ 0.000999] Security Framework initialized
> [ 0.000999] SELinux: Initializing.
> [ 0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152
> bytes)
> [ 0.001408] Inode-cache hash table entries: 131072 (order: 8, 1048576
> bytes)
> [ 0.001635] Mount-cache hash table entries: 256
> [ 0.002195] Initializing cgroup subsys ns
> [ 0.002208] ns_cgroup deprecated: consider using the
> 'clone_children' flag without the ns_cgroup.
> [ 0.002220] Initializing cgroup subsys cpuacct
> [ 0.002231] Initializing cgroup subsys freezer
> [ 0.002356] CPU: Unsupported number of siblings 16
> [ 0.002362] Performance Events: unsupported p6 CPU model 26 no PMU
> driver, software events only.
> [ 0.002483] SMP alternatives: switching to UP code
> [ 0.004119] cpu 0 spinlock event irq 318
> [ 0.004765] NMI watchdog failed to create perf event on cpu0:
> fffffffffffffffe
> [ 0.005050] installing Xen timer for CPU 1
> [ 0.005157] cpu 1 spinlock event irq 312
> [ 0.005194] SMP alternatives: switching to SMP code
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.006158] NMI watchdog failed to create perf event on cpu1:
> fffffffffffffffe
> [ 0.006474] installing Xen timer for CPU 2
> [ 0.006585] cpu 2 spinlock event irq 306
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.006816] NMI watchdog failed to create perf event on cpu2:
> fffffffffffffffe
> [ 0.006999] installing Xen timer for CPU 3
> [ 0.007037] cpu 3 spinlock event irq 300
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.007300] NMI watchdog failed to create perf event on cpu3:
> fffffffffffffffe
> [ 0.007534] installing Xen timer for CPU 4
> [ 0.007636] cpu 4 spinlock event irq 294
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.007874] NMI watchdog failed to create perf event on cpu4:
> fffffffffffffffe
> [ 0.008041] installing Xen timer for CPU 5
> [ 0.008083] cpu 5 spinlock event irq 288
> [ 0.000999] CPU: Unsupported number of siblings 16
> [ 0.008344] NMI watchdog failed to create perf event on cpu5:
> fffffffffffffffe
> [ 0.008356] Brought up 6 CPUs
> [ 0.009181] kworker/u:0 used greatest stack depth: 5696 bytes left
> [ 0.012051] Grant table initialized
> [ 0.030978] Time: 165:165:165 Date: 165/165/65
> [ 0.031229] NET: Registered protocol family 16
> [ 0.036672] PCI: setting up Xen PCI frontend stub
> [ 0.037225] kworker/u:0 used greatest stack depth: 5528 bytes left
> [ 0.074302] bio: create slab <bio-0> at 0
> [ 0.075385] ACPI: Interpreter disabled.
> [ 0.077056] xen_balloon: Initialising balloon driver.
> [ 0.079323] vgaarb: loaded
> [ 0.081270] usbcore: registered new interface driver usbfs
> [ 0.081283] usbcore: registered new interface driver hub
> [ 0.081283] usbcore: registered new device driver usb
> [ 0.083183] PCI: System does not support PCI
> [ 0.084007] PCI: System does not support PCI
> [ 0.085236] NetLabel: Initializing
> [ 0.085236] NetLabel: domain hash size = 128
> [ 0.085236] NetLabel: protocols = UNLABELED CIPSOv4
> [ 0.085313] NetLabel: unlabeled traffic allowed by default
> [ 0.086544] Switching to clocksource xen
> [ 0.106256] pnp: PnP ACPI: disabled
> [ 0.129799] NET: Registered protocol family 2
> [ 0.130174] IP route cache hash table entries: 65536 (order: 7, 524288
> bytes)
> [ 0.131882] TCP established hash table entries: 262144 (order: 10,
> 4194304 bytes)
> [ 0.132891] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [ 0.133179] TCP: Hash tables configured (established 262144 bind 65536)
> [ 0.133191] TCP reno registered
> [ 0.133301] UDP hash table entries: 1024 (order: 3, 32768 bytes)
> [ 0.133407] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
> [ 0.133914] NET: Registered protocol family 1
> [ 0.134582] RPC: Registered udp transport module.
> [ 0.134590] RPC: Registered tcp transport module.
> [ 0.134595] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [ 0.134892] Trying to unpack rootfs image as initramfs...
> [ 0.178824] Freeing initrd memory: 18512k freed
> [ 0.268798] DMA-API: preallocated 32768 debug entries
> [ 0.268810] DMA-API: debugging enabled by kernel config
> [ 0.271181] platform rtc_cmos: registered platform RTC device (no
> PNP device found)
> [ 0.285410] Machine check injector initialized
> [ 0.294732] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294803] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294893] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294944] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.294998] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.295043] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
> [ 0.295590] microcode: Microcode Update Driver: v2.00
> <tigran@aivazian.fsnet.co.uk>, Peter Oruba
> [ 0.295607] Scanning for low memory corruption every 60 seconds
> [ 0.298813] audit: initializing netlink socket (disabled)
> [ 0.298940] type=2000 audit(1289403082.795:1): initialized
> [ 0.333762] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [ 0.376985] VFS: Disk quotas dquot_6.5.2
> [ 0.377853] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [ 0.383121] NTFS driver 2.1.29 [Flags: R/W].
> [ 0.385014] msgmni has been set to 3864
> [ 0.389953] Block layer SCSI generic (bsg) driver version 0.4
> loaded (major 253)
> [ 0.389970] io scheduler noop registered
> [ 0.389979] io scheduler deadline registered
> [ 0.390666] io scheduler cfq registered (default)
> [ 0.392486] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [ 0.406238] pcifront pci-0: Installing PCI frontend
> [ 0.406967] pcifront pci-0: Creating PCI Frontend Bus 0000:00
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:00.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:01.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:02.0/4
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/0
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/2
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/3
> [ 0.417634] pcifront pci-0: claiming resource 0000:00:03.0/4
> [ 0.731775] Non-volatile memory driver v1.3
> [ 0.731790] Linux agpgart interface v0.103
> [ 0.733731] [drm] Initialized drm 1.1.0 20060810
> [ 0.733752] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [ 0.827202] brd: module loaded
> [ 0.840788] loop: module loaded
> [ 0.841965] Fixed MDIO Bus: probed
> [ 0.844025] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> [ 0.844651] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> [ 0.845194] uhci_hcd: USB Universal Host Controller Interface driver
> [ 0.846985] usbcore: registered new interface driver usblp
> [ 0.847763] usbcore: registered new interface driver libusual
> [ 0.849433] PNP: No PS/2 controller found. Probing ports directly.
> [ 0.850279] i8042.c: No controller found.
> [ 0.850883] mice: PS/2 mouse device common for all mice
> [ 0.853446] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [ 0.854066] rtc_cmos: probe of rtc_cmos failed with error -38
> [ 0.855928] cpuidle: using governor ladder
> [ 0.855941] cpuidle: using governor menu
> [ 0.856029] Netfilter messages via NETLINK v0.30.
> [ 0.856136] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
> [ 0.857888] ctnetlink v0.93: registering with nfnetlink.
> [ 0.860285] ip_tables: (C) 2000-2006 Netfilter Core Team
> [ 0.860677] TCP cubic registered
> [ 0.860687] Initializing XFRM netlink socket
> [ 0.863286] NET: Registered protocol family 10
> [ 0.867216] ip6_tables: (C) 2000-2006 Netfilter Core Team
> [ 0.867612] IPv6 over IPv4 tunneling driver
> [ 0.871326] NET: Registered protocol family 17
> [ 0.871566] Registering the dns_resolver key type
> [ 0.873668] registered taskstats version 1
> [ 0.873912] kmemleak: Kernel memory leak detector initialized
> [ 0.873928] kmemleak: Automatic memory scanning thread started
> [ 0.875057] XENBUS: Device with no driver: device/vbd/51712
> [ 0.875057] XENBUS: Device with no driver: device/vif/0
> [ 0.875057] XENBUS: Device with no driver: device/console/0
> [ 0.875057] Magic number: 1:252:3141
> [ 13.388958] Freeing unused kernel memory: 756k freed
> [ 13.389190] Write protecting the kernel read-only data: 10240k
> [ 13.398774] Freeing unused kernel memory: 1724k freed
> [ 13.400663] Freeing unused kernel memory: 1992k freed
> [ 13.494573] all_generic_ide used greatest stack depth: 5472 bytes left
> [ 13.519388] udevd (1111): /proc/1111/oom_adj is deprecated, please
> use /proc/1111/oom_score_adj instead.
> Begin: Loading essential drivers... ...
> [ 13.559435] blkid used greatest stack depth: 5408 bytes left
> [ 13.568391] blkfront: xvda: barriers disabled
> [ 13.574762] blkid used greatest stack depth: 5264 bytes left
> [ 13.580813] xvda: xvda1 xvda2 < xvda5 >
> [ 13.645562] modprobe used greatest stack depth: 5088 bytes left
> [ 13.653160] Initialising Xen virtual ethernet driver.
> [ 13.659540] blkid used greatest stack depth: 4960 bytes left
> [ 13.735467] blkid used greatest stack depth: 4752 bytes left
> Done.
> Begin: Running /scripts/init-premount ...
> Done.
> Begin: Mounting root file system... ...
> Begin: Running /scripts/local-top ...
> Done.
> Begin: Running /scripts/local-premount ...
> Done.
> [ 13.903774] EXT4-fs (xvda1): INFO: recovery required on readonly
> filesystem
> [ 13.903799] EXT4-fs (xvda1): write access will be enabled during
> recovery
> [ 14.033719] EXT4-fs (xvda1): recovery complete
> [ 14.034966] EXT4-fs (xvda1): mounted filesystem with ordered data
> mode. Opts: (null)
> [ 14.035510] exe used greatest stack depth: 4112 bytes left
> Begin: Running /scripts/local-bottom ...
> Done.
> Done.
> Begin: Running /scripts/init-bottom ...
> Done.
> init: ureadahead main process (1214) terminated with status 5
> fsck from util-linux-ng 2.17.2
> e2fsck 1.41.11 (14-Mar-2010)
> /dev/xvda1: clean, 56314/288576 files, 287743/1153280 blocks
> * Enable login on console [
> OK ]
> * Check for hostname changes... [
> OK ]
> ...done.
> * Setting sensors limits
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> No sensors found!
> Make sure you loaded all the kernel drivers you need.
> Try sensors-detect to find out which these are.
> [
> OK ]
> * Initializing random number generator... [
> OK ]
> * Setting up X server socket directory /tmp/.X11-unix... [
> OK ]
> * Setting up ICE socket directory /tmp/.ICE-unix... [
> OK ]
> * Not starting fancontrol; run pwmconfig first.
> * Starting internet superserver inetd [
> OK ]
> Starting openntpd: ntpd.
> * Starting network management services:
> * Setting core file location... [
> OK ]
> * Restoring resolver state... [
> OK ]
> * Checking tomcat configuration
> ...done.
> * Starting Tomcat servlet engine tomcat6
> Using CATALINA_BASE: /var/lib/tomcat6
> Using CATALINA_HOME: /usr/share/tomcat6
> Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
> Using JRE_HOME: /usr/lib/jvm/java-6-openjdk
> Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar
> [
> OK ]
> collectd has been disabled - see /etc/default/collectd.
> * Recording successful boot for GRUB
> ...done.
> * Running local boot scripts (/etc/rc.local) [
> OK ]
> mountall: Plymouth command failed
> mountall: Disconnected from Plymouth
>
> 6.0.16-01
> kaan-04-dpm login: root
> Password:
> [ 42.646538] ------------[ cut here ]------------
> [ 42.646556] kernel BUG at arch/x86/xen/mmu.c:1831!
> [ 42.646566] invalid opcode: 0000 [#1] SMP
> [ 42.646582] last sysfs file: /sys/kernel/uevent_seqnum
> [ 42.646592] CPU 0
> [ 42.646598] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront
> xen_netfront xen_blkfront
> [ 42.646665]
> [ 42.646674] Pid: 3916, comm: lsb_release Not tainted
> 2.6.37-rc1-kernel.org-domu-6.0.bedge-debug #1 /
> [ 42.646688] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
> pin_pagetable_pfn+0x31/0x37
> [ 42.646711] RSP: e02b:ffff88007595bce8 EFLAGS: 00010282
> [ 42.646721] RAX: 00000000ffffffea RBX: 000000000007751e RCX:
> 0000000000000001
> [ 42.646732] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
> 00000000deadbeef
> [ 42.646744] RBP: ffff88007595bd08 R08: 00003ffffffff000 R09:
> ffff880000000000
> [ 42.646756] R10: 00000000deadbeef R11: dead000000200200 R12:
> 0000000000000003
> [ 42.646768] R13: ffff880075c90150 R14: ffff8800781d9398 R15:
> ffff880075161d40
> [ 42.646784] FS: 00007fee63b01700(0000) GS:ffff88007ff40000(0000)
> knlGS:0000000000000000
> [ 42.646797] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 42.646807] CR2: 00007faac54182b8 CR3: 00000000771dd000 CR4:
> 0000000000002660
> [ 42.646819] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [ 42.646832] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [ 42.646844] Process lsb_release (pid: 3916, threadinfo
> ffff88007595a000, task ffff880075161d40)
> [ 42.646857] Stack:
> [ 42.646864] ffff880000000000 00000000000484f0 ffff8800777c6d80
> 000000000007751e
> [ 42.646892] ffff88007595bd28 ffffffff810066a8 000000000007751e
> ffff8800777c6d80
> [ 42.646919] ffff88007595bd38 ffffffff810066db ffff88007595bd78
> ffffffff810e048f
> [ 42.646946] Call Trace:
> [ 42.646959] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
> [ 42.646973] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
> [ 42.646989] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
> [ 42.647003] [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
> [ 42.647017] [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
> [ 42.647033] [<ffffffff8100eb84>] ?
> arch_get_unmapped_area_topdown+0x4a/0x28a
> [ 42.647052] [<ffffffff814463fe>] do_page_fault+0x374/0x396
> [ 42.647059] [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
> [ 42.647059] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
> [ 42.647059] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
> [ 42.647059] [<ffffffff81443615>] page_fault+0x25/0x30
> [ 42.647059] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d
> e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff
> 85 c0 74 04 <0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48
> 8b 3d
> [ 42.647059] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
> [ 42.647059] RSP <ffff88007595bce8>
> [ 42.647059] ---[ end trace 3fbbffee414a5166 ]---
>
Here's another full boot that I just let sit at the login prompt. This is
using the pcifront-fixes kernel:
Started domain dpm (id=1)
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug (root@opal) (gcc
version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Wed Nov 10 17:11:41 PST 2010
[ 0.000000] Command line: root=UUID=1e67e680-4108-4e11-8bc4-844dad448902
ro swiotlb=force noirqbalance noirqbalance console=hvc0,115200n8
--flow_control=r ip=:127.0.255.255::::eth0:dhcp
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] released 0 pages of unused memory
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
[ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
[ 0.000000] Xen: 0000000000100000 - 0000000080800000 (usable)
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] DMI not present or invalid.
[ 0.000000] No AGP bridge found
[ 0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000
[ 0.000000] Scanning 0 areas for low memory corruption
[ 0.000000] init_memory_mapping: 0000000000000000-0000000080800000
[ 0.000000] RAMDISK: 01d95000 - 02db9000
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at 0000000000000000-0000000080800000
[ 0.000000] Initmem setup node 0 0000000000000000-0000000080800000
[ 0.000000] NODE_DATA [000000007fffb000 - 000000007fffffff]
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal empty
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[2] active PFN ranges
[ 0.000000] 0: 0x00000010 -> 0x000000a0
[ 0.000000] 0: 0x00000100 -> 0x00080800
[ 0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs
[ 0.000000] No local APIC present
[ 0.000000] APIC: disable apic facility
[ 0.000000] APIC: switched to apic NOOP
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 -
0000000000100000
[ 0.000000] Allocating PCI resources starting at 80800000 (gap:
80800000:7f800000)
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.1-unstable (preserve-AD)
[ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:6
nr_node_ids:1
[ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752 r8192
d23744 u114688
[ 0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096
[ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5
[ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total
pages: 519022
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line:
root=UUID=1e67e680-4108-4e11-8bc4-844dad448902 ro swiotlb=force
noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r
ip=:127.0.255.255::::eth0:dhcp
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Placing 64MB software IO TLB between ffff880079400000 -
ffff88007d400000
[ 0.000000] software IO TLB at phys 0x79400000 - 0x7d400000
[ 0.000000] Memory: 1961880k/2105344k available (4398k kernel code, 448k
absent, 143016k reserved, 6327k data, 756k init)
[ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0,
CPUs=6, Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] RCU-based detection of stalled CPUs is disabled.
[ 0.000000] NR_IRQS:4352 nr_irqs:320 16
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] console [hvc0] enabled
[ 0.000000] installing Xen timer for CPU 0
[ 0.000000] Detected 2660.054 MHz processor.
[ 0.000999] Calibrating delay loop (skipped), value calculated using
timer frequency.. 5320.10 BogoMIPS (lpj=2660054)
[ 0.000999] pid_max: default: 32768 minimum: 301
[ 0.000999] Security Framework initialized
[ 0.000999] SELinux: Initializing.
[ 0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152
bytes)
[ 0.001410] Inode-cache hash table entries: 131072 (order: 8, 1048576
bytes)
[ 0.001641] Mount-cache hash table entries: 256
[ 0.002191] Initializing cgroup subsys ns
[ 0.002204] ns_cgroup deprecated: consider using the 'clone_children'
flag without the ns_cgroup.
[ 0.002216] Initializing cgroup subsys cpuacct
[ 0.002227] Initializing cgroup subsys freezer
[ 0.002354] CPU: Unsupported number of siblings 16
[ 0.002361] Performance Events: unsupported p6 CPU model 26 no PMU
driver, software events only.
[ 0.002524] SMP alternatives: switching to UP code
[ 0.004910] cpu 0 spinlock event irq 318
[ 0.005490] NMI watchdog failed to create perf event on cpu0:
fffffffffffffffe
[ 0.005817] installing Xen timer for CPU 1
[ 0.005857] cpu 1 spinlock event irq 312
[ 0.005893] SMP alternatives: switching to SMP code
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.007158] NMI watchdog failed to create perf event on cpu1:
fffffffffffffffe
[ 0.007474] installing Xen timer for CPU 2
[ 0.007585] cpu 2 spinlock event irq 306
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.007909] NMI watchdog failed to create perf event on cpu2:
fffffffffffffffe
[ 0.008078] installing Xen timer for CPU 3
[ 0.008193] cpu 3 spinlock event irq 300
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.008450] NMI watchdog failed to create perf event on cpu3:
fffffffffffffffe
[ 0.008685] installing Xen timer for CPU 4
[ 0.008789] cpu 4 spinlock event irq 294
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.008999] NMI watchdog failed to create perf event on cpu4:
fffffffffffffffe
[ 0.009210] installing Xen timer for CPU 5
[ 0.009252] cpu 5 spinlock event irq 288
[ 0.000999] CPU: Unsupported number of siblings 16
[ 0.009530] NMI watchdog failed to create perf event on cpu5:
fffffffffffffffe
[ 0.009541] Brought up 6 CPUs
[ 0.010295] kworker/u:0 used greatest stack depth: 5696 bytes left
[ 0.013138] Grant table initialized
[ 0.032051] Time: 165:165:165 Date: 165/165/65
[ 0.032378] NET: Registered protocol family 16
[ 0.037699] PCI: setting up Xen PCI frontend stub
[ 0.039132] kworker/u:0 used greatest stack depth: 5528 bytes left
[ 0.067272] bio: create slab <bio-0> at 0
[ 0.068203] ACPI: Interpreter disabled.
[ 0.069064] xen_balloon: Initialising balloon driver.
[ 0.070332] vgaarb: loaded
[ 0.072248] usbcore: registered new interface driver usbfs
[ 0.072279] usbcore: registered new interface driver hub
[ 0.073080] usbcore: registered new device driver usb
[ 0.075072] PCI: System does not support PCI
[ 0.075137] PCI: System does not support PCI
[ 0.076112] NetLabel: Initializing
[ 0.076120] NetLabel: domain hash size = 128
[ 0.076126] NetLabel: protocols = UNLABELED CIPSOv4
[ 0.076223] NetLabel: unlabeled traffic allowed by default
[ 0.077209] Switching to clocksource xen
[ 0.093954] pnp: PnP ACPI: disabled
[ 0.138850] NET: Registered protocol family 2
[ 0.139329] IP route cache hash table entries: 65536 (order: 7, 524288
bytes)
[ 0.141816] TCP established hash table entries: 262144 (order: 10,
4194304 bytes)
[ 0.143253] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 0.143474] TCP: Hash tables configured (established 262144 bind 65536)
[ 0.143487] TCP reno registered
[ 0.143537] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[ 0.143593] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[ 0.144245] NET: Registered protocol family 1
[ 0.145189] RPC: Registered udp transport module.
[ 0.145203] RPC: Registered tcp transport module.
[ 0.145213] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.145645] Trying to unpack rootfs image as initramfs...
[ 0.186054] Freeing initrd memory: 16528k freed
[ 0.270158] DMA-API: preallocated 32768 debug entries
[ 0.270170] DMA-API: debugging enabled by kernel config
[ 0.272584] platform rtc_cmos: registered platform RTC device (no PNP
device found)
[ 0.285338] Machine check injector initialized
[ 0.294148] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294217] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294308] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294359] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294427] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294473] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa
[ 0.294990] microcode: Microcode Update Driver: v2.00 <
tigran@aivazian.fsnet.co.uk>, Peter Oruba
[ 0.295008] Scanning for low memory corruption every 60 seconds
[ 0.297474] audit: initializing netlink socket (disabled)
[ 0.297559] type=2000 audit(1289430805.716:1): initialized
[ 0.331826] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.375119] VFS: Disk quotas dquot_6.5.2
[ 0.375896] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.381318] NTFS driver 2.1.29 [Flags: R/W].
[ 0.383311] msgmni has been set to 3864
[ 0.387906] Block layer SCSI generic (bsg) driver version 0.4 loaded
(major 253)
[ 0.387924] io scheduler noop registered
[ 0.387933] io scheduler deadline registered
[ 0.388724] io scheduler cfq registered (default)
[ 0.390543] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.403338] pcifront pci-0: Installing PCI frontend
[ 0.404082] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[ 0.406134] pcifront pci-0: claiming resource 0000:00:00.0/0
[ 0.406134] pcifront pci-0: claiming resource 0000:00:00.0/2
[ 0.406134] pcifront pci-0: claiming resource 0000:00:00.0/3
[ 0.406134] pcifront pci-0: claiming resource 0000:00:00.0/4
[ 0.406134] pcifront pci-0: claiming resource 0000:00:01.0/0
[ 0.406134] pcifront pci-0: claiming resource 0000:00:01.0/2
[ 0.406134] pcifront pci-0: claiming resource 0000:00:01.0/3
[ 0.406134] pcifront pci-0: claiming resource 0000:00:01.0/4
[ 0.406134] pcifront pci-0: claiming resource 0000:00:02.0/0
[ 0.406134] pcifront pci-0: claiming resource 0000:00:02.0/2
[ 0.406134] pcifront pci-0: claiming resource 0000:00:02.0/3
[ 0.406134] pcifront pci-0: claiming resource 0000:00:02.0/4
[ 0.406134] pcifront pci-0: claiming resource 0000:00:03.0/0
[ 0.406134] pcifront pci-0: claiming resource 0000:00:03.0/2
[ 0.406134] pcifront pci-0: claiming resource 0000:00:03.0/3
[ 0.406134] pcifront pci-0: claiming resource 0000:00:03.0/4
[ 0.734074] Non-volatile memory driver v1.3
[ 0.734089] Linux agpgart interface v0.103
[ 0.735535] [drm] Initialized drm 1.1.0 20060810
[ 0.735556] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.829048] brd: module loaded
[ 0.842342] loop: module loaded
[ 0.843465] Fixed MDIO Bus: probed
[ 0.845827] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.846398] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.846882] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.848574] usbcore: registered new interface driver usblp
[ 0.849103] usbcore: registered new interface driver libusual
[ 0.850786] PNP: No PS/2 controller found. Probing ports directly.
[ 0.851626] i8042.c: No controller found.
[ 0.852258] mice: PS/2 mouse device common for all mice
[ 0.855068] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[ 0.855717] rtc_cmos: probe of rtc_cmos failed with error -38
[ 0.857602] cpuidle: using governor ladder
[ 0.857619] cpuidle: using governor menu
[ 0.857713] Netfilter messages via NETLINK v0.30.
[ 0.857800] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 0.859562] ctnetlink v0.93: registering with nfnetlink.
[ 0.861913] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 0.862342] TCP cubic registered
[ 0.862354] Initializing XFRM netlink socket
[ 0.864995] NET: Registered protocol family 10
[ 0.869020] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 0.869455] IPv6 over IPv4 tunneling driver
[ 0.873134] NET: Registered protocol family 17
[ 0.873381] Registering the dns_resolver key type
[ 0.875443] registered taskstats version 1
[ 0.875615] kmemleak: Kernel memory leak detector initialized
[ 0.875634] kmemleak: Automatic memory scanning thread started
[ 0.877640] XENBUS: Device with no driver: device/vbd/51712
[ 0.877654] XENBUS: Device with no driver: device/vif/0
[ 0.877664] XENBUS: Device with no driver: device/console/0
[ 0.877692] Magic number: 1:252:3141
[ 13.391082] Freeing unused kernel memory: 756k freed
[ 13.391322] Write protecting the kernel read-only data: 10240k
[ 13.401075] Freeing unused kernel memory: 1724k freed
[ 13.402500] Freeing unused kernel memory: 1992k freed
Loading, please wait...
mount: mounting none on /dev failed: No such device
[ 13.498289] all_generic_ide used greatest stack depth: 5472 bytes left
[ 13.523488] udevd (1111): /proc/1111/oom_adj is deprecated, please use
/proc/1111/oom_score_adj instead.
Begin: Loading essential drivers... ...
[ 13.546062] udevadm used greatest stack depth: 5088 bytes left
[ 13.580639] blkfront: xvda: barriers disabled
[ 13.594779] xvda: xvda1 xvda2 < xvda5 >
[ 13.669872] Initialising Xen virtual ethernet driver.
[ 13.672323] blkid used greatest stack depth: 4960 bytes left
[ 13.756667] blkid used greatest stack depth: 4752 bytes left
Done.
Begin: Running /scripts/init-premount ...
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
Done.
Begin: Running /scripts/local-premount ...
Done.
[ 13.903338] EXT4-fs (xvda1): INFO: recovery required on readonly
filesystem
[ 13.903363] EXT4-fs (xvda1): write access will be enabled during recovery
[ 14.190114] EXT4-fs (xvda1): recovery complete
[ 14.191367] EXT4-fs (xvda1): mounted filesystem with ordered data mode.
Opts: (null)
[ 14.191951] exe used greatest stack depth: 4112 bytes left
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
init: ureadahead main process (1208) terminated with status 5
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda1: clean, 56107/288576 files, 296111/1153280 blocks
* Enable login on console [
OK ]
* Check for hostname changes... [
OK ]
...done.
* Setting sensors
limits No sensors
found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
No sensors found!
Make sure you loaded all the kernel drivers you need.
Try sensors-detect to find out which these are.
[
OK ]
* Initializing random number generator... [
OK ]
* Setting up X server socket directory /tmp/.X11-unix... [
OK ]
* Setting up ICE socket directory /tmp/.ICE-unix... [
OK ]
* Not starting fancontrol; run pwmconfig first.
* Starting internet superserver inetd [
OK ]
Starting openntpd: ntpd.
* Starting network management
services: * Starting the system
activity data collector
sadc
Cannot open /var/log/sysstat/sa10: No such file or directory
[fail]
* Setting core file location... [
OK ]
* Restoring resolver state... [
OK ]
* Checking tomcat configuration
...done.
* Starting Tomcat servlet engine
tomcat6 Using CATALINA_BASE:
/var/lib/tomcat6
Using CATALINA_HOME: /usr/share/tomcat6
Using CATALINA_TMPDIR: /tmp/tomcat6-tmp
Using JRE_HOME: /usr/lib/jvm/java-6-openjdk
Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar
[
OK ]
collectd has been disabled - see /etc/default/collectd.
* Recording successful boot for GRUB
...done.
* Running local boot scripts (/etc/rc.local) [
OK ]
mountall: Plymouth command failed
mountall: Disconnected from Plymouth
6.0.16-01
kaan-04-dpm login: [ 47.548041] ------------[ cut here ]------------
[ 47.548078] kernel BUG at arch/x86/xen/mmu.c:1831!
[ 47.548093] invalid opcode: 0000 [#1] SMP
[ 47.548114] last sysfs file: /sys/kernel/uevent_seqnum
[ 47.548124] CPU 0
[ 47.548130] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[ 47.548199]
[ 47.548208] Pid: 3815, comm: java Not tainted
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[ 47.548222] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[ 47.548245] RSP: e02b:ffff880077fadce8 EFLAGS: 00010282
[ 47.548255] RAX: 00000000ffffffea RBX: 0000000000078312 RCX:
0000000000000001
[ 47.548266] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[ 47.548278] RBP: ffff880077fadd08 R08: 00003ffffffff000 R09:
ffff880000000000
[ 47.548290] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[ 47.548302] R13: ffff880077f5e838 R14: ffff88007629ab80 R15:
ffff880078288750
[ 47.548319] FS: 00007f2e8058c710(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[ 47.548332] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 47.548342] CR2: 00007f2e60fe8728 CR3: 0000000077ec9000 CR4:
0000000000002660
[ 47.548354] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 47.548366] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[ 47.548379] Process java (pid: 3815, threadinfo ffff880077fac000, task
ffff880078288750)
[ 47.548391] Stack:
[ 47.548398] ffff880000000000 00000000000476fc ffff88007603ea00
0000000000078312
[ 47.548426] ffff880077fadd28 ffffffff810066a8 0000000000078312
ffff88007603ea00
[ 47.548454] ffff880077fadd38 ffffffff810066db ffff880077fadd78
ffffffff810e048f
[ 47.548481] Call Trace:
[ 47.548494] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[ 47.548509] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[ 47.548525] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[ 47.548539] [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[ 47.548554] [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
[ 47.548571] [<ffffffff81442ef2>] ? _raw_spin_unlock_irqrestore+0x19/0x1c
[ 47.548588] [<ffffffff81048583>] ? try_to_wake_up+0x207/0x219
[ 47.548603] [<ffffffff814463fe>] do_page_fault+0x374/0x396
[ 47.548617] [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
[ 47.548632] [<ffffffff81442ef2>] ? _raw_spin_unlock_irqrestore+0x19/0x1c
[ 47.548648] [<ffffffff81202bce>] ? rwsem_wake+0x39/0x43
[ 47.548661] [<ffffffff81206bc7>] ? call_rwsem_wake+0x17/0x30
[ 47.548677] [<ffffffff8106aa3d>] ? up_write+0x18/0x1a
[ 47.548690] [<ffffffff810e7378>] ? sys_mmap_pgoff+0x126/0x159
[ 47.548705] [<ffffffff81443615>] page_fault+0x25/0x30
[ 47.548715] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48
89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04
<0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
[ 47.548794] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[ 47.548794] RSP <ffff880077fadce8>
[ 47.548794] ---[ end trace ceae70ee7c69e40a ]---
[ 63.942338] kmemleak: 4 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)
[ 92.806778] ------------[ cut here ]------------
[ 92.806802] kernel BUG at arch/x86/xen/mmu.c:1831!
[ 92.806817] invalid opcode: 0000 [#2] SMP
[ 92.806841] last sysfs file: /sys/kernel/uevent_seqnum
[ 92.806856] CPU 0
[ 92.806865] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[ 92.806967]
[ 92.806981] Pid: 3914, comm: rsyslogd Tainted: G D
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[ 92.807005] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[ 92.807036] RSP: e02b:ffff880077869ce8 EFLAGS: 00010282
[ 92.807053] RAX: 00000000ffffffea RBX: 0000000000077c61 RCX:
0000000000000001
[ 92.807063] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[ 92.807063] RBP: ffff880077869d08 R08: 00003ffffffff000 R09:
ffff880000000000
[ 92.807063] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[ 92.807063] R13: ffff8800760e44e0 R14: ffff880075ce8958 R15:
ffff880077015f10
[ 92.807063] FS: 00007ffd53e39700(0000) GS:ffff88007ff40000(0000)
knlGS:0000000000000000
[ 92.807063] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 92.807063] CR2: 00007ffd53801330 CR3: 000000007821e000 CR4:
0000000000002660
[ 92.807063] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 92.807063] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[ 92.807063] Process rsyslogd (pid: 3914, threadinfo ffff880077868000,
task ffff880077015f10)
[ 92.807063] Stack:
[ 92.807063] ffff880000000000 0000000000047dad ffff880076138700
0000000000077c61
[ 92.807063] ffff880077869d28 ffffffff810066a8 0000000000077c61
ffff880076138700
[ 92.807063] ffff880077869d38 ffffffff810066db ffff880077869d78
ffffffff810e048f
[ 92.807063] Call Trace:
[ 92.807063] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[ 92.807063] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[ 92.807063] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[ 92.807063] [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[ 92.807063] [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[ 92.807063] [<ffffffff814463fe>] do_page_fault+0x374/0x396
[ 92.807063] [<ffffffff81007bc9>] ? xen_spin_unlock+0x11/0x30
[ 92.807063] [<ffffffff8105b253>] ? do_sigaction+0x175/0x18d
[ 92.807063] [<ffffffff811039a7>] ? fput+0x1ed/0x1fc
[ 92.807063] [<ffffffff8105d855>] ? sys_rt_sigaction+0x5e/0x9d
[ 92.807063] [<ffffffff81443615>] page_fault+0x25/0x30
[ 92.807063] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48
89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04
<0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
[ 92.807063] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[ 92.807063] RSP <ffff880077869ce8>
[ 92.807063] ---[ end trace ceae70ee7c69e40b ]---
[ 693.101054] kmemleak: 86 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)
[ 695.855546] ------------[ cut here ]------------
[ 695.855557] kernel BUG at arch/x86/xen/mmu.c:1831!
[ 695.855563] invalid opcode: 0000 [#3] SMP
[ 695.855574] last sysfs file: /sys/kernel/uevent_seqnum
[ 695.855581] CPU 1
[ 695.855585] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[ 695.855629]
[ 695.855635] Pid: 3946, comm: cron Tainted: G D
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[ 695.855645] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[ 695.855664] RSP: e02b:ffff880067a29ce8 EFLAGS: 00010282
[ 695.855671] RAX: 00000000ffffffea RBX: 0000000000077c58 RCX:
0000000000000001
[ 695.855680] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[ 695.855689] RBP: ffff880067a29d08 R08: 00003ffffffff000 R09:
ffff880000000000
[ 695.855696] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[ 695.855703] R13: ffff880067a2cb40 R14: ffff880077c7e678 R15:
ffff8800682395f0
[ 695.855716] FS: 00007fa1afde67a0(0000) GS:ffff88007ff5c000(0000)
knlGS:0000000000000000
[ 695.855725] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 695.855734] CR2: 00007fa1ad1ac758 CR3: 0000000067890000 CR4:
0000000000002660
[ 695.855742] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 695.855749] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[ 695.855759] Process cron (pid: 3946, threadinfo ffff880067a28000, task
ffff8800682395f0)
[ 695.855767] Stack:
[ 695.855774] ffff880000000000 0000000000047db6 ffff8800760e3480
0000000000077c58
[ 695.855793] ffff880067a29d28 ffffffff810066a8 0000000000077c58
ffff8800760e3480
[ 695.855814] ffff880067a29d38 ffffffff810066db ffff880067a29d78
ffffffff810e048f
[ 695.855834] Call Trace:
[ 695.855842] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[ 695.855852] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[ 695.855865] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[ 695.855877] [<ffffffff810e2717>] handle_mm_fault+0x131/0x8c1
[ 695.855887] [<ffffffff810e6e36>] ? mmap_region+0x391/0x4bf
[ 695.855900] [<ffffffff8100eb84>] ?
arch_get_unmapped_area_topdown+0x4a/0x28a
[ 695.855915] [<ffffffff814463fe>] do_page_fault+0x374/0x396
[ 695.855925] [<ffffffff8102b78a>] ? pvclock_clocksource_read+0x4b/0xb4
[ 695.855937] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[ 695.855949] [<ffffffff81007e85>] ? __xen_spin_lock+0xb7/0xcd
[ 695.855959] [<ffffffff81443615>] page_fault+0x25/0x30
[ 695.855967] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48
89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04
<0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
[ 695.856056] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[ 695.856056] RSP <ffff880067a29ce8>
[ 695.856056] ---[ end trace ceae70ee7c69e40c ]---
[ 696.704202] cron[3928] general protection ip:7fa1af4ce588 sp:7ffff1875200
error:0 in libc-2.11.1.so[7fa1af424000+17a000]
[ 697.853416] ------------[ cut here ]------------
[ 697.853427] kernel BUG at arch/x86/xen/mmu.c:1831!
[ 697.853433] invalid opcode: 0000 [#4] SMP
[ 697.853443] last sysfs file: /sys/kernel/uevent_seqnum
[ 697.853450] CPU 4
[ 697.853453] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[ 697.853497]
[ 697.853503] Pid: 3947, comm: sh Tainted: G D
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[ 697.853512] RIP: e030:[<ffffffff81004bce>] [<ffffffff81004bce>]
pin_pagetable_pfn+0x31/0x37
[ 697.853526] RSP: e02b:ffff880067aa5b38 EFLAGS: 00010282
[ 697.853532] RAX: 00000000ffffffea RBX: 0000000000075203 RCX:
0000000000000001
[ 697.853540] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI:
00000000deadbeef
[ 697.853547] RBP: ffff880067aa5b58 R08: 00003ffffffff000 R09:
ffff880000000000
[ 697.853554] R10: 00000000deadbeef R11: dead000000200200 R12:
0000000000000003
[ 697.853561] R13: ffff880075204000 R14: ffff880075204000 R15:
ffff880078a26ff8
[ 697.853571] FS: 00007fa1afde67a0(0000) GS:ffff88007ffb0000(0000)
knlGS:0000000000000000
[ 697.853580] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 697.853586] CR2: 00007fa1af7bd560 CR3: 0000000077a98000 CR4:
0000000000002660
[ 697.853594] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 697.853602] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[ 697.853609] Process sh (pid: 3947, threadinfo ffff880067aa4000, task
ffff880068239d40)
[ 697.853616] Stack:
[ 697.853621] ffff880000000000 000000000004a80b ffff8800760e1500
0000000000075203
[ 697.853640] ffff880067aa5b78 ffffffff810066a8 0000000000075203
ffff8800760e1500
[ 697.853658] ffff880067aa5b88 ffffffff810066db ffff880067aa5bc8
ffffffff810e048f
[ 697.853676] Call Trace:
[ 697.853685] [<ffffffff810066a8>] xen_alloc_ptpage+0x68/0x6d
[ 697.853693] [<ffffffff810066db>] xen_alloc_pte+0xe/0x10
[ 697.853704] [<ffffffff810e048f>] __pte_alloc+0x72/0xd7
[ 697.853713] [<ffffffff810e7faa>] move_page_tables+0x1cb/0x4c1
[ 697.853722] [<ffffffff81007bc9>] ? xen_spin_unlock+0x11/0x30
[ 697.853732] [<ffffffff8110870a>] setup_arg_pages+0x1cb/0x355
[ 697.853743] [<ffffffff8113ce7a>] load_elf_binary+0x418/0x16ec
[ 697.853752] [<ffffffff8113b319>] ? load_misc_binary+0x5c/0x33c
[ 697.853761] [<ffffffff810e3336>] ? get_user_pages+0x44/0x46
[ 697.853772] [<ffffffff81107355>] ? get_arg_page+0x50/0xa7
[ 697.853781] [<ffffffff81107793>] search_binary_handler+0xbb/0x24d
[ 697.853790] [<ffffffff8113ca62>] ? load_elf_binary+0x0/0x16ec
[ 697.853799] [<ffffffff81108e2d>] do_execve+0x19e/0x28d
[ 697.853809] [<ffffffff810118dd>] sys_execve+0x3e/0x55
[ 697.853818] [<ffffffff8100af1c>] stub_execve+0x6c/0xc0
[ 697.853824] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48
89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 76 c7 ff ff 85 c0 74 04
<0f> 0b eb fe c9 c3 55 48 89 e5 41 54 49 89 fc 53 31 db 48 8b 3d
[ 697.854013] RIP [<ffffffff81004bce>] pin_pagetable_pfn+0x31/0x37
[ 697.854024] RSP <ffff880067aa5b38>
[ 697.854031] ---[ end trace ceae70ee7c69e40d ]---
[ 755.706246] BUG: unable to handle kernel paging request at
ffff880075202b80
[ 755.706263] IP: [<ffffffff810e24b3>] do_wp_page+0x58b/0x6be
[ 755.706278] PGD 1a04067 PUD 39d9067 PMD 3b83067 PTE 8010000075202065
[ 755.706294] Oops: 0003 [#5] SMP
[ 755.706305] last sysfs file: /sys/kernel/uevent_seqnum
[ 755.706312] CPU 4
[ 755.706316] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront
fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront
xen_blkfront
[ 755.706364]
[ 755.706370] Pid: 3948, comm: cron Tainted: G D
2.6.37-rc1-pvops-pcifront-fixes-domu-6.0.bedge-debug #1 /
[ 755.706379] RIP: e030:[<ffffffff810e24b3>] [<ffffffff810e24b3>]
do_wp_page+0x58b/0x6be
[ 755.706391] RSP: e02b:ffff880067439ce8 EFLAGS: 00010246
[ 755.706397] RAX: 0000000000000000 RBX: ffffea00016f7d98 RCX:
0000000000000001
[ 755.706404] RDX: 6db6db6db6db6db7 RSI: ffffffff81b460c0 RDI:
ffff8800760a1f80
[ 755.706411] RBP: ffff880067439d78 R08: 0000000000000000 R09:
0000000000000041
[ 755.706419] R10: 0000000000000100 R11: ffff880078277030 R12:
80000000407c3067
[ 755.706426] R13: ffffea0001bd0068 R14: ffff880075202b80 R15:
ffff880067a77508
[ 755.706436] FS: 00007fa1afde67a0(0000) GS:ffff88007ffb0000(0000)
knlGS:0000000000000000
[ 755.706444] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 755.706451] CR2: ffff880075202b80 CR3: 0000000077ea9000 CR4:
0000000000002660
[ 755.706458] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 755.706466] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[ 755.706474] Process cron (pid: 3948, threadinfo ffff880067438000, task
ffff88006823a490)
[ 755.706481] Stack:
[ 755.706486] ffff880067439d08 ffffffff81007c09 ffffea000199f080
0000000000000000
[ 755.706504] 01ff880067439d58 ffffea000199f080 ffff8800760a1f80
0000000000d703b0
[ 755.706523] 00007fa1afde6a70 ffff880078277030 ffff880067a77508
ffff8800760a1f80
[ 755.706542] Call Trace:
[ 755.706552] [<ffffffff81007c09>] ? __spin_time_accum+0x21/0x37
[ 755.706562] [<ffffffff810e2e60>] handle_mm_fault+0x87a/0x8c1
[ 755.706571] [<ffffffff8103ed4d>] ? enqueue_entity+0x2ac/0x2b8
[ 755.706582] [<ffffffff814463fe>] do_page_fault+0x374/0x396
[ 755.706590] [<ffffffff8100728f>] ? xen_restore_fl_direct_end+0x0/0x1
[ 755.706600] [<ffffffff81004274>] ? xen_mc_flush+0x1cd/0x1f2
[ 755.706609] [<ffffffff8103ed4d>] ? enqueue_entity+0x2ac/0x2b8
[ 755.706618] [<ffffffff81007bc9>] ? xen_spin_unlock+0x11/0x30
[ 755.706628] [<ffffffff81045b20>] ? finish_task_switch+0x4f/0x94
[ 755.706637] [<ffffffff81443615>] page_fault+0x25/0x30
[ 755.706644] Code: 48 0f af fa 48 c1 e7 0c 48 09 c7 ff 14 25 28 13 a1 81
49 89 c4 48 83 c8 42 49 8b 3f 49 83 cc 40 41 f6 47 30 02 4c 0f 45 e0 31 c0
<49> 87 06 48 8b 75 a8 4c 89 f2 0f 1f 80 00 00 00 00 48 8b 75 a8
[ 755.706830] RIP [<ffffffff810e24b3>] do_wp_page+0x58b/0x6be
[ 755.706841] RSP <ffff880067439ce8>
[ 755.706847] CR2: ffff880075202b80
[ 755.706854] ---[ end trace ceae70ee7c69e40e ]---
[-- Attachment #1.2: Type: text/html, Size: 61166 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 45+ messages in thread