[BUG] Xen vm kernel crash in get_free_entries.

[Astarta <astarta@rat.ru>]

[-- Attachment #1: Type: text/plain, Size: 818 bytes --]

Hello,

This is a some kind of a follow up to the 
http://lists.xen.org/archives/html/xen-devel/2013-09/msg03109.html (
Linux 3.11.1 HVM guest kernel crash when started with xl 
(get_free_entries)).

Looks like we've here the similar issue.

Xen VM with 3.8.13 kernel crashes in get_free_entries (kernel BUG at 
drivers/xen/grant-table.c:1181!), i.e in:
BUG_ON(grefs_per_grant_frame == 0);. 3.11.5 kernel behaves in exactly 
the same way.

Whole kernel log is attached. It does not contain  any "Grant tables 
using version" message.

Seems that gnttab_request_version() has was never executed, so 
grefs_per_grant_frame was never set.

Could you please advice on this issue?
I'm ready to provide any further information if required.

I'm not subscribed, so please add me to CC, so that I can see replies.


--
Marina

[-- Attachment #2: xen.log --]
[-- Type: text/x-log, Size: 13545 bytes --]

[root@xenserver62 ~]# xl console 117
Linux version 3.8.13-x86_64 (astarta@domain) (gcc version 4.4.3 ) #1 SMP Fri Oct 4 15:23:59 MSD 2013
Command line: initrd=ramdisk.dat root=/dev/ram0 console=ttyS0,115200 console=tty0 vga=0x314 BOOT_IMAGE=kernel.dat 
e820: BIOS-provided physical RAM map:
BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable
BIOS-e820: [mem 0x000000000009e000-0x000000000009ffff] reserved
BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
BIOS-e820: [mem 0x0000000000100000-0x000000003fbfffff] usable
BIOS-e820: [mem 0x00000000fc000000-0x00000000ffffffff] reserved
NX (Execute Disable) protection: active
SMBIOS 2.4 present.                                                                                                           
Hypervisor detected: Xen HVM                                                                                                  
Xen version 4.1.                                                                                                              
Netfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated NICs.                            
Blkfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated disks.                           
You might have to change the root device                                                                                      
from /dev/hd[a-d] to /dev/xvd[a-d]                                                                                            
in your root= kernel command line option
No AGP bridge found
e820: last_pfn = 0x3fc00 max_arch_pfn = 0x400000000
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
found SMP MP-table at [mem 0x000fb710-0x000fb71f] mapped at [ffff8800000fb710]
init_memory_mapping: [mem 0x00000000-0x3fbfffff]
RAMDISK: [mem 0x3c6d0000-0x3fbdffff]
ACPI: RSDP 00000000000ea020 00024 (v02    Xen)
ACPI: XSDT 00000000fc00ef80 00044 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: FACP 00000000fc00ed40 000F4 (v04    Xen      HVM 00000000 HVML 00000000)
ACPI: DSDT 00000000fc003040 0BC75 (v02    Xen      HVM 00000000 INTL 20090123)
ACPI: FACS 00000000fc003000 00040
ACPI: APIC 00000000fc00ee40 000D8 (v02    Xen      HVM 00000000 HVML 00000000)
ACPI: HPET 00000000fc00ef20 00038 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: WAET 00000000fc00ef58 00028 (v01    Xen      HVM 00000000 HVML 00000000)
No NUMA configuration found
Faking a node at [mem 0x0000000000000000-0x000000003fbfffff]
Initmem setup node 0 [mem 0x00000000-0x3fbfffff]
  NODE_DATA [mem 0x3fbec000-0x3fbfffff]
Zone ranges:
  DMA      [mem 0x00010000-0x00ffffff]
  DMA32    [mem 0x01000000-0xffffffff]
  Normal   empty
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x00010000-0x0009dfff]
  node   0: [mem 0x00100000-0x3fbfffff]
ACPI: PM-Timer IO Port: 0x1f48
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] disabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] disabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] disabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] disabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] disabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x0c] disabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0e] disabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x10] disabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x12] disabled)
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x14] disabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x16] disabled)
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x18] disabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x1a] disabled)
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x1c] disabled)
ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 low level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 low level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 low level)
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8086a201 base: 0xfed00000
smpboot: Allowing 15 CPUs, 14 hotplug CPUs
e820: [mem 0x3fc00000-0xfbffffff] available for PCI devices
Booting paravirtualized kernel on Xen HVM
setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:15 nr_node_ids:1
PERCPU: Embedded 27 pages/cpu @ffff88003c400000 s78656 r8192 d23744 u131072
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 256920
Policy zone: DMA32
Kernel command line: initrd=ramdisk.dat root=/dev/ram0 console=ttyS0,115200 console=tty0 vga=0x314 BOOT_IMAGE=kernel.dat 
PID hash table entries: 4096 (order: 3, 32768 bytes)
__ex_table already sorted, skipping sort
Checking aperture...
No AGP bridge found
Memory: 963504k/1044480k available (3992k kernel code, 456k absent, 80520k reserved, 2581k data, 768k init)
SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=15, Nodes=1
Hierarchical RCU implementation.
        RCU restricting CPUs from NR_CPUS=128 to nr_cpu_ids=15.
NR_IRQS:8448 nr_irqs:1208 16
Xen HVM callback vector for event delivery is enabled
Console: colour dummy device 80x25
console [tty0] enabled
Cannot get hvm parameter CONSOLE_EVTCHN (18): -22!
console [ttyS0] enabled
tsc: Detected 1864.851 MHz processor
Calibrating delay loop (skipped), value calculated using timer frequency.. 3729.70 BogoMIPS (lpj=1864851)
pid_max: default: 32768 minimum: 301
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
Mount-cache hash table entries: 256
mce: CPU supports 0 MCE banks
Last level iTLB entries: 4KB 128, 2MB 4, 4MB 4
Last level dTLB entries: 4KB 256, 2MB 0, 4MB 32
tlb_flushall_shift: -1
ACPI: Core revision 20121018
Switched APIC routing to physical flat.
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0
smpboot: CPU0: Intel(R) Core(TM)2 CPU          6300  @ 1.86GHz (fam: 06, model: 0f, stepping: 02)
installing Xen timer for CPU 0
Performance Events: unsupported p6 CPU model 15 no PMU driver, software events only.
Brought up 1 CPUs
smpboot: Total of 1 processors activated (3729.70 BogoMIPS)
devtmpfs: initialized
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: Added _OSI(Module Device)
ACPI: Added _OSI(Processor Device)
ACPI: Added _OSI(3.0 _SCP Extensions)
ACPI: Added _OSI(Processor Aggregator Device)
ACPI: Interpreter enabled
ACPI: (supports S0 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfbffffff]
* Found PM-Timer Bug on the chipset. Due to workarounds for a bug,
* this clock source is slow. Consider trying other clock sources
pci 0000:00:01.3: quirk: [io  0x1f40-0x1f7f] claimed by PIIX4 ACPI
 pci0000:00: ACPI _OSC support notification failed, disabling PCIe ASPM
 pci0000:00: Unable to request _OSC control (_OSC support mask: 0x08)
ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11)
xen/balloon: Initialising balloon driver.
xen-balloon: Initialising balloon driver.
vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:00:02.0
PCI: Using ACPI for IRQ routing
HPET: 3 timers in total, 0 timers will be used for per-cpu timer
Switching to clocksource xen
pnp: PnP ACPI init
ACPI: bus type pnp registered
system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved
system 00:01: [io  0x10c0-0x1141] has been reserved
system 00:01: [io  0xb044-0xb047] has been reserved
system 00:03: [io  0x08a0-0x08a3] has been reserved
system 00:03: [io  0x0cc0-0x0ccf] has been reserved
system 00:03: [io  0x04d0-0x04d1] has been reserved
pnp: PnP ACPI: found 12 devices
ACPI: ACPI bus type pnp unregistered
Loading, please wait...
NET: Registered protocol family 2
TCP established hash table entries: 8192 (order: 5, 131072 bytes)
TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
TCP: Hash tables configured (established 8192 bind 8192)
TCP: reno registered
UDP hash table entries: 512 (order: 2, 16384 bytes)
UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
pci 0000:00:00.0: Limiting direct PCI/PCI transfers
pci 0000:00:01.0: PIIX3: Enabling Passive Release
pci 0000:00:01.0: Activating ISA DMA hang workarounds
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 54336k freed
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
ROMFS MTD (C) 2007 Red Hat, Inc.
msgmni has been set to 1987
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
vesafb: mode is 800x600x16, linelength=1600, pages=3
vesafb: scrolling: redraw
vesafb: Truecolor: size=0:5:6:5, shift=0:11:5:0
vesafb: framebuffer at 0xf0000000, mapped to 0xffffc90000200000, using 1875k, total 4096k
Console: switching to colour frame buffer device 100x37
fb0: VESA VGA frame buffer device
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
ACPI: Power Button [PWRF]
input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
ACPI: Sleep Button [SLPF]
GHES: HEST is not enabled!
Event-channel device installed.
Cannot get hvm parameter CONSOLE_EVTCHN (18): -22!
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
00:0a: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
Linux agpgart interface v0.103
[drm] Initialized drm 1.1.0 20060810
brd: module loaded
------------[ cut here ]------------
kernel BUG at drivers/xen/grant-table.c:1181!
invalid opcode: 0000 [#1] SMP 
Modules linked in:
CPU 0 
Pid: 1, comm: swapper/0 Not tainted 3.8.13-x86_64 #1 Xen HVM domU
RIP: 0010:[<ffffffff81251200>]  [<ffffffff81251200>] get_free_entries+0x2e0/0x300
RSP: 0000:ffff88003c205bd8  EFLAGS: 00010046
RAX: 0000000000000296 RBX: 0000000000000001 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 000000000003f5ea RDI: ffffffff817c0150
RBP: ffff88003c205c38 R08: 0000000000000000 R09: ffff88003f608060
R10: ffff88003fbec6c0 R11: 0000000000000008 R12: 0000000000000296
R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff81643298
FS:  0000000000000000(0000) GS:ffff88003c400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 000000000160c000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper/0 (pid: 1, threadinfo ffff88003c204000, task ffff88003c208000)
Stack:
 ffff88003fbec6c0 0000000000000002 000000103c205c88 0000000000000000
 ffff88003fbec000 0000000000000000 ffff88003c017440 ffff88003f608000
 000000000003f5ea 0000000000000000 0000000000000000 ffffffff81643298
Call Trace:
 [<ffffffff812512ce>] gnttab_grant_foreign_access+0x2e/0x70
 [<ffffffff812555ef>] xenbus_grant_ring+0x1f/0x50
 [<ffffffff812bf7ca>] talk_to_blkback+0xaa/0x350
 [<ffffffff812c1db6>] blkfront_probe+0x186/0x2b0
 [<ffffffff81258127>] xenbus_dev_probe+0x77/0x130
 [<ffffffff812afb13>] driver_probe_device+0x93/0x250
 [<ffffffff812afd63>] __driver_attach+0x93/0xa0
 [<ffffffff812afcd0>] ? driver_probe_device+0x250/0x250
 [<ffffffff812ae133>] bus_for_each_dev+0x83/0xa0
 [<ffffffff812af939>] driver_attach+0x19/0x20
 [<ffffffff812aead0>] bus_add_driver+0x1c0/0x250
 [<ffffffff816afac8>] ? brd_init+0x1d2/0x1d2
 [<ffffffff812b03b8>] driver_register+0x78/0x160
 [<ffffffff816afac8>] ? brd_init+0x1d2/0x1d2
 [<ffffffff81257f95>] xenbus_register_driver_common+0x15/0x20
 [<ffffffff8125a053>] xenbus_register_frontend+0x23/0x40
 [<ffffffff816afac8>] ? brd_init+0x1d2/0x1d2
 [<ffffffff816afb2a>] xlblk_init+0x62/0x86
 [<ffffffff816af8f6>] ? ramdisk_size+0x1a/0x1a
 [<ffffffff8100203d>] do_one_initcall+0x3d/0x170
 [<ffffffff816816df>] kernel_init_freeable+0x108/0x192
 [<ffffffff81681769>] ? kernel_init_freeable+0x192/0x192
 [<ffffffff813ceaa0>] ? rest_init+0x80/0x80
 [<ffffffff813ceaa9>] kernel_init+0x9/0xf0
 [<ffffffff813e1f3c>] ret_from_fork+0x7c/0xb0
 [<ffffffff813ceaa0>] ? rest_init+0x80/0x80
Code: 48 89 10 48 89 05 b1 ef 56 00 4d 85 f6 0f 84 cf fd ff ff 44 8b 2d 79 ef 56 00 4c 89 f0 eb ce 4c 8b 3d 5d ef 56 00 e9 2e ff ff ff <0f> 0b eb fe 0f 0b eb fe 48 c7 00 00 00 00 00 48 8b 78 10 ff 50 
RIP  [<ffffffff81251200>] get_free_entries+0x2e0/0x300
 RSP <ffff88003c205bd8>
---[ end trace aeae77e70a304f5f ]---
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b




[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel