All of lore.kernel.org
 help / color / mirror / Atom feed
* Fwd: Debugging "Xen BUG at ... list.h" on ARM
       [not found] <CAG4opy8h2ZBpFhODSiS8OmGP-M_UCLPn3bPpWzt1wm1UHprDSw@mail.gmail.com>
@ 2014-04-14  8:02 ` Thomas Leonard
  2014-04-14  8:06   ` Thomas Leonard
  2014-04-14  9:12   ` Fwd: " Julien Grall
  0 siblings, 2 replies; 14+ messages in thread
From: Thomas Leonard @ 2014-04-14  8:02 UTC (permalink / raw)
  To: xen-devel

[-- Attachment #1: Type: text/plain, Size: 2747 bytes --]

Hi all,

I'm new to Xen, and trying to get the arm32 port of Mini-OS working on
a Cubieboard2. I have the board running an Ubuntu Dom0 and an Ubuntu
guest happily, and I've written up the instructions for that here:

  https://github.com/talex5/mirage-www/blob/master/tmpl/wiki/xen-on-cubieboard2.md

However, while trying to get a Mini-OS domU guest working I can now
reliably crash the hypervisor. No doubt this is a bug in my guest
code, but I assume it shouldn't be possible for a buggy guest to take
down Xen itself.

Here's a simplified version of my code which shows the problem:

  https://github.com/talex5/xen-paratest

I'm building Xen from the stable-4.4 Git branch (03eb51340), compiled
with debug on.

The code is rather simple: it's just trying to initialise the data
cache, as described in the "ARM Cortex-A Series Programmer’s Guide".
I'm not actually sure whether this is necessary for a paravirtualised
guest, but I was getting some odd problems which looked cache related,
so I added this code in the hope of fixing that.

The cfg file is just:

kernel = "/root/paratest.img"
memory = 128
name = "paratest"
vcpus = 1
serial="pty"

When the guest is started, Xen always crashes. Sometimes silently, but
usually writing a stack-trace to the console. I've included a
selection of them in the commit messages, but a typical one looks like
this:

    (XEN) Assertion 'svc ==
CSCHED_VCPU(curr_on_cpu(svc->vcpu->processor))' failed, line 859, file
sche
    (XEN) Xen BUG at sched_credit.c:859
    (XEN) CPU1: Unexpected Trap: Undefined Instruction
    (XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
    (XEN) CPU:    1
    (XEN) PC:     002417b4 __bug+0x28/0x44
    (XEN) CPSR:   2000015a MODE:Hypervisor
    (XEN)      R0: 002656dc R1: 00000000 R2: 3fd2bd80 R3: 2000015a
    (XEN)      R4: 0000035b R5: 0025d9d0 R6: 40006230 R7: 47fe4be0
    (XEN)      R8: 00265e00 R9: 002b1ff0 R10:002e4254 R11:47fd7e9c R12:00000001
    (XEN) HYP: SP: 47fd7e94 LR: 002417b4
[...]
    (XEN) Xen call trace:
    (XEN)    [<002417b4>] __bug+0x28/0x44 (PC)
    (XEN)    [<002417b4>] __bug+0x28/0x44 (LR)
    (XEN)    [<0021bac0>] csched_tick+0x1ac/0x4d4
    (XEN)    [<0022d820>] execute_timer+0xcc/0xe0
    (XEN)    [<0022d910>] timer_softirq_action+0xdc/0x230
    (XEN)    [<00229b88>] __do_softirq+0xf4/0x100
    (XEN)    [<00229c20>] do_softirq+0x14/0x18
    (XEN)    [<0024f100>] leave_hypervisor_tail+0x50/0x6c
    (XEN)    [<00251450>] return_to_guest+0xc/0xb8


Any hints welcome.

Thanks,


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

[-- Attachment #2: xen-bug.txt --]
[-- Type: text/plain, Size: 12917 bytes --]

(XEN) Assertion 'entry->next->prev == entry' failed, line 175, file /home/tal/work/cubieboard/xen/xen/include/xen/list.h
(XEN) Xen BUG at /home/tal/work/cubieboard/xen/xen/include/xen/list.h:175
(XEN) CPU0: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) PC:     002414cc __bug+0x28/0x44
(XEN) CPSR:   200001da MODE:Hypervisor
(XEN)      R0: 002656dc R1: 00000002 R2: 00000000 R3: 200001da
(XEN)      R4: 000000af R5: 0025d238 R6: 00000000 R7: 00000000
(XEN)      R8: 002e6380 R9: 002e6380 R10:002e6380 R11:40037e3c R12:00000003
(XEN) HYP: SP: 40037e34 LR: 002414cc
(XEN)
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 000100007ec06000
(XEN)
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282835
(XEN)  TTBR0_EL2: 000000007fedf000
(XEN)
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 000000000001c810
(XEN)      HDFAR: c8800f00
(XEN)      HIFAR: 0000000c
(XEN)
(XEN) Xen stack trace from sp=40037e34:
(XEN)    00000000 40037e6c 0022e708 4003b940 800001da 00007fff 4003b000 4003b928
(XEN)    00282835 4001c000 00000002 4001202c 00000001 40037e7c 002504c4 4003b000
(XEN)    4003a000 40037e94 002440c0 002e6500 4001c000 4003b000 4003b000 40037ea4
(XEN)    002445b8 74972ede 000015e8 40037f0c 002266e8 00000000 4003b848 00000001
(XEN)    40012028 ffffffff ffffffff 002e6588 93840047 40037eec 0022884c 4001c000
(XEN)    c000e43c ffffffff ffffffff 40037f00 c07063f8 40037f54 00000001 002b1ff4
(XEN)    002ae000 002e7614 00265b80 002b1ff0 002e4254 40037f3c 00229b28 00000000
(XEN)    ffffffff 002e7614 002ae000 002e7614 c07063f8 c07414d7 c050078c c06fe000
(XEN)    c07414d7 40037f44 00229bc0 40037f54 0024ee00 c0706474 c06fe000 40037f58
(XEN)    00251150 ffffffed 077de000 c070746c 00000000 c0706474 c06fe000 c07063f8
(XEN)    c07414d7 c050078c c06fe000 c07414d7 c06fe000 00000000 ffffffff b6def1d9
(XEN)    c0019d28 60070093 00000000 bedd8850 c0742080 c0012400 c06fffa8 c000f600
(XEN)    c074208c c0012540 c0742098 c00125e0 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 40070030 20000193 80070093 60000193 00000000
(XEN)    00000000 030c3c80 651a8d04
(XEN) Xen call trace:
(XEN)    [<002414cc>] __bug+0x28/0x44 (PC)
(XEN)    [<002414cc>] __bug+0x28/0x44 (LR)
(XEN)    [<0022e708>] migrate_timer+0x1d4/0x3a8
(XEN)    [<002504c4>] virt_timer_restore+0x3c/0x78
(XEN)    [<002440c0>] schedule_tail+0x298/0x2f4
(XEN)    [<002445b8>] context_switch+0x114/0x118
(XEN)    [<002266e8>] schedule+0x8a0/0x8c8
(XEN)    [<00229b28>] __do_softirq+0xf4/0x100
(XEN)    [<00229bc0>] do_softirq+0x14/0x18
(XEN)    [<0024ee00>] leave_hypervisor_tail+0x50/0x6c
(XEN)    [<00251150>] return_to_guest+0xc/0xb8
(XEN)





(XEN) grant_table.c:1249:d2 Expanding dom (2) grant table from (1) to (2) frames.
(XEN) mm.c:1192:d0 gnttab_mark_dirty not implemented yet
(XEN) Assertion 'svc == CSCHED_VCPU(curr_on_cpu(svc->vcpu->processor))' failed, line 351, file sched_credit.c
(XEN) Xen BUG at sched_credit.c:351
(XEN) CPU1: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) PC:     002414cc __bug+0x28/0x44
(XEN) CPSR:   200001da MODE:Hypervisor
(XEN)      R0: 002656dc R1: 00000001 R2: 3fd2bd80 R3: 200001da
(XEN)      R4: 0000015f R5: 0025d9d0 R6: 00000000 R7: 00000001
(XEN)      R8: 40027790 R9: 0000004f R10:00000001 R11:47fd7dd4 R12:00000002
(XEN) HYP: SP: 47fd7dcc LR: 002414cc
(XEN)
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 0002000045ca0000
(XEN)
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282835
(XEN)  TTBR0_EL2: 0000000076011000
(XEN)
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 000000000001c810
(XEN)      HDFAR: c8800f00
(XEN)      HIFAR: 83740628
(XEN)
(XEN) Xen stack trace from sp=47fd7dcc:
(XEN)    00000000 47fd7df4 00219cdc 0000004f 40027790 40006be8 00000000 00000001
(XEN)    61253ed2 47fd7ea4 0021beec 00000000 00000002 00000000 00000001 00000000
(XEN)    00000000 00006b68 00009afc 000000f4 00000000 6b689afc 0000004f 00000000
(XEN)    40006230 00000000 00000000 000000f4 00002400 47fd7ed0 a5ed4c00 40012100
(XEN)    00000001 00000001 002e4020 002e6380 002e6380 600001da 40012058 47fd7e7c
(XEN)    0022d4c8 3fd2bd80 40012040 47fd7ea4 0022e52c 4001202c 6b689afc 0000004f
(XEN)    40072000 002e4020 40012040 4001202c 00000001 47fd7f0c 00225f84 00000000
(XEN)    002524dc 47fd7eb8 40012028 00000000 ffffffff 5584561d 0000004f 40012100
(XEN)    40010038 4001c000 002b1ff0 002e6298 40012100 00000001 0fe20400 47fd7f54
(XEN)    00000001 002b1ff4 002ae000 002e7614 00265b80 002b1ff0 002e4254 47fd7f3c
(XEN)    00229b28 00000000 ffffffff 00000000 002ae000 002e7614 00000000 00000000
(XEN)    00000000 00000000 00000000 47fd7f44 00229bc0 47fd7f54 0024ee00 87fff000
(XEN)    00000000 47fd7f58 00251150 5ef90c06 800257bc 80028000 0000007f 87fff000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    ffffffff 00000000 800080e0 800001d3 c2c2c2c2 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 c2c2c2c2 00000000 00000000 00000000
(XEN)    00000000 00000000 c2c2c2c2 c2c2c2c2 c2c2c2c2
(XEN) Xen call trace:
(XEN)    [<002414cc>] __bug+0x28/0x44 (PC)
(XEN)    [<002414cc>] __bug+0x28/0x44 (LR)
(XEN)    [<00219cdc>] burn_credits+0x70/0x180
(XEN)    [<0021beec>] csched_schedule+0x164/0xee8
(XEN)    [<00225f84>] schedule+0x13c/0x8c8
(XEN)    [<00229b28>] __do_softirq+0xf4/0x100
(XEN)    [<00229bc0>] do_softirq+0x14/0x18
(XEN)    [<0024ee00>] leave_hypervisor_tail+0x50/0x6c
(XEN)    [<00251150>] return_to_guest+0xc/0xb8
(XEN)



(XEN) Xen BUG at sched_credit.c:232
(XEN) CPU0: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) PC:     00241e5c __bug+0x28/0x44
(XEN) CPSR:   200001da MODE:Hypervisor
(XEN)      R0: 002666dc R1: 00000001 R2: 00000000 R3: 200001da
(XEN)      R4: 000000e8 R5: 0025e9f0 R6: 00000000 R7: 00000000
(XEN)      R8: 01cab509 R9: 00000000 R10:00000000 R11:40077df4 R12:00000002
(XEN) HYP: SP: 40077dec LR: 00241e5c
(XEN)
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 000300004dce8000
(XEN)
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 00000000002828b5
(XEN)  TTBR0_EL2: 000000007fedf000
(XEN)
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 000000000001c810
(XEN)      HDFAR: c8800f00
(XEN)      HIFAR: 0040218a
(XEN)
(XEN) Xen stack trace from sp=40077dec:
(XEN)    00000000 40077ea4 0021c2dc 00000000 00000002 0000001e 00000000 00000000
(XEN)    00000000 000011b1 0000d3b3 40077eb8 00000000 11b1d3b3 0000000c 00000000
(XEN)    40006230 00000000 00000000 000000f4 00002400 40077ed0 2d85e600 0000000c
(XEN)    00000000 00000000 00000000 016e3600 002e4254 0000ffff 002e6380 40077e7c
(XEN)    0022a9a0 00000000 002e62c0 40077ea4 0022eb18 002e62ac 11b1d3b3 0000000c
(XEN)    4003b000 002e4020 002e62c0 002e62ac 00000000 40077f0c 00226454 00000000
(XEN)    00252f5c 40077eb8 002e62a8 00000000 00000000 11b1ba4f 0000000c 002e6380
(XEN)    40006c48 40006538 002b1ff0 40077ee4 0024d078 00000001 11b1ba4f 40077f0c
(XEN)    00000000 002b1ff4 002ae000 002e7614 00266b80 002b1ff0 002e4254 40077f3c
(XEN)    0022a078 00000000 ffffffff 002e7614 002ae000 002e7614 00000000 00000000
(XEN)    00000000 00000000 00000000 40077f44 0022a114 40077f54 0024f884 87fff000
(XEN)    00000000 40077f58 00251bd0 00000000 deadbeef deadbeef 00000021 87fff000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 deadbeef
(XEN)    ffffffff 00000000 8000813c 600001d3 c2c2c2c2 00000000 00000000 00000000
(XEN)    00000000 8000813c 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 c2c2c2c2 00000000 00000000 00000000
(XEN)    00000000 00000000 c2c2c2c2 c2c2c2c2 c2c2c2c2
(XEN) Xen call trace:
(XEN)    [<00241e5c>] __bug+0x28/0x44 (PC)
(XEN)    [<00241e5c>] __bug+0x28/0x44 (LR)
(XEN)    [<0021c2dc>] csched_schedule+0x2c4/0xf0c
(XEN)    [<00226454>] schedule+0x13c/0x8e8
(XEN)    [<0022a078>] __do_softirq+0xf8/0x104
(XEN)    [<0022a114>] do_softirq+0x14/0x18
(XEN)    [<0024f884>] leave_hypervisor_tail+0x50/0x6c
(XEN)    [<00251bd0>] return_to_guest+0xc/0xb8
(XEN)



(XEN) CPU0: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) PC:     00219ec8 burn_credits+0x3c/0x188
(XEN) CPSR:   8000015a MODE:Hypervisor
(XEN)      R0: 00000000 R1: 47fc24a0 R2: 29d34fbb R3: 00000016
(XEN)      R4: 40006528 R5: 00000000 R6: 40006230 R7: 400277e8
(XEN)      R8: 400277e8 R9: 002b1ff0 R10:002e4254 R11:40077e9c R12:00000000
(XEN) HYP: SP: 40077e7c LR: 0021bcf8
(XEN)
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 000300004dce8000
(XEN)
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282835
(XEN)  TTBR0_EL2: 000000007fedf000
(XEN)
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 00000000002c0010
(XEN)      HDFAR: e0800f00
(XEN)      HIFAR: 211000d5
(XEN)
(XEN) Xen stack trace from sp=40077e7c:
(XEN)    400277e8 00000016 40006528 00000000 40006230 400277e8 00266e00 40077ed4
(XEN)    0021bcf8 016e3600 002e4254 40077edc 00252f5c 40077eb8 002e6380 0021bb90
(XEN)    00000000 40006c48 40006538 002b1ff0 002e4254 40077eec 0022dcfc 00000016
(XEN)    29d322cf 00000016 002e6380 40077f0c 0022ddec 00000000 002b1ff0 002ae000
(XEN)    002e7614 00266b80 002b1ff0 40077f3c 0022a078 00000000 ffffffff 002e7614
(XEN)    002ae000 002e7614 00000000 00000000 00000000 00000000 00000000 40077f44
(XEN)    0022a114 40077f54 0024f884 87fff000 00000000 40077f58 00251bd0 00000000
(XEN)    deadbeef deadbeef 00000021 87fff000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 deadbeef ffffffff 00000000 8000813c 600001d3
(XEN)    c2c2c2c2 00000000 00000000 00000000 00000000 8000813c 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    c2c2c2c2 00000000 00000000 00000000 00000000 00000000 c2c2c2c2 c2c2c2c2
(XEN)    c2c2c2c2
(XEN) Xen call trace:
(XEN)    [<00219ec8>] burn_credits+0x3c/0x188 (PC)
(XEN)    [<0021bcf8>] csched_tick+0x168/0x488 (LR)
(XEN)    [<0021bcf8>] csched_tick+0x168/0x488
(XEN)    [<0022dcfc>] execute_timer+0x48/0x5c
(XEN)    [<0022ddec>] timer_softirq_action+0xdc/0x230
(XEN)    [<0022a078>] __do_softirq+0xf8/0x104
(XEN)    [<0022a114>] do_softirq+0x14/0x18
(XEN)    [<0024f884>] leave_hypervisor_tail+0x50/0x6c
(XEN)    [<00251bd0>] return_to_guest+0xc/0xb8
(XEN)
(XEN) CPU1: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) PC:     0022ed08 migrate_timer+0x1e8/0x3bc
(XEN) CPSR:   200001da MODE:Hypervisor
(XEN)      R0: 40006540 R1: 4007e930 R2: 002e6398 R3: 47fdf8f8
(XEN)      R4: 47fdf8f0 R5: 00000000 R6: 00000001 R7: 00000000
(XEN)      R8: 002e6380 R9: 002e6380 R10:002e6380 R11:47fd7e6c R12:00000000
(XEN) HYP: SP: 47fd7e3c LR: 0022ec4c
(XEN)
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 000200007ec30000
(XEN)
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282835
(XEN)  TTBR0_EL2: 0000000076011000
(XEN)
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 00000000002c0010
(XEN)      HDFAR: e0800f00
(XEN)      HIFAR: 83740400
(XEN)
(XEN) Xen stack trace from sp=47fd7e3c:
(XEN)    0022ec4c 47fdf908 200001da 00007fff 47fdf000 47fdf928 00282835 4007e000
(XEN)    00000001 002e62ac 00000000 47fd7e7c 00250f58 47fdf000 47fde000 47fd7e94
(XEN)    00244a60 002e6500 4007e000 47fdf000 47fdf000 47fd7ea4 00244f6c 29ad035e
(XEN)    00000016 47fd7f0c 00226bd8 00000000 00252f5c 47fd7eb8 002e62a8 01c9c380
(XEN)    00000000 29ac9722 00000016 002e6380 40006c48 4007e000 002b1ff0 01c9c380
(XEN)    00000000 00000000 29ac9722 47fd7f0c 00000000 002b1ff4 002ae000 002e7614
(XEN)    00266b80 002b1ff0 002e4254 47fd7f3c 0022a078 00000000 ffffffff 002e7614
(XEN)    002ae000 002e7614 b6e7e4cc 000000a0 b6fde6b0 00000058 000000a8 47fd7f44
(XEN)    0022a114 47fd7f54 0024f884 b6fde6b0 b6fde6a8 47fd7f58 00251bd0 b6fde6b0
(XEN)    00000001 00000000 00000001 b6fde6b0 b6fde6a8 b6e7e4cc 000000a0 b6fde6b0
(XEN)    00000058 000000a8 bee268f0 b6ecf0b0 ffffffff b6df37b1 b6df37be 600f0030
(XEN)    c2c2c2c2 bee267a8 c0723b40 c0012400 de455ff8 b6e2502c c0723b4c c0012540
(XEN)    c0723b58 c00125e0 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    c2c2c2c2 400f0010 a00f0193 80000093 600f0193 00000000 c2c2c2c2 c2c2c2c2
(XEN)    c2c2c2c2
(XEN) Xen call trace:
(XEN)    [<0022ed08>] migrate_timer+0x1e8/0x3bc (PC)
(XEN)    [<0022ec4c>] migrate_timer+0x12c/0x3bc (LR)
(XEN)    [<00250f58>] virt_timer_restore+0x48/0x78
(XEN)    [<00244a60>] schedule_tail+0x298/0x2f4
(XEN)    [<00244f6c>] context_switch+0x120/0x124
(XEN)    [<00226bd8>] schedule+0x8c0/0x8e8
(XEN)    [<0022a078>] __do_softirq+0xf8/0x104
(XEN)    [<0022a114>] do_softirq+0x14/0x18
(XEN)    [<0024f884>] leave_hypervisor_tail+0x50/0x6c
(XEN)    [<00251bd0>] return_to_guest+0xc/0xb8


[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14  8:02 ` Fwd: Debugging "Xen BUG at ... list.h" on ARM Thomas Leonard
@ 2014-04-14  8:06   ` Thomas Leonard
  2014-04-14  9:12   ` Fwd: " Julien Grall
  1 sibling, 0 replies; 14+ messages in thread
From: Thomas Leonard @ 2014-04-14  8:06 UTC (permalink / raw)
  To: xen-devel

On 14 April 2014 09:02, Thomas Leonard <tal36@cam.ac.uk> wrote:
[...]

Note: please ignore the attachment on my previous message, which
belonged to an earlier draft. It shows some Xen console output for my
actual code, rather than the simplified test case.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14  8:02 ` Fwd: Debugging "Xen BUG at ... list.h" on ARM Thomas Leonard
  2014-04-14  8:06   ` Thomas Leonard
@ 2014-04-14  9:12   ` Julien Grall
  2014-04-14  9:30     ` Thomas Leonard
  1 sibling, 1 reply; 14+ messages in thread
From: Julien Grall @ 2014-04-14  9:12 UTC (permalink / raw)
  To: Thomas Leonard, xen-devel, Ian Campbell, george.dunlap,
	Dario Faggioli

(Adding a couple of people in cc)

On 14/04/14 09:02, Thomas Leonard wrote:
> Hi all,

Hello,

> I'm new to Xen, and trying to get the arm32 port of Mini-OS working on
> a Cubieboard2. I have the board running an Ubuntu Dom0 and an Ubuntu
> guest happily, and I've written up the instructions for that here:
>
>    https://github.com/talex5/mirage-www/blob/master/tmpl/wiki/xen-on-cubieboard2.md
>
> However, while trying to get a Mini-OS domU guest working I can now
> reliably crash the hypervisor. No doubt this is a bug in my guest
> code, but I assume it shouldn't be possible for a buggy guest to take
> down Xen itself.
>
> Here's a simplified version of my code which shows the problem:
>
>    https://github.com/talex5/xen-paratest
>
> I'm building Xen from the stable-4.4 Git branch (03eb51340), compiled
> with debug on.
>
> The code is rather simple: it's just trying to initialise the data
> cache, as described in the "ARM Cortex-A Series Programmer’s Guide".
> I'm not actually sure whether this is necessary for a paravirtualised
> guest, but I was getting some odd problems which looked cache related,
> so I added this code in the hope of fixing that.

Does it crash if you remove your cache code?

>
> The cfg file is just:
>
> kernel = "/root/paratest.img"
> memory = 128
> name = "paratest"
> vcpus = 1
> serial="pty"
>
> When the guest is started, Xen always crashes. Sometimes silently, but
> usually writing a stack-trace to the console. I've included a
> selection of them in the commit messages, but a typical one looks like
> this:

hmmm, I've just tried your kernel on midway and Xen didn't crash.

Do you have more input? Number of physical CPUs used, state of the VCPU...?

>      (XEN) Assertion 'svc ==
> CSCHED_VCPU(curr_on_cpu(svc->vcpu->processor))' failed, line 859, file
> sche
>      (XEN) Xen BUG at sched_credit.c:859

I can't find any assert at this line. Did you modify Xen code?

>      (XEN) CPU1: Unexpected Trap: Undefined Instruction
>      (XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
>      (XEN) CPU:    1
>      (XEN) PC:     002417b4 __bug+0x28/0x44
>      (XEN) CPSR:   2000015a MODE:Hypervisor
>      (XEN)      R0: 002656dc R1: 00000000 R2: 3fd2bd80 R3: 2000015a
>      (XEN)      R4: 0000035b R5: 0025d9d0 R6: 40006230 R7: 47fe4be0
>      (XEN)      R8: 00265e00 R9: 002b1ff0 R10:002e4254 R11:47fd7e9c R12:00000001
>      (XEN) HYP: SP: 47fd7e94 LR: 002417b4
> [...]
>      (XEN) Xen call trace:
>      (XEN)    [<002417b4>] __bug+0x28/0x44 (PC)
>      (XEN)    [<002417b4>] __bug+0x28/0x44 (LR)
>      (XEN)    [<0021bac0>] csched_tick+0x1ac/0x4d4
>      (XEN)    [<0022d820>] execute_timer+0xcc/0xe0
>      (XEN)    [<0022d910>] timer_softirq_action+0xdc/0x230
>      (XEN)    [<00229b88>] __do_softirq+0xf4/0x100
>      (XEN)    [<00229c20>] do_softirq+0x14/0x18
>      (XEN)    [<0024f100>] leave_hypervisor_tail+0x50/0x6c
>      (XEN)    [<00251450>] return_to_guest+0xc/0xb8

Regards,


-- 
Julien Grall

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14  9:12   ` Fwd: " Julien Grall
@ 2014-04-14  9:30     ` Thomas Leonard
  2014-04-14  9:43       ` Ian Campbell
  2014-04-14 12:00       ` Ian Campbell
  0 siblings, 2 replies; 14+ messages in thread
From: Thomas Leonard @ 2014-04-14  9:30 UTC (permalink / raw)
  To: Julien Grall; +Cc: Dario Faggioli, george.dunlap, Ian Campbell, xen-devel

On 14 April 2014 10:12, Julien Grall <julien.grall@linaro.org> wrote:
> (Adding a couple of people in cc)
>
> On 14/04/14 09:02, Thomas Leonard wrote:
>>
>> Hi all,
>
>
> Hello,
>
>
>> I'm new to Xen, and trying to get the arm32 port of Mini-OS working on
>> a Cubieboard2. I have the board running an Ubuntu Dom0 and an Ubuntu
>> guest happily, and I've written up the instructions for that here:
>>
>>
>> https://github.com/talex5/mirage-www/blob/master/tmpl/wiki/xen-on-cubieboard2.md
>>
>> However, while trying to get a Mini-OS domU guest working I can now
>> reliably crash the hypervisor. No doubt this is a bug in my guest
>> code, but I assume it shouldn't be possible for a buggy guest to take
>> down Xen itself.
>>
>> Here's a simplified version of my code which shows the problem:
>>
>>    https://github.com/talex5/xen-paratest
>>
>> I'm building Xen from the stable-4.4 Git branch (03eb51340), compiled
>> with debug on.
>>
>> The code is rather simple: it's just trying to initialise the data
>> cache, as described in the "ARM Cortex-A Series Programmer’s Guide".
>> I'm not actually sure whether this is necessary for a paravirtualised
>> guest, but I was getting some odd problems which looked cache related,
>> so I added this code in the hope of fixing that.
>
>
> Does it crash if you remove your cache code?

No, it just produces the wrong output in that case.

For background: I was making a hypervisor call to write out a string,
then modifying a second string, then make a call to write that (both
using HYPERVISOR_console_io).

Xen printed the second string with its first part containing the old
text and the second part containing the new text. It looked like the
old part was the bytes on the same cache line as the first string I
printed, and I was trying to debug that.

>> The cfg file is just:
>>
>> kernel = "/root/paratest.img"
>> memory = 128
>> name = "paratest"
>> vcpus = 1
>> serial="pty"
>>
>> When the guest is started, Xen always crashes. Sometimes silently, but
>> usually writing a stack-trace to the console. I've included a
>> selection of them in the commit messages, but a typical one looks like
>> this:
>
> hmmm, I've just tried your kernel on midway and Xen didn't crash.
>
> Do you have more input? Number of physical CPUs used, state of the VCPU...?

There are 2 physical CPUs. I'm not sure what you mean by the VCPU
state (beyond what's in the dump).

>>      (XEN) Assertion 'svc ==
>> CSCHED_VCPU(curr_on_cpu(svc->vcpu->processor))' failed, line 859, file
>> sche
>>      (XEN) Xen BUG at sched_credit.c:859
>
>
> I can't find any assert at this line. Did you modify Xen code?

Sorry, you're right, I added an extra assert there. Here's a stack
trace with a clean build (stable-4.4 branch + debug=y):

(XEN) Assertion 'entry->next->prev == entry' failed, line 175, file
/home/tal/work/cubieboard/xen/xen/include/xen/list.h
(XEN) Xen BUG at /home/tal/work/cubieboard/xen/xen/include/xen/list.h:175
(XEN) CPU1: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) PC:     002414cc __bug+0x28/0x44
(XEN) CPSR:   200e01da MODE:Hypervisor
(XEN)      R0: 002656dc R1: 00000001 R2: 3fd2bd80 R3: 200e01da
(XEN)      R4: 000000af R5: 0025d238 R6: 002e4020 R7: 002e6380
(XEN)      R8: 002e6380 R9: 600e01da R10:4003a940 R11:40037e34 R12:00000002
(XEN) HYP: SP: 40037e2c LR: 002414cc
(XEN)
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 000100007ec06000
(XEN)
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282835
(XEN)  TTBR0_EL2: 0000000076011000
(XEN)
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 000000000001c810
(XEN)      HDFAR: c8800f00
(XEN)      HIFAR: c87400a0
(XEN)
(XEN) Xen stack trace from sp=40037e2c:
(XEN)    00000000 40037e64 0022e32c 15882183 00000007 062e43e6 062e43e6 00000000
(XEN)    4003a000 40004000 00000002 002e62ac 00000000 40037e7c 00250480 077e6000
(XEN)    40004000 4003a000 4003b000 40037e94 00243e70 002e6500 40004000 4003b000
(XEN)    4003b000 40037ea4 002445b8 14f1129d 00000007 40037f0c 002266e8 00000000
(XEN)    4003b7c8 40037efc 002e62a8 ffffffff ffffffff 002e6588 0024f9d0 40037eec
(XEN)    0022884c 40004000 00000000 ffffffff ffffffff 40037f00 c07063f8 40037f54
(XEN)    00000000 002b1ff4 002ae000 002e7614 00265b80 002b1ff0 002e4254 40037f3c
(XEN)    00229b28 00000000 ffffffff 002e7614 002ae000 002e7614 c07063f8 c07414d7
(XEN)    c050078c c06fe000 c07414d7 40037f44 00229bc0 40037f54 0024ee00 c0706474
(XEN)    c06fe000 40037f58 00251150 ffffffed 077de000 c070746c 00000000 c0706474
(XEN)    c06fe000 c07063f8 c07414d7 c050078c c06fe000 c07414d7 c06fe000 c070d900
(XEN)    ffffffff 0000d43b c0019d28 600e0093 00000000 be9695f4 c0742080 c0012400
(XEN)    c06fffa8 c000f600 c074208c c0012760 c0742098 c00125e0 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 600e0013 800f0193 800f0093
(XEN)    600e0193 00000000 00000000 030c3080 451a8c04
(XEN) Xen call trace:
(XEN)    [<002414cc>] __bug+0x28/0x44 (PC)
(XEN)    [<002414cc>] __bug+0x28/0x44 (LR)
(XEN)    [<0022e32c>] set_timer+0x1d0/0x278
(XEN)    [<00250480>] virt_timer_save+0x88/0x90
(XEN)    [<00243e70>] schedule_tail+0x48/0x2f4
(XEN)    [<002445b8>] context_switch+0x114/0x118
(XEN)    [<002266e8>] schedule+0x8a0/0x8c8
(XEN)    [<00229b28>] __do_softirq+0xf4/0x100
(XEN)    [<00229bc0>] do_softirq+0x14/0x18
(XEN)    [<0024ee00>] leave_hypervisor_tail+0x50/0x6c
(XEN)    [<00251150>] return_to_guest+0xc/0xb8

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14  9:30     ` Thomas Leonard
@ 2014-04-14  9:43       ` Ian Campbell
  2014-04-14 10:10         ` Thomas Leonard
  2014-04-14 12:00       ` Ian Campbell
  1 sibling, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2014-04-14  9:43 UTC (permalink / raw)
  To: Thomas Leonard; +Cc: Dario Faggioli, Julien Grall, george.dunlap, xen-devel

On Mon, 2014-04-14 at 10:30 +0100, Thomas Leonard wrote:
> On 14 April 2014 10:12, Julien Grall <julien.grall@linaro.org> wrote:
> > (Adding a couple of people in cc)
> >
> > On 14/04/14 09:02, Thomas Leonard wrote:
> >>
> >> Hi all,
> >
> >
> > Hello,
> >
> >
> >> I'm new to Xen, and trying to get the arm32 port of Mini-OS working on
> >> a Cubieboard2. I have the board running an Ubuntu Dom0 and an Ubuntu
> >> guest happily, and I've written up the instructions for that here:
> >>
> >>
> >> https://github.com/talex5/mirage-www/blob/master/tmpl/wiki/xen-on-cubieboard2.md
> >>
> >> However, while trying to get a Mini-OS domU guest working I can now
> >> reliably crash the hypervisor. No doubt this is a bug in my guest
> >> code, but I assume it shouldn't be possible for a buggy guest to take
> >> down Xen itself.
> >>
> >> Here's a simplified version of my code which shows the problem:
> >>
> >>    https://github.com/talex5/xen-paratest
> >>
> >> I'm building Xen from the stable-4.4 Git branch (03eb51340), compiled
> >> with debug on.
> >>
> >> The code is rather simple: it's just trying to initialise the data
> >> cache, as described in the "ARM Cortex-A Series Programmer’s Guide".
> >> I'm not actually sure whether this is necessary for a paravirtualised
> >> guest, but I was getting some odd problems which looked cache related,
> >> so I added this code in the hope of fixing that.
> >
> >
> > Does it crash if you remove your cache code?
> 
> No, it just produces the wrong output in that case.
> 
> For background: I was making a hypervisor call to write out a string,
> then modifying a second string, then make a call to write that (both
> using HYPERVISOR_console_io).
> 
> Xen printed the second string with its first part containing the old
> text and the second part containing the new text. It looked like the
> old part was the bytes on the same cache line as the first string I
> printed, and I was trying to debug that.

Has your guest enabled caching (SCTLR.C)? This is required by the
hypercall ABI. See
http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi

(the hypervisor still shouldn't crash though)

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14  9:43       ` Ian Campbell
@ 2014-04-14 10:10         ` Thomas Leonard
  2014-04-14 10:29           ` Julien Grall
  0 siblings, 1 reply; 14+ messages in thread
From: Thomas Leonard @ 2014-04-14 10:10 UTC (permalink / raw)
  To: Ian Campbell; +Cc: Dario Faggioli, Julien Grall, george.dunlap, xen-devel

On 14 April 2014 10:43, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2014-04-14 at 10:30 +0100, Thomas Leonard wrote:
>> On 14 April 2014 10:12, Julien Grall <julien.grall@linaro.org> wrote:
>> > (Adding a couple of people in cc)
>> >
>> > On 14/04/14 09:02, Thomas Leonard wrote:
>> >>
>> >> Hi all,
>> >
>> >
>> > Hello,
>> >
>> >
>> >> I'm new to Xen, and trying to get the arm32 port of Mini-OS working on
>> >> a Cubieboard2. I have the board running an Ubuntu Dom0 and an Ubuntu
>> >> guest happily, and I've written up the instructions for that here:
>> >>
>> >>
>> >> https://github.com/talex5/mirage-www/blob/master/tmpl/wiki/xen-on-cubieboard2.md
>> >>
>> >> However, while trying to get a Mini-OS domU guest working I can now
>> >> reliably crash the hypervisor. No doubt this is a bug in my guest
>> >> code, but I assume it shouldn't be possible for a buggy guest to take
>> >> down Xen itself.
>> >>
>> >> Here's a simplified version of my code which shows the problem:
>> >>
>> >>    https://github.com/talex5/xen-paratest
>> >>
>> >> I'm building Xen from the stable-4.4 Git branch (03eb51340), compiled
>> >> with debug on.
>> >>
>> >> The code is rather simple: it's just trying to initialise the data
>> >> cache, as described in the "ARM Cortex-A Series Programmer’s Guide".
>> >> I'm not actually sure whether this is necessary for a paravirtualised
>> >> guest, but I was getting some odd problems which looked cache related,
>> >> so I added this code in the hope of fixing that.
>> >
>> >
>> > Does it crash if you remove your cache code?
>>
>> No, it just produces the wrong output in that case.
>>
>> For background: I was making a hypervisor call to write out a string,
>> then modifying a second string, then make a call to write that (both
>> using HYPERVISOR_console_io).
>>
>> Xen printed the second string with its first part containing the old
>> text and the second part containing the new text. It looked like the
>> old part was the bytes on the same cache line as the first string I
>> printed, and I was trying to debug that.
>
> Has your guest enabled caching (SCTLR.C)? This is required by the
> hypercall ABI. See
> http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi

Here's my attempt to set up the translation table and enable caching:

https://github.com/talex5/xen-paratest/tree/cache-problem

It prints:

(d11) 012
(d11) ....++++++++++++++++++++++

The second line should be all "+"s.

> (the hypervisor still shouldn't crash though)
>
> Ian.



-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14 10:10         ` Thomas Leonard
@ 2014-04-14 10:29           ` Julien Grall
  2014-04-14 10:53             ` Ian Campbell
  2014-04-14 11:03             ` Thomas Leonard
  0 siblings, 2 replies; 14+ messages in thread
From: Julien Grall @ 2014-04-14 10:29 UTC (permalink / raw)
  To: Thomas Leonard; +Cc: Dario Faggioli, george.dunlap, Ian Campbell, xen-devel

On 04/14/2014 11:10 AM, Thomas Leonard wrote:
>> Has your guest enabled caching (SCTLR.C)? This is required by the
>> hypercall ABI. See
>> http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi
> 
> Here's my attempt to set up the translation table and enable caching:
> 
> https://github.com/talex5/xen-paratest/tree/cache-problem

Your page table looks wrong for me. Each page table should be write-back
or write-through.

Regards,

-- 
Julien Grall

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14 10:29           ` Julien Grall
@ 2014-04-14 10:53             ` Ian Campbell
  2014-04-14 11:03             ` Thomas Leonard
  1 sibling, 0 replies; 14+ messages in thread
From: Ian Campbell @ 2014-04-14 10:53 UTC (permalink / raw)
  To: Julien Grall; +Cc: Dario Faggioli, Thomas Leonard, george.dunlap, xen-devel

On Mon, 2014-04-14 at 11:29 +0100, Julien Grall wrote:
> On 04/14/2014 11:10 AM, Thomas Leonard wrote:
> >> Has your guest enabled caching (SCTLR.C)? This is required by the
> >> hypercall ABI. See
> >> http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi
> > 
> > Here's my attempt to set up the translation table and enable caching:
> > 
> > https://github.com/talex5/xen-paratest/tree/cache-problem
> 
> Your page table looks wrong for me. Each page table should be write-back
> or write-through.

I also don't think it's strictly speaking necessary to enable paging to
enable the caches (although I appreciate you might want to do so for
reasons unrelated to using HYPERVISOR_console_io)

Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14 10:29           ` Julien Grall
  2014-04-14 10:53             ` Ian Campbell
@ 2014-04-14 11:03             ` Thomas Leonard
  2014-04-14 11:47               ` Ian Campbell
  1 sibling, 1 reply; 14+ messages in thread
From: Thomas Leonard @ 2014-04-14 11:03 UTC (permalink / raw)
  To: Julien Grall; +Cc: Dario Faggioli, george.dunlap, Ian Campbell, xen-devel

On 14 April 2014 11:29, Julien Grall <julien.grall@linaro.org> wrote:
> On 04/14/2014 11:10 AM, Thomas Leonard wrote:
>>> Has your guest enabled caching (SCTLR.C)? This is required by the
>>> hypercall ABI. See
>>> http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi
>>
>> Here's my attempt to set up the translation table and enable caching:
>>
>> https://github.com/talex5/xen-paratest/tree/cache-problem
>
> Your page table looks wrong for me. Each page table should be write-back
> or write-through.

Thanks - that fixed my test case!


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14 11:03             ` Thomas Leonard
@ 2014-04-14 11:47               ` Ian Campbell
  2014-04-14 11:53                 ` Thomas Leonard
  0 siblings, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2014-04-14 11:47 UTC (permalink / raw)
  To: Thomas Leonard; +Cc: Dario Faggioli, Julien Grall, george.dunlap, xen-devel

On Mon, 2014-04-14 at 12:03 +0100, Thomas Leonard wrote:
> On 14 April 2014 11:29, Julien Grall <julien.grall@linaro.org> wrote:
> > On 04/14/2014 11:10 AM, Thomas Leonard wrote:
> >>> Has your guest enabled caching (SCTLR.C)? This is required by the
> >>> hypercall ABI. See
> >>> http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi
> >>
> >> Here's my attempt to set up the translation table and enable caching:
> >>
> >> https://github.com/talex5/xen-paratest/tree/cache-problem
> >
> > Your page table looks wrong for me. Each page table should be write-back
> > or write-through.
> 
> Thanks - that fixed my test case!

Excellent.

Can you also just confirm for my peace of mind that the hypervisor crash
was only reproducible with a locally modified hypervisor?

Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14 11:47               ` Ian Campbell
@ 2014-04-14 11:53                 ` Thomas Leonard
  2014-04-14 11:57                   ` Ian Campbell
  0 siblings, 1 reply; 14+ messages in thread
From: Thomas Leonard @ 2014-04-14 11:53 UTC (permalink / raw)
  To: Ian Campbell; +Cc: Dario Faggioli, Julien Grall, george.dunlap, xen-devel

On 14 April 2014 12:47, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2014-04-14 at 12:03 +0100, Thomas Leonard wrote:
>> On 14 April 2014 11:29, Julien Grall <julien.grall@linaro.org> wrote:
>> > On 04/14/2014 11:10 AM, Thomas Leonard wrote:
>> >>> Has your guest enabled caching (SCTLR.C)? This is required by the
>> >>> hypercall ABI. See
>> >>> http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi
>> >>
>> >> Here's my attempt to set up the translation table and enable caching:
>> >>
>> >> https://github.com/talex5/xen-paratest/tree/cache-problem
>> >
>> > Your page table looks wrong for me. Each page table should be write-back
>> > or write-through.
>>
>> Thanks - that fixed my test case!
>
> Excellent.
>
> Can you also just confirm for my peace of mind that the hypervisor crash
> was only reproducible with a locally modified hypervisor?

The crash happens with the unmodified hypervisor too. I added the
extra asserts only because it was crashing. Here's an example:

http://lists.xen.org/archives/html/xen-devel/2014-04/msg01702.html


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14 11:53                 ` Thomas Leonard
@ 2014-04-14 11:57                   ` Ian Campbell
  0 siblings, 0 replies; 14+ messages in thread
From: Ian Campbell @ 2014-04-14 11:57 UTC (permalink / raw)
  To: Thomas Leonard; +Cc: Dario Faggioli, Julien Grall, george.dunlap, xen-devel

On Mon, 2014-04-14 at 12:53 +0100, Thomas Leonard wrote:
> On 14 April 2014 12:47, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Mon, 2014-04-14 at 12:03 +0100, Thomas Leonard wrote:
> >> On 14 April 2014 11:29, Julien Grall <julien.grall@linaro.org> wrote:
> >> > On 04/14/2014 11:10 AM, Thomas Leonard wrote:
> >> >>> Has your guest enabled caching (SCTLR.C)? This is required by the
> >> >>> hypercall ABI. See
> >> >>> http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi
> >> >>
> >> >> Here's my attempt to set up the translation table and enable caching:
> >> >>
> >> >> https://github.com/talex5/xen-paratest/tree/cache-problem
> >> >
> >> > Your page table looks wrong for me. Each page table should be write-back
> >> > or write-through.
> >>
> >> Thanks - that fixed my test case!
> >
> > Excellent.
> >
> > Can you also just confirm for my peace of mind that the hypervisor crash
> > was only reproducible with a locally modified hypervisor?
> 
> The crash happens with the unmodified hypervisor too. I added the
> extra asserts only because it was crashing. Here's an example:
> 
> http://lists.xen.org/archives/html/xen-devel/2014-04/msg01702.html

OK, thanks.

Ian.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14  9:30     ` Thomas Leonard
  2014-04-14  9:43       ` Ian Campbell
@ 2014-04-14 12:00       ` Ian Campbell
  2014-04-14 12:25         ` Thomas Leonard
  1 sibling, 1 reply; 14+ messages in thread
From: Ian Campbell @ 2014-04-14 12:00 UTC (permalink / raw)
  To: Thomas Leonard, Stefano Stabellini
  Cc: Dario Faggioli, Julien Grall, george.dunlap, xen-devel

On Mon, 2014-04-14 at 10:30 +0100, Thomas Leonard wrote:
> Sorry, you're right, I added an extra assert there. Here's a stack
> trace with a clean build (stable-4.4 branch + debug=y):
> 
> (XEN) Assertion 'entry->next->prev == entry' failed, line 175, file
> /home/tal/work/cubieboard/xen/xen/include/xen/list.h
> (XEN) Xen BUG at /home/tal/work/cubieboard/xen/xen/include/xen/list.h:175
> (XEN) CPU1: Unexpected Trap: Undefined Instruction
> (XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
> (XEN) CPU:    1
> (XEN) PC:     002414cc __bug+0x28/0x44
> (XEN) CPSR:   200e01da MODE:Hypervisor
> (XEN)      R0: 002656dc R1: 00000001 R2: 3fd2bd80 R3: 200e01da
> (XEN)      R4: 000000af R5: 0025d238 R6: 002e4020 R7: 002e6380
> (XEN)      R8: 002e6380 R9: 600e01da R10:4003a940 R11:40037e34 R12:00000002
> (XEN) HYP: SP: 40037e2c LR: 002414cc
> (XEN)
> (XEN)   VTCR_EL2: 80002558
> (XEN)  VTTBR_EL2: 000100007ec06000
> (XEN)
> (XEN)  SCTLR_EL2: 30cd187f
> (XEN)    HCR_EL2: 0000000000282835
> (XEN)  TTBR0_EL2: 0000000076011000
> (XEN)
> (XEN)    ESR_EL2: 00000000
> (XEN)  HPFAR_EL2: 000000000001c810
> (XEN)      HDFAR: c8800f00
> (XEN)      HIFAR: c87400a0
> (XEN)
> (XEN) Xen stack trace from sp=40037e2c:
> (XEN)    00000000 40037e64 0022e32c 15882183 00000007 062e43e6 062e43e6 00000000
> (XEN)    4003a000 40004000 00000002 002e62ac 00000000 40037e7c 00250480 077e6000
> (XEN)    40004000 4003a000 4003b000 40037e94 00243e70 002e6500 40004000 4003b000
> (XEN)    4003b000 40037ea4 002445b8 14f1129d 00000007 40037f0c 002266e8 00000000
> (XEN)    4003b7c8 40037efc 002e62a8 ffffffff ffffffff 002e6588 0024f9d0 40037eec
> (XEN)    0022884c 40004000 00000000 ffffffff ffffffff 40037f00 c07063f8 40037f54
> (XEN)    00000000 002b1ff4 002ae000 002e7614 00265b80 002b1ff0 002e4254 40037f3c
> (XEN)    00229b28 00000000 ffffffff 002e7614 002ae000 002e7614 c07063f8 c07414d7
> (XEN)    c050078c c06fe000 c07414d7 40037f44 00229bc0 40037f54 0024ee00 c0706474
> (XEN)    c06fe000 40037f58 00251150 ffffffed 077de000 c070746c 00000000 c0706474
> (XEN)    c06fe000 c07063f8 c07414d7 c050078c c06fe000 c07414d7 c06fe000 c070d900
> (XEN)    ffffffff 0000d43b c0019d28 600e0093 00000000 be9695f4 c0742080 c0012400
> (XEN)    c06fffa8 c000f600 c074208c c0012760 c0742098 c00125e0 00000000 00000000
> (XEN)    00000000 00000000 00000000 00000000 00000000 600e0013 800f0193 800f0093
> (XEN)    600e0193 00000000 00000000 030c3080 451a8c04
> (XEN) Xen call trace:
> (XEN)    [<002414cc>] __bug+0x28/0x44 (PC)
> (XEN)    [<002414cc>] __bug+0x28/0x44 (LR)
> (XEN)    [<0022e32c>] set_timer+0x1d0/0x278
> (XEN)    [<00250480>] virt_timer_save+0x88/0x90


This presumably implies that the timer is already active, which it
really shouldn't be while the vcpu is scheduled.

The guest here is the same stuff as
https://github.com/talex5/xen-paratest ?

Ian.

> (XEN)    [<00243e70>] schedule_tail+0x48/0x2f4
> (XEN)    [<002445b8>] context_switch+0x114/0x118
> (XEN)    [<002266e8>] schedule+0x8a0/0x8c8
> (XEN)    [<00229b28>] __do_softirq+0xf4/0x100
> (XEN)    [<00229bc0>] do_softirq+0x14/0x18
> (XEN)    [<0024ee00>] leave_hypervisor_tail+0x50/0x6c
> (XEN)    [<00251150>] return_to_guest+0xc/0xb8

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Fwd: Debugging "Xen BUG at ... list.h" on ARM
  2014-04-14 12:00       ` Ian Campbell
@ 2014-04-14 12:25         ` Thomas Leonard
  0 siblings, 0 replies; 14+ messages in thread
From: Thomas Leonard @ 2014-04-14 12:25 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Dario Faggioli, Julien Grall, Stefano Stabellini, george.dunlap,
	xen-devel

[-- Attachment #1: Type: text/plain, Size: 1319 bytes --]

On 14 April 2014 13:00, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Mon, 2014-04-14 at 10:30 +0100, Thomas Leonard wrote:
>> Sorry, you're right, I added an extra assert there. Here's a stack
>> trace with a clean build (stable-4.4 branch + debug=y):
>>
>> (XEN) Assertion 'entry->next->prev == entry' failed, line 175, file
>> /home/tal/work/cubieboard/xen/xen/include/xen/list.h
>> (XEN) Xen BUG at /home/tal/work/cubieboard/xen/xen/include/xen/list.h:175
>> (XEN) CPU1: Unexpected Trap: Undefined Instruction
>> (XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
[...]
>> (XEN) Xen call trace:
>> (XEN)    [<002414cc>] __bug+0x28/0x44 (PC)
>> (XEN)    [<002414cc>] __bug+0x28/0x44 (LR)
>> (XEN)    [<0022e32c>] set_timer+0x1d0/0x278
>> (XEN)    [<00250480>] virt_timer_save+0x88/0x90
>
>
> This presumably implies that the timer is already active, which it
> really shouldn't be while the vcpu is scheduled.
>
> The guest here is the same stuff as
> https://github.com/talex5/xen-paratest ?

Yes (the master branch, commit 22e3b9bf). It doesn't always give the
same output, even from clean boot. Here are some more samples
(attached).


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

[-- Attachment #2: xen-crashes.txt --]
[-- Type: text/plain, Size: 13265 bytes --]

[   43.332044] Alignment trap: not handling instruction e1903f9f at [<c003c660>]
[   43.332092] Unhandled fault: alignment exception (0x001) at 0xc5b4a2ff
[   43.332118] Internal error: : 1 [#1] SMP ARM
[   43.332136] Modules linked in:
[   43.332161] CPU: 1 PID: 1063 Comm: kworker/u4:0 Not tainted 3.14.0-rc6-00163-g4c89db4 #7
[   43.332191] task: c58aed00 ti: c591a000 task.ti: c591a000
[   43.332222] PC is at commit_creds+0x44/0x250
[   43.332244] LR is at ____call_usermodehelper+0x130/0x16c
[   43.332264] pc : [<c003c664>]    lr : [<c002e814>]    psr: 200f0013
[   43.332264] sp : c591bf78  ip : 00000000  fp : 00000000
[   43.332296] r10: 00000000  r9 : 00000000  r8 : 00000000
[   43.332314] r7 : 00000000  r6 : c58aed00  r5 : c5b4a2ff  r4 : c5815c00
[   43.332334] r3 : 00000001  r2 : 00000001  r1 : 00000000  r0 : c5b4a2ff
[   43.332357] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
[   43.332380] Control: 10c5387d  Table: 6d9d006a  DAC: 00000015
[   43.332400] Process kworker/u4:0 (pid: 1063, stack limit = 0xc591a240)
[   43.332422] Stack: (0xc591bf78 to 0xc591c000)
[   43.332441] bf60:                                                       00000000 0000000
0
[   43.332468] bf80: 00000000 c5b4a2ff c5b4a2ff c589e300 00000000 00000000 00000000 c002e81
4
[   43.332496] bfa0: c589e300 c002e6e4 00000000 c000eab8 00000000 00000000 00000000 0000000
0
[   43.332523] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000
0
[   43.332550] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 0000000
0
[   43.332591] [<c003c664>] (commit_creds) from [<c002e814>] (____call_usermodehelper+0x130
/0x16c)
[   43.332626] [<c002e814>] (____call_usermodehelper) from [<c000eab8>] (ret_from_fork+0x14
/0x3c)
[   43.332657] Code: e3530000 da000082 f590f000 e1903f9f (e2833001)
[   43.332682] ---[ end trace 84ae17060566a4e5 ]---
[   43.339266] Alignment trap: not handling instruction e1931f9f at [<c003c358>]
[   43.339320] Unhandled fault: alignment exception (0x001) at 0xc5815379
[   43.339345] Internal error: : 1 [#2] SMP ARM
[   43.339363] Modules linked in:
[   43.339389] CPU: 0 PID: 730 Comm: rs:main Q:Reg Tainted: G      D      3.14.0-rc6-00163-g4c89db4 #7
[   43.339417] task: c3ec43c0 ti: c3c24000 task.ti: c3c24000
[   43.339449] PC is at override_creds+0x24/0x58
[   43.339471] LR is at SyS_faccessat+0x74/0x1cc
[   43.339490] pc : [<c003c35c>]    lr : [<c00b71c0>]    psr: 20070013
[   43.339490] sp : c3c25f70  ip : 00000000  fp : 00000000
[   43.339522] r10: 00000000  r9 : 00000000  r8 : 00000010
[   43.339541] r7 : b6f6a868  r6 : ffffff9c  r5 : c5815379  r4 : 00000001
[   43.339561] r3 : c5815379  r2 : c594f880  r1 : 00000001  r0 : c5815379
[   43.339584] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   43.339606] Control: 10c5387d  Table: 6d90806a  DAC: 00000015
[   43.339627] Process rs:main Q:Reg (pid: 730, stack limit = 0xc3c24240)
[   43.339648] Stack: (0xc3c25f70 to 0xc3c26000)
[   43.339668] 5f60:                                     00000000 0000004e c000eba4 b6df43f0
[   43.339696] 5f80: 00000008 b6f6a868 b6f7a970 00039690 00000021 c000eba4 c3c24000 00000000
[   43.339724] 5fa0: 00000000 c000ea20 b6f6a868 b6f7a970 b6f6a868 00000000 00000000 00000000
[   43.339752] 5fc0: b6f6a868 b6f7a970 00039690 00000021 000609e8 b5400468 00000001 00000000
[   43.339779] 5fe0: 0004b1ec b6df3d64 b6f4e1c5 b6f20a06 60070030 b6f6a868 00000000 00000000
[   43.339820] [<c003c35c>] (override_creds) from [<c00b71c0>] (SyS_faccessat+0x74/0x1cc)
[   43.339855] [<c00b71c0>] (SyS_faccessat) from [<c000ea20>] (ret_fast_syscall+0x0/0x30)
[   43.339884] Code: e592200c e5922294 f590f000 e1931f9f (e2811001)
[   43.339910] ---[ end trace 84ae17060566a4e6 ]---
[   43.345056] Kernel panic - not syncing: CRED: put_cred_rcu() sees c5815a80 with usage -5
[   43.345056]
[   43.354872] CPU1: stopping
[   43.354895] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G      D      3.14.0-rc6-00163-g4c89db4 #7
[   43.354943] [<c0014aac>] (unwind_backtrace) from [<c00118d8>] (show_stack+0x10/0x14)
[   43.354977] [<c00118d8>] (show_stack) from [<c04f9fc0>] (dump_stack+0x84/0x94)
[   43.355008] [<c04f9fc0>] (dump_stack) from [<c0013808>] (handle_IPI+0x14c/0x174)
[   43.355037] [<c0013808>] (handle_IPI) from [<c0008584>] (gic_handle_irq+0x58/0x5c)
[   43.355066] [<c0008584>] (gic_handle_irq) from [<c0012440>] (__irq_svc+0x40/0x50)
[   43.355088] Exception stack(0xc6495ec8 to 0xc6495f10)
[   43.355110] 5ec0:                   00000001 ffff9bc0 00000100 00000001 00000002 0000001b
[   43.355138] 5ee0: 00000000 c6494000 c050078c c0700080 c07414d7 c6494000 c0742880 c6495f10
[   43.355163] 5f00: 00000000 c0020c34 200f0113 ffffffff
[   43.355190] [<c0012440>] (__irq_svc) from [<c0020c34>] (__do_softirq+0xcc/0x24c)
[   43.355221] [<c0020c34>] (__do_softirq) from [<c0021038>] (irq_exit+0xac/0xf4)
[   43.355251] [<c0021038>] (irq_exit) from [<c000f300>] (handle_IRQ+0x44/0x90)
[   43.355279] [<c000f300>] (handle_IRQ) from [<c0008558>] (gic_handle_irq+0x2c/0x5c)
[   43.355307] [<c0008558>] (gic_handle_irq) from [<c0012440>] (__irq_svc+0x40/0x50)
[   43.355329] Exception stack(0xc6495f98 to 0xc6495fe0)
[   43.355348] 5f80:                                                       ffffffed 077e6000
[   43.355376] 5fa0: c070746c 00000000 c0706474 c6494000 c07063f8 c07414d7 c050078c c6494000
[   43.355404] 5fc0: c07414d7 c6494000 c070d900 c6495fe0 c000f600 c000f5f8 600f0113 ffffffff
[   43.355434] [<c0012440>] (__irq_svc) from [<c000f5f8>] (arch_cpu_idle+0x2c/0x38)
[   43.355464] [<c000f5f8>] (arch_cpu_idle) from [<c0059c0c>] (cpu_startup_entry+0x100/0x144)
[   43.355494] [<c0059c0c>] (cpu_startup_entry) from [<68008624>] (0x68008624)


(XEN) Assertion 'svc == CSCHED_VCPU(curr_on_cpu(svc->vcpu->processor))' failed, line 351, file sched_credit.c
(XEN) Xen BUG at sched_credit.c:351
(XEN) CPU1: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) PC:     002414cc __bug+0x28/0x44
(XEN) CPSR:   2000015a MODE:Hypervisor
(XEN)      R0: 002656dc R1: 00000000 R2: 3fd2bd80 R3: 2000015a
(XEN)      R4: 0000015f R5: 0025d9d0 R6: 40006230 R7: 47fe4be0
(XEN)      R8: 47fe4be0 R9: 002b1ff0 R10:002e4254 R11:47fd7e7c R12:00000001
(XEN) HYP: SP: 47fd7e74 LR: 002414cc
(XEN) 
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 000200007ec30000
(XEN) 
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282835
(XEN)  TTBR0_EL2: 0000000076011000
(XEN) 
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 000000000001c810
(XEN)      HDFAR: c8800f00
(XEN)      HIFAR: 81741022
(XEN) 
(XEN) Xen stack trace from sp=47fd7e74:
(XEN)    00000000 47fd7e9c 00219cdc 00000007 40006be8 00000001 40006230 47fe4be0
(XEN)    00265e00 47fd7ed4 0021ba74 016e3600 002e4254 47fd7edc 002524dc 47fd7eb8
(XEN)    40012100 0021b914 00000001 40010038 40006bf8 002b1ff0 002e4254 47fd7eec
(XEN)    0022d738 00000007 5cdc0182 00000007 40012100 47fd7f0c 0022d828 00000001
(XEN)    002b1ff0 002ae000 002e7614 00265b80 002b1ff0 47fd7f3c 00229b28 00000000
(XEN)    ffffffff 002e7614 002ae000 002e7614 00000000 00000000 00000000 00000000
(XEN)    00000000 47fd7f44 00229bc0 47fd7f54 0024ee00 00000000 00000000 47fd7f58
(XEN)    00251150 0000007f 00000000 00000fc0 0000007f 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 ffffffff 00000000
(XEN)    8000805c 600001d3 c2c2c2c2 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 c2c2c2c2 00000000 00000000 00000000 00000000 00000000
(XEN)    c2c2c2c2 c2c2c2c2 c2c2c2c2
(XEN) Xen call trace:
(XEN)    [<002414cc>] __bug+0x28/0x44 (PC)
(XEN)    [<002414cc>] __bug+0x28/0x44 (LR)
(XEN)    [<00219cdc>] burn_credits+0x70/0x180
(XEN)    [<0021ba74>] csched_tick+0x160/0x474
(XEN)    [<0022d738>] execute_timer+0x48/0x5c
(XEN)    [<0022d828>] timer_softirq_action+0xdc/0x230
(XEN)    [<00229b28>] __do_softirq+0xf4/0x100
(XEN)    [<00229bc0>] do_softirq+0x14/0x18
(XEN)    [<0024ee00>] leave_hypervisor_tail+0x5


[  110.614905] INFO: rcu_sched detected stalls on CPUs/tasks: { 1} (detected by 0, t=2102 jiffies, g=314, c=313, q=315)
then, on second attempt:
(XEN) Xen BUG at sched_credit.c:1690
(XEN) CPU1: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) PC:     002414cc __bug+0x28/0x44
(XEN) CPSR:   200e01da MODE:Hypervisor
(XEN)      R0: 002656dc R1: 00000001 R2: 3fd2bd80 R3: 200e01da
(XEN)      R4: 0000069a R5: 0025d9d0 R6: 00000000 R7: 0000001e
(XEN)      R8: acfa22c1 R9: 0000001b R10:00000001 R11:40037df4 R12:00000002
(XEN) HYP: SP: 40037dec LR: 002414cc
(XEN) 
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 000100007ec06000
(XEN) 
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282835
(XEN)  TTBR0_EL2: 0000000076011000
(XEN) 
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 000000000001c810
(XEN)      HDFAR: c8800f00
(XEN)      HIFAR: c07410a2
(XEN) 
(XEN) Xen stack trace from sp=40037dec:
(XEN)    00000000 40037ea4 0021c13c 00000000 00000002 0000001e 00000001 00000000
(XEN)    00000000 0000ad28 0000ba54 4003a000 00000000 ad28ba54 0000001b 00000000
(XEN)    40006230 00000000 00000000 00000000 00000000 40037ed0 fb65b800 40012100
(XEN)    00000001 00000001 002e4020 002e6380 002e6380 600e01da 40012058 40037e7c
(XEN)    0022d4c8 3fd2bd80 40012040 40037ea4 0022e52c 4001202c ad28ba54 0000001b
(XEN)    4003b000 002e4020 40012040 4001202c 00000001 40037f0c 00225f84 00000000
(XEN)    4003b4a8 40037efc 40012028 00000000 40037f20 4003b000 93840047 40037eec
(XEN)    0022884c 00000000 93840047 40037eec 07e00000 40037f58 c07063f8 40037f54
(XEN)    00000001 002b1ff4 002ae000 002e7614 00265b80 002b1ff0 002e4254 40037f3c
(XEN)    00229b28 00000000 ffffffff 01c81000 002ae000 002e7614 c07063f8 c07414d7
(XEN)    c050078c c06fe000 c07414d7 40037f44 00229bc0 40037f54 0024ee00 c0706474
(XEN)    c06fe000 40037f58 00251150 ffffffed 077de000 c070746c 00000000 c0706474
(XEN)    c06fe000 c07063f8 c07414d7 c050078c c06fe000 c07414d7 c06fe000 c070d900
(XEN)    ffffffff b6ba0eeb c0019d28 600e0093 00000000 bee930e8 c0742080 c0012400
(XEN)    c06fffa8 c000f600 c074208c c00123a0 c0742098 c00125e0 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 600e0030 800e0193 20070093
(XEN)    20000193 00000000 00000000 030c3080 651a8c04
(XEN) Xen call trace:
(XEN)    [<002414cc>] __bug+0x28/0x44 (PC)
(XEN)    [<002414cc>] __bug+0x28/0x44 (LR)
(XEN)    [<0021c13c>] csched_schedule+0x3b4/0xee8
(XEN)    [<00225f84>] schedule+0x13c/0x8c8
(XEN)    [<00229b28>] __do_softirq+0xf4/0x100
(XEN)    [<00229bc0>] do_softirq+0x14/0x18
(XEN)    [<0024ee00>] leave_hypervisor_tail+0x50/0x6c
(XEN)    [<00251150>] return_to_guest+0xc/0xb8




(XEN) Assertion 'svc == CSCHED_VCPU(curr_on_cpu(svc->vcpu->processor))' failed, line 351, file sched_credit.c
(XEN) Xen BUG at sched_credit.c:351
(XEN) CPU0: Unexpected Trap: Undefined Instruction
(XEN) ----[ Xen-4.4.1-pre  arm32  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) PC:     002414cc __bug+0x28/0x44
(XEN) CPSR:   2000015a MODE:Hypervisor
(XEN)      R0: 002656dc R1: 00000000 R2: 00000000 R3: 2000015a
(XEN)      R4: 0000015f R5: 0025d9d0 R6: 40006230 R7: 400277e8
(XEN)      R8: 400277e8 R9: 002b1ff0 R10:002e4254 R11:47fd7e7c R12:00000001
(XEN) HYP: SP: 47fd7e74 LR: 002414cc
(XEN) 
(XEN)   VTCR_EL2: 80002558
(XEN)  VTTBR_EL2: 000200007ec30000
(XEN) 
(XEN)  SCTLR_EL2: 30cd187f
(XEN)    HCR_EL2: 0000000000282835
(XEN)  TTBR0_EL2: 000000007fedf000
(XEN) 
(XEN)    ESR_EL2: 00000000
(XEN)  HPFAR_EL2: 000000000001c810
(XEN)      HDFAR: c8800f00
(XEN)      HIFAR: 01930940
(XEN) 
(XEN) Xen stack trace from sp=47fd7e74:
(XEN)    00000000 47fd7e9c 00219cdc 00000008 40006528 00000000 40006230 400277e8
(XEN)    00265e00 47fd7ed4 0021ba74 016e3600 002e4254 47fd7edc 002524dc 47fd7eb8
(XEN)    002e6380 0021b914 00000000 40006c48 40006538 002b1ff0 002e4254 47fd7eec
(XEN)    0022d738 00000008 e18d709f 00000008 002e6380 47fd7f0c 0022d828 00000000
(XEN)    002b1ff0 002ae000 002e7614 00265b80 002b1ff0 47fd7f3c 00229b28 00000000
(XEN)    ffffffff 002e7614 002ae000 002e7614 00000000 00000000 00000000 00000000
(XEN)    00000000 47fd7f44 00229bc0 47fd7f54 0024ee00 00000000 00000000 47fd7f58
(XEN)    00251150 0000007f 00000000 00000fc0 0000007f 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 ffffffff 00000000
(XEN)    8000805c 600001d3 c2c2c2c2 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)    00000000 00000000 c2c2c2c2 00000000 00000000 00000000 00000000 00000000
(XEN)    c2c2c2c2 c2c2c2c2 c2c2c2c2
(XEN) Xen call trace:
(XEN)    [<002414cc>] __bug+0x28/0x44 (PC)
(XEN)    [<002414cc>] __bug+0x28/0x44 (LR)
(XEN)    [<00219cdc>] burn_credits+0x70/0x180
(XEN)    [<0021ba74>] csched_tick+0x160/0x474
(XEN)    [<0022d738>] execute_timer+0x48/0x5c
(XEN)    [<0022d828>] timer_softirq_action+0xdc/0x230
(XEN)    [<00229b28>] __do_softirq+0xf4/0x100
(XEN)    [<00229bc0>] do_softirq+0x14/0x18
(XEN)    [<0024ee00>] leave_hypervisor_tail+0x50/0x6c
(XEN)    [<00251150>] return_to_guest+0xc/0xb8


[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2014-04-14 12:25 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <CAG4opy8h2ZBpFhODSiS8OmGP-M_UCLPn3bPpWzt1wm1UHprDSw@mail.gmail.com>
2014-04-14  8:02 ` Fwd: Debugging "Xen BUG at ... list.h" on ARM Thomas Leonard
2014-04-14  8:06   ` Thomas Leonard
2014-04-14  9:12   ` Fwd: " Julien Grall
2014-04-14  9:30     ` Thomas Leonard
2014-04-14  9:43       ` Ian Campbell
2014-04-14 10:10         ` Thomas Leonard
2014-04-14 10:29           ` Julien Grall
2014-04-14 10:53             ` Ian Campbell
2014-04-14 11:03             ` Thomas Leonard
2014-04-14 11:47               ` Ian Campbell
2014-04-14 11:53                 ` Thomas Leonard
2014-04-14 11:57                   ` Ian Campbell
2014-04-14 12:00       ` Ian Campbell
2014-04-14 12:25         ` Thomas Leonard

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.