Re: XEN[ARM] Master not working on Allwinner A20

["peter" <peter@perkbv.com>]

Thank you for your fast reaction Ian.

I have tested d9639236 with the smaller dom0 allocation.
256M is still giving me the same problem.
But 128M is working and booting the dom0 kernel.

Bootlog:

WARNING: ignoring attempt to set core boot address 7ff88454 on core -1
WARNING: Not kicking secondary CPUs

Starting kernel ...

- UART enabled -
- CPU 00000000 booting -
- Xen starting in Hyp mode -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
Checking for initrd in /chosen
RAM: 0000000040000000 - 000000007fffffff

MODULE[1]: 00000000402ef700 - 00000000402f1c57 
MODULE[2]: 0000000050000000 - 0000000050f00000 
MODULE[3]: 0000000060000000 - 0000000066000000 

Command line: dom0_mem=128M sync_console console=dtuart dtuart=serial0
Placing Xen at 0x000000007fe00000-0x0000000080000000
Xen heap: 0000000076000000-000000007e000000 (32768 pages)
Dom heap: 229376 pages
Looking for UART console serial0
 Xen 4.4-unstable
(XEN) Xen version 4.4-unstable (peter@) (arm-linux-gnueabihf-gcc
(Ubuntu/Linaro 4.8.1-10ubuntu7) 4.8.1) debug=y Mon Dec 16 11:36:32 CET 2013
(XEN) Latest ChangeSet: Mon Dec 9 12:13:48 2013 +0000 git:d963923
(XEN) Console output is synchronous.
(XEN) Processor: 410fc074: "ARM Limited", variant: 0x0, part 0xc07, rev 0x4
(XEN) 32-bit Execution:
(XEN)   Processor Features: 00001131:00011011
(XEN)     Instruction Sets: AArch32 Thumb Thumb-2 ThumbEE Jazelle
(XEN)     Extensions: GenericTimer Security
(XEN)   Debug Features: 02010555
(XEN)   Auxiliary Features: 00000000
(XEN)   Memory Model Features: 10101105 40000000 01240000 02102211
(XEN)  ISA Features: 02101110 13112111 21232041 11112131 10011142 00000000
(XEN) Platform: Allwinner A20
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27
(XEN) Using generic timer at 24000 KHz
(XEN) GIC initialization:
(XEN)         gic_dist_addr=0000000001c81000
(XEN)         gic_cpu_addr=0000000001c82000
(XEN)         gic_hyp_addr=0000000001c84000
(XEN)         gic_vcpu_addr=0000000001c86000
(XEN)         gic_maintenance_irq=25
(XEN) GIC: 160 lines, 2 cpus, secure (IID 0100143b).
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Allocated console ring of 16 KiB.
(XEN) VFP implementer 0x41 architecture 2 part 0x30 variant 0x7 rev 0x4
(XEN) Brought up 1 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Populate P2M 0x68000000->0x70000000 (1:1 mapping for dom0)
(XEN) Loading kernel from boot module 2
(XEN) Loading zImage from 0000000050000000 to
000000006fc00000-000000006ff36190
(XEN) Loading dom0 initrd from 0000000060000000 to
0x0000000069c00000-0x000000006fc00000
(XEN) Loading dom0 DTB to 0x0000000069a00000-0x0000000069a01ee0
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) **********************************************
(XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) ******* This option is intended to aid debugging of Xen by ensuring
(XEN) ******* that all output is synchronously delivered on the serial line.
(XEN) ******* However it can introduce SIGNIFICANT latencies and affect
(XEN) ******* timekeeping. It is NOT recommended for production use!
(XEN) **********************************************
(XEN) 3... 2... 1... 
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
to Xen)
(XEN) Freed 264kB init memory.
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 3.12.0-rc1-87467-g70d9501-dirty
(peter@behemoth) (gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu7) ) #74
SMP Thu Dec 12 09:25:43 CET 2013
[    0.000000] CPU: ARMv7 Processor [410fc074] revision 4 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
instruction cache
[    0.000000] Machine: Allwinner A1X (Device Tree), model: Cubietech
Cubieboard2
[    0.000000] debug: ignoring loglevel setting.
[    0.000000] Memory policy: ECC disabled, Data cache writealloc
[    0.000000] On node 0 totalpages: 32768
[    0.000000] free_area_init_node: node 0, pgdat c0668b00, node_mem_map
c06bf000
[    0.000000]   Normal zone: 256 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 32768 pages, LIFO batch:7
[    0.000000] psci: probing function IDs from device-tree
[    0.000000] PERCPU: Embedded 7 pages/cpu @c07c5000 s7808 r8192 d12672
u32768
[    0.000000] pcpu-alloc: s7808 r8192 d12672 u32768 alloc=8*4096
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on. 
Total pages: 32512
[    0.000000] Kernel command line: console=hvc0,115200n8 debug
ignore_loglevel rw rootwait earlyprintk=xen clk_ignore_unused
[    0.000000] PID hash table entries: 512 (order: -1, 2048 bytes)
[    0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
[    0.000000] Memory: 24672K/131072K available (4726K kernel code, 229K
rwdata, 1364K rodata, 211K init, 341K bss, 106400K reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
[    0.000000]     vmalloc : 0xc8800000 - 0xff000000   ( 872 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xc8000000   ( 128 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0008000 - 0xc05faaa8   (6091 kB)
[    0.000000]       .init : 0xc05fb000 - 0xc062fe80   ( 212 kB)
[    0.000000]       .data : 0xc0630000 - 0xc0669600   ( 230 kB)
[    0.000000]        .bss : 0xc0669608 - 0xc06bec24   ( 342 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] Architected cp15 timer(s) running at 24.00MHz (virt).
[    0.000000] Switching to timer-based delay loop
[    0.000000] sched_clock: ARM arch timer >56 bits at 24000kHz,
resolution 41ns
[    0.000000] sched_clock: 32 bits at 100 Hz, resolution 10000000ns,
wraps every 4294967286ms
[    0.000000] Console: colour dummy device 80x30
[    3.910036] Calibrating delay loop (skipped), value calculated using
timer frequency.. 48.00 BogoMIPS (lpj=240000)
[    3.910050] pid_max: default: 32768 minimum: 301
[    3.910216] Mount-cache hash table entries: 512
[    3.912329] CPU: Testing write buffer coherency: ok
[    3.912657] /cpus/cpu@0 missing clock-frequency property
[    3.912678] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[    3.912714] Setting up static identity map for 0xc047ee88 - 0xc047eee0
[    3.913298] Brought up 1 CPUs
[    3.913310] SMP: Total of 1 processors activated.
[    3.913318] CPU: All CPU(s) started in SVC mode.
[    3.913852] devtmpfs: initialized
[    3.917823] VFP support v0.3: implementor 41 architecture 2 part 30
variant 7 rev 4
[    3.917924] Xen 4.4 support found, events_irq=31 gnttab_frame_pfn=1d00
[    3.918018] xen:grant_table: Grant tables using version 1 layout
[    3.918081] Grant table initialized
[    3.918318] pinctrl core: initialized pinctrl subsystem
[    3.918726] regulator-dummy: no parameters
[    3.919821] NET: Registered protocol family 16
[    3.920196] Xen: initializing cpu0
[    3.920516] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    3.922793] Init eGon pin module V2.0
[    3.927962] bio: create slab <bio-0> at 0
[    3.928486] xen:balloon: Initialising balloon driver
[    3.928608] reg-fixed-voltage usb1-vbus.4: could not find pctldev for
node /soc@01c00000/pinctrl@01c20800/usb1_vbus_pin@0, deferring probe
[    3.928632] platform usb1-vbus.4: Driver reg-fixed-voltage requests
probe deferral
[    3.928655] reg-fixed-voltage usb2-vbus.5: could not find pctldev for
node /soc@01c00000/pinctrl@01c20800/usb2_vbus_pin@0, deferring probe
[    3.928671] platform usb2-vbus.5: Driver reg-fixed-voltage requests
probe deferral
[    3.929371] SCSI subsystem initialized
[    3.929741] libata version 3.00 loaded.
[    3.929961] usbcore: registered new interface driver usbfs
[    3.930023] usbcore: registered new interface driver hub
[    3.930131] usbcore: registered new device driver usb
[    3.930372] pps_core: LinuxPPS API ver. 1 registered
[    3.930383] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo
Giometti <giometti@linux.it>
[    3.930412] PTP clock support registered
[    3.930446] EDAC MC: Ver: 3.0.0
[    3.931450] Switched to clocksource arch_sys_counter
[    3.939651] NET: Registered protocol family 2
[    3.940251] TCP established hash table entries: 1024 (order: 1, 8192 bytes)
[    3.940282] TCP bind hash table entries: 1024 (order: 1, 8192 bytes)
[    3.940307] TCP: Hash tables configured (established 1024 bind 1024)
[    3.940373] TCP: reno registered
[    3.940386] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    3.940420] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    3.940646] NET: Registered protocol family 1
[    3.941095] RPC: Registered named UNIX socket transport module.
[    3.941109] RPC: Registered udp transport module.
[    3.941115] RPC: Registered tcp transport module.
[    3.941121] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    3.941332] Trying to unpack rootfs image as initramfs...
[    3.941553] rootfs image is not initramfs (junk in compressed archive);
looks like an initrd
[    4.089121] Freeing initrd memory: 98304K (c1c00000 - c7c00000)
[    4.099151] NFS: Registering the id_resolver key type
[    4.099236] Key type id_resolver registered
[    4.099244] Key type id_legacy registered
[    4.099301] msgmni has been set to 240
[    4.100682] Block layer SCSI generic (bsg) driver version 0.4 loaded
(major 251)
[    4.100698] io scheduler noop registered
[    4.100706] io scheduler deadline registered
[    4.101027] io scheduler cfq registered (default)
[    4.102991] sunxi-pinctrl 1c20800.pinctrl: initialized sunXi PIO driver
[    4.103494] sunxi disp driver loaded (/dev/disp api 1.0)
[    4.103882] xen:xen_evtchn: Event-channel device installed
[    4.759574] console [hvc0] enabled
[    4.763072] serial: Freescale lpuart driver
[    4.767308] [drm] Initialized drm 1.1.0 20060810
[    4.772824] brd: module loaded
[    4.779254] loop: module loaded
[    4.782823] sun4i-mdio 1c0b080.mdio: no regulator found
[    4.788041] libphy: sun4i_mii_bus: probed
[    4.827833] sun4i-emac 1c0b000.ethernet: using random MAC address
5e:f1:b1:00:b9:96
[    4.837231] sun4i-emac 1c0b000.ethernet: eth0: at c8832000, IRQ 87 MAC:
5e:f1:b1:00:b9:96
[    4.845465] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    4.851868] ehci-platform: EHCI generic platform driver
[    4.857125] sunxi-ehci: Allwinner sunXi EHCI driver
[    4.861981] platform 1c14000.ehci0: Driver sunxi-ehci requests probe
deferral
[    4.869015] platform 1c1c000.ehci1: Driver sunxi-ehci requests probe
deferral
[    4.876272] usbcore: registered new interface driver usb-storage
[    4.882673] mousedev: PS/2 mouse device common for all mice
[    4.888577] sdhci: Secure Digital Host Controller Interface driver
[    4.894580] sdhci: Copyright(c) Pierre Ossman
[    4.898960] sdhci-pltfm: SDHCI platform and OF driver helper
[    4.905076] usbcore: registered new interface driver usbhid
[    4.910528] usbhid: USB HID core driver
[    4.914602] TCP: cubic registered
[    4.918386] NET: Registered protocol family 10
[    4.923494] sit: IPv6 over IPv4 tunneling driver
[    4.928814] Key type dns_resolver registered
[    4.933124] Registering SWP/SWPB emulation handler
[    4.938920] usb1-vbus: 3300 mV 
[    4.942308] usb2-vbus: 3300 mV 
[    4.945754] sunxi-ehci 1c14000.ehci0: EHCI Host Controller
[    4.951102] sunxi-ehci 1c14000.ehci0: new USB bus registered, assigned
bus number 1
[    4.958839] sunxi-ehci 1c14000.ehci0: irq 71, io mem 0x01c14000
[    4.974643] sunxi-ehci 1c14000.ehci0: USB 2.0 started, EHCI 1.00
[    4.981308] hub 1-0:1.0: USB hub found
[    4.985037] hub 1-0:1.0: 1 port detected
[    4.989438] sunxi-ehci 1c1c000.ehci1: EHCI Host Controller
[    4.994851] sunxi-ehci 1c1c000.ehci1: new USB bus registered, assigned
bus number 2
[    5.002472] sunxi-ehci 1c1c000.ehci1: irq 72, io mem 0x01c1c000
[    5.023777] sunxi-ehci 1c1c000.ehci1: USB 2.0 started, EHCI 1.00
[    5.030405] hub 2-0:1.0: USB hub found
[    5.034103] hub 2-0:1.0: 1 port detected
[    5.038273] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[    5.044478] clk: Not disabling unused clocks
[    5.049441] RAMDISK: Couldn't find valid RAM disk image starting at 0.
[    5.055895] Waiting for root device ...


Reverting 6c21cb36 didn't help for me.

But I have also found another bug between 720f45ad..d9639236 in the DT
bootargs parsing.
When my DTS is:
xen,dom0-bootargs = "console=hvc0,115200n8 debug ignore_loglevel rw
rootwait earlyprintk=xen clk_ignore_unused ";
It successfully boots (but I can't mount my rootfs :( ), but when
dom0-bootargs is:
xen,dom0-bootargs = "console=hvc0,115200n8 debug ignore_loglevel rw
rootwait earlyprintk=xen clk_ignore_unused root=/dev/nfs
nfsroot=10.31.5.161:/export/cubie/rootfs,v3,tcp
ip=10.31.5.165:10.31.5.161:10.31.4.1:255.255.252.0:cubie:eth0:off";
It stays stuck on:
(XEN) **********************************************
(XEN) 3... 2... 1... 
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
to Xen)
(XEN) Freed 264kB init memory.

Because of this I can't test my system on master because I can't mount my
rootfs.

720f45ad Is working for me I can create multiple vm's but I get some
kernel messages when creating them:
[  278.215653] Failed to unmap pfn:5e035 rc:-2
[  278.219786] Failed to unmap pfn:5e035 rc:-2
[  278.233617] Failed to unmap pfn:5f033 rc:-2
[  278.237857] Failed to unmap pfn:5e035 rc:-2
[  278.248553] Failed to unmap pfn:5e033 rc:-2
[  278.256166] Failed to unmap pfn:5e032 rc:-2

And when I destroy a VM I get this:
(XEN) mm.c:1250:d0 gnttab_mark_dirty not implemented yet

The rootfs of dom0 is Ubuntu 13.04 and is mounted using NFS.


Peter.

-----Original Message-----
From: Ian Campbell <Ian.Campbell@citrix.com>
To: peter <peter@perkbv.com>
Cc: <xen-devel@lists.xen.org>, Julien Grall <julien.grall@citrix.com>
Date: Mon, 16 Dec 2013 10:09:37 +0000
Subject: Re: [Xen-devel] XEN[ARM] Master not working on Allwinner A20

> On Mon, 2013-12-16 at 08:54 +0100, peter wrote:
> > Goodmorning building the latest master of XEN
> > d9639236: Ian Campbell, xen: arm: correct return value of
> > raw_copy_{to/from}_guest_*, raw_clear_guest
> > For the Allwinner A20 arm board XEN is giving me this error:
> > [...]
> > (XEN) ****************************************
> > (XEN) Panic on CPU 0:
> > (XEN) Failed to allocate contiguous memory for dom0
> > (XEN) ****************************************
> > (XEN) 
> > (XEN) Reboot in five seconds...
> > 
> > When i follow the tutorial for the Allwinner board
> >
> (http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions/Allwin
> ner)
> > And checkout to: 720f45ad: Julien Grall, xen/evtchn: Fix build on ARM
> > XEN is working fine for me.
> 
> 720f45ad..d9639236 is quite a big range but it does contains some 1:1
> mapping changes. You are giving dom0 512M out of 1024M of RAM -- there
> shuold be that much free at this stage of the boot but I wonder if we
> are trying to allocate the memory aligned to 512M as well -- which
> equates to exactly two region and allocating any one page from one of
> them will cause us to fail to find 512M.
> 
> Can you try a smaller dom0 allocation, e.g. 256M or 128M?
> 
> To what extent did 720f45ad work for you? Did it actually mount its
> rootfs (which device? mmc?) and run guests without I/O errors?
> 
> You could also try reverting 6c21cb36 although I can't immediately see
> what would be wrong with it.
> 
> Ian.