How to create a PVHv2 guest

How to create a PVHv2 guest

[Chao Gao]

Hi, Roger.

I meet an error when creating a pvh guest. I am using commit 6e2a4c73564a. 
From the error log, I found bootlate_pv()->pin_table always failed. And the
failure was caused by is_pv_domain(pg_owner) in do_mmuext_op(). Do you have
any idea on this?

The guest config file is:
builder = "hvm"
name = "vVTD_ASS_02_1467338450"
memory=3300
vcpus=4
disk = [ '/home/gao/performance_tuning/env/xen_centos2.qcow2,qcow2,hda,rw' ]
boot='c'
device_model_version='none'
kernel="/home/gao/hvmlite/vmlinuz-4.13.0-rc1+"
ramdisk="/home/gao/hvmlite/initramfs-4.13.0-rc1+.img"
cmdline="root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb console=tty0 console=ttyS0,115200 x2apic_phys xen_nopv LANG=en_US.UTF-8"


The error log is:
Parsing config from /home/gao/Downloads/config.vmxVTD_ASS_02
{
    "c_info": {
        "type": "hvm",
        "name": "vVTD_ASS_02_1467338450",
        "uuid": "721df691-dae5-401d-94a3-cffacb67e6b5",
        "run_hotplug_scripts": "True"
    },
    "b_info": {
        "max_vcpus": 4,
        "avail_vcpus": [
            0,
            1,
            2,
            3
        ],
        "max_memkb": 3379200,
        "target_memkb": 3379200,
        "shadow_memkb": 30496,
        "device_model_version": "none",
        "sched_params": {

        },
        "claim_mode": "True",
        "kernel": "/home/gao/hvmlite/vmlinuz-4.13.0-rc1+",
        "cmdline": "root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb console=tty0 console=ttyS0,115200 x2apic_phys xen_nopv LANG=en_US.UTF-8",
        "ramdisk": "/home/gao/hvmlite/initramfs-4.13.0-rc1+.img",
        "type.hvm": {
            "vga": {

            },
            "vnc": {

            },
            "sdl": {

            },
            "spice": {

            },
            "boot": "c",
            "rdm": {

            }
        },
        "arch_arm": {

        }
    },
    "disks": [
        {
            "pdev_path": "/home/gao/performance_tuning/env/xen_centos2.qcow2",
            "vdev": "hda",
            "format": "qcow2",
            "readwrite": 1
        }
    ],
    "on_reboot": "restart",
    "on_soft_reset": "soft_reset"
}
libxl: debug: libxl_create.c:1609:do_domain_create: Domain 0:ao 0x7f4cc0: create: how=(nil) callback=(nil) poller=0x7f4ac0
libxl: debug: libxl_device.c:361:libxl__device_disk_set_backend: Disk vdev=hda spec.backend=unknown
libxl: debug: libxl_device.c:324:disk_try_backend: Disk vdev=hda, backend phy unsuitable due to format qcow2
libxl: debug: libxl_device.c:396:libxl__device_disk_set_backend: Disk vdev=hda, using backend qdisk
libxl: debug: libxl_create.c:965:initiate_domain_create: Domain 19:running bootloader
libxl: debug: libxl_bootloader.c:328:libxl__bootloader_run: Domain 19:not a PV domain, skipping bootloader
libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x7f7628: deregister unregistered
libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=272, nr_vcpus=132, free_memkb=87413
libxl: detail: libxl_dom.c:182:numa_place_domain: NUMA placement candidate with 1 nodes, 272 cpus and 87413 KB free selected
domainbuilder: detail: xc_dom_allocate: cmdline="root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb console=tty0 console=ttyS0,115200 x2apic_phys xen_nopv LANG=en_US.UTF-8", features=""
domainbuilder: detail: xc_dom_kernel_file: filename="/home/gao/hvmlite/vmlinuz-4.13.0-rc1+"
domainbuilder: detail: xc_dom_malloc_filemap    : 6130 kB
domainbuilder: detail: xc_dom_ramdisk_file: filename="/home/gao/hvmlite/initramfs-4.13.0-rc1+.img"
domainbuilder: detail: xc_dom_malloc_filemap    : 50 MB
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.10, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... 
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying HVM-generic loader ... 
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... 
domainbuilder: detail: xc_dom_malloc            : 20922 kB
domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x5eee00 -> 0x146e9b8
domainbuilder: detail: loader probe OK
xc: detail: ELF: phdr: paddr=0x1000000 memsz=0xb5c000
xc: detail: ELF: phdr: paddr=0x1c00000 memsz=0x235000
xc: detail: ELF: phdr: paddr=0x1e35000 memsz=0x1ce58
xc: detail: ELF: phdr: paddr=0x1e52000 memsz=0x52a000
xc: detail: ELF: memory: 0x1000000 -> 0x237c000
xc: detail: ELF: note: GUEST_OS = "linux"
xc: detail: ELF: note: GUEST_VERSION = "2.6"
xc: detail: ELF: note: XEN_VERSION = "xen-3.0"
xc: detail: ELF: note: VIRT_BASE = 0xffffffff80000000
xc: detail: ELF: note: INIT_P2M = 0x8000000000
xc: detail: ELF: note: ENTRY = 0xffffffff81e52180
xc: detail: ELF: note: HYPERCALL_PAGE = 0xffffffff81001000
xc: detail: ELF: note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
xc: detail: ELF: note: SUPPORTED_FEATURES = 0x801
xc: detail: ELF: note: PAE_MODE = "yes"
xc: detail: ELF: note: LOADER = "generic"
xc: detail: ELF: note: unknown (0xd)
xc: detail: ELF: note: SUSPEND_CANCEL = 0x1
xc: detail: ELF: note: MOD_START_PFN = 0x1
xc: detail: ELF: note: HV_START_LOW = 0xffff800000000000
xc: detail: ELF: note: PADDR_OFFSET = 0
xc: detail: ELF: addresses:
xc: detail:     virt_base        = 0xffffffff80000000
xc: detail:     elf_paddr_offset = 0x0
xc: detail:     virt_offset      = 0xffffffff80000000
xc: detail:     virt_kstart      = 0xffffffff81000000
xc: detail:     virt_kend        = 0xffffffff8237c000
xc: detail:     virt_entry       = 0xffffffff81e52180
xc: detail:     p2m_base         = 0x8000000000
domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0xffffffff81000000 -> 0xffffffff8237c000
domainbuilder: detail: xc_dom_mem_init: mem 3300 MB, pages 0xce400 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0xce400 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64
domainbuilder: detail: xc_dom_malloc            : 6600 kB
S3 disabled
S4 disabled
CONV disabled
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_malloc            : 116 kB
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x1000+0x137c at 0x7fb644ff6000
domainbuilder: detail: xc_dom_alloc_segment:   kernel       : 0xffffffff81000000 -> 0xffffffff8237c000  (pfn 0x1000 + 0x137c pages)
xc: detail: ELF: phdr 0 at 0x7fb644ff6000 -> 0x7fb645b52000
xc: detail: ELF: phdr 1 at 0x7fb645bf6000 -> 0x7fb645e2b000
xc: detail: ELF: phdr 2 at 0x7fb645e2b000 -> 0x7fb645e47e58
xc: detail: ELF: phdr 3 at 0x7fb645e48000 -> 0x7fb646064000
domainbuilder: detail: xc_dom_load_acpi: 64 bytes at address fffc0

domainbuilder: detail: xc_dom_load_acpi: 4096 bytes at address fc000000

domainbuilder: detail: xc_dom_load_acpi: 28672 bytes at address fc001000

domainbuilder: detail: xc_dom_alloc_page   :   start info   : 0xffffffff8237c000 (pfn 0x237c)
domainbuilder: detail: xc_dom_alloc_page   :   xenstore     : 0xffffffff8237d000 (pfn 0x237d)
domainbuilder: detail: xc_dom_alloc_page   :   console      : 0xffffffff8237e000 (pfn 0x237e)
domainbuilder: detail: xc_dom_alloc_page   :   shared info  : 0xffffffff8237f000 (pfn 0x237f)
domainbuilder: detail: count_pgtables: 0x0000007fffffffff/39: 0x0000ff8000000000 -> 0x0000ffffffffffff, 1 table(s)
domainbuilder: detail: count_pgtables: 0x000000003fffffff/30: 0x0000ffff80000000 -> 0x0000ffffbfffffff, 1 table(s)
domainbuilder: detail: count_pgtables: 0x00000000001fffff/21: 0x0000ffff80000000 -> 0x0000ffff827fffff, 20 table(s)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x2380+0x17 at 0x7fb64e2ba000
domainbuilder: detail: xc_dom_alloc_segment:   page tables  : 0xffffffff82380000 -> 0xffffffff82397000  (pfn 0x2380 + 0x17 pages)
domainbuilder: detail: xc_dom_alloc_page   :   boot stack   : 0xffffffff82397000 (pfn 0x2397)
domainbuilder: detail: xc_dom_build_image  : virt_alloc_end : 0xffffffff82398000
domainbuilder: detail: xc_dom_build_image  : virt_pgtab_end : 0xffffffff82800000
domainbuilder: detail: xc_dom_malloc            : 804 kB
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x2800+0x8621 at 0x7fb63c9d5000
domainbuilder: detail: xc_dom_alloc_segment:   ramdisk      : 0xffffffff82800000 -> 0xffffffff8ae21000  (pfn 0x2800 + 0x8621 pages)
domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x329bd2f -> 0x8620610
domainbuilder: detail: count_pgtables: 0x0000007fffffffff/39: 0x0000008000000000 -> 0x000000ffffffffff, 1 table(s)
domainbuilder: detail: count_pgtables: 0x000000003fffffff/30: 0x0000008000000000 -> 0x000000803fffffff, 1 table(s)
domainbuilder: detail: count_pgtables: 0x00000000001fffff/21: 0x0000008000000000 -> 0x00000080007fffff, 4 table(s)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xae21+0x678 at 0x7fb63c35d000
domainbuilder: detail: xc_dom_alloc_segment:   phys2mach    : 0xffffffff8ae21000 -> 0xffffffff8b499000  (pfn 0xae21 + 0x678 pages)
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0xce400
domainbuilder: detail: clear_page: pfn 0x237e, mfn 0x237e
domainbuilder: detail: clear_page: pfn 0x237d, mfn 0x237d
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x237c+0x1 at 0x7fb64e1e5000
domainbuilder: detail: start_info_x86_64: called
domainbuilder: detail: setup_hypercall_page: vaddr=0xffffffff81001000 pfn=0x1001
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail:    allocated
domainbuilder: detail:       malloc             : 28487 kB
domainbuilder: detail:       anon mmap          : 0 bytes
domainbuilder: detail:    mapped
domainbuilder: detail:       file mmap          : 56 MB
domainbuilder: detail:       domU mmap          : 160 MB
xc: error: panic: xc_dom_x86.c:1587: bootlate_pv: pin_table failed (pfn 0x2380, rc=1): Internal error
libxl: error: libxl_dom.c:698:libxl__build_dom: xc_dom_boot_image failed: Invalid argument
domainbuilder: detail: xc_dom_release: called
libxl: error: libxl_create.c:1217:domcreate_rebuild_done: Domain 19:cannot (re-)build domain: -3
libxl: debug: libxl_domain.c:1140:devices_destroy_cb: Domain 19:Forked pid 5625 for destroy of domain
libxl: debug: libxl_create.c:1646:do_domain_create: Domain 0:ao 0x7f4cc0: inprogress: poller=0x7f4ac0, flags=i
libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x7f4cc0: complete, rc=-3
libxl: debug: libxl_event.c:1838:libxl__ao__destroy: ao 0x7f4cc0: destroy
libxl: debug: libxl_domain.c:871:libxl_domain_destroy: Domain 19:ao 0x7f6960: create: how=(nil) callback=(nil) poller=0x7f4ac0
libxl: error: libxl_domain.c:1003:libxl__destroy_domid: Domain 19:Non-existant domain
libxl: error: libxl_domain.c:962:domain_destroy_callback: Domain 19:Unable to destroy guest
libxl: error: libxl_domain.c:889:domain_destroy_cb: Domain 19:Destruction of domain failed
libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x7f6960: complete, rc=-21
libxl: debug: libxl_domain.c:880:libxl_domain_destroy: Domain 19:ao 0x7f6960: inprogress: poller=0x7f4ac0, flags=ic
libxl: debug: libxl_event.c:1838:libxl__ao__destroy: ao 0x7f6960: destroy
xencall:buffer: debug: total allocations:677 total releases:677
xencall:buffer: debug: current allocations:0 maximum allocations:3
xencall:buffer: debug: cache current size:3
xencall:buffer: debug: cache hits:656 misses:3 toobig:18
xencall:buffer: debug: total allocations:0 total releases:0
xencall:buffer: debug: current allocations:0 maximum allocations:0
xencall:buffer: debug: cache current size:0
xencall:buffer: debug: cache hits:0 misses:0 toobig:0

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: How to create a PVHv2 guest

[Roger Pau Monné]

On Mon, Sep 11, 2017 at 09:58:01AM +0800, Chao Gao wrote:
> Hi, Roger.
> 
> I meet an error when creating a pvh guest. I am using commit 6e2a4c73564a. 
> From the error log, I found bootlate_pv()->pin_table always failed. And the
> failure was caused by is_pv_domain(pg_owner) in do_mmuext_op(). Do you have
> any idea on this?

IIRC bootlate_pv should never be called for a PVH guest. I'm not
really sure how you can get there, can you paste/attach your Linux
kernel config file?

Also I'm not sure using a qcow2 disk is going to work properly, the
current PVHv2 tools implementation will not spawn a QEMU instance to
act as the backend.

In any case I'm working on a new interface that can be found at:

https://lists.xenproject.org/archives/html/xen-devel/2017-09/msg00647.html

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: How to create a PVHv2 guest

[Chao Gao]

On Mon, Sep 11, 2017 at 10:14:15AM +0100, Roger Pau Monné wrote:
>On Mon, Sep 11, 2017 at 09:58:01AM +0800, Chao Gao wrote:
>> Hi, Roger.
>> 
>> I meet an error when creating a pvh guest. I am using commit 6e2a4c73564a. 
>> From the error log, I found bootlate_pv()->pin_table always failed. And the
>> failure was caused by is_pv_domain(pg_owner) in do_mmuext_op(). Do you have
>> any idea on this?
>
>IIRC bootlate_pv should never be called for a PVH guest. I'm not
>really sure how you can get there, can you paste/attach your Linux
>kernel config file?

I really got there. I don't set XEN_PVH in config file. After setting
XEN_PVH, the error disappears. From the output of xc_dom_compat_check(),
I guess the guest was wrongly treated as a pv guest.

>
>Also I'm not sure using a qcow2 disk is going to work properly, the
>current PVHv2 tools implementation will not spawn a QEMU instance to
>act as the backend.

Ok. I will try to use a raw image.

Thanks
Chao

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: How to create a PVHv2 guest

[Roger Pau Monné]

On Mon, Sep 11, 2017 at 04:45:59PM +0800, Chao Gao wrote:
> On Mon, Sep 11, 2017 at 10:14:15AM +0100, Roger Pau Monné wrote:
> >On Mon, Sep 11, 2017 at 09:58:01AM +0800, Chao Gao wrote:
> >> Hi, Roger.
> >> 
> >> I meet an error when creating a pvh guest. I am using commit 6e2a4c73564a. 
> >> From the error log, I found bootlate_pv()->pin_table always failed. And the
> >> failure was caused by is_pv_domain(pg_owner) in do_mmuext_op(). Do you have
> >> any idea on this?
> >
> >IIRC bootlate_pv should never be called for a PVH guest. I'm not
> >really sure how you can get there, can you paste/attach your Linux
> >kernel config file?
> 
> I really got there. I don't set XEN_PVH in config file. After setting
> XEN_PVH, the error disappears. From the output of xc_dom_compat_check(),
> I guess the guest was wrongly treated as a pv guest.

OK, I'm not really thrilled to fix this because this interface is
hopefully going away really soon, we'll see if the same thing happens
with the new one and then we can hopefully fix it.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel