Re: [Xen-users] DomU not starting under pvhv2

Re: [Xen-users] DomU not starting under pvhv2

[George Dunlap]

Moving to xen-devel, and cc'ing Roger and Boris (who developed PVH)

On Mon, Jan 8, 2018 at 5:24 AM, Peter <xen@randomwebstuff.com> wrote:
> Hi.
>
> Running Xen 4.10.0

What version of Linux are you using?

>
> A VM is not starting with type = 'pvh'.  The VM starts, but exits prior to
> any data being read off the domU disk image.
>
> The same VMs works if the type is changed to 'pv'.
>
> Other VMs on other hosts (with the exact same setup) work with pvh fine.
>
> The main difference between hosts where type=pvh does not work and ones
> where it does is that the pvh does not work on older CPUs:
>  Intel(R) Core(TM)2 Duo CPU     E4600  @ 2.40GHz
>  Intel(R) Xeon(R) CPU           E3110  @ 3.00GHz
>  Intel(R) Xeon(R) CPU           E5310  @ 1.60GHz
>  Intel(R) Xeon(R) CPU           E5320  @ 1.86GHz
>  Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
>  Intel(R) Xeon(R) CPU           X3220  @ 2.40GHz
>
>
> Newer CPUs seem to be fine (start VMs with type = pvh):
>  Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
>  Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
>  Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz
>  Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz
>  Intel(R) Xeon(R) CPU E3-1240 v5 @ 3.50GHz
>  Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
>  Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
>  Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
>  Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
>  Intel(R) Xeon(R) CPU           E5506  @ 2.13GHz
>  Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz
>  Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
>  Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
>  Intel(R) Xeon(R) CPU           X3440  @ 2.53GHz
>
> Dom0 command line is: dom0_max_vcpus=2 dom0_vcpus_pin loglvl=info
> vcpu_migration_delay=1000
>
> I see (all?) the older CPUs don't have VT-d.  However some of the CPUs where
> pvh is working do not report hvm_directio in xl info virt_caps (e.g. the
> E5506 servers) though they do have VT-d available per ark.intel.com.
>
> And ideas on what may be stopping type=pvh VMs starting on those hosts?  And
> how we could get it working?
>
> Else we'll likely be dumping some hardware....
>
> I did a diff of xl info; xl dmesg and xl create startup messages.  Between
> one of the non-pvh-able hosts (the - line) and a host where pvh works (the +
> line):
> diff host of xl -v create and xlinfo:
> -hw_caps                :
> bfebfbff:0004e3bd:20100800:00000001:00000000:00000000:00000000:00000000
> -virt_caps              : hvm
> +hw_caps                :
> bfebfbff:009ee3fd:2c100800:00000001:00000000:00000000:00000000:00000100
> +virt_caps              : hvm hvm_directio
>
> +libxl: detail: libxl_dom.c:204:numa_place_domain: NUMA placement candidate
> with 1 nodes, 8 cpus and 16819 KB free selected
>
> - (XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
> + (XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
>
> - (XEN) I/O virtualisation disabled
> + (XEN) Intel VT-d Snoop Control enabled.
> + (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> + (XEN) Intel VT-d Queued Invalidation enabled.
> + (XEN) Intel VT-d Interrupt Remapping enabled.
> + (XEN) Intel VT-d Posted Interrupt not enabled.
> + (XEN) Intel VT-d Shared EPT tables not enabled.
> + (XEN) I/O virtualisation enabled
>
> -(XEN)  -> Using new ACK method
> +(XEN)  -> Using old ACK method
>
>   (XEN) VMX: Supported advanced features:
> + (XEN)  - APIC MMIO access virtualisation
> + (XEN)  - Extended Page Tables (EPT)
> + (XEN)  - Virtual-Processor Identifiers (VPID)
> + (XEN)  - Virtual NMI
> + (XEN)  - Unrestricted Guest
>
>
> -(XEN) HVM: Hardware Assisted Paging (HAP) not detected
> +(XEN) HVM: Hardware Assisted Paging (HAP) detected
>
> +(XEN) Found masked UR signaling on 0000:00:00.0
> +(XEN) Masked UR signaling on 0000:00:01.0
> +(XEN) Masked UR signaling on 0000:00:03.0
> +(XEN) Masked UR signaling on 0000:00:05.0
> +(XEN) Masked UR signaling on 0000:00:07.0
> +(XEN) Masked VT-d error signaling on 0000:00:14.0
>
>
> Regards, Peter
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-users

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-users] DomU not starting under pvhv2

[Peter]

On 2018-01-09 07:01, George Dunlap wrote:
> Moving to xen-devel, and cc'ing Roger and Boris (who developed PVH)
> 
> On Mon, Jan 8, 2018 at 5:24 AM, Peter <xen@randomwebstuff.com> wrote:
>> Hi.
>> 
>> Running Xen 4.10.0
> 
> What version of Linux are you using?

  4.14.12-1.el6xen.x86_64 #2 SMP Fri Jan 5 22:32:01 UTC 2018 x86_64 
x86_64 x86_64 GNU/Linux

# cat /etc/redhat-release
CentOS release 6.9 (Final)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-users] DomU not starting under pvhv2

[Roger Pau Monné]

On Mon, Jan 08, 2018 at 06:01:57PM +0000, George Dunlap wrote:
> Moving to xen-devel, and cc'ing Roger and Boris (who developed PVH)
> 
> On Mon, Jan 8, 2018 at 5:24 AM, Peter <xen@randomwebstuff.com> wrote:
> > Hi.
> >
> > Running Xen 4.10.0
> 
> What version of Linux are you using?
> 
> >
> > A VM is not starting with type = 'pvh'.  The VM starts, but exits prior to
> > any data being read off the domU disk image.

Can you append earlyprintk=xen to your guest command line and try with
a debug build of the hypervisor?

Is there anything relevant in the hypervisor serial console output?
(xl dmesg if you don't have the serial console hooked up)

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-users] DomU not starting under pvhv2

[M A Young]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]

On Wed, 17 Jan 2018, Roger Pau Monné wrote:

> On Mon, Jan 08, 2018 at 06:01:57PM +0000, George Dunlap wrote:
> > Moving to xen-devel, and cc'ing Roger and Boris (who developed PVH)
> > 
> > On Mon, Jan 8, 2018 at 5:24 AM, Peter <xen@randomwebstuff.com> wrote:
> > > Hi.
> > >
> > > Running Xen 4.10.0
> > 
> > What version of Linux are you using?
> > 
> > >
> > > A VM is not starting with type = 'pvh'.  The VM starts, but exits prior to
> > > any data being read off the domU disk image.
> 
> Can you append earlyprintk=xen to your guest command line and try with
> a debug build of the hypervisor?
> 
> Is there anything relevant in the hypervisor serial console output?
> (xl dmesg if you don't have the serial console hooked up)

Also are you using qemu-xen or qemu-xen-traditional? If it is the former 
and you have the error
xen emulation not implemented (yet)
in the relevant qemu-dm log file in /var/log/xen then you presumably need 
the patch at
https://lists.xenproject.org/archives/html/xen-devel/2018-01/msg01289.html

	Michael Young

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel