Re: xen guest hang with TSC_DEADLINE

[Christoph Egger <Christoph.Egger@amd.com>]

On Sunday 19 December 2010 15:26:30 Wei, Gang wrote:
> Christoph Egger wrote on 2010-12-17:
> > Hi,
> >
> > c/s 22529 and 22530 cause a xen guest hang.
> >
> > While "normal" guests like Linux and NetBSD boot fine I boot Xen
> > itself as a xen guest for my nested virtualization.
> >
> > When I do that then the guest dom0 hangs at boot when it tries to
> > initialize the first vcpu.
> > The bug is introduced somewhere in c/s 22529 and triggers with c/s 22530.
>
> Can you enable apic_timer debug info var hvm_debug and give more serial
> port log around the guest dom0 hangs? I used to test xen guest, it works
> well expect that it boot a little bit slowly.

This is the log output I get with TSC_DEADLINE feature enabled:

(XEN) [HVM:1.0] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.0] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.0] <vlapic_get_tmcct> timer initial count 1000000000, timer 
current count 999546729, offset 453271
(XEN) [HVM:1.0] <vlapic_get_tmcct> timer initial count 1000000000, timer 
current count 989547039, offset 10452961
(XEN) [HVM:1.0] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.0] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.0] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.0] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.0] <vlapic_tdt_msr_set> ignore tsc deadline msr write
(XEN) [HVM:1.1] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.0] <vlapic_tdt_msr_set> ignore tsc deadline msr write
(XEN) [HVM:1.1] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.1] <vlapic_tdt_msr_set> ignore tsc deadline msr write
(XEN) [HVM:1.1] <vlapic_tdt_msr_set> ignore tsc deadline msr write
(XEN) [HVM:1.2] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.2] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.2] <vlapic_tdt_msr_set> ignore tsc deadline msr write
(XEN) [HVM:1.2] <vlapic_tdt_msr_set> ignore tsc deadline msr write
(XEN) [HVM:1.3] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.3] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.3] <vlapic_tdt_msr_set> ignore tsc deadline msr write
(XEN) [HVM:1.3] <vlapic_tdt_msr_set> ignore tsc deadline msr write

The guest dom0 output right before the hang:

ioapic0 at mainbus0 apid 1, virtual wire mode
hypervisor0 at mainbus0: Xen version 4.1
vcpu0 at hypervisor0

The vcpu driver tries to detect the tsc frequency here.
The dom0 uses the xen clock timer, the same a PV guest
uses.


This is the log output I get with TSC_DEADLINE feature disabled:

(XEN) [HVM:1.0] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.0] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.0] <vlapic_get_tmcct> timer initial count 1000000000, timer 
current count 999716563, offset 283437
(XEN) [HVM:1.0] <vlapic_get_tmcct> timer initial count 1000000000, timer 
current count 989716153, offset 10283847
(XEN) [HVM:1.0] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.0] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.0] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.0] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.1] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.1] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.2] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.2] <vlapic_reg_write> timer divisor is 0x1
(XEN) [HVM:1.3] <vlapic_set_tdcr> timer_divisor: 1
(XEN) [HVM:1.3] <vlapic_reg_write> timer divisor is 0x1


The guest dom0 output:

ioapic0 at mainbus0 apid 1, virtual wire mode
hypervisor0 at mainbus0: Xen version 4.1
vcpu0 at hypervisor0: AMD 686-class, 1895MHz
xenbus0 at hypervisor0: Xen Virtual Bus Interface
[...]


-- 
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632