[AMD PVH]: Partial/domU xen patches...

[AMD PVH]: Partial/domU xen patches...

[Mukesh Rathor]

Hi Elena/Boris:

Actually, I forgot I had already made and tested AMD SVM and other changes 
for domU support, please find the patches.

So the only thing remaining for AMD would be iommu support and SVM vmexit 
for CR reads and writes which currently calls handle_mmio (which patch #3
attempted, but improperly).  I believe Jan or Roger is already looking 
into that path, so you could start down the iommu path... the last I 
remember was :

(XEN) AMD-Vi: IO_PAGE_FAULT: domain = 0, device id = 0x99, fault address = 0xffffffc0, flags = 0
(XEN) AMD-Vi: IO_PAGE_FAULT: domain = 0, device id = 0x91, fault address = 0xffffffc0, flags = 0
(XEN) AMD-Vi: IO_PAGE_FAULT: domain = 0, device id = 0x90, fault address = 0xffffffc0, flags = 0

(XEN) Xen call trace:
(XEN)    [<ffff82d08014d600>] parse_event_log_entry+0/0x140
(XEN)    [<ffff82d08014dfdf>] iommu_read_log.constprop.6+0x8f/0x100
(XEN)    [<ffff82d08014e165>] do_amd_iommu_irq+0x115/0x1f0
(XEN)    [<ffff82d0801296d0>] do_tasklet_work+0x60/0xa0
(XEN)    [<ffff82d080129750>] tasklet_softirq_action+0x40/0x70
(XEN)    [<ffff82d080126ee5>] __do_softirq+0x65/0xa0
(XEN)    [<ffff82d0802abe68>] _setup_hwdom_pci_devices+0xa8/0x190
(XEN)    [<ffff82d0802abdc0>] _setup_hwdom_pci_devices+0/0x190
(XEN)    [<ffff82d0801435cb>] pci_segments_iterate+0x2b/0x70
(XEN)    [<ffff82d0802ac2e8>] setup_hwdom_pci_devices+0x28/0x40
(XEN)    [<ffff82d0802aee00>] amd_iommu_setup_hwdom_device+0/0xc0
(XEN)    [<ffff82d0802cbfab>] construct_dom0+0x247b/0x3390
(XEN)    [<ffff82d0802bb810>] bootstrap_map+0/0x10c
(XEN)    [<ffff82d0802bf136>] __start_xen+0x35a6/0x3aa0
(XEN)    [<ffff82d080100067>] __high_start+0x53/0x5c


Thanks,
Mukesh

[V0 PATCH 1/6] AMD-PVH: construct vmcb changes

[Mukesh Rathor]

PVH guest starts in Long 64bit paging mode. This patch modifies
construct_vmcb for that.

Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com>
---
 xen/arch/x86/hvm/svm/vmcb.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/vmcb.c b/xen/arch/x86/hvm/svm/vmcb.c
index 21292bb..5df5f36 100644
--- a/xen/arch/x86/hvm/svm/vmcb.c
+++ b/xen/arch/x86/hvm/svm/vmcb.c
@@ -138,6 +138,8 @@ static int construct_vmcb(struct vcpu *v)
 
     /* Guest EFER. */
     v->arch.hvm_vcpu.guest_efer = 0;
+    if ( is_pvh_vcpu(v) )
+        v->arch.hvm_vcpu.guest_efer |= EFER_LMA;   /* PVH 32bitfixme */
     hvm_update_guest_efer(v);
 
     /* Guest segment limits. */
@@ -162,7 +164,12 @@ static int construct_vmcb(struct vcpu *v)
     vmcb->ds.attr.bytes = 0xc93;
     vmcb->fs.attr.bytes = 0xc93;
     vmcb->gs.attr.bytes = 0xc93;
-    vmcb->cs.attr.bytes = 0xc9b; /* exec/read, accessed */
+
+    if ( is_pvh_vcpu(v) )
+        /* CS.L == 1, exec, read/write, accessed. PVH 32bitfixme. */
+        vmcb->cs.attr.bytes = 0xa9b;
+    else
+        vmcb->cs.attr.bytes = 0xc9b; /* exec/read, accessed */
 
     /* Guest IDT. */
     vmcb->idtr.base = 0;
@@ -184,12 +191,17 @@ static int construct_vmcb(struct vcpu *v)
     vmcb->tr.limit = 0xff;
 
     v->arch.hvm_vcpu.guest_cr[0] = X86_CR0_PE | X86_CR0_ET;
+    /* PVH domains start in paging mode */
+    if ( is_pvh_vcpu(v) )
+        v->arch.hvm_vcpu.guest_cr[0] |= X86_CR0_PG;
     hvm_update_guest_cr(v, 0);
 
-    v->arch.hvm_vcpu.guest_cr[4] = 0;
+    v->arch.hvm_vcpu.guest_cr[4] = is_pvh_vcpu(v) ? X86_CR4_PAE : 0;
     hvm_update_guest_cr(v, 4);
 
-    paging_update_paging_modes(v);
+    /* For pvh, paging mode is updated by arch_set_info_guest(). */
+    if ( is_hvm_vcpu(v) )
+        paging_update_paging_modes(v);
 
     vmcb->_exception_intercepts =
         HVM_TRAP_MASK
-- 
1.8.3.1

[V0 PATCH 2/6] AMD-PVH: cpuid intercept

[Mukesh Rathor]

Call pv_cpuid for pvh cpuid intercept. Note, we modify
svm_vmexit_do_cpuid instead of the intercept switch because the guest
eip needs to be adjusted for pvh also.

Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com>
---
 xen/arch/x86/hvm/svm/svm.c | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 71b8a6a..4ff4a96 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -1517,18 +1517,22 @@ static void svm_vmexit_do_cpuid(struct cpu_user_regs *regs)
     if ( (inst_len = __get_instruction_length(current, INSTR_CPUID)) == 0 )
         return;
 
-    eax = regs->eax;
-    ebx = regs->ebx;
-    ecx = regs->ecx;
-    edx = regs->edx;
-
-    svm_cpuid_intercept(&eax, &ebx, &ecx, &edx);
+    if ( is_pvh_vcpu(current) )
+        pv_cpuid(regs);
+    else
+    {
+        eax = regs->eax;
+        ebx = regs->ebx;
+        ecx = regs->ecx;
+        edx = regs->edx;
 
-    regs->eax = eax;
-    regs->ebx = ebx;
-    regs->ecx = ecx;
-    regs->edx = edx;
+        svm_cpuid_intercept(&eax, &ebx, &ecx, &edx);
 
+        regs->eax = eax;
+        regs->ebx = ebx;
+        regs->ecx = ecx;
+        regs->edx = edx;
+    }
     __update_guest_eip(regs, inst_len);
 }
 
-- 
1.8.3.1

[V0 PATCH 3/6] AMD-PVH: call hvm_emulate_one instead of handle_mmio

[Mukesh Rathor]

Certain IOIO instructions and CR access instructions like
lmsw/clts etc need to be emulated. handle_mmio is incorrectly called to
accomplish this. Create svm_emulate() to call hvm_emulate_one which is more
appropriate, and works for pvh as well. handle_mmio call is
forbidden for pvh.

Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com>
---
 xen/arch/x86/hvm/svm/svm.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 4ff4a96..dac16f4 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -2209,6 +2209,18 @@ static struct hvm_function_table __initdata svm_function_table = {
     .nhvm_hap_walk_L1_p2m = nsvm_hap_walk_L1_p2m,
 };
 
+static void svm_emulate(struct cpu_user_regs *regs)
+{
+    int rc;
+    struct hvm_emulate_ctxt ctxt;
+
+    hvm_emulate_prepare(&ctxt, regs);
+    rc = hvm_emulate_one(&ctxt);
+
+    if ( rc != X86EMUL_OKAY )
+        hvm_inject_hw_exception(TRAP_gp_fault, 0);
+}
+
 void svm_vmexit_handler(struct cpu_user_regs *regs)
 {
     uint64_t exit_reason;
@@ -2470,16 +2482,16 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
             if ( handle_pio(port, bytes, dir) )
                 __update_guest_eip(regs, vmcb->exitinfo2 - vmcb->rip);
         }
-        else if ( !handle_mmio() )
-            hvm_inject_hw_exception(TRAP_gp_fault, 0);
+        else
+            svm_emulate(regs);
         break;
 
     case VMEXIT_CR0_READ ... VMEXIT_CR15_READ:
     case VMEXIT_CR0_WRITE ... VMEXIT_CR15_WRITE:
         if ( cpu_has_svm_decode && (vmcb->exitinfo1 & (1ULL << 63)) )
             svm_vmexit_do_cr_access(vmcb, regs);
-        else if ( !handle_mmio() ) 
-            hvm_inject_hw_exception(TRAP_gp_fault, 0);
+        else
+            svm_emulate(regs);
         break;
 
     case VMEXIT_INVLPG:
-- 
1.8.3.1

[V0 PATCH 4/6] AMD-PVH: Do not get/set vlapic TPR

[Mukesh Rathor]

PVH doesn't use apic emulation hence vlapic->regs ptr is not set for it.

Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com>
---
 xen/arch/x86/hvm/svm/svm.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index dac16f4..4bb4ff2 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -1052,7 +1052,7 @@ static void noreturn svm_do_resume(struct vcpu *v)
         hvm_asid_flush_vcpu(v);
     }
 
-    if ( !vcpu_guestmode )
+    if ( !vcpu_guestmode && vcpu_vlapic(v)->regs )
     {
         vintr_t intr;
 
@@ -2247,7 +2247,7 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
      * NB. We need to preserve the low bits of the TPR to make checked builds
      * of Windows work, even though they don't actually do anything.
      */
-    if ( !vcpu_guestmode ) {
+    if ( !vcpu_guestmode && vcpu_vlapic(v)->regs ) {
         intr = vmcb_get_vintr(vmcb);
         vlapic_set_reg(vcpu_vlapic(v), APIC_TASKPRI,
                    ((intr.fields.tpr & 0x0F) << 4) |
@@ -2628,15 +2628,18 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
     }
 
   out:
-    if ( vcpu_guestmode )
-        /* Don't clobber TPR of the nested guest. */
-        return;
-
-    /* The exit may have updated the TPR: reflect this in the hardware vtpr */
-    intr = vmcb_get_vintr(vmcb);
-    intr.fields.tpr =
-        (vlapic_get_reg(vcpu_vlapic(v), APIC_TASKPRI) & 0xFF) >> 4;
-    vmcb_set_vintr(vmcb, intr);
+    /* Don't clobber TPR of the nested guest. */
+    if ( vcpu_guestmode && vcpu_vlapic(v)->regs )
+    {
+        /*
+         * The exit may have updated the TPR: reflect this in the hardware
+         * vtpr.
+         */
+        intr = vmcb_get_vintr(vmcb);
+        intr.fields.tpr =
+            (vlapic_get_reg(vcpu_vlapic(v), APIC_TASKPRI) & 0xFF) >> 4;
+        vmcb_set_vintr(vmcb, intr);
+    }
 }
 
 void svm_trace_vmentry(void)
-- 
1.8.3.1

[V0 PATCH 5/6] AMD-PVH: Support TSC_MODE_NEVER_EMULATE for PVH

[Mukesh Rathor]

On AMD, MSR_AMD64_TSC_RATIO must be set for rdtsc instruction in guest
to properly read the cpu tsc. To that end, set tsc_khz in struct domain.

Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com>
---
 xen/arch/x86/time.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index bd89219..7512aa4 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -1908,6 +1908,7 @@ void tsc_set_info(struct domain *d,
          * but "always_emulate" does not for some reason.  Figure out
          * why.
          */
+        d->arch.tsc_khz = cpu_khz;
         switch ( tsc_mode )
         {
         case TSC_MODE_NEVER_EMULATE:
-- 
1.8.3.1

[V0 PATCH 6/6] AMD-PVH: enable pvh if requirements met

[Mukesh Rathor]

Finally, enable pvh if the cpu supports NPT and svm decode.

Signed-off-by: Mukesh Rathor <mukesh.rathor@oracle.com>
---
 xen/arch/x86/hvm/svm/svm.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 4bb4ff2..8b27a76 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -1390,6 +1390,9 @@ const struct hvm_function_table * __init start_svm(void)
     svm_function_table.hap_capabilities = HVM_HAP_SUPERPAGE_2MB |
         ((cpuid_edx(0x80000001) & 0x04000000) ? HVM_HAP_SUPERPAGE_1GB : 0);
 
+    if ( cpu_has_svm_npt  && cpu_has_svm_decode )
+        svm_function_table.pvh_supported = 1;
+
     return &svm_function_table;
 }
 
-- 
1.8.3.1