Re: LWP Interrupt Handler

[Wei Huang <wei.huang2@amd.com>]

[-- Attachment #1: Type: text/plain, Size: 3802 bytes --]

Thanks. I created the LWP interrupt patch based on your solution. It is 
attached in this email. This patch was tested using Hans Rosenfeld's LWP 
tree (http://www.amd64.org/gitweb/?p=linux/lwp.git;a=summary).

=================
AMD_LWP: add interrupt support for AMD LWP

This patch adds interrupt support for AMD lightweight profiling. It
registers interrupt handler using alloc_direct_apic_vector(). When
notified, SVM reinjects virtual interrupts into guest VM using
guest's virtual local APIC.

Signed-off-by: Wei Huang <wei.huang2@amd.com>
=================

-Wei

On 03/30/2012 03:21 AM, Keir Fraser wrote:
> On 29/03/2012 23:54, "Wei Huang"<wei.huang2@amd.com>  wrote:
>
>> How about the approach in attached patches? A interrupt handler is
>> registered at vector 0xf6 as a delegate for guest VM. Guest VCPU can
>> register interrupt handler to receive notification. I will prepare
>> formal ones after receiving your comments.
> Ugh, no, these ad-hoc dynamically-registered subhandlers are getting beyond
> the pale. Therefore, I've just spent half hour routing all our interrupts,
> even the direct APIC ones, through do_IRQ(), and now allow everything to be
> dynamically allocated.
>
> Please pull up to xen-unstable tip (>= 25113) and use
> alloc_direct_apic_vector(). You can find a few callers of it that I added
> myself, to give you an idea how to use it. In essence, you can call it from
> your SVM-specific code, to allocate a vector for your irq handler. And you
> can do it immediately before you poke the vector into your special MSR. You
> can call alloc_direct_apic_vector() unconditionally -- it will deal with
> ensuring the allocation happens only once.
>
>   -- Keir
>
>> Thanks,
>> -Wei
>>
>>
>> On 03/25/2012 07:08 PM, Zhang, Xiantao wrote:
>>> Please make sure the per-cpu vector is considered in your case.  For CPU's
>>> built-in event, it always happens on all cpus,  but IRQ-based events may only
>>> happen on some special cpus which are determined by apic's mode.
>>> Xiantao
>>>
>>>> -----Original Message-----
>>>> From: xen-devel-bounces@lists.xen.org [mailto:xen-devel-
>>>> bounces@lists.xen.org] On Behalf Of Keir Fraser
>>>> Sent: Saturday, March 24, 2012 6:50 AM
>>>> To: wei.huang2@amd.com; xen-devel@lists.xen.org; Jan Beulich
>>>> Subject: Re: [Xen-devel] LWP Interrupt Handler
>>>>
>>>> On 23/03/2012 22:03, "Wei Huang"<wei.huang2@amd.com>   wrote:
>>>>
>>>>> I am adding interrupt support for LWP, whose spec is available at
>>>>> http://support.amd.com/us/Processor_TechDocs/43724.pdf. Basically OS
>>>>> can specify an interrupt vector in LWP_CFG MSR; the interrupt will be
>>>>> triggered when event buffer overflows. For HVM guests, I want to
>>>>> re-inject this interrupt back into the guest VM. Here is one idea
>>>>> similar to virtualized PMU: It first registers a special interrupt
>>>>> handler (say on vector 0xf6) using set_intr_gate(). When triggered,
>>>>> this handler injects an IRQ (with vector copied from LWP_CFG) into
>>>>> guest VM via virtual local APIC. This worked from my test.
>>>>>
>>>>> But adding a interrupt handler seems to be an overkill. Is there any
>>>>> better way to create a dummy interrupt receiver on be-behalf of guest
>>>>> VMs? I also looked into IRQ and MSI solutions inside Xen. But most of
>>>>> them assume that interrupts are from physical device (but not in this
>>>>> LWP case, where interrupt is initiated from CPU itself); so they don't
>>>>> fit very well.
>>>> I think just allocating a vector is fine. If we get too many we could move
>>>> to
>>>> dynamic allocation of them.
>>>>
>>>>    -- Keir
>>>>
>>>>> Thanks,
>>>>> -Wei
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@lists.xen.org
>>>> http://lists.xen.org/xen-devel
>
>


[-- Attachment #2: amd_lwp_interrupt.txt --]
[-- Type: text/plain, Size: 3635 bytes --]

# HG changeset patch
# User Wei Huang <wei.huang2@amd.com>
# Date 1333136756 18000
# Node ID b3d45e0ffd4b7e2ea409e6d376951f33ca7feadf
# Parent  6765a2510ee7ad899dcb87eefdf206f8c8ae34ae
AMD_LWP: add interrupt support for AMD LWP

This patch adds interrupt support for AMD lightweight profiling. It
registers interrupt handler using alloc_direct_apic_vector(). When
notified, SVM reinjects virtual interrupts into guest VM using
guest's virtual local APIC.

Signed-off-by: Wei Huang <wei.huang2@amd.com>

diff -r 6765a2510ee7 -r b3d45e0ffd4b xen/arch/x86/hvm/svm/svm.c
--- a/xen/arch/x86/hvm/svm/svm.c	Fri Mar 30 10:01:15 2012 +0100
+++ b/xen/arch/x86/hvm/svm/svm.c	Fri Mar 30 14:45:56 2012 -0500
@@ -87,6 +87,9 @@
 static uint64_t osvw_length, osvw_status;
 static DEFINE_SPINLOCK(osvw_lock);
 
+/* LWP Vector */
+static uint8_t lwp_intr_vector;
+
 void __update_guest_eip(struct cpu_user_regs *regs, unsigned int inst_len)
 {
     struct vcpu *curr = current;
@@ -745,6 +748,26 @@
     *(u16 *)(hypercall_page + (__HYPERVISOR_iret * 32)) = 0x0b0f; /* ud2 */
 }
 
+/* LWP interrupt handler */
+static void svm_lwp_interrupt(struct cpu_user_regs *regs)
+{
+    struct vcpu *v = current;
+
+    ack_APIC_irq();
+    vlapic_set_irq(vcpu_vlapic(v),
+                   (v->arch.hvm_svm.guest_lwp_cfg >> 40) && 0xff, 0);    
+}
+
+/* Init LWP interrupt handler */
+static void svm_init_lwp_intr(void)
+{
+    /* return if already allocated */
+    if ( lwp_intr_vector > 0 )
+        return;
+
+    alloc_direct_apic_vector(&lwp_intr_vector, svm_lwp_interrupt);
+}
+
 static inline void svm_lwp_save(struct vcpu *v)
 {
     /* Don't mess up with other guests. Disable LWP for next VCPU. */
@@ -759,7 +782,7 @@
 {
     /* Only LWP_CFG is reloaded. LWP_CBADDR will be reloaded via xrstor. */
    if ( v->arch.hvm_svm.guest_lwp_cfg ) 
-       wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.guest_lwp_cfg);
+       wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.cpu_lwp_cfg);
 }
 
 /* Update LWP_CFG MSR (0xc0000105). Return -1 if error; otherwise returns 0. */
@@ -776,11 +799,23 @@
         /* generate #GP if guest tries to turn on unsupported features. */
         if ( msr_low & ~edx)
             return -1;
+
+        v->arch.hvm_svm.guest_lwp_cfg = msr_content;
+
+        /* setup interrupt handler if needed */
+        if ( (msr_content & 0x80000000) && ((msr_content >> 40) & 0xff) )
+        {
+            svm_init_lwp_intr();
+            v->arch.hvm_svm.cpu_lwp_cfg = (msr_content & 0xffff00ffffffffffULL)
+                | ((uint64_t)lwp_intr_vector << 40);
+        }
+        else
+        {
+            /* otherwise disable it */
+            v->arch.hvm_svm.cpu_lwp_cfg = msr_content & 0xffff00ff7fffffffULL;
+        }
         
-        wrmsrl(MSR_AMD64_LWP_CFG, msr_content);
-        /* CPU might automatically correct reserved bits. So read it back. */
-        rdmsrl(MSR_AMD64_LWP_CFG, msr_content);
-        v->arch.hvm_svm.guest_lwp_cfg = msr_content;
+        wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.cpu_lwp_cfg);
 
         /* track nonalzy state if LWP_CFG is non-zero. */
         v->arch.nonlazy_xstate_used = !!(msr_content);
diff -r 6765a2510ee7 -r b3d45e0ffd4b xen/include/asm-x86/hvm/svm/vmcb.h
--- a/xen/include/asm-x86/hvm/svm/vmcb.h	Fri Mar 30 10:01:15 2012 +0100
+++ b/xen/include/asm-x86/hvm/svm/vmcb.h	Fri Mar 30 14:45:56 2012 -0500
@@ -515,7 +515,8 @@
     uint64_t guest_sysenter_eip;
     
     /* AMD lightweight profiling MSR */
-    uint64_t guest_lwp_cfg;
+    uint64_t guest_lwp_cfg;      /* guest version */
+    uint64_t cpu_lwp_cfg;        /* CPU version */
 
     /* OSVW MSRs */
     struct {

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel