From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wei Huang Subject: Re: LWP Interrupt Handler Date: Thu, 29 Mar 2012 17:54:29 -0500 Message-ID: <4F74E825.9060909@amd.com> References: <4F6CF344.30802@amd.com> Reply-To: wei.huang2@amd.com Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------030609080306070606070401" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Zhang, Xiantao" Cc: Keir Fraser , Jan Beulich , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org --------------030609080306070606070401 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit How about the approach in attached patches? A interrupt handler is registered at vector 0xf6 as a delegate for guest VM. Guest VCPU can register interrupt handler to receive notification. I will prepare formal ones after receiving your comments. Thanks, -Wei On 03/25/2012 07:08 PM, Zhang, Xiantao wrote: > Please make sure the per-cpu vector is considered in your case. For CPU's built-in event, it always happens on all cpus, but IRQ-based events may only happen on some special cpus which are determined by apic's mode. > Xiantao > >> -----Original Message----- >> From: xen-devel-bounces@lists.xen.org [mailto:xen-devel- >> bounces@lists.xen.org] On Behalf Of Keir Fraser >> Sent: Saturday, March 24, 2012 6:50 AM >> To: wei.huang2@amd.com; xen-devel@lists.xen.org; Jan Beulich >> Subject: Re: [Xen-devel] LWP Interrupt Handler >> >> On 23/03/2012 22:03, "Wei Huang" wrote: >> >>> I am adding interrupt support for LWP, whose spec is available at >>> http://support.amd.com/us/Processor_TechDocs/43724.pdf. Basically OS >>> can specify an interrupt vector in LWP_CFG MSR; the interrupt will be >>> triggered when event buffer overflows. For HVM guests, I want to >>> re-inject this interrupt back into the guest VM. Here is one idea >>> similar to virtualized PMU: It first registers a special interrupt >>> handler (say on vector 0xf6) using set_intr_gate(). When triggered, >>> this handler injects an IRQ (with vector copied from LWP_CFG) into >>> guest VM via virtual local APIC. This worked from my test. >>> >>> But adding a interrupt handler seems to be an overkill. Is there any >>> better way to create a dummy interrupt receiver on be-behalf of guest >>> VMs? I also looked into IRQ and MSI solutions inside Xen. But most of >>> them assume that interrupts are from physical device (but not in this >>> LWP case, where interrupt is initiated from CPU itself); so they don't >>> fit very well. >> I think just allocating a vector is fine. If we get too many we could move to >> dynamic allocation of them. >> >> -- Keir >> >>> Thanks, >>> -Wei >>> >>> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel --------------030609080306070606070401 Content-Type: text/plain; name="RFC_amd_lwp_interrupt.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="RFC_amd_lwp_interrupt.txt" Content-Description: RFC_amd_lwp_interrupt.txt # HG changeset patch # User Wei Huang # Date 1333061831 18000 # Node ID 07f17f70ffe4e733eff1965e011be5693622d129 # Parent d7fdc948f6540427c4c20da216b304fc7a11cd76 AMD_LWP: add interrupt support for AMD LWP This patch adds interrupt support for AMD lightweight profiling. It registers interrupt handler using CPU affiliated interrupt delegate. Hypervisor reinjects virtual interrupt into guest VM using local APIC. Signed-off-by: Wei Huang diff -r d7fdc948f654 -r 07f17f70ffe4 xen/arch/x86/hvm/svm/svm.c --- a/xen/arch/x86/hvm/svm/svm.c Thu Mar 29 17:51:16 2012 -0500 +++ b/xen/arch/x86/hvm/svm/svm.c Thu Mar 29 17:57:11 2012 -0500 @@ -759,7 +759,17 @@ { /* Only LWP_CFG is reloaded. LWP_CBADDR will be reloaded via xrstor. */ if ( v->arch.hvm_svm.guest_lwp_cfg ) - wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.guest_lwp_cfg); + wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.cpu_lwp_cfg); +} + +int svm_lwp_intr_handler(struct cpu_user_regs *regs) +{ + struct vcpu *v = current; + + vlapic_set_irq(vcpu_vlapic(v), + (v->arch.hvm_svm.guest_lwp_cfg >> 40) && 0xff, 0); + + return 1; } /* Update LWP_CFG MSR (0xc0000105). Return -1 if error; otherwise returns 0. */ @@ -776,12 +786,24 @@ /* generate #GP if guest tries to turn on unsupported features. */ if ( msr_low & ~edx) return -1; - - wrmsrl(MSR_AMD64_LWP_CFG, msr_content); - /* CPU might automatically correct reserved bits. So read it back. */ - rdmsrl(MSR_AMD64_LWP_CFG, msr_content); + v->arch.hvm_svm.guest_lwp_cfg = msr_content; + /* setup interrupt handler */ + if ( (msr_content & 0x80000000) && ((msr_content >> 40) & 0xff) ) + { + v->arch.hvm_svm.cpu_lwp_cfg = (msr_content & 0xffff00ffffffffffULL) + | ((uint64_t)CPU_AFFILIATE_VECTOR << 40); + v->arch.hvm_vcpu.affiliate_intr_handler = svm_lwp_intr_handler; + } + else + { + /* disable interrupt */ + v->arch.hvm_svm.cpu_lwp_cfg = msr_content & 0xffff00ff7fffffffULL; + v->arch.hvm_vcpu.affiliate_intr_handler = NULL; + } + wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.cpu_lwp_cfg); + /* track nonalzy state if LWP_CFG is non-zero. */ v->arch.nonlazy_xstate_used = !!(msr_content); } diff -r d7fdc948f654 -r 07f17f70ffe4 xen/include/asm-x86/hvm/svm/vmcb.h --- a/xen/include/asm-x86/hvm/svm/vmcb.h Thu Mar 29 17:51:16 2012 -0500 +++ b/xen/include/asm-x86/hvm/svm/vmcb.h Thu Mar 29 17:57:11 2012 -0500 @@ -515,7 +515,8 @@ uint64_t guest_sysenter_eip; /* AMD lightweight profiling MSR */ - uint64_t guest_lwp_cfg; + uint64_t guest_lwp_cfg; /* guest version */ + uint64_t cpu_lwp_cfg; /* cpu version */ /* OSVW MSRs */ struct { --------------030609080306070606070401 Content-Type: text/plain; name="RFC_cpu_interrupt_delegate.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="RFC_cpu_interrupt_delegate.txt" Content-Description: RFC_cpu_interrupt_delegate.txt # HG changeset patch # User Wei Huang # Date 1333061476 18000 # Node ID d7fdc948f6540427c4c20da216b304fc7a11cd76 # Parent 86fac5ca5d9234d293047fc7b310127f7eaa91f3 x86: add cpu affiliated interrupt delegate for guest VM This patch adds a delegate on-behalf-of guest VM for interrupts which are initiated by CPU. When this interrupt is triggered, hypervisor calls registered interrupt handler to notify guest that the interrupt has arrived. Guest VM can disable receiving this notification by setting handler as NULL. Signed-off-by: Wei Huang diff -r 86fac5ca5d92 -r d7fdc948f654 xen/arch/x86/apic.c --- a/xen/arch/x86/apic.c Thu Mar 29 17:46:18 2012 -0500 +++ b/xen/arch/x86/apic.c Thu Mar 29 17:51:16 2012 -0500 @@ -111,6 +111,13 @@ BUILD_SMP_INTERRUPT(thermal_interrupt,THERMAL_APIC_VECTOR) #endif +/* The following vector is received on-behalf-of guest VM for interrupts which + * are initiated by CPU locally. One example is light-weight profiling (LWP), + * which can trigger an interrupt when event buffer overflows. Hypervisor will + * notify guest VM about the arrival of interrupts. + */ +BUILD_SMP_INTERRUPT(cpu_affiliate_interrupt, CPU_AFFILIATE_VECTOR) + static int modern_apic(void) { unsigned int lvr, version; diff -r 86fac5ca5d92 -r d7fdc948f654 xen/arch/x86/hvm/hvm.c --- a/xen/arch/x86/hvm/hvm.c Thu Mar 29 17:46:18 2012 -0500 +++ b/xen/arch/x86/hvm/hvm.c Thu Mar 29 17:51:16 2012 -0500 @@ -1438,6 +1438,19 @@ return -1; } +int hvm_handle_affiliate_intr(struct cpu_user_regs *regs) +{ + struct vcpu *curr = current; + + if ( !curr->arch.hvm_vcpu.affiliate_intr_handler ) + { + printk("HVM: received an unregistered CPU affiliated interrupt!\n"); + return 0; + } + else + return curr->arch.hvm_vcpu.affiliate_intr_handler(regs); +} + int hvm_set_efer(uint64_t value) { struct vcpu *v = current; diff -r 86fac5ca5d92 -r d7fdc948f654 xen/arch/x86/irq.c --- a/xen/arch/x86/irq.c Thu Mar 29 17:46:18 2012 -0500 +++ b/xen/arch/x86/irq.c Thu Mar 29 17:51:16 2012 -0500 @@ -608,6 +608,16 @@ desc->handler->enable(desc); } +fastcall void smp_cpu_affiliate_interrupt(struct cpu_user_regs *regs) +{ + struct cpu_user_regs *old_regs = set_irq_regs(regs); + + ack_APIC_irq(); + this_cpu(irq_count)++; + hvm_handle_affiliate_intr(regs); + set_irq_regs(old_regs); +} + static void dump_irqs(unsigned char key); fastcall void smp_irq_move_cleanup_interrupt(struct cpu_user_regs *regs) diff -r 86fac5ca5d92 -r d7fdc948f654 xen/arch/x86/smpboot.c --- a/xen/arch/x86/smpboot.c Thu Mar 29 17:46:18 2012 -0500 +++ b/xen/arch/x86/smpboot.c Thu Mar 29 17:51:16 2012 -0500 @@ -1037,4 +1037,7 @@ /* IPI for generic function call */ set_intr_gate(CALL_FUNCTION_VECTOR, call_function_interrupt); + + /* CPU affiliate interrupt delegate */ + set_intr_gate(CPU_AFFILIATE_VECTOR, cpu_affiliate_interrupt); } diff -r 86fac5ca5d92 -r d7fdc948f654 xen/include/asm-x86/hvm/hvm.h --- a/xen/include/asm-x86/hvm/hvm.h Thu Mar 29 17:46:18 2012 -0500 +++ b/xen/include/asm-x86/hvm/hvm.h Thu Mar 29 17:51:16 2012 -0500 @@ -309,6 +309,7 @@ void hvm_migrate_pirqs(struct vcpu *v); void hvm_inject_exception(unsigned int trapnr, int errcode, unsigned long cr2); +int hvm_handle_affiliate_intr(struct cpu_user_regs *regs); static inline int hvm_event_pending(struct vcpu *v) { diff -r 86fac5ca5d92 -r d7fdc948f654 xen/include/asm-x86/hvm/vcpu.h --- a/xen/include/asm-x86/hvm/vcpu.h Thu Mar 29 17:46:18 2012 -0500 +++ b/xen/include/asm-x86/hvm/vcpu.h Thu Mar 29 17:51:16 2012 -0500 @@ -170,6 +170,10 @@ unsigned long inject_cr2; struct viridian_vcpu viridian; + + /* callback handler for CPU affiliated interrupt. Guest VM can setup this + * handler if it wants to receive notification. NULL means no handler. */ + int (*affiliate_intr_handler)(struct cpu_user_regs *regs); }; #endif /* __ASM_X86_HVM_VCPU_H__ */ diff -r 86fac5ca5d92 -r d7fdc948f654 xen/include/asm-x86/irq.h --- a/xen/include/asm-x86/irq.h Thu Mar 29 17:46:18 2012 -0500 +++ b/xen/include/asm-x86/irq.h Thu Mar 29 17:51:16 2012 -0500 @@ -89,6 +89,7 @@ fastcall void thermal_interrupt(void); fastcall void cmci_interrupt(void); fastcall void irq_move_cleanup_interrupt(void); +fastcall void cpu_affiliate_interrupt(void); fastcall void smp_event_check_interrupt(struct cpu_user_regs *regs); fastcall void smp_invalidate_interrupt(void); @@ -100,6 +101,7 @@ fastcall void smp_thermal_interrupt(struct cpu_user_regs *regs); fastcall void smp_cmci_interrupt(struct cpu_user_regs *regs); fastcall void smp_irq_move_cleanup_interrupt(struct cpu_user_regs *regs); +fastcall void smp_cpu_affiliate_interrupt(struct cpu_user_regs *regs); void do_IRQ(struct cpu_user_regs *regs); diff -r 86fac5ca5d92 -r d7fdc948f654 xen/include/asm-x86/mach-default/irq_vectors.h --- a/xen/include/asm-x86/mach-default/irq_vectors.h Thu Mar 29 17:46:18 2012 -0500 +++ b/xen/include/asm-x86/mach-default/irq_vectors.h Thu Mar 29 17:51:16 2012 -0500 @@ -11,12 +11,18 @@ #define LOCAL_TIMER_VECTOR 0xf9 #define PMU_APIC_VECTOR 0xf8 #define CMCI_APIC_VECTOR 0xf7 + +/* CPU affiliate interrupt vector, which is used for interrupts that are + * initiated by CPU locally. Hypervisor receives on-behalf-of guest VM. + */ +#define CPU_AFFILIATE_VECTOR 0xf6 + /* * High-priority dynamically-allocated vectors. For interrupts that * must be higher priority than any guest-bound interrupt. */ #define FIRST_HIPRIORITY_VECTOR 0xf0 -#define LAST_HIPRIORITY_VECTOR 0xf6 +#define LAST_HIPRIORITY_VECTOR 0xf5 /* Legacy PIC uses vectors 0xe0-0xef. */ #define FIRST_LEGACY_VECTOR 0xe0 --------------030609080306070606070401 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --------------030609080306070606070401--