From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wei Huang Subject: Re: LWP Interrupt Handler Date: Fri, 30 Mar 2012 14:44:43 -0500 Message-ID: <4F760D2B.2090206@amd.com> References: Reply-To: wei.huang2@amd.com Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------010701050204090300050405" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Keir Fraser Cc: Jan Beulich , "Zhang, Xiantao" , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org --------------010701050204090300050405 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Thanks. I created the LWP interrupt patch based on your solution. It is attached in this email. This patch was tested using Hans Rosenfeld's LWP tree (http://www.amd64.org/gitweb/?p=linux/lwp.git;a=summary). ================= AMD_LWP: add interrupt support for AMD LWP This patch adds interrupt support for AMD lightweight profiling. It registers interrupt handler using alloc_direct_apic_vector(). When notified, SVM reinjects virtual interrupts into guest VM using guest's virtual local APIC. Signed-off-by: Wei Huang ================= -Wei On 03/30/2012 03:21 AM, Keir Fraser wrote: > On 29/03/2012 23:54, "Wei Huang" wrote: > >> How about the approach in attached patches? A interrupt handler is >> registered at vector 0xf6 as a delegate for guest VM. Guest VCPU can >> register interrupt handler to receive notification. I will prepare >> formal ones after receiving your comments. > Ugh, no, these ad-hoc dynamically-registered subhandlers are getting beyond > the pale. Therefore, I've just spent half hour routing all our interrupts, > even the direct APIC ones, through do_IRQ(), and now allow everything to be > dynamically allocated. > > Please pull up to xen-unstable tip (>= 25113) and use > alloc_direct_apic_vector(). You can find a few callers of it that I added > myself, to give you an idea how to use it. In essence, you can call it from > your SVM-specific code, to allocate a vector for your irq handler. And you > can do it immediately before you poke the vector into your special MSR. You > can call alloc_direct_apic_vector() unconditionally -- it will deal with > ensuring the allocation happens only once. > > -- Keir > >> Thanks, >> -Wei >> >> >> On 03/25/2012 07:08 PM, Zhang, Xiantao wrote: >>> Please make sure the per-cpu vector is considered in your case. For CPU's >>> built-in event, it always happens on all cpus, but IRQ-based events may only >>> happen on some special cpus which are determined by apic's mode. >>> Xiantao >>> >>>> -----Original Message----- >>>> From: xen-devel-bounces@lists.xen.org [mailto:xen-devel- >>>> bounces@lists.xen.org] On Behalf Of Keir Fraser >>>> Sent: Saturday, March 24, 2012 6:50 AM >>>> To: wei.huang2@amd.com; xen-devel@lists.xen.org; Jan Beulich >>>> Subject: Re: [Xen-devel] LWP Interrupt Handler >>>> >>>> On 23/03/2012 22:03, "Wei Huang" wrote: >>>> >>>>> I am adding interrupt support for LWP, whose spec is available at >>>>> http://support.amd.com/us/Processor_TechDocs/43724.pdf. Basically OS >>>>> can specify an interrupt vector in LWP_CFG MSR; the interrupt will be >>>>> triggered when event buffer overflows. For HVM guests, I want to >>>>> re-inject this interrupt back into the guest VM. Here is one idea >>>>> similar to virtualized PMU: It first registers a special interrupt >>>>> handler (say on vector 0xf6) using set_intr_gate(). When triggered, >>>>> this handler injects an IRQ (with vector copied from LWP_CFG) into >>>>> guest VM via virtual local APIC. This worked from my test. >>>>> >>>>> But adding a interrupt handler seems to be an overkill. Is there any >>>>> better way to create a dummy interrupt receiver on be-behalf of guest >>>>> VMs? I also looked into IRQ and MSI solutions inside Xen. But most of >>>>> them assume that interrupts are from physical device (but not in this >>>>> LWP case, where interrupt is initiated from CPU itself); so they don't >>>>> fit very well. >>>> I think just allocating a vector is fine. If we get too many we could move >>>> to >>>> dynamic allocation of them. >>>> >>>> -- Keir >>>> >>>>> Thanks, >>>>> -Wei >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Xen-devel mailing list >>>> Xen-devel@lists.xen.org >>>> http://lists.xen.org/xen-devel > > --------------010701050204090300050405 Content-Type: text/plain; name="amd_lwp_interrupt.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="amd_lwp_interrupt.txt" Content-Description: amd_lwp_interrupt.txt # HG changeset patch # User Wei Huang # Date 1333136756 18000 # Node ID b3d45e0ffd4b7e2ea409e6d376951f33ca7feadf # Parent 6765a2510ee7ad899dcb87eefdf206f8c8ae34ae AMD_LWP: add interrupt support for AMD LWP This patch adds interrupt support for AMD lightweight profiling. It registers interrupt handler using alloc_direct_apic_vector(). When notified, SVM reinjects virtual interrupts into guest VM using guest's virtual local APIC. Signed-off-by: Wei Huang diff -r 6765a2510ee7 -r b3d45e0ffd4b xen/arch/x86/hvm/svm/svm.c --- a/xen/arch/x86/hvm/svm/svm.c Fri Mar 30 10:01:15 2012 +0100 +++ b/xen/arch/x86/hvm/svm/svm.c Fri Mar 30 14:45:56 2012 -0500 @@ -87,6 +87,9 @@ static uint64_t osvw_length, osvw_status; static DEFINE_SPINLOCK(osvw_lock); +/* LWP Vector */ +static uint8_t lwp_intr_vector; + void __update_guest_eip(struct cpu_user_regs *regs, unsigned int inst_len) { struct vcpu *curr = current; @@ -745,6 +748,26 @@ *(u16 *)(hypercall_page + (__HYPERVISOR_iret * 32)) = 0x0b0f; /* ud2 */ } +/* LWP interrupt handler */ +static void svm_lwp_interrupt(struct cpu_user_regs *regs) +{ + struct vcpu *v = current; + + ack_APIC_irq(); + vlapic_set_irq(vcpu_vlapic(v), + (v->arch.hvm_svm.guest_lwp_cfg >> 40) && 0xff, 0); +} + +/* Init LWP interrupt handler */ +static void svm_init_lwp_intr(void) +{ + /* return if already allocated */ + if ( lwp_intr_vector > 0 ) + return; + + alloc_direct_apic_vector(&lwp_intr_vector, svm_lwp_interrupt); +} + static inline void svm_lwp_save(struct vcpu *v) { /* Don't mess up with other guests. Disable LWP for next VCPU. */ @@ -759,7 +782,7 @@ { /* Only LWP_CFG is reloaded. LWP_CBADDR will be reloaded via xrstor. */ if ( v->arch.hvm_svm.guest_lwp_cfg ) - wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.guest_lwp_cfg); + wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.cpu_lwp_cfg); } /* Update LWP_CFG MSR (0xc0000105). Return -1 if error; otherwise returns 0. */ @@ -776,11 +799,23 @@ /* generate #GP if guest tries to turn on unsupported features. */ if ( msr_low & ~edx) return -1; + + v->arch.hvm_svm.guest_lwp_cfg = msr_content; + + /* setup interrupt handler if needed */ + if ( (msr_content & 0x80000000) && ((msr_content >> 40) & 0xff) ) + { + svm_init_lwp_intr(); + v->arch.hvm_svm.cpu_lwp_cfg = (msr_content & 0xffff00ffffffffffULL) + | ((uint64_t)lwp_intr_vector << 40); + } + else + { + /* otherwise disable it */ + v->arch.hvm_svm.cpu_lwp_cfg = msr_content & 0xffff00ff7fffffffULL; + } - wrmsrl(MSR_AMD64_LWP_CFG, msr_content); - /* CPU might automatically correct reserved bits. So read it back. */ - rdmsrl(MSR_AMD64_LWP_CFG, msr_content); - v->arch.hvm_svm.guest_lwp_cfg = msr_content; + wrmsrl(MSR_AMD64_LWP_CFG, v->arch.hvm_svm.cpu_lwp_cfg); /* track nonalzy state if LWP_CFG is non-zero. */ v->arch.nonlazy_xstate_used = !!(msr_content); diff -r 6765a2510ee7 -r b3d45e0ffd4b xen/include/asm-x86/hvm/svm/vmcb.h --- a/xen/include/asm-x86/hvm/svm/vmcb.h Fri Mar 30 10:01:15 2012 +0100 +++ b/xen/include/asm-x86/hvm/svm/vmcb.h Fri Mar 30 14:45:56 2012 -0500 @@ -515,7 +515,8 @@ uint64_t guest_sysenter_eip; /* AMD lightweight profiling MSR */ - uint64_t guest_lwp_cfg; + uint64_t guest_lwp_cfg; /* guest version */ + uint64_t cpu_lwp_cfg; /* CPU version */ /* OSVW MSRs */ struct { --------------010701050204090300050405 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --------------010701050204090300050405--