Re: Branch Trace Storage for guests and VPMU initialization

[Boris Ostrovsky <boris.ostrovsky@oracle.com>]

On 02/24/2015 10:27 AM, Kevin.Mayer@gdata.de wrote:
>
> Hi guys
>
> I`m trying to set up the BTS so that I can log the branches taken in 
> the guest using Xen 4.4.1 with a WinXP SP3 guest on a Core i7 Sandy 
> Bridge.
>
> I added the vpmu=bts boot parameter to my grub2 configuration and 
> extended the libxl,libxc,domctl,… with an own command so that I can 
> trigger the activation of the BTS whenever I want.
>


I am not sure why you are doing all these changes to Xen code. BTS is 
supposed to be managed from the guest. For example, a Fedora HVM guest 
will produce this:

[root@dhcp-burlington7-2nd-B-east-10-152-55-140 ~]# perf record -e 
branches:u -c 1 -d sleep 1
[ perf record: Woken up 3838 times to write data ]
[ perf record: Captured and wrote 0.704 MB perf.data (~30756 samples) ]
[root@dhcp-burlington7-2nd-B-east-10-152-55-140 ~]# perf script -f 
ip,addr,sym,dso,symoff --show-kernel-path
  ffffffff8167c347 native_irq_return_iret+0x0 (/proc/kcore) =>       
328c001590 [unknown] (/proc/kcore)
  ffffffff8167c347 native_irq_return_iret+0x0 (/proc/kcore) =>       
328c001590 [unknown] ([unknown])
        328c001593 [unknown] ([unknown]) =>       328c004b70 [unknown] 
([unknown])
...

> In this command I do the following:
>
> I set up the memory region for the BTS Buffer and the DS Buffer 
> Management Area using xzalloc_bytes
>


I don't think you should be allocating BTS buffers in the hypervisor, 
they are in guest's memory.


> Then I write the pointer to the BTS Buffer into the DS Buffer 
> Management Area at +0x0 and +0x8 (BTS Buffer Base and BTS Index)
>
> When I use vmx_msr_write_intercept to store the value in 
> MSR_IA32_DS_AREA the host reboots (my idea is he tries to access a 
> vpmu-struct that isn´t there in the current vcpu and panics).
>

Can you post hypervisor log? (hard to say how helpful it will be without 
seeing your code changes though)

> When I use a modified version of vmx_msr_write_intercept I don’t get 
> any crashes as long as I don’t enable BTS and TR in the 
> GUEST_IA32_DEBUGCTL (BTR works). When I enable the BTS (and TR) the 
> guest crashes. I suppose he gets killed by the hypervisor for 
> accessing forbidden memory.
>

Possibly because DS area point to hypervisor memory.


Having said all this, I am not sure how well BTS works. You did notice 
this in the hypervisor log:

(XEN) ******************************************************
(XEN) ** WARNING: Emulation of BTS Feature is switched on **
(XEN) ** Using this processor feature in a virtualized **
(XEN) ** environment is not 100% safe. **
(XEN) ** Setting the DS buffer address with wrong values **
(XEN) ** may lead to hypervisor hangs or crashes. **
(XEN) ** It is NOT recommended for production use! **
(XEN) ******************************************************


-boris


> The modified version of vmx_msr_write_intercept takes a vcpu-struct as 
> a parameter and uses this instead of the current vcpu.
>
> Instead of
>
> staticint vmx_msr_write_intercept(unsigned int msr, uint64_t msr_content)
>
> {
>
>     struct vcpu *v = current;
>
> I just have
>
> staticint own_vmx_msr_write_intercept(unsigned int msr, uint64_t 
> msr_content, struct vcpu *v)
>
> I get this vcpu by d->vcpu[0] as I have limited my guest domain to one 
> vcpu atm.
>
> Of course I also use similarly modified version of the called 
> functions(vpmu_do_wrmsr,…).
>
> I´m pretty sure that my problem is with a wrong scope/usage of the 
> vcpus/memory, but I have no idea how to fix this.
>
> I can see a potential problem with the memory allocation (in the host) 
> into which the cpu in guest-mode is supposed to write.
>
> Or maybe I got the principle of a vcpu/vpmu all wrong.
>
> Since I couldn’t find any project that uses the BTS for the guest, I 
> am wondering if anyone has ever done this and if it is possible at all.
>
> Any input is welcome as I am pretty much stuck atm…
>
> Cheers
>
> Kevin
>
>
> ____________
> Virus checked by G Data MailSecurity
> Version: AVA 25.404 dated 24.02.2015
> Virus news: www.antiviruslab.com <http://www.antiviruslab.com>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel