a). 32/64 compat is missing backtrace support. If you run with a 32-bit dom0 and try to set the backtrace, the hypervisor sets is as -some-huge-number. It might be there are some other hypercalls that need some compat tweaks.

b). 32-bit dom0 oprofile toolstack truncates the EIP of 64-bit guests (or hypervisor). I am not really sure how to solve that except just not run 64-bit guests/hypervisor with a 32-bit dom0. Or make oprofile and its tools capable of doing 64-bit architecture. The vice-versa condition does not exist - so I can profile 32-bit guests using a 64-bit dom0.

# opreport -lwc #(functions
                  calling other functions):

CPU: Core 2, speed 2493.77 MHz (estimated)

Counted CPU_CLK_UNHALTED
                  events (Clock cycles when not halted) with a unit mask
                  of 0x00 (Unhalted
                  core cycles) count 1000000

vma      samples  %       
                  image name               app name                
                  symbol name

-------------------------------------------------------------------------------

  00000000000b9610
                1         2.0000  libc-2.5.so             
                  python                   getaddrinfo

  000000000000fad0
                1         2.0000  libpthread-2.5.so       
                  python                   _fini

000000000000e790

3 6.0000 ld-2.5

.so               
                python                   _dl_fini

  000000000002aec0
                12       24.0000  libc-2.5.so             
                  python                   msort_with_tmp

  0000000000004480 25       50.0000  _xslib.so               
                  python                   init_xslib

0000000000000000
                415746   22.5841  libpython2.4.so.1.0      python                  
                  /usr/lib/libpython2.4.so.1.0

  0000000000000000 415746   100.000  libpython2.4.so.1.0      python                  
                  /usr/lib/libpython2.4.so.1.0 [self]

...

-------------------------------------------------------------------------------

ffff82c480170470 36        0.1587  xen-syms                
                  qemu-dm                  send_IPI_mask_flat

  ffff82c480170470
                36       100.000  xen-syms                
                  qemu-dm                  send_IPI_mask_flat [self]

-------------------------------------------------------------------------------

ffff82c480120d40 33        0.1455  xen-syms                
                  qemu-dm                  cpumask_raise_softirq

  ffff82c480120d40
                33       100.000  xen-syms                
                  qemu-dm                  cpumask_raise_softirq [self]
                

h). There are some counters in the hypervisor for the oprofile statistics, like lost samples, etc. I does not look like they are exported/printed anywhere. Perhaps an 'register_keyhandler' should be written to dump those (and also which domains are profiled).

i). opreports often tells me warning: /domain1-apps could not be found. warning: /domain1-modules could not be found. warning: /domain1-xen-unknown could not be found. warning: /domain2-apps could not be found. warning: /domain2-modules could not be found. warning: /domain2-xen-unknown could not be found. warning: /domain3-apps could not be found. warning: /domain3-modules could not be found. warning: /domain3-xen-unknown could not be found. warning: /vmlinux-unknown could not be found. warning: /xen-unknown could not be found.

And it occurs to me it could be possible be to make some inroads on making performance monitoring easier: 1). fix the glaring omissions in oprofile for the new CPUs 2). Add a register keyhandle to get some debug info. 3). piggyback on oprofile hypercalls and insert some bridge in perf (lots of handwaving here). Or perhaps emulate in the Linux kernel the wmsrs (so xen_safe_wrmsrs) and have the pvops kernel based on the MSRs make the hypercalls to setup the buffers, etc. 3a). new hypercalls? intercept rdmsr/wrmsrs and stuff the right data in the initial domain? Other thoughts? 4). Extend perf to have '--xen' so it can also look at the xen-hypervisor ELF file.