Re: KVM PMU virtualization

[Ingo Molnar <mingo@elte.hu>]

* Avi Kivity <avi@redhat.com> wrote:

> On 02/26/2010 03:44 PM, Ingo Molnar wrote:
> >* Avi Kivity<avi@redhat.com>  wrote:
> >
> >>On 02/26/2010 03:06 PM, Ingo Molnar wrote:
> >>>>>Firstly, an emulated PMU was only the second-tier option i suggested. By far
> >>>>>the best approach is native API to the host regarding performance events and
> >>>>>good guest side integration.
> >>>>>
> >>>>>Secondly, the PMU cannot be 'given' to the guest in the general case. Those
> >>>>>are privileged registers. They can expose sensitive host execution details,
> >>>>>etc. etc. So if you emulate a PMU you have to exit out of most PMU accesses
> >>>>>anyway for a secure solution. (RDPMC can still be supported, but in close
> >>>>>cooperation with the host)
> >>>>There is nothing secret in the host PMU, and it's easy to clear out the
> >>>>counters before passing them off to the guest.
> >>>That's wrong. On some CPUs the host PMU can be used to say sample aspects of
> >>>another CPU, allowing statistical attacks to recover crypto keys. It can be
> >>>used to sample memory access patterns of another node.
> >>>
> >>>There's a good reason PMU configuration registers are privileged and there's
> >>>good value in only giving a certain sub-set to less privileged entities by
> >>>default.
> >>Even if there were no security considerations, if the guest can observe host
> >>data in the pmu, it means the pmu is inaccurate.  We should expose guest
> >>data only in the guest pmu.  That's not difficult to do, you stop the pmu on
> >>exit and swap the counters on context switches.
> >Again you are making an incorrect assumption: that information leakage via the
> >PMU only occurs while the host is running on that CPU. It does not - the PMU
> >can leak general system details _while the guest is running_.
> 
> You mean like bus transactions on a multicore?  Well, we're already
> exposed to cache timing attacks.

If you give a full PMU to a guest it's a whole different dimension and quality 
of information. Literally hundreds of different events about all sorts of 
aspects of the CPU and the hardware in general.

> >So for this and for the many other reasons we dont want to give a raw PMU to
> >guests:
> >
> >  - A paravirt event driver is more compatible and more transparent in the long
> >    run: it allows hardware upgrade and upgraded PMU functionality (for Linux)
> >    without having to upgrade the guest OS. Via that a guest OS could even be
> >    live-migrated to a different PMU, without noticing anything about it.
> 
> What about Windows?

What is your question? Why should i limit Linux kernel design decisions based 
on any aspect of Windows? You might want to support it, but _please_ dont let 
the design be dictated by it ...

> >    In contrast, a 'stolen', 'raw' PMU directly programmed by the guest OS
> >    always assumes the guest OS is upgraded to the host. Also, 'raw' PMU state
> >    cannot be live-migrated. (save/restore doesnt help)
> 
> Why not?  So long as the source and destination are compatible?

'As long as it works' is certainly a good enough filter for quality ;-)

> >  - It's far cleaner on the host side as well: more granular, per event usage
> >    is possible. The guest can use portion of the PMU (managed by the host),
> >    and the host can use a portion too.
> >
> >    In contrast, a 'stolen', 'raw' PMU directly programmed by the guest OS
> >    precludes the host OS from running some different piece of instrumentation
> >    at the same time.
> 
> Right, time slicing is something we want.
> 
> >  - It's more secure: the host can have a finegrained policy about what kinds of
> >    events it exposes to the guest. It might chose to only expose software
> >    events for example.
> >
> >    In contrast, a 'stolen', 'raw' PMU directly programmed by the guest OS is
> >    an all-or-nothing policy affair: either you fully allow the guest (and live
> >    with whatever consequences the piece of hardware that takes up a fair chunk
> >    on the CPU die causes), or you allow none of it.
> 
> No, we can hide insecure events with a full pmu.  Trap the control register 
> and don't pass it on to the hardware.

So you basically concede partial emulation ...

> >  - A proper paravirt event driver gives more features as well: it can exposes
> >    host software events and tracepoints, probes - not restricting itself to
> >    the 'hardware PMU' abstraction.
> 
> But it is limited to whatever the host stack supports.  At least
> that's our control, but things like PEBS will take a ton of work.

PEBS support is being implemented for perf, as a transparent feature. So once 
it's available, PEBS support will magically improve the quality of guest OS 
samples, if a paravirt driver approach is used and if sys_perf_event_open() is 
taught about that driver. Without any other change needed on the guest side.

> >  - There's proper event scheduling and event allocation. Time-slicing, etc.
> >
> >
> > The thing is, we made quite similar arguments in the past, during the 
> > perfmon vs. perfcounters discussions. There's really a big advantage to 
> > proper abstractions, both on the host and on the guest side.
> 
> We only control half of the equation.  That's very different compared to 
> tools/perf.

You mean Windows?

For heaven's sake, why dont you think like Linus thought 20 years ago. To the 
hell with Windows suckiness and lets make sure our stuff works well. Then the 
users will come, developers will come, and people will profile Linux under 
Linux and maybe the tools will be so good that they'll profile under Linux 
using Wine just to be able to use those good tools...

If you gut Linux capabilities like that to accomodate for the suckiness of 
Windows, without giving a technological edge to Linux, and then we are bound 
to fail in the long run ...

	Ingo