Re: One (possible) x86 get_user_pages bug

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 01/27/2011 06:49 AM, Jan Beulich wrote:
> However, going through all the comments in gup.c again I wonder
> whether pv Xen guests don't violate the major assumption: There
> is talk about interrupts being off preventing (or sufficiently
> deferring) remote CPUs doing TLB flushes. In pv Xen guests,
> however, non-local TLB flushes do not happen by sending IPIs -
> the hypercall interface gets used instead

Yes, I was aware of that synchronization mechanism, and I think I'd
convinced myself we were OK.  But I can't think was that reasoning was -
perhaps it was something as simple as "gupf isn't used under Xen" (which
may have been true at the time, but isn't now).

As clever as it is, the whole "disable interrupts -> hold off IPI ->
prevent TLB flush -> delay freeing" chain seems pretty fragile.  I guess
its OK if we assume that x86 will forever have IPI-based cross-cpu TLB
flushes, but one could imagine some kind of "remote tlb shootdown using
bus transaction" appearing in the architecture.

And even just considering virtualization, having non-IPI-based tlb
shootdown is a measurable performance win, since a hypervisor can
optimise away a cross-VCPU shootdown if it knows no physical TLB
contains the target VCPU's entries.  I can imagine the KVM folks could
get some benefit from that as well.

So is there some way we can preserve the current scheme's benefits while
making it a bit more general?  (If anyone else has non-IPI-based
shootdown, it would be s390; is there some inspiration there?  An
instruction perhaps?)

    J