Re: [Qemu-devel] Help on TLB Flush

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Mark Burton (mark.burton@greensocs.com) wrote:
> 
> > On 12 Feb 2015, at 16:01, Peter Maydell <peter.maydell@linaro.org> wrote:
> > 
> > On 12 February 2015 at 14:45, Alexander Graf <agraf@suse.de> wrote:
> >> 
> >>> On 12.02.2015, at 15:35, Mark Burton <mark.burton@greensocs.com> wrote:
> >>> We are proposing to implement this by signalling all other CPU???s
> >>> to exit (and requesting they flush before re-starting). In other
> >>> words, this would happen asynchronously.
> >> 
> >> For global flushes, give them a pointer payload along with the flush
> >> request and tell all cpus to increment it atomically. In your main
> >> thread, wait until *ptr == nKickedCpus.
> > 
> > I bet this will not be the only situation where you want to
> > do an "get all other CPUs to do $something and wait til they
> > have done so" kind of operation, so some lightweight but generic
> > infrastructure for doing that would not be a bad plan. (Similarly
> > "get all other CPUs to stop, then I can do $something and let
> > the others continue???.)
> 
> We tried this - we ended up in knots.
> We had 2 CPU???s trying to flush at about the same time, both waiting for the other.
> We had CPU???s trying to get the global mutex to finish what they were doing, while being told to flush, 
> We had CPU???s in the global mutex trying to do something that would cause a flush??? etc....
> We had spaghetti with extra Bolognese sauce???

This is the hard problem of multithreaded emulation.
You've always got to let CPUs get back to a point where you can
invalidate a mapping/page quickly.

Thus you've also got to be very careful about where any CPU might
get into a loop or take another lock that would stop another CPU
causing an invalidate.  Either that or you need a way of somehow
breaking locks or recovering from the situation.

> We eventually concluded, yes - in an infinite universe everything is possible, but if we could simply do this ???asynchronously??? then our lives would be a LOT easier.
> e.g.  - ask all CPU???s to ???exit and do something??? is easy -  wait for them to do that is a whole other problem???

Which is why you've got to bound how long it might take
those CPUs to get back to you, and optimise out cases where
it's not really needed later.

> Our question is - do we need this ???sync??? (before the flush), or can we actually allow CPU???s to flush themselves asynchronously???.

Always assume the worst.

Dave

> 
> Cheers
> 
> Mark.
> 
> 
> 
> > 
> > -- PMM
> 
> 
> 	 +44 (0)20 7100 3485 x 210
>  +33 (0)5 33 52 01 77x 210
> 
> 	+33 (0)603762104
> 	mark.burton
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK