Re: [patch 13/14] x86_64: Use common functions in cluster and physflat mode

[Andi Kleen <ak@muc.de>]

On Sun, Sep 11, 2005 at 09:44:16AM -0700, Zwane Mwaikambo wrote:
> On Fri, 9 Sep 2005, Ashok Raj wrote:
> 
> > In general we need
> > 
> > flat_mode - #cpus <= 8 (Hotplug defined or not, so we use mask version 
> >                        for safety)
> > 
> > physflat or cluster_mode when #cpus >8.
> 
> I agree there.
> 
> > If we choose physflat as default for #cpus <=8 (with hotplug) would make
> > IPI performance worse, since it would do one cpu at a time, and requires 2 
> > writes per cpu for each IPI v.s just 2 for a flat mode mask version of the API.
> 
> I don't see the benefit then :/ I certainly hope we don't go that route.

I originally made that objection, but Ashok then did some benchmarks
that showed essentially no difference. I can see the point - it's 
likely that an access to the local APIC (which is in the CPU) is fast
(we know it is) and all the time is dominated by sending the 
requests over the wires between the CPUs. So it shouldn't matter 
much if you use sequence mode or masks.

That is why I changed my mind and just made physflat default for the hotplug
case (which will be essentially everywhere because I expect most kernels
to have hotplug enabled in the future) 

It's a bit of a mess in mm right now because me and Ashok have been fixing 
similar problems in a different way (e.g. the patch in flat to
use the sequence sending path is also not needed anymore with that) 
Need to clean this up a bit.

Handling it properly for i386 is also still needed e.g. the older physflat
patch I did needs to be merged with bigsmp and cleaned up a bit.
But I don't have time to do it before monday so it'll miss the 2.6.14
window anyways.

-Andi