Re: [patch] genapic: optimize & fix APIC mode setup

[Andi Kleen <ak@suse.de>]

> 
> Secondly, in the physical case, /all/ IPI sending goes through this 
> code:
> 
>         for_each_cpu_mask(query_cpu, mask) {
> 
> yes, even the single-IPI calls which give the overwhelming majority of 
> the use of IPIs. Even on systems that have only 2 CPUs to begin with. 
> This should be measurable.

I thought so too originally, but it wasn't. My original thinking
was that logical must be faster because it can in theory send less messages
on large systems.

On the two CPU case it is basically always the same anyways because
the loop is very cheap compared to the IPIs (IPIs tend to be thousands
of cycles). for_each_cpu_mask is essentially just BSF with some glue.

For the > 2 CPU case it is not that obvious -- in theory the hardware
could optimize it to be more efficient, but it doesn't seem to. Or at least
not in a good enough way to show significant differences.

> you are still not getting it i think. The IO-APIC is still in logical 
> delivery mode on small systems, 

The IO-APIC delivery mode that is configured comes from genapic, and should be 
physical for physflat unless I'm totally confused about the code.

> the right solution is to use pure physical mode (both local APIC and 
> IO-APIC) only on large systems, and to use pure logical mode on small 
> systems - maybe with the combination of clustered mode as well.

I disagree.  I think we should just use physical mode everywhere,
except on the old i386 systems.
 
> as i said it before, what you are suggesting is not a 'fix', it's a 
> workaround for a design flaw in the hotplug code which flaw is hitting 
> us in other places and architectures anyway, and which workaround makes 
> us use an inferior IRQ delivery method on small systems ...

It isn't inferior as far as I know.

-Andi