[PATCH] i386 IOAPIC: de-fang IRQ compression

[PATCH] i386 IOAPIC: de-fang IRQ compression

[Len Brown]

    commit c434b7a6aedfe428ad17cd61b21b125a7b7a29ce
    (x86: avoid wasting IRQs for PCI devices)
    created a concept of "IRQ compression" on i386
    to conserve IRQ numbers on systems with many
    sparsely populated IO APICs.
    
    The same scheme was also added to x86_64,
    but later removed when x86_64 recieved an IRQ over-haul
    that made it unnecessary -- including per-CPU
    IRQ vectors that greatly increased the IRQ capacity
    on the machine.
    
    i386 has not received the analogous over-haul,
    and thus a previous attempt to delete IRQ compression
    from i386 was rejected on the theory that there may
    exist machines that actually need it.  The fact is
    that the author of IRQ compression patch was unable
    to confirm the actual existence of such a system.
    
    As a result, all i386 kernels with IOAPIC support
    pay the following:
    
    1. confusion
    
    IRQ compression re-names the traditional IOAPIC
    pin numbers (aka ACPI GSI's) into sequential IRQ #s:
    
    ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 20 (level, low) -> IRQ 16
    ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI 21 (level, low) -> IRQ 17
    ACPI: PCI Interrupt 0000:00:1c.2[C] -> GSI 22 (level, low) -> IRQ 18
    ACPI: PCI Interrupt 0000:00:1c.3[D] -> GSI 23 (level, low) -> IRQ 19
    ACPI: PCI Interrupt 0000:00:1c.4[A] -> GSI 20 (level, low) -> IRQ 16
    
    This makes /proc/interrupts look different
    depending on system configuration and device probe order.
    It is also different than the x86_64 kernel running
    on the exact same system.  As a result, programmers
    get confused when comparing systems.
    
    2. complexity
    
    The IRQ code in Linux is already overly complex,
    and IRQ compression makes it worse.  There have
    already been two bug workarounds related to IRQ
    compression -- the IRQ0 timer workaround and
    the VIA PCI IRQ workaround.
    
    3. size
    
    All i386 kernels with IOAPIC support contain an int[4096] --
    a 4 page array to contain the renamed IRQs.
    
    So while the irq compression code on i386 should really
    be deleted -- even before merging the x86_64 irq-overhaul,
    this patch simply disables it on all high volume systems
    to avoid problems #1 and #2 on most all i386 systems.
    
    A large system with pin numbers >=64 will still have compression
    to conserve limited IRQ numbers for sparse IOAPICS.  However,
    the vast majority of the planet, those with only pin numbers < 64
    will use an identity GSI -> IRQ mapping.
    
    Signed-off-by: Len Brown <len.brown@intel.com>

diff --git a/arch/x86/kernel/mpparse_32.c b/arch/x86/kernel/mpparse_32.c
index 7a05a7f..468d6ed 100644
--- a/arch/x86/kernel/mpparse_32.c
+++ b/arch/x86/kernel/mpparse_32.c
@@ -1041,13 +1041,14 @@ void __init mp_config_acpi_legacy_irqs (void)
 }
 
 #define MAX_GSI_NUM	4096
+#define IRQ_COMPRESSION_START	64
 
 int mp_register_gsi(u32 gsi, int triggering, int polarity)
 {
 	int ioapic = -1;
 	int ioapic_pin = 0;
 	int idx, bit = 0;
-	static int pci_irq = 16;
+	static int pci_irq = IRQ_COMPRESSION_START;
 	/*
 	 * Mapping between Global System Interrups, which
 	 * represent all possible interrupts, and IRQs
@@ -1086,12 +1087,16 @@ int mp_register_gsi(u32 gsi, int triggering, int polarity)
 	if ((1<<bit) & mp_ioapic_routing[ioapic].pin_programmed[idx]) {
 		Dprintk(KERN_DEBUG "Pin %d-%d already programmed\n",
 			mp_ioapic_routing[ioapic].apic_id, ioapic_pin);
-		return gsi_to_irq[gsi];
+		return (gsi < IRQ_COMPRESSION_START ? gsi : gsi_to_irq[gsi]);
 	}
 
 	mp_ioapic_routing[ioapic].pin_programmed[idx] |= (1<<bit);
 
-	if (triggering == ACPI_LEVEL_SENSITIVE) {
+	/*
+	 * For GSI >= 64, use IRQ compression
+	 */
+	if ((gsi >= IRQ_COMPRESSION_START)
+		&& (triggering == ACPI_LEVEL_SENSITIVE)) {
 		/*
 		 * For PCI devices assign IRQs in order, avoiding gaps
 		 * due to unused I/O APIC pins.

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Eric W. Biederman]

Len Brown <lenb@kernel.org> writes:

>     commit c434b7a6aedfe428ad17cd61b21b125a7b7a29ce
>     (x86: avoid wasting IRQs for PCI devices)
>     created a concept of "IRQ compression" on i386
>     to conserve IRQ numbers on systems with many
>     sparsely populated IO APICs.
>     
>     The same scheme was also added to x86_64,
>     but later removed when x86_64 recieved an IRQ over-haul
>     that made it unnecessary -- including per-CPU
>     IRQ vectors that greatly increased the IRQ capacity
>     on the machine.
>     
>     i386 has not received the analogous over-haul,
>     and thus a previous attempt to delete IRQ compression
>     from i386 was rejected on the theory that there may
>     exist machines that actually need it.  The fact is
>     that the author of IRQ compression patch was unable
>     to confirm the actual existence of such a system.
>     
>     As a result, all i386 kernels with IOAPIC support
>     pay the following:
>     
>     1. confusion
>     
>     IRQ compression re-names the traditional IOAPIC
>     pin numbers (aka ACPI GSI's) into sequential IRQ #s:
>     
>     ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 20 (level, low) -> IRQ 16
>     ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI 21 (level, low) -> IRQ 17
>     ACPI: PCI Interrupt 0000:00:1c.2[C] -> GSI 22 (level, low) -> IRQ 18
>     ACPI: PCI Interrupt 0000:00:1c.3[D] -> GSI 23 (level, low) -> IRQ 19
>     ACPI: PCI Interrupt 0000:00:1c.4[A] -> GSI 20 (level, low) -> IRQ 16
>     
>     This makes /proc/interrupts look different
>     depending on system configuration and device probe order.
>     It is also different than the x86_64 kernel running
>     on the exact same system.  As a result, programmers
>     get confused when comparing systems.
>     
>     2. complexity
>     
>     The IRQ code in Linux is already overly complex,
>     and IRQ compression makes it worse.  There have
>     already been two bug workarounds related to IRQ
>     compression -- the IRQ0 timer workaround and
>     the VIA PCI IRQ workaround.
>     
>     3. size
>     
>     All i386 kernels with IOAPIC support contain an int[4096] --
>     a 4 page array to contain the renamed IRQs.
>     
>     So while the irq compression code on i386 should really
>     be deleted -- even before merging the x86_64 irq-overhaul,
>     this patch simply disables it on all high volume systems
>     to avoid problems #1 and #2 on most all i386 systems.
>     
>     A large system with pin numbers >=64 will still have compression
>     to conserve limited IRQ numbers for sparse IOAPICS.  However,
>     the vast majority of the planet, those with only pin numbers < 64
>     will use an identity GSI -> IRQ mapping.
>     
>     Signed-off-by: Len Brown <len.brown@intel.com>

Looks reasonable.  As a further cleanup it might be worth yanking the
Via workaround because we simply can not hit it with your change of
disabling irq compression for the first 64 gsis.

I honestly don't understand the "(gsi == 0 && !timer_uses_ioapic_pin_0)"
test but I do know killing irq compression was safe and worked on
x86_64 so I don't expect any problems there.

Acked-by: "Eric W. Biederman" <ebiederm@xmission.com>




> diff --git a/arch/x86/kernel/mpparse_32.c b/arch/x86/kernel/mpparse_32.c
> index 7a05a7f..468d6ed 100644
> --- a/arch/x86/kernel/mpparse_32.c
> +++ b/arch/x86/kernel/mpparse_32.c
> @@ -1041,13 +1041,14 @@ void __init mp_config_acpi_legacy_irqs (void)
>  }
>  
>  #define MAX_GSI_NUM	4096
> +#define IRQ_COMPRESSION_START	64
>  
>  int mp_register_gsi(u32 gsi, int triggering, int polarity)
>  {
>  	int ioapic = -1;
>  	int ioapic_pin = 0;
>  	int idx, bit = 0;
> -	static int pci_irq = 16;
> +	static int pci_irq = IRQ_COMPRESSION_START;
>  	/*
>  	 * Mapping between Global System Interrups, which
>  	 * represent all possible interrupts, and IRQs
> @@ -1086,12 +1087,16 @@ int mp_register_gsi(u32 gsi, int triggering, int
> polarity)
>  	if ((1<<bit) & mp_ioapic_routing[ioapic].pin_programmed[idx]) {
>  		Dprintk(KERN_DEBUG "Pin %d-%d already programmed\n",
>  			mp_ioapic_routing[ioapic].apic_id, ioapic_pin);
> -		return gsi_to_irq[gsi];
> +		return (gsi < IRQ_COMPRESSION_START ? gsi : gsi_to_irq[gsi]);
>  	}
>  
>  	mp_ioapic_routing[ioapic].pin_programmed[idx] |= (1<<bit);
>  
> -	if (triggering == ACPI_LEVEL_SENSITIVE) {
> +	/*
> +	 * For GSI >= 64, use IRQ compression
> +	 */
> +	if ((gsi >= IRQ_COMPRESSION_START)
> +		&& (triggering == ACPI_LEVEL_SENSITIVE)) {
>  		/*
>  		 * For PCI devices assign IRQs in order, avoiding gaps
>  		 * due to unused I/O APIC pins.

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Ingo Molnar]

* Len Brown <lenb@kernel.org> wrote:

>     So while the irq compression code on i386 should really
>     be deleted -- even before merging the x86_64 irq-overhaul,
>     this patch simply disables it on all high volume systems
>     to avoid problems #1 and #2 on most all i386 systems.
>     
>     A large system with pin numbers >=64 will still have compression
>     to conserve limited IRQ numbers for sparse IOAPICS.  However,
>     the vast majority of the planet, those with only pin numbers < 64
>     will use an identity GSI -> IRQ mapping.
>     
>     Signed-off-by: Len Brown <len.brown@intel.com>

thanks for the patch and the extensive description. I've applied this to 
x86.git. Do you agree that this has no urgency for v2.6.24 and is thus 
v2.6.25 material?

would you be interested in doing a follow-up patch as well that just 
yanks all of the irq compression code? That would keep things nicely 
bisectable and testable - the second, larger patch would be a NOP in 
theory on most systems.

	Ingo

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Eric W. Biederman]

Ingo Molnar <mingo@elte.hu> writes:

> * Len Brown <lenb@kernel.org> wrote:
>
>>     So while the irq compression code on i386 should really
>>     be deleted -- even before merging the x86_64 irq-overhaul,
>>     this patch simply disables it on all high volume systems
>>     to avoid problems #1 and #2 on most all i386 systems.
>>     
>>     A large system with pin numbers >=64 will still have compression
>>     to conserve limited IRQ numbers for sparse IOAPICS.  However,
>>     the vast majority of the planet, those with only pin numbers < 64
>>     will use an identity GSI -> IRQ mapping.
>>     
>>     Signed-off-by: Len Brown <len.brown@intel.com>
>
> thanks for the patch and the extensive description. I've applied this to 
> x86.git. Do you agree that this has no urgency for v2.6.24 and is thus 
> v2.6.25 material?
>
> would you be interested in doing a follow-up patch as well that just 
> yanks all of the irq compression code? That would keep things nicely 
> bisectable and testable - the second, larger patch would be a NOP in 
> theory on most systems.

With respect to a follow on patch that removes irq compression.
We must raise NR_IRQs.

I have seen systems with > 256 GSIs (in the 4k ballpark) that
people occasionally boot 32bit kernels on.

However those systems seemed to use fewer then 200 or so of those irqs.
So as long as we are allocating vectors to irqs on demand (i.e. when
we find that the irq is hooked up or later) we should be ok.

Eric

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Christian Kujau]

On Tue, 4 Dec 2007, Ingo Molnar wrote:
> thanks for the patch and the extensive description. I've applied this to
> x86.git. Do you agree that this has no urgency for v2.6.24 and is thus
> v2.6.25 material?

Pardon my ignorance, but: aren't we in -rc already? I was under the 
assumption that during the -rc phase only (release-)critical bugfixes 
would be applied? So this one *must* be 2.6.25 material, right?

Thanks,
Christian.
-- 
BOFH excuse #54:

Evil dogs hypnotised the night shift

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Ingo Molnar]

* Christian Kujau <christian@g-house.de> wrote:

> On Tue, 4 Dec 2007, Ingo Molnar wrote:
>> thanks for the patch and the extensive description. I've applied this to
>> x86.git. Do you agree that this has no urgency for v2.6.24 and is thus
>> v2.6.25 material?
>
> Pardon my ignorance, but: aren't we in -rc already? I was under the 
> assumption that during the -rc phase only (release-)critical bugfixes 
> would be applied? So this one *must* be 2.6.25 material, right?

that was just a boilerplate question from me to make sure there's no 
misunderstanding about where it got queued and what the planned timeline 
is.

	Ingo

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Andrew Morton]

On Tue, 4 Dec 2007 13:26:49 +0100
Ingo Molnar <mingo@elte.hu> wrote:

> * Len Brown <lenb@kernel.org> wrote:
> 
> >     So while the irq compression code on i386 should really
> >     be deleted -- even before merging the x86_64 irq-overhaul,
> >     this patch simply disables it on all high volume systems
> >     to avoid problems #1 and #2 on most all i386 systems.
> >     
> >     A large system with pin numbers >=64 will still have compression
> >     to conserve limited IRQ numbers for sparse IOAPICS.  However,
> >     the vast majority of the planet, those with only pin numbers < 64
> >     will use an identity GSI -> IRQ mapping.
> >     
> >     Signed-off-by: Len Brown <len.brown@intel.com>
> 
> thanks for the patch and the extensive description. I've applied this to 
> x86.git.

Len applied it to his tree too.

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Ingo Molnar]

* Andrew Morton <akpm@linux-foundation.org> wrote:

> On Tue, 4 Dec 2007 13:26:49 +0100
> Ingo Molnar <mingo@elte.hu> wrote:
> 
> > * Len Brown <lenb@kernel.org> wrote:
> > 
> > >     So while the irq compression code on i386 should really
> > >     be deleted -- even before merging the x86_64 irq-overhaul,
> > >     this patch simply disables it on all high volume systems
> > >     to avoid problems #1 and #2 on most all i386 systems.
> > >     
> > >     A large system with pin numbers >=64 will still have compression
> > >     to conserve limited IRQ numbers for sparse IOAPICS.  However,
> > >     the vast majority of the planet, those with only pin numbers < 64
> > >     will use an identity GSI -> IRQ mapping.
> > >     
> > >     Signed-off-by: Len Brown <len.brown@intel.com>
> > 
> > thanks for the patch and the extensive description. I've applied this to 
> > x86.git.
> 
> Len applied it to his tree too.

Len, i think this belongs into x86.git a bit more (especially with the 
unification activities going on all around the tree) - do you agree? 
Andrew, i'd suggest to apply a reverted patch to between git.acpi and 
git.x86 until this gets sorted out.

	Ingo

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Len Brown]

On Wednesday 05 December 2007 04:48, Ingo Molnar wrote:
> 
> * Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> > On Tue, 4 Dec 2007 13:26:49 +0100
> > Ingo Molnar <mingo@elte.hu> wrote:
> > 
> > > * Len Brown <lenb@kernel.org> wrote:
> > > 
> > > >     So while the irq compression code on i386 should really
> > > >     be deleted -- even before merging the x86_64 irq-overhaul,
> > > >     this patch simply disables it on all high volume systems
> > > >     to avoid problems #1 and #2 on most all i386 systems.
> > > >     
> > > >     A large system with pin numbers >=64 will still have compression
> > > >     to conserve limited IRQ numbers for sparse IOAPICS.  However,
> > > >     the vast majority of the planet, those with only pin numbers < 64
> > > >     will use an identity GSI -> IRQ mapping.
> > > >     
> > > >     Signed-off-by: Len Brown <len.brown@intel.com>
> > > 
> > > thanks for the patch and the extensive description. I've applied this to 
> > > x86.git.
> > 
> > Len applied it to his tree too.
> 
> Len, i think this belongs into x86.git a bit more (especially with the 
> unification activities going on all around the tree) - do you agree? 
> Andrew, i'd suggest to apply a reverted patch to between git.acpi and 
> git.x86 until this gets sorted out.

Sure.
I'm re-basing my test branch right now and can exclude this one
since it is in x86.git. (and yes, I'm interested in unifying mpparse_*.c some day)

yes, your understanding is correct -- this is not urgent 2.6.24 material,
it is just a 'regular patch':-)

Re: making the VIA part into dead-code
I had avoided that originally because I was going to nominate
this patch for the highest check-in-comment length/code-change ratio
But once I went over 1 line I blew the budget;-)

Eric,
What do you suggest we do with NR_IRQS on i386 so that we
can delete the compression code entirely?

thanks,
-Len

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Eric W. Biederman]

Len Brown <lenb@kernel.org> writes:

> On Wednesday 05 December 2007 04:48, Ingo Molnar wrote:
>> 
>> * Andrew Morton <akpm@linux-foundation.org> wrote:
>> 
>> > On Tue, 4 Dec 2007 13:26:49 +0100
>> > Ingo Molnar <mingo@elte.hu> wrote:
>> > 
>> > > * Len Brown <lenb@kernel.org> wrote:
>> > > 
>> > > >     So while the irq compression code on i386 should really
>> > > >     be deleted -- even before merging the x86_64 irq-overhaul,
>> > > >     this patch simply disables it on all high volume systems
>> > > >     to avoid problems #1 and #2 on most all i386 systems.
>> > > >     
>> > > >     A large system with pin numbers >=64 will still have compression
>> > > >     to conserve limited IRQ numbers for sparse IOAPICS.  However,
>> > > >     the vast majority of the planet, those with only pin numbers < 64
>> > > >     will use an identity GSI -> IRQ mapping.
>> > > >     
>> > > >     Signed-off-by: Len Brown <len.brown@intel.com>
>> > > 
>> > > thanks for the patch and the extensive description. I've applied this to 
>> > > x86.git.
>> > 
>> > Len applied it to his tree too.
>> 
>> Len, i think this belongs into x86.git a bit more (especially with the 
>> unification activities going on all around the tree) - do you agree? 
>> Andrew, i'd suggest to apply a reverted patch to between git.acpi and 
>> git.x86 until this gets sorted out.
>
> Sure.
> I'm re-basing my test branch right now and can exclude this one
> since it is in x86.git. (and yes, I'm interested in unifying mpparse_*.c some
> day)
>
> yes, your understanding is correct -- this is not urgent 2.6.24 material,
> it is just a 'regular patch':-)
>
> Re: making the VIA part into dead-code
> I had avoided that originally because I was going to nominate
> this patch for the highest check-in-comment length/code-change ratio
> But once I went over 1 line I blew the budget;-)
>
> Eric,
> What do you suggest we do with NR_IRQS on i386 so that we
> can delete the compression code entirely?
>
> thanks,
> -Len

Attached below is my patch from last time I was looking at this
problem, it doesn't quite apply but it is gives a good idea of where
I think we should go.

For the current 64bit kernel we do have a point where if we run out
of vectors we just can't use the irqs.  Because that point is
about 192*nr_online_cpus() we never hit it in practice.

If we just rip out the vector irq compression logic (without
implementing the per cpu vector logic) we will theoretically
have systems that will not run out of vectors and so some of their
irqs (50?) won't work, but it is likely those systems will still boot.

Unless there are really nasty ioapic bugs on that don't show up on
the 64bit systems we should be able to move the  per cpu vector
logic onto the 32bit side as well now that we understand what needs
to happen.  I didn't want to do the port until we had stabalized
things on the 64bit side, which largely we seem to have managed.

So as long as we are going to move the per cpu vector logic eventually
to the 32bit kernels I don't think we will have a problem if we
increase NR_IRQs, then kill the compression, and then get around to
implementing the per cpu vector logic.


>From b91d576ba0f88e5416f8737c4f2dca3fb814f2fe Mon Sep 17 00:00:00 2001
From: Eric W. Biederman <ebiederm@xmission.com>
Date: Fri, 16 Feb 2007 02:18:28 -0700
Subject: [PATCH] i386 irq: Kill NR_IRQ_VECTORS and increase NR_IRQS

Due to hardware and software implementation limits the i386 kernel
can only use 224 or so different IRQs at one time.  However except
for actually having the irqs delivered there are no limits except
the arbitrary number NR_IRQS on how many irqs we can talk about
and deal with.

Frequently not all io_apics inputs are connected to interrupt traces
going to real devices, and since even when they are devices don't
 always use all of interrupt traces routed to them.  So it makes
sense to be able to talk about many more irq sources then we can
actually use.

So this patch consolidates NR_IRQS and NR_IRQ_VECTORS into just
NR_IRQS and raises NR_IRQS where appropriate above the number of
irqs we can use at one time.

Allowing the kernel to work on big machines without complicated
and error prone irq remapping logic.

Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
 arch/i386/kernel/acpi/boot.c                       |    4 ++--
 arch/i386/kernel/io_apic.c                         |    6 +++---
 include/asm-i386/mach-default/irq_vectors_limits.h |    8 +++-----
 include/asm-i386/mach-generic/irq_vectors_limits.h |    3 +--
 include/asm-i386/mach-summit/irq_vectors_limits.h  |    3 +--
 include/asm-i386/mach-visws/irq_vectors.h          |    1 -
 include/asm-i386/mach-voyager/irq_vectors.h        |    1 -
 7 files changed, 10 insertions(+), 16 deletions(-)

diff --git a/arch/i386/kernel/acpi/boot.c b/arch/i386/kernel/acpi/boot.c
index 9ea5b8e..859fb03 100644
--- a/arch/i386/kernel/acpi/boot.c
+++ b/arch/i386/kernel/acpi/boot.c
@@ -811,7 +811,7 @@ static int __init acpi_parse_madt_ioapic_entries(void)
 
 	count =
 	    acpi_table_parse_madt(ACPI_MADT_TYPE_INTERRUPT_OVERRIDE, acpi_parse_int_src_ovr,
-				  NR_IRQ_VECTORS);
+				  NR_IRQS);
 	if (count < 0) {
 		printk(KERN_ERR PREFIX
 		       "Error parsing interrupt source overrides entry\n");
@@ -831,7 +831,7 @@ static int __init acpi_parse_madt_ioapic_entries(void)
 
 	count =
 	    acpi_table_parse_madt(ACPI_MADT_TYPE_NMI_SOURCE, acpi_parse_nmi_src,
-				  NR_IRQ_VECTORS);
+				  NR_IRQS);
 	if (count < 0) {
 		printk(KERN_ERR PREFIX "Error parsing NMI SRC entry\n");
 		/* TBD: Cleanup to allow fallback to MPS */
diff --git a/arch/i386/kernel/io_apic.c b/arch/i386/kernel/io_apic.c
index 816a8aa..9578ca9 100644
--- a/arch/i386/kernel/io_apic.c
+++ b/arch/i386/kernel/io_apic.c
@@ -1223,14 +1223,14 @@ static inline int IO_APIC_irq_trigger(int irq)
 }
 
 /* irq_vectors is indexed by the sum of all RTEs in all I/O APICs. */
-static u8 irq_vector[NR_IRQ_VECTORS] __read_mostly = { FIRST_DEVICE_VECTOR , 0 };
+static u8 irq_vector[NR_IRQS] __read_mostly = { FIRST_DEVICE_VECTOR , 0 };
 
 static int __assign_irq_vector(int irq)
 {
 	static int current_vector = FIRST_DEVICE_VECTOR, current_offset = 0;
 	int vector, offset, i;
 
-	BUG_ON((unsigned)irq >= NR_IRQ_VECTORS);
+	BUG_ON((unsigned)irq >= NR_IRQS);
 
 	if (irq_vector[irq] > 0)
 		return irq_vector[irq];
@@ -1247,7 +1247,7 @@ next:
 		return -ENOSPC;
 	if (vector == SYSCALL_VECTOR)
 		goto next;
-	for (i = 0; i < NR_IRQ_VECTORS; i++)
+	for (i = 0; i < NR_IRQS; i++)
 		if (irq_vector[i] == vector)
 			goto next;
 
diff --git a/include/asm-i386/mach-default/irq_vectors_limits.h b/include/asm-i386/mach-default/irq_vectors_limits.h
index 7f161e7..ad4e05c 100644
--- a/include/asm-i386/mach-default/irq_vectors_limits.h
+++ b/include/asm-i386/mach-default/irq_vectors_limits.h
@@ -2,15 +2,13 @@
 #define _ASM_IRQ_VECTORS_LIMITS_H
 
 #ifdef CONFIG_X86_IO_APIC
-#define NR_IRQS 224
-# if (224 >= 32 * NR_CPUS)
-# define NR_IRQ_VECTORS NR_IRQS
+# if (200 >= 32 * NR_CPUS)
+#  define NR_IRQS 200
 # else
-# define NR_IRQ_VECTORS (32 * NR_CPUS)
+#  define NR_IRQS (32 * NR_CPUS)
 # endif
 #else
 #define NR_IRQS 16
-#define NR_IRQ_VECTORS NR_IRQS
 #endif
 
 #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
diff --git a/include/asm-i386/mach-generic/irq_vectors_limits.h b/include/asm-i386/mach-generic/irq_vectors_limits.h
index 890ce3f..f16f727 100644
--- a/include/asm-i386/mach-generic/irq_vectors_limits.h
+++ b/include/asm-i386/mach-generic/irq_vectors_limits.h
@@ -8,7 +8,6 @@
  * This value should be the same in both the generic and summit subarches.
  * Change one, change 'em both.
  */
-#define NR_IRQS	224
-#define NR_IRQ_VECTORS	1024
+#define NR_IRQS	1024
 
 #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
diff --git a/include/asm-i386/mach-summit/irq_vectors_limits.h b/include/asm-i386/mach-summit/irq_vectors_limits.h
index 890ce3f..f16f727 100644
--- a/include/asm-i386/mach-summit/irq_vectors_limits.h
+++ b/include/asm-i386/mach-summit/irq_vectors_limits.h
@@ -8,7 +8,6 @@
  * This value should be the same in both the generic and summit subarches.
  * Change one, change 'em both.
  */
-#define NR_IRQS	224
-#define NR_IRQ_VECTORS	1024
+#define NR_IRQS	1024
 
 #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
diff --git a/include/asm-i386/mach-visws/irq_vectors.h b/include/asm-i386/mach-visws/irq_vectors.h
index cb572d8..c32ebdf 100644
--- a/include/asm-i386/mach-visws/irq_vectors.h
+++ b/include/asm-i386/mach-visws/irq_vectors.h
@@ -51,7 +51,6 @@
  */
 #define NR_VECTORS 256
 #define NR_IRQS 224
-#define NR_IRQ_VECTORS NR_IRQS
 
 #define FPU_IRQ			13
 
diff --git a/include/asm-i386/mach-voyager/irq_vectors.h b/include/asm-i386/mach-voyager/irq_vectors.h
index 165421f..abaf5cf 100644
--- a/include/asm-i386/mach-voyager/irq_vectors.h
+++ b/include/asm-i386/mach-voyager/irq_vectors.h
@@ -57,7 +57,6 @@
 
 #define NR_VECTORS 256
 #define NR_IRQS 224
-#define NR_IRQ_VECTORS NR_IRQS
 
 #define FPU_IRQ				13
 
-- 
1.5.3.rc6.17.g1911

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Ingo Molnar]

* Eric W. Biederman <ebiederm@xmission.com> wrote:

> Attached below is my patch from last time I was looking at this 
> problem, it doesn't quite apply but it is gives a good idea of where I 
> think we should go.

merged it up to x86.git - see below. Alan, have no tried to build, let 
alone boot it. Does it look good to you?

	Ingo

----------------->
Subject: x83: i386 IOAPIC: de-fang IRQ compression
From: ebiederm@xmission.com (Eric W. Biederman)

Due to hardware and software implementation limits the i386 kernel
can only use 224 or so different IRQs at one time.  However except
for actually having the irqs delivered there are no limits except
the arbitrary number NR_IRQS on how many irqs we can talk about
and deal with.

Frequently not all io_apics inputs are connected to interrupt traces
going to real devices, and since even when they are devices don't
 always use all of interrupt traces routed to them.  So it makes
sense to be able to talk about many more irq sources then we can
actually use.

So this patch consolidates NR_IRQS and NR_IRQ_VECTORS into just
NR_IRQS and raises NR_IRQS where appropriate above the number of
irqs we can use at one time.

Allowing the kernel to work on big machines without complicated
and error prone irq remapping logic.

Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/acpi/boot.c                       |    4 ++--
 arch/x86/kernel/io_apic_32.c                      |    4 ++--
 include/asm-x86/mach-default/irq_vectors_limits.h |    8 +++-----
 include/asm-x86/mach-generic/irq_vectors_limits.h |    3 +--
 include/asm-x86/mach-summit/irq_vectors_limits.h  |    3 +--
 include/asm-x86/mach-visws/irq_vectors.h          |    1 -
 include/asm-x86/mach-voyager/irq_vectors.h        |    1 -
 7 files changed, 9 insertions(+), 15 deletions(-)

Index: linux-x86.q/arch/x86/kernel/acpi/boot.c
===================================================================
--- linux-x86.q.orig/arch/x86/kernel/acpi/boot.c
+++ linux-x86.q/arch/x86/kernel/acpi/boot.c
@@ -855,7 +855,7 @@ static int __init acpi_parse_madt_ioapic
 
 	count =
 	    acpi_table_parse_madt(ACPI_MADT_TYPE_INTERRUPT_OVERRIDE, acpi_parse_int_src_ovr,
-				  NR_IRQ_VECTORS);
+				  NR_IRQS);
 	if (count < 0) {
 		printk(KERN_ERR PREFIX
 		       "Error parsing interrupt source overrides entry\n");
@@ -875,7 +875,7 @@ static int __init acpi_parse_madt_ioapic
 
 	count =
 	    acpi_table_parse_madt(ACPI_MADT_TYPE_NMI_SOURCE, acpi_parse_nmi_src,
-				  NR_IRQ_VECTORS);
+				  NR_IRQS);
 	if (count < 0) {
 		printk(KERN_ERR PREFIX "Error parsing NMI SRC entry\n");
 		/* TBD: Cleanup to allow fallback to MPS */
Index: linux-x86.q/arch/x86/kernel/io_apic_32.c
===================================================================
--- linux-x86.q.orig/arch/x86/kernel/io_apic_32.c
+++ linux-x86.q/arch/x86/kernel/io_apic_32.c
@@ -1191,14 +1191,14 @@ static inline int IO_APIC_irq_trigger(in
 }
 
 /* irq_vectors is indexed by the sum of all RTEs in all I/O APICs. */
-static u8 irq_vector[NR_IRQ_VECTORS] __read_mostly = { FIRST_DEVICE_VECTOR , 0 };
+static u8 irq_vector[NR_IRQS] __read_mostly = { FIRST_DEVICE_VECTOR , 0 };
 
 static int __assign_irq_vector(int irq)
 {
 	static int current_vector = FIRST_DEVICE_VECTOR, current_offset = 0;
 	int vector, offset;
 
-	BUG_ON((unsigned)irq >= NR_IRQ_VECTORS);
+	BUG_ON((unsigned)irq >= NR_IRQS);
 
 	if (irq_vector[irq] > 0)
 		return irq_vector[irq];
Index: linux-x86.q/include/asm-x86/mach-default/irq_vectors_limits.h
===================================================================
--- linux-x86.q.orig/include/asm-x86/mach-default/irq_vectors_limits.h
+++ linux-x86.q/include/asm-x86/mach-default/irq_vectors_limits.h
@@ -2,15 +2,13 @@
 #define _ASM_IRQ_VECTORS_LIMITS_H
 
 #if defined(CONFIG_X86_IO_APIC) || defined(CONFIG_PARAVIRT)
-#define NR_IRQS 224
-# if (224 >= 32 * NR_CPUS)
-# define NR_IRQ_VECTORS NR_IRQS
+# if (200 >= 32 * NR_CPUS)
+#  define NR_IRQS 200
 # else
-# define NR_IRQ_VECTORS (32 * NR_CPUS)
+#  define NR_IRQS (32 * NR_CPUS)
 # endif
 #else
 #define NR_IRQS 16
-#define NR_IRQ_VECTORS NR_IRQS
 #endif
 
 #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
Index: linux-x86.q/include/asm-x86/mach-generic/irq_vectors_limits.h
===================================================================
--- linux-x86.q.orig/include/asm-x86/mach-generic/irq_vectors_limits.h
+++ linux-x86.q/include/asm-x86/mach-generic/irq_vectors_limits.h
@@ -8,7 +8,6 @@
  * This value should be the same in both the generic and summit subarches.
  * Change one, change 'em both.
  */
-#define NR_IRQS	224
-#define NR_IRQ_VECTORS	1024
+#define NR_IRQS	1024
 
 #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
Index: linux-x86.q/include/asm-x86/mach-summit/irq_vectors_limits.h
===================================================================
--- linux-x86.q.orig/include/asm-x86/mach-summit/irq_vectors_limits.h
+++ linux-x86.q/include/asm-x86/mach-summit/irq_vectors_limits.h
@@ -8,7 +8,6 @@
  * This value should be the same in both the generic and summit subarches.
  * Change one, change 'em both.
  */
-#define NR_IRQS	224
-#define NR_IRQ_VECTORS	1024
+#define NR_IRQS	1024
 
 #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
Index: linux-x86.q/include/asm-x86/mach-visws/irq_vectors.h
===================================================================
--- linux-x86.q.orig/include/asm-x86/mach-visws/irq_vectors.h
+++ linux-x86.q/include/asm-x86/mach-visws/irq_vectors.h
@@ -51,7 +51,6 @@
  */
 #define NR_VECTORS 256
 #define NR_IRQS 224
-#define NR_IRQ_VECTORS NR_IRQS
 
 #define FPU_IRQ			13
 
Index: linux-x86.q/include/asm-x86/mach-voyager/irq_vectors.h
===================================================================
--- linux-x86.q.orig/include/asm-x86/mach-voyager/irq_vectors.h
+++ linux-x86.q/include/asm-x86/mach-voyager/irq_vectors.h
@@ -57,7 +57,6 @@
 
 #define NR_VECTORS 256
 #define NR_IRQS 224
-#define NR_IRQ_VECTORS NR_IRQS
 
 #define FPU_IRQ				13
 

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Ingo Molnar]

* Ingo Molnar <mingo@elte.hu> wrote:

> 
> * Eric W. Biederman <ebiederm@xmission.com> wrote:
> 
> > Attached below is my patch from last time I was looking at this 
> > problem, it doesn't quite apply but it is gives a good idea of where I 
> > think we should go.
> 
> merged it up to x86.git - see below. Alan, have no tried to build, let 
> alone boot it. Does it look good to you?

s/Alan/Alas

	Ingo

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Eric W. Biederman]

Ingo Molnar <mingo@elte.hu> writes:

> * Eric W. Biederman <ebiederm@xmission.com> wrote:
>
>> Attached below is my patch from last time I was looking at this 
>> problem, it doesn't quite apply but it is gives a good idea of where I 
>> think we should go.
>
> merged it up to x86.git - see below. Alas, have no tried to build, let 
> alone boot it. Does it look good to you?

Well the forward port looks good.  Assuming nothing has changed in
this area since February we should be good.

We have one now unnecessary definition of IRQ_VECTORS in irqs-64.h

Nothing actually changes with the patch below until we actually
disable the irq compression.


> ----------------->
> Subject: x83: i386 IOAPIC: de-fang IRQ compression
> From: ebiederm@xmission.com (Eric W. Biederman)
>
> Due to hardware and software implementation limits the i386 kernel
> can only use 224 or so different IRQs at one time.  However except
> for actually having the irqs delivered there are no limits except
> the arbitrary number NR_IRQS on how many irqs we can talk about
> and deal with.
>
> Frequently not all io_apics inputs are connected to interrupt traces
> going to real devices, and since even when they are devices don't
>  always use all of interrupt traces routed to them.  So it makes
> sense to be able to talk about many more irq sources then we can
> actually use.
>
> So this patch consolidates NR_IRQS and NR_IRQ_VECTORS into just
> NR_IRQS and raises NR_IRQS where appropriate above the number of
> irqs we can use at one time.
>
> Allowing the kernel to work on big machines without complicated
> and error prone irq remapping logic.
>
> Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
> Signed-off-by: Ingo Molnar <mingo@elte.hu>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  arch/x86/kernel/acpi/boot.c                       |    4 ++--
>  arch/x86/kernel/io_apic_32.c                      |    4 ++--
>  include/asm-x86/mach-default/irq_vectors_limits.h |    8 +++-----
>  include/asm-x86/mach-generic/irq_vectors_limits.h |    3 +--
>  include/asm-x86/mach-summit/irq_vectors_limits.h  |    3 +--
>  include/asm-x86/mach-visws/irq_vectors.h          |    1 -
>  include/asm-x86/mach-voyager/irq_vectors.h        |    1 -
>  7 files changed, 9 insertions(+), 15 deletions(-)
>
> Index: linux-x86.q/arch/x86/kernel/acpi/boot.c
> ===================================================================
> --- linux-x86.q.orig/arch/x86/kernel/acpi/boot.c
> +++ linux-x86.q/arch/x86/kernel/acpi/boot.c
> @@ -855,7 +855,7 @@ static int __init acpi_parse_madt_ioapic
>  
>  	count =
>  	    acpi_table_parse_madt(ACPI_MADT_TYPE_INTERRUPT_OVERRIDE,
> acpi_parse_int_src_ovr,
> -				  NR_IRQ_VECTORS);
> +				  NR_IRQS);
>  	if (count < 0) {
>  		printk(KERN_ERR PREFIX
>  		       "Error parsing interrupt source overrides entry\n");
> @@ -875,7 +875,7 @@ static int __init acpi_parse_madt_ioapic
>  
>  	count =
>  	    acpi_table_parse_madt(ACPI_MADT_TYPE_NMI_SOURCE, acpi_parse_nmi_src,
> -				  NR_IRQ_VECTORS);
> +				  NR_IRQS);
>  	if (count < 0) {
>  		printk(KERN_ERR PREFIX "Error parsing NMI SRC entry\n");
>  		/* TBD: Cleanup to allow fallback to MPS */
> Index: linux-x86.q/arch/x86/kernel/io_apic_32.c
> ===================================================================
> --- linux-x86.q.orig/arch/x86/kernel/io_apic_32.c
> +++ linux-x86.q/arch/x86/kernel/io_apic_32.c
> @@ -1191,14 +1191,14 @@ static inline int IO_APIC_irq_trigger(in
>  }
>  
>  /* irq_vectors is indexed by the sum of all RTEs in all I/O APICs. */
> -static u8 irq_vector[NR_IRQ_VECTORS] __read_mostly = { FIRST_DEVICE_VECTOR , 0
> };
> +static u8 irq_vector[NR_IRQS] __read_mostly = { FIRST_DEVICE_VECTOR , 0 };
>  
>  static int __assign_irq_vector(int irq)
>  {
>  	static int current_vector = FIRST_DEVICE_VECTOR, current_offset = 0;
>  	int vector, offset;
>  
> -	BUG_ON((unsigned)irq >= NR_IRQ_VECTORS);
> +	BUG_ON((unsigned)irq >= NR_IRQS);
>  
>  	if (irq_vector[irq] > 0)
>  		return irq_vector[irq];
> Index: linux-x86.q/include/asm-x86/mach-default/irq_vectors_limits.h
> ===================================================================
> --- linux-x86.q.orig/include/asm-x86/mach-default/irq_vectors_limits.h
> +++ linux-x86.q/include/asm-x86/mach-default/irq_vectors_limits.h
> @@ -2,15 +2,13 @@
>  #define _ASM_IRQ_VECTORS_LIMITS_H
>  
>  #if defined(CONFIG_X86_IO_APIC) || defined(CONFIG_PARAVIRT)
> -#define NR_IRQS 224
> -# if (224 >= 32 * NR_CPUS)
> -# define NR_IRQ_VECTORS NR_IRQS
> +# if (200 >= 32 * NR_CPUS)
> +#  define NR_IRQS 200
>  # else
> -# define NR_IRQ_VECTORS (32 * NR_CPUS)
> +#  define NR_IRQS (32 * NR_CPUS)
>  # endif
>  #else
>  #define NR_IRQS 16
> -#define NR_IRQ_VECTORS NR_IRQS
>  #endif
>  
>  #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
> Index: linux-x86.q/include/asm-x86/mach-generic/irq_vectors_limits.h
> ===================================================================
> --- linux-x86.q.orig/include/asm-x86/mach-generic/irq_vectors_limits.h
> +++ linux-x86.q/include/asm-x86/mach-generic/irq_vectors_limits.h
> @@ -8,7 +8,6 @@
>   * This value should be the same in both the generic and summit subarches.
>   * Change one, change 'em both.
>   */
> -#define NR_IRQS	224
> -#define NR_IRQ_VECTORS	1024
> +#define NR_IRQS	1024
>  
>  #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
> Index: linux-x86.q/include/asm-x86/mach-summit/irq_vectors_limits.h
> ===================================================================
> --- linux-x86.q.orig/include/asm-x86/mach-summit/irq_vectors_limits.h
> +++ linux-x86.q/include/asm-x86/mach-summit/irq_vectors_limits.h
> @@ -8,7 +8,6 @@
>   * This value should be the same in both the generic and summit subarches.
>   * Change one, change 'em both.
>   */
> -#define NR_IRQS	224
> -#define NR_IRQ_VECTORS	1024
> +#define NR_IRQS	1024
>  
>  #endif /* _ASM_IRQ_VECTORS_LIMITS_H */
> Index: linux-x86.q/include/asm-x86/mach-visws/irq_vectors.h
> ===================================================================
> --- linux-x86.q.orig/include/asm-x86/mach-visws/irq_vectors.h
> +++ linux-x86.q/include/asm-x86/mach-visws/irq_vectors.h
> @@ -51,7 +51,6 @@
>   */
>  #define NR_VECTORS 256
>  #define NR_IRQS 224
> -#define NR_IRQ_VECTORS NR_IRQS
>  
>  #define FPU_IRQ			13
>  
> Index: linux-x86.q/include/asm-x86/mach-voyager/irq_vectors.h
> ===================================================================
> --- linux-x86.q.orig/include/asm-x86/mach-voyager/irq_vectors.h
> +++ linux-x86.q/include/asm-x86/mach-voyager/irq_vectors.h
> @@ -57,7 +57,6 @@
>  
>  #define NR_VECTORS 256
>  #define NR_IRQS 224
> -#define NR_IRQ_VECTORS NR_IRQS
>  
>  #define FPU_IRQ				13
>  

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Natalie Protasevich]

On Nov 27, 2007 10:21 PM, Len Brown <lenb@kernel.org> wrote:
>     commit c434b7a6aedfe428ad17cd61b21b125a7b7a29ce
>     (x86: avoid wasting IRQs for PCI devices)
>     created a concept of "IRQ compression" on i386
>     to conserve IRQ numbers on systems with many
>     sparsely populated IO APICs.
>
>     The same scheme was also added to x86_64,
>     but later removed when x86_64 recieved an IRQ over-haul
>     that made it unnecessary -- including per-CPU
>     IRQ vectors that greatly increased the IRQ capacity
>     on the machine.
>
>     i386 has not received the analogous over-haul,
>     and thus a previous attempt to delete IRQ compression
>     from i386 was rejected on the theory that there may
>     exist machines that actually need it.  The fact is
>     that the author of IRQ compression patch was unable
>     to confirm the actual existence of such a system.

Those systems did exist (and still exist actually). They used over 200
irqs sometimes and with "normal" IRQ allocation they were failing even
before reaching half of their I/O configuration. So simple removal
wouldn't work for those, dynamic allocation sure would. They "scrolled
off the topic" though because new generations of such machines are not
32 bit anymore. So the author didn't actually object :) it was the
other users of large 32 bit platforms that did.

However, this patch is great, it should've been in there from the
start. However, as of today, I am not aware of systems that would
still be interested in large 32 bit configuration so wouldn't object
totall removal.

(As for irq0 workarounds - nothing special about those. They were
dictated by the code flow and the practical cases, and the fact that
is was "quick" workaround over pretty inflexible piece of code at the
time. They were truly safe no-ops for all "normal" boxes...)

>
>     As a result, all i386 kernels with IOAPIC support
>     pay the following:
>
>     1. confusion
>
>     IRQ compression re-names the traditional IOAPIC
>     pin numbers (aka ACPI GSI's) into sequential IRQ #s:
>
>     ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 20 (level, low) -> IRQ 16
>     ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI 21 (level, low) -> IRQ 17
>     ACPI: PCI Interrupt 0000:00:1c.2[C] -> GSI 22 (level, low) -> IRQ 18
>     ACPI: PCI Interrupt 0000:00:1c.3[D] -> GSI 23 (level, low) -> IRQ 19
>     ACPI: PCI Interrupt 0000:00:1c.4[A] -> GSI 20 (level, low) -> IRQ 16
>
>     This makes /proc/interrupts look different
>     depending on system configuration and device probe order.
>     It is also different than the x86_64 kernel running
>     on the exact same system.  As a result, programmers
>     get confused when comparing systems.
>
>     2. complexity
>
>     The IRQ code in Linux is already overly complex,
>     and IRQ compression makes it worse.  There have
>     already been two bug workarounds related to IRQ
>     compression -- the IRQ0 timer workaround and
>     the VIA PCI IRQ workaround.
>
>     3. size
>
>     All i386 kernels with IOAPIC support contain an int[4096] --
>     a 4 page array to contain the renamed IRQs.
>
>     So while the irq compression code on i386 should really
>     be deleted -- even before merging the x86_64 irq-overhaul,
>     this patch simply disables it on all high volume systems
>     to avoid problems #1 and #2 on most all i386 systems.
>
>     A large system with pin numbers >=64 will still have compression
>     to conserve limited IRQ numbers for sparse IOAPICS.  However,
>     the vast majority of the planet, those with only pin numbers < 64
>     will use an identity GSI -> IRQ mapping.
>
>     Signed-off-by: Len Brown <len.brown@intel.com>
>
> diff --git a/arch/x86/kernel/mpparse_32.c b/arch/x86/kernel/mpparse_32.c
> index 7a05a7f..468d6ed 100644
> --- a/arch/x86/kernel/mpparse_32.c
> +++ b/arch/x86/kernel/mpparse_32.c
> @@ -1041,13 +1041,14 @@ void __init mp_config_acpi_legacy_irqs (void)
>  }
>
>  #define MAX_GSI_NUM    4096
> +#define IRQ_COMPRESSION_START  64
>
>  int mp_register_gsi(u32 gsi, int triggering, int polarity)
>  {
>         int ioapic = -1;
>         int ioapic_pin = 0;
>         int idx, bit = 0;
> -       static int pci_irq = 16;
> +       static int pci_irq = IRQ_COMPRESSION_START;
>         /*
>          * Mapping between Global System Interrups, which
>          * represent all possible interrupts, and IRQs
> @@ -1086,12 +1087,16 @@ int mp_register_gsi(u32 gsi, int triggering, int polarity)
>         if ((1<<bit) & mp_ioapic_routing[ioapic].pin_programmed[idx]) {
>                 Dprintk(KERN_DEBUG "Pin %d-%d already programmed\n",
>                         mp_ioapic_routing[ioapic].apic_id, ioapic_pin);
> -               return gsi_to_irq[gsi];
> +               return (gsi < IRQ_COMPRESSION_START ? gsi : gsi_to_irq[gsi]);
>         }
>
>         mp_ioapic_routing[ioapic].pin_programmed[idx] |= (1<<bit);
>
> -       if (triggering == ACPI_LEVEL_SENSITIVE) {
> +       /*
> +        * For GSI >= 64, use IRQ compression
> +        */
> +       if ((gsi >= IRQ_COMPRESSION_START)
> +               && (triggering == ACPI_LEVEL_SENSITIVE)) {
>                 /*
>                  * For PCI devices assign IRQs in order, avoiding gaps
>                  * due to unused I/O APIC pins.
>

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Eric W. Biederman]

"Natalie Protasevich" <protasnb@gmail.com> writes:

> On Nov 27, 2007 10:21 PM, Len Brown <lenb@kernel.org> wrote:
>>     commit c434b7a6aedfe428ad17cd61b21b125a7b7a29ce
>>     (x86: avoid wasting IRQs for PCI devices)
>>     created a concept of "IRQ compression" on i386
>>     to conserve IRQ numbers on systems with many
>>     sparsely populated IO APICs.
>>
>>     The same scheme was also added to x86_64,
>>     but later removed when x86_64 recieved an IRQ over-haul
>>     that made it unnecessary -- including per-CPU
>>     IRQ vectors that greatly increased the IRQ capacity
>>     on the machine.
>>
>>     i386 has not received the analogous over-haul,
>>     and thus a previous attempt to delete IRQ compression
>>     from i386 was rejected on the theory that there may
>>     exist machines that actually need it.  The fact is
>>     that the author of IRQ compression patch was unable
>>     to confirm the actual existence of such a system.
>
> Those systems did exist (and still exist actually). They used over 200
> irqs sometimes and with "normal" IRQ allocation they were failing even
> before reaching half of their I/O configuration. So simple removal
> wouldn't work for those, dynamic allocation sure would. They "scrolled
> off the topic" though because new generations of such machines are not
> 32 bit anymore. So the author didn't actually object :) it was the
> other users of large 32 bit platforms that did.

Natalie.  Did they just have over 200 irqs/gsis or did they actually
use over 200 irqs?

In particular is a large NR_IRQS plus dynamic vector allocation
sufficient for all cases you know about?

Eric

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Natalie Protasevich]

On Dec 5, 2007 3:25 PM, Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> "Natalie Protasevich" <protasnb@gmail.com> writes:
>
> > On Nov 27, 2007 10:21 PM, Len Brown <lenb@kernel.org> wrote:
> >>     commit c434b7a6aedfe428ad17cd61b21b125a7b7a29ce
> >>     (x86: avoid wasting IRQs for PCI devices)
> >>     created a concept of "IRQ compression" on i386
> >>     to conserve IRQ numbers on systems with many
> >>     sparsely populated IO APICs.
> >>
> >>     The same scheme was also added to x86_64,
> >>     but later removed when x86_64 recieved an IRQ over-haul
> >>     that made it unnecessary -- including per-CPU
> >>     IRQ vectors that greatly increased the IRQ capacity
> >>     on the machine.
> >>
> >>     i386 has not received the analogous over-haul,
> >>     and thus a previous attempt to delete IRQ compression
> >>     from i386 was rejected on the theory that there may
> >>     exist machines that actually need it.  The fact is
> >>     that the author of IRQ compression patch was unable
> >>     to confirm the actual existence of such a system.
> >
> > Those systems did exist (and still exist actually). They used over 200
> > irqs sometimes and with "normal" IRQ allocation they were failing even
> > before reaching half of their I/O configuration. So simple removal
> > wouldn't work for those, dynamic allocation sure would. They "scrolled
> > off the topic" though because new generations of such machines are not
> > 32 bit anymore. So the author didn't actually object :) it was the
> > other users of large 32 bit platforms that did.
>
> Natalie.  Did they just have over 200 irqs/gsis or did they actually
> use over 200 irqs?
>

I think we counted them in the order of 1400 external IRQs (actual
ioapics/slots plus possible on-card bridges), and yes numbers for used
IRQs were close to 250. Actual customer configurations could've big
bigger, I don't have such data.

> In particular is a large NR_IRQS plus dynamic vector allocation
> sufficient for all cases you know about?

Yes, since x86_64 boxes never had a problem once dynamic vectors were
incorporated.

>
> Eric
>

Re: [PATCH] i386 IOAPIC: de-fang IRQ compression

[Eric W. Biederman]

"Natalie Protasevich" <protasnb@gmail.com> writes:

> I think we counted them in the order of 1400 external IRQs (actual
> ioapics/slots plus possible on-card bridges), and yes numbers for used
> IRQs were close to 250. Actual customer configurations could've big
> bigger, I don't have such data.
>
>> In particular is a large NR_IRQS plus dynamic vector allocation
>> sufficient for all cases you know about?
>
> Yes, since x86_64 boxes never had a problem once dynamic vectors were
> incorporated.

I was wondering if we could avoid making the vectors per cpu and still be
in good shape on x86_32.  From your description it looks like we can't
quite support everything on x86_32 if we don't do the per cpu vector
thing.  However we will likely have everything interesting supported.

Eric