Re: [tip:x86/irq] x86: apic: Fix mismerge, add arch_probe_nr_irqs() again

[ebiederm@xmission.com (Eric W. Biederman)]

Ian Campbell <ijc@hellion.org.uk> writes:

> On Fri, 2010-02-26 at 10:19 -0800, Yinghai Lu wrote:
>> 
>> 
>> for x86, with radix tree based irq_to_desc(),
>> removing arch_probe_nr_irqs is intentional. so we get more irq that
>> could be used. 
>
> This sounds interesting for the Xen dom0 use case where we can have
> essentially arbitrary numbers of interrupt sources.
>
> Is there a tree somewhere that I can be looking at?

Ingo's x86 tip tree I imagine.  I think this patch is slated for coming
in sometime this merge window so you should be able to see it
in 2.6.34-rc1.

This is a truly frustrating question.  At this point all of the
practical limitations are in the Xen code itself, and it has been that
way for the last couple of years. Looking at the x86_64 tree it looks
like the core work went in at the end of 2006 about 2.6.19.  Last I
looked the Xen dom0 kernel was stuck at the kernel just before the irq
scaling changes went in.  Every time I look the Xen code is doing
something stupid and unnecessary that makes scaling to large numbers
of irqs hard.

The changes YH is working on are the last couple of changes so that
we can seriously remove NR_IRQs and not have to pay a penalty of
a large static array on small machines when we have a kernel built
to support large machines and a large number of interrupts.

As of 2.6.33 the limitations in DomU support are:
- xen_evtchn_do_upcall starts with the irq number instead of
  the irq_desc, and happens to unnecessarily call into arch
  specific code.  
- Xen has an array irq_info[NR_IRQS] one of the last static arrays
  sized at NR_IRQs in the entire kernel.

One of the big reasons Xen dom0 irq support was not merged was because
merging it would effectively be a revert of the irq scaling work.

If you can fix the Xen code so it isn't dragging the rest of the
kernel down when it comes to large numbers of irqs more power to you.
For now I will continue to write little patches every once in a while
that bonk Xen domU on the head when something Xen domU is doing becomes a
bottleneck for the rest of the kernel.

Eric