[PATCH 0/2] xen: events: a couple of minor cleanups

[PATCH 0/2] xen: events: a couple of minor cleanups

[Ian Campbell]

The following two patches:

Remove the "allocating IRQs backwards" workaround which was useful
before the core irq allocator infrastructure was available but isn't any
longer.

Stop panicking on irq allocation failure, it's not necessarily fatal to
the host to not be able to allocate an irq. I did a brief sweep of the
callers and they appear to mostly do something sane with the failure.

These are on top of Konrad's stable/irq.rework branch.

Ian.

[PATCH 1/2] xen: events: do not workaround too-small nr_irqs

[Ian Campbell]

This workaround was somewhat useful prior to the introduction of the
core irq allocator and 026c9d2d0d75 "xen: events: allocate GSIs and
dynamic IRQs from separate IRQ ranges." but should not be unnecessary now.

If nr_irqs turns out to be too small under Xen then we use (or define
if necessary) the interfaces to increase nr_irqs rather than working
around the core allocator in this way.

In my configuration NR_IRQS ends up being 2304 with nr_irq_gsi 272
which is sufficient.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
---
 drivers/xen/events.c |    7 -------
 1 files changed, 0 insertions(+), 7 deletions(-)

diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 975e90f..ce33061 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -393,15 +393,8 @@ static int xen_allocate_irq_dynamic(void)
 		first = get_nr_irqs_gsi();
 #endif
 
-retry:
 	irq = irq_alloc_desc_from(first, -1);
 
-	if (irq == -ENOMEM && first > NR_IRQS_LEGACY) {
-		printk(KERN_ERR "Out of dynamic IRQ space and eating into GSI space. You should increase nr_irqs\n");
-		first = max(NR_IRQS_LEGACY, first - NR_IRQS_LEGACY);
-		goto retry;
-	}
-
 	if (irq < 0)
 		panic("No available IRQ to bind to: increase nr_irqs!\n");
 
-- 
1.5.6.5

[PATCH 2/2] xen: events: propagate irq allocation failure instead of panicing

[Ian Campbell]

Running out of IRQs need not be fatal to the machine as a whole.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
---
 drivers/xen/events.c |   17 ++++++++++-------
 1 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index ce33061..51051cf 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -376,7 +376,7 @@ static void unmask_evtchn(int port)
 	put_cpu();
 }
 
-static int xen_allocate_irq_dynamic(void)
+static int __must_check xen_allocate_irq_dynamic(void)
 {
 	int first = 0;
 	int irq;
@@ -395,13 +395,10 @@ static int xen_allocate_irq_dynamic(void)
 
 	irq = irq_alloc_desc_from(first, -1);
 
-	if (irq < 0)
-		panic("No available IRQ to bind to: increase nr_irqs!\n");
-
 	return irq;
 }
 
-static int xen_allocate_irq_gsi(unsigned gsi)
+static int __must_check xen_allocate_irq_gsi(unsigned gsi)
 {
 	int irq;
 
@@ -419,8 +416,6 @@ static int xen_allocate_irq_gsi(unsigned gsi)
 		return gsi;
 
 	irq = irq_alloc_desc_at(gsi, -1);
-	if (irq < 0)
-		panic("Unable to allocate to IRQ%d (%d)\n", gsi, irq);
 
 	return irq;
 }
@@ -609,6 +604,8 @@ int xen_map_pirq_gsi(unsigned pirq, unsigned gsi, int shareable, char *name)
 	}
 
 	irq = xen_allocate_irq_gsi(gsi);
+	if (irq < 0)
+		goto out;
 
 	set_irq_chip_and_handler_name(irq, &xen_pirq_chip,
 				      handle_level_irq, name);
@@ -795,6 +792,8 @@ int bind_evtchn_to_irq(unsigned int evtchn)
 
 	if (irq == -1) {
 		irq = xen_allocate_irq_dynamic();
+		if (irq == -1)
+			goto out;
 
 		set_irq_chip_and_handler_name(irq, &xen_dynamic_chip,
 					      handle_fasteoi_irq, "event");
@@ -803,6 +802,7 @@ int bind_evtchn_to_irq(unsigned int evtchn)
 		irq_info[irq] = mk_evtchn_info(evtchn);
 	}
 
+out:
 	spin_unlock(&irq_mapping_update_lock);
 
 	return irq;
@@ -856,6 +856,8 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu)
 
 	if (irq == -1) {
 		irq = xen_allocate_irq_dynamic();
+		if (irq == -1)
+			goto out;
 
 		set_irq_chip_and_handler_name(irq, &xen_percpu_chip,
 					      handle_percpu_irq, "virq");
@@ -875,6 +877,7 @@ int bind_virq_to_irq(unsigned int virq, unsigned int cpu)
 		bind_evtchn_to_cpu(evtchn, cpu);
 	}
 
+out:
 	spin_unlock(&irq_mapping_update_lock);
 
 	return irq;
-- 
1.5.6.5

Re: [PATCH 1/2] xen: events: do not workaround too-small nr_irqs

[Konrad Rzeszutek Wilk]

On Wed, Feb 09, 2011 at 11:10:44AM +0000, Ian Campbell wrote:
> This workaround was somewhat useful prior to the introduction of the
> core irq allocator and 026c9d2d0d75 "xen: events: allocate GSIs and
> dynamic IRQs from separate IRQ ranges." but should not be unnecessary now.
> 
> If nr_irqs turns out to be too small under Xen then we use (or define
> if necessary) the interfaces to increase nr_irqs rather than working
> around the core allocator in this way.
> 
> In my configuration NR_IRQS ends up being 2304 with nr_irq_gsi 272
> which is sufficient.

Is this configuration where you run Xen + Linux under QEMU?
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: Jeremy Fitzhardinge <jeremy@goop.org>
> ---
>  drivers/xen/events.c |    7 -------
>  1 files changed, 0 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/xen/events.c b/drivers/xen/events.c
> index 975e90f..ce33061 100644
> --- a/drivers/xen/events.c
> +++ b/drivers/xen/events.c
> @@ -393,15 +393,8 @@ static int xen_allocate_irq_dynamic(void)
>  		first = get_nr_irqs_gsi();
>  #endif
>  
> -retry:
>  	irq = irq_alloc_desc_from(first, -1);
>  
> -	if (irq == -ENOMEM && first > NR_IRQS_LEGACY) {
> -		printk(KERN_ERR "Out of dynamic IRQ space and eating into GSI space. You should increase nr_irqs\n");
> -		first = max(NR_IRQS_LEGACY, first - NR_IRQS_LEGACY);
> -		goto retry;
> -	}
> -
>  	if (irq < 0)
>  		panic("No available IRQ to bind to: increase nr_irqs!\n");
>  
> -- 
> 1.5.6.5

Re: [PATCH 1/2] xen: events: do not workaround too-small nr_irqs

[Ian Campbell]

On Wed, 2011-02-09 at 16:21 +0000, Konrad Rzeszutek Wilk wrote: 
> On Wed, Feb 09, 2011 at 11:10:44AM +0000, Ian Campbell wrote:
> > This workaround was somewhat useful prior to the introduction of the
> > core irq allocator and 026c9d2d0d75 "xen: events: allocate GSIs and
> > dynamic IRQs from separate IRQ ranges." but should not be unnecessary now.
> > 
> > If nr_irqs turns out to be too small under Xen then we use (or define
> > if necessary) the interfaces to increase nr_irqs rather than working
> > around the core allocator in this way.
> > 
> > In my configuration NR_IRQS ends up being 2304 with nr_irq_gsi 272
> > which is sufficient.
> 
> Is this configuration where you run Xen + Linux under QEMU?

No, it was my physical test box.

If this change causes problems under qemu then we need to work with the
x86 guys to find a way to allow us to increase nr_irqs at start of day
when running under Xen (e.g. a generic interface to add headroom for
dynamic IRQs etc).

Ian.

> > 
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > Cc: Jeremy Fitzhardinge <jeremy@goop.org>
> > ---
> >  drivers/xen/events.c |    7 -------
> >  1 files changed, 0 insertions(+), 7 deletions(-)
> > 
> > diff --git a/drivers/xen/events.c b/drivers/xen/events.c
> > index 975e90f..ce33061 100644
> > --- a/drivers/xen/events.c
> > +++ b/drivers/xen/events.c
> > @@ -393,15 +393,8 @@ static int xen_allocate_irq_dynamic(void)
> >  		first = get_nr_irqs_gsi();
> >  #endif
> >  
> > -retry:
> >  	irq = irq_alloc_desc_from(first, -1);
> >  
> > -	if (irq == -ENOMEM && first > NR_IRQS_LEGACY) {
> > -		printk(KERN_ERR "Out of dynamic IRQ space and eating into GSI space. You should increase nr_irqs\n");
> > -		first = max(NR_IRQS_LEGACY, first - NR_IRQS_LEGACY);
> > -		goto retry;
> > -	}
> > -
> >  	if (irq < 0)
> >  		panic("No available IRQ to bind to: increase nr_irqs!\n");
> >  
> > -- 
> > 1.5.6.5

Re: [PATCH 1/2] xen: events: do not workaround too-small nr_irqs

[Ian Campbell]

On Wed, 2011-02-09 at 16:21 +0000, Konrad Rzeszutek Wilk wrote:
> On Wed, Feb 09, 2011 at 11:10:44AM +0000, Ian Campbell wrote:
> > This workaround was somewhat useful prior to the introduction of the
> > core irq allocator and 026c9d2d0d75 "xen: events: allocate GSIs and
> > dynamic IRQs from separate IRQ ranges." but should not be unnecessary now.
> > 
> > If nr_irqs turns out to be too small under Xen then we use (or define
> > if necessary) the interfaces to increase nr_irqs rather than working
> > around the core allocator in this way.
> > 
> > In my configuration NR_IRQS ends up being 2304 with nr_irq_gsi 272
> > which is sufficient.
> 
> Is this configuration where you run Xen + Linux under QEMU?

I've just tried these two patches under qemu, booting a 64 bit
xen-unstable hypervisor with a 32 bit domain 0 kernel, pxeboot using:
        qemu-system-x86_64 -m 256 -vnc 0.0.0.0:1 -k en-gb -serial stdio -boot nc \
        	-usb -usbdevice tablet -net nic,vlan=0,macaddr=00:16:3e:f7:c4:1d,model=e1000 \
        	-net tap,vlan=0,ifname=tapQEMU.0 -hda /dev/VG/debian-HVM-1 
        
There I appear to get:
        nr_irqs_gsi: 272
        NR_IRQS:2304 nr_irqs:256 16

which leads to a failure due to not being able to allocate a dynamic IRQ
(because 272 > 256!), I'll dig into this...

Ian.

Re: Re: [PATCH 1/2] xen: events: do not workaround too-small nr_irqs

[Ian Campbell]

On Tue, 2011-02-15 at 15:15 +0000, Ian Campbell wrote:
> On Wed, 2011-02-09 at 16:21 +0000, Konrad Rzeszutek Wilk wrote:
> > On Wed, Feb 09, 2011 at 11:10:44AM +0000, Ian Campbell wrote:
> > > This workaround was somewhat useful prior to the introduction of the
> > > core irq allocator and 026c9d2d0d75 "xen: events: allocate GSIs and
> > > dynamic IRQs from separate IRQ ranges." but should not be unnecessary now.
> > > 
> > > If nr_irqs turns out to be too small under Xen then we use (or define
> > > if necessary) the interfaces to increase nr_irqs rather than working
> > > around the core allocator in this way.
> > > 
> > > In my configuration NR_IRQS ends up being 2304 with nr_irq_gsi 272
> > > which is sufficient.
> > 
> > Is this configuration where you run Xen + Linux under QEMU?
> 
> I've just tried these two patches under qemu, booting a 64 bit
> xen-unstable hypervisor with a 32 bit domain 0 kernel, pxeboot using:
>         qemu-system-x86_64 -m 256 -vnc 0.0.0.0:1 -k en-gb -serial stdio -boot nc \
>         	-usb -usbdevice tablet -net nic,vlan=0,macaddr=00:16:3e:f7:c4:1d,model=e1000 \
>         	-net tap,vlan=0,ifname=tapQEMU.0 -hda /dev/VG/debian-HVM-1 
>         
> There I appear to get:
>         nr_irqs_gsi: 272
>         NR_IRQS:2304 nr_irqs:256 16
> 
> which leads to a failure due to not being able to allocate a dynamic IRQ
> (because 272 > 256!), I'll dig into this...

The solution is to increase nr_irqs, which is currently static after
boot even with sparseirqs.

Fortunately tglx intends to make this dynamically growable in the IRQ
core in the 2.6.39 time frame which nicely ties in with this patch.

Ian.

Re: Re: [PATCH 1/2] xen: events: do not workaround too-small nr_irqs

[Konrad Rzeszutek Wilk]

On Wed, Feb 16, 2011 at 04:14:17PM +0000, Ian Campbell wrote:
> On Tue, 2011-02-15 at 15:15 +0000, Ian Campbell wrote:
> > On Wed, 2011-02-09 at 16:21 +0000, Konrad Rzeszutek Wilk wrote:
> > > On Wed, Feb 09, 2011 at 11:10:44AM +0000, Ian Campbell wrote:
> > > > This workaround was somewhat useful prior to the introduction of the
> > > > core irq allocator and 026c9d2d0d75 "xen: events: allocate GSIs and
> > > > dynamic IRQs from separate IRQ ranges." but should not be unnecessary now.
> > > > 
> > > > If nr_irqs turns out to be too small under Xen then we use (or define
> > > > if necessary) the interfaces to increase nr_irqs rather than working
> > > > around the core allocator in this way.
> > > > 
> > > > In my configuration NR_IRQS ends up being 2304 with nr_irq_gsi 272
> > > > which is sufficient.
> > > 
> > > Is this configuration where you run Xen + Linux under QEMU?
> > 
> > I've just tried these two patches under qemu, booting a 64 bit
> > xen-unstable hypervisor with a 32 bit domain 0 kernel, pxeboot using:
> >         qemu-system-x86_64 -m 256 -vnc 0.0.0.0:1 -k en-gb -serial stdio -boot nc \
> >         	-usb -usbdevice tablet -net nic,vlan=0,macaddr=00:16:3e:f7:c4:1d,model=e1000 \
> >         	-net tap,vlan=0,ifname=tapQEMU.0 -hda /dev/VG/debian-HVM-1 
> >         
> > There I appear to get:
> >         nr_irqs_gsi: 272
> >         NR_IRQS:2304 nr_irqs:256 16
> > 
> > which leads to a failure due to not being able to allocate a dynamic IRQ
> > (because 272 > 256!), I'll dig into this...
> 
> The solution is to increase nr_irqs, which is currently static after
> boot even with sparseirqs.
> 
> Fortunately tglx intends to make this dynamically growable in the IRQ
> core in the 2.6.39 time frame which nicely ties in with this patch.

OK. Can you point me to tglx's git tree that brings these goodies in?

Re: Re: [PATCH 1/2] xen: events: do not workaround too-small nr_irqs

[Ian Campbell]

On Wed, 2011-02-16 at 16:55 +0000, Konrad Rzeszutek Wilk wrote:
> On Wed, Feb 16, 2011 at 04:14:17PM +0000, Ian Campbell wrote:
> > On Tue, 2011-02-15 at 15:15 +0000, Ian Campbell wrote:
> > Fortunately tglx intends to make this dynamically growable in the IRQ
> > core in the 2.6.39 time frame which nicely ties in with this patch.
> 
> OK. Can you point me to tglx's git tree that brings these goodies in?

I don't think it actually exists yet. I've asked him to point me to it
when it happens.

Ian.