Interrupt controller updates

Interrupt controller updates

[Benjamin Herrenschmidt]

Hi Jan !

David (CC) want to make some progress with our in-kernel PIC. From
memory, one of the outcomes of the BOF was that we need to move the
existing "enable in-kernel PIC" from generic KVM init to machine init in
order to be able to add an argument indicating the "model" use by the
arch/platform since some like ours support several different models and
since that all needs to be selected before the VCPUs are created.

Again, from memory, you were volunteered to do the initial x86 change so
we could piggy back on it :-) Or do I remember wrong ?

Cheers,
Ben.

[Qemu-devel] Interrupt controller updates

[Benjamin Herrenschmidt]

Hi Jan !

David (CC) want to make some progress with our in-kernel PIC. From
memory, one of the outcomes of the BOF was that we need to move the
existing "enable in-kernel PIC" from generic KVM init to machine init in
order to be able to add an argument indicating the "model" use by the
arch/platform since some like ours support several different models and
since that all needs to be selected before the VCPUs are created.

Again, from memory, you were volunteered to do the initial x86 change so
we could piggy back on it :-) Or do I remember wrong ?

Cheers,
Ben.

Re: Interrupt controller updates

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

On 2012-11-21 02:07, Benjamin Herrenschmidt wrote:
> Hi Jan !
> 
> David (CC) want to make some progress with our in-kernel PIC. From
> memory, one of the outcomes of the BOF was that we need to move the
> existing "enable in-kernel PIC" from generic KVM init to machine init in
> order to be able to add an argument indicating the "model" use by the
> arch/platform since some like ours support several different models and
> since that all needs to be selected before the VCPUs are created.
> 
> Again, from memory, you were volunteered to do the initial x86 change so
> we could piggy back on it :-) Or do I remember wrong ?

I guess I wasn't in the room at that time ;).

I don't have much bandwidth the very next weeks to work on this, so I
guess it's better you come up with a patch proposal.

Jan



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: [Qemu-devel] Interrupt controller updates

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

On 2012-11-21 02:07, Benjamin Herrenschmidt wrote:
> Hi Jan !
> 
> David (CC) want to make some progress with our in-kernel PIC. From
> memory, one of the outcomes of the BOF was that we need to move the
> existing "enable in-kernel PIC" from generic KVM init to machine init in
> order to be able to add an argument indicating the "model" use by the
> arch/platform since some like ours support several different models and
> since that all needs to be selected before the VCPUs are created.
> 
> Again, from memory, you were volunteered to do the initial x86 change so
> we could piggy back on it :-) Or do I remember wrong ?

I guess I wasn't in the room at that time ;).

I don't have much bandwidth the very next weeks to work on this, so I
guess it's better you come up with a patch proposal.

Jan



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: Interrupt controller updates

[Paolo Bonzini]

Il 21/11/2012 02:07, Benjamin Herrenschmidt ha scritto:
> David (CC) want to make some progress with our in-kernel PIC. From
> memory, one of the outcomes of the BOF was that we need to move the
> existing "enable in-kernel PIC" from generic KVM init to machine init in
> order to be able to add an argument indicating the "model" use by the
> arch/platform since some like ours support several different models and
> since that all needs to be selected before the VCPUs are created.
> 
> Again, from memory, you were volunteered to do the initial x86 change so
> we could piggy back on it :-) Or do I remember wrong ?

Please suggest an API, then we can work out the x86 changes.  I can
volunteer myself, but I wasn't in the BOF so I need something more concrete.

Paolo

Re: [Qemu-devel] Interrupt controller updates

[Paolo Bonzini]

Il 21/11/2012 02:07, Benjamin Herrenschmidt ha scritto:
> David (CC) want to make some progress with our in-kernel PIC. From
> memory, one of the outcomes of the BOF was that we need to move the
> existing "enable in-kernel PIC" from generic KVM init to machine init in
> order to be able to add an argument indicating the "model" use by the
> arch/platform since some like ours support several different models and
> since that all needs to be selected before the VCPUs are created.
> 
> Again, from memory, you were volunteered to do the initial x86 change so
> we could piggy back on it :-) Or do I remember wrong ?

Please suggest an API, then we can work out the x86 changes.  I can
volunteer myself, but I wasn't in the BOF so I need something more concrete.

Paolo

Re: Interrupt controller updates

[Benjamin Herrenschmidt]

On Thu, 2012-11-22 at 11:25 +0100, Paolo Bonzini wrote:
> > Again, from memory, you were volunteered to do the initial x86
> change so
> > we could piggy back on it :-) Or do I remember wrong ?
> 
> Please suggest an API, then we can work out the x86 changes.  I can
> volunteer myself, but I wasn't in the BOF so I need something more
> concrete. 

Oh it's simple enough initially, just move the ioctl call from generic
kvm init to machine init. The problem is then to add an argument, since
that essentially means changing the ioctl number, but we need that for
all archs where the interrupt subsystem can be fundamentally different
based on the platform.

Basically, what was discussed in the BOF was that we split the init:

 * The existing ioctl moves to early machine init (before VCPUs) and
gets that argument to define the type of interrupt subsystem to use. It
causes powerpc to instanciate ICPs per VCPUs for example. On archs that
don't have a per-vcpu structure (equivalent of local APIC or ICP), all
it does is enable subsequent irq related ioctls to work (it's just an
"enable" flag).

 * A new ioctl is used to actually instanciate external interrupt
controllers (GIC on ARM, ICS for ppc/pseries, MPIC for ppc/mpic, ...).
This is used later by the PIC code itself when the former ioctl has
enabled "in kernel PIC"

 * A new ioctl is used for platforms that need to be able to adjust the
base address of a PIC (arm/GIC, ppc/mpic)

We have other things to look at (mostly along the MSI routing calls in
qemu that need to be changed to be PCI bridge hooks populated by the
platform) but that's the starting point.

Cheers,
Ben.

Re: [Qemu-devel] Interrupt controller updates

[Benjamin Herrenschmidt]

On Thu, 2012-11-22 at 11:25 +0100, Paolo Bonzini wrote:
> > Again, from memory, you were volunteered to do the initial x86
> change so
> > we could piggy back on it :-) Or do I remember wrong ?
> 
> Please suggest an API, then we can work out the x86 changes.  I can
> volunteer myself, but I wasn't in the BOF so I need something more
> concrete. 

Oh it's simple enough initially, just move the ioctl call from generic
kvm init to machine init. The problem is then to add an argument, since
that essentially means changing the ioctl number, but we need that for
all archs where the interrupt subsystem can be fundamentally different
based on the platform.

Basically, what was discussed in the BOF was that we split the init:

 * The existing ioctl moves to early machine init (before VCPUs) and
gets that argument to define the type of interrupt subsystem to use. It
causes powerpc to instanciate ICPs per VCPUs for example. On archs that
don't have a per-vcpu structure (equivalent of local APIC or ICP), all
it does is enable subsequent irq related ioctls to work (it's just an
"enable" flag).

 * A new ioctl is used to actually instanciate external interrupt
controllers (GIC on ARM, ICS for ppc/pseries, MPIC for ppc/mpic, ...).
This is used later by the PIC code itself when the former ioctl has
enabled "in kernel PIC"

 * A new ioctl is used for platforms that need to be able to adjust the
base address of a PIC (arm/GIC, ppc/mpic)

We have other things to look at (mostly along the MSI routing calls in
qemu that need to be changed to be PCI bridge hooks populated by the
platform) but that's the starting point.

Cheers,
Ben.

Re: Interrupt controller updates

[Peter Maydell]

On 22 November 2012 21:00, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> Oh it's simple enough initially, just move the ioctl call from generic
> kvm init to machine init. The problem is then to add an argument, since
> that essentially means changing the ioctl number, but we need that for
> all archs where the interrupt subsystem can be fundamentally different
> based on the platform.

I cynically suspect there may need to be some disentangling of x86/qemu
code assumptions about what happens when, in order to do this "just
move" step :-)

> Basically, what was discussed in the BOF was that we split the init:
>
>  * The existing ioctl moves to early machine init (before VCPUs) and
> gets that argument to define the type of interrupt subsystem to use. It
> causes powerpc to instanciate ICPs per VCPUs for example. On archs that
> don't have a per-vcpu structure (equivalent of local APIC or ICP), all
> it does is enable subsequent irq related ioctls to work (it's just an
> "enable" flag).
>
>  * A new ioctl is used to actually instanciate external interrupt
> controllers (GIC on ARM, ICS for ppc/pseries, MPIC for ppc/mpic, ...).
> This is used later by the PIC code itself when the former ioctl has
> enabled "in kernel PIC"

For ARM we could move to use this but it would just be for the
benefit of nicer fallback behaviour (you could say "no in kernel
GIC" if the user runs qemu with a guest CPU which doesn't have a
GIC, rather than having to exit saying "incompatible options"
if 'in-kernel irqchip' and 'cpu with no irqchip' were both
specified).

>  * A new ioctl is used for platforms that need to be able to adjust the
> base address of a PIC (arm/GIC, ppc/mpic)

We have the ABI for this already in the kvm/arm patches which are
heading into final review, by the way.

-- PMM

Re: [Qemu-devel] Interrupt controller updates

[Peter Maydell]

On 22 November 2012 21:00, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> Oh it's simple enough initially, just move the ioctl call from generic
> kvm init to machine init. The problem is then to add an argument, since
> that essentially means changing the ioctl number, but we need that for
> all archs where the interrupt subsystem can be fundamentally different
> based on the platform.

I cynically suspect there may need to be some disentangling of x86/qemu
code assumptions about what happens when, in order to do this "just
move" step :-)

> Basically, what was discussed in the BOF was that we split the init:
>
>  * The existing ioctl moves to early machine init (before VCPUs) and
> gets that argument to define the type of interrupt subsystem to use. It
> causes powerpc to instanciate ICPs per VCPUs for example. On archs that
> don't have a per-vcpu structure (equivalent of local APIC or ICP), all
> it does is enable subsequent irq related ioctls to work (it's just an
> "enable" flag).
>
>  * A new ioctl is used to actually instanciate external interrupt
> controllers (GIC on ARM, ICS for ppc/pseries, MPIC for ppc/mpic, ...).
> This is used later by the PIC code itself when the former ioctl has
> enabled "in kernel PIC"

For ARM we could move to use this but it would just be for the
benefit of nicer fallback behaviour (you could say "no in kernel
GIC" if the user runs qemu with a guest CPU which doesn't have a
GIC, rather than having to exit saying "incompatible options"
if 'in-kernel irqchip' and 'cpu with no irqchip' were both
specified).

>  * A new ioctl is used for platforms that need to be able to adjust the
> base address of a PIC (arm/GIC, ppc/mpic)

We have the ABI for this already in the kvm/arm patches which are
heading into final review, by the way.

-- PMM