Re: [Qemu-devel] [Qemu-discuss] IRQ per CPU

Re: [Qemu-devel] [Qemu-discuss] IRQ per CPU

[Dongli Zhang]

On 07/04/2018 10:32 PM, Probir Roy wrote:
> I am writing a virtual device that would generate IRQ per CPU basis. I

Does 'per CPU basis' indicates irq per cpu, or irq per device queue?

AFAIK, the device may create multiple queues in the driver (in OS) and we would
have one irq (vector) per queue?

If you are talking about irq (vector) per queue, I would suggest read about
following on nvme which involves per-queue vector.

https://github.com/qemu/qemu/blob/master/hw/block/nvme.c

Although I am not expert on qemu, in my opinion the qemu nvme code is very
helpful for understanding per-queue vector.

> have written a PCI device from the template which does not generate
> IRQ per CPU. How can write such device in Qemu?
> 
> The code of current device is here:
> https://gist.github.com/proywm/6ca98d3e8ca001965e2c8792fcf97911
> 
> Regards,
> Probir
> 

Dongli Zhang

Re: [Qemu-devel] [Qemu-discuss] IRQ per CPU

[Probir Roy]

> Does 'per CPU basis' indicates irq per cpu, or irq per device queue?

IRQ per CPU core, meaning that IRQ will be raised at and served by
that CPU. Does IRQ per queue mean the same thing?

Re: [Qemu-devel] [Qemu-discuss] IRQ per CPU

[Dongli Zhang]

On 07/05/2018 12:12 PM, Probir Roy wrote:
>> Does 'per CPU basis' indicates irq per cpu, or irq per device queue?
> 
> IRQ per CPU core, meaning that IRQ will be raised at and served by
> that CPU. Does IRQ per queue mean the same thing?
> 

About 'IRQ per queue', the device may create multiple queue in the OS driver.
The number of queues are always proportional to the number of CPU core/thread
(although this is also always configurable by OS driver).

Usually the per-queue irq/vector is bound to each CPU. As the number of
queue/irq is always the same as the number of CPU, it is sort of 'per CPU
basis', as each CPU will be serving irq for its own queue.

Dongli Zhang

Re: [Qemu-devel] [Qemu-discuss] IRQ per CPU

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

On Thu, Jul 05, 2018 at 03:16:44PM +0800, Dongli Zhang wrote:
> 
> 
> On 07/05/2018 12:12 PM, Probir Roy wrote:
> >> Does 'per CPU basis' indicates irq per cpu, or irq per device queue?
> > 
> > IRQ per CPU core, meaning that IRQ will be raised at and served by
> > that CPU. Does IRQ per queue mean the same thing?
> > 
> 
> About 'IRQ per queue', the device may create multiple queue in the OS driver.
> The number of queues are always proportional to the number of CPU core/thread
> (although this is also always configurable by OS driver).
> 
> Usually the per-queue irq/vector is bound to each CPU. As the number of
> queue/irq is always the same as the number of CPU, it is sort of 'per CPU
> basis', as each CPU will be serving irq for its own queue.

Some more background on devices with multiple irqs:

It's the guest kernel that configures interrupt routing on modern
systems where the interrupt controller hardware supports that.

With a PCI device you would need multiple Message-Signalled Interrupt
vectors.  Then the guest driver can set the irq affinity so that the
vcpu of your choice receives each of these interrupts.

Interrupts are a limited resource, even when using Message-Signalled
Interrupts, so you'll have to choose a maximum number of interrupts that
the device supports.

https://en.wikipedia.org/wiki/Message_Signaled_Interrupts#MSI-X

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]