[PATCH RFC 0/2] qemu-kvm: MSI-X support

[PATCH RFC 0/2] qemu-kvm: MSI-X support

[Michael S. Tsirkin]

Here's a draft MSI-X support patch. Among missing features:
save/load support, and command-line flag to control the
feature. This is on top of qemu-kvm: msi-x is disabled
without kvm interrupt injection support for now.

Michael S. Tsirkin (2):
  qemu-kvm: add MSI-X support
  qemu-kvm: use common code for assigned msix

 Makefile.target        |    2 +-
 hw/device-assignment.c |  336 +++++++++++---------------------------------
 hw/device-assignment.h |    8 +-
 hw/msix.c              |  371 ++++++++++++++++++++++++++++++++++++++++++++++++
 hw/msix.h              |   33 +++++
 hw/pci.c               |   35 ++++--
 hw/pci.h               |   57 +++++++-
 hw/virtio-balloon.c    |    2 +-
 hw/virtio-blk.c        |    3 +-
 hw/virtio-console.c    |    3 +-
 hw/virtio-net.c        |    3 +-
 hw/virtio.c            |  167 +++++++++++++++++-----
 hw/virtio.h            |    4 +-
 13 files changed, 709 insertions(+), 315 deletions(-)
 create mode 100644 hw/msix.c
 create mode 100644 hw/msix.h

Re: [PATCH RFC 0/2] qemu-kvm: MSI-X support

[Anthony Liguori]

Michael S. Tsirkin wrote:
> Here's a draft MSI-X support patch. Among missing features:
> save/load support, and command-line flag to control the
> feature. This is on top of qemu-kvm: msi-x is disabled
> without kvm interrupt injection support for now.
>   

What's your impression of how much work would be to get this going on 
top of upstream QEMU?

I'm willing to borrow a few cycles to help out here.  I'd really like to 
see this series go in via QEMU if possible.

Regards,

Anthony Liguori

> Michael S. Tsirkin (2):
>   qemu-kvm: add MSI-X support
>   qemu-kvm: use common code for assigned msix
>
>  Makefile.target        |    2 +-
>  hw/device-assignment.c |  336 +++++++++++---------------------------------
>  hw/device-assignment.h |    8 +-
>  hw/msix.c              |  371 ++++++++++++++++++++++++++++++++++++++++++++++++
>  hw/msix.h              |   33 +++++
>  hw/pci.c               |   35 ++++--
>  hw/pci.h               |   57 +++++++-
>  hw/virtio-balloon.c    |    2 +-
>  hw/virtio-blk.c        |    3 +-
>  hw/virtio-console.c    |    3 +-
>  hw/virtio-net.c        |    3 +-
>  hw/virtio.c            |  167 +++++++++++++++++-----
>  hw/virtio.h            |    4 +-
>  13 files changed, 709 insertions(+), 315 deletions(-)
>  create mode 100644 hw/msix.c
>  create mode 100644 hw/msix.h
>   

Re: [PATCH RFC 0/2] qemu-kvm: MSI-X support

[Michael S. Tsirkin]

On Mon, May 11, 2009 at 05:24:25PM -0500, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> Here's a draft MSI-X support patch. Among missing features:
>> save/load support, and command-line flag to control the
>> feature. This is on top of qemu-kvm: msi-x is disabled
>> without kvm interrupt injection support for now.
>>   
>
> What's your impression of how much work would be to get this going on  
> top of upstream QEMU?
>
> I'm willing to borrow a few cycles to help out here.  I'd really like to  
> see this series go in via QEMU if possible.

It seems that if I just call apic_deliver_irq each time
I want to send MSI, things will work.

However, large part of the msix code is managing IRQs versus kernel,
and I'm not sure it's a wise investment of effort to rip it all out. So
IMHO, what's missing is API that abstracts managing irq routes in kvm,
specifically abstract this stuff in some way:
	kvm_get_irq_route_gsi
	kvm_add_routing_entry
	kvm_del_routing_entry
	kvm_commit_irq_routes
	kvm_set_irq
How hard is that?

For now, this API could be a stub that just stores the routes somewhere,
and set_irq would call the local apic emulation, along the lines of:

	uint8_t dest = (addr_lo & MSI_ADDR_DEST_ID_MASK)
		>> MSI_ADDR_DEST_ID_SHIFT;
	uint8_t vector = (addr_hi & MSI_DATA_VECTOR_MASK)
		>> MSI_DATA_VECTOR_SHIFT;
	uint8_t dest_mode = (addr_lo >> MSI_ADDR_DEST_MODE_SHIFT) & 0x1;
	uint8_t trigger_mode = (data >> MSI_DATA_TRIGGER_SHIFT) & 0x1;
	uint8_t delivery_mode = (data >> MSI_DATA_DELIVERY_MODE_SHIFT) &
		0x7;
	apic_deliver_irq(dest, dest_mode, delivery_mode, vector, 0,
			 trigger_mode);

I would be happy to port my msix code to work on top of this API.
Willing to help?


-- 
MST

Re: [PATCH RFC 0/2] qemu-kvm: MSI-X support

[Avi Kivity]

Michael S. Tsirkin wrote:
> It seems that if I just call apic_deliver_irq each time
> I want to send MSI, things will work.
>
> However, large part of the msix code is managing IRQs versus kernel,
> and I'm not sure it's a wise investment of effort to rip it all out. So
> IMHO, what's missing is API that abstracts managing irq routes in kvm,
> specifically abstract this stuff in some way:
> 	kvm_get_irq_route_gsi
> 	kvm_add_routing_entry
> 	kvm_del_routing_entry
> 	kvm_commit_irq_routes
>   

All these are just games with qemu_irq objects.  Should be a lot simpler 
in userspace.


> 	kvm_set_irq
>   

qemu_set_irq().

> How hard is that?
>   

Should be pretty easy, once you get the hang of qemu_irq.

> For now, this API could be a stub that just stores the routes somewhere,
> and set_irq would call the local apic emulation, along the lines of:
>
> 	uint8_t dest = (addr_lo & MSI_ADDR_DEST_ID_MASK)
> 		>> MSI_ADDR_DEST_ID_SHIFT;
> 	uint8_t vector = (addr_hi & MSI_DATA_VECTOR_MASK)
> 		>> MSI_DATA_VECTOR_SHIFT;
> 	uint8_t dest_mode = (addr_lo >> MSI_ADDR_DEST_MODE_SHIFT) & 0x1;
> 	uint8_t trigger_mode = (data >> MSI_DATA_TRIGGER_SHIFT) & 0x1;
> 	uint8_t delivery_mode = (data >> MSI_DATA_DELIVERY_MODE_SHIFT) &
> 		0x7;
> 	apic_deliver_irq(dest, dest_mode, delivery_mode, vector, 0,
> 			 trigger_mode);
>   

qemu_set_irq() eventually calls a callback that you specify; just set it 
do look up the entry and call apic_deliver_irq.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.