Extensions for KVM MSI related ioctls

Extensions for KVM MSI related ioctls

[Andre Przywara]

Hi Paolo,

we (Pavel, Eric and me) need your quick advice on how to proceed with a
small API extension:
(tl;dr: skip to the numbered list at the end)

For using MSIs in a guest when running on an ARM(64) system using a
GICv3 interrupt controller we need to have a device ID available. On
real hardware this information is sampled from the bus by the ITS part
of the interrupt controller.
To make this work for guests, we need to extend two ioctls which deal
with MSIs: KVM_SIGNAL_MSI and KVM_SET_GSI_ROUTING.

The idea that we sketched so far in [1] and [2] is to use a new
capability (KVM_CAP_MSI_DEVID) to advertise both a flag bit for
KVM_SIGNAL_MSI and a new type for KVM_SET_GSI_ROUTING.

Since current kernels bail out on any flag value != 0 in KVM_SIGNAL_MSI,
we need the new capability to tell userland about it in a reliable and
portable way (to avoid hacks like #ifdef ARM && USES_IRQ_ROUTING in
userland).

For KVM_SET_GSI_ROUTING there was the idea of using the very same flag
value in it's own flag field, but I find it saner to use a new routing
type instead (KVM_IRQ_ROUTING_EXTENDED_MSI). Both approaches would
piggy-back on the existing struct kvm_irq_routing_msi and re-purpose the
pad field in there.

Summarized:
1) Add a new KVM_CAP_MSI_DEVID to advertise the device ID extension.
2) Use a KVM_MSI_VALID_DEVID flag in KVM_SIGNAL_MSI to re-purpose part
   of struct kvm_msi.
3) Add a new routing type KVM_IRQ_ROUTING_EXTENDED_MSI for
   KVM_SET_GSI_ROUTING to add device IDs in struct kvm_irq_routing_msi.

Is that a sensible way to extend the KVM API?

Cheers,
Andre.

[1]:https://lists.cs.columbia.edu/pipermail/kvmarm/2015-July/015622.html
[2]:https://lists.cs.columbia.edu/pipermail/kvmarm/2015-July/015689.html

Re: Extensions for KVM MSI related ioctls

[Paolo Bonzini]

On 13/07/2015 12:25, Andre Przywara wrote:
> 
> For using MSIs in a guest when running on an ARM(64) system using a
> GICv3 interrupt controller we need to have a device ID available. On
> real hardware this information is sampled from the bus by the ITS part
> of the interrupt controller.
> To make this work for guests, we need to extend two ioctls which deal
> with MSIs: KVM_SIGNAL_MSI and KVM_SET_GSI_ROUTING.
> 
> The idea that we sketched so far in [1] and [2] is to use a new
> capability (KVM_CAP_MSI_DEVID) to advertise both a flag bit for
> KVM_SIGNAL_MSI and a new type for KVM_SET_GSI_ROUTING.
> 
> Since current kernels bail out on any flag value != 0 in KVM_SIGNAL_MSI,
> we need the new capability to tell userland about it in a reliable and
> portable way (to avoid hacks like #ifdef ARM && USES_IRQ_ROUTING in
> userland).
> 
> For KVM_SET_GSI_ROUTING there was the idea of using the very same flag
> value in it's own flag field, but I find it saner to use a new routing
> type instead (KVM_IRQ_ROUTING_EXTENDED_MSI). Both approaches would
> piggy-back on the existing struct kvm_irq_routing_msi and re-purpose the
> pad field in there.

I think I prefer the flag.  Offhand it sounds easier to add support for
it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.

> Summarized:
> 1) Add a new KVM_CAP_MSI_DEVID to advertise the device ID extension.
> 2) Use a KVM_MSI_VALID_DEVID flag in KVM_SIGNAL_MSI to re-purpose part
>    of struct kvm_msi.

These two sounds good.

Paolo

> 3) Add a new routing type KVM_IRQ_ROUTING_EXTENDED_MSI for
>    KVM_SET_GSI_ROUTING to add device IDs in struct kvm_irq_routing_msi.
> 
> Is that a sensible way to extend the KVM API?

RE: Extensions for KVM MSI related ioctls

[Pavel Fedin]

 Hello!

> I think I prefer the flag.  Offhand it sounds easier to add support for
> it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.

 Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
adding new definitions where we can reuse existing ones. IMHO using flag would make an API more
consistent.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

Re: Extensions for KVM MSI related ioctls

[Eric Auger]

On 07/13/2015 03:32 PM, Pavel Fedin wrote:
>  Hello!
> 
>> I think I prefer the flag.  Offhand it sounds easier to add support for
>> it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.
> 
>  Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
> adding new definitions where we can reuse existing ones. IMHO using flag would make an API more
> consistent.

OK I will respin with user space flag.

Andre, what about the kernel routing entry struct. You wanted me to get
rid of  KVM_IRQ_ROUTING_EXTENDED_MSI there too. Will you be able to
manage a usespace wrong setting if the type is not set?

Best Regards

Eric
> 
> Kind regards,
> Pavel Fedin
> Expert Engineer
> Samsung Electronics Research center Russia
> 
> 

Re: Extensions for KVM MSI related ioctls

[Andre Przywara]

Hi,

On 13/07/15 15:24, Eric Auger wrote:
> On 07/13/2015 03:32 PM, Pavel Fedin wrote:
>>  Hello!
>>
>>> I think I prefer the flag.  Offhand it sounds easier to add support for
>>> it to non-ARM architectures, compared to KVM_IRQ_ROUTING_EXTENDED_MSI.
>>
>>  Actually i also voted for flag, because it is already introduced in (2), and i'm not a fan of
>> adding new definitions where we can reuse existing ones. IMHO using flag would make an API more
>> consistent.
> 
> OK I will respin with user space flag.
> 
> Andre, what about the kernel routing entry struct. You wanted me to get
> rid of  KVM_IRQ_ROUTING_EXTENDED_MSI there too. Will you be able to
> manage a usespace wrong setting if the type is not set?

I am about to see how this all fits together, but I don't expect any
serious problems, at least on the kvmtool side.
Instead of setting a different type I just set the flag and guard that
by the capability: should be not an issue.

Cheers,
Andre.