Re: [PATCH v2 6/7] KVM: arm/arm64: enable MSI routing

[Andre Przywara <andre.przywara@arm.com>]

On 09/07/15 09:22, Eric Auger wrote:
> Up to now, only irqchip routing entries could be set. This patch
> adds the capability to insert MSI routing entries, with or without
> device id. Although standard MSI entries can be set, their
> injection still is not supported. For ARM64, let's also increase
> KVM_MAX_IRQ_ROUTES to 4096: include SPI irqchip flat routes plus
> MSI routes. In the future this might be extended.
> 
> The new MSI routing entry type also must be managed similarly to
> legacy KVM_IRQ_ROUTING_MSI in eventfd irqfd_wakeup and irqfd_update.
> 
> Signed-off-by: Eric Auger <eric.auger@linaro.org>
> 
> ---
> 
> v1 -> v2:
> - adapt to new routing entry types
> 
> RFC -> PATCH:
> - move api MSI routing updates into that patch file
> - use new devid field of user api struct
> ---
>  Documentation/virtual/kvm/api.txt | 10 ++++++++++
>  include/linux/kvm_host.h          |  2 ++
>  virt/kvm/arm/vgic.c               | 13 +++++++++++++
>  virt/kvm/eventfd.c                |  6 ++++--
>  4 files changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
> index 9276dac..686b121 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -1431,6 +1431,11 @@ Sets the GSI routing table entries, overwriting any previously set entries.
>  On arm/arm64, GSI routing has the following limitation:
>  - GSI routing does not apply to KVM_IRQ_LINE but only to KVM_IRQFD.
>  
> +On arm/arm64, MSI routing through in-kernel GICv3 ITS must use
> +KVM_IRQ_ROUTING_EXTENDED_MSI routing type and device ID must be set
> +in msi struct. Otherwise, KVM_IRQ_ROUTING_MSI must be used without
> +populating the msi devid field.
> +
>  struct kvm_irq_routing {
>  	__u32 nr;
>  	__u32 flags;
> @@ -2342,6 +2347,11 @@ On arm/arm64, gsi routing being supported, the following can happen:
>  - in case no routing entry is associated to this gsi, injection fails
>  - in case the gsi is associated to an irqchip routing entry,
>    irqchip.pin + 32 corresponds to the injected SPI ID.
> +- in case the gsi is associated to an MSI routing entry,
> +  * without GICv3 ITS in-kernel emulation, MSI data matches the SPI ID
> +    of the injected SPI
> +  * with GICv3 ITS in-kernel emulation, the MSI message and device ID
> +    are translated into an LPI.
>  
>  4.76 KVM_PPC_ALLOCATE_HTAB
>  
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index cac0fe4..ea1a810 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -976,6 +976,8 @@ static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
>  
>  #ifdef CONFIG_S390
>  #define KVM_MAX_IRQ_ROUTES 4096 //FIXME: we can have more than that...
> +#elif defined(CONFIG_ARM64)
> +#define KVM_MAX_IRQ_ROUTES 4096
>  #else
>  #define KVM_MAX_IRQ_ROUTES 1024
>  #endif
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 63be67e..c5546d8 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -2252,6 +2252,19 @@ int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
>  		    (e->irqchip.irqchip >= KVM_NR_IRQCHIPS))
>  			goto out;
>  		break;

what about:

+	case KVM_IRQ_ROUTING_EXTENDED_MSI:
+		e->devid = ue->u.msi.devid;
+		/* fall through */
> +	case KVM_IRQ_ROUTING_MSI:
...

Also if we avoid KVM_IRQ_ROUTING_EXTENDED_MSI in the kernel, we could:
+		e->type = KVM_IRQ_ROUTING_MSI;
(before the fall through) and then get rid of the rest of this patch at all.

Cheers,
Andre.

> +		e->set = kvm_set_msi;
> +		e->msi.address_lo = ue->u.msi.address_lo;
> +		e->msi.address_hi = ue->u.msi.address_hi;
> +		e->msi.data = ue->u.msi.data;
> +		break;



> +	case KVM_IRQ_ROUTING_EXTENDED_MSI:
> +		e->set = kvm_set_msi;
> +		e->msi.address_lo = ue->u.msi.address_lo;
> +		e->msi.address_hi = ue->u.msi.address_hi;
> +		e->msi.data = ue->u.msi.data;
> +		e->devid = ue->u.msi.devid;
> +		break;
>  	default:
>  		goto out;
>  	}
> diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
> index 9ff4193..d76d05d 100644
> --- a/virt/kvm/eventfd.c
> +++ b/virt/kvm/eventfd.c
> @@ -238,7 +238,8 @@ irqfd_wakeup(wait_queue_t *wait, unsigned mode, int sync, void *key)
>  			irq = irqfd->irq_entry;
>  		} while (read_seqcount_retry(&irqfd->irq_entry_sc, seq));
>  		/* An event has been signaled, inject an interrupt */
> -		if (irq.type == KVM_IRQ_ROUTING_MSI)
> +		if (irq.type == KVM_IRQ_ROUTING_MSI ||
> +		    irq.type == KVM_IRQ_ROUTING_EXTENDED_MSI)
>  			kvm_set_msi(&irq, kvm, KVM_USERSPACE_IRQ_SOURCE_ID, 1,
>  					false);
>  		else
> @@ -294,7 +295,8 @@ static void irqfd_update(struct kvm *kvm, struct _irqfd *irqfd)
>  	e = entries;
>  	for (i = 0; i < n_entries; ++i, ++e) {
>  		/* Only fast-path MSI. */
> -		if (e->type == KVM_IRQ_ROUTING_MSI)
> +		if (e->type == KVM_IRQ_ROUTING_MSI ||
> +		    e->type == KVM_IRQ_ROUTING_EXTENDED_MSI)
>  			irqfd->irq_entry = *e;
>  	}
>  
>