[Qemu-devel] [PATCH v5 0/3] Make KVM/MSI code device-ID-aware

[Qemu-devel] [PATCH v5 0/3] Make KVM/MSI code device-ID-aware

[Pavel Fedin]

On ARM architecture ITS (Interrupt Translation Service), additionally to
normal MSI data, uses also side-band device IDs. This series prepares the
infrastructure to handling them.

This small series is actually an extraction from vITS support RFC
(https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg07074.html).
Nobody reviewed it so far, however some bits of it could actually be
applied early. This series consists only of those parts, which do not
depend on any new unreleased kernel APIs.

v4 => v5:
- pci_requester_id() moved back to architecture-independent PCI code.
- Use pci_requester_id() in more places.

v3 => v4:
- Rename msi_requester_id() to pci_requester_id() and move from MSI to PCI
  code

v2 => v3:
- Use official "requester ID" denotion everywhere. Renamed getter function
  and MemTxAttrs field.

v1 => v2:
- Improved commit messages, more explanations
- msi_device_id() function made architecture-specific

Pavel Fedin (3):
  kvm: Make KVM_CAP_SIGNAL_MSI globally available
  hw/pci: Introduce pci_requester_id()
  kvm: Pass PCI device pointer to MSI routing functions

 hw/i386/kvm/pci-assign.c | 11 ++++++-----
 hw/pci/msi.c             |  2 +-
 hw/pci/pcie_aer.c        |  2 +-
 hw/vfio/pci.c            | 11 ++++++-----
 hw/virtio/virtio-pci.c   |  5 +++--
 include/exec/memattrs.h  |  4 ++--
 include/hw/pci/pci.h     |  5 +++++
 include/sysemu/kvm.h     | 14 ++++++++++++--
 kvm-all.c                | 15 ++++++++-------
 kvm-stub.c               |  5 +++--
 10 files changed, 47 insertions(+), 27 deletions(-)

-- 
2.4.4

[Qemu-devel] [PATCH v5 1/3] kvm: Make KVM_CAP_SIGNAL_MSI globally available

[Pavel Fedin]

This capability is useful to determine whether we can use KVM ITS
emulation on ARM

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
---
 include/sysemu/kvm.h |  9 +++++++++
 kvm-all.c            | 10 +++++-----
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 2a58b4d..d42c464 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -52,6 +52,7 @@ extern bool kvm_msi_via_irqfd_allowed;
 extern bool kvm_gsi_routing_allowed;
 extern bool kvm_gsi_direct_mapping;
 extern bool kvm_readonly_mem_allowed;
+extern bool kvm_direct_msi_allowed;
 
 #if defined CONFIG_KVM || !defined NEED_CPU_H
 #define kvm_enabled()           (kvm_allowed)
@@ -145,6 +146,13 @@ extern bool kvm_readonly_mem_allowed;
  */
 #define kvm_readonly_mem_enabled() (kvm_readonly_mem_allowed)
 
+/**
+ * kvm_direct_msi_enabled:
+ *
+ * Returns: true if KVM allows direct MSI injection.
+ */
+#define kvm_direct_msi_enabled() (kvm_direct_msi_allowed)
+
 #else
 #define kvm_enabled()           (0)
 #define kvm_irqchip_in_kernel() (false)
@@ -157,6 +165,7 @@ extern bool kvm_readonly_mem_allowed;
 #define kvm_gsi_routing_allowed() (false)
 #define kvm_gsi_direct_mapping() (false)
 #define kvm_readonly_mem_enabled() (false)
+#define kvm_direct_msi_enabled() (false)
 #endif
 
 struct kvm_run;
diff --git a/kvm-all.c b/kvm-all.c
index 0be4615..4931b27 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -93,7 +93,6 @@ struct KVMState
     uint32_t *used_gsi_bitmap;
     unsigned int gsi_count;
     QTAILQ_HEAD(msi_hashtab, KVMMSIRoute) msi_hashtab[KVM_MSI_HASHTAB_SIZE];
-    bool direct_msi;
 #endif
     KVMMemoryListener memory_listener;
 };
@@ -111,6 +110,7 @@ bool kvm_gsi_direct_mapping;
 bool kvm_allowed;
 bool kvm_readonly_mem_allowed;
 bool kvm_vm_attributes_allowed;
+bool kvm_direct_msi_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
@@ -979,7 +979,7 @@ void kvm_init_irq_routing(KVMState *s)
     s->irq_routes = g_malloc0(sizeof(*s->irq_routes));
     s->nr_allocated_irq_routes = 0;
 
-    if (!s->direct_msi) {
+    if (!kvm_direct_msi_allowed) {
         for (i = 0; i < KVM_MSI_HASHTAB_SIZE; i++) {
             QTAILQ_INIT(&s->msi_hashtab[i]);
         }
@@ -1113,7 +1113,7 @@ static int kvm_irqchip_get_virq(KVMState *s)
      * number can succeed even though a new route entry cannot be added.
      * When this happens, flush dynamic MSI entries to free IRQ route entries.
      */
-    if (!s->direct_msi && s->irq_routes->nr == s->gsi_count) {
+    if (!kvm_direct_msi_allowed && s->irq_routes->nr == s->gsi_count) {
         kvm_flush_dynamic_msi_routes(s);
     }
 
@@ -1150,7 +1150,7 @@ int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg)
     struct kvm_msi msi;
     KVMMSIRoute *route;
 
-    if (s->direct_msi) {
+    if (kvm_direct_msi_allowed) {
         msi.address_lo = (uint32_t)msg.address;
         msi.address_hi = msg.address >> 32;
         msi.data = le32_to_cpu(msg.data);
@@ -1598,7 +1598,7 @@ static int kvm_init(MachineState *ms)
 #endif
 
 #ifdef KVM_CAP_IRQ_ROUTING
-    s->direct_msi = (kvm_check_extension(s, KVM_CAP_SIGNAL_MSI) > 0);
+    kvm_direct_msi_allowed = (kvm_check_extension(s, KVM_CAP_SIGNAL_MSI) > 0);
 #endif
 
     s->intx_set_mask = kvm_check_extension(s, KVM_CAP_PCI_2_3);
-- 
2.4.4

[Qemu-devel] [PATCH v5 2/3] hw/pci: Introduce pci_requester_id()

[Pavel Fedin]

For GICv3 ITS implementation we are going to use requester IDs in KVM IRQ
routing code. This patch introduces reusable convenient way to obtain this
ID from the device pointer. The new function is now used in some places,
where the same calculation was used.

MemTxAttrs.stream_id also renamed to requester_id in order to better
reflect semantics of the field.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
---
 hw/i386/kvm/pci-assign.c | 2 +-
 hw/pci/msi.c             | 2 +-
 hw/pci/pcie_aer.c        | 2 +-
 include/exec/memattrs.h  | 4 ++--
 include/hw/pci/pci.h     | 5 +++++
 5 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
index 44beee3..e48cae6 100644
--- a/hw/i386/kvm/pci-assign.c
+++ b/hw/i386/kvm/pci-assign.c
@@ -1483,7 +1483,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
          * error bits, leave the rest. */
         status = pci_get_long(pci_dev->config + pos + PCI_X_STATUS);
         status &= ~(PCI_X_STATUS_BUS | PCI_X_STATUS_DEVFN);
-        status |= (pci_bus_num(pci_dev->bus) << 8) | pci_dev->devfn;
+        status |= pci_requester_id(pci_dev);
         status &= ~(PCI_X_STATUS_SPL_DISC | PCI_X_STATUS_UNX_SPL |
                     PCI_X_STATUS_SPL_ERR);
         pci_set_long(pci_dev->config + pos + PCI_X_STATUS, status);
diff --git a/hw/pci/msi.c b/hw/pci/msi.c
index f9c0484..c1dd531 100644
--- a/hw/pci/msi.c
+++ b/hw/pci/msi.c
@@ -294,7 +294,7 @@ void msi_send_message(PCIDevice *dev, MSIMessage msg)
 {
     MemTxAttrs attrs = {};
 
-    attrs.stream_id = (pci_bus_num(dev->bus) << 8) | dev->devfn;
+    attrs.requester_id = pci_requester_id(dev);
     address_space_stl_le(&dev->bus_master_as, msg.address, msg.data,
                          attrs, NULL);
 }
diff --git a/hw/pci/pcie_aer.c b/hw/pci/pcie_aer.c
index 46e0ad8..98d2c18 100644
--- a/hw/pci/pcie_aer.c
+++ b/hw/pci/pcie_aer.c
@@ -979,7 +979,7 @@ static int do_pcie_aer_inject_error(Monitor *mon,
         }
     }
     err.status = error_status;
-    err.source_id = (pci_bus_num(dev->bus) << 8) | dev->devfn;
+    err.source_id = pci_requester_id(dev);
 
     err.flags = 0;
     if (correctable) {
diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
index f8537a8..e601061 100644
--- a/include/exec/memattrs.h
+++ b/include/exec/memattrs.h
@@ -35,8 +35,8 @@ typedef struct MemTxAttrs {
     unsigned int secure:1;
     /* Memory access is usermode (unprivileged) */
     unsigned int user:1;
-    /* Stream ID (for MSI for example) */
-    unsigned int stream_id:16;
+    /* Requester ID (for MSI for example) */
+    unsigned int requester_id:16;
 } MemTxAttrs;
 
 /* Bus masters which don't specify any attributes will get this,
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index 551cb3d..f5e7fd8 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -677,6 +677,11 @@ static inline uint32_t pci_config_size(const PCIDevice *d)
     return pci_is_express(d) ? PCIE_CONFIG_SPACE_SIZE : PCI_CONFIG_SPACE_SIZE;
 }
 
+static inline uint16_t pci_requester_id(PCIDevice *dev)
+{
+    return (pci_bus_num(dev->bus) << 8) | dev->devfn;
+}
+
 /* DMA access functions */
 static inline AddressSpace *pci_get_address_space(PCIDevice *dev)
 {
-- 
2.4.4

Re: [Qemu-devel] [PATCH v5 2/3] hw/pci: Introduce pci_requester_id()

[Michael S. Tsirkin]

On Thu, Oct 15, 2015 at 02:05:16PM +0300, Pavel Fedin wrote:
> For GICv3 ITS implementation we are going to use requester IDs in KVM IRQ
> routing code. This patch introduces reusable convenient way to obtain this
> ID from the device pointer. The new function is now used in some places,
> where the same calculation was used.
> 
> MemTxAttrs.stream_id also renamed to requester_id in order to better
> reflect semantics of the field.
> 
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>

You are merging all this through the KVM tree, right?

In that case:
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>

> ---
>  hw/i386/kvm/pci-assign.c | 2 +-
>  hw/pci/msi.c             | 2 +-
>  hw/pci/pcie_aer.c        | 2 +-
>  include/exec/memattrs.h  | 4 ++--
>  include/hw/pci/pci.h     | 5 +++++
>  5 files changed, 10 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
> index 44beee3..e48cae6 100644
> --- a/hw/i386/kvm/pci-assign.c
> +++ b/hw/i386/kvm/pci-assign.c
> @@ -1483,7 +1483,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
>           * error bits, leave the rest. */
>          status = pci_get_long(pci_dev->config + pos + PCI_X_STATUS);
>          status &= ~(PCI_X_STATUS_BUS | PCI_X_STATUS_DEVFN);
> -        status |= (pci_bus_num(pci_dev->bus) << 8) | pci_dev->devfn;
> +        status |= pci_requester_id(pci_dev);
>          status &= ~(PCI_X_STATUS_SPL_DISC | PCI_X_STATUS_UNX_SPL |
>                      PCI_X_STATUS_SPL_ERR);
>          pci_set_long(pci_dev->config + pos + PCI_X_STATUS, status);
> diff --git a/hw/pci/msi.c b/hw/pci/msi.c
> index f9c0484..c1dd531 100644
> --- a/hw/pci/msi.c
> +++ b/hw/pci/msi.c
> @@ -294,7 +294,7 @@ void msi_send_message(PCIDevice *dev, MSIMessage msg)
>  {
>      MemTxAttrs attrs = {};
>  
> -    attrs.stream_id = (pci_bus_num(dev->bus) << 8) | dev->devfn;
> +    attrs.requester_id = pci_requester_id(dev);
>      address_space_stl_le(&dev->bus_master_as, msg.address, msg.data,
>                           attrs, NULL);
>  }
> diff --git a/hw/pci/pcie_aer.c b/hw/pci/pcie_aer.c
> index 46e0ad8..98d2c18 100644
> --- a/hw/pci/pcie_aer.c
> +++ b/hw/pci/pcie_aer.c
> @@ -979,7 +979,7 @@ static int do_pcie_aer_inject_error(Monitor *mon,
>          }
>      }
>      err.status = error_status;
> -    err.source_id = (pci_bus_num(dev->bus) << 8) | dev->devfn;
> +    err.source_id = pci_requester_id(dev);
>  
>      err.flags = 0;
>      if (correctable) {
> diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
> index f8537a8..e601061 100644
> --- a/include/exec/memattrs.h
> +++ b/include/exec/memattrs.h
> @@ -35,8 +35,8 @@ typedef struct MemTxAttrs {
>      unsigned int secure:1;
>      /* Memory access is usermode (unprivileged) */
>      unsigned int user:1;
> -    /* Stream ID (for MSI for example) */
> -    unsigned int stream_id:16;
> +    /* Requester ID (for MSI for example) */
> +    unsigned int requester_id:16;
>  } MemTxAttrs;
>  
>  /* Bus masters which don't specify any attributes will get this,
> diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
> index 551cb3d..f5e7fd8 100644
> --- a/include/hw/pci/pci.h
> +++ b/include/hw/pci/pci.h
> @@ -677,6 +677,11 @@ static inline uint32_t pci_config_size(const PCIDevice *d)
>      return pci_is_express(d) ? PCIE_CONFIG_SPACE_SIZE : PCI_CONFIG_SPACE_SIZE;
>  }
>  
> +static inline uint16_t pci_requester_id(PCIDevice *dev)
> +{
> +    return (pci_bus_num(dev->bus) << 8) | dev->devfn;
> +}
> +
>  /* DMA access functions */
>  static inline AddressSpace *pci_get_address_space(PCIDevice *dev)
>  {
> -- 
> 2.4.4

Re: [Qemu-devel] [PATCH v5 2/3] hw/pci: Introduce pci_requester_id()

[Pavel Fedin]

 Hello!

> You are merging all this through the KVM tree, right?

 To tell the truth, i don't know. :) I cc'ed everybody related, but from maintainers only you responded. You could agree, whose tree
will take it. Or you could rip the set apart and each maintainer merges own pieces, it doesn't matter. As i said before, these
patches do not depend on each other.

> Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
> Acked-by: Michael S. Tsirkin <mst@redhat.com>

 Thank you very much. I'll post v6 though, because of Cornelia's notice on 3/3.

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

Re: [Qemu-devel] [PATCH v5 2/3] hw/pci: Introduce pci_requester_id()

[Michael S. Tsirkin]

On Thu, Oct 15, 2015 at 02:05:16PM +0300, Pavel Fedin wrote:
> For GICv3 ITS implementation we are going to use requester IDs in KVM IRQ
> routing code. This patch introduces reusable convenient way to obtain this
> ID from the device pointer. The new function is now used in some places,
> where the same calculation was used.
> 
> MemTxAttrs.stream_id also renamed to requester_id in order to better
> reflect semantics of the field.
> 
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>


Acked-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>

> ---
>  hw/i386/kvm/pci-assign.c | 2 +-
>  hw/pci/msi.c             | 2 +-
>  hw/pci/pcie_aer.c        | 2 +-
>  include/exec/memattrs.h  | 4 ++--
>  include/hw/pci/pci.h     | 5 +++++
>  5 files changed, 10 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
> index 44beee3..e48cae6 100644
> --- a/hw/i386/kvm/pci-assign.c
> +++ b/hw/i386/kvm/pci-assign.c
> @@ -1483,7 +1483,7 @@ static int assigned_device_pci_cap_init(PCIDevice *pci_dev, Error **errp)
>           * error bits, leave the rest. */
>          status = pci_get_long(pci_dev->config + pos + PCI_X_STATUS);
>          status &= ~(PCI_X_STATUS_BUS | PCI_X_STATUS_DEVFN);
> -        status |= (pci_bus_num(pci_dev->bus) << 8) | pci_dev->devfn;
> +        status |= pci_requester_id(pci_dev);
>          status &= ~(PCI_X_STATUS_SPL_DISC | PCI_X_STATUS_UNX_SPL |
>                      PCI_X_STATUS_SPL_ERR);
>          pci_set_long(pci_dev->config + pos + PCI_X_STATUS, status);
> diff --git a/hw/pci/msi.c b/hw/pci/msi.c
> index f9c0484..c1dd531 100644
> --- a/hw/pci/msi.c
> +++ b/hw/pci/msi.c
> @@ -294,7 +294,7 @@ void msi_send_message(PCIDevice *dev, MSIMessage msg)
>  {
>      MemTxAttrs attrs = {};
>  
> -    attrs.stream_id = (pci_bus_num(dev->bus) << 8) | dev->devfn;
> +    attrs.requester_id = pci_requester_id(dev);
>      address_space_stl_le(&dev->bus_master_as, msg.address, msg.data,
>                           attrs, NULL);
>  }
> diff --git a/hw/pci/pcie_aer.c b/hw/pci/pcie_aer.c
> index 46e0ad8..98d2c18 100644
> --- a/hw/pci/pcie_aer.c
> +++ b/hw/pci/pcie_aer.c
> @@ -979,7 +979,7 @@ static int do_pcie_aer_inject_error(Monitor *mon,
>          }
>      }
>      err.status = error_status;
> -    err.source_id = (pci_bus_num(dev->bus) << 8) | dev->devfn;
> +    err.source_id = pci_requester_id(dev);
>  
>      err.flags = 0;
>      if (correctable) {
> diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
> index f8537a8..e601061 100644
> --- a/include/exec/memattrs.h
> +++ b/include/exec/memattrs.h
> @@ -35,8 +35,8 @@ typedef struct MemTxAttrs {
>      unsigned int secure:1;
>      /* Memory access is usermode (unprivileged) */
>      unsigned int user:1;
> -    /* Stream ID (for MSI for example) */
> -    unsigned int stream_id:16;
> +    /* Requester ID (for MSI for example) */
> +    unsigned int requester_id:16;
>  } MemTxAttrs;
>  
>  /* Bus masters which don't specify any attributes will get this,
> diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
> index 551cb3d..f5e7fd8 100644
> --- a/include/hw/pci/pci.h
> +++ b/include/hw/pci/pci.h
> @@ -677,6 +677,11 @@ static inline uint32_t pci_config_size(const PCIDevice *d)
>      return pci_is_express(d) ? PCIE_CONFIG_SPACE_SIZE : PCI_CONFIG_SPACE_SIZE;
>  }
>  
> +static inline uint16_t pci_requester_id(PCIDevice *dev)
> +{
> +    return (pci_bus_num(dev->bus) << 8) | dev->devfn;
> +}
> +
>  /* DMA access functions */
>  static inline AddressSpace *pci_get_address_space(PCIDevice *dev)
>  {
> -- 
> 2.4.4

[Qemu-devel] [PATCH v5 3/3] kvm: Pass PCI device pointer to MSI routing functions

[Pavel Fedin]

In-kernel ITS emulation on ARM64 will require to supply requester IDs.
These IDs can now be retrieved from the device pointer using new
pci_requester_id() function.

This patch adds pci_dev pointer to KVM GSI routing functions and makes
callers passing it.

x86 architecture does not use requester IDs, but hw/i386/kvm/pci-assign.c
also made passing PCI device pointer instead of NULL for consistency with
the rest of the code.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
---
 hw/i386/kvm/pci-assign.c |  9 +++++----
 hw/vfio/pci.c            | 11 ++++++-----
 hw/virtio/virtio-pci.c   |  5 +++--
 include/sysemu/kvm.h     |  5 +++--
 kvm-all.c                |  5 +++--
 kvm-stub.c               |  5 +++--
 6 files changed, 23 insertions(+), 17 deletions(-)

diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
index e48cae6..6c33af0 100644
--- a/hw/i386/kvm/pci-assign.c
+++ b/hw/i386/kvm/pci-assign.c
@@ -979,7 +979,7 @@ static void assigned_dev_update_msi(PCIDevice *pci_dev)
         MSIMessage msg = msi_get_message(pci_dev, 0);
         int virq;
 
-        virq = kvm_irqchip_add_msi_route(kvm_state, msg);
+        virq = kvm_irqchip_add_msi_route(kvm_state, msg, pci_dev);
         if (virq < 0) {
             perror("assigned_dev_update_msi: kvm_irqchip_add_msi_route");
             return;
@@ -1017,7 +1017,7 @@ static void assigned_dev_update_msi_msg(PCIDevice *pci_dev)
     }
 
     kvm_irqchip_update_msi_route(kvm_state, assigned_dev->msi_virq[0],
-                                 msi_get_message(pci_dev, 0));
+                                 msi_get_message(pci_dev, 0), pci_dev);
 }
 
 static bool assigned_dev_msix_masked(MSIXTableEntry *entry)
@@ -1083,7 +1083,7 @@ static int assigned_dev_update_msix_mmio(PCIDevice *pci_dev)
 
         msg.address = entry->addr_lo | ((uint64_t)entry->addr_hi << 32);
         msg.data = entry->data;
-        r = kvm_irqchip_add_msi_route(kvm_state, msg);
+        r = kvm_irqchip_add_msi_route(kvm_state, msg, pci_dev);
         if (r < 0) {
             return r;
         }
@@ -1602,7 +1602,8 @@ static void assigned_dev_msix_mmio_write(void *opaque, hwaddr addr,
                 msg.data = entry->data;
 
                 ret = kvm_irqchip_update_msi_route(kvm_state,
-                                                   adev->msi_virq[i], msg);
+                                                   adev->msi_virq[i], msg,
+                                                   pci_dev);
                 if (ret) {
                     error_report("Error updating irq routing entry (%d)", ret);
                 }
diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index dcabb6d..8fadbcf 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -424,7 +424,7 @@ static void vfio_add_kvm_msi_virq(VFIOPCIDevice *vdev, VFIOMSIVector *vector,
         return;
     }
 
-    virq = kvm_irqchip_add_msi_route(kvm_state, *msg);
+    virq = kvm_irqchip_add_msi_route(kvm_state, *msg, &vdev->pdev);
     if (virq < 0) {
         event_notifier_cleanup(&vector->kvm_interrupt);
         return;
@@ -449,9 +449,10 @@ static void vfio_remove_kvm_msi_virq(VFIOMSIVector *vector)
     event_notifier_cleanup(&vector->kvm_interrupt);
 }
 
-static void vfio_update_kvm_msi_virq(VFIOMSIVector *vector, MSIMessage msg)
+static void vfio_update_kvm_msi_virq(VFIOMSIVector *vector, MSIMessage msg,
+                                     PCIDevice *pdev)
 {
-    kvm_irqchip_update_msi_route(kvm_state, vector->virq, msg);
+    kvm_irqchip_update_msi_route(kvm_state, vector->virq, msg, pdev);
 }
 
 static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
@@ -486,7 +487,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
         if (!msg) {
             vfio_remove_kvm_msi_virq(vector);
         } else {
-            vfio_update_kvm_msi_virq(vector, *msg);
+            vfio_update_kvm_msi_virq(vector, *msg, pdev);
         }
     } else {
         vfio_add_kvm_msi_virq(vdev, vector, msg, true);
@@ -760,7 +761,7 @@ static void vfio_update_msi(VFIOPCIDevice *vdev)
         }
 
         msg = msi_get_message(&vdev->pdev, i);
-        vfio_update_kvm_msi_virq(vector, msg);
+        vfio_update_kvm_msi_virq(vector, msg, &vdev->pdev);
     }
 }
 
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index e5c406d..f55dd2b 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -590,7 +590,7 @@ static int kvm_virtio_pci_vq_vector_use(VirtIOPCIProxy *proxy,
     int ret;
 
     if (irqfd->users == 0) {
-        ret = kvm_irqchip_add_msi_route(kvm_state, msg);
+        ret = kvm_irqchip_add_msi_route(kvm_state, msg, &proxy->pci_dev);
         if (ret < 0) {
             return ret;
         }
@@ -726,7 +726,8 @@ static int virtio_pci_vq_vector_unmask(VirtIOPCIProxy *proxy,
     if (proxy->vector_irqfd) {
         irqfd = &proxy->vector_irqfd[vector];
         if (irqfd->msg.data != msg.data || irqfd->msg.address != msg.address) {
-            ret = kvm_irqchip_update_msi_route(kvm_state, irqfd->virq, msg);
+            ret = kvm_irqchip_update_msi_route(kvm_state, irqfd->virq, msg,
+                                               &proxy->pci_dev);
             if (ret < 0) {
                 return ret;
             }
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index d42c464..4d26571 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -447,8 +447,9 @@ static inline void cpu_clean_state(CPUState *cpu)
     }
 }
 
-int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg);
-int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg);
+int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg, PCIDevice *dev);
+int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
+                                 PCIDevice *dev);
 void kvm_irqchip_release_virq(KVMState *s, int virq);
 
 int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter);
diff --git a/kvm-all.c b/kvm-all.c
index 4931b27..4632138 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -1189,7 +1189,7 @@ int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg)
     return kvm_set_irq(s, route->kroute.gsi, 1);
 }
 
-int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
+int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg, PCIDevice *dev)
 {
     struct kvm_irq_routing_entry kroute = {};
     int virq;
@@ -1224,7 +1224,8 @@ int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
     return virq;
 }
 
-int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg)
+int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
+                                 PCIDevice *dev)
 {
     struct kvm_irq_routing_entry kroute = {};
 
diff --git a/kvm-stub.c b/kvm-stub.c
index d9ad624..08bcc32 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -115,7 +115,7 @@ int kvm_on_sigbus(int code, void *addr)
 }
 
 #ifndef CONFIG_USER_ONLY
-int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
+int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg, PCIDevice *dev)
 {
     return -ENOSYS;
 }
@@ -128,7 +128,8 @@ void kvm_irqchip_release_virq(KVMState *s, int virq)
 {
 }
 
-int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg)
+int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
+                                 PCIDevice *dev)
 {
     return -ENOSYS;
 }
-- 
2.4.4

Re: [Qemu-devel] [PATCH v5 3/3] kvm: Pass PCI device pointer to MSI routing functions

[Cornelia Huck]

On Thu, 15 Oct 2015 14:05:17 +0300
Pavel Fedin <p.fedin@samsung.com> wrote:

> In-kernel ITS emulation on ARM64 will require to supply requester IDs.
> These IDs can now be retrieved from the device pointer using new
> pci_requester_id() function.
> 
> This patch adds pci_dev pointer to KVM GSI routing functions and makes
> callers passing it.

Should the pci_dev be passed to kvm_arch_fixup_msi_route() as well?
Currently, the only implementer of this function (s390x) does not need
it; but it shouldn't hurt?

> 
> x86 architecture does not use requester IDs, but hw/i386/kvm/pci-assign.c
> also made passing PCI device pointer instead of NULL for consistency with
> the rest of the code.
> 
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> ---
>  hw/i386/kvm/pci-assign.c |  9 +++++----
>  hw/vfio/pci.c            | 11 ++++++-----
>  hw/virtio/virtio-pci.c   |  5 +++--
>  include/sysemu/kvm.h     |  5 +++--
>  kvm-all.c                |  5 +++--
>  kvm-stub.c               |  5 +++--
>  6 files changed, 23 insertions(+), 17 deletions(-)

Re: [Qemu-devel] [PATCH v5 3/3] kvm: Pass PCI device pointer to MSI routing functions

[Pavel Fedin]

 Hello!

> Should the pci_dev be passed to kvm_arch_fixup_msi_route() as well?
> Currently, the only implementer of this function (s390x) does not need
> it; but it shouldn't hurt?

 Yes, it shouldn't. I didn't pass it there exactly because of this reason. However, &kroute is also passed to this function, so
setting ARM-specific data could be moved there. I mean this repeating sequence (see vITS RFC 4/6 -
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg07075.html):
--- cut ---
    kroute.flags = kvm_msi_flags;
    if (kroute.flags & KVM_MSI_VALID_DEVID) {
        kroute.u.msi.devid = pci_requester_id(dev);
    }
--- cut ---

Kind regards,
Pavel Fedin
Expert Engineer
Samsung Electronics Research center Russia

Re: [Qemu-devel] [PATCH v5 0/3] Make KVM/MSI code device-ID-aware

[Michael S. Tsirkin]

On Thu, Oct 15, 2015 at 02:05:14PM +0300, Pavel Fedin wrote:
> On ARM architecture ITS (Interrupt Translation Service), additionally to
> normal MSI data, uses also side-band device IDs. This series prepares the
> infrastructure to handling them.
> 
> This small series is actually an extraction from vITS support RFC
> (https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg07074.html).
> Nobody reviewed it so far, however some bits of it could actually be
> applied early. This series consists only of those parts, which do not
> depend on any new unreleased kernel APIs.

Looks sane
Acked-by: Michael S. Tsirkin <mst@redhat.com>

I also reviewed the pci patch.

Thanks for your patience!


> v4 => v5:
> - pci_requester_id() moved back to architecture-independent PCI code.
> - Use pci_requester_id() in more places.
> 
> v3 => v4:
> - Rename msi_requester_id() to pci_requester_id() and move from MSI to PCI
>   code
> 
> v2 => v3:
> - Use official "requester ID" denotion everywhere. Renamed getter function
>   and MemTxAttrs field.
> 
> v1 => v2:
> - Improved commit messages, more explanations
> - msi_device_id() function made architecture-specific
> 
> Pavel Fedin (3):
>   kvm: Make KVM_CAP_SIGNAL_MSI globally available
>   hw/pci: Introduce pci_requester_id()
>   kvm: Pass PCI device pointer to MSI routing functions
> 
>  hw/i386/kvm/pci-assign.c | 11 ++++++-----
>  hw/pci/msi.c             |  2 +-
>  hw/pci/pcie_aer.c        |  2 +-
>  hw/vfio/pci.c            | 11 ++++++-----
>  hw/virtio/virtio-pci.c   |  5 +++--
>  include/exec/memattrs.h  |  4 ++--
>  include/hw/pci/pci.h     |  5 +++++
>  include/sysemu/kvm.h     | 14 ++++++++++++--
>  kvm-all.c                | 15 ++++++++-------
>  kvm-stub.c               |  5 +++--
>  10 files changed, 47 insertions(+), 27 deletions(-)
> 
> -- 
> 2.4.4