[PATCH v12 0/6] VT-d Device-TLB flush issue

["Xu, Quan" <quan.xu@intel.com>]

From: Quan Xu <quan.xu@intel.com>

This patches fix current timeout concern and also allow limited ATS support:

1. add a timeout parameter for device IOTLB invalidation

    The parameter 'iommu_dev_iotlb_timeout' specifies the timeout
    of device IOTLB invalidation in milliseconds. By default, the
    timeout is 1000 milliseconds, which can be boot-time changed.

    We also confirmed with VT-d hardware team that 1 milliseconds
    is large enough for VT-d IOMMU internal invalidation.

    the existing panic() is eliminated and we bubble up the timeout
    of device IOTLB invalidation for further processing, as the
    PCI-e Address Translation Services (ATS) mandates a timeout of
    60 seconds for device IOTLB invalidation. Obviously we can't
    spin for 60 seconds or otherwise Xen hypervisor hangs.

2. today we do Device-TLB flush synchronization after issuing flush
    requests for all ATS devices belonging to a VM. Doing so however
    imposes a limitation, i.e. that we can not figure out which flush
    request is blocked in the flush queue list, based on VT-d spec.

    To prepare correct Device-TLB flush timeout handling in next patch,
    we change the behavior to synchronize for every Device-TLB flush
    request. So the Device-TLB flush interface is changed a little bit,
    by checking timeout within the function instead of outside of function.

    Accordingly we also do a similar change for flush interfaces of
    IOTLB/IEC/Context, i.e. moving synchronization into the function.
    Since there is no user of a non-synced interface, we just rename
    existing ones with _sync suffix.

3. move the domain crash logic up to the generic IOMMU layer

4. If Device-TLB flush timed out, we hide the target ATS device
    immediately and crash the domain owning this ATS device. If
    impacted domain is hardware domain, just throw out a warning.

    By hiding the device, we make sure it can't be assigned to any
    domain any longer (see device_assigned).

---
Not covered in this series:

    a) Eliminate the panic() in IOMMU_WAIT_OP, used only in VT-d register read/write.
       Further discussion is required on whether and how to improve it.
    b) Handle IOTLB/Context/IEC flush timeout (after v12 review, I try to send out
       brain storming).
---
Quan Xu (6):
  IOMMU: add a timeout parameter for device IOTLB invalidation
  vt-d: synchronize for Device-TLB flush one by one
  vt-d: convert conditionals of qi_ctrl->qinval_maddr into ASSERT()s
  IOMMU/x86: using a struct pci_dev* instead of SBDF
  IOMMU: move the domain crash logic up to the generic IOMMU layer
  vt-d: fix vt-d Device-TLB flush timeout issue

 docs/misc/xen-command-line.markdown         |   9 ++
 xen/drivers/passthrough/amd/iommu_cmd.c     |  11 +-
 xen/drivers/passthrough/amd/pci_amd_iommu.c |   2 +-
 xen/drivers/passthrough/ats.h               |   6 +-
 xen/drivers/passthrough/iommu.c             |  33 +++++-
 xen/drivers/passthrough/pci.c               |   6 +-
 xen/drivers/passthrough/vtd/extern.h        |   6 +-
 xen/drivers/passthrough/vtd/iommu.c         |  19 +++-
 xen/drivers/passthrough/vtd/qinval.c        | 168 +++++++++++++++++++---------
 xen/drivers/passthrough/vtd/x86/ats.c       |  23 ++--
 xen/drivers/passthrough/x86/ats.c           |  52 ++++++---
 xen/include/xen/iommu.h                     |   2 +
 xen/include/xen/pci.h                       |   1 +
 13 files changed, 238 insertions(+), 100 deletions(-)

-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel