[Qemu-devel] [PULL v2 0/8] vfio pull request

[Qemu-devel] [PULL v2 0/8] vfio pull request

[Alex Williamson]

Anthony,

My last vfio pull request didn't seem to make the most recent round of
merges.  The only difference in this request is trivial patch 8/8, the
rest is a resend.  I've not rebased in order to maintain the commit
IDs from my previous tag, the merge to current HEAD is still clean.
Please pull.  Thanks,

Alex

The following changes since commit 1cf892ca2689c84960b4ce4d2723b6bee453711c:

  SPARC: Fix LEON3 power down instruction (2014-01-15 15:37:33 +1000)

are available in the git repository at:

  git://github.com/awilliam/qemu-vfio.git tags/vfio-pci-for-qemu-20140128.0

for you to fetch changes up to 8b6d14087d487203f4d1a67aeaddc3be6c73f49f:

  vfio: correct debug macro typo (2014-01-28 08:23:19 -0700)

----------------------------------------------------------------
vfio-pci updates include:
 - Destroy MemoryRegions on device teardown
 - Print warnings around PCI option ROM failures
 - Skip bogus mappings from 64bit BAR sizing
 - Act on DMA mapping failures
 - Fix alignment to avoid MSI-X table mapping
 - Fix debug macro typo

----------------------------------------------------------------
Alex Williamson (3):
      vfio: Destroy memory regions
      vfio: Filter out bogus mappings
      vfio-pci: Fail initfn on DMA mapping errors

Alexey Kardashevskiy (2):
      kvm: initialize qemu_host_page_size
      vfio: fix mapping of MSIX bar

Bandan Das (3):
      vfio: warn if host device rom can't be read
      vfio: Do not reattempt a failed rom read
      vfio: correct debug macro typo

 hw/misc/vfio.c          | 78 ++++++++++++++++++++++++++++++++++++++++++-------
 include/exec/exec-all.h |  1 +
 kvm-all.c               |  1 +
 translate-all.c         | 14 +++++----
 4 files changed, 77 insertions(+), 17 deletions(-)

[Qemu-devel] [PULL v2 1/8] vfio: Destroy memory regions

[Alex Williamson]

Somehow this has been lurking for a while; we remove our subregions
from the base BAR and VGA region mappings, but we don't destroy them,
creating a leak and more serious problems when we try to migrate after
removing these devices.  Add the trivial bit of final cleanup to
remove these entirely.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/misc/vfio.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c
index 9aecaa8..ec9f41b 100644
--- a/hw/misc/vfio.c
+++ b/hw/misc/vfio.c
@@ -1968,6 +1968,7 @@ static void vfio_vga_quirk_teardown(VFIODevice *vdev)
         while (!QLIST_EMPTY(&vdev->vga.region[i].quirks)) {
             VFIOQuirk *quirk = QLIST_FIRST(&vdev->vga.region[i].quirks);
             memory_region_del_subregion(&vdev->vga.region[i].mem, &quirk->mem);
+            memory_region_destroy(&quirk->mem);
             QLIST_REMOVE(quirk, next);
             g_free(quirk);
         }
@@ -1990,6 +1991,7 @@ static void vfio_bar_quirk_teardown(VFIODevice *vdev, int nr)
     while (!QLIST_EMPTY(&bar->quirks)) {
         VFIOQuirk *quirk = QLIST_FIRST(&bar->quirks);
         memory_region_del_subregion(&bar->mem, &quirk->mem);
+        memory_region_destroy(&quirk->mem);
         QLIST_REMOVE(quirk, next);
         g_free(quirk);
     }
@@ -2412,10 +2414,12 @@ static void vfio_unmap_bar(VFIODevice *vdev, int nr)
 
     memory_region_del_subregion(&bar->mem, &bar->mmap_mem);
     munmap(bar->mmap, memory_region_size(&bar->mmap_mem));
+    memory_region_destroy(&bar->mmap_mem);
 
     if (vdev->msix && vdev->msix->table_bar == nr) {
         memory_region_del_subregion(&bar->mem, &vdev->msix->mmap_mem);
         munmap(vdev->msix->mmap, memory_region_size(&vdev->msix->mmap_mem));
+        memory_region_destroy(&vdev->msix->mmap_mem);
     }
 
     memory_region_destroy(&bar->mem);

[Qemu-devel] [PULL v2 2/8] vfio: warn if host device rom can't be read

[Alex Williamson]

From: Bandan Das <bsd@redhat.com>

If the device rom can't be read, report an error to the
user. This alerts the user that the device has a bad
state that is causing rom read failure or option rom
loading has been disabled from the device boot menu
(among other reasons).

Signed-off-by: Bandan Das <bsd@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/misc/vfio.c |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c
index ec9f41b..ef615fc 100644
--- a/hw/misc/vfio.c
+++ b/hw/misc/vfio.c
@@ -1125,6 +1125,13 @@ static void vfio_pci_load_rom(VFIODevice *vdev)
     vdev->rom_offset = reg_info.offset;
 
     if (!vdev->rom_size) {
+        error_report("vfio-pci: Cannot read device rom at "
+                    "%04x:%02x:%02x.%x\n",
+                    vdev->host.domain, vdev->host.bus, vdev->host.slot,
+                    vdev->host.function);
+        error_printf("Device option ROM contents are probably invalid "
+                    "(check dmesg).\nSkip option ROM probe with rombar=0, "
+                    "or load from file with romfile=\n");
         return;
     }
 

[Qemu-devel] [PULL v2 3/8] vfio: Do not reattempt a failed rom read

[Alex Williamson]

From: Bandan Das <bsd@redhat.com>

During lazy rom loading, if rom read fails, and the
guest attempts a read again, vfio will again attempt it.
Add a boolean to prevent this. There could be a case where
a failed rom read might succeed the next time because of
a device reset or such, but it's best to exclude unpredictable
behavior

Signed-off-by: Bandan Das <bsd@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
 hw/misc/vfio.c |    6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c
index ef615fc..30b1a78 100644
--- a/hw/misc/vfio.c
+++ b/hw/misc/vfio.c
@@ -191,6 +191,7 @@ typedef struct VFIODevice {
     bool has_flr;
     bool has_pm_reset;
     bool needs_reset;
+    bool rom_read_failed;
 } VFIODevice;
 
 typedef struct VFIOGroup {
@@ -1125,6 +1126,7 @@ static void vfio_pci_load_rom(VFIODevice *vdev)
     vdev->rom_offset = reg_info.offset;
 
     if (!vdev->rom_size) {
+        vdev->rom_read_failed = true;
         error_report("vfio-pci: Cannot read device rom at "
                     "%04x:%02x:%02x.%x\n",
                     vdev->host.domain, vdev->host.bus, vdev->host.slot,
@@ -1163,6 +1165,9 @@ static uint64_t vfio_rom_read(void *opaque, hwaddr addr, unsigned size)
     /* Load the ROM lazily when the guest tries to read it */
     if (unlikely(!vdev->rom)) {
         vfio_pci_load_rom(vdev);
+        if (unlikely(!vdev->rom && !vdev->rom_read_failed)) {
+            vfio_pci_load_rom(vdev);
+        }
     }
 
     memcpy(&val, vdev->rom + addr,
@@ -1230,6 +1235,7 @@ static void vfio_pci_size_rom(VFIODevice *vdev)
                      PCI_BASE_ADDRESS_SPACE_MEMORY, &vdev->pdev.rom);
 
     vdev->pdev.has_rom = true;
+    vdev->rom_read_failed = false;
 }
 
 static void vfio_vga_write(void *opaque, hwaddr addr,

[Qemu-devel] [PULL v2 4/8] vfio: Filter out bogus mappings

[Alex Williamson]

Since 57271d63 we now see spurious mappings with the upper bits set
if 64bit PCI BARs are sized while enabled.  The guest writes a mask
of 0xffffffff to the lower BAR to size it, then restores it, then
writes the same mask to the upper BAR resulting in a spurious BAR
mapping into the last 4G of the 64bit address space.  Most
architectures do not support or make use of the full 64bits address
space for PCI BARs, so we filter out mappings with the high bit set.
Long term, we probably need to think about vfio telling us the
address width limitations of the IOMMU.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/misc/vfio.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c
index 30b1a78..d304213 100644
--- a/hw/misc/vfio.c
+++ b/hw/misc/vfio.c
@@ -2156,7 +2156,14 @@ static int vfio_dma_map(VFIOContainer *container, hwaddr iova,
 
 static bool vfio_listener_skipped_section(MemoryRegionSection *section)
 {
-    return !memory_region_is_ram(section->mr);
+    return !memory_region_is_ram(section->mr) ||
+           /*
+            * Sizing an enabled 64-bit BAR can cause spurious mappings to
+            * addresses in the upper part of the 64-bit address space.  These
+            * are never accessed by the CPU and beyond the address width of
+            * some IOMMU hardware.  TODO: VFIO should tell us the IOMMU width.
+            */
+           section->offset_within_address_space & (1ULL << 63);
 }
 
 static void vfio_listener_region_add(MemoryListener *listener,

Re: [Qemu-devel] [PULL v2 0/8] vfio pull request

[Alex Williamson]

I guess I should be addressing these to both Anthony and Peter now.
Thanks for stepping in, Peter.  Thanks,

Alex

On Tue, 2014-01-28 at 08:44 -0700, Alex Williamson wrote:
> Anthony,
> 
> My last vfio pull request didn't seem to make the most recent round of
> merges.  The only difference in this request is trivial patch 8/8, the
> rest is a resend.  I've not rebased in order to maintain the commit
> IDs from my previous tag, the merge to current HEAD is still clean.
> Please pull.  Thanks,
> 
> Alex
> 
> The following changes since commit 1cf892ca2689c84960b4ce4d2723b6bee453711c:
> 
>   SPARC: Fix LEON3 power down instruction (2014-01-15 15:37:33 +1000)
> 
> are available in the git repository at:
> 
>   git://github.com/awilliam/qemu-vfio.git tags/vfio-pci-for-qemu-20140128.0
> 
> for you to fetch changes up to 8b6d14087d487203f4d1a67aeaddc3be6c73f49f:
> 
>   vfio: correct debug macro typo (2014-01-28 08:23:19 -0700)
> 
> ----------------------------------------------------------------
> vfio-pci updates include:
>  - Destroy MemoryRegions on device teardown
>  - Print warnings around PCI option ROM failures
>  - Skip bogus mappings from 64bit BAR sizing
>  - Act on DMA mapping failures
>  - Fix alignment to avoid MSI-X table mapping
>  - Fix debug macro typo
> 
> ----------------------------------------------------------------
> Alex Williamson (3):
>       vfio: Destroy memory regions
>       vfio: Filter out bogus mappings
>       vfio-pci: Fail initfn on DMA mapping errors
> 
> Alexey Kardashevskiy (2):
>       kvm: initialize qemu_host_page_size
>       vfio: fix mapping of MSIX bar
> 
> Bandan Das (3):
>       vfio: warn if host device rom can't be read
>       vfio: Do not reattempt a failed rom read
>       vfio: correct debug macro typo
> 
>  hw/misc/vfio.c          | 78 ++++++++++++++++++++++++++++++++++++++++++-------
>  include/exec/exec-all.h |  1 +
>  kvm-all.c               |  1 +
>  translate-all.c         | 14 +++++----
>  4 files changed, 77 insertions(+), 17 deletions(-)

Re: [Qemu-devel] [PULL v2 0/8] vfio pull request

[Peter Maydell]

Applied, thanks.

For the record, it doesn't matter if you cc me or not, because I've set my
mail client to look for the magic words "for you to fetch changes up to" :-)

thanks
-- PMM

On 28 January 2014 15:58, Alex Williamson <alex.williamson@redhat.com> wrote:
>
> I guess I should be addressing these to both Anthony and Peter now.
> Thanks for stepping in, Peter.  Thanks,
>
> Alex
>
> On Tue, 2014-01-28 at 08:44 -0700, Alex Williamson wrote:
>> Anthony,
>>
>> My last vfio pull request didn't seem to make the most recent round of
>> merges.  The only difference in this request is trivial patch 8/8, the
>> rest is a resend.  I've not rebased in order to maintain the commit
>> IDs from my previous tag, the merge to current HEAD is still clean.
>> Please pull.  Thanks,
>>
>> Alex
>>
>> The following changes since commit 1cf892ca2689c84960b4ce4d2723b6bee453711c:
>>
>>   SPARC: Fix LEON3 power down instruction (2014-01-15 15:37:33 +1000)
>>
>> are available in the git repository at:
>>
>>   git://github.com/awilliam/qemu-vfio.git tags/vfio-pci-for-qemu-20140128.0
>>
>> for you to fetch changes up to 8b6d14087d487203f4d1a67aeaddc3be6c73f49f:
>>
>>   vfio: correct debug macro typo (2014-01-28 08:23:19 -0700)
>>
>> ----------------------------------------------------------------
>> vfio-pci updates include:
>>  - Destroy MemoryRegions on device teardown
>>  - Print warnings around PCI option ROM failures
>>  - Skip bogus mappings from 64bit BAR sizing
>>  - Act on DMA mapping failures
>>  - Fix alignment to avoid MSI-X table mapping
>>  - Fix debug macro typo
>>
>> ----------------------------------------------------------------
>> Alex Williamson (3):
>>       vfio: Destroy memory regions
>>       vfio: Filter out bogus mappings
>>       vfio-pci: Fail initfn on DMA mapping errors
>>
>> Alexey Kardashevskiy (2):
>>       kvm: initialize qemu_host_page_size
>>       vfio: fix mapping of MSIX bar
>>
>> Bandan Das (3):
>>       vfio: warn if host device rom can't be read
>>       vfio: Do not reattempt a failed rom read
>>       vfio: correct debug macro typo
>>
>>  hw/misc/vfio.c          | 78 ++++++++++++++++++++++++++++++++++++++++++-------
>>  include/exec/exec-all.h |  1 +
>>  kvm-all.c               |  1 +
>>  translate-all.c         | 14 +++++----
>>  4 files changed, 77 insertions(+), 17 deletions(-)