[PULL 0/3] Xen queue 2024-03-12

[PULL 0/3] Xen queue 2024-03-12

[Anthony PERARD]

The following changes since commit 8f3f329f5e0117bd1a23a79ab751f8a7d3471e4b:

  Merge tag 'migration-20240311-pull-request' of https://gitlab.com/peterx/qemu into staging (2024-03-12 11:35:41 +0000)

are available in the Git repository at:

  https://xenbits.xen.org/git-http/people/aperard/qemu-dm.git tags/pull-xen-20240312

for you to fetch changes up to 918a7f706b69a8c725bac0694971d2831f688ebb:

  i386: load kernel on xen using DMA (2024-03-12 14:13:08 +0000)

----------------------------------------------------------------
Xen queue:

* In Xen PCI passthrough, emulate multifunction bit.
* Fix in Xen mapcache.
* Improve performance of kernel+initrd loading in an Xen HVM Direct
  Kernel Boot scenario.

----------------------------------------------------------------
Marek Marczykowski-Górecki (1):
      i386: load kernel on xen using DMA

Peng Fan (1):
      xen: Drop out of coroutine context xen_invalidate_map_cache_entry

Ross Lagerwall (1):
      xen/pt: Emulate multifunction bit in header type

 hw/i386/pc.c                |  3 ++-
 hw/xen/xen-mapcache.c       | 30 ++++++++++++++++++++++++++++--
 hw/xen/xen_pt_config_init.c |  7 +++++--
 3 files changed, 35 insertions(+), 5 deletions(-)

[PULL 1/3] xen/pt: Emulate multifunction bit in header type

[Anthony PERARD]

From: Ross Lagerwall <ross.lagerwall@citrix.com>

The intention of the code appears to have been to unconditionally set
the multifunction bit but since the emulation mask is 0x00 it has no
effect. Instead, emulate the bit and set it based on the multifunction
property of the PCIDevice (which can be set using QAPI).

This allows making passthrough devices appear as functions in a Xen
guest.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Message-Id: <20231103172601.1319375-1-ross.lagerwall@citrix.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 hw/xen/xen_pt_config_init.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c
index ba4cd78238..3edaeab1e3 100644
--- a/hw/xen/xen_pt_config_init.c
+++ b/hw/xen/xen_pt_config_init.c
@@ -292,7 +292,10 @@ static int xen_pt_header_type_reg_init(XenPCIPassthroughState *s,
                                        uint32_t *data)
 {
     /* read PCI_HEADER_TYPE */
-    *data = reg->init_val | 0x80;
+    *data = reg->init_val;
+    if ((PCI_DEVICE(s)->cap_present & QEMU_PCI_CAP_MULTIFUNCTION)) {
+        *data |= PCI_HEADER_TYPE_MULTI_FUNCTION;
+    }
     return 0;
 }
 
@@ -677,7 +680,7 @@ static XenPTRegInfo xen_pt_emu_reg_header0[] = {
         .size       = 1,
         .init_val   = 0x00,
         .ro_mask    = 0xFF,
-        .emu_mask   = 0x00,
+        .emu_mask   = PCI_HEADER_TYPE_MULTI_FUNCTION,
         .init       = xen_pt_header_type_reg_init,
         .u.b.read   = xen_pt_byte_reg_read,
         .u.b.write  = xen_pt_byte_reg_write,
-- 
Anthony PERARD

[PULL 2/3] xen: Drop out of coroutine context xen_invalidate_map_cache_entry

[Anthony PERARD]

From: Peng Fan <peng.fan@nxp.com>

xen_invalidate_map_cache_entry is not expected to run in a
coroutine. Without this, there is crash:

    signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
    threadid=<optimized out>) at pthread_kill.c:78
    at /usr/src/debug/glibc/2.38+git-r0/sysdeps/posix/raise.c:26
    fmt=0xffff9e1ca8a8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0xaaaae0d25740 "!qemu_in_coroutine()",
    file=file@entry=0xaaaae0d301a8 "../qemu-xen-dir-remote/block/graph-lock.c", line=line@entry=260,
    function=function@entry=0xaaaae0e522c0 <__PRETTY_FUNCTION__.3> "bdrv_graph_rdlock_main_loop") at assert.c:92
    assertion=assertion@entry=0xaaaae0d25740 "!qemu_in_coroutine()",
    file=file@entry=0xaaaae0d301a8 "../qemu-xen-dir-remote/block/graph-lock.c", line=line@entry=260,
    function=function@entry=0xaaaae0e522c0 <__PRETTY_FUNCTION__.3> "bdrv_graph_rdlock_main_loop") at assert.c:101
    at ../qemu-xen-dir-remote/block/graph-lock.c:260
    at /home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/include/block/graph-lock.h:259
    host=host@entry=0xffff742c8000, size=size@entry=2097152)
    at ../qemu-xen-dir-remote/block/io.c:3362
    host=0xffff742c8000, size=2097152)
    at ../qemu-xen-dir-remote/block/block-backend.c:2859
    host=<optimized out>, size=<optimized out>, max_size=<optimized out>)
    at ../qemu-xen-dir-remote/block/block-ram-registrar.c:33
    size=2097152, max_size=2097152)
    at ../qemu-xen-dir-remote/hw/core/numa.c:883
    buffer=buffer@entry=0xffff743c5000 "")
    at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:475
    buffer=buffer@entry=0xffff743c5000 "")
    at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:487
    as=as@entry=0xaaaae1ca3ae8 <address_space_memory>, buffer=0xffff743c5000,
    len=<optimized out>, is_write=is_write@entry=true,
    access_len=access_len@entry=32768)
    at ../qemu-xen-dir-remote/system/physmem.c:3199
    dir=DMA_DIRECTION_FROM_DEVICE, len=<optimized out>,
    buffer=<optimized out>, as=0xaaaae1ca3ae8 <address_space_memory>)
    at /home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/include/sysemu/dma.h:236
    elem=elem@entry=0xaaaaf620aa30, len=len@entry=32769)
    at ../qemu-xen-dir-remote/hw/virtio/virtio.c:758
    elem=elem@entry=0xaaaaf620aa30, len=len@entry=32769, idx=idx@entry=0)
    at ../qemu-xen-dir-remote/hw/virtio/virtio.c:919
    elem=elem@entry=0xaaaaf620aa30, len=32769)
    at ../qemu-xen-dir-remote/hw/virtio/virtio.c:994
    req=req@entry=0xaaaaf620aa30, status=status@entry=0 '\000')
    at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:67
    ret=0) at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:136
    at ../qemu-xen-dir-remote/block/block-backend.c:1559
--Type <RET> for more, q to quit, c to continue without paging--
    at ../qemu-xen-dir-remote/block/block-backend.c:1614
    i1=<optimized out>) at ../qemu-xen-dir-remote/util/coroutine-ucontext.c:177
    at ../sysdeps/unix/sysv/linux/aarch64/setcontext.S:123

Signed-off-by: Peng Fan <peng.fan@nxp.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Message-Id: <20240124021450.21656-1-peng.fan@oss.nxp.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 hw/xen/xen-mapcache.c | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index 4f956d048e..7f59080ba7 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -476,11 +476,37 @@ static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
     g_free(entry);
 }
 
-void xen_invalidate_map_cache_entry(uint8_t *buffer)
+typedef struct XenMapCacheData {
+    Coroutine *co;
+    uint8_t *buffer;
+} XenMapCacheData;
+
+static void xen_invalidate_map_cache_entry_bh(void *opaque)
 {
+    XenMapCacheData *data = opaque;
+
     mapcache_lock();
-    xen_invalidate_map_cache_entry_unlocked(buffer);
+    xen_invalidate_map_cache_entry_unlocked(data->buffer);
     mapcache_unlock();
+
+    aio_co_wake(data->co);
+}
+
+void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
+{
+    if (qemu_in_coroutine()) {
+        XenMapCacheData data = {
+            .co = qemu_coroutine_self(),
+            .buffer = buffer,
+        };
+        aio_bh_schedule_oneshot(qemu_get_current_aio_context(),
+                                xen_invalidate_map_cache_entry_bh, &data);
+        qemu_coroutine_yield();
+    } else {
+        mapcache_lock();
+        xen_invalidate_map_cache_entry_unlocked(buffer);
+        mapcache_unlock();
+    }
 }
 
 void xen_invalidate_map_cache(void)
-- 
Anthony PERARD

Re: [PULL 2/3] xen: Drop out of coroutine context xen_invalidate_map_cache_entry

[Michael Tokarev]

12.03.2024 17:27, Anthony PERARD wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> xen_invalidate_map_cache_entry is not expected to run in a
> coroutine. Without this, there is crash:

Hi!  Is this a stable material? (It applies cleanly and builds on 8.2 and 7.2)

Thanks,

/mjt

Re: [PULL 2/3] xen: Drop out of coroutine context xen_invalidate_map_cache_entry

[Michael Tokarev]

13.03.2024 20:21, Michael Tokarev:
> 12.03.2024 17:27, Anthony PERARD wrote:
>> From: Peng Fan <peng.fan@nxp.com>
>>
>> xen_invalidate_map_cache_entry is not expected to run in a
>> coroutine. Without this, there is crash:
> 
> Hi!  Is this a stable material? (It applies cleanly and builds on 8.2 and 7.2)

Actually for 7.2 it needed a minor tweak:

-void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
+void xen_invalidate_map_cache_entry(uint8_t *buffer)

but the rest is okay.

/mjt

RE: [PULL 2/3] xen: Drop out of coroutine context xen_invalidate_map_cache_entry

[Peng Fan]

> Subject: Re: [PULL 2/3] xen: Drop out of coroutine context
> xen_invalidate_map_cache_entry
> 
> 13.03.2024 20:21, Michael Tokarev:
> > 12.03.2024 17:27, Anthony PERARD wrote:
> >> From: Peng Fan <peng.fan@nxp.com>
> >>
> >> xen_invalidate_map_cache_entry is not expected to run in a coroutine.
> >> Without this, there is crash:
> >
> > Hi!  Is this a stable material? (It applies cleanly and builds on 8.2
> > and 7.2)
> 
> Actually for 7.2 it needed a minor tweak:
> 
> -void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
> +void xen_invalidate_map_cache_entry(uint8_t *buffer)

I only tested 8.2 with xen virtio enabled. Not sure whether 7.2 has the issue
or not.

Thanks,
Peng.

> 
> but the rest is okay.
> 
> /mjt

[PULL 3/3] i386: load kernel on xen using DMA

[Anthony PERARD]

From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Kernel on Xen is loaded via fw_cfg. Previously it used non-DMA version,
which loaded the kernel (and initramfs) byte by byte. Change this
to DMA, to load in bigger chunks.
This change alone reduces load time of a (big) kernel+initramfs from
~10s down to below 1s.

This change was suggested initially here:
https://lore.kernel.org/xen-devel/20180216204031.000052e9@gmail.com/
Apparently this alone is already enough to get massive speedup.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Message-Id: <20210426034709.595432-1-marmarek@invisiblethingslab.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 hw/i386/pc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index f5ff970acf..4f322e0856 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -718,7 +718,8 @@ void xen_load_linux(PCMachineState *pcms)
 
     assert(MACHINE(pcms)->kernel_filename != NULL);
 
-    fw_cfg = fw_cfg_init_io(FW_CFG_IO_BASE);
+    fw_cfg = fw_cfg_init_io_dma(FW_CFG_IO_BASE, FW_CFG_IO_BASE + 4,
+                                &address_space_memory);
     fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, x86ms->boot_cpus);
     rom_set_fw(fw_cfg);
 
-- 
Anthony PERARD

Re: [PULL 0/3] Xen queue 2024-03-12

[Peter Maydell]

On Tue, 12 Mar 2024 at 14:29, Anthony PERARD <anthony.perard@citrix.com> wrote:
>
> The following changes since commit 8f3f329f5e0117bd1a23a79ab751f8a7d3471e4b:
>
>   Merge tag 'migration-20240311-pull-request' of https://gitlab.com/peterx/qemu into staging (2024-03-12 11:35:41 +0000)
>
> are available in the Git repository at:
>
>   https://xenbits.xen.org/git-http/people/aperard/qemu-dm.git tags/pull-xen-20240312
>
> for you to fetch changes up to 918a7f706b69a8c725bac0694971d2831f688ebb:
>
>   i386: load kernel on xen using DMA (2024-03-12 14:13:08 +0000)
>
> ----------------------------------------------------------------
> Xen queue:
>
> * In Xen PCI passthrough, emulate multifunction bit.
> * Fix in Xen mapcache.
> * Improve performance of kernel+initrd loading in an Xen HVM Direct
>   Kernel Boot scenario.



Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/9.0
for any user-visible changes.

-- PMM