Re: [PULL 00/38] Migration 20231016 patches

[Juan Quintela <quintela@redhat.com>]

Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Mon, 16 Oct 2023 at 06:11, Juan Quintela <quintela@redhat.com> wrote:
>>
>> The following changes since commit 63011373ad22c794a013da69663c03f1297a5c56:
>>
>>   Merge tag 'pull-riscv-to-apply-20231012-1' of https://github.com/alistair23/qemu into staging (2023-10-12 10:24:44 -0400)
>>
>> are available in the Git repository at:
>>
>>   https://gitlab.com/juan.quintela/qemu.git tags/migration-20231016-pull-request
>>
>> for you to fetch changes up to f39b0f42753635b0f2d8b00a26d11bb197bf51e2:
>>
>>   migration/multifd: Clarify Error usage in multifd_channel_connect (2023-10-16 11:01:33 +0200)
>>
>> ----------------------------------------------------------------
>> Migration Pull request (20231016)
>>
>> In this pull request:
>> - rdma cleanups
>> - removal of QEMUFileHook
>> - test for analyze-migration.py
>> - test for multifd file
>> - multifd cleanups
>> - available switchover bandwidth
>> - lots of cleanups.
>>
>> CI: https://gitlab.com/juan.quintela/qemu/-/pipelines/1037878829
>>
>> Please, apply.
>
> This CI failure looks migration-related:

It is.

> MALLOC_PERTURB_=96
> PYTHON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/pyvenv/bin/python3
> QTEST_QEMU_BINARY=./qemu-system-i386
> G_TEST_DBUS_DAEMON=/home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_IMG=./qemu-img
> QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon
> /home/gitlab-runner/builds/-LCfcJ2T/0/qemu-project/qemu/build/tests/qtest/migration-test
> --tap -k
> ――――――――――――――――――――――――――――――――――――― ✀ ―――――――――――――――――――――――――――――――――――――
> stderr:
> **
> ERROR:../tests/qtest/migration-test.c:1969:file_offset_finish_hook:
> assertion failed (cpu_to_be32(*p) == QEMU_VM_FILE_MAGIC): (3 ==
> 1363498573)
>
> https://gitlab.com/qemu-project/qemu/-/jobs/5301793548

But what I am doing wrong here?
This time I even posnted the CI link that I passed O:-)

I thought that if I pass the:

$ git push -o ci.variable=QEMU_CI=2

on my branch, I was doing all the testing that you are doing.  Clearly
not.

Thanks, Juan.


> Stefan
>
>>
>> ----------------------------------------------------------------
>>
>> Dmitry Frolov (1):
>>   migration: fix RAMBlock add NULL check
>>
>> Elena Ufimtseva (3):
>>   migration: check for rate_limit_max for RATE_LIMIT_DISABLED
>>   multifd: fix counters in multifd_send_thread
>>   multifd: reset next_packet_len after sending pages
>>
>> Fabiano Rosas (13):
>>   migration: Fix analyze-migration.py 'configuration' parsing
>>   migration: Add capability parsing to analyze-migration.py
>>   migration: Fix analyze-migration.py when ignore-shared is used
>>   migration: Fix analyze-migration read operation signedness
>>   tests/qtest/migration: Add a test for the analyze-migration script
>>   tests/qtest: migration-test: Add tests for file-based migration
>>   migration/ram: Remove RAMState from xbzrle_cache_zero_page
>>   migration/ram: Stop passing QEMUFile around in save_zero_page
>>   migration/ram: Move xbzrle zero page handling into save_zero_page
>>   migration/ram: Merge save_zero_page functions
>>   migration/multifd: Remove direct "socket" references
>>   migration/multifd: Unify multifd_send_thread error paths
>>   migration/multifd: Clarify Error usage in multifd_channel_connect
>>
>> Fiona Ebner (1):
>>   migration: hold the BQL during setup
>>
>> Juan Quintela (15):
>>   migration: Non multifd migration don't care about multifd flushes
>>   migration: Create migrate_rdma()
>>   migration/rdma: Unfold ram_control_before_iterate()
>>   migration/rdma: Unfold ram_control_after_iterate()
>>   migration/rdma: Remove all uses of RAM_CONTROL_HOOK
>>   migration/rdma: Unfold hook_ram_load()
>>   migration/rdma: Create rdma_control_save_page()
>>   qemu-file: Remove QEMUFileHooks
>>   migration/rdma: Move rdma constants from qemu-file.h to rdma.h
>>   migration/rdma: Remove qemu_ prefix from exported functions
>>   migration/rdma: Check sooner if we are in postcopy for save_page()
>>   migration/rdma: Use i as for index instead of idx
>>   migration/rdma: Declare for index variables local
>>   migration/rdma: Remove all "ret" variables that are used only once
>>   migration: Improve json and formatting
>>
>> Nikolay Borisov (2):
>>   migration: Add the configuration vmstate to the json writer
>>   migration/ram: Refactor precopy ram loading code
>>
>> Peter Xu (1):
>>   migration: Allow user to specify available switchover bandwidth
>>
>> Philippe Mathieu-Daudé (1):
>>   migration: Use g_autofree to simplify ram_dirty_bitmap_reload()
>>
>> Wei Wang (1):
>>   migration: refactor migration_completion
>>
>>  qapi/migration.json            |  41 ++++-
>>  include/migration/register.h   |   2 +-
>>  migration/migration.h          |   4 +-
>>  migration/options.h            |   2 +
>>  migration/qemu-file.h          |  49 ------
>>  migration/rdma.h               |  42 +++++
>>  migration/block-dirty-bitmap.c |   3 -
>>  migration/block.c              |   5 -
>>  migration/migration-hmp-cmds.c |  14 ++
>>  migration/migration-stats.c    |   9 +-
>>  migration/migration.c          | 199 +++++++++++++--------
>>  migration/multifd.c            | 101 +++++------
>>  migration/options.c            |  35 ++++
>>  migration/qemu-file.c          |  61 +------
>>  migration/ram.c                | 306 ++++++++++++++++++---------------
>>  migration/rdma.c               | 259 ++++++++++++----------------
>>  migration/savevm.c             |  22 ++-
>>  tests/qtest/migration-test.c   | 207 ++++++++++++++++++++++
>>  migration/trace-events         |  33 ++--
>>  scripts/analyze-migration.py   |  67 +++++++-
>>  tests/qtest/meson.build        |   2 +
>>  21 files changed, 895 insertions(+), 568 deletions(-)
>>
>> --
>> 2.41.0
>>
>>