[PULL v2 00/18] Migration 20241030 patches

[PULL v2 00/18] Migration 20241030 patches

[Peter Xu]

The following changes since commit ea8ae47bdd2024dc2596f16b27f27fd4dcc08776:

  Merge tag 'pull-target-arm-20241029' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2024-10-31 13:28:57 +0000)

are available in the Git repository at:

  https://gitlab.com/peterx/qemu.git tags/migration-20241030-pull-request

for you to fetch changes up to 00b4b216534d84ace7b0583cec70a3aaf256cb25:

  migration/multifd: Zero p->flags before starting filling a packet (2024-10-31 15:48:18 -0400)

----------------------------------------------------------------
Migration pull request for softfreeze

v2:
- Patch "migration: Move cpu-throttle.c from system to migration",
  fix build on MacOS, and subject spelling

NOTE: checkpatch.pl could report a false positive on this branch:

  WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
  #21:
   {include/sysemu => migration}/cpu-throttle.h | 0

That's covered by "F: migration/" entry.

Changelog:

- Peter's cleanup patch on migrate_fd_cleanup()
- Peter's cleanup patch to introduce thread name macros
- Hanna's error path fix for vmstate subsection save()s
- Hyman's auto converge enhancement on background dirty sync
- Peter's additional tracepoints for save state entries
- Thomas's build fix for OpenBSD in dirtyrate.c
- Peter's deprecation of query-migrationthreads command
- Peter's cleanup/fixes from the "export misc.h" series
- Maciej's two small patches from multifd+vfio series

----------------------------------------------------------------

Hanna Czenczek (1):
  migration: Ensure vmstate_save() sets errp

Hyman Huang (6):
  accel/tcg/icount-common: Remove the reference to the unused header
    file
  migration: Stop CPU throttling conditionally
  migration: Move cpu-throttle.c from system to migration
  migration: Remove "rs" parameter in migration_bitmap_sync_precopy
  migration: Support periodic RAMBlock dirty bitmap sync
  tests/migration: Add case for periodic ramblock dirty sync

Maciej S. Szmigiero (2):
  migration/ram: Add load start trace event
  migration/multifd: Zero p->flags before starting filling a packet

Peter Xu (8):
  migration: Cleanup migrate_fd_cleanup() on accessing to_dst_file
  migration: Put thread names together with macros
  migration: Deprecate query-migrationthreads command
  migration: Take migration object refcount earlier for threads
  migration: Unexport dirty_bitmap_mig_init()
  migration: Unexport ram_mig_init()
  migration: Drop migration_is_setup_or_active()
  migration: Drop migration_is_idle()

Thomas Huth (1):
  migration/dirtyrate: Silence warning about strcpy() on OpenBSD

 docs/about/deprecated.rst            |   8 ++
 qapi/migration.json                  |   7 +-
 include/migration/misc.h             |   9 +-
 include/sysemu/cpu-throttle.h        |  14 +++
 migration/migration.h                |  19 ++++
 migration/ram.h                      |   1 +
 accel/tcg/icount-common.c            |   1 -
 hw/vfio/common.c                     |   2 +-
 hw/virtio/virtio-mem.c               |   2 +-
 migration/colo.c                     |   3 +-
 {system => migration}/cpu-throttle.c |  70 ++++++++++++++-
 migration/dirtyrate.c                |  11 ++-
 migration/migration.c                | 128 +++++++++++----------------
 migration/multifd.c                  |   8 +-
 migration/postcopy-ram.c             |   6 +-
 migration/ram.c                      |  19 ++--
 migration/savevm.c                   |   3 +-
 migration/vmstate.c                  |  13 +--
 net/vhost-vdpa.c                     |   3 +-
 system/cpu-timers.c                  |   3 -
 system/qdev-monitor.c                |   4 +-
 tests/qtest/migration-test.c         |  32 +++++++
 migration/meson.build                |   1 +
 migration/trace-events               |   5 ++
 system/meson.build                   |   1 -
 system/trace-events                  |   3 -
 26 files changed, 254 insertions(+), 122 deletions(-)
 rename {system => migration}/cpu-throttle.c (65%)

-- 
2.45.0

[PULL v2 06/18] migration: Move cpu-throttle.c from system to migration

[Peter Xu]

From: Hyman Huang <yong.huang@smartx.com>

Move cpu-throttle.c from system to migration since it's
only used for migration; this makes us avoid exporting the
util functions and variables in misc.h but export them in
migration.h when implementing the periodic ramblock dirty
sync feature in the upcoming commits.

Since CPU throttle timers are only used in migration, move
their registry to migration_object_init.

Signed-off-by: Hyman Huang <yong.huang@smartx.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/c1b3efaa0cb49e03d422e9da97bdb65cc3d234d1.1729146786.git.yong.huang@smartx.com
[peterx: Fix build on MacOS on cocoa.m, not move cpu-throttle.h yet]
[peterx: Fix subject spelling, per pm215]
Signed-off-by: Peter Xu <peterx@redhat.com>
---
 {system => migration}/cpu-throttle.c | 0
 migration/migration.c                | 3 +++
 system/cpu-timers.c                  | 3 ---
 migration/meson.build                | 1 +
 migration/trace-events               | 3 +++
 system/meson.build                   | 1 -
 system/trace-events                  | 3 ---
 7 files changed, 7 insertions(+), 7 deletions(-)
 rename {system => migration}/cpu-throttle.c (100%)

diff --git a/system/cpu-throttle.c b/migration/cpu-throttle.c
similarity index 100%
rename from system/cpu-throttle.c
rename to migration/cpu-throttle.c
diff --git a/migration/migration.c b/migration/migration.c
index e81c70b9d2..64c1d0c1db 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -263,6 +263,9 @@ void migration_object_init(void)
 
     ram_mig_init();
     dirty_bitmap_mig_init();
+
+    /* Initialize cpu throttle timers */
+    cpu_throttle_init();
 }
 
 typedef struct {
diff --git a/system/cpu-timers.c b/system/cpu-timers.c
index 0b31c9a1b6..856e502e34 100644
--- a/system/cpu-timers.c
+++ b/system/cpu-timers.c
@@ -35,7 +35,6 @@
 #include "sysemu/runstate.h"
 #include "hw/core/cpu.h"
 #include "sysemu/cpu-timers.h"
-#include "sysemu/cpu-throttle.h"
 #include "sysemu/cpu-timers-internal.h"
 
 /* clock and ticks */
@@ -272,6 +271,4 @@ void cpu_timers_init(void)
     seqlock_init(&timers_state.vm_clock_seqlock);
     qemu_spin_init(&timers_state.vm_clock_lock);
     vmstate_register(NULL, 0, &vmstate_timers, &timers_state);
-
-    cpu_throttle_init();
 }
diff --git a/migration/meson.build b/migration/meson.build
index 66d3de86f0..d53cf3417a 100644
--- a/migration/meson.build
+++ b/migration/meson.build
@@ -13,6 +13,7 @@ system_ss.add(files(
   'block-dirty-bitmap.c',
   'channel.c',
   'channel-block.c',
+  'cpu-throttle.c',
   'dirtyrate.c',
   'exec.c',
   'fd.c',
diff --git a/migration/trace-events b/migration/trace-events
index c65902f042..9a19599804 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -378,3 +378,6 @@ migration_block_progression(unsigned percent) "Completed %u%%"
 # page_cache.c
 migration_pagecache_init(int64_t max_num_items) "Setting cache buckets to %" PRId64
 migration_pagecache_insert(void) "Error allocating page"
+
+# cpu-throttle.c
+cpu_throttle_set(int new_throttle_pct)  "set guest CPU throttled by %d%%"
diff --git a/system/meson.build b/system/meson.build
index a296270cb0..4952f4b2c7 100644
--- a/system/meson.build
+++ b/system/meson.build
@@ -10,7 +10,6 @@ system_ss.add(files(
   'balloon.c',
   'bootdevice.c',
   'cpus.c',
-  'cpu-throttle.c',
   'cpu-timers.c',
   'datadir.c',
   'dirtylimit.c',
diff --git a/system/trace-events b/system/trace-events
index 074d001e90..2ed1d59b1f 100644
--- a/system/trace-events
+++ b/system/trace-events
@@ -44,6 +44,3 @@ dirtylimit_state_finalize(void)
 dirtylimit_throttle_pct(int cpu_index, uint64_t pct, int64_t time_us) "CPU[%d] throttle percent: %" PRIu64 ", throttle adjust time %"PRIi64 " us"
 dirtylimit_set_vcpu(int cpu_index, uint64_t quota) "CPU[%d] set dirty page rate limit %"PRIu64
 dirtylimit_vcpu_execute(int cpu_index, int64_t sleep_time_us) "CPU[%d] sleep %"PRIi64 " us"
-
-# cpu-throttle.c
-cpu_throttle_set(int new_throttle_pct)  "set guest CPU throttled by %d%%"
-- 
2.45.0

Re: [PULL v2 00/18] Migration 20241030 patches

[Peter Maydell]

On Fri, 1 Nov 2024 at 13:46, Peter Xu <peterx@redhat.com> wrote:
>
> The following changes since commit ea8ae47bdd2024dc2596f16b27f27fd4dcc08776:
>
>   Merge tag 'pull-target-arm-20241029' of https://git.linaro.org/people/pmaydell/qemu-arm into staging (2024-10-31 13:28:57 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/peterx/qemu.git tags/migration-20241030-pull-request
>
> for you to fetch changes up to 00b4b216534d84ace7b0583cec70a3aaf256cb25:
>
>   migration/multifd: Zero p->flags before starting filling a packet (2024-10-31 15:48:18 -0400)
>
> ----------------------------------------------------------------
> Migration pull request for softfreeze
>
> v2:
> - Patch "migration: Move cpu-throttle.c from system to migration",
>   fix build on MacOS, and subject spelling
>
> NOTE: checkpatch.pl could report a false positive on this branch:
>
>   WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
>   #21:
>    {include/sysemu => migration}/cpu-throttle.h | 0
>
> That's covered by "F: migration/" entry.
>
> Changelog:
>
> - Peter's cleanup patch on migrate_fd_cleanup()
> - Peter's cleanup patch to introduce thread name macros
> - Hanna's error path fix for vmstate subsection save()s
> - Hyman's auto converge enhancement on background dirty sync
> - Peter's additional tracepoints for save state entries
> - Thomas's build fix for OpenBSD in dirtyrate.c
> - Peter's deprecation of query-migrationthreads command
> - Peter's cleanup/fixes from the "export misc.h" series
> - Maciej's two small patches from multifd+vfio series
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/9.2
for any user-visible changes.

-- PMM