* [PATCH] introduce dirty ring size for guestperf tool @ 2022-03-03 10:45 huangy81 2022-03-03 10:45 ` [PATCH] migration: Introduce dirtylimit capability huangy81 0 siblings, 1 reply; 3+ messages in thread From: huangy81 @ 2022-03-03 10:45 UTC (permalink / raw) To: qemu-devel Cc: Hyman Huang, Daniel P. Berrangé, Dr. David Alan Gilbert, Juan Quintela From: Hyman Huang(黄勇) <huangy81@chinatelecom.cn> Dirtylimit implementation has been reviewed in the past few months, if things go well, it will be merged in the near future, which is the first step to implement a new live migration feature. For more details refer to: https://lore.kernel.org/qemu-devel/cover.1646247968.git.huangy81@chinatelecom.cn/ The second step is to implement the "dirtylimit" capability of live migration basing on the implementation in first step, and the main logic is almost done, refer to: https://github.com/newfriday/qemu/commits/migration_dirtylimit_v1 Since "dirtylimit" capability of live migration is basing on dirty ring, so if we want compare live migration used "dirtylimit" capability with other capabilities such as "auto-converge" in performance, set dirty ring size when start vm using guestperf tool is an convenient way. So let's introduce dirty ring size for guestperf tool. Hyman Huang (1): tests/migration: Introduce dirty-ring-size option into guestperf tests/migration/guestperf/engine.py | 6 +++++- tests/migration/guestperf/hardware.py | 8 ++++++-- tests/migration/guestperf/shell.py | 6 +++++- 3 files changed, 16 insertions(+), 4 deletions(-) -- 1.8.3.1 ^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH] migration: Introduce dirtylimit capability 2022-03-03 10:45 [PATCH] introduce dirty ring size for guestperf tool huangy81 @ 2022-03-03 10:45 ` huangy81 2022-03-03 10:49 ` Hyman Huang 0 siblings, 1 reply; 3+ messages in thread From: huangy81 @ 2022-03-03 10:45 UTC (permalink / raw) To: qemu-devel Cc: Hyman Huang, Daniel P. Berrangé, Dr. David Alan Gilbert, Juan Quintela From: Hyman Huang(黄勇) <huangy81@chinatelecom.cn> Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn> --- migration/migration.c | 19 +++++++++++++++++++ migration/migration.h | 1 + migration/ram.c | 24 ++++++++++++++++++++++++ qapi/migration.json | 36 +++++++++++++++++++++++++++++++----- 4 files changed, 75 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index bcc385b..0b89468 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -116,6 +116,9 @@ #define DEFAULT_MIGRATE_ANNOUNCE_ROUNDS 5 #define DEFAULT_MIGRATE_ANNOUNCE_STEP 100 +#define DEFAULT_MIGRATE_MIN_DIRTYLIMIT_RATE 1 +#define DEFAULT_MIGRATE_DIRTYLIMIT_STEP_SIZE 10 + static NotifierList migration_state_notifiers = NOTIFIER_LIST_INITIALIZER(migration_state_notifiers); @@ -2397,6 +2400,15 @@ bool migrate_auto_converge(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_AUTO_CONVERGE]; } +bool migrate_dirtylimit(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTYLIMIT]; +} + bool migrate_zero_blocks(void) { MigrationState *s; @@ -4215,6 +4227,12 @@ static Property migration_properties[] = { DEFINE_PROP_SIZE("announce-step", MigrationState, parameters.announce_step, DEFAULT_MIGRATE_ANNOUNCE_STEP), + DEFINE_PROP_UINT64("min-dirtylimit-rate", MigrationState, + parameters.min_dirtylimit_rate, + DEFAULT_MIGRATE_ANNOUNCE_STEP), + DEFINE_PROP_UINT64("dirtylimit-step-size", MigrationState, + parameters.dirtylimit_step_size, + DEFAULT_MIGRATE_ANNOUNCE_STEP), /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -4231,6 +4249,7 @@ static Property migration_properties[] = { DEFINE_PROP_MIG_CAP("x-multifd", MIGRATION_CAPABILITY_MULTIFD), DEFINE_PROP_MIG_CAP("x-background-snapshot", MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT), + DEFINE_PROP_MIG_CAP("x-dirtylimit", MIGRATION_CAPABILITY_DIRTYLIMIT), DEFINE_PROP_END_OF_LIST(), }; diff --git a/migration/migration.h b/migration/migration.h index 8130b70..f2bf226 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -332,6 +332,7 @@ bool migrate_ignore_shared(void); bool migrate_validate_uuid(void); bool migrate_auto_converge(void); +bool migrate_dirtylimit(void); bool migrate_use_multifd(void); bool migrate_pause_before_switchover(void); int migrate_multifd_channels(void); diff --git a/migration/ram.c b/migration/ram.c index 781f074..cf18536 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -56,6 +56,7 @@ #include "qemu/iov.h" #include "multifd.h" #include "sysemu/runstate.h" +#include "sysemu/dirtylimit.h" #include "hw/boards.h" /* for machine_dump_guest_core() */ @@ -1082,6 +1083,27 @@ static void migration_update_rates(RAMState *rs, int64_t end_time) } } +/** + * mig_dirtylimit_guest_down: enable dirtylimit to throttle down the guest + */ +static void mig_dirtylimit_guest_down(void) +{ + MigrationState *s = migrate_get_current(); + uint64_t current_dirtyrate = s->mbps / 8; + uint64_t min_dirtyrate = s->parameters.min_dirtylimit_rate; + uint64_t step_size = s->parameters.dirtylimit_step_size; + static uint64_t quota_dirtyrate; + + if (!dirtylimit_in_service()) { + quota_dirtyrate = MAX(current_dirtyrate, min_dirtyrate); + dirtylimit_set_all(quota_dirtyrate, true); + } else { + quota_dirtyrate -= step_size; + quota_dirtyrate = MAX(current_dirtyrate, min_dirtyrate); + dirtylimit_set_all(quota_dirtyrate, true); + } +} + static void migration_trigger_throttle(RAMState *rs) { MigrationState *s = migrate_get_current(); @@ -1108,6 +1130,8 @@ static void migration_trigger_throttle(RAMState *rs) mig_throttle_guest_down(bytes_dirty_period, bytes_dirty_threshold); } + } else if (migrate_dirtylimit() && kvm_dirty_ring_enabled()) { + mig_dirtylimit_guest_down(); } } diff --git a/qapi/migration.json b/qapi/migration.json index 2ccbb92..e074dc8 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -463,6 +463,9 @@ # procedure starts. The VM RAM is saved with running VM. # (since 6.0) # +# @dirtylimit: Use dirtylimit to throttle down guest if enabled. +# (since 6.1) +# # Features: # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -476,7 +479,7 @@ 'block', 'return-path', 'pause-before-switchover', 'multifd', 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, - 'validate-uuid', 'background-snapshot'] } + 'validate-uuid', 'background-snapshot', 'dirtylimit'] } ## # @MigrationCapabilityStatus: @@ -760,6 +763,12 @@ # block device name if there is one, and to their node name # otherwise. (Since 5.2) # +# @min-dirtylimit-rate: minimum qouta for limiting dirty page rate. +# Defaults to 1. (Since 6.1) +# +# @dirtylimit-step-size: step size (MB/s) for limiting dirty page rate. +# Defaults to 10. (Since 6.1) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -779,8 +788,9 @@ 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-compression', - 'multifd-zlib-level' ,'multifd-zstd-level', - 'block-bitmap-mapping' ] } + 'multifd-zlib-level', 'multifd-zstd-level', + 'block-bitmap-mapping', 'min-dirtylimit-rate', + 'dirtylimit-step-size' ] } ## # @MigrateSetParameters: @@ -925,6 +935,12 @@ # block device name if there is one, and to their node name # otherwise. (Since 5.2) # +# @min-dirtylimit-rate: minimum qouta for limiting dirty page rate. +# Defaults to 1. (Since 6.1) +# +# @dirtylimit-step-size: step size (MB/s) for limiting dirty page rate. +# Defaults to 10. (Since 6.1) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -960,7 +976,9 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', - '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], + '*min-dirtylimit-rate': 'uint64', + '*dirtylimit-step-size': 'uint64' } } ## # @migrate-set-parameters: @@ -1125,6 +1143,12 @@ # block device name if there is one, and to their node name # otherwise. (Since 5.2) # +# @min-dirtylimit-rate: minimum qouta for limiting dirty page rate. +# Defaults to 1. (Since 6.1) +# +# @dirtylimit-step-size: step size (MB/s) for limiting dirty page rate. +# Defaults to 10. (Since 6.1) +# # Features: # @unstable: Member @x-checkpoint-delay is experimental. # @@ -1158,7 +1182,9 @@ '*multifd-compression': 'MultiFDCompression', '*multifd-zlib-level': 'uint8', '*multifd-zstd-level': 'uint8', - '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], + '*min-dirtylimit-rate': 'uint64', + '*dirtylimit-step-size': 'uint64' } } ## # @query-migrate-parameters: -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] migration: Introduce dirtylimit capability 2022-03-03 10:45 ` [PATCH] migration: Introduce dirtylimit capability huangy81 @ 2022-03-03 10:49 ` Hyman Huang 0 siblings, 0 replies; 3+ messages in thread From: Hyman Huang @ 2022-03-03 10:49 UTC (permalink / raw) To: qemu-devel; +Cc: Daniel P. Berrangé, Dr. David Alan Gilbert, Juan Quintela "Sent by accident, please ignore, I'll send v1 when ready." 在 2022/3/3 18:45, huangy81@chinatelecom.cn 写道: > From: Hyman Huang(黄勇) <huangy81@chinatelecom.cn> > > Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn> > --- > migration/migration.c | 19 +++++++++++++++++++ > migration/migration.h | 1 + > migration/ram.c | 24 ++++++++++++++++++++++++ > qapi/migration.json | 36 +++++++++++++++++++++++++++++++----- > 4 files changed, 75 insertions(+), 5 deletions(-) > > diff --git a/migration/migration.c b/migration/migration.c > index bcc385b..0b89468 100644 > --- a/migration/migration.c > +++ b/migration/migration.c > @@ -116,6 +116,9 @@ > #define DEFAULT_MIGRATE_ANNOUNCE_ROUNDS 5 > #define DEFAULT_MIGRATE_ANNOUNCE_STEP 100 > > +#define DEFAULT_MIGRATE_MIN_DIRTYLIMIT_RATE 1 > +#define DEFAULT_MIGRATE_DIRTYLIMIT_STEP_SIZE 10 > + > static NotifierList migration_state_notifiers = > NOTIFIER_LIST_INITIALIZER(migration_state_notifiers); > > @@ -2397,6 +2400,15 @@ bool migrate_auto_converge(void) > return s->enabled_capabilities[MIGRATION_CAPABILITY_AUTO_CONVERGE]; > } > > +bool migrate_dirtylimit(void) > +{ > + MigrationState *s; > + > + s = migrate_get_current(); > + > + return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTYLIMIT]; > +} > + > bool migrate_zero_blocks(void) > { > MigrationState *s; > @@ -4215,6 +4227,12 @@ static Property migration_properties[] = { > DEFINE_PROP_SIZE("announce-step", MigrationState, > parameters.announce_step, > DEFAULT_MIGRATE_ANNOUNCE_STEP), > + DEFINE_PROP_UINT64("min-dirtylimit-rate", MigrationState, > + parameters.min_dirtylimit_rate, > + DEFAULT_MIGRATE_ANNOUNCE_STEP), > + DEFINE_PROP_UINT64("dirtylimit-step-size", MigrationState, > + parameters.dirtylimit_step_size, > + DEFAULT_MIGRATE_ANNOUNCE_STEP), > > /* Migration capabilities */ > DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), > @@ -4231,6 +4249,7 @@ static Property migration_properties[] = { > DEFINE_PROP_MIG_CAP("x-multifd", MIGRATION_CAPABILITY_MULTIFD), > DEFINE_PROP_MIG_CAP("x-background-snapshot", > MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT), > + DEFINE_PROP_MIG_CAP("x-dirtylimit", MIGRATION_CAPABILITY_DIRTYLIMIT), > > DEFINE_PROP_END_OF_LIST(), > }; > diff --git a/migration/migration.h b/migration/migration.h > index 8130b70..f2bf226 100644 > --- a/migration/migration.h > +++ b/migration/migration.h > @@ -332,6 +332,7 @@ bool migrate_ignore_shared(void); > bool migrate_validate_uuid(void); > > bool migrate_auto_converge(void); > +bool migrate_dirtylimit(void); > bool migrate_use_multifd(void); > bool migrate_pause_before_switchover(void); > int migrate_multifd_channels(void); > diff --git a/migration/ram.c b/migration/ram.c > index 781f074..cf18536 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -56,6 +56,7 @@ > #include "qemu/iov.h" > #include "multifd.h" > #include "sysemu/runstate.h" > +#include "sysemu/dirtylimit.h" > > #include "hw/boards.h" /* for machine_dump_guest_core() */ > > @@ -1082,6 +1083,27 @@ static void migration_update_rates(RAMState *rs, int64_t end_time) > } > } > > +/** > + * mig_dirtylimit_guest_down: enable dirtylimit to throttle down the guest > + */ > +static void mig_dirtylimit_guest_down(void) > +{ > + MigrationState *s = migrate_get_current(); > + uint64_t current_dirtyrate = s->mbps / 8; > + uint64_t min_dirtyrate = s->parameters.min_dirtylimit_rate; > + uint64_t step_size = s->parameters.dirtylimit_step_size; > + static uint64_t quota_dirtyrate; > + > + if (!dirtylimit_in_service()) { > + quota_dirtyrate = MAX(current_dirtyrate, min_dirtyrate); > + dirtylimit_set_all(quota_dirtyrate, true); > + } else { > + quota_dirtyrate -= step_size; > + quota_dirtyrate = MAX(current_dirtyrate, min_dirtyrate); > + dirtylimit_set_all(quota_dirtyrate, true); > + } > +} > + > static void migration_trigger_throttle(RAMState *rs) > { > MigrationState *s = migrate_get_current(); > @@ -1108,6 +1130,8 @@ static void migration_trigger_throttle(RAMState *rs) > mig_throttle_guest_down(bytes_dirty_period, > bytes_dirty_threshold); > } > + } else if (migrate_dirtylimit() && kvm_dirty_ring_enabled()) { > + mig_dirtylimit_guest_down(); > } > } > > diff --git a/qapi/migration.json b/qapi/migration.json > index 2ccbb92..e074dc8 100644 > --- a/qapi/migration.json > +++ b/qapi/migration.json > @@ -463,6 +463,9 @@ > # procedure starts. The VM RAM is saved with running VM. > # (since 6.0) > # > +# @dirtylimit: Use dirtylimit to throttle down guest if enabled. > +# (since 6.1) > +# > # Features: > # @unstable: Members @x-colo and @x-ignore-shared are experimental. > # > @@ -476,7 +479,7 @@ > 'block', 'return-path', 'pause-before-switchover', 'multifd', > 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', > { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, > - 'validate-uuid', 'background-snapshot'] } > + 'validate-uuid', 'background-snapshot', 'dirtylimit'] } > > ## > # @MigrationCapabilityStatus: > @@ -760,6 +763,12 @@ > # block device name if there is one, and to their node name > # otherwise. (Since 5.2) > # > +# @min-dirtylimit-rate: minimum qouta for limiting dirty page rate. > +# Defaults to 1. (Since 6.1) > +# > +# @dirtylimit-step-size: step size (MB/s) for limiting dirty page rate. > +# Defaults to 10. (Since 6.1) > +# > # Features: > # @unstable: Member @x-checkpoint-delay is experimental. > # > @@ -779,8 +788,9 @@ > 'multifd-channels', > 'xbzrle-cache-size', 'max-postcopy-bandwidth', > 'max-cpu-throttle', 'multifd-compression', > - 'multifd-zlib-level' ,'multifd-zstd-level', > - 'block-bitmap-mapping' ] } > + 'multifd-zlib-level', 'multifd-zstd-level', > + 'block-bitmap-mapping', 'min-dirtylimit-rate', > + 'dirtylimit-step-size' ] } > > ## > # @MigrateSetParameters: > @@ -925,6 +935,12 @@ > # block device name if there is one, and to their node name > # otherwise. (Since 5.2) > # > +# @min-dirtylimit-rate: minimum qouta for limiting dirty page rate. > +# Defaults to 1. (Since 6.1) > +# > +# @dirtylimit-step-size: step size (MB/s) for limiting dirty page rate. > +# Defaults to 10. (Since 6.1) > +# > # Features: > # @unstable: Member @x-checkpoint-delay is experimental. > # > @@ -960,7 +976,9 @@ > '*multifd-compression': 'MultiFDCompression', > '*multifd-zlib-level': 'uint8', > '*multifd-zstd-level': 'uint8', > - '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } > + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], > + '*min-dirtylimit-rate': 'uint64', > + '*dirtylimit-step-size': 'uint64' } } > > ## > # @migrate-set-parameters: > @@ -1125,6 +1143,12 @@ > # block device name if there is one, and to their node name > # otherwise. (Since 5.2) > # > +# @min-dirtylimit-rate: minimum qouta for limiting dirty page rate. > +# Defaults to 1. (Since 6.1) > +# > +# @dirtylimit-step-size: step size (MB/s) for limiting dirty page rate. > +# Defaults to 10. (Since 6.1) > +# > # Features: > # @unstable: Member @x-checkpoint-delay is experimental. > # > @@ -1158,7 +1182,9 @@ > '*multifd-compression': 'MultiFDCompression', > '*multifd-zlib-level': 'uint8', > '*multifd-zstd-level': 'uint8', > - '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ] } } > + '*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ], > + '*min-dirtylimit-rate': 'uint64', > + '*dirtylimit-step-size': 'uint64' } } > > ## > # @query-migrate-parameters: -- Best regard Hyman Huang(黄勇) ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-03-03 10:51 UTC | newest] Thread overview: 3+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-03-03 10:45 [PATCH] introduce dirty ring size for guestperf tool huangy81 2022-03-03 10:45 ` [PATCH] migration: Introduce dirtylimit capability huangy81 2022-03-03 10:49 ` Hyman Huang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).