* [PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment
2024-08-09 7:41 [PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment yong.huang
@ 2024-08-09 7:43 ` yong.huang
2024-08-09 7:43 ` [PATCH v1 1/5] tests/migration: Move the guestperf tool to scripts directory yong.huang
` (4 subsequent siblings)
5 siblings, 0 replies; 12+ messages in thread
From: yong.huang @ 2024-08-09 7:43 UTC (permalink / raw)
To: qemu-devel; +Cc: Fabiano Rosas, Peter Xu, yong.huang
From: Hyman Huang <yong.huang@smartx.com>
The previous patchset:
https://lore.kernel.org/qemu-devel/cover.1722957352.git.yong.huang@smartx.com/
does not made the necessary changes and tests for the upstream version.
This patchset works for that:
1. Move the guestperf to scripts directory suggested by Fabiano Rosas
2. Make initrd-stress.img built by default suggested by Fabiano Rosas
3. Make the necessary changes to adapt the latest multifd behavior
4. A nitpick for multifd migration
5. Support multifd compression option
Please review,
Thanks
Yong
Hyman Huang (5):
tests/migration: Move the guestperf tool to scripts directory
tests/migration: Make initrd-stress.img built by default
guestperf: Support deferred migration for multifd
guestperf: Nitpick the inconsistent parameters
guestperf: Introduce multifd compression option
MAINTAINERS | 5 ++++
{tests => scripts}/migration/guestperf-batch.py | 0
{tests => scripts}/migration/guestperf-plot.py | 0
{tests => scripts}/migration/guestperf.py | 0
{tests => scripts}/migration/guestperf/__init__.py | 0
.../migration/guestperf/comparison.py | 15 +++++++++-
{tests => scripts}/migration/guestperf/engine.py | 32 +++++++++++++++++++---
{tests => scripts}/migration/guestperf/hardware.py | 0
{tests => scripts}/migration/guestperf/plot.py | 0
{tests => scripts}/migration/guestperf/progress.py | 0
{tests => scripts}/migration/guestperf/report.py | 0
{tests => scripts}/migration/guestperf/scenario.py | 7 +++--
{tests => scripts}/migration/guestperf/shell.py | 3 ++
{tests => scripts}/migration/guestperf/timings.py | 0
tests/migration/meson.build | 5 ++--
15 files changed, 58 insertions(+), 9 deletions(-)
rename {tests => scripts}/migration/guestperf-batch.py (100%)
rename {tests => scripts}/migration/guestperf-plot.py (100%)
rename {tests => scripts}/migration/guestperf.py (100%)
rename {tests => scripts}/migration/guestperf/__init__.py (100%)
rename {tests => scripts}/migration/guestperf/comparison.py (89%)
rename {tests => scripts}/migration/guestperf/engine.py (93%)
rename {tests => scripts}/migration/guestperf/hardware.py (100%)
rename {tests => scripts}/migration/guestperf/plot.py (100%)
rename {tests => scripts}/migration/guestperf/progress.py (100%)
rename {tests => scripts}/migration/guestperf/report.py (100%)
rename {tests => scripts}/migration/guestperf/scenario.py (93%)
rename {tests => scripts}/migration/guestperf/shell.py (98%)
rename {tests => scripts}/migration/guestperf/timings.py (100%)
--
2.11.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH v1 1/5] tests/migration: Move the guestperf tool to scripts directory
2024-08-09 7:41 [PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment yong.huang
2024-08-09 7:43 ` yong.huang
@ 2024-08-09 7:43 ` yong.huang
2024-08-09 12:57 ` Fabiano Rosas
2024-08-09 7:43 ` [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default yong.huang
` (3 subsequent siblings)
5 siblings, 1 reply; 12+ messages in thread
From: yong.huang @ 2024-08-09 7:43 UTC (permalink / raw)
To: qemu-devel; +Cc: Fabiano Rosas, Peter Xu, yong.huang
From: Hyman Huang <yong.huang@smartx.com>
Guestperf was designed to test the performance of migration,
with a loose connection to the fundamental test cases of QEMU.
To improve the repository's structure, move it to the scripts
directory.
Signed-off-by: Hyman Huang <yong.huang@smartx.com>
---
MAINTAINERS | 5 +++++
{tests => scripts}/migration/guestperf-batch.py | 0
{tests => scripts}/migration/guestperf-plot.py | 0
{tests => scripts}/migration/guestperf.py | 0
{tests => scripts}/migration/guestperf/__init__.py | 0
{tests => scripts}/migration/guestperf/comparison.py | 0
{tests => scripts}/migration/guestperf/engine.py | 0
{tests => scripts}/migration/guestperf/hardware.py | 0
{tests => scripts}/migration/guestperf/plot.py | 0
{tests => scripts}/migration/guestperf/progress.py | 0
{tests => scripts}/migration/guestperf/report.py | 0
{tests => scripts}/migration/guestperf/scenario.py | 0
{tests => scripts}/migration/guestperf/shell.py | 0
{tests => scripts}/migration/guestperf/timings.py | 0
14 files changed, 5 insertions(+)
rename {tests => scripts}/migration/guestperf-batch.py (100%)
rename {tests => scripts}/migration/guestperf-plot.py (100%)
rename {tests => scripts}/migration/guestperf.py (100%)
rename {tests => scripts}/migration/guestperf/__init__.py (100%)
rename {tests => scripts}/migration/guestperf/comparison.py (100%)
rename {tests => scripts}/migration/guestperf/engine.py (100%)
rename {tests => scripts}/migration/guestperf/hardware.py (100%)
rename {tests => scripts}/migration/guestperf/plot.py (100%)
rename {tests => scripts}/migration/guestperf/progress.py (100%)
rename {tests => scripts}/migration/guestperf/report.py (100%)
rename {tests => scripts}/migration/guestperf/scenario.py (100%)
rename {tests => scripts}/migration/guestperf/shell.py (100%)
rename {tests => scripts}/migration/guestperf/timings.py (100%)
diff --git a/MAINTAINERS b/MAINTAINERS
index 10af212632..dae1535dae 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3449,6 +3449,11 @@ F: migration/dirtyrate.h
F: include/sysemu/dirtyrate.h
F: docs/devel/migration/dirty-limit.rst
+Migration performance test tool
+M: Hyman Huang <yong.huang@smartx.com>
+S: Maintained
+F: scripts/migration/
+
Detached LUKS header
M: Hyman Huang <yong.huang@smartx.com>
S: Maintained
diff --git a/tests/migration/guestperf-batch.py b/scripts/migration/guestperf-batch.py
similarity index 100%
rename from tests/migration/guestperf-batch.py
rename to scripts/migration/guestperf-batch.py
diff --git a/tests/migration/guestperf-plot.py b/scripts/migration/guestperf-plot.py
similarity index 100%
rename from tests/migration/guestperf-plot.py
rename to scripts/migration/guestperf-plot.py
diff --git a/tests/migration/guestperf.py b/scripts/migration/guestperf.py
similarity index 100%
rename from tests/migration/guestperf.py
rename to scripts/migration/guestperf.py
diff --git a/tests/migration/guestperf/__init__.py b/scripts/migration/guestperf/__init__.py
similarity index 100%
rename from tests/migration/guestperf/__init__.py
rename to scripts/migration/guestperf/__init__.py
diff --git a/tests/migration/guestperf/comparison.py b/scripts/migration/guestperf/comparison.py
similarity index 100%
rename from tests/migration/guestperf/comparison.py
rename to scripts/migration/guestperf/comparison.py
diff --git a/tests/migration/guestperf/engine.py b/scripts/migration/guestperf/engine.py
similarity index 100%
rename from tests/migration/guestperf/engine.py
rename to scripts/migration/guestperf/engine.py
diff --git a/tests/migration/guestperf/hardware.py b/scripts/migration/guestperf/hardware.py
similarity index 100%
rename from tests/migration/guestperf/hardware.py
rename to scripts/migration/guestperf/hardware.py
diff --git a/tests/migration/guestperf/plot.py b/scripts/migration/guestperf/plot.py
similarity index 100%
rename from tests/migration/guestperf/plot.py
rename to scripts/migration/guestperf/plot.py
diff --git a/tests/migration/guestperf/progress.py b/scripts/migration/guestperf/progress.py
similarity index 100%
rename from tests/migration/guestperf/progress.py
rename to scripts/migration/guestperf/progress.py
diff --git a/tests/migration/guestperf/report.py b/scripts/migration/guestperf/report.py
similarity index 100%
rename from tests/migration/guestperf/report.py
rename to scripts/migration/guestperf/report.py
diff --git a/tests/migration/guestperf/scenario.py b/scripts/migration/guestperf/scenario.py
similarity index 100%
rename from tests/migration/guestperf/scenario.py
rename to scripts/migration/guestperf/scenario.py
diff --git a/tests/migration/guestperf/shell.py b/scripts/migration/guestperf/shell.py
similarity index 100%
rename from tests/migration/guestperf/shell.py
rename to scripts/migration/guestperf/shell.py
diff --git a/tests/migration/guestperf/timings.py b/scripts/migration/guestperf/timings.py
similarity index 100%
rename from tests/migration/guestperf/timings.py
rename to scripts/migration/guestperf/timings.py
--
2.11.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v1 1/5] tests/migration: Move the guestperf tool to scripts directory
2024-08-09 7:43 ` [PATCH v1 1/5] tests/migration: Move the guestperf tool to scripts directory yong.huang
@ 2024-08-09 12:57 ` Fabiano Rosas
2024-08-10 2:32 ` Hyman Huang
0 siblings, 1 reply; 12+ messages in thread
From: Fabiano Rosas @ 2024-08-09 12:57 UTC (permalink / raw)
To: yong.huang, qemu-devel; +Cc: Peter Xu, yong.huang
yong.huang@smartx.com writes:
> From: Hyman Huang <yong.huang@smartx.com>
>
> Guestperf was designed to test the performance of migration,
> with a loose connection to the fundamental test cases of QEMU.
>
> To improve the repository's structure, move it to the scripts
> directory.
>
> Signed-off-by: Hyman Huang <yong.huang@smartx.com>
> ---
> MAINTAINERS | 5 +++++
> {tests => scripts}/migration/guestperf-batch.py | 0
> {tests => scripts}/migration/guestperf-plot.py | 0
> {tests => scripts}/migration/guestperf.py | 0
> {tests => scripts}/migration/guestperf/__init__.py | 0
> {tests => scripts}/migration/guestperf/comparison.py | 0
> {tests => scripts}/migration/guestperf/engine.py | 0
> {tests => scripts}/migration/guestperf/hardware.py | 0
> {tests => scripts}/migration/guestperf/plot.py | 0
> {tests => scripts}/migration/guestperf/progress.py | 0
> {tests => scripts}/migration/guestperf/report.py | 0
> {tests => scripts}/migration/guestperf/scenario.py | 0
> {tests => scripts}/migration/guestperf/shell.py | 0
> {tests => scripts}/migration/guestperf/timings.py | 0
> 14 files changed, 5 insertions(+)
> rename {tests => scripts}/migration/guestperf-batch.py (100%)
> rename {tests => scripts}/migration/guestperf-plot.py (100%)
> rename {tests => scripts}/migration/guestperf.py (100%)
> rename {tests => scripts}/migration/guestperf/__init__.py (100%)
> rename {tests => scripts}/migration/guestperf/comparison.py (100%)
> rename {tests => scripts}/migration/guestperf/engine.py (100%)
> rename {tests => scripts}/migration/guestperf/hardware.py (100%)
> rename {tests => scripts}/migration/guestperf/plot.py (100%)
> rename {tests => scripts}/migration/guestperf/progress.py (100%)
> rename {tests => scripts}/migration/guestperf/report.py (100%)
> rename {tests => scripts}/migration/guestperf/scenario.py (100%)
> rename {tests => scripts}/migration/guestperf/shell.py (100%)
> rename {tests => scripts}/migration/guestperf/timings.py (100%)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 10af212632..dae1535dae 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3449,6 +3449,11 @@ F: migration/dirtyrate.h
> F: include/sysemu/dirtyrate.h
> F: docs/devel/migration/dirty-limit.rst
>
> +Migration performance test tool
> +M: Hyman Huang <yong.huang@smartx.com>
> +S: Maintained
> +F: scripts/migration/
This probably needs to be scripts/migration/guestperf* unless of course
you intend to maintain all scripts that show up under
scripts/migration/. We have analyze-migration.py and
vmstate-static-checker.py which are candidates to be put in that same
directory.
> +
> Detached LUKS header
> M: Hyman Huang <yong.huang@smartx.com>
> S: Maintained
> diff --git a/tests/migration/guestperf-batch.py b/scripts/migration/guestperf-batch.py
> similarity index 100%
> rename from tests/migration/guestperf-batch.py
> rename to scripts/migration/guestperf-batch.py
> diff --git a/tests/migration/guestperf-plot.py b/scripts/migration/guestperf-plot.py
> similarity index 100%
> rename from tests/migration/guestperf-plot.py
> rename to scripts/migration/guestperf-plot.py
> diff --git a/tests/migration/guestperf.py b/scripts/migration/guestperf.py
> similarity index 100%
> rename from tests/migration/guestperf.py
> rename to scripts/migration/guestperf.py
> diff --git a/tests/migration/guestperf/__init__.py b/scripts/migration/guestperf/__init__.py
> similarity index 100%
> rename from tests/migration/guestperf/__init__.py
> rename to scripts/migration/guestperf/__init__.py
> diff --git a/tests/migration/guestperf/comparison.py b/scripts/migration/guestperf/comparison.py
> similarity index 100%
> rename from tests/migration/guestperf/comparison.py
> rename to scripts/migration/guestperf/comparison.py
> diff --git a/tests/migration/guestperf/engine.py b/scripts/migration/guestperf/engine.py
> similarity index 100%
> rename from tests/migration/guestperf/engine.py
> rename to scripts/migration/guestperf/engine.py
> diff --git a/tests/migration/guestperf/hardware.py b/scripts/migration/guestperf/hardware.py
> similarity index 100%
> rename from tests/migration/guestperf/hardware.py
> rename to scripts/migration/guestperf/hardware.py
> diff --git a/tests/migration/guestperf/plot.py b/scripts/migration/guestperf/plot.py
> similarity index 100%
> rename from tests/migration/guestperf/plot.py
> rename to scripts/migration/guestperf/plot.py
> diff --git a/tests/migration/guestperf/progress.py b/scripts/migration/guestperf/progress.py
> similarity index 100%
> rename from tests/migration/guestperf/progress.py
> rename to scripts/migration/guestperf/progress.py
> diff --git a/tests/migration/guestperf/report.py b/scripts/migration/guestperf/report.py
> similarity index 100%
> rename from tests/migration/guestperf/report.py
> rename to scripts/migration/guestperf/report.py
> diff --git a/tests/migration/guestperf/scenario.py b/scripts/migration/guestperf/scenario.py
> similarity index 100%
> rename from tests/migration/guestperf/scenario.py
> rename to scripts/migration/guestperf/scenario.py
> diff --git a/tests/migration/guestperf/shell.py b/scripts/migration/guestperf/shell.py
> similarity index 100%
> rename from tests/migration/guestperf/shell.py
> rename to scripts/migration/guestperf/shell.py
> diff --git a/tests/migration/guestperf/timings.py b/scripts/migration/guestperf/timings.py
> similarity index 100%
> rename from tests/migration/guestperf/timings.py
> rename to scripts/migration/guestperf/timings.py
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v1 1/5] tests/migration: Move the guestperf tool to scripts directory
2024-08-09 12:57 ` Fabiano Rosas
@ 2024-08-10 2:32 ` Hyman Huang
0 siblings, 0 replies; 12+ messages in thread
From: Hyman Huang @ 2024-08-10 2:32 UTC (permalink / raw)
To: Fabiano Rosas, qemu-devel; +Cc: Peter Xu
在 2024/8/9 20:57, Fabiano Rosas 写道:
> yong.huang@smartx.com writes:
>
>> From: Hyman Huang <yong.huang@smartx.com>
>>
>> Guestperf was designed to test the performance of migration,
>> with a loose connection to the fundamental test cases of QEMU.
>>
>> To improve the repository's structure, move it to the scripts
>> directory.
>>
>> Signed-off-by: Hyman Huang <yong.huang@smartx.com>
>> ---
>> MAINTAINERS | 5 +++++
>> {tests => scripts}/migration/guestperf-batch.py | 0
>> {tests => scripts}/migration/guestperf-plot.py | 0
>> {tests => scripts}/migration/guestperf.py | 0
>> {tests => scripts}/migration/guestperf/__init__.py | 0
>> {tests => scripts}/migration/guestperf/comparison.py | 0
>> {tests => scripts}/migration/guestperf/engine.py | 0
>> {tests => scripts}/migration/guestperf/hardware.py | 0
>> {tests => scripts}/migration/guestperf/plot.py | 0
>> {tests => scripts}/migration/guestperf/progress.py | 0
>> {tests => scripts}/migration/guestperf/report.py | 0
>> {tests => scripts}/migration/guestperf/scenario.py | 0
>> {tests => scripts}/migration/guestperf/shell.py | 0
>> {tests => scripts}/migration/guestperf/timings.py | 0
>> 14 files changed, 5 insertions(+)
>> rename {tests => scripts}/migration/guestperf-batch.py (100%)
>> rename {tests => scripts}/migration/guestperf-plot.py (100%)
>> rename {tests => scripts}/migration/guestperf.py (100%)
>> rename {tests => scripts}/migration/guestperf/__init__.py (100%)
>> rename {tests => scripts}/migration/guestperf/comparison.py (100%)
>> rename {tests => scripts}/migration/guestperf/engine.py (100%)
>> rename {tests => scripts}/migration/guestperf/hardware.py (100%)
>> rename {tests => scripts}/migration/guestperf/plot.py (100%)
>> rename {tests => scripts}/migration/guestperf/progress.py (100%)
>> rename {tests => scripts}/migration/guestperf/report.py (100%)
>> rename {tests => scripts}/migration/guestperf/scenario.py (100%)
>> rename {tests => scripts}/migration/guestperf/shell.py (100%)
>> rename {tests => scripts}/migration/guestperf/timings.py (100%)
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 10af212632..dae1535dae 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -3449,6 +3449,11 @@ F: migration/dirtyrate.h
>> F: include/sysemu/dirtyrate.h
>> F: docs/devel/migration/dirty-limit.rst
>>
>> +Migration performance test tool
>> +M: Hyman Huang <yong.huang@smartx.com>
>> +S: Maintained
>> +F: scripts/migration/
> This probably needs to be scripts/migration/guestperf* unless of course
> you intend to maintain all scripts that show up under
> scripts/migration/. We have analyze-migration.py and
> vmstate-static-checker.py which are candidates to be put in that same
> directory.
Ok, I'll update this section in the next version.
>
>> +
>> Detached LUKS header
>> M: Hyman Huang <yong.huang@smartx.com>
>> S: Maintained
>> diff --git a/tests/migration/guestperf-batch.py b/scripts/migration/guestperf-batch.py
>> similarity index 100%
>> rename from tests/migration/guestperf-batch.py
>> rename to scripts/migration/guestperf-batch.py
>> diff --git a/tests/migration/guestperf-plot.py b/scripts/migration/guestperf-plot.py
>> similarity index 100%
>> rename from tests/migration/guestperf-plot.py
>> rename to scripts/migration/guestperf-plot.py
>> diff --git a/tests/migration/guestperf.py b/scripts/migration/guestperf.py
>> similarity index 100%
>> rename from tests/migration/guestperf.py
>> rename to scripts/migration/guestperf.py
>> diff --git a/tests/migration/guestperf/__init__.py b/scripts/migration/guestperf/__init__.py
>> similarity index 100%
>> rename from tests/migration/guestperf/__init__.py
>> rename to scripts/migration/guestperf/__init__.py
>> diff --git a/tests/migration/guestperf/comparison.py b/scripts/migration/guestperf/comparison.py
>> similarity index 100%
>> rename from tests/migration/guestperf/comparison.py
>> rename to scripts/migration/guestperf/comparison.py
>> diff --git a/tests/migration/guestperf/engine.py b/scripts/migration/guestperf/engine.py
>> similarity index 100%
>> rename from tests/migration/guestperf/engine.py
>> rename to scripts/migration/guestperf/engine.py
>> diff --git a/tests/migration/guestperf/hardware.py b/scripts/migration/guestperf/hardware.py
>> similarity index 100%
>> rename from tests/migration/guestperf/hardware.py
>> rename to scripts/migration/guestperf/hardware.py
>> diff --git a/tests/migration/guestperf/plot.py b/scripts/migration/guestperf/plot.py
>> similarity index 100%
>> rename from tests/migration/guestperf/plot.py
>> rename to scripts/migration/guestperf/plot.py
>> diff --git a/tests/migration/guestperf/progress.py b/scripts/migration/guestperf/progress.py
>> similarity index 100%
>> rename from tests/migration/guestperf/progress.py
>> rename to scripts/migration/guestperf/progress.py
>> diff --git a/tests/migration/guestperf/report.py b/scripts/migration/guestperf/report.py
>> similarity index 100%
>> rename from tests/migration/guestperf/report.py
>> rename to scripts/migration/guestperf/report.py
>> diff --git a/tests/migration/guestperf/scenario.py b/scripts/migration/guestperf/scenario.py
>> similarity index 100%
>> rename from tests/migration/guestperf/scenario.py
>> rename to scripts/migration/guestperf/scenario.py
>> diff --git a/tests/migration/guestperf/shell.py b/scripts/migration/guestperf/shell.py
>> similarity index 100%
>> rename from tests/migration/guestperf/shell.py
>> rename to scripts/migration/guestperf/shell.py
>> diff --git a/tests/migration/guestperf/timings.py b/scripts/migration/guestperf/timings.py
>> similarity index 100%
>> rename from tests/migration/guestperf/timings.py
>> rename to scripts/migration/guestperf/timings.py
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default
2024-08-09 7:41 [PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment yong.huang
2024-08-09 7:43 ` yong.huang
2024-08-09 7:43 ` [PATCH v1 1/5] tests/migration: Move the guestperf tool to scripts directory yong.huang
@ 2024-08-09 7:43 ` yong.huang
2024-08-09 8:34 ` Daniel P. Berrangé
2024-08-09 12:59 ` Fabiano Rosas
2024-08-09 7:43 ` [PATCH v1 3/5] guestperf: Support deferred migration for multifd yong.huang
` (2 subsequent siblings)
5 siblings, 2 replies; 12+ messages in thread
From: yong.huang @ 2024-08-09 7:43 UTC (permalink / raw)
To: qemu-devel; +Cc: Fabiano Rosas, Peter Xu, yong.huang
From: Hyman Huang <yong.huang@smartx.com>
The initrd-stress.img was compiled by specifying the
target, to make it easier for developers to play the
guestperf tool, make it built by default.
Signed-off-by: Hyman Huang <yong.huang@smartx.com>
---
tests/migration/meson.build | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/tests/migration/meson.build b/tests/migration/meson.build
index a91aa61c65..393e401800 100644
--- a/tests/migration/meson.build
+++ b/tests/migration/meson.build
@@ -7,12 +7,13 @@ stress = executable(
files('stress.c'),
dependencies: [glib_static, sysprof],
link_args: ['-static'],
- build_by_default: false,
+ build_by_default: true,
)
custom_target(
'initrd-stress.img',
output: 'initrd-stress.img',
input: stress,
- command: [find_program('initrd-stress.sh'), '@OUTPUT@', '@INPUT@']
+ command: [find_program('initrd-stress.sh'), '@OUTPUT@', '@INPUT@'],
+ build_by_default: true,
)
--
2.11.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default
2024-08-09 7:43 ` [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default yong.huang
@ 2024-08-09 8:34 ` Daniel P. Berrangé
2024-08-09 9:00 ` Yong Huang
2024-08-09 12:59 ` Fabiano Rosas
1 sibling, 1 reply; 12+ messages in thread
From: Daniel P. Berrangé @ 2024-08-09 8:34 UTC (permalink / raw)
To: yong.huang; +Cc: qemu-devel, Fabiano Rosas, Peter Xu
On Fri, Aug 09, 2024 at 03:43:33PM +0800, yong.huang@smartx.com wrote:
> From: Hyman Huang <yong.huang@smartx.com>
>
> The initrd-stress.img was compiled by specifying the
> target, to make it easier for developers to play the
> guestperf tool, make it built by default.
If you're going to do this, then you will need to make sure it is only
built on a Linux host, and when glib static & sysprof are found. eg
if host_os == 'linux' && glib_static.found() && sysprof.found()
>
> Signed-off-by: Hyman Huang <yong.huang@smartx.com>
> ---
> tests/migration/meson.build | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/tests/migration/meson.build b/tests/migration/meson.build
> index a91aa61c65..393e401800 100644
> --- a/tests/migration/meson.build
> +++ b/tests/migration/meson.build
> @@ -7,12 +7,13 @@ stress = executable(
> files('stress.c'),
> dependencies: [glib_static, sysprof],
> link_args: ['-static'],
> - build_by_default: false,
> + build_by_default: true,
> )
>
> custom_target(
> 'initrd-stress.img',
> output: 'initrd-stress.img',
> input: stress,
> - command: [find_program('initrd-stress.sh'), '@OUTPUT@', '@INPUT@']
> + command: [find_program('initrd-stress.sh'), '@OUTPUT@', '@INPUT@'],
> + build_by_default: true,
> )
> --
> 2.11.0
>
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default
2024-08-09 8:34 ` Daniel P. Berrangé
@ 2024-08-09 9:00 ` Yong Huang
0 siblings, 0 replies; 12+ messages in thread
From: Yong Huang @ 2024-08-09 9:00 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel, Fabiano Rosas, Peter Xu
[-- Attachment #1: Type: text/plain, Size: 1904 bytes --]
On Fri, Aug 9, 2024 at 4:34 PM Daniel P. Berrangé <berrange@redhat.com>
wrote:
> On Fri, Aug 09, 2024 at 03:43:33PM +0800, yong.huang@smartx.com wrote:
> > From: Hyman Huang <yong.huang@smartx.com>
> >
> > The initrd-stress.img was compiled by specifying the
> > target, to make it easier for developers to play the
> > guestperf tool, make it built by default.
>
> If you're going to do this, then you will need to make sure it is only
> built on a Linux host, and when glib static & sysprof are found. eg
>
> if host_os == 'linux' && glib_static.found() && sysprof.found()
Ok, thanks for the comments, I'll add the check in the next version.
>
>
>
> > Signed-off-by: Hyman Huang <yong.huang@smartx.com>
> > ---
> > tests/migration/meson.build | 5 +++--
> > 1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/tests/migration/meson.build b/tests/migration/meson.build
> > index a91aa61c65..393e401800 100644
> > --- a/tests/migration/meson.build
> > +++ b/tests/migration/meson.build
> > @@ -7,12 +7,13 @@ stress = executable(
> > files('stress.c'),
> > dependencies: [glib_static, sysprof],
> > link_args: ['-static'],
> > - build_by_default: false,
> > + build_by_default: true,
> > )
> >
> > custom_target(
> > 'initrd-stress.img',
> > output: 'initrd-stress.img',
> > input: stress,
> > - command: [find_program('initrd-stress.sh'), '@OUTPUT@', '@INPUT@']
> > + command: [find_program('initrd-stress.sh'), '@OUTPUT@', '@INPUT@'],
> > + build_by_default: true,
> > )
> > --
> > 2.11.0
> >
> >
>
> With regards,
> Daniel
> --
> |: https://berrange.com -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o-
> https://www.instagram.com/dberrange :|
>
>
--
Best regards
[-- Attachment #2: Type: text/html, Size: 4019 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default
2024-08-09 7:43 ` [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default yong.huang
2024-08-09 8:34 ` Daniel P. Berrangé
@ 2024-08-09 12:59 ` Fabiano Rosas
1 sibling, 0 replies; 12+ messages in thread
From: Fabiano Rosas @ 2024-08-09 12:59 UTC (permalink / raw)
To: yong.huang, qemu-devel; +Cc: Peter Xu, yong.huang
yong.huang@smartx.com writes:
> From: Hyman Huang <yong.huang@smartx.com>
>
> The initrd-stress.img was compiled by specifying the
> target, to make it easier for developers to play the
> guestperf tool, make it built by default.
>
> Signed-off-by: Hyman Huang <yong.huang@smartx.com>
> ---
> tests/migration/meson.build | 5 +++--
This could have moved along with the rest of guestperf. There's already
a scripts/meson.build where these could go.
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v1 3/5] guestperf: Support deferred migration for multifd
2024-08-09 7:41 [PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment yong.huang
` (2 preceding siblings ...)
2024-08-09 7:43 ` [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default yong.huang
@ 2024-08-09 7:43 ` yong.huang
2024-08-09 7:43 ` [PATCH v1 4/5] guestperf: Nitpick the inconsistent parameters yong.huang
2024-08-09 7:43 ` [PATCH v1 5/5] guestperf: Introduce multifd compression option yong.huang
5 siblings, 0 replies; 12+ messages in thread
From: yong.huang @ 2024-08-09 7:43 UTC (permalink / raw)
To: qemu-devel; +Cc: Fabiano Rosas, Peter Xu, yong.huang
From: Hyman Huang <yong.huang@smartx.com>
The way to enable multifd migration has been changed by commit,
82137e6c8c (migration: enforce multifd and postcopy preempt to
be set before incoming), and guestperf has not made the
necessary changes. If multifd migration had been enabled in the
previous manner, the following error would have occurred:
Multifd must be set before incoming starts
Supporting deferred migration will fix it.
Signed-off-by: Hyman Huang <yong.huang@smartx.com>
---
scripts/migration/guestperf/engine.py | 18 ++++++++++++++----
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/scripts/migration/guestperf/engine.py b/scripts/migration/guestperf/engine.py
index 608d7270f6..4cddd05a9d 100644
--- a/scripts/migration/guestperf/engine.py
+++ b/scripts/migration/guestperf/engine.py
@@ -106,7 +106,8 @@ def _migrate_progress(self, vm):
info.get("dirty-limit-ring-full-time", 0),
)
- def _migrate(self, hardware, scenario, src, dst, connect_uri):
+ def _migrate(self, hardware, scenario, src,
+ dst, connect_uri, defer_migrate):
src_qemu_time = []
src_vcpu_time = []
src_pid = src.get_pid()
@@ -220,6 +221,8 @@ def _migrate(self, hardware, scenario, src, dst, connect_uri):
resp = src.cmd("migrate-set-parameters",
vcpu_dirty_limit=scenario._vcpu_dirty_limit)
+ if defer_migrate:
+ resp = dst.cmd("migrate-incoming", uri=connect_uri)
resp = src.cmd("migrate", uri=connect_uri)
post_copy = False
@@ -373,11 +376,14 @@ def _get_common_args(self, hardware, tunnelled=False):
def _get_src_args(self, hardware):
return self._get_common_args(hardware)
- def _get_dst_args(self, hardware, uri):
+ def _get_dst_args(self, hardware, uri, defer_migrate):
tunnelled = False
if self._dst_host != "localhost":
tunnelled = True
argv = self._get_common_args(hardware, tunnelled)
+
+ if defer_migrate:
+ return argv + ["-incoming", "defer"]
return argv + ["-incoming", uri]
@staticmethod
@@ -439,6 +445,9 @@ def run(self, hardware, scenario, result_dir=os.getcwd()):
except:
pass
+ if scenario._multifd:
+ defer_migrate = True
+
if self._dst_host != "localhost":
dstmonaddr = ("localhost", 9001)
else:
@@ -452,7 +461,7 @@ def run(self, hardware, scenario, result_dir=os.getcwd()):
monitor_address=srcmonaddr)
dst = QEMUMachine(self._binary,
- args=self._get_dst_args(hardware, uri),
+ args=self._get_dst_args(hardware, uri, defer_migrate),
wrapper=self._get_dst_wrapper(hardware),
name="qemu-dst-%d" % os.getpid(),
monitor_address=dstmonaddr)
@@ -461,7 +470,8 @@ def run(self, hardware, scenario, result_dir=os.getcwd()):
src.launch()
dst.launch()
- ret = self._migrate(hardware, scenario, src, dst, uri)
+ ret = self._migrate(hardware, scenario, src,
+ dst, uri, defer_migrate)
progress_history = ret[0]
qemu_timings = ret[1]
vcpu_timings = ret[2]
--
2.11.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v1 4/5] guestperf: Nitpick the inconsistent parameters
2024-08-09 7:41 [PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment yong.huang
` (3 preceding siblings ...)
2024-08-09 7:43 ` [PATCH v1 3/5] guestperf: Support deferred migration for multifd yong.huang
@ 2024-08-09 7:43 ` yong.huang
2024-08-09 7:43 ` [PATCH v1 5/5] guestperf: Introduce multifd compression option yong.huang
5 siblings, 0 replies; 12+ messages in thread
From: yong.huang @ 2024-08-09 7:43 UTC (permalink / raw)
To: qemu-devel; +Cc: Fabiano Rosas, Peter Xu, yong.huang
From: Hyman Huang <yong.huang@smartx.com>
Signed-off-by: Hyman Huang <yong.huang@smartx.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
---
scripts/migration/guestperf/comparison.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/migration/guestperf/comparison.py b/scripts/migration/guestperf/comparison.py
index 42cc0372d1..40e9d2eb1d 100644
--- a/scripts/migration/guestperf/comparison.py
+++ b/scripts/migration/guestperf/comparison.py
@@ -127,7 +127,7 @@ def __init__(self, name, scenarios):
# varying numbers of channels
Comparison("compr-multifd", scenarios = [
Scenario("compr-multifd-channels-4",
- multifd=True, multifd_channels=2),
+ multifd=True, multifd_channels=4),
Scenario("compr-multifd-channels-8",
multifd=True, multifd_channels=8),
Scenario("compr-multifd-channels-32",
--
2.11.0
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v1 5/5] guestperf: Introduce multifd compression option
2024-08-09 7:41 [PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment yong.huang
` (4 preceding siblings ...)
2024-08-09 7:43 ` [PATCH v1 4/5] guestperf: Nitpick the inconsistent parameters yong.huang
@ 2024-08-09 7:43 ` yong.huang
5 siblings, 0 replies; 12+ messages in thread
From: yong.huang @ 2024-08-09 7:43 UTC (permalink / raw)
To: qemu-devel; +Cc: Fabiano Rosas, Peter Xu, yong.huang
From: Hyman Huang <yong.huang@smartx.com>
Guestperf tool does not cover the multifd compression option
currently, it is worth supporting so that developers can
analysis the migration performance with different
compression algorithms.
Multifd support 4 compression algorithms currently:
zlib, zstd, qpl, uadk
To request that multifd with the specified compression
algorithm such as zlib:
$ ./scripts/migration/guestperf.py \
--multifd --multifd-channels 4 --multifd-compression zlib \
--output output.json
To run the entire standardized set of multifd compression
comparisons, with unix migration:
$ ./scripts/migration/guestperf-batch.py \
--dst-host localhost --transport unix \
--filter compr-multifd-compression* --output outputdir
Signed-off-by: Hyman Huang <yong.huang@smartx.com>
---
scripts/migration/guestperf/comparison.py | 13 +++++++++++++
scripts/migration/guestperf/engine.py | 14 ++++++++++++++
scripts/migration/guestperf/scenario.py | 7 +++++--
scripts/migration/guestperf/shell.py | 3 +++
4 files changed, 35 insertions(+), 2 deletions(-)
diff --git a/scripts/migration/guestperf/comparison.py b/scripts/migration/guestperf/comparison.py
index 40e9d2eb1d..e209a68ce5 100644
--- a/scripts/migration/guestperf/comparison.py
+++ b/scripts/migration/guestperf/comparison.py
@@ -158,4 +158,17 @@ def __init__(self, name, scenarios):
Scenario("compr-dirty-limit-50MB",
dirty_limit=True, vcpu_dirty_limit=50),
]),
+
+ # Looking at effect of multifd with
+ # different compression algorithms
+ Comparison("compr-multifd-compression", scenarios = [
+ Scenario("compr-multifd-compression-zlib",
+ multifd=True, multifd_channels=4, multifd_compression="zlib"),
+ Scenario("compr-multifd-compression-zstd",
+ multifd=True, multifd_channels=4, multifd_compression="zstd"),
+ Scenario("compr-multifd-compression-qpl",
+ multifd=True, multifd_channels=4, multifd_compression="qpl"),
+ Scenario("compr-multifd-compression-uadk",
+ multifd=True, multifd_channels=4, multifd_compression="uadk"),
+ ]),
]
diff --git a/scripts/migration/guestperf/engine.py b/scripts/migration/guestperf/engine.py
index 4cddd05a9d..59e77edb72 100644
--- a/scripts/migration/guestperf/engine.py
+++ b/scripts/migration/guestperf/engine.py
@@ -31,6 +31,8 @@
'..', '..', '..', 'python'))
from qemu.machine import QEMUMachine
+# multifd supported compression algorithms
+MULTIFD_CMP_ALGS = ("zlib", "zstd", "qpl", "uadk")
class Engine(object):
@@ -191,6 +193,12 @@ def _migrate(self, hardware, scenario, src,
scenario._compression_xbzrle_cache))
if scenario._multifd:
+ if (scenario._multifd_compression and
+ (scenario._multifd_compression not in MULTIFD_CMP_ALGS)):
+ raise Exception("unsupported multifd compression "
+ "algorithm: %s" %
+ scenario._multifd_compression)
+
resp = src.cmd("migrate-set-capabilities",
capabilities = [
{ "capability": "multifd",
@@ -206,6 +214,12 @@ def _migrate(self, hardware, scenario, src,
resp = dst.cmd("migrate-set-parameters",
multifd_channels=scenario._multifd_channels)
+ if scenario._multifd_compression:
+ resp = src.cmd("migrate-set-parameters",
+ multifd_compression=scenario._multifd_compression)
+ resp = dst.cmd("migrate-set-parameters",
+ multifd_compression=scenario._multifd_compression)
+
if scenario._dirty_limit:
if not hardware._dirty_ring_size:
raise Exception("dirty ring size must be configured when "
diff --git a/scripts/migration/guestperf/scenario.py b/scripts/migration/guestperf/scenario.py
index 154c4f5d5f..4be7fafebf 100644
--- a/scripts/migration/guestperf/scenario.py
+++ b/scripts/migration/guestperf/scenario.py
@@ -30,7 +30,7 @@ def __init__(self, name,
auto_converge=False, auto_converge_step=10,
compression_mt=False, compression_mt_threads=1,
compression_xbzrle=False, compression_xbzrle_cache=10,
- multifd=False, multifd_channels=2,
+ multifd=False, multifd_channels=2, multifd_compression="",
dirty_limit=False, x_vcpu_dirty_limit_period=500,
vcpu_dirty_limit=1):
@@ -61,6 +61,7 @@ def __init__(self, name,
self._multifd = multifd
self._multifd_channels = multifd_channels
+ self._multifd_compression = multifd_compression
self._dirty_limit = dirty_limit
self._x_vcpu_dirty_limit_period = x_vcpu_dirty_limit_period
@@ -85,6 +86,7 @@ def serialize(self):
"compression_xbzrle_cache": self._compression_xbzrle_cache,
"multifd": self._multifd,
"multifd_channels": self._multifd_channels,
+ "multifd_compression": self._multifd_compression,
"dirty_limit": self._dirty_limit,
"x_vcpu_dirty_limit_period": self._x_vcpu_dirty_limit_period,
"vcpu_dirty_limit": self._vcpu_dirty_limit,
@@ -109,4 +111,5 @@ def deserialize(cls, data):
data["compression_xbzrle"],
data["compression_xbzrle_cache"],
data["multifd"],
- data["multifd_channels"])
+ data["multifd_channels"],
+ data["multifd_compression"])
diff --git a/scripts/migration/guestperf/shell.py b/scripts/migration/guestperf/shell.py
index c85d89efec..1452eb8a33 100644
--- a/scripts/migration/guestperf/shell.py
+++ b/scripts/migration/guestperf/shell.py
@@ -130,6 +130,8 @@ def __init__(self):
action="store_true")
parser.add_argument("--multifd-channels", dest="multifd_channels",
default=2, type=int)
+ parser.add_argument("--multifd-compression", dest="multifd_compression",
+ default="")
parser.add_argument("--dirty-limit", dest="dirty_limit", default=False,
action="store_true")
@@ -166,6 +168,7 @@ def get_scenario(self, args):
multifd=args.multifd,
multifd_channels=args.multifd_channels,
+ multifd_compression=args.multifd_compression,
dirty_limit=args.dirty_limit,
x_vcpu_dirty_limit_period=\
--
2.11.0
^ permalink raw reply related [flat|nested] 12+ messages in thread