* [PATCH v9 1/5] docs/migration: add qatzip compression feature
2024-08-30 23:27 [PATCH v9 0/5] Implement QATzip compression method Yichen Wang
@ 2024-08-30 23:27 ` Yichen Wang
2024-08-30 23:27 ` [PATCH v9 2/5] meson: Introduce 'qatzip' feature to the build system Yichen Wang
` (4 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Yichen Wang @ 2024-08-30 23:27 UTC (permalink / raw)
To: Peter Xu, Fabiano Rosas, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel
Cc: Hao Xiang, Liu, Yuan1, Zou, Nanhai, Ho-Ren (Jack) Chuang,
Yichen Wang, Xiaoning Ding
From: Yuan Liu <yuan1.liu@intel.com>
add Intel QATzip compression method introduction
Reviewed-by: Nanhai Zou <nanhai.zou@intel.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Yuan Liu <yuan1.liu@intel.com>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
---
docs/devel/migration/features.rst | 1 +
docs/devel/migration/qatzip-compression.rst | 165 ++++++++++++++++++++
2 files changed, 166 insertions(+)
create mode 100644 docs/devel/migration/qatzip-compression.rst
diff --git a/docs/devel/migration/features.rst b/docs/devel/migration/features.rst
index 58f8fd9e16..8f431d52f9 100644
--- a/docs/devel/migration/features.rst
+++ b/docs/devel/migration/features.rst
@@ -14,3 +14,4 @@ Migration has plenty of features to support different use cases.
CPR
qpl-compression
uadk-compression
+ qatzip-compression
diff --git a/docs/devel/migration/qatzip-compression.rst b/docs/devel/migration/qatzip-compression.rst
new file mode 100644
index 0000000000..862b383164
--- /dev/null
+++ b/docs/devel/migration/qatzip-compression.rst
@@ -0,0 +1,165 @@
+==================
+QATzip Compression
+==================
+In scenarios with limited network bandwidth, the ``QATzip`` solution can help
+users save a lot of host CPU resources by accelerating compression and
+decompression through the Intel QuickAssist Technology(``QAT``) hardware.
+
+
+The following test was conducted using 8 multifd channels and 10Gbps network
+bandwidth. The results show that, compared to zstd, ``QATzip`` significantly
+saves CPU resources on the sender and reduces migration time. Compared to the
+uncompressed solution, ``QATzip`` greatly improves the dirty page processing
+capability, indicated by the Pages per Second metric, and also reduces the
+total migration time.
+
+::
+
+ VM Configuration: 16 vCPU and 64G memory
+ VM Workload: all vCPUs are idle and 54G memory is filled with Silesia data.
+ QAT Devices: 4
+ |-----------|--------|---------|----------|----------|------|------|
+ |8 Channels |Total |down |throughput|pages per | send | recv |
+ | |time(ms)|time(ms) |(mbps) |second | cpu %| cpu% |
+ |-----------|--------|---------|----------|----------|------|------|
+ |qatzip | 16630| 28| 10467| 2940235| 160| 360|
+ |-----------|--------|---------|----------|----------|------|------|
+ |zstd | 20165| 24| 8579| 2391465| 810| 340|
+ |-----------|--------|---------|----------|----------|------|------|
+ |none | 46063| 40| 10848| 330240| 45| 85|
+ |-----------|--------|---------|----------|----------|------|------|
+
+
+QATzip Compression Framework
+============================
+
+``QATzip`` is a user space library which builds on top of the Intel QuickAssist
+Technology to provide extended accelerated compression and decompression
+services.
+
+For more ``QATzip`` introduction, please refer to `QATzip Introduction
+<https://github.com/intel/QATzip?tab=readme-ov-file#introductionl>`_
+
+::
+
+ +----------------+
+ | MultiFd Thread |
+ +-------+--------+
+ |
+ | compress/decompress
+ +-------+--------+
+ | QATzip library |
+ +-------+--------+
+ |
+ +-------+--------+
+ | QAT library |
+ +-------+--------+
+ | user space
+ --------+---------------------
+ | kernel space
+ +------+-------+
+ | QAT Driver |
+ +------+-------+
+ |
+ +------+-------+
+ | QAT Devices |
+ +--------------+
+
+
+QATzip Installation
+-------------------
+
+The ``QATzip`` installation package has been integrated into some Linux
+distributions and can be installed directly. For example, the Ubuntu Server
+24.04 LTS system can be installed using below command
+
+.. code-block:: shell
+
+ #apt search qatzip
+ libqatzip-dev/noble 1.2.0-0ubuntu3 amd64
+ Intel QuickAssist user space library development files
+
+ libqatzip3/noble 1.2.0-0ubuntu3 amd64
+ Intel QuickAssist user space library
+
+ qatzip/noble,now 1.2.0-0ubuntu3 amd64 [installed]
+ Compression user-space tool for Intel QuickAssist Technology
+
+ #sudo apt install libqatzip-dev libqatzip3 qatzip
+
+If your system does not support the ``QATzip`` installation package, you can
+use the source code to build and install, please refer to `QATzip source code installation
+<https://github.com/intel/QATzip?tab=readme-ov-file#build-intel-quickassist-technology-driver>`_
+
+QAT Hardware Deployment
+-----------------------
+
+``QAT`` supports physical functions(PFs) and virtual functions(VFs) for
+deployment, and users can configure ``QAT`` resources for migration according
+to actual needs. For more details about ``QAT`` deployment, please refer to
+`Intel QuickAssist Technology Documentation
+<https://intel.github.io/quickassist/index.html>`_
+
+For more ``QAT`` hardware introduction, please refer to `intel-quick-assist-technology-overview
+<https://www.intel.com/content/www/us/en/architecture-and-technology/intel-quick-assist-technology-overview.html>`_
+
+How To Use QATzip Compression
+=============================
+
+1 - Install ``QATzip`` library
+
+2 - Build ``QEMU`` with ``--enable-qatzip`` parameter
+
+ E.g. configure --target-list=x86_64-softmmu --enable-kvm ``--enable-qatzip``
+
+3 - Set ``migrate_set_parameter multifd-compression qatzip``
+
+4 - Set ``migrate_set_parameter multifd-qatzip-level comp_level``, the default
+comp_level value is 1, and it supports levels from 1 to 9
+
+QAT Memory Requirements
+=======================
+
+The user needs to reserve system memory for the QAT memory management to
+allocate DMA memory. The size of the reserved system memory depends on the
+number of devices used for migration and the number of multifd channels.
+
+Because memory usage depends on QAT configuration, please refer to `QAT Memory
+Driver Queries
+<https://intel.github.io/quickassist/PG/infrastructure_debugability.html?highlight=memory>`_
+for memory usage calculation.
+
+.. list-table:: An example of a PF used for migration
+ :header-rows: 1
+
+ * - Number of channels
+ - Sender memory usage
+ - Receiver memory usage
+ * - 2
+ - 10M
+ - 10M
+ * - 4
+ - 12M
+ - 14M
+ * - 8
+ - 16M
+ - 20M
+
+How To Choose Between QATzip and QPL
+====================================
+Starting from 4th Gen Intel Xeon Scalable processors, codenamed Sapphire Rapids
+processor(``SPR``), multiple built-in accelerators are supported including
+``QAT`` and ``IAA``. The former can accelerate ``QATzip`` and the latter is
+used to accelerate ``QPL``.
+
+Here are some suggestions:
+
+1 - If the live migration scenario is limited by network bandwidth and ``QAT``
+hardware resources exceed ``IAA``, use the ``QATzip`` method, which can save a
+lot of host CPU resources for compression.
+
+2 - If the system cannot support shared virtual memory (SVM) technology, use
+the ``QATzip`` method because ``QPL`` performance is not good without SVM
+support.
+
+3 - For other scenarios, use the ``QPL`` method first.
--
Yichen Wang
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v9 2/5] meson: Introduce 'qatzip' feature to the build system
2024-08-30 23:27 [PATCH v9 0/5] Implement QATzip compression method Yichen Wang
2024-08-30 23:27 ` [PATCH v9 1/5] docs/migration: add qatzip compression feature Yichen Wang
@ 2024-08-30 23:27 ` Yichen Wang
2024-08-30 23:27 ` [PATCH v9 3/5] migration: Add migration parameters for QATzip Yichen Wang
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Yichen Wang @ 2024-08-30 23:27 UTC (permalink / raw)
To: Peter Xu, Fabiano Rosas, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel
Cc: Hao Xiang, Liu, Yuan1, Zou, Nanhai, Ho-Ren (Jack) Chuang,
Yichen Wang, Xiaoning Ding, Bryan Zhang
From: Bryan Zhang <bryan.zhang@bytedance.com>
Add a 'qatzip' feature, which is automatically disabled, and which
depends on the QATzip library if enabled.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
---
meson.build | 10 ++++++++++
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh | 3 +++
3 files changed, 15 insertions(+)
diff --git a/meson.build b/meson.build
index fbda17c987..b89b713e79 100644
--- a/meson.build
+++ b/meson.build
@@ -1262,6 +1262,14 @@ if not get_option('uadk').auto() or have_system
uadk = declare_dependency(dependencies: [libwd, libwd_comp])
endif
endif
+
+qatzip = not_found
+if not get_option('qatzip').auto() or have_system
+ qatzip = dependency('qatzip', version: '>=1.1.2',
+ required: get_option('qatzip'),
+ method: 'pkg-config')
+endif
+
virgl = not_found
have_vhost_user_gpu = have_tools and host_os == 'linux' and pixman.found()
@@ -2412,6 +2420,7 @@ config_host_data.set('CONFIG_STATX_MNT_ID', has_statx_mnt_id)
config_host_data.set('CONFIG_ZSTD', zstd.found())
config_host_data.set('CONFIG_QPL', qpl.found())
config_host_data.set('CONFIG_UADK', uadk.found())
+config_host_data.set('CONFIG_QATZIP', qatzip.found())
config_host_data.set('CONFIG_FUSE', fuse.found())
config_host_data.set('CONFIG_FUSE_LSEEK', fuse_lseek.found())
config_host_data.set('CONFIG_SPICE_PROTOCOL', spice_protocol.found())
@@ -4535,6 +4544,7 @@ summary_info += {'lzfse support': liblzfse}
summary_info += {'zstd support': zstd}
summary_info += {'Query Processing Library support': qpl}
summary_info += {'UADK Library support': uadk}
+summary_info += {'qatzip support': qatzip}
summary_info += {'NUMA host support': numa}
summary_info += {'capstone': capstone}
summary_info += {'libpmem support': libpmem}
diff --git a/meson_options.txt b/meson_options.txt
index 0269fa0f16..f7b652b30d 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -261,6 +261,8 @@ option('qpl', type : 'feature', value : 'auto',
description: 'Query Processing Library support')
option('uadk', type : 'feature', value : 'auto',
description: 'UADK Library support')
+option('qatzip', type: 'feature', value: 'auto',
+ description: 'QATzip compression support')
option('fuse', type: 'feature', value: 'auto',
description: 'FUSE block device export')
option('fuse_lseek', type : 'feature', value : 'auto',
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index c97079a38c..5f377a6d81 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -163,6 +163,7 @@ meson_options_help() {
printf "%s\n" ' pixman pixman support'
printf "%s\n" ' plugins TCG plugins via shared library loading'
printf "%s\n" ' png PNG support with libpng'
+ printf "%s\n" ' qatzip QATzip compression support'
printf "%s\n" ' qcow1 qcow1 image format support'
printf "%s\n" ' qed qed image format support'
printf "%s\n" ' qga-vss build QGA VSS support (broken with MinGW)'
@@ -427,6 +428,8 @@ _meson_option_parse() {
--enable-png) printf "%s" -Dpng=enabled ;;
--disable-png) printf "%s" -Dpng=disabled ;;
--prefix=*) quote_sh "-Dprefix=$2" ;;
+ --enable-qatzip) printf "%s" -Dqatzip=enabled ;;
+ --disable-qatzip) printf "%s" -Dqatzip=disabled ;;
--enable-qcow1) printf "%s" -Dqcow1=enabled ;;
--disable-qcow1) printf "%s" -Dqcow1=disabled ;;
--enable-qed) printf "%s" -Dqed=enabled ;;
--
Yichen Wang
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v9 3/5] migration: Add migration parameters for QATzip
2024-08-30 23:27 [PATCH v9 0/5] Implement QATzip compression method Yichen Wang
2024-08-30 23:27 ` [PATCH v9 1/5] docs/migration: add qatzip compression feature Yichen Wang
2024-08-30 23:27 ` [PATCH v9 2/5] meson: Introduce 'qatzip' feature to the build system Yichen Wang
@ 2024-08-30 23:27 ` Yichen Wang
2024-09-04 17:18 ` Fabiano Rosas
2024-09-05 6:11 ` Prasad Pandit
2024-08-30 23:27 ` [PATCH v9 4/5] migration: Introduce 'qatzip' compression method Yichen Wang
` (2 subsequent siblings)
5 siblings, 2 replies; 10+ messages in thread
From: Yichen Wang @ 2024-08-30 23:27 UTC (permalink / raw)
To: Peter Xu, Fabiano Rosas, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel
Cc: Hao Xiang, Liu, Yuan1, Zou, Nanhai, Ho-Ren (Jack) Chuang,
Yichen Wang, Xiaoning Ding, Bryan Zhang
From: Bryan Zhang <bryan.zhang@bytedance.com>
Adds support for migration parameters to control QATzip compression
level.
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
---
migration/migration-hmp-cmds.c | 4 ++++
migration/options.c | 34 ++++++++++++++++++++++++++++++++++
migration/options.h | 1 +
qapi/migration.json | 18 ++++++++++++++++++
4 files changed, 57 insertions(+)
diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c
index 7d608d26e1..28165cfc9e 100644
--- a/migration/migration-hmp-cmds.c
+++ b/migration/migration-hmp-cmds.c
@@ -576,6 +576,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
p->has_multifd_zlib_level = true;
visit_type_uint8(v, param, &p->multifd_zlib_level, &err);
break;
+ case MIGRATION_PARAMETER_MULTIFD_QATZIP_LEVEL:
+ p->has_multifd_qatzip_level = true;
+ visit_type_uint8(v, param, &p->multifd_qatzip_level, &err);
+ break;
case MIGRATION_PARAMETER_MULTIFD_ZSTD_LEVEL:
p->has_multifd_zstd_level = true;
visit_type_uint8(v, param, &p->multifd_zstd_level, &err);
diff --git a/migration/options.c b/migration/options.c
index 645f55003d..147cd2b8fd 100644
--- a/migration/options.c
+++ b/migration/options.c
@@ -55,6 +55,13 @@
#define DEFAULT_MIGRATE_MULTIFD_COMPRESSION MULTIFD_COMPRESSION_NONE
/* 0: means nocompress, 1: best speed, ... 9: best compress ratio */
#define DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL 1
+/*
+ * 1: best speed, ... 9: best compress ratio
+ * There is some nuance here. Refer to QATzip documentation to understand
+ * the mapping of QATzip levels to standard deflate levels.
+ */
+#define DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL 1
+
/* 0: means nocompress, 1: best speed, ... 20: best compress ratio */
#define DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL 1
@@ -123,6 +130,9 @@ Property migration_properties[] = {
DEFINE_PROP_UINT8("multifd-zlib-level", MigrationState,
parameters.multifd_zlib_level,
DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL),
+ DEFINE_PROP_UINT8("multifd-qatzip-level", MigrationState,
+ parameters.multifd_qatzip_level,
+ DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL),
DEFINE_PROP_UINT8("multifd-zstd-level", MigrationState,
parameters.multifd_zstd_level,
DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL),
@@ -787,6 +797,13 @@ int migrate_multifd_zlib_level(void)
return s->parameters.multifd_zlib_level;
}
+int migrate_multifd_qatzip_level(void)
+{
+ MigrationState *s = migrate_get_current();
+
+ return s->parameters.multifd_qatzip_level;
+}
+
int migrate_multifd_zstd_level(void)
{
MigrationState *s = migrate_get_current();
@@ -892,6 +909,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp)
params->multifd_compression = s->parameters.multifd_compression;
params->has_multifd_zlib_level = true;
params->multifd_zlib_level = s->parameters.multifd_zlib_level;
+ params->has_multifd_qatzip_level = true;
+ params->multifd_qatzip_level = s->parameters.multifd_qatzip_level;
params->has_multifd_zstd_level = true;
params->multifd_zstd_level = s->parameters.multifd_zstd_level;
params->has_xbzrle_cache_size = true;
@@ -946,6 +965,7 @@ void migrate_params_init(MigrationParameters *params)
params->has_multifd_channels = true;
params->has_multifd_compression = true;
params->has_multifd_zlib_level = true;
+ params->has_multifd_qatzip_level = true;
params->has_multifd_zstd_level = true;
params->has_xbzrle_cache_size = true;
params->has_max_postcopy_bandwidth = true;
@@ -1038,6 +1058,14 @@ bool migrate_params_check(MigrationParameters *params, Error **errp)
return false;
}
+ if (params->has_multifd_qatzip_level &&
+ ((params->multifd_qatzip_level > 9) ||
+ (params->multifd_qatzip_level < 1))) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_qatzip_level",
+ "a value between 1 and 9");
+ return false;
+ }
+
if (params->has_multifd_zstd_level &&
(params->multifd_zstd_level > 20)) {
error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_zstd_level",
@@ -1195,6 +1223,9 @@ static void migrate_params_test_apply(MigrateSetParameters *params,
if (params->has_multifd_compression) {
dest->multifd_compression = params->multifd_compression;
}
+ if (params->has_multifd_qatzip_level) {
+ dest->multifd_qatzip_level = params->multifd_qatzip_level;
+ }
if (params->has_multifd_zlib_level) {
dest->multifd_zlib_level = params->multifd_zlib_level;
}
@@ -1315,6 +1346,9 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp)
if (params->has_multifd_compression) {
s->parameters.multifd_compression = params->multifd_compression;
}
+ if (params->has_multifd_qatzip_level) {
+ s->parameters.multifd_qatzip_level = params->multifd_qatzip_level;
+ }
if (params->has_multifd_zlib_level) {
s->parameters.multifd_zlib_level = params->multifd_zlib_level;
}
diff --git a/migration/options.h b/migration/options.h
index a2397026db..a0bd6edc06 100644
--- a/migration/options.h
+++ b/migration/options.h
@@ -78,6 +78,7 @@ uint64_t migrate_max_postcopy_bandwidth(void);
int migrate_multifd_channels(void);
MultiFDCompression migrate_multifd_compression(void);
int migrate_multifd_zlib_level(void);
+int migrate_multifd_qatzip_level(void);
int migrate_multifd_zstd_level(void);
uint8_t migrate_throttle_trigger_threshold(void);
const char *migrate_tls_authz(void);
diff --git a/qapi/migration.json b/qapi/migration.json
index 7324571e92..f4c27426c8 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -792,6 +792,11 @@
# speed, and 9 means best compression ratio which will consume
# more CPU. Defaults to 1. (Since 5.0)
#
+# @multifd-qatzip-level: Set the compression level to be used in live
+# migration. The level is an integer between 1 and 9, where 1 means
+# the best compression speed, and 9 means the best compression
+# ratio which will consume more CPU. Defaults to 1. (Since 9.2)
+#
# @multifd-zstd-level: Set the compression level to be used in live
# migration, the compression level is an integer between 0 and 20,
# where 0 means no compression, 1 means the best compression
@@ -852,6 +857,7 @@
'xbzrle-cache-size', 'max-postcopy-bandwidth',
'max-cpu-throttle', 'multifd-compression',
'multifd-zlib-level', 'multifd-zstd-level',
+ 'multifd-qatzip-level',
'block-bitmap-mapping',
{ 'name': 'x-vcpu-dirty-limit-period', 'features': ['unstable'] },
'vcpu-dirty-limit',
@@ -967,6 +973,11 @@
# speed, and 9 means best compression ratio which will consume
# more CPU. Defaults to 1. (Since 5.0)
#
+# @multifd-qatzip-level: Set the compression level to be used in live
+# migration. The level is an integer between 1 and 9, where 1 means
+# the best compression speed, and 9 means the best compression
+# ratio which will consume more CPU. Defaults to 1. (Since 9.2)
+#
# @multifd-zstd-level: Set the compression level to be used in live
# migration, the compression level is an integer between 0 and 20,
# where 0 means no compression, 1 means the best compression
@@ -1040,6 +1051,7 @@
'*max-cpu-throttle': 'uint8',
'*multifd-compression': 'MultiFDCompression',
'*multifd-zlib-level': 'uint8',
+ '*multifd-qatzip-level': 'uint8',
'*multifd-zstd-level': 'uint8',
'*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ],
'*x-vcpu-dirty-limit-period': { 'type': 'uint64',
@@ -1171,6 +1183,11 @@
# speed, and 9 means best compression ratio which will consume
# more CPU. Defaults to 1. (Since 5.0)
#
+# @multifd-qatzip-level: Set the compression level to be used in live
+# migration. The level is an integer between 1 and 9, where 1 means
+# the best compression speed, and 9 means the best compression
+# ratio which will consume more CPU. Defaults to 1. (Since 9.2)
+#
# @multifd-zstd-level: Set the compression level to be used in live
# migration, the compression level is an integer between 0 and 20,
# where 0 means no compression, 1 means the best compression
@@ -1241,6 +1258,7 @@
'*max-cpu-throttle': 'uint8',
'*multifd-compression': 'MultiFDCompression',
'*multifd-zlib-level': 'uint8',
+ '*multifd-qatzip-level': 'uint8',
'*multifd-zstd-level': 'uint8',
'*block-bitmap-mapping': [ 'BitmapMigrationNodeAlias' ],
'*x-vcpu-dirty-limit-period': { 'type': 'uint64',
--
Yichen Wang
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v9 3/5] migration: Add migration parameters for QATzip
2024-08-30 23:27 ` [PATCH v9 3/5] migration: Add migration parameters for QATzip Yichen Wang
@ 2024-09-04 17:18 ` Fabiano Rosas
2024-09-05 6:11 ` Prasad Pandit
1 sibling, 0 replies; 10+ messages in thread
From: Fabiano Rosas @ 2024-09-04 17:18 UTC (permalink / raw)
To: Yichen Wang, Peter Xu, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel
Cc: Hao Xiang, Liu, Yuan1, Zou, Nanhai, Ho-Ren (Jack) Chuang,
Yichen Wang, Xiaoning Ding, Bryan Zhang
Yichen Wang <yichen.wang@bytedance.com> writes:
> From: Bryan Zhang <bryan.zhang@bytedance.com>
>
> Adds support for migration parameters to control QATzip compression
> level.
>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
> Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
> Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v9 3/5] migration: Add migration parameters for QATzip
2024-08-30 23:27 ` [PATCH v9 3/5] migration: Add migration parameters for QATzip Yichen Wang
2024-09-04 17:18 ` Fabiano Rosas
@ 2024-09-05 6:11 ` Prasad Pandit
2024-09-05 22:39 ` [External] " Yichen Wang
1 sibling, 1 reply; 10+ messages in thread
From: Prasad Pandit @ 2024-09-05 6:11 UTC (permalink / raw)
To: Yichen Wang
Cc: Peter Xu, Fabiano Rosas, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel, Hao Xiang, Liu, Yuan1, Zou, Nanhai,
Ho-Ren (Jack) Chuang, Xiaoning Ding, Bryan Zhang
On Sat, 31 Aug 2024 at 04:58, Yichen Wang <yichen.wang@bytedance.com> wrote:
> +/*
> + * 1: best speed, ... 9: best compress ratio
> + * There is some nuance here. Refer to QATzip documentation to understand
> + * the mapping of QATzip levels to standard deflate levels.
> + */
> +#define DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL 1
> +
...
> +# @multifd-qatzip-level: Set the compression level to be used in live
> +# migration. The level is an integer between 1 and 9, where 1 means
> +# the best compression speed, and 9 means the best compression
> +# ratio which will consume more CPU. Defaults to 1. (Since 9.2)
> +#
* Should the default compression level be at the median of 1 - 9 => 5
or 6 ? A compression method (QATzip) choosing speed (1) over
compression as default seems contradictory.
Otherwise:
Reviewed-by: Prasad Pandit <pjp@fedoraproject.org>
Thank you.
---
- Prasad
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [External] Re: [PATCH v9 3/5] migration: Add migration parameters for QATzip
2024-09-05 6:11 ` Prasad Pandit
@ 2024-09-05 22:39 ` Yichen Wang
0 siblings, 0 replies; 10+ messages in thread
From: Yichen Wang @ 2024-09-05 22:39 UTC (permalink / raw)
To: Prasad Pandit
Cc: Peter Xu, Fabiano Rosas, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel, Hao Xiang, Liu, Yuan1, Zou, Nanhai,
Ho-Ren (Jack) Chuang, Xiaoning Ding, Bryan Zhang
On Wed, Sep 4, 2024 at 11:11 PM Prasad Pandit <ppandit@redhat.com> wrote:
>
> On Sat, 31 Aug 2024 at 04:58, Yichen Wang <yichen.wang@bytedance.com> wrote:
> > +/*
> > + * 1: best speed, ... 9: best compress ratio
> > + * There is some nuance here. Refer to QATzip documentation to understand
> > + * the mapping of QATzip levels to standard deflate levels.
> > + */
> > +#define DEFAULT_MIGRATE_MULTIFD_QATZIP_LEVEL 1
> > +
> ...
> > +# @multifd-qatzip-level: Set the compression level to be used in live
> > +# migration. The level is an integer between 1 and 9, where 1 means
> > +# the best compression speed, and 9 means the best compression
> > +# ratio which will consume more CPU. Defaults to 1. (Since 9.2)
> > +#
>
> * Should the default compression level be at the median of 1 - 9 => 5
> or 6 ? A compression method (QATzip) choosing speed (1) over
> compression as default seems contradictory.
This is being discussed on Patch V2,
https://lore.kernel.org/all/20240326224221.3623014-3-bryan.zhang@bytedance.com/.
The conclusion of the discussion with Intel is to keep 1 as the
default level.
>
> Otherwise:
> Reviewed-by: Prasad Pandit <pjp@fedoraproject.org>
>
> Thank you.
> ---
> - Prasad
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v9 4/5] migration: Introduce 'qatzip' compression method
2024-08-30 23:27 [PATCH v9 0/5] Implement QATzip compression method Yichen Wang
` (2 preceding siblings ...)
2024-08-30 23:27 ` [PATCH v9 3/5] migration: Add migration parameters for QATzip Yichen Wang
@ 2024-08-30 23:27 ` Yichen Wang
2024-08-30 23:27 ` [PATCH v9 5/5] tests/migration: Add integration test for " Yichen Wang
2024-09-09 14:56 ` [PATCH v9 0/5] Implement QATzip " Peter Xu
5 siblings, 0 replies; 10+ messages in thread
From: Yichen Wang @ 2024-08-30 23:27 UTC (permalink / raw)
To: Peter Xu, Fabiano Rosas, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel
Cc: Hao Xiang, Liu, Yuan1, Zou, Nanhai, Ho-Ren (Jack) Chuang,
Yichen Wang, Xiaoning Ding, Bryan Zhang, Prasad Pandit
From: Bryan Zhang <bryan.zhang@bytedance.com>
Adds support for 'qatzip' as an option for the multifd compression
method parameter, and implements using QAT for 'qatzip' compression and
decompression.
Acked-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Prasad Pandit <pjp@fedoraproject.org>
Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
---
hw/core/qdev-properties-system.c | 2 +-
migration/meson.build | 1 +
migration/multifd-qatzip.c | 394 +++++++++++++++++++++++++++++++
migration/multifd.h | 5 +-
qapi/migration.json | 3 +
5 files changed, 402 insertions(+), 3 deletions(-)
create mode 100644 migration/multifd-qatzip.c
diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
index f13350b4fb..a56fbf728d 100644
--- a/hw/core/qdev-properties-system.c
+++ b/hw/core/qdev-properties-system.c
@@ -659,7 +659,7 @@ const PropertyInfo qdev_prop_fdc_drive_type = {
const PropertyInfo qdev_prop_multifd_compression = {
.name = "MultiFDCompression",
.description = "multifd_compression values, "
- "none/zlib/zstd/qpl/uadk",
+ "none/zlib/zstd/qpl/uadk/qatzip",
.enum_table = &MultiFDCompression_lookup,
.get = qdev_propinfo_get_enum,
.set = qdev_propinfo_set_enum,
diff --git a/migration/meson.build b/migration/meson.build
index 5ce2acb41e..c9454c26ae 100644
--- a/migration/meson.build
+++ b/migration/meson.build
@@ -41,6 +41,7 @@ system_ss.add(when: rdma, if_true: files('rdma.c'))
system_ss.add(when: zstd, if_true: files('multifd-zstd.c'))
system_ss.add(when: qpl, if_true: files('multifd-qpl.c'))
system_ss.add(when: uadk, if_true: files('multifd-uadk.c'))
+system_ss.add(when: qatzip, if_true: files('multifd-qatzip.c'))
specific_ss.add(when: 'CONFIG_SYSTEM_ONLY',
if_true: files('ram.c',
diff --git a/migration/multifd-qatzip.c b/migration/multifd-qatzip.c
new file mode 100644
index 0000000000..3c787ed879
--- /dev/null
+++ b/migration/multifd-qatzip.c
@@ -0,0 +1,394 @@
+/*
+ * Multifd QATzip compression implementation
+ *
+ * Copyright (c) Bytedance
+ *
+ * Authors:
+ * Bryan Zhang <bryan.zhang@bytedance.com>
+ * Hao Xiang <hao.xiang@bytedance.com>
+ * Yichen Wang <yichen.wang@bytedance.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "exec/ramblock.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "qapi/qapi-types-migration.h"
+#include "options.h"
+#include "multifd.h"
+#include <qatzip.h>
+
+typedef struct {
+ /*
+ * Unique session for use with QATzip API
+ */
+ QzSession_T sess;
+
+ /*
+ * For compression: Buffer for pages to compress
+ * For decompression: Buffer for data to decompress
+ */
+ uint8_t *in_buf;
+ uint32_t in_len;
+
+ /*
+ * For compression: Output buffer of compressed data
+ * For decompression: Output buffer of decompressed data
+ */
+ uint8_t *out_buf;
+ uint32_t out_len;
+} QatzipData;
+
+/**
+ * qatzip_send_setup: Set up QATzip session and private buffers.
+ *
+ * @param p Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return 0 on success, -1 on error (and *errp will be set)
+ */
+static int qatzip_send_setup(MultiFDSendParams *p, Error **errp)
+{
+ QatzipData *q;
+ QzSessionParamsDeflate_T params;
+ const char *err_msg;
+ int ret;
+
+ q = g_new0(QatzipData, 1);
+ p->compress_data = q;
+ /* We need one extra place for the packet header */
+ p->iov = g_new0(struct iovec, 2);
+
+ /*
+ * Initialize QAT device with software fallback by default. This allows
+ * QATzip to use CPU path when QAT hardware reaches maximum throughput.
+ */
+ ret = qzInit(&q->sess, true);
+ if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+ err_msg = "qzInit failed";
+ goto err;
+ }
+
+ ret = qzGetDefaultsDeflate(¶ms);
+ if (ret != QZ_OK) {
+ err_msg = "qzGetDefaultsDeflate failed";
+ goto err;
+ }
+
+ /* Make sure to use configured QATzip compression level. */
+ params.common_params.comp_lvl = migrate_multifd_qatzip_level();
+ ret = qzSetupSessionDeflate(&q->sess, ¶ms);
+ if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+ err_msg = "qzSetupSessionDeflate failed";
+ goto err;
+ }
+
+ if (MULTIFD_PACKET_SIZE > UINT32_MAX) {
+ err_msg = "packet size too large for QAT";
+ goto err;
+ }
+
+ q->in_len = MULTIFD_PACKET_SIZE;
+ /*
+ * PINNED_MEM is an enum from qatzip headers, which means to use
+ * kzalloc_node() to allocate memory for QAT DMA purposes. When QAT device
+ * is not available or software fallback is used, the malloc flag needs to
+ * be set as COMMON_MEM.
+ */
+ q->in_buf = qzMalloc(q->in_len, 0, PINNED_MEM);
+ if (!q->in_buf) {
+ q->in_buf = qzMalloc(q->in_len, 0, COMMON_MEM);
+ if (!q->in_buf) {
+ err_msg = "qzMalloc failed";
+ goto err;
+ }
+ }
+
+ q->out_len = qzMaxCompressedLength(MULTIFD_PACKET_SIZE, &q->sess);
+ q->out_buf = qzMalloc(q->out_len, 0, PINNED_MEM);
+ if (!q->out_buf) {
+ q->out_buf = qzMalloc(q->out_len, 0, COMMON_MEM);
+ if (!q->out_buf) {
+ err_msg = "qzMalloc failed";
+ goto err;
+ }
+ }
+
+ return 0;
+
+err:
+ error_setg(errp, "multifd %u: [sender] %s", p->id, err_msg);
+ return -1;
+}
+
+/**
+ * qatzip_send_cleanup: Tear down QATzip session and release private buffers.
+ *
+ * @param p Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return None
+ */
+static void qatzip_send_cleanup(MultiFDSendParams *p, Error **errp)
+{
+ QatzipData *q = p->compress_data;
+
+ if (q) {
+ if (q->in_buf) {
+ qzFree(q->in_buf);
+ }
+ if (q->out_buf) {
+ qzFree(q->out_buf);
+ }
+ (void)qzTeardownSession(&q->sess);
+ (void)qzClose(&q->sess);
+ g_free(q);
+ }
+
+ g_free(p->iov);
+ p->iov = NULL;
+ p->compress_data = NULL;
+}
+
+/**
+ * qatzip_send_prepare: Compress pages and update IO channel info.
+ *
+ * @param p Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return 0 on success, -1 on error (and *errp will be set)
+ */
+static int qatzip_send_prepare(MultiFDSendParams *p, Error **errp)
+{
+ MultiFDPages_t *pages = p->pages;
+ QatzipData *q = p->compress_data;
+ int ret;
+ unsigned int in_len, out_len;
+
+ if (!multifd_send_prepare_common(p)) {
+ goto out;
+ }
+
+ /*
+ * Unlike other multifd compression implementations, we use a non-streaming
+ * API and place all the data into one buffer, rather than sending each
+ * page to the compression API at a time. Based on initial benchmarks, the
+ * non-streaming API outperforms the streaming API. Plus, the logic in QEMU
+ * is friendly to using the non-streaming API anyway. If either of these
+ * statements becomes no longer true, we can revisit adding a streaming
+ * implementation.
+ */
+ for (int i = 0; i < pages->normal_num; i++) {
+ memcpy(q->in_buf + (i * p->page_size),
+ pages->block->host + pages->offset[i],
+ p->page_size);
+ }
+
+ in_len = pages->normal_num * p->page_size;
+ if (in_len > q->in_len) {
+ error_setg(errp, "multifd %u: unexpectedly large input", p->id);
+ return -1;
+ }
+ out_len = q->out_len;
+
+ ret = qzCompress(&q->sess, q->in_buf, &in_len, q->out_buf, &out_len, 1);
+ if (ret != QZ_OK) {
+ error_setg(errp, "multifd %u: QATzip returned %d instead of QZ_OK",
+ p->id, ret);
+ return -1;
+ }
+ if (in_len != pages->normal_num * p->page_size) {
+ error_setg(errp, "multifd %u: QATzip failed to compress all input",
+ p->id);
+ return -1;
+ }
+
+ p->iov[p->iovs_num].iov_base = q->out_buf;
+ p->iov[p->iovs_num].iov_len = out_len;
+ p->iovs_num++;
+ p->next_packet_size = out_len;
+
+out:
+ p->flags |= MULTIFD_FLAG_QATZIP;
+ multifd_send_fill_packet(p);
+ return 0;
+}
+
+/**
+ * qatzip_recv_setup: Set up QATzip session and allocate private buffers.
+ *
+ * @param p Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return 0 on success, -1 on error (and *errp will be set)
+ */
+static int qatzip_recv_setup(MultiFDRecvParams *p, Error **errp)
+{
+ QatzipData *q;
+ QzSessionParamsDeflate_T params;
+ const char *err_msg;
+ int ret;
+
+ q = g_new0(QatzipData, 1);
+ p->compress_data = q;
+
+ /*
+ * Initialize QAT device with software fallback by default. This allows
+ * QATzip to use CPU path when QAT hardware reaches maximum throughput.
+ */
+ ret = qzInit(&q->sess, true);
+ if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+ err_msg = "qzInit failed";
+ goto err;
+ }
+
+ ret = qzGetDefaultsDeflate(¶ms);
+ if (ret != QZ_OK) {
+ err_msg = "qzGetDefaultsDeflate failed";
+ goto err;
+ }
+
+ ret = qzSetupSessionDeflate(&q->sess, ¶ms);
+ if (ret != QZ_OK && ret != QZ_DUPLICATE) {
+ err_msg = "qzSetupSessionDeflate failed";
+ goto err;
+ }
+
+ /*
+ * Reserve extra spaces for the incoming packets. Current implementation
+ * doesn't send uncompressed pages in case the compression gets too big.
+ */
+ q->in_len = MULTIFD_PACKET_SIZE * 2;
+ /*
+ * PINNED_MEM is an enum from qatzip headers, which means to use
+ * kzalloc_node() to allocate memory for QAT DMA purposes. When QAT device
+ * is not available or software fallback is used, the malloc flag needs to
+ * be set as COMMON_MEM.
+ */
+ q->in_buf = qzMalloc(q->in_len, 0, PINNED_MEM);
+ if (!q->in_buf) {
+ q->in_buf = qzMalloc(q->in_len, 0, COMMON_MEM);
+ if (!q->in_buf) {
+ err_msg = "qzMalloc failed";
+ goto err;
+ }
+ }
+
+ q->out_len = MULTIFD_PACKET_SIZE;
+ q->out_buf = qzMalloc(q->out_len, 0, PINNED_MEM);
+ if (!q->out_buf) {
+ q->out_buf = qzMalloc(q->out_len, 0, COMMON_MEM);
+ if (!q->out_buf) {
+ err_msg = "qzMalloc failed";
+ goto err;
+ }
+ }
+
+ return 0;
+
+err:
+ error_setg(errp, "multifd %u: [receiver] %s", p->id, err_msg);
+ return -1;
+}
+
+/**
+ * qatzip_recv_cleanup: Tear down QATzip session and release private buffers.
+ *
+ * @param p Multifd channel params
+ * @return None
+ */
+static void qatzip_recv_cleanup(MultiFDRecvParams *p)
+{
+ QatzipData *q = p->compress_data;
+
+ if (q) {
+ if (q->in_buf) {
+ qzFree(q->in_buf);
+ }
+ if (q->out_buf) {
+ qzFree(q->out_buf);
+ }
+ (void)qzTeardownSession(&q->sess);
+ (void)qzClose(&q->sess);
+ g_free(q);
+ }
+ p->compress_data = NULL;
+}
+
+
+/**
+ * qatzip_recv: Decompress pages and copy them to the appropriate
+ * locations.
+ *
+ * @param p Multifd channel params
+ * @param errp Pointer to error, which will be set in case of error
+ * @return 0 on success, -1 on error (and *errp will be set)
+ */
+static int qatzip_recv(MultiFDRecvParams *p, Error **errp)
+{
+ QatzipData *q = p->compress_data;
+ int ret;
+ unsigned int in_len, out_len;
+ uint32_t in_size = p->next_packet_size;
+ uint32_t expected_size = p->normal_num * p->page_size;
+ uint32_t flags = p->flags & MULTIFD_FLAG_COMPRESSION_MASK;
+
+ if (in_size > q->in_len) {
+ error_setg(errp, "multifd %u: received unexpectedly large packet",
+ p->id);
+ return -1;
+ }
+
+ if (flags != MULTIFD_FLAG_QATZIP) {
+ error_setg(errp, "multifd %u: flags received %x flags expected %x",
+ p->id, flags, MULTIFD_FLAG_QATZIP);
+ return -1;
+ }
+
+ multifd_recv_zero_page_process(p);
+ if (!p->normal_num) {
+ assert(in_size == 0);
+ return 0;
+ }
+
+ ret = qio_channel_read_all(p->c, (void *)q->in_buf, in_size, errp);
+ if (ret != 0) {
+ return ret;
+ }
+
+ in_len = in_size;
+ out_len = q->out_len;
+ ret = qzDecompress(&q->sess, q->in_buf, &in_len, q->out_buf, &out_len);
+ if (ret != QZ_OK) {
+ error_setg(errp, "multifd %u: qzDecompress failed", p->id);
+ return -1;
+ }
+ if (out_len != expected_size) {
+ error_setg(errp, "multifd %u: packet size received %u size expected %u",
+ p->id, out_len, expected_size);
+ return -1;
+ }
+
+ /* Copy each page to its appropriate location. */
+ for (int i = 0; i < p->normal_num; i++) {
+ memcpy(p->host + p->normal[i],
+ q->out_buf + p->page_size * i,
+ p->page_size);
+ }
+ return 0;
+}
+
+static MultiFDMethods multifd_qatzip_ops = {
+ .send_setup = qatzip_send_setup,
+ .send_cleanup = qatzip_send_cleanup,
+ .send_prepare = qatzip_send_prepare,
+ .recv_setup = qatzip_recv_setup,
+ .recv_cleanup = qatzip_recv_cleanup,
+ .recv = qatzip_recv
+};
+
+static void multifd_qatzip_register(void)
+{
+ multifd_register_ops(MULTIFD_COMPRESSION_QATZIP, &multifd_qatzip_ops);
+}
+
+migration_init(multifd_qatzip_register);
diff --git a/migration/multifd.h b/migration/multifd.h
index 0ecd6f47d7..adceb65050 100644
--- a/migration/multifd.h
+++ b/migration/multifd.h
@@ -34,14 +34,15 @@ MultiFDRecvData *multifd_get_recv_data(void);
/* Multifd Compression flags */
#define MULTIFD_FLAG_SYNC (1 << 0)
-/* We reserve 4 bits for compression methods */
-#define MULTIFD_FLAG_COMPRESSION_MASK (0xf << 1)
+/* We reserve 5 bits for compression methods */
+#define MULTIFD_FLAG_COMPRESSION_MASK (0x1f << 1)
/* we need to be compatible. Before compression value was 0 */
#define MULTIFD_FLAG_NOCOMP (0 << 1)
#define MULTIFD_FLAG_ZLIB (1 << 1)
#define MULTIFD_FLAG_ZSTD (2 << 1)
#define MULTIFD_FLAG_QPL (4 << 1)
#define MULTIFD_FLAG_UADK (8 << 1)
+#define MULTIFD_FLAG_QATZIP (16 << 1)
/* This value needs to be a multiple of qemu_target_page_size() */
#define MULTIFD_PACKET_SIZE (512 * 1024)
diff --git a/qapi/migration.json b/qapi/migration.json
index f4c27426c8..f1b7103dc8 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -561,6 +561,8 @@
#
# @zstd: use zstd compression method.
#
+# @qatzip: use qatzip compression method. (Since 9.2)
+#
# @qpl: use qpl compression method. Query Processing Library(qpl) is
# based on the deflate compression algorithm and use the Intel
# In-Memory Analytics Accelerator(IAA) accelerated compression and
@@ -573,6 +575,7 @@
{ 'enum': 'MultiFDCompression',
'data': [ 'none', 'zlib',
{ 'name': 'zstd', 'if': 'CONFIG_ZSTD' },
+ { 'name': 'qatzip', 'if': 'CONFIG_QATZIP'},
{ 'name': 'qpl', 'if': 'CONFIG_QPL' },
{ 'name': 'uadk', 'if': 'CONFIG_UADK' } ] }
--
Yichen Wang
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH v9 5/5] tests/migration: Add integration test for 'qatzip' compression method
2024-08-30 23:27 [PATCH v9 0/5] Implement QATzip compression method Yichen Wang
` (3 preceding siblings ...)
2024-08-30 23:27 ` [PATCH v9 4/5] migration: Introduce 'qatzip' compression method Yichen Wang
@ 2024-08-30 23:27 ` Yichen Wang
2024-09-09 14:56 ` [PATCH v9 0/5] Implement QATzip " Peter Xu
5 siblings, 0 replies; 10+ messages in thread
From: Yichen Wang @ 2024-08-30 23:27 UTC (permalink / raw)
To: Peter Xu, Fabiano Rosas, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel
Cc: Hao Xiang, Liu, Yuan1, Zou, Nanhai, Ho-Ren (Jack) Chuang,
Yichen Wang, Xiaoning Ding, Bryan Zhang
From: Bryan Zhang <bryan.zhang@bytedance.com>
Adds an integration test for 'qatzip'.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Bryan Zhang <bryan.zhang@bytedance.com>
Signed-off-by: Hao Xiang <hao.xiang@linux.dev>
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
---
tests/qtest/migration-test.c | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 6c06100d91..9f74b1722c 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -2996,6 +2996,18 @@ test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from,
}
#endif /* CONFIG_ZSTD */
+#ifdef CONFIG_QATZIP
+static void *
+test_migrate_precopy_tcp_multifd_qatzip_start(QTestState *from,
+ QTestState *to)
+{
+ migrate_set_parameter_int(from, "multifd-qatzip-level", 2);
+ migrate_set_parameter_int(to, "multifd-qatzip-level", 2);
+
+ return test_migrate_precopy_tcp_multifd_start_common(from, to, "qatzip");
+}
+#endif
+
#ifdef CONFIG_QPL
static void *
test_migrate_precopy_tcp_multifd_qpl_start(QTestState *from,
@@ -3093,6 +3105,17 @@ static void test_multifd_tcp_zstd(void)
}
#endif
+#ifdef CONFIG_QATZIP
+static void test_multifd_tcp_qatzip(void)
+{
+ MigrateCommon args = {
+ .listen_uri = "defer",
+ .start_hook = test_migrate_precopy_tcp_multifd_qatzip_start,
+ };
+ test_precopy_common(&args);
+}
+#endif
+
#ifdef CONFIG_QPL
static void test_multifd_tcp_qpl(void)
{
@@ -3996,6 +4019,10 @@ int main(int argc, char **argv)
migration_test_add("/migration/multifd/tcp/plain/zstd",
test_multifd_tcp_zstd);
#endif
+#ifdef CONFIG_QATZIP
+ migration_test_add("/migration/multifd/tcp/plain/qatzip",
+ test_multifd_tcp_qatzip);
+#endif
#ifdef CONFIG_QPL
migration_test_add("/migration/multifd/tcp/plain/qpl",
test_multifd_tcp_qpl);
--
Yichen Wang
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH v9 0/5] Implement QATzip compression method
2024-08-30 23:27 [PATCH v9 0/5] Implement QATzip compression method Yichen Wang
` (4 preceding siblings ...)
2024-08-30 23:27 ` [PATCH v9 5/5] tests/migration: Add integration test for " Yichen Wang
@ 2024-09-09 14:56 ` Peter Xu
5 siblings, 0 replies; 10+ messages in thread
From: Peter Xu @ 2024-09-09 14:56 UTC (permalink / raw)
To: Yichen Wang
Cc: Fabiano Rosas, Paolo Bonzini, Daniel P. Berrangé,
Eduardo Habkost, Marc-André Lureau, Thomas Huth,
Philippe Mathieu-Daudé, Eric Blake, Markus Armbruster,
qemu-devel, Hao Xiang, Liu, Yuan1, Zou, Nanhai,
Ho-Ren (Jack) Chuang, Xiaoning Ding
On Fri, Aug 30, 2024 at 04:27:17PM -0700, Yichen Wang wrote:
> v9:
> - Rebase changes on top of cec99171931ea79215c79661d33423ac84e63b6e;
> - Address comments and add Review-by in commit messages;
queued, thanks.
--
Peter Xu
^ permalink raw reply [flat|nested] 10+ messages in thread