* [PATCH v2 01/10] migration: Create migrate_rdma()
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 12:49 ` [PATCH v2 02/10] migration/rdma: Unfold ram_control_before_iterate() Juan Quintela
` (9 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
Helper to say if we are doing a migration over rdma.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/migration.h | 2 ++
migration/options.h | 1 +
migration/migration.c | 1 +
migration/options.c | 7 +++++++
migration/rdma.c | 4 +++-
5 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/migration/migration.h b/migration/migration.h
index 972597f4de..4543e36081 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -463,6 +463,8 @@ struct MigrationState {
* switchover has been received.
*/
bool switchover_acked;
+ /* Is this a rdma migration */
+ bool rdma_migration;
};
void migrate_set_state(int *state, int old_state, int new_state);
diff --git a/migration/options.h b/migration/options.h
index 045e2a41a2..a26fd1680b 100644
--- a/migration/options.h
+++ b/migration/options.h
@@ -56,6 +56,7 @@ bool migrate_zero_copy_send(void);
bool migrate_multifd_flush_after_each_section(void);
bool migrate_postcopy(void);
+bool migrate_rdma(void);
bool migrate_tls(void);
/* capabilities helpers */
diff --git a/migration/migration.c b/migration/migration.c
index 585d3c8f55..1cb8898e6b 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1439,6 +1439,7 @@ int migrate_init(MigrationState *s, Error **errp)
s->iteration_initial_bytes = 0;
s->threshold_size = 0;
s->switchover_acked = false;
+ s->rdma_migration = false;
/*
* set mig_stats compression_counters memory to zero for a
* new migration
diff --git a/migration/options.c b/migration/options.c
index 1d1e1321b0..0107d63b9b 100644
--- a/migration/options.c
+++ b/migration/options.c
@@ -376,6 +376,13 @@ bool migrate_postcopy(void)
return migrate_postcopy_ram() || migrate_dirty_bitmaps();
}
+bool migrate_rdma(void)
+{
+ MigrationState *s = migrate_get_current();
+
+ return s->rdma_migration;
+}
+
bool migrate_tls(void)
{
MigrationState *s = migrate_get_current();
diff --git a/migration/rdma.c b/migration/rdma.c
index cd5e1afe60..6219227915 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -4126,6 +4126,7 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp)
int ret;
RDMAContext *rdma;
Error *local_err = NULL;
+ MigrationState *s = migrate_get_current();
trace_rdma_start_incoming_migration();
@@ -4156,7 +4157,7 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp)
}
trace_rdma_start_incoming_migration_after_rdma_listen();
-
+ s->rdma_migration = true;
qemu_set_fd_handler(rdma->channel->fd, rdma_accept_incoming_migration,
NULL, (void *)(intptr_t)rdma);
return;
@@ -4233,6 +4234,7 @@ void rdma_start_outgoing_migration(void *opaque,
trace_rdma_start_outgoing_migration_after_rdma_connect();
s->to_dst_file = rdma_new_output(rdma);
+ s->rdma_migration = true;
migrate_fd_connect(s, NULL);
return;
return_path_err:
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 02/10] migration/rdma: Unfold ram_control_before_iterate()
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
2023-10-04 12:49 ` [PATCH v2 01/10] migration: Create migrate_rdma() Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 17:00 ` Fabiano Rosas
2023-10-04 12:49 ` [PATCH v2 03/10] migration/rdma: Unfold ram_control_after_iterate() Juan Quintela
` (8 subsequent siblings)
10 siblings, 1 reply; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
Once there:
- Remove unused data parameter
- unfold it in its callers.
- change all callers to call qemu_rdma_registration_start()
- We need to call QIO_CHANNEL_RDMA() after we check for migrate_rdma()
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
--
initilazize rioc after checknig that rdma is enabled.
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/qemu-file.h | 2 --
migration/rdma.h | 7 +++++++
migration/qemu-file.c | 13 +------------
migration/ram.c | 16 +++++++++++++---
migration/rdma.c | 12 ++++--------
5 files changed, 25 insertions(+), 25 deletions(-)
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index 03e718c264..d6a370c569 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -55,7 +55,6 @@ typedef int (QEMURamSaveFunc)(QEMUFile *f,
size_t size);
typedef struct QEMUFileHooks {
- QEMURamHookFunc *before_ram_iterate;
QEMURamHookFunc *after_ram_iterate;
QEMURamHookFunc *hook_ram_load;
QEMURamSaveFunc *save_page;
@@ -127,7 +126,6 @@ void qemu_fflush(QEMUFile *f);
void qemu_file_set_blocking(QEMUFile *f, bool block);
int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size);
-void ram_control_before_iterate(QEMUFile *f, uint64_t flags);
void ram_control_after_iterate(QEMUFile *f, uint64_t flags);
void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data);
diff --git a/migration/rdma.h b/migration/rdma.h
index de2ba09dc5..670c67a8cb 100644
--- a/migration/rdma.h
+++ b/migration/rdma.h
@@ -22,4 +22,11 @@ void rdma_start_outgoing_migration(void *opaque, const char *host_port,
void rdma_start_incoming_migration(const char *host_port, Error **errp);
+
+#ifdef CONFIG_RDMA
+int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags);
+#else
+static inline
+int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; }
+#endif
#endif
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 5e8207dae4..1a8170421f 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -32,6 +32,7 @@
#include "trace.h"
#include "options.h"
#include "qapi/error.h"
+#include "rdma.h"
#define IO_BUF_SIZE 32768
#define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64)
@@ -288,18 +289,6 @@ void qemu_fflush(QEMUFile *f)
f->iovcnt = 0;
}
-void ram_control_before_iterate(QEMUFile *f, uint64_t flags)
-{
- int ret = 0;
-
- if (f->hooks && f->hooks->before_ram_iterate) {
- ret = f->hooks->before_ram_iterate(f, flags, NULL);
- if (ret < 0) {
- qemu_file_set_error(f, ret);
- }
- }
-}
-
void ram_control_after_iterate(QEMUFile *f, uint64_t flags)
{
int ret = 0;
diff --git a/migration/ram.c b/migration/ram.c
index e4bfd39f08..407760b3a2 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -59,6 +59,7 @@
#include "qemu/iov.h"
#include "multifd.h"
#include "sysemu/runstate.h"
+#include "rdma.h"
#include "options.h"
#include "sysemu/dirtylimit.h"
#include "sysemu/kvm.h"
@@ -3054,7 +3055,10 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
}
}
- ram_control_before_iterate(f, RAM_CONTROL_SETUP);
+ ret = qemu_rdma_registration_start(f, RAM_CONTROL_SETUP);
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
ram_control_after_iterate(f, RAM_CONTROL_SETUP);
migration_ops = g_malloc0(sizeof(MigrationOps));
@@ -3114,7 +3118,10 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
/* Read version before ram_list.blocks */
smp_rmb();
- ram_control_before_iterate(f, RAM_CONTROL_ROUND);
+ ret = qemu_rdma_registration_start(f, RAM_CONTROL_ROUND);
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
i = 0;
@@ -3219,7 +3226,10 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
migration_bitmap_sync_precopy(rs, true);
}
- ram_control_before_iterate(f, RAM_CONTROL_FINISH);
+ ret = qemu_rdma_registration_start(f, RAM_CONTROL_FINISH);
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
/* try transferring iterative blocks of memory */
diff --git a/migration/rdma.c b/migration/rdma.c
index 6219227915..a93d8d705d 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3868,18 +3868,15 @@ static int rdma_load_hook(QEMUFile *f, uint64_t flags, void *data)
}
}
-static int qemu_rdma_registration_start(QEMUFile *f,
- uint64_t flags, void *data)
+int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags)
{
- QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
- RDMAContext *rdma;
-
- if (migration_in_postcopy()) {
+ if (!migrate_rdma () || migration_in_postcopy()) {
return 0;
}
+ QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
RCU_READ_LOCK_GUARD();
- rdma = qatomic_rcu_read(&rioc->rdmaout);
+ RDMAContext *rdma = qatomic_rcu_read(&rioc->rdmaout);
if (!rdma) {
return -EIO;
}
@@ -4012,7 +4009,6 @@ static const QEMUFileHooks rdma_read_hooks = {
};
static const QEMUFileHooks rdma_write_hooks = {
- .before_ram_iterate = qemu_rdma_registration_start,
.after_ram_iterate = qemu_rdma_registration_stop,
.save_page = qemu_rdma_save_page,
};
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v2 02/10] migration/rdma: Unfold ram_control_before_iterate()
2023-10-04 12:49 ` [PATCH v2 02/10] migration/rdma: Unfold ram_control_before_iterate() Juan Quintela
@ 2023-10-04 17:00 ` Fabiano Rosas
0 siblings, 0 replies; 14+ messages in thread
From: Fabiano Rosas @ 2023-10-04 17:00 UTC (permalink / raw)
To: Juan Quintela, qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Leonardo Bras
Juan Quintela <quintela@redhat.com> writes:
> Once there:
> - Remove unused data parameter
> - unfold it in its callers.
> - change all callers to call qemu_rdma_registration_start()
> - We need to call QIO_CHANNEL_RDMA() after we check for migrate_rdma()
>
> Reviewed-by: Peter Xu <peterx@redhat.com>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
>
> --
>
> initilazize rioc after checknig that rdma is enabled.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
> migration/qemu-file.h | 2 --
> migration/rdma.h | 7 +++++++
> migration/qemu-file.c | 13 +------------
> migration/ram.c | 16 +++++++++++++---
> migration/rdma.c | 12 ++++--------
> 5 files changed, 25 insertions(+), 25 deletions(-)
>
> diff --git a/migration/qemu-file.h b/migration/qemu-file.h
> index 03e718c264..d6a370c569 100644
> --- a/migration/qemu-file.h
> +++ b/migration/qemu-file.h
> @@ -55,7 +55,6 @@ typedef int (QEMURamSaveFunc)(QEMUFile *f,
> size_t size);
>
> typedef struct QEMUFileHooks {
> - QEMURamHookFunc *before_ram_iterate;
> QEMURamHookFunc *after_ram_iterate;
> QEMURamHookFunc *hook_ram_load;
> QEMURamSaveFunc *save_page;
> @@ -127,7 +126,6 @@ void qemu_fflush(QEMUFile *f);
> void qemu_file_set_blocking(QEMUFile *f, bool block);
> int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size);
>
> -void ram_control_before_iterate(QEMUFile *f, uint64_t flags);
> void ram_control_after_iterate(QEMUFile *f, uint64_t flags);
> void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data);
>
> diff --git a/migration/rdma.h b/migration/rdma.h
> index de2ba09dc5..670c67a8cb 100644
> --- a/migration/rdma.h
> +++ b/migration/rdma.h
> @@ -22,4 +22,11 @@ void rdma_start_outgoing_migration(void *opaque, const char *host_port,
>
> void rdma_start_incoming_migration(const char *host_port, Error **errp);
>
> +
> +#ifdef CONFIG_RDMA
> +int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags);
> +#else
> +static inline
> +int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; }
> +#endif
> #endif
> diff --git a/migration/qemu-file.c b/migration/qemu-file.c
> index 5e8207dae4..1a8170421f 100644
> --- a/migration/qemu-file.c
> +++ b/migration/qemu-file.c
> @@ -32,6 +32,7 @@
> #include "trace.h"
> #include "options.h"
> #include "qapi/error.h"
> +#include "rdma.h"
>
> #define IO_BUF_SIZE 32768
> #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64)
> @@ -288,18 +289,6 @@ void qemu_fflush(QEMUFile *f)
> f->iovcnt = 0;
> }
>
> -void ram_control_before_iterate(QEMUFile *f, uint64_t flags)
> -{
> - int ret = 0;
> -
> - if (f->hooks && f->hooks->before_ram_iterate) {
> - ret = f->hooks->before_ram_iterate(f, flags, NULL);
> - if (ret < 0) {
> - qemu_file_set_error(f, ret);
> - }
> - }
> -}
> -
> void ram_control_after_iterate(QEMUFile *f, uint64_t flags)
> {
> int ret = 0;
> diff --git a/migration/ram.c b/migration/ram.c
> index e4bfd39f08..407760b3a2 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -59,6 +59,7 @@
> #include "qemu/iov.h"
> #include "multifd.h"
> #include "sysemu/runstate.h"
> +#include "rdma.h"
> #include "options.h"
> #include "sysemu/dirtylimit.h"
> #include "sysemu/kvm.h"
> @@ -3054,7 +3055,10 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
> }
> }
>
> - ram_control_before_iterate(f, RAM_CONTROL_SETUP);
> + ret = qemu_rdma_registration_start(f, RAM_CONTROL_SETUP);
> + if (ret < 0) {
> + qemu_file_set_error(f, ret);
Markus' patch 23 will turn the return code from
qemu_rdma_registration_start() from -EIO into -1. Any code that uses
strerr to report it will now see an EPERM (-1).
We should someday give an Error argument to savevm functions and stop
using QEMUFile as an error carrier. Or perhaps add an Error pointer to
RAMState?
Anyway, out scope for this patch.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v2 03/10] migration/rdma: Unfold ram_control_after_iterate()
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
2023-10-04 12:49 ` [PATCH v2 01/10] migration: Create migrate_rdma() Juan Quintela
2023-10-04 12:49 ` [PATCH v2 02/10] migration/rdma: Unfold ram_control_before_iterate() Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 12:49 ` [PATCH v2 04/10] migration/rdma: Remove all uses of RAM_CONTROL_HOOK Juan Quintela
` (7 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
Once there:
- Remove unused data parameter
- unfold it in its callers
- change all callers to call qemu_rdma_registration_stop()
- We need to call QIO_CHANNEL_RDMA() after we check for migrate_rdma()
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
This function has goto's. So I don't change the place where we
declare variables, althought I think that it is correct, just don't
start that discussion.
---
migration/qemu-file.h | 2 --
migration/rdma.h | 3 +++
migration/qemu-file.c | 12 ------------
migration/ram.c | 17 ++++++++++++++---
migration/rdma.c | 9 ++++-----
5 files changed, 21 insertions(+), 22 deletions(-)
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index d6a370c569..35e671a01e 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -55,7 +55,6 @@ typedef int (QEMURamSaveFunc)(QEMUFile *f,
size_t size);
typedef struct QEMUFileHooks {
- QEMURamHookFunc *after_ram_iterate;
QEMURamHookFunc *hook_ram_load;
QEMURamSaveFunc *save_page;
} QEMUFileHooks;
@@ -126,7 +125,6 @@ void qemu_fflush(QEMUFile *f);
void qemu_file_set_blocking(QEMUFile *f, bool block);
int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size);
-void ram_control_after_iterate(QEMUFile *f, uint64_t flags);
void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data);
/* Whenever this is found in the data stream, the flags
diff --git a/migration/rdma.h b/migration/rdma.h
index 670c67a8cb..c13b94c782 100644
--- a/migration/rdma.h
+++ b/migration/rdma.h
@@ -25,8 +25,11 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp);
#ifdef CONFIG_RDMA
int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags);
+int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags);
#else
static inline
int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; }
+static inline
+int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; }
#endif
#endif
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 1a8170421f..35c5c9109b 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -289,18 +289,6 @@ void qemu_fflush(QEMUFile *f)
f->iovcnt = 0;
}
-void ram_control_after_iterate(QEMUFile *f, uint64_t flags)
-{
- int ret = 0;
-
- if (f->hooks && f->hooks->after_ram_iterate) {
- ret = f->hooks->after_ram_iterate(f, flags, NULL);
- if (ret < 0) {
- qemu_file_set_error(f, ret);
- }
- }
-}
-
void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data)
{
if (f->hooks && f->hooks->hook_ram_load) {
diff --git a/migration/ram.c b/migration/ram.c
index 407760b3a2..7bbdb69501 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -3059,7 +3059,11 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
if (ret < 0) {
qemu_file_set_error(f, ret);
}
- ram_control_after_iterate(f, RAM_CONTROL_SETUP);
+
+ ret = qemu_rdma_registration_stop(f, RAM_CONTROL_SETUP);
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
migration_ops = g_malloc0(sizeof(MigrationOps));
migration_ops->ram_save_target_page = ram_save_target_page_legacy;
@@ -3178,7 +3182,10 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
* Must occur before EOS (or any QEMUFile operation)
* because of RDMA protocol.
*/
- ram_control_after_iterate(f, RAM_CONTROL_ROUND);
+ ret = qemu_rdma_registration_stop(f, RAM_CONTROL_ROUND);
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
out:
if (ret >= 0
@@ -3251,7 +3258,11 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
qemu_mutex_unlock(&rs->bitmap_mutex);
ram_flush_compressed_data(rs);
- ram_control_after_iterate(f, RAM_CONTROL_FINISH);
+
+ int ret = qemu_rdma_registration_stop(f, RAM_CONTROL_FINISH);
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
}
if (ret < 0) {
diff --git a/migration/rdma.c b/migration/rdma.c
index a93d8d705d..0eff308577 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3894,19 +3894,19 @@ int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags)
* Inform dest that dynamic registrations are done for now.
* First, flush writes, if any.
*/
-static int qemu_rdma_registration_stop(QEMUFile *f,
- uint64_t flags, void *data)
+int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags)
{
- QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
+ QIOChannelRDMA *rioc;
RDMAContext *rdma;
RDMAControlHeader head = { .len = 0, .repeat = 1 };
int ret = 0;
- if (migration_in_postcopy()) {
+ if (!migrate_rdma() || migration_in_postcopy()) {
return 0;
}
RCU_READ_LOCK_GUARD();
+ rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
rdma = qatomic_rcu_read(&rioc->rdmaout);
if (!rdma) {
return -EIO;
@@ -4009,7 +4009,6 @@ static const QEMUFileHooks rdma_read_hooks = {
};
static const QEMUFileHooks rdma_write_hooks = {
- .after_ram_iterate = qemu_rdma_registration_stop,
.save_page = qemu_rdma_save_page,
};
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 04/10] migration/rdma: Remove all uses of RAM_CONTROL_HOOK
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
` (2 preceding siblings ...)
2023-10-04 12:49 ` [PATCH v2 03/10] migration/rdma: Unfold ram_control_after_iterate() Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 12:49 ` [PATCH v2 05/10] migration/rdma: Unfold hook_ram_load() Juan Quintela
` (6 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
Instead of going trhough ram_control_load_hook(), call
qemu_rdma_registration_handle() directly.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/qemu-file.h | 1 -
migration/rdma.h | 3 +++
migration/ram.c | 5 ++++-
migration/rdma.c | 12 +++++++-----
4 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index 35e671a01e..14ff0d9cc4 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -41,7 +41,6 @@ typedef int (QEMURamHookFunc)(QEMUFile *f, uint64_t flags, void *data);
*/
#define RAM_CONTROL_SETUP 0
#define RAM_CONTROL_ROUND 1
-#define RAM_CONTROL_HOOK 2
#define RAM_CONTROL_FINISH 3
#define RAM_CONTROL_BLOCK_REG 4
diff --git a/migration/rdma.h b/migration/rdma.h
index c13b94c782..8bd277efb9 100644
--- a/migration/rdma.h
+++ b/migration/rdma.h
@@ -24,10 +24,13 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp);
#ifdef CONFIG_RDMA
+int qemu_rdma_registration_handle(QEMUFile *f);
int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags);
int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags);
#else
static inline
+int qemu_rdma_registration_handle(QEMUFile *f) { return 0; }
+static inline
int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; }
static inline
int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; }
diff --git a/migration/ram.c b/migration/ram.c
index 7bbdb69501..b3abcad1b4 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4064,7 +4064,10 @@ static int ram_load_precopy(QEMUFile *f)
}
break;
case RAM_SAVE_FLAG_HOOK:
- ram_control_load_hook(f, RAM_CONTROL_HOOK, NULL);
+ ret = qemu_rdma_registration_handle(f);
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
break;
default:
error_report("Unknown combination of migration flags: 0x%x", flags);
diff --git a/migration/rdma.c b/migration/rdma.c
index 0eff308577..6208f1f818 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3535,7 +3535,7 @@ static int dest_ram_sort_func(const void *a, const void *b)
*
* Keep doing this until the source tells us to stop.
*/
-static int qemu_rdma_registration_handle(QEMUFile *f)
+int qemu_rdma_registration_handle(QEMUFile *f)
{
RDMAControlHeader reg_resp = { .len = sizeof(RDMARegisterResult),
.type = RDMA_CONTROL_REGISTER_RESULT,
@@ -3547,7 +3547,7 @@ static int qemu_rdma_registration_handle(QEMUFile *f)
};
RDMAControlHeader blocks = { .type = RDMA_CONTROL_RAM_BLOCKS_RESULT,
.repeat = 1 };
- QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
+ QIOChannelRDMA *rioc;
RDMAContext *rdma;
RDMALocalBlocks *local;
RDMAControlHeader head;
@@ -3562,7 +3562,12 @@ static int qemu_rdma_registration_handle(QEMUFile *f)
int count = 0;
int i = 0;
+ if (!migrate_rdma()) {
+ return 0;
+ }
+
RCU_READ_LOCK_GUARD();
+ rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
rdma = qatomic_rcu_read(&rioc->rdmain);
if (!rdma) {
@@ -3859,9 +3864,6 @@ static int rdma_load_hook(QEMUFile *f, uint64_t flags, void *data)
case RAM_CONTROL_BLOCK_REG:
return rdma_block_notification_handle(f, data);
- case RAM_CONTROL_HOOK:
- return qemu_rdma_registration_handle(f);
-
default:
/* Shouldn't be called with any other values */
abort();
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 05/10] migration/rdma: Unfold hook_ram_load()
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
` (3 preceding siblings ...)
2023-10-04 12:49 ` [PATCH v2 04/10] migration/rdma: Remove all uses of RAM_CONTROL_HOOK Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 12:49 ` [PATCH v2 06/10] migration/rdma: Create rdma_control_save_page() Juan Quintela
` (5 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
There is only one flag called with: RAM_CONTROL_BLOCK_REG.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/qemu-file.h | 11 -----------
migration/rdma.h | 3 +++
migration/qemu-file.c | 10 ----------
migration/ram.c | 6 ++++--
migration/rdma.c | 34 +++++++++++-----------------------
5 files changed, 18 insertions(+), 46 deletions(-)
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index 14ff0d9cc4..80c30631dc 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -29,20 +29,12 @@
#include "exec/cpu-common.h"
#include "io/channel.h"
-/*
- * This function provides hooks around different
- * stages of RAM migration.
- * 'data' is call specific data associated with the 'flags' value
- */
-typedef int (QEMURamHookFunc)(QEMUFile *f, uint64_t flags, void *data);
-
/*
* Constants used by ram_control_* hooks
*/
#define RAM_CONTROL_SETUP 0
#define RAM_CONTROL_ROUND 1
#define RAM_CONTROL_FINISH 3
-#define RAM_CONTROL_BLOCK_REG 4
/*
* This function allows override of where the RAM page
@@ -54,7 +46,6 @@ typedef int (QEMURamSaveFunc)(QEMUFile *f,
size_t size);
typedef struct QEMUFileHooks {
- QEMURamHookFunc *hook_ram_load;
QEMURamSaveFunc *save_page;
} QEMUFileHooks;
@@ -124,8 +115,6 @@ void qemu_fflush(QEMUFile *f);
void qemu_file_set_blocking(QEMUFile *f, bool block);
int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size);
-void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data);
-
/* Whenever this is found in the data stream, the flags
* will be passed to ram_control_load_hook in the incoming-migration
* side. This lets before_ram_iterate/after_ram_iterate add
diff --git a/migration/rdma.h b/migration/rdma.h
index 8bd277efb9..8df8b4089a 100644
--- a/migration/rdma.h
+++ b/migration/rdma.h
@@ -27,6 +27,7 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp);
int qemu_rdma_registration_handle(QEMUFile *f);
int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags);
int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags);
+int rdma_block_notification_handle(QEMUFile *f, const char *name);
#else
static inline
int qemu_rdma_registration_handle(QEMUFile *f) { return 0; }
@@ -34,5 +35,7 @@ static inline
int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; }
static inline
int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; }
+static inline
+int rdma_block_notification_handle(QEMUFile *f, const char *name) { return 0; }
#endif
#endif
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 35c5c9109b..a15d1f6635 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -289,16 +289,6 @@ void qemu_fflush(QEMUFile *f)
f->iovcnt = 0;
}
-void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data)
-{
- if (f->hooks && f->hooks->hook_ram_load) {
- int ret = f->hooks->hook_ram_load(f, flags, data);
- if (ret < 0) {
- qemu_file_set_error(f, ret);
- }
- }
-}
-
int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset,
ram_addr_t offset, size_t size)
{
diff --git a/migration/ram.c b/migration/ram.c
index b3abcad1b4..656e1dd96c 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4015,8 +4015,10 @@ static int ram_load_precopy(QEMUFile *f)
ret = -EINVAL;
}
}
- ram_control_load_hook(f, RAM_CONTROL_BLOCK_REG,
- block->idstr);
+ ret = rdma_block_notification_handle(f, block->idstr);
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
} else {
error_report("Unknown ramblock \"%s\", cannot "
"accept migration", id);
diff --git a/migration/rdma.c b/migration/rdma.c
index 6208f1f818..65514999a3 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3817,22 +3817,23 @@ out:
}
/* Destination:
- * Called via a ram_control_load_hook during the initial RAM load section which
- * lists the RAMBlocks by name. This lets us know the order of the RAMBlocks
- * on the source.
- * We've already built our local RAMBlock list, but not yet sent the list to
- * the source.
+ * Called during the initial RAM load section which lists the
+ * RAMBlocks by name. This lets us know the order of the RAMBlocks on
+ * the source. We've already built our local RAMBlock list, but not
+ * yet sent the list to the source.
*/
-static int
-rdma_block_notification_handle(QEMUFile *f, const char *name)
+int rdma_block_notification_handle(QEMUFile *f, const char *name)
{
- RDMAContext *rdma;
- QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
int curr;
int found = -1;
+ if (!migrate_rdma()) {
+ return 0;
+ }
+
RCU_READ_LOCK_GUARD();
- rdma = qatomic_rcu_read(&rioc->rdmain);
+ QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(qemu_file_get_ioc(f));
+ RDMAContext *rdma = qatomic_rcu_read(&rioc->rdmain);
if (!rdma) {
return -EIO;
@@ -3858,18 +3859,6 @@ rdma_block_notification_handle(QEMUFile *f, const char *name)
return 0;
}
-static int rdma_load_hook(QEMUFile *f, uint64_t flags, void *data)
-{
- switch (flags) {
- case RAM_CONTROL_BLOCK_REG:
- return rdma_block_notification_handle(f, data);
-
- default:
- /* Shouldn't be called with any other values */
- abort();
- }
-}
-
int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags)
{
if (!migrate_rdma () || migration_in_postcopy()) {
@@ -4007,7 +3996,6 @@ err:
}
static const QEMUFileHooks rdma_read_hooks = {
- .hook_ram_load = rdma_load_hook,
};
static const QEMUFileHooks rdma_write_hooks = {
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 06/10] migration/rdma: Create rdma_control_save_page()
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
` (4 preceding siblings ...)
2023-10-04 12:49 ` [PATCH v2 05/10] migration/rdma: Unfold hook_ram_load() Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 12:49 ` [PATCH v2 07/10] qemu-file: Remove QEMUFileHooks Juan Quintela
` (4 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
The only user of ram_control_save_page() and save_page() hook was
rdma. Just move the function to rdma.c, rename it to
rdma_control_save_page().
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/qemu-file.h | 12 ------------
migration/rdma.h | 10 ++++++++++
migration/qemu-file.c | 20 --------------------
migration/ram.c | 4 ++--
migration/rdma.c | 20 +++++++++++++++++++-
5 files changed, 31 insertions(+), 35 deletions(-)
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index 80c30631dc..60510a2819 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -36,17 +36,7 @@
#define RAM_CONTROL_ROUND 1
#define RAM_CONTROL_FINISH 3
-/*
- * This function allows override of where the RAM page
- * is saved (such as RDMA, for example.)
- */
-typedef int (QEMURamSaveFunc)(QEMUFile *f,
- ram_addr_t block_offset,
- ram_addr_t offset,
- size_t size);
-
typedef struct QEMUFileHooks {
- QEMURamSaveFunc *save_page;
} QEMUFileHooks;
QEMUFile *qemu_file_new_input(QIOChannel *ioc);
@@ -125,8 +115,6 @@ int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size);
#define RAM_SAVE_CONTROL_NOT_SUPP -1000
#define RAM_SAVE_CONTROL_DELAYED -2000
-int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset,
- ram_addr_t offset, size_t size);
QIOChannel *qemu_file_get_ioc(QEMUFile *file);
#endif
diff --git a/migration/rdma.h b/migration/rdma.h
index 8df8b4089a..09a16c1e3c 100644
--- a/migration/rdma.h
+++ b/migration/rdma.h
@@ -17,6 +17,8 @@
#ifndef QEMU_MIGRATION_RDMA_H
#define QEMU_MIGRATION_RDMA_H
+#include "exec/memory.h"
+
void rdma_start_outgoing_migration(void *opaque, const char *host_port,
Error **errp);
@@ -28,6 +30,8 @@ int qemu_rdma_registration_handle(QEMUFile *f);
int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags);
int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags);
int rdma_block_notification_handle(QEMUFile *f, const char *name);
+int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset,
+ ram_addr_t offset, size_t size);
#else
static inline
int qemu_rdma_registration_handle(QEMUFile *f) { return 0; }
@@ -37,5 +41,11 @@ static inline
int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; }
static inline
int rdma_block_notification_handle(QEMUFile *f, const char *name) { return 0; }
+static inline
+int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset,
+ ram_addr_t offset, size_t size)
+{
+ return RAM_SAVE_CONTROL_NOT_SUPP;
+}
#endif
#endif
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index a15d1f6635..e3514740fb 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -289,26 +289,6 @@ void qemu_fflush(QEMUFile *f)
f->iovcnt = 0;
}
-int ram_control_save_page(QEMUFile *f, ram_addr_t block_offset,
- ram_addr_t offset, size_t size)
-{
- if (f->hooks && f->hooks->save_page) {
- int ret = f->hooks->save_page(f, block_offset, offset, size);
- /*
- * RAM_SAVE_CONTROL_* are negative values
- */
- if (ret != RAM_SAVE_CONTROL_DELAYED &&
- ret != RAM_SAVE_CONTROL_NOT_SUPP) {
- if (ret < 0) {
- qemu_file_set_error(f, ret);
- }
- }
- return ret;
- }
-
- return RAM_SAVE_CONTROL_NOT_SUPP;
-}
-
/*
* Attempt to fill the buffer from the underlying file
* Returns the number of bytes read, or negative value for an error.
diff --git a/migration/ram.c b/migration/ram.c
index 656e1dd96c..e5d443daf2 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1189,8 +1189,8 @@ static bool control_save_page(PageSearchStatus *pss, RAMBlock *block,
{
int ret;
- ret = ram_control_save_page(pss->pss_channel, block->offset, offset,
- TARGET_PAGE_SIZE);
+ ret = rdma_control_save_page(pss->pss_channel, block->offset, offset,
+ TARGET_PAGE_SIZE);
if (ret == RAM_SAVE_CONTROL_NOT_SUPP) {
return false;
}
diff --git a/migration/rdma.c b/migration/rdma.c
index 65514999a3..7f10bfb81b 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3324,6 +3324,25 @@ err:
return ret;
}
+int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset,
+ ram_addr_t offset, size_t size)
+{
+ if (!migrate_rdma()) {
+ return RAM_SAVE_CONTROL_NOT_SUPP;
+ }
+
+ int ret = qemu_rdma_save_page(f, block_offset, offset, size);
+
+ if (ret != RAM_SAVE_CONTROL_DELAYED &&
+ ret != RAM_SAVE_CONTROL_NOT_SUPP) {
+ if (ret < 0) {
+ qemu_file_set_error(f, ret);
+ }
+ }
+ return ret;
+}
+
+
static void rdma_accept_incoming_migration(void *opaque);
static void rdma_cm_poll_handler(void *opaque)
@@ -3999,7 +4018,6 @@ static const QEMUFileHooks rdma_read_hooks = {
};
static const QEMUFileHooks rdma_write_hooks = {
- .save_page = qemu_rdma_save_page,
};
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 07/10] qemu-file: Remove QEMUFileHooks
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
` (5 preceding siblings ...)
2023-10-04 12:49 ` [PATCH v2 06/10] migration/rdma: Create rdma_control_save_page() Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 12:49 ` [PATCH v2 08/10] migration/rdma: Move rdma constants from qemu-file.h to rdma.h Juan Quintela
` (3 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
The only user was rdma, and its use is gone.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/qemu-file.h | 4 ----
migration/qemu-file.c | 6 ------
migration/rdma.c | 9 ---------
3 files changed, 19 deletions(-)
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index 60510a2819..0b22d8335f 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -36,12 +36,8 @@
#define RAM_CONTROL_ROUND 1
#define RAM_CONTROL_FINISH 3
-typedef struct QEMUFileHooks {
-} QEMUFileHooks;
-
QEMUFile *qemu_file_new_input(QIOChannel *ioc);
QEMUFile *qemu_file_new_output(QIOChannel *ioc);
-void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
int qemu_fclose(QEMUFile *f);
/*
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index e3514740fb..352bb320a2 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -38,7 +38,6 @@
#define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64)
struct QEMUFile {
- const QEMUFileHooks *hooks;
QIOChannel *ioc;
bool is_writable;
@@ -133,11 +132,6 @@ QEMUFile *qemu_file_new_input(QIOChannel *ioc)
return qemu_file_new_impl(ioc, false);
}
-void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks)
-{
- f->hooks = hooks;
-}
-
/*
* Get last error for stream f with optional Error*
*
diff --git a/migration/rdma.c b/migration/rdma.c
index 7f10bfb81b..bcb6de14e7 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -4014,13 +4014,6 @@ err:
return ret;
}
-static const QEMUFileHooks rdma_read_hooks = {
-};
-
-static const QEMUFileHooks rdma_write_hooks = {
-};
-
-
static void qio_channel_rdma_finalize(Object *obj)
{
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(obj);
@@ -4072,7 +4065,6 @@ static QEMUFile *rdma_new_input(RDMAContext *rdma)
rioc->file = qemu_file_new_input(QIO_CHANNEL(rioc));
rioc->rdmain = rdma;
rioc->rdmaout = rdma->return_path;
- qemu_file_set_hooks(rioc->file, &rdma_read_hooks);
return rioc->file;
}
@@ -4084,7 +4076,6 @@ static QEMUFile *rdma_new_output(RDMAContext *rdma)
rioc->file = qemu_file_new_output(QIO_CHANNEL(rioc));
rioc->rdmaout = rdma;
rioc->rdmain = rdma->return_path;
- qemu_file_set_hooks(rioc->file, &rdma_write_hooks);
return rioc->file;
}
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 08/10] migration/rdma: Move rdma constants from qemu-file.h to rdma.h
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
` (6 preceding siblings ...)
2023-10-04 12:49 ` [PATCH v2 07/10] qemu-file: Remove QEMUFileHooks Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 12:49 ` [PATCH v2 09/10] migration/rdma: Remove qemu_ prefix from exported functions Juan Quintela
` (2 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/qemu-file.h | 17 -----------------
migration/rdma.h | 16 ++++++++++++++++
migration/ram.c | 2 +-
3 files changed, 17 insertions(+), 18 deletions(-)
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index 0b22d8335f..a29c37b0d0 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -29,13 +29,6 @@
#include "exec/cpu-common.h"
#include "io/channel.h"
-/*
- * Constants used by ram_control_* hooks
- */
-#define RAM_CONTROL_SETUP 0
-#define RAM_CONTROL_ROUND 1
-#define RAM_CONTROL_FINISH 3
-
QEMUFile *qemu_file_new_input(QIOChannel *ioc);
QEMUFile *qemu_file_new_output(QIOChannel *ioc);
int qemu_fclose(QEMUFile *f);
@@ -101,16 +94,6 @@ void qemu_fflush(QEMUFile *f);
void qemu_file_set_blocking(QEMUFile *f, bool block);
int qemu_file_get_to_fd(QEMUFile *f, int fd, size_t size);
-/* Whenever this is found in the data stream, the flags
- * will be passed to ram_control_load_hook in the incoming-migration
- * side. This lets before_ram_iterate/after_ram_iterate add
- * transport-specific sections to the RAM migration data.
- */
-#define RAM_SAVE_FLAG_HOOK 0x80
-
-#define RAM_SAVE_CONTROL_NOT_SUPP -1000
-#define RAM_SAVE_CONTROL_DELAYED -2000
-
QIOChannel *qemu_file_get_ioc(QEMUFile *file);
#endif
diff --git a/migration/rdma.h b/migration/rdma.h
index 09a16c1e3c..1ff3718a76 100644
--- a/migration/rdma.h
+++ b/migration/rdma.h
@@ -24,6 +24,22 @@ void rdma_start_outgoing_migration(void *opaque, const char *host_port,
void rdma_start_incoming_migration(const char *host_port, Error **errp);
+/*
+ * Constants used by rdma return codes
+ */
+#define RAM_CONTROL_SETUP 0
+#define RAM_CONTROL_ROUND 1
+#define RAM_CONTROL_FINISH 3
+
+/*
+ * Whenever this is found in the data stream, the flags
+ * will be passed to rdma functions in the incoming-migration
+ * side.
+ */
+#define RAM_SAVE_FLAG_HOOK 0x80
+
+#define RAM_SAVE_CONTROL_NOT_SUPP -1000
+#define RAM_SAVE_CONTROL_DELAYED -2000
#ifdef CONFIG_RDMA
int qemu_rdma_registration_handle(QEMUFile *f);
diff --git a/migration/ram.c b/migration/ram.c
index e5d443daf2..f3bf7b9185 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -89,7 +89,7 @@
#define RAM_SAVE_FLAG_EOS 0x10
#define RAM_SAVE_FLAG_CONTINUE 0x20
#define RAM_SAVE_FLAG_XBZRLE 0x40
-/* 0x80 is reserved in qemu-file.h for RAM_SAVE_FLAG_HOOK */
+/* 0x80 is reserved in rdma.h for RAM_SAVE_FLAG_HOOK */
#define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100
#define RAM_SAVE_FLAG_MULTIFD_FLUSH 0x200
/* We can't use any flag that is bigger than 0x200 */
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 09/10] migration/rdma: Remove qemu_ prefix from exported functions
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
` (7 preceding siblings ...)
2023-10-04 12:49 ` [PATCH v2 08/10] migration/rdma: Move rdma constants from qemu-file.h to rdma.h Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 12:49 ` [PATCH v2 10/10] migration/rdma: Check sooner if we are in postcopy for save_page() Juan Quintela
2023-10-04 17:11 ` [PATCH v2 00/10] Removal of QEMUFileHooks Peter Xu
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
Functions are long enough even without this.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/rdma.h | 12 ++++++------
migration/ram.c | 14 +++++++-------
migration/rdma.c | 40 +++++++++++++++++++---------------------
migration/trace-events | 28 ++++++++++++++--------------
4 files changed, 46 insertions(+), 48 deletions(-)
diff --git a/migration/rdma.h b/migration/rdma.h
index 1ff3718a76..30b15b4466 100644
--- a/migration/rdma.h
+++ b/migration/rdma.h
@@ -42,19 +42,19 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp);
#define RAM_SAVE_CONTROL_DELAYED -2000
#ifdef CONFIG_RDMA
-int qemu_rdma_registration_handle(QEMUFile *f);
-int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags);
-int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags);
+int rdma_registration_handle(QEMUFile *f);
+int rdma_registration_start(QEMUFile *f, uint64_t flags);
+int rdma_registration_stop(QEMUFile *f, uint64_t flags);
int rdma_block_notification_handle(QEMUFile *f, const char *name);
int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset,
ram_addr_t offset, size_t size);
#else
static inline
-int qemu_rdma_registration_handle(QEMUFile *f) { return 0; }
+int rdma_registration_handle(QEMUFile *f) { return 0; }
static inline
-int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; }
+int rdma_registration_start(QEMUFile *f, uint64_t flags) { return 0; }
static inline
-int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; }
+int rdma_registration_stop(QEMUFile *f, uint64_t flags) { return 0; }
static inline
int rdma_block_notification_handle(QEMUFile *f, const char *name) { return 0; }
static inline
diff --git a/migration/ram.c b/migration/ram.c
index f3bf7b9185..abfc57a58c 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -3055,12 +3055,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
}
}
- ret = qemu_rdma_registration_start(f, RAM_CONTROL_SETUP);
+ ret = rdma_registration_start(f, RAM_CONTROL_SETUP);
if (ret < 0) {
qemu_file_set_error(f, ret);
}
- ret = qemu_rdma_registration_stop(f, RAM_CONTROL_SETUP);
+ ret = rdma_registration_stop(f, RAM_CONTROL_SETUP);
if (ret < 0) {
qemu_file_set_error(f, ret);
}
@@ -3122,7 +3122,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
/* Read version before ram_list.blocks */
smp_rmb();
- ret = qemu_rdma_registration_start(f, RAM_CONTROL_ROUND);
+ ret = rdma_registration_start(f, RAM_CONTROL_ROUND);
if (ret < 0) {
qemu_file_set_error(f, ret);
}
@@ -3182,7 +3182,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
* Must occur before EOS (or any QEMUFile operation)
* because of RDMA protocol.
*/
- ret = qemu_rdma_registration_stop(f, RAM_CONTROL_ROUND);
+ ret = rdma_registration_stop(f, RAM_CONTROL_ROUND);
if (ret < 0) {
qemu_file_set_error(f, ret);
}
@@ -3233,7 +3233,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
migration_bitmap_sync_precopy(rs, true);
}
- ret = qemu_rdma_registration_start(f, RAM_CONTROL_FINISH);
+ ret = rdma_registration_start(f, RAM_CONTROL_FINISH);
if (ret < 0) {
qemu_file_set_error(f, ret);
}
@@ -3259,7 +3259,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
ram_flush_compressed_data(rs);
- int ret = qemu_rdma_registration_stop(f, RAM_CONTROL_FINISH);
+ int ret = rdma_registration_stop(f, RAM_CONTROL_FINISH);
if (ret < 0) {
qemu_file_set_error(f, ret);
}
@@ -4066,7 +4066,7 @@ static int ram_load_precopy(QEMUFile *f)
}
break;
case RAM_SAVE_FLAG_HOOK:
- ret = qemu_rdma_registration_handle(f);
+ ret = rdma_registration_handle(f);
if (ret < 0) {
qemu_file_set_error(f, ret);
}
diff --git a/migration/rdma.c b/migration/rdma.c
index bcb6de14e7..c73e610416 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3554,7 +3554,7 @@ static int dest_ram_sort_func(const void *a, const void *b)
*
* Keep doing this until the source tells us to stop.
*/
-int qemu_rdma_registration_handle(QEMUFile *f)
+int rdma_registration_handle(QEMUFile *f)
{
RDMAControlHeader reg_resp = { .len = sizeof(RDMARegisterResult),
.type = RDMA_CONTROL_REGISTER_RESULT,
@@ -3597,7 +3597,7 @@ int qemu_rdma_registration_handle(QEMUFile *f)
local = &rdma->local_ram_blocks;
do {
- trace_qemu_rdma_registration_handle_wait();
+ trace_rdma_registration_handle_wait();
ret = qemu_rdma_exchange_recv(rdma, &head, RDMA_CONTROL_NONE);
@@ -3617,9 +3617,9 @@ int qemu_rdma_registration_handle(QEMUFile *f)
comp = (RDMACompress *) rdma->wr_data[idx].control_curr;
network_to_compress(comp);
- trace_qemu_rdma_registration_handle_compress(comp->length,
- comp->block_idx,
- comp->offset);
+ trace_rdma_registration_handle_compress(comp->length,
+ comp->block_idx,
+ comp->offset);
if (comp->block_idx >= rdma->local_ram_blocks.nb_blocks) {
error_report("rdma: 'compress' bad block index %u (vs %d)",
(unsigned int)comp->block_idx,
@@ -3636,11 +3636,11 @@ int qemu_rdma_registration_handle(QEMUFile *f)
break;
case RDMA_CONTROL_REGISTER_FINISHED:
- trace_qemu_rdma_registration_handle_finished();
+ trace_rdma_registration_handle_finished();
goto out;
case RDMA_CONTROL_RAM_BLOCKS_REQUEST:
- trace_qemu_rdma_registration_handle_ram_blocks();
+ trace_rdma_registration_handle_ram_blocks();
/* Sort our local RAM Block list so it's the same as the source,
* we can do this since we've filled in a src_index in the list
@@ -3680,7 +3680,7 @@ int qemu_rdma_registration_handle(QEMUFile *f)
rdma->dest_blocks[i].length = local->block[i].length;
dest_block_to_network(&rdma->dest_blocks[i]);
- trace_qemu_rdma_registration_handle_ram_blocks_loop(
+ trace_rdma_registration_handle_ram_blocks_loop(
local->block[i].block_name,
local->block[i].offset,
local->block[i].length,
@@ -3702,7 +3702,7 @@ int qemu_rdma_registration_handle(QEMUFile *f)
break;
case RDMA_CONTROL_REGISTER_REQUEST:
- trace_qemu_rdma_registration_handle_register(head.repeat);
+ trace_rdma_registration_handle_register(head.repeat);
reg_resp.repeat = head.repeat;
registers = (RDMARegister *) rdma->wr_data[idx].control_curr;
@@ -3716,7 +3716,7 @@ int qemu_rdma_registration_handle(QEMUFile *f)
reg_result = &results[count];
- trace_qemu_rdma_registration_handle_register_loop(count,
+ trace_rdma_registration_handle_register_loop(count,
reg->current_index, reg->key.current_addr, reg->chunks);
if (reg->current_index >= rdma->local_ram_blocks.nb_blocks) {
@@ -3768,8 +3768,7 @@ int qemu_rdma_registration_handle(QEMUFile *f)
reg_result->host_addr = (uintptr_t)block->local_host_addr;
- trace_qemu_rdma_registration_handle_register_rkey(
- reg_result->rkey);
+ trace_rdma_registration_handle_register_rkey(reg_result->rkey);
result_to_network(reg_result);
}
@@ -3783,7 +3782,7 @@ int qemu_rdma_registration_handle(QEMUFile *f)
}
break;
case RDMA_CONTROL_UNREGISTER_REQUEST:
- trace_qemu_rdma_registration_handle_unregister(head.repeat);
+ trace_rdma_registration_handle_unregister(head.repeat);
unreg_resp.repeat = head.repeat;
registers = (RDMARegister *) rdma->wr_data[idx].control_curr;
@@ -3791,7 +3790,7 @@ int qemu_rdma_registration_handle(QEMUFile *f)
reg = ®isters[count];
network_to_register(reg);
- trace_qemu_rdma_registration_handle_unregister_loop(count,
+ trace_rdma_registration_handle_unregister_loop(count,
reg->current_index, reg->key.chunk);
block = &(rdma->local_ram_blocks.block[reg->current_index]);
@@ -3807,8 +3806,7 @@ int qemu_rdma_registration_handle(QEMUFile *f)
rdma->total_registrations--;
- trace_qemu_rdma_registration_handle_unregister_success(
- reg->key.chunk);
+ trace_rdma_registration_handle_unregister_success(reg->key.chunk);
}
ret = qemu_rdma_post_send_control(rdma, NULL, &unreg_resp);
@@ -3878,7 +3876,7 @@ int rdma_block_notification_handle(QEMUFile *f, const char *name)
return 0;
}
-int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags)
+int rdma_registration_start(QEMUFile *f, uint64_t flags)
{
if (!migrate_rdma () || migration_in_postcopy()) {
return 0;
@@ -3893,7 +3891,7 @@ int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags)
CHECK_ERROR_STATE();
- trace_qemu_rdma_registration_start(flags);
+ trace_rdma_registration_start(flags);
qemu_put_be64(f, RAM_SAVE_FLAG_HOOK);
qemu_fflush(f);
@@ -3904,7 +3902,7 @@ int qemu_rdma_registration_start(QEMUFile *f, uint64_t flags)
* Inform dest that dynamic registrations are done for now.
* First, flush writes, if any.
*/
-int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags)
+int rdma_registration_stop(QEMUFile *f, uint64_t flags)
{
QIOChannelRDMA *rioc;
RDMAContext *rdma;
@@ -3937,7 +3935,7 @@ int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags)
int reg_result_idx, i, nb_dest_blocks;
head.type = RDMA_CONTROL_RAM_BLOCKS_REQUEST;
- trace_qemu_rdma_registration_stop_ram();
+ trace_rdma_registration_stop_ram();
/*
* Make sure that we parallelize the pinning on both sides.
@@ -3999,7 +3997,7 @@ int qemu_rdma_registration_stop(QEMUFile *f, uint64_t flags)
}
}
- trace_qemu_rdma_registration_stop(flags);
+ trace_rdma_registration_stop(flags);
head.type = RDMA_CONTROL_REGISTER_FINISHED;
ret = qemu_rdma_exchange_send(rdma, &head, NULL, NULL, NULL, NULL);
diff --git a/migration/trace-events b/migration/trace-events
index 002abe3a4e..3f86575f41 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -229,20 +229,6 @@ qemu_rdma_post_send_control(const char *desc) "CONTROL: sending %s.."
qemu_rdma_register_and_get_keys(uint64_t len, void *start) "Registering %" PRIu64 " bytes @ %p"
qemu_rdma_register_odp_mr(const char *name) "Try to register On-Demand Paging memory region: %s"
qemu_rdma_advise_mr(const char *name, uint32_t len, uint64_t addr, const char *res) "Try to advise block %s prefetch at %" PRIu32 "@0x%" PRIx64 ": %s"
-qemu_rdma_registration_handle_compress(int64_t length, int index, int64_t offset) "Zapping zero chunk: %" PRId64 " bytes, index %d, offset %" PRId64
-qemu_rdma_registration_handle_finished(void) ""
-qemu_rdma_registration_handle_ram_blocks(void) ""
-qemu_rdma_registration_handle_ram_blocks_loop(const char *name, uint64_t offset, uint64_t length, void *local_host_addr, unsigned int src_index) "%s: @0x%" PRIx64 "/%" PRIu64 " host:@%p src_index: %u"
-qemu_rdma_registration_handle_register(int requests) "%d requests"
-qemu_rdma_registration_handle_register_loop(int req, int index, uint64_t addr, uint64_t chunks) "Registration request (%d): index %d, current_addr %" PRIu64 " chunks: %" PRIu64
-qemu_rdma_registration_handle_register_rkey(int rkey) "0x%x"
-qemu_rdma_registration_handle_unregister(int requests) "%d requests"
-qemu_rdma_registration_handle_unregister_loop(int count, int index, uint64_t chunk) "Unregistration request (%d): index %d, chunk %" PRIu64
-qemu_rdma_registration_handle_unregister_success(uint64_t chunk) "%" PRIu64
-qemu_rdma_registration_handle_wait(void) ""
-qemu_rdma_registration_start(uint64_t flags) "%" PRIu64
-qemu_rdma_registration_stop(uint64_t flags) "%" PRIu64
-qemu_rdma_registration_stop_ram(void) ""
qemu_rdma_resolve_host_trying(const char *host, const char *ip) "Trying %s => %s"
qemu_rdma_signal_unregister_append(uint64_t chunk, int pos) "Appending unregister chunk %" PRIu64 " at position %d"
qemu_rdma_signal_unregister_already(uint64_t chunk) "Unregister chunk %" PRIu64 " already in queue"
@@ -261,6 +247,20 @@ qemu_rdma_write_one_zero(uint64_t chunk, int len, int index, int64_t offset) "En
rdma_add_block(const char *block_name, int block, uint64_t addr, uint64_t offset, uint64_t len, uint64_t end, uint64_t bits, int chunks) "Added Block: '%s':%d, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " bits %" PRIu64 " chunks %d"
rdma_block_notification_handle(const char *name, int index) "%s at %d"
rdma_delete_block(void *block, uint64_t addr, uint64_t offset, uint64_t len, uint64_t end, uint64_t bits, int chunks) "Deleted Block: %p, addr: %" PRIu64 ", offset: %" PRIu64 " length: %" PRIu64 " end: %" PRIu64 " bits %" PRIu64 " chunks %d"
+rdma_registration_handle_compress(int64_t length, int index, int64_t offset) "Zapping zero chunk: %" PRId64 " bytes, index %d, offset %" PRId64
+rdma_registration_handle_finished(void) ""
+rdma_registration_handle_ram_blocks(void) ""
+rdma_registration_handle_ram_blocks_loop(const char *name, uint64_t offset, uint64_t length, void *local_host_addr, unsigned int src_index) "%s: @0x%" PRIx64 "/%" PRIu64 " host:@%p src_index: %u"
+rdma_registration_handle_register(int requests) "%d requests"
+rdma_registration_handle_register_loop(int req, int index, uint64_t addr, uint64_t chunks) "Registration request (%d): index %d, current_addr %" PRIu64 " chunks: %" PRIu64
+rdma_registration_handle_register_rkey(int rkey) "0x%x"
+rdma_registration_handle_unregister(int requests) "%d requests"
+rdma_registration_handle_unregister_loop(int count, int index, uint64_t chunk) "Unregistration request (%d): index %d, chunk %" PRIu64
+rdma_registration_handle_unregister_success(uint64_t chunk) "%" PRIu64
+rdma_registration_handle_wait(void) ""
+rdma_registration_start(uint64_t flags) "%" PRIu64
+rdma_registration_stop(uint64_t flags) "%" PRIu64
+rdma_registration_stop_ram(void) ""
rdma_start_incoming_migration(void) ""
rdma_start_incoming_migration_after_dest_init(void) ""
rdma_start_incoming_migration_after_rdma_listen(void) ""
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 10/10] migration/rdma: Check sooner if we are in postcopy for save_page()
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
` (8 preceding siblings ...)
2023-10-04 12:49 ` [PATCH v2 09/10] migration/rdma: Remove qemu_ prefix from exported functions Juan Quintela
@ 2023-10-04 12:49 ` Juan Quintela
2023-10-04 17:11 ` [PATCH v2 00/10] Removal of QEMUFileHooks Peter Xu
10 siblings, 0 replies; 14+ messages in thread
From: Juan Quintela @ 2023-10-04 12:49 UTC (permalink / raw)
To: qemu-devel
Cc: Li Zhijian, Juan Quintela, Peter Xu, Fabiano Rosas, Leonardo Bras
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/rdma.c | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/migration/rdma.c b/migration/rdma.c
index c73e610416..62293ab1d6 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3253,10 +3253,6 @@ static int qemu_rdma_save_page(QEMUFile *f, ram_addr_t block_offset,
RDMAContext *rdma;
int ret;
- if (migration_in_postcopy()) {
- return RAM_SAVE_CONTROL_NOT_SUPP;
- }
-
RCU_READ_LOCK_GUARD();
rdma = qatomic_rcu_read(&rioc->rdmaout);
@@ -3327,7 +3323,7 @@ err:
int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset,
ram_addr_t offset, size_t size)
{
- if (!migrate_rdma()) {
+ if (!migrate_rdma() || migration_in_postcopy()) {
return RAM_SAVE_CONTROL_NOT_SUPP;
}
--
2.41.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v2 00/10] Removal of QEMUFileHooks
2023-10-04 12:49 [PATCH v2 00/10] Removal of QEMUFileHooks Juan Quintela
` (9 preceding siblings ...)
2023-10-04 12:49 ` [PATCH v2 10/10] migration/rdma: Check sooner if we are in postcopy for save_page() Juan Quintela
@ 2023-10-04 17:11 ` Peter Xu
2023-10-04 17:26 ` Juan Quintela
10 siblings, 1 reply; 14+ messages in thread
From: Peter Xu @ 2023-10-04 17:11 UTC (permalink / raw)
To: Juan Quintela; +Cc: qemu-devel, Li Zhijian, Fabiano Rosas, Leonardo Bras
On Wed, Oct 04, 2023 at 02:49:03PM +0200, Juan Quintela wrote:
> I remove the reviewed by from peter for 2-6 because I had to touch
> those files.
You yet didn't? :)
I saw you plan to rebase yours onto Markus's. I'll wait for a new version.
Thanks,
--
Peter Xu
^ permalink raw reply [flat|nested] 14+ messages in thread