From: Juan Quintela <quintela@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Daniel Henrique Barboza" <danielhb413@gmail.com>,
"Christian Borntraeger" <borntraeger@linux.ibm.com>,
"David Hildenbrand" <david@redhat.com>,
"Stefan Hajnoczi" <stefanha@redhat.com>,
qemu-block@nongnu.org, "Eric Blake" <eblake@redhat.com>,
"Vladimir Sementsov-Ogievskiy" <vsementsov@yandex-team.ru>,
"John Snow" <jsnow@redhat.com>,
"Juan Quintela" <quintela@redhat.com>,
"Halil Pasic" <pasic@linux.ibm.com>,
"Peter Xu" <peterx@redhat.com>,
"Richard Henderson" <richard.henderson@linaro.org>,
"David Gibson" <david@gibson.dropbear.id.au>,
"Harsh Prateek Bora" <harshpb@linux.ibm.com>,
"Eric Farman" <farman@linux.ibm.com>,
"Greg Kurz" <groug@kaod.org>,
qemu-ppc@nongnu.org, qemu-s390x@nongnu.org,
"Fam Zheng" <fam@euphon.net>, "Thomas Huth" <thuth@redhat.com>,
"Cédric Le Goater" <clg@kaod.org>,
"Leonardo Bras" <leobras@redhat.com>,
"Ilya Leoshkevich" <iii@linux.ibm.com>
Subject: [PATCH 15/21] migration: Don't abuse qemu_file transferred for RDMA
Date: Mon, 8 May 2023 15:09:03 +0200 [thread overview]
Message-ID: <20230508130909.65420-16-quintela@redhat.com> (raw)
In-Reply-To: <20230508130909.65420-1-quintela@redhat.com>
Just create a variable for it, the same way that multifd does. This
way it is safe to use for other thread, etc, etc.
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
migration/migration-stats.c | 5 +++--
migration/migration-stats.h | 4 ++++
migration/rdma.c | 22 ++++++++++++++++++++--
migration/trace-events | 2 +-
4 files changed, 28 insertions(+), 5 deletions(-)
diff --git a/migration/migration-stats.c b/migration/migration-stats.c
index a42b5d953e..1c2c0b3077 100644
--- a/migration/migration-stats.c
+++ b/migration/migration-stats.c
@@ -65,8 +65,9 @@ void migration_rate_limit_reset(QEMUFile *f)
uint64_t migration_transferred_bytes(QEMUFile *f)
{
uint64_t multifd = stat64_get(&mig_stats.multifd_bytes);
+ uint64_t rdma = stat64_get(&mig_stats.rdma_bytes);
uint64_t qemu_file = qemu_file_transferred(f);
- trace_migration_transferred_bytes(qemu_file, multifd);
- return qemu_file + multifd;
+ trace_migration_transferred_bytes(qemu_file, multifd, rdma);
+ return qemu_file + multifd + rdma;
}
diff --git a/migration/migration-stats.h b/migration/migration-stats.h
index ab4cc15a74..cbab99cfdc 100644
--- a/migration/migration-stats.h
+++ b/migration/migration-stats.h
@@ -77,6 +77,10 @@ typedef struct {
* Maximum amount of data we can send in a cycle.
*/
Stat64 rate_limit_max;
+ /*
+ * Number of bytes sent through RDMA.
+ */
+ Stat64 rdma_bytes;
/*
* How long has the setup stage took.
*/
diff --git a/migration/rdma.c b/migration/rdma.c
index 2cd8f1cc66..941797506a 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2122,9 +2122,18 @@ retry:
return -EIO;
}
+ /*
+ * TODO: Here we are sending something, but we are not
+ * accounting for anything transferred. The following is wrong:
+ *
+ * stat64_add(&mig_stats.rdma_bytes, sge.length);
+ *
+ * because we are using some kind of compression. I
+ * would think that head.len would be the more similar
+ * thing to a correct value.
+ */
stat64_add(&mig_stats.zero_pages,
sge.length / qemu_target_page_size());
-
return 1;
}
@@ -2232,8 +2241,17 @@ retry:
set_bit(chunk, block->transit_bitmap);
stat64_add(&mig_stats.normal_pages, sge.length / qemu_target_page_size());
+ /*
+ * We are adding to transferred the amount of data written, but no
+ * overhead at all. I will asume that RDMA is magicaly and don't
+ * need to transfer (at least) the addresses where it wants to
+ * write the pages. Here it looks like it should be something
+ * like:
+ * sizeof(send_wr) + sge.length
+ * but this being RDMA, who knows.
+ */
+ stat64_add(&mig_stats.rdma_bytes, sge.length);
ram_transferred_add(sge.length);
- qemu_file_credit_transfer(f, sge.length);
rdma->total_writes++;
return 0;
diff --git a/migration/trace-events b/migration/trace-events
index 4b6e802833..800cfce547 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -187,7 +187,7 @@ process_incoming_migration_co_postcopy_end_main(void) ""
postcopy_preempt_enabled(bool value) "%d"
# migration-stats
-migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd) "qemu_file %" PRIu64 " multifd %" PRIu64
+migration_transferred_bytes(uint64_t qemu_file, uint64_t multifd, uint64_t rdma) "qemu_file %" PRIu64 " multifd %" PRIu64 " RDMA %" PRIu64
# channel.c
migration_set_incoming_channel(void *ioc, const char *ioctype) "ioc=%p ioctype=%s"
--
2.40.0
next prev parent reply other threads:[~2023-05-08 13:12 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-08 13:08 [PATCH 00/21] Migration: More migration atomic counters Juan Quintela
2023-05-08 13:08 ` [PATCH 01/21] migration: A rate limit value of 0 is valid Juan Quintela
2023-05-15 8:33 ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 02/21] migration: Don't use INT64_MAX for unlimited rate Juan Quintela
2023-05-09 11:41 ` Harsh Prateek Bora
2023-05-09 11:51 ` Juan Quintela
2023-05-09 12:02 ` Harsh Prateek Bora
2023-05-15 8:34 ` Cédric Le Goater
2023-05-15 11:18 ` Juan Quintela
2023-05-08 13:08 ` [PATCH 03/21] migration: We set the rate_limit by a second Juan Quintela
2023-05-15 8:38 ` Cédric Le Goater
2023-05-15 11:18 ` Juan Quintela
2023-05-08 13:08 ` [PATCH 04/21] qemu-file: make qemu_file_[sg]et_rate_limit() use an uint64_t Juan Quintela
2023-05-15 8:38 ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 05/21] qemu-file: Make rate_limit_used " Juan Quintela
2023-05-15 8:40 ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 06/21] qemu-file: Remove total from qemu_file_total_transferred_*() Juan Quintela
2023-05-15 9:33 ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 07/21] migration: Correct transferred bytes value Juan Quintela
2023-05-09 12:08 ` Harsh Prateek Bora
2023-05-09 14:17 ` Juan Quintela
2023-05-08 13:08 ` [PATCH 08/21] migration: Move setup_time to mig_stats Juan Quintela
2023-05-15 10:35 ` Cédric Le Goater
2023-05-15 11:23 ` Juan Quintela
2023-05-08 13:08 ` [PATCH 09/21] qemu-file: Account for rate_limit usage on qemu_fflush() Juan Quintela
2023-05-15 12:15 ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 10/21] migration: Move rate_limit_max and rate_limit_used to migration_stats Juan Quintela
2023-05-09 10:27 ` Harsh Prateek Bora
2023-05-09 11:10 ` Juan Quintela
2023-05-15 8:51 ` Harsh Prateek Bora
2023-05-15 13:02 ` Cédric Le Goater
2023-05-15 13:09 ` Juan Quintela
2023-05-15 13:28 ` Cédric Le Goater
2023-05-15 13:33 ` Juan Quintela
2023-05-15 17:16 ` Juan Quintela
2023-05-08 13:08 ` [PATCH 11/21] migration: Move migration_total_bytes() to migration-stats.c Juan Quintela
2023-05-15 13:02 ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 12/21] migration: Add a trace for migration_transferred_bytes Juan Quintela
2023-05-15 13:02 ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 13/21] migration: Use migration_transferred_bytes() to calculate rate_limit Juan Quintela
2023-05-15 13:02 ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 14/21] migration: We don't need the field rate_limit_used anymore Juan Quintela
2023-05-15 13:02 ` Cédric Le Goater
2023-05-08 13:09 ` Juan Quintela [this message]
2023-05-08 13:09 ` [PATCH 16/21] migration/RDMA: It is accounting for zero/normal pages in two places Juan Quintela
2023-05-08 13:09 ` [PATCH 17/21] migration/rdma: Remove QEMUFile parameter when not used Juan Quintela
2023-05-08 13:09 ` [PATCH 18/21] migration/rdma: Don't use imaginary transfers Juan Quintela
2023-05-08 13:09 ` [PATCH 19/21] migration: Remove unused qemu_file_credit_transfer() Juan Quintela
2023-05-08 13:09 ` [PATCH 20/21] migration/rdma: Simplify the function that saves a page Juan Quintela
2023-05-08 13:09 ` [PATCH 21/21] migration/multifd: Compute transferred bytes correctly Juan Quintela
2023-05-18 16:32 ` Peter Xu
2023-05-18 16:40 ` Juan Quintela
2023-05-18 18:32 ` Peter Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230508130909.65420-16-quintela@redhat.com \
--to=quintela@redhat.com \
--cc=borntraeger@linux.ibm.com \
--cc=clg@kaod.org \
--cc=danielhb413@gmail.com \
--cc=david@gibson.dropbear.id.au \
--cc=david@redhat.com \
--cc=eblake@redhat.com \
--cc=fam@euphon.net \
--cc=farman@linux.ibm.com \
--cc=groug@kaod.org \
--cc=harshpb@linux.ibm.com \
--cc=iii@linux.ibm.com \
--cc=jsnow@redhat.com \
--cc=leobras@redhat.com \
--cc=pasic@linux.ibm.com \
--cc=peterx@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=qemu-ppc@nongnu.org \
--cc=qemu-s390x@nongnu.org \
--cc=richard.henderson@linaro.org \
--cc=stefanha@redhat.com \
--cc=thuth@redhat.com \
--cc=vsementsov@yandex-team.ru \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).