qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Juan Quintela <quintela@redhat.com>
To: Peter Xu <peterx@redhat.com>
Cc: qemu-devel@nongnu.org,
	"Daniel Henrique Barboza" <danielhb413@gmail.com>,
	"Christian Borntraeger" <borntraeger@linux.ibm.com>,
	"David Hildenbrand" <david@redhat.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	qemu-block@nongnu.org, "Eric Blake" <eblake@redhat.com>,
	"Vladimir Sementsov-Ogievskiy" <vsementsov@yandex-team.ru>,
	"John Snow" <jsnow@redhat.com>,
	"Halil Pasic" <pasic@linux.ibm.com>,
	"Richard Henderson" <richard.henderson@linaro.org>,
	"David Gibson" <david@gibson.dropbear.id.au>,
	"Harsh Prateek Bora" <harshpb@linux.ibm.com>,
	"Eric Farman" <farman@linux.ibm.com>,
	"Greg Kurz" <groug@kaod.org>,
	qemu-ppc@nongnu.org, qemu-s390x@nongnu.org,
	"Fam Zheng" <fam@euphon.net>, "Thomas Huth" <thuth@redhat.com>,
	"Cédric Le Goater" <clg@kaod.org>,
	"Leonardo Bras" <leobras@redhat.com>,
	"Ilya Leoshkevich" <iii@linux.ibm.com>
Subject: Re: [PATCH 21/21] migration/multifd: Compute transferred bytes correctly
Date: Thu, 18 May 2023 18:40:18 +0200	[thread overview]
Message-ID: <87o7mhbnkd.fsf@secure.mitica> (raw)
In-Reply-To: <ZGZTCAHV5M/+w3VS@x1n> (Peter Xu's message of "Thu, 18 May 2023 12:32:08 -0400")

Peter Xu <peterx@redhat.com> wrote:
> On Mon, May 08, 2023 at 03:09:09PM +0200, Juan Quintela wrote:
>> In the past, we had to put the in the main thread all the operations
>> related with sizes due to qemu_file not beeing thread safe.  As now
>> all counters are atomic, we can update the counters just after the
>> do the write.  As an aditional bonus, we are able to use the right
>> value for the compression methods.  Right now we were assuming that
>> there were no compression at all.
>
> Maybe worth mention that initial packet is also accounted after this.

Ok.
>>  
>> +            stat64_add(&mig_stats.multifd_bytes, p->next_packet_size);
>> +            stat64_add(&mig_stats.transferred, p->next_packet_size);
>
> Two nits:
>
> Maybe merge the two so half atomic operations?

On my tree, to send after this got in:

77fdd3475c migration: Remove transferred atomic counter

O:-)

> Also maybe also worth having a inline helper for adding both multifd_bytes
> and transferred?

I am removing it.

After next set of packates:

rate limit is calulated as:

begining_period = migration_transferred_bytes();
...

bytes_this_period = migration_transferred_bytes() - begining_period;

transferred is calculated as:
- multifd_bytes + qemu_file_bytes;

So things get really simple.  As all counters are atomic, you do a
write and after the write to increse the write size to the qemu_file or
to the multifd_bytes.  And that is it.


> With/without that, all look good:
>
> Reviewed-by: Peter Xu <peterx@redhat.com>

Thanks, Juan.



  reply	other threads:[~2023-05-18 16:41 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-05-08 13:08 [PATCH 00/21] Migration: More migration atomic counters Juan Quintela
2023-05-08 13:08 ` [PATCH 01/21] migration: A rate limit value of 0 is valid Juan Quintela
2023-05-15  8:33   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 02/21] migration: Don't use INT64_MAX for unlimited rate Juan Quintela
2023-05-09 11:41   ` Harsh Prateek Bora
2023-05-09 11:51     ` Juan Quintela
2023-05-09 12:02       ` Harsh Prateek Bora
2023-05-15  8:34       ` Cédric Le Goater
2023-05-15 11:18         ` Juan Quintela
2023-05-08 13:08 ` [PATCH 03/21] migration: We set the rate_limit by a second Juan Quintela
2023-05-15  8:38   ` Cédric Le Goater
2023-05-15 11:18     ` Juan Quintela
2023-05-08 13:08 ` [PATCH 04/21] qemu-file: make qemu_file_[sg]et_rate_limit() use an uint64_t Juan Quintela
2023-05-15  8:38   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 05/21] qemu-file: Make rate_limit_used " Juan Quintela
2023-05-15  8:40   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 06/21] qemu-file: Remove total from qemu_file_total_transferred_*() Juan Quintela
2023-05-15  9:33   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 07/21] migration: Correct transferred bytes value Juan Quintela
2023-05-09 12:08   ` Harsh Prateek Bora
2023-05-09 14:17     ` Juan Quintela
2023-05-08 13:08 ` [PATCH 08/21] migration: Move setup_time to mig_stats Juan Quintela
2023-05-15 10:35   ` Cédric Le Goater
2023-05-15 11:23     ` Juan Quintela
2023-05-08 13:08 ` [PATCH 09/21] qemu-file: Account for rate_limit usage on qemu_fflush() Juan Quintela
2023-05-15 12:15   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 10/21] migration: Move rate_limit_max and rate_limit_used to migration_stats Juan Quintela
2023-05-09 10:27   ` Harsh Prateek Bora
2023-05-09 11:10     ` Juan Quintela
2023-05-15  8:51       ` Harsh Prateek Bora
2023-05-15 13:02   ` Cédric Le Goater
2023-05-15 13:09     ` Juan Quintela
2023-05-15 13:28       ` Cédric Le Goater
2023-05-15 13:33         ` Juan Quintela
2023-05-15 17:16         ` Juan Quintela
2023-05-08 13:08 ` [PATCH 11/21] migration: Move migration_total_bytes() to migration-stats.c Juan Quintela
2023-05-15 13:02   ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 12/21] migration: Add a trace for migration_transferred_bytes Juan Quintela
2023-05-15 13:02   ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 13/21] migration: Use migration_transferred_bytes() to calculate rate_limit Juan Quintela
2023-05-15 13:02   ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 14/21] migration: We don't need the field rate_limit_used anymore Juan Quintela
2023-05-15 13:02   ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 15/21] migration: Don't abuse qemu_file transferred for RDMA Juan Quintela
2023-05-08 13:09 ` [PATCH 16/21] migration/RDMA: It is accounting for zero/normal pages in two places Juan Quintela
2023-05-08 13:09 ` [PATCH 17/21] migration/rdma: Remove QEMUFile parameter when not used Juan Quintela
2023-05-08 13:09 ` [PATCH 18/21] migration/rdma: Don't use imaginary transfers Juan Quintela
2023-05-08 13:09 ` [PATCH 19/21] migration: Remove unused qemu_file_credit_transfer() Juan Quintela
2023-05-08 13:09 ` [PATCH 20/21] migration/rdma: Simplify the function that saves a page Juan Quintela
2023-05-08 13:09 ` [PATCH 21/21] migration/multifd: Compute transferred bytes correctly Juan Quintela
2023-05-18 16:32   ` Peter Xu
2023-05-18 16:40     ` Juan Quintela [this message]
2023-05-18 18:32       ` Peter Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87o7mhbnkd.fsf@secure.mitica \
    --to=quintela@redhat.com \
    --cc=borntraeger@linux.ibm.com \
    --cc=clg@kaod.org \
    --cc=danielhb413@gmail.com \
    --cc=david@gibson.dropbear.id.au \
    --cc=david@redhat.com \
    --cc=eblake@redhat.com \
    --cc=fam@euphon.net \
    --cc=farman@linux.ibm.com \
    --cc=groug@kaod.org \
    --cc=harshpb@linux.ibm.com \
    --cc=iii@linux.ibm.com \
    --cc=jsnow@redhat.com \
    --cc=leobras@redhat.com \
    --cc=pasic@linux.ibm.com \
    --cc=peterx@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-ppc@nongnu.org \
    --cc=qemu-s390x@nongnu.org \
    --cc=richard.henderson@linaro.org \
    --cc=stefanha@redhat.com \
    --cc=thuth@redhat.com \
    --cc=vsementsov@yandex-team.ru \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).