All of lore.kernel.org
 help / color / mirror / Atom feed
From: Juan Quintela <quintela@redhat.com>
To: Peter Xu <peterx@redhat.com>
Cc: qemu-devel@nongnu.org,
	"Daniel Henrique Barboza" <danielhb413@gmail.com>,
	"Christian Borntraeger" <borntraeger@linux.ibm.com>,
	"David Hildenbrand" <david@redhat.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	qemu-block@nongnu.org, "Eric Blake" <eblake@redhat.com>,
	"Vladimir Sementsov-Ogievskiy" <vsementsov@yandex-team.ru>,
	"John Snow" <jsnow@redhat.com>,
	"Halil Pasic" <pasic@linux.ibm.com>,
	"Richard Henderson" <richard.henderson@linaro.org>,
	"David Gibson" <david@gibson.dropbear.id.au>,
	"Harsh Prateek Bora" <harshpb@linux.ibm.com>,
	"Eric Farman" <farman@linux.ibm.com>,
	"Greg Kurz" <groug@kaod.org>,
	qemu-ppc@nongnu.org, qemu-s390x@nongnu.org,
	"Fam Zheng" <fam@euphon.net>, "Thomas Huth" <thuth@redhat.com>,
	"Cédric Le Goater" <clg@kaod.org>,
	"Leonardo Bras" <leobras@redhat.com>,
	"Ilya Leoshkevich" <iii@linux.ibm.com>
Subject: Re: [PATCH 21/21] migration/multifd: Compute transferred bytes correctly
Date: Thu, 18 May 2023 18:40:18 +0200	[thread overview]
Message-ID: <87o7mhbnkd.fsf@secure.mitica> (raw)
In-Reply-To: <ZGZTCAHV5M/+w3VS@x1n> (Peter Xu's message of "Thu, 18 May 2023 12:32:08 -0400")

Peter Xu <peterx@redhat.com> wrote:
> On Mon, May 08, 2023 at 03:09:09PM +0200, Juan Quintela wrote:
>> In the past, we had to put the in the main thread all the operations
>> related with sizes due to qemu_file not beeing thread safe.  As now
>> all counters are atomic, we can update the counters just after the
>> do the write.  As an aditional bonus, we are able to use the right
>> value for the compression methods.  Right now we were assuming that
>> there were no compression at all.
>
> Maybe worth mention that initial packet is also accounted after this.

Ok.
>>  
>> +            stat64_add(&mig_stats.multifd_bytes, p->next_packet_size);
>> +            stat64_add(&mig_stats.transferred, p->next_packet_size);
>
> Two nits:
>
> Maybe merge the two so half atomic operations?

On my tree, to send after this got in:

77fdd3475c migration: Remove transferred atomic counter

O:-)

> Also maybe also worth having a inline helper for adding both multifd_bytes
> and transferred?

I am removing it.

After next set of packates:

rate limit is calulated as:

begining_period = migration_transferred_bytes();
...

bytes_this_period = migration_transferred_bytes() - begining_period;

transferred is calculated as:
- multifd_bytes + qemu_file_bytes;

So things get really simple.  As all counters are atomic, you do a
write and after the write to increse the write size to the qemu_file or
to the multifd_bytes.  And that is it.


> With/without that, all look good:
>
> Reviewed-by: Peter Xu <peterx@redhat.com>

Thanks, Juan.



  reply	other threads:[~2023-05-18 16:41 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-05-08 13:08 [PATCH 00/21] Migration: More migration atomic counters Juan Quintela
2023-05-08 13:08 ` [PATCH 01/21] migration: A rate limit value of 0 is valid Juan Quintela
2023-05-15  8:33   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 02/21] migration: Don't use INT64_MAX for unlimited rate Juan Quintela
2023-05-09 11:41   ` Harsh Prateek Bora
2023-05-09 11:51     ` Juan Quintela
2023-05-09 12:02       ` Harsh Prateek Bora
2023-05-15  8:34       ` Cédric Le Goater
2023-05-15 11:18         ` Juan Quintela
2023-05-08 13:08 ` [PATCH 03/21] migration: We set the rate_limit by a second Juan Quintela
2023-05-15  8:38   ` Cédric Le Goater
2023-05-15 11:18     ` Juan Quintela
2023-05-08 13:08 ` [PATCH 04/21] qemu-file: make qemu_file_[sg]et_rate_limit() use an uint64_t Juan Quintela
2023-05-15  8:38   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 05/21] qemu-file: Make rate_limit_used " Juan Quintela
2023-05-15  8:40   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 06/21] qemu-file: Remove total from qemu_file_total_transferred_*() Juan Quintela
2023-05-15  9:33   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 07/21] migration: Correct transferred bytes value Juan Quintela
2023-05-09 12:08   ` Harsh Prateek Bora
2023-05-09 14:17     ` Juan Quintela
2023-05-08 13:08 ` [PATCH 08/21] migration: Move setup_time to mig_stats Juan Quintela
2023-05-15 10:35   ` Cédric Le Goater
2023-05-15 11:23     ` Juan Quintela
2023-05-08 13:08 ` [PATCH 09/21] qemu-file: Account for rate_limit usage on qemu_fflush() Juan Quintela
2023-05-15 12:15   ` Cédric Le Goater
2023-05-08 13:08 ` [PATCH 10/21] migration: Move rate_limit_max and rate_limit_used to migration_stats Juan Quintela
2023-05-09 10:27   ` Harsh Prateek Bora
2023-05-09 11:10     ` Juan Quintela
2023-05-15  8:51       ` Harsh Prateek Bora
2023-05-15 13:02   ` Cédric Le Goater
2023-05-15 13:09     ` Juan Quintela
2023-05-15 13:28       ` Cédric Le Goater
2023-05-15 13:33         ` Juan Quintela
2023-05-15 17:16         ` Juan Quintela
2023-05-08 13:08 ` [PATCH 11/21] migration: Move migration_total_bytes() to migration-stats.c Juan Quintela
2023-05-15 13:02   ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 12/21] migration: Add a trace for migration_transferred_bytes Juan Quintela
2023-05-15 13:02   ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 13/21] migration: Use migration_transferred_bytes() to calculate rate_limit Juan Quintela
2023-05-15 13:02   ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 14/21] migration: We don't need the field rate_limit_used anymore Juan Quintela
2023-05-15 13:02   ` Cédric Le Goater
2023-05-08 13:09 ` [PATCH 15/21] migration: Don't abuse qemu_file transferred for RDMA Juan Quintela
2023-05-08 13:09 ` [PATCH 16/21] migration/RDMA: It is accounting for zero/normal pages in two places Juan Quintela
2023-05-08 13:09 ` [PATCH 17/21] migration/rdma: Remove QEMUFile parameter when not used Juan Quintela
2023-05-08 13:09 ` [PATCH 18/21] migration/rdma: Don't use imaginary transfers Juan Quintela
2023-05-08 13:09 ` [PATCH 19/21] migration: Remove unused qemu_file_credit_transfer() Juan Quintela
2023-05-08 13:09 ` [PATCH 20/21] migration/rdma: Simplify the function that saves a page Juan Quintela
2023-05-08 13:09 ` [PATCH 21/21] migration/multifd: Compute transferred bytes correctly Juan Quintela
2023-05-18 16:32   ` Peter Xu
2023-05-18 16:40     ` Juan Quintela [this message]
2023-05-18 18:32       ` Peter Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87o7mhbnkd.fsf@secure.mitica \
    --to=quintela@redhat.com \
    --cc=borntraeger@linux.ibm.com \
    --cc=clg@kaod.org \
    --cc=danielhb413@gmail.com \
    --cc=david@gibson.dropbear.id.au \
    --cc=david@redhat.com \
    --cc=eblake@redhat.com \
    --cc=fam@euphon.net \
    --cc=farman@linux.ibm.com \
    --cc=groug@kaod.org \
    --cc=harshpb@linux.ibm.com \
    --cc=iii@linux.ibm.com \
    --cc=jsnow@redhat.com \
    --cc=leobras@redhat.com \
    --cc=pasic@linux.ibm.com \
    --cc=peterx@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-ppc@nongnu.org \
    --cc=qemu-s390x@nongnu.org \
    --cc=richard.henderson@linaro.org \
    --cc=stefanha@redhat.com \
    --cc=thuth@redhat.com \
    --cc=vsementsov@yandex-team.ru \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.