From: Juan Quintela <quintela@redhat.com>
To: Avi Kivity <avi@redhat.com>
Cc: Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>,
Anthony Liguori <anthony@codemonkey.ws>,
Paolo Bonzini <pbonzini@redhat.com>,
qemu-devel@nongnu.org, kvm-devel <kvm@vger.kernel.org>
Subject: Re: [PATCH 09/10] Exit loop if we have been there too long
Date: Wed, 01 Dec 2010 14:45:28 +0100 [thread overview]
Message-ID: <m3zkspy5c7.fsf@trasno.mitica> (raw)
In-Reply-To: <4CF6412D.5030601@redhat.com> (Avi Kivity's message of "Wed, 01 Dec 2010 14:35:57 +0200")
Avi Kivity <avi@redhat.com> wrote:
> On 12/01/2010 03:52 AM, Juan Quintela wrote:
>> > - 512GB guest is really the target?
>>
>> no, problems exist with smaller amounts of RAM. with 16GB guest it is
>> trivial to get 1s stalls, 64GB guest, 3-4s, with more memory, migration
>> is flaky to say the less.
>>
>> > - how much cpu time can we use for these things?
>>
>> the problem here is that we are forced to walk the bitmap too many
>> times, we want to do it less times.
>
> How much time is spent walking bitmaps? Are you sure this is the problem?
see my 10/10 patch, that one makes ram_save_live() to move from 80% of the
time in the profile to the second one with ~5% or so.
The important bit is:
static uint64_t ram_save_remaining(void)
{
- RAMBlock *block;
- uint64_t count = 0;
-
- QLIST_FOREACH(block, &ram_list.blocks, next) {
- ram_addr_t addr;
- for (addr = block->offset; addr < block->offset + block->length;
- addr += TARGET_PAGE_SIZE) {
- if (cpu_physical_memory_get_dirty(addr, MIGRATION_DIRTY_FLAG)) {
- count++;
- }
- }
- }
-
- return count;
+ return ram_list.dirty_pages;
}
We dont' need to walk all bitmap to see how much memory is remaining.
syncing the bitmap is also expensive, but just now we have other issues
that hide it. That is the reason that I didn't started trynig to get a
better interface with kvm, 1st I will try to remove the bigger
bottlenecks.
>> > - how many dirty pages do we have to care?
>>
>> default values and assuming 1Gigabit ethernet for ourselves ~9.5MB of
>> dirty pages to have only 30ms of downtime.
>
> 1Gb/s * 30ms = 100 MB/s * 30 ms = 3 MB.
I will learn to make math with time and bytes at some point O:-)
Later, Juan.
WARNING: multiple messages have this Message-ID (diff)
From: Juan Quintela <quintela@redhat.com>
To: Avi Kivity <avi@redhat.com>
Cc: Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>,
Paolo Bonzini <pbonzini@redhat.com>,
qemu-devel@nongnu.org, kvm-devel <kvm@vger.kernel.org>
Subject: [Qemu-devel] Re: [PATCH 09/10] Exit loop if we have been there too long
Date: Wed, 01 Dec 2010 14:45:28 +0100 [thread overview]
Message-ID: <m3zkspy5c7.fsf@trasno.mitica> (raw)
In-Reply-To: <4CF6412D.5030601@redhat.com> (Avi Kivity's message of "Wed, 01 Dec 2010 14:35:57 +0200")
Avi Kivity <avi@redhat.com> wrote:
> On 12/01/2010 03:52 AM, Juan Quintela wrote:
>> > - 512GB guest is really the target?
>>
>> no, problems exist with smaller amounts of RAM. with 16GB guest it is
>> trivial to get 1s stalls, 64GB guest, 3-4s, with more memory, migration
>> is flaky to say the less.
>>
>> > - how much cpu time can we use for these things?
>>
>> the problem here is that we are forced to walk the bitmap too many
>> times, we want to do it less times.
>
> How much time is spent walking bitmaps? Are you sure this is the problem?
see my 10/10 patch, that one makes ram_save_live() to move from 80% of the
time in the profile to the second one with ~5% or so.
The important bit is:
static uint64_t ram_save_remaining(void)
{
- RAMBlock *block;
- uint64_t count = 0;
-
- QLIST_FOREACH(block, &ram_list.blocks, next) {
- ram_addr_t addr;
- for (addr = block->offset; addr < block->offset + block->length;
- addr += TARGET_PAGE_SIZE) {
- if (cpu_physical_memory_get_dirty(addr, MIGRATION_DIRTY_FLAG)) {
- count++;
- }
- }
- }
-
- return count;
+ return ram_list.dirty_pages;
}
We dont' need to walk all bitmap to see how much memory is remaining.
syncing the bitmap is also expensive, but just now we have other issues
that hide it. That is the reason that I didn't started trynig to get a
better interface with kvm, 1st I will try to remove the bigger
bottlenecks.
>> > - how many dirty pages do we have to care?
>>
>> default values and assuming 1Gigabit ethernet for ourselves ~9.5MB of
>> dirty pages to have only 30ms of downtime.
>
> 1Gb/s * 30ms = 100 MB/s * 30 ms = 3 MB.
I will learn to make math with time and bytes at some point O:-)
Later, Juan.
next prev parent reply other threads:[~2010-12-01 13:46 UTC|newest]
Thread overview: 92+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-23 23:02 [Qemu-devel] [PATCH 00/10] Fix migration with lots of memory Juan Quintela
2010-11-23 23:02 ` [Qemu-devel] [PATCH 01/10] Add spent time to migration Juan Quintela
2010-11-23 23:02 ` [Qemu-devel] [PATCH 02/10] Add buffered_file_internal constant Juan Quintela
2010-11-24 10:40 ` [Qemu-devel] " Michael S. Tsirkin
2010-11-24 10:52 ` Juan Quintela
2010-11-24 11:04 ` Michael S. Tsirkin
2010-11-24 11:13 ` Juan Quintela
2010-11-24 11:19 ` Michael S. Tsirkin
[not found] ` <4CF46012.2060804@codemonkey.ws>
2010-11-30 11:56 ` Juan Quintela
2010-11-30 14:02 ` Anthony Liguori
2010-11-30 14:11 ` Michael S. Tsirkin
2010-11-30 14:22 ` Anthony Liguori
2010-11-30 15:40 ` Juan Quintela
2010-11-30 16:10 ` Michael S. Tsirkin
2010-11-30 16:32 ` Juan Quintela
2010-11-30 16:44 ` Anthony Liguori
2010-11-30 18:04 ` Juan Quintela
2010-11-30 18:54 ` Anthony Liguori
2010-11-30 19:15 ` Juan Quintela
2010-11-30 20:23 ` Anthony Liguori
2010-11-30 20:56 ` Juan Quintela
2010-11-23 23:03 ` [Qemu-devel] [PATCH 03/10] Add printf debug to savevm Juan Quintela
[not found] ` <4CF45AB2.7050506@codemonkey.ws>
2010-11-30 10:36 ` Stefan Hajnoczi
2010-11-30 22:40 ` [Qemu-devel] " Juan Quintela
2010-12-01 7:50 ` Stefan Hajnoczi
2010-11-23 23:03 ` [Qemu-devel] [PATCH 04/10] No need to iterate if we already are over the limit Juan Quintela
2010-11-23 23:03 ` [Qemu-devel] [PATCH 05/10] KVM don't care about TLB handling Juan Quintela
2010-11-23 23:03 ` [Qemu-devel] [PATCH 06/10] Only calculate expected_time for stage 2 Juan Quintela
2010-11-23 23:03 ` [Qemu-devel] [PATCH 07/10] ram_save_remaining() returns an uint64_t Juan Quintela
[not found] ` <4CF45C0C.705@codemonkey.ws>
2010-11-30 7:21 ` [Qemu-devel] " Paolo Bonzini
2010-11-30 13:44 ` Anthony Liguori
2010-11-30 14:38 ` Juan Quintela
2010-11-23 23:03 ` [Qemu-devel] [PATCH 08/10] Count nanoseconds with uint64_t not doubles Juan Quintela
2010-11-30 7:17 ` [Qemu-devel] " Paolo Bonzini
[not found] ` <4CF45C5B.9080507@codemonkey.ws>
2010-11-30 14:40 ` Juan Quintela
2010-11-23 23:03 ` [Qemu-devel] [PATCH 09/10] Exit loop if we have been there too long Juan Quintela
2010-11-24 10:40 ` [Qemu-devel] " Michael S. Tsirkin
2010-11-24 11:01 ` Juan Quintela
2010-11-24 11:14 ` Michael S. Tsirkin
2010-11-24 15:16 ` Paolo Bonzini
2010-11-24 15:59 ` Michael S. Tsirkin
[not found] ` <4CF45E3F.4040609@codemonkey.ws>
2010-11-30 8:10 ` Paolo Bonzini
2010-11-30 13:26 ` Juan Quintela
[not found] ` <4CF45D67.5010906@codemonkey.ws>
2010-11-30 7:15 ` Paolo Bonzini
2010-11-30 13:47 ` Anthony Liguori
2010-11-30 13:47 ` [Qemu-devel] " Anthony Liguori
2010-11-30 13:58 ` Avi Kivity
2010-11-30 13:58 ` [Qemu-devel] " Avi Kivity
2010-11-30 14:17 ` Anthony Liguori
2010-11-30 14:17 ` [Qemu-devel] " Anthony Liguori
2010-11-30 14:27 ` Avi Kivity
2010-11-30 14:27 ` [Qemu-devel] " Avi Kivity
2010-11-30 14:50 ` Anthony Liguori
2010-11-30 14:50 ` [Qemu-devel] " Anthony Liguori
2010-12-01 12:40 ` Avi Kivity
2010-12-01 12:40 ` [Qemu-devel] " Avi Kivity
2010-11-30 17:43 ` Juan Quintela
2010-11-30 17:43 ` [Qemu-devel] " Juan Quintela
2010-12-01 1:20 ` Takuya Yoshikawa
2010-12-01 1:20 ` [Qemu-devel] " Takuya Yoshikawa
2010-12-01 1:52 ` Juan Quintela
2010-12-01 1:52 ` [Qemu-devel] " Juan Quintela
2010-12-01 2:22 ` Takuya Yoshikawa
2010-12-01 2:22 ` [Qemu-devel] " Takuya Yoshikawa
2010-12-01 12:35 ` Avi Kivity
2010-12-01 12:35 ` [Qemu-devel] " Avi Kivity
2010-12-01 13:45 ` Juan Quintela [this message]
2010-12-01 13:45 ` Juan Quintela
2010-12-02 1:31 ` Takuya Yoshikawa
2010-12-02 1:31 ` [Qemu-devel] " Takuya Yoshikawa
2010-12-02 8:37 ` Avi Kivity
2010-12-02 8:37 ` [Qemu-devel] " Avi Kivity
2010-11-30 14:12 ` Paolo Bonzini
2010-11-30 14:12 ` [Qemu-devel] " Paolo Bonzini
2010-11-30 15:00 ` Anthony Liguori
2010-11-30 15:00 ` [Qemu-devel] " Anthony Liguori
2010-11-30 17:59 ` Juan Quintela
2010-11-30 17:59 ` [Qemu-devel] " Juan Quintela
2010-11-23 23:03 ` [Qemu-devel] [PATCH 10/10] Maintaing number of dirty pages Juan Quintela
[not found] ` <4CF45DE0.8020701@codemonkey.ws>
2010-11-30 14:46 ` [Qemu-devel] " Juan Quintela
2010-12-01 14:46 ` Avi Kivity
2010-12-01 15:51 ` Juan Quintela
2010-12-01 15:55 ` Anthony Liguori
2010-12-01 16:25 ` Juan Quintela
2010-12-01 16:33 ` Anthony Liguori
2010-12-01 16:43 ` Avi Kivity
2010-12-01 16:49 ` Anthony Liguori
2010-12-01 16:52 ` Avi Kivity
2010-12-01 16:56 ` Anthony Liguori
2010-12-01 17:01 ` Avi Kivity
2010-12-01 17:05 ` Anthony Liguori
2010-12-01 18:51 ` Juan Quintela
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=m3zkspy5c7.fsf@trasno.mitica \
--to=quintela@redhat.com \
--cc=anthony@codemonkey.ws \
--cc=avi@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=yoshikawa.takuya@oss.ntt.co.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.