* [PATCH v1 0/1] COLO: only flush dirty ram pages from colo cache @ 2020-09-10 10:46 Derek Su 2020-09-10 10:46 ` [PATCH v1 1/1] " Derek Su 0 siblings, 1 reply; 9+ messages in thread From: Derek Su @ 2020-09-10 10:46 UTC (permalink / raw) To: qemu-devel; +Cc: Derek Su, zhang.zhanghailiang, dgilbert, quintela In the secondary side, the colo_flush_ram_cache() calls migration_bitmap_find_dirty() finding the dirty pages and flushes them to host. But ram_state's ram_bulk_stage flag is always enabled in secondary side, it leads to the whole ram pages copy instead of only dirty pages. In the test VM with 4GB RAM under the steady state, the colo_flush_ram_cache() consumes 650 ms. Here, the ram_bulk_stage flag in secondary side is disabled in the preparation of COLO incoming process to avoid the whole dirty ram pages flush. After patching, the time consumption of colo_flush_ram_cache() is reduced to 10 ms averagely. Please help to review and give comments, thanks a lot! Derek Su (1): COLO: only flush dirty ram pages from colo cache migration/colo.c | 6 +++++- migration/ram.c | 10 ++++++++++ migration/ram.h | 3 +++ 3 files changed, 18 insertions(+), 1 deletion(-) -- 2.25.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache 2020-09-10 10:46 [PATCH v1 0/1] COLO: only flush dirty ram pages from colo cache Derek Su @ 2020-09-10 10:46 ` Derek Su 2020-09-22 3:31 ` Derek Su 0 siblings, 1 reply; 9+ messages in thread From: Derek Su @ 2020-09-10 10:46 UTC (permalink / raw) To: qemu-devel; +Cc: Derek Su, zhang.zhanghailiang, dgilbert, quintela In secondary side, the colo_flush_ram_cache() calls migration_bitmap_find_dirty() to finding the dirty pages and flush them to host. But ram_state's ram_bulk_stage flag is always enabled in secondary side, it leads to the whole ram pages copy instead of only dirty pages. Here, the ram_bulk_stage in secondary side is disabled in the preparation of COLO incoming process to avoid the whole dirty ram pages flush. Signed-off-by: Derek Su <dereksu@qnap.com> --- migration/colo.c | 6 +++++- migration/ram.c | 10 ++++++++++ migration/ram.h | 3 +++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c index ea7d1e9d4e..6e644db306 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -844,6 +844,8 @@ void *colo_process_incoming_thread(void *opaque) return NULL; } + colo_disable_ram_bulk_stage(); + failover_init_state(); mis->to_src_file = qemu_file_get_return_path(mis->from_src_file); @@ -873,7 +875,7 @@ void *colo_process_incoming_thread(void *opaque) goto out; } #else - abort(); + abort(); #endif vm_start(); trace_colo_vm_state_change("stop", "run"); @@ -924,6 +926,8 @@ out: qemu_fclose(fb); } + colo_enable_ram_bulk_stage(); + /* Hope this not to be too long to loop here */ qemu_sem_wait(&mis->colo_incoming_sem); qemu_sem_destroy(&mis->colo_incoming_sem); diff --git a/migration/ram.c b/migration/ram.c index 76d4fee5d5..65e9b12058 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3357,6 +3357,16 @@ static bool postcopy_is_running(void) return ps >= POSTCOPY_INCOMING_LISTENING && ps < POSTCOPY_INCOMING_END; } +void colo_enable_ram_bulk_stage(void) +{ + ram_state->ram_bulk_stage = true; +} + +void colo_disable_ram_bulk_stage(void) +{ + ram_state->ram_bulk_stage = false; +} + /* * Flush content of RAM cache into SVM's memory. * Only flush the pages that be dirtied by PVM or SVM or both. diff --git a/migration/ram.h b/migration/ram.h index 2eeaacfa13..c1c0ebbe0f 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -69,4 +69,7 @@ void colo_flush_ram_cache(void); void colo_release_ram_cache(void); void colo_incoming_start_dirty_log(void); +void colo_enable_ram_bulk_stage(void); +void colo_disable_ram_bulk_stage(void); + #endif -- 2.25.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache 2020-09-10 10:46 ` [PATCH v1 1/1] " Derek Su @ 2020-09-22 3:31 ` Derek Su 2020-09-22 3:41 ` Zhang, Chen 0 siblings, 1 reply; 9+ messages in thread From: Derek Su @ 2020-09-22 3:31 UTC (permalink / raw) To: qemu-devel; +Cc: Zhang, Chen, zhang.zhanghailiang, dgilbert, quintela [-- Attachment #1: Type: text/plain, Size: 2717 bytes --] Hello, all Ping... Regards, Derek Su Derek Su <dereksu@qnap.com> 於 2020年9月10日 週四 下午6:47寫道： > In secondary side, the colo_flush_ram_cache() calls > migration_bitmap_find_dirty() to finding the dirty pages and > flush them to host. But ram_state's ram_bulk_stage flag is always > enabled in secondary side, it leads to the whole ram pages copy > instead of only dirty pages. > > Here, the ram_bulk_stage in secondary side is disabled in the > preparation of COLO incoming process to avoid the whole dirty > ram pages flush. > > Signed-off-by: Derek Su <dereksu@qnap.com> > --- > migration/colo.c | 6 +++++- > migration/ram.c | 10 ++++++++++ > migration/ram.h | 3 +++ > 3 files changed, 18 insertions(+), 1 deletion(-) > > diff --git a/migration/colo.c b/migration/colo.c > index ea7d1e9d4e..6e644db306 100644 > --- a/migration/colo.c > +++ b/migration/colo.c > @@ -844,6 +844,8 @@ void *colo_process_incoming_thread(void *opaque) > return NULL; > } > > + colo_disable_ram_bulk_stage(); > + > failover_init_state(); > > mis->to_src_file = qemu_file_get_return_path(mis->from_src_file); > @@ -873,7 +875,7 @@ void *colo_process_incoming_thread(void *opaque) > goto out; > } > #else > - abort(); > + abort(); > #endif > vm_start(); > trace_colo_vm_state_change("stop", "run"); > @@ -924,6 +926,8 @@ out: > qemu_fclose(fb); > } > > + colo_enable_ram_bulk_stage(); > + > /* Hope this not to be too long to loop here */ > qemu_sem_wait(&mis->colo_incoming_sem); > qemu_sem_destroy(&mis->colo_incoming_sem); > diff --git a/migration/ram.c b/migration/ram.c > index 76d4fee5d5..65e9b12058 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -3357,6 +3357,16 @@ static bool postcopy_is_running(void) > return ps >= POSTCOPY_INCOMING_LISTENING && ps < > POSTCOPY_INCOMING_END; > } > > +void colo_enable_ram_bulk_stage(void) > +{ > + ram_state->ram_bulk_stage = true; > +} > + > +void colo_disable_ram_bulk_stage(void) > +{ > + ram_state->ram_bulk_stage = false; > +} > + > /* > * Flush content of RAM cache into SVM's memory. > * Only flush the pages that be dirtied by PVM or SVM or both. > diff --git a/migration/ram.h b/migration/ram.h > index 2eeaacfa13..c1c0ebbe0f 100644 > --- a/migration/ram.h > +++ b/migration/ram.h > @@ -69,4 +69,7 @@ void colo_flush_ram_cache(void); > void colo_release_ram_cache(void); > void colo_incoming_start_dirty_log(void); > > +void colo_enable_ram_bulk_stage(void); > +void colo_disable_ram_bulk_stage(void); > + > #endif > -- > 2.25.1 > > [-- Attachment #2: Type: text/html, Size: 3714 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache 2020-09-22 3:31 ` Derek Su @ 2020-09-22 3:41 ` Zhang, Chen 2020-09-22 3:48 ` Derek Su 0 siblings, 1 reply; 9+ messages in thread From: Zhang, Chen @ 2020-09-22 3:41 UTC (permalink / raw) To: Derek Su, qemu-devel, Rao, Lei Cc: zhang.zhanghailiang@huawei.com, dgilbert@redhat.com, quintela@redhat.com [-- Attachment #1: Type: text/plain, Size: 3087 bytes --] Hi Derek and Lei, It looks same with Lei’ patch: [PATCH 2/3] Reduce the time of checkpoint for COLO Can you discuss to merge it into one patch? Thanks Zhang Chen From: Derek Su <dereksu@qnap.com> Sent: Tuesday, September 22, 2020 11:31 AM To: qemu-devel <qemu-devel@nongnu.org> Cc: zhang.zhanghailiang@huawei.com; quintela@redhat.com; dgilbert@redhat.com; Zhang, Chen <chen.zhang@intel.com> Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache Hello, all Ping... Regards, Derek Su Derek Su <dereksu@qnap.com<mailto:dereksu@qnap.com>> 於 2020年9月10日 週四 下午6:47寫道： In secondary side, the colo_flush_ram_cache() calls migration_bitmap_find_dirty() to finding the dirty pages and flush them to host. But ram_state's ram_bulk_stage flag is always enabled in secondary side, it leads to the whole ram pages copy instead of only dirty pages. Here, the ram_bulk_stage in secondary side is disabled in the preparation of COLO incoming process to avoid the whole dirty ram pages flush. Signed-off-by: Derek Su <dereksu@qnap.com<mailto:dereksu@qnap.com>> --- migration/colo.c | 6 +++++- migration/ram.c | 10 ++++++++++ migration/ram.h | 3 +++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c index ea7d1e9d4e..6e644db306 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -844,6 +844,8 @@ void *colo_process_incoming_thread(void *opaque) return NULL; } + colo_disable_ram_bulk_stage(); + failover_init_state(); mis->to_src_file = qemu_file_get_return_path(mis->from_src_file); @@ -873,7 +875,7 @@ void *colo_process_incoming_thread(void *opaque) goto out; } #else - abort(); + abort(); #endif vm_start(); trace_colo_vm_state_change("stop", "run"); @@ -924,6 +926,8 @@ out: qemu_fclose(fb); } + colo_enable_ram_bulk_stage(); + /* Hope this not to be too long to loop here */ qemu_sem_wait(&mis->colo_incoming_sem); qemu_sem_destroy(&mis->colo_incoming_sem); diff --git a/migration/ram.c b/migration/ram.c index 76d4fee5d5..65e9b12058 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3357,6 +3357,16 @@ static bool postcopy_is_running(void) return ps >= POSTCOPY_INCOMING_LISTENING && ps < POSTCOPY_INCOMING_END; } +void colo_enable_ram_bulk_stage(void) +{ + ram_state->ram_bulk_stage = true; +} + +void colo_disable_ram_bulk_stage(void) +{ + ram_state->ram_bulk_stage = false; +} + /* * Flush content of RAM cache into SVM's memory. * Only flush the pages that be dirtied by PVM or SVM or both. diff --git a/migration/ram.h b/migration/ram.h index 2eeaacfa13..c1c0ebbe0f 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -69,4 +69,7 @@ void colo_flush_ram_cache(void); void colo_release_ram_cache(void); void colo_incoming_start_dirty_log(void); +void colo_enable_ram_bulk_stage(void); +void colo_disable_ram_bulk_stage(void); + #endif -- 2.25.1 [-- Attachment #2: Type: text/html, Size: 7789 bytes --] ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache 2020-09-22 3:41 ` Zhang, Chen @ 2020-09-22 3:48 ` Derek Su 2020-09-22 5:04 ` Rao, Lei 0 siblings, 1 reply; 9+ messages in thread From: Derek Su @ 2020-09-22 3:48 UTC (permalink / raw) To: Zhang, Chen Cc: Rao, Lei, quintela@redhat.com, qemu-devel, dgilbert@redhat.com, zhang.zhanghailiang@huawei.com [-- Attachment #1: Type: text/plain, Size: 3615 bytes --] Hi, Chen Sure. BTW, I just went through Lei's patch. ram_bulk_stage() might need to reset to true after stopping COLO service as my patch. How about your opinion? Thanks. Best regards, Derek Zhang, Chen <chen.zhang@intel.com> 於 2020年9月22日 週二 上午11:41寫道： > Hi Derek and Lei, > > > > It looks same with Lei’ patch: > > [PATCH 2/3] Reduce the time of checkpoint for COLO > > Can you discuss to merge it into one patch? > > > > Thanks > > Zhang Chen > > > > *From:* Derek Su <dereksu@qnap.com> > *Sent:* Tuesday, September 22, 2020 11:31 AM > *To:* qemu-devel <qemu-devel@nongnu.org> > *Cc:* zhang.zhanghailiang@huawei.com; quintela@redhat.com; > dgilbert@redhat.com; Zhang, Chen <chen.zhang@intel.com> > *Subject:* Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo > cache > > > > Hello, all > > > > Ping... > > > > Regards, > > Derek Su > > > > Derek Su <dereksu@qnap.com> 於 2020年9月10日 週四 下午6:47寫道： > > In secondary side, the colo_flush_ram_cache() calls > migration_bitmap_find_dirty() to finding the dirty pages and > flush them to host. But ram_state's ram_bulk_stage flag is always > enabled in secondary side, it leads to the whole ram pages copy > instead of only dirty pages. > > Here, the ram_bulk_stage in secondary side is disabled in the > preparation of COLO incoming process to avoid the whole dirty > ram pages flush. > > Signed-off-by: Derek Su <dereksu@qnap.com> > --- > migration/colo.c | 6 +++++- > migration/ram.c | 10 ++++++++++ > migration/ram.h | 3 +++ > 3 files changed, 18 insertions(+), 1 deletion(-) > > diff --git a/migration/colo.c b/migration/colo.c > index ea7d1e9d4e..6e644db306 100644 > --- a/migration/colo.c > +++ b/migration/colo.c > @@ -844,6 +844,8 @@ void *colo_process_incoming_thread(void *opaque) > return NULL; > } > > + colo_disable_ram_bulk_stage(); > + > failover_init_state(); > > mis->to_src_file = qemu_file_get_return_path(mis->from_src_file); > @@ -873,7 +875,7 @@ void *colo_process_incoming_thread(void *opaque) > goto out; > } > #else > - abort(); > + abort(); > #endif > vm_start(); > trace_colo_vm_state_change("stop", "run"); > @@ -924,6 +926,8 @@ out: > qemu_fclose(fb); > } > > + colo_enable_ram_bulk_stage(); > + > /* Hope this not to be too long to loop here */ > qemu_sem_wait(&mis->colo_incoming_sem); > qemu_sem_destroy(&mis->colo_incoming_sem); > diff --git a/migration/ram.c b/migration/ram.c > index 76d4fee5d5..65e9b12058 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -3357,6 +3357,16 @@ static bool postcopy_is_running(void) > return ps >= POSTCOPY_INCOMING_LISTENING && ps < > POSTCOPY_INCOMING_END; > } > > +void colo_enable_ram_bulk_stage(void) > +{ > + ram_state->ram_bulk_stage = true; > +} > + > +void colo_disable_ram_bulk_stage(void) > +{ > + ram_state->ram_bulk_stage = false; > +} > + > /* > * Flush content of RAM cache into SVM's memory. > * Only flush the pages that be dirtied by PVM or SVM or both. > diff --git a/migration/ram.h b/migration/ram.h > index 2eeaacfa13..c1c0ebbe0f 100644 > --- a/migration/ram.h > +++ b/migration/ram.h > @@ -69,4 +69,7 @@ void colo_flush_ram_cache(void); > void colo_release_ram_cache(void); > void colo_incoming_start_dirty_log(void); > > +void colo_enable_ram_bulk_stage(void); > +void colo_disable_ram_bulk_stage(void); > + > #endif > -- > 2.25.1 > > [-- Attachment #2: Type: text/html, Size: 7113 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache 2020-09-22 3:48 ` Derek Su @ 2020-09-22 5:04 ` Rao, Lei 2020-09-22 5:18 ` Derek Su 0 siblings, 1 reply; 9+ messages in thread From: Rao, Lei @ 2020-09-22 5:04 UTC (permalink / raw) To: Derek Su, Zhang, Chen Cc: quintela@redhat.com, qemu-devel, dgilbert@redhat.com, zhang.zhanghailiang@huawei.com Hi, Derek and Chen ram_bulk_stage is false by default before Hailiang's patch. For COLO, it does not seem to be used, so I think there is no need to reset it to true. Thanks, Lei. From: Derek Su <dereksu@qnap.com> Sent: Tuesday, September 22, 2020 11:48 AM To: Zhang, Chen <chen.zhang@intel.com> Cc: qemu-devel <qemu-devel@nongnu.org>; Rao, Lei <lei.rao@intel.com>; zhang.zhanghailiang@huawei.com; quintela@redhat.com; dgilbert@redhat.com Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache Hi, Chen Sure. BTW, I just went through Lei's patch. ram_bulk_stage() might need to reset to true after stopping COLO service as my patch. How about your opinion? Thanks. Best regards, Derek Zhang, Chen <mailto:chen.zhang@intel.com> 於 2020年9月22日 週二 上午11:41寫道： Hi Derek and Lei, It looks same with Lei’ patch: [PATCH 2/3] Reduce the time of checkpoint for COLO Can you discuss to merge it into one patch? Thanks Zhang Chen From: Derek Su <mailto:dereksu@qnap.com> Sent: Tuesday, September 22, 2020 11:31 AM To: qemu-devel <mailto:qemu-devel@nongnu.org> Cc: mailto:zhang.zhanghailiang@huawei.com; mailto:quintela@redhat.com; mailto:dgilbert@redhat.com; Zhang, Chen <mailto:chen.zhang@intel.com> Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache Hello, all Ping... Regards, Derek Su Derek Su <mailto:dereksu@qnap.com> 於 2020年9月10日 週四 下午6:47寫道： In secondary side, the colo_flush_ram_cache() calls migration_bitmap_find_dirty() to finding the dirty pages and flush them to host. But ram_state's ram_bulk_stage flag is always enabled in secondary side, it leads to the whole ram pages copy instead of only dirty pages. Here, the ram_bulk_stage in secondary side is disabled in the preparation of COLO incoming process to avoid the whole dirty ram pages flush. Signed-off-by: Derek Su <mailto:dereksu@qnap.com> --- migration/colo.c | 6 +++++- migration/ram.c | 10 ++++++++++ migration/ram.h | 3 +++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c index ea7d1e9d4e..6e644db306 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -844,6 +844,8 @@ void *colo_process_incoming_thread(void *opaque) return NULL; } + colo_disable_ram_bulk_stage(); + failover_init_state(); mis->to_src_file = qemu_file_get_return_path(mis->from_src_file); @@ -873,7 +875,7 @@ void *colo_process_incoming_thread(void *opaque) goto out; } #else - abort(); + abort(); #endif vm_start(); trace_colo_vm_state_change("stop", "run"); @@ -924,6 +926,8 @@ out: qemu_fclose(fb); } + colo_enable_ram_bulk_stage(); + /* Hope this not to be too long to loop here */ qemu_sem_wait(&mis->colo_incoming_sem); qemu_sem_destroy(&mis->colo_incoming_sem); diff --git a/migration/ram.c b/migration/ram.c index 76d4fee5d5..65e9b12058 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3357,6 +3357,16 @@ static bool postcopy_is_running(void) return ps >= POSTCOPY_INCOMING_LISTENING && ps < POSTCOPY_INCOMING_END; } +void colo_enable_ram_bulk_stage(void) +{ + ram_state->ram_bulk_stage = true; +} + +void colo_disable_ram_bulk_stage(void) +{ + ram_state->ram_bulk_stage = false; +} + /* * Flush content of RAM cache into SVM's memory. * Only flush the pages that be dirtied by PVM or SVM or both. diff --git a/migration/ram.h b/migration/ram.h index 2eeaacfa13..c1c0ebbe0f 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -69,4 +69,7 @@ void colo_flush_ram_cache(void); void colo_release_ram_cache(void); void colo_incoming_start_dirty_log(void); +void colo_enable_ram_bulk_stage(void); +void colo_disable_ram_bulk_stage(void); + #endif -- 2.25.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache 2020-09-22 5:04 ` Rao, Lei @ 2020-09-22 5:18 ` Derek Su 2020-09-22 5:37 ` Zhang, Chen 0 siblings, 1 reply; 9+ messages in thread From: Derek Su @ 2020-09-22 5:18 UTC (permalink / raw) To: Rao, Lei Cc: Zhang, Chen, quintela@redhat.com, qemu-devel, dgilbert@redhat.com, zhang.zhanghailiang@huawei.com [-- Attachment #1: Type: text/plain, Size: 4331 bytes --] Hi, Lei Got it. Thanks. Regards, Derek Rao, Lei <lei.rao@intel.com> 於 2020年9月22日 週二 下午1:04寫道： > Hi, Derek and Chen > > ram_bulk_stage is false by default before Hailiang's patch. > For COLO, it does not seem to be used, so I think there is no need to > reset it to true. > > Thanks, > Lei. > > From: Derek Su <dereksu@qnap.com> > Sent: Tuesday, September 22, 2020 11:48 AM > To: Zhang, Chen <chen.zhang@intel.com> > Cc: qemu-devel <qemu-devel@nongnu.org>; Rao, Lei <lei.rao@intel.com>; > zhang.zhanghailiang@huawei.com; quintela@redhat.com; dgilbert@redhat.com > Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo > cache > > Hi, Chen > > Sure. > > BTW, I just went through Lei's patch. > ram_bulk_stage() might need to reset to true after stopping COLO service > as my patch. > How about your opinion? > > Thanks. > > > Best regards, > Derek > > > Zhang, Chen <mailto:chen.zhang@intel.com> 於 2020年9月22日 週二 上午11:41寫道： > Hi Derek and Lei, > > It looks same with Lei’ patch: > [PATCH 2/3] Reduce the time of checkpoint for COLO > Can you discuss to merge it into one patch? > > Thanks > Zhang Chen > > From: Derek Su <mailto:dereksu@qnap.com> > Sent: Tuesday, September 22, 2020 11:31 AM > To: qemu-devel <mailto:qemu-devel@nongnu.org> > Cc: mailto:zhang.zhanghailiang@huawei.com; mailto:quintela@redhat.com; > mailto:dgilbert@redhat.com; Zhang, Chen <mailto:chen.zhang@intel.com> > Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo > cache > > Hello, all > > Ping... > > Regards, > Derek Su > > Derek Su <mailto:dereksu@qnap.com> 於 2020年9月10日 週四 下午6:47寫道： > In secondary side, the colo_flush_ram_cache() calls > migration_bitmap_find_dirty() to finding the dirty pages and > flush them to host. But ram_state's ram_bulk_stage flag is always > enabled in secondary side, it leads to the whole ram pages copy > instead of only dirty pages. > > Here, the ram_bulk_stage in secondary side is disabled in the > preparation of COLO incoming process to avoid the whole dirty > ram pages flush. > > Signed-off-by: Derek Su <mailto:dereksu@qnap.com> > --- > migration/colo.c | 6 +++++- > migration/ram.c | 10 ++++++++++ > migration/ram.h | 3 +++ > 3 files changed, 18 insertions(+), 1 deletion(-) > > diff --git a/migration/colo.c b/migration/colo.c > index ea7d1e9d4e..6e644db306 100644 > --- a/migration/colo.c > +++ b/migration/colo.c > @@ -844,6 +844,8 @@ void *colo_process_incoming_thread(void *opaque) > return NULL; > } > > + colo_disable_ram_bulk_stage(); > + > failover_init_state(); > > mis->to_src_file = qemu_file_get_return_path(mis->from_src_file); > @@ -873,7 +875,7 @@ void *colo_process_incoming_thread(void *opaque) > goto out; > } > #else > - abort(); > + abort(); > #endif > vm_start(); > trace_colo_vm_state_change("stop", "run"); > @@ -924,6 +926,8 @@ out: > qemu_fclose(fb); > } > > + colo_enable_ram_bulk_stage(); > + > /* Hope this not to be too long to loop here */ > qemu_sem_wait(&mis->colo_incoming_sem); > qemu_sem_destroy(&mis->colo_incoming_sem); > diff --git a/migration/ram.c b/migration/ram.c > index 76d4fee5d5..65e9b12058 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -3357,6 +3357,16 @@ static bool postcopy_is_running(void) > return ps >= POSTCOPY_INCOMING_LISTENING && ps < > POSTCOPY_INCOMING_END; > } > > +void colo_enable_ram_bulk_stage(void) > +{ > + ram_state->ram_bulk_stage = true; > +} > + > +void colo_disable_ram_bulk_stage(void) > +{ > + ram_state->ram_bulk_stage = false; > +} > + > /* > * Flush content of RAM cache into SVM's memory. > * Only flush the pages that be dirtied by PVM or SVM or both. > diff --git a/migration/ram.h b/migration/ram.h > index 2eeaacfa13..c1c0ebbe0f 100644 > --- a/migration/ram.h > +++ b/migration/ram.h > @@ -69,4 +69,7 @@ void colo_flush_ram_cache(void); > void colo_release_ram_cache(void); > void colo_incoming_start_dirty_log(void); > > +void colo_enable_ram_bulk_stage(void); > +void colo_disable_ram_bulk_stage(void); > + > #endif > -- > 2.25.1 > [-- Attachment #2: Type: text/html, Size: 6278 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache 2020-09-22 5:18 ` Derek Su @ 2020-09-22 5:37 ` Zhang, Chen 2020-09-22 5:46 ` Derek Su 0 siblings, 1 reply; 9+ messages in thread From: Zhang, Chen @ 2020-09-22 5:37 UTC (permalink / raw) To: Derek Su, Rao, Lei Cc: quintela@redhat.com, qemu-devel, dgilbert@redhat.com, zhang.zhanghailiang@huawei.com [-- Attachment #1: Type: text/plain, Size: 4959 bytes --] So, Derek, you will send new version patch? Thanks Zhang Chen From: Derek Su <dereksu@qnap.com> Sent: Tuesday, September 22, 2020 1:18 PM To: Rao, Lei <lei.rao@intel.com> Cc: Zhang, Chen <chen.zhang@intel.com>; qemu-devel <qemu-devel@nongnu.org>; zhang.zhanghailiang@huawei.com; quintela@redhat.com; dgilbert@redhat.com Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache Hi, Lei Got it. Thanks. Regards, Derek Rao, Lei <lei.rao@intel.com<mailto:lei.rao@intel.com>> 於 2020年9月22日 週二 下午1:04寫道： Hi, Derek and Chen ram_bulk_stage is false by default before Hailiang's patch. For COLO, it does not seem to be used, so I think there is no need to reset it to true. Thanks, Lei. From: Derek Su <dereksu@qnap.com<mailto:dereksu@qnap.com>> Sent: Tuesday, September 22, 2020 11:48 AM To: Zhang, Chen <chen.zhang@intel.com<mailto:chen.zhang@intel.com>> Cc: qemu-devel <qemu-devel@nongnu.org<mailto:qemu-devel@nongnu.org>>; Rao, Lei <lei.rao@intel.com<mailto:lei.rao@intel.com>>; zhang.zhanghailiang@huawei.com<mailto:zhang.zhanghailiang@huawei.com>; quintela@redhat.com<mailto:quintela@redhat.com>; dgilbert@redhat.com<mailto:dgilbert@redhat.com> Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache Hi, Chen Sure. BTW, I just went through Lei's patch. ram_bulk_stage() might need to reset to true after stopping COLO service as my patch. How about your opinion? Thanks. Best regards, Derek Zhang, Chen <mailto:chen.zhang@intel.com<mailto:chen.zhang@intel.com>> 於 2020年9月22日 週二 上午11:41寫道： Hi Derek and Lei, It looks same with Lei’ patch: [PATCH 2/3] Reduce the time of checkpoint for COLO Can you discuss to merge it into one patch? Thanks Zhang Chen From: Derek Su <mailto:dereksu@qnap.com<mailto:dereksu@qnap.com>> Sent: Tuesday, September 22, 2020 11:31 AM To: qemu-devel <mailto:qemu-devel@nongnu.org<mailto:qemu-devel@nongnu.org>> Cc: mailto:zhang.zhanghailiang@huawei.com<mailto:zhang.zhanghailiang@huawei.com>; mailto:quintela@redhat.com<mailto:quintela@redhat.com>; mailto:dgilbert@redhat.com<mailto:dgilbert@redhat.com>; Zhang, Chen <mailto:chen.zhang@intel.com<mailto:chen.zhang@intel.com>> Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache Hello, all Ping... Regards, Derek Su Derek Su <mailto:dereksu@qnap.com<mailto:dereksu@qnap.com>> 於 2020年9月10日 週四 下午6:47寫道： In secondary side, the colo_flush_ram_cache() calls migration_bitmap_find_dirty() to finding the dirty pages and flush them to host. But ram_state's ram_bulk_stage flag is always enabled in secondary side, it leads to the whole ram pages copy instead of only dirty pages. Here, the ram_bulk_stage in secondary side is disabled in the preparation of COLO incoming process to avoid the whole dirty ram pages flush. Signed-off-by: Derek Su <mailto:dereksu@qnap.com<mailto:dereksu@qnap.com>> --- migration/colo.c | 6 +++++- migration/ram.c | 10 ++++++++++ migration/ram.h | 3 +++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/migration/colo.c b/migration/colo.c index ea7d1e9d4e..6e644db306 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -844,6 +844,8 @@ void *colo_process_incoming_thread(void *opaque) return NULL; } + colo_disable_ram_bulk_stage(); + failover_init_state(); mis->to_src_file = qemu_file_get_return_path(mis->from_src_file); @@ -873,7 +875,7 @@ void *colo_process_incoming_thread(void *opaque) goto out; } #else - abort(); + abort(); #endif vm_start(); trace_colo_vm_state_change("stop", "run"); @@ -924,6 +926,8 @@ out: qemu_fclose(fb); } + colo_enable_ram_bulk_stage(); + /* Hope this not to be too long to loop here */ qemu_sem_wait(&mis->colo_incoming_sem); qemu_sem_destroy(&mis->colo_incoming_sem); diff --git a/migration/ram.c b/migration/ram.c index 76d4fee5d5..65e9b12058 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3357,6 +3357,16 @@ static bool postcopy_is_running(void) return ps >= POSTCOPY_INCOMING_LISTENING && ps < POSTCOPY_INCOMING_END; } +void colo_enable_ram_bulk_stage(void) +{ + ram_state->ram_bulk_stage = true; +} + +void colo_disable_ram_bulk_stage(void) +{ + ram_state->ram_bulk_stage = false; +} + /* * Flush content of RAM cache into SVM's memory. * Only flush the pages that be dirtied by PVM or SVM or both. diff --git a/migration/ram.h b/migration/ram.h index 2eeaacfa13..c1c0ebbe0f 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -69,4 +69,7 @@ void colo_flush_ram_cache(void); void colo_release_ram_cache(void); void colo_incoming_start_dirty_log(void); +void colo_enable_ram_bulk_stage(void); +void colo_disable_ram_bulk_stage(void); + #endif -- 2.25.1 [-- Attachment #2: Type: text/html, Size: 11007 bytes --] ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo cache 2020-09-22 5:37 ` Zhang, Chen @ 2020-09-22 5:46 ` Derek Su 0 siblings, 0 replies; 9+ messages in thread From: Derek Su @ 2020-09-22 5:46 UTC (permalink / raw) To: Zhang, Chen Cc: Rao, Lei, quintela@redhat.com, qemu-devel, dgilbert@redhat.com, zhang.zhanghailiang@huawei.com [-- Attachment #1: Type: text/plain, Size: 5095 bytes --] Hi, Chen and Lei Using Lei's patch is OK to me. Please help to add "Signed-off-by: Derek Su <dereksu@qnap.com>" for merging it. Thank you. :) Regards Derek Zhang, Chen <chen.zhang@intel.com> 於 2020年9月22日 週二 下午1:37寫道： > So, Derek, you will send new version patch? > > > > Thanks > > Zhang Chen > > > > *From:* Derek Su <dereksu@qnap.com> > *Sent:* Tuesday, September 22, 2020 1:18 PM > *To:* Rao, Lei <lei.rao@intel.com> > *Cc:* Zhang, Chen <chen.zhang@intel.com>; qemu-devel < > qemu-devel@nongnu.org>; zhang.zhanghailiang@huawei.com; > quintela@redhat.com; dgilbert@redhat.com > *Subject:* Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo > cache > > > > Hi, Lei > > > > Got it. Thanks. > > > > Regards, > > Derek > > > > Rao, Lei <lei.rao@intel.com> 於 2020年9月22日 週二 下午1:04寫道： > > Hi, Derek and Chen > > ram_bulk_stage is false by default before Hailiang's patch. > For COLO, it does not seem to be used, so I think there is no need to > reset it to true. > > Thanks, > Lei. > > From: Derek Su <dereksu@qnap.com> > Sent: Tuesday, September 22, 2020 11:48 AM > To: Zhang, Chen <chen.zhang@intel.com> > Cc: qemu-devel <qemu-devel@nongnu.org>; Rao, Lei <lei.rao@intel.com>; > zhang.zhanghailiang@huawei.com; quintela@redhat.com; dgilbert@redhat.com > Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo > cache > > Hi, Chen > > Sure. > > BTW, I just went through Lei's patch. > ram_bulk_stage() might need to reset to true after stopping COLO service > as my patch. > How about your opinion? > > Thanks. > > > Best regards, > Derek > > > Zhang, Chen <mailto:chen.zhang@intel.com> 於 2020年9月22日 週二 上午11:41寫道： > Hi Derek and Lei, > > It looks same with Lei’ patch: > [PATCH 2/3] Reduce the time of checkpoint for COLO > Can you discuss to merge it into one patch? > > Thanks > Zhang Chen > > From: Derek Su <mailto:dereksu@qnap.com> > Sent: Tuesday, September 22, 2020 11:31 AM > To: qemu-devel <mailto:qemu-devel@nongnu.org> > Cc: mailto:zhang.zhanghailiang@huawei.com; mailto:quintela@redhat.com; > mailto:dgilbert@redhat.com; Zhang, Chen <mailto:chen.zhang@intel.com> > Subject: Re: [PATCH v1 1/1] COLO: only flush dirty ram pages from colo > cache > > Hello, all > > Ping... > > Regards, > Derek Su > > Derek Su <mailto:dereksu@qnap.com> 於 2020年9月10日 週四 下午6:47寫道： > In secondary side, the colo_flush_ram_cache() calls > migration_bitmap_find_dirty() to finding the dirty pages and > flush them to host. But ram_state's ram_bulk_stage flag is always > enabled in secondary side, it leads to the whole ram pages copy > instead of only dirty pages. > > Here, the ram_bulk_stage in secondary side is disabled in the > preparation of COLO incoming process to avoid the whole dirty > ram pages flush. > > Signed-off-by: Derek Su <mailto:dereksu@qnap.com> > --- > migration/colo.c | 6 +++++- > migration/ram.c | 10 ++++++++++ > migration/ram.h | 3 +++ > 3 files changed, 18 insertions(+), 1 deletion(-) > > diff --git a/migration/colo.c b/migration/colo.c > index ea7d1e9d4e..6e644db306 100644 > --- a/migration/colo.c > +++ b/migration/colo.c > @@ -844,6 +844,8 @@ void *colo_process_incoming_thread(void *opaque) > return NULL; > } > > + colo_disable_ram_bulk_stage(); > + > failover_init_state(); > > mis->to_src_file = qemu_file_get_return_path(mis->from_src_file); > @@ -873,7 +875,7 @@ void *colo_process_incoming_thread(void *opaque) > goto out; > } > #else > - abort(); > + abort(); > #endif > vm_start(); > trace_colo_vm_state_change("stop", "run"); > @@ -924,6 +926,8 @@ out: > qemu_fclose(fb); > } > > + colo_enable_ram_bulk_stage(); > + > /* Hope this not to be too long to loop here */ > qemu_sem_wait(&mis->colo_incoming_sem); > qemu_sem_destroy(&mis->colo_incoming_sem); > diff --git a/migration/ram.c b/migration/ram.c > index 76d4fee5d5..65e9b12058 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -3357,6 +3357,16 @@ static bool postcopy_is_running(void) > return ps >= POSTCOPY_INCOMING_LISTENING && ps < > POSTCOPY_INCOMING_END; > } > > +void colo_enable_ram_bulk_stage(void) > +{ > + ram_state->ram_bulk_stage = true; > +} > + > +void colo_disable_ram_bulk_stage(void) > +{ > + ram_state->ram_bulk_stage = false; > +} > + > /* > * Flush content of RAM cache into SVM's memory. > * Only flush the pages that be dirtied by PVM or SVM or both. > diff --git a/migration/ram.h b/migration/ram.h > index 2eeaacfa13..c1c0ebbe0f 100644 > --- a/migration/ram.h > +++ b/migration/ram.h > @@ -69,4 +69,7 @@ void colo_flush_ram_cache(void); > void colo_release_ram_cache(void); > void colo_incoming_start_dirty_log(void); > > +void colo_enable_ram_bulk_stage(void); > +void colo_disable_ram_bulk_stage(void); > + > #endif > -- > 2.25.1 > > [-- Attachment #2: Type: text/html, Size: 10166 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-09-22 5:47 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-09-10 10:46 [PATCH v1 0/1] COLO: only flush dirty ram pages from colo cache Derek Su 2020-09-10 10:46 ` [PATCH v1 1/1] " Derek Su 2020-09-22 3:31 ` Derek Su 2020-09-22 3:41 ` Zhang, Chen 2020-09-22 3:48 ` Derek Su 2020-09-22 5:04 ` Rao, Lei 2020-09-22 5:18 ` Derek Su 2020-09-22 5:37 ` Zhang, Chen 2020-09-22 5:46 ` Derek Su
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.