* [Qemu-devel] [PATCH] vl: Adjust the place of calling mlockall to speedup VM's startup
@ 2014-09-23 7:57 zhanghailiang
2014-09-23 8:30 ` Michael S. Tsirkin
0 siblings, 1 reply; 5+ messages in thread
From: zhanghailiang @ 2014-09-23 7:57 UTC (permalink / raw)
To: qemu-devel
Cc: zhanghailiang, mst, hutao, luonengjun, peter.huangpeng, imammedo
If we configure mlock=on and memory policy=bind at the same time,
It will consume lots of time for system to treat with memory,
especially when call mbind after mlockall.
Adjust the place of calling mlockall, calling mbind before mlockall
can remarkably reduce the time of VM's startup.
Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
---
Hi,
Actually, for mbind and mlockall, i have made a test about the time consuming
for the different call sequence.
The results is shown below. It is obviously that mlockall called before mbind is
more time-consuming.
Besides, this patch is OK with memory hotplug.
TEST CODE:
if (mbind_first) {
printf("mbind --> mlockall\n");
mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2,
MPOL_MF_STRICT | MPOL_MF_MOVE);
mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
MPOL_MF_STRICT | MPOL_MF_MOVE);
mlockall(MCL_CURRENT | MCL_FUTURE);
} else {
printf("mlockall --> mbind\n");
mlockall(MCL_CURRENT | MCL_FUTURE);
mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 ,
MPOL_MF_STRICT | MPOL_MF_MOVE);
mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
MPOL_MF_STRICT | MPOL_MF_MOVE);
}
RESULT 1:
#time /home/test_mbind 10240 0
memroy size 10737418240
mlockall --> mbind
real 0m11.886s
user 0m0.004s
sys 0m11.865s
#time /home/test_mbind 10240 1
memroy size 10737418240
mbind --> mlockall
real 0m5.334s
user 0m0.000s
sys 0m5.324s
RESULT 2:
#time /home/test_mbind 4096 0
memroy size 4294967296
mlockall --> mbind
real 0m5.503s
user 0m0.000s
sys 0m5.492s
#time /home/test_mbind 4096 1
memroy size 4294967296
mbind --> mlockall
real 0m2.139s
user 0m0.000s
sys 0m2.132s
Best Regards,
zhanghailiang
---
vl.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/vl.c b/vl.c
index dc792fe..adf4770 100644
--- a/vl.c
+++ b/vl.c
@@ -134,6 +134,7 @@ const char* keyboard_layout = NULL;
ram_addr_t ram_size;
const char *mem_path = NULL;
int mem_prealloc = 0; /* force preallocation of physical target memory */
+int enable_mlock = false;
int nb_nics;
NICInfo nd_table[MAX_NICS];
int autostart;
@@ -1421,12 +1422,8 @@ static void smp_parse(QemuOpts *opts)
}
-static void configure_realtime(QemuOpts *opts)
+static void realtime_init(void)
{
- bool enable_mlock;
-
- enable_mlock = qemu_opt_get_bool(opts, "mlock", true);
-
if (enable_mlock) {
if (os_mlock() < 0) {
fprintf(stderr, "qemu: locking memory failed\n");
@@ -3973,7 +3970,7 @@ int main(int argc, char **argv, char **envp)
if (!opts) {
exit(1);
}
- configure_realtime(opts);
+ enable_mlock = qemu_opt_get_bool(opts, "mlock", true);
break;
case QEMU_OPTION_msg:
opts = qemu_opts_parse(qemu_find_opts("msg"), optarg, 0);
@@ -4441,6 +4438,8 @@ int main(int argc, char **argv, char **envp)
machine_class->init(current_machine);
+ realtime_init();
+
audio_init();
cpu_synchronize_all_post_init();
--
1.7.12.4
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [Qemu-devel] [PATCH] vl: Adjust the place of calling mlockall to speedup VM's startup
2014-09-23 7:57 [Qemu-devel] [PATCH] vl: Adjust the place of calling mlockall to speedup VM's startup zhanghailiang
@ 2014-09-23 8:30 ` Michael S. Tsirkin
2014-09-23 8:35 ` Hu Tao
2014-09-23 10:19 ` zhanghailiang
0 siblings, 2 replies; 5+ messages in thread
From: Michael S. Tsirkin @ 2014-09-23 8:30 UTC (permalink / raw)
To: zhanghailiang; +Cc: imammedo, peter.huangpeng, luonengjun, qemu-devel, hutao
On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote:
> If we configure mlock=on and memory policy=bind at the same time,
> It will consume lots of time for system to treat with memory,
> especially when call mbind after mlockall.
>
> Adjust the place of calling mlockall, calling mbind before mlockall
> can remarkably reduce the time of VM's startup.
>
> Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
The idea makes absolute sense to me:
bind after lock will force data copy of
all pages. bind before lock gives us an
indication where to put data on fault in.
Acked-by: Michael S. Tsirkin <mst@redhat.com>
> ---
> Hi,
>
> Actually, for mbind and mlockall, i have made a test about the time consuming
> for the different call sequence.
>
> The results is shown below. It is obviously that mlockall called before mbind is
> more time-consuming.
>
> Besides, this patch is OK with memory hotplug.
>
> TEST CODE:
> if (mbind_first) {
> printf("mbind --> mlockall\n");
> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2,
> MPOL_MF_STRICT | MPOL_MF_MOVE);
> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
> MPOL_MF_STRICT | MPOL_MF_MOVE);
> mlockall(MCL_CURRENT | MCL_FUTURE);
> } else {
> printf("mlockall --> mbind\n");
> mlockall(MCL_CURRENT | MCL_FUTURE);
> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 ,
> MPOL_MF_STRICT | MPOL_MF_MOVE);
> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
> MPOL_MF_STRICT | MPOL_MF_MOVE);
> }
>
> RESULT 1:
> #time /home/test_mbind 10240 0
> memroy size 10737418240
> mlockall --> mbind
>
> real 0m11.886s
> user 0m0.004s
> sys 0m11.865s
> #time /home/test_mbind 10240 1
> memroy size 10737418240
> mbind --> mlockall
>
> real 0m5.334s
> user 0m0.000s
> sys 0m5.324s
>
> RESULT 2:
> #time /home/test_mbind 4096 0
> memroy size 4294967296
> mlockall --> mbind
>
> real 0m5.503s
> user 0m0.000s
> sys 0m5.492s
> #time /home/test_mbind 4096 1
> memroy size 4294967296
> mbind --> mlockall
>
> real 0m2.139s
> user 0m0.000s
> sys 0m2.132s
>
> Best Regards,
> zhanghailiang
> ---
> vl.c | 11 +++++------
> 1 file changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/vl.c b/vl.c
> index dc792fe..adf4770 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL;
> ram_addr_t ram_size;
> const char *mem_path = NULL;
> int mem_prealloc = 0; /* force preallocation of physical target memory */
> +int enable_mlock = false;
> int nb_nics;
> NICInfo nd_table[MAX_NICS];
> int autostart;
> @@ -1421,12 +1422,8 @@ static void smp_parse(QemuOpts *opts)
>
> }
>
> -static void configure_realtime(QemuOpts *opts)
> +static void realtime_init(void)
> {
> - bool enable_mlock;
> -
> - enable_mlock = qemu_opt_get_bool(opts, "mlock", true);
> -
> if (enable_mlock) {
> if (os_mlock() < 0) {
> fprintf(stderr, "qemu: locking memory failed\n");
> @@ -3973,7 +3970,7 @@ int main(int argc, char **argv, char **envp)
> if (!opts) {
> exit(1);
> }
> - configure_realtime(opts);
> + enable_mlock = qemu_opt_get_bool(opts, "mlock", true);
> break;
> case QEMU_OPTION_msg:
> opts = qemu_opts_parse(qemu_find_opts("msg"), optarg, 0);
> @@ -4441,6 +4438,8 @@ int main(int argc, char **argv, char **envp)
>
> machine_class->init(current_machine);
>
> + realtime_init();
> +
> audio_init();
>
> cpu_synchronize_all_post_init();
> --
> 1.7.12.4
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [Qemu-devel] [PATCH] vl: Adjust the place of calling mlockall to speedup VM's startup
2014-09-23 8:30 ` Michael S. Tsirkin
@ 2014-09-23 8:35 ` Hu Tao
2014-09-23 9:06 ` zhanghailiang
2014-09-23 10:19 ` zhanghailiang
1 sibling, 1 reply; 5+ messages in thread
From: Hu Tao @ 2014-09-23 8:35 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: imammedo, peter.huangpeng, luonengjun, zhanghailiang, qemu-devel
On Tue, Sep 23, 2014 at 11:30:26AM +0300, Michael S. Tsirkin wrote:
> On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote:
> > If we configure mlock=on and memory policy=bind at the same time,
> > It will consume lots of time for system to treat with memory,
> > especially when call mbind after mlockall.
> >
> > Adjust the place of calling mlockall, calling mbind before mlockall
> > can remarkably reduce the time of VM's startup.
> >
> > Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
>
> The idea makes absolute sense to me:
> bind after lock will force data copy of
> all pages. bind before lock gives us an
> indication where to put data on fault in.
Agreed.
>
> Acked-by: Michael S. Tsirkin <mst@redhat.com>
>
>
> > ---
> > Hi,
> >
> > Actually, for mbind and mlockall, i have made a test about the time consuming
> > for the different call sequence.
> >
> > The results is shown below. It is obviously that mlockall called before mbind is
> > more time-consuming.
> >
> > Besides, this patch is OK with memory hotplug.
> >
> > TEST CODE:
> > if (mbind_first) {
> > printf("mbind --> mlockall\n");
> > mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2,
> > MPOL_MF_STRICT | MPOL_MF_MOVE);
> > mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
> > MPOL_MF_STRICT | MPOL_MF_MOVE);
> > mlockall(MCL_CURRENT | MCL_FUTURE);
> > } else {
> > printf("mlockall --> mbind\n");
> > mlockall(MCL_CURRENT | MCL_FUTURE);
> > mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 ,
> > MPOL_MF_STRICT | MPOL_MF_MOVE);
> > mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
> > MPOL_MF_STRICT | MPOL_MF_MOVE);
> > }
> >
> > RESULT 1:
> > #time /home/test_mbind 10240 0
> > memroy size 10737418240
> > mlockall --> mbind
> >
> > real 0m11.886s
> > user 0m0.004s
> > sys 0m11.865s
> > #time /home/test_mbind 10240 1
> > memroy size 10737418240
> > mbind --> mlockall
> >
> > real 0m5.334s
> > user 0m0.000s
> > sys 0m5.324s
> >
> > RESULT 2:
> > #time /home/test_mbind 4096 0
> > memroy size 4294967296
> > mlockall --> mbind
> >
> > real 0m5.503s
> > user 0m0.000s
> > sys 0m5.492s
> > #time /home/test_mbind 4096 1
> > memroy size 4294967296
> > mbind --> mlockall
> >
> > real 0m2.139s
> > user 0m0.000s
> > sys 0m2.132s
> >
> > Best Regards,
> > zhanghailiang
> > ---
> > vl.c | 11 +++++------
> > 1 file changed, 5 insertions(+), 6 deletions(-)
> >
> > diff --git a/vl.c b/vl.c
> > index dc792fe..adf4770 100644
> > --- a/vl.c
> > +++ b/vl.c
> > @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL;
> > ram_addr_t ram_size;
> > const char *mem_path = NULL;
> > int mem_prealloc = 0; /* force preallocation of physical target memory */
> > +int enable_mlock = false;
Why not bool?
Regards,
Hu
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [Qemu-devel] [PATCH] vl: Adjust the place of calling mlockall to speedup VM's startup
2014-09-23 8:35 ` Hu Tao
@ 2014-09-23 9:06 ` zhanghailiang
0 siblings, 0 replies; 5+ messages in thread
From: zhanghailiang @ 2014-09-23 9:06 UTC (permalink / raw)
To: Hu Tao, Michael S. Tsirkin
Cc: imammedo, luonengjun, qemu-devel, peter.huangpeng
On 2014/9/23 16:35, Hu Tao wrote:
> On Tue, Sep 23, 2014 at 11:30:26AM +0300, Michael S. Tsirkin wrote:
>> On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote:
>>> If we configure mlock=on and memory policy=bind at the same time,
>>> It will consume lots of time for system to treat with memory,
>>> especially when call mbind after mlockall.
>>>
>>> Adjust the place of calling mlockall, calling mbind before mlockall
>>> can remarkably reduce the time of VM's startup.
>>>
>>> Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
>>
>> The idea makes absolute sense to me:
>> bind after lock will force data copy of
>> all pages. bind before lock gives us an
>> indication where to put data on fault in.
>
> Agreed.
>
>>
>> Acked-by: Michael S. Tsirkin <mst@redhat.com>
>>
>>
>>> ---
>>> Hi,
>>>
>>> Actually, for mbind and mlockall, i have made a test about the time consuming
>>> for the different call sequence.
>>>
>>> The results is shown below. It is obviously that mlockall called before mbind is
>>> more time-consuming.
>>>
>>> Besides, this patch is OK with memory hotplug.
>>>
>>> TEST CODE:
>>> if (mbind_first) {
>>> printf("mbind --> mlockall\n");
>>> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2,
>>> MPOL_MF_STRICT | MPOL_MF_MOVE);
>>> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
>>> MPOL_MF_STRICT | MPOL_MF_MOVE);
>>> mlockall(MCL_CURRENT | MCL_FUTURE);
>>> } else {
>>> printf("mlockall --> mbind\n");
>>> mlockall(MCL_CURRENT | MCL_FUTURE);
>>> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 ,
>>> MPOL_MF_STRICT | MPOL_MF_MOVE);
>>> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
>>> MPOL_MF_STRICT | MPOL_MF_MOVE);
>>> }
>>>
>>> RESULT 1:
>>> #time /home/test_mbind 10240 0
>>> memroy size 10737418240
>>> mlockall --> mbind
>>>
>>> real 0m11.886s
>>> user 0m0.004s
>>> sys 0m11.865s
>>> #time /home/test_mbind 10240 1
>>> memroy size 10737418240
>>> mbind --> mlockall
>>>
>>> real 0m5.334s
>>> user 0m0.000s
>>> sys 0m5.324s
>>>
>>> RESULT 2:
>>> #time /home/test_mbind 4096 0
>>> memroy size 4294967296
>>> mlockall --> mbind
>>>
>>> real 0m5.503s
>>> user 0m0.000s
>>> sys 0m5.492s
>>> #time /home/test_mbind 4096 1
>>> memroy size 4294967296
>>> mbind --> mlockall
>>>
>>> real 0m2.139s
>>> user 0m0.000s
>>> sys 0m2.132s
>>>
>>> Best Regards,
>>> zhanghailiang
>>> ---
>>> vl.c | 11 +++++------
>>> 1 file changed, 5 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/vl.c b/vl.c
>>> index dc792fe..adf4770 100644
>>> --- a/vl.c
>>> +++ b/vl.c
>>> @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL;
>>> ram_addr_t ram_size;
>>> const char *mem_path = NULL;
>>> int mem_prealloc = 0; /* force preallocation of physical target memory */
>>> +int enable_mlock = false;
>
> Why not bool?
>
Er, that is my fault, Will fix it and submit V2, Thanks;)
> Regards,
> Hu
>
> .
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH] vl: Adjust the place of calling mlockall to speedup VM's startup
2014-09-23 8:30 ` Michael S. Tsirkin
2014-09-23 8:35 ` Hu Tao
@ 2014-09-23 10:19 ` zhanghailiang
1 sibling, 0 replies; 5+ messages in thread
From: zhanghailiang @ 2014-09-23 10:19 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: imammedo, peter.huangpeng, luonengjun, qemu-devel, hutao
On 2014/9/23 16:30, Michael S. Tsirkin wrote:
> On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote:
>> If we configure mlock=on and memory policy=bind at the same time,
>> It will consume lots of time for system to treat with memory,
>> especially when call mbind after mlockall.
>>
>> Adjust the place of calling mlockall, calling mbind before mlockall
>> can remarkably reduce the time of VM's startup.
>>
>> Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com>
>
> The idea makes absolute sense to me:
> bind after lock will force data copy of
> all pages. bind before lock gives us an
> indication where to put data on fault in.
>
> Acked-by: Michael S. Tsirkin <mst@redhat.com>
>
>
Thanks for your quick reviewing..
Best Regards,
zhanghailiang
>> ---
>> Hi,
>>
>> Actually, for mbind and mlockall, i have made a test about the time consuming
>> for the different call sequence.
>>
>> The results is shown below. It is obviously that mlockall called before mbind is
>> more time-consuming.
>>
>> Besides, this patch is OK with memory hotplug.
>>
>> TEST CODE:
>> if (mbind_first) {
>> printf("mbind --> mlockall\n");
>> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2,
>> MPOL_MF_STRICT | MPOL_MF_MOVE);
>> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
>> MPOL_MF_STRICT | MPOL_MF_MOVE);
>> mlockall(MCL_CURRENT | MCL_FUTURE);
>> } else {
>> printf("mlockall --> mbind\n");
>> mlockall(MCL_CURRENT | MCL_FUTURE);
>> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 ,
>> MPOL_MF_STRICT | MPOL_MF_MOVE);
>> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
>> MPOL_MF_STRICT | MPOL_MF_MOVE);
>> }
>>
>> RESULT 1:
>> #time /home/test_mbind 10240 0
>> memroy size 10737418240
>> mlockall --> mbind
>>
>> real 0m11.886s
>> user 0m0.004s
>> sys 0m11.865s
>> #time /home/test_mbind 10240 1
>> memroy size 10737418240
>> mbind --> mlockall
>>
>> real 0m5.334s
>> user 0m0.000s
>> sys 0m5.324s
>>
>> RESULT 2:
>> #time /home/test_mbind 4096 0
>> memroy size 4294967296
>> mlockall --> mbind
>>
>> real 0m5.503s
>> user 0m0.000s
>> sys 0m5.492s
>> #time /home/test_mbind 4096 1
>> memroy size 4294967296
>> mbind --> mlockall
>>
>> real 0m2.139s
>> user 0m0.000s
>> sys 0m2.132s
>>
>> Best Regards,
>> zhanghailiang
>> ---
>> vl.c | 11 +++++------
>> 1 file changed, 5 insertions(+), 6 deletions(-)
>>
>> diff --git a/vl.c b/vl.c
>> index dc792fe..adf4770 100644
>> --- a/vl.c
>> +++ b/vl.c
>> @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL;
>> ram_addr_t ram_size;
>> const char *mem_path = NULL;
>> int mem_prealloc = 0; /* force preallocation of physical target memory */
>> +int enable_mlock = false;
>> int nb_nics;
>> NICInfo nd_table[MAX_NICS];
>> int autostart;
>> @@ -1421,12 +1422,8 @@ static void smp_parse(QemuOpts *opts)
>>
>> }
>>
>> -static void configure_realtime(QemuOpts *opts)
>> +static void realtime_init(void)
>> {
>> - bool enable_mlock;
>> -
>> - enable_mlock = qemu_opt_get_bool(opts, "mlock", true);
>> -
>> if (enable_mlock) {
>> if (os_mlock() < 0) {
>> fprintf(stderr, "qemu: locking memory failed\n");
>> @@ -3973,7 +3970,7 @@ int main(int argc, char **argv, char **envp)
>> if (!opts) {
>> exit(1);
>> }
>> - configure_realtime(opts);
>> + enable_mlock = qemu_opt_get_bool(opts, "mlock", true);
>> break;
>> case QEMU_OPTION_msg:
>> opts = qemu_opts_parse(qemu_find_opts("msg"), optarg, 0);
>> @@ -4441,6 +4438,8 @@ int main(int argc, char **argv, char **envp)
>>
>> machine_class->init(current_machine);
>>
>> + realtime_init();
>> +
>> audio_init();
>>
>> cpu_synchronize_all_post_init();
>> --
>> 1.7.12.4
>>
>
> .
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2014-09-23 10:20 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-09-23 7:57 [Qemu-devel] [PATCH] vl: Adjust the place of calling mlockall to speedup VM's startup zhanghailiang
2014-09-23 8:30 ` Michael S. Tsirkin
2014-09-23 8:35 ` Hu Tao
2014-09-23 9:06 ` zhanghailiang
2014-09-23 10:19 ` zhanghailiang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).