* [LTP] [PATCH] Set tunable value of min_free_kbytes lower @ 2012-08-23 6:43 shuang.qiu 2012-08-23 8:59 ` Zhouping Liu 2012-08-24 8:11 ` Zhouping Liu 0 siblings, 2 replies; 16+ messages in thread From: shuang.qiu @ 2012-08-23 6:43 UTC (permalink / raw) To: ltp-list From: Shuang Qiu <shuang.qiu@oracle.com> The value of min_free_kbytes is set too high(half of mem_free) during testing min_free_kbytes tunable.It often cause the system to become out-of-memory.Using 20% of mem_free to avoid oom and system hang. --- testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c b/testcases/kernel/mem/tunable/min_free_kbytes.c index 00ead04..8a03f85 100644 --- a/testcases/kernel/mem/tunable/min_free_kbytes.c +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c @@ -128,7 +128,7 @@ static void test_tune(unsigned long overcommit_policy) /* case2 */ else { memfree = read_meminfo("MemFree:"); - tune = memfree / 2; + tune = memfree / 5; set_sys_tune("min_free_kbytes", tune, 1); } -- 1.7.7 ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-23 6:43 [LTP] [PATCH] Set tunable value of min_free_kbytes lower shuang.qiu @ 2012-08-23 8:59 ` Zhouping Liu 2012-08-23 9:00 ` Wanlong Gao 2012-08-24 8:11 ` Zhouping Liu 1 sibling, 1 reply; 16+ messages in thread From: Zhouping Liu @ 2012-08-23 8:59 UTC (permalink / raw) To: shuang qiu; +Cc: ltp-list ----- Original Message ----- > From: "shuang qiu" <shuang.qiu@oracle.com> > To: ltp-list@lists.sourceforge.net > Sent: Thursday, August 23, 2012 2:43:01 PM > Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > From: Shuang Qiu <shuang.qiu@oracle.com> > > The value of min_free_kbytes is set too high(half of mem_free) during > testing min_free_kbytes tunable.It often cause the system to become > out-of-memory.Using 20% of mem_free to avoid oom and system hang. Thanks for the patch, I will test the patch, will provide feedback ASAP. > --- > testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c > b/testcases/kernel/mem/tunable/min_free_kbytes.c > index 00ead04..8a03f85 100644 > --- a/testcases/kernel/mem/tunable/min_free_kbytes.c > +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c > @@ -128,7 +128,7 @@ static void test_tune(unsigned long > overcommit_policy) > /* case2 */ > else { > memfree = read_meminfo("MemFree:"); > - tune = memfree / 2; > + tune = memfree / 5; > set_sys_tune("min_free_kbytes", tune, 1); > } > > -- > 1.7.7 > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. > Discussions > will include endpoint security, mobile security and the latest in > malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Ltp-list mailing list > Ltp-list@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ltp-list > -- Thanks, Zhouping ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-23 8:59 ` Zhouping Liu @ 2012-08-23 9:00 ` Wanlong Gao 0 siblings, 0 replies; 16+ messages in thread From: Wanlong Gao @ 2012-08-23 9:00 UTC (permalink / raw) To: Zhouping Liu; +Cc: ltp-list On 08/23/2012 04:59 PM, Zhouping Liu wrote: > > ----- Original Message ----- >> > From: "shuang qiu" <shuang.qiu@oracle.com> >> > To: ltp-list@lists.sourceforge.net >> > Sent: Thursday, August 23, 2012 2:43:01 PM >> > Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower >> > >> > From: Shuang Qiu <shuang.qiu@oracle.com> >> > >> > The value of min_free_kbytes is set too high(half of mem_free) during >> > testing min_free_kbytes tunable.It often cause the system to become >> > out-of-memory.Using 20% of mem_free to avoid oom and system hang. > Thanks for the patch, I will test the patch, will provide feedback ASAP. > Great! thank you Zhouping. Wanlong Gao ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-23 6:43 [LTP] [PATCH] Set tunable value of min_free_kbytes lower shuang.qiu 2012-08-23 8:59 ` Zhouping Liu @ 2012-08-24 8:11 ` Zhouping Liu 2012-08-24 10:18 ` Shuang Qiu 2012-08-27 5:16 ` Zhouping Liu 1 sibling, 2 replies; 16+ messages in thread From: Zhouping Liu @ 2012-08-24 8:11 UTC (permalink / raw) To: shuang qiu; +Cc: ltp-list ----- Original Message ----- > From: "shuang qiu" <shuang.qiu@oracle.com> > To: ltp-list@lists.sourceforge.net > Sent: Thursday, August 23, 2012 2:43:01 PM > Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > From: Shuang Qiu <shuang.qiu@oracle.com> > > The value of min_free_kbytes is set too high(half of mem_free) during > testing min_free_kbytes tunable.It often cause the system to become > out-of-memory.Using 20% of mem_free to avoid oom and system hang. Hi, shuang I met the issue that min_free_kbytes caused system hang before, and what you said made sense. but after tested the patch, I found it still make the system hang with the patch, so I don't think this patch can fix the issue completely, maybe there are other factors, or need lower tune value. always suppose it was because of the high tune value, it's more better to add an optional to control it. also I will investigate it deeper to look after the real reason. please comment if you have other ideas. Thanks, Zhouping > --- > testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c > b/testcases/kernel/mem/tunable/min_free_kbytes.c > index 00ead04..8a03f85 100644 > --- a/testcases/kernel/mem/tunable/min_free_kbytes.c > +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c > @@ -128,7 +128,7 @@ static void test_tune(unsigned long > overcommit_policy) > /* case2 */ > else { > memfree = read_meminfo("MemFree:"); > - tune = memfree / 2; > + tune = memfree / 5; > set_sys_tune("min_free_kbytes", tune, 1); > } > > -- > 1.7.7 > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. > Discussions > will include endpoint security, mobile security and the latest in > malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Ltp-list mailing list > Ltp-list@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ltp-list > ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-24 8:11 ` Zhouping Liu @ 2012-08-24 10:18 ` Shuang Qiu 2012-08-24 13:48 ` Zhouping Liu 2012-08-27 5:16 ` Zhouping Liu 1 sibling, 1 reply; 16+ messages in thread From: Shuang Qiu @ 2012-08-24 10:18 UTC (permalink / raw) To: Zhouping Liu; +Cc: ltp-list On 08/24/2012 04:11 PM, Zhouping Liu wrote: > > ----- Original Message ----- >> From: "shuang qiu" <shuang.qiu@oracle.com> >> To: ltp-list@lists.sourceforge.net >> Sent: Thursday, August 23, 2012 2:43:01 PM >> Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower >> >> From: Shuang Qiu <shuang.qiu@oracle.com> >> >> The value of min_free_kbytes is set too high(half of mem_free) during >> testing min_free_kbytes tunable.It often cause the system to become >> out-of-memory.Using 20% of mem_free to avoid oom and system hang. > Hi, shuang > > I met the issue that min_free_kbytes caused system hang before, and what you said made sense. > > but after tested the patch, I found it still make the system hang with the patch, so I don't think > this patch can fix the issue completely, maybe there are other factors, or need lower tune value. > always suppose it was because of the high tune value, it's more better to add an optional to control it. > also I will investigate it deeper to look after the real reason. please comment if you have other ideas. > > Thanks, > Zhouping Thanks for testing,Zhouping. I find it says "However, setting this parameter to a value that is too high (5-10% of total system memory) will cause your system to become out-of-memory immediately." in redhat memory-tunables document. https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-tunables.html Set the value less than 5% of total memory will make it more safe. Let's think about a better option to control it. It is great that you will investigate it deeper. Seems there are some other problems in this case,it always failed with i386. The type of total_mem "unsigned long" is not suitable with nowadays memory ,it will overflow(with i386) in line 177:map_count = total_mem * KB / MAP_SIZE and the value of map_count is not correct. Using "unsigned long long" could fix this issue. But the case still failed after fix it. Could you help to investigate it too. Thanks Shuang >> --- >> testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c >> b/testcases/kernel/mem/tunable/min_free_kbytes.c >> index 00ead04..8a03f85 100644 >> --- a/testcases/kernel/mem/tunable/min_free_kbytes.c >> +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c >> @@ -128,7 +128,7 @@ static void test_tune(unsigned long >> overcommit_policy) >> /* case2 */ >> else { >> memfree = read_meminfo("MemFree:"); >> - tune = memfree / 2; >> + tune = memfree / 5; >> set_sys_tune("min_free_kbytes", tune, 1); >> } >> >> -- >> 1.7.7 >> >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. >> Discussions >> will include endpoint security, mobile security and the latest in >> malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Ltp-list mailing list >> Ltp-list@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/ltp-list >> ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-24 10:18 ` Shuang Qiu @ 2012-08-24 13:48 ` Zhouping Liu 2012-08-27 3:03 ` Shuang Qiu 0 siblings, 1 reply; 16+ messages in thread From: Zhouping Liu @ 2012-08-24 13:48 UTC (permalink / raw) To: Shuang Qiu; +Cc: ltp-list ----- Original Message ----- > From: "Shuang Qiu" <shuang.qiu@oracle.com> > To: "Zhouping Liu" <zliu@redhat.com> > Cc: ltp-list@lists.sourceforge.net > Sent: Friday, August 24, 2012 6:18:00 PM > Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > On 08/24/2012 04:11 PM, Zhouping Liu wrote: > > > > ----- Original Message ----- > >> From: "shuang qiu" <shuang.qiu@oracle.com> > >> To: ltp-list@lists.sourceforge.net > >> Sent: Thursday, August 23, 2012 2:43:01 PM > >> Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > >> > >> From: Shuang Qiu <shuang.qiu@oracle.com> > >> > >> The value of min_free_kbytes is set too high(half of mem_free) > >> during > >> testing min_free_kbytes tunable.It often cause the system to > >> become > >> out-of-memory.Using 20% of mem_free to avoid oom and system hang. > > Hi, shuang > > > > I met the issue that min_free_kbytes caused system hang before, and > > what you said made sense. > > > > but after tested the patch, I found it still make the system hang > > with the patch, so I don't think > > this patch can fix the issue completely, maybe there are other > > factors, or need lower tune value. > > always suppose it was because of the high tune value, it's more > > better to add an optional to control it. > > also I will investigate it deeper to look after the real reason. > > please comment if you have other ideas. > > > > Thanks, > > Zhouping > Thanks for testing,Zhouping. > > I find it says "However, setting this parameter to a value that is > too > high (5-10% of total system memory) will cause your system to become > out-of-memory immediately." in redhat memory-tunables document. > https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-tunables.html > Set the value less than 5% of total memory will make it more safe. Thanks for sharing this! I will take your ideas into account. > Let's think about a better option to control it. Ok, I will prepare a patch for this. > > It is great that you will investigate it deeper. > Seems there are some other problems in this case,it always failed > with i386. > The type of total_mem "unsigned long" is not suitable with nowadays > memory ,it will overflow(with i386) in line 177:map_count = total_mem > * > KB / MAP_SIZE and the value of map_count is not correct. > Using "unsigned long long" could fix this issue. > But the case still failed after fix it. > Could you help to investigate it too. sure. -- Thanks, Zhouping ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-24 13:48 ` Zhouping Liu @ 2012-08-27 3:03 ` Shuang Qiu 2012-08-27 5:01 ` Zhouping Liu 0 siblings, 1 reply; 16+ messages in thread From: Shuang Qiu @ 2012-08-27 3:03 UTC (permalink / raw) To: Zhouping Liu; +Cc: ltp-list On 08/24/2012 09:48 PM, Zhouping Liu wrote: > > ----- Original Message ----- >> From: "Shuang Qiu" <shuang.qiu@oracle.com> >> To: "Zhouping Liu" <zliu@redhat.com> >> Cc: ltp-list@lists.sourceforge.net >> Sent: Friday, August 24, 2012 6:18:00 PM >> Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower >> >> On 08/24/2012 04:11 PM, Zhouping Liu wrote: >>> ----- Original Message ----- >>>> From: "shuang qiu" <shuang.qiu@oracle.com> >>>> To: ltp-list@lists.sourceforge.net >>>> Sent: Thursday, August 23, 2012 2:43:01 PM >>>> Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower >>>> >>>> From: Shuang Qiu <shuang.qiu@oracle.com> >>>> >>>> The value of min_free_kbytes is set too high(half of mem_free) >>>> during >>>> testing min_free_kbytes tunable.It often cause the system to >>>> become >>>> out-of-memory.Using 20% of mem_free to avoid oom and system hang. >>> Hi, shuang >>> >>> I met the issue that min_free_kbytes caused system hang before, and >>> what you said made sense. >>> >>> but after tested the patch, I found it still make the system hang >>> with the patch, so I don't think >>> this patch can fix the issue completely, maybe there are other >>> factors, or need lower tune value. >>> always suppose it was because of the high tune value, it's more >>> better to add an optional to control it. >>> also I will investigate it deeper to look after the real reason. >>> please comment if you have other ideas. >>> >>> Thanks, >>> Zhouping >> Thanks for testing,Zhouping. >> >> I find it says "However, setting this parameter to a value that is >> too >> high (5-10% of total system memory) will cause your system to become >> out-of-memory immediately." in redhat memory-tunables document. >> https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-tunables.html >> Set the value less than 5% of total memory will make it more safe. > Thanks for sharing this! I will take your ideas into account. > >> Let's think about a better option to control it. > Ok, I will prepare a patch for this. > > >> It is great that you will investigate it deeper. >> Seems there are some other problems in this case,it always failed >> with i386. >> The type of total_mem "unsigned long" is not suitable with nowadays >> memory ,it will overflow(with i386) in line 177:map_count = total_mem >> * >> KB / MAP_SIZE and the value of map_count is not correct. >> Using "unsigned long long" could fix this issue. >> But the case still failed after fix it. >> Could you help to investigate it too. > sure. > Hi Zhouping, I think the reason for the failures which I met is also that it just catched oom_killer scenario when overcommit_memory=1. I found your patch before: commit 4f267b6fd11749c91211c9dbfc706cf46919650b Author: Zhouping Liu <zliu@redhat.com> Date: Fri Mar 30 22:29:58 2012 +0800 mm/tunable: fix unexpected error log in min_free_kbytes when overcommit_memory=0, mmap() will be destroyed in two below scenario: 1) MAP_FAILED with ENOMEM, 2) oom_killer kill it. without the patch, it just catched oom_killer scenario, the patch fixed it. Could this patch also for when overcommit_memory=1 scenario? Thanks Shuang ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-27 3:03 ` Shuang Qiu @ 2012-08-27 5:01 ` Zhouping Liu 0 siblings, 0 replies; 16+ messages in thread From: Zhouping Liu @ 2012-08-27 5:01 UTC (permalink / raw) To: Shuang Qiu; +Cc: ltp-list ----- Original Message ----- > From: "Shuang Qiu" <shuang.qiu@oracle.com> > To: "Zhouping Liu" <zliu@redhat.com> > Cc: ltp-list@lists.sourceforge.net > Sent: Monday, August 27, 2012 11:03:37 AM > Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > On 08/24/2012 09:48 PM, Zhouping Liu wrote: > > > > ----- Original Message ----- > >> From: "Shuang Qiu" <shuang.qiu@oracle.com> > >> To: "Zhouping Liu" <zliu@redhat.com> > >> Cc: ltp-list@lists.sourceforge.net > >> Sent: Friday, August 24, 2012 6:18:00 PM > >> Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes > >> lower > >> > >> On 08/24/2012 04:11 PM, Zhouping Liu wrote: > >>> ----- Original Message ----- > >>>> From: "shuang qiu" <shuang.qiu@oracle.com> > >>>> To: ltp-list@lists.sourceforge.net > >>>> Sent: Thursday, August 23, 2012 2:43:01 PM > >>>> Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes > >>>> lower > >>>> > >>>> From: Shuang Qiu <shuang.qiu@oracle.com> > >>>> > >>>> The value of min_free_kbytes is set too high(half of mem_free) > >>>> during > >>>> testing min_free_kbytes tunable.It often cause the system to > >>>> become > >>>> out-of-memory.Using 20% of mem_free to avoid oom and system > >>>> hang. > >>> Hi, shuang > >>> > >>> I met the issue that min_free_kbytes caused system hang before, > >>> and > >>> what you said made sense. > >>> > >>> but after tested the patch, I found it still make the system hang > >>> with the patch, so I don't think > >>> this patch can fix the issue completely, maybe there are other > >>> factors, or need lower tune value. > >>> always suppose it was because of the high tune value, it's more > >>> better to add an optional to control it. > >>> also I will investigate it deeper to look after the real reason. > >>> please comment if you have other ideas. > >>> > >>> Thanks, > >>> Zhouping > >> Thanks for testing,Zhouping. > >> > >> I find it says "However, setting this parameter to a value that is > >> too > >> high (5-10% of total system memory) will cause your system to > >> become > >> out-of-memory immediately." in redhat memory-tunables document. > >> https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-tunables.html > >> Set the value less than 5% of total memory will make it more safe. > > Thanks for sharing this! I will take your ideas into account. > > > >> Let's think about a better option to control it. > > Ok, I will prepare a patch for this. > > > > > >> It is great that you will investigate it deeper. > >> Seems there are some other problems in this case,it always failed > >> with i386. > >> The type of total_mem "unsigned long" is not suitable with > >> nowadays > >> memory ,it will overflow(with i386) in line 177:map_count = > >> total_mem > >> * > >> KB / MAP_SIZE and the value of map_count is not correct. > >> Using "unsigned long long" could fix this issue. > >> But the case still failed after fix it. > >> Could you help to investigate it too. > > sure. > > Hi, Shuang I think the below patch can't fix the issue, it looks more like a bug related to oom-killer, I have opened a bug to trace it, please see https://bugzilla.redhat.com/show_bug.cgi?id=851938 > Hi Zhouping, > I think the reason for the failures which I met is also that it just > catched oom_killer scenario when overcommit_memory=1. > I found your patch before: > commit 4f267b6fd11749c91211c9dbfc706cf46919650b > Author: Zhouping Liu <zliu@redhat.com> > Date: Fri Mar 30 22:29:58 2012 +0800 > > mm/tunable: fix unexpected error log in min_free_kbytes > > when overcommit_memory=0, mmap() will be destroyed in two below > scenario: > 1) MAP_FAILED with ENOMEM, > 2) oom_killer kill it. > without the patch, it just catched oom_killer scenario, > the patch fixed it. > > > Could this patch also for when overcommit_memory=1 scenario? > > Thanks > Shuang > > > -- Thanks, Zhouping ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-24 8:11 ` Zhouping Liu 2012-08-24 10:18 ` Shuang Qiu @ 2012-08-27 5:16 ` Zhouping Liu 2012-08-28 3:11 ` Shuang Qiu 2012-08-29 14:03 ` Zhouping Liu 1 sibling, 2 replies; 16+ messages in thread From: Zhouping Liu @ 2012-08-27 5:16 UTC (permalink / raw) To: shuang qiu; +Cc: ltp-list ----- Original Message ----- > From: "Zhouping Liu" <zliu@redhat.com> > To: "shuang qiu" <shuang.qiu@oracle.com> > Cc: ltp-list@lists.sourceforge.net > Sent: Friday, August 24, 2012 4:11:41 PM > Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > > > ----- Original Message ----- > > From: "shuang qiu" <shuang.qiu@oracle.com> > > To: ltp-list@lists.sourceforge.net > > Sent: Thursday, August 23, 2012 2:43:01 PM > > Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > > > From: Shuang Qiu <shuang.qiu@oracle.com> > > > > The value of min_free_kbytes is set too high(half of mem_free) > > during > > testing min_free_kbytes tunable.It often cause the system to become > > out-of-memory.Using 20% of mem_free to avoid oom and system hang. > > Hi, shuang > > I met the issue that min_free_kbytes caused system hang before, and > what you said made sense. For the issue that min_free_kbytes caused system hang, I prepared a patch for it, at the same time, in order to enlarge the case's coverage, I add a new case(2x default_tune) I have tested it in my box, it's good. Shuang, can you review and test the patch? diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c b/testcases/kernel/mem/tunable/min_free_kbytes.c index 00ead04..ded0e45 100644 --- a/testcases/kernel/mem/tunable/min_free_kbytes.c +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c @@ -10,7 +10,8 @@ * the current free memory with the tunable value repeatedly. * * a) default min_free_kbytes with all overcommit memory policy - * b) half of mem_free with all overcommit memory policy + * b) 2x default value with all overcommit memory policy + * c) 10% of MemFree or %5 MemTotal with all overcommit memory policy * ******************************************************************** * Copyright (C) 2012 Red Hat, Inc. @@ -115,20 +116,27 @@ int main(int argc, char *argv[]) static void test_tune(unsigned long overcommit_policy) { int status; - int pid[2]; + int pid[3]; int ret, i; - unsigned long tune, memfree; + unsigned long tune, memfree, memtotal; set_sys_tune("overcommit_memory", overcommit_policy, 1); - for (i = 0; i < 2; i++) { + for (i = 0; i < 3; i++) { /* case1 */ if (i == 0) set_sys_tune("min_free_kbytes", default_tune, 1); /* case2 */ - else { + else if (i == 1) { + set_sys_tune("min_free_kbytes", 2 * default_tune, 1); + /* case3 */ + } else { memfree = read_meminfo("MemFree:"); - tune = memfree / 2; + memtotal = read_meminfo("MemTotal:"); + tune = memfree / 10; + if (tune > (memtotal / 20)) + tune = memtotal / 20; + set_sys_tune("min_free_kbytes", tune, 1); } > > but after tested the patch, I found it still make the system hang > with the patch, so I don't think > this patch can fix the issue completely, maybe there are other > factors, or need lower tune value. > always suppose it was because of the high tune value, it's more > better to add an optional to control it. > also I will investigate it deeper to look after the real reason. > please comment if you have other ideas. > > Thanks, > Zhouping > > > --- > > testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +- > > 1 files changed, 1 insertions(+), 1 deletions(-) > > > > diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c > > b/testcases/kernel/mem/tunable/min_free_kbytes.c > > index 00ead04..8a03f85 100644 > > --- a/testcases/kernel/mem/tunable/min_free_kbytes.c > > +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c > > @@ -128,7 +128,7 @@ static void test_tune(unsigned long > > overcommit_policy) > > /* case2 */ > > else { > > memfree = read_meminfo("MemFree:"); > > - tune = memfree / 2; > > + tune = memfree / 5; > > set_sys_tune("min_free_kbytes", tune, 1); > > } > > > > -- > > 1.7.7 > > > > > > ------------------------------------------------------------------------------ > > Live Security Virtual Conference > > Exclusive live event will cover all the ways today's security and > > threat landscape has changed and how IT managers can respond. > > Discussions > > will include endpoint security, mobile security and the latest in > > malware > > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > > _______________________________________________ > > Ltp-list mailing list > > Ltp-list@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/ltp-list > > > -- Thanks, Zhouping ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-27 5:16 ` Zhouping Liu @ 2012-08-28 3:11 ` Shuang Qiu 2012-08-28 12:51 ` ZhouPing Liu 2012-08-29 14:03 ` Zhouping Liu 1 sibling, 1 reply; 16+ messages in thread From: Shuang Qiu @ 2012-08-28 3:11 UTC (permalink / raw) To: Zhouping Liu; +Cc: ltp-list On 08/27/2012 01:16 PM, Zhouping Liu wrote: > > ----- Original Message ----- > For the issue that min_free_kbytes caused system hang, I prepared a patch for it, > at the same time, in order to enlarge the case's coverage, I add a new case(2x default_tune) > I have tested it in my box, it's good. > > Shuang, can you review and test the patch? > > diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c b/testcases/kernel/mem/tunable/min_free_kbytes.c > index 00ead04..ded0e45 100644 > --- a/testcases/kernel/mem/tunable/min_free_kbytes.c > +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c > @@ -10,7 +10,8 @@ > * the current free memory with the tunable value repeatedly. > * > * a) default min_free_kbytes with all overcommit memory policy > - * b) half of mem_free with all overcommit memory policy > + * b) 2x default value with all overcommit memory policy > + * c) 10% of MemFree or %5 MemTotal with all overcommit memory policy > * > ******************************************************************** > * Copyright (C) 2012 Red Hat, Inc. > @@ -115,20 +116,27 @@ int main(int argc, char *argv[]) > static void test_tune(unsigned long overcommit_policy) > { > int status; > - int pid[2]; > + int pid[3]; > int ret, i; > - unsigned long tune, memfree; > + unsigned long tune, memfree, memtotal; > > set_sys_tune("overcommit_memory", overcommit_policy, 1); > > - for (i = 0; i < 2; i++) { > + for (i = 0; i < 3; i++) { > /* case1 */ > if (i == 0) > set_sys_tune("min_free_kbytes", default_tune, 1); > /* case2 */ > - else { > + else if (i == 1) { > + set_sys_tune("min_free_kbytes", 2 * default_tune, 1); > + /* case3 */ > + } else { > memfree = read_meminfo("MemFree:"); > - tune = memfree / 2; > + memtotal = read_meminfo("MemTotal:"); > + tune = memfree / 10; > + if (tune > (memtotal / 20)) > + tune = memtotal / 20; > + > set_sys_tune("min_free_kbytes", tune, 1); > } > Hi Zhouping, I have tested this patch,it works. But I found another problem with this case that it is not work expectedly with the scenario that i386 system(PAE kernel) which has >4GB memory. i.e. In i386 system with 7GB free memory,it could only eat about 3GB memory in one mem-hog fork process. And we can also patch the overflow issue which I mentioned before: The type of total_mem "unsigned long" is not suitable with nowadays memory ,it will overflow(with i386) in line 177:map_count = total_mem * KB / MAP_SIZE and the value of map_count is not correct. Using "unsigned long long" could fix this issue. Thanks Shuang ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-28 3:11 ` Shuang Qiu @ 2012-08-28 12:51 ` ZhouPing Liu 2012-08-29 8:08 ` Shuang Qiu 2012-08-29 8:29 ` Wanlong Gao 0 siblings, 2 replies; 16+ messages in thread From: ZhouPing Liu @ 2012-08-28 12:51 UTC (permalink / raw) To: Shuang Qiu; +Cc: ltp-list On 08/28/2012 11:11 AM, Shuang Qiu wrote: > On 08/27/2012 01:16 PM, Zhouping Liu wrote: >> >> ----- Original Message ----- >> For the issue that min_free_kbytes caused system hang, I prepared a >> patch for it, >> at the same time, in order to enlarge the case's coverage, I add a >> new case(2x default_tune) >> I have tested it in my box, it's good. >> >> Shuang, can you review and test the patch? >> >> diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c >> b/testcases/kernel/mem/tunable/min_free_kbytes.c >> index 00ead04..ded0e45 100644 >> --- a/testcases/kernel/mem/tunable/min_free_kbytes.c >> +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c >> @@ -10,7 +10,8 @@ >> * the current free memory with the tunable value repeatedly. >> * >> * a) default min_free_kbytes with all overcommit memory policy >> - * b) half of mem_free with all overcommit memory policy >> + * b) 2x default value with all overcommit memory policy >> + * c) 10% of MemFree or %5 MemTotal with all overcommit memory policy >> * >> ******************************************************************** >> * Copyright (C) 2012 Red Hat, Inc. >> @@ -115,20 +116,27 @@ int main(int argc, char *argv[]) >> static void test_tune(unsigned long overcommit_policy) >> { >> int status; >> - int pid[2]; >> + int pid[3]; >> int ret, i; >> - unsigned long tune, memfree; >> + unsigned long tune, memfree, memtotal; >> set_sys_tune("overcommit_memory", overcommit_policy, 1); >> - for (i = 0; i < 2; i++) { >> + for (i = 0; i < 3; i++) { >> /* case1 */ >> if (i == 0) >> set_sys_tune("min_free_kbytes", >> default_tune, 1); >> /* case2 */ >> - else { >> + else if (i == 1) { >> + set_sys_tune("min_free_kbytes", 2 * >> default_tune, 1); >> + /* case3 */ >> + } else { >> memfree = read_meminfo("MemFree:"); >> - tune = memfree / 2; >> + memtotal = read_meminfo("MemTotal:"); >> + tune = memfree / 10; >> + if (tune > (memtotal / 20)) >> + tune = memtotal / 20; >> + >> set_sys_tune("min_free_kbytes", tune, 1); >> } > Hi Zhouping, > I have tested this patch,it works. > > But I found another problem with this case that it is not work > expectedly with the scenario that i386 system(PAE kernel) which has > >4GB memory. > i.e. In i386 system with 7GB free memory,it could only eat about 3GB > memory in one mem-hog fork process. yes, you are right. I tried to fork several processes to hog memory, but I don't think it's a better way, so a good solution(I think) is included in below patch. > > And we can also patch the overflow issue which I mentioned before: > > The type of total_mem "unsigned long" is not suitable with nowadays > memory ,it will overflow(with i386) in line 177:map_count = total_mem > * > KB / MAP_SIZE and the value of map_count is not correct. > Using "unsigned long long" could fix this issue. after re-check the code, I found map_count was not necessary, so I removed it, at the same time, it fixed your the overflow issue. please review the patch again, if you agree the fix, I will send a formal patch set in new subject: diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c b/testcases/kernel/mem/tunable/min_free_kbytes.c index ded0e45..af25597 100644 --- a/testcases/kernel/mem/tunable/min_free_kbytes.c +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c @@ -158,8 +158,18 @@ static void test_tune(unsigned long overcommit_policy) "child unexpectedly failed: %d", status); } else if (overcommit_policy == 1) { if (!WIFSIGNALED(status) || WTERMSIG(status) != SIGKILL) + { +#if __WORDSIZE == 32 + if (total_mem < 3145728UL) +#endif tst_resm(TFAIL, "child unexpectedly failed: %d", status); +#if __WORDSIZE == 32 + else + tst_resm(TINFO, "Child can't allocate " + ">3Gb memory in 32bit system"); +#endif + } } else { if (WIFEXITED(status)) { if (WEXITSTATUS(status) != 0) { @@ -177,31 +187,23 @@ static void test_tune(unsigned long overcommit_policy) static int eatup_mem(unsigned long overcommit_policy) { - int map_count, i; int ret = 0; unsigned long memfree; - void **addrs; - - map_count = total_mem * KB / MAP_SIZE; - addrs = (void **)malloc(map_count * sizeof(void *)); - if (addrs == NULL) { - perror("malloc"); - return -1; - } + void *addrs; memfree = read_meminfo("MemFree:"); printf("memfree is %lu kB before eatup mem\n", memfree); - for (i = 0; i < map_count; i++) { - addrs[i] = mmap(NULL, MAP_SIZE, PROT_READ|PROT_WRITE, + while (1) { + addrs = mmap(NULL, MAP_SIZE, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); - if (addrs[i] == MAP_FAILED) { + if (addrs == MAP_FAILED) { if (overcommit_policy != 1 && errno != ENOMEM) { perror("mmap"); ret = -1; } break; } - memset(addrs[i], i, MAP_SIZE); + memset(addrs, 1, MAP_SIZE); } memfree = read_meminfo("MemFree:"); printf("memfree is %lu kB after eatup mem\n", memfree); Thanks, Zhouping ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-28 12:51 ` ZhouPing Liu @ 2012-08-29 8:08 ` Shuang Qiu 2012-08-29 8:29 ` Wanlong Gao 1 sibling, 0 replies; 16+ messages in thread From: Shuang Qiu @ 2012-08-29 8:08 UTC (permalink / raw) To: ZhouPing Liu; +Cc: ltp-list On 08/28/2012 08:51 PM, ZhouPing Liu wrote: > On 08/28/2012 11:11 AM, Shuang Qiu wrote: >> On 08/27/2012 01:16 PM, Zhouping Liu wrote: >>> >>> ----- Original Message ----- >>> For the issue that min_free_kbytes caused system hang, I prepared a >>> patch for it, >>> at the same time, in order to enlarge the case's coverage, I add a >>> new case(2x default_tune) >>> I have tested it in my box, it's good. >>> >>> Shuang, can you review and test the patch? >>> >>> diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c >>> b/testcases/kernel/mem/tunable/min_free_kbytes.c >>> index 00ead04..ded0e45 100644 >>> --- a/testcases/kernel/mem/tunable/min_free_kbytes.c >>> +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c >>> @@ -10,7 +10,8 @@ >>> * the current free memory with the tunable value repeatedly. >>> * >>> * a) default min_free_kbytes with all overcommit memory policy >>> - * b) half of mem_free with all overcommit memory policy >>> + * b) 2x default value with all overcommit memory policy >>> + * c) 10% of MemFree or %5 MemTotal with all overcommit memory policy >>> * >>> ******************************************************************** >>> * Copyright (C) 2012 Red Hat, Inc. >>> @@ -115,20 +116,27 @@ int main(int argc, char *argv[]) >>> static void test_tune(unsigned long overcommit_policy) >>> { >>> int status; >>> - int pid[2]; >>> + int pid[3]; >>> int ret, i; >>> - unsigned long tune, memfree; >>> + unsigned long tune, memfree, memtotal; >>> set_sys_tune("overcommit_memory", overcommit_policy, 1); >>> - for (i = 0; i < 2; i++) { >>> + for (i = 0; i < 3; i++) { >>> /* case1 */ >>> if (i == 0) >>> set_sys_tune("min_free_kbytes", >>> default_tune, 1); >>> /* case2 */ >>> - else { >>> + else if (i == 1) { >>> + set_sys_tune("min_free_kbytes", 2 * >>> default_tune, 1); >>> + /* case3 */ >>> + } else { >>> memfree = read_meminfo("MemFree:"); >>> - tune = memfree / 2; >>> + memtotal = read_meminfo("MemTotal:"); >>> + tune = memfree / 10; >>> + if (tune > (memtotal / 20)) >>> + tune = memtotal / 20; >>> + >>> set_sys_tune("min_free_kbytes", tune, 1); >>> } >> Hi Zhouping, >> I have tested this patch,it works. >> >> But I found another problem with this case that it is not work >> expectedly with the scenario that i386 system(PAE kernel) which has >> >4GB memory. >> i.e. In i386 system with 7GB free memory,it could only eat about 3GB >> memory in one mem-hog fork process. > > yes, you are right. > I tried to fork several processes to hog memory, but I don't think > it's a better way, so a good solution(I think) is included in below > patch. > >> >> And we can also patch the overflow issue which I mentioned before: >> >> The type of total_mem "unsigned long" is not suitable with nowadays >> memory ,it will overflow(with i386) in line 177:map_count = total_mem >> * >> KB / MAP_SIZE and the value of map_count is not correct. >> Using "unsigned long long" could fix this issue. > > after re-check the code, I found map_count was not necessary, so I > removed it, > at the same time, it fixed your the overflow issue. > > please review the patch again, if you agree the fix, I will send a > formal patch set in new subject: > > diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c > b/testcases/kernel/mem/tunable/min_free_kbytes.c > index ded0e45..af25597 100644 > --- a/testcases/kernel/mem/tunable/min_free_kbytes.c > +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c > @@ -158,8 +158,18 @@ static void test_tune(unsigned long > overcommit_policy) > "child unexpectedly failed: %d", > status); > } else if (overcommit_policy == 1) { > if (!WIFSIGNALED(status) || WTERMSIG(status) > != SIGKILL) > + { > +#if __WORDSIZE == 32 > + if (total_mem < 3145728UL) > +#endif > tst_resm(TFAIL, > "child unexpectedly failed: %d", > status); > +#if __WORDSIZE == 32 > + else > + tst_resm(TINFO, "Child can't > allocate " > + ">3Gb memory in 32bit > system"); > +#endif > + } > } else { > if (WIFEXITED(status)) { > if (WEXITSTATUS(status) != 0) { > @@ -177,31 +187,23 @@ static void test_tune(unsigned long > overcommit_policy) > > static int eatup_mem(unsigned long overcommit_policy) > { > - int map_count, i; > int ret = 0; > unsigned long memfree; > - void **addrs; > - > - map_count = total_mem * KB / MAP_SIZE; > - addrs = (void **)malloc(map_count * sizeof(void *)); > - if (addrs == NULL) { > - perror("malloc"); > - return -1; > - } > + void *addrs; > > memfree = read_meminfo("MemFree:"); > printf("memfree is %lu kB before eatup mem\n", memfree); > - for (i = 0; i < map_count; i++) { > - addrs[i] = mmap(NULL, MAP_SIZE, PROT_READ|PROT_WRITE, > + while (1) { > + addrs = mmap(NULL, MAP_SIZE, PROT_READ|PROT_WRITE, > MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); > - if (addrs[i] == MAP_FAILED) { > + if (addrs == MAP_FAILED) { > if (overcommit_policy != 1 && errno != ENOMEM) { > perror("mmap"); > ret = -1; > } > break; > } > - memset(addrs[i], i, MAP_SIZE); > + memset(addrs, 1, MAP_SIZE); > } > memfree = read_meminfo("MemFree:"); > printf("memfree is %lu kB after eatup mem\n", memfree); > I agree the fix and it works fine. Thanks Shuang > Thanks, > Zhouping ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-28 12:51 ` ZhouPing Liu 2012-08-29 8:08 ` Shuang Qiu @ 2012-08-29 8:29 ` Wanlong Gao 2012-08-29 8:56 ` Zhouping Liu 1 sibling, 1 reply; 16+ messages in thread From: Wanlong Gao @ 2012-08-29 8:29 UTC (permalink / raw) To: ZhouPing Liu; +Cc: ltp-list > yes, you are right. > I tried to fork several processes to hog memory, but I don't think it's > a better way, so a good solution(I think) is included in below patch. > >> >> And we can also patch the overflow issue which I mentioned before: >> >> The type of total_mem "unsigned long" is not suitable with nowadays >> memory ,it will overflow(with i386) in line 177:map_count = total_mem >> * >> KB / MAP_SIZE and the value of map_count is not correct. >> Using "unsigned long long" could fix this issue. > > after re-check the code, I found map_count was not necessary, so I > removed it, > at the same time, it fixed your the overflow issue. > > please review the patch again, if you agree the fix, I will send a > formal patch set in new subject: So, please, thank you. Wanlong Gao ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-29 8:29 ` Wanlong Gao @ 2012-08-29 8:56 ` Zhouping Liu 0 siblings, 0 replies; 16+ messages in thread From: Zhouping Liu @ 2012-08-29 8:56 UTC (permalink / raw) To: gaowanlong; +Cc: ltp-list ----- Original Message ----- > From: "Wanlong Gao" <gaowanlong@cn.fujitsu.com> > To: "ZhouPing Liu" <zliu@redhat.com> > Cc: "Shuang Qiu" <shuang.qiu@oracle.com>, ltp-list@lists.sourceforge.net > Sent: Wednesday, August 29, 2012 4:29:52 PM > Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > > > yes, you are right. > > I tried to fork several processes to hog memory, but I don't think > > it's > > a better way, so a good solution(I think) is included in below > > patch. > > > >> > >> And we can also patch the overflow issue which I mentioned before: > >> > >> The type of total_mem "unsigned long" is not suitable with > >> nowadays > >> memory ,it will overflow(with i386) in line 177:map_count = > >> total_mem > >> * > >> KB / MAP_SIZE and the value of map_count is not correct. > >> Using "unsigned long long" could fix this issue. > > > > after re-check the code, I found map_count was not necessary, so I > > removed it, > > at the same time, it fixed your the overflow issue. > > > > please review the patch again, if you agree the fix, I will send a > > formal patch set in new subject: > > > So, please, thank you. will do later, thanks for care :) -- Thanks, Zhouping ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-27 5:16 ` Zhouping Liu 2012-08-28 3:11 ` Shuang Qiu @ 2012-08-29 14:03 ` Zhouping Liu 2012-08-30 2:53 ` Shuang Qiu 1 sibling, 1 reply; 16+ messages in thread From: Zhouping Liu @ 2012-08-29 14:03 UTC (permalink / raw) To: shuang qiu; +Cc: ltp-list ----- Original Message ----- > From: "Zhouping Liu" <zliu@redhat.com> > To: "shuang qiu" <shuang.qiu@oracle.com> > Cc: ltp-list@lists.sourceforge.net > Sent: Monday, August 27, 2012 1:16:45 PM > Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > > > ----- Original Message ----- > > From: "Zhouping Liu" <zliu@redhat.com> > > To: "shuang qiu" <shuang.qiu@oracle.com> > > Cc: ltp-list@lists.sourceforge.net > > Sent: Friday, August 24, 2012 4:11:41 PM > > Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes > > lower > > > > > > > > ----- Original Message ----- > > > From: "shuang qiu" <shuang.qiu@oracle.com> > > > To: ltp-list@lists.sourceforge.net > > > Sent: Thursday, August 23, 2012 2:43:01 PM > > > Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower > > > > > > From: Shuang Qiu <shuang.qiu@oracle.com> > > > > > > The value of min_free_kbytes is set too high(half of mem_free) > > > during > > > testing min_free_kbytes tunable.It often cause the system to > > > become > > > out-of-memory.Using 20% of mem_free to avoid oom and system hang. > > > > Hi, shuang > > > > I met the issue that min_free_kbytes caused system hang before, and > > what you said made sense. > > For the issue that min_free_kbytes caused system hang, I prepared a > patch for it, > at the same time, in order to enlarge the case's coverage, I add a > new case(2x default_tune) > I have tested it in my box, it's good. > > Shuang, can you review and test the patch? > > diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c > b/testcases/kernel/mem/tunable/min_free_kbytes.c > index 00ead04..ded0e45 100644 > --- a/testcases/kernel/mem/tunable/min_free_kbytes.c > +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c > @@ -10,7 +10,8 @@ > * the current free memory with the tunable value repeatedly. > * > * a) default min_free_kbytes with all overcommit memory policy > - * b) half of mem_free with all overcommit memory policy > + * b) 2x default value with all overcommit memory policy > + * c) 10% of MemFree or %5 MemTotal with all overcommit memory Hi, Shuang after setting 10% of MemFree or 5% MemTotal, it still cause a system hang, which has 16Gb memory. so I adjusted the value, reduced the value to 5% of MemFree or 2% MemTotal, which I think is more reasonable. Thanks, Zhouping > policy > * > ******************************************************************** > * Copyright (C) 2012 Red Hat, Inc. > @@ -115,20 +116,27 @@ int main(int argc, char *argv[]) > static void test_tune(unsigned long overcommit_policy) > { > int status; > - int pid[2]; > + int pid[3]; > int ret, i; > - unsigned long tune, memfree; > + unsigned long tune, memfree, memtotal; > > set_sys_tune("overcommit_memory", overcommit_policy, 1); > > - for (i = 0; i < 2; i++) { > + for (i = 0; i < 3; i++) { > /* case1 */ > if (i == 0) > set_sys_tune("min_free_kbytes", default_tune, > 1); > /* case2 */ > - else { > + else if (i == 1) { > + set_sys_tune("min_free_kbytes", 2 * > default_tune, 1); > + /* case3 */ > + } else { > memfree = read_meminfo("MemFree:"); > - tune = memfree / 2; > + memtotal = read_meminfo("MemTotal:"); > + tune = memfree / 10; > + if (tune > (memtotal / 20)) > + tune = memtotal / 20; > + > set_sys_tune("min_free_kbytes", tune, 1); > } > > > > > > > but after tested the patch, I found it still make the system hang > > with the patch, so I don't think > > this patch can fix the issue completely, maybe there are other > > factors, or need lower tune value. > > always suppose it was because of the high tune value, it's more > > better to add an optional to control it. > > also I will investigate it deeper to look after the real reason. > > please comment if you have other ideas. > > > > Thanks, > > Zhouping > > > > > --- > > > testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +- > > > 1 files changed, 1 insertions(+), 1 deletions(-) > > > > > > diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c > > > b/testcases/kernel/mem/tunable/min_free_kbytes.c > > > index 00ead04..8a03f85 100644 > > > --- a/testcases/kernel/mem/tunable/min_free_kbytes.c > > > +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c > > > @@ -128,7 +128,7 @@ static void test_tune(unsigned long > > > overcommit_policy) > > > /* case2 */ > > > else { > > > memfree = read_meminfo("MemFree:"); > > > - tune = memfree / 2; > > > + tune = memfree / 5; > > > set_sys_tune("min_free_kbytes", tune, 1); > > > } > > > > > > -- > > > 1.7.7 > > > > > > > > > ------------------------------------------------------------------------------ > > > Live Security Virtual Conference > > > Exclusive live event will cover all the ways today's security and > > > threat landscape has changed and how IT managers can respond. > > > Discussions > > > will include endpoint security, mobile security and the latest in > > > malware > > > threats. > > > http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > > > _______________________________________________ > > > Ltp-list mailing list > > > Ltp-list@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/ltp-list > > > > > > > -- > Thanks, > Zhouping > ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower 2012-08-29 14:03 ` Zhouping Liu @ 2012-08-30 2:53 ` Shuang Qiu 0 siblings, 0 replies; 16+ messages in thread From: Shuang Qiu @ 2012-08-30 2:53 UTC (permalink / raw) To: Zhouping Liu; +Cc: ltp-list On 08/29/2012 10:03 PM, Zhouping Liu wrote: > > ----- Original Message ----- >> From: "Zhouping Liu" <zliu@redhat.com> >> To: "shuang qiu" <shuang.qiu@oracle.com> >> Cc: ltp-list@lists.sourceforge.net >> Sent: Monday, August 27, 2012 1:16:45 PM >> Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes lower >> >> >> >> ----- Original Message ----- >>> From: "Zhouping Liu" <zliu@redhat.com> >>> To: "shuang qiu" <shuang.qiu@oracle.com> >>> Cc: ltp-list@lists.sourceforge.net >>> Sent: Friday, August 24, 2012 4:11:41 PM >>> Subject: Re: [LTP] [PATCH] Set tunable value of min_free_kbytes >>> lower >>> >>> >>> >>> ----- Original Message ----- >>>> From: "shuang qiu" <shuang.qiu@oracle.com> >>>> To: ltp-list@lists.sourceforge.net >>>> Sent: Thursday, August 23, 2012 2:43:01 PM >>>> Subject: [LTP] [PATCH] Set tunable value of min_free_kbytes lower >>>> >>>> From: Shuang Qiu <shuang.qiu@oracle.com> >>>> >>>> The value of min_free_kbytes is set too high(half of mem_free) >>>> during >>>> testing min_free_kbytes tunable.It often cause the system to >>>> become >>>> out-of-memory.Using 20% of mem_free to avoid oom and system hang. >>> Hi, shuang >>> >>> I met the issue that min_free_kbytes caused system hang before, and >>> what you said made sense. >> For the issue that min_free_kbytes caused system hang, I prepared a >> patch for it, >> at the same time, in order to enlarge the case's coverage, I add a >> new case(2x default_tune) >> I have tested it in my box, it's good. >> >> Shuang, can you review and test the patch? >> >> diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c >> b/testcases/kernel/mem/tunable/min_free_kbytes.c >> index 00ead04..ded0e45 100644 >> --- a/testcases/kernel/mem/tunable/min_free_kbytes.c >> +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c >> @@ -10,7 +10,8 @@ >> * the current free memory with the tunable value repeatedly. >> * >> * a) default min_free_kbytes with all overcommit memory policy >> - * b) half of mem_free with all overcommit memory policy >> + * b) 2x default value with all overcommit memory policy >> + * c) 10% of MemFree or %5 MemTotal with all overcommit memory > Hi, Shuang > > after setting 10% of MemFree or 5% MemTotal, it still cause a system > hang, which has 16Gb memory. so I adjusted the value, reduced the value > to 5% of MemFree or 2% MemTotal, which I think is more reasonable. OK. Thanks. > > Thanks, > Zhouping > >> policy >> * >> ******************************************************************** >> * Copyright (C) 2012 Red Hat, Inc. >> @@ -115,20 +116,27 @@ int main(int argc, char *argv[]) >> static void test_tune(unsigned long overcommit_policy) >> { >> int status; >> - int pid[2]; >> + int pid[3]; >> int ret, i; >> - unsigned long tune, memfree; >> + unsigned long tune, memfree, memtotal; >> >> set_sys_tune("overcommit_memory", overcommit_policy, 1); >> >> - for (i = 0; i < 2; i++) { >> + for (i = 0; i < 3; i++) { >> /* case1 */ >> if (i == 0) >> set_sys_tune("min_free_kbytes", default_tune, >> 1); >> /* case2 */ >> - else { >> + else if (i == 1) { >> + set_sys_tune("min_free_kbytes", 2 * >> default_tune, 1); >> + /* case3 */ >> + } else { >> memfree = read_meminfo("MemFree:"); >> - tune = memfree / 2; >> + memtotal = read_meminfo("MemTotal:"); >> + tune = memfree / 10; >> + if (tune > (memtotal / 20)) >> + tune = memtotal / 20; >> + >> set_sys_tune("min_free_kbytes", tune, 1); >> } >> >> >> >>> but after tested the patch, I found it still make the system hang >>> with the patch, so I don't think >>> this patch can fix the issue completely, maybe there are other >>> factors, or need lower tune value. >>> always suppose it was because of the high tune value, it's more >>> better to add an optional to control it. >>> also I will investigate it deeper to look after the real reason. >>> please comment if you have other ideas. >>> >>> Thanks, >>> Zhouping >>> >>>> --- >>>> testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +- >>>> 1 files changed, 1 insertions(+), 1 deletions(-) >>>> >>>> diff --git a/testcases/kernel/mem/tunable/min_free_kbytes.c >>>> b/testcases/kernel/mem/tunable/min_free_kbytes.c >>>> index 00ead04..8a03f85 100644 >>>> --- a/testcases/kernel/mem/tunable/min_free_kbytes.c >>>> +++ b/testcases/kernel/mem/tunable/min_free_kbytes.c >>>> @@ -128,7 +128,7 @@ static void test_tune(unsigned long >>>> overcommit_policy) >>>> /* case2 */ >>>> else { >>>> memfree = read_meminfo("MemFree:"); >>>> - tune = memfree / 2; >>>> + tune = memfree / 5; >>>> set_sys_tune("min_free_kbytes", tune, 1); >>>> } >>>> >>>> -- >>>> 1.7.7 >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Live Security Virtual Conference >>>> Exclusive live event will cover all the ways today's security and >>>> threat landscape has changed and how IT managers can respond. >>>> Discussions >>>> will include endpoint security, mobile security and the latest in >>>> malware >>>> threats. >>>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>>> _______________________________________________ >>>> Ltp-list mailing list >>>> Ltp-list@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/ltp-list >>>> >> -- >> Thanks, >> Zhouping >> ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2012-08-30 2:53 UTC | newest] Thread overview: 16+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-08-23 6:43 [LTP] [PATCH] Set tunable value of min_free_kbytes lower shuang.qiu 2012-08-23 8:59 ` Zhouping Liu 2012-08-23 9:00 ` Wanlong Gao 2012-08-24 8:11 ` Zhouping Liu 2012-08-24 10:18 ` Shuang Qiu 2012-08-24 13:48 ` Zhouping Liu 2012-08-27 3:03 ` Shuang Qiu 2012-08-27 5:01 ` Zhouping Liu 2012-08-27 5:16 ` Zhouping Liu 2012-08-28 3:11 ` Shuang Qiu 2012-08-28 12:51 ` ZhouPing Liu 2012-08-29 8:08 ` Shuang Qiu 2012-08-29 8:29 ` Wanlong Gao 2012-08-29 8:56 ` Zhouping Liu 2012-08-29 14:03 ` Zhouping Liu 2012-08-30 2:53 ` Shuang Qiu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox