All of lore.kernel.org
 help / color / mirror / Atom feed
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
To: rientjes@google.com
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	akpm@linux-foundation.org, caiqian@redhat.com, hughd@google.com,
	kamezawa.hiroyu@jp.fujitsu.com, minchan.kim@gmail.com,
	oleg@redhat.com
Subject: Re: [PATCH 3/5] oom: oom-killer don't use proportion of system-ram internally
Date: Tue, 24 May 2011 11:07:57 +0900	[thread overview]
Message-ID: <4DDB12FD.2000208@jp.fujitsu.com> (raw)
In-Reply-To: <alpine.DEB.2.00.1105231522410.17840@chino.kir.corp.google.com>

(2011/05/24 7:28), David Rientjes wrote:
> On Fri, 20 May 2011, KOSAKI Motohiro wrote:
>
>> CAI Qian reported his kernel did hang-up if he ran fork intensive
>> workload and then invoke oom-killer.
>>
>> The problem is, current oom calculation uses 0-1000 normalized value
>> (The unit is a permillage of system-ram). Its low precision make
>> a lot of same oom score. IOW, in his case, all processes have smaller
>> oom score than 1 and internal calculation round it to 1.
>>
>> Thus oom-killer kill ineligible process. This regression is caused by
>> commit a63d83f427 (oom: badness heuristic rewrite).
>>
>> The solution is, the internal calculation just use number of pages
>> instead of permillage of system-ram. And convert it to permillage
>> value at displaying time.
>>
>> This patch doesn't change any ABI (included  /proc/<pid>/oom_score_adj)
>> even though current logic has a lot of my dislike thing.
>>
>
> Same response as when you initially proposed this patch:
> http://marc.info/?l=linux-kernel&m=130507086613317 -- you never replied to
> that.

I did replay. Why don't you read?
http://www.gossamer-threads.com/lists/linux/kernel/1378837#1378837

If you haven't understand the issue, you can apply following patch and
run it.


diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index b01fa64..f35909b 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -718,6 +718,9 @@ void out_of_memory(struct zonelist *zonelist, gfp_t gfp_mask,
  	 */
  	constraint = constrained_alloc(zonelist, gfp_mask, nodemask,
  						&totalpages);
+
+	totalpages *= 10;
+
  	mpol_mask = (constraint == CONSTRAINT_MEMORY_POLICY) ? nodemask : NULL;
  	check_panic_on_oom(constraint, gfp_mask, order, mpol_mask);



> The changelog doesn't accurately represent CAI Qian's problem; the issue
> is that root processes are given too large of a bonus in comparison to
> other threads that are using at most 1.9% of available memory.  That can
> be fixed, as I suggested by giving 1% bonus per 10% of memory used so that
> the process would have to be using 10% before it even receives a bonus.
>
> I already suggested an alternative patch to CAI Qian to greatly increase
> the granularity of the oom score from a range of 0-1000 to 0-10000 to
> differentiate between tasks within 0.01% of available memory (16MB on CAI
> Qian's 16GB system).  I'll propose this officially in a separate email.
>
> This patch also includes undocumented changes such as changing the bonus
> given to root processes.





WARNING: multiple messages have this Message-ID (diff)
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
To: rientjes@google.com
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	akpm@linux-foundation.org, caiqian@redhat.com, hughd@google.com,
	kamezawa.hiroyu@jp.fujitsu.com, minchan.kim@gmail.com,
	oleg@redhat.com
Subject: Re: [PATCH 3/5] oom: oom-killer don't use proportion of system-ram internally
Date: Tue, 24 May 2011 11:07:57 +0900	[thread overview]
Message-ID: <4DDB12FD.2000208@jp.fujitsu.com> (raw)
In-Reply-To: <alpine.DEB.2.00.1105231522410.17840@chino.kir.corp.google.com>

(2011/05/24 7:28), David Rientjes wrote:
> On Fri, 20 May 2011, KOSAKI Motohiro wrote:
>
>> CAI Qian reported his kernel did hang-up if he ran fork intensive
>> workload and then invoke oom-killer.
>>
>> The problem is, current oom calculation uses 0-1000 normalized value
>> (The unit is a permillage of system-ram). Its low precision make
>> a lot of same oom score. IOW, in his case, all processes have smaller
>> oom score than 1 and internal calculation round it to 1.
>>
>> Thus oom-killer kill ineligible process. This regression is caused by
>> commit a63d83f427 (oom: badness heuristic rewrite).
>>
>> The solution is, the internal calculation just use number of pages
>> instead of permillage of system-ram. And convert it to permillage
>> value at displaying time.
>>
>> This patch doesn't change any ABI (included  /proc/<pid>/oom_score_adj)
>> even though current logic has a lot of my dislike thing.
>>
>
> Same response as when you initially proposed this patch:
> http://marc.info/?l=linux-kernel&m=130507086613317 -- you never replied to
> that.

I did replay. Why don't you read?
http://www.gossamer-threads.com/lists/linux/kernel/1378837#1378837

If you haven't understand the issue, you can apply following patch and
run it.


diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index b01fa64..f35909b 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -718,6 +718,9 @@ void out_of_memory(struct zonelist *zonelist, gfp_t gfp_mask,
  	 */
  	constraint = constrained_alloc(zonelist, gfp_mask, nodemask,
  						&totalpages);
+
+	totalpages *= 10;
+
  	mpol_mask = (constraint == CONSTRAINT_MEMORY_POLICY) ? nodemask : NULL;
  	check_panic_on_oom(constraint, gfp_mask, order, mpol_mask);



> The changelog doesn't accurately represent CAI Qian's problem; the issue
> is that root processes are given too large of a bonus in comparison to
> other threads that are using at most 1.9% of available memory.  That can
> be fixed, as I suggested by giving 1% bonus per 10% of memory used so that
> the process would have to be using 10% before it even receives a bonus.
>
> I already suggested an alternative patch to CAI Qian to greatly increase
> the granularity of the oom score from a range of 0-1000 to 0-10000 to
> differentiate between tasks within 0.01% of available memory (16MB on CAI
> Qian's 16GB system).  I'll propose this officially in a separate email.
>
> This patch also includes undocumented changes such as changing the bonus
> given to root processes.




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2011-05-24  2:08 UTC|newest]

Thread overview: 118+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-05-20  8:00 [PATCH v2 0/5] Fix oom killer doesn't work at all if system have > gigabytes memory (aka CAI founded issue) KOSAKI Motohiro
2011-05-20  8:00 ` KOSAKI Motohiro
2011-05-20  8:01 ` [PATCH 1/5] oom: improve dump_tasks() show items KOSAKI Motohiro
2011-05-20  8:01   ` KOSAKI Motohiro
2011-05-23 22:16   ` David Rientjes
2011-05-23 22:16     ` David Rientjes
2011-05-20  8:02 ` [PATCH 2/5] oom: kill younger process first KOSAKI Motohiro
2011-05-20  8:02   ` KOSAKI Motohiro
2011-05-23  2:37   ` Minchan Kim
2011-05-23  2:37     ` Minchan Kim
2011-05-23 22:20   ` David Rientjes
2011-05-23 22:20     ` David Rientjes
2011-05-20  8:03 ` [PATCH 3/5] oom: oom-killer don't use proportion of system-ram internally KOSAKI Motohiro
2011-05-20  8:03   ` KOSAKI Motohiro
2011-05-23  3:59   ` Minchan Kim
2011-05-23  3:59     ` Minchan Kim
2011-05-24  1:14     ` KOSAKI Motohiro
2011-05-24  1:14       ` KOSAKI Motohiro
2011-05-24  1:32       ` Minchan Kim
2011-05-24  1:32         ` Minchan Kim
2011-05-23  4:02   ` Minchan Kim
2011-05-23  4:02     ` Minchan Kim
2011-05-24  1:44     ` KOSAKI Motohiro
2011-05-24  1:44       ` KOSAKI Motohiro
2011-05-24  3:11       ` KOSAKI Motohiro
2011-05-24  3:11         ` KOSAKI Motohiro
2011-05-23 22:28   ` David Rientjes
2011-05-23 22:28     ` David Rientjes
2011-05-23 22:48     ` David Rientjes
2011-05-23 22:48       ` David Rientjes
2011-05-24  1:21       ` KOSAKI Motohiro
2011-05-24  1:21         ` KOSAKI Motohiro
2011-05-24  8:32       ` CAI Qian
2011-05-24  8:32         ` CAI Qian
2011-05-26  7:08       ` CAI Qian
2011-05-26  7:08         ` CAI Qian
2011-05-27 19:12         ` David Rientjes
2011-05-27 19:12           ` David Rientjes
2011-05-24  2:07     ` KOSAKI Motohiro [this message]
2011-05-24  2:07       ` KOSAKI Motohiro
2011-05-26  9:34   ` CAI Qian
2011-05-26  9:34     ` CAI Qian
2011-05-26  9:56     ` KOSAKI Motohiro
2011-05-26  9:56       ` KOSAKI Motohiro
2011-05-20  8:04 ` [PATCH 4/5] oom: don't kill random process KOSAKI Motohiro
2011-05-20  8:04   ` KOSAKI Motohiro
2011-05-23  4:31   ` Minchan Kim
2011-05-23  4:31     ` Minchan Kim
2011-05-24  1:53     ` KOSAKI Motohiro
2011-05-24  1:53       ` KOSAKI Motohiro
2011-05-24  8:46       ` Minchan Kim
2011-05-24  8:46         ` Minchan Kim
2011-05-24  8:49         ` KOSAKI Motohiro
2011-05-24  8:49           ` KOSAKI Motohiro
2011-05-24  9:04           ` Minchan Kim
2011-05-24  9:04             ` Minchan Kim
2011-05-24  9:09             ` KOSAKI Motohiro
2011-05-24  9:09               ` KOSAKI Motohiro
2011-05-24  9:20               ` Minchan Kim
2011-05-24  9:20                 ` Minchan Kim
2011-05-24  9:38                 ` KOSAKI Motohiro
2011-05-24  9:38                   ` KOSAKI Motohiro
2011-05-23 22:32   ` David Rientjes
2011-05-23 22:32     ` David Rientjes
2011-05-24  1:35     ` KOSAKI Motohiro
2011-05-24  1:35       ` KOSAKI Motohiro
2011-05-24  1:39       ` David Rientjes
2011-05-24  1:39         ` David Rientjes
2011-05-24  1:55         ` KOSAKI Motohiro
2011-05-24  1:55           ` KOSAKI Motohiro
2011-05-24  1:58           ` David Rientjes
2011-05-24  1:58             ` David Rientjes
2011-05-24  2:03             ` KOSAKI Motohiro
2011-05-24  2:03               ` KOSAKI Motohiro
2011-05-25 23:50               ` David Rientjes
2011-05-25 23:50                 ` David Rientjes
2011-05-30  1:17                 ` KOSAKI Motohiro
2011-05-30  1:17                   ` KOSAKI Motohiro
2011-05-31  4:48                   ` David Rientjes
2011-05-31  4:48                     ` David Rientjes
2011-05-31  4:54                     ` KOSAKI Motohiro
2011-05-31  4:54                       ` KOSAKI Motohiro
2011-05-20  8:05 ` [PATCH 5/5] oom: merge oom_kill_process() with oom_kill_task() KOSAKI Motohiro
2011-05-20  8:05   ` KOSAKI Motohiro
2011-05-31  1:33 ` [PATCH v2 0/5] Fix oom killer doesn't work at all if system have > gigabytes memory (aka CAI founded issue) CAI Qian
2011-05-31  1:33   ` CAI Qian
2011-05-31  4:10   ` KOSAKI Motohiro
2011-05-31  4:10     ` KOSAKI Motohiro
2011-05-31  4:14     ` CAI Qian
2011-05-31  4:14       ` CAI Qian
2011-05-31  4:34       ` KOSAKI Motohiro
2011-05-31  4:34         ` KOSAKI Motohiro
2011-05-31  4:49       ` KOSAKI Motohiro
2011-05-31  4:49         ` KOSAKI Motohiro
2011-05-31  4:32     ` KOSAKI Motohiro
2011-05-31  4:32       ` KOSAKI Motohiro
2011-05-31  4:52     ` CAI Qian
2011-05-31  4:52       ` CAI Qian
2011-05-31  7:04       ` KOSAKI Motohiro
2011-05-31  7:04         ` KOSAKI Motohiro
2011-05-31  7:50         ` CAI Qian
2011-05-31  7:50           ` CAI Qian
2011-05-31  7:56           ` KOSAKI Motohiro
2011-05-31  7:56             ` KOSAKI Motohiro
2011-05-31  7:59             ` CAI Qian
2011-05-31  7:59               ` CAI Qian
2011-05-31  8:11               ` KOSAKI Motohiro
2011-05-31  8:11                 ` KOSAKI Motohiro
2011-05-31 10:01                 ` KOSAKI Motohiro
2011-05-31 10:01                   ` KOSAKI Motohiro
2011-06-01  1:17                   ` CAI Qian
2011-06-01  1:17                     ` CAI Qian
2011-06-01  3:32                   ` Minchan Kim
2011-06-01  3:32                     ` Minchan Kim
2011-06-06  3:07                     ` KOSAKI Motohiro
2011-06-06  3:07                       ` KOSAKI Motohiro
2011-06-06 14:44                       ` Minchan Kim
2011-06-06 14:44                         ` Minchan Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4DDB12FD.2000208@jp.fujitsu.com \
    --to=kosaki.motohiro@jp.fujitsu.com \
    --cc=akpm@linux-foundation.org \
    --cc=caiqian@redhat.com \
    --cc=hughd@google.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=minchan.kim@gmail.com \
    --cc=oleg@redhat.com \
    --cc=rientjes@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.