All of lore.kernel.org
 help / color / mirror / Atom feed
From: Balbir Singh <balbir-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
To: KOSAKI Motohiro
	<kosaki.motohiro-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
Cc: Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	David.Singleton-FCV4sgi5zeUQrrorzV6ljw@public.gmane.org,
	YAMAMOTO Takashi
	<yamamoto-jCdQPDEk3idL9jVzuh4AOg@public.gmane.org>,
	Daisuke Nishimura
	<nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>,
	Linux MM <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>,
	Linux Containers
	<containers-qjLDD68F18O7TbgM5vRIOg@public.gmane.org>,
	Hugh Dickins <hugh-DTz5qymZ9yRBDgjK7y7TUQ@public.gmane.org>,
	Pavel Emelyanov <xemul-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH 0/4] swapcgroup(v2)
Date: Fri, 23 May 2008 12:15:15 +0530	[thread overview]
Message-ID: <483667FB.1030702@linux.vnet.ibm.com> (raw)
In-Reply-To: <20080523145947.84F4.KOSAKI.MOTOHIRO-+CUm20s59erQFUHtdCDX3A@public.gmane.org>

KOSAKI Motohiro wrote:
>>> Have you seen any real world example of this? 
>> At the unsophisticated end, there are lots of (Fortran) HPC applications
>> with very large static array declarations but only "use" a small fraction
>> of that.  Those users know they only need a small fraction and are happy
>> to volunteer small physical memory limits that we (admins/queuing
>> systems) can apply.
>>
>> At the sophisticated end, the use of numerous large memory maps in
>> parallel HPC applications to gain visibility into other processes is
>> growing.  We have processes with VSZ > 400GB just because they have
>> 4GB maps into 127 other processes.  Their physical page use is of
>> the order 2GB.
> 
> Ah, agreed.
> Fujitsu HPC user said similar things ago.

OK, so this use case is HPC specific. I am not against the swap controller, but
overcommit can lead to problems if not controlled - such as OOM kill. The
virtual address space limit helps applications fail gracefully rather than swap
out excessively or OOM.

I suspect there'll be applications that swing both ways.

-- 
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL

WARNING: multiple messages have this Message-ID (diff)
From: Balbir Singh <balbir@linux.vnet.ibm.com>
To: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: David.Singleton@anu.edu.au, Rik van Riel <riel@redhat.com>,
	YAMAMOTO Takashi <yamamoto@valinux.co.jp>,
	Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>,
	Linux MM <linux-mm@kvack.org>,
	Linux Containers <containers@lists.osdl.org>,
	Hugh Dickins <hugh@veritas.com>,
	Pavel Emelyanov <xemul@openvz.org>
Subject: Re: [PATCH 0/4] swapcgroup(v2)
Date: Fri, 23 May 2008 12:15:15 +0530	[thread overview]
Message-ID: <483667FB.1030702@linux.vnet.ibm.com> (raw)
In-Reply-To: <20080523145947.84F4.KOSAKI.MOTOHIRO@jp.fujitsu.com>

KOSAKI Motohiro wrote:
>>> Have you seen any real world example of this? 
>> At the unsophisticated end, there are lots of (Fortran) HPC applications
>> with very large static array declarations but only "use" a small fraction
>> of that.  Those users know they only need a small fraction and are happy
>> to volunteer small physical memory limits that we (admins/queuing
>> systems) can apply.
>>
>> At the sophisticated end, the use of numerous large memory maps in
>> parallel HPC applications to gain visibility into other processes is
>> growing.  We have processes with VSZ > 400GB just because they have
>> 4GB maps into 127 other processes.  Their physical page use is of
>> the order 2GB.
> 
> Ah, agreed.
> Fujitsu HPC user said similar things ago.

OK, so this use case is HPC specific. I am not against the swap controller, but
overcommit can lead to problems if not controlled - such as OOM kill. The
virtual address space limit helps applications fail gracefully rather than swap
out excessively or OOM.

I suspect there'll be applications that swing both ways.

-- 
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2008-05-23  6:45 UTC|newest]

Thread overview: 84+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-05-22  6:13 [PATCH 0/4] swapcgroup(v2) Daisuke Nishimura
2008-05-22  6:13 ` Daisuke Nishimura
     [not found] ` <48350F15.9070007-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-22  6:17   ` [PATCH 1/4] swapcgroup: add cgroup files Daisuke Nishimura
2008-05-22  6:17     ` Daisuke Nishimura
2008-05-22  6:18   ` [PATCH 2/4] swapcgroup: add member to swap_info_struct for cgroup Daisuke Nishimura
2008-05-22  6:18     ` Daisuke Nishimura
     [not found]     ` <4835104B.4040405-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-22  7:23       ` KAMEZAWA Hiroyuki
2008-05-22  7:23         ` KAMEZAWA Hiroyuki
     [not found]         ` <20080522162312.a60d914b.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2008-05-22  8:46           ` Daisuke Nishimura
2008-05-22  8:46             ` Daisuke Nishimura
     [not found]             ` <483532FE.9080707-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-22  9:35               ` KAMEZAWA Hiroyuki
2008-05-22  9:35                 ` KAMEZAWA Hiroyuki
2008-05-22  6:20   ` [PATCH 3/4] swapcgroup: implement charge/uncharge Daisuke Nishimura
2008-05-22  6:20     ` Daisuke Nishimura
     [not found]     ` <48351095.3040009-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-22  7:37       ` KAMEZAWA Hiroyuki
2008-05-22  7:37         ` KAMEZAWA Hiroyuki
     [not found]         ` <20080522163748.74e9bd4f.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2008-05-23 11:52           ` Daisuke Nishimura
2008-05-23 11:52             ` Daisuke Nishimura
     [not found]             ` <4836AFFD.3060605-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-26  0:57               ` KAMEZAWA Hiroyuki
2008-05-26  0:57                 ` KAMEZAWA Hiroyuki
     [not found]                 ` <20080526095706.c90a0afb.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2008-05-27 13:42                   ` KAMEZAWA Hiroyuki
2008-05-27 13:42                     ` KAMEZAWA Hiroyuki
2008-05-22  6:22   ` [PATCH 4/4] swapcgroup: modify vm_swap_full for cgroup Daisuke Nishimura
2008-05-22  6:22     ` Daisuke Nishimura
     [not found]     ` <48351120.6000800-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-22  6:45       ` YAMAMOTO Takashi
2008-05-22  6:45         ` YAMAMOTO Takashi
     [not found]         ` <20080522064507.AB6A35A0A-Pcsii4f/SVk@public.gmane.org>
2008-05-22 12:34           ` Daisuke Nishimura
2008-05-22 12:34             ` Daisuke Nishimura
     [not found]             ` <4835686A.9000106-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-25 23:35               ` YAMAMOTO Takashi
2008-05-25 23:35                 ` YAMAMOTO Takashi
2008-05-22  7:39       ` KAMEZAWA Hiroyuki
2008-05-22  7:39         ` KAMEZAWA Hiroyuki
2008-05-22  8:00       ` KOSAKI Motohiro
2008-05-22  8:00         ` KOSAKI Motohiro
     [not found]         ` <20080522165322.F516.KOSAKI.MOTOHIRO-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2008-05-22 12:22           ` Daisuke Nishimura
2008-05-22 12:22             ` Daisuke Nishimura
     [not found]             ` <4835656D.4020706-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-22 12:32               ` KOSAKI Motohiro
2008-05-22 12:32                 ` KOSAKI Motohiro
     [not found]                 ` <2f11576a0805220532l668ca59emd37afb60f50b703-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2008-05-23 12:26                   ` Daisuke Nishimura
2008-05-23 12:26                     ` Daisuke Nishimura
2008-05-22  7:44   ` [PATCH 0/4] swapcgroup(v2) KAMEZAWA Hiroyuki
2008-05-22  7:44     ` KAMEZAWA Hiroyuki
     [not found]     ` <20080522164421.84849565.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2008-05-23  2:10       ` Daisuke Nishimura
2008-05-23  2:10         ` Daisuke Nishimura
     [not found]         ` <48362795.9020709-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-23  2:42           ` Daisuke Nishimura
2008-05-23  2:42             ` Daisuke Nishimura
2008-05-22 21:27   ` Balbir Singh
2008-05-22 21:27     ` Balbir Singh
     [not found]     ` <4835E55A.1000308-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2008-05-23  4:27       ` Daisuke Nishimura
2008-05-23  4:27         ` Daisuke Nishimura
     [not found]         ` <483647AB.8090104-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-27  7:31           ` YAMAMOTO Takashi
2008-05-27  7:31             ` YAMAMOTO Takashi
     [not found]             ` <20080527073118.0D92B5A0E-Pcsii4f/SVk@public.gmane.org>
2008-05-27  7:42               ` Balbir Singh
2008-05-27  7:42                 ` Balbir Singh
     [not found]                 ` <483BBB4C.3040501-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2008-05-27  8:30                   ` Daisuke Nishimura
2008-05-27  8:30                     ` Daisuke Nishimura
     [not found]                     ` <483BC690.6010206-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-27 13:18                       ` Balbir Singh
2008-05-27 13:18                         ` Balbir Singh
     [not found]                         ` <483C0A0D.50909-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2008-05-27 13:42                           ` Daisuke Nishimura
2008-05-27 13:42                             ` Daisuke Nishimura
     [not found]                             ` <483C0FB2.7080706-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2008-05-27 13:46                               ` Balbir Singh
2008-05-27 13:46                                 ` Balbir Singh
     [not found]                                 ` <483C10C0.7040503-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2008-05-27 14:00                                   ` Daisuke Nishimura
2008-05-27 14:00                                     ` Daisuke Nishimura
2008-05-23  2:26   ` Rik van Riel
2008-05-23  2:26     ` Rik van Riel
     [not found]     ` <20080522222655.166657da-Fuq27k0DHcCSkoNiqTzCLQ@public.gmane.org>
2008-05-23  3:10       ` KAMEZAWA Hiroyuki
2008-05-23  3:10         ` KAMEZAWA Hiroyuki
     [not found]         ` <20080523121027.b0eecfa0.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2008-05-23  3:32           ` Rik van Riel
2008-05-23  3:32             ` Rik van Riel
2008-05-23  3:59           ` Balbir Singh
2008-05-23  3:59             ` Balbir Singh
     [not found]             ` <4836411B.2030601-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2008-05-23  4:30               ` KOSAKI Motohiro
2008-05-23  4:30                 ` KOSAKI Motohiro
     [not found]                 ` <20080523131812.84F1.KOSAKI.MOTOHIRO-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2008-05-23  4:51                   ` Balbir Singh
2008-05-23  4:51                     ` Balbir Singh
     [not found]                     ` <48364D38.7000304-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
2008-05-23  5:23                       ` KAMEZAWA Hiroyuki
2008-05-23  5:23                         ` KAMEZAWA Hiroyuki
2008-05-23  5:29                       ` David Singleton
2008-05-23  5:29                         ` David Singleton
     [not found]                         ` <4836563B.4060603-FCV4sgi5zeUQrrorzV6ljw@public.gmane.org>
2008-05-23  6:00                           ` KOSAKI Motohiro
2008-05-23  6:00                             ` KOSAKI Motohiro
     [not found]                             ` <20080523145947.84F4.KOSAKI.MOTOHIRO-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2008-05-23  6:45                               ` Balbir Singh [this message]
2008-05-23  6:45                                 ` Balbir Singh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=483667FB.1030702@linux.vnet.ibm.com \
    --to=balbir-23vcf4htsmix0ybbhkvfkdbpr1lh4cv8@public.gmane.org \
    --cc=David.Singleton-FCV4sgi5zeUQrrorzV6ljw@public.gmane.org \
    --cc=containers-qjLDD68F18O7TbgM5vRIOg@public.gmane.org \
    --cc=hugh-DTz5qymZ9yRBDgjK7y7TUQ@public.gmane.org \
    --cc=kosaki.motohiro-+CUm20s59erQFUHtdCDX3A@public.gmane.org \
    --cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
    --cc=nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org \
    --cc=riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
    --cc=xemul-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org \
    --cc=yamamoto-jCdQPDEk3idL9jVzuh4AOg@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.