All of lore.kernel.org
 help / color / mirror / Atom feed
From: Manfred Spraul <manfred@colorfullife.com>
To: Michael Kerrisk <mtk.manpages@gmail.com>,
	Davidlohr Bueso <davidlohr@hp.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
	Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Greg Thelen <gthelen@google.com>,
	aswin@hp.com, LKML <linux-kernel@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default
Date: Thu, 17 Apr 2014 18:22:26 +0200	[thread overview]
Message-ID: <534FFFC2.6050601@colorfullife.com> (raw)
In-Reply-To: <CAHO5Pa3BOgJGCm7NvE4xbm3O1WbRLRBS0pgvErPudypP_iiZ3g@mail.gmail.com>

Hi Michael,

On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>> From: Davidlohr Bueso <davidlohr@hp.com>
>>
>> The default size for shmmax is, and always has been, 32Mb.
>> Today, in the XXI century, it seems that this value is rather small,
>> making users have to increase it via sysctl, which can cause
>> unnecessary work and userspace application workarounds[1].
>>
>> Instead of choosing yet another arbitrary value, larger than 32Mb,
>> this patch disables the use of both shmmax and shmall by default,
>> allowing users to create segments of unlimited sizes. Users and
>> applications that already explicitly set these values through sysctl
>> are left untouched, and thus does not change any of the behavior.
>>
>> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
>> implies unlimited memory, as opposed to disabling sysv shared memory.
>> This is safe as 0 cannot possibly be used previously as SHMMIN is
>> hardcoded to 1 and cannot be modified.
>>
>> This change allows Linux to treat shm just as regular anonymous memory.
>> One important difference between them, though, is handling out-of-memory
>> conditions: as opposed to regular anon memory, the OOM killer will not
>> free the memory as it is shm, allowing users to potentially abuse this.
>> To overcome this situation, the shm_rmid_forced option must be enabled.
>>
>> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
>>
>> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
>> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Of the two proposed approaches (the other being
> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
> me, since it allows strange users to maintain historical behavior
> (i.e., the ability to set a limit) if they really want it, so:
>
> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>
> One or two comments below, that you might consider for your v3 patch.
I don't understand what you mean.

After a
     # echo 33554432 > /proc/sys/kernel/shmmax
     # echo 2097152 > /proc/sys/kernel/shmmax

both patches behave exactly identical.

There are only two differences:
- Davidlohr's patch handles
     # echo <really huge number that doesn't fit into 64-bit> > 
/proc/sys/kernel/shmmax
    With my patch, shmmax would end up as 0 and all allocations fail.

- My patch handles the case if some startup code/installer checks
    shmmax and complains if it is below the requirement of the application.

--
     Manfred

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Manfred Spraul <manfred@colorfullife.com>
To: Michael Kerrisk <mtk.manpages@gmail.com>,
	Davidlohr Bueso <davidlohr@hp.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
	Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Greg Thelen <gthelen@google.com>,
	aswin@hp.com, LKML <linux-kernel@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default
Date: Thu, 17 Apr 2014 18:22:26 +0200	[thread overview]
Message-ID: <534FFFC2.6050601@colorfullife.com> (raw)
In-Reply-To: <CAHO5Pa3BOgJGCm7NvE4xbm3O1WbRLRBS0pgvErPudypP_iiZ3g@mail.gmail.com>

Hi Michael,

On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>> From: Davidlohr Bueso <davidlohr@hp.com>
>>
>> The default size for shmmax is, and always has been, 32Mb.
>> Today, in the XXI century, it seems that this value is rather small,
>> making users have to increase it via sysctl, which can cause
>> unnecessary work and userspace application workarounds[1].
>>
>> Instead of choosing yet another arbitrary value, larger than 32Mb,
>> this patch disables the use of both shmmax and shmall by default,
>> allowing users to create segments of unlimited sizes. Users and
>> applications that already explicitly set these values through sysctl
>> are left untouched, and thus does not change any of the behavior.
>>
>> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
>> implies unlimited memory, as opposed to disabling sysv shared memory.
>> This is safe as 0 cannot possibly be used previously as SHMMIN is
>> hardcoded to 1 and cannot be modified.
>>
>> This change allows Linux to treat shm just as regular anonymous memory.
>> One important difference between them, though, is handling out-of-memory
>> conditions: as opposed to regular anon memory, the OOM killer will not
>> free the memory as it is shm, allowing users to potentially abuse this.
>> To overcome this situation, the shm_rmid_forced option must be enabled.
>>
>> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
>>
>> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
>> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Of the two proposed approaches (the other being
> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
> me, since it allows strange users to maintain historical behavior
> (i.e., the ability to set a limit) if they really want it, so:
>
> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>
> One or two comments below, that you might consider for your v3 patch.
I don't understand what you mean.

After a
     # echo 33554432 > /proc/sys/kernel/shmmax
     # echo 2097152 > /proc/sys/kernel/shmmax

both patches behave exactly identical.

There are only two differences:
- Davidlohr's patch handles
     # echo <really huge number that doesn't fit into 64-bit> > 
/proc/sys/kernel/shmmax
    With my patch, shmmax would end up as 0 and all allocations fail.

- My patch handles the case if some startup code/installer checks
    shmmax and complains if it is below the requirement of the application.

--
     Manfred

  reply	other threads:[~2014-04-17 16:22 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-04-12  3:22 [PATCH v2] ipc,shm: disable shmmax and shmall by default Davidlohr Bueso
2014-04-12  3:22 ` Davidlohr Bueso
2014-04-17 10:53 ` Michael Kerrisk
2014-04-17 10:53   ` Michael Kerrisk
2014-04-17 16:22   ` Manfred Spraul [this message]
2014-04-17 16:22     ` Manfred Spraul
2014-04-17 20:23     ` Michael Kerrisk (man-pages)
2014-04-17 20:23       ` Michael Kerrisk (man-pages)
2014-04-17 22:31       ` Davidlohr Bueso
2014-04-17 22:31         ` Davidlohr Bueso
2014-04-18  5:28         ` Michael Kerrisk (man-pages)
2014-04-18  5:28           ` Michael Kerrisk (man-pages)
2014-04-18 16:29           ` Davidlohr Bueso
2014-04-18 16:29             ` Davidlohr Bueso
2014-04-19  6:22             ` Michael Kerrisk (man-pages)
2014-04-19  6:22               ` Michael Kerrisk (man-pages)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=534FFFC2.6050601@colorfullife.com \
    --to=manfred@colorfullife.com \
    --cc=akpm@linux-foundation.org \
    --cc=aswin@hp.com \
    --cc=davidlohr@hp.com \
    --cc=gthelen@google.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=kosaki.motohiro@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mtk.manpages@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.