From mboxrd@z Thu Jan 1 00:00:00 1970 From: Minchan Kim Subject: Re: MADV_DONTNEED semantics? Was: [RFC PATCH] mm: madvise: Ignore repeated MADV_DONTNEED hints Date: Mon, 9 Feb 2015 15:50:45 +0900 Message-ID: <20150209065045.GB32300@blaptop> References: <54D08483.40209@suse.cz> <20150203105301.GC14259@node.dhcp.inet.fi> <54D0B43D.8000209@suse.cz> <54D0F56A.9050003@gmail.com> <54D22298.3040504@suse.cz> <54D2508A.9030804@suse.cz> <20150205154102.GA20607@dhcp22.suse.cz> <54D4E47E.4020509@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: Content-Disposition: inline In-Reply-To: <54D4E47E.4020509-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> Sender: linux-man-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: "Michael Kerrisk (man-pages)" Cc: Michal Hocko , Vlastimil Babka , "Kirill A. Shutemov" , Dave Hansen , Mel Gorman , "linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org" , Andrew Morton , lkml , Linux API , linux-man , Hugh Dickins List-Id: linux-api@vger.kernel.org On Fri, Feb 06, 2015 at 04:57:50PM +0100, Michael Kerrisk (man-pages) w= rote: > Hi Michael >=20 > On 02/05/2015 04:41 PM, Michal Hocko wrote: > > On Wed 04-02-15 20:24:27, Michael Kerrisk wrote: > > [...] > >> So, how about this text: > >> > >> After a successful MADV_DONTNEED operation, the sema= n=E2=80=90 > >> tics of memory access in the specified region a= re > >> changed: subsequent accesses of pages in the ran= ge > >> will succeed, but will result in either reloading = of > >> the memory contents from the underlying mapped fi= le > >=20 > > " > > result in either providing the up-to-date contents of the underlyin= g > > mapped file > > " >=20 > Thanks! I did something like that. See below. >=20 > > Would be more precise IMO because reload might be interpreted as a = major > > fault which is not necessarily the case (see below). > >=20 > >> (for shared file mappings, shared anonymous mapping= s, > >> and shmem-based techniques such as System V shar= ed > >> memory segments) or zero-fill-on-demand pages f= or > >> anonymous private mappings. > >=20 > > Yes, this wording is better because many users are not aware of > > MAP_ANON|MAP_SHARED being file backed in fact and mmap man page doe= sn't > > mention that. >=20 > (Michal, would you have a text to propose to add to the mmap(2) page? > Maybe it would be useful to add something there.) >=20 > >=20 > > I am just wondering whether it makes sense to mention that MADV_DON= TNEED > > for shared mappings might be surprising and not freeing the backing > > pages thus not really freeing memory until there is a memory > > pressure. But maybe this is too implementation specific for a man > > page. What about the following wording on top of yours? > > " > > Please note that the MADV_DONTNEED hint on shared mappings might no= t > > lead to immediate freeing of pages in the range. The kernel is free= to > > delay this until an appropriate moment. RSS of the calling process = will > > be reduced however. > > " >=20 > Thanks! I added this, but dropped in the word "immediately" in the la= st=20 > sentence, since I assume that was implied. So now we have: >=20 > After a successful MADV_DONTNEED operation, the seman= =E2=80=90 > tics of memory access in the specified region ar= e > changed: subsequent accesses of pages in the range wil= l > succeed, but will result in either repopulating the mem= =E2=80=90 > ory contents from the up-to-date contents of the under= =E2=80=90 > lying mapped file (for shared file mappings, share= d > anonymous mappings, and shmem-based techniques such a= s > System V shared memory segments) or zero-fill-on-deman= d > pages for anonymous private mappings. >=20 > Note that, when applied to shared mappings, MADV_DONT= =E2=80=90 > NEED might not lead to immediate freeing of the pages i= n > the range. The kernel is free to delay freeing th= e > pages until an appropriate moment. The resident se= t > size (RSS) of the calling process will be immediatel= y > reduced however. Looks good. So, I can parse it that anonymous private mappings will lea= d to immediate freeing of the pages in the range so it's clearly differen= t with MADV_FREE. >=20 > The current draft of the page can be found in a branch, > http://git.kernel.org/cgit/docs/man-pages/man-pages.git/log/?h=3Ddraf= t_madvise >=20 > Thanks, >=20 > Michael >=20 >=20 >=20 > --=20 > Michael Kerrisk > Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ > Linux/UNIX System Programming Training: http://man7.org/training/ --=20 Kind regards, Minchan Kim -- To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by kanga.kvack.org (Postfix) with ESMTP id 19E486B0032 for ; Mon, 9 Feb 2015 01:50:57 -0500 (EST) Received: by mail-pa0-f54.google.com with SMTP id kx10so16647515pab.13 for ; Sun, 08 Feb 2015 22:50:56 -0800 (PST) Received: from mail-pd0-f173.google.com (mail-pd0-f173.google.com. [209.85.192.173]) by mx.google.com with ESMTPS id zt10si20895927pbc.18.2015.02.08.22.50.56 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 08 Feb 2015 22:50:56 -0800 (PST) Received: by pdjz10 with SMTP id z10so11693138pdj.9 for ; Sun, 08 Feb 2015 22:50:56 -0800 (PST) Date: Mon, 9 Feb 2015 15:50:45 +0900 From: Minchan Kim Subject: Re: MADV_DONTNEED semantics? Was: [RFC PATCH] mm: madvise: Ignore repeated MADV_DONTNEED hints Message-ID: <20150209065045.GB32300@blaptop> References: <54D08483.40209@suse.cz> <20150203105301.GC14259@node.dhcp.inet.fi> <54D0B43D.8000209@suse.cz> <54D0F56A.9050003@gmail.com> <54D22298.3040504@suse.cz> <54D2508A.9030804@suse.cz> <20150205154102.GA20607@dhcp22.suse.cz> <54D4E47E.4020509@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <54D4E47E.4020509@gmail.com> Sender: owner-linux-mm@kvack.org List-ID: To: "Michael Kerrisk (man-pages)" Cc: Michal Hocko , Vlastimil Babka , "Kirill A. Shutemov" , Dave Hansen , Mel Gorman , "linux-mm@kvack.org" , Andrew Morton , lkml , Linux API , linux-man , Hugh Dickins On Fri, Feb 06, 2015 at 04:57:50PM +0100, Michael Kerrisk (man-pages) wrote: > Hi Michael > > On 02/05/2015 04:41 PM, Michal Hocko wrote: > > On Wed 04-02-15 20:24:27, Michael Kerrisk wrote: > > [...] > >> So, how about this text: > >> > >> After a successful MADV_DONTNEED operation, the semana?? > >> tics of memory access in the specified region are > >> changed: subsequent accesses of pages in the range > >> will succeed, but will result in either reloading of > >> the memory contents from the underlying mapped file > > > > " > > result in either providing the up-to-date contents of the underlying > > mapped file > > " > > Thanks! I did something like that. See below. > > > Would be more precise IMO because reload might be interpreted as a major > > fault which is not necessarily the case (see below). > > > >> (for shared file mappings, shared anonymous mappings, > >> and shmem-based techniques such as System V shared > >> memory segments) or zero-fill-on-demand pages for > >> anonymous private mappings. > > > > Yes, this wording is better because many users are not aware of > > MAP_ANON|MAP_SHARED being file backed in fact and mmap man page doesn't > > mention that. > > (Michal, would you have a text to propose to add to the mmap(2) page? > Maybe it would be useful to add something there.) > > > > > I am just wondering whether it makes sense to mention that MADV_DONTNEED > > for shared mappings might be surprising and not freeing the backing > > pages thus not really freeing memory until there is a memory > > pressure. But maybe this is too implementation specific for a man > > page. What about the following wording on top of yours? > > " > > Please note that the MADV_DONTNEED hint on shared mappings might not > > lead to immediate freeing of pages in the range. The kernel is free to > > delay this until an appropriate moment. RSS of the calling process will > > be reduced however. > > " > > Thanks! I added this, but dropped in the word "immediately" in the last > sentence, since I assume that was implied. So now we have: > > After a successful MADV_DONTNEED operation, the semana?? > tics of memory access in the specified region are > changed: subsequent accesses of pages in the range will > succeed, but will result in either repopulating the mema?? > ory contents from the up-to-date contents of the undera?? > lying mapped file (for shared file mappings, shared > anonymous mappings, and shmem-based techniques such as > System V shared memory segments) or zero-fill-on-demand > pages for anonymous private mappings. > > Note that, when applied to shared mappings, MADV_DONTa?? > NEED might not lead to immediate freeing of the pages in > the range. The kernel is free to delay freeing the > pages until an appropriate moment. The resident set > size (RSS) of the calling process will be immediately > reduced however. Looks good. So, I can parse it that anonymous private mappings will lead to immediate freeing of the pages in the range so it's clearly different with MADV_FREE. > > The current draft of the page can be found in a branch, > http://git.kernel.org/cgit/docs/man-pages/man-pages.git/log/?h=draft_madvise > > Thanks, > > Michael > > > > -- > Michael Kerrisk > Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ > Linux/UNIX System Programming Training: http://man7.org/training/ -- Kind regards, Minchan Kim -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759440AbbBIGu7 (ORCPT ); Mon, 9 Feb 2015 01:50:59 -0500 Received: from mail-pd0-f174.google.com ([209.85.192.174]:36053 "EHLO mail-pd0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754175AbbBIGu4 (ORCPT ); Mon, 9 Feb 2015 01:50:56 -0500 Date: Mon, 9 Feb 2015 15:50:45 +0900 From: Minchan Kim To: "Michael Kerrisk (man-pages)" Cc: Michal Hocko , Vlastimil Babka , "Kirill A. Shutemov" , Dave Hansen , Mel Gorman , "linux-mm@kvack.org" , Andrew Morton , lkml , Linux API , linux-man , Hugh Dickins Subject: Re: MADV_DONTNEED semantics? Was: [RFC PATCH] mm: madvise: Ignore repeated MADV_DONTNEED hints Message-ID: <20150209065045.GB32300@blaptop> References: <54D08483.40209@suse.cz> <20150203105301.GC14259@node.dhcp.inet.fi> <54D0B43D.8000209@suse.cz> <54D0F56A.9050003@gmail.com> <54D22298.3040504@suse.cz> <54D2508A.9030804@suse.cz> <20150205154102.GA20607@dhcp22.suse.cz> <54D4E47E.4020509@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <54D4E47E.4020509@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Feb 06, 2015 at 04:57:50PM +0100, Michael Kerrisk (man-pages) wrote: > Hi Michael > > On 02/05/2015 04:41 PM, Michal Hocko wrote: > > On Wed 04-02-15 20:24:27, Michael Kerrisk wrote: > > [...] > >> So, how about this text: > >> > >> After a successful MADV_DONTNEED operation, the seman‐ > >> tics of memory access in the specified region are > >> changed: subsequent accesses of pages in the range > >> will succeed, but will result in either reloading of > >> the memory contents from the underlying mapped file > > > > " > > result in either providing the up-to-date contents of the underlying > > mapped file > > " > > Thanks! I did something like that. See below. > > > Would be more precise IMO because reload might be interpreted as a major > > fault which is not necessarily the case (see below). > > > >> (for shared file mappings, shared anonymous mappings, > >> and shmem-based techniques such as System V shared > >> memory segments) or zero-fill-on-demand pages for > >> anonymous private mappings. > > > > Yes, this wording is better because many users are not aware of > > MAP_ANON|MAP_SHARED being file backed in fact and mmap man page doesn't > > mention that. > > (Michal, would you have a text to propose to add to the mmap(2) page? > Maybe it would be useful to add something there.) > > > > > I am just wondering whether it makes sense to mention that MADV_DONTNEED > > for shared mappings might be surprising and not freeing the backing > > pages thus not really freeing memory until there is a memory > > pressure. But maybe this is too implementation specific for a man > > page. What about the following wording on top of yours? > > " > > Please note that the MADV_DONTNEED hint on shared mappings might not > > lead to immediate freeing of pages in the range. The kernel is free to > > delay this until an appropriate moment. RSS of the calling process will > > be reduced however. > > " > > Thanks! I added this, but dropped in the word "immediately" in the last > sentence, since I assume that was implied. So now we have: > > After a successful MADV_DONTNEED operation, the seman‐ > tics of memory access in the specified region are > changed: subsequent accesses of pages in the range will > succeed, but will result in either repopulating the mem‐ > ory contents from the up-to-date contents of the under‐ > lying mapped file (for shared file mappings, shared > anonymous mappings, and shmem-based techniques such as > System V shared memory segments) or zero-fill-on-demand > pages for anonymous private mappings. > > Note that, when applied to shared mappings, MADV_DONT‐ > NEED might not lead to immediate freeing of the pages in > the range. The kernel is free to delay freeing the > pages until an appropriate moment. The resident set > size (RSS) of the calling process will be immediately > reduced however. Looks good. So, I can parse it that anonymous private mappings will lead to immediate freeing of the pages in the range so it's clearly different with MADV_FREE. > > The current draft of the page can be found in a branch, > http://git.kernel.org/cgit/docs/man-pages/man-pages.git/log/?h=draft_madvise > > Thanks, > > Michael > > > > -- > Michael Kerrisk > Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ > Linux/UNIX System Programming Training: http://man7.org/training/ -- Kind regards, Minchan Kim