man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

[Michael Kerrisk]

Hello Hugh, Izik

For the MADV_MERGEABLE + MADV_UNMERGEABLE changes added in 2.6.32,
I've written the following man-pages text. Could you please
review/fix/ACK.

Thanks to Andi and Andrew for pointers that this documentation was needed.

Thanks,

Michael


--- a/man2/madvise.2
+++ b/man2/madvise.2
@@ -156,6 +157,42 @@ and the page being unmapped.
 This feature is intended for memory testing.
 This feature is only available if the kernel was configured with
 .BR CONFIG_MEMORY_FAILURE .
+.TP
+.BR MADV_MERGEABLE " (since Linux 2.6.32)"
+Enable Kernel Samepage Merging (KSM) for the pages in the range specified by
+.I addr
+and
+.IR len .
+The KSM daemon
+.RI ( ksmd )
+periodically scans those areas of user memory that have
+been marked as mergeable,
+looking for pages with identical content.
+These are replaced by a single write-protected page (which is automatically
+copied if a process later wants to update the content of the page).
+KSM only merges private anonymous pages (see
+.BR mmap (2)).
+The KSM feature is intended for applications that generate many
+instances of the same data (e.g., virtualization systems such as KVM).
+It can consume a lot of processing power; use with care.
+See the kernel source file
+.I Documentation/vm/ksm.txt
+for more details.
+The
+.BR MADV_MERGEABLE
+and
+.BR MADV_UNMERGEABLE
+operations are only available if the kernel was configured with
+.BR CONFIG_KSM.
+.TP
+.BR MADV_UNMERGEABLE " (since Linux 2.6.32)"
+Undo the effect of an earlier
+.BR MADV_MERGEABLE
+operation on the specified address range;
+KSM unmerges whatever pages it had merged in the address rnage specified by
+.IR addr
+and
+.IR length .
 .SH "RETURN VALUE"
 On success
 .BR madvise ()
@@ -189,6 +226,14 @@ is not a valid value
 .IP *
 The application is attempting to release locked or shared pages (with
 .BR MADV_DONTNEED ).
+.IP *
+.BR MADV_MERGEABLE
+or
+.BR MADV_UNMERGEABLE
+was specified in
+.IR advice ,
+but the kernel was not configured with
+.BR CONFIG_KSM .
 .RE
 .TP
 .B EIO
@@ -221,8 +266,10 @@ for file access.
 .BR MADV_REMOVE ,
 .BR MADV_DONTFORK ,
 .BR MADV_DOFORK ,
+.BR MAD_HWPOISON ,
+.BR MADVISE_MERGEABLE ,
 and
-.BR MAD_HWPOISON
+.BR MADVISE_UNMERGEABLE
 are Linux-specific.
 .SH NOTES
 .SS "Linux Notes"

Re: man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

[Andi Kleen]

> +.IR len .
> +The KSM daemon
> +.RI ( ksmd )

You should probably make clear it's a kernel thread, not a classic
daemon.

> +periodically scans those areas of user memory that have
> +been marked as mergeable,
> +looking for pages with identical content.

Overall it seems like an internal implementation detail.

-Andi
-- 
ak@linux.intel.com -- Speaking for myself only.

Re: man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

[Michael Kerrisk]

Hi Andi,

On Sat, Jun 19, 2010 at 10:58 AM, Andi Kleen <andi@firstfloor.org> wrote:
>> +.IR len .
>> +The KSM daemon
>> +.RI ( ksmd )
>
> You should probably make clear it's a kernel thread, not a classic
> daemon.

Thanks. I changed it to "KSM kernel thread"

>> +periodically scans those areas of user memory that have
>> +been marked as mergeable,
>> +looking for pages with identical content.
>
> Overall it seems like an internal implementation detail.

Not sure how I could improve this. Did you have some specific idea in mind?

Thanks,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface" http://blog.man7.org/

Re: man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

[Andi Kleen]

On Sat, Jun 19, 2010 at 02:22:33PM +0200, Michael Kerrisk wrote:
> Hi Andi,
> 
> On Sat, Jun 19, 2010 at 10:58 AM, Andi Kleen <andi@firstfloor.org> wrote:
> >> +.IR len .
> >> +The KSM daemon
> >> +.RI ( ksmd )
> >
> > You should probably make clear it's a kernel thread, not a classic
> > daemon.
> 
> Thanks. I changed it to "KSM kernel thread"
> 
> >> +periodically scans those areas of user memory that have
> >> +been marked as mergeable,
> >> +looking for pages with identical content.
> >
> > Overall it seems like an internal implementation detail.
> 
> Not sure how I could improve this. Did you have some specific idea in mind?

Just dropping the sentence or using soft language like "the kernel regularly
attempts to merge pages" 

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

[Michael Kerrisk]

Hi Andi,

On Sat, Jun 19, 2010 at 2:29 PM, Andi Kleen <andi@firstfloor.org> wrote:
> On Sat, Jun 19, 2010 at 02:22:33PM +0200, Michael Kerrisk wrote:
>> Hi Andi,
>>
>> On Sat, Jun 19, 2010 at 10:58 AM, Andi Kleen <andi@firstfloor.org> wrote:
>> >> +.IR len .
>> >> +The KSM daemon
>> >> +.RI ( ksmd )
>> >
>> > You should probably make clear it's a kernel thread, not a classic
>> > daemon.
>>
>> Thanks. I changed it to "KSM kernel thread"
>>
>> >> +periodically scans those areas of user memory that have
>> >> +been marked as mergeable,
>> >> +looking for pages with identical content.
>> >
>> > Overall it seems like an internal implementation detail.
>>
>> Not sure how I could improve this. Did you have some specific idea in mind?
>
> Just dropping the sentence or using soft language like "the kernel regularly
> attempts to merge pages"

Thanks. I reworded like as you suggest.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface" http://blog.man7.org/

Re: man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

[Hugh Dickins]

Hi Michael,

On Sat, 19 Jun 2010, Michael Kerrisk wrote:

> Hello Hugh, Izik
> 
> For the MADV_MERGEABLE + MADV_UNMERGEABLE changes added in 2.6.32,
> I've written the following man-pages text. Could you please
> review/fix/ACK.

Ah, good, thanks for writing this (but I recognize many of the words ;))

> 
> Thanks to Andi and Andrew for pointers that this documentation was needed.

(We did Cc you when ksm.txt went into Andrew's tree, and when it went
on into Linus's tree: is there something more we should have done?)

> 
> Thanks,
> 
> Michael
> 
> 
> --- a/man2/madvise.2
> +++ b/man2/madvise.2
> @@ -156,6 +157,42 @@ and the page being unmapped.
>  This feature is intended for memory testing.
>  This feature is only available if the kernel was configured with
>  .BR CONFIG_MEMORY_FAILURE .
> +.TP
> +.BR MADV_MERGEABLE " (since Linux 2.6.32)"
> +Enable Kernel Samepage Merging (KSM) for the pages in the range specified by
> +.I addr
> +and
> +.IR len .
> +The KSM daemon
> +.RI ( ksmd )
> +periodically scans those areas of user memory that have
> +been marked as mergeable,
> +looking for pages with identical content.

I was rather puzzled by Andi calling that an internal implementation
detail.  Okay, I suppose the "KSM daemon periodically scans" is an
internal implementation detail, though I think it helps to warn the
programmer of the overhead of MADV_MERGEABLE; but without "pages
with identical content", aren't we left mystified as to just what
this "merging" is all about?

> +These are replaced by a single write-protected page (which is automatically
> +copied if a process later wants to update the content of the page).
> +KSM only merges private anonymous pages (see
> +.BR mmap (2)).
> +The KSM feature is intended for applications that generate many
> +instances of the same data (e.g., virtualization systems such as KVM).
> +It can consume a lot of processing power; use with care.

                                           :

> +See the kernel source file
> +.I Documentation/vm/ksm.txt
> +for more details.
> +The
> +.BR MADV_MERGEABLE
> +and
> +.BR MADV_UNMERGEABLE
> +operations are only available if the kernel was configured with
> +.BR CONFIG_KSM.
> +.TP
> +.BR MADV_UNMERGEABLE " (since Linux 2.6.32)"
> +Undo the effect of an earlier
> +.BR MADV_MERGEABLE
> +operation on the specified address range;
> +KSM unmerges whatever pages it had merged in the address rnage specified by

                                                            range

> +.IR addr
> +and
> +.IR length .
>  .SH "RETURN VALUE"
>  On success
>  .BR madvise ()
> @@ -189,6 +226,14 @@ is not a valid value
>  .IP *
>  The application is attempting to release locked or shared pages (with
>  .BR MADV_DONTNEED ).
> +.IP *
> +.BR MADV_MERGEABLE
> +or
> +.BR MADV_UNMERGEABLE
> +was specified in
> +.IR advice ,
> +but the kernel was not configured with
> +.BR CONFIG_KSM .
>  .RE
>  .TP
>  .B EIO
> @@ -221,8 +266,10 @@ for file access.
>  .BR MADV_REMOVE ,
>  .BR MADV_DONTFORK ,
>  .BR MADV_DOFORK ,
> +.BR MAD_HWPOISON ,

Oh, I absolutely agree with you, MAD_HWPOISON indeed;
but Andi might prefer MADV_HWPOISON.

> +.BR MADVISE_MERGEABLE,

       MADV_MERGEABLE,

>  and
> -.BR MAD_HWPOISON
> +.BR MADVISE_UNMERGEABLE

       MADV_UNMERGEABLE

>  are Linux-specific.
>  .SH NOTES
>  .SS "Linux Notes"

Thanks!
Hugh

Re: man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

[Michael Kerrisk]

Hi Hugh,

On Sun, Jun 20, 2010 at 2:02 AM, Hugh Dickins <hughd@google.com> wrote:
> Hi Michael,
>
> On Sat, 19 Jun 2010, Michael Kerrisk wrote:
>
>> Hello Hugh, Izik
>>
>> For the MADV_MERGEABLE + MADV_UNMERGEABLE changes added in 2.6.32,
>> I've written the following man-pages text. Could you please
>> review/fix/ACK.
>
> Ah, good, thanks for writing this (but I recognize many of the words ;))

(Yes ;-).)

>> Thanks to Andi and Andrew for pointers that this documentation was needed.
>
> (We did Cc you when ksm.txt went into Andrew's tree, and when it went
> on into Linus's tree: is there something more we should have done?)

Ahh -- I missed that the CC came from you. (There is currently a big backlog.)

>> --- a/man2/madvise.2
>> +++ b/man2/madvise.2
>> @@ -156,6 +157,42 @@ and the page being unmapped.
>>  This feature is intended for memory testing.
>>  This feature is only available if the kernel was configured with
>>  .BR CONFIG_MEMORY_FAILURE .
>> +.TP
>> +.BR MADV_MERGEABLE " (since Linux 2.6.32)"
>> +Enable Kernel Samepage Merging (KSM) for the pages in the range specified by
>> +.I addr
>> +and
>> +.IR len .
>> +The KSM daemon
>> +.RI ( ksmd )
>> +periodically scans those areas of user memory that have
>> +been marked as mergeable,
>> +looking for pages with identical content.
>
> I was rather puzzled by Andi calling that an internal implementation
> detail.  Okay, I suppose the "KSM daemon periodically scans" is an
> internal implementation detail, though I think it helps to warn the
> programmer of the overhead of MADV_MERGEABLE; but without "pages
> with identical content", aren't we left mystified as to just what
> this "merging" is all about?

I agree. All I had changed was to remove mention of the KSM daemon --
I just say "the kernel".

>> +These are replaced by a single write-protected page (which is automatically
>> +copied if a process later wants to update the content of the page).
>> +KSM only merges private anonymous pages (see
>> +.BR mmap (2)).
>> +The KSM feature is intended for applications that generate many
>> +instances of the same data (e.g., virtualization systems such as KVM).
>> +It can consume a lot of processing power; use with care.
>
>                                           :
>
>> +See the kernel source file
>> +.I Documentation/vm/ksm.txt
>> +for more details.
>> +The
>> +.BR MADV_MERGEABLE
>> +and
>> +.BR MADV_UNMERGEABLE
>> +operations are only available if the kernel was configured with
>> +.BR CONFIG_KSM.
>> +.TP
>> +.BR MADV_UNMERGEABLE " (since Linux 2.6.32)"
>> +Undo the effect of an earlier
>> +.BR MADV_MERGEABLE
>> +operation on the specified address range;
>> +KSM unmerges whatever pages it had merged in the address rnage specified by
>
>                                                            range

Thanks.

>> +.IR addr
>> +and
>> +.IR length .
>>  .SH "RETURN VALUE"
>>  On success
>>  .BR madvise ()
>> @@ -189,6 +226,14 @@ is not a valid value
>>  .IP *
>>  The application is attempting to release locked or shared pages (with
>>  .BR MADV_DONTNEED ).
>> +.IP *
>> +.BR MADV_MERGEABLE
>> +or
>> +.BR MADV_UNMERGEABLE
>> +was specified in
>> +.IR advice ,
>> +but the kernel was not configured with
>> +.BR CONFIG_KSM .
>>  .RE
>>  .TP
>>  .B EIO
>> @@ -221,8 +266,10 @@ for file access.
>>  .BR MADV_REMOVE ,
>>  .BR MADV_DONTFORK ,
>>  .BR MADV_DOFORK ,
>> +.BR MAD_HWPOISON ,
>
> Oh, I absolutely agree with you, MAD_HWPOISON indeed;
> but Andi might prefer MADV_HWPOISON.
>
>> +.BR MADVISE_MERGEABLE,
>
>       MADV_MERGEABLE,
>
>>  and
>> -.BR MAD_HWPOISON
>> +.BR MADVISE_UNMERGEABLE
>
>       MADV_UNMERGEABLE

D'oh!  All of the above fixed now.

Thanks!

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface" http://blog.man7.org/

Re: man-pages text for MAV_MERGEABLE and MADV_UNMERGEABLE

[Hugh Dickins]

On Sat, Jun 19, 2010 at 10:36 PM, Michael Kerrisk
<mtk.manpages@gmail.com> wrote:
> On Sun, Jun 20, 2010 at 2:02 AM, Hugh Dickins <hughd@google.com> wrote:
>> On Sat, 19 Jun 2010, Michael Kerrisk wrote:
>>> +The KSM daemon
>>> +.RI ( ksmd )
>>> +periodically scans those areas of user memory that have
>>> +been marked as mergeable,
>>> +looking for pages with identical content.
>>
>> I was rather puzzled by Andi calling that an internal implementation
>> detail.  Okay, I suppose the "KSM daemon periodically scans" is an
>> internal implementation detail, though I think it helps to warn the
>> programmer of the overhead of MADV_MERGEABLE; but without "pages
>> with identical content", aren't we left mystified as to just what
>> this "merging" is all about?
>
> I agree. All I had changed was to remove mention of the KSM daemon --
> I just say "the kernel".

Great, that's just right: thanks.

Hugh