[PATCH 1/2] pipe.2: mention fcntl and F_GETPIPE_SZ/F_SETPIPE_SZ

[PATCH 1/2] pipe.2: mention fcntl and F_GETPIPE_SZ/F_SETPIPE_SZ

[Vegard Nossum]

Signed-off-by: Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 man2/pipe.2 | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git man2/pipe.2 man2/pipe.2
index 88835ca..e817ba9 100644
--- man2/pipe.2
+++ man2/pipe.2
@@ -130,6 +130,14 @@ file status flag on the two new open file descriptions.
 Using this flag saves extra calls to
 .BR fcntl (2)
 to achieve the same result.
+.P
+On Linux, the size of the pipe buffer can be obtained or changed using
+.B F_GETPIPE_SZ
+and
+.B F_SETPIPE_SZ;
+see
+.BR fcntl (2)
+for more information.
 .SH RETURN VALUE
 On success, zero is returned.
 On error, \-1 is returned, and
@@ -244,6 +252,7 @@ main(int argc, char *argv[])
 }
 .fi
 .SH SEE ALSO
+.BR fcntl (2),
 .BR fork (2),
 .BR read (2),
 .BR socketpair (2),
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] pipe.2: document sysctls

[Vegard Nossum]

Cc: Willy Tarreau <w@1wt.eu>
Cc: Jens Axboe <jens.axboe-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 man2/pipe.2 | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git man2/pipe.2 man2/pipe.2
index e817ba9..ecf2f34 100644
--- man2/pipe.2
+++ man2/pipe.2
@@ -184,6 +184,35 @@ POSIX.1-2001, POSIX.1-2008.
 
 .BR pipe2 ()
 is Linux-specific.
+
+.SH FILES
+On Linux, the following files control how much memory can be used for pipes:
+.SS /proc/sys/fs/pipe-max-pages " (since Linux 2.6.34)"
+.\" commit b492e95be0ae672922f4734acf3f5d35c30be948
+Deprecated since Linux 2.6.35.
+.SS /proc/sys/fs/pipe-max-size " (since Linux 2.6.35)"
+.\" commit ff9da691c0498ff81fdd014e7a0731dab2337dac
+The maximum size (in bytes) of individual pipes created or set by users
+without the
+.B CAP_SYS_RESOURCE
+capability.
+.SS /proc/sys/fs/pipe-user-pages-hard " (since Linux 4.4)"
+.\" commit 759c01142a5d0f364a462346168a56de28a80f52
+The combined maximum size (in pages) of all pipes created or set by a
+single user without both the
+.B CAP_SYS_RESOURCE
+and
+.B CAP_SYS_ADMIN
+capabilities.
+.SS /proc/sys/fs/pipe-user-pages-soft " (since Linux 4.4)"
+.\" commit 759c01142a5d0f364a462346168a56de28a80f52
+The combined size (in pages) of all pipes created or set by a single
+user (without both the
+.B CAP_SYS_RESOURCE
+and
+.B CAP_SYS_ADMIN
+capabilities) after which individual pipes will be limited to 1 page.
+
 .SH EXAMPLE
 .\" fork.2 refers to this example program.
 The following program creates a pipe, and then
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] pipe.2: document sysctls

[Michael Kerrisk (man-pages)]

[Adding a typo fix, and correct email for Jens]

On 08/14/2016 11:13 AM, Michael Kerrisk (man-pages) wrote:
> Hello Vegard,
>
> On 08/13/2016 03:35 AM, Vegard Nossum wrote:
>> Cc: Willy Tarreau <w@1wt.eu>
>> Cc: Jens Axboe <jens.axboe-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>> Signed-off-by: Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>> ---
>>  man2/pipe.2 | 29 +++++++++++++++++++++++++++++
>>  1 file changed, 29 insertions(+)
>> diff --git man2/pipe.2 man2/pipe.2
>> index e817ba9..ecf2f34 100644
>> --- man2/pipe.2
>> +++ man2/pipe.2
>> @@ -184,6 +184,35 @@ POSIX.1-2001, POSIX.1-2008.
>>
>>  .BR pipe2 ()
>>  is Linux-specific.
>> +
>> +.SH FILES
>> +On Linux, the following files control how much memory can be used for pipes:
>> +.SS /proc/sys/fs/pipe-max-pages " (since Linux 2.6.34)"
>> +.\" commit b492e95be0ae672922f4734acf3f5d35c30be948
>> +Deprecated since Linux 2.6.35.
>> +.SS /proc/sys/fs/pipe-max-size " (since Linux 2.6.35)"
>> +.\" commit ff9da691c0498ff81fdd014e7a0731dab2337dac
>> +The maximum size (in bytes) of individual pipes created or set by users
>> +without the
>> +.B CAP_SYS_RESOURCE
>> +capability.
>> +.SS /proc/sys/fs/pipe-user-pages-hard " (since Linux 4.4)"
>> +.\" commit 759c01142a5d0f364a462346168a56de28a80f52
>> +The combined maximum size (in pages) of all pipes created or set by a
>> +single user without both the
>> +.B CAP_SYS_RESOURCE
>> +and
>> +.B CAP_SYS_ADMIN
>> +capabilities.
>> +.SS /proc/sys/fs/pipe-user-pages-soft " (since Linux 4.4)"
>> +.\" commit 759c01142a5d0f364a462346168a56de28a80f52
>> +The combined size (in pages) of all pipes created or set by a single
>> +user (without both the
>> +.B CAP_SYS_RESOURCE
>> +and
>> +.B CAP_SYS_ADMIN
>> +capabilities) after which individual pipes will be limited to 1 page.
>> +
>>  .SH EXAMPLE
>>  .\" fork.2 refers to this example program.
>>  The following program creates a pipe, and then
>
> Thanks. I've added this text to pipe(7) (rather than pipe(2)), and
> expanded it considerably:
>
>     /proc files
>         On  Linux, the following files control how much memory can be used
>         for pipes:
>
>         /proc/sys/fs/pipe-max-pages (only in Linux 2.6.34)
>                An upper limit, in pages, on the capacity that an  unprivi‐
>                leged  user  (one without the CAP_SYS_ADMIN capability) can

s/CAP_SYS_ADMIN/CAP_SYS_RESOURCE/
(Along the way, I injected that as a typo.)

>                set for a pipe.  The default value for  this  limit  is  16
>                times  the  default  pipe  capacity  (see above); the lower
>                limit is two pages.  This interface was  removed  in  Linux
>                2.6.35, in favor of /proc/sys/fs/pipe-max-size.
>
>         /proc/sys/fs/pipe-max-size (since Linux 2.6.35)
>                The  maximum size (in bytes) of individual pipes created or
>                set by users without the CAP_SYS_RESOURCE capability.   The
>                default  value for this file is 1048576.  Attempts to set a
>                limit less than the page size cause write(2) to  fail  with
>                the error EINVAL.
>
>         /proc/sys/fs/pipe-user-pages-hard (since Linux 4.5)
>                The  hard  limit  on the total size (in pages) of all pipes
>                created or set by a single  unprivileged  user  (i.e.,  one
>                with  neither  the  CAP_SYS_RESOURCE  nor the CAP_SYS_ADMIN
>                capability).  So long as the total number  of  pages  allo‐
>                cated  to  pipe  buffers  for  this  user is at this limit,
>                attempts to create new pipes will be denied,  and  attempts
>                to increase a pipe's capacity will be denied.
>
>                When  the  value  of  this  limit  is  zero  (which  is the
>                default), no hard limit is applied.
>
>         /proc/sys/fs/pipe-user-pages-soft (since Linux 4.5)
>                The soft limit on the total size (in pages)  of  all  pipes
>                created or set by a single unprivileged user (.e., one with
>                neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capabil‐
>                ity).   So  long  as the total number of pages allocated to
>                pipe buffers for this user is  at  this  limit,  individual
>                pipes  created  by  a user will be limited to one page, and
>                attempts to increase a pipe's capacity will be denied.
>
>                When the value of this limit is  zero,  no  soft  limit  is
>                applied.   The  default value for this file is 16384, which
>                permits creating up to 1024 pipes with the  default  capac‐
>                ity.
>
> How does the above seem?

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] pipe.2: document sysctls

[Vegard Nossum]

On 08/14/2016 01:13 AM, Michael Kerrisk (man-pages) wrote:
>
> Thanks. I've added this text to pipe(7) (rather than pipe(2)), and
> expanded it considerably:

Good idea!

>
>     /proc files
>         On  Linux, the following files control how much memory can be used
>         for pipes:
>
>         /proc/sys/fs/pipe-max-pages (only in Linux 2.6.34)
>                An upper limit, in pages, on the capacity that an  unprivi‐
>                leged  user  (one without the CAP_SYS_RESOURCE capability) can
>                set for a pipe.  The default value for  this  limit  is  16
>                times  the  default  pipe  capacity  (see above); the lower
>                limit is two pages.  This interface was  removed  in  Linux
>                2.6.35, in favor of /proc/sys/fs/pipe-max-size.
>
>         /proc/sys/fs/pipe-max-size (since Linux 2.6.35)
>                The  maximum size (in bytes) of individual pipes created or

I think I had a mistake in my original text because this limit is not
checked at creation time (the default pipe size, PIPE_DEF_BUFFERS, is
used even when it is greater than pipe-max-size), so maybe we can take
out the "created or".

>                set by users without the CAP_SYS_RESOURCE capability.   The
>                default  value for this file is 1048576.  Attempts to set a
>                limit less than the page size cause write(2) to  fail  with
>                the error EINVAL.

Maybe add in a paranthesis that 1048576 is 1 MiB?

>
>         /proc/sys/fs/pipe-user-pages-hard (since Linux 4.5)
>                The  hard  limit  on the total size (in pages) of all pipes
>                created or set by a single  unprivileged  user  (i.e.,  one
>                with  neither  the  CAP_SYS_RESOURCE  nor the CAP_SYS_ADMIN
>                capability).  So long as the total number  of  pages  allo‐
>                cated  to  pipe  buffers  for  this  user is at this limit,
>                attempts to create new pipes will be denied,  and  attempts
>                to increase a pipe's capacity will be denied.
>
>                When  the  value  of  this  limit  is  zero  (which  is the
>                default), no hard limit is applied.
>
>         /proc/sys/fs/pipe-user-pages-soft (since Linux 4.5)
>                The soft limit on the total size (in pages)  of  all  pipes
>                created or set by a single unprivileged user (.e., one with
>                neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capabil‐
>                ity).   So  long  as the total number of pages allocated to
>                pipe buffers for this user is  at  this  limit,  individual
>                pipes  created  by  a user will be limited to one page, and
>                attempts to increase a pipe's capacity will be denied.
>
>                When the value of this limit is  zero,  no  soft  limit  is
>                applied.   The  default value for this file is 16384, which
>                permits creating up to 1024 pipes with the  default  capac‐
>                ity.
>
> How does the above seem?

It looks great, thanks for cleaning up my measly attempt.


Vegard
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] pipe.2: document sysctls

[Willy Tarreau]

On Sun, Aug 14, 2016 at 09:41:28AM +0200, Vegard Nossum wrote:
(...)
> It looks great, thanks for cleaning up my measly attempt.

Looks great to me as well, thanks guys.

Willy
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] pipe.2: document sysctls

[Michael Kerrisk (man-pages)]

Hello Vegard,

On 08/14/2016 07:41 PM, Vegard Nossum wrote:
> On 08/14/2016 01:13 AM, Michael Kerrisk (man-pages) wrote:
>>
>> Thanks. I've added this text to pipe(7) (rather than pipe(2)), and
>> expanded it considerably:
> 
> Good idea!
> 
>>
>>     /proc files
>>         On  Linux, the following files control how much memory can be used
>>         for pipes:
>>
>>         /proc/sys/fs/pipe-max-pages (only in Linux 2.6.34)
>>                An upper limit, in pages, on the capacity that an  unprivi‐
>>                leged  user  (one without the CAP_SYS_RESOURCE capability) can
>>                set for a pipe.  The default value for  this  limit  is  16
>>                times  the  default  pipe  capacity  (see above); the lower
>>                limit is two pages.  This interface was  removed  in  Linux
>>                2.6.35, in favor of /proc/sys/fs/pipe-max-size.
>>
>>         /proc/sys/fs/pipe-max-size (since Linux 2.6.35)
>>                The  maximum size (in bytes) of individual pipes created or
> 
> I think I had a mistake in my original text because this limit is not
> checked at creation time (the default pipe size, PIPE_DEF_BUFFERS, is
> used even when it is greater than pipe-max-size), so maybe we can take
> out the "created or".

Hmmm -- that's true. It seems a little weird that one can create a pipe
whose initial size exceeds pipe-max-size. Should we fix that in the kernel?
(I can add a third patch to my series.) Setting pipe-max-size lower than
PIPE_DEF_BUFFERS is an admittedly an odd set up, so maybe it's not worth
changing the code(?).

I removed the "created or".

>>                set by users without the CAP_SYS_RESOURCE capability.   The
>>                default  value for this file is 1048576.  Attempts to set a
>>                limit less than the page size cause write(2) to  fail  with
>>                the error EINVAL.
> 
> Maybe add in a paranthesis that 1048576 is 1 MiB?

Done.

>>
>>         /proc/sys/fs/pipe-user-pages-hard (since Linux 4.5)
>>                The  hard  limit  on the total size (in pages) of all pipes
>>                created or set by a single  unprivileged  user  (i.e.,  one
>>                with  neither  the  CAP_SYS_RESOURCE  nor the CAP_SYS_ADMIN
>>                capability).  So long as the total number  of  pages  allo‐
>>                cated  to  pipe  buffers  for  this  user is at this limit,
>>                attempts to create new pipes will be denied,  and  attempts
>>                to increase a pipe's capacity will be denied.
>>
>>                When  the  value  of  this  limit  is  zero  (which  is the
>>                default), no hard limit is applied.
>>
>>         /proc/sys/fs/pipe-user-pages-soft (since Linux 4.5)
>>                The soft limit on the total size (in pages)  of  all  pipes
>>                created or set by a single unprivileged user (.e., one with
>>                neither the CAP_SYS_RESOURCE nor the CAP_SYS_ADMIN capabil‐
>>                ity).   So  long  as the total number of pages allocated to
>>                pipe buffers for this user is  at  this  limit,  individual
>>                pipes  created  by  a user will be limited to one page, and
>>                attempts to increase a pipe's capacity will be denied.
>>
>>                When the value of this limit is  zero,  no  soft  limit  is
>>                applied.   The  default value for this file is 16384, which
>>                permits creating up to 1024 pipes with the  default  capac‐
>>                ity.
>>
>> How does the above seem?
> 
> It looks great, thanks for cleaning up my measly attempt.

No problem. Yet again, documenting the API well finds some interesting
bugs ;-).

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] pipe.2: mention fcntl and F_GETPIPE_SZ/F_SETPIPE_SZ

[Michael Kerrisk (man-pages)]

Hello Vegard,

On 08/13/2016 03:35 AM, Vegard Nossum wrote:
> Signed-off-by: Vegard Nossum <vegard.nossum-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> ---
>  man2/pipe.2 | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git man2/pipe.2 man2/pipe.2
> index 88835ca..e817ba9 100644
> --- man2/pipe.2
> +++ man2/pipe.2
> @@ -130,6 +130,14 @@ file status flag on the two new open file descriptions.
>  Using this flag saves extra calls to
>  .BR fcntl (2)
>  to achieve the same result.
> +.P
> +On Linux, the size of the pipe buffer can be obtained or changed using
> +.B F_GETPIPE_SZ
> +and
> +.B F_SETPIPE_SZ;
> +see
> +.BR fcntl (2)
> +for more information.

This detail is covered (with rather more context), in pipe(7), which is
referenced in the first paragraph of the pipe(2) page.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html