rseq(2) man page

rseq(2) man page

[Mathieu Desnoyers]

Hi!

I would like to contribute a man page for the rseq(2) system call to the
man-pages project. I maintain this system call which appeared in Linux 4.18.
I have done several attempts to contribute a man page for it in the past,
so let's hope we will have more luck this time.

I have just done some improvements to the man page, here is its current
location:

https://git.kernel.org/pub/scm/libs/librseq/librseq.git/tree/doc/man/rseq.2

Please let me know if this is a good time to contribute it, and if I need
to do significant changes before submitting again.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: rseq(2) man page

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 15176 bytes --]

Hi Mathieu!

On 1/6/23 18:16, Mathieu Desnoyers wrote:
> Hi!
> 
> I would like to contribute a man page for the rseq(2) system call to the
> man-pages project. I maintain this system call which appeared in Linux 4.18.
> I have done several attempts to contribute a man page for it in the past,
> so let's hope we will have more luck this time.

:)

> 
> I have just done some improvements to the man page, here is its current
> location:
> 
> https://git.kernel.org/pub/scm/libs/librseq/librseq.git/tree/doc/man/rseq.2

Thanks!  I'll copy it before to answer to it.

> 
> Please let me know if this is a good time to contribute it,

Yes, it is.

> and if I need
> to do significant changes before submitting again.

BTW, would you mind sending links to the previous submissions?  I couldn't find 
them.  I know there are a few patches that we received when Michael was 
on-and-off that I deferred for a later time, and never did, and maybe this is 
one of those.  I tried to find such patches some time ago, but with no luck.

> 
> Thanks,
> 
> Mathieu
> 

Cheers,

Alex


---

.\" Copyright 2015-2020 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
.\"
.\" %%%LICENSE_START(VERBATIM)
.\" Permission is granted to make and distribute verbatim copies of this
.\" manual provided the copyright notice and this permission notice are
.\" preserved on all copies.
.\"
.\" Permission is granted to copy and distribute modified versions of this
.\" manual under the conditions for verbatim copying, provided that the
.\" entire resulting derived work is distributed under the terms of a
.\" permission notice identical to this one.
.\"
.\" Since the Linux kernel and libraries are constantly changing, this
.\" manual page may be incorrect or out-of-date.  The author(s) assume no
.\" responsibility for errors or omissions, or for damages resulting from
.\" the use of the information contained herein.  The author(s) may not
.\" have taken the same level of care in the production of this manual,
.\" which is licensed free of charge, as they might when working
.\" professionally.
.\"
.\" Formatted or processed versions of this manual, if unaccompanied by
.\" the source, must acknowledge the copyright and authors of this work.
.\" %%%LICENSE_END
.\"
.TH RSEQ 2 2020-06-05 "Linux" "Linux Programmer's Manual"
.SH NAME
rseq \- Restartable sequences system call
.SH SYNOPSIS
.nf
.B #include <linux/rseq.h>
.sp
.BI "int rseq(struct rseq * " rseq ", uint32_t " rseq_len ", int " flags ", 
uint32_t " sig ");
.sp
.SH DESCRIPTION

The
.BR rseq ()
ABI accelerates specific user-space operations by registering a
per-thread data structure shared between kernel and user-space. This
data structure can be read from or written to by user-space to skip
otherwise expensive system calls.

A restartable sequence is a sequence of instructions guaranteed to be executed
atomically with respect to other threads and signal handlers on the current
CPU. If its execution does not complete atomically, the kernel changes the
execution flow by jumping to an abort handler defined by user-space for that
restartable sequence.

Using restartable sequences requires to register a
rseq ABI per-thread data structure (struct rseq) through the
.BR rseq ()
system call. Only one rseq ABI can be registered per thread, so
user-space libraries and applications must follow a user-space ABI
defining how to share this resource. The ABI defining how to share this
resource between applications and libraries is defined by the C library.
Allocation of the per-thread rseq ABI and its registration to the kernel
is handled by glibc since version 2.35.

The rseq ABI per-thread data structure contains a
.I rseq_cs
field which points to the currently executing critical section. For each
thread, a single rseq critical section can run at any given point. Each
critical section need to be implemented in assembly.

The
.BR rseq ()
ABI accelerates user-space operations on per-cpu data by defining a
shared data structure ABI between each user-space thread and the kernel.

It allows user-space to perform update operations on per-cpu data
without requiring heavy-weight atomic operations.

The term CPU used in this documentation refers to a hardware execution
context. For instance, each CPU number returned by
.BR sched_getcpu ()
is a CPU. The current CPU means to the CPU on which the registered thread is
running.

Restartable sequences are atomic with respect to preemption (making it
atomic with respect to other threads running on the same CPU), as well
as signal delivery (user-space execution contexts nested over the same
thread). They either complete atomically with respect to preemption on
the current CPU and signal delivery, or they are aborted.

Restartable sequences are suited for update operations on per-cpu data.

Restartable sequences can be used on data structures shared between threads
within a process, and on data structures shared between threads across
different processes.

.PP
Some examples of operations that can be accelerated or improved
by this ABI:
.IP \[bu] 2
Memory allocator per-cpu free-lists,
.IP \[bu] 2
Querying the current CPU number,
.IP \[bu] 2
Incrementing per-CPU counters,
.IP \[bu] 2
Modifying data protected by per-CPU spinlocks,
.IP \[bu] 2
Inserting/removing elements in per-CPU linked-lists,
.IP \[bu] 2
Writing/reading per-CPU ring buffers content.
.IP \[bu] 2
Accurately reading performance monitoring unit counters
with respect to thread migration.

.PP
Restartable sequences must not perform system calls. Doing so may result
in termination of the process by a segmentation fault.

.PP
The
.I rseq
argument is a pointer to the thread-local rseq structure to be shared
between kernel and user-space.

.PP
The structure
.B struct rseq
is an extensible structure. Additional feature fields can be added in
future kernel versions. Its layout is as follows:
.TP
.B Structure alignment
This structure is aligned on either 32-byte boundary, or on the
alignment value returned by
.I getauxval(AT_RSEQ_ALIGN)
if the structure size differs from 32 bytes.
.TP
.B Structure size
This structure size needs to be at least 32 bytes. It can be either
32 bytes, or it needs to be large enough to hold the result of
.I getauxval(AT_RSEQ_FEATURE_SIZE) .
Its size is passed as parameter to the rseq system call.
.PP
.in +8n
.EX
struct rseq {
     __u32 cpu_id_start;
     __u32 cpu_id;
     union {
         /* Edited out for conciseness. [...] */
     } rseq_cs;
     __u32 flags;
     __u32 node_id;
     __u32 mm_cid;
} __attribute__((aligned(32)));
.EE
.TP
.B Fields

.TP
.in +4n
.I cpu_id_start
Always-updated value of the CPU number on which the registered thread is
running. Its value is guaranteed to always be a possible CPU number,
even when rseq is not registered. Its value should always be confirmed by
reading the cpu_id field before user-space performs any side-effect (e.g.
storing to memory).

This field is always guaranteed to hold a valid CPU number in the range
[ 0 ..  nr_possible_cpus - 1 ]. It can therefore be loaded by user-space
and used as an offset in per-cpu data structures without having to check
whether its value is within the valid bounds compared to the number of
possible CPUs in the system.

Initialized by user-space to a possible CPU number (e.g., 0), updated
by the kernel for threads registered with rseq.

For user-space applications executed on a kernel without rseq support,
the cpu_id_start field stays initialized at 0, which is indeed a valid
CPU number. It is therefore valid to use it as an offset in per-cpu data
structures, and only validate whether it's actually the current CPU
number by comparing it with the cpu_id field within the rseq critical
section. If the kernel does not provide rseq support, that cpu_id field
stays initialized at -1, so the comparison always fails, as intended.

This field should only be read by the thread which registered this data
structure. Aligned on 32-bit.

It is up to user-space to implement a fall-back mechanism for scenarios where
rseq is not available.
.in
.TP
.in +4n
.I cpu_id
Always-updated value of the CPU number on which the registered thread is
running. Initialized by user-space to -1, updated by the kernel for
threads registered with rseq.

This field should only be read by the thread which registered this data
structure. Aligned on 32-bit.
.in
.TP
.in +4n
.I rseq_cs
The rseq_cs field is a pointer to a struct rseq_cs. Is is NULL when no
rseq assembly block critical section is active for the registered thread.
Setting it to point to a critical section descriptor (struct rseq_cs)
marks the beginning of the critical section.

Initialized by user-space to NULL.

Updated by user-space, which sets the address of the currently
active rseq_cs at the beginning of assembly instruction sequence
block, and set to NULL by the kernel when it restarts an assembly
instruction sequence block, as well as when the kernel detects that
it is preempting or delivering a signal outside of the range
targeted by the rseq_cs. Also needs to be set to NULL by user-space
before reclaiming memory that contains the targeted struct rseq_cs.

Read and set by the kernel.

This field should only be updated by the thread which registered this
data structure. Aligned on 64-bit.
.in
.TP
.in +4n
.I flags
Flags indicating the restart behavior for the registered thread. This is
mainly used for debugging purposes. Can be a combination of:
.IP \[bu]
RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT: Inhibit instruction sequence block restart
on preemption for this thread. This flag is deprecated since kernel 6.1.
.IP \[bu]
RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL: Inhibit instruction sequence block restart
on signal delivery for this thread. This flag is deprecated since kernel 6.1.
.IP \[bu]
RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE: Inhibit instruction sequence block restart
on migration for this thread. This flag is deprecated since kernel 6.1.

Initialized by user-space, used by the kernel.
.in
.TP
.in +4n
.I node_id
Always-updated value of the current NUMA node ID.

Initialized by user-space to 0.

Updated by the kernel. Read by user-space with single-copy atomicity
semantics. This field should only be read by the thread which registered
this data structure. Aligned on 32-bit.
.in
.TP
.in +4n
.I mm_cid
Contains the current thread's concurrency ID (allocated uniquely within
a memory map).

Updated by the kernel. Read by user-space with single-copy atomicity
semantics. This field should only be read by the thread which registered
this data structure. Aligned on 32-bit.

This concurrency ID is within the possible cpus range, and is
temporarily (and uniquely) assigned while threads are actively running
within a memory map. If a memory map has fewer threads than cores, or is
limited to run on few cores concurrently through sched affinity or
cgroup cpusets, the concurrency IDs will be values close to 0, thus
allowing efficient use of user-space memory for per-cpu data structures.

.PP
The layout of
.B struct rseq_cs
version 0 is as follows:
.TP
.B Structure alignment
This structure is aligned on 32-byte boundary.
.TP
.B Structure size
This structure has a fixed size of 32 bytes.
.PP
.in +8n
.EX
struct rseq_cs {
     __u32   version;
     __u32   flags;
     __u64   start_ip;
     __u64   post_commit_offset;
     __u64   abort_ip;
} __attribute__((aligned(32)));
.EE
.TP
.B Fields

.TP
.in +4n
.I version
Version of this structure. Should be initialized to 0.
.in
.TP
.in +4n
.I flags
Flags indicating the restart behavior of this structure. Can be a combination
of:
.IP \[bu]
RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT: Inhibit instruction sequence block restart
on preemption for this critical section. This flag is deprecated since kernel
6.1.
.IP \[bu]
RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL: Inhibit instruction sequence block restart
on signal delivery for this critical section. This flag is deprecated since
kernel 6.1.
.IP \[bu]
RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE: Inhibit instruction sequence block restart
on migration for this critical section. This flag is deprecated since kernel
6.1.
.TP
.in +4n
.I start_ip
Instruction pointer address of the first instruction of the sequence of
consecutive assembly instructions.
.in
.TP
.in +4n
.I post_commit_offset
Offset (from start_ip address) of the address after the last instruction
of the sequence of consecutive assembly instructions.
.in
.TP
.in +4n
.I abort_ip
Instruction pointer address where to move the execution flow in case of
abort of the sequence of consecutive assembly instructions.
.in

.PP
The
.I rseq_len
argument is the size of the
.I struct rseq
to register.

.PP
The
.I flags
argument is 0 for registration, and
.IR RSEQ_FLAG_UNREGISTER
for unregistration.

.PP
The
.I sig
argument is the 32-bit signature to be expected before the abort
handler code.

.PP
A single library per process should keep the rseq structure in a
per-thread data structure.
The
.I cpu_id
field should be initialized to -1, and the
.I cpu_id_start
field should be initialized to a possible CPU value (typically 0).

.PP
Each thread is responsible for registering and unregistering its rseq
structure. No more than one rseq structure address can be registered
per thread at a given time.

.PP
Reclaim of rseq object's memory must only be done after either an
explicit rseq unregistration is performed or after the thread exits.

.PP
In a typical usage scenario, the thread registering the rseq
structure will be performing loads and stores from/to that structure. It
is however also allowed to read that structure from other threads.
The rseq field updates performed by the kernel provide relaxed atomicity
semantics (atomic store, without memory ordering), which guarantee that other
threads performing relaxed atomic reads (atomic load, without memory ordering)
of the cpu number fields will always observe a consistent value.

.SH RETURN VALUE
A return value of 0 indicates success. On error, \-1 is returned, and
.I errno
is set appropriately.

.SH ERRORS
.TP
.B EINVAL
Either
.I flags
contains an invalid value, or
.I rseq
contains an address which is not appropriately aligned, or
.I rseq_len
contains an incorrect size.
.TP
.B ENOSYS
The
.BR rseq ()
system call is not implemented by this kernel.
.TP
.B EFAULT
.I rseq
is an invalid address.
.TP
.B EBUSY
Restartable sequence is already registered for this thread.
.TP
.B EPERM
The
.I sig
argument on unregistration does not match the signature received
on registration.

.SH VERSIONS
The
.BR rseq ()
system call was added in Linux 4.18.

.SH CONFORMING TO
.BR rseq ()
is Linux-specific.

.in
.SH SEE ALSO
.BR sched_getcpu (3) ,
.BR membarrier (2) ,
.BR getauxval (3)


-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: rseq(2) man page

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 19607 bytes --]

Hi Mathieu,

See some comments below.

Cheers,

Alex

> .\" Copyright 2015-2020 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> .\"
> .\" %%%LICENSE_START(VERBATIM)
> .\" Permission is granted to make and distribute verbatim copies of this
> .\" manual provided the copyright notice and this permission notice are
> .\" preserved on all copies.
> .\"
> .\" Permission is granted to copy and distribute modified versions of this
> .\" manual under the conditions for verbatim copying, provided that the
> .\" entire resulting derived work is distributed under the terms of a
> .\" permission notice identical to this one.
> .\"
> .\" Since the Linux kernel and libraries are constantly changing, this
> .\" manual page may be incorrect or out-of-date.  The author(s) assume no
> .\" responsibility for errors or omissions, or for damages resulting from
> .\" the use of the information contained herein.  The author(s) may not
> .\" have taken the same level of care in the production of this manual,
> .\" which is licensed free of charge, as they might when working
> .\" professionally.
> .\"
> .\" Formatted or processed versions of this manual, if unaccompanied by
> .\" the source, must acknowledge the copyright and authors of this work.
> .\" %%%LICENSE_END

We now use SPDX-License-Identifier.

> .\"
> .TH RSEQ 2 2020-06-05 "Linux" "Linux Programmer's Manual"

We use lowercase for the function name (or to be more precise, the same case 
that the function name uses.

The date is specified with a placeholder that is replaced at the time of 
creation of the tarball.

The 5th argument is unspecified.

The 4th argument is now the project name and a placeholder for the version.

See an example:

$ cat man2/membarrier.2 | grep '^.TH'
.TH membarrier 2 (date) "Linux man-pages (unreleased)"


> .SH NAME
> rseq \- Restartable sequences system call

We use lowercase here, so s/Restartable/restartable/

> .SH SYNOPSIS
> .nf
> .B #include <linux/rseq.h>

Is there a glibc wrapper for this syscall?  Do you expect that it will be added 
relatively soon?  Or is it expected that this syscall will be called through 
syscall(2) for many years?

If so, it may be better to document it directly as such, like for example 
membarrier:

SYNOPSIS
        #include <linux/membarrier.h> /* Definition of MEMBARRIER_* constants */
        #include <sys/syscall.h>      /* Definition of SYS_* constants */
        #include <unistd.h>

        int syscall(SYS_membarrier, int cmd, unsigned int flags, int cpu_id);

        Note: glibc provides no wrapper for membarrier(), necessitating the use
        of syscall(2).

> .sp

s/sp/PP/

> .BI "int rseq(struct rseq * " rseq ", uint32_t " rseq_len ", int " flags ", 

Is it valid to pass NULL in 'rseq'?  If it is, we now document that using 
_Nullable.  See for example recv(2):

SYNOPSIS
        #include <sys/socket.h>

        ssize_t recv(int sockfd, void buf[.len], size_t len,
                         int flags);
        ssize_t recvfrom(int sockfd, void buf[restrict .len], size_t len,
                         int flags,
                         struct sockaddr *_Nullable restrict src_addr,
                         socklen_t *_Nullable restrict addrlen);
        ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);




> uint32_t " sig ");
> .sp

.sp is unnecessary here.

.fi is missing (it's the "closing" pair to .nf).

> .SH DESCRIPTION
> 

Use .PP instead of blank lines.

> The
> .BR rseq ()
> ABI accelerates specific user-space operations by registering a
> per-thread data structure shared between kernel and user-space. This

Use semantic newlines.  See man-pages(7):

    Use semantic newlines
        In  the source of a manual page, new sentences should be started on new
        lines, long sentences should be split into lines at clause breaks (com‐
        mas, semicolons, colons, and so on), and long clauses should  be  split
        at  phrase  boundaries.   This convention, sometimes known as "semantic
        newlines", makes it easier to see the effect of  patches,  which  often
        operate at the level of individual sentences, clauses, or phrases.


> data structure can be read from or written to by user-space to skip
> otherwise expensive system calls.
> 
> A restartable sequence is a sequence of instructions guaranteed to be executed
> atomically with respect to other threads and signal handlers on the current
> CPU. If its execution does not complete atomically, the kernel changes the
> execution flow by jumping to an abort handler defined by user-space for that
> restartable sequence.
> 
> Using restartable sequences requires to register a
> rseq ABI per-thread data structure (struct rseq) through the
> .BR rseq ()
> system call. Only one rseq ABI can be registered per thread, so
> user-space libraries and applications must follow a user-space ABI
> defining how to share this resource. The ABI defining how to share this
> resource between applications and libraries is defined by the C library.
> Allocation of the per-thread rseq ABI and its registration to the kernel
> is handled by glibc since version 2.35.
> 
> The rseq ABI per-thread data structure contains a
> .I rseq_cs
> field which points to the currently executing critical section. For each
> thread, a single rseq critical section can run at any given point. Each
> critical section need to be implemented in assembly.
> 
> The
> .BR rseq ()
> ABI accelerates user-space operations on per-cpu data by defining a
> shared data structure ABI between each user-space thread and the kernel.
> 
> It allows user-space to perform update operations on per-cpu data
> without requiring heavy-weight atomic operations.
> 
> The term CPU used in this documentation refers to a hardware execution
> context. For instance, each CPU number returned by
> .BR sched_getcpu ()
> is a CPU. The current CPU means to the CPU on which the registered thread is
> running.
> 
> Restartable sequences are atomic with respect to preemption (making it
> atomic with respect to other threads running on the same CPU), as well
> as signal delivery (user-space execution contexts nested over the same
> thread). They either complete atomically with respect to preemption on
> the current CPU and signal delivery, or they are aborted.
> 
> Restartable sequences are suited for update operations on per-cpu data.
> 
> Restartable sequences can be used on data structures shared between threads
> within a process, and on data structures shared between threads across
> different processes.
> 
> .PP
> Some examples of operations that can be accelerated or improved
> by this ABI:
> .IP \[bu] 2

Use 3 instead of 2.  See man-pages:

    Lists
        There are different kinds of lists:

        [...]

        Bullet lists
               Elements  are  preceded by bullet symbols (\(bu).  Anything that
               doesn’t fit elsewhere is usually covered by this type of list.

        [...]

        There should always be exactly 2 spaces between the list symbol and the
        elements.  This doesn’t apply to "tagged paragraphs", which use the de‐
        fault indentation rules.


The rationale for that was that if you use 1 space, then the list introducer can 
be confused with the list contents.  Two spaces makes the difference more clear.


Also, we use \(bu instead of \[bu].  I'm not particularly worried by using it, 
but I prefer being consistent at which one we use.


> Memory allocator per-cpu free-lists,
> .IP \[bu] 2
> Querying the current CPU number,
> .IP \[bu] 2
> Incrementing per-CPU counters,
> .IP \[bu] 2
> Modifying data protected by per-CPU spinlocks,
> .IP \[bu] 2
> Inserting/removing elements in per-CPU linked-lists,
> .IP \[bu] 2
> Writing/reading per-CPU ring buffers content.
> .IP \[bu] 2
> Accurately reading performance monitoring unit counters
> with respect to thread migration.
> 
> .PP
> Restartable sequences must not perform system calls. Doing so may result
> in termination of the process by a segmentation fault.
> 
> .PP
> The
> .I rseq
> argument is a pointer to the thread-local rseq structure to be shared
> between kernel and user-space.
> 
> .PP
> The structure
> .B struct rseq
> is an extensible structure. Additional feature fields can be added in
> future kernel versions. Its layout is as follows:
> .TP
> .B Structure alignment
> This structure is aligned on either 32-byte boundary, or on the
> alignment value returned by
> .I getauxval(AT_RSEQ_ALIGN)
> if the structure size differs from 32 bytes.
> .TP
> .B Structure size
> This structure size needs to be at least 32 bytes. It can be either
> 32 bytes, or it needs to be large enough to hold the result of
> .I getauxval(AT_RSEQ_FEATURE_SIZE) .
> Its size is passed as parameter to the rseq system call.
> .PP
> .in +8n
> .EX
> struct rseq {
>      __u32 cpu_id_start;
>      __u32 cpu_id;
>      union {
>          /* Edited out for conciseness. [...] */
>      } rseq_cs;
>      __u32 flags;
>      __u32 node_id;
>      __u32 mm_cid;
> } __attribute__((aligned(32)));
> .EE
> .TP
> .B Fields
> 
> .TP
> .in +4n

I guess you're looking for .RS/.RE.  It would wrap all the indented stuff, 
replacing .in.

> .I cpu_id_start
> Always-updated value of the CPU number on which the registered thread is
> running. Its value is guaranteed to always be a possible CPU number,
> even when rseq is not registered. Its value should always be confirmed by

rseq (and maybe other cases around too) should be formatted in italics, since 
it's a variable name (.I).

> reading the cpu_id field before user-space performs any side-effect (e.g.
> storing to memory).
> 
> This field is always guaranteed to hold a valid CPU number in the range
> [ 0 ..  nr_possible_cpus - 1 ]. It can therefore be loaded by user-space
> and used as an offset in per-cpu data structures without having to check
> whether its value is within the valid bounds compared to the number of
> possible CPUs in the system.
> 
> Initialized by user-space to a possible CPU number (e.g., 0), updated
> by the kernel for threads registered with rseq.
> 
> For user-space applications executed on a kernel without rseq support,
> the cpu_id_start field stays initialized at 0, which is indeed a valid
> CPU number. It is therefore valid to use it as an offset in per-cpu data
> structures, and only validate whether it's actually the current CPU
> number by comparing it with the cpu_id field within the rseq critical
> section. If the kernel does not provide rseq support, that cpu_id field
> stays initialized at -1, so the comparison always fails, as intended.
> 
> This field should only be read by the thread which registered this data
> structure. Aligned on 32-bit.
> 
> It is up to user-space to implement a fall-back mechanism for scenarios where
> rseq is not available.
> .in
> .TP
> .in +4n
> .I cpu_id
> Always-updated value of the CPU number on which the registered thread is
> running. Initialized by user-space to -1, updated by the kernel for
> threads registered with rseq.
> 
> This field should only be read by the thread which registered this data
> structure. Aligned on 32-bit.
> .in
> .TP
> .in +4n
> .I rseq_cs
> The rseq_cs field is a pointer to a struct rseq_cs. Is is NULL when no
> rseq assembly block critical section is active for the registered thread.
> Setting it to point to a critical section descriptor (struct rseq_cs)
> marks the beginning of the critical section.
> 
> Initialized by user-space to NULL.
> 
> Updated by user-space, which sets the address of the currently
> active rseq_cs at the beginning of assembly instruction sequence
> block, and set to NULL by the kernel when it restarts an assembly
> instruction sequence block, as well as when the kernel detects that
> it is preempting or delivering a signal outside of the range
> targeted by the rseq_cs. Also needs to be set to NULL by user-space
> before reclaiming memory that contains the targeted struct rseq_cs.
> 
> Read and set by the kernel.
> 
> This field should only be updated by the thread which registered this
> data structure. Aligned on 64-bit.
> .in
> .TP
> .in +4n
> .I flags
> Flags indicating the restart behavior for the registered thread. This is
> mainly used for debugging purposes. Can be a combination of:
> .IP \[bu]
> RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT: Inhibit instruction sequence block restart
> on preemption for this thread. This flag is deprecated since kernel 6.1.
> .IP \[bu]
> RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL: Inhibit instruction sequence block restart
> on signal delivery for this thread. This flag is deprecated since kernel 6.1.
> .IP \[bu]
> RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE: Inhibit instruction sequence block restart
> on migration for this thread. This flag is deprecated since kernel 6.1.
> 
> Initialized by user-space, used by the kernel.
> .in
> .TP
> .in +4n
> .I node_id
> Always-updated value of the current NUMA node ID.
> 
> Initialized by user-space to 0.
> 
> Updated by the kernel. Read by user-space with single-copy atomicity
> semantics. This field should only be read by the thread which registered
> this data structure. Aligned on 32-bit.
> .in
> .TP
> .in +4n
> .I mm_cid
> Contains the current thread's concurrency ID (allocated uniquely within
> a memory map).
> 
> Updated by the kernel. Read by user-space with single-copy atomicity
> semantics. This field should only be read by the thread which registered
> this data structure. Aligned on 32-bit.
> 
> This concurrency ID is within the possible cpus range, and is
> temporarily (and uniquely) assigned while threads are actively running
> within a memory map. If a memory map has fewer threads than cores, or is
> limited to run on few cores concurrently through sched affinity or
> cgroup cpusets, the concurrency IDs will be values close to 0, thus
> allowing efficient use of user-space memory for per-cpu data structures.
> 
> .PP
> The layout of
> .B struct rseq_cs
> version 0 is as follows:
> .TP
> .B Structure alignment
> This structure is aligned on 32-byte boundary.
> .TP
> .B Structure size
> This structure has a fixed size of 32 bytes.
> .PP
> .in +8n
> .EX
> struct rseq_cs {
>      __u32   version;
>      __u32   flags;
>      __u64   start_ip;
>      __u64   post_commit_offset;
>      __u64   abort_ip;
> } __attribute__((aligned(32)));
> .EE
> .TP
> .B Fields
> 
> .TP
> .in +4n
> .I version
> Version of this structure. Should be initialized to 0.
> .in
> .TP
> .in +4n
> .I flags
> Flags indicating the restart behavior of this structure. Can be a combination
> of:
> .IP \[bu]

This list should be a tagged paragraph instead.  See man-pages(7):

    Lists
        There are different kinds of lists:

        Tagged paragraphs
               These  are used for a list of tags and their descriptions.  When
               the tags are constants (either macros or numbers)  they  are  in
               bold.  Use the .TP macro.

               An example is this "Tagged paragraphs" subsection is itself.

        [...]

        Bullet lists
               Elements  are  preceded by bullet symbols (\(bu).  Anything that
               doesn’t fit elsewhere is usually covered by this type of list.

        [...]


> RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT: Inhibit instruction sequence block restart
> on preemption for this critical section. This flag is deprecated since kernel
> 6.1.
> .IP \[bu]
> RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL: Inhibit instruction sequence block restart
> on signal delivery for this critical section. This flag is deprecated since
> kernel 6.1.
> .IP \[bu]
> RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE: Inhibit instruction sequence block restart
> on migration for this critical section. This flag is deprecated since kernel
> 6.1.
> .TP
> .in +4n
> .I start_ip
> Instruction pointer address of the first instruction of the sequence of
> consecutive assembly instructions.
> .in
> .TP
> .in +4n
> .I post_commit_offset
> Offset (from start_ip address) of the address after the last instruction
> of the sequence of consecutive assembly instructions.
> .in
> .TP
> .in +4n
> .I abort_ip
> Instruction pointer address where to move the execution flow in case of
> abort of the sequence of consecutive assembly instructions.
> .in
> 
> .PP
> The
> .I rseq_len
> argument is the size of the
> .I struct rseq
> to register.
> 
> .PP
> The
> .I flags
> argument is 0 for registration, and
> .IR RSEQ_FLAG_UNREGISTER
> for unregistration.
> 
> .PP
> The
> .I sig
> argument is the 32-bit signature to be expected before the abort
> handler code.
> 
> .PP
> A single library per process should keep the rseq structure in a
> per-thread data structure.
> The
> .I cpu_id
> field should be initialized to -1, and the
> .I cpu_id_start
> field should be initialized to a possible CPU value (typically 0).
> 
> .PP
> Each thread is responsible for registering and unregistering its rseq
> structure. No more than one rseq structure address can be registered
> per thread at a given time.
> 
> .PP
> Reclaim of rseq object's memory must only be done after either an
> explicit rseq unregistration is performed or after the thread exits.
> 
> .PP
> In a typical usage scenario, the thread registering the rseq
> structure will be performing loads and stores from/to that structure. It
> is however also allowed to read that structure from other threads.
> The rseq field updates performed by the kernel provide relaxed atomicity
> semantics (atomic store, without memory ordering), which guarantee that other
> threads performing relaxed atomic reads (atomic load, without memory ordering)
> of the cpu number fields will always observe a consistent value.
> 
> .SH RETURN VALUE
> A return value of 0 indicates success. On error, \-1 is returned, and
> .I errno
> is set appropriately.
> 
> .SH ERRORS
> .TP
> .B EINVAL
> Either
> .I flags
> contains an invalid value, or
> .I rseq
> contains an address which is not appropriately aligned, or
> .I rseq_len
> contains an incorrect size.
> .TP
> .B ENOSYS
> The
> .BR rseq ()
> system call is not implemented by this kernel.
> .TP
> .B EFAULT
> .I rseq
> is an invalid address.

Doesn't this result in a SEGV?  It's trying to access invalid memory.  We had 
some discussion about this in other syscalls, and concluded that that's 
undefined behavior, and a crash is valid behavior (and probably a good thing to 
do), right?  I'm just curious about the view from the kernel point of view.

> .TP
> .B EBUSY
> Restartable sequence is already registered for this thread.
> .TP
> .B EPERM
> The
> .I sig
> argument on unregistration does not match the signature received
> on registration.
> 
> .SH VERSIONS
> The
> .BR rseq ()
> system call was added in Linux 4.18.
> 
> .SH CONFORMING TO

We call that section STANDARDS now.

> .BR rseq ()
> is Linux-specific.
> 
> .in
> .SH SEE ALSO
> .BR sched_getcpu (3) ,
> .BR membarrier (2) ,
> .BR getauxval (3)
> 

Cheers,

Alex

> 

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: rseq(2) man page

[Mathieu Desnoyers]

On 2023-01-06 12:50, Alejandro Colomar wrote:
> Hi Mathieu,
> 
> See some comments below.
> 
> Cheers,
> 
> Alex

[...]

> 
> We now use SPDX-License-Identifier.

OK

> 
>> .\"
>> .TH RSEQ 2 2020-06-05 "Linux" "Linux Programmer's Manual"
> 
> We use lowercase for the function name (or to be more precise, the same 
> case that the function name uses.
> 
> The date is specified with a placeholder that is replaced at the time of 
> creation of the tarball.
> 
> The 5th argument is unspecified.
> 
> The 4th argument is now the project name and a placeholder for the version.
> 
> See an example:
> 
> $ cat man2/membarrier.2 | grep '^.TH'
> .TH membarrier 2 (date) "Linux man-pages (unreleased)"

OK

> 
> 
>> .SH NAME
>> rseq \- Restartable sequences system call
> 
> We use lowercase here, so s/Restartable/restartable/

OK

> 
>> .SH SYNOPSIS
>> .nf
>> .B #include <linux/rseq.h>
> 
> Is there a glibc wrapper for this syscall?  Do you expect that it will 
> be added relatively soon?  Or is it expected that this syscall will be 
> called through syscall(2) for many years?

The system call is only for registering/unregistering the per-thread 
area, so I expect it will only be used through syscall(2), and typically 
only from libc.

User applications/libraries are expected to use the per-thread area to 
communicate with the kernel, but that does not require use of rseq(2).

> 
> If so, it may be better to document it directly as such, like for 
> example membarrier:
> 
> SYNOPSIS
>         #include <linux/membarrier.h> /* Definition of MEMBARRIER_* 
> constants */
>         #include <sys/syscall.h>      /* Definition of SYS_* constants */
>         #include <unistd.h>
> 
>         int syscall(SYS_membarrier, int cmd, unsigned int flags, int 
> cpu_id);
> 
>         Note: glibc provides no wrapper for membarrier(), necessitating 
> the use
>         of syscall(2).
> 

OK

>> .sp
> 
> s/sp/PP/

OK

> 
>> .BI "int rseq(struct rseq * " rseq ", uint32_t " rseq_len ", int " 
>> flags ", 
> 
> Is it valid to pass NULL in 'rseq'?  If it is, we now document that 
> using _Nullable.  See for example recv(2):
> 
> SYNOPSIS
>         #include <sys/socket.h>
> 
>         ssize_t recv(int sockfd, void buf[.len], size_t len,
>                          int flags);
>         ssize_t recvfrom(int sockfd, void buf[restrict .len], size_t len,
>                          int flags,
>                          struct sockaddr *_Nullable restrict src_addr,
>                          socklen_t *_Nullable restrict addrlen);
>         ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
> 

OK

> 
> 
> 
>> uint32_t " sig ");
>> .sp
> 
> .sp is unnecessary here.

OK

> 
> .fi is missing (it's the "closing" pair to .nf).

OK

> 
>> .SH DESCRIPTION
>>
> 
> Use .PP instead of blank lines.

OK

> 
>> The
>> .BR rseq ()
>> ABI accelerates specific user-space operations by registering a
>> per-thread data structure shared between kernel and user-space. This
> 
> Use semantic newlines.  See man-pages(7):
> 
>     Use semantic newlines
>         In  the source of a manual page, new sentences should be started 
> on new
>         lines, long sentences should be split into lines at clause 
> breaks (com‐
>         mas, semicolons, colons, and so on), and long clauses should  
> be  split
>         at  phrase  boundaries.   This convention, sometimes known as 
> "semantic
>         newlines", makes it easier to see the effect of  patches,  
> which  often
>         operate at the level of individual sentences, clauses, or phrases.
> 

OK, done across the entire the manual page source.

[...]

>> .PP
>> Some examples of operations that can be accelerated or improved
>> by this ABI:
>> .IP \[bu] 2
> 
> Use 3 instead of 2.  See man-pages:
> 
>     Lists
>         There are different kinds of lists:
> 
>         [...]
> 
>         Bullet lists
>                Elements  are  preceded by bullet symbols (\(bu).  
> Anything that
>                doesn’t fit elsewhere is usually covered by this type of 
> list.
> 
>         [...]
> 
>         There should always be exactly 2 spaces between the list symbol 
> and the
>         elements.  This doesn’t apply to "tagged paragraphs", which use 
> the de‐
>         fault indentation rules.
> 
> 
> The rationale for that was that if you use 1 space, then the list 
> introducer can be confused with the list contents.  Two spaces makes the 
> difference more clear.

OK

> 
> 
> Also, we use \(bu instead of \[bu].  I'm not particularly worried by 
> using it, but I prefer being consistent at which one we use.

OK

[...]

>>
>> .TP
>> .in +4n
> 
> I guess you're looking for .RS/.RE.  It would wrap all the indented 
> stuff, replacing .in.

OK.


> 
>> .I cpu_id_start
>> Always-updated value of the CPU number on which the registered thread is
>> running. Its value is guaranteed to always be a possible CPU number,
>> even when rseq is not registered. Its value should always be confirmed by
> 
> rseq (and maybe other cases around too) should be formatted in italics, 
> since it's a variable name (.I).

"rseq" here does not refer to a variable name, but rather to the system 
call. Should it be formatted in italics ? I format it as ".BR" elsewhere 
in the man page.

I'll push a first update without the italics changes, please let me know 
if there are still issues left.

[...]

>> .in +4n
>> .I flags
>> Flags indicating the restart behavior of this structure. Can be a 
>> combination
>> of:
>> .IP \[bu]
> 
> This list should be a tagged paragraph instead.  See man-pages(7):
> 
>     Lists
>         There are different kinds of lists:
> 
>         Tagged paragraphs
>                These  are used for a list of tags and their 
> descriptions.  When
>                the tags are constants (either macros or numbers)  they  
> are  in
>                bold.  Use the .TP macro.
> 
>                An example is this "Tagged paragraphs" subsection is itself.
> 
>         [...]
> 
>         Bullet lists
>                Elements  are  preceded by bullet symbols (\(bu).  
> Anything that
>                doesn’t fit elsewhere is usually covered by this type of 
> list.
> 
>         [...]

OK

[...]

>> .SH RETURN VALUE
>> A return value of 0 indicates success. On error, \-1 is returned, and
>> .I errno
>> is set appropriately.
>>
>> .SH ERRORS
>> .TP
>> .B EINVAL
>> Either
>> .I flags
>> contains an invalid value, or
>> .I rseq
>> contains an address which is not appropriately aligned, or
>> .I rseq_len
>> contains an incorrect size.
>> .TP
>> .B ENOSYS
>> The
>> .BR rseq ()
>> system call is not implemented by this kernel.
>> .TP
>> .B EFAULT
>> .I rseq
>> is an invalid address.
> 
> Doesn't this result in a SEGV?  It's trying to access invalid memory.  
> We had some discussion about this in other syscalls, and concluded that 
> that's undefined behavior, and a crash is valid behavior (and probably a 
> good thing to do), right?  I'm just curious about the view from the 
> kernel point of view.

If the registered rseq pointer / size points to invalid memory on rseq 
registration, the rseq registration system call will return -1, 
errno=EFAULT. If at some point _after_ registration the mapping becomes 
invalid (e.g. unmapped without prior unregistration), then a SIGSEGV can 
be triggered.

I was not aware of this discussion a regarding returning EFAULT errno vs 
SIGSEGV. If this is becoming a consensus across system calls to segfault 
  rather than return EFAULT errno, I'm open to improve sys_rseq accordingly.

> 
>> .TP
>> .B EBUSY
>> Restartable sequence is already registered for this thread.
>> .TP
>> .B EPERM
>> The
>> .I sig
>> argument on unregistration does not match the signature received
>> on registration.
>>
>> .SH VERSIONS
>> The
>> .BR rseq ()
>> system call was added in Linux 4.18.
>>
>> .SH CONFORMING TO
> 
> We call that section STANDARDS now.

OK

I've done a first pass applying all the relevant changes. I've pushed 
the changes into the librseq master branch. I may have omitted 
bold/italic for some identifiers. If it's the case, please let me know.

Thanks!

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: rseq(2) man page

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 22308 bytes --]

Hi Mathieu,

On 1/6/23 21:57, Mathieu Desnoyers wrote:
> On 2023-01-06 12:50, Alejandro Colomar wrote:
>>> .I cpu_id_start
>>> Always-updated value of the CPU number on which the registered thread is
>>> running. Its value is guaranteed to always be a possible CPU number,
>>> even when rseq is not registered. Its value should always be confirmed by
>>
>> rseq (and maybe other cases around too) should be formatted in italics, since 
>> it's a variable name (.I).
> 
> "rseq" here does not refer to a variable name, but rather to the system call. 
> Should it be formatted in italics ? I format it as ".BR" elsewhere in the man page.

For referring to a function (or syscall), we use a bold name and non-bold 
(roman) trailing parens; empty if it's defined in the same page.

So, it would be:

.BR rseq ()

>>> .TP
>>> .B EFAULT
>>> .I rseq
>>> is an invalid address.
>>
>> Doesn't this result in a SEGV?  It's trying to access invalid memory. We had 
>> some discussion about this in other syscalls, and concluded that that's 
>> undefined behavior, and a crash is valid behavior (and probably a good thing 
>> to do), right?  I'm just curious about the view from the kernel point of view.
> 
> If the registered rseq pointer / size points to invalid memory on rseq 
> registration, the rseq registration system call will return -1, errno=EFAULT. If 
> at some point _after_ registration the mapping becomes invalid (e.g. unmapped 
> without prior unregistration), then a SIGSEGV can be triggered.
> 
> I was not aware of this discussion a regarding returning EFAULT errno vs 
> SIGSEGV. If this is becoming a consensus across system calls to segfault  rather 
> than return EFAULT errno, I'm open to improve sys_rseq accordingly.

The discussion was in libc-alpha:

<https://inbox.sourceware.org/libc-alpha/b55075c8-e55b-e492-7f4f-d6feb0ee2432@gmail.com/T/#u>

The reason is that, since holding a pointer to invalid memory is already 
undefined behavior (the standard defines that as overflow[1]), passing it to a 
syscall necessarily implies that you hold such a pointer, so UB was invoked 
prior to the call, and thus the call is already within UB.

[1]:  <https://port70.net/%7Ensz/c/c11/n1570.html#6.5.6p8>

However, that discussion was in glibc; I don't know what kernel people might 
think of that.  If you discuss that in the kernel, please link to that glibc 
thread, and CC the people from that thread.


Now I'll add some comments for v2.

Cheers,

Alex


--- v2:

> '\" t

This is a comment for pages that use tables (tbl(1)), so that man(1) knows if it 
needs to run tbl(1) before roff(1).  It seems that your page doesn't use tbl(1) 
features, so you should remove it.

> .\" Copyright 2015-2023 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> .\"
> .\" SPDX-License-Identifier: Linux-man-pages-copyleft
> .\"
> .TH rseq 2 (date) "Linux man-pages (unreleased)"
> .SH NAME
> rseq \- restartable sequences system call

I forgot; there's a new section: LIBRARY.

It specifies the library in which the function is defined, or in the case of 
syscalls, the wrapper (since we call it through syscall(2), it would be libc).

BTW, I wonder what librseq is.  Is librseq something that users should care about?

> .SH SYNOPSIS
> .nf
> .PP
> .BR "#include <linux/rseq.h>" \
> " /* Definition of " RSEQ_* " constants and rseq types */"

The line above goes beyond column 80 in formatted output.  That's a hard limit 
for manual pages.  If you add this page to the linux man-pages repo, and run the 
linters, you'll see a warning about that.  In case you're interested in linting 
manual pages in the future, you can do something similar to what I do in the 
man-pages[2]

I decided to not include types in the comments.  The reason, apart from having 
shorter comments, is that the man-pages now document types in their own pages in 
subsection 3type.

Since I guess you don't want to go so far to write a new manual page for the 
types (but if you want, feel free), since those types are specific to this call, 
what you can do is specify the include that provides the type, in the same piece 
of code where you document the type.

> .BR "#include #include <sys/syscall.h>" "	* Definition of " SYS_* " constants */"

The comment above seems broken.  See '/*'.

> .B #include <unistd.h>
> .PP
> .BI "int syscall(SYS_rseq, struct rseq *_Nullable " rseq ", uint32_t " rseq_len \

What's the meaning for NULL?  Does it have a valid sentinel meaning, or is it an 
invalid address?  If it's just interpreted as an invalid address (for which from 
a user-space perspective a crash would be legitimate), then I'd remove _Nullable.

> ", int " flags ", uint32_t " sig ");

The last '"' is unmatched.

Also, this goes beyond 80-col.  Please cut after rseq_len (my mailer will 
probably put the comma in a separate line; it should go at the end of the line, 
preceeded by a space):

.BI "int syscall(SYS_rseq, struct rseq *_Nullable " rseq ", uint32_t " rseq_len ,
.BI "            int " flags ", uint32_t " sig );


> .fi
> .PP
> .IR Note :
> glibc provides no wrapper for
> .BR rseq (),
> necessitating the use of
> .BR syscall (2).
> .SH DESCRIPTION
> .PP

.PP after .SH or .SS is redundant, and should be removed.

> The
> .BR rseq ()
> ABI accelerates specific user-space operations by registering a
> per-thread data structure shared between kernel and user-space.
> This data structure can be read from or written to by user-space to skip
> otherwise expensive system calls.
> .PP
> A restartable sequence is a sequence of instructions guaranteed to be executed
> atomically with respect to other threads and signal handlers on the current
> CPU.

I'd break after "instructions", and after "respect to".

> If its execution does not complete atomically, the kernel changes the

Breaking after commas is strongly preferred.

> execution flow by jumping to an abort handler defined by user-space for
> that restartable sequence.
> .PP
> Using restartable sequences requires to register a
> rseq ABI per-thread data structure (

This produces a space after the '('.

> .B struct rseq

Instead, use:

.RB ( "struct rseq" )

> ) through the
> .BR rseq ()
> system call.
> Only one rseq ABI can be registered per thread, so user-space libraries
> and applications must follow a user-space ABI defining how to share this
> resource.
> The ABI defining how to share this resource between applications and
> libraries is defined by the C library.
> Allocation of the per-thread rseq ABI and its registration to the kernel
> is handled by glibc since version 2.35.
> .PP
> The rseq ABI per-thread data structure contains a
> .I rseq_cs
> field which points to the currently executing critical section.
> For each thread, a single rseq critical section can run at any given
> point.
> Each critical section need to be implemented in assembly.
> .PP
> The
> .BR rseq ()
> ABI accelerates user-space operations on per-cpu data by defining a
> shared data structure ABI between each user-space thread and the kernel.
> .PP
> It allows user-space to perform update operations on per-cpu data
> without requiring heavy-weight atomic operations.
> .PP
> The term CPU used in this documentation refers to a hardware execution
> context.
> For instance, each CPU number returned by
> .BR sched_getcpu ()
> is a CPU.
> The current CPU means to the CPU on which the registered thread is
> running.
> .PP
> Restartable sequences are atomic with respect to preemption (making it
> atomic with respect to other threads running on the same CPU),
> as well as signal delivery (user-space execution contexts nested over
> the same thread).
> They either complete atomically with respect to preemption on the
> current CPU and signal delivery, or they are aborted.
> .PP
> Restartable sequences are suited for update operations on per-cpu data.
> .PP
> Restartable sequences can be used on data structures shared between threads
> within a process,
> and on data structures shared between threads across different
> processes.
> .PP
> Some examples of operations that can be accelerated or improved by this ABI:
> .IP \(bu 3
> Memory allocator per-cpu free-lists,
> .IP \(bu 3
> Querying the current CPU number,
> .IP \(bu 3
> Incrementing per-CPU counters,
> .IP \(bu 3
> Modifying data protected by per-CPU spinlocks,
> .IP \(bu 3
> Inserting/removing elements in per-CPU linked-lists,
> .IP \(bu 3
> Writing/reading per-CPU ring buffers content.
> .IP \(bu 3
> Accurately reading performance monitoring unit counters with respect to
> thread migration.
> .PP
> Restartable sequences must not perform system calls.
> Doing so may result in termination of the process by a segmentation
> fault.
> .PP
> The
> .I rseq
> argument is a pointer to the thread-local rseq structure to be shared
> between kernel and user-space.
> .PP
> The structure
> .B struct rseq
> is an extensible structure.
> Additional feature fields can be added in future kernel versions.
> Its layout is as follows:
> .TP
> .B Structure alignment
> This structure is aligned on either 32-byte boundary,
> or on the alignment value returned by
> .I getauxval(
> .B AT_RSEQ_ALIGN
> )
> if the structure size differs from 32 bytes.
> .TP
> .B Structure size
> This structure size needs to be at least 32 bytes.
> It can be either 32 bytes,
> or it needs to be large enough to hold the result of
> .I getauxval(
> .B AT_RSEQ_FEATURE_SIZE
> ) .
> Its size is passed as parameter to the rseq system call.
> .RS

For code examples we use .in +4n rather than .RS.  I don't remember the exact 
reason, but it had some glitches in some cases.

> .PP

This should be .IP, since it's a continuation of the tagged paragraph (TP).  IP 
continues the indentation, while PP start a clean paragraph that is not indented 
as the previous one.

> .EX

You could add the #include <linux/rseq.h> here.

> struct rseq {
>     __u32 cpu_id_start;
>     __u32 cpu_id;
>     union {
>         /* Edited out for conciseness. [...] */

You can simply say /* ... */

We use that in a few other places.

>     } rseq_cs;
>     __u32 flags;
>     __u32 node_id;
>     __u32 mm_cid;
> } __attribute__((aligned(32)));
> .EE
> .RE
> .TP
> .B Fields
> .RS

Here there should be another .TP

> .I cpu_id_start
> .RS

And that TP would make this RS unnecessary, since it would indent the paragraph.

> Always-updated value of the CPU number on which the registered thread is
> running.
> Its value is guaranteed to always be a possible CPU number,
> even when rseq is not registered.
> Its value should always be confirmed by reading the cpu_id field before
> user-space performs any side-effect
> (e.g. storing to memory).
> .PP

This should be a IP, to continue the indentation of the previous TP.

> This field is always guaranteed to hold a valid CPU number in the range
> [ 0 ..  nr_possible_cpus - 1 ].
> It can therefore be loaded by user-space and used as an offset in
> per-cpu data structures without having to check whether its value is
> within the valid bounds compared to the number of possible CPUs in the
> system.

Having a word alone in a line is usually a hint that there exists a better point 
to break the sentence (except where .I or .B ask for it).

> .PP
> Initialized by user-space to a possible CPU number (e.g., 0),
> updated by the kernel for threads registered with rseq.

rseq shoudl be formatted.

> .PP
> For user-space applications executed on a kernel without rseq support,
> the cpu_id_start field stays initialized at 0, which is indeed a valid
> CPU number.
> It is therefore valid to use it as an offset in per-cpu data structures,
> and only validate whether it's actually the current CPU number by
> comparing it with the cpu_id field within the rseq critical section.
> If the kernel does not provide rseq support, that cpu_id field stays
> initialized at -1,
> so the comparison always fails, as intended.
> .PP
> This field should only be read by the thread which registered this data
> structure.
> Aligned on 32-bit.
> .PP
> It is up to user-space to implement a fall-back mechanism for scenarios where

s/user-space/user space/

The - is used as in natural English, for form an adjective.  Since here it acts 
as a subject, it goes with a space.

> rseq is not available.
> .RE
> .PP
> .I cpu_id
> .RS
> Always-updated value of the CPU number on which the registered thread is
> running.
> Initialized by user-space to -1,
> updated by the kernel for threads registered with rseq.
> .PP
> This field should only be read by the thread which registered this data
> structure.
> Aligned on 32-bit.
> .RE
> .PP
> .I rseq_cs
> .RS
> The rseq_cs field is a pointer to a
> .B struct rseq_cs .
> Is is NULL when no rseq assembly block critical section is active for
> the registered thread.
> Setting it to point to a critical section descriptor (
> .B struct rseq_cs
> ) marks the beginning of the critical section.
> .PP
> Initialized by user-space to NULL.
> .PP
> Updated by user-space, which sets the address of the currently
> active rseq_cs at the beginning of assembly instruction sequence
> block,
> and set to NULL by the kernel when it restarts an assembly instruction
> sequence block,
> as well as when the kernel detects that it is preempting or delivering a
> signal outside of the range targeted by the rseq_cs.
> Also needs to be set to NULL by user-space before reclaiming memory that
> contains the targeted
> .B struct rseq_cs .
> .PP
> Read and set by the kernel.
> .PP
> This field should only be updated by the thread which registered this
> data structure.
> Aligned on 64-bit.
> .RE
> .PP
> .I flags
> .RS
> Flags indicating the restart behavior for the registered thread.
> This is mainly used for debugging purposes.
> Can be a combination of:
> .TP
> .B RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT
> Inhibit instruction sequence block restart on preemption for this
> thread.
> This flag is deprecated since kernel 6.1.

I did some consistency fixes regarding linux versions.  Now we always use "Linux 
x.y", rather than "Linux kernel x.y", or "kernel x.y".  That helps grepping.

> .TP
> .B RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL
> Inhibit instruction sequence block restart on signal delivery for this
> thread.
> This flag is deprecated since kernel 6.1.
> .TP
> .B RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE
> Inhibit instruction sequence block restart on migration for this thread.
> This flag is deprecated since kernel 6.1.
> .PP
> Initialized by user-space, used by the kernel.
> .RE
> .PP
> .I node_id
> .RS
> Always-updated value of the current NUMA node ID.
> .PP
> Initialized by user-space to 0.
> .PP
> Updated by the kernel.
> Read by user-space with single-copy atomicity semantics.
> This field should only be read by the thread which registered
> this data structure.
> Aligned on 32-bit.
> .RE
> .PP
> .I mm_cid
> .RS
> Contains the current thread's concurrency ID
> (allocated uniquely within a memory map).
> .PP
> Updated by the kernel.
> Read by user-space with single-copy atomicity semantics.
> This field should only be read by the thread which registered this data
> structure.
> Aligned on 32-bit.
> .PP
> This concurrency ID is within the possible cpus range,
> and is temporarily (and uniquely) assigned while threads are actively
> running within a memory map.
> If a memory map has fewer threads than cores,
> or is limited to run on few cores concurrently through sched affinity or
> cgroup cpusets,
> the concurrency IDs will be values close to 0,
> thus allowing efficient use of user-space memory for per-cpu data
> structures.
> .RE
> .RE
> .RE
> .PP
> The layout of
> .B struct rseq_cs
> version 0 is as follows:
> .TP
> .B Structure alignment
> This structure is aligned on 32-byte boundary.
> .TP
> .B Structure size
> This structure has a fixed size of 32 bytes.
> .RS
> .EX
> struct rseq_cs {
>     __u32   version;
>     __u32   flags;
>     __u64   start_ip;
>     __u64   post_commit_offset;
>     __u64   abort_ip;
> } __attribute__((aligned(32)));
> .EE
> .RE
> .PP
> .B Fields
> .RS
> .I version
> .RS
> Version of this structure.
> Should be initialized to 0.
> .RE
> .PP
> .I flags
> .RS
> Flags indicating the restart behavior of this structure.
> Can be a combination of:
> .TP
> .B RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT
> Inhibit instruction sequence block restart on preemption for this
> critical section.
> This flag is deprecated since kernel 6.1.
> .TP
> .B RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL
> Inhibit instruction sequence block restart on signal delivery for this
> critical section.
> This flag is deprecated since kernel 6.1.
> .TP
> .B RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE
> Inhibit instruction sequence block restart on migration for this
> critical section.
> This flag is deprecated since kernel 6.1.
> .RE
> .PP
> .I start_ip
> .RS
> Instruction pointer address of the first instruction of the sequence of
> consecutive assembly instructions.
> .RE
> .PP
> .I post_commit_offset
> .RS
> Offset (from start_ip address) of the address after the last instruction
> of the sequence of consecutive assembly instructions.
> .RE
> .PP
> .I abort_ip
> .RS
> Instruction pointer address where to move the execution flow in case of
> abort of the sequence of consecutive assembly instructions.
> .RE
> .RE
> .PP
> The
> .I rseq_len
> argument is the size of the
> .B struct rseq
> to register.
> .PP
> The
> .I flags
> argument is 0 for registration, and
> .B RSEQ_FLAG_UNREGISTER
> for unregistration.
> .PP
> The
> .I sig
> argument is the 32-bit signature to be expected before the abort
> handler code.
> .PP
> A single library per process should keep the rseq structure in a
> per-thread data structure.
> The
> .I cpu_id
> field should be initialized to -1, and the
> .I cpu_id_start
> field should be initialized to a possible CPU value (typically 0).
> .PP
> Each thread is responsible for registering and unregistering its rseq
> structure.
> No more than one rseq structure address can be registered per thread at
> a given time.
> .PP
> Reclaim of rseq object's memory must only be done after either an
> explicit rseq unregistration is performed or after the thread exits.
> .PP
> In a typical usage scenario, the thread registering the rseq
> structure will be performing loads and stores from/to that structure.
> It is however also allowed to read that structure from other threads.
> The rseq field updates performed by the kernel provide relaxed atomicity
> semantics (atomic store, without memory ordering),
> which guarantee that other threads performing relaxed atomic reads
> (atomic load, without memory ordering) of the cpu number fields will
> always observe a consistent value.
> .PP
> .SH RETURN VALUE
> A return value of 0 indicates success.
> On error, \-1 is returned, and
> .I errno
> is set appropriately.
> .PP
> .SH ERRORS
> .TP
> .B EINVAL
> Either
> .I flags
> contains an invalid value, or
> .I rseq
> contains an address which is not appropriately aligned, or
> .I rseq_len
> contains an incorrect size.
> .TP
> .B ENOSYS
> The
> .BR rseq ()
> system call is not implemented by this kernel.
> .TP
> .B EFAULT
> .I rseq
> is an invalid address.
> .TP
> .B EBUSY
> Restartable sequence is already registered for this thread.
> .TP
> .B EPERM
> The
> .I sig
> argument on unregistration does not match the signature received
> on registration.
> .PP
> .SH VERSIONS
> The
> .BR rseq ()
> system call was added in Linux 4.18.
> .PP
> .SH STANDARDS
> .BR rseq ()
> is Linux-specific.
> .PP
> .SH SEE ALSO
> .BR sched_getcpu (3) ,
> .BR membarrier (2) ,
> .BR getauxval (3)



[2]:  (linting manual pages)


If you add a new manual page to the man-pages repo, it will automatically be 
included in the build system, so you'll be able to lint it, as described in the 
CONTRIBUTING file (tl;dr: `make lint`).  But if you want to add something 
similar to your build system, you can inspect the following commands being run:


alx@asus5775:~/src/linux/man-pages/man-pages/main$ make lint -ij >/dev/null 
2>/dev/null
alx@asus5775:~/src/linux/man-pages/man-pages/main$ touch man2/rseq.2
alx@asus5775:~/src/linux/man-pages/man-pages/main$ make lint -k
LINT (groff)	tmp/lint/man2/rseq.2.lint-man.groff.touch
an.tmac:man2/rseq.2:297: style: unbalanced .RE
found style problems; aborting
man2/rseq.2:7:       #include <linux/rseq.h> /* Definition of RSEQ_* constants 
and rseq types */
man2/rseq.2:11:       int syscall(SYS_rseq, struct rseq *_Nullable rseq, 
uint32_t rseq_len, int flags, uint32_t sig);
make: *** [lib/lint-man.mk:78: tmp/lint/man2/rseq.2.lint-man.groff.touch] Error 1
LINT (mandoc)	tmp/lint/man2/rseq.2.lint-man.mandoc.touch
mandoc: man2/rseq.2:18:114: STYLE: unterminated quoted argument
mandoc: man2/rseq.2:297:2: ERROR: skipping end of block that is not open: RE
mandoc: man2/rseq.2:409:2: WARNING: skipping paragraph macro: PP empty
mandoc: man2/rseq.2:27:2: WARNING: skipping paragraph macro: PP after SH
mandoc: man2/rseq.2:415:2: WARNING: skipping paragraph macro: PP empty
mandoc: man2/rseq.2:444:2: WARNING: skipping paragraph macro: PP empty
mandoc: man2/rseq.2:449:2: WARNING: skipping paragraph macro: PP empty
mandoc: man2/rseq.2:453:2: WARNING: skipping paragraph macro: PP empty
make: *** [lib/lint-man.mk:88: tmp/lint/man2/rseq.2.lint-man.mandoc.touch] Error 1
LINT (tbl)	tmp/lint/man2/rseq.2.lint-man.tbl.touch
man2/rseq.2:1: spurious '\" t' comment:
'\" t
make: *** [lib/lint-man.mk:101: tmp/lint/man2/rseq.2.lint-man.tbl.touch] Error 1
make: Target 'lint' not remade because of errors.


If you want to see the commands being run, you can use `make lint V=1`.  Feel 
free to incorporate any of that to your Makefile.


-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: rseq(2) man page

[Mathieu Desnoyers]

On 2023-01-06 17:57, Alejandro Colomar wrote:
> Hi Mathieu,
> 
> On 1/6/23 21:57, Mathieu Desnoyers wrote:
>> On 2023-01-06 12:50, Alejandro Colomar wrote:
>>>> .I cpu_id_start
>>>> Always-updated value of the CPU number on which the registered 
>>>> thread is
>>>> running. Its value is guaranteed to always be a possible CPU number,
>>>> even when rseq is not registered. Its value should always be 
>>>> confirmed by
>>>
>>> rseq (and maybe other cases around too) should be formatted in 
>>> italics, since it's a variable name (.I).
>>
>> "rseq" here does not refer to a variable name, but rather to the 
>> system call. Should it be formatted in italics ? I format it as ".BR" 
>> elsewhere in the man page.
> 
> For referring to a function (or syscall), we use a bold name and 
> non-bold (roman) trailing parens; empty if it's defined in the same page.
> 
> So, it would be:
> 
> .BR rseq ()

OK

> 
>>>> .TP
>>>> .B EFAULT
>>>> .I rseq
>>>> is an invalid address.
>>>
>>> Doesn't this result in a SEGV?  It's trying to access invalid memory. 
>>> We had some discussion about this in other syscalls, and concluded 
>>> that that's undefined behavior, and a crash is valid behavior (and 
>>> probably a good thing to do), right?  I'm just curious about the view 
>>> from the kernel point of view.
>>
>> If the registered rseq pointer / size points to invalid memory on rseq 
>> registration, the rseq registration system call will return -1, 
>> errno=EFAULT. If at some point _after_ registration the mapping 
>> becomes invalid (e.g. unmapped without prior unregistration), then a 
>> SIGSEGV can be triggered.
>>
>> I was not aware of this discussion a regarding returning EFAULT errno 
>> vs SIGSEGV. If this is becoming a consensus across system calls to 
>> segfault  rather than return EFAULT errno, I'm open to improve 
>> sys_rseq accordingly.
> 
> The discussion was in libc-alpha:
> 
> <https://inbox.sourceware.org/libc-alpha/b55075c8-e55b-e492-7f4f-d6feb0ee2432@gmail.com/T/#u>
> 
> The reason is that, since holding a pointer to invalid memory is already 
> undefined behavior (the standard defines that as overflow[1]), passing 
> it to a syscall necessarily implies that you hold such a pointer, so UB 
> was invoked prior to the call, and thus the call is already within UB.
> 
> [1]:  <https://port70.net/%7Ensz/c/c11/n1570.html#6.5.6p8>
> 
> However, that discussion was in glibc; I don't know what kernel people 
> might think of that.  If you discuss that in the kernel, please link to 
> that glibc thread, and CC the people from that thread.

OK, let's keep this for a separate discussion.

> 
> 
> Now I'll add some comments for v2.
> 
> Cheers,
> 
> Alex
> 
> 
> --- v2:
> 
>> '\" t
> 
> This is a comment for pages that use tables (tbl(1)), so that man(1) 
> knows if it needs to run tbl(1) before roff(1).  It seems that your page 
> doesn't use tbl(1) features, so you should remove it.
> 
>> .\" Copyright 2015-2023 Mathieu Desnoyers 
>> <mathieu.desnoyers@efficios.com>
>> .\"
>> .\" SPDX-License-Identifier: Linux-man-pages-copyleft
>> .\"
>> .TH rseq 2 (date) "Linux man-pages (unreleased)"
>> .SH NAME
>> rseq \- restartable sequences system call
> 
> I forgot; there's a new section: LIBRARY.
> 
> It specifies the library in which the function is defined, or in the 
> case of syscalls, the wrapper (since we call it through syscall(2), it 
> would be libc).

OK. Similar to futex(2).

> 
> BTW, I wonder what librseq is.  Is librseq something that users should 
> care about?

Users are not required to use librseq to use the rseq system call,
but it's very convenient to use this C-level API rather than have
each user reimplement the per-architecture assembly code required to
create the RSEQ critical sections.

librseq did not have an official release yet, so that's mainly why I
think it too early to refer to it in manual pages.

> 
>> .SH SYNOPSIS
>> .nf
>> .PP
>> .BR "#include <linux/rseq.h>" \
>> " /* Definition of " RSEQ_* " constants and rseq types */"
> 
> The line above goes beyond column 80 in formatted output.  That's a hard 
> limit for manual pages.  If you add this page to the linux man-pages 
> repo, and run the linters, you'll see a warning about that.  In case 
> you're interested in linting manual pages in the future, you can do 
> something similar to what I do in the man-pages[2]

OK. I've used it, it's quite useful! I have fixed all warnings except for
"mandoc: man2/rseq.2:5:12: WARNING: cannot parse date, using it verbatim: (date)"
which I suspect is expected.

> 
> I decided to not include types in the comments.  The reason, apart from 
> having shorter comments, is that the man-pages now document types in 
> their own pages in subsection 3type.
> 
> Since I guess you don't want to go so far to write a new manual page for 
> the types (but if you want, feel free), since those types are specific 
> to this call, what you can do is specify the include that provides the 
> type, in the same piece of code where you document the type.
> 
>> .BR "#include #include <sys/syscall.h>" "    * Definition of " SYS_* " 
>> constants */"
> 
> The comment above seems broken.  See '/*'.

OK

> 
>> .B #include <unistd.h>
>> .PP
>> .BI "int syscall(SYS_rseq, struct rseq *_Nullable " rseq ", uint32_t " 
>> rseq_len \
> 
> What's the meaning for NULL?  Does it have a valid sentinel meaning, or 
> is it an invalid address?  If it's just interpreted as an invalid 
> address (for which from a user-space perspective a crash would be 
> legitimate), then I'd remove _Nullable.

With the flags that are currently implemented (0 or RSEQ_FLAG_UNREGISTER),
the rseq argument is not expected to be legitimately NULL (it would return
-1, errno=EFAULT on registration, or -1, errno=EINVAL on unregister attempt).

We may add new flags in the future which would not care about the rseq address
(it could very well be null then). Do you recommend that we only add the
_Nullable tag when this occurs ?

> 
>> ", int " flags ", uint32_t " sig ");
> 
> The last '"' is unmatched.
> 
> Also, this goes beyond 80-col.  Please cut after rseq_len (my mailer 
> will probably put the comma in a separate line; it should go at the end 
> of the line, preceeded by a space):
> 
> .BI "int syscall(SYS_rseq, struct rseq *_Nullable " rseq ", uint32_t " 
> rseq_len ,
> .BI "            int " flags ", uint32_t " sig );

OK

> 
> 
>> .fi
>> .PP
>> .IR Note :
>> glibc provides no wrapper for
>> .BR rseq (),
>> necessitating the use of
>> .BR syscall (2).
>> .SH DESCRIPTION
>> .PP
> 
> .PP after .SH or .SS is redundant, and should be removed.

OK

> 
>> The
>> .BR rseq ()
>> ABI accelerates specific user-space operations by registering a
>> per-thread data structure shared between kernel and user-space.
>> This data structure can be read from or written to by user-space to skip
>> otherwise expensive system calls.
>> .PP
>> A restartable sequence is a sequence of instructions guaranteed to be 
>> executed
>> atomically with respect to other threads and signal handlers on the 
>> current
>> CPU.
> 
> I'd break after "instructions", and after "respect to".

OK

> 
>> If its execution does not complete atomically, the kernel changes the
> 
> Breaking after commas is strongly preferred.

OK

> 
>> execution flow by jumping to an abort handler defined by user-space for
>> that restartable sequence.
>> .PP
>> Using restartable sequences requires to register a
>> rseq ABI per-thread data structure (
> 
> This produces a space after the '('.
> 
>> .B struct rseq
> 
> Instead, use:
> 
> .RB ( "struct rseq" )

OK

> 
>> ) through the
>> .BR rseq ()
>> system call.
>> Only one rseq ABI can be registered per thread, so user-space libraries
>> and applications must follow a user-space ABI defining how to share this
>> resource.
>> The ABI defining how to share this resource between applications and
>> libraries is defined by the C library.
>> Allocation of the per-thread rseq ABI and its registration to the kernel
>> is handled by glibc since version 2.35.
>> .PP
>> The rseq ABI per-thread data structure contains a
>> .I rseq_cs
>> field which points to the currently executing critical section.
>> For each thread, a single rseq critical section can run at any given
>> point.
>> Each critical section need to be implemented in assembly.
>> .PP
>> The
>> .BR rseq ()
>> ABI accelerates user-space operations on per-cpu data by defining a
>> shared data structure ABI between each user-space thread and the kernel.
>> .PP
>> It allows user-space to perform update operations on per-cpu data
>> without requiring heavy-weight atomic operations.
>> .PP
>> The term CPU used in this documentation refers to a hardware execution
>> context.
>> For instance, each CPU number returned by
>> .BR sched_getcpu ()
>> is a CPU.
>> The current CPU means to the CPU on which the registered thread is
>> running.
>> .PP
>> Restartable sequences are atomic with respect to preemption (making it
>> atomic with respect to other threads running on the same CPU),
>> as well as signal delivery (user-space execution contexts nested over
>> the same thread).
>> They either complete atomically with respect to preemption on the
>> current CPU and signal delivery, or they are aborted.
>> .PP
>> Restartable sequences are suited for update operations on per-cpu data.
>> .PP
>> Restartable sequences can be used on data structures shared between 
>> threads
>> within a process,
>> and on data structures shared between threads across different
>> processes.
>> .PP
>> Some examples of operations that can be accelerated or improved by 
>> this ABI:
>> .IP \(bu 3
>> Memory allocator per-cpu free-lists,
>> .IP \(bu 3
>> Querying the current CPU number,
>> .IP \(bu 3
>> Incrementing per-CPU counters,
>> .IP \(bu 3
>> Modifying data protected by per-CPU spinlocks,
>> .IP \(bu 3
>> Inserting/removing elements in per-CPU linked-lists,
>> .IP \(bu 3
>> Writing/reading per-CPU ring buffers content.
>> .IP \(bu 3
>> Accurately reading performance monitoring unit counters with respect to
>> thread migration.
>> .PP
>> Restartable sequences must not perform system calls.
>> Doing so may result in termination of the process by a segmentation
>> fault.
>> .PP
>> The
>> .I rseq
>> argument is a pointer to the thread-local rseq structure to be shared
>> between kernel and user-space.
>> .PP
>> The structure
>> .B struct rseq
>> is an extensible structure.
>> Additional feature fields can be added in future kernel versions.
>> Its layout is as follows:
>> .TP
>> .B Structure alignment
>> This structure is aligned on either 32-byte boundary,
>> or on the alignment value returned by
>> .I getauxval(
>> .B AT_RSEQ_ALIGN
>> )
>> if the structure size differs from 32 bytes.
>> .TP
>> .B Structure size
>> This structure size needs to be at least 32 bytes.
>> It can be either 32 bytes,
>> or it needs to be large enough to hold the result of
>> .I getauxval(
>> .B AT_RSEQ_FEATURE_SIZE
>> ) .
>> Its size is passed as parameter to the rseq system call.
>> .RS
> 
> For code examples we use .in +4n rather than .RS.  I don't remember the 
> exact reason, but it had some glitches in some cases.
OK

> 
>> .PP
> 
> This should be .IP, since it's a continuation of the tagged paragraph 
> (TP).  IP continues the indentation, while PP start a clean paragraph 
> that is not indented as the previous one.

OK

> 
>> .EX
> 
> You could add the #include <linux/rseq.h> here.

OK

> 
>> struct rseq {
>>     __u32 cpu_id_start;
>>     __u32 cpu_id;
>>     union {
>>         /* Edited out for conciseness. [...] */
> 
> You can simply say /* ... */
> 
> We use that in a few other places.

OK

> 
>>     } rseq_cs;
>>     __u32 flags;
>>     __u32 node_id;
>>     __u32 mm_cid;
>> } __attribute__((aligned(32)));
>> .EE
>> .RE
>> .TP
>> .B Fields
>> .RS
> 
> Here there should be another .TP
> 
>> .I cpu_id_start
>> .RS
> 
> And that TP would make this RS unnecessary, since it would indent the 
> paragraph.

OK

> 
>> Always-updated value of the CPU number on which the registered thread is
>> running.
>> Its value is guaranteed to always be a possible CPU number,
>> even when rseq is not registered.
>> Its value should always be confirmed by reading the cpu_id field before
>> user-space performs any side-effect
>> (e.g. storing to memory).
>> .PP
> 
> This should be a IP, to continue the indentation of the previous TP.

OK

> 
>> This field is always guaranteed to hold a valid CPU number in the range
>> [ 0 ..  nr_possible_cpus - 1 ].
>> It can therefore be loaded by user-space and used as an offset in
>> per-cpu data structures without having to check whether its value is
>> within the valid bounds compared to the number of possible CPUs in the
>> system.
> 
> Having a word alone in a line is usually a hint that there exists a 
> better point to break the sentence (except where .I or .B ask for it).

OK

> 
>> .PP
>> Initialized by user-space to a possible CPU number (e.g., 0),
>> updated by the kernel for threads registered with rseq.
> 
> rseq shoudl be formatted.

OK

> 
>> .PP
>> For user-space applications executed on a kernel without rseq support,
>> the cpu_id_start field stays initialized at 0, which is indeed a valid
>> CPU number.
>> It is therefore valid to use it as an offset in per-cpu data structures,
>> and only validate whether it's actually the current CPU number by
>> comparing it with the cpu_id field within the rseq critical section.
>> If the kernel does not provide rseq support, that cpu_id field stays
>> initialized at -1,
>> so the comparison always fails, as intended.
>> .PP
>> This field should only be read by the thread which registered this data
>> structure.
>> Aligned on 32-bit.
>> .PP
>> It is up to user-space to implement a fall-back mechanism for 
>> scenarios where
> 
> s/user-space/user space/

OK

> 
> The - is used as in natural English, for form an adjective.  Since here 
> it acts as a subject, it goes with a space.

Yes.

> 
>> rseq is not available.
>> .RE
>> .PP
>> .I cpu_id
>> .RS
>> Always-updated value of the CPU number on which the registered thread is
>> running.
>> Initialized by user-space to -1,
>> updated by the kernel for threads registered with rseq.
>> .PP
>> This field should only be read by the thread which registered this data
>> structure.
>> Aligned on 32-bit.
>> .RE
>> .PP
>> .I rseq_cs
>> .RS
>> The rseq_cs field is a pointer to a
>> .B struct rseq_cs .
>> Is is NULL when no rseq assembly block critical section is active for
>> the registered thread.
>> Setting it to point to a critical section descriptor (
>> .B struct rseq_cs
>> ) marks the beginning of the critical section.
>> .PP
>> Initialized by user-space to NULL.
>> .PP
>> Updated by user-space, which sets the address of the currently
>> active rseq_cs at the beginning of assembly instruction sequence
>> block,
>> and set to NULL by the kernel when it restarts an assembly instruction
>> sequence block,
>> as well as when the kernel detects that it is preempting or delivering a
>> signal outside of the range targeted by the rseq_cs.
>> Also needs to be set to NULL by user-space before reclaiming memory that
>> contains the targeted
>> .B struct rseq_cs .
>> .PP
>> Read and set by the kernel.
>> .PP
>> This field should only be updated by the thread which registered this
>> data structure.
>> Aligned on 64-bit.
>> .RE
>> .PP
>> .I flags
>> .RS
>> Flags indicating the restart behavior for the registered thread.
>> This is mainly used for debugging purposes.
>> Can be a combination of:
>> .TP
>> .B RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT
>> Inhibit instruction sequence block restart on preemption for this
>> thread.
>> This flag is deprecated since kernel 6.1.
> 
> I did some consistency fixes regarding linux versions.  Now we always 
> use "Linux x.y", rather than "Linux kernel x.y", or "kernel x.y".  That 
> helps grepping.

OK

[...]

Updated version based on your comments pushed into my repo, thanks!

Mathieu


-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: rseq(2) man page

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 3637 bytes --]



On 1/10/23 17:54, Mathieu Desnoyers wrote:

[...]

[EFAULT / SEGV]

> 
> OK, let's keep this for a separate discussion.

Sure.

[...]

>> I forgot; there's a new section: LIBRARY.
>>
>> It specifies the library in which the function is defined, or in the case of 
>> syscalls, the wrapper (since we call it through syscall(2), it would be libc).
> 
> OK. Similar to futex(2).

Yep.

> 
>>
>> BTW, I wonder what librseq is.  Is librseq something that users should care 
>> about?
> 
> Users are not required to use librseq to use the rseq system call,
> but it's very convenient to use this C-level API rather than have
> each user reimplement the per-architecture assembly code required to
> create the RSEQ critical sections.
> 
> librseq did not have an official release yet, so that's mainly why I
> think it too early to refer to it in manual pages.

Okay; when you feel it's ready, you could document it like libkeyutils in keyctl(2).

> 
>>
>>> .SH SYNOPSIS
>>> .nf
>>> .PP
>>> .BR "#include <linux/rseq.h>" \
>>> " /* Definition of " RSEQ_* " constants and rseq types */"
>>
>> The line above goes beyond column 80 in formatted output.  That's a hard limit 
>> for manual pages.  If you add this page to the linux man-pages repo, and run 
>> the linters, you'll see a warning about that.  In case you're interested in 
>> linting manual pages in the future, you can do something similar to what I do 
>> in the man-pages[2]
> 
> OK. I've used it, it's quite useful! I have fixed all warnings except for
> "mandoc: man2/rseq.2:5:12: WARNING: cannot parse date, using it verbatim: (date)"
> which I suspect is expected.

Uhh, I had that one explicitly silenced:

$ make lint-man-mandoc V=1
LINT (mandoc)	tmp/lint/man2/perf_event_open.2.lint-man.mandoc.touch
! (mandoc -man -Tlint  man2/perf_event_open.2 2>&1 \
    | grep -v 'STYLE: lower case character in document title:' \
    | grep -v 'UNSUPP: ignoring macro in table:' \
    | grep -v 'WARNING: cannot parse date, using it verbatim: TH (date)' \
    | grep -v 'WARNING: empty block: UR' \
    ||:; \
) \
| grep '.' >&2
touch tmp/lint/man2/perf_event_open.2.lint-man.mandoc.touch


For some reason you didn't have "TH" in the error message, which is why my grep 
didn't discard it.

[...]

>>> .BI "int syscall(SYS_rseq, struct rseq *_Nullable " rseq ", uint32_t " 
>>> rseq_len \
>>
>> What's the meaning for NULL?  Does it have a valid sentinel meaning, or is it 
>> an invalid address?  If it's just interpreted as an invalid address (for which 
>> from a user-space perspective a crash would be legitimate), then I'd remove 
>> _Nullable.
> 
> With the flags that are currently implemented (0 or RSEQ_FLAG_UNREGISTER),
> the rseq argument is not expected to be legitimately NULL (it would return
> -1, errno=EFAULT on registration, or -1, errno=EINVAL on unregister attempt).
> 
> We may add new flags in the future which would not care about the rseq address
> (it could very well be null then). Do you recommend that we only add the
> _Nullable tag when this occurs ?

Yes; since it's what the user can pass, it makes sense to be as constrained as 
possible.  If it were some return that the user would have to inspect, it would 
make sense to be cautious on the NULL side of things an use _Nullable 
preventively, but for an input, non-null is preferred for now.


[...]

> 
> Updated version based on your comments pushed into my repo, thanks!

Cool! I'll have a look.

Cheers,

Alex

> 
> Mathieu
> 
> 

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: rseq(2) man page

[Mathieu Desnoyers]

On 2023-01-10 14:28, Alejandro Colomar wrote:
> 
> 
> On 1/10/23 17:54, Mathieu Desnoyers wrote:
> 

> 
> [...]
> 
>>>> .BI "int syscall(SYS_rseq, struct rseq *_Nullable " rseq ", uint32_t 
>>>> " rseq_len \
>>>
>>> What's the meaning for NULL?  Does it have a valid sentinel meaning, 
>>> or is it an invalid address?  If it's just interpreted as an invalid 
>>> address (for which from a user-space perspective a crash would be 
>>> legitimate), then I'd remove _Nullable.
>>
>> With the flags that are currently implemented (0 or 
>> RSEQ_FLAG_UNREGISTER),
>> the rseq argument is not expected to be legitimately NULL (it would 
>> return
>> -1, errno=EFAULT on registration, or -1, errno=EINVAL on unregister 
>> attempt).
>>
>> We may add new flags in the future which would not care about the rseq 
>> address
>> (it could very well be null then). Do you recommend that we only add the
>> _Nullable tag when this occurs ?
> 
> Yes; since it's what the user can pass, it makes sense to be as 
> constrained as possible.  If it were some return that the user would 
> have to inspect, it would make sense to be cautious on the NULL side of 
> things an use _Nullable preventively, but for an input, non-null is 
> preferred for now.

OK, updated.

> 
> 
> [...]
> 
>>
>> Updated version based on your comments pushed into my repo, thanks!
> 
> Cool! I'll have a look.

Thanks! Once you find it to your liking, I plan to sent it as a patch 
against the man-pages project.

Mathieu

> 
> Cheers,
> 
> Alex
> 
>>
>> Mathieu
>>
>>
> 

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

man page width limit and example indentation (was: rseq(2) man page)

[G. Branden Robinson]

[-- Attachment #1.1: Type: text/plain, Size: 6793 bytes --]

[formatting issues; CC list trimmed, added groff@]

At 2023-01-06T23:57:35+0100, Alejandro Colomar wrote:
> On 1/6/23 21:57, Mathieu Desnoyers wrote:
> The line above goes beyond column 80 in formatted output.  That's a
> hard limit for manual pages.

The hard limit should be thought of as somewhat lower--as low as 76--in
pages that use tbl(1).  This is due to a groff design issue that goes
back 30 years.[1]  I will try to resolve it for groff 1.24 but it might
be the biggest challenge I've yet undertaken with the codebase.  It will
require a redesign of how tbl(1) gets information through to the
terminal output driver, grotty(1).

(My former advice was 78 columns; I am revising that advice in a more
conservative direction having dug more deeply into tbl(1) behavior.)

Here are the details.  Some readers will want to skip to the "Practical
Advice for Man Page Authors" heading below.

1.  When a table is "boxed", meaning it uses any of the "box",
    "doublebox", or "allbox" region options, 3 extra character cells
    consumed by the leading | character and a trailing space and | at
    the end of the table row.  tbl(1) sets up a warning to be emitted at
    formatting time if this happens.  (In its role as a preprocessor,
    tbl itself doesn't know what the line length of the output device
    will be.)

2.  When a table row uses a vertical rule (|) as a column descriptor,
    tbl(1) is designed such it considers the rule as taking up
    effectively no space.  And on typesetting devices, this is true for
    practical purposes.  On terminals, though, they eat an entire
    character cell, and this matters.

    a.  _Interior_ vertical rules, as with a format of "L | L | L.", are
        not a problem in practice because the columns of the table are
        already separated by 3n of space by default.  So what usually
        happens is that the middle space of the 3 gets
        replaced/overstruck with the "|" and the table formats happily
        without getting any wider and causing problems.

    b.  However the amount of inter-column space is both
        user-configurable--you can suffix any column descriptor with a
        number to change the column separation.  Also, the "expand"
        region option will _compress_ (reduce) the amount of
        inter-column space if it has to to fit the table within the
        available line length.  Fortunately, "expand" is not that
        popular a region option; it doesn't expand _columns_, but only
        the separation between them.  What people usually want is the
        "x" modifier on a column descriptor, as with "L L Lx.".

    c.  But this means that an inter-column space amount of 0 is in
        conflict with the presence of a vertical rule in that place (on
        terminal devices).  So, on terminals, the smallest practical
        spacing after any column that contains a vertical rule is not 0,
        but 1.

3.  Another issue is related to these and I may tackle it in parallel.
    It is not possible in terminal output to direct the inclusion or
    exclusion of space adjacent to vertical rules.  Presently, their
    handling is inconsistent; a leading vertical rule as a column
    classifier gets no space after it, but a trailing vertical rule as a
    column classifier _does_ get space _before_ it.[2]

4.  In preparing the attached exhibit of all the above weirdness, I
    noticed two further bugs.

    a.  I need to revisit my fix for Savannah #63449;[3] I'm still
        seeing unnecessary drawing of vertical rules 1v above the top of
        the table in circumstances where that isn't warranted (i.e.,
        there is no intersection with a horizontal rule).  More cases
        for the regression test.

    b.  The third exhibit (line 17) overruns the line length but doesn't
        produce a warning.

I think that's all I have on tables for now.  I've fixed 19 bugs in tbl
for groff 1.23, but it's not enough.  :-|

Practical Advice for Man Page Authors
=====================================

A.  The 80-column limit on output line length for terminals is "safe" if
    tbl(1) is not used.

B.  If tbl(1) _is_ used, you're still safe if you avoid:

    i.    using boxed tables ("allbox", "box", "doublebox");
    ii.   placing vertical rules at the left and right edges of tables
          (in a row description); and
    iii.  reducing the column separation to zero before a vertical rule
          (in a row description).

C.  If you've read this far and your table is still overrunning the
    line, you may need to reduce your table's width by up to 3 character
    cells.  Since the default column separation is 3n, any table with
    more than 2 columns is guaranteed to have enough total column
    separation for 3 character cells to be "borrowed" from two of the
    columns while still having at least 1n of separation.

D.  If even that doesn't work out, or you have a two-column table that
    is overrunning the line, you need to either:

    i.   recast the content of the table to shorten it; or
    ii.  use text blocks ("T{ T}") to house some of the table entries.

> For code examples we use .in +4n rather than .RS.  I don't remember
> the exact reason, but it had some glitches in some cases.

There were no glitches that I recall, but Michael wanted the man(7)
source to be in a form where examples could be moved freely between
various contexts without needing any internal alteration.  The
discussion was in November 2020.  The following message and his reply
capture the boundaries of the problem.

https://lore.kernel.org/linux-man/20201113094755.bg6pl7g2s5h2w4mu@localhost.localdomain/

There is a limit to Michael's solution, but if the Linux man pages don't
go bonkers with deeply nested regions, it's workable.  (If they do, then
migrating examples may need to be reformatted due to changes in the
available line length; the more you indent, the less there is.)

It would please me to come up with a way, possibly a groff man(7)
extension (smaller than most, like recognition of a second argument to
the `RS` macro), that would enable "purer" man(7) input without recourse
to troff requests.

...but it might not be worth it.  If you relocate an example from a
heavily-indented region to a less-indented one, you _gain_ some line
length, and this is not an error condition.  Abbreviations or
compromises you made in an example's formatting might no longer be
necessary, and only an engaged human brain will detect the fact that the
example can be recast to leverage the added line capacity.

I don't think ChatGPT is to up that sort of thing.  Yet.

Regards,
Branden

[1] https://savannah.gnu.org/bugs/index.php?62471
[2] https://savannah.gnu.org/bugs/index.php?61597
[3] https://savannah.gnu.org/bugs/index.php?63449

[-- Attachment #1.2: table64.roff --]
[-- Type: text/troff, Size: 485 bytes --]

.ll 64n
.nf
1234567890123456789012345678901234567890123456789012345678901234
.fi
.TS
L L L L L.
abandoning	babysitter	cablecasts	dachshunds	earmarkingX
.TE
.
.TS
| L L L L L.
abandoning	babysitter	cablecasts	dachshunds	earmarkingX
.TE
.
.TS
L L L L L |.
abandoning	babysitter	cablecasts	dachshunds	earmarkingX
.TE
.
.
.TS
| L L L L L |.
abandoning	babysitter	cablecasts	dachshunds	earmarkingX
.TE
.
.TS
doublebox;
L L L L L.
abandoning	babysitter	cablecasts	dachshunds	earmarkingX
.TE

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: man page width limit and example indentation (was: rseq(2) man page)

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 4379 bytes --]

Hi Branden,

On 1/10/23 20:03, G. Branden Robinson wrote:
> [formatting issues; CC list trimmed, added groff@]
> 
> At 2023-01-06T23:57:35+0100, Alejandro Colomar wrote:
>> On 1/6/23 21:57, Mathieu Desnoyers wrote:
>> The line above goes beyond column 80 in formatted output.  That's a
>> hard limit for manual pages.
> 
> The hard limit should be thought of as somewhat lower--as low as 76--in
> pages that use tbl(1).  This is due to a groff design issue that goes
> back 30 years.[1]  I will try to resolve it for groff 1.24 but it might
> be the biggest challenge I've yet undertaken with the codebase.  It will
> require a redesign of how tbl(1) gets information through to the
> terminal output driver, grotty(1).
> 
> (My former advice was 78 columns; I am revising that advice in a more
> conservative direction having dug more deeply into tbl(1) behavior.)

The terminal is 80 cols, so I expect that setting a hard limit of 80 to the 
output should be fin, right?


tbl man2/perf_event_open.2 \
| eqn -Tutf8  \
| troff -man -t -M ./etc/groff/tmac -m checkstyle -rCHECKSTYLE=3 -ww -Tutf8 
-rLL=78n  \
| grotty -c  \
| col -b -p -x  \
| (! grep -n '.\{80\}.' | sed 's,^,man2/perf_event_open.2:,' >&2)


Or do you think that I should change that rule?  In the end, that grep only 
tests what users will see.  It can't measure anything else.

If you think I can add any other test, please suggest it, because I'm not sure I 
understand it.

As for your example, I put it in an actual man page (added TH and SH), and 
removed ll, and it still showed weird no matter what the terminal width was, so 
I don't really understand it.  However, it shows badly, so I hope whatever the 
issue is, you fix it for 1.24 :)

[...]

> Practical Advice for Man Page Authors
> =====================================
> 
> A.  The 80-column limit on output line length for terminals is "safe" if
>      tbl(1) is not used.

If it is used, it is also the only limit that makes sense, isn't it?  Please 
show some actual manual page with which I can reproduce a bug when the terminal 
isn't wide enough.

[...]

>> For code examples we use .in +4n rather than .RS.  I don't remember
>> the exact reason, but it had some glitches in some cases.
> 
> There were no glitches that I recall, but Michael wanted the man(7)
> source to be in a form where examples could be moved freely between
> various contexts without needing any internal alteration.  The
> discussion was in November 2020.  The following message and his reply
> capture the boundaries of the problem.
> 
> https://lore.kernel.org/linux-man/20201113094755.bg6pl7g2s5h2w4mu@localhost.localdomain/

Ahh, thanks.  So yes, there was a glitch: RS forced a blank line, which may not 
be desirable in all cases.

> 
> There is a limit to Michael's solution, but if the Linux man pages don't
> go bonkers with deeply nested regions, it's workable.  (If they do, then
> migrating examples may need to be reformatted due to changes in the
> available line length; the more you indent, the less there is.)
> 
> It would please me to come up with a way, possibly a groff man(7)
> extension (smaller than most, like recognition of a second argument to
> the `RS` macro), that would enable "purer" man(7) input without recourse
> to troff requests.
> 
> ...but it might not be worth it.  If you relocate an example from a
> heavily-indented region to a less-indented one, you _gain_ some line
> length, and this is not an error condition.  Abbreviations or
> compromises you made in an example's formatting might no longer be
> necessary, and only an engaged human brain will detect the fact that the
> example can be recast to leverage the added line capacity.
> 
> I don't think ChatGPT is to up that sort of thing.  Yet.

Fun thing: My dad talked to me about ChatGPT a couple of weeks ago (I had never 
heard of it).  We tried it with some code; I showed a function I had written, 
and asked it to review it.  ChatGPT proposed an alternative implementation, but 
it introduced two bugs.  :P

> 
> Regards,
> Branden

Cheers,

Alex

> 
> [1] https://savannah.gnu.org/bugs/index.php?62471
> [2] https://savannah.gnu.org/bugs/index.php?61597
> [3] https://savannah.gnu.org/bugs/index.php?63449

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: man page width limit and example indentation (was: rseq(2) man page)

[G. Branden Robinson]

[-- Attachment #1.1: Type: text/plain, Size: 4090 bytes --]

Hi Alex,

At 2023-01-10T21:29:39+0100, Alejandro Colomar wrote:
> On 1/10/23 20:03, G. Branden Robinson wrote:
> > (My former advice was 78 columns; I am revising that advice in a
> > more conservative direction having dug more deeply into tbl(1)
> > behavior.)
> 
> The terminal is 80 cols,

Not necessarily; it could be much wider.  The point I'm making is that
thanks to tbl(1) bugs, composing your table for within what you _think_
is 80 columns might cause it to overrun anyway.

> so I expect that setting a hard limit of 80 to the output should be
> fin, right?

No.  I would stay at 78n or even go down to 77 or 76 if you wanted to be
really paranoid.  But you kind of have to work at it to get to the 77
case, and the 76 is because of something I think I might do for groff
1.23.  But I'm not sure yet.

> tbl man2/perf_event_open.2 \
> | eqn -Tutf8  \
> | troff -man -t -M ./etc/groff/tmac -m checkstyle -rCHECKSTYLE=3 -ww -Tutf8
> -rLL=78n  \
> | grotty -c  \
> | col -b -p -x  \
> | (! grep -n '.\{80\}.' | sed 's,^,man2/perf_event_open.2:,' >&2)
> 
> 
> Or do you think that I should change that rule?  In the end, that grep only
> tests what users will see.  It can't measure anything else.

That's an excellent approach.  If all your man pages stay within the
80-column limit under that test, then you're fine--don't change a thing.

> If you think I can add any other test, please suggest it, because I'm
> not sure I understand it.

My lengthy exploration of the issues is because I am more the kind of
paranoid person who likes Ada and Haskell than the sort who slouches off
the end of a `switch` statement without defining a `default` case
because I'm sure I handled every scenario and if I didn't it's not
important anyway.  :-P

> As for your example, I put it in an actual man page (added TH and SH),
> and removed ll, and it still showed weird no matter what the terminal
> width was, so I don't really understand it.  However, it shows badly,
> so I hope whatever the issue is, you fix it for 1.24 :)

Well, some of it, I'm still trying to fix for 1.23.  I _still_ have not
heard back from Bertrand.  It's been two weeks.  I need to start
considering begging Werner to come out of retirement just long enough to
tag and push some tar archives.  :-O

> > Practical Advice for Man Page Authors
> > =====================================
> > 
> > A.  The 80-column limit on output line length for terminals is "safe" if
> >      tbl(1) is not used.
> 
> If it is used, it is also the only limit that makes sense, isn't it?
> Please show some actual manual page with which I can reproduce a bug
> when the terminal isn't wide enough.

Okay.  Is a mocked-up manual page good enough?  I'm attaching one.

> > > For code examples we use .in +4n rather than .RS.  I don't remember
> > > the exact reason, but it had some glitches in some cases.
> > 
> > There were no glitches that I recall, but Michael wanted the man(7)
> > source to be in a form where examples could be moved freely between
> > various contexts without needing any internal alteration.  The
> > discussion was in November 2020.  The following message and his reply
> > capture the boundaries of the problem.
> > 
> > https://lore.kernel.org/linux-man/20201113094755.bg6pl7g2s5h2w4mu@localhost.localdomain/
> 
> Ahh, thanks.  So yes, there was a glitch: RS forced a blank line, which may
> not be desirable in all cases.

Er, I don't think that's true.  `RS` forces a _break_, but not vertical
space ("blank line(s)").  I've added a demonstration to the attached man
page.

If you can point me to what you're referring to, maybe I can clarify.

> Fun thing: My dad talked to me about ChatGPT a couple of weeks ago (I
> had never heard of it).  We tried it with some code; I showed a
> function I had written, and asked it to review it.  ChatGPT proposed
> an alternative implementation, but it introduced two bugs.  :P

Wow, it really is as good as the average programmer at a tech company!

Regards,
Branden

[-- Attachment #1.2: wide-table.man --]
[-- Type: application/x-troff-man, Size: 1322 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: man page width limit and example indentation (was: rseq(2) man page)

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 4782 bytes --]

Hi Branden,

On 1/10/23 22:39, G. Branden Robinson wrote:
[...]

> No.  I would stay at 78n or even go down to 77 or 76 if you wanted to be
> really paranoid.  But you kind of have to work at it to get to the 77
> case, and the 76 is because of something I think I might do for groff
> 1.23.  But I'm not sure yet.
> 
>> tbl man2/perf_event_open.2 \
>> | eqn -Tutf8  \
>> | troff -man -t -M ./etc/groff/tmac -m checkstyle -rCHECKSTYLE=3 -ww -Tutf8
>> -rLL=78n  \
>> | grotty -c  \
>> | col -b -p -x  \
>> | (! grep -n '.\{80\}.' | sed 's,^,man2/perf_event_open.2:,' >&2)
>>
>>
>> Or do you think that I should change that rule?  In the end, that grep only
>> tests what users will see.  It can't measure anything else.
> 
> That's an excellent approach.

Merit of the original implementation goes to Ralph.

>  If all your man pages stay within the
> 80-column limit under that test, then you're fine--don't change a thing.

They do (I don't remember if all, but I think so; or at most there are a few 
exceptions).  However, I don't run `make lint` before every commit, so I may 
introduce issues from time to time, then fix them when I notice :)

> 
>> If you think I can add any other test, please suggest it, because I'm
>> not sure I understand it.
> 
> My lengthy exploration of the issues is because I am more the kind of
> paranoid person who likes Ada and Haskell than the sort who slouches off
> the end of a `switch` statement without defining a `default` case
> because I'm sure I handled every scenario and if I didn't it's not
> important anyway.  :-P
> 
>> As for your example, I put it in an actual man page (added TH and SH),
>> and removed ll, and it still showed weird no matter what the terminal
>> width was, so I don't really understand it.  However, it shows badly,
>> so I hope whatever the issue is, you fix it for 1.24 :)
> 
> Well, some of it, I'm still trying to fix for 1.23.  I _still_ have not
> heard back from Bertrand.  It's been two weeks.  I need to start
> considering begging Werner to come out of retirement just long enough to
> tag and push some tar archives.  :-O

You can't do it, right?  Or do you?

> 
>>> Practical Advice for Man Page Authors
>>> =====================================
>>>
>>> A.  The 80-column limit on output line length for terminals is "safe" if
>>>       tbl(1) is not used.
>>
>> If it is used, it is also the only limit that makes sense, isn't it?
>> Please show some actual manual page with which I can reproduce a bug
>> when the terminal isn't wide enough.
> 
> Okay.  Is a mocked-up manual page good enough?  I'm attaching one.

Yep.  Now I understand.  So yeah, I never try to guess how much the page will 
take up on screen, and just check experimentally.  And the test works nicely to 
make sure I don't forget to check (but I still may forget to run the test :).

> 
>>>> For code examples we use .in +4n rather than .RS.  I don't remember
>>>> the exact reason, but it had some glitches in some cases.
>>>
>>> There were no glitches that I recall, but Michael wanted the man(7)
>>> source to be in a form where examples could be moved freely between
>>> various contexts without needing any internal alteration.  The
>>> discussion was in November 2020.  The following message and his reply
>>> capture the boundaries of the problem.
>>>
>>> https://lore.kernel.org/linux-man/20201113094755.bg6pl7g2s5h2w4mu@localhost.localdomain/
>>
>> Ahh, thanks.  So yes, there was a glitch: RS forced a blank line, which may
>> not be desirable in all cases.
> 
> Er, I don't think that's true.  `RS` forces a _break_, but not vertical
> space ("blank line(s)").  I've added a demonstration to the attached man
> page.

You're right.  I don't remember what was the exact issue we had with it.
Anyway, .in just works so far. :)

> 
> If you can point me to what you're referring to, maybe I can clarify.
> 
>> Fun thing: My dad talked to me about ChatGPT a couple of weeks ago (I
>> had never heard of it).  We tried it with some code; I showed a
>> function I had written, and asked it to review it.  ChatGPT proposed
>> an alternative implementation, but it introduced two bugs.  :P
> 
> Wow, it really is as good as the average programmer at a tech company!

Quite right.  That's what I said to him! :P
AI learns from average, and so it produces average.

He then had a distopian theory that it could replace the average programmer in 
the future, and then good programmers would be hired to fix the bugs, costing 
much less to companies.  While I don't like the idea, I find it likely to happen 
at some point.

Cheers,

Alex

> 
> Regards,
> Branden

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: man page width limit and example indentation (was: rseq(2) man page)

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 2081 bytes --]

At 2023-01-11T00:30:16+0100, Alejandro Colomar wrote:
> On 1/10/23 22:39, G. Branden Robinson wrote:
> > Well, some of it, I'm still trying to fix for 1.23.  I _still_ have not
> > heard back from Bertrand.  It's been two weeks.  I need to start
> > considering begging Werner to come out of retirement just long enough to
> > tag and push some tar archives.  :-O
> 
> You can't do it, right?  Or do you?

I can create the tag in Git, but I don't have a GPG public key that the
FSF recognizes as having a "maintainer" bit, so I cannot validly upload
a distribution archive to alpha.gnu.org.  Apart from signing issues, I
don't have authorization to upload there, or know the correct procedure.
Since this is an RC, I don't need to be on a whitelist (if one exists)
to email info-gnu, since only final releases are announced there.  The
plan was to announce RC2 to the groff list and the GNU platform-testers
list.

I suspect the only things that _strictly_ require an official GNU
maintainer are generation of the signature for the distribution archive,
and upload of that archive to alpha.gnu.org.

If I were in Bertrand or Werner's position, I would prefer to perform
the Git tagging and archive generation ("make dist" or preferably "make
distcheck") myself.  But AFAIK there's no _technical_ barrier to me
doing those things.  (I already "distcheck" before _every_ set of
commits I push.)  But in theory I could just hand either of them an
archive of 1.23.0.rc2 and say "trust me".

> Yep.  Now I understand.  So yeah, I never try to guess how much the
> page will take up on screen, and just check experimentally.

That's a sound approach.  Nevertheless I would like to make tbl(1)'s
output more predictable on terminals for all groff users.  That should
lower user frustration.

> You're right.  I don't remember what was the exact issue we had with
> it.  Anyway, .in just works so far. :)

I really like weaning man page authors off of *roff requests; I may have
to take another look at this issue someday.  :D

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: man page width limit and example indentation (was: rseq(2) man page)

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 989 bytes --]

At 2023-01-10T13:03:14-0600, G. Branden Robinson wrote:
> 4.  In preparing the attached exhibit of all the above weirdness, I
>     noticed two further bugs.
> 
>     a.  I need to revisit my fix for Savannah #63449;[3] I'm still
>         seeing unnecessary drawing of vertical rules 1v above the top of
>         the table in circumstances where that isn't warranted (i.e.,
>         there is no intersection with a horizontal rule).  More cases
>         for the regression test.
[...]
> [3] https://savannah.gnu.org/bugs/index.php?63449

I've pushed a fix for this and it will be in groff 1.23.  The
description above is a little bit garbled, confused once again with the
highly annoying but difficult-to-fix Savannah #62471.

The thrust of matters is that

.TS
| L.
foo
.TE

now behaves the same as

.TS
L |.
foo
.TE

and

.TS
L | L.
foo
.TE

with respect to vertical space above the table.

See the URL above for details.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: rseq(2) man page

[Mathieu Desnoyers]

On 2023-01-06 12:22, Alejandro Colomar wrote:
> Hi Mathieu!
[...]
> 
> BTW, would you mind sending links to the previous submissions?  I 
> couldn't find them.  I know there are a few patches that we received 
> when Michael was on-and-off that I deferred for a later time, and never 
> did, and maybe this is one of those.  I tried to find such patches some 
> time ago, but with no luck.

Here is what I could find in my archives (from most recent to oldest submission):

https://lore.kernel.org/r/1094083824.1256.1591899797181.JavaMail.zimbra@efficios.com
https://lore.kernel.org/r/20181206144228.9656-1-mathieu.desnoyers@efficios.com
https://lore.kernel.org/r/20180919144028.10863-1-mathieu.desnoyers@efficios.com
https://lore.kernel.org/r/20171116151711.19001-3-mathieu.desnoyers@efficios.com
https://lore.kernel.org/r/20171115191316.828-3-mathieu.desnoyers@efficios.com

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com