Re: Restartable Sequences system call merged into Linux

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

----- On Jun 13, 2018, at 7:48 AM, heiko carstens heiko.carstens@de.ibm.com wrote:

> On Mon, Jun 11, 2018 at 03:49:18PM -0400, Mathieu Desnoyers wrote:
>> Hi!
>> 
>> Good news! The restartable sequences (rseq) system call is now merged into the
>> master
>> branch of the Linux kernel within the 4.18 merge window:
>> 
>> https://github.com/torvalds/linux/commit/d82991a8688ad128b46db1b42d5d84396487a508
>> 
>> It would be important to discuss how we should proceed to integrate the library
>> part
>> of rseq (see tools/testing/selftests/rseq/rseq*.{ch}) into glibc, or if it
>> should
>> live in a standalone project.
> 
> Is there any documentation available of what is the exact semantics of the
> functions that have to be implemented for additional architectures?

It's documented on top of kernel/rseq.c:

/*
 *
 * Restartable sequences are a lightweight interface that allows
 * user-level code to be executed atomically relative to scheduler
 * preemption and signal delivery. Typically used for implementing
 * per-cpu operations.
 *
 * It allows user-space to perform update operations on per-cpu data
 * without requiring heavy-weight atomic operations.
 *
 * Detailed algorithm of rseq user-space assembly sequences:
 *
 *                     init(rseq_cs)
 *                     cpu = TLS->rseq::cpu_id_start
 *   [1]               TLS->rseq::rseq_cs = rseq_cs
 *   [start_ip]        ----------------------------
 *   [2]               if (cpu != TLS->rseq::cpu_id)
 *                             goto abort_ip;
 *   [3]               <last_instruction_in_cs>
 *   [post_commit_ip]  ----------------------------
 *
 *   The address of jump target abort_ip must be outside the critical
 *   region, i.e.:
 *
 *     [abort_ip] < [start_ip]  || [abort_ip] >= [post_commit_ip]
 *
 *   Steps [2]-[3] (inclusive) need to be a sequence of instructions in
 *   userspace that can handle being interrupted between any of those
 *   instructions, and then resumed to the abort_ip.
 *
 *   1.  Userspace stores the address of the struct rseq_cs assembly
 *       block descriptor into the rseq_cs field of the registered
 *       struct rseq TLS area. This update is performed through a single
 *       store within the inline assembly instruction sequence.
 *       [start_ip]
 *
 *   2.  Userspace tests to check whether the current cpu_id field match
 *       the cpu number loaded before start_ip, branching to abort_ip
 *       in case of a mismatch.
 *
 *       If the sequence is preempted or interrupted by a signal
 *       at or after start_ip and before post_commit_ip, then the kernel
 *       clears TLS->__rseq_abi::rseq_cs, and sets the user-space return
 *       ip to abort_ip before returning to user-space, so the preempted
 *       execution resumes at abort_ip.
 *
 *   3.  Userspace critical section final instruction before
 *       post_commit_ip is the commit. The critical section is
 *       self-terminating.
 *       [post_commit_ip]
 *
 *   4.  <success>
 *
 *   On failure at [2], or if interrupted by preempt or signal delivery
 *   between [1] and [3]:
 *
 *       [abort_ip]
 *   F1. <failure>
 */

> I could look at rseq-skip.h and e.g. rseq-x86.h and try to figure out what
> would be the correct implementation for s390. But having that somewhere
> written down, e.g. as comments in one of the implementations, would be very
> helpful.

The first architecture implemented was rseq-x86.h. Boqun derived rseq-ppc.h
from it, and I derived rseq-arm.h from it. Feel free to start from whichever
architecture has the instruction set which is most similar to yours.

Thanks!

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com