Re: [RFC PATCH v7 7/7] Restartable sequences: self-tests

[Mathieu Desnoyers <mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org>]

----- On Aug 11, 2016, at 11:10 PM, Mathieu Desnoyers mathieu.desnoyers-vg+e7yoeK/dWk0Htik3J/w@public.gmane.org wrote:

> ----- On Aug 11, 2016, at 9:28 PM, Boqun Feng boqun.feng-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org wrote:
> 
>> On Thu, Aug 11, 2016 at 11:26:30PM +0000, Mathieu Desnoyers wrote:
>>> ----- On Jul 24, 2016, at 2:01 PM, Dave Watson davejwatson-b10kYP2dOMg@public.gmane.org wrote:
>>> 
>>> >>> +static inline __attribute__((always_inline))
>>> >>> +bool rseq_finish(struct rseq_lock *rlock,
>>> >>> + intptr_t *p, intptr_t to_write,
>>> >>> + struct rseq_state start_value)
>>> > 
>>> >>> This ABI looks like it will work fine for our use case. I don't think it
>>> >>> has been mentioned yet, but we may still need multiple asm blocks
>>> >>> for differing numbers of writes. For example, an array-based freelist push:
>>> > 
>>> >>> void push(void *obj) {
>>> >>> if (index < maxlen) {
>>> >>> freelist[index++] = obj;
>>> >>> }
>>> >>> }
>>> > 
>>> >>> would be more efficiently implemented with a two-write rseq_finish:
>>> > 
>>> >>> rseq_finish2(&freelist[index], obj, // first write
>>> >>> &index, index + 1, // second write
>>> >>> ...);
>>> > 
>>> >> Would pairing one rseq_start with two rseq_finish do the trick
>>> >> there ?
>>> > 
>>> > Yes, two rseq_finish works, as long as the extra rseq management overhead
>>> > is not substantial.
>>> 
>>> I've added a commit implementing rseq_finish2() in my rseq volatile
>>> dev branch. You can fetch it at:
>>> 
>>> https://github.com/compudj/linux-percpu-dev/tree/rseq-fallback
>>> 
>>> I also have a separate test and benchmark tree in addition to the
>>> kernel selftests here:
>>> 
>>> https://github.com/compudj/rseq-test
>>> 
>>> I named the first write a "speculative" write, and the second write
>>> the "final" write.
>>> 
>> 
>> Maybe I miss something subtle, but if the first write is only a
>> "speculative" write, why can't we put it in the rseq critical section
>> rather than asm block? Like this:
>> 
>>	do_rseq(..., result, targetptr, newval
>>		{
>>			newval = index;
>>			targetptr = &index;
>>			if (newval < maxlen)
>>				freelist[newval++] = obj;
>>			else
>>				result = false;
>>		}
>> 
>> No extra rseq_finish() is needed here, but maybe a little more
>> "speculative" writes?
> 
> This won't work unfortunately. The speculative stores need to be
> between the rseq_event_counter comparison instruction in the rseq_finish
> asm sequence and the final store. The ip fixup is really needed for
> correctness of speculative stores. The sequence number scheme only works
> for loads.
> 
> Putting it in the C code between rseq_start and rseq_finish would lead
> to races such as:
> 
> thread A                                thread B
> rseq_start
> <preempted>
>                                        <sched in>
>                                        rseq_start
>                                        freelist[offset + 1] = obj
>                                        rseq_finish
>                                           offset++
>                                        <preempted>
> <sched in>
> freelist[newval + 1] = obj  <--- corrupts the list content.
> 

Small clarification to the scenario:

thread A                                thread B
rseq_start
load offset into (register 1)
<preempted>
                                       <sched in>
                                       rseq_start
                                       freelist[offset + 1] = obj
                                       rseq_finish
                                          offset++
                                       <preempted>
<sched in>
freelist[(register 1) + 1] = obj  <--- corrupts the list content.

Thanks,

Mathieu


> <snip>
> 
>> Besides, do we allow userspace programs do read-only access to the
>> memory objects modified by do_rseq(). If so, we have a problem when
>> there are two writes in a do_rseq()(either in the rseq critical section
>> or in the asm block), because in current implemetation, these two writes
>> are unordered, which makes the readers outside a do_rseq() could observe
>> the ordering of writes differently.
>> 
>> For rseq_finish2(), a simple solution would be making the "final" write
>> a RELEASE.
> 
> Indeed, we would need a release semantic for the final store here if this
> is the common use. Or we could duplicate the "flavors" of rseq_finish2 and
> add a rseq_finish2_release. We should find a way to eliminate code duplication
> there. I suspect we'll end up doing macros.
> 
> Thanks,
> 
> Mathieu
> 
>> 
>> Regards,
>> Boqun
>> 
>>> Thanks,
>>> 
>>> Mathieu
>>> 
>>> --
>>> Mathieu Desnoyers
>>> EfficiOS Inc.
>> > http://www.efficios.com
> 
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com