Re: slow sync rcu_tasks_trace

[Alexei Starovoitov <ast@fb.com>]

On 9/9/20 10:27 PM, Paul E. McKenney wrote:
> On Wed, Sep 09, 2020 at 02:22:12PM -0700, Paul E. McKenney wrote:
>> On Wed, Sep 09, 2020 at 02:04:47PM -0700, Paul E. McKenney wrote:
>>> On Wed, Sep 09, 2020 at 12:48:28PM -0700, Alexei Starovoitov wrote:
>>>> On Wed, Sep 09, 2020 at 12:39:00PM -0700, Paul E. McKenney wrote:
> 
> [ . . . ]
> 
>>>>> My plan is to try the following:
>>>>>
>>>>> 1.	Parameterize the backoff sequence so that RCU Tasks Trace
>>>>> 	uses faster rechecking than does RCU Tasks.  Experiment as
>>>>> 	needed to arrive at a good backoff value.
>>>>>
>>>>> 2.	If the tasks-list scan turns out to be a tighter bottleneck
>>>>> 	than the backoff waits, look into parallelizing this scan.
>>>>> 	(This seems unlikely, but the fact remains that RCU Tasks
>>>>> 	Trace must do a bit more work per task than RCU Tasks.)
>>>>>
>>>>> 3.	If these two approaches, still don't get the update-side
>>>>> 	latency where it needs to be, improvise.
>>>>>
>>>>> The exact path into mainline will of course depend on how far down this
>>>>> list I must go, but first to get a solution.
>>>>
>>>> I think there is a case of 4. Nothing is inside rcu_trace critical section.
>>>> I would expect single ipi would confirm that.
>>>
>>> Unless the task moves, yes.  So a single IPI should suffice in the
>>> common case.
>>
>> And what I am doing now is checking code paths.
> 
> And the following diff from a set of three patches gets my average
> RCU Tasks Trace grace-period latencies down to about 20 milliseconds,
> almost a 50x improvement from earlier today.
> 
> These are still quite rough and not yet suited for production use, but
> I will be testing.  If that goes well, I hope to send a more polished
> set of patches by end of day tomorrow, Pacific Time.  But if you get a
> chance to test them, I would value any feedback that you might have.
> 
> These patches do not require hand-tuning, they instead adjust the
> behavior according to CONFIG_TASKS_TRACE_RCU_READ_MB, which in turn
> adjusts according to CONFIG_PREEMPT_RT.  So you should get the desired
> latency reductions "out of the box", again, without tuning.

Great. Confirming improvement :)

time ./test_progs -t trampoline_count
#101 trampoline_count:OK
Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED

real	0m2.897s
user	0m0.128s
sys	0m1.527s

This is without CONFIG_TASKS_TRACE_RCU_READ_MB, of course.