[PATCH v28 0/8] Optimized Donor Migration for Proxy Execution

[John Stultz <jstultz@google.com>]

Hey All,

This is just the next step for Proxy Execution, allowing
proxy-return-migration to occur from the try_to_wake_up()
callpath. This however introduces some complexity as it means
the rq->donor can change from a try_to_wake_up() call, and thus
there are further changes needed to make this safe.

As always, I’m trying to submit this larger work in smallish
digestible pieces, so in this portion of the series, I’m only
submitting for review and consideration these patches which
minimize the use of passing “prev” (ie: the rq->donor) down
through the class schedulers since this cached value can become
invalid across runqueue lock drops. From there, there are some
fixups the deadline class scheduler to avoid issues when the
donor changes, then the actual patch to do return migration
from ttwu() followed by some optimization patches so we
migrate the entire task chain in proxy_migrate_task().  

While there’s not a whole lot new in this revision, I still want
to give a big thanks to K Prateek for his feedback and
suggestions on the new changes.

New in this iteration:
* Dropped the patch adding a rq->donor->sched_class->yield_task
  check, as with changes from K Prateek we shouldn’t temporarily
  run with idle as donor anymore.

* Folded in change suggested by K Prateek to introduce
  proxy_reset_donor() to reset the donor to current task when
  the donor is woken up.

* Drop an unnecessary PROXY_WAKING assignment which was noted
  by K Prateek

I’d love to get further feedback on any place where these
patches are confusing, or could use additional clarifications.

Additionally I’d appreciate any testing or comments that folks
have with the full Proxy Execution series! You can find the full
Proxy Exec series here:
  https://github.com/johnstultz-work/linux-dev/tree/proxy-exec-v28-7.1-rc0
  https://github.com/johnstultz-work/linux-dev.git proxy-exec-v28-7.1-rc0

New changes to the full patch series in this revision include:
* I found in the sleeping owner handling patch, we end up
  calling put_task() from __proxy_remove_from_sleeping_owner(),
  which might free the owner while we are still holing the
  owners->blocked_lock. So be sure to get_task()/put_task()
  around the owner usage to ensure we don't prematurely free the
  task.

* Similarly I also re-orderd the put_task/unlock lines in
  activate_blocked_waiters(), to avoid the same problem

Issues still to address with the full series:
* Continue stress testing with the recent ttwu rework to ensure
  the whole stack is working as intended (so far so good).

* Testing to double check I sorted the rwsem collisions w/
  7.1-rc properly

* Continue working to get sched_ext to be ok with Proxy
  Execution enabled.

* Reevaluate performance regression K Prateek Nayak found with
  the full series.

* The chain migration functionality needs further iterations and
  better validation to ensure it truly maintains the RT/DL load
  balancing invariants (despite this being broken in vanilla
  upstream with RT_PUSH_IPI currently)

Future work:
* Expand to more locking primitives: Figuring out pi-futexes
  would be good, using proxy for Binder PI is something else
  we’re exploring.

* Eventually: Work to replace rt_mutexes and get things happy
  with PREEMPT_RT


Credit/Disclaimer:
—--------------------
As always, this Proxy Execution series has a long history with
lots of developers that deserve credit: 

First described in a paper[1] by Watkins, Straub, Niehaus, then
from patches from Peter Zijlstra, extended with lots of work by
Juri Lelli, Valentin Schneider, and Connor O'Brien. (and thank
you to Steven Rostedt for providing additional details here!).
Thanks also to Joel Fernandes, Dietmar Eggemann, Metin Kaya,
K Prateek Nayak and Suleiman Souhlal for their substantial
review, suggestion, and patch contributions.

So again, many thanks to those above, as all the credit for this
series really is due to them - while the mistakes are surely mine.

Thanks so much!
-john

[1] https://static.lwn.net/images/conf/rtlws11/papers/proc/p38.pdf

Cc: Joel Fernandes <joelagnelf@nvidia.com>
Cc: Qais Yousef <qyousef@layalina.io>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Valentin Schneider <vschneid@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ben Segall <bsegall@google.com>
Cc: Zimuzo Ezeozue <zezeozue@google.com>
Cc: Will Deacon <will@kernel.org>
Cc: Waiman Long <longman@redhat.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Metin Kaya <Metin.Kaya@arm.com>
Cc: Xuewen Yan <xuewen.yan94@gmail.com>
Cc: K Prateek Nayak <kprateek.nayak@amd.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Suleiman Souhlal <suleiman@google.com>
Cc: kuyo chang <kuyo.chang@mediatek.com>
Cc: hupu <hupu.gm@gmail.com>
Cc: kernel-team@android.com

John Stultz (7):
  sched: Rework pick_next_task() and prev_balance() to avoid stale prev
    references
  sched: deadline: Add some helper variables to cleanup deadline logic
  sched: deadline: Add dl_rq->curr pointer to address issues with Proxy
    Exec
  sched: Rework block_task so it can be directly called
  sched: Have try_to_wake_up() handle return-migration for PROXY_WAKING
    case
  sched: Break out core of attach_tasks() helper into sched.h
  sched: Migrate whole chain in proxy_migrate_task()

Peter Zijlstra (1):
  sched: Add blocked_donor link to task for smarter mutex handoffs

 include/linux/sched.h    |   3 +-
 init/init_task.c         |   1 +
 kernel/fork.c            |   1 +
 kernel/locking/mutex.c   |  43 +++++-
 kernel/sched/core.c      | 307 +++++++++++++++++++++------------------
 kernel/sched/deadline.c  |  44 ++++--
 kernel/sched/fair.c      |  25 +---
 kernel/sched/idle.c      |   2 +-
 kernel/sched/rt.c        |   8 +-
 kernel/sched/sched.h     |  30 +++-
 kernel/sched/stop_task.c |   2 +-
 11 files changed, 280 insertions(+), 186 deletions(-)

-- 
2.54.0.rc2.533.g4f5dca5207-goog