[GIT PULL] Scheduler changes for v6.5

[Ingo Molnar <mingo@kernel.org>]

Linus,

Please pull the latest sched/core git tree from:

   git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched-core-2023-06-27

   # HEAD: ebb83d84e49b54369b0db67136a5fe1087124dcc sched/core: Avoid multiple calling update_rq_clock() in __cfsb_csd_unthrottle()

NOTE:

  When merging this tree you'll get a new conflict in 
  drivers/clocksource/hyperv_timer.c, due to overlapping changes.

  In case you want to double check your conflict resolution, our -tip CI 
  conflict resolution can be found at the e31a421069a6 merge commit in the 
  core/merge -tip branch:

      git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core/merge

Scheduler changes for v6.5:

- Scheduler SMP load-balancer improvements:

    - Avoid unnecessary migrations within SMT domains on hybrid systems.

      Problem:

        On hybrid CPU systems, (processors with a mixture of higher-frequency
	SMT cores and lower-frequency non-SMT cores), under the old code
	lower-priority CPUs pulled tasks from the higher-priority cores if
	more than one SMT sibling was busy - resulting in many unnecessary
	task migrations.

      Solution:

        The new code improves the load balancer to recognize SMT cores with more
        than one busy sibling and allows lower-priority CPUs to pull tasks, which
        avoids superfluous migrations and lets lower-priority cores inspect all SMT
        siblings for the busiest queue.

    - Implement the 'runnable boosting' feature in the EAS balancer: consider CPU
      contention in frequency, EAS max util & load-balance busiest CPU selection.

      This improves CPU utilization for certain workloads, while leaves other key
      workloads unchanged.

- Scheduler infrastructure improvements:

    - Rewrite the scheduler topology setup code by consolidating it
      into the build_sched_topology() helper function and building
      it dynamically on the fly.

    - Resolve the local_clock() vs. noinstr complications by rewriting
      the code: provide separate sched_clock_noinstr() and
      local_clock_noinstr() functions to be used in instrumentation code,
      and make sure it is all instrumentation-safe.

- Fixes:

    - Fix a kthread_park() race with wait_woken()

    - Fix misc wait_task_inactive() bugs unearthed by the -rt merge:
       - Fix UP PREEMPT bug by unifying the SMP and UP implementations.
       - Fix task_struct::saved_state handling.

    - Fix various rq clock update bugs, unearthed by turning on the rq clock
      debugging code.

    - Fix the PSI WINDOW_MIN_US trigger limit, which was easy to trigger by
      creating enough cgroups, by removing the warnign and restricting
      window size triggers to PSI file write-permission or CAP_SYS_RESOURCE.

    - Propagate SMT flags in the topology when removing degenerate domain

    - Fix grub_reclaim() calculation bug in the deadline scheduler code

    - Avoid resetting the min update period when it is unnecessary, in
      psi_trigger_destroy().

    - Don't balance a task to its current running CPU in load_balance(),
      which was possible on certain NUMA topologies with overlapping
      groups.

    - Fix the sched-debug printing of rq->nr_uninterruptible

- Cleanups:

    - Address various -Wmissing-prototype warnings, as a preparation
      to (maybe) enable this warning in the future.

    - Remove unused code

    - Mark more functions __init

    - Fix shadow-variable warnings

Signed-off-by: Ingo Molnar <mingo@kernel.org>
 Thanks,

	Ingo

------------------>
Arnd Bergmann (5):
      sched: Hide unused sched_update_scaling()
      sched: Add schedule_user() declaration
      sched/fair: Hide unused init_cfs_bandwidth() stub
      sched: Make task_vruntime_update() prototype visible
      sched/fair: Move unused stub functions to header

Arve Hjønnevåg (1):
      sched/wait: Fix a kthread_park race with wait_woken()

Chen Yu (1):
      x86/sched: Add the SD_ASYM_PACKING flag to the die domain of hybrid processors

Dietmar Eggemann (2):
      sched/fair: Refactor CPU utilization functions
      sched/fair, cpufreq: Introduce 'runnable boosting'

Hao Jia (3):
      sched/core: Fixed missing rq clock update before calling set_rq_offline()
      sched/core: Avoid double calling update_rq_clock() in __balance_push_cpu_stop()
      sched/core: Avoid multiple calling update_rq_clock() in __cfsb_csd_unthrottle()

Miaohe Lin (2):
      sched/deadline: remove unused dl_bandwidth
      sched/topology: Mark set_sched_topology() __init

Peter Zijlstra (17):
      sched: Unconditionally use full-fat wait_task_inactive()
      sched: Consider task_struct::saved_state in wait_task_inactive()
      x86/sched: Rewrite topology setup
      seqlock/latch: Provide raw_read_seqcount_latch_retry()
      time/sched_clock: Provide sched_clock_noinstr()
      arm64/io: Always inline all of __raw_{read,write}[bwlq]()
      arm64/arch_timer: Provide noinstr sched_clock_read() functions
      loongarch: Provide noinstr sched_clock_read()
      s390/time: Provide sched_clock_noinstr()
      math64: Always inline u128 version of mul_u64_u64_shr()
      x86/vdso: Fix gettimeofday masking
      clocksource: hyper-v: Adjust hv_read_tsc_page_tsc() to avoid special casing U64_MAX
      clocksource: hyper-v: Provide noinstr sched_clock()
      x86/tsc: Provide sched_clock_noinstr()
      sched/clock: Provide local_clock_noinstr()
      cpuidle: Use local_clock_noinstr()
      arm64/arch_timer: Fix MMIO byteswap

Ricardo Neri (11):
      sched/fair: Move is_core_idle() out of CONFIG_NUMA
      sched/fair: Only do asym_packing load balancing from fully idle SMT cores
      sched/fair: Simplify asym_packing logic for SMT cores
      sched/fair: Let low-priority cores help high-priority busy SMT cores
      sched/fair: Keep a fully_busy SMT sched group as busiest
      sched/fair: Use the busiest group to set prefer_sibling
      sched/fair: Do not even the number of busy CPUs via asym_packing
      sched/topology: Check SDF_SHARED_CHILD in highest_flag_domain()
      sched/topology: Remove SHARED_CHILD from ASYM_PACKING
      x86/sched: Remove SD_ASYM_PACKING from the SMT domain flags
      x86/sched/itmt: Give all SMT siblings of a core the same priority

Suren Baghdasaryan (1):
      psi: remove 500ms min window size limitation for triggers

Tim C Chen (1):
      sched/topology: Propagate SMT flags when removing degenerate domain

Tom Rix (1):
      sched/fair: Rename variable cpu_util eff_util

Vineeth Pillai (2):
      sched/deadline: Fix bandwidth reclaim equation in GRUB
      sched/deadline: Update GRUB description in the documentation

Yang Yang (1):
      sched/psi: Avoid resetting the min update period when it is unnecessary

Yicong Yang (1):
      sched/fair: Don't balance task to its current running CPU

晏艳(采苓) (1):
      sched/debug: Correct printing for rq->nr_uninterruptible


 Documentation/scheduler/sched-deadline.rst |   5 +-
 arch/arm64/include/asm/arch_timer.h        |   8 +-
 arch/arm64/include/asm/io.h                |  12 +-
 arch/loongarch/include/asm/loongarch.h     |   2 +-
 arch/loongarch/kernel/time.c               |   6 +-
 arch/s390/include/asm/timex.h              |  13 +-
 arch/s390/kernel/time.c                    |   5 +
 arch/x86/include/asm/mshyperv.h            |   5 +
 arch/x86/include/asm/vdso/gettimeofday.h   |  41 +++-
 arch/x86/kernel/itmt.c                     |  23 +-
 arch/x86/kernel/kvmclock.c                 |   4 +-
 arch/x86/kernel/smpboot.c                  | 102 +++++----
 arch/x86/kernel/tsc.c                      |  38 +++-
 arch/x86/kvm/x86.c                         |   7 +-
 arch/x86/xen/time.c                        |   3 +-
 drivers/clocksource/arm_arch_timer.c       |  54 +++--
 drivers/clocksource/hyperv_timer.c         |  44 ++--
 drivers/cpuidle/cpuidle.c                  |   8 +-
 drivers/cpuidle/poll_state.c               |   4 +-
 include/clocksource/hyperv_timer.h         |  24 +--
 include/linux/kthread.h                    |   1 +
 include/linux/math64.h                     |   2 +-
 include/linux/rbtree_latch.h               |   2 +-
 include/linux/sched.h                      |   7 +-
 include/linux/sched/clock.h                |  17 +-
 include/linux/sched/sd_flags.h             |   5 +-
 include/linux/sched/topology.h             |   2 +-
 include/linux/seqlock.h                    |  15 +-
 kernel/cgroup/cgroup.c                     |  12 ++
 kernel/kthread.c                           |  10 +
 kernel/printk/printk.c                     |   2 +-
 kernel/sched/clock.c                       |  19 +-
 kernel/sched/core.c                        | 278 +++++++++++++-----------
 kernel/sched/cpufreq_schedutil.c           |   3 +-
 kernel/sched/deadline.c                    |  57 ++---
 kernel/sched/debug.c                       |   2 +-
 kernel/sched/fair.c                        | 329 ++++++++++++++++++-----------
 kernel/sched/psi.c                         |  19 +-
 kernel/sched/sched.h                       | 105 +++++----
 kernel/sched/topology.c                    |  15 +-
 kernel/sched/wait.c                        |   7 +-
 kernel/time/sched_clock.c                  |  24 ++-
 kernel/time/timekeeping.c                  |   4 +-
 43 files changed, 777 insertions(+), 568 deletions(-)