[PATCH 0/2] sched/cache: Fix use after free mm access in account_mm_sched()

[Tim Chen <tim.c.chen@linux.intel.com>]

Hyunwoo Kim reported a KASAN use-after-free in account_mm_sched():

  https://lore.kernel.org/lkml/apPb-Dr4nPYuHQOK@v4bel/

Cache-aware scheduling keeps its per-address-space statistics inside the
mm:

        struct mm_struct {
                ...
                struct sched_cache_stat sc_stat;
        };

so their lifetime is the mm's lifetime.  mm_alloc_sched() allocates
sc_stat.pcpu_sched from mm_init(), and mm_destroy_sched() frees it from
__mmdrop().

The scheduler, though, reaches that object from contexts that hold no
reference on the mm:

  - account_mm_sched() runs from update_curr() holding only the rq lock,
    and dereferences rq->curr->mm.

  - the load-balance predicates - can_migrate_llc_task() ->
    invalid_llc_nr() / exceed_llc_capacity() - and the task_cache_work()
    LLC occupancy scan read p->mm of *remote* tasks.

Nothing on those paths keeps the mm alive, so an exit, or an exec_mmap()
installing a new mm, can free pcpu_sched underneath a concurrent reader.

Serializing the two sides - taking the rq lock in the mm teardown path -
would put a scheduler lock in the middle of __mmdrop(), which is a lot
of coupling to pay for a statistics object.  Give the object its own
lifetime instead.

Patch 1 lifts sched_cache_stat out of mm_struct, renames it
sched_cache_group, and turns it into a refcounted object freed via
call_rcu(); the mm now merely points at it.

Patch 2 gives every task its own reference in
task_struct->sched_cache_grp - taken in copy_mm() and exec_mmap(),
dropped in exit_mm() - and converts the scheduler to read
p->sched_cache_grp rather than p->mm->sc_stat.  Readers access
the sched_cache_grp without worry that it was freed as it
has a ref count on the object.

The two patches are one fix.  Patch 1 does not stand alone, so they need
to be applied, and backported, as a pair.

A welcome side effect of the decoupling is that the group is no longer
welded to an address space, so a later series can key it on a cgroup, a
core-scheduling cookie or a numa_group instead of on a single mm.

These are the first two patches of the cache-aware prctl RFC series

  https://lore.kernel.org/lkml/cover.1787955777.git.tim.c.chen@linux.intel.com/

reposted on their own with the changelogs rewritten and minor updates around the
use-after-free, so that they can be considered ahead of the rest of that
series.  Hyunwoo confirmed the splat is gone; his Tested-by is on both
patches.

Tim

---

Tim Chen (2):
  sched/cache: Decouple sched_cache_group from mm
  sched/cache: Introduce task_struct->sched_cache_grp

 fs/exec.c                    |  14 +++
 include/linux/mm_types.h     |  15 +--
 include/linux/sched.h        |  11 +-
 kernel/exit.c                |  28 ++++--
 kernel/fork.c                |  23 +++++
 kernel/sched/build_utility.c |   4 +
 kernel/sched/cache_sched.c   |  39 ++++++++
 kernel/sched/fair.c          | 188 ++++++++++++++++++++++-------------
 kernel/sched/sched.h         |   3 +
 9 files changed, 239 insertions(+), 86 deletions(-)
 create mode 100644 kernel/sched/cache_sched.c

-- 
2.32.0