Re: [PATCH v4 0/3] sched: make migrate_enable/migrate_disable inline

Menglong Dong
2025-09-15T01:50:12Z

On 2025/8/28 14:03 Menglong Dong <menglong.dong@linux.dev> write:
> In this series, we make migrate_enable/migrate_disable inline to obtain
> better performance in some case.
>
> In the first patch, we add the macro "COMPILE_OFFSETS" to all the
> asm-offset.c to avoid circular dependency in the 2nd patch.
>
> In the 2nd patch, we generate the offset of nr_pinned in "struct rq" with
> rq-offsets.c, as the "struct rq" is defined internally and we need to
> access the "nr_pinned" field in migrate_enable and migrate_disable. Then,
> we move the definition of migrate_enable/migrate_disable from
> kernel/sched/core.c to include/linux/sched.h.
>
> In the 3rd patch, we fix some typos in include/linux/preempt.h.

Hi, everyone. Do we have any thoughts on this series?

Thanks!
Menglong Dong

>
> One of the beneficiaries of this series is BPF trampoline. Without this
> series, the migrate_enable/migrate_disable is hot when we run the
> benchmark for FENTRY, FEXIT, MODIFY_RETURN, etc:
>
> 54.63% bpf_prog_2dcccf652aac1793_bench_trigger_fentry [k]
> bpf_prog_2dcccf652aac1793_bench_trigger_fentry
> 10.43% [kernel] [k] migrate_enable
> 10.07% bpf_trampoline_6442517037 [k] bpf_trampoline_6442517037
> 8.06% [kernel] [k] __bpf_prog_exit_recur
> 4.11% libc.so.6 [.] syscall
> 2.15% [kernel] [k] entry_SYSCALL_64
> 1.48% [kernel] [k] memchr_inv
> 1.32% [kernel] [k] fput
> 1.16% [kernel] [k] _copy_to_user
> 0.73% [kernel] [k] bpf_prog_test_run_raw_tp
>
> Before this patch, the performance of BPF FENTRY is:
>
> fentry : 113.030 ± 0.149M/s
> fentry : 112.501 ± 0.187M/s
> fentry : 112.828 ± 0.267M/s
> fentry : 115.287 ± 0.241M/s
>
> After this patch, the performance of BPF FENTRY increases to:
>
> fentry : 143.644 ± 0.670M/s
> fentry : 149.764 ± 0.362M/s
> fentry : 149.642 ± 0.156M/s
> fentry : 145.263 ± 0.221M/s
>
> Changes since V3:
> * some modification on the 2nd patch, as Alexei advised:
> - rename CREATE_MIGRATE_DISABLE to INSTANTIATE_EXPORTED_MIGRATE_DISABLE
> - add document for INSTANTIATE_EXPORTED_MIGRATE_DISABLE
>
> Changes since V2:
> * some modification on the 2nd patch, as Peter advised:
> - don't export runqueues, define migrate_enable and migrate_disable in
> kernel/sched/core.c and use them for kernel modules instead
> - define the macro this_rq_pinned()
> - add some comment for this_rq_raw()
>
> Changes since V1:
> * use PERCPU_PTR() for this_rq_raw() if !CONFIG_SMP in the 2nd patch
>
> Menglong Dong (3):
> arch: add the macro COMPILE_OFFSETS to all the asm-offsets.c
> sched: make migrate_enable/migrate_disable inline
> sched: fix some typos in include/linux/preempt.h
>
> Kbuild | 13 ++-
> arch/alpha/kernel/asm-offsets.c | 1 +
> arch/arc/kernel/asm-offsets.c | 1 +
> arch/arm/kernel/asm-offsets.c | 2 +
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/csky/kernel/asm-offsets.c | 1 +
> arch/hexagon/kernel/asm-offsets.c | 1 +
> arch/loongarch/kernel/asm-offsets.c | 2 +
> arch/m68k/kernel/asm-offsets.c | 1 +
> arch/microblaze/kernel/asm-offsets.c | 1 +
> arch/mips/kernel/asm-offsets.c | 2 +
> arch/nios2/kernel/asm-offsets.c | 1 +
> arch/openrisc/kernel/asm-offsets.c | 1 +
> arch/parisc/kernel/asm-offsets.c | 1 +
> arch/powerpc/kernel/asm-offsets.c | 1 +
> arch/riscv/kernel/asm-offsets.c | 1 +
> arch/s390/kernel/asm-offsets.c | 1 +
> arch/sh/kernel/asm-offsets.c | 1 +
> arch/sparc/kernel/asm-offsets.c | 1 +
> arch/um/kernel/asm-offsets.c | 2 +
> arch/xtensa/kernel/asm-offsets.c | 1 +
> include/linux/preempt.h | 11 +--
> include/linux/sched.h | 113 +++++++++++++++++++++++++++
> kernel/bpf/verifier.c | 1 +
> kernel/sched/core.c | 63 ++++-----------
> kernel/sched/rq-offsets.c | 12 +++
> 26 files changed, 180 insertions(+), 57 deletions(-)
> create mode 100644 kernel/sched/rq-offsets.c
>
> --
> 2.51.0
>
>