[PATCH 0/4] sched_ext: Compatible testing of SCX_ENQ_CPU_SELECTED

[PATCH 0/4] sched_ext: Compatible testing of SCX_ENQ_CPU_SELECTED

[Changwoo Min]

This patchset is for testing `SCX_ENQ_CPU_SELECTED` in a compatible way.
More specifically, it handles two cases:
  1. a BPF scheduler is compiled against vmlinux.h where
  SCX_ENQ_CPU_SELECTED is defined, but it runs on a kernel that does not
  have SCX_ENQ_CPU_SELECTED.
  2. a BPF scheduler is compiling against vmlinux.h where
  SCX_ENQ_CPU_SELECTED is not defined.

The patch consists of three parts:
  1. Add enum_defs.autogen.h, which has macros denoting whether SCX
  enums are defined in the vmlinux.h or not.
  2. Implement __COMPAT_is_enq_cpu_selected(), which provide the test of
  SCX_ENQ_CPU_SELECTED in a compatible way.
  3. Use  __COMPAT_is_enq_cpu_selected() in scx_qmap.

Also, this patchset is a sync of the relevant PR [1] in the scx repo.

[1] https://github.com/sched-ext/scx/pull/1314

Changwoo Min (4):
  sched_ext: Add enum_defs.autogen.h
  sched_ext: Include enum_defs.autogen.h from common/common.bpf.h
  sched_ext: Add __COMPAT_is_enq_cpu_selected() for BPF compatability
  sched_ext: Use __COMPAT_is_enq_cpu_selected() in scx_qmap

 tools/sched_ext/include/scx/common.bpf.h      |   1 +
 tools/sched_ext/include/scx/common.h          |   1 +
 tools/sched_ext/include/scx/compat.bpf.h      |  52 ++++++++
 .../sched_ext/include/scx/enum_defs.autogen.h | 120 ++++++++++++++++++
 tools/sched_ext/scx_qmap.bpf.c                |   2 +-
 5 files changed, 175 insertions(+), 1 deletion(-)
 create mode 100644 tools/sched_ext/include/scx/enum_defs.autogen.h

-- 
2.48.1

[PATCH 1/4] sched_ext: Add enum_defs.autogen.h

[Changwoo Min]

Add enum_defs.autogen.h, which has macros denoting whether particular
enums in the sched_ext are defined in the kernel or not. The file is
generated by gen_enum_def.py [1].

[1] https://github.com/sched-ext/scx/blob/main/scripts/gen_enum_defs.py

Signed-off-by: Changwoo Min <changwoo@igalia.com>
---
 .../sched_ext/include/scx/enum_defs.autogen.h | 120 ++++++++++++++++++
 1 file changed, 120 insertions(+)
 create mode 100644 tools/sched_ext/include/scx/enum_defs.autogen.h

diff --git a/tools/sched_ext/include/scx/enum_defs.autogen.h b/tools/sched_ext/include/scx/enum_defs.autogen.h
new file mode 100644
index 000000000000..a32e534a4012
--- /dev/null
+++ b/tools/sched_ext/include/scx/enum_defs.autogen.h
@@ -0,0 +1,120 @@
+/*
+ * WARNING: This file is autogenerated from scripts/gen_enum_defs.py.
+ */
+
+#ifndef __ENUM_DEFS_AUTOGEN_H__
+#define __ENUM_DEFS_AUTOGEN_H__
+
+
+#define HAVE_SCX_DSP_DFL_MAX_BATCH
+#define HAVE_SCX_DSP_MAX_LOOPS
+#define HAVE_SCX_WATCHDOG_MAX_TIMEOUT
+#define HAVE_SCX_EXIT_BT_LEN
+#define HAVE_SCX_EXIT_MSG_LEN
+#define HAVE_SCX_EXIT_DUMP_DFL_LEN
+#define HAVE_SCX_CPUPERF_ONE
+#define HAVE_SCX_OPS_TASK_ITER_BATCH
+#define HAVE_SCX_CPU_PREEMPT_RT
+#define HAVE_SCX_CPU_PREEMPT_DL
+#define HAVE_SCX_CPU_PREEMPT_STOP
+#define HAVE_SCX_CPU_PREEMPT_UNKNOWN
+#define HAVE_SCX_DEQ_SLEEP
+#define HAVE_SCX_DEQ_CORE_SCHED_EXEC
+#define HAVE_SCX_DSQ_FLAG_BUILTIN
+#define HAVE_SCX_DSQ_FLAG_LOCAL_ON
+#define HAVE_SCX_DSQ_INVALID
+#define HAVE_SCX_DSQ_GLOBAL
+#define HAVE_SCX_DSQ_LOCAL
+#define HAVE_SCX_DSQ_LOCAL_ON
+#define HAVE_SCX_DSQ_LOCAL_CPU_MASK
+#define HAVE_SCX_DSQ_ITER_REV
+#define HAVE___SCX_DSQ_ITER_HAS_SLICE
+#define HAVE___SCX_DSQ_ITER_HAS_VTIME
+#define HAVE___SCX_DSQ_ITER_USER_FLAGS
+#define HAVE___SCX_DSQ_ITER_ALL_FLAGS
+#define HAVE_SCX_DSQ_LNODE_ITER_CURSOR
+#define HAVE___SCX_DSQ_LNODE_PRIV_SHIFT
+#define HAVE_SCX_ENQ_WAKEUP
+#define HAVE_SCX_ENQ_HEAD
+#define HAVE_SCX_ENQ_CPU_SELECTED
+#define HAVE_SCX_ENQ_PREEMPT
+#define HAVE_SCX_ENQ_REENQ
+#define HAVE_SCX_ENQ_LAST
+#define HAVE___SCX_ENQ_INTERNAL_MASK
+#define HAVE_SCX_ENQ_CLEAR_OPSS
+#define HAVE_SCX_ENQ_DSQ_PRIQ
+#define HAVE_SCX_TASK_DSQ_ON_PRIQ
+#define HAVE_SCX_TASK_QUEUED
+#define HAVE_SCX_TASK_RESET_RUNNABLE_AT
+#define HAVE_SCX_TASK_DEQD_FOR_SLEEP
+#define HAVE_SCX_TASK_STATE_SHIFT
+#define HAVE_SCX_TASK_STATE_BITS
+#define HAVE_SCX_TASK_STATE_MASK
+#define HAVE_SCX_TASK_CURSOR
+#define HAVE_SCX_ECODE_RSN_HOTPLUG
+#define HAVE_SCX_ECODE_ACT_RESTART
+#define HAVE_SCX_EXIT_NONE
+#define HAVE_SCX_EXIT_DONE
+#define HAVE_SCX_EXIT_UNREG
+#define HAVE_SCX_EXIT_UNREG_BPF
+#define HAVE_SCX_EXIT_UNREG_KERN
+#define HAVE_SCX_EXIT_SYSRQ
+#define HAVE_SCX_EXIT_ERROR
+#define HAVE_SCX_EXIT_ERROR_BPF
+#define HAVE_SCX_EXIT_ERROR_STALL
+#define HAVE_SCX_KF_UNLOCKED
+#define HAVE_SCX_KF_CPU_RELEASE
+#define HAVE_SCX_KF_DISPATCH
+#define HAVE_SCX_KF_ENQUEUE
+#define HAVE_SCX_KF_SELECT_CPU
+#define HAVE_SCX_KF_REST
+#define HAVE___SCX_KF_RQ_LOCKED
+#define HAVE___SCX_KF_TERMINAL
+#define HAVE_SCX_KICK_IDLE
+#define HAVE_SCX_KICK_PREEMPT
+#define HAVE_SCX_KICK_WAIT
+#define HAVE_SCX_OPI_BEGIN
+#define HAVE_SCX_OPI_NORMAL_BEGIN
+#define HAVE_SCX_OPI_NORMAL_END
+#define HAVE_SCX_OPI_CPU_HOTPLUG_BEGIN
+#define HAVE_SCX_OPI_CPU_HOTPLUG_END
+#define HAVE_SCX_OPI_END
+#define HAVE_SCX_OPS_ENABLING
+#define HAVE_SCX_OPS_ENABLED
+#define HAVE_SCX_OPS_DISABLING
+#define HAVE_SCX_OPS_DISABLED
+#define HAVE_SCX_OPS_KEEP_BUILTIN_IDLE
+#define HAVE_SCX_OPS_ENQ_LAST
+#define HAVE_SCX_OPS_ENQ_EXITING
+#define HAVE_SCX_OPS_SWITCH_PARTIAL
+#define HAVE_SCX_OPS_HAS_CGROUP_WEIGHT
+#define HAVE_SCX_OPS_ALL_FLAGS
+#define HAVE_SCX_OPSS_NONE
+#define HAVE_SCX_OPSS_QUEUEING
+#define HAVE_SCX_OPSS_QUEUED
+#define HAVE_SCX_OPSS_DISPATCHING
+#define HAVE_SCX_OPSS_QSEQ_SHIFT
+#define HAVE_SCX_PICK_IDLE_CORE
+#define HAVE_SCX_OPS_NAME_LEN
+#define HAVE_SCX_SLICE_DFL
+#define HAVE_SCX_SLICE_INF
+#define HAVE_SCX_RQ_ONLINE
+#define HAVE_SCX_RQ_CAN_STOP_TICK
+#define HAVE_SCX_RQ_BAL_PENDING
+#define HAVE_SCX_RQ_BAL_KEEP
+#define HAVE_SCX_RQ_BYPASSING
+#define HAVE_SCX_RQ_IN_WAKEUP
+#define HAVE_SCX_RQ_IN_BALANCE
+#define HAVE_SCX_TASK_NONE
+#define HAVE_SCX_TASK_INIT
+#define HAVE_SCX_TASK_READY
+#define HAVE_SCX_TASK_ENABLED
+#define HAVE_SCX_TASK_NR_STATES
+#define HAVE_SCX_TG_ONLINE
+#define HAVE_SCX_TG_INITED
+#define HAVE_SCX_WAKE_FORK
+#define HAVE_SCX_WAKE_TTWU
+#define HAVE_SCX_WAKE_SYNC
+
+#endif /* __ENUM_DEFS_AUTOGEN_H__ */
+
-- 
2.48.1

[PATCH 2/4] sched_ext: Include enum_defs.autogen.h from common/common.bpf.h

[Changwoo Min]

Update common.h and common.bpf.h to include enum_defs.autogen.h,
so C and BPF code can use HAVE_{enum name} macros.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
---
 tools/sched_ext/include/scx/common.bpf.h | 1 +
 tools/sched_ext/include/scx/common.h     | 1 +
 2 files changed, 2 insertions(+)

diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/include/scx/common.bpf.h
index ae717f4d6ede..f1caf9fc8f8c 100644
--- a/tools/sched_ext/include/scx/common.bpf.h
+++ b/tools/sched_ext/include/scx/common.bpf.h
@@ -18,6 +18,7 @@
 #include <bpf/bpf_tracing.h>
 #include <asm-generic/errno.h>
 #include "user_exit_info.h"
+#include "enum_defs.autogen.h"
 
 #define PF_WQ_WORKER			0x00000020	/* I'm a workqueue worker */
 #define PF_KTHREAD			0x00200000	/* I am a kernel thread */
diff --git a/tools/sched_ext/include/scx/common.h b/tools/sched_ext/include/scx/common.h
index dc18b99e55cd..1dc76bd84296 100644
--- a/tools/sched_ext/include/scx/common.h
+++ b/tools/sched_ext/include/scx/common.h
@@ -16,6 +16,7 @@
 #include <stdlib.h>
 #include <stdint.h>
 #include <errno.h>
+#include "enum_defs.autogen.h"
 
 typedef uint8_t u8;
 typedef uint16_t u16;
-- 
2.48.1

[PATCH 3/4] sched_ext: Add __COMPAT_is_enq_cpu_selected() for BPF compatability

[Changwoo Min]

Suppose a BPF scheduler is compiled against the vmlinux.h that has
SCX_ENQ_CPU_SELECTED, but it runs on an old kernel where SCX_ENQ_CPU_SELECTED
is not defined. In this case, the test result of
'enq_flags & SCX_ENQ_CPU_SELECTED' will always be false. That test result is
semantically incorrect because the kernel before SCX_ENQ_CPU_SELECTED has
never skipped select_task_rq_scx(), so the result should be true.
To hide such complexity, introduce __COMPAT_is_enq_cpu_selected(), which
checks if SCX_ENQ_CPU_SELECTED exists in runtime using BPF CO-RE.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
---
 tools/sched_ext/include/scx/compat.bpf.h | 52 ++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/tools/sched_ext/include/scx/compat.bpf.h b/tools/sched_ext/include/scx/compat.bpf.h
index 50e1499ae093..e5fa72f9bf22 100644
--- a/tools/sched_ext/include/scx/compat.bpf.h
+++ b/tools/sched_ext/include/scx/compat.bpf.h
@@ -125,6 +125,58 @@ bool scx_bpf_dispatch_vtime_from_dsq___compat(struct bpf_iter_scx_dsq *it__iter,
 	false;									\
 })
 
+/**
+ * __COMPAT_is_enq_cpu_selected - Test if SCX_ENQ_CPU_SELECTED is on
+ * in a compatible way. We will preserve this __COMPAT helper until v6.16.
+ *
+ * @enq_flags: enqueue flags from ops.enqueue()
+ *
+ * Return: True if SCX_ENQ_CPU_SELECTED is turned on in @enq_flags
+ */
+static inline bool __COMPAT_is_enq_cpu_selected(u64 enq_flags)
+{
+#ifdef HAVE_SCX_ENQ_CPU_SELECTED
+	/*
+	 * This is the case that a BPF code compiled against vmlinux.h
+	 * where the enum SCX_ENQ_CPU_SELECTED exists.
+	 */
+
+	/*
+	 * We should temporarily suspend the macro expansion of
+	 * 'SCX_ENQ_CPU_SELECTED'. This avoids 'SCX_ENQ_CPU_SELECTED' being
+	 * rewritten to '__SCX_ENQ_CPU_SELECTED' when 'SCX_ENQ_CPU_SELECTED'
+	 * is defined in 'scripts/gen_enums.py'.
+	 */
+#pragma push_macro("SCX_ENQ_CPU_SELECTED")
+#undef SCX_ENQ_CPU_SELECTED
+	u64 flag;
+
+	/*
+	 * When the kernel did not have SCX_ENQ_CPU_SELECTED,
+	 * select_task_rq_scx() has never been skipped. Thus, this case
+	 * should be considered that the CPU has already been selected.
+	 */
+	if (!bpf_core_enum_value_exists(enum scx_enq_flags,
+					SCX_ENQ_CPU_SELECTED))
+		return true;
+
+	flag = bpf_core_enum_value(enum scx_enq_flags, SCX_ENQ_CPU_SELECTED);
+	return enq_flags & flag;
+
+	/*
+	 * Once done, resume the macro expansion of 'SCX_ENQ_CPU_SELECTED'.
+	 */
+#pragma pop_macro("SCX_ENQ_CPU_SELECTED")
+#else
+	/*
+	 * This is the case that a BPF code compiled against vmlinux.h
+	 * where the enum SCX_ENQ_CPU_SELECTED does NOT exist.
+	 */
+	return true;
+#endif /* HAVE_SCX_ENQ_CPU_SELECTED */
+}
+
+
 #define scx_bpf_now()								\
 	(bpf_ksym_exists(scx_bpf_now) ?						\
 	 scx_bpf_now() :							\
-- 
2.48.1

[PATCH 4/4] sched_ext: Use __COMPAT_is_enq_cpu_selected() in scx_qmap

[Changwoo Min]

Instead of directly using SCX_ENQ_CPU_SELECTED, which may not be defined
in old kernels, use the __COMPAT_is_enq_cpu_selected() helper function
so the BPF scheduler can behave correctly regardless of kernel versions.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
---
 tools/sched_ext/scx_qmap.bpf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/sched_ext/scx_qmap.bpf.c b/tools/sched_ext/scx_qmap.bpf.c
index 5edb79742e37..267fe15d934a 100644
--- a/tools/sched_ext/scx_qmap.bpf.c
+++ b/tools/sched_ext/scx_qmap.bpf.c
@@ -231,7 +231,7 @@ void BPF_STRUCT_OPS(qmap_enqueue, struct task_struct *p, u64 enq_flags)
 	}
 
 	/* if select_cpu() wasn't called, try direct dispatch */
-	if (!(enq_flags & SCX_ENQ_CPU_SELECTED) &&
+	if (!__COMPAT_is_enq_cpu_selected(enq_flags) &&
 	    (cpu = pick_direct_dispatch_cpu(p, scx_bpf_task_cpu(p))) >= 0) {
 		__sync_fetch_and_add(&nr_ddsp_from_enq, 1);
 		scx_bpf_dsq_insert(p, SCX_DSQ_LOCAL_ON | cpu, slice_ns, enq_flags);
-- 
2.48.1

Re: [PATCH 0/4] sched_ext: Compatible testing of SCX_ENQ_CPU_SELECTED

[Tejun Heo]

On Sat, Feb 08, 2025 at 01:43:28PM +0900, Changwoo Min wrote:
> This patchset is for testing `SCX_ENQ_CPU_SELECTED` in a compatible way.
> More specifically, it handles two cases:
>   1. a BPF scheduler is compiled against vmlinux.h where
>   SCX_ENQ_CPU_SELECTED is defined, but it runs on a kernel that does not
>   have SCX_ENQ_CPU_SELECTED.
>   2. a BPF scheduler is compiling against vmlinux.h where
>   SCX_ENQ_CPU_SELECTED is not defined.
> 
> The patch consists of three parts:
>   1. Add enum_defs.autogen.h, which has macros denoting whether SCX
>   enums are defined in the vmlinux.h or not.
>   2. Implement __COMPAT_is_enq_cpu_selected(), which provide the test of
>   SCX_ENQ_CPU_SELECTED in a compatible way.
>   3. Use  __COMPAT_is_enq_cpu_selected() in scx_qmap.
> 
> Also, this patchset is a sync of the relevant PR [1] in the scx repo.
> 
> [1] https://github.com/sched-ext/scx/pull/1314
> 
> Changwoo Min (4):
>   sched_ext: Add enum_defs.autogen.h
>   sched_ext: Include enum_defs.autogen.h from common/common.bpf.h
>   sched_ext: Add __COMPAT_is_enq_cpu_selected() for BPF compatability
>   sched_ext: Use __COMPAT_is_enq_cpu_selected() in scx_qmap

Can you collapse the four patches into a single patch? I don't think this
needs to be split this granularly.

Thanks.

-- 
tejun

Re: [PATCH 0/4] sched_ext: Compatible testing of SCX_ENQ_CPU_SELECTED

[Changwoo Min]

Hello,

Thanks for the quick review!

On 25. 2. 8. 17:13, Tejun Heo wrote:
> On Sat, Feb 08, 2025 at 01:43:28PM +0900, Changwoo Min wrote:
>> Changwoo Min (4):
>>    sched_ext: Add enum_defs.autogen.h
>>    sched_ext: Include enum_defs.autogen.h from common/common.bpf.h
>>    sched_ext: Add __COMPAT_is_enq_cpu_selected() for BPF compatability
>>    sched_ext: Use __COMPAT_is_enq_cpu_selected() in scx_qmap
> 
> Can you collapse the four patches into a single patch? I don't think this
> needs to be split this granularly.

Sure, I will send out a collapsed version.

Regards,
Changwoo Min