* [RFC PATCH v2 05/15] s390/vdso: Avoid emitting DWARF CFI for non-vDSO
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
This replicates Josh's x86 commit TODO ("x86/asm: Avoid emitting DWARF
CFI for non-VDSO") for s390. It also aligns asm/dwarf.h to x86
asm/dwarf2.h.
It was decided years ago that .cfi_* annotations aren't maintainable in
the kernel. For the kernel proper, ensure the CFI_* macros don't do
anything.
On the other hand the vDSO library *does* use them, so user space can
unwind through it.
Make sure these macros only work for vDSO. They aren't actually being
used outside of vDSO anyway, so there's no functional change.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Link to latest x86 patch:
https://lore.kernel.org/all/20250425024022.477374378@goodmis.org/
arch/s390/include/asm/dwarf.h | 45 ++++++++++++++++++++++-------------
1 file changed, 29 insertions(+), 16 deletions(-)
diff --git a/arch/s390/include/asm/dwarf.h b/arch/s390/include/asm/dwarf.h
index df9f467910f7..6bcf37256feb 100644
--- a/arch/s390/include/asm/dwarf.h
+++ b/arch/s390/include/asm/dwarf.h
@@ -6,6 +6,18 @@
#warning "asm/dwarf.h should be only included in pure assembly files"
#endif
+.macro nocfi args:vararg
+.endm
+
+#ifdef BUILD_VDSO
+
+ /*
+ * For the vDSO, emit both runtime unwind information and debug
+ * symbols for the .dbg file.
+ */
+
+ .cfi_sections .eh_frame, .debug_frame
+
#define CFI_STARTPROC .cfi_startproc
#define CFI_ENDPROC .cfi_endproc
#define CFI_DEF_CFA_OFFSET .cfi_def_cfa_offset
@@ -16,23 +28,24 @@
#ifdef CONFIG_AS_CFI_VAL_OFFSET
#define CFI_VAL_OFFSET .cfi_val_offset
#else
-#define CFI_VAL_OFFSET #
+#define CFI_VAL_OFFSET nocfi
#endif
-#ifndef BUILD_VDSO
- /*
- * Emit CFI data in .debug_frame sections and not in .eh_frame
- * sections. The .eh_frame CFI is used for runtime unwind
- * information that is not being used. Hence, vmlinux.lds.S
- * can discard the .eh_frame sections.
- */
- .cfi_sections .debug_frame
-#else
- /*
- * For vDSO, emit CFI data in both, .eh_frame and .debug_frame
- * sections.
- */
- .cfi_sections .eh_frame, .debug_frame
-#endif
+#else /* !BUILD_VDSO */
+
+/*
+ * On s390, these macros aren't used outside vDSO. As well they shouldn't be:
+ * they're fragile and very difficult to maintain.
+ */
+
+#define CFI_STARTPROC nocfi
+#define CFI_ENDPROC nocfi
+#define CFI_DEF_CFA_OFFSET nocfi
+#define CFI_ADJUST_CFA_OFFSET nocfi
+#define CFI_RESTORE nocfi
+#define CFI_REL_OFFSET nocfi
+#define CFI_VAL_OFFSET nocfi
+
+#endif /* !BUILD_VDSO */
#endif /* _ASM_S390_DWARF_H */
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 04/15] s390: asm/dwarf.h should only be included in assembly files
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
Align to x86 and add a compile-time check that asm/dwarf.h is only
included in pure assembly files.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in RFC v2:
- Adjust to upstream change of __ASSEMBLY__ to __ASSEMBLER__.
arch/s390/include/asm/dwarf.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/s390/include/asm/dwarf.h b/arch/s390/include/asm/dwarf.h
index e3ad6798d0cd..df9f467910f7 100644
--- a/arch/s390/include/asm/dwarf.h
+++ b/arch/s390/include/asm/dwarf.h
@@ -2,7 +2,9 @@
#ifndef _ASM_S390_DWARF_H
#define _ASM_S390_DWARF_H
-#ifdef __ASSEMBLER__
+#ifndef __ASSEMBLER__
+#warning "asm/dwarf.h should be only included in pure assembly files"
+#endif
#define CFI_STARTPROC .cfi_startproc
#define CFI_ENDPROC .cfi_endproc
@@ -33,6 +35,4 @@
.cfi_sections .eh_frame, .debug_frame
#endif
-#endif /* __ASSEMBLER__ */
-
#endif /* _ASM_S390_DWARF_H */
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 11/15] unwind_user/sframe: Enable archs with encoded SFrame CFA offsets
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
Enable architectures, such as s390, which store SFrame CFA offset values
encoded, to e.g. make (better) use of unsigned 8-bit SFrame offsets.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in RFC v2:
- Rename generic_sframe_cfa_offset_decode() to
sframe_cfa_offset_decode(). (Josh)
include/asm-generic/unwind_user_sframe.h | 8 ++++++++
kernel/unwind/sframe.c | 1 +
2 files changed, 9 insertions(+)
diff --git a/include/asm-generic/unwind_user_sframe.h b/include/asm-generic/unwind_user_sframe.h
index 163961ca5252..80ae9bfaa88d 100644
--- a/include/asm-generic/unwind_user_sframe.h
+++ b/include/asm-generic/unwind_user_sframe.h
@@ -10,6 +10,14 @@
#define SFRAME_SP_OFFSET 0
#endif
+#ifndef sframe_cfa_offset_decode
+static inline s32 sframe_cfa_offset_decode(s32 offset)
+{
+ return offset;
+}
+#define sframe_cfa_offset_decode sframe_cfa_offset_decode
+#endif
+
#ifndef sframe_init_reginfo
static inline void
sframe_init_reginfo(struct unwind_user_reginfo *reginfo, s32 offset)
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index 45cd7380ac38..92f770fc21f6 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -226,6 +226,7 @@ static __always_inline int __read_fre(struct sframe_section *sec,
UNSAFE_GET_USER_INC(cfa_off, cur, offset_size, Efault);
offset_count--;
+ cfa_off = sframe_cfa_offset_decode(cfa_off);
ra_off = sec->ra_off;
if (!ra_off && offset_count) {
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 08/15] unwind_user: Enable archs that define CFA = SP_callsite + offset
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
Most architectures define their CFA as the value of the stack pointer
(SP) at the call site in the previous frame, as suggested by the DWARF
standard. Therefore the SP at call site can be unwound using an
implicitly assumed value offset from CFA rule with an offset of zero:
.cfi_val_offset <SP>, 0
As a result the SP at call site computes as follows:
SP = CFA
Enable unwinding of user space for architectures, such as s390, which
define their CFA as the value of the SP at the call site in the previous
frame with an offset. Do so by enabling architectures to override the
default SP value offset from CFA of zero with an architecture-specific
one:
.cfi_val_offset <SP>, offset
So that the SP at call site computes as follows:
SP = CFA + offset
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in RFC v2:
- Reword commit message. (Josh)
- Use term "sp_off" instead of "sp_val_off". (Josh)
- Move definition, initialization, and setting of sp_off field to
happen right after the cfa_off field.
- Use SFRAME_SP_OFFSET macro instead of sframe_sp_off() function,
which can be overridden by an architecture, such as s390.
- Drop lengthy sframe_sp_[val_]off() comment.
arch/x86/include/asm/unwind_user.h | 2 ++
include/asm-generic/Kbuild | 1 +
include/asm-generic/unwind_user_sframe.h | 12 ++++++++++++
include/linux/unwind_user_types.h | 1 +
kernel/unwind/sframe.c | 2 ++
kernel/unwind/user.c | 11 ++++++-----
6 files changed, 24 insertions(+), 5 deletions(-)
create mode 100644 include/asm-generic/unwind_user_sframe.h
diff --git a/arch/x86/include/asm/unwind_user.h b/arch/x86/include/asm/unwind_user.h
index 4d699e4954ed..dbdbad0beaf9 100644
--- a/arch/x86/include/asm/unwind_user.h
+++ b/arch/x86/include/asm/unwind_user.h
@@ -25,6 +25,7 @@ static inline int unwind_user_word_size(struct pt_regs *regs)
#define ARCH_INIT_USER_FP_FRAME(ws) \
.cfa_off = 2*(ws), \
+ .sp_off = 0, \
.ra_off = -1*(ws), \
.fp_off = -2*(ws), \
.use_fp = true, \
@@ -32,6 +33,7 @@ static inline int unwind_user_word_size(struct pt_regs *regs)
#define ARCH_INIT_USER_FP_ENTRY_FRAME(ws) \
.cfa_off = 1*(ws), \
+ .sp_off = 0, \
.ra_off = -1*(ws), \
.fp_off = 0, \
.use_fp = false, \
diff --git a/include/asm-generic/Kbuild b/include/asm-generic/Kbuild
index 295c94a3ccc1..b1d448ef4a50 100644
--- a/include/asm-generic/Kbuild
+++ b/include/asm-generic/Kbuild
@@ -60,6 +60,7 @@ mandatory-y += topology.h
mandatory-y += trace_clock.h
mandatory-y += uaccess.h
mandatory-y += unwind_user.h
+mandatory-y += unwind_user_sframe.h
mandatory-y += vermagic.h
mandatory-y += vga.h
mandatory-y += video.h
diff --git a/include/asm-generic/unwind_user_sframe.h b/include/asm-generic/unwind_user_sframe.h
new file mode 100644
index 000000000000..8c9ac47bc8bd
--- /dev/null
+++ b/include/asm-generic/unwind_user_sframe.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_GENERIC_UNWIND_USER_SFRAME_H
+#define _ASM_GENERIC_UNWIND_USER_SFRAME_H
+
+#include <linux/types.h>
+
+#ifndef SFRAME_SP_OFFSET
+/* Most archs/ABIs define CFA as SP at call site, so that SP = CFA + 0. */
+#define SFRAME_SP_OFFSET 0
+#endif
+
+#endif /* _ASM_GENERIC_UNWIND_USER_SFRAME_H */
diff --git a/include/linux/unwind_user_types.h b/include/linux/unwind_user_types.h
index 616cc5ee4586..4656aa08a7db 100644
--- a/include/linux/unwind_user_types.h
+++ b/include/linux/unwind_user_types.h
@@ -29,6 +29,7 @@ struct unwind_stacktrace {
struct unwind_user_frame {
s32 cfa_off;
+ s32 sp_off;
s32 ra_off;
s32 fp_off;
bool use_fp;
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index 6465e7a315bc..7952b041dd23 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -12,6 +12,7 @@
#include <linux/mm.h>
#include <linux/string_helpers.h>
#include <linux/sframe.h>
+#include <asm/unwind_user_sframe.h>
#include <linux/unwind_user_types.h>
#include "sframe.h"
@@ -307,6 +308,7 @@ static __always_inline int __find_fre(struct sframe_section *sec,
fre = prev_fre;
frame->cfa_off = fre->cfa_off;
+ frame->sp_off = SFRAME_SP_OFFSET;
frame->ra_off = fre->ra_off;
frame->fp_off = fre->fp_off;
frame->use_fp = SFRAME_FRE_CFA_BASE_REG_ID(fre->info) == SFRAME_BASE_REG_FP;
diff --git a/kernel/unwind/user.c b/kernel/unwind/user.c
index fdb1001e3750..6c75a7411871 100644
--- a/kernel/unwind/user.c
+++ b/kernel/unwind/user.c
@@ -30,7 +30,7 @@ get_user_word(unsigned long *word, unsigned long base, int off, unsigned int ws)
static int unwind_user_next_common(struct unwind_user_state *state,
const struct unwind_user_frame *frame)
{
- unsigned long cfa, fp, ra;
+ unsigned long cfa, sp, fp, ra;
/* Stop unwinding when reaching an outermost frame. */
if (frame->outermost) {
@@ -48,12 +48,13 @@ static int unwind_user_next_common(struct unwind_user_state *state,
}
cfa += frame->cfa_off;
+ /* Get the Stack Pointer (SP) */
+ sp = cfa + frame->sp_off;
/* Make sure that stack is not going in wrong direction */
- if (cfa <= state->sp)
+ if (sp <= state->sp)
return -EINVAL;
-
/* Make sure that the address is word aligned */
- if (cfa & (state->ws - 1))
+ if (sp & (state->ws - 1))
return -EINVAL;
/* Get the Return Address (RA) */
@@ -65,7 +66,7 @@ static int unwind_user_next_common(struct unwind_user_state *state,
return -EINVAL;
state->ip = ra;
- state->sp = cfa;
+ state->sp = sp;
if (frame->fp_off)
state->fp = fp;
state->topmost = false;
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 02/15] unwind_user/fp: Use dummies instead of ifdef
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
This simplifies the code. unwind_user_next_fp() does not need to
return -EINVAL if config option HAVE_UNWIND_USER_FP is disabled, as
unwind_user_start() will then not select this unwind method and
unwind_user_next() will therefore not call it.
Provide (1) a dummy definition of ARCH_INIT_USER_FP_FRAME, if the unwind
user method HAVE_UNWIND_USER_FP is not enabled, (2) a common fallback
definition of unwind_user_at_function_start() which returns false, and
(3) a common dummy definition of ARCH_INIT_USER_FP_ENTRY_FRAME.
Note that enabling the config option HAVE_UNWIND_USER_FP without
defining ARCH_INIT_USER_FP_FRAME triggers a compile error, which is
helpful when implementing support for this unwind user method in an
architecture. Enabling the config option when providing an arch-
specific unwind_user_at_function_start() definition makes it necessary
to also provide an arch-specific ARCH_INIT_USER_FP_ENTRY_FRAME
definition.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in v3:
- Remove comment on #endif. (Ingo)
Changes in v2:
- Add parameter ws to ARCH_INIT_USER_{FP_FRAME|FP_ENTRY_FRAME}.
- Provide common fallback of unwind_user_at_function_start().
- Provide common dummy of ARCH_INIT_USER_FP_ENTRY_FRAME.
- Reword commit message accordingly.
arch/x86/include/asm/unwind_user.h | 1 +
include/linux/unwind_user.h | 18 ++++++++++++++++--
kernel/unwind/user.c | 4 ----
3 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/arch/x86/include/asm/unwind_user.h b/arch/x86/include/asm/unwind_user.h
index f9a1c460150d..a528eee80dd6 100644
--- a/arch/x86/include/asm/unwind_user.h
+++ b/arch/x86/include/asm/unwind_user.h
@@ -37,6 +37,7 @@ static inline bool unwind_user_at_function_start(struct pt_regs *regs)
{
return is_uprobe_at_func_entry(regs);
}
+#define unwind_user_at_function_start unwind_user_at_function_start
#endif /* CONFIG_HAVE_UNWIND_USER_FP */
diff --git a/include/linux/unwind_user.h b/include/linux/unwind_user.h
index 7f7282516bf5..64618618febd 100644
--- a/include/linux/unwind_user.h
+++ b/include/linux/unwind_user.h
@@ -5,8 +5,22 @@
#include <linux/unwind_user_types.h>
#include <asm/unwind_user.h>
-#ifndef ARCH_INIT_USER_FP_FRAME
- #define ARCH_INIT_USER_FP_FRAME
+#ifndef CONFIG_HAVE_UNWIND_USER_FP
+
+#define ARCH_INIT_USER_FP_FRAME(ws)
+
+#endif
+
+#ifndef ARCH_INIT_USER_FP_ENTRY_FRAME
+#define ARCH_INIT_USER_FP_ENTRY_FRAME(ws)
+#endif
+
+#ifndef unwind_user_at_function_start
+static inline bool unwind_user_at_function_start(struct pt_regs *regs)
+{
+ return false;
+}
+#define unwind_user_at_function_start unwind_user_at_function_start
#endif
int unwind_user(struct unwind_stacktrace *trace, unsigned int max_entries);
diff --git a/kernel/unwind/user.c b/kernel/unwind/user.c
index f81c36ab2861..fdb1001e3750 100644
--- a/kernel/unwind/user.c
+++ b/kernel/unwind/user.c
@@ -74,7 +74,6 @@ static int unwind_user_next_common(struct unwind_user_state *state,
static int unwind_user_next_fp(struct unwind_user_state *state)
{
-#ifdef CONFIG_HAVE_UNWIND_USER_FP
struct pt_regs *regs = task_pt_regs(current);
if (state->topmost && unwind_user_at_function_start(regs)) {
@@ -88,9 +87,6 @@ static int unwind_user_next_fp(struct unwind_user_state *state)
ARCH_INIT_USER_FP_FRAME(state->ws)
};
return unwind_user_next_common(state, &fp_frame);
-#else
- return -EINVAL;
-#endif
}
static int unwind_user_next_sframe(struct unwind_user_state *state)
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 03/15] x86/unwind_user: Guard unwind_user_word_size() by UNWIND_USER
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
The unwind user framework in general requires an architecture-specific
implementation of unwind_user_word_size() to be present for any unwind
method, whether that is fp or a future other method, such as potentially
sframe.
Guard unwind_user_word_size() by the availability of the UNWIND_USER
framework instead of the specific HAVE_UNWIND_USER_FP method.
This facilitates to selectively disable HAVE_UNWIND_USER_FP on x86
(e.g. for test purposes) once a new unwind method is added to unwind
user.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in v3:
- Move includes into more common UNWIND_USER guard at the top of the
source. asm/ptrace.h is required for struct pt_regs.
arch/x86/include/asm/unwind_user.h | 30 +++++++++++++++++-------------
1 file changed, 17 insertions(+), 13 deletions(-)
diff --git a/arch/x86/include/asm/unwind_user.h b/arch/x86/include/asm/unwind_user.h
index a528eee80dd6..4d699e4954ed 100644
--- a/arch/x86/include/asm/unwind_user.h
+++ b/arch/x86/include/asm/unwind_user.h
@@ -2,11 +2,27 @@
#ifndef _ASM_X86_UNWIND_USER_H
#define _ASM_X86_UNWIND_USER_H
-#ifdef CONFIG_HAVE_UNWIND_USER_FP
+#ifdef CONFIG_UNWIND_USER
#include <asm/ptrace.h>
#include <asm/uprobes.h>
+static inline int unwind_user_word_size(struct pt_regs *regs)
+{
+ /* We can't unwind VM86 stacks */
+ if (regs->flags & X86_VM_MASK)
+ return 0;
+#ifdef CONFIG_X86_64
+ if (!user_64bit_mode(regs))
+ return sizeof(int);
+#endif
+ return sizeof(long);
+}
+
+#endif /* CONFIG_UNWIND_USER */
+
+#ifdef CONFIG_HAVE_UNWIND_USER_FP
+
#define ARCH_INIT_USER_FP_FRAME(ws) \
.cfa_off = 2*(ws), \
.ra_off = -1*(ws), \
@@ -21,18 +37,6 @@
.use_fp = false, \
.outermost = false,
-static inline int unwind_user_word_size(struct pt_regs *regs)
-{
- /* We can't unwind VM86 stacks */
- if (regs->flags & X86_VM_MASK)
- return 0;
-#ifdef CONFIG_X86_64
- if (!user_64bit_mode(regs))
- return sizeof(int);
-#endif
- return sizeof(long);
-}
-
static inline bool unwind_user_at_function_start(struct pt_regs *regs)
{
return is_uprobe_at_func_entry(regs);
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 10/15] unwind_user: Enable archs that save RA/FP in other registers
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
Enable unwinding of user space for architectures, such as s390, that
save the return address (RA) and/or frame pointer (FP) in other
registers. This is only valid in the topmost frame, for instance when
in a leaf function.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in RFC v2:
- Reword HAVE_UNWIND_USER_LOC_REG help text.
- Rename struct unwind_user_reginfo field frame_off to offset. (Josh)
- Move dummy unwind_user_get_reg() from asm-generic/unwind_user.h
to linux/unwind_user.h, drop its function comment, warn once,
return -EINVAL, and guard by !HAVE_UNWIND_USER_LOC_REG. (Josh)
- Rename generic_sframe_set_frame_reginfo() to sframe_init_reginfo()
and drop its function comment. (Josh)
- Do not check FP/RA offset for zero for UNWIND_USER_LOC_STACK. (Josh)
- Do not check for !IS_ENABLED(CONFIG_HAVE_UNWIND_USER_LOC_REG), as
the dummy implementation of unwind_user_get_reg() returns -EINVAL.
- Drop config option HAVE_UNWIND_USER_LOC_REG, as it is no longer of
any value.
- Keep checking for topmost for UNWIND_USER_LOC_REG. (Jens)
- Explicitly preserve FP if UNWIND_USER_LOC_NONE and drop later test
for frame->fp.loc != UNWIND_USER_LOC_NONE. (Josh)
Would it make sense to rename UNWIND_USER_LOC_NONE to one of the
following to clarify its meaning for the unwinder?
- UNWIND_USER_LOC_UNCHANGED
- UNWIND_USER_LOC_RETAIN
- UNWIND_USER_LOC_PRESERVED
- UNWIND_USER_LOC_IDENTITY
arch/x86/include/asm/unwind_user.h | 21 +++++++++++---
include/asm-generic/unwind_user_sframe.h | 15 ++++++++++
include/linux/unwind_user.h | 9 ++++++
include/linux/unwind_user_types.h | 18 ++++++++++--
kernel/unwind/sframe.c | 4 +--
kernel/unwind/user.c | 37 +++++++++++++++++++-----
6 files changed, 89 insertions(+), 15 deletions(-)
diff --git a/arch/x86/include/asm/unwind_user.h b/arch/x86/include/asm/unwind_user.h
index dbdbad0beaf9..61a9ae9b07ea 100644
--- a/arch/x86/include/asm/unwind_user.h
+++ b/arch/x86/include/asm/unwind_user.h
@@ -26,16 +26,27 @@ static inline int unwind_user_word_size(struct pt_regs *regs)
#define ARCH_INIT_USER_FP_FRAME(ws) \
.cfa_off = 2*(ws), \
.sp_off = 0, \
- .ra_off = -1*(ws), \
- .fp_off = -2*(ws), \
+ .ra = { \
+ .loc = UNWIND_USER_LOC_STACK,\
+ .offset = -1*(ws), \
+ }, \
+ .fp = { \
+ .loc = UNWIND_USER_LOC_STACK,\
+ .offset = -2*(ws), \
+ }, \
.use_fp = true, \
.outermost = false,
#define ARCH_INIT_USER_FP_ENTRY_FRAME(ws) \
.cfa_off = 1*(ws), \
.sp_off = 0, \
- .ra_off = -1*(ws), \
- .fp_off = 0, \
+ .ra = { \
+ .loc = UNWIND_USER_LOC_STACK,\
+ .offset = -1*(ws), \
+ }, \
+ .fp = { \
+ .loc = UNWIND_USER_LOC_NONE, \
+ }, \
.use_fp = false, \
.outermost = false,
@@ -47,4 +58,6 @@ static inline bool unwind_user_at_function_start(struct pt_regs *regs)
#endif /* CONFIG_HAVE_UNWIND_USER_FP */
+#include <asm-generic/unwind_user.h>
+
#endif /* _ASM_X86_UNWIND_USER_H */
diff --git a/include/asm-generic/unwind_user_sframe.h b/include/asm-generic/unwind_user_sframe.h
index 8c9ac47bc8bd..163961ca5252 100644
--- a/include/asm-generic/unwind_user_sframe.h
+++ b/include/asm-generic/unwind_user_sframe.h
@@ -2,6 +2,7 @@
#ifndef _ASM_GENERIC_UNWIND_USER_SFRAME_H
#define _ASM_GENERIC_UNWIND_USER_SFRAME_H
+#include <linux/unwind_user_types.h>
#include <linux/types.h>
#ifndef SFRAME_SP_OFFSET
@@ -9,4 +10,18 @@
#define SFRAME_SP_OFFSET 0
#endif
+#ifndef sframe_init_reginfo
+static inline void
+sframe_init_reginfo(struct unwind_user_reginfo *reginfo, s32 offset)
+{
+ if (offset) {
+ reginfo->loc = UNWIND_USER_LOC_STACK;
+ reginfo->offset = offset;
+ } else {
+ reginfo->loc = UNWIND_USER_LOC_NONE;
+ }
+}
+#define sframe_init_reginfo sframe_init_reginfo
+#endif
+
#endif /* _ASM_GENERIC_UNWIND_USER_SFRAME_H */
diff --git a/include/linux/unwind_user.h b/include/linux/unwind_user.h
index bc2edae39955..61fd5c05d0f0 100644
--- a/include/linux/unwind_user.h
+++ b/include/linux/unwind_user.h
@@ -32,6 +32,15 @@ static inline int unwind_user_get_ra_reg(unsigned long *val)
#define unwind_user_get_ra_reg unwind_user_get_ra_reg
#endif
+#ifndef unwind_user_get_reg
+static inline int unwind_user_get_reg(unsigned long *val, int regnum)
+{
+ WARN_ON_ONCE(1);
+ return -EINVAL;
+}
+#define unwind_user_get_reg unwind_user_get_reg
+#endif
+
int unwind_user(struct unwind_stacktrace *trace, unsigned int max_entries);
#endif /* _LINUX_UNWIND_USER_H */
diff --git a/include/linux/unwind_user_types.h b/include/linux/unwind_user_types.h
index 4656aa08a7db..6efc12b6e831 100644
--- a/include/linux/unwind_user_types.h
+++ b/include/linux/unwind_user_types.h
@@ -27,11 +27,25 @@ struct unwind_stacktrace {
unsigned long *entries;
};
+enum unwind_user_loc {
+ UNWIND_USER_LOC_NONE,
+ UNWIND_USER_LOC_STACK,
+ UNWIND_USER_LOC_REG,
+};
+
+struct unwind_user_reginfo {
+ enum unwind_user_loc loc;
+ union {
+ s32 offset;
+ int regnum;
+ };
+};
+
struct unwind_user_frame {
s32 cfa_off;
s32 sp_off;
- s32 ra_off;
- s32 fp_off;
+ struct unwind_user_reginfo ra;
+ struct unwind_user_reginfo fp;
bool use_fp;
bool outermost;
};
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index 38b3577f5253..45cd7380ac38 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -307,8 +307,8 @@ static __always_inline int __find_fre(struct sframe_section *sec,
frame->cfa_off = fre->cfa_off;
frame->sp_off = SFRAME_SP_OFFSET;
- frame->ra_off = fre->ra_off;
- frame->fp_off = fre->fp_off;
+ sframe_init_reginfo(&frame->ra, fre->ra_off);
+ sframe_init_reginfo(&frame->fp, fre->fp_off);
frame->use_fp = SFRAME_FRE_CFA_BASE_REG_ID(fre->info) == SFRAME_BASE_REG_FP;
frame->outermost = fre->ra_undefined;
diff --git a/kernel/unwind/user.c b/kernel/unwind/user.c
index 58e1549cd9f4..122045cb411f 100644
--- a/kernel/unwind/user.c
+++ b/kernel/unwind/user.c
@@ -62,22 +62,45 @@ static int unwind_user_next_common(struct unwind_user_state *state,
return -EINVAL;
/* Get the Return Address (RA) */
- if (frame->ra_off) {
- if (get_user_word(&ra, cfa, frame->ra_off, state->ws))
- return -EINVAL;
- } else {
+ switch (frame->ra.loc) {
+ case UNWIND_USER_LOC_NONE:
if (!state->topmost || unwind_user_get_ra_reg(&ra))
return -EINVAL;
+ break;
+ case UNWIND_USER_LOC_STACK:
+ if (get_user_word(&ra, cfa, frame->ra.offset, state->ws))
+ return -EINVAL;
+ break;
+ case UNWIND_USER_LOC_REG:
+ if (!state->topmost || unwind_user_get_reg(&ra, frame->ra.regnum))
+ return -EINVAL;
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ return -EINVAL;
}
/* Get the Frame Pointer (FP) */
- if (frame->fp_off && get_user_word(&fp, cfa, frame->fp_off, state->ws))
+ switch (frame->fp.loc) {
+ case UNWIND_USER_LOC_NONE:
+ fp = state->fp;
+ break;
+ case UNWIND_USER_LOC_STACK:
+ if (get_user_word(&fp, cfa, frame->fp.offset, state->ws))
+ return -EINVAL;
+ break;
+ case UNWIND_USER_LOC_REG:
+ if (!state->topmost || unwind_user_get_reg(&fp, frame->fp.regnum))
+ return -EINVAL;
+ break;
+ default:
+ WARN_ON_ONCE(1);
return -EINVAL;
+ }
state->ip = ra;
state->sp = sp;
- if (frame->fp_off)
- state->fp = fp;
+ state->fp = fp;
state->topmost = false;
return 0;
}
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 00/15] s390: SFrame user space unwinding
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
This RFC series adds s390 support for unwinding of user space using
SFrame. It is based on Josh's, Steven's, and my work (see prerequisites
below). The generic unwind user (sframe) frameworks are extended to
enable support for a few s390-particularities (see patches 8-11),
including unwinding of user space using back chain (see patch 12).
The latter could be broken apart as a separate patch series.
Changes in RFC v2:
- Rebased on latest "unwind user" enhancements from Peter Zijlstra and
my latest "unwind user sframe" series v12.
- Incorporated RFC v1 review feedback.
- No new config options (except for unwind user backchain).
Motivation:
On s390 unwinding using frame pointer (FP) is unsupported, because of
lack of proper s390 64-bit (s390x) ABI specification and compiler
support. The ABI does only specify a "preferred" FP register. Both GCC
and Clang, regardless of compiler option -fno-omit-frame-pointer, setup
the preferred FP register as late as possible, which usually is after
static stack allocation, so that the CFA cannot be deduced from the FP
without any further data, such as provided by DWARF CFI or SFrame.
In theory there is a s390-specific alternative of unwinding using
back chain (compiler option -mbackchain), but this has its own
limitations. Ubuntu is currently the only distribution that that
builds user space with back chain.
As a consequence the Kernel stack tracer cannot unwind user space
(except if it is built with back chain). Recording call graphs of user
space using perf is limited to stack dump sampling (i.e. perf record
--call-graph dwarf), which generates a fairly large amount of data and
has limitations.
Initial testing of recording call graphs using perf using the s390
support for SFrame provided by RFC v1 of this series shows that
data size notably improves:
perf record data size is greatly reduced (smaller perf.data):
SFrame (--call-graph fp):
# perf record -F 9999 --call-graph fp objdump -wdWF objdump
[ perf record: Woken up 9 times to write data ]
[ perf record: Captured and wrote 2.498 MB perf.data (10891 samples) ]
Stack sampling (--call-graph dwarf) with a default stack size of 8192:
# perf record -F 9999 --call-graph dwarf objdump -wdWF objdump
[ perf record: Woken up 270 times to write data ]
[ perf record: Captured and wrote 67.467 MB perf.data (8241 samples) ]
Prerequirements:
This RFC series applies on top of the latest unwind user sframe series
"[PATCH v12 00/13] unwind_deferred: Implement sframe handling":
https://lore.kernel.org/all/20251119132323.1281768-1-jremus@linux.ibm.com/
It depends on binutils 2.45 to build executables and libraries
(e.g. vDSO) with SFrame on s390, including the latest SFrame V2 with
PC-relative FDE encoding. Optionally a binutils mainline build is
required for SFrame V2 with outermost frame indication.
The unwind user sframe series depends on a Glibc patch from Josh, that
adds support for the prctls introduced in the Kernel:
https://lore.kernel.org/all/20250122023517.lmztuocecdjqzfhc@jpoimboe/
Note that Josh's Glibc patch needs to be adjusted for the updated prctl
numbers from "[PATCH v12 13/13] unwind_user/sframe: Add prctl()
interface for registering .sframe sections":
https://lore.kernel.org/all/20251119132323.1281768-14-jremus@linux.ibm.com/
Overview:
Patches 1-3 originate from my "[PATCH v3 0/3] unwind_user: Cleanups"
series and can be ignored here (please review in the respective series):
https://lore.kernel.org/all/20251202150814.3505869-1-jremus@linux.ibm.com/
Patch 4 aligns asm/dwarf.h to x86 asm/dwarf2.h.
Patch 5 replicates Josh's x86 patch "x86/asm: Avoid emitting DWARF
CFI for non-VDSO" for s390.
Patch 6 changes the build of the vDSO on s390 to keep the function
symbols for stack tracing purposes.
Patch 7 replicates Josh's patch "x86/vdso: Enable sframe generation
in VDSO" for s390. It enables generation of SFrame stack trace
information (.sframe section) for the vDSO if the assembler supports it.
Patches 8-11 enable Josh's generic unwind user (sframe) frameworks to
support the following s390 particularities:
- Patch 8 adds support for architectures that define their CFA as SP at
callsite + offset.
- Patch 9 adds support support for architectures that pass the return
address (RA) in a register instead of on the stack and that do not
necessarily save the RA on the stack (or in another register) in the
topmost frame (e.g. in the prologue or in leaf functions).
- Patch 10 adds support for architectures that save RA/FP in other
registers instead of on the stack, e.g. in leaf functions.
- Patch 11 adds support for architectures that store the CFA offset
from CFA base register (e.g. SP or FP) in SFrame encoded. For
instance on s390 the CFA offset is stored adjusted by -160 and
then scaled down by 8 to enable and improve the use of signed 8-bit
SFrame offsets (i.e. CFA, RA, and FP offset).
Patch 12 introduces frame_pointer() in ptrace on s390, which is a
prerequisite for enabling unwind user.
Patch 13 adds support for unwinding of user space using SFrame on
s390. It leverages the extensions of the generic unwind user (sframe)
frameworks from patches 8-11.
Patch 14 introduces unwind user backchain to perform unwinding of user
space using back chain, such as on s390.
Patch 15 adds support for unwinding of user space using back chain on
s390. Main reasons to support back chain on s390 are:
- With Ubuntu there is a major distribution that builds user space with
back chain.
- Java JREs, such as OpenJDK, do maintain the back chain in jitted code.
Limitations:
Unwinding of user space using back chain cannot - by design - restore
the FP. Therefore unwinding of subsequent frames using e.g. SFrame may
fail, if the FP is the CFA base register.
Thanks and regards,
Jens
Jens Remus (15):
unwind_user: Enhance comments on get CFA, FP, and RA
unwind_user/fp: Use dummies instead of ifdef
x86/unwind_user: Guard unwind_user_word_size() by UNWIND_USER
s390: asm/dwarf.h should only be included in assembly files
s390/vdso: Avoid emitting DWARF CFI for non-vDSO
s390/vdso: Keep function symbols in vDSO
s390/vdso: Enable SFrame generation in vDSO
unwind_user: Enable archs that define CFA = SP_callsite + offset
unwind_user: Enable archs that pass RA in a register
unwind_user: Enable archs that save RA/FP in other registers
unwind_user/sframe: Enable archs with encoded SFrame CFA offsets
s390/ptrace: Provide frame_pointer()
s390/unwind_user/sframe: Enable HAVE_UNWIND_USER_SFRAME
unwind_user/backchain: Introduce back chain user space unwinding
s390/unwind_user/backchain: Enable HAVE_UNWIND_USER_BACKCHAIN
arch/Kconfig | 9 ++
arch/s390/Kconfig | 2 +
arch/s390/include/asm/dwarf.h | 53 ++++++----
arch/s390/include/asm/ptrace.h | 18 +++-
arch/s390/include/asm/unwind_user.h | 100 ++++++++++++++++++
arch/s390/include/asm/unwind_user_sframe.h | 33 ++++++
arch/s390/kernel/Makefile | 2 +
arch/s390/kernel/unwind_user_backchain.c | 112 +++++++++++++++++++++
arch/s390/kernel/vdso64/Makefile | 9 +-
arch/s390/kernel/vdso64/vdso64.lds.S | 9 ++
arch/x86/include/asm/unwind_user.h | 50 ++++++---
include/asm-generic/Kbuild | 1 +
include/asm-generic/unwind_user_sframe.h | 35 +++++++
include/linux/unwind_user.h | 36 ++++++-
include/linux/unwind_user_backchain.h | 20 ++++
include/linux/unwind_user_types.h | 21 +++-
kernel/unwind/sframe.c | 13 +--
kernel/unwind/sframe.h | 14 +++
kernel/unwind/user.c | 75 ++++++++++----
19 files changed, 548 insertions(+), 64 deletions(-)
create mode 100644 arch/s390/include/asm/unwind_user.h
create mode 100644 arch/s390/include/asm/unwind_user_sframe.h
create mode 100644 arch/s390/kernel/unwind_user_backchain.c
create mode 100644 include/asm-generic/unwind_user_sframe.h
create mode 100644 include/linux/unwind_user_backchain.h
--
2.51.0
^ permalink raw reply
* [RFC PATCH v2 07/15] s390/vdso: Enable SFrame generation in vDSO
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
This replicates Josh's x86 patch "x86/vdso: Enable sframe generation
in VDSO" [1] for s390.
Test whether the assembler supports generating SFrame stack trace
information. Note that it is insufficient to test whether the assembler
supports option --gsframe, as GNU assembler supports that regardless
of whether it is actually capable of generating SFrame stack trace
information for the architecture.
If so enable SFrame stack trace information generation in the vDSO
library so kernel and user space can unwind through it.
[1]: x86/vdso: Enable sframe generation in VDSO,
https://lore.kernel.org/all/20250425024023.173709192@goodmis.org/
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in RFC v2:
- Introduce config option AS_SFRAME instead of requiring Josh's x86
patch as pre-requisite.
- Reword commit message.
Link to Josh's latest x86 patch:
https://lore.kernel.org/all/20250425024023.173709192@goodmis.org/
arch/Kconfig | 3 +++
arch/s390/include/asm/dwarf.h | 4 ++++
arch/s390/kernel/vdso64/Makefile | 7 ++++++-
arch/s390/kernel/vdso64/vdso64.lds.S | 9 +++++++++
4 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/arch/Kconfig b/arch/Kconfig
index 06c4f909398c..7fa89d70b244 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -482,6 +482,9 @@ config HAVE_HARDLOCKUP_DETECTOR_ARCH
It uses the same command line parameters, and sysctl interface,
as the generic hardlockup detectors.
+config AS_SFRAME
+ def_bool $(as-instr,.cfi_sections .sframe\n.cfi_startproc\n.cfi_endproc)
+
config UNWIND_USER
bool
diff --git a/arch/s390/include/asm/dwarf.h b/arch/s390/include/asm/dwarf.h
index 6bcf37256feb..2f148b15fd7d 100644
--- a/arch/s390/include/asm/dwarf.h
+++ b/arch/s390/include/asm/dwarf.h
@@ -16,7 +16,11 @@
* symbols for the .dbg file.
*/
+#ifdef CONFIG_AS_SFRAME
+ .cfi_sections .eh_frame, .debug_frame, .sframe
+#else
.cfi_sections .eh_frame, .debug_frame
+#endif
#define CFI_STARTPROC .cfi_startproc
#define CFI_ENDPROC .cfi_endproc
diff --git a/arch/s390/kernel/vdso64/Makefile b/arch/s390/kernel/vdso64/Makefile
index 8e78dc3ba025..f597f3b863d7 100644
--- a/arch/s390/kernel/vdso64/Makefile
+++ b/arch/s390/kernel/vdso64/Makefile
@@ -20,7 +20,11 @@ targets := $(obj-vdso64) $(obj-cvdso64) vdso64.so vdso64.so.dbg
obj-vdso64 := $(addprefix $(obj)/, $(obj-vdso64))
obj-cvdso64 := $(addprefix $(obj)/, $(obj-cvdso64))
-KBUILD_AFLAGS += -DBUILD_VDSO
+ifeq ($(CONFIG_AS_SFRAME),y)
+ SFRAME_CFLAGS := -Wa,--gsframe
+endif
+
+KBUILD_AFLAGS += -DBUILD_VDSO $(SFRAME_CFLAGS)
KBUILD_CFLAGS += -DBUILD_VDSO -DDISABLE_BRANCH_PROFILING
KBUILD_AFLAGS_64 := $(filter-out -m64,$(KBUILD_AFLAGS))
@@ -32,6 +36,7 @@ KBUILD_CFLAGS_64 := $(filter-out -mno-pic-data-is-text-relative,$(KBUILD_CFLAGS_
KBUILD_CFLAGS_64 := $(filter-out -munaligned-symbols,$(KBUILD_CFLAGS_64))
KBUILD_CFLAGS_64 := $(filter-out -fno-asynchronous-unwind-tables,$(KBUILD_CFLAGS_64))
KBUILD_CFLAGS_64 += -m64 -fPIC -fno-common -fno-builtin -fasynchronous-unwind-tables
+KBUILD_CFLAGS_64 += $(SFRAME_CFLAGS)
ldflags-y := -shared -soname=linux-vdso64.so.1 \
--hash-style=both --build-id=sha1 -T
diff --git a/arch/s390/kernel/vdso64/vdso64.lds.S b/arch/s390/kernel/vdso64/vdso64.lds.S
index e4f6551ae898..0205d84369ca 100644
--- a/arch/s390/kernel/vdso64/vdso64.lds.S
+++ b/arch/s390/kernel/vdso64/vdso64.lds.S
@@ -50,6 +50,11 @@ SECTIONS
.eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr
.eh_frame : { KEEP (*(.eh_frame)) } :text
+
+#ifdef CONFIG_AS_SFRAME
+ .sframe : { *(.sframe) } :text :sframe
+#endif
+
.gcc_except_table : { *(.gcc_except_table .gcc_except_table.*) }
.rela.dyn ALIGN(8) : { *(.rela.dyn) }
@@ -114,6 +119,7 @@ SECTIONS
* Very old versions of ld do not recognize this name token; use the constant.
*/
#define PT_GNU_EH_FRAME 0x6474e550
+#define PT_GNU_SFRAME 0x6474e554
/*
* We must supply the ELF program headers explicitly to get just one
@@ -125,6 +131,9 @@ PHDRS
dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
note PT_NOTE FLAGS(4); /* PF_R */
eh_frame_hdr PT_GNU_EH_FRAME;
+#ifdef CONFIG_AS_SFRAME
+ sframe PT_GNU_SFRAME;
+#endif
}
/*
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 13/15] s390/unwind_user/sframe: Enable HAVE_UNWIND_USER_SFRAME
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
Add s390 support for unwinding of user space using SFrame. This
leverages the previous commits to address the following s390
particularities:
- The CFA is defined as the value of the stack pointer (SP) at call
site in the previous frame + 160. Therefore the SP unwinds as
SP = CFA - 160. Therefore use a SP value offset from CFA of -160.
- The return address (RA) is not saved on the stack at function entry.
It is also not saved in the function prologue, when in leaf functions.
Therefore the RA does not necessarily need to be unwound in the first
unwinding step for the topmost frame.
- The frame pointer (FP) and/or return address (RA) may be saved in
other registers when in leaf functions. GCC effectively uses
floating-point registers (FPR) for this purpose. Therefore DWARF
register numbers may be encoded in the SFrame FP/RA offsets.
- To make use of the signed 8-bit SFrame offset size and effectively
reduce the .sframe section size the SFrame CFA offset values are
encoded as (CFA - 160) / 8. This is because the lowest CFA offset
value on s390 is by definition +160 (= value at function entry),
which does not fit into a signed 8-bit SFrame offset. Therefore
the CFA offset values are stored adjusted by -160. Additionally
they are scaled by the s390-specific DWARF data scaling factor of 8.
The s390x ELF ABI [1] guarantees that the CFA offset values are
always aligned on an 8-byte boundary.
Add s390-specific SFrame format definitions. Note that SFRAME_ABI_*
(and thus SFRAME_ABI_S390_ENDIAN_BIG) is currently unused.
Include <asm/unwind_user_sframe.h> after "sframe.h" to make those
s390-specific definitions available to architecture-specific unwind
user sframe code, particularly the s390-specific one.
[1]: s390x ELF ABI, https://github.com/IBM/s390x-abi/releases
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in RFC v2:
- Provide unwind_user_word_size() to satisfy new unwind user need. Note
that support for COMPAT has not been implemented as s390 support for
COMPAT is expected to be removed with v6.19:
https://lore.kernel.org/all/20251201102713.22472A5b-hca@linux.ibm.com/
- Adjust to changes in preceding patches in this series that enable
support in unwind user (sframe) for s390 particularities.
Alternatively the s390-specific definitions could also be added to the
s390-specific unwind user sframe header. The current implementation
follows Binutils approach to have all SFrame format definitions in one
central header file.
arch/s390/Kconfig | 1 +
arch/s390/include/asm/unwind_user.h | 100 +++++++++++++++++++++
arch/s390/include/asm/unwind_user_sframe.h | 33 +++++++
kernel/unwind/sframe.c | 2 +-
kernel/unwind/sframe.h | 14 +++
5 files changed, 149 insertions(+), 1 deletion(-)
create mode 100644 arch/s390/include/asm/unwind_user.h
create mode 100644 arch/s390/include/asm/unwind_user_sframe.h
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index df22b10d9141..52d3f3b3e086 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -246,6 +246,7 @@ config S390
select HAVE_SETUP_PER_CPU_AREA
select HAVE_SOFTIRQ_ON_OWN_STACK
select HAVE_SYSCALL_TRACEPOINTS
+ select HAVE_UNWIND_USER_SFRAME
select HAVE_VIRT_CPU_ACCOUNTING
select HAVE_VIRT_CPU_ACCOUNTING_IDLE
select HOTPLUG_SMT
diff --git a/arch/s390/include/asm/unwind_user.h b/arch/s390/include/asm/unwind_user.h
new file mode 100644
index 000000000000..38dff49d5f86
--- /dev/null
+++ b/arch/s390/include/asm/unwind_user.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_UNWIND_USER_H
+#define _ASM_S390_UNWIND_USER_H
+
+#include <linux/sched/task_stack.h>
+#include <linux/types.h>
+#include <asm/fpu-insn.h>
+
+#ifdef CONFIG_UNWIND_USER
+
+static inline int unwind_user_word_size(struct pt_regs *regs)
+{
+ return sizeof(long);
+}
+
+static inline int arch_unwind_user_get_ra_reg(unsigned long *val)
+{
+ struct pt_regs *regs = task_pt_regs(current);
+ *val = regs->gprs[14];
+ return 0;
+}
+#define unwind_user_get_ra_reg arch_unwind_user_get_ra_reg
+
+static inline int __s390_get_dwarf_fpr(unsigned long *val, int regnum)
+{
+ switch (regnum) {
+ case 16:
+ fpu_std(0, (freg_t *)val);
+ break;
+ case 17:
+ fpu_std(2, (freg_t *)val);
+ break;
+ case 18:
+ fpu_std(4, (freg_t *)val);
+ break;
+ case 19:
+ fpu_std(6, (freg_t *)val);
+ break;
+ case 20:
+ fpu_std(1, (freg_t *)val);
+ break;
+ case 21:
+ fpu_std(3, (freg_t *)val);
+ break;
+ case 22:
+ fpu_std(5, (freg_t *)val);
+ break;
+ case 23:
+ fpu_std(7, (freg_t *)val);
+ break;
+ case 24:
+ fpu_std(8, (freg_t *)val);
+ break;
+ case 25:
+ fpu_std(10, (freg_t *)val);
+ break;
+ case 26:
+ fpu_std(12, (freg_t *)val);
+ break;
+ case 27:
+ fpu_std(14, (freg_t *)val);
+ break;
+ case 28:
+ fpu_std(9, (freg_t *)val);
+ break;
+ case 29:
+ fpu_std(11, (freg_t *)val);
+ break;
+ case 30:
+ fpu_std(13, (freg_t *)val);
+ break;
+ case 31:
+ fpu_std(15, (freg_t *)val);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static inline int arch_unwind_user_get_reg(unsigned long *val, int regnum)
+{
+ if (0 <= regnum && regnum <= 15) {
+ struct pt_regs *regs = task_pt_regs(current);
+ *val = regs->gprs[regnum];
+ return 0;
+ } else if (16 <= regnum && regnum <= 31) {
+ return __s390_get_dwarf_fpr(val, regnum);
+ }
+
+ return -EINVAL;
+}
+#define unwind_user_get_reg arch_unwind_user_get_reg
+
+#endif /* CONFIG_UNWIND_USER */
+
+#include <asm-generic/unwind_user.h>
+
+#endif /* _ASM_S390_UNWIND_USER_H */
diff --git a/arch/s390/include/asm/unwind_user_sframe.h b/arch/s390/include/asm/unwind_user_sframe.h
new file mode 100644
index 000000000000..e3b80c48f326
--- /dev/null
+++ b/arch/s390/include/asm/unwind_user_sframe.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_UNWIND_USER_SFRAME_H
+#define _ASM_S390_UNWIND_USER_SFRAME_H
+
+#include <linux/unwind_user.h>
+#include <linux/types.h>
+
+#define SFRAME_SP_OFFSET SFRAME_S390X_SP_VAL_OFFSET
+
+static inline s32 arch_sframe_cfa_offset_decode(s32 offset)
+{
+ return SFRAME_V2_S390X_CFA_OFFSET_DECODE(offset);
+}
+#define sframe_cfa_offset_decode arch_sframe_cfa_offset_decode
+
+static inline void
+arch_sframe_init_reginfo(struct unwind_user_reginfo *reginfo, s32 offset)
+{
+ if (SFRAME_V2_S390X_OFFSET_IS_REGNUM(offset)) {
+ reginfo->loc = UNWIND_USER_LOC_REG;
+ reginfo->regnum = SFRAME_V2_S390X_OFFSET_DECODE_REGNUM(offset);
+ } else if (offset) {
+ reginfo->loc = UNWIND_USER_LOC_STACK;
+ reginfo->offset = offset;
+ } else {
+ reginfo->loc = UNWIND_USER_LOC_NONE;
+ }
+}
+#define sframe_init_reginfo arch_sframe_init_reginfo
+
+#include <asm-generic/unwind_user_sframe.h>
+
+#endif /* _ASM_S390_UNWIND_USER_SFRAME_H */
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index 92f770fc21f6..bd446d55b552 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -12,11 +12,11 @@
#include <linux/mm.h>
#include <linux/string_helpers.h>
#include <linux/sframe.h>
-#include <asm/unwind_user_sframe.h>
#include <linux/unwind_user_types.h>
#include "sframe.h"
#include "sframe_debug.h"
+#include <asm/unwind_user_sframe.h>
struct sframe_fde_internal {
unsigned long func_start_addr;
diff --git a/kernel/unwind/sframe.h b/kernel/unwind/sframe.h
index 69ce0d5b9694..c09f25fbaa2f 100644
--- a/kernel/unwind/sframe.h
+++ b/kernel/unwind/sframe.h
@@ -18,6 +18,7 @@
#define SFRAME_ABI_AARCH64_ENDIAN_BIG 1
#define SFRAME_ABI_AARCH64_ENDIAN_LITTLE 2
#define SFRAME_ABI_AMD64_ENDIAN_LITTLE 3
+#define SFRAME_ABI_S390X_ENDIAN_BIG 4 /* s390 64-bit (s390x) */
#define SFRAME_FDE_TYPE_PCINC 0
#define SFRAME_FDE_TYPE_PCMASK 1
@@ -69,4 +70,17 @@ struct sframe_fde {
#define SFRAME_FRE_OFFSET_SIZE(data) ((data >> 5) & 0x3)
#define SFRAME_FRE_MANGLED_RA_P(data) ((data >> 7) & 0x1)
+/* s390 64-bit (s390x) */
+
+#define SFRAME_S390X_SP_VAL_OFFSET (-160)
+
+#define SFRAME_S390X_CFA_OFFSET_ADJUSTMENT SFRAME_S390X_SP_VAL_OFFSET
+#define SFRAME_S390X_CFA_OFFSET_ALIGNMENT_FACTOR 8
+#define SFRAME_V2_S390X_CFA_OFFSET_DECODE(offset) \
+ (((offset) * SFRAME_S390X_CFA_OFFSET_ALIGNMENT_FACTOR) \
+ - SFRAME_S390X_CFA_OFFSET_ADJUSTMENT)
+
+#define SFRAME_V2_S390X_OFFSET_IS_REGNUM(offset) ((offset) & 1)
+#define SFRAME_V2_S390X_OFFSET_DECODE_REGNUM(offset) ((offset) >> 1)
+
#endif /* _SFRAME_H */
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 01/15] unwind_user: Enhance comments on get CFA, FP, and RA
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
Move the comment "Get the Canonical Frame Address (CFA)" to the top
of the sequence of statements that actually get the CFA. Reword the
comment "Find the Return Address (RA)" to "Get ...", as the statements
actually get the RA. Add a respective comment to the statements that
get the FP. This will be useful once future commits extend the logic
to get the RA and FP.
While at it align the comment on the "stack going in wrong direction"
check to the following one on the "address is word aligned" check.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
kernel/unwind/user.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/kernel/unwind/user.c b/kernel/unwind/user.c
index d053295b1f7e..f81c36ab2861 100644
--- a/kernel/unwind/user.c
+++ b/kernel/unwind/user.c
@@ -38,6 +38,7 @@ static int unwind_user_next_common(struct unwind_user_state *state,
return 0;
}
+ /* Get the Canonical Frame Address (CFA) */
if (frame->use_fp) {
if (state->fp < state->sp)
return -EINVAL;
@@ -45,11 +46,9 @@ static int unwind_user_next_common(struct unwind_user_state *state,
} else {
cfa = state->sp;
}
-
- /* Get the Canonical Frame Address (CFA) */
cfa += frame->cfa_off;
- /* stack going in wrong direction? */
+ /* Make sure that stack is not going in wrong direction */
if (cfa <= state->sp)
return -EINVAL;
@@ -57,10 +56,11 @@ static int unwind_user_next_common(struct unwind_user_state *state,
if (cfa & (state->ws - 1))
return -EINVAL;
- /* Find the Return Address (RA) */
+ /* Get the Return Address (RA) */
if (get_user_word(&ra, cfa, frame->ra_off, state->ws))
return -EINVAL;
+ /* Get the Frame Pointer (FP) */
if (frame->fp_off && get_user_word(&fp, cfa, frame->fp_off, state->ws))
return -EINVAL;
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 15/15] s390/unwind_user/backchain: Enable HAVE_UNWIND_USER_BACKCHAIN
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
Enable unwinding of user space using back chain on s390. Based on
arch_stack_walk_user_common() in arch/s390/kernel/stacktrace.c.
Note that an invalid RA obtained from the stack frame pointed to by the
back chain is not a valid indication that the IP is still early in the
function prologue and a fallback to the RA and SP register r14 and r15
contents should be made.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in RFC v2:
- Adjusted to latest unwind user changes.
- Use struct stack_frame_user and struct stack_frame_vdso_wrapper from
asm/stacktrace.h.
- In topmost frame do not fallback to RA (and SP) register values if
RA is invalid. This is not a valid indication for early prologue.
- In topmost frame use RA and SP register values if they match those
saved in the frame. This indicates early prologue.
arch/s390/Kconfig | 1 +
arch/s390/kernel/Makefile | 2 +
arch/s390/kernel/unwind_user_backchain.c | 112 +++++++++++++++++++++++
3 files changed, 115 insertions(+)
create mode 100644 arch/s390/kernel/unwind_user_backchain.c
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 52d3f3b3e086..5aeb2abd390f 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -246,6 +246,7 @@ config S390
select HAVE_SETUP_PER_CPU_AREA
select HAVE_SOFTIRQ_ON_OWN_STACK
select HAVE_SYSCALL_TRACEPOINTS
+ select HAVE_UNWIND_USER_BACKCHAIN
select HAVE_UNWIND_USER_SFRAME
select HAVE_VIRT_CPU_ACCOUNTING
select HAVE_VIRT_CPU_ACCOUNTING_IDLE
diff --git a/arch/s390/kernel/Makefile b/arch/s390/kernel/Makefile
index eb06ff888314..eb662e95c5fd 100644
--- a/arch/s390/kernel/Makefile
+++ b/arch/s390/kernel/Makefile
@@ -83,6 +83,8 @@ obj-$(CONFIG_PERF_EVENTS) += perf_pai_crypto.o perf_pai_ext.o
obj-$(CONFIG_TRACEPOINTS) += trace.o
+obj-$(CONFIG_HAVE_UNWIND_USER_BACKCHAIN) += unwind_user_backchain.o
+
# vdso
obj-y += vdso64/
obj-$(CONFIG_COMPAT) += vdso32/
diff --git a/arch/s390/kernel/unwind_user_backchain.c b/arch/s390/kernel/unwind_user_backchain.c
new file mode 100644
index 000000000000..4e10ca43ea36
--- /dev/null
+++ b/arch/s390/kernel/unwind_user_backchain.c
@@ -0,0 +1,112 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#define pr_fmt(fmt) "backchain: " fmt
+
+#include <asm/asm-offsets.h>
+#include <asm/stacktrace.h>
+#include <linux/security.h>
+#include <linux/unwind_user.h>
+#include <linux/unwind_user_backchain.h>
+
+/**
+ * ip_invalid - Perform some basic checks whether an instruction pointer (IP)
+ * taken from an unreliable source is invalid
+ * @ip: The instruction pointer to be validated
+ *
+ * returns whether the instruction pointer is invalid
+ */
+static inline bool ip_invalid(unsigned long ip)
+{
+ /* Architecture requires IP to be 2-byte aligned. */
+ if (ip & 1)
+ return true;
+ if (ip < mmap_min_addr)
+ return true;
+ if (ip >= current->mm->context.asce_limit)
+ return true;
+ return false;
+}
+
+/**
+ * ip_within_vdso - Check whether an instruction pointer (IP) is within vDSO
+ * @ip: The instruction pointer
+ *
+ * returns whether the instruction pointer is within vDSO
+ */
+static inline bool ip_within_vdso(unsigned long ip)
+{
+ return in_range(ip, current->mm->context.vdso_base, vdso_text_size());
+}
+
+/**
+ * arch_unwind_user_next_backchain - Unwind one frame using s390 back chain
+ * @state: The unwind user state
+ *
+ * returns zero when successful, otherwise -EINVAL.
+ */
+int arch_unwind_user_next_backchain(struct unwind_user_state *state)
+{
+ struct stack_frame_user __user *sf;
+ unsigned long sp, ra;
+
+ sf = (void __user *)state->sp;
+
+ /*
+ * In topmost frame check whether IP in early prologue, RA and SP
+ * registers saved, and no new stack frame allocated.
+ */
+ if (state->topmost) {
+ unsigned long ra_reg;
+
+ if (__get_user(ra, (unsigned long __user *)&sf->gprs[8]))
+ return -EINVAL;
+ if (__get_user(sp, (unsigned long __user *)&sf->gprs[9]))
+ return -EINVAL;
+ if (unwind_user_get_ra_reg(&ra_reg))
+ return -EINVAL;
+ if (ra == ra_reg && sp == state->sp)
+ goto done;
+ }
+
+ if (__get_user(sp, (unsigned long __user *)&sf->back_chain))
+ return -EINVAL;
+ if (!sp && ip_within_vdso(state->ip)) {
+ /*
+ * Assume non-standard vDSO user wrapper stack frame.
+ * See vDSO user wrapper code for details.
+ */
+ struct stack_frame_vdso_wrapper *sf_vdso = (void __user *)sf;
+
+ if (__get_user(ra, (unsigned long __user *)&sf_vdso->return_address))
+ return -EINVAL;
+ sf = (void __user *)((unsigned long)sf + STACK_FRAME_VDSO_OVERHEAD);
+ if (__get_user(sp, (unsigned long __user *)&sf->back_chain))
+ return -EINVAL;
+ } else if (!sp) {
+ /* Assume outermost frame reached. */
+ state->done = true;
+ return 0;
+ } else {
+ /*
+ * Assume IP past prologue and new stack frame allocated.
+ * Follow back chain, which then equals the SP at entry.
+ * Skips caller if wrong in topmost frame.
+ */
+ sf = (void __user *)sp;
+ if (__get_user(ra, (unsigned long __user *)&sf->gprs[8]))
+ return -EINVAL;
+ /* Skip validation: ABI requires SP to be saved as well. */
+ }
+
+done:
+ /* Validate SP and RA (ABI requires SP to be 8-byte aligned). */
+ if (sp & 7 || ip_invalid(ra))
+ return -EINVAL;
+
+ state->ip = ra;
+ state->sp = sp;
+ state->fp = 0; /* Cannot unwind FP. */
+ state->topmost = false;
+
+ return 0;
+}
--
2.51.0
^ permalink raw reply related
* [RFC PATCH v2 14/15] unwind_user/backchain: Introduce back chain user space unwinding
From: Jens Remus @ 2025-12-05 17:14 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, linux-s390, bpf, x86,
Steven Rostedt
Cc: Jens Remus, Heiko Carstens, Vasily Gorbik, Ilya Leoshkevich,
Josh Poimboeuf, Masami Hiramatsu, Mathieu Desnoyers,
Peter Zijlstra, Ingo Molnar, Jiri Olsa, Arnaldo Carvalho de Melo,
Namhyung Kim, Thomas Gleixner, Andrii Nakryiko, Indu Bhagat,
Jose E. Marchesi, Beau Belgrave, Linus Torvalds, Andrew Morton,
Florian Weimer, Kees Cook, Carlos O'Donell, Sam James,
Dylan Hatch
In-Reply-To: <20251205171446.2814872-1-jremus@linux.ibm.com>
Add support for unwinding of user space using back chain to the
unwind user interface. Use it as secondary fallback for unwinding
using SFrame, if that fails and the primary fallback using frame
pointer is not available.
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
Notes (jremus):
Changes in RFC v2:
- Adjusted to latest unwind user enhancements. Does hopefully no longer
appear grafted on.
arch/Kconfig | 6 ++++++
include/linux/unwind_user_backchain.h | 20 ++++++++++++++++++++
include/linux/unwind_user_types.h | 2 ++
kernel/unwind/user.c | 12 ++++++++++++
4 files changed, 40 insertions(+)
create mode 100644 include/linux/unwind_user_backchain.h
diff --git a/arch/Kconfig b/arch/Kconfig
index 7fa89d70b244..37fb78a5e876 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -488,6 +488,12 @@ config AS_SFRAME
config UNWIND_USER
bool
+config HAVE_UNWIND_USER_BACKCHAIN
+ bool
+ select UNWIND_USER
+ help
+ The arch supports unwinding of user space using back chain.
+
config HAVE_UNWIND_USER_FP
bool
select UNWIND_USER
diff --git a/include/linux/unwind_user_backchain.h b/include/linux/unwind_user_backchain.h
new file mode 100644
index 000000000000..e7a8e584b13f
--- /dev/null
+++ b/include/linux/unwind_user_backchain.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_UNWIND_USER_BACKCHAIN_H
+#define _LINUX_UNWIND_USER_BACKCHAIN_H
+
+struct unwind_user_state;
+
+#ifdef CONFIG_HAVE_UNWIND_USER_BACKCHAIN
+
+extern int arch_unwind_user_next_backchain(struct unwind_user_state *state);
+
+#else /* !CONFIG_HAVE_UNWIND_USER_BACKCHAIN */
+
+static inline int arch_unwind_user_next_backchain(struct unwind_user_state *state)
+{
+ return -EINVAL;
+}
+
+#endif /* !CONFIG_HAVE_UNWIND_USER_BACKCHAIN */
+
+#endif /* _LINUX_UNWIND_USER_BACKCHAIN_H */
diff --git a/include/linux/unwind_user_types.h b/include/linux/unwind_user_types.h
index 6efc12b6e831..b44502e90b7f 100644
--- a/include/linux/unwind_user_types.h
+++ b/include/linux/unwind_user_types.h
@@ -11,6 +11,7 @@
enum unwind_user_type_bits {
UNWIND_USER_TYPE_SFRAME_BIT = 0,
UNWIND_USER_TYPE_FP_BIT = 1,
+ UNWIND_USER_TYPE_BACKCHAIN_BIT = 2,
NR_UNWIND_USER_TYPE_BITS,
};
@@ -20,6 +21,7 @@ enum unwind_user_type {
UNWIND_USER_TYPE_NONE = 0,
UNWIND_USER_TYPE_SFRAME = BIT(UNWIND_USER_TYPE_SFRAME_BIT),
UNWIND_USER_TYPE_FP = BIT(UNWIND_USER_TYPE_FP_BIT),
+ UNWIND_USER_TYPE_BACKCHAIN = BIT(UNWIND_USER_TYPE_BACKCHAIN_BIT),
};
struct unwind_stacktrace {
diff --git a/kernel/unwind/user.c b/kernel/unwind/user.c
index 122045cb411f..5b4649bc91ba 100644
--- a/kernel/unwind/user.c
+++ b/kernel/unwind/user.c
@@ -6,6 +6,7 @@
#include <linux/sched.h>
#include <linux/sched/task_stack.h>
#include <linux/unwind_user.h>
+#include <linux/unwind_user_backchain.h>
#include <linux/uaccess.h>
#include <linux/sframe.h>
@@ -105,6 +106,11 @@ static int unwind_user_next_common(struct unwind_user_state *state,
return 0;
}
+static int unwind_user_next_backchain(struct unwind_user_state *state)
+{
+ return arch_unwind_user_next_backchain(state);
+}
+
static int unwind_user_next_fp(struct unwind_user_state *state)
{
struct pt_regs *regs = task_pt_regs(current);
@@ -159,6 +165,10 @@ static int unwind_user_next(struct unwind_user_state *state)
if (!unwind_user_next_fp(state))
return 0;
continue;
+ case UNWIND_USER_TYPE_BACKCHAIN:
+ if (!unwind_user_next_backchain(state))
+ return 0;
+ continue; /* Try next method. */
default:
WARN_ONCE(1, "Undefined unwind bit %d", bit);
break;
@@ -187,6 +197,8 @@ static int unwind_user_start(struct unwind_user_state *state)
state->available_types |= UNWIND_USER_TYPE_SFRAME;
if (IS_ENABLED(CONFIG_HAVE_UNWIND_USER_FP))
state->available_types |= UNWIND_USER_TYPE_FP;
+ if (IS_ENABLED(CONFIG_HAVE_UNWIND_USER_BACKCHAIN))
+ state->available_types |= UNWIND_USER_TYPE_BACKCHAIN;
state->ip = instruction_pointer(regs);
state->sp = user_stack_pointer(regs);
--
2.51.0
^ permalink raw reply related
* [PATCH v1 4/4] tools/rtla: Remove unneeded nr_cpus from for_each_monitored_cpu
From: Costa Shulyupin @ 2025-12-05 15:19 UTC (permalink / raw)
To: Steven Rostedt, Tomas Glozar, Crystal Wood, Wander Lairson Costa,
Costa Shulyupin, Ivan Pravdin, John Kacur, Tiezhu Yang,
linux-trace-kernel, linux-kernel, bpf
In-Reply-To: <20251205151924.2250142-1-costa.shul@redhat.com>
nr_cpus does not change at runtime, so passing it through the macro
argument is unnecessary.
Remove the argument and use the global nr_cpus instead.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
---
tools/tracing/rtla/src/common.h | 2 +-
tools/tracing/rtla/src/osnoise_hist.c | 15 +++++++--------
tools/tracing/rtla/src/osnoise_top.c | 2 +-
tools/tracing/rtla/src/timerlat.c | 4 ++--
tools/tracing/rtla/src/timerlat_hist.c | 16 ++++++++--------
tools/tracing/rtla/src/timerlat_top.c | 2 +-
6 files changed, 20 insertions(+), 21 deletions(-)
diff --git a/tools/tracing/rtla/src/common.h b/tools/tracing/rtla/src/common.h
index 2765e2a9f85f..a597e7591e20 100644
--- a/tools/tracing/rtla/src/common.h
+++ b/tools/tracing/rtla/src/common.h
@@ -109,7 +109,7 @@ struct common_params {
extern int nr_cpus;
-#define for_each_monitored_cpu(cpu, nr_cpus, common) \
+#define for_each_monitored_cpu(cpu, common) \
for (cpu = 0; cpu < nr_cpus; cpu++) \
if (!(common)->cpus || CPU_ISSET(cpu, &(common)->monitored_cpus))
diff --git a/tools/tracing/rtla/src/osnoise_hist.c b/tools/tracing/rtla/src/osnoise_hist.c
index 7514b29ec559..56bca13b991f 100644
--- a/tools/tracing/rtla/src/osnoise_hist.c
+++ b/tools/tracing/rtla/src/osnoise_hist.c
@@ -245,7 +245,7 @@ static void osnoise_hist_header(struct osnoise_tool *tool)
if (!params->common.hist.no_index)
trace_seq_printf(s, "Index");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -274,8 +274,7 @@ osnoise_print_summary(struct osnoise_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "count:");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
-
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -286,7 +285,7 @@ osnoise_print_summary(struct osnoise_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "min: ");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -299,7 +298,7 @@ osnoise_print_summary(struct osnoise_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "avg: ");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -315,7 +314,7 @@ osnoise_print_summary(struct osnoise_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "max: ");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -350,7 +349,7 @@ osnoise_print_stats(struct osnoise_tool *tool)
trace_seq_printf(trace->seq, "%-6d",
bucket * data->bucket_size);
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -386,7 +385,7 @@ osnoise_print_stats(struct osnoise_tool *tool)
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "over: ");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
diff --git a/tools/tracing/rtla/src/osnoise_top.c b/tools/tracing/rtla/src/osnoise_top.c
index e5a13f1bdfb6..0f5b86f11486 100644
--- a/tools/tracing/rtla/src/osnoise_top.c
+++ b/tools/tracing/rtla/src/osnoise_top.c
@@ -236,7 +236,7 @@ osnoise_print_stats(struct osnoise_tool *top)
osnoise_top_header(top);
- for_each_monitored_cpu(i, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(i, ¶ms->common) {
osnoise_top_print(top, i);
}
diff --git a/tools/tracing/rtla/src/timerlat.c b/tools/tracing/rtla/src/timerlat.c
index 7503e18b905c..181d72de3197 100644
--- a/tools/tracing/rtla/src/timerlat.c
+++ b/tools/tracing/rtla/src/timerlat.c
@@ -124,7 +124,7 @@ int timerlat_enable(struct osnoise_tool *tool)
return -1;
}
- for_each_monitored_cpu(i, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(i, ¶ms->common) {
if (save_cpu_idle_disable_state(i) < 0) {
err_msg("Could not save cpu idle state.\n");
return -1;
@@ -217,7 +217,7 @@ void timerlat_free(struct osnoise_tool *tool)
if (dma_latency_fd >= 0)
close(dma_latency_fd);
if (params->deepest_idle_state >= -1) {
- for_each_monitored_cpu(i, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(i, ¶ms->common) {
restore_cpu_idle_disable_state(i);
}
}
diff --git a/tools/tracing/rtla/src/timerlat_hist.c b/tools/tracing/rtla/src/timerlat_hist.c
index 207359f1cec4..e93476e18ad8 100644
--- a/tools/tracing/rtla/src/timerlat_hist.c
+++ b/tools/tracing/rtla/src/timerlat_hist.c
@@ -298,7 +298,7 @@ static void timerlat_hist_header(struct osnoise_tool *tool)
if (!params->common.hist.no_index)
trace_seq_printf(s, "Index");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -350,7 +350,7 @@ timerlat_print_summary(struct timerlat_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "count:");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -372,7 +372,7 @@ timerlat_print_summary(struct timerlat_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "min: ");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -400,7 +400,7 @@ timerlat_print_summary(struct timerlat_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "avg: ");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -428,7 +428,7 @@ timerlat_print_summary(struct timerlat_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "max: ");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -473,7 +473,7 @@ timerlat_print_stats_all(struct timerlat_params *params,
sum.min_thread = ~0;
sum.min_user = ~0;
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -620,7 +620,7 @@ timerlat_print_stats(struct osnoise_tool *tool)
trace_seq_printf(trace->seq, "%-6d",
bucket * data->bucket_size);
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -658,7 +658,7 @@ timerlat_print_stats(struct osnoise_tool *tool)
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "over: ");
- for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
diff --git a/tools/tracing/rtla/src/timerlat_top.c b/tools/tracing/rtla/src/timerlat_top.c
index 34c8a0010828..c48f02075197 100644
--- a/tools/tracing/rtla/src/timerlat_top.c
+++ b/tools/tracing/rtla/src/timerlat_top.c
@@ -447,7 +447,7 @@ timerlat_print_stats(struct osnoise_tool *top)
timerlat_top_header(params, top);
- for_each_monitored_cpu(i, nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(i, ¶ms->common) {
timerlat_top_print(top, i);
timerlat_top_update_sum(top, i, &summary);
}
--
2.52.0
^ permalink raw reply related
* [PATCH v1 3/4] tools/rtla: Remove unneeded nr_cpus members
From: Costa Shulyupin @ 2025-12-05 15:19 UTC (permalink / raw)
To: Steven Rostedt, Tomas Glozar, Crystal Wood, Wander Lairson Costa,
Costa Shulyupin, Ivan Pravdin, John Kacur, Tiezhu Yang,
linux-trace-kernel, linux-kernel, bpf
In-Reply-To: <20251205151924.2250142-1-costa.shul@redhat.com>
nr_cpus does not change at runtime, so keeping it in struct members is
unnecessary.
Use the global nr_cpus instead of struct members.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
---
tools/tracing/rtla/src/osnoise_hist.c | 18 ++++++------
tools/tracing/rtla/src/osnoise_top.c | 3 --
tools/tracing/rtla/src/timerlat_aa.c | 10 +++----
tools/tracing/rtla/src/timerlat_hist.c | 40 ++++++++++++--------------
tools/tracing/rtla/src/timerlat_top.c | 19 ++++++------
5 files changed, 39 insertions(+), 51 deletions(-)
diff --git a/tools/tracing/rtla/src/osnoise_hist.c b/tools/tracing/rtla/src/osnoise_hist.c
index ae773334e700..7514b29ec559 100644
--- a/tools/tracing/rtla/src/osnoise_hist.c
+++ b/tools/tracing/rtla/src/osnoise_hist.c
@@ -30,7 +30,6 @@ struct osnoise_hist_data {
struct osnoise_hist_cpu *hist;
int entries;
int bucket_size;
- int nr_cpus;
};
/*
@@ -42,7 +41,7 @@ osnoise_free_histogram(struct osnoise_hist_data *data)
int cpu;
/* one histogram for IRQ and one for thread, per CPU */
- for (cpu = 0; cpu < data->nr_cpus; cpu++) {
+ for (cpu = 0; cpu < nr_cpus; cpu++) {
if (data->hist[cpu].samples)
free(data->hist[cpu].samples);
}
@@ -74,7 +73,6 @@ static struct osnoise_hist_data
data->entries = entries;
data->bucket_size = bucket_size;
- data->nr_cpus = nr_cpus;
data->hist = calloc(1, sizeof(*data->hist) * nr_cpus);
if (!data->hist)
@@ -247,7 +245,7 @@ static void osnoise_hist_header(struct osnoise_tool *tool)
if (!params->common.hist.no_index)
trace_seq_printf(s, "Index");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -276,7 +274,7 @@ osnoise_print_summary(struct osnoise_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "count:");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -288,7 +286,7 @@ osnoise_print_summary(struct osnoise_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "min: ");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -301,7 +299,7 @@ osnoise_print_summary(struct osnoise_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "avg: ");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -317,7 +315,7 @@ osnoise_print_summary(struct osnoise_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "max: ");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -352,7 +350,7 @@ osnoise_print_stats(struct osnoise_tool *tool)
trace_seq_printf(trace->seq, "%-6d",
bucket * data->bucket_size);
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
@@ -388,7 +386,7 @@ osnoise_print_stats(struct osnoise_tool *tool)
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "over: ");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].count)
continue;
diff --git a/tools/tracing/rtla/src/osnoise_top.c b/tools/tracing/rtla/src/osnoise_top.c
index 367a765387c8..e5a13f1bdfb6 100644
--- a/tools/tracing/rtla/src/osnoise_top.c
+++ b/tools/tracing/rtla/src/osnoise_top.c
@@ -31,7 +31,6 @@ struct osnoise_top_cpu {
struct osnoise_top_data {
struct osnoise_top_cpu *cpu_data;
- int nr_cpus;
};
/*
@@ -59,8 +58,6 @@ static struct osnoise_top_data *osnoise_alloc_top(void)
if (!data)
return NULL;
- data->nr_cpus = nr_cpus;
-
/* one set of histograms per CPU */
data->cpu_data = calloc(1, sizeof(*data->cpu_data) * nr_cpus);
if (!data->cpu_data)
diff --git a/tools/tracing/rtla/src/timerlat_aa.c b/tools/tracing/rtla/src/timerlat_aa.c
index 5766d58709eb..59b219a1503e 100644
--- a/tools/tracing/rtla/src/timerlat_aa.c
+++ b/tools/tracing/rtla/src/timerlat_aa.c
@@ -102,7 +102,6 @@ struct timerlat_aa_data {
* The analysis context and system wide view
*/
struct timerlat_aa_context {
- int nr_cpus;
int dump_tasks;
/* per CPU data */
@@ -738,7 +737,7 @@ void timerlat_auto_analysis(int irq_thresh, int thread_thresh)
irq_thresh = irq_thresh * 1000;
thread_thresh = thread_thresh * 1000;
- for (cpu = 0; cpu < taa_ctx->nr_cpus; cpu++) {
+ for (cpu = 0; cpu < nr_cpus; cpu++) {
taa_data = timerlat_aa_get_data(taa_ctx, cpu);
if (irq_thresh && taa_data->tlat_irq_latency >= irq_thresh) {
@@ -766,7 +765,7 @@ void timerlat_auto_analysis(int irq_thresh, int thread_thresh)
printf("\n");
printf("Printing CPU tasks:\n");
- for (cpu = 0; cpu < taa_ctx->nr_cpus; cpu++) {
+ for (cpu = 0; cpu < nr_cpus; cpu++) {
taa_data = timerlat_aa_get_data(taa_ctx, cpu);
tep = taa_ctx->tool->trace.tep;
@@ -792,7 +791,7 @@ static void timerlat_aa_destroy_seqs(struct timerlat_aa_context *taa_ctx)
if (!taa_ctx->taa_data)
return;
- for (i = 0; i < taa_ctx->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
taa_data = timerlat_aa_get_data(taa_ctx, i);
if (taa_data->prev_irqs_seq) {
@@ -842,7 +841,7 @@ static int timerlat_aa_init_seqs(struct timerlat_aa_context *taa_ctx)
struct timerlat_aa_data *taa_data;
int i;
- for (i = 0; i < taa_ctx->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
taa_data = timerlat_aa_get_data(taa_ctx, i);
@@ -1031,7 +1030,6 @@ int timerlat_aa_init(struct osnoise_tool *tool, int dump_tasks)
__timerlat_aa_ctx = taa_ctx;
- taa_ctx->nr_cpus = nr_cpus;
taa_ctx->tool = tool;
taa_ctx->dump_tasks = dump_tasks;
diff --git a/tools/tracing/rtla/src/timerlat_hist.c b/tools/tracing/rtla/src/timerlat_hist.c
index d4a9dcd67d48..207359f1cec4 100644
--- a/tools/tracing/rtla/src/timerlat_hist.c
+++ b/tools/tracing/rtla/src/timerlat_hist.c
@@ -44,7 +44,6 @@ struct timerlat_hist_data {
struct timerlat_hist_cpu *hist;
int entries;
int bucket_size;
- int nr_cpus;
};
/*
@@ -56,7 +55,7 @@ timerlat_free_histogram(struct timerlat_hist_data *data)
int cpu;
/* one histogram for IRQ and one for thread, per CPU */
- for (cpu = 0; cpu < data->nr_cpus; cpu++) {
+ for (cpu = 0; cpu < nr_cpus; cpu++) {
if (data->hist[cpu].irq)
free(data->hist[cpu].irq);
@@ -94,7 +93,6 @@ static struct timerlat_hist_data
data->entries = entries;
data->bucket_size = bucket_size;
- data->nr_cpus = nr_cpus;
/* one set of histograms per CPU */
data->hist = calloc(1, sizeof(*data->hist) * nr_cpus);
@@ -204,17 +202,17 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
{
struct timerlat_hist_data *data = tool->data;
int i, j, err;
- long long value_irq[data->nr_cpus],
- value_thread[data->nr_cpus],
- value_user[data->nr_cpus];
+ long long value_irq[nr_cpus],
+ value_thread[nr_cpus],
+ value_user[nr_cpus];
/* Pull histogram */
for (i = 0; i < data->entries; i++) {
err = timerlat_bpf_get_hist_value(i, value_irq, value_thread,
- value_user, data->nr_cpus);
+ value_user, nr_cpus);
if (err)
return err;
- for (j = 0; j < data->nr_cpus; j++) {
+ for (j = 0; j < nr_cpus; j++) {
data->hist[j].irq[i] = value_irq[j];
data->hist[j].thread[i] = value_thread[j];
data->hist[j].user[i] = value_user[j];
@@ -226,7 +224,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->hist[i].irq_count = value_irq[i];
data->hist[i].thread_count = value_thread[i];
data->hist[i].user_count = value_user[i];
@@ -236,7 +234,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->hist[i].min_irq = value_irq[i];
data->hist[i].min_thread = value_thread[i];
data->hist[i].min_user = value_user[i];
@@ -246,7 +244,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->hist[i].max_irq = value_irq[i];
data->hist[i].max_thread = value_thread[i];
data->hist[i].max_user = value_user[i];
@@ -256,7 +254,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->hist[i].sum_irq = value_irq[i];
data->hist[i].sum_thread = value_thread[i];
data->hist[i].sum_user = value_user[i];
@@ -266,7 +264,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->hist[i].irq[data->entries] = value_irq[i];
data->hist[i].thread[data->entries] = value_thread[i];
data->hist[i].user[data->entries] = value_user[i];
@@ -300,7 +298,7 @@ static void timerlat_hist_header(struct osnoise_tool *tool)
if (!params->common.hist.no_index)
trace_seq_printf(s, "Index");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -352,7 +350,7 @@ timerlat_print_summary(struct timerlat_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "count:");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -374,7 +372,7 @@ timerlat_print_summary(struct timerlat_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "min: ");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -402,7 +400,7 @@ timerlat_print_summary(struct timerlat_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "avg: ");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -430,7 +428,7 @@ timerlat_print_summary(struct timerlat_params *params,
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "max: ");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -475,7 +473,7 @@ timerlat_print_stats_all(struct timerlat_params *params,
sum.min_thread = ~0;
sum.min_user = ~0;
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -622,7 +620,7 @@ timerlat_print_stats(struct osnoise_tool *tool)
trace_seq_printf(trace->seq, "%-6d",
bucket * data->bucket_size);
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
@@ -660,7 +658,7 @@ timerlat_print_stats(struct osnoise_tool *tool)
if (!params->common.hist.no_index)
trace_seq_printf(trace->seq, "over: ");
- for_each_monitored_cpu(cpu, data->nr_cpus, ¶ms->common) {
+ for_each_monitored_cpu(cpu, nr_cpus, ¶ms->common) {
if (!data->hist[cpu].irq_count && !data->hist[cpu].thread_count)
continue;
diff --git a/tools/tracing/rtla/src/timerlat_top.c b/tools/tracing/rtla/src/timerlat_top.c
index 7b62549f69e3..34c8a0010828 100644
--- a/tools/tracing/rtla/src/timerlat_top.c
+++ b/tools/tracing/rtla/src/timerlat_top.c
@@ -42,7 +42,6 @@ struct timerlat_top_cpu {
struct timerlat_top_data {
struct timerlat_top_cpu *cpu_data;
- int nr_cpus;
};
/*
@@ -72,8 +71,6 @@ static struct timerlat_top_data *timerlat_alloc_top(void)
if (!data)
return NULL;
- data->nr_cpus = nr_cpus;
-
/* one set of histograms per CPU */
data->cpu_data = calloc(1, sizeof(*data->cpu_data) * nr_cpus);
if (!data->cpu_data)
@@ -191,16 +188,16 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
{
struct timerlat_top_data *data = tool->data;
int i, err;
- long long value_irq[data->nr_cpus],
- value_thread[data->nr_cpus],
- value_user[data->nr_cpus];
+ long long value_irq[nr_cpus],
+ value_thread[nr_cpus],
+ value_user[nr_cpus];
/* Pull summary */
err = timerlat_bpf_get_summary_value(SUMMARY_CURRENT,
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->cpu_data[i].cur_irq = value_irq[i];
data->cpu_data[i].cur_thread = value_thread[i];
data->cpu_data[i].cur_user = value_user[i];
@@ -210,7 +207,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->cpu_data[i].irq_count = value_irq[i];
data->cpu_data[i].thread_count = value_thread[i];
data->cpu_data[i].user_count = value_user[i];
@@ -220,7 +217,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->cpu_data[i].min_irq = value_irq[i];
data->cpu_data[i].min_thread = value_thread[i];
data->cpu_data[i].min_user = value_user[i];
@@ -230,7 +227,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->cpu_data[i].max_irq = value_irq[i];
data->cpu_data[i].max_thread = value_thread[i];
data->cpu_data[i].max_user = value_user[i];
@@ -240,7 +237,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
value_irq, value_thread, value_user);
if (err)
return err;
- for (i = 0; i < data->nr_cpus; i++) {
+ for (i = 0; i < nr_cpus; i++) {
data->cpu_data[i].sum_irq = value_irq[i];
data->cpu_data[i].sum_thread = value_thread[i];
data->cpu_data[i].sum_user = value_user[i];
--
2.52.0
^ permalink raw reply related
* [PATCH v1 2/4] tools/rtla: Remove unneeded nr_cpus arguments
From: Costa Shulyupin @ 2025-12-05 15:19 UTC (permalink / raw)
To: Steven Rostedt, Tomas Glozar, Crystal Wood, Wander Lairson Costa,
Costa Shulyupin, Ivan Pravdin, John Kacur, Tiezhu Yang,
linux-trace-kernel, linux-kernel, bpf
In-Reply-To: <20251205151924.2250142-1-costa.shul@redhat.com>
nr_cpus does not change at runtime, so passing it through function
arguments is unnecessary.
Use the global nr_cpus instead of propagating it via parameters.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
---
tools/tracing/rtla/src/osnoise_hist.c | 4 ++--
tools/tracing/rtla/src/osnoise_top.c | 4 ++--
tools/tracing/rtla/src/timerlat_bpf.c | 5 ++---
tools/tracing/rtla/src/timerlat_bpf.h | 6 ++----
tools/tracing/rtla/src/timerlat_hist.c | 19 +++++++------------
tools/tracing/rtla/src/timerlat_top.c | 19 +++++++------------
tools/tracing/rtla/src/timerlat_u.c | 6 +++---
7 files changed, 25 insertions(+), 38 deletions(-)
diff --git a/tools/tracing/rtla/src/osnoise_hist.c b/tools/tracing/rtla/src/osnoise_hist.c
index 0bed9717cef6..ae773334e700 100644
--- a/tools/tracing/rtla/src/osnoise_hist.c
+++ b/tools/tracing/rtla/src/osnoise_hist.c
@@ -63,7 +63,7 @@ static void osnoise_free_hist_tool(struct osnoise_tool *tool)
* osnoise_alloc_histogram - alloc runtime data
*/
static struct osnoise_hist_data
-*osnoise_alloc_histogram(int nr_cpus, int entries, int bucket_size)
+*osnoise_alloc_histogram(int entries, int bucket_size)
{
struct osnoise_hist_data *data;
int cpu;
@@ -704,7 +704,7 @@ static struct osnoise_tool
if (!tool)
return NULL;
- tool->data = osnoise_alloc_histogram(nr_cpus, params->hist.entries,
+ tool->data = osnoise_alloc_histogram(params->hist.entries,
params->hist.bucket_size);
if (!tool->data)
goto out_err;
diff --git a/tools/tracing/rtla/src/osnoise_top.c b/tools/tracing/rtla/src/osnoise_top.c
index 8fa0046f0136..367a765387c8 100644
--- a/tools/tracing/rtla/src/osnoise_top.c
+++ b/tools/tracing/rtla/src/osnoise_top.c
@@ -51,7 +51,7 @@ static void osnoise_free_top_tool(struct osnoise_tool *tool)
/*
* osnoise_alloc_histogram - alloc runtime data
*/
-static struct osnoise_top_data *osnoise_alloc_top(int nr_cpus)
+static struct osnoise_top_data *osnoise_alloc_top(void)
{
struct osnoise_top_data *data;
@@ -548,7 +548,7 @@ struct osnoise_tool *osnoise_init_top(struct common_params *params)
if (!tool)
return NULL;
- tool->data = osnoise_alloc_top(nr_cpus);
+ tool->data = osnoise_alloc_top();
if (!tool->data) {
osnoise_destroy_tool(tool);
return NULL;
diff --git a/tools/tracing/rtla/src/timerlat_bpf.c b/tools/tracing/rtla/src/timerlat_bpf.c
index e97d16646bcd..4b623a904802 100644
--- a/tools/tracing/rtla/src/timerlat_bpf.c
+++ b/tools/tracing/rtla/src/timerlat_bpf.c
@@ -169,12 +169,11 @@ int timerlat_bpf_get_hist_value(int key,
int timerlat_bpf_get_summary_value(enum summary_field key,
long long *value_irq,
long long *value_thread,
- long long *value_user,
- int cpus)
+ long long *value_user)
{
return get_value(bpf->maps.summary_irq,
bpf->maps.summary_thread,
bpf->maps.summary_user,
- key, value_irq, value_thread, value_user, cpus);
+ key, value_irq, value_thread, value_user, nr_cpus);
}
#endif /* HAVE_BPF_SKEL */
diff --git a/tools/tracing/rtla/src/timerlat_bpf.h b/tools/tracing/rtla/src/timerlat_bpf.h
index 118487436d30..2b18f3061c32 100644
--- a/tools/tracing/rtla/src/timerlat_bpf.h
+++ b/tools/tracing/rtla/src/timerlat_bpf.h
@@ -27,8 +27,7 @@ int timerlat_bpf_get_hist_value(int key,
int timerlat_bpf_get_summary_value(enum summary_field key,
long long *value_irq,
long long *value_thread,
- long long *value_user,
- int cpus);
+ long long *value_user);
static inline int have_libbpf_support(void) { return 1; }
#else
@@ -52,8 +51,7 @@ static inline int timerlat_bpf_get_hist_value(int key,
static inline int timerlat_bpf_get_summary_value(enum summary_field key,
long long *value_irq,
long long *value_thread,
- long long *value_user,
- int cpus)
+ long long *value_user)
{
return -1;
}
diff --git a/tools/tracing/rtla/src/timerlat_hist.c b/tools/tracing/rtla/src/timerlat_hist.c
index 37bb9b931c8c..d4a9dcd67d48 100644
--- a/tools/tracing/rtla/src/timerlat_hist.c
+++ b/tools/tracing/rtla/src/timerlat_hist.c
@@ -83,7 +83,7 @@ static void timerlat_free_histogram_tool(struct osnoise_tool *tool)
* timerlat_alloc_histogram - alloc runtime data
*/
static struct timerlat_hist_data
-*timerlat_alloc_histogram(int nr_cpus, int entries, int bucket_size)
+*timerlat_alloc_histogram(int entries, int bucket_size)
{
struct timerlat_hist_data *data;
int cpu;
@@ -223,8 +223,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
/* Pull summary */
err = timerlat_bpf_get_summary_value(SUMMARY_COUNT,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -234,8 +233,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
}
err = timerlat_bpf_get_summary_value(SUMMARY_MIN,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -245,8 +243,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
}
err = timerlat_bpf_get_summary_value(SUMMARY_MAX,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -256,8 +253,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
}
err = timerlat_bpf_get_summary_value(SUMMARY_SUM,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -267,8 +263,7 @@ static int timerlat_hist_bpf_pull_data(struct osnoise_tool *tool)
}
err = timerlat_bpf_get_summary_value(SUMMARY_OVERFLOW,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -1082,7 +1077,7 @@ static struct osnoise_tool
if (!tool)
return NULL;
- tool->data = timerlat_alloc_histogram(nr_cpus, params->hist.entries,
+ tool->data = timerlat_alloc_histogram(params->hist.entries,
params->hist.bucket_size);
if (!tool->data)
goto out_err;
diff --git a/tools/tracing/rtla/src/timerlat_top.c b/tools/tracing/rtla/src/timerlat_top.c
index 8b15f4439c6c..7b62549f69e3 100644
--- a/tools/tracing/rtla/src/timerlat_top.c
+++ b/tools/tracing/rtla/src/timerlat_top.c
@@ -63,7 +63,7 @@ static void timerlat_free_top_tool(struct osnoise_tool *tool)
/*
* timerlat_alloc_histogram - alloc runtime data
*/
-static struct timerlat_top_data *timerlat_alloc_top(int nr_cpus)
+static struct timerlat_top_data *timerlat_alloc_top(void)
{
struct timerlat_top_data *data;
int cpu;
@@ -197,8 +197,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
/* Pull summary */
err = timerlat_bpf_get_summary_value(SUMMARY_CURRENT,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -208,8 +207,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
}
err = timerlat_bpf_get_summary_value(SUMMARY_COUNT,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -219,8 +217,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
}
err = timerlat_bpf_get_summary_value(SUMMARY_MIN,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -230,8 +227,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
}
err = timerlat_bpf_get_summary_value(SUMMARY_MAX,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -241,8 +237,7 @@ static int timerlat_top_bpf_pull_data(struct osnoise_tool *tool)
}
err = timerlat_bpf_get_summary_value(SUMMARY_SUM,
- value_irq, value_thread, value_user,
- data->nr_cpus);
+ value_irq, value_thread, value_user);
if (err)
return err;
for (i = 0; i < data->nr_cpus; i++) {
@@ -828,7 +823,7 @@ static struct osnoise_tool
if (!top)
return NULL;
- top->data = timerlat_alloc_top(nr_cpus);
+ top->data = timerlat_alloc_top();
if (!top->data)
goto out_err;
diff --git a/tools/tracing/rtla/src/timerlat_u.c b/tools/tracing/rtla/src/timerlat_u.c
index a569fe7f93aa..03b4e68e8b1e 100644
--- a/tools/tracing/rtla/src/timerlat_u.c
+++ b/tools/tracing/rtla/src/timerlat_u.c
@@ -99,7 +99,7 @@ static int timerlat_u_main(int cpu, struct timerlat_u_params *params)
*
* Return the number of processes that received the kill.
*/
-static int timerlat_u_send_kill(pid_t *procs, int nr_cpus)
+static int timerlat_u_send_kill(pid_t *procs)
{
int killed = 0;
int i, retval;
@@ -169,7 +169,7 @@ void *timerlat_u_dispatcher(void *data)
/* parent */
if (pid == -1) {
- timerlat_u_send_kill(procs, nr_cpus);
+ timerlat_u_send_kill(procs);
debug_msg("Failed to create child processes");
pthread_exit(&retval);
}
@@ -196,7 +196,7 @@ void *timerlat_u_dispatcher(void *data)
sleep(1);
}
- timerlat_u_send_kill(procs, nr_cpus);
+ timerlat_u_send_kill(procs);
while (procs_count) {
pid = waitpid(-1, &wstatus, 0);
--
2.52.0
^ permalink raw reply related
* [PATCH v1 1/4] tools/rtla: Consolidate nr_cpus usage across all tools
From: Costa Shulyupin @ 2025-12-05 15:19 UTC (permalink / raw)
To: Steven Rostedt, Tomas Glozar, Crystal Wood, Wander Lairson Costa,
Costa Shulyupin, Ivan Pravdin, John Kacur, Tiezhu Yang,
linux-trace-kernel, linux-kernel, bpf
sysconf(_SC_NPROCESSORS_CONF) (via get_nprocs_conf) reflects
cpu_possible_mask, which is fixed at boot time, so querying it
repeatedly is unnecessary.
Replace multiple calls to sysconf(_SC_NPROCESSORS_CONF) with a single
global nr_cpus variable initialized once at startup.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
---
tools/tracing/rtla/src/common.c | 6 ++++--
tools/tracing/rtla/src/common.h | 2 ++
tools/tracing/rtla/src/osnoise_hist.c | 3 ---
tools/tracing/rtla/src/osnoise_top.c | 7 -------
tools/tracing/rtla/src/timerlat.c | 5 +----
tools/tracing/rtla/src/timerlat_aa.c | 1 -
tools/tracing/rtla/src/timerlat_hist.c | 3 ---
tools/tracing/rtla/src/timerlat_top.c | 7 -------
tools/tracing/rtla/src/timerlat_u.c | 3 +--
tools/tracing/rtla/src/utils.c | 10 +---------
10 files changed, 9 insertions(+), 38 deletions(-)
diff --git a/tools/tracing/rtla/src/common.c b/tools/tracing/rtla/src/common.c
index b197037fc58b..38ee2ad21e65 100644
--- a/tools/tracing/rtla/src/common.c
+++ b/tools/tracing/rtla/src/common.c
@@ -4,11 +4,12 @@
#include <pthread.h>
#include <signal.h>
#include <stdlib.h>
-#include <unistd.h>
+#include <sys/sysinfo.h>
#include "common.h"
struct trace_instance *trace_inst;
int stop_tracing;
+int nr_cpus;
static void stop_trace(int sig)
{
@@ -55,7 +56,7 @@ common_apply_config(struct osnoise_tool *tool, struct common_params *params)
}
if (!params->cpus) {
- for (i = 0; i < sysconf(_SC_NPROCESSORS_CONF); i++)
+ for (i = 0; i < nr_cpus; i++)
CPU_SET(i, ¶ms->monitored_cpus);
}
@@ -103,6 +104,7 @@ int run_tool(struct tool_ops *ops, int argc, char *argv[])
bool stopped;
int retval;
+ nr_cpus = get_nprocs_conf();
params = ops->parse_args(argc, argv);
if (!params)
exit(1);
diff --git a/tools/tracing/rtla/src/common.h b/tools/tracing/rtla/src/common.h
index 9ec2b7632c37..2765e2a9f85f 100644
--- a/tools/tracing/rtla/src/common.h
+++ b/tools/tracing/rtla/src/common.h
@@ -107,6 +107,8 @@ struct common_params {
struct timerlat_u_params user;
};
+extern int nr_cpus;
+
#define for_each_monitored_cpu(cpu, nr_cpus, common) \
for (cpu = 0; cpu < nr_cpus; cpu++) \
if (!(common)->cpus || CPU_ISSET(cpu, &(common)->monitored_cpus))
diff --git a/tools/tracing/rtla/src/osnoise_hist.c b/tools/tracing/rtla/src/osnoise_hist.c
index ff8c231e47c4..0bed9717cef6 100644
--- a/tools/tracing/rtla/src/osnoise_hist.c
+++ b/tools/tracing/rtla/src/osnoise_hist.c
@@ -699,9 +699,6 @@ static struct osnoise_tool
*osnoise_init_hist(struct common_params *params)
{
struct osnoise_tool *tool;
- int nr_cpus;
-
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
tool = osnoise_init_tool("osnoise_hist");
if (!tool)
diff --git a/tools/tracing/rtla/src/osnoise_top.c b/tools/tracing/rtla/src/osnoise_top.c
index 04c699bdd736..8fa0046f0136 100644
--- a/tools/tracing/rtla/src/osnoise_top.c
+++ b/tools/tracing/rtla/src/osnoise_top.c
@@ -232,12 +232,8 @@ osnoise_print_stats(struct osnoise_tool *top)
{
struct osnoise_params *params = to_osnoise_params(top->params);
struct trace_instance *trace = &top->trace;
- static int nr_cpus = -1;
int i;
- if (nr_cpus == -1)
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
-
if (!params->common.quiet)
clear_terminal(trace->seq);
@@ -547,9 +543,6 @@ osnoise_top_apply_config(struct osnoise_tool *tool)
struct osnoise_tool *osnoise_init_top(struct common_params *params)
{
struct osnoise_tool *tool;
- int nr_cpus;
-
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
tool = osnoise_init_tool("osnoise_top");
if (!tool)
diff --git a/tools/tracing/rtla/src/timerlat.c b/tools/tracing/rtla/src/timerlat.c
index df4f9bfe3433..7503e18b905c 100644
--- a/tools/tracing/rtla/src/timerlat.c
+++ b/tools/tracing/rtla/src/timerlat.c
@@ -108,7 +108,7 @@ timerlat_apply_config(struct osnoise_tool *tool, struct timerlat_params *params)
int timerlat_enable(struct osnoise_tool *tool)
{
struct timerlat_params *params = to_timerlat_params(tool->params);
- int retval, nr_cpus, i;
+ int retval, i;
if (params->dma_latency >= 0) {
dma_latency_fd = set_cpu_dma_latency(params->dma_latency);
@@ -124,8 +124,6 @@ int timerlat_enable(struct osnoise_tool *tool)
return -1;
}
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
-
for_each_monitored_cpu(i, nr_cpus, ¶ms->common) {
if (save_cpu_idle_disable_state(i) < 0) {
err_msg("Could not save cpu idle state.\n");
@@ -213,7 +211,6 @@ void timerlat_analyze(struct osnoise_tool *tool, bool stopped)
void timerlat_free(struct osnoise_tool *tool)
{
struct timerlat_params *params = to_timerlat_params(tool->params);
- int nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
int i;
timerlat_aa_destroy();
diff --git a/tools/tracing/rtla/src/timerlat_aa.c b/tools/tracing/rtla/src/timerlat_aa.c
index 31e66ea2b144..5766d58709eb 100644
--- a/tools/tracing/rtla/src/timerlat_aa.c
+++ b/tools/tracing/rtla/src/timerlat_aa.c
@@ -1022,7 +1022,6 @@ void timerlat_aa_destroy(void)
*/
int timerlat_aa_init(struct osnoise_tool *tool, int dump_tasks)
{
- int nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
struct timerlat_aa_context *taa_ctx;
int retval;
diff --git a/tools/tracing/rtla/src/timerlat_hist.c b/tools/tracing/rtla/src/timerlat_hist.c
index 1fb471a787b7..37bb9b931c8c 100644
--- a/tools/tracing/rtla/src/timerlat_hist.c
+++ b/tools/tracing/rtla/src/timerlat_hist.c
@@ -1077,9 +1077,6 @@ static struct osnoise_tool
*timerlat_init_hist(struct common_params *params)
{
struct osnoise_tool *tool;
- int nr_cpus;
-
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
tool = osnoise_init_tool("timerlat_hist");
if (!tool)
diff --git a/tools/tracing/rtla/src/timerlat_top.c b/tools/tracing/rtla/src/timerlat_top.c
index 29c2c1f717ed..8b15f4439c6c 100644
--- a/tools/tracing/rtla/src/timerlat_top.c
+++ b/tools/tracing/rtla/src/timerlat_top.c
@@ -443,15 +443,11 @@ timerlat_print_stats(struct osnoise_tool *top)
struct timerlat_params *params = to_timerlat_params(top->params);
struct trace_instance *trace = &top->trace;
struct timerlat_top_cpu summary;
- static int nr_cpus = -1;
int i;
if (params->common.aa_only)
return;
- if (nr_cpus == -1)
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
-
if (!params->common.quiet)
clear_terminal(trace->seq);
@@ -827,9 +823,6 @@ static struct osnoise_tool
*timerlat_init_top(struct common_params *params)
{
struct osnoise_tool *top;
- int nr_cpus;
-
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
top = osnoise_init_tool("timerlat_top");
if (!top)
diff --git a/tools/tracing/rtla/src/timerlat_u.c b/tools/tracing/rtla/src/timerlat_u.c
index ce68e39d25fd..a569fe7f93aa 100644
--- a/tools/tracing/rtla/src/timerlat_u.c
+++ b/tools/tracing/rtla/src/timerlat_u.c
@@ -16,7 +16,7 @@
#include <sys/wait.h>
#include <sys/prctl.h>
-#include "utils.h"
+#include "common.h"
#include "timerlat_u.h"
/*
@@ -131,7 +131,6 @@ static int timerlat_u_send_kill(pid_t *procs, int nr_cpus)
*/
void *timerlat_u_dispatcher(void *data)
{
- int nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
struct timerlat_u_params *params = data;
char proc_name[128];
int procs_count = 0;
diff --git a/tools/tracing/rtla/src/utils.c b/tools/tracing/rtla/src/utils.c
index 9cf5a0098e9a..afb32d2431c3 100644
--- a/tools/tracing/rtla/src/utils.c
+++ b/tools/tracing/rtla/src/utils.c
@@ -18,7 +18,7 @@
#include <sched.h>
#include <stdio.h>
-#include "utils.h"
+#include "common.h"
#define MAX_MSG_LENGTH 1024
int config_debug;
@@ -118,14 +118,11 @@ int parse_cpu_set(char *cpu_list, cpu_set_t *set)
{
const char *p;
int end_cpu;
- int nr_cpus;
int cpu;
int i;
CPU_ZERO(set);
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
-
for (p = cpu_list; *p; ) {
cpu = atoi(p);
if (cpu < 0 || (!cpu && *p != '0') || cpu >= nr_cpus)
@@ -552,7 +549,6 @@ int save_cpu_idle_disable_state(unsigned int cpu)
unsigned int nr_states;
unsigned int state;
int disabled;
- int nr_cpus;
nr_states = cpuidle_state_count(cpu);
@@ -560,7 +556,6 @@ int save_cpu_idle_disable_state(unsigned int cpu)
return 0;
if (saved_cpu_idle_disable_state == NULL) {
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
saved_cpu_idle_disable_state = calloc(nr_cpus, sizeof(unsigned int *));
if (!saved_cpu_idle_disable_state)
return -1;
@@ -637,13 +632,10 @@ int restore_cpu_idle_disable_state(unsigned int cpu)
void free_cpu_idle_disable_states(void)
{
int cpu;
- int nr_cpus;
if (!saved_cpu_idle_disable_state)
return;
- nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
-
for (cpu = 0; cpu < nr_cpus; cpu++) {
free(saved_cpu_idle_disable_state[cpu]);
saved_cpu_idle_disable_state[cpu] = NULL;
--
2.52.0
^ permalink raw reply related
* Re: [PATCH] kprobes: Call check_ftrace_location() on CONFIG_KPROBES_ON_FTRACE
From: Steven Rostedt @ 2025-12-05 15:08 UTC (permalink / raw)
To: qingwei.hu; +Cc: naveen, davem, mhiramat, linux-kernel, linux-trace-kernel
In-Reply-To: <20251205092933.3889547-1-qingwei.hu@bytedance.com>
On Fri, 5 Dec 2025 17:29:33 +0800
"qingwei.hu" <qingwei.hu@bytedance.com> wrote:
> From: Qingwei Hu <qingwei.hu@bytedance.com>
>
> There is a possible configuration dependency:
>
> KPROBES_ON_FTRACE [=n]
> ^----- KPROBES [=y]
> |--- HAVE_KPROBES_ON_FTRACE [=n]
> |--- DYNAMIC_FTRACE_WITH_REGS [=n]
> ^----- FTRACE [=y]
> |--- DYNAMIC_FTRACE [=y]
> |--- HAVE_DYNAMIC_FTRACE_WITH_REGS [=n]
>
> With DYNAMIC_FTRACE=y, ftrace_location() is meaningful and may
> return the same address as the probe target.
>
> However, when KPROBES_ON_FTRACE=n, the current implementation
> returns -EINVAL after calling check_ftrace_location(), causing
> the validation to fail.
This is a feature not a bug.
The reason is if you put a kprobe on a ftrace location, it can cause ftrace
to trigger a bug, as kprobes will modify the location and ftrace will see
something it doesn't expect and think the system is corrupted. We don't want
that either.
If you say "KPROBES_ON_FTRACE=n" and place a kprobe on a location that is
controlled by ftrace, it had better fail!
NAK
-- Steve
^ permalink raw reply
* Re: [PATCH v3 09/13] sched: Add deadline tracepoints
From: Phil Auld @ 2025-12-05 13:41 UTC (permalink / raw)
To: Gabriele Monaco
Cc: linux-kernel, Steven Rostedt, Nam Cao, Masami Hiramatsu,
Ingo Molnar, Peter Zijlstra, linux-trace-kernel, Tomas Glozar,
Juri Lelli, Clark Williams, John Kacur
In-Reply-To: <20251205131621.135513-10-gmonaco@redhat.com>
On Fri, Dec 05, 2025 at 02:16:17PM +0100 Gabriele Monaco wrote:
> Add the following tracepoints:
>
> * sched_dl_throttle(dl_se, cpu):
> Called when a deadline entity is throttled
> * sched_dl_replenish(dl_se, cpu):
> Called when a deadline entity's runtime is replenished
> * sched_dl_server_start(dl_se, cpu):
> Called when a deadline server is started
> * sched_dl_server_stop(dl_se, cpu):
> Called when a deadline server is stopped
>
> Those tracepoints can be useful to validate the deadline scheduler with
> RV and are not exported to tracefs.
>
> Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
These seem like they might be useful outside of the RV context as well.
Reviewed-by: Phil Auld <pauld@redhat.com>
Cheers,
Phil
> ---
>
> Notes:
> V3:
> * Rename dl argument to dl_se in tracepoints
>
> include/trace/events/sched.h | 16 ++++++++++++++++
> kernel/sched/core.c | 4 ++++
> kernel/sched/deadline.c | 7 +++++++
> 3 files changed, 27 insertions(+)
>
> diff --git a/include/trace/events/sched.h b/include/trace/events/sched.h
> index 7b2645b50e78..366b2e8ec40c 100644
> --- a/include/trace/events/sched.h
> +++ b/include/trace/events/sched.h
> @@ -896,6 +896,22 @@ DECLARE_TRACE(sched_set_need_resched,
> TP_PROTO(struct task_struct *tsk, int cpu, int tif),
> TP_ARGS(tsk, cpu, tif));
>
> +DECLARE_TRACE(sched_dl_throttle,
> + TP_PROTO(struct sched_dl_entity *dl_se, int cpu),
> + TP_ARGS(dl_se, cpu));
> +
> +DECLARE_TRACE(sched_dl_replenish,
> + TP_PROTO(struct sched_dl_entity *dl_se, int cpu),
> + TP_ARGS(dl_se, cpu));
> +
> +DECLARE_TRACE(sched_dl_server_start,
> + TP_PROTO(struct sched_dl_entity *dl_se, int cpu),
> + TP_ARGS(dl_se, cpu));
> +
> +DECLARE_TRACE(sched_dl_server_stop,
> + TP_PROTO(struct sched_dl_entity *dl_se, int cpu),
> + TP_ARGS(dl_se, cpu));
> +
> #endif /* _TRACE_SCHED_H */
>
> /* This part must be outside protection */
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 5516778c19eb..3aed0ed6f2e9 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -122,6 +122,10 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(sched_compute_energy_tp);
> EXPORT_TRACEPOINT_SYMBOL_GPL(sched_entry_tp);
> EXPORT_TRACEPOINT_SYMBOL_GPL(sched_exit_tp);
> EXPORT_TRACEPOINT_SYMBOL_GPL(sched_set_need_resched_tp);
> +EXPORT_TRACEPOINT_SYMBOL_GPL(sched_dl_throttle_tp);
> +EXPORT_TRACEPOINT_SYMBOL_GPL(sched_dl_replenish_tp);
> +EXPORT_TRACEPOINT_SYMBOL_GPL(sched_dl_server_start_tp);
> +EXPORT_TRACEPOINT_SYMBOL_GPL(sched_dl_server_stop_tp);
>
> DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
> DEFINE_PER_CPU(struct rnd_state, sched_rnd_state);
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index 319439fe1870..2192e9f42e02 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -733,6 +733,7 @@ static inline void replenish_dl_new_period(struct sched_dl_entity *dl_se,
> dl_se->dl_throttled = 1;
> dl_se->dl_defer_armed = 1;
> }
> + trace_sched_dl_replenish_tp(dl_se, cpu_of(rq));
> }
>
> /*
> @@ -850,6 +851,8 @@ static void replenish_dl_entity(struct sched_dl_entity *dl_se)
> if (dl_se->dl_throttled)
> dl_se->dl_throttled = 0;
>
> + trace_sched_dl_replenish_tp(dl_se, cpu_of(rq));
> +
> /*
> * If this is the replenishment of a deferred reservation,
> * clear the flag and return.
> @@ -1341,6 +1344,7 @@ static inline void dl_check_constrained_dl(struct sched_dl_entity *dl_se)
> dl_time_before(rq_clock(rq), dl_next_period(dl_se))) {
> if (unlikely(is_dl_boosted(dl_se) || !start_dl_timer(dl_se)))
> return;
> + trace_sched_dl_throttle_tp(dl_se, cpu_of(rq));
> dl_se->dl_throttled = 1;
> if (dl_se->runtime > 0)
> dl_se->runtime = 0;
> @@ -1504,6 +1508,7 @@ static void update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64
>
> throttle:
> if (dl_runtime_exceeded(dl_se) || dl_se->dl_yielded) {
> + trace_sched_dl_throttle_tp(dl_se, cpu_of(rq));
> dl_se->dl_throttled = 1;
>
> /* If requested, inform the user about runtime overruns. */
> @@ -1795,6 +1800,7 @@ void dl_server_start(struct sched_dl_entity *dl_se)
> if (WARN_ON_ONCE(!cpu_online(cpu_of(rq))))
> return;
>
> + trace_sched_dl_server_start_tp(dl_se, cpu_of(rq));
> dl_se->dl_server_active = 1;
> enqueue_dl_entity(dl_se, ENQUEUE_WAKEUP);
> if (!dl_task(dl_se->rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl))
> @@ -1806,6 +1812,7 @@ void dl_server_stop(struct sched_dl_entity *dl_se)
> if (!dl_server(dl_se) || !dl_server_active(dl_se))
> return;
>
> + trace_sched_dl_server_stop_tp(dl_se, cpu_of(dl_se->rq));
> dequeue_dl_entity(dl_se, DEQUEUE_SLEEP);
> hrtimer_try_to_cancel(&dl_se->dl_timer);
> dl_se->dl_defer_armed = 0;
> --
> 2.52.0
>
>
--
^ permalink raw reply
* Re: [PATCH v3 08/13] sched: Export hidden tracepoints to modules
From: Phil Auld @ 2025-12-05 13:35 UTC (permalink / raw)
To: Gabriele Monaco
Cc: linux-kernel, Steven Rostedt, Nam Cao, Ingo Molnar,
Peter Zijlstra, Tomas Glozar, Juri Lelli, Clark Williams,
John Kacur, linux-trace-kernel
In-Reply-To: <20251205131621.135513-9-gmonaco@redhat.com>
On Fri, Dec 05, 2025 at 02:16:16PM +0100 Gabriele Monaco wrote:
> The tracepoints sched_entry, sched_exit and sched_set_need_resched
> are not exported to tracefs as trace events, this allows only kernel
> code to access them. Helper modules like [1] can be used to still have
> the tracepoints available to ftrace for debugging purposes, but they do
> rely on the tracepoints being exported.
>
> Export the 3 not exported tracepoints.
> Note that sched_set_state is already exported as the macro is called
> from modules.
>
> [1] - https://github.com/qais-yousef/sched_tp.git
>
> Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
This makes sense.
Reviewed-by: Phil Auld <pauld@redhat.com>
Cheers,
Phil
> ---
> kernel/sched/core.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 41ba0be16911..5516778c19eb 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -119,6 +119,9 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(sched_util_est_cfs_tp);
> EXPORT_TRACEPOINT_SYMBOL_GPL(sched_util_est_se_tp);
> EXPORT_TRACEPOINT_SYMBOL_GPL(sched_update_nr_running_tp);
> EXPORT_TRACEPOINT_SYMBOL_GPL(sched_compute_energy_tp);
> +EXPORT_TRACEPOINT_SYMBOL_GPL(sched_entry_tp);
> +EXPORT_TRACEPOINT_SYMBOL_GPL(sched_exit_tp);
> +EXPORT_TRACEPOINT_SYMBOL_GPL(sched_set_need_resched_tp);
>
> DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
> DEFINE_PER_CPU(struct rnd_state, sched_rnd_state);
> --
> 2.52.0
>
>
--
^ permalink raw reply
* [PATCH v3 13/13] rv: Add dl_server specific monitors
From: Gabriele Monaco @ 2025-12-05 13:16 UTC (permalink / raw)
To: linux-kernel, Steven Rostedt, Nam Cao, Gabriele Monaco,
Jonathan Corbet, Masami Hiramatsu, linux-trace-kernel, linux-doc
Cc: Tomas Glozar, Juri Lelli, Clark Williams, John Kacur
In-Reply-To: <20251205131621.135513-1-gmonaco@redhat.com>
Add monitors to validate the behaviour of the deadline server.
The currently implemented monitors are:
* boost
fair tasks run either independently or boosted
* laxity
deferrable servers wait for zero-laxity and run
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
---
Documentation/trace/rv/monitor_deadline.rst | 115 ++++++++
kernel/trace/rv/Kconfig | 2 +
kernel/trace/rv/Makefile | 2 +
kernel/trace/rv/monitors/boost/Kconfig | 15 +
kernel/trace/rv/monitors/boost/boost.c | 279 ++++++++++++++++++
kernel/trace/rv/monitors/boost/boost.h | 159 ++++++++++
kernel/trace/rv/monitors/boost/boost_trace.h | 19 ++
kernel/trace/rv/monitors/laxity/Kconfig | 14 +
kernel/trace/rv/monitors/laxity/laxity.c | 226 ++++++++++++++
kernel/trace/rv/monitors/laxity/laxity.h | 126 ++++++++
.../trace/rv/monitors/laxity/laxity_trace.h | 19 ++
kernel/trace/rv/rv_trace.h | 2 +
tools/verification/models/deadline/boost.dot | 51 ++++
tools/verification/models/deadline/laxity.dot | 34 +++
14 files changed, 1063 insertions(+)
create mode 100644 kernel/trace/rv/monitors/boost/Kconfig
create mode 100644 kernel/trace/rv/monitors/boost/boost.c
create mode 100644 kernel/trace/rv/monitors/boost/boost.h
create mode 100644 kernel/trace/rv/monitors/boost/boost_trace.h
create mode 100644 kernel/trace/rv/monitors/laxity/Kconfig
create mode 100644 kernel/trace/rv/monitors/laxity/laxity.c
create mode 100644 kernel/trace/rv/monitors/laxity/laxity.h
create mode 100644 kernel/trace/rv/monitors/laxity/laxity_trace.h
create mode 100644 tools/verification/models/deadline/boost.dot
create mode 100644 tools/verification/models/deadline/laxity.dot
diff --git a/Documentation/trace/rv/monitor_deadline.rst b/Documentation/trace/rv/monitor_deadline.rst
index 481748adaac3..e5cf5db91650 100644
--- a/Documentation/trace/rv/monitor_deadline.rst
+++ b/Documentation/trace/rv/monitor_deadline.rst
@@ -149,3 +149,118 @@ replenish on an idle CPU, meaningful only for servers::
+---------------+ <-+ ^
| |
+---------------- dl_throttle;is_constr_dl == 1 -------------+
+
+Monitor boost
+~~~~~~~~~~~~~
+
+The boost monitor ensures tasks associated to a server (e.g. fair tasks) run
+either independently or boosted in a timely manner.
+Unlike other models, the ``running`` state (and the ``switch_in/out`` events)
+indicates that any fair task is running, this needs to happen within a
+threshold that depends on server deadline and remaining runtime, whenever a
+task is ready.
+
+The following chart is simplified to avoid confusion, several less important
+self-loops on states have been removed and event names have been simplified:
+
+* ``idle`` (``dl_server_idle``) occurs when the CPU runs the idle task.
+* ``start/stop`` (``dl_server_start/stop``) start and stop the server.
+* ``switch`` (``sched_switch_in/out``) represented as a double arrow to
+ indicate both events are present: ``ready/throttled -- switch_in ->
+ running/throttled_running`` and vice versa with ``switch_out``. As stated
+ above this fires when any fair task starting or stopping to run.
+* ``resume/resume_throttle``: a fair task woke up, potentially when the server
+ is throttled (no runtime left), this event is especially frequent on self
+ loops (no state change during a wakeup) but is removed here for clarity.
+* arrows merges with an ``x`` sign to indicate they are the same event going to
+ the same state (but with different origins). The ``+`` sign indicates normal
+ crossings or corners.
+
+Refer to the dot file for the full specification::
+
+ |
+ v
+ #=========# idle +----------+
+ H H <----- switch_out ----- | |
+ +-->H stopped H | stopping |
+ | H H --+ | |
+ | #=========# | +----------+
+ | ^ | ^
+ | | | stop;reset(clk) | replenish;reset(clk)
+ | stop | | +--+
+ | | start;reset(clk) +--------------------+ | |
+ | | v | | v
+ | +--------- +---------------+ <---------- switch --------> +---------+
+ | | ready | | |
+ | +- resume -> | | -replenish;reset(clk) | running |
+ | | | clk < thesh() | | | |
+ | | +- idle - +---------------+ <-+ +---------------- +---------+
+ | | | | ^ | ^ |
+ | | | | | throttle | |
+ | | | | |replenish;reset(clk) | | |
+ | | | throttle | | replenish;reset(clk) |
+ | | | | | | | |
+ | | | v | v | |
+ | | | +---------+ switch +-------------------+ | |
+ x---+--+-- | | <----------> | throttled_running | --------+ |
+ | | | |throttled| +-------------------+ |
+ | | | | | -----+ | |
+ | | | +---------+ | | |
+ | | | ^ | | |
+ | | | resume_throttle | | |
+ stop | | | | | |
+ | | v | | | |
+ | +---------+ <-----------x--- idle ---x-----------------------------+
+ | | |
+ +-- | idle | <--+
+ | | | replenish;reset(clk)
+ +---------+ ---+
+
+Monitor laxity
+~~~~~~~~~~~~~~
+
+The laxity monitor ensure deferrable servers go to a zero-laxity wait unless
+already running and run in starvation cases. The model can stay in the
+zero-laxity wait only for a limited time depending on deadline and runtime,
+then the server either prepares to stop (after ``idle_wait``) or prepares to
+boost a task (``running``). Boosting (``sched_switch_in``) is only allowed in
+the ``running`` state::
+
+ |
+ +----- dl_server_stop -----+ |
+ | v v
+ | #=======================================#
+ | +------- H stopped H
+ | | #=======================================#
+ | | | ^
+ | | dl_server_start_running; dl_server_stop
+ | | reset(clk) |
+ | | v | dl_replenish_running;
+ | | +-------------------------------------+ clk < REPLENISH_NS
+ | | | | -------------+
+ | | | running | |
+ | | | | <------------+
+ | | +-------------------------------------+
+ | | | ^ ^
+ | | dl_throttle dl_replenish_running |
+ | | v | |
+ | | +-----------------+ | |
+ | | | replenish_wait | -+ |
+ | | +-----------------+ |
+ | | | |
+ | | dl_replenish;reset(clk) | dl_replenish_running
+ | | dl_replenish_idle;reset(clk) |
+ | | v |
+ | dl_server_start; +--------------------------+ dl_replenish;reset(clk)
+ | reset(clk) | zero_laxity_wait | -------------+
+ | | | clk < laxity_ns() | |
+ | +---------------> | | <------------+
+ | +--------------------------+
+ | | ^
+ | dl_replenish_idle;reset(clk) | dl_replenish;reset(clk)
+ | v |
+ | +-------------------------+ |
+ +------------------ | idle_wait | -+
+ +-------------------------+
+ ^ dl_replenish_idle;reset(clk)
+ +-------------+
diff --git a/kernel/trace/rv/Kconfig b/kernel/trace/rv/Kconfig
index 719cdcfb6d41..139443e0e51c 100644
--- a/kernel/trace/rv/Kconfig
+++ b/kernel/trace/rv/Kconfig
@@ -82,6 +82,8 @@ source "kernel/trace/rv/monitors/stall/Kconfig"
source "kernel/trace/rv/monitors/deadline/Kconfig"
source "kernel/trace/rv/monitors/nomiss/Kconfig"
source "kernel/trace/rv/monitors/throttle/Kconfig"
+source "kernel/trace/rv/monitors/boost/Kconfig"
+source "kernel/trace/rv/monitors/laxity/Kconfig"
# Add new deadline monitors here
# Add new monitors here
diff --git a/kernel/trace/rv/Makefile b/kernel/trace/rv/Makefile
index 15a1edc8bd0f..4cf15c189a96 100644
--- a/kernel/trace/rv/Makefile
+++ b/kernel/trace/rv/Makefile
@@ -21,6 +21,8 @@ obj-$(CONFIG_RV_MON_STALL) += monitors/stall/stall.o
obj-$(CONFIG_RV_MON_DEADLINE) += monitors/deadline/deadline.o
obj-$(CONFIG_RV_MON_NOMISS) += monitors/nomiss/nomiss.o
obj-$(CONFIG_RV_MON_THROTTLE) += monitors/throttle/throttle.o
+obj-$(CONFIG_RV_MON_BOOST) += monitors/boost/boost.o
+obj-$(CONFIG_RV_MON_LAXITY) += monitors/laxity/laxity.o
# Add new monitors here
obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
diff --git a/kernel/trace/rv/monitors/boost/Kconfig b/kernel/trace/rv/monitors/boost/Kconfig
new file mode 100644
index 000000000000..3fa121f77729
--- /dev/null
+++ b/kernel/trace/rv/monitors/boost/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_BOOST
+ depends on RV
+ depends on RV_MON_DEADLINE
+ default y
+ select HA_MON_EVENTS_ID
+ bool "boost monitor"
+ help
+ Monitor to ensure tasks associated to a server (e.g. fair tasks) run
+ either independently or boosted in a timely manner.
+ This monitor is part of the deadline monitors collection.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/boost/boost.c b/kernel/trace/rv/monitors/boost/boost.c
new file mode 100644
index 000000000000..1f940dc7a9c5
--- /dev/null
+++ b/kernel/trace/rv/monitors/boost/boost.c
@@ -0,0 +1,279 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+
+#define MODULE_NAME "boost"
+
+#include <trace/events/syscalls.h>
+#include <trace/events/sched.h>
+#include <rv_trace.h>
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define DA_SKIP_AUTO_ALLOC
+#define HA_TIMER_TYPE HA_TIMER_WHEEL
+typedef struct sched_dl_entity *monitor_target;
+#include "boost.h"
+#include <rv/ha_monitor.h>
+#include <monitors/deadline/deadline.h>
+
+#define STOPPING_NS TICK_NSEC
+
+static inline u64 server_threshold_ns(struct ha_monitor *ha_mon)
+{
+ struct sched_dl_entity *dl_se = ha_get_target(ha_mon);
+
+ return dl_se->dl_deadline + TICK_NSEC - dl_se->runtime;
+}
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_boost env, u64 time_ns)
+{
+ if (env == clk_boost)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ return ENV_INVALID_VALUE;
+}
+
+static void ha_reset_env(struct ha_monitor *ha_mon, enum envs_boost env, u64 time_ns)
+{
+ if (env == clk_boost)
+ ha_reset_clk_ns(ha_mon, env, time_ns);
+}
+
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == ready_boost)
+ return ha_check_invariant_ns(ha_mon, clk_boost, time_ns);
+ else if (curr_state == stopping_boost)
+ return ha_check_invariant_ns(ha_mon, clk_boost, time_ns);
+ return true;
+}
+
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ bool res = true;
+
+ if (curr_state == idle_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == ready_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == running_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == running_boost && event == dl_server_stop_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == stopped_boost && event == dl_server_start_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == throttled_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == throttled_running_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ return res;
+}
+
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == curr_state && event != dl_replenish_boost)
+ return;
+ if (next_state == ready_boost)
+ ha_start_timer_ns(ha_mon, clk_boost, server_threshold_ns(ha_mon), time_ns);
+ else if (next_state == stopping_boost)
+ ha_start_timer_ns(ha_mon, clk_boost, STOPPING_NS, time_ns);
+ else if (curr_state == ready_boost)
+ ha_cancel_timer(ha_mon);
+ else if (curr_state == stopping_boost)
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+static void handle_dl_replenish(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_replenish_boost);
+}
+
+static void handle_sched_switch(void *data, bool preempt,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ struct sched_dl_entity *dl_se = get_fair_server(next);
+ int cpu = task_cpu(next);
+
+ /*
+ * The server is available in next only if the next task is boosted,
+ * otherwise we need to retrieve it.
+ * This monitor considers switch in/out whenever a task related to the
+ * server (i.e. fair) is scheduled in or out, boosted or not.
+ * Any switch to the same policy is ignored.
+ * Note: idle may race with concurrent wakeup/migration events.
+ */
+ if (!dl_se || (next->policy == prev->policy && !is_idle_task(next) &&
+ !is_idle_task(prev)))
+ return;
+ if (is_idle_task(next))
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_server_idle_boost);
+ else if (next->policy == SCHED_NORMAL)
+ da_handle_event(EXPAND_ID(dl_se, cpu), sched_switch_in_boost);
+ else if (prev->policy == SCHED_NORMAL && !is_idle_task(prev))
+ da_handle_event(EXPAND_ID(dl_se, cpu), sched_switch_out_boost);
+}
+
+static void handle_syscall(void *data, struct pt_regs *regs, long id)
+{
+ struct sched_dl_entity *dl_se;
+ struct task_struct *p;
+ int new_policy = -1;
+
+ new_policy = extract_params(regs, id, &p);
+ if (new_policy < 0 || new_policy == p->policy)
+ return;
+ dl_se = get_fair_server(p);
+ if (!dl_se || !p->on_rq)
+ return;
+ /*
+ * Note: this attaches to the syscall entry and is unstable:
+ * - the syscall may fail
+ * - the task may be preempted before changing the policy
+ * A more robust solution can be written using enqueue/dequeue events.
+ */
+ if (p->policy == SCHED_NORMAL)
+ da_handle_event(EXPAND_ID(dl_se, task_cpu(p)), sched_switch_out_boost);
+ else if (new_policy == SCHED_NORMAL)
+ da_handle_event(EXPAND_ID(dl_se, task_cpu(p)), dl_server_resume_boost);
+}
+
+static void handle_sched_wakeup(void *data, struct task_struct *tsk)
+{
+ struct sched_dl_entity *dl_se = get_fair_server(tsk);
+
+ if (dl_se && tsk->policy == SCHED_NORMAL) {
+ da_handle_event(EXPAND_ID(dl_se, task_cpu(tsk)),
+ dl_se->runtime > 0 ?
+ dl_server_resume_boost :
+ dl_server_resume_throttled_boost);
+ }
+}
+
+static void handle_sched_migrate(void *data, struct task_struct *p, int dest_cpu)
+{
+ struct sched_dl_entity *orig_dl_se, *dest_dl_se;
+
+ if (p->policy != SCHED_NORMAL)
+ return;
+ orig_dl_se = get_fair_server(p);
+ dest_dl_se = da_get_target_by_id(fair_server_id(dest_cpu));
+ if (orig_dl_se && idle_cpu(task_cpu(p)))
+ da_handle_event(EXPAND_ID(orig_dl_se, task_cpu(p)), dl_server_idle_boost);
+ if (dest_dl_se) {
+ da_handle_event(EXPAND_ID(dest_dl_se, dest_cpu),
+ dest_dl_se->runtime > 0 ?
+ dl_server_resume_boost :
+ dl_server_resume_throttled_boost);
+ }
+}
+
+static void handle_dl_server_start(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_server_start_boost);
+}
+
+static void handle_dl_server_stop(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_start_event(EXPAND_ID(dl_se, cpu), dl_server_stop_boost);
+}
+
+static void handle_dl_throttle(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_throttle_boost);
+}
+
+static int enable_boost(void)
+{
+ int retval;
+
+ retval = da_monitor_init();
+ if (retval)
+ return retval;
+
+ retval = init_storage(true);
+ if (retval)
+ return retval;
+ rv_attach_trace_probe("boost", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_attach_trace_probe("boost", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_attach_trace_probe("boost", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_attach_trace_probe("boost", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_attach_trace_probe("boost", sched_wakeup, handle_sched_wakeup);
+ rv_attach_trace_probe("boost", sched_wakeup_new, handle_sched_wakeup);
+ rv_attach_trace_probe("boost", sched_migrate_task, handle_sched_migrate);
+ rv_attach_trace_probe("boost", sched_switch, handle_sched_switch);
+ if (!should_skip_syscall_handle())
+ rv_attach_trace_probe("boost", sys_enter, handle_syscall);
+
+ return 0;
+}
+
+static void disable_boost(void)
+{
+ rv_this.enabled = 0;
+
+ rv_detach_trace_probe("boost", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_detach_trace_probe("boost", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_detach_trace_probe("boost", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_detach_trace_probe("boost", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_detach_trace_probe("boost", sched_wakeup, handle_sched_wakeup);
+ rv_detach_trace_probe("boost", sched_wakeup_new, handle_sched_wakeup);
+ rv_detach_trace_probe("boost", sched_migrate_task, handle_sched_migrate);
+ rv_detach_trace_probe("boost", sched_switch, handle_sched_switch);
+ if (!should_skip_syscall_handle())
+ rv_detach_trace_probe("boost", sys_enter, handle_syscall);
+
+ da_monitor_destroy();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "boost",
+ .description = "fair tasks run either independently or boosted.",
+ .enable = enable_boost,
+ .disable = disable_boost,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_boost(void)
+{
+ return rv_register_monitor(&rv_this, &rv_deadline);
+}
+
+static void __exit unregister_boost(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_boost);
+module_exit(unregister_boost);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("boost: fair tasks run either independently or boosted.");
diff --git a/kernel/trace/rv/monitors/boost/boost.h b/kernel/trace/rv/monitors/boost/boost.h
new file mode 100644
index 000000000000..2d7b6116b1d0
--- /dev/null
+++ b/kernel/trace/rv/monitors/boost/boost.h
@@ -0,0 +1,159 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Automatically generated C representation of boost automaton
+ * For further information about this format, see kernel documentation:
+ * Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#define MONITOR_NAME boost
+
+enum states_boost {
+ stopped_boost,
+ idle_boost,
+ ready_boost,
+ running_boost,
+ stopping_boost,
+ throttled_boost,
+ throttled_running_boost,
+ state_max_boost,
+};
+
+#define INVALID_STATE state_max_boost
+
+enum events_boost {
+ dl_replenish_boost,
+ dl_server_idle_boost,
+ dl_server_resume_boost,
+ dl_server_resume_throttled_boost,
+ dl_server_start_boost,
+ dl_server_stop_boost,
+ dl_throttle_boost,
+ sched_switch_in_boost,
+ sched_switch_out_boost,
+ event_max_boost,
+};
+
+enum envs_boost {
+ clk_boost,
+ env_max_boost,
+ env_max_stored_boost = env_max_boost,
+};
+
+_Static_assert(env_max_stored_boost <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_boost {
+ char *state_names[state_max_boost];
+ char *event_names[event_max_boost];
+ char *env_names[env_max_boost];
+ unsigned char function[state_max_boost][event_max_boost];
+ unsigned char initial_state;
+ bool final_states[state_max_boost];
+};
+
+static const struct automaton_boost automaton_boost = {
+ .state_names = {
+ "stopped",
+ "idle",
+ "ready",
+ "running",
+ "stopping",
+ "throttled",
+ "throttled_running",
+ },
+ .event_names = {
+ "dl_replenish",
+ "dl_server_idle",
+ "dl_server_resume",
+ "dl_server_resume_throttled",
+ "dl_server_start",
+ "dl_server_stop",
+ "dl_throttle",
+ "sched_switch_in",
+ "sched_switch_out",
+ },
+ .env_names = {
+ "clk",
+ },
+ .function = {
+ {
+ INVALID_STATE,
+ stopped_boost,
+ stopped_boost,
+ stopped_boost,
+ ready_boost,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ stopped_boost,
+ },
+ {
+ idle_boost,
+ idle_boost,
+ ready_boost,
+ throttled_boost,
+ INVALID_STATE,
+ stopped_boost,
+ idle_boost,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ {
+ ready_boost,
+ idle_boost,
+ ready_boost,
+ ready_boost,
+ INVALID_STATE,
+ stopped_boost,
+ throttled_boost,
+ running_boost,
+ ready_boost,
+ },
+ {
+ running_boost,
+ idle_boost,
+ running_boost,
+ running_boost,
+ INVALID_STATE,
+ stopping_boost,
+ throttled_running_boost,
+ INVALID_STATE,
+ ready_boost,
+ },
+ {
+ INVALID_STATE,
+ stopped_boost,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ stopped_boost,
+ },
+ {
+ ready_boost,
+ idle_boost,
+ INVALID_STATE,
+ throttled_boost,
+ INVALID_STATE,
+ stopped_boost,
+ throttled_boost,
+ throttled_running_boost,
+ INVALID_STATE,
+ },
+ {
+ running_boost,
+ idle_boost,
+ INVALID_STATE,
+ throttled_running_boost,
+ INVALID_STATE,
+ INVALID_STATE,
+ throttled_running_boost,
+ INVALID_STATE,
+ throttled_boost,
+ },
+ },
+ .initial_state = stopped_boost,
+ .final_states = { 1, 0, 0, 0, 0, 0, 0 },
+};
diff --git a/kernel/trace/rv/monitors/boost/boost_trace.h b/kernel/trace/rv/monitors/boost/boost_trace.h
new file mode 100644
index 000000000000..7e422b0e586d
--- /dev/null
+++ b/kernel/trace/rv/monitors/boost/boost_trace.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_BOOST
+DEFINE_EVENT(event_da_monitor_id, event_boost,
+ TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_boost,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_boost,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+#endif /* CONFIG_RV_MON_BOOST */
diff --git a/kernel/trace/rv/monitors/laxity/Kconfig b/kernel/trace/rv/monitors/laxity/Kconfig
new file mode 100644
index 000000000000..7ba69405d09b
--- /dev/null
+++ b/kernel/trace/rv/monitors/laxity/Kconfig
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_LAXITY
+ depends on RV
+ depends on RV_MON_DEADLINE
+ default y
+ select HA_MON_EVENTS_ID
+ bool "laxity monitor"
+ help
+ Monitor to ensure deferrable servers go to a zero-laxity wait unless
+ already running and run in starvation cases.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/laxity/laxity.c b/kernel/trace/rv/monitors/laxity/laxity.c
new file mode 100644
index 000000000000..5a2ef4a4ec9b
--- /dev/null
+++ b/kernel/trace/rv/monitors/laxity/laxity.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+
+#define MODULE_NAME "laxity"
+
+#include <trace/events/syscalls.h>
+#include <trace/events/sched.h>
+#include <rv_trace.h>
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define HA_TIMER_TYPE HA_TIMER_WHEEL
+/* The start condition is on server_stop, allocation likely fails on PREEMPT_RT */
+#define DA_SKIP_AUTO_ALLOC
+typedef struct sched_dl_entity *monitor_target;
+#include "laxity.h"
+#include <rv/ha_monitor.h>
+#include <monitors/deadline/deadline.h>
+
+/* with sched_feat(HRTICK_DL) the threshold can be lower */
+#define RUNTIME_THRESH TICK_NSEC
+
+/* allow replenish when running only right after server start */
+#define REPLENISH_NS TICK_NSEC
+
+static inline u64 laxity_ns(struct ha_monitor *ha_mon)
+{
+ struct sched_dl_entity *dl_se = pi_of(ha_get_target(ha_mon));
+
+ return dl_se->dl_deadline - dl_se->runtime + RUNTIME_THRESH;
+}
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_laxity env, u64 time_ns)
+{
+ if (env == clk_laxity)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ return ENV_INVALID_VALUE;
+}
+
+static void ha_reset_env(struct ha_monitor *ha_mon, enum envs_laxity env, u64 time_ns)
+{
+ if (env == clk_laxity)
+ ha_reset_clk_ns(ha_mon, env, time_ns);
+}
+
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == zero_laxity_wait_laxity)
+ return ha_check_invariant_ns(ha_mon, clk_laxity, time_ns);
+ return true;
+}
+
+static inline void ha_convert_inv_guard(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == next_state)
+ return;
+ if (curr_state == zero_laxity_wait_laxity)
+ ha_inv_to_guard(ha_mon, clk_laxity, laxity_ns(ha_mon), time_ns);
+}
+
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ bool res = true;
+
+ if (curr_state == idle_wait_laxity && event == dl_replenish_idle_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == idle_wait_laxity && event == dl_replenish_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == replenish_wait_laxity && event == dl_replenish_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == replenish_wait_laxity && event == dl_replenish_idle_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == running_laxity && event == dl_replenish_running_laxity)
+ res = ha_monitor_env_invalid(ha_mon, clk_laxity) ||
+ ha_get_env(ha_mon, clk_laxity, time_ns) < REPLENISH_NS;
+ else if (curr_state == stopped_laxity && event == dl_server_start_running_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == stopped_laxity && event == dl_server_start_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == zero_laxity_wait_laxity && event == dl_replenish_idle_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == zero_laxity_wait_laxity && event == dl_replenish_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ return res;
+}
+
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == curr_state && event != dl_replenish_idle_laxity &&
+ event != dl_replenish_laxity)
+ return;
+ if (next_state == zero_laxity_wait_laxity)
+ ha_start_timer_ns(ha_mon, clk_laxity, laxity_ns(ha_mon), time_ns);
+ else if (curr_state == zero_laxity_wait_laxity)
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_convert_inv_guard(ha_mon, curr_state, event, next_state, time_ns);
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+static void handle_dl_replenish(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ /* Special replenish happening after throttle, ignore it */
+ if (dl_se->dl_defer_running && dl_se->dl_throttled)
+ return;
+ if (dl_se->dl_defer_running)
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_replenish_running_laxity);
+ else if (idle_cpu(cpu) || (smp_processor_id() == cpu && is_idle_task(current)))
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_replenish_idle_laxity);
+ else
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_replenish_laxity);
+}
+
+static void handle_dl_server_start(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ if (dl_se->dl_defer_running)
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_server_start_running_laxity);
+ else
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_server_start_laxity);
+}
+
+static void handle_dl_server_stop(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_start_event(EXPAND_ID(dl_se, cpu), dl_server_stop_laxity);
+}
+
+static void handle_dl_throttle(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_throttle_laxity);
+}
+
+static void handle_sched_switch(void *data, bool preempt,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ if (next->dl_server)
+ da_handle_start_event(EXPAND_ID(next->dl_server, task_cpu(next)),
+ sched_switch_in_laxity);
+}
+
+static int enable_laxity(void)
+{
+ int retval;
+
+ retval = da_monitor_init();
+ if (retval)
+ return retval;
+
+ retval = init_storage(true);
+ if (retval)
+ return retval;
+ rv_attach_trace_probe("laxity", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_attach_trace_probe("laxity", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_attach_trace_probe("laxity", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_attach_trace_probe("laxity", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_attach_trace_probe("laxity", sched_switch, handle_sched_switch);
+
+ return 0;
+}
+
+static void disable_laxity(void)
+{
+ rv_this.enabled = 0;
+
+ rv_detach_trace_probe("laxity", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_detach_trace_probe("laxity", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_detach_trace_probe("laxity", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_detach_trace_probe("laxity", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_detach_trace_probe("laxity", sched_switch, handle_sched_switch);
+
+ da_monitor_destroy();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "laxity",
+ .description = "deferrable servers wait for zero-laxity and run.",
+ .enable = enable_laxity,
+ .disable = disable_laxity,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_laxity(void)
+{
+ return rv_register_monitor(&rv_this, &rv_deadline);
+}
+
+static void __exit unregister_laxity(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_laxity);
+module_exit(unregister_laxity);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("laxity: deferrable servers wait for zero-laxity and run.");
diff --git a/kernel/trace/rv/monitors/laxity/laxity.h b/kernel/trace/rv/monitors/laxity/laxity.h
new file mode 100644
index 000000000000..3fdab88be535
--- /dev/null
+++ b/kernel/trace/rv/monitors/laxity/laxity.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Automatically generated C representation of laxity automaton
+ * For further information about this format, see kernel documentation:
+ * Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#define MONITOR_NAME laxity
+
+enum states_laxity {
+ stopped_laxity,
+ idle_wait_laxity,
+ replenish_wait_laxity,
+ running_laxity,
+ zero_laxity_wait_laxity,
+ state_max_laxity,
+};
+
+#define INVALID_STATE state_max_laxity
+
+enum events_laxity {
+ dl_replenish_laxity,
+ dl_replenish_idle_laxity,
+ dl_replenish_running_laxity,
+ dl_server_start_laxity,
+ dl_server_start_running_laxity,
+ dl_server_stop_laxity,
+ dl_throttle_laxity,
+ sched_switch_in_laxity,
+ event_max_laxity,
+};
+
+enum envs_laxity {
+ clk_laxity,
+ env_max_laxity,
+ env_max_stored_laxity = env_max_laxity,
+};
+
+_Static_assert(env_max_stored_laxity <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_laxity {
+ char *state_names[state_max_laxity];
+ char *event_names[event_max_laxity];
+ char *env_names[env_max_laxity];
+ unsigned char function[state_max_laxity][event_max_laxity];
+ unsigned char initial_state;
+ bool final_states[state_max_laxity];
+};
+
+static const struct automaton_laxity automaton_laxity = {
+ .state_names = {
+ "stopped",
+ "idle_wait",
+ "replenish_wait",
+ "running",
+ "zero_laxity_wait",
+ },
+ .event_names = {
+ "dl_replenish",
+ "dl_replenish_idle",
+ "dl_replenish_running",
+ "dl_server_start",
+ "dl_server_start_running",
+ "dl_server_stop",
+ "dl_throttle",
+ "sched_switch_in",
+ },
+ .env_names = {
+ "clk",
+ },
+ .function = {
+ {
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ zero_laxity_wait_laxity,
+ running_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ {
+ zero_laxity_wait_laxity,
+ idle_wait_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ stopped_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ {
+ zero_laxity_wait_laxity,
+ zero_laxity_wait_laxity,
+ running_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ {
+ INVALID_STATE,
+ INVALID_STATE,
+ running_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ stopped_laxity,
+ replenish_wait_laxity,
+ running_laxity,
+ },
+ {
+ zero_laxity_wait_laxity,
+ idle_wait_laxity,
+ running_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ },
+ .initial_state = stopped_laxity,
+ .final_states = { 1, 0, 0, 0, 0 },
+};
diff --git a/kernel/trace/rv/monitors/laxity/laxity_trace.h b/kernel/trace/rv/monitors/laxity/laxity_trace.h
new file mode 100644
index 000000000000..32580dba8f42
--- /dev/null
+++ b/kernel/trace/rv/monitors/laxity/laxity_trace.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_LAXITY
+DEFINE_EVENT(event_da_monitor_id, event_laxity,
+ TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_laxity,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_laxity,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+#endif /* CONFIG_RV_MON_LAXITY */
diff --git a/kernel/trace/rv/rv_trace.h b/kernel/trace/rv/rv_trace.h
index 1bf0f3666ee4..f1d55c39dc48 100644
--- a/kernel/trace/rv/rv_trace.h
+++ b/kernel/trace/rv/rv_trace.h
@@ -190,6 +190,8 @@ DECLARE_EVENT_CLASS(error_env_da_monitor_id,
#include <monitors/stall/stall_trace.h>
#include <monitors/nomiss/nomiss_trace.h>
#include <monitors/throttle/throttle_trace.h>
+#include <monitors/boost/boost_trace.h>
+#include <monitors/laxity/laxity_trace.h>
// Add new monitors based on CONFIG_HA_MON_EVENTS_ID here
#endif
diff --git a/tools/verification/models/deadline/boost.dot b/tools/verification/models/deadline/boost.dot
new file mode 100644
index 000000000000..ee7bae1e8feb
--- /dev/null
+++ b/tools/verification/models/deadline/boost.dot
@@ -0,0 +1,51 @@
+digraph state_automaton {
+ center = true;
+ size = "7,11";
+ {node [shape = circle] "idle"};
+ {node [shape = circle] "ready"};
+ {node [shape = circle] "running"};
+ {node [shape = plaintext, style=invis, label=""] "__init_stopped"};
+ {node [shape = doublecircle] "stopped"};
+ {node [shape = circle] "stopped"};
+ {node [shape = circle] "stopping"};
+ {node [shape = circle] "throttled"};
+ {node [shape = circle] "throttled_running"};
+ "__init_stopped" -> "stopped";
+ "idle" [label = "idle"];
+ "idle" -> "idle" [ label = "dl_server_idle\ndl_replenish;reset(clk)\ndl_throttle" ];
+ "idle" -> "ready" [ label = "dl_server_resume" ];
+ "idle" -> "stopped" [ label = "dl_server_stop" ];
+ "idle" -> "throttled" [ label = "dl_server_resume_throttled" ];
+ "ready" [label = "ready\nclk < server_threshold_ns()"];
+ "ready" -> "idle" [ label = "dl_server_idle" ];
+ "ready" -> "ready" [ label = "sched_switch_out\ndl_server_resume_throttled\ndl_server_resume\ndl_replenish;reset(clk)" ];
+ "ready" -> "running" [ label = "sched_switch_in" ];
+ "ready" -> "stopped" [ label = "dl_server_stop" ];
+ "ready" -> "throttled" [ label = "dl_throttle" ];
+ "running" [label = "running"];
+ "running" -> "idle" [ label = "dl_server_idle" ];
+ "running" -> "ready" [ label = "sched_switch_out" ];
+ "running" -> "running" [ label = "dl_server_resume_throttled\ndl_server_resume\ndl_replenish;reset(clk)" ];
+ "running" -> "stopping" [ label = "dl_server_stop;reset(clk)" ];
+ "running" -> "throttled_running" [ label = "dl_throttle" ];
+ "stopped" [label = "stopped", color = green3];
+ "stopped" -> "ready" [ label = "dl_server_start;reset(clk)" ];
+ "stopped" -> "stopped" [ label = "dl_server_idle\nsched_switch_out\ndl_server_resume\ndl_server_resume_throttled" ];
+ "stopping" [label = "stopping\nclk < STOPPING_NS"];
+ "stopping" -> "stopped" [ label = "dl_server_idle\nsched_switch_out" ];
+ "throttled" [label = "throttled"];
+ "throttled" -> "idle" [ label = "dl_server_idle" ];
+ "throttled" -> "ready" [ label = "dl_replenish;reset(clk)" ];
+ "throttled" -> "stopped" [ label = "dl_server_stop" ];
+ "throttled" -> "throttled" [ label = "dl_throttle\ndl_server_resume_throttled" ];
+ "throttled" -> "throttled_running" [ label = "sched_switch_in" ];
+ "throttled_running" [label = "throttled_running"];
+ "throttled_running" -> "idle" [ label = "dl_server_idle" ];
+ "throttled_running" -> "running" [ label = "dl_replenish;reset(clk)" ];
+ "throttled_running" -> "throttled" [ label = "sched_switch_out" ];
+ "throttled_running" -> "throttled_running" [ label = "dl_throttle\ndl_server_resume_throttled" ];
+ { rank = min ;
+ "__init_stopped";
+ "stopped";
+ }
+}
diff --git a/tools/verification/models/deadline/laxity.dot b/tools/verification/models/deadline/laxity.dot
new file mode 100644
index 000000000000..e423a9de573c
--- /dev/null
+++ b/tools/verification/models/deadline/laxity.dot
@@ -0,0 +1,34 @@
+digraph state_automaton {
+ center = true;
+ size = "7,11";
+ {node [shape = circle] "idle_wait"};
+ {node [shape = circle] "replenish_wait"};
+ {node [shape = circle] "running"};
+ {node [shape = plaintext, style=invis, label=""] "__init_stopped"};
+ {node [shape = doublecircle] "stopped"};
+ {node [shape = circle] "stopped"};
+ {node [shape = circle] "zero_laxity_wait"};
+ "__init_stopped" -> "stopped";
+ "idle_wait" [label = "idle_wait"];
+ "idle_wait" -> "idle_wait" [ label = "dl_replenish_idle;reset(clk)" ];
+ "idle_wait" -> "stopped" [ label = "dl_server_stop" ];
+ "idle_wait" -> "zero_laxity_wait" [ label = "dl_replenish;reset(clk)" ];
+ "replenish_wait" [label = "replenish_wait"];
+ "replenish_wait" -> "running" [ label = "dl_replenish_running" ];
+ "replenish_wait" -> "zero_laxity_wait" [ label = "dl_replenish;reset(clk)\ndl_replenish_idle;reset(clk)" ];
+ "running" [label = "running"];
+ "running" -> "replenish_wait" [ label = "dl_throttle" ];
+ "running" -> "running" [ label = "dl_replenish_running;clk < REPLENISH_NS" ];
+ "running" -> "stopped" [ label = "dl_server_stop" ];
+ "stopped" [label = "stopped", color = green3];
+ "stopped" -> "running" [ label = "dl_server_start_running;reset(clk)" ];
+ "stopped" -> "zero_laxity_wait" [ label = "dl_server_start;reset(clk)" ];
+ "zero_laxity_wait" [label = "zero_laxity_wait\nclk < laxity_ns()"];
+ "zero_laxity_wait" -> "idle_wait" [ label = "dl_replenish_idle;reset(clk)" ];
+ "zero_laxity_wait" -> "running" [ label = "dl_replenish_running" ];
+ "zero_laxity_wait" -> "zero_laxity_wait" [ label = "dl_replenish;reset(clk)" ];
+ { rank = min ;
+ "__init_stopped";
+ "stopped";
+ }
+}
--
2.52.0
^ permalink raw reply related
* [PATCH v3 12/13] rv: Add deadline monitors
From: Gabriele Monaco @ 2025-12-05 13:16 UTC (permalink / raw)
To: linux-kernel, Steven Rostedt, Nam Cao, Gabriele Monaco,
Jonathan Corbet, Masami Hiramatsu, linux-trace-kernel, linux-doc
Cc: Tomas Glozar, Juri Lelli, Clark Williams, John Kacur
In-Reply-To: <20251205131621.135513-1-gmonaco@redhat.com>
Add the deadline monitors collection to validate the deadline scheduler,
both for deadline tasks and servers.
The currently implemented monitors are:
* throttle:
validate dl entities are throttled when they use up their runtime
* nomiss:
validate dl entities run to completion before their deadiline
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
---
Notes:
V3:
* Adapt models to new dl server behaviour
* Rename dl argument to dl_se in tracepoints
* Use __COUNTER__ in dl monitor syscall helpers
Documentation/trace/rv/index.rst | 1 +
Documentation/trace/rv/monitor_deadline.rst | 151 +++++++++
kernel/trace/rv/Kconfig | 5 +
kernel/trace/rv/Makefile | 3 +
kernel/trace/rv/monitors/deadline/Kconfig | 10 +
kernel/trace/rv/monitors/deadline/deadline.c | 35 +++
kernel/trace/rv/monitors/deadline/deadline.h | 203 ++++++++++++
kernel/trace/rv/monitors/nomiss/Kconfig | 15 +
kernel/trace/rv/monitors/nomiss/nomiss.c | 289 ++++++++++++++++++
kernel/trace/rv/monitors/nomiss/nomiss.h | 137 +++++++++
.../trace/rv/monitors/nomiss/nomiss_trace.h | 19 ++
kernel/trace/rv/monitors/throttle/Kconfig | 15 +
kernel/trace/rv/monitors/throttle/throttle.c | 248 +++++++++++++++
kernel/trace/rv/monitors/throttle/throttle.h | 116 +++++++
.../rv/monitors/throttle/throttle_trace.h | 19 ++
kernel/trace/rv/rv_trace.h | 2 +
tools/verification/models/deadline/nomiss.dot | 43 +++
.../verification/models/deadline/throttle.dot | 44 +++
18 files changed, 1355 insertions(+)
create mode 100644 Documentation/trace/rv/monitor_deadline.rst
create mode 100644 kernel/trace/rv/monitors/deadline/Kconfig
create mode 100644 kernel/trace/rv/monitors/deadline/deadline.c
create mode 100644 kernel/trace/rv/monitors/deadline/deadline.h
create mode 100644 kernel/trace/rv/monitors/nomiss/Kconfig
create mode 100644 kernel/trace/rv/monitors/nomiss/nomiss.c
create mode 100644 kernel/trace/rv/monitors/nomiss/nomiss.h
create mode 100644 kernel/trace/rv/monitors/nomiss/nomiss_trace.h
create mode 100644 kernel/trace/rv/monitors/throttle/Kconfig
create mode 100644 kernel/trace/rv/monitors/throttle/throttle.c
create mode 100644 kernel/trace/rv/monitors/throttle/throttle.h
create mode 100644 kernel/trace/rv/monitors/throttle/throttle_trace.h
create mode 100644 tools/verification/models/deadline/nomiss.dot
create mode 100644 tools/verification/models/deadline/throttle.dot
diff --git a/Documentation/trace/rv/index.rst b/Documentation/trace/rv/index.rst
index bf9962f49959..29769f06bb0f 100644
--- a/Documentation/trace/rv/index.rst
+++ b/Documentation/trace/rv/index.rst
@@ -17,3 +17,4 @@ Runtime Verification
monitor_sched.rst
monitor_rtapp.rst
monitor_stall.rst
+ monitor_deadline.rst
diff --git a/Documentation/trace/rv/monitor_deadline.rst b/Documentation/trace/rv/monitor_deadline.rst
new file mode 100644
index 000000000000..481748adaac3
--- /dev/null
+++ b/Documentation/trace/rv/monitor_deadline.rst
@@ -0,0 +1,151 @@
+Scheduler monitors
+==================
+
+- Name: deadline
+- Type: container for multiple monitors
+- Author: Gabriele Monaco <gmonaco@redhat.com>
+
+Description
+-----------
+
+The deadline monitor is a set of specifications to describe the deadline
+scheduler behaviour. It includes monitors per scheduling entity (deadline tasks
+and servers) that work independently to verify different specifications the
+deadline scheduler should follow.
+
+Specifications
+--------------
+
+Monitor throttle
+~~~~~~~~~~~~~~~~
+
+The throttle monitor ensures deadline entities are throttled when they use up
+their runtime. Deadline tasks can be only ``running``, ``preempted`` and
+``throttled``, the runtime is enforced only in ``running`` based on an internal
+clock and the runtime value in the deadline entity.
+
+Servers can be also in the ``armed`` state, which corresponds to when the
+server is consuming bandwidth in background (e.g. idle or normal tasks are
+running without any boost). From this state the server can be throttled but it
+can also use more runtime than available. A server is considered ``running``
+when it's actively boosting a task, only there the runtime is enforced. The
+server is preempted if the running task is not in the server's runqueue (e.g. a
+FIFO task for the fair server).
+Events like ``dl_armed`` and ``sched_switch_in`` can occur sequentially for
+servers since they are related to the current task (e.g. a 2 fair tasks can be
+switched in sequentially, that corresponds to multiple ``dl_armed``).
+
+Any task or server in the ``throttled`` state must leave it shortly, e.g.
+become ``preempted``::
+
+ |
+ |
+ dl_replenish;reset(clk) v
+ sched_switch_in #=========================# sched_switch_in;
+ +--------------- H H reset(clk)
+ | H H <----------------+
+ +--------------> H running H |
+ dl_throttle;reset(clk) H clk < runtime_left_ns() H |
+ +--------------------------- H H sched_switch_out |
+ | +------------------> H H -------------+ |
+ | dl_replenish;reset(clk) #=========================# | |
+ | | | ^ | |
+ v | dl_defer_arm | | |
+ +-------------------------+ | | | |
+ | throttled | | sched_switch_in;reset(clk) | |
+ | clk < THROTTLED_TIME_NS | v | | |
+ +-------------------------+ +----------------+ | |
+ | | | | sched_switch_out | |
+ | | +---------- | | -------------+ | |
+ | | dl_replenish | armed | | | |
+ | | dl_defer_arm | | <--------+ | | |
+ | | +---------> | | dl_defer_arm | | |
+ | | +----------------+ | | | |
+ | | | ^ | | | |
+ | | dl_throttle dl_replenish | | | |
+ | | v | | | | |
+ | | dl_defer_arm +-------------------+ | v v |
+ | | +---------- | | +--------------+
+ | | | | | | |
+ | | +---------> | armed_throttled | | preempted |
+ | | | | | |
+ | +-----------------> | | +--------------+
+ | dl_defer_arm +-------------------+ sched_switch_out ^ | ^
+ | | ^ dl_replenish | | |
+ | sched_switch_out dl_defer_arm +----+ | |
+ | v | | |
+ | sched_switch_out +-----------------------+ | |
+ | +-------------- | | dl_throttle; | |
+ | | | | is_constr_dl==1 | |
+ | +-------------> | preempted_throttled | <-----------------+ |
+ | | | |
+ +-----------------------> | | -- dl_replenish -----+
+ sched_switch_out +-----------------------+
+
+The value of ``runtime_left_ns()`` is directly read from the deadline entity
+and updated as the task runs. It is increased by 1 tick to account for the
+maximum delay to throttle (not valid if ``sched_feat(HRTICK_DL)`` is active).
+
+Monitor nomiss
+~~~~~~~~~~~~~~
+
+The nomiss monitor ensures dl entities get to run *and* run to completion
+before their deadline, although deferrable servers may not run. An entity is
+considered done if ``throttled``, either because it yielded or used up its
+runtime, or when it voluntarily starts ``sleeping``.
+The monitor includes a user configurable deadline threshold. If the total
+utilisation of deadline tasks is larger than 1, they are only guaranteed
+bounded tardiness. See Documentation/scheduler/sched-deadline.rst for more
+details. The threshold (module parameter ``nomiss.deadline_thresh``) can be
+configured to avoid the monitor to fail based on the acceptable tardiness in
+the system. Since ``dl_throttle`` is a valid outcome for the entity to be done,
+the minimum tardiness needs be 1 tick to consider the throttle delay.
+
+Servers have also an intermediate ``idle`` state, occurring as soon as no
+runnable task is available. The special event ``dl_replenish_idle`` is a common
+replenish on an idle CPU, meaningful only for servers::
+
+ sched_wakeup |
+ dl_server_start v
+ dl_replenish;reset(clk) -- #=========================#
+ | H H dl_replenish;reset(clk)
+ +-----------> H H <--------------------+
+ H H |
+ +- dl_server_stop ---- H ready H |
+ | +-----------------> H clk < DEADLINE_NS() H dl_throttle; |
+ | | H H is_defer == 1 |
+ | | sched_switch_in - H H ------------------+ |
+ | | | #=========================# | |
+ | | | | ^ | |
+ | | | dl_server_idle dl_replenish;reset(clk) | |
+ | | | v | | |
+ | | | dl_server_idle +--------------+ | |
+ | | | dl_server_start --- | | | |
+ | | | dl_replenish_idle; | | dl_throttle | |
+ | | | reset(clk) | idle | ------------------+ | |
+ | | | +--------> | | | | |
+ +-----+--+---+--- dl_server_idle -> | | dl_server_idle | | |
+ | | | | | | <--------------+ | | |
+ | +--+--+---+--- dl_server_stop -- +--------------+ | | | |
+ | | | | | | ^ | | | |
+ | | | | | sched_switch_in dl_server_idle | | | |
+ | | | | | v | | | | |
+ | | | | | +---------- +---------------------+ | | | |
+ | | | | | sched_switch_in | | | | | |
+ | | | | | sched_wakeup | | | | | |
+ | | | | | dl_replenish; | running | -------+ | | | |
+ | | | | | reset(clk) | clk < DEADLINE_NS() | | | | | |
+ | | | | | +---------> | | dl_throttle | | | |
+ | | | | +----------------> | | | | | | |
+ | | | | +---------------------+ | | | | |
+ | | | sched_wakeup ^ sched_switch_suspend | | | | |
+ | v v dl_replenish*;reset(clk) | dl_server_stop | | | | |
+ +---------------+ | | v | v v |
+ | | - sched_switch_in + | +---------------+
+ | | <---------------------+ dl_throttle +-- | |
+ | sleeping | sched_wakeup | | throttled |
+ | | --+ dl_server_start +-> | |
+ | | dl_server_start sched_switch_suspend +---------------+
+ +---------------+ <-+ ^
+ | |
+ +---------------- dl_throttle;is_constr_dl == 1 -------------+
diff --git a/kernel/trace/rv/Kconfig b/kernel/trace/rv/Kconfig
index 720fbe4935f8..719cdcfb6d41 100644
--- a/kernel/trace/rv/Kconfig
+++ b/kernel/trace/rv/Kconfig
@@ -79,6 +79,11 @@ source "kernel/trace/rv/monitors/sleep/Kconfig"
# Add new rtapp monitors here
source "kernel/trace/rv/monitors/stall/Kconfig"
+source "kernel/trace/rv/monitors/deadline/Kconfig"
+source "kernel/trace/rv/monitors/nomiss/Kconfig"
+source "kernel/trace/rv/monitors/throttle/Kconfig"
+# Add new deadline monitors here
+
# Add new monitors here
config RV_REACTORS
diff --git a/kernel/trace/rv/Makefile b/kernel/trace/rv/Makefile
index 51c95e2d2da6..15a1edc8bd0f 100644
--- a/kernel/trace/rv/Makefile
+++ b/kernel/trace/rv/Makefile
@@ -18,6 +18,9 @@ obj-$(CONFIG_RV_MON_NRP) += monitors/nrp/nrp.o
obj-$(CONFIG_RV_MON_SSSW) += monitors/sssw/sssw.o
obj-$(CONFIG_RV_MON_OPID) += monitors/opid/opid.o
obj-$(CONFIG_RV_MON_STALL) += monitors/stall/stall.o
+obj-$(CONFIG_RV_MON_DEADLINE) += monitors/deadline/deadline.o
+obj-$(CONFIG_RV_MON_NOMISS) += monitors/nomiss/nomiss.o
+obj-$(CONFIG_RV_MON_THROTTLE) += monitors/throttle/throttle.o
# Add new monitors here
obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
diff --git a/kernel/trace/rv/monitors/deadline/Kconfig b/kernel/trace/rv/monitors/deadline/Kconfig
new file mode 100644
index 000000000000..38804a6ad91d
--- /dev/null
+++ b/kernel/trace/rv/monitors/deadline/Kconfig
@@ -0,0 +1,10 @@
+config RV_MON_DEADLINE
+ depends on RV
+ bool "deadline monitor"
+ help
+ Collection of monitors to check the deadline scheduler and server
+ behave according to specifications. Enable this to enable all
+ scheduler specification supported by the current kernel.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/deadline/deadline.c b/kernel/trace/rv/monitors/deadline/deadline.c
new file mode 100644
index 000000000000..45aed62c1371
--- /dev/null
+++ b/kernel/trace/rv/monitors/deadline/deadline.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+
+#define MODULE_NAME "deadline"
+
+#include "deadline.h"
+
+struct rv_monitor rv_deadline = {
+ .name = "deadline",
+ .description = "container for several deadline scheduler specifications.",
+ .enable = NULL,
+ .disable = NULL,
+ .reset = NULL,
+ .enabled = 0,
+};
+
+static int __init register_deadline(void)
+{
+ return rv_register_monitor(&rv_deadline, NULL);
+}
+
+static void __exit unregister_deadline(void)
+{
+ rv_unregister_monitor(&rv_deadline);
+}
+
+module_init(register_deadline);
+module_exit(unregister_deadline);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("deadline: container for several deadline scheduler specifications.");
diff --git a/kernel/trace/rv/monitors/deadline/deadline.h b/kernel/trace/rv/monitors/deadline/deadline.h
new file mode 100644
index 000000000000..f6c9689d1c7d
--- /dev/null
+++ b/kernel/trace/rv/monitors/deadline/deadline.h
@@ -0,0 +1,203 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#include <linux/kernel.h>
+#include <linux/uaccess.h>
+#include <asm/syscall.h>
+#include <uapi/linux/sched/types.h>
+
+/*
+ * Dummy values if not available
+ */
+#ifndef __NR_sched_setscheduler
+#define __NR_sched_setscheduler -__COUNTER__
+#endif
+#ifndef __NR_sched_setattr
+#define __NR_sched_setattr -__COUNTER__
+#endif
+
+extern struct rv_monitor rv_deadline;
+
+static inline struct task_struct *dl_task_of(struct sched_dl_entity *dl_se)
+{
+ if (WARN_ONCE(dl_se->dl_server, "Call this only on a DL task!"))
+ return NULL;
+ return container_of(dl_se, struct task_struct, dl);
+}
+
+#ifdef CONFIG_RT_MUTEXES
+static inline struct sched_dl_entity *pi_of(struct sched_dl_entity *dl_se)
+{
+ return dl_se->pi_se;
+}
+
+static inline bool is_dl_boosted(struct sched_dl_entity *dl_se)
+{
+ return pi_of(dl_se) != dl_se;
+}
+#else /* !CONFIG_RT_MUTEXES: */
+static inline struct sched_dl_entity *pi_of(struct sched_dl_entity *dl_se)
+{
+ return dl_se;
+}
+
+static inline bool is_dl_boosted(struct sched_dl_entity *dl_se)
+{
+ return false;
+}
+#endif /* !CONFIG_RT_MUTEXES */
+
+static inline bool dl_is_implicit(struct sched_dl_entity *dl_se)
+{
+ return dl_se->dl_deadline == dl_se->dl_period;
+}
+
+/*
+ * If both have dummy values, the syscalls are not supported and we don't even
+ * need to register the handler.
+ */
+static inline bool should_skip_syscall_handle(void)
+{
+ return __NR_sched_setattr < 0 && __NR_sched_setscheduler < 0;
+}
+
+/*
+ * Use negative numbers for the server.
+ * Currently only one fair server per CPU, may change in the future.
+ */
+#define fair_server_id(cpu) (-cpu)
+/*
+ * Get a unique id used for dl entities
+ *
+ * The cpu is not required for tasks as the pid is used there, if this function
+ * is called on a dl_se that for sure corresponds to a task, DL_TASK can be
+ * used in place of cpu.
+ * We need the cpu for servers as it is provided in the tracepoint and we
+ * cannot easily retrieve it from the dl_se (requires the struct rq definition).
+ */
+static inline int get_entity_id(struct sched_dl_entity *dl_se, int cpu)
+{
+ if (dl_se->dl_server)
+ return fair_server_id(cpu);
+ return dl_task_of(dl_se)->pid;
+}
+
+/* Expand id and target as arguments for da functions */
+#define EXPAND_ID(dl_se, cpu) get_entity_id(dl_se, cpu), dl_se
+
+/* Use this as the cpu in EXPAND_ID in case the dl_se is surely from a task */
+#define DL_TASK -1
+
+static inline int extract_params(struct pt_regs *regs, long id, struct task_struct **p)
+{
+ size_t size = offsetof(struct sched_attr, sched_nice);
+ struct sched_attr __user *uattr, attr;
+ int new_policy = -1, ret;
+ unsigned long args[6];
+ pid_t pid;
+
+ switch (id) {
+ case __NR_sched_setscheduler:
+ syscall_get_arguments(current, regs, args);
+ pid = args[0];
+ new_policy = args[1];
+ break;
+ case __NR_sched_setattr:
+ syscall_get_arguments(current, regs, args);
+ pid = args[0];
+ uattr = (void *)args[1];
+ /*
+ * Just copy up to sched_flags, we are not interested after that
+ */
+ ret = copy_struct_from_user(&attr, size, uattr, size);
+ if (ret)
+ return ret;
+ if (attr.sched_flags & SCHED_FLAG_KEEP_POLICY)
+ return -EINVAL;
+ new_policy = attr.sched_policy;
+ break;
+ default:
+ return -EINVAL;
+ }
+ if (!pid)
+ *p = current;
+ else {
+ /*
+ * Required for find_task_by_vpid, make sure the caller doesn't
+ * need to get_task_struct().
+ */
+ guard(rcu)();
+ *p = find_task_by_vpid(pid);
+ if (unlikely(!p))
+ return -EINVAL;
+ }
+
+ return new_policy & ~SCHED_RESET_ON_FORK;
+}
+
+/* Helper functions requiring DA/HA utilities */
+#ifdef RV_MON_TYPE
+
+/*
+ * get_fair_server - get the fair server associated to a task
+ *
+ * If the task is a boosted task, the server is available in the task_struct,
+ * otherwise grab the dl entity saved for the CPU where the task is enqueued.
+ * This function assumes the task is enqueued somewhere.
+ */
+static inline struct sched_dl_entity *get_fair_server(struct task_struct *tsk)
+{
+ if (tsk->dl_server)
+ return tsk->dl_server;
+ return da_get_target_by_id(fair_server_id(task_cpu(tsk)));
+}
+
+/*
+ * Initialise monitors for all tasks and pre-allocate the storage for servers.
+ * This is necessary since we don't have access to the servers here and
+ * allocation can cause deadlocks from their tracepoints. We can only fill
+ * pre-initialised storage from there.
+ */
+static inline int init_storage(bool skip_tasks)
+{
+ struct task_struct *g, *p;
+ int cpu;
+
+ for_each_possible_cpu(cpu) {
+ if (!da_create_empty_storage(fair_server_id(cpu)))
+ goto fail;
+ }
+
+ if (skip_tasks)
+ return 0;
+
+ read_lock(&tasklist_lock);
+ for_each_process_thread(g, p) {
+ if (p->policy == SCHED_DEADLINE) {
+ if (!da_create_storage(EXPAND_ID(&p->dl, DL_TASK), NULL)) {
+ read_unlock(&tasklist_lock);
+ goto fail;
+ }
+ }
+ }
+ read_unlock(&tasklist_lock);
+ return 0;
+
+fail:
+ da_monitor_destroy();
+ return -ENOMEM;
+}
+
+static void __maybe_unused handle_newtask(void *data, struct task_struct *task, u64 flags)
+{
+ /* Might be superfluous as tasks are not started with this policy.. */
+ if (task->policy == SCHED_DEADLINE)
+ da_create_storage(EXPAND_ID(&task->dl, DL_TASK), NULL);
+}
+
+static void __maybe_unused handle_exit(void *data, struct task_struct *p, bool group_dead)
+{
+ if (p->policy == SCHED_DEADLINE)
+ da_destroy_storage(get_entity_id(&p->dl, DL_TASK));
+}
+
+#endif
diff --git a/kernel/trace/rv/monitors/nomiss/Kconfig b/kernel/trace/rv/monitors/nomiss/Kconfig
new file mode 100644
index 000000000000..e1886c3a0dd9
--- /dev/null
+++ b/kernel/trace/rv/monitors/nomiss/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_NOMISS
+ depends on RV
+ depends on HAVE_SYSCALL_TRACEPOINTS
+ depends on RV_MON_DEADLINE
+ default y
+ select HA_MON_EVENTS_ID
+ bool "nomiss monitor"
+ help
+ Monitor to ensure dl entities run to completion before their deadiline.
+ This monitor is part of the deadline monitors collection.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/nomiss/nomiss.c b/kernel/trace/rv/monitors/nomiss/nomiss.c
new file mode 100644
index 000000000000..1b0b79d39ddf
--- /dev/null
+++ b/kernel/trace/rv/monitors/nomiss/nomiss.c
@@ -0,0 +1,289 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+
+#define MODULE_NAME "nomiss"
+
+#include <uapi/linux/sched/types.h>
+#include <trace/events/syscalls.h>
+#include <trace/events/sched.h>
+#include <trace/events/task.h>
+#include <rv_trace.h>
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define HA_TIMER_TYPE HA_TIMER_WHEEL
+/* The start condition is on sched_switch, it's dangerous to allocate there */
+#define DA_SKIP_AUTO_ALLOC
+typedef struct sched_dl_entity *monitor_target;
+#include "nomiss.h"
+#include <rv/ha_monitor.h>
+#include <monitors/deadline/deadline.h>
+
+/*
+ * User configurable deadline threshold. If the total utilisation of deadline
+ * tasks is larger than 1, they are only guaranteed bounded tardiness. See
+ * Documentation/scheduler/sched-deadline.rst for more details.
+ * The minimum tardiness without sched_feat(HRTICK_DL) is 1 tick to accommodate
+ * for throttle enforced on the next tick.
+ */
+static u64 deadline_thresh = TICK_NSEC;
+module_param(deadline_thresh, ullong, 0644);
+#define DEADLINE_NS(ha_mon) (pi_of(ha_get_target(ha_mon))->dl_deadline + deadline_thresh)
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_nomiss env, u64 time_ns)
+{
+ if (env == clk_nomiss)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ else if (env == is_constr_dl_nomiss)
+ return !dl_is_implicit(ha_get_target(ha_mon));
+ else if (env == is_defer_nomiss)
+ return ha_get_target(ha_mon)->dl_defer;
+ return ENV_INVALID_VALUE;
+}
+
+static void ha_reset_env(struct ha_monitor *ha_mon, enum envs_nomiss env, u64 time_ns)
+{
+ if (env == clk_nomiss)
+ ha_reset_clk_ns(ha_mon, env, time_ns);
+}
+
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == ready_nomiss)
+ return ha_check_invariant_ns(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == running_nomiss)
+ return ha_check_invariant_ns(ha_mon, clk_nomiss, time_ns);
+ return true;
+}
+
+static inline void ha_convert_inv_guard(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == next_state)
+ return;
+ if (curr_state == ready_nomiss)
+ ha_inv_to_guard(ha_mon, clk_nomiss, DEADLINE_NS(ha_mon), time_ns);
+ else if (curr_state == running_nomiss)
+ ha_inv_to_guard(ha_mon, clk_nomiss, DEADLINE_NS(ha_mon), time_ns);
+}
+
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ bool res = true;
+
+ if (curr_state == idle_nomiss && event == dl_replenish_idle_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == idle_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == ready_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == ready_nomiss && event == dl_throttle_nomiss)
+ res = ha_get_env(ha_mon, is_defer_nomiss, time_ns) == 1ull;
+ else if (curr_state == running_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == sleeping_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == sleeping_nomiss && event == dl_replenish_idle_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == sleeping_nomiss && event == dl_throttle_nomiss)
+ res = ha_get_env(ha_mon, is_constr_dl_nomiss, time_ns) == 1ull;
+ else if (curr_state == throttled_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ return res;
+}
+
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == curr_state && event != dl_replenish_nomiss &&
+ event != dl_replenish_idle_nomiss)
+ return;
+ if (next_state == ready_nomiss)
+ ha_start_timer_ns(ha_mon, clk_nomiss, DEADLINE_NS(ha_mon), time_ns);
+ else if (next_state == running_nomiss)
+ ha_start_timer_ns(ha_mon, clk_nomiss, DEADLINE_NS(ha_mon), time_ns);
+ else if (curr_state == ready_nomiss)
+ ha_cancel_timer(ha_mon);
+ else if (curr_state == running_nomiss)
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_convert_inv_guard(ha_mon, curr_state, event, next_state, time_ns);
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+static void handle_dl_replenish(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ /* Special case of replenish with idle CPU */
+ if ((idle_cpu(cpu) || (smp_processor_id() == cpu && is_idle_task(current))) &&
+ dl_se->dl_server) {
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_replenish_idle_nomiss);
+ return;
+ }
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_replenish_nomiss);
+}
+
+static void handle_dl_throttle(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_throttle_nomiss);
+}
+
+static void handle_dl_server_start(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_server_start_nomiss);
+}
+
+static void handle_dl_server_stop(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ /*
+ * This isn't the standard use of da_handle_start_run_event since those
+ * events cannot only occur from the initial states.
+ * It is fine to use here because they always bring to a known state
+ * and the fact we "pretend" the transition starts from the initial
+ * state has no side effect.
+ */
+ da_handle_start_run_event(EXPAND_ID(dl_se, cpu), dl_server_stop_nomiss);
+}
+
+static void handle_sched_switch(void *data, bool preempt,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ struct sched_dl_entity *dl_se;
+ int cpu = task_cpu(next);
+
+ if (prev_state != TASK_RUNNING && !preempt && prev->policy == SCHED_DEADLINE)
+ da_handle_event(EXPAND_ID(&prev->dl, cpu), sched_switch_suspend_nomiss);
+ if (next->policy == SCHED_DEADLINE)
+ da_handle_start_run_event(EXPAND_ID(&next->dl, cpu), sched_switch_in_nomiss);
+
+ /*
+ * The server is available in next only if the next task is boosted,
+ * otherwise we need to retrieve it.
+ * Here the server continues in the state running/armed until actually
+ * stopped, this works since we continue expecting a throttle.
+ */
+ dl_se = get_fair_server(next);
+ if (!dl_se)
+ return;
+ if (next->dl_server)
+ da_handle_start_run_event(EXPAND_ID(next->dl_server, cpu), sched_switch_in_nomiss);
+ else if (is_idle_task(next))
+ da_handle_start_run_event(EXPAND_ID(dl_se, cpu), dl_server_idle_nomiss);
+}
+
+static void handle_syscall(void *data, struct pt_regs *regs, long id)
+{
+ struct task_struct *p;
+ int new_policy = -1;
+
+ new_policy = extract_params(regs, id, &p);
+ if (new_policy < 0 || new_policy == p->policy)
+ return;
+ if (p->policy == SCHED_DEADLINE)
+ da_reset(EXPAND_ID(&p->dl, DL_TASK));
+ else if (new_policy == SCHED_DEADLINE)
+ da_create_or_get(EXPAND_ID(&p->dl, DL_TASK));
+}
+
+static void handle_sched_wakeup(void *data, struct task_struct *tsk)
+{
+ if (tsk->policy == SCHED_DEADLINE)
+ da_handle_event(EXPAND_ID(&tsk->dl, task_cpu(tsk)), sched_wakeup_nomiss);
+}
+
+static int enable_nomiss(void)
+{
+ int retval;
+
+ retval = da_monitor_init();
+ if (retval)
+ return retval;
+
+ retval = init_storage(false);
+ if (retval)
+ return retval;
+ rv_attach_trace_probe("nomiss", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_attach_trace_probe("nomiss", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_attach_trace_probe("nomiss", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_attach_trace_probe("nomiss", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_attach_trace_probe("nomiss", sched_switch, handle_sched_switch);
+ rv_attach_trace_probe("nomiss", sched_wakeup, handle_sched_wakeup);
+ if (!should_skip_syscall_handle())
+ rv_attach_trace_probe("nomiss", sys_enter, handle_syscall);
+ rv_attach_trace_probe("nomiss", task_newtask, handle_newtask);
+ rv_attach_trace_probe("nomiss", sched_process_exit, handle_exit);
+
+ return 0;
+}
+
+static void disable_nomiss(void)
+{
+ rv_this.enabled = 0;
+
+ /* Those are RCU writers, detach earlier hoping to close a bit faster */
+ rv_detach_trace_probe("nomiss", task_newtask, handle_newtask);
+ rv_detach_trace_probe("nomiss", sched_process_exit, handle_exit);
+ if (!should_skip_syscall_handle())
+ rv_detach_trace_probe("nomiss", sys_enter, handle_syscall);
+
+ rv_detach_trace_probe("nomiss", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_detach_trace_probe("nomiss", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_detach_trace_probe("nomiss", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_detach_trace_probe("nomiss", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_detach_trace_probe("nomiss", sched_switch, handle_sched_switch);
+ rv_detach_trace_probe("nomiss", sched_wakeup, handle_sched_wakeup);
+
+ da_monitor_destroy();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "nomiss",
+ .description = "dl entities run to completion before their deadiline.",
+ .enable = enable_nomiss,
+ .disable = disable_nomiss,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_nomiss(void)
+{
+ return rv_register_monitor(&rv_this, &rv_deadline);
+}
+
+static void __exit unregister_nomiss(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_nomiss);
+module_exit(unregister_nomiss);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("nomiss: dl entities run to completion before their deadiline.");
diff --git a/kernel/trace/rv/monitors/nomiss/nomiss.h b/kernel/trace/rv/monitors/nomiss/nomiss.h
new file mode 100644
index 000000000000..c30c064aacdf
--- /dev/null
+++ b/kernel/trace/rv/monitors/nomiss/nomiss.h
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Automatically generated C representation of nomiss automaton
+ * For further information about this format, see kernel documentation:
+ * Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#define MONITOR_NAME nomiss
+
+enum states_nomiss {
+ ready_nomiss,
+ idle_nomiss,
+ running_nomiss,
+ sleeping_nomiss,
+ throttled_nomiss,
+ state_max_nomiss,
+};
+
+#define INVALID_STATE state_max_nomiss
+
+enum events_nomiss {
+ dl_replenish_nomiss,
+ dl_replenish_idle_nomiss,
+ dl_server_idle_nomiss,
+ dl_server_start_nomiss,
+ dl_server_stop_nomiss,
+ dl_throttle_nomiss,
+ sched_switch_in_nomiss,
+ sched_switch_suspend_nomiss,
+ sched_wakeup_nomiss,
+ event_max_nomiss,
+};
+
+enum envs_nomiss {
+ clk_nomiss,
+ is_constr_dl_nomiss,
+ is_defer_nomiss,
+ env_max_nomiss,
+ env_max_stored_nomiss = is_constr_dl_nomiss,
+};
+
+_Static_assert(env_max_stored_nomiss <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_nomiss {
+ char *state_names[state_max_nomiss];
+ char *event_names[event_max_nomiss];
+ char *env_names[env_max_nomiss];
+ unsigned char function[state_max_nomiss][event_max_nomiss];
+ unsigned char initial_state;
+ bool final_states[state_max_nomiss];
+};
+
+static const struct automaton_nomiss automaton_nomiss = {
+ .state_names = {
+ "ready",
+ "idle",
+ "running",
+ "sleeping",
+ "throttled",
+ },
+ .event_names = {
+ "dl_replenish",
+ "dl_replenish_idle",
+ "dl_server_idle",
+ "dl_server_start",
+ "dl_server_stop",
+ "dl_throttle",
+ "sched_switch_in",
+ "sched_switch_suspend",
+ "sched_wakeup",
+ },
+ .env_names = {
+ "clk",
+ "is_constr_dl",
+ "is_defer",
+ },
+ .function = {
+ {
+ ready_nomiss,
+ INVALID_STATE,
+ idle_nomiss,
+ ready_nomiss,
+ sleeping_nomiss,
+ throttled_nomiss,
+ running_nomiss,
+ INVALID_STATE,
+ ready_nomiss,
+ },
+ {
+ ready_nomiss,
+ idle_nomiss,
+ idle_nomiss,
+ idle_nomiss,
+ sleeping_nomiss,
+ throttled_nomiss,
+ running_nomiss,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ {
+ running_nomiss,
+ INVALID_STATE,
+ idle_nomiss,
+ INVALID_STATE,
+ sleeping_nomiss,
+ throttled_nomiss,
+ running_nomiss,
+ sleeping_nomiss,
+ running_nomiss,
+ },
+ {
+ ready_nomiss,
+ ready_nomiss,
+ idle_nomiss,
+ sleeping_nomiss,
+ INVALID_STATE,
+ throttled_nomiss,
+ running_nomiss,
+ INVALID_STATE,
+ ready_nomiss,
+ },
+ {
+ ready_nomiss,
+ INVALID_STATE,
+ idle_nomiss,
+ throttled_nomiss,
+ INVALID_STATE,
+ throttled_nomiss,
+ INVALID_STATE,
+ throttled_nomiss,
+ throttled_nomiss,
+ },
+ },
+ .initial_state = ready_nomiss,
+ .final_states = { 1, 0, 0, 0, 0 },
+};
diff --git a/kernel/trace/rv/monitors/nomiss/nomiss_trace.h b/kernel/trace/rv/monitors/nomiss/nomiss_trace.h
new file mode 100644
index 000000000000..42e7efaca4e7
--- /dev/null
+++ b/kernel/trace/rv/monitors/nomiss/nomiss_trace.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_NOMISS
+DEFINE_EVENT(event_da_monitor_id, event_nomiss,
+ TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_nomiss,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_nomiss,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+#endif /* CONFIG_RV_MON_NOMISS */
diff --git a/kernel/trace/rv/monitors/throttle/Kconfig b/kernel/trace/rv/monitors/throttle/Kconfig
new file mode 100644
index 000000000000..d9bd2dc903cd
--- /dev/null
+++ b/kernel/trace/rv/monitors/throttle/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_THROTTLE
+ depends on RV
+ depends on HAVE_SYSCALL_TRACEPOINTS
+ depends on RV_MON_DEADLINE
+ default y
+ select HA_MON_EVENTS_ID
+ bool "throttle monitor"
+ help
+ Monitor to ensure dl entities are throttled when they use up their runtime.
+ This monitor is part of the deadline monitors collection.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/throttle/throttle.c b/kernel/trace/rv/monitors/throttle/throttle.c
new file mode 100644
index 000000000000..eb3cf4365416
--- /dev/null
+++ b/kernel/trace/rv/monitors/throttle/throttle.c
@@ -0,0 +1,248 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+
+#define MODULE_NAME "throttle"
+
+#include <uapi/linux/sched/types.h>
+#include <trace/events/syscalls.h>
+#include <trace/events/sched.h>
+#include <trace/events/task.h>
+#include <rv_trace.h>
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define HA_TIMER_TYPE HA_TIMER_WHEEL
+/* The start condition is on sched_switch, it's dangerous to allocate there */
+#define DA_SKIP_AUTO_ALLOC
+typedef struct sched_dl_entity *monitor_target;
+#include "throttle.h"
+#include <rv/ha_monitor.h>
+#include <monitors/deadline/deadline.h>
+
+#define THROTTLED_TIME_NS TICK_NSEC
+/* with sched_feat(HRTICK_DL) the threshold can be lower */
+#define RUNTIME_THRESH TICK_NSEC
+
+static inline u64 runtime_left_ns(struct ha_monitor *ha_mon)
+{
+ return pi_of(ha_get_target(ha_mon))->runtime + RUNTIME_THRESH;
+}
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_throttle env, u64 time_ns)
+{
+ if (env == clk_throttle)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ else if (env == is_constr_dl_throttle)
+ return !dl_is_implicit(ha_get_target(ha_mon));
+ return ENV_INVALID_VALUE;
+}
+
+static void ha_reset_env(struct ha_monitor *ha_mon, enum envs_throttle env, u64 time_ns)
+{
+ if (env == clk_throttle)
+ ha_reset_clk_ns(ha_mon, env, time_ns);
+}
+
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == running_throttle)
+ return ha_check_invariant_ns(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == throttled_throttle)
+ return ha_check_invariant_ns(ha_mon, clk_throttle, time_ns);
+ return true;
+}
+
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ bool res = true;
+
+ if (curr_state == armed_throttle && event == sched_switch_in_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == preempted_throttle && event == dl_throttle_throttle)
+ res = ha_get_env(ha_mon, is_constr_dl_throttle, time_ns) == 1ull;
+ else if (curr_state == preempted_throttle && event == sched_switch_in_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == running_throttle && event == dl_replenish_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == running_throttle && event == dl_throttle_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == throttled_throttle && event == dl_replenish_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ return res;
+}
+
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == curr_state && event != dl_replenish_throttle)
+ return;
+ if (next_state == running_throttle)
+ ha_start_timer_ns(ha_mon, clk_throttle, runtime_left_ns(ha_mon), time_ns);
+ else if (next_state == throttled_throttle)
+ ha_start_timer_ns(ha_mon, clk_throttle, THROTTLED_TIME_NS, time_ns);
+ else if (curr_state == running_throttle)
+ ha_cancel_timer(ha_mon);
+ else if (curr_state == throttled_throttle)
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+static void handle_dl_replenish(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_replenish_throttle);
+}
+
+static void handle_dl_throttle(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_throttle_throttle);
+}
+
+static void handle_dl_server_stop(void *data, struct sched_dl_entity *dl_se, int cpu)
+{
+ da_handle_start_run_event(EXPAND_ID(dl_se, cpu), sched_switch_out_throttle);
+}
+
+static void handle_sched_switch(void *data, bool preempt,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ struct sched_dl_entity *dl_se;
+ int cpu = task_cpu(next);
+
+ if (prev->policy == SCHED_DEADLINE)
+ da_handle_event(EXPAND_ID(&prev->dl, cpu), sched_switch_out_throttle);
+ if (next->policy == SCHED_DEADLINE)
+ da_handle_start_event(EXPAND_ID(&next->dl, cpu), sched_switch_in_throttle);
+
+ /*
+ * The server is available in next only if the next task is boosted,
+ * otherwise we need to retrieve it.
+ * Here the server continues in the state running/armed until actually
+ * stopped, this works since we continue expecting a throttle.
+ */
+ dl_se = get_fair_server(next);
+ if (!dl_se)
+ return;
+ if (next->dl_server)
+ da_handle_start_event(EXPAND_ID(next->dl_server, cpu), sched_switch_in_throttle);
+ else if (is_idle_task(next) || next->policy == SCHED_NORMAL)
+ da_handle_event(EXPAND_ID(dl_se, cpu), dl_defer_arm_throttle);
+}
+
+static void handle_syscall(void *data, struct pt_regs *regs, long id)
+{
+ struct task_struct *p;
+ int new_policy = -1;
+
+ new_policy = extract_params(regs, id, &p);
+ if (new_policy < 0 || new_policy == p->policy)
+ return;
+ if (p->policy == SCHED_DEADLINE) {
+ da_reset(EXPAND_ID(&p->dl, DL_TASK));
+ /*
+ * When a task changes from SCHED_DEADLINE to SCHED_NORMAL, the
+ * runtime after the change is counted in the fair server.
+ */
+ if (new_policy == SCHED_NORMAL) {
+ struct sched_dl_entity *dl_se = get_fair_server(p);
+
+ if (!dl_se || !p->on_cpu)
+ return;
+ da_handle_event(EXPAND_ID(dl_se, task_cpu(p)), dl_defer_arm_throttle);
+ }
+ } else if (new_policy == SCHED_DEADLINE) {
+ da_create_or_get(EXPAND_ID(&p->dl, DL_TASK));
+ }
+}
+
+static int enable_throttle(void)
+{
+ int retval;
+
+ retval = da_monitor_init();
+ if (retval)
+ return retval;
+
+ retval = init_storage(false);
+ if (retval)
+ return retval;
+ rv_attach_trace_probe("throttle", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_attach_trace_probe("throttle", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_attach_trace_probe("throttle", sched_switch, handle_sched_switch);
+ if (!should_skip_syscall_handle())
+ rv_attach_trace_probe("throttle", sys_enter, handle_syscall);
+ rv_attach_trace_probe("throttle", task_newtask, handle_newtask);
+ rv_attach_trace_probe("throttle", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_attach_trace_probe("throttle", sched_process_exit, handle_exit);
+
+ return 0;
+}
+
+static void disable_throttle(void)
+{
+ rv_this.enabled = 0;
+
+ /* Those are RCU writers, detach earlier hoping to close a bit faster */
+ rv_detach_trace_probe("throttle", task_newtask, handle_newtask);
+ rv_detach_trace_probe("throttle", sched_process_exit, handle_exit);
+ if (!should_skip_syscall_handle())
+ rv_detach_trace_probe("throttle", sys_enter, handle_syscall);
+
+ rv_detach_trace_probe("throttle", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_detach_trace_probe("throttle", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_detach_trace_probe("throttle", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_detach_trace_probe("throttle", sched_switch, handle_sched_switch);
+
+ da_monitor_destroy();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "throttle",
+ .description = "throttle dl entities when they use up their runtime.",
+ .enable = enable_throttle,
+ .disable = disable_throttle,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_throttle(void)
+{
+ return rv_register_monitor(&rv_this, &rv_deadline);
+}
+
+static void __exit unregister_throttle(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_throttle);
+module_exit(unregister_throttle);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("throttle: throttle dl entities when they use up their runtime.");
diff --git a/kernel/trace/rv/monitors/throttle/throttle.h b/kernel/trace/rv/monitors/throttle/throttle.h
new file mode 100644
index 000000000000..3ab6d73280d2
--- /dev/null
+++ b/kernel/trace/rv/monitors/throttle/throttle.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Automatically generated C representation of throttle automaton
+ * For further information about this format, see kernel documentation:
+ * Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#define MONITOR_NAME throttle
+
+enum states_throttle {
+ running_throttle,
+ armed_throttle,
+ armed_throttled_throttle,
+ preempted_throttle,
+ preempted_throttled_throttle,
+ throttled_throttle,
+ state_max_throttle,
+};
+
+#define INVALID_STATE state_max_throttle
+
+enum events_throttle {
+ dl_defer_arm_throttle,
+ dl_replenish_throttle,
+ dl_throttle_throttle,
+ sched_switch_in_throttle,
+ sched_switch_out_throttle,
+ event_max_throttle,
+};
+
+enum envs_throttle {
+ clk_throttle,
+ is_constr_dl_throttle,
+ env_max_throttle,
+ env_max_stored_throttle = is_constr_dl_throttle,
+};
+
+_Static_assert(env_max_stored_throttle <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_throttle {
+ char *state_names[state_max_throttle];
+ char *event_names[event_max_throttle];
+ char *env_names[env_max_throttle];
+ unsigned char function[state_max_throttle][event_max_throttle];
+ unsigned char initial_state;
+ bool final_states[state_max_throttle];
+};
+
+static const struct automaton_throttle automaton_throttle = {
+ .state_names = {
+ "running",
+ "armed",
+ "armed_throttled",
+ "preempted",
+ "preempted_throttled",
+ "throttled",
+ },
+ .event_names = {
+ "dl_defer_arm",
+ "dl_replenish",
+ "dl_throttle",
+ "sched_switch_in",
+ "sched_switch_out",
+ },
+ .env_names = {
+ "clk",
+ "is_constr_dl",
+ },
+ .function = {
+ {
+ armed_throttle,
+ running_throttle,
+ throttled_throttle,
+ running_throttle,
+ preempted_throttle,
+ },
+ {
+ armed_throttle,
+ armed_throttle,
+ armed_throttled_throttle,
+ running_throttle,
+ preempted_throttle,
+ },
+ {
+ armed_throttled_throttle,
+ armed_throttle,
+ INVALID_STATE,
+ INVALID_STATE,
+ preempted_throttled_throttle,
+ },
+ {
+ armed_throttle,
+ preempted_throttle,
+ preempted_throttled_throttle,
+ running_throttle,
+ preempted_throttle,
+ },
+ {
+ armed_throttled_throttle,
+ preempted_throttle,
+ INVALID_STATE,
+ INVALID_STATE,
+ preempted_throttled_throttle,
+ },
+ {
+ armed_throttled_throttle,
+ running_throttle,
+ INVALID_STATE,
+ INVALID_STATE,
+ preempted_throttled_throttle,
+ },
+ },
+ .initial_state = running_throttle,
+ .final_states = { 1, 0, 0, 0, 0, 0 },
+};
diff --git a/kernel/trace/rv/monitors/throttle/throttle_trace.h b/kernel/trace/rv/monitors/throttle/throttle_trace.h
new file mode 100644
index 000000000000..7e376d3aec60
--- /dev/null
+++ b/kernel/trace/rv/monitors/throttle/throttle_trace.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_THROTTLE
+DEFINE_EVENT(event_da_monitor_id, event_throttle,
+ TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_throttle,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_throttle,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+#endif /* CONFIG_RV_MON_THROTTLE */
diff --git a/kernel/trace/rv/rv_trace.h b/kernel/trace/rv/rv_trace.h
index 9e8072d863a2..1bf0f3666ee4 100644
--- a/kernel/trace/rv/rv_trace.h
+++ b/kernel/trace/rv/rv_trace.h
@@ -188,6 +188,8 @@ DECLARE_EVENT_CLASS(error_env_da_monitor_id,
);
#include <monitors/stall/stall_trace.h>
+#include <monitors/nomiss/nomiss_trace.h>
+#include <monitors/throttle/throttle_trace.h>
// Add new monitors based on CONFIG_HA_MON_EVENTS_ID here
#endif
diff --git a/tools/verification/models/deadline/nomiss.dot b/tools/verification/models/deadline/nomiss.dot
new file mode 100644
index 000000000000..ff51ce8be8e5
--- /dev/null
+++ b/tools/verification/models/deadline/nomiss.dot
@@ -0,0 +1,43 @@
+digraph state_automaton {
+ center = true;
+ size = "7,11";
+ {node [shape = circle] "idle"};
+ {node [shape = plaintext, style=invis, label=""] "__init_ready"};
+ {node [shape = doublecircle] "ready"};
+ {node [shape = circle] "ready"};
+ {node [shape = circle] "running"};
+ {node [shape = circle] "sleeping"};
+ {node [shape = circle] "throttled"};
+ "__init_ready" -> "ready";
+ "idle" [label = "idle"];
+ "idle" -> "idle" [ label = "dl_server_idle\ndl_server_start\ndl_replenish_idle;reset(clk)" ];
+ "idle" -> "ready" [ label = "dl_replenish;reset(clk)" ];
+ "idle" -> "running" [ label = "sched_switch_in" ];
+ "idle" -> "sleeping" [ label = "dl_server_stop" ];
+ "idle" -> "throttled" [ label = "dl_throttle" ];
+ "ready" [label = "ready\nclk < DEADLINE_NS()", color = green3];
+ "ready" -> "idle" [ label = "dl_server_idle" ];
+ "ready" -> "ready" [ label = "sched_wakeup\ndl_server_start\ndl_replenish;reset(clk)" ];
+ "ready" -> "running" [ label = "sched_switch_in" ];
+ "ready" -> "sleeping" [ label = "dl_server_stop" ];
+ "ready" -> "throttled" [ label = "dl_throttle;is_defer == 1" ];
+ "running" [label = "running\nclk < DEADLINE_NS()"];
+ "running" -> "idle" [ label = "dl_server_idle" ];
+ "running" -> "running" [ label = "dl_replenish;reset(clk)\nsched_switch_in\nsched_wakeup" ];
+ "running" -> "sleeping" [ label = "sched_switch_suspend\ndl_server_stop" ];
+ "running" -> "throttled" [ label = "dl_throttle" ];
+ "sleeping" [label = "sleeping"];
+ "sleeping" -> "idle" [ label = "dl_server_idle" ];
+ "sleeping" -> "ready" [ label = "sched_wakeup\ndl_replenish_idle;reset(clk)\ndl_replenish;reset(clk)" ];
+ "sleeping" -> "running" [ label = "sched_switch_in" ];
+ "sleeping" -> "sleeping" [ label = "dl_server_start" ];
+ "sleeping" -> "throttled" [ label = "dl_throttle;is_constr_dl == 1" ];
+ "throttled" [label = "throttled"];
+ "throttled" -> "idle" [ label = "dl_server_idle" ];
+ "throttled" -> "ready" [ label = "dl_replenish;reset(clk)" ];
+ "throttled" -> "throttled" [ label = "sched_switch_suspend\nsched_wakeup\ndl_server_start\ndl_throttle" ];
+ { rank = min ;
+ "__init_ready";
+ "ready";
+ }
+}
diff --git a/tools/verification/models/deadline/throttle.dot b/tools/verification/models/deadline/throttle.dot
new file mode 100644
index 000000000000..c24fc3f291a9
--- /dev/null
+++ b/tools/verification/models/deadline/throttle.dot
@@ -0,0 +1,44 @@
+digraph state_automaton {
+ center = true;
+ size = "7,11";
+ {node [shape = circle] "armed"};
+ {node [shape = circle] "armed_throttled"};
+ {node [shape = circle] "preempted"};
+ {node [shape = circle] "preempted_throttled"};
+ {node [shape = plaintext, style=invis, label=""] "__init_running"};
+ {node [shape = doublecircle] "running"};
+ {node [shape = circle] "running"};
+ {node [shape = circle] "throttled"};
+ "__init_running" -> "running";
+ "armed" [label = "armed"];
+ "armed" -> "armed" [ label = "dl_replenish\ndl_defer_arm" ];
+ "armed" -> "armed_throttled" [ label = "dl_throttle" ];
+ "armed" -> "preempted" [ label = "sched_switch_out" ];
+ "armed" -> "running" [ label = "sched_switch_in;reset(clk)" ];
+ "armed_throttled" [label = "armed_throttled"];
+ "armed_throttled" -> "armed" [ label = "dl_replenish" ];
+ "armed_throttled" -> "armed_throttled" [ label = "dl_defer_arm" ];
+ "armed_throttled" -> "preempted_throttled" [ label = "sched_switch_out" ];
+ "preempted" [label = "preempted"];
+ "preempted" -> "armed" [ label = "dl_defer_arm" ];
+ "preempted" -> "preempted" [ label = "dl_replenish\nsched_switch_out" ];
+ "preempted" -> "preempted_throttled" [ label = "dl_throttle;is_constr_dl == 1" ];
+ "preempted" -> "running" [ label = "sched_switch_in;reset(clk)" ];
+ "preempted_throttled" [label = "preempted_throttled"];
+ "preempted_throttled" -> "armed_throttled" [ label = "dl_defer_arm" ];
+ "preempted_throttled" -> "preempted" [ label = "dl_replenish" ];
+ "preempted_throttled" -> "preempted_throttled" [ label = "sched_switch_out" ];
+ "running" [label = "running\nclk < runtime_left_ns()", color = green3];
+ "running" -> "armed" [ label = "dl_defer_arm" ];
+ "running" -> "preempted" [ label = "sched_switch_out" ];
+ "running" -> "running" [ label = "dl_replenish;reset(clk)\nsched_switch_in" ];
+ "running" -> "throttled" [ label = "dl_throttle;reset(clk)" ];
+ "throttled" [label = "throttled\nclk < THROTTLED_TIME_NS"];
+ "throttled" -> "armed_throttled" [ label = "dl_defer_arm" ];
+ "throttled" -> "preempted_throttled" [ label = "sched_switch_out" ];
+ "throttled" -> "running" [ label = "dl_replenish;reset(clk)" ];
+ { rank = min ;
+ "__init_running";
+ "running";
+ }
+}
--
2.52.0
^ permalink raw reply related
* [PATCH v3 11/13] verification/rvgen: Add support for per-obj monitors
From: Gabriele Monaco @ 2025-12-05 13:16 UTC (permalink / raw)
To: linux-kernel, Steven Rostedt, Nam Cao, Gabriele Monaco,
linux-trace-kernel
Cc: Tomas Glozar, Juri Lelli, Clark Williams, John Kacur
In-Reply-To: <20251205131621.135513-1-gmonaco@redhat.com>
The special per-object monitor type was just introduced in RV, this
requires the user to define some functions and type specific to the
object.
Adapt rvgen to add stub definitions for the monitor_target type and
other modifications required to create per-object monitors.
Reviewed-by: Nam Cao <namcao@linutronix.de>
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
---
Notes:
V3:
* Add _is_id_monitor() in dot2k to handle per-obj together with per-task
tools/verification/rvgen/rvgen/dot2k.py | 17 +++++++++++++----
tools/verification/rvgen/rvgen/generator.py | 2 +-
2 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/tools/verification/rvgen/rvgen/dot2k.py b/tools/verification/rvgen/rvgen/dot2k.py
index e72369ec1ad7..42111b477847 100644
--- a/tools/verification/rvgen/rvgen/dot2k.py
+++ b/tools/verification/rvgen/rvgen/dot2k.py
@@ -27,6 +27,8 @@ class dot2k(Monitor, Dot2c):
def fill_monitor_type(self) -> str:
buff = [ self.monitor_type.upper() ]
buff += self._fill_timer_type()
+ if self.monitor_type == "per_obj":
+ buff.append("typedef /* XXX: define the target type */ *monitor_target;")
return "\n".join(buff)
def fill_tracepoint_handlers_skel(self) -> str:
@@ -45,6 +47,10 @@ class dot2k(Monitor, Dot2c):
if self.monitor_type == "per_task":
buff.append("\tstruct task_struct *p = /* XXX: how do I get p? */;");
buff.append("\tda_%s(p, %s%s);" % (handle, event, self.enum_suffix));
+ elif self.monitor_type == "per_obj":
+ buff.append("\tint id = /* XXX: how do I get the id? */;");
+ buff.append("\tmonitor_target t = /* XXX: how do I get t? */;");
+ buff.append("\tda_%s(id, t, %s%s);" % (handle, event, self.enum_suffix));
else:
buff.append("\tda_%s(%s%s);" % (handle, event, self.enum_suffix));
buff.append("}")
@@ -92,13 +98,16 @@ class dot2k(Monitor, Dot2c):
return '\n'.join(buff)
+ def _is_id_monitor(self) -> bool:
+ return self.monitor_type in ("per_task", "per_obj")
+
def fill_monitor_class_type(self) -> str:
- if self.monitor_type == "per_task":
+ if self._is_id_monitor():
return "DA_MON_EVENTS_ID"
return "DA_MON_EVENTS_IMPLICIT"
def fill_monitor_class(self) -> str:
- if self.monitor_type == "per_task":
+ if self._is_id_monitor():
return "da_monitor_id"
return "da_monitor"
@@ -122,7 +131,7 @@ class dot2k(Monitor, Dot2c):
}
tp_args_id = ("int ", "id")
tp_args = tp_args_dict[tp_type]
- if self.monitor_type == "per_task":
+ if self._is_id_monitor():
tp_args.insert(0, tp_args_id)
tp_proto_c = ", ".join([a+b for a,b in tp_args])
tp_args_c = ", ".join([b for a,b in tp_args])
@@ -169,7 +178,7 @@ class ha2k(dot2k):
self.__parse_constraints()
def fill_monitor_class_type(self) -> str:
- if self.monitor_type == "per_task":
+ if self._is_id_monitor():
return "HA_MON_EVENTS_ID"
return "HA_MON_EVENTS_IMPLICIT"
diff --git a/tools/verification/rvgen/rvgen/generator.py b/tools/verification/rvgen/rvgen/generator.py
index b80af3fd6701..5eac12e110dc 100644
--- a/tools/verification/rvgen/rvgen/generator.py
+++ b/tools/verification/rvgen/rvgen/generator.py
@@ -243,7 +243,7 @@ obj-$(CONFIG_RV_MON_%s) += monitors/%s/%s.o
class Monitor(RVGenerator):
- monitor_types = { "global" : 1, "per_cpu" : 2, "per_task" : 3 }
+ monitor_types = { "global" : 1, "per_cpu" : 2, "per_task" : 3, "per_obj" : 4 }
def __init__(self, extra_params={}):
super().__init__(extra_params)
--
2.52.0
^ permalink raw reply related
* [PATCH v3 10/13] rv: Add support for per-object monitors in DA/HA
From: Gabriele Monaco @ 2025-12-05 13:16 UTC (permalink / raw)
To: linux-kernel, Steven Rostedt, Nam Cao, Gabriele Monaco,
linux-trace-kernel
Cc: Tomas Glozar, Juri Lelli, Clark Williams, John Kacur
In-Reply-To: <20251205131621.135513-1-gmonaco@redhat.com>
RV deterministic and hybrid automata currently only support global,
per-cpu and per-task monitors. It isn't possible to write a model that
would follow some different type of object, like a deadline entity or a
lock.
Define the generic per-object monitor implementation which shares part
of the implementation with the per-task monitors.
The user needs to provide an id for the object (e.g. pid for tasks) and
define the data type for the monitor_target (e.g. struct task_struct *
for tasks). Both are supplied to the event handlers, as the id may not
be easily available in the target.
The monitor storage (e.g. the rv monitor, pointer to the target, etc.)
is stored in a hash table indexed by id. Monitor storage objects are
automatically allocated unless specified otherwise (e.g. if the creation
context is unsafe for allocation).
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
---
Notes:
V3:
* Use kmalloc_nolock for per-obj storage allocation
* Improve function name consistency
include/linux/rv.h | 1 +
include/rv/da_monitor.h | 300 +++++++++++++++++++++++++++++++++++++++-
include/rv/ha_monitor.h | 5 +-
3 files changed, 300 insertions(+), 6 deletions(-)
diff --git a/include/linux/rv.h b/include/linux/rv.h
index 0aef9e3c785c..541ba404926a 100644
--- a/include/linux/rv.h
+++ b/include/linux/rv.h
@@ -13,6 +13,7 @@
#define RV_MON_GLOBAL 0
#define RV_MON_PER_CPU 1
#define RV_MON_PER_TASK 2
+#define RV_MON_PER_OBJ 3
#ifdef CONFIG_RV
#include <linux/array_size.h>
diff --git a/include/rv/da_monitor.h b/include/rv/da_monitor.h
index 4d0f9fb60e3c..6830b5d6f34a 100644
--- a/include/rv/da_monitor.h
+++ b/include/rv/da_monitor.h
@@ -19,6 +19,8 @@
#include <linux/stringify.h>
#include <linux/bug.h>
#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/hashtable.h>
static struct rv_monitor rv_this;
@@ -51,6 +53,9 @@ static struct rv_monitor rv_this;
/*
* Type for the target id, default to int but can be overridden.
+ * A long type can work as hash table key (PER_OBJ) but will be downgraded to
+ * int in the event tracepoint.
+ * Unused for implicit monitors.
*/
#ifndef da_id_type
#define da_id_type int
@@ -239,9 +244,9 @@ static inline struct da_monitor *da_get_monitor(struct task_struct *tsk)
}
/*
- * da_get_task - return the task associated to the monitor
+ * da_get_target - return the task associated to the monitor
*/
-static inline struct task_struct *da_get_task(struct da_monitor *da_mon)
+static inline struct task_struct *da_get_target(struct da_monitor *da_mon)
{
return container_of(da_mon, struct task_struct, rv[task_mon_slot].da_mon);
}
@@ -253,7 +258,7 @@ static inline struct task_struct *da_get_task(struct da_monitor *da_mon)
*/
static inline da_id_type da_get_id(struct da_monitor *da_mon)
{
- return da_get_task(da_mon)->pid;
+ return da_get_target(da_mon)->pid;
}
static void da_monitor_reset_all(void)
@@ -303,6 +308,221 @@ static inline void da_monitor_destroy(void)
da_monitor_reset_all();
}
+
+#elif RV_MON_TYPE == RV_MON_PER_OBJ
+/*
+ * Functions to define, init and get a per-object monitor.
+ */
+
+struct da_monitor_storage {
+ da_id_type id;
+ monitor_target target;
+ union rv_task_monitor rv;
+ struct hlist_node node;
+ struct rcu_head rcu;
+};
+
+#ifndef DA_MONITOR_HT_BITS
+#define DA_MONITOR_HT_BITS 10
+#endif
+static DEFINE_HASHTABLE(da_monitor_ht, DA_MONITOR_HT_BITS);
+
+/*
+ * da_create_empty_storage - pre-allocate an empty storage
+ */
+static inline struct da_monitor_storage *da_create_empty_storage(da_id_type id)
+{
+ struct da_monitor_storage *mon_storage;
+
+ mon_storage = kmalloc_nolock(sizeof(struct da_monitor_storage),
+ __GFP_ZERO, NUMA_NO_NODE);
+ if (!mon_storage)
+ return NULL;
+
+ hash_add_rcu(da_monitor_ht, &mon_storage->node, id);
+ mon_storage->id = id;
+ return mon_storage;
+}
+
+/*
+ * da_create_storage - create the per-object storage
+ *
+ * The caller is responsible to synchronise writers, either with locks or
+ * implicitly. For instance, if da_create_storage is only called from a single
+ * event for target (e.g. sched_switch), it's safe to call this without locks.
+ */
+static inline struct da_monitor *da_create_storage(da_id_type id,
+ monitor_target target,
+ struct da_monitor *da_mon)
+{
+ struct da_monitor_storage *mon_storage;
+
+ if (da_mon)
+ return da_mon;
+
+ mon_storage = da_create_empty_storage(id);
+ if (!mon_storage)
+ return NULL;
+
+ mon_storage->target = target;
+ return &mon_storage->rv.da_mon;
+}
+
+/*
+ * __da_get_mon_storage - get the monitor storage from the hash table
+ */
+static inline struct da_monitor_storage *__da_get_mon_storage(da_id_type id)
+{
+ struct da_monitor_storage *mon_storage;
+
+ lockdep_assert_in_rcu_read_lock();
+ hash_for_each_possible_rcu(da_monitor_ht, mon_storage, node, id) {
+ if (mon_storage->id == id)
+ return mon_storage;
+ }
+
+ return NULL;
+}
+
+/*
+ * da_get_monitor - return the monitor for target
+ */
+static struct da_monitor *da_get_monitor(da_id_type id, monitor_target target)
+{
+ struct da_monitor_storage *mon_storage;
+
+ mon_storage = __da_get_mon_storage(id);
+ return mon_storage ? &mon_storage->rv.da_mon : NULL;
+}
+
+/*
+ * da_get_target - return the object associated to the monitor
+ */
+static inline monitor_target da_get_target(struct da_monitor *da_mon)
+{
+ return container_of(da_mon, struct da_monitor_storage, rv.da_mon)->target;
+}
+
+/*
+ * da_get_id - return the id associated to the monitor
+ */
+static inline da_id_type da_get_id(struct da_monitor *da_mon)
+{
+ return container_of(da_mon, struct da_monitor_storage, rv.da_mon)->id;
+}
+
+/*
+ * da_create_or_get - create the per-object storage if not already there
+ *
+ * This needs a lookup so should be guarded by RCU, the condition is checked
+ * directly in da_create_storage()
+ */
+static inline void da_create_or_get(da_id_type id, monitor_target target)
+{
+ guard(rcu)();
+ da_create_storage(id, target, da_get_monitor(id, target));
+}
+
+/*
+ * da_fill_empty_storage - store the target in a pre-allocated storage
+ *
+ * Can be used as a substitute of da_create_storage when starting a monitor in
+ * an environment where allocation is unsafe.
+ */
+static inline struct da_monitor *da_fill_empty_storage(da_id_type id,
+ monitor_target target,
+ struct da_monitor *da_mon)
+{
+ if (unlikely(da_mon && !da_get_target(da_mon)))
+ container_of(da_mon, struct da_monitor_storage, rv.da_mon)->target = target;
+ return da_mon;
+}
+
+/*
+ * da_get_target_by_id - return the object associated to the id
+ */
+static inline monitor_target da_get_target_by_id(da_id_type id)
+{
+ struct da_monitor_storage *mon_storage;
+
+ guard(rcu)();
+ mon_storage = __da_get_mon_storage(id);
+
+ if (unlikely(!mon_storage))
+ return NULL;
+ return mon_storage->target;
+}
+
+/*
+ * da_destroy_storage - destroy the per-object storage
+ *
+ * The caller is responsible to synchronise writers, either with locks or
+ * implicitly. For instance, if da_destroy_storage is called at sched_exit and
+ * da_create_storage can never occur after that, it's safe to call this without
+ * locks.
+ * This function includes an RCU read-side critical section to synchronise
+ * against da_monitor_destroy().
+ */
+static inline void da_destroy_storage(da_id_type id)
+{
+ struct da_monitor_storage *mon_storage;
+
+ guard(rcu)();
+ mon_storage = __da_get_mon_storage(id);
+
+ if (!mon_storage)
+ return;
+ da_monitor_reset_hook(&mon_storage->rv.da_mon);
+ hash_del_rcu(&mon_storage->node);
+ kfree_rcu(mon_storage, rcu);
+}
+
+static void da_monitor_reset_all(void)
+{
+ struct da_monitor_storage *mon_storage;
+ int bkt;
+
+ rcu_read_lock();
+ hash_for_each_rcu(da_monitor_ht, bkt, mon_storage, node)
+ da_monitor_reset(&mon_storage->rv.da_mon);
+ rcu_read_unlock();
+}
+
+static inline int da_monitor_init(void)
+{
+ hash_init(da_monitor_ht);
+ return 0;
+}
+
+static inline void da_monitor_destroy(void)
+{
+ struct da_monitor_storage *mon_storage;
+ struct hlist_node *tmp;
+ int bkt;
+
+ /*
+ * This function is called after all probes are disabled, we need only
+ * worry about concurrency against old events.
+ */
+ synchronize_rcu();
+ hash_for_each_safe(da_monitor_ht, bkt, tmp, mon_storage, node) {
+ da_monitor_reset_hook(&mon_storage->rv.da_mon);
+ hash_del_rcu(&mon_storage->node);
+ kfree(mon_storage);
+ }
+}
+
+/*
+ * Allow the per-object monitors to run allocation manually, necessary if the
+ * start condition is in a context problematic for allocation (e.g. scheduling).
+ * In such case, if the storage was pre-allocated without a target, set it now.
+ */
+#ifdef DA_SKIP_AUTO_ALLOC
+#define da_prepare_storage da_fill_empty_storage
+#else
+#define da_prepare_storage da_create_storage
+#endif /* DA_SKIP_AUTO_ALLOC */
+
#endif /* RV_MON_TYPE */
#if RV_MON_TYPE == RV_MON_GLOBAL || RV_MON_TYPE == RV_MON_PER_CPU
@@ -336,9 +556,9 @@ static inline da_id_type da_get_id(struct da_monitor *da_mon)
return 0;
}
-#elif RV_MON_TYPE == RV_MON_PER_TASK
+#elif RV_MON_TYPE == RV_MON_PER_TASK || RV_MON_TYPE == RV_MON_PER_OBJ
/*
- * Trace events for per_task monitors, report the PID of the task.
+ * Trace events for per_task/per_object monitors, report the target id.
*/
static inline void da_trace_event(struct da_monitor *da_mon,
@@ -519,6 +739,76 @@ static inline bool da_handle_start_run_event(struct task_struct *tsk,
{
return __da_handle_start_run_event(da_get_monitor(tsk), event, tsk->pid);
}
+
+#elif RV_MON_TYPE == RV_MON_PER_OBJ
+/*
+ * Handle event for per object.
+ */
+
+/*
+ * da_handle_event - handle an event
+ */
+static inline void da_handle_event(da_id_type id, monitor_target target, enum events event)
+{
+ struct da_monitor *da_mon;
+
+ guard(rcu)();
+ da_mon = da_get_monitor(id, target);
+ if (likely(da_mon))
+ __da_handle_event(da_mon, event, id);
+}
+
+/*
+ * da_handle_start_event - start monitoring or handle event
+ *
+ * This function is used to notify the monitor that the system is returning
+ * to the initial state, so the monitor can start monitoring in the next event.
+ * Thus:
+ *
+ * If the monitor already started, handle the event.
+ * If the monitor did not start yet, start the monitor but skip the event.
+ */
+static inline bool da_handle_start_event(da_id_type id, monitor_target target,
+ enum events event)
+{
+ struct da_monitor *da_mon;
+
+ guard(rcu)();
+ da_mon = da_get_monitor(id, target);
+ da_mon = da_prepare_storage(id, target, da_mon);
+ if (unlikely(!da_mon))
+ return 0;
+ return __da_handle_start_event(da_mon, event, id);
+}
+
+/*
+ * da_handle_start_run_event - start monitoring and handle event
+ *
+ * This function is used to notify the monitor that the system is in the
+ * initial state, so the monitor can start monitoring and handling event.
+ */
+static inline bool da_handle_start_run_event(da_id_type id, monitor_target target,
+ enum events event)
+{
+ struct da_monitor *da_mon;
+
+ guard(rcu)();
+ da_mon = da_get_monitor(id, target);
+ da_mon = da_prepare_storage(id, target, da_mon);
+ if (unlikely(!da_mon))
+ return 0;
+ return __da_handle_start_run_event(da_mon, event, id);
+}
+
+static inline void da_reset(da_id_type id, monitor_target target)
+{
+ struct da_monitor *da_mon;
+
+ guard(rcu)();
+ da_mon = da_get_monitor(id, target);
+ if (likely(da_mon))
+ da_monitor_reset(da_mon);
+}
#endif /* RV_MON_TYPE */
#endif
diff --git a/include/rv/ha_monitor.h b/include/rv/ha_monitor.h
index c8a434cb46ac..047059e54ae8 100644
--- a/include/rv/ha_monitor.h
+++ b/include/rv/ha_monitor.h
@@ -191,7 +191,10 @@ static inline void ha_trace_error_env(struct ha_monitor *ha_mon,
{
CONCATENATE(trace_error_env_, MONITOR_NAME)(curr_state, event, env);
}
-#elif RV_MON_TYPE == RV_MON_PER_TASK
+#elif RV_MON_TYPE == RV_MON_PER_TASK || RV_MON_TYPE == RV_MON_PER_OBJ
+
+#define ha_get_target(ha_mon) da_get_target(&ha_mon->da_mon)
+
static inline void ha_trace_error_env(struct ha_monitor *ha_mon,
char *curr_state, char *event, char *env,
da_id_type id)
--
2.52.0
^ permalink raw reply related
* [PATCH v3 09/13] sched: Add deadline tracepoints
From: Gabriele Monaco @ 2025-12-05 13:16 UTC (permalink / raw)
To: linux-kernel, Steven Rostedt, Nam Cao, Masami Hiramatsu,
Ingo Molnar, Peter Zijlstra, linux-trace-kernel
Cc: Gabriele Monaco, Tomas Glozar, Juri Lelli, Clark Williams,
John Kacur
In-Reply-To: <20251205131621.135513-1-gmonaco@redhat.com>
Add the following tracepoints:
* sched_dl_throttle(dl_se, cpu):
Called when a deadline entity is throttled
* sched_dl_replenish(dl_se, cpu):
Called when a deadline entity's runtime is replenished
* sched_dl_server_start(dl_se, cpu):
Called when a deadline server is started
* sched_dl_server_stop(dl_se, cpu):
Called when a deadline server is stopped
Those tracepoints can be useful to validate the deadline scheduler with
RV and are not exported to tracefs.
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
---
Notes:
V3:
* Rename dl argument to dl_se in tracepoints
include/trace/events/sched.h | 16 ++++++++++++++++
kernel/sched/core.c | 4 ++++
kernel/sched/deadline.c | 7 +++++++
3 files changed, 27 insertions(+)
diff --git a/include/trace/events/sched.h b/include/trace/events/sched.h
index 7b2645b50e78..366b2e8ec40c 100644
--- a/include/trace/events/sched.h
+++ b/include/trace/events/sched.h
@@ -896,6 +896,22 @@ DECLARE_TRACE(sched_set_need_resched,
TP_PROTO(struct task_struct *tsk, int cpu, int tif),
TP_ARGS(tsk, cpu, tif));
+DECLARE_TRACE(sched_dl_throttle,
+ TP_PROTO(struct sched_dl_entity *dl_se, int cpu),
+ TP_ARGS(dl_se, cpu));
+
+DECLARE_TRACE(sched_dl_replenish,
+ TP_PROTO(struct sched_dl_entity *dl_se, int cpu),
+ TP_ARGS(dl_se, cpu));
+
+DECLARE_TRACE(sched_dl_server_start,
+ TP_PROTO(struct sched_dl_entity *dl_se, int cpu),
+ TP_ARGS(dl_se, cpu));
+
+DECLARE_TRACE(sched_dl_server_stop,
+ TP_PROTO(struct sched_dl_entity *dl_se, int cpu),
+ TP_ARGS(dl_se, cpu));
+
#endif /* _TRACE_SCHED_H */
/* This part must be outside protection */
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5516778c19eb..3aed0ed6f2e9 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -122,6 +122,10 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(sched_compute_energy_tp);
EXPORT_TRACEPOINT_SYMBOL_GPL(sched_entry_tp);
EXPORT_TRACEPOINT_SYMBOL_GPL(sched_exit_tp);
EXPORT_TRACEPOINT_SYMBOL_GPL(sched_set_need_resched_tp);
+EXPORT_TRACEPOINT_SYMBOL_GPL(sched_dl_throttle_tp);
+EXPORT_TRACEPOINT_SYMBOL_GPL(sched_dl_replenish_tp);
+EXPORT_TRACEPOINT_SYMBOL_GPL(sched_dl_server_start_tp);
+EXPORT_TRACEPOINT_SYMBOL_GPL(sched_dl_server_stop_tp);
DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
DEFINE_PER_CPU(struct rnd_state, sched_rnd_state);
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 319439fe1870..2192e9f42e02 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -733,6 +733,7 @@ static inline void replenish_dl_new_period(struct sched_dl_entity *dl_se,
dl_se->dl_throttled = 1;
dl_se->dl_defer_armed = 1;
}
+ trace_sched_dl_replenish_tp(dl_se, cpu_of(rq));
}
/*
@@ -850,6 +851,8 @@ static void replenish_dl_entity(struct sched_dl_entity *dl_se)
if (dl_se->dl_throttled)
dl_se->dl_throttled = 0;
+ trace_sched_dl_replenish_tp(dl_se, cpu_of(rq));
+
/*
* If this is the replenishment of a deferred reservation,
* clear the flag and return.
@@ -1341,6 +1344,7 @@ static inline void dl_check_constrained_dl(struct sched_dl_entity *dl_se)
dl_time_before(rq_clock(rq), dl_next_period(dl_se))) {
if (unlikely(is_dl_boosted(dl_se) || !start_dl_timer(dl_se)))
return;
+ trace_sched_dl_throttle_tp(dl_se, cpu_of(rq));
dl_se->dl_throttled = 1;
if (dl_se->runtime > 0)
dl_se->runtime = 0;
@@ -1504,6 +1508,7 @@ static void update_curr_dl_se(struct rq *rq, struct sched_dl_entity *dl_se, s64
throttle:
if (dl_runtime_exceeded(dl_se) || dl_se->dl_yielded) {
+ trace_sched_dl_throttle_tp(dl_se, cpu_of(rq));
dl_se->dl_throttled = 1;
/* If requested, inform the user about runtime overruns. */
@@ -1795,6 +1800,7 @@ void dl_server_start(struct sched_dl_entity *dl_se)
if (WARN_ON_ONCE(!cpu_online(cpu_of(rq))))
return;
+ trace_sched_dl_server_start_tp(dl_se, cpu_of(rq));
dl_se->dl_server_active = 1;
enqueue_dl_entity(dl_se, ENQUEUE_WAKEUP);
if (!dl_task(dl_se->rq->curr) || dl_entity_preempt(dl_se, &rq->curr->dl))
@@ -1806,6 +1812,7 @@ void dl_server_stop(struct sched_dl_entity *dl_se)
if (!dl_server(dl_se) || !dl_server_active(dl_se))
return;
+ trace_sched_dl_server_stop_tp(dl_se, cpu_of(dl_se->rq));
dequeue_dl_entity(dl_se, DEQUEUE_SLEEP);
hrtimer_try_to_cancel(&dl_se->dl_timer);
dl_se->dl_defer_armed = 0;
--
2.52.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox