* Re: [PATCH v6 14/16] sched_ext: Export task_is_scx_enabled() for verification
From: Tejun Heo @ 2026-02-25 17:08 UTC (permalink / raw)
To: Gabriele Monaco
Cc: linux-kernel, Andrea Righi, Steven Rostedt, Nam Cao, Juri Lelli,
Ingo Molnar, Peter Zijlstra, sched-ext, Tomas Glozar,
Clark Williams, John Kacur, linux-trace-kernel
In-Reply-To: <20260225095122.80683-15-gmonaco@redhat.com>
On Wed, Feb 25, 2026 at 10:51:20AM +0100, Gabriele Monaco wrote:
> When a sched_ext scheduler is loaded, depending on the configuration, it
> can handle also fair tasks, however it isn't trivial to understand if a
> certain task is currently handled by fair or ext outside of scheduler
> code. This can be a problem when writing verification or observability
> tools like RV monitors.
>
> Export a task_is_scx_enabled() to allow quick classification by using
> the scx state SCX_TASK_ENABLED.
This test already exists - task_on_scx(). Please feel free to move that out
to include/linux/sched/ext.h.
Thanks.
--
tejun
^ permalink raw reply
* Re: [External] Re: [RFC PATCH bpf-next 2/3] ftrace: Use kallsyms binary search for single-symbol lookup
From: Andrey Grodzovsky @ 2026-02-25 15:25 UTC (permalink / raw)
To: Steven Rostedt
Cc: bpf, ast, daniel, andrii, jolsa, linux-trace-kernel,
linux-open-source
In-Reply-To: <20260225064723.53bcce3e@fedora>
On Wed, Feb 25, 2026 at 6:47 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Mon, 23 Feb 2026 16:51:12 -0500
> Andrey Grodzovsky <andrey.grodzovsky@crowdstrike.com> wrote:
>
> > When ftrace_lookup_symbols() is called with a single symbol (cnt == 1),
> > use kallsyms_lookup_name() for O(log N) binary search instead of the
> > full linear scan via kallsyms_on_each_symbol().
>
> So this patch looks like it should go through the tracing tree, not bpf.
Hey Steve, are there any extra steps required on my side to make this go
through your tree?
Andrey
>
> >
> > ftrace_lookup_symbols() was designed for batch resolution of many
> > symbols in a single pass. For large cnt this is efficient: a single
> > O(N) walk over all symbols with O(log cnt) binary search into the
> > sorted input array. But for cnt == 1 it still decompresses all ~200K
> > kernel symbols only to match one.
> >
> > kallsyms_lookup_name() uses the sorted kallsyms index and needs only
> > ~17 decompressions for a single lookup.
> >
> > This is the common path for kprobe.session with exact function names,
> > where libbpf sends one symbol per BPF_LINK_CREATE syscall.
> >
> > If binary lookup fails (duplicate symbol names where the first match
> > is not ftrace-instrumented, or module symbols), the function falls
> > through to the existing linear scan path.
> >
> > Before (cnt=1, 50 kprobe.session programs):
> > Attach: 858 ms (kallsyms_expand_symbol 25% of CPU)
> >
> > After:
> > Attach: 52 ms (16x faster)
> >
> > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@crowdstrike.com>
> > ---
> > kernel/trace/ftrace.c | 28 ++++++++++++++++++++++++++++
> > 1 file changed, 28 insertions(+)
> >
> > diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> > index 827fb9a0bf0d..bfd7670669c2 100644
> > --- a/kernel/trace/ftrace.c
> > +++ b/kernel/trace/ftrace.c
> > @@ -9263,6 +9263,19 @@ static int kallsyms_callback(void *data, const char *name, unsigned long addr)
> > * @addrs array, which needs to be big enough to store at least @cnt
> > * addresses.
> > *
> > + * For a single symbol (cnt == 1), uses kallsyms_lookup_name() which
> > + * performs an O(log N) binary search via the sorted kallsyms index.
> > + * This avoids the full O(N) linear scan over all kernel symbols that
> > + * the multi-symbol path requires.
> > + *
> > + * For multiple symbols, uses a single-pass linear scan via
> > + * kallsyms_on_each_symbol() with binary search into the sorted input
> > + * array.
>
> The above is fine.
>
> > While individual lookups are O(log N), doing K lookups
> > + * totals O(K * log N) which loses to a single sequential O(N) pass
> > + * at scale due to cache-friendly memory access patterns of the linear
> > + * walk. Empirical testing shows the linear scan is faster for batch
> > + * lookups even well below 10K symbols.
>
> The above is unneeded for a comment in the code and just belongs in the
> change log.
>
> -- Steve
>
> > + *
> > * Returns: 0 if all provided symbols are found, -ESRCH otherwise.
> > */
> > int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *addrs)
> > @@ -9270,6 +9283,21 @@ int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *a
> > struct kallsyms_data args;
> > int found_all;
> >
> > + /* Fast path: single symbol uses O(log N) binary search */
> > + if (cnt == 1) {
> > + addrs[0] = kallsyms_lookup_name(sorted_syms[0]);
> > + if (addrs[0])
> > + addrs[0] = ftrace_location(addrs[0]);
> > + if (addrs[0])
> > + return 0;
> > + /*
> > + * Binary lookup can fail for duplicate symbol names
> > + * where the first match is not ftrace-instrumented,
> > + * or for module symbols. Retry with linear scan.
> > + */
> > + }
> > +
> > + /* Batch path: single-pass O(N) linear scan */
> > memset(addrs, 0, sizeof(*addrs) * cnt);
> > args.addrs = addrs;
> > args.syms = sorted_syms;
>
^ permalink raw reply
* [PATCH v2 4/4] mm: add tracepoints for zone lock
From: Dmitry Ilvokhin @ 2026-02-25 14:43 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Brendan Jackman,
Johannes Weiner, Zi Yan, Oscar Salvador, Qi Zheng, Shakeel Butt,
Axel Rasmussen, Yuanchu Xie, Wei Xu
Cc: linux-kernel, linux-mm, linux-trace-kernel, linux-cxl,
kernel-team, Benjamin Cheatham, Dmitry Ilvokhin
In-Reply-To: <cover.1772030186.git.d@ilvokhin.com>
Add tracepoint instrumentation to zone lock acquire/release operations
via the previously introduced wrappers.
The implementation follows the mmap_lock tracepoint pattern: a
lightweight inline helper checks whether the tracepoint is enabled and
calls into an out-of-line helper when tracing is active. When
CONFIG_TRACING is disabled, helpers compile to empty inline stubs.
The fast path is unaffected when tracing is disabled.
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
---
MAINTAINERS | 2 +
include/linux/zone_lock.h | 64 +++++++++++++++++++++++++++++++-
include/trace/events/zone_lock.h | 64 ++++++++++++++++++++++++++++++++
mm/Makefile | 2 +-
mm/zone_lock.c | 31 ++++++++++++++++
5 files changed, 161 insertions(+), 2 deletions(-)
create mode 100644 include/trace/events/zone_lock.h
create mode 100644 mm/zone_lock.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 61e3d1f5bf43..b5aa2bb5d2ba 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -16681,6 +16681,7 @@ F: include/linux/ptdump.h
F: include/linux/vmpressure.h
F: include/linux/vmstat.h
F: include/linux/zone_lock.h
+F: include/trace/events/zone_lock.h
F: kernel/fork.c
F: mm/Kconfig
F: mm/debug.c
@@ -16700,6 +16701,7 @@ F: mm/sparse.c
F: mm/util.c
F: mm/vmpressure.c
F: mm/vmstat.c
+F: mm/zone_lock.c
N: include/linux/page[-_]*
MEMORY MANAGEMENT - EXECMEM
diff --git a/include/linux/zone_lock.h b/include/linux/zone_lock.h
index c531e26280e6..cea41dd56324 100644
--- a/include/linux/zone_lock.h
+++ b/include/linux/zone_lock.h
@@ -4,6 +4,53 @@
#include <linux/mmzone.h>
#include <linux/spinlock.h>
+#include <linux/tracepoint-defs.h>
+
+DECLARE_TRACEPOINT(zone_lock_start_locking);
+DECLARE_TRACEPOINT(zone_lock_acquire_returned);
+DECLARE_TRACEPOINT(zone_lock_released);
+
+#ifdef CONFIG_TRACING
+
+void __zone_lock_do_trace_start_locking(struct zone *zone);
+void __zone_lock_do_trace_acquire_returned(struct zone *zone, bool success);
+void __zone_lock_do_trace_released(struct zone *zone);
+
+static inline void __zone_lock_trace_start_locking(struct zone *zone)
+{
+ if (tracepoint_enabled(zone_lock_start_locking))
+ __zone_lock_do_trace_start_locking(zone);
+}
+
+static inline void __zone_lock_trace_acquire_returned(struct zone *zone,
+ bool success)
+{
+ if (tracepoint_enabled(zone_lock_acquire_returned))
+ __zone_lock_do_trace_acquire_returned(zone, success);
+}
+
+static inline void __zone_lock_trace_released(struct zone *zone)
+{
+ if (tracepoint_enabled(zone_lock_released))
+ __zone_lock_do_trace_released(zone);
+}
+
+#else /* !CONFIG_TRACING */
+
+static inline void __zone_lock_trace_start_locking(struct zone *zone)
+{
+}
+
+static inline void __zone_lock_trace_acquire_returned(struct zone *zone,
+ bool success)
+{
+}
+
+static inline void __zone_lock_trace_released(struct zone *zone)
+{
+}
+
+#endif /* CONFIG_TRACING */
static inline void zone_lock_init(struct zone *zone)
{
@@ -12,26 +59,41 @@ static inline void zone_lock_init(struct zone *zone)
#define zone_lock_irqsave(zone, flags) \
do { \
+ bool success = true; \
+ \
+ __zone_lock_trace_start_locking(zone); \
spin_lock_irqsave(&(zone)->lock, flags); \
+ __zone_lock_trace_acquire_returned(zone, success); \
} while (0)
#define zone_trylock_irqsave(zone, flags) \
({ \
- spin_trylock_irqsave(&(zone)->lock, flags); \
+ bool success; \
+ \
+ __zone_lock_trace_start_locking(zone); \
+ success = spin_trylock_irqsave(&(zone)->lock, flags); \
+ __zone_lock_trace_acquire_returned(zone, success); \
+ success; \
})
static inline void zone_unlock_irqrestore(struct zone *zone, unsigned long flags)
{
+ __zone_lock_trace_released(zone);
spin_unlock_irqrestore(&zone->lock, flags);
}
static inline void zone_lock_irq(struct zone *zone)
{
+ bool success = true;
+
+ __zone_lock_trace_start_locking(zone);
spin_lock_irq(&zone->lock);
+ __zone_lock_trace_acquire_returned(zone, success);
}
static inline void zone_unlock_irq(struct zone *zone)
{
+ __zone_lock_trace_released(zone);
spin_unlock_irq(&zone->lock);
}
diff --git a/include/trace/events/zone_lock.h b/include/trace/events/zone_lock.h
new file mode 100644
index 000000000000..3df82a8c0160
--- /dev/null
+++ b/include/trace/events/zone_lock.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM zone_lock
+
+#if !defined(_TRACE_ZONE_LOCK_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_ZONE_LOCK_H
+
+#include <linux/tracepoint.h>
+#include <linux/types.h>
+
+struct zone;
+
+DECLARE_EVENT_CLASS(zone_lock,
+
+ TP_PROTO(struct zone *zone),
+
+ TP_ARGS(zone),
+
+ TP_STRUCT__entry(
+ __field(struct zone *, zone)
+ ),
+
+ TP_fast_assign(
+ __entry->zone = zone;
+ ),
+
+ TP_printk("zone=%p", __entry->zone)
+);
+
+#define DEFINE_ZONE_LOCK_EVENT(name) \
+ DEFINE_EVENT(zone_lock, name, \
+ TP_PROTO(struct zone *zone), \
+ TP_ARGS(zone))
+
+DEFINE_ZONE_LOCK_EVENT(zone_lock_start_locking);
+DEFINE_ZONE_LOCK_EVENT(zone_lock_released);
+
+TRACE_EVENT(zone_lock_acquire_returned,
+
+ TP_PROTO(struct zone *zone, bool success),
+
+ TP_ARGS(zone, success),
+
+ TP_STRUCT__entry(
+ __field(struct zone *, zone)
+ __field(bool, success)
+ ),
+
+ TP_fast_assign(
+ __entry->zone = zone;
+ __entry->success = success;
+ ),
+
+ TP_printk(
+ "zone=%p success=%s",
+ __entry->zone,
+ __entry->success ? "true" : "false"
+ )
+);
+
+#endif /* _TRACE_ZONE_LOCK_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/mm/Makefile b/mm/Makefile
index 8ad2ab08244e..ffd06cf7a04e 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -55,7 +55,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \
mm_init.o percpu.o slab_common.o \
compaction.o show_mem.o \
interval_tree.o list_lru.o workingset.o \
- debug.o gup.o mmap_lock.o vma_init.o $(mmu-y)
+ debug.o gup.o mmap_lock.o zone_lock.o vma_init.o $(mmu-y)
# Give 'page_alloc' its own module-parameter namespace
page-alloc-y := page_alloc.o
diff --git a/mm/zone_lock.c b/mm/zone_lock.c
new file mode 100644
index 000000000000..f647fd2aca48
--- /dev/null
+++ b/mm/zone_lock.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#define CREATE_TRACE_POINTS
+#include <trace/events/zone_lock.h>
+
+#include <linux/zone_lock.h>
+
+EXPORT_TRACEPOINT_SYMBOL(zone_lock_start_locking);
+EXPORT_TRACEPOINT_SYMBOL(zone_lock_acquire_returned);
+EXPORT_TRACEPOINT_SYMBOL(zone_lock_released);
+
+#ifdef CONFIG_TRACING
+
+void __zone_lock_do_trace_start_locking(struct zone *zone)
+{
+ trace_zone_lock_start_locking(zone);
+}
+EXPORT_SYMBOL(__zone_lock_do_trace_start_locking);
+
+void __zone_lock_do_trace_acquire_returned(struct zone *zone, bool success)
+{
+ trace_zone_lock_acquire_returned(zone, success);
+}
+EXPORT_SYMBOL(__zone_lock_do_trace_acquire_returned);
+
+void __zone_lock_do_trace_released(struct zone *zone)
+{
+ trace_zone_lock_released(zone);
+}
+EXPORT_SYMBOL(__zone_lock_do_trace_released);
+
+#endif /* CONFIG_TRACING */
--
2.47.3
^ permalink raw reply related
* [PATCH v2 2/4] mm: convert zone lock users to wrappers
From: Dmitry Ilvokhin @ 2026-02-25 14:43 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Brendan Jackman,
Johannes Weiner, Zi Yan, Oscar Salvador, Qi Zheng, Shakeel Butt,
Axel Rasmussen, Yuanchu Xie, Wei Xu
Cc: linux-kernel, linux-mm, linux-trace-kernel, linux-cxl,
kernel-team, Benjamin Cheatham, Dmitry Ilvokhin
In-Reply-To: <cover.1772030186.git.d@ilvokhin.com>
Replace direct zone lock acquire/release operations with the
newly introduced wrappers.
The changes are purely mechanical substitutions. No functional change
intended. Locking semantics and ordering remain unchanged.
The compaction path is left unchanged for now and will be
handled separately in the following patch due to additional
non-trivial modifications.
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
---
mm/compaction.c | 25 +++++++++-------
mm/memory_hotplug.c | 9 +++---
mm/mm_init.c | 3 +-
mm/page_alloc.c | 73 +++++++++++++++++++++++----------------------
mm/page_isolation.c | 19 ++++++------
mm/page_reporting.c | 13 ++++----
mm/show_mem.c | 5 ++--
mm/vmscan.c | 5 ++--
mm/vmstat.c | 9 +++---
9 files changed, 86 insertions(+), 75 deletions(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 1e8f8eca318c..47b26187a5df 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -24,6 +24,7 @@
#include <linux/page_owner.h>
#include <linux/psi.h>
#include <linux/cpuset.h>
+#include <linux/zone_lock.h>
#include "internal.h"
#ifdef CONFIG_COMPACTION
@@ -530,11 +531,14 @@ static bool compact_lock_irqsave(spinlock_t *lock, unsigned long *flags,
* Returns true if compaction should abort due to fatal signal pending.
* Returns false when compaction can continue.
*/
-static bool compact_unlock_should_abort(spinlock_t *lock,
- unsigned long flags, bool *locked, struct compact_control *cc)
+
+static bool compact_unlock_should_abort(struct zone *zone,
+ unsigned long flags,
+ bool *locked,
+ struct compact_control *cc)
{
if (*locked) {
- spin_unlock_irqrestore(lock, flags);
+ zone_unlock_irqrestore(zone, flags);
*locked = false;
}
@@ -582,9 +586,8 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
* contention, to give chance to IRQs. Abort if fatal signal
* pending.
*/
- if (!(blockpfn % COMPACT_CLUSTER_MAX)
- && compact_unlock_should_abort(&cc->zone->lock, flags,
- &locked, cc))
+ if (!(blockpfn % COMPACT_CLUSTER_MAX) &&
+ compact_unlock_should_abort(cc->zone, flags, &locked, cc))
break;
nr_scanned++;
@@ -649,7 +652,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
}
if (locked)
- spin_unlock_irqrestore(&cc->zone->lock, flags);
+ zone_unlock_irqrestore(cc->zone, flags);
/*
* Be careful to not go outside of the pageblock.
@@ -1555,7 +1558,7 @@ static void fast_isolate_freepages(struct compact_control *cc)
if (!area->nr_free)
continue;
- spin_lock_irqsave(&cc->zone->lock, flags);
+ zone_lock_irqsave(cc->zone, flags);
freelist = &area->free_list[MIGRATE_MOVABLE];
list_for_each_entry_reverse(freepage, freelist, buddy_list) {
unsigned long pfn;
@@ -1614,7 +1617,7 @@ static void fast_isolate_freepages(struct compact_control *cc)
}
}
- spin_unlock_irqrestore(&cc->zone->lock, flags);
+ zone_unlock_irqrestore(cc->zone, flags);
/* Skip fast search if enough freepages isolated */
if (cc->nr_freepages >= cc->nr_migratepages)
@@ -1988,7 +1991,7 @@ static unsigned long fast_find_migrateblock(struct compact_control *cc)
if (!area->nr_free)
continue;
- spin_lock_irqsave(&cc->zone->lock, flags);
+ zone_lock_irqsave(cc->zone, flags);
freelist = &area->free_list[MIGRATE_MOVABLE];
list_for_each_entry(freepage, freelist, buddy_list) {
unsigned long free_pfn;
@@ -2021,7 +2024,7 @@ static unsigned long fast_find_migrateblock(struct compact_control *cc)
break;
}
}
- spin_unlock_irqrestore(&cc->zone->lock, flags);
+ zone_unlock_irqrestore(cc->zone, flags);
}
cc->total_migrate_scanned += nr_scanned;
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index bc805029da51..cfc0103fa50e 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -36,6 +36,7 @@
#include <linux/rmap.h>
#include <linux/module.h>
#include <linux/node.h>
+#include <linux/zone_lock.h>
#include <asm/tlbflush.h>
@@ -1190,9 +1191,9 @@ int online_pages(unsigned long pfn, unsigned long nr_pages,
* Fixup the number of isolated pageblocks before marking the sections
* onlining, such that undo_isolate_page_range() works correctly.
*/
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
zone->nr_isolate_pageblock += nr_pages / pageblock_nr_pages;
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
/*
* If this zone is not populated, then it is not in zonelist.
@@ -2041,9 +2042,9 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
* effectively stale; nobody should be touching them. Fixup the number
* of isolated pageblocks, memory onlining will properly revert this.
*/
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
zone->nr_isolate_pageblock -= nr_pages / pageblock_nr_pages;
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
lru_cache_enable();
zone_pcp_enable(zone);
diff --git a/mm/mm_init.c b/mm/mm_init.c
index 61d983d23f55..6dd37621248b 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -32,6 +32,7 @@
#include <linux/vmstat.h>
#include <linux/kexec_handover.h>
#include <linux/hugetlb.h>
+#include <linux/zone_lock.h>
#include "internal.h"
#include "slab.h"
#include "shuffle.h"
@@ -1425,7 +1426,7 @@ static void __meminit zone_init_internals(struct zone *zone, enum zone_type idx,
zone_set_nid(zone, nid);
zone->name = zone_names[idx];
zone->zone_pgdat = NODE_DATA(nid);
- spin_lock_init(&zone->lock);
+ zone_lock_init(zone);
zone_seqlock_init(zone);
zone_pcp_init(zone);
}
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index fcc32737f451..c5d13fe9b79f 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -54,6 +54,7 @@
#include <linux/delayacct.h>
#include <linux/cacheinfo.h>
#include <linux/pgalloc_tag.h>
+#include <linux/zone_lock.h>
#include <asm/div64.h>
#include "internal.h"
#include "shuffle.h"
@@ -1500,7 +1501,7 @@ static void free_pcppages_bulk(struct zone *zone, int count,
/* Ensure requested pindex is drained first. */
pindex = pindex - 1;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
while (count > 0) {
struct list_head *list;
@@ -1533,7 +1534,7 @@ static void free_pcppages_bulk(struct zone *zone, int count,
} while (count > 0 && !list_empty(list));
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
}
/* Split a multi-block free page into its individual pageblocks. */
@@ -1577,12 +1578,12 @@ static void free_one_page(struct zone *zone, struct page *page,
unsigned long flags;
if (unlikely(fpi_flags & FPI_TRYLOCK)) {
- if (!spin_trylock_irqsave(&zone->lock, flags)) {
+ if (!zone_trylock_irqsave(zone, flags)) {
add_page_to_zone_llist(zone, page, order);
return;
}
} else {
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
}
/* The lock succeeded. Process deferred pages. */
@@ -1600,7 +1601,7 @@ static void free_one_page(struct zone *zone, struct page *page,
}
}
split_large_buddy(zone, page, pfn, order, fpi_flags);
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
__count_vm_events(PGFREE, 1 << order);
}
@@ -2553,10 +2554,10 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
int i;
if (unlikely(alloc_flags & ALLOC_TRYLOCK)) {
- if (!spin_trylock_irqsave(&zone->lock, flags))
+ if (!zone_trylock_irqsave(zone, flags))
return 0;
} else {
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
}
for (i = 0; i < count; ++i) {
struct page *page = __rmqueue(zone, order, migratetype,
@@ -2576,7 +2577,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
*/
list_add_tail(&page->pcp_list, list);
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return i;
}
@@ -3246,10 +3247,10 @@ struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone,
do {
page = NULL;
if (unlikely(alloc_flags & ALLOC_TRYLOCK)) {
- if (!spin_trylock_irqsave(&zone->lock, flags))
+ if (!zone_trylock_irqsave(zone, flags))
return NULL;
} else {
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
}
if (alloc_flags & ALLOC_HIGHATOMIC)
page = __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC);
@@ -3268,11 +3269,11 @@ struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone,
page = __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC);
if (!page) {
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return NULL;
}
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
} while (check_new_pages(page, order));
__count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order);
@@ -3459,7 +3460,7 @@ static void reserve_highatomic_pageblock(struct page *page, int order,
if (zone->nr_reserved_highatomic >= max_managed)
return;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
/* Recheck the nr_reserved_highatomic limit under the lock */
if (zone->nr_reserved_highatomic >= max_managed)
@@ -3481,7 +3482,7 @@ static void reserve_highatomic_pageblock(struct page *page, int order,
}
out_unlock:
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
}
/*
@@ -3514,7 +3515,7 @@ static bool unreserve_highatomic_pageblock(const struct alloc_context *ac,
pageblock_nr_pages)
continue;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
for (order = 0; order < NR_PAGE_ORDERS; order++) {
struct free_area *area = &(zone->free_area[order]);
unsigned long size;
@@ -3562,11 +3563,11 @@ static bool unreserve_highatomic_pageblock(const struct alloc_context *ac,
*/
WARN_ON_ONCE(ret == -1);
if (ret > 0) {
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return ret;
}
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
}
return false;
@@ -6446,7 +6447,7 @@ static void __setup_per_zone_wmarks(void)
for_each_zone(zone) {
u64 tmp;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
tmp = (u64)pages_min * zone_managed_pages(zone);
tmp = div64_ul(tmp, lowmem_pages);
if (is_highmem(zone) || zone_idx(zone) == ZONE_MOVABLE) {
@@ -6487,7 +6488,7 @@ static void __setup_per_zone_wmarks(void)
zone->_watermark[WMARK_PROMO] = high_wmark_pages(zone) + tmp;
trace_mm_setup_per_zone_wmarks(zone);
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
}
/* update totalreserve_pages */
@@ -7257,7 +7258,7 @@ struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages,
zonelist = node_zonelist(nid, gfp_mask);
for_each_zone_zonelist_nodemask(zone, z, zonelist,
gfp_zone(gfp_mask), nodemask) {
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
pfn = ALIGN(zone->zone_start_pfn, nr_pages);
while (zone_spans_last_pfn(zone, pfn, nr_pages)) {
@@ -7271,18 +7272,18 @@ struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages,
* allocation spinning on this lock, it may
* win the race and cause allocation to fail.
*/
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
ret = alloc_contig_frozen_range_noprof(pfn,
pfn + nr_pages,
ACR_FLAGS_NONE,
gfp_mask);
if (!ret)
return pfn_to_page(pfn);
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
}
pfn += nr_pages;
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
}
/*
* If we failed, retry the search, but treat regions with HugeTLB pages
@@ -7436,7 +7437,7 @@ unsigned long __offline_isolated_pages(unsigned long start_pfn,
offline_mem_sections(pfn, end_pfn);
zone = page_zone(pfn_to_page(pfn));
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
while (pfn < end_pfn) {
page = pfn_to_page(pfn);
/*
@@ -7466,7 +7467,7 @@ unsigned long __offline_isolated_pages(unsigned long start_pfn,
del_page_from_free_list(page, zone, order, MIGRATE_ISOLATE);
pfn += (1 << order);
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return end_pfn - start_pfn - already_offline;
}
@@ -7542,7 +7543,7 @@ bool take_page_off_buddy(struct page *page)
unsigned int order;
bool ret = false;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
for (order = 0; order < NR_PAGE_ORDERS; order++) {
struct page *page_head = page - (pfn & ((1 << order) - 1));
int page_order = buddy_order(page_head);
@@ -7563,7 +7564,7 @@ bool take_page_off_buddy(struct page *page)
if (page_count(page_head) > 0)
break;
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return ret;
}
@@ -7576,7 +7577,7 @@ bool put_page_back_buddy(struct page *page)
unsigned long flags;
bool ret = false;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
if (put_page_testzero(page)) {
unsigned long pfn = page_to_pfn(page);
int migratetype = get_pfnblock_migratetype(page, pfn);
@@ -7587,7 +7588,7 @@ bool put_page_back_buddy(struct page *page)
ret = true;
}
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return ret;
}
@@ -7636,7 +7637,7 @@ static void __accept_page(struct zone *zone, unsigned long *flags,
account_freepages(zone, -MAX_ORDER_NR_PAGES, MIGRATE_MOVABLE);
__mod_zone_page_state(zone, NR_UNACCEPTED, -MAX_ORDER_NR_PAGES);
__ClearPageUnaccepted(page);
- spin_unlock_irqrestore(&zone->lock, *flags);
+ zone_unlock_irqrestore(zone, *flags);
accept_memory(page_to_phys(page), PAGE_SIZE << MAX_PAGE_ORDER);
@@ -7648,9 +7649,9 @@ void accept_page(struct page *page)
struct zone *zone = page_zone(page);
unsigned long flags;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
if (!PageUnaccepted(page)) {
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return;
}
@@ -7663,11 +7664,11 @@ static bool try_to_accept_memory_one(struct zone *zone)
unsigned long flags;
struct page *page;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
page = list_first_entry_or_null(&zone->unaccepted_pages,
struct page, lru);
if (!page) {
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return false;
}
@@ -7724,12 +7725,12 @@ static bool __free_unaccepted(struct page *page)
if (!lazy_accept)
return false;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
list_add_tail(&page->lru, &zone->unaccepted_pages);
account_freepages(zone, MAX_ORDER_NR_PAGES, MIGRATE_MOVABLE);
__mod_zone_page_state(zone, NR_UNACCEPTED, MAX_ORDER_NR_PAGES);
__SetPageUnaccepted(page);
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return true;
}
diff --git a/mm/page_isolation.c b/mm/page_isolation.c
index c48ff5c00244..56a272f38b66 100644
--- a/mm/page_isolation.c
+++ b/mm/page_isolation.c
@@ -10,6 +10,7 @@
#include <linux/hugetlb.h>
#include <linux/page_owner.h>
#include <linux/migrate.h>
+#include <linux/zone_lock.h>
#include "internal.h"
#define CREATE_TRACE_POINTS
@@ -173,7 +174,7 @@ static int set_migratetype_isolate(struct page *page, enum pb_isolate_mode mode,
if (PageUnaccepted(page))
accept_page(page);
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
/*
* We assume the caller intended to SET migrate type to isolate.
@@ -181,7 +182,7 @@ static int set_migratetype_isolate(struct page *page, enum pb_isolate_mode mode,
* set it before us.
*/
if (is_migrate_isolate_page(page)) {
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return -EBUSY;
}
@@ -200,15 +201,15 @@ static int set_migratetype_isolate(struct page *page, enum pb_isolate_mode mode,
mode);
if (!unmovable) {
if (!pageblock_isolate_and_move_free_pages(zone, page)) {
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return -EBUSY;
}
zone->nr_isolate_pageblock++;
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
return 0;
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
if (mode == PB_ISOLATE_MODE_MEM_OFFLINE) {
/*
* printk() with zone->lock held will likely trigger a
@@ -229,7 +230,7 @@ static void unset_migratetype_isolate(struct page *page)
struct page *buddy;
zone = page_zone(page);
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
if (!is_migrate_isolate_page(page))
goto out;
@@ -280,7 +281,7 @@ static void unset_migratetype_isolate(struct page *page)
}
zone->nr_isolate_pageblock--;
out:
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
}
static inline struct page *
@@ -641,9 +642,9 @@ int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn,
/* Check all pages are free or marked as ISOLATED */
zone = page_zone(page);
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
pfn = __test_page_isolated_in_pageblock(start_pfn, end_pfn, mode);
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
ret = pfn < end_pfn ? -EBUSY : 0;
diff --git a/mm/page_reporting.c b/mm/page_reporting.c
index f0042d5743af..37e54e16538b 100644
--- a/mm/page_reporting.c
+++ b/mm/page_reporting.c
@@ -7,6 +7,7 @@
#include <linux/module.h>
#include <linux/delay.h>
#include <linux/scatterlist.h>
+#include <linux/zone_lock.h>
#include "page_reporting.h"
#include "internal.h"
@@ -161,7 +162,7 @@ page_reporting_cycle(struct page_reporting_dev_info *prdev, struct zone *zone,
if (list_empty(list))
return err;
- spin_lock_irq(&zone->lock);
+ zone_lock_irq(zone);
/*
* Limit how many calls we will be making to the page reporting
@@ -219,7 +220,7 @@ page_reporting_cycle(struct page_reporting_dev_info *prdev, struct zone *zone,
list_rotate_to_front(&page->lru, list);
/* release lock before waiting on report processing */
- spin_unlock_irq(&zone->lock);
+ zone_unlock_irq(zone);
/* begin processing pages in local list */
err = prdev->report(prdev, sgl, PAGE_REPORTING_CAPACITY);
@@ -231,7 +232,7 @@ page_reporting_cycle(struct page_reporting_dev_info *prdev, struct zone *zone,
budget--;
/* reacquire zone lock and resume processing */
- spin_lock_irq(&zone->lock);
+ zone_lock_irq(zone);
/* flush reported pages from the sg list */
page_reporting_drain(prdev, sgl, PAGE_REPORTING_CAPACITY, !err);
@@ -251,7 +252,7 @@ page_reporting_cycle(struct page_reporting_dev_info *prdev, struct zone *zone,
if (!list_entry_is_head(next, list, lru) && !list_is_first(&next->lru, list))
list_rotate_to_front(&next->lru, list);
- spin_unlock_irq(&zone->lock);
+ zone_unlock_irq(zone);
return err;
}
@@ -296,9 +297,9 @@ page_reporting_process_zone(struct page_reporting_dev_info *prdev,
err = prdev->report(prdev, sgl, leftover);
/* flush any remaining pages out from the last report */
- spin_lock_irq(&zone->lock);
+ zone_lock_irq(zone);
page_reporting_drain(prdev, sgl, leftover, !err);
- spin_unlock_irq(&zone->lock);
+ zone_unlock_irq(zone);
}
return err;
diff --git a/mm/show_mem.c b/mm/show_mem.c
index 24078ac3e6bc..245beca127af 100644
--- a/mm/show_mem.c
+++ b/mm/show_mem.c
@@ -14,6 +14,7 @@
#include <linux/mmzone.h>
#include <linux/swap.h>
#include <linux/vmstat.h>
+#include <linux/zone_lock.h>
#include "internal.h"
#include "swap.h"
@@ -363,7 +364,7 @@ static void show_free_areas(unsigned int filter, nodemask_t *nodemask, int max_z
show_node(zone);
printk(KERN_CONT "%s: ", zone->name);
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
for (order = 0; order < NR_PAGE_ORDERS; order++) {
struct free_area *area = &zone->free_area[order];
int type;
@@ -377,7 +378,7 @@ static void show_free_areas(unsigned int filter, nodemask_t *nodemask, int max_z
types[order] |= 1 << type;
}
}
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
for (order = 0; order < NR_PAGE_ORDERS; order++) {
printk(KERN_CONT "%lu*%lukB ",
nr[order], K(1UL) << order);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 0fc9373e8251..b369e00e8415 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -58,6 +58,7 @@
#include <linux/random.h>
#include <linux/mmu_notifier.h>
#include <linux/parser.h>
+#include <linux/zone_lock.h>
#include <asm/tlbflush.h>
#include <asm/div64.h>
@@ -7139,9 +7140,9 @@ static int balance_pgdat(pg_data_t *pgdat, int order, int highest_zoneidx)
/* Increments are under the zone lock */
zone = pgdat->node_zones + i;
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
zone->watermark_boost -= min(zone->watermark_boost, zone_boosts[i]);
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
}
/*
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 86b14b0f77b5..299b461a6b4b 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -28,6 +28,7 @@
#include <linux/mm_inline.h>
#include <linux/page_owner.h>
#include <linux/sched/isolation.h>
+#include <linux/zone_lock.h>
#include "internal.h"
@@ -1535,10 +1536,10 @@ static void walk_zones_in_node(struct seq_file *m, pg_data_t *pgdat,
continue;
if (!nolock)
- spin_lock_irqsave(&zone->lock, flags);
+ zone_lock_irqsave(zone, flags);
print(m, pgdat, zone);
if (!nolock)
- spin_unlock_irqrestore(&zone->lock, flags);
+ zone_unlock_irqrestore(zone, flags);
}
}
#endif
@@ -1603,9 +1604,9 @@ static void pagetypeinfo_showfree_print(struct seq_file *m,
}
}
seq_printf(m, "%s%6lu ", overflow ? ">" : "", freecount);
- spin_unlock_irq(&zone->lock);
+ zone_unlock_irq(zone);
cond_resched();
- spin_lock_irq(&zone->lock);
+ zone_lock_irq(zone);
}
seq_putc(m, '\n');
}
--
2.47.3
^ permalink raw reply related
* [PATCH v2 3/4] mm: convert compaction to zone lock wrappers
From: Dmitry Ilvokhin @ 2026-02-25 14:43 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Brendan Jackman,
Johannes Weiner, Zi Yan, Oscar Salvador, Qi Zheng, Shakeel Butt,
Axel Rasmussen, Yuanchu Xie, Wei Xu
Cc: linux-kernel, linux-mm, linux-trace-kernel, linux-cxl,
kernel-team, Benjamin Cheatham, Dmitry Ilvokhin
In-Reply-To: <cover.1772030186.git.d@ilvokhin.com>
Compaction uses compact_lock_irqsave(), which currently operates
on a raw spinlock_t pointer so that it can be used for both
zone->lock and lru_lock. Since zone lock operations are now wrapped,
compact_lock_irqsave() can no longer operate directly on a spinlock_t
when the lock belongs to a zone.
Introduce struct compact_lock to abstract the underlying lock type. The
structure carries a lock type enum and a union holding either a zone
pointer or a raw spinlock_t pointer, and dispatches to the appropriate
lock/unlock helper.
No functional change intended.
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
---
mm/compaction.c | 73 +++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 64 insertions(+), 9 deletions(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 47b26187a5df..c3b97379a963 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -494,6 +494,53 @@ static bool test_and_set_skip(struct compact_control *cc, struct page *page)
}
#endif /* CONFIG_COMPACTION */
+enum compact_lock_type {
+ COMPACT_LOCK_ZONE,
+ COMPACT_LOCK_RAW_SPINLOCK,
+};
+
+struct compact_lock {
+ enum compact_lock_type type;
+ union {
+ struct zone *zone;
+ spinlock_t *lock; /* Reference to lru lock */
+ };
+};
+
+static bool compact_do_trylock_irqsave(struct compact_lock lock,
+ unsigned long *flags)
+{
+ if (lock.type == COMPACT_LOCK_ZONE)
+ return zone_trylock_irqsave(lock.zone, *flags);
+
+ return spin_trylock_irqsave(lock.lock, *flags);
+}
+
+static void compact_do_zone_lock_irqsave(struct zone *zone,
+ unsigned long *flags)
+__acquires(zone->lock)
+{
+ zone_lock_irqsave(zone, *flags);
+}
+
+static void compact_do_raw_lock_irqsave(spinlock_t *lock,
+ unsigned long *flags)
+__acquires(lock)
+{
+ spin_lock_irqsave(lock, *flags);
+}
+
+static void compact_do_lock_irqsave(struct compact_lock lock,
+ unsigned long *flags)
+{
+ if (lock.type == COMPACT_LOCK_ZONE) {
+ compact_do_zone_lock_irqsave(lock.zone, flags);
+ return;
+ }
+
+ compact_do_raw_lock_irqsave(lock.lock, flags);
+}
+
/*
* Compaction requires the taking of some coarse locks that are potentially
* very heavily contended. For async compaction, trylock and record if the
@@ -503,19 +550,19 @@ static bool test_and_set_skip(struct compact_control *cc, struct page *page)
*
* Always returns true which makes it easier to track lock state in callers.
*/
-static bool compact_lock_irqsave(spinlock_t *lock, unsigned long *flags,
- struct compact_control *cc)
- __acquires(lock)
+static bool compact_lock_irqsave(struct compact_lock lock,
+ unsigned long *flags,
+ struct compact_control *cc)
{
/* Track if the lock is contended in async mode */
if (cc->mode == MIGRATE_ASYNC && !cc->contended) {
- if (spin_trylock_irqsave(lock, *flags))
+ if (compact_do_trylock_irqsave(lock, flags))
return true;
cc->contended = true;
}
- spin_lock_irqsave(lock, *flags);
+ compact_do_lock_irqsave(lock, flags);
return true;
}
@@ -531,7 +578,6 @@ static bool compact_lock_irqsave(spinlock_t *lock, unsigned long *flags,
* Returns true if compaction should abort due to fatal signal pending.
* Returns false when compaction can continue.
*/
-
static bool compact_unlock_should_abort(struct zone *zone,
unsigned long flags,
bool *locked,
@@ -616,8 +662,12 @@ static unsigned long isolate_freepages_block(struct compact_control *cc,
/* If we already hold the lock, we can skip some rechecking. */
if (!locked) {
- locked = compact_lock_irqsave(&cc->zone->lock,
- &flags, cc);
+ struct compact_lock zol = {
+ .type = COMPACT_LOCK_ZONE,
+ .zone = cc->zone,
+ };
+
+ locked = compact_lock_irqsave(zol, &flags, cc);
/* Recheck this is a buddy page under lock */
if (!PageBuddy(page))
@@ -1160,10 +1210,15 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
/* If we already hold the lock, we can skip some rechecking */
if (lruvec != locked) {
+ struct compact_lock zol = {
+ .type = COMPACT_LOCK_RAW_SPINLOCK,
+ .lock = &lruvec->lru_lock,
+ };
+
if (locked)
unlock_page_lruvec_irqrestore(locked, flags);
- compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);
+ compact_lock_irqsave(zol, &flags, cc);
locked = lruvec;
lruvec_memcg_debug(lruvec, folio);
--
2.47.3
^ permalink raw reply related
* [PATCH v2 1/4] mm: introduce zone lock wrappers
From: Dmitry Ilvokhin @ 2026-02-25 14:43 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Brendan Jackman,
Johannes Weiner, Zi Yan, Oscar Salvador, Qi Zheng, Shakeel Butt,
Axel Rasmussen, Yuanchu Xie, Wei Xu
Cc: linux-kernel, linux-mm, linux-trace-kernel, linux-cxl,
kernel-team, Benjamin Cheatham, Dmitry Ilvokhin
In-Reply-To: <cover.1772030186.git.d@ilvokhin.com>
Add thin wrappers around zone lock acquire/release operations. This
prepares the code for future tracepoint instrumentation without
modifying individual call sites.
Centralizing zone lock operations behind wrappers allows future
instrumentation or debugging hooks to be added without touching
all users.
No functional change intended. The wrappers are introduced in
preparation for subsequent patches and are not yet used.
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
---
MAINTAINERS | 1 +
include/linux/zone_lock.h | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 39 insertions(+)
create mode 100644 include/linux/zone_lock.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 55af015174a5..61e3d1f5bf43 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -16680,6 +16680,7 @@ F: include/linux/pgtable.h
F: include/linux/ptdump.h
F: include/linux/vmpressure.h
F: include/linux/vmstat.h
+F: include/linux/zone_lock.h
F: kernel/fork.c
F: mm/Kconfig
F: mm/debug.c
diff --git a/include/linux/zone_lock.h b/include/linux/zone_lock.h
new file mode 100644
index 000000000000..c531e26280e6
--- /dev/null
+++ b/include/linux/zone_lock.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_ZONE_LOCK_H
+#define _LINUX_ZONE_LOCK_H
+
+#include <linux/mmzone.h>
+#include <linux/spinlock.h>
+
+static inline void zone_lock_init(struct zone *zone)
+{
+ spin_lock_init(&zone->lock);
+}
+
+#define zone_lock_irqsave(zone, flags) \
+do { \
+ spin_lock_irqsave(&(zone)->lock, flags); \
+} while (0)
+
+#define zone_trylock_irqsave(zone, flags) \
+({ \
+ spin_trylock_irqsave(&(zone)->lock, flags); \
+})
+
+static inline void zone_unlock_irqrestore(struct zone *zone, unsigned long flags)
+{
+ spin_unlock_irqrestore(&zone->lock, flags);
+}
+
+static inline void zone_lock_irq(struct zone *zone)
+{
+ spin_lock_irq(&zone->lock);
+}
+
+static inline void zone_unlock_irq(struct zone *zone)
+{
+ spin_unlock_irq(&zone->lock);
+}
+
+#endif /* _LINUX_ZONE_LOCK_H */
--
2.47.3
^ permalink raw reply related
* [PATCH v2 0/4] mm: zone lock tracepoint instrumentation
From: Dmitry Ilvokhin @ 2026-02-25 14:43 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Brendan Jackman,
Johannes Weiner, Zi Yan, Oscar Salvador, Qi Zheng, Shakeel Butt,
Axel Rasmussen, Yuanchu Xie, Wei Xu
Cc: linux-kernel, linux-mm, linux-trace-kernel, linux-cxl,
kernel-team, Benjamin Cheatham, Dmitry Ilvokhin
Zone lock contention can significantly impact allocation and
reclaim latency, as it is a central synchronization point in
the page allocator and reclaim paths. Improved visibility into
its behavior is therefore important for diagnosing performance
issues in memory-intensive workloads.
On some production workloads at Meta, we have observed noticeable
zone lock contention. Deeper analysis of lock holders and waiters
is currently difficult with existing instrumentation.
While generic lock contention_begin/contention_end tracepoints
cover the slow path, they do not provide sufficient visibility
into lock hold times. In particular, the lack of a release-side
event makes it difficult to identify long lock holders and
correlate them with waiters. As a result, distinguishing between
short bursts of contention and pathological long hold times
requires additional instrumentation.
This patch series adds dedicated tracepoint instrumentation to
zone lock, following the existing mmap_lock tracing model.
The goal is to enable detailed holder/waiter analysis and lock
hold time measurements without affecting the fast path when
tracing is disabled.
The series is structured as follows:
1. Introduce zone lock wrappers.
2. Mechanically convert zone lock users to the wrappers.
3. Convert compaction to use the wrappers (requires minor
restructuring of compact_lock_irqsave()).
4. Add zone lock tracepoints.
The tracepoints are added via lightweight inline helpers in the
wrappers. When tracing is disabled, the fast path remains
unchanged.
The compaction changes required abstracting compact_lock_irqsave() away from
raw spinlock_t. I chose a small tagged struct to handle both zone and LRU
locks uniformly. If there is a preferred alternative (e.g. splitting helpers
or using a different abstraction), I would appreciate feedback.
Changes in v2:
- Move mecanical changes from mm/compaction.c to different commit.
- Removed compact_do_zone_trylock() and compact_do_raw_trylock_irqsave().
Dmitry Ilvokhin (4):
mm: introduce zone lock wrappers
mm: convert zone lock users to wrappers
mm: convert compaction to zone lock wrappers
mm: add tracepoints for zone lock
MAINTAINERS | 3 +
include/linux/zone_lock.h | 100 +++++++++++++++++++++++++++++++
include/trace/events/zone_lock.h | 64 ++++++++++++++++++++
mm/Makefile | 2 +-
mm/compaction.c | 96 +++++++++++++++++++++++------
mm/memory_hotplug.c | 9 +--
mm/mm_init.c | 3 +-
mm/page_alloc.c | 73 +++++++++++-----------
mm/page_isolation.c | 19 +++---
mm/page_reporting.c | 13 ++--
mm/show_mem.c | 5 +-
mm/vmscan.c | 5 +-
mm/vmstat.c | 9 +--
mm/zone_lock.c | 31 ++++++++++
14 files changed, 348 insertions(+), 84 deletions(-)
create mode 100644 include/linux/zone_lock.h
create mode 100644 include/trace/events/zone_lock.h
create mode 100644 mm/zone_lock.c
--
2.47.3
^ permalink raw reply
* Re: [LSF/MM/BPF TOPIC][RFC PATCH v4 00/27] Private Memory Nodes (w/ Compressed RAM)
From: Gregory Price @ 2026-02-25 14:43 UTC (permalink / raw)
To: Alejandro Lucero Palau
Cc: lsf-pc, linux-kernel, linux-cxl, cgroups, linux-mm,
linux-trace-kernel, damon, kernel-team, gregkh, rafael, dakr,
dave, jonathan.cameron, dave.jiang, alison.schofield,
vishal.l.verma, ira.weiny, dan.j.williams, longman, akpm, david,
lorenzo.stoakes, Liam.Howlett, vbabka, rppt, surenb, mhocko,
osalvador, ziy, matthew.brost, joshua.hahnjy, rakie.kim,
byungchul, ying.huang, apopple, axelrasmussen, yuanchu, weixugc,
yury.norov, linux, mhiramat, mathieu.desnoyers, tj, hannes,
mkoutny, jackmanb, sj, baolin.wang, npache, ryan.roberts,
dev.jain, baohua, lance.yang, muchun.song, xu.xin16,
chengming.zhou, jannh, linmiaohe, nao.horiguchi, pfalcato,
rientjes, shakeel.butt, riel, harry.yoo, cl, roman.gushchin,
chrisl, kasong, shikemeng, nphamcs, bhe, zhengqi.arch,
terry.bowman
In-Reply-To: <b704b05e-3e65-4a73-84c0-21557b0cc38f@amd.com>
On Wed, Feb 25, 2026 at 12:40:09PM +0000, Alejandro Lucero Palau wrote:
>
> > /* This is my memory. There are many like it, but this one is mine. */
> > rc = add_private_memory_driver_managed(nid, start, size, name, flags,
> > online_type, private_context);
> >
> > page = alloc_pages_node(nid, __GFP_PRIVATE, 0);
>
>
> Hi Gregory,
>
>
> I can see the nid param is just a "preferred nid" with alloc pages. Using
> __GFP_PRIVATE will restrict the allocation to private nodes but I think the
> idea here is:
>
>
> 1) I own this node
>
> 2) Do not give me memory from another private node but from mine.
>
>
> Should not this be ensure somehow?
>
A right I set up GFP_PRIVATE for this
#define GFP_PRIVATE (__GFP_PRIVATE | __GFP_THISNODE)
If your service hides the interface to get to this node behind something
it controls, and it doesn't enable things like reclaim/compaction etc,
then it's responsible for dealing with an out-of-memory situation.
__THISNODE was insufficient alone to allow isolation since it is used in
a variety of scenarios around the kernel. v2 or v3 explored that.
~Gregory
^ permalink raw reply
* [PATCH net-next v2 10/10] net/mlx5: Add a shared devlink instance for PFs on same chip
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Use the previously introduced shared devlink infrastructure to create
a shared devlink instance for mlx5 PFs that reside on the same physical
chip. The shared instance is identified by the chip's serial number
extracted from PCI VPD (V3 keyword, with fallback to serial number
for older devices).
Each PF that probes calls mlx5_shd_init() which extracts the chip serial
number and uses devlink_shd_get() to get or create the shared instance.
When a PF is removed, mlx5_shd_uninit() calls devlink_shd_put()
to release the reference. The shared instance is automatically destroyed
when the last PF is removed.
Make the PF devlink instances nested in this shared devlink instance,
allowing userspace to identify which PFs belong to the same physical
chip.
Example:
pci/0000:08:00.0: index 0
nested_devlink:
auxiliary/mlx5_core.eth.0
devlink_index/1: index 1
nested_devlink:
pci/0000:08:00.0
pci/0000:08:00.1
auxiliary/mlx5_core.eth.0: index 2
pci/0000:08:00.1: index 3
nested_devlink:
auxiliary/mlx5_core.eth.1
auxiliary/mlx5_core.eth.1: index 4
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/core/Makefile | 5 +-
.../net/ethernet/mellanox/mlx5/core/main.c | 17 +++++
.../ethernet/mellanox/mlx5/core/sh_devlink.c | 62 +++++++++++++++++++
.../ethernet/mellanox/mlx5/core/sh_devlink.h | 12 ++++
include/linux/mlx5/driver.h | 1 +
5 files changed, 95 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index 8ffa286a18f5..d39fe9c4a87c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -16,8 +16,9 @@ mlx5_core-y := main.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \
transobj.o vport.o sriov.o fs_cmd.o fs_core.o pci_irq.o \
fs_counters.o fs_ft_pool.o rl.o lag/debugfs.o lag/lag.o dev.o events.o wq.o lib/gid.o \
lib/devcom.o lib/pci_vsc.o lib/dm.o lib/fs_ttc.o diag/fs_tracepoint.o \
- diag/fw_tracer.o diag/crdump.o devlink.o diag/rsc_dump.o diag/reporter_vnic.o \
- fw_reset.o qos.o lib/tout.o lib/aso.o wc.o fs_pool.o lib/nv_param.o
+ diag/fw_tracer.o diag/crdump.o devlink.o sh_devlink.o diag/rsc_dump.o \
+ diag/reporter_vnic.o fw_reset.o qos.o lib/tout.o lib/aso.o wc.o fs_pool.o \
+ lib/nv_param.o
#
# Netdev basic
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 55b4e0cceae2..0aefcbaf4761 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -74,6 +74,7 @@
#include "mlx5_irq.h"
#include "hwmon.h"
#include "lag/lag.h"
+#include "sh_devlink.h"
MODULE_AUTHOR("Eli Cohen <eli@mellanox.com>");
MODULE_DESCRIPTION("Mellanox 5th generation network adapters (ConnectX series) core driver");
@@ -1520,10 +1521,16 @@ int mlx5_init_one(struct mlx5_core_dev *dev)
int err;
devl_lock(devlink);
+ if (dev->shd) {
+ err = devl_nested_devlink_set(dev->shd, devlink);
+ if (err)
+ goto unlock;
+ }
devl_register(devlink);
err = mlx5_init_one_devl_locked(dev);
if (err)
devl_unregister(devlink);
+unlock:
devl_unlock(devlink);
return err;
}
@@ -2005,6 +2012,13 @@ static int probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
goto pci_init_err;
}
+ err = mlx5_shd_init(dev);
+ if (err) {
+ mlx5_core_err(dev, "mlx5_shd_init failed with error code %d\n",
+ err);
+ goto shd_init_err;
+ }
+
err = mlx5_init_one(dev);
if (err) {
mlx5_core_err(dev, "mlx5_init_one failed with error code %d\n",
@@ -2018,6 +2032,8 @@ static int probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
return 0;
err_init_one:
+ mlx5_shd_uninit(dev);
+shd_init_err:
mlx5_pci_close(dev);
pci_init_err:
mlx5_mdev_uninit(dev);
@@ -2039,6 +2055,7 @@ static void remove_one(struct pci_dev *pdev)
mlx5_drain_health_wq(dev);
mlx5_sriov_disable(pdev, false);
mlx5_uninit_one(dev);
+ mlx5_shd_uninit(dev);
mlx5_pci_close(dev);
mlx5_mdev_uninit(dev);
mlx5_adev_idx_free(dev->priv.adev_idx);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c
new file mode 100644
index 000000000000..f0561c40f52a
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c
@@ -0,0 +1,62 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#include <linux/mlx5/driver.h>
+#include <net/devlink.h>
+
+#include "sh_devlink.h"
+
+static const struct devlink_ops mlx5_shd_ops = {
+};
+
+int mlx5_shd_init(struct mlx5_core_dev *dev)
+{
+ u8 *vpd_data __free(kfree) = NULL;
+ struct pci_dev *pdev = dev->pdev;
+ unsigned int vpd_size, kw_len;
+ struct devlink *devlink;
+ const char *sn;
+ char *end;
+ int start;
+ int err;
+
+ if (!mlx5_core_is_pf(dev))
+ return 0;
+
+ vpd_data = pci_vpd_alloc(pdev, &vpd_size);
+ if (IS_ERR(vpd_data)) {
+ err = PTR_ERR(vpd_data);
+ return err == -ENODEV ? 0 : err;
+ }
+ start = pci_vpd_find_ro_info_keyword(vpd_data, vpd_size, "V3", &kw_len);
+ if (start < 0) {
+ /* Fall-back to SN for older devices. */
+ start = pci_vpd_find_ro_info_keyword(vpd_data, vpd_size,
+ PCI_VPD_RO_KEYWORD_SERIALNO, &kw_len);
+ if (start < 0)
+ return -ENOENT;
+ }
+ sn = kstrndup(vpd_data + start, kw_len, GFP_KERNEL);
+ if (!sn)
+ return -ENOMEM;
+ /* Firmware may return spaces at the end of the string, strip it. */
+ end = strchrnul(sn, ' ');
+ *end = '\0';
+
+ /* Get or create shared devlink instance */
+ devlink = devlink_shd_get(sn, &mlx5_shd_ops, 0);
+ kfree(sn);
+ if (!devlink)
+ return -ENOMEM;
+
+ dev->shd = devlink;
+ return 0;
+}
+
+void mlx5_shd_uninit(struct mlx5_core_dev *dev)
+{
+ if (!dev->shd)
+ return;
+
+ devlink_shd_put(dev->shd);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h
new file mode 100644
index 000000000000..8ab8d6940227
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#ifndef __MLX5_SH_DEVLINK_H__
+#define __MLX5_SH_DEVLINK_H__
+
+#include <linux/mlx5/driver.h>
+
+int mlx5_shd_init(struct mlx5_core_dev *dev);
+void mlx5_shd_uninit(struct mlx5_core_dev *dev);
+
+#endif /* __MLX5_SH_DEVLINK_H__ */
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index 04dcd09f7517..1268fcf35ec7 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -798,6 +798,7 @@ struct mlx5_core_dev {
enum mlx5_wc_state wc_state;
/* sync write combining state */
struct mutex wc_state_lock;
+ struct devlink *shd;
};
struct mlx5_db {
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 09/10] documentation: networking: add shared devlink documentation
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Document shared devlink instances for multiple PFs on the same chip.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
v1->v2:
- fixed number of "="'s
---
.../networking/devlink/devlink-shared.rst | 89 +++++++++++++++++++
Documentation/networking/devlink/index.rst | 1 +
2 files changed, 90 insertions(+)
create mode 100644 Documentation/networking/devlink/devlink-shared.rst
diff --git a/Documentation/networking/devlink/devlink-shared.rst b/Documentation/networking/devlink/devlink-shared.rst
new file mode 100644
index 000000000000..4043f6647243
--- /dev/null
+++ b/Documentation/networking/devlink/devlink-shared.rst
@@ -0,0 +1,89 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+========================
+Devlink Shared Instances
+========================
+
+Overview
+========
+
+Shared devlink instances allow multiple physical functions (PFs) on the same
+chip to share an additional devlink instance for chip-wide operations. This
+is implemented within individual drivers alongside the individual PF devlink
+instances, not replacing them.
+
+Multiple PFs may reside on the same physical chip, running a single firmware.
+Some of the resources and configurations may be shared among these PFs. The
+shared devlink instance provides an object to pin configuration knobs on.
+
+The shared devlink instance is not backed by any struct *device*.
+
+Implementation
+==============
+
+Architecture
+------------
+
+The implementation uses:
+
+* **Chip identification**: PFs are grouped by chip using a driver-specific identifier
+* **Shared instance management**: Global list of shared instances with reference counting
+
+API Functions
+-------------
+
+The following functions are provided for managing shared devlink instances:
+
+* ``devlink_shd_get()``: Get or create a shared devlink instance identified by a string ID
+* ``devlink_shd_put()``: Release a reference on a shared devlink instance
+* ``devlink_shd_get_priv()``: Get private data from shared devlink instance
+
+Initialization Flow
+-------------------
+
+1. **PF calls shared devlink init** during driver probe
+2. **Chip identification** using driver-specific method to determine device identity
+3. **Get or create shared instance** using ``devlink_shd_get()``:
+
+ * The function looks up existing instance by identifier
+ * If none exists, creates new instance:
+ - Allocates and registers devlink instance
+ - Adds to global shared instances list
+ - Increments reference count
+
+4. **Set nested devlink instance** for the PF devlink instance using
+ ``devl_nested_devlink_set()`` before registering the PF devlink instance
+
+Cleanup Flow
+------------
+
+1. **Cleanup** when PF is removed
+2. **Call** ``devlink_shd_put()`` to release reference (decrements reference count)
+3. **Shared instance is automatically destroyed** when the last PF removes (reference count reaches zero)
+
+Chip Identification
+-------------------
+
+PFs belonging to the same chip are identified using a driver-specific method.
+The driver is free to choose any identifier that is suitable for determining
+whether two PFs are part of the same device. Examples include:
+
+* **PCI VPD serial numbers**: Extract from PCI VPD
+* **Device tree properties**: Read chip identifier from device tree
+* **Other hardware-specific identifiers**: Any unique identifier that groups PFs by chip
+
+Locking
+-------
+
+A global mutex (``shd_mutex``) protects the shared instances list during registration/deregistration.
+
+Similarly to other nested devlink instance relationships, devlink lock of
+the shared instance should be always taken after the devlink lock of PF.
+
+Reference Counting
+------------------
+
+Each shared devlink instance maintains a reference count (``refcount_t refcount``).
+The reference count is incremented when ``devlink_shd_get()`` is called and decremented
+when ``devlink_shd_put()`` is called. When the reference count reaches zero, the shared
+instance is automatically destroyed.
diff --git a/Documentation/networking/devlink/index.rst b/Documentation/networking/devlink/index.rst
index 35b12a2bfeba..f7ba7dcf477d 100644
--- a/Documentation/networking/devlink/index.rst
+++ b/Documentation/networking/devlink/index.rst
@@ -68,6 +68,7 @@ general.
devlink-resource
devlink-selftests
devlink-trap
+ devlink-shared
Driver-specific documentation
-----------------------------
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 08/10] devlink: introduce shared devlink instance for PFs on same chip
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Multiple PFs may reside on the same physical chip, running a single
firmware. Some of the resources and configurations may be shared among
these PFs. Currently, there is no good object to pin the configuration
knobs on.
Introduce a shared devlink instance, instantiated upon probe of
the first PF and removed during remove of the last PF. The shared
devlink instance is not backed by any device device, as there is
no PCI device related to it.
The implementation uses reference counting to manage the lifecycle:
each PF that probes calls devlink_shd_get() to get or create
the shared instance, and calls devlink_shd_put() when it removes.
The shared instance is automatically destroyed when the last PF removes.
Example:
pci/0000:08:00.0: index 0
nested_devlink:
auxiliary/mlx5_core.eth.0
devlink_index/1: index 1
nested_devlink:
pci/0000:08:00.0
pci/0000:08:00.1
auxiliary/mlx5_core.eth.0: index 2
pci/0000:08:00.1: index 3
nested_devlink:
auxiliary/mlx5_core.eth.1
auxiliary/mlx5_core.eth.1: index 4
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
v1->v2:
- s/err_kstrdup_id/err_devlink_free/
- fixed kernel-doc comment of devlink_shd_get()
- removed NULL arg check in devlink_shd_get/put()
---
include/net/devlink.h | 6 ++
net/devlink/Makefile | 2 +-
net/devlink/sh_dev.c | 142 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 149 insertions(+), 1 deletion(-)
create mode 100644 net/devlink/sh_dev.c
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 45dec7067a8e..e97ee09af5d6 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -1647,6 +1647,12 @@ void devlink_register(struct devlink *devlink);
void devlink_unregister(struct devlink *devlink);
void devlink_free(struct devlink *devlink);
+struct devlink *devlink_shd_get(const char *id,
+ const struct devlink_ops *ops,
+ size_t priv_size);
+void devlink_shd_put(struct devlink *devlink);
+void *devlink_shd_get_priv(struct devlink *devlink);
+
/**
* struct devlink_port_ops - Port operations
* @port_split: Callback used to split the port into multiple ones.
diff --git a/net/devlink/Makefile b/net/devlink/Makefile
index 000da622116a..8f2adb5e5836 100644
--- a/net/devlink/Makefile
+++ b/net/devlink/Makefile
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: GPL-2.0
obj-y := core.o netlink.o netlink_gen.o dev.o port.o sb.o dpipe.o \
- resource.o param.o region.o health.o trap.o rate.o linecard.o
+ resource.o param.o region.o health.o trap.o rate.o linecard.o sh_dev.o
diff --git a/net/devlink/sh_dev.c b/net/devlink/sh_dev.c
new file mode 100644
index 000000000000..f4da25e8f8bc
--- /dev/null
+++ b/net/devlink/sh_dev.c
@@ -0,0 +1,142 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#include <net/devlink.h>
+
+#include "devl_internal.h"
+
+static LIST_HEAD(shd_list);
+static DEFINE_MUTEX(shd_mutex); /* Protects shd_list and shd->list */
+
+/* This structure represents a shared devlink instance,
+ * there is one created per identifier (e.g., serial number).
+ */
+struct devlink_shd {
+ struct list_head list; /* Node in shd list */
+ const char *id; /* Identifier string (e.g., serial number) */
+ refcount_t refcount; /* Reference count */
+ char priv[] __aligned(NETDEV_ALIGN); /* Driver private data */
+};
+
+static struct devlink_shd *devlink_shd_lookup(const char *id)
+{
+ struct devlink_shd *shd;
+
+ list_for_each_entry(shd, &shd_list, list) {
+ if (!strcmp(shd->id, id))
+ return shd;
+ }
+
+ return NULL;
+}
+
+static struct devlink_shd *devlink_shd_create(const char *id,
+ const struct devlink_ops *ops,
+ size_t priv_size)
+{
+ struct devlink_shd *shd;
+ struct devlink *devlink;
+
+ devlink = devlink_alloc(ops, sizeof(struct devlink_shd) + priv_size,
+ NULL);
+ if (!devlink)
+ return NULL;
+ shd = devlink_priv(devlink);
+
+ shd->id = kstrdup(id, GFP_KERNEL);
+ if (!shd->id)
+ goto err_devlink_free;
+ refcount_set(&shd->refcount, 1);
+
+ devl_lock(devlink);
+ devl_register(devlink);
+ devl_unlock(devlink);
+
+ list_add_tail(&shd->list, &shd_list);
+
+ return shd;
+
+err_devlink_free:
+ devlink_free(devlink);
+ return NULL;
+}
+
+static void devlink_shd_destroy(struct devlink_shd *shd)
+{
+ struct devlink *devlink = priv_to_devlink(shd);
+
+ list_del(&shd->list);
+ devl_lock(devlink);
+ devl_unregister(devlink);
+ devl_unlock(devlink);
+ kfree(shd->id);
+ devlink_free(devlink);
+}
+
+/**
+ * devlink_shd_get - Get or create a shared devlink instance
+ * @id: Identifier string (e.g., serial number) for the shared instance
+ * @ops: Devlink operations structure
+ * @priv_size: Size of private data structure
+ *
+ * Get an existing shared devlink instance identified by @id, or create
+ * a new one if it doesn't exist. Return the devlink instance with a
+ * reference held. The caller must call devlink_shd_put() when done.
+ *
+ * Return: Pointer to the shared devlink instance on success,
+ * NULL on failure
+ */
+struct devlink *devlink_shd_get(const char *id,
+ const struct devlink_ops *ops,
+ size_t priv_size)
+{
+ struct devlink_shd *shd;
+
+ mutex_lock(&shd_mutex);
+
+ shd = devlink_shd_lookup(id);
+ if (!shd)
+ shd = devlink_shd_create(id, ops, priv_size);
+ else
+ refcount_inc(&shd->refcount);
+
+ mutex_unlock(&shd_mutex);
+ return shd ? priv_to_devlink(shd) : NULL;
+}
+EXPORT_SYMBOL_GPL(devlink_shd_get);
+
+/**
+ * devlink_shd_put - Release a reference on a shared devlink instance
+ * @devlink: Shared devlink instance
+ *
+ * Release a reference on a shared devlink instance obtained via
+ * devlink_shd_get().
+ */
+void devlink_shd_put(struct devlink *devlink)
+{
+ struct devlink_shd *shd;
+
+ mutex_lock(&shd_mutex);
+ shd = devlink_priv(devlink);
+ if (refcount_dec_and_test(&shd->refcount))
+ devlink_shd_destroy(shd);
+ mutex_unlock(&shd_mutex);
+}
+EXPORT_SYMBOL_GPL(devlink_shd_put);
+
+/**
+ * devlink_shd_get_priv - Get private data from shared devlink instance
+ * @devlink: Devlink instance
+ *
+ * Returns a pointer to the driver's private data structure within
+ * the shared devlink instance.
+ *
+ * Return: Pointer to private data
+ */
+void *devlink_shd_get_priv(struct devlink *devlink)
+{
+ struct devlink_shd *shd = devlink_priv(devlink);
+
+ return shd->priv;
+}
+EXPORT_SYMBOL_GPL(devlink_shd_get_priv);
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 07/10] devlink: allow devlink instance allocation without a backing device
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Allow devlink_alloc_ns() to be called with dev=NULL to support
device-less devlink instances. When dev is NULL, the instance is
identified over netlink using "devlink_index" as bus_name and
the decimal index value as dev_name.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
v1->v2:
- moved DEVLINK_INDEX_BUS_NAME definition to patch #5
- added comment to dev arg that it can be NULL
- fixed the index sprintf for dev-less
---
net/devlink/core.c | 25 +++++++++++++++++++------
net/devlink/dev.c | 11 +++++++----
net/devlink/devl_internal.h | 4 ++--
net/devlink/port.c | 14 +++++++++-----
4 files changed, 37 insertions(+), 17 deletions(-)
diff --git a/net/devlink/core.c b/net/devlink/core.c
index 85ea5856d523..e931e66aa3a2 100644
--- a/net/devlink/core.c
+++ b/net/devlink/core.c
@@ -331,7 +331,10 @@ static void devlink_release(struct work_struct *work)
mutex_destroy(&devlink->lock);
lockdep_unregister_key(&devlink->lock_key);
- put_device(devlink->dev);
+ if (devlink->dev)
+ put_device(devlink->dev);
+ else
+ kfree(devlink->dev_name);
kvfree(devlink);
}
@@ -433,7 +436,7 @@ EXPORT_SYMBOL_GPL(devlink_unregister);
* @ops: ops
* @priv_size: size of user private data
* @net: net namespace
- * @dev: parent device
+ * @dev: parent device, or NULL for a device-less devlink instance
*
* Allocate new devlink instance resources, including devlink index
* and name.
@@ -446,7 +449,7 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops,
static u32 last_id;
int ret;
- WARN_ON(!ops || !dev);
+ WARN_ON(!ops);
if (!devlink_reload_actions_valid(ops))
return NULL;
@@ -459,9 +462,17 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops,
if (ret < 0)
goto err_xa_alloc;
- devlink->dev = get_device(dev);
- devlink->bus_name = dev->bus->name;
- devlink->dev_name = dev_name(dev);
+ if (dev) {
+ devlink->dev = get_device(dev);
+ devlink->bus_name = dev->bus->name;
+ devlink->dev_name = dev_name(dev);
+ } else {
+ devlink->bus_name = DEVLINK_INDEX_BUS_NAME;
+ devlink->dev_name = kasprintf(GFP_KERNEL, "%u", devlink->index);
+ if (!devlink->dev_name)
+ goto err_kasprintf;
+ }
+
devlink->ops = ops;
xa_init_flags(&devlink->ports, XA_FLAGS_ALLOC);
xa_init_flags(&devlink->params, XA_FLAGS_ALLOC);
@@ -486,6 +497,8 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops,
return devlink;
+err_kasprintf:
+ xa_erase(&devlinks, devlink->index);
err_xa_alloc:
kvfree(devlink);
return NULL;
diff --git a/net/devlink/dev.c b/net/devlink/dev.c
index e3a36de4f4ae..b63597312bbd 100644
--- a/net/devlink/dev.c
+++ b/net/devlink/dev.c
@@ -453,7 +453,8 @@ int devlink_reload(struct devlink *devlink, struct net *dest_net,
* (e.g., PCI reset) and to close possible races between these
* operations and probe/remove.
*/
- device_lock_assert(devlink->dev);
+ if (devlink->dev)
+ device_lock_assert(devlink->dev);
memcpy(remote_reload_stats, devlink->stats.remote_reload_stats,
sizeof(remote_reload_stats));
@@ -892,9 +893,11 @@ devlink_nl_info_fill(struct sk_buff *msg, struct devlink *devlink,
goto err_cancel_msg;
}
- err = devlink_nl_driver_info_get(dev->driver, &req);
- if (err)
- goto err_cancel_msg;
+ if (dev) {
+ err = devlink_nl_driver_info_get(dev->driver, &req);
+ if (err)
+ goto err_cancel_msg;
+ }
genlmsg_end(msg, hdr);
return 0;
diff --git a/net/devlink/devl_internal.h b/net/devlink/devl_internal.h
index 67425c5d8cfc..89d08fd511cb 100644
--- a/net/devlink/devl_internal.h
+++ b/net/devlink/devl_internal.h
@@ -107,7 +107,7 @@ static inline bool devl_is_registered(struct devlink *devlink)
static inline void devl_dev_lock(struct devlink *devlink, bool dev_lock)
{
- if (dev_lock)
+ if (dev_lock && devlink->dev)
device_lock(devlink->dev);
devl_lock(devlink);
}
@@ -115,7 +115,7 @@ static inline void devl_dev_lock(struct devlink *devlink, bool dev_lock)
static inline void devl_dev_unlock(struct devlink *devlink, bool dev_lock)
{
devl_unlock(devlink);
- if (dev_lock)
+ if (dev_lock && devlink->dev)
device_unlock(devlink->dev);
}
diff --git a/net/devlink/port.c b/net/devlink/port.c
index 1d4a79c6d4d3..f19b690ebe7e 100644
--- a/net/devlink/port.c
+++ b/net/devlink/port.c
@@ -976,7 +976,9 @@ static void devlink_port_type_warn(struct work_struct *work)
struct devlink_port *port = container_of(to_delayed_work(work),
struct devlink_port,
type_warn_dw);
- dev_warn(port->devlink->dev, "Type was not set for devlink port.");
+ if (port->devlink->dev)
+ dev_warn(port->devlink->dev,
+ "Type was not set for devlink port.");
}
static bool devlink_port_type_should_warn(struct devlink_port *devlink_port)
@@ -1242,9 +1244,10 @@ static void __devlink_port_type_set(struct devlink_port *devlink_port,
*/
void devlink_port_type_eth_set(struct devlink_port *devlink_port)
{
- dev_warn(devlink_port->devlink->dev,
- "devlink port type for port %d set to Ethernet without a software interface reference, device type not supported by the kernel?\n",
- devlink_port->index);
+ if (devlink_port->devlink->dev)
+ dev_warn(devlink_port->devlink->dev,
+ "devlink port type for port %d set to Ethernet without a software interface reference, device type not supported by the kernel?\n",
+ devlink_port->index);
__devlink_port_type_set(devlink_port, DEVLINK_PORT_TYPE_ETH, NULL);
}
EXPORT_SYMBOL_GPL(devlink_port_type_eth_set);
@@ -1272,7 +1275,8 @@ EXPORT_SYMBOL_GPL(devlink_port_type_ib_set);
*/
void devlink_port_type_clear(struct devlink_port *devlink_port)
{
- if (devlink_port->type == DEVLINK_PORT_TYPE_ETH)
+ if (devlink_port->type == DEVLINK_PORT_TYPE_ETH &&
+ devlink_port->devlink->dev)
dev_warn(devlink_port->devlink->dev,
"devlink port type for port %d cleared without a software interface reference, device type not supported by the kernel?\n",
devlink_port->index);
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 06/10] devlink: add devlink_dev_driver_name() helper and use it in trace events
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
In preparation to dev-less devlinks, add devlink_dev_driver_name()
that safely returns the driver name or NULL, and use it in all trace
events. The trace __string() macro handles NULL via __string_src(),
recording "(null)".
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
v1->v2:
- added missing symbol export
---
include/net/devlink.h | 1 +
include/trace/events/devlink.h | 12 ++++++------
net/devlink/core.c | 8 ++++++++
3 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 0afb0958b910..45dec7067a8e 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -1613,6 +1613,7 @@ struct devlink *priv_to_devlink(void *priv);
struct device *devlink_to_dev(const struct devlink *devlink);
const char *devlink_bus_name(const struct devlink *devlink);
const char *devlink_dev_name(const struct devlink *devlink);
+const char *devlink_dev_driver_name(const struct devlink *devlink);
/* Devlink instance explicit locking */
void devl_lock(struct devlink *devlink);
diff --git a/include/trace/events/devlink.h b/include/trace/events/devlink.h
index 32304ce9ad15..4f8edf77dfbe 100644
--- a/include/trace/events/devlink.h
+++ b/include/trace/events/devlink.h
@@ -23,7 +23,7 @@ TRACE_EVENT(devlink_hwmsg,
TP_STRUCT__entry(
__string(bus_name, devlink_bus_name(devlink))
__string(dev_name, devlink_dev_name(devlink))
- __string(driver_name, devlink_to_dev(devlink)->driver->name)
+ __string(driver_name, devlink_dev_driver_name(devlink))
__field(bool, incoming)
__field(unsigned long, type)
__dynamic_array(u8, buf, len)
@@ -57,7 +57,7 @@ TRACE_EVENT(devlink_hwerr,
TP_STRUCT__entry(
__string(bus_name, devlink_bus_name(devlink))
__string(dev_name, devlink_dev_name(devlink))
- __string(driver_name, devlink_to_dev(devlink)->driver->name)
+ __string(driver_name, devlink_dev_driver_name(devlink))
__field(int, err)
__string(msg, msg)
),
@@ -87,7 +87,7 @@ TRACE_EVENT(devlink_health_report,
TP_STRUCT__entry(
__string(bus_name, devlink_bus_name(devlink))
__string(dev_name, devlink_dev_name(devlink))
- __string(driver_name, devlink_to_dev(devlink)->driver->name)
+ __string(driver_name, devlink_dev_driver_name(devlink))
__string(reporter_name, reporter_name)
__string(msg, msg)
),
@@ -118,7 +118,7 @@ TRACE_EVENT(devlink_health_recover_aborted,
TP_STRUCT__entry(
__string(bus_name, devlink_bus_name(devlink))
__string(dev_name, devlink_dev_name(devlink))
- __string(driver_name, devlink_to_dev(devlink)->driver->name)
+ __string(driver_name, devlink_dev_driver_name(devlink))
__string(reporter_name, reporter_name)
__field(bool, health_state)
__field(u64, time_since_last_recover)
@@ -152,7 +152,7 @@ TRACE_EVENT(devlink_health_reporter_state_update,
TP_STRUCT__entry(
__string(bus_name, devlink_bus_name(devlink))
__string(dev_name, devlink_dev_name(devlink))
- __string(driver_name, devlink_to_dev(devlink)->driver->name)
+ __string(driver_name, devlink_dev_driver_name(devlink))
__string(reporter_name, reporter_name)
__field(u8, new_state)
),
@@ -183,7 +183,7 @@ TRACE_EVENT(devlink_trap_report,
TP_STRUCT__entry(
__string(bus_name, devlink_bus_name(devlink))
__string(dev_name, devlink_dev_name(devlink))
- __string(driver_name, devlink_to_dev(devlink)->driver->name)
+ __string(driver_name, devlink_dev_driver_name(devlink))
__string(trap_name, metadata->trap_name)
__string(trap_group_name, metadata->trap_group_name)
__array(char, input_dev_name, IFNAMSIZ)
diff --git a/net/devlink/core.c b/net/devlink/core.c
index 2dd6d45bec18..85ea5856d523 100644
--- a/net/devlink/core.c
+++ b/net/devlink/core.c
@@ -260,6 +260,14 @@ const char *devlink_dev_name(const struct devlink *devlink)
}
EXPORT_SYMBOL_GPL(devlink_dev_name);
+const char *devlink_dev_driver_name(const struct devlink *devlink)
+{
+ struct device *dev = devlink->dev;
+
+ return dev ? dev->driver->name : NULL;
+}
+EXPORT_SYMBOL_GPL(devlink_dev_driver_name);
+
struct net *devlink_net(const struct devlink *devlink)
{
return read_pnet(&devlink->_net);
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 05/10] devlink: support index-based lookup via bus_name/dev_name handle
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Devlink instances without a backing device use bus_name
"devlink_index" and dev_name set to the decimal index string.
When user space sends this handle, detect the pattern and perform
a direct xarray lookup by index instead of iterating all instances.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
v1->v2:
- moved DEVLINK_INDEX_BUS_NAME definition here from patch #7
---
include/uapi/linux/devlink.h | 2 ++
net/devlink/netlink.c | 9 +++++++++
2 files changed, 11 insertions(+)
diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h
index 1ba3436db4ae..7de2d8cc862f 100644
--- a/include/uapi/linux/devlink.h
+++ b/include/uapi/linux/devlink.h
@@ -19,6 +19,8 @@
#define DEVLINK_GENL_VERSION 0x1
#define DEVLINK_GENL_MCGRP_CONFIG_NAME "config"
+#define DEVLINK_INDEX_BUS_NAME "devlink_index"
+
enum devlink_command {
/* don't change the order or add anything between, this is ABI! */
DEVLINK_CMD_UNSPEC,
diff --git a/net/devlink/netlink.c b/net/devlink/netlink.c
index b73cec31089f..127f337489c0 100644
--- a/net/devlink/netlink.c
+++ b/net/devlink/netlink.c
@@ -200,6 +200,15 @@ devlink_get_from_attrs_lock(struct net *net, struct nlattr **attrs,
busname = nla_data(attrs[DEVLINK_ATTR_BUS_NAME]);
devname = nla_data(attrs[DEVLINK_ATTR_DEV_NAME]);
+ if (!strcmp(busname, DEVLINK_INDEX_BUS_NAME)) {
+ if (kstrtoul(devname, 10, &index))
+ return ERR_PTR(-EINVAL);
+ devlink = devlinks_xa_lookup_get(net, index);
+ if (!devlink)
+ return ERR_PTR(-ENODEV);
+ goto found;
+ }
+
devlinks_xa_for_each_registered_get(net, index, devlink) {
if (strcmp(devlink->bus_name, busname) == 0 &&
strcmp(devlink->dev_name, devname) == 0)
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 04/10] devlink: allow to use devlink index as a command handle
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Currently devlink instances are addressed bus_name/dev_name tuple.
Allow the newly introduced DEVLINK_ATTR_INDEX to be used as
an alternative handle for all devlink commands.
When DEVLINK_ATTR_INDEX is present in the request, use it for a direct
xarray lookup instead of iterating over all instances comparing
bus_name/dev_name strings.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
Documentation/netlink/specs/devlink.yaml | 51 ++++
net/devlink/core.c | 16 +-
net/devlink/devl_internal.h | 1 +
net/devlink/netlink.c | 11 +-
net/devlink/netlink_gen.c | 350 ++++++++++++++---------
5 files changed, 286 insertions(+), 143 deletions(-)
diff --git a/Documentation/netlink/specs/devlink.yaml b/Documentation/netlink/specs/devlink.yaml
index 1bed67a0eefb..a73a86ec595d 100644
--- a/Documentation/netlink/specs/devlink.yaml
+++ b/Documentation/netlink/specs/devlink.yaml
@@ -1310,6 +1310,7 @@ operations:
attributes: &dev-id-attrs
- bus-name
- dev-name
+ - index
reply: &get-reply
value: 3
attributes:
@@ -1334,6 +1335,7 @@ operations:
attributes: &port-id-attrs
- bus-name
- dev-name
+ - index
- port-index
reply:
value: 7
@@ -1358,6 +1360,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
- port-type
- port-function
@@ -1375,6 +1378,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
- port-flavour
- port-pci-pf-number
@@ -1409,6 +1413,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
- port-split-count
@@ -1437,6 +1442,7 @@ operations:
attributes: &sb-id-attrs
- bus-name
- dev-name
+ - index
- sb-index
reply: &sb-get-reply
value: 13
@@ -1459,6 +1465,7 @@ operations:
attributes: &sb-pool-id-attrs
- bus-name
- dev-name
+ - index
- sb-index
- sb-pool-index
reply: &sb-pool-get-reply
@@ -1482,6 +1489,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- sb-index
- sb-pool-index
- sb-pool-threshold-type
@@ -1500,6 +1508,7 @@ operations:
attributes: &sb-port-pool-id-attrs
- bus-name
- dev-name
+ - index
- port-index
- sb-index
- sb-pool-index
@@ -1524,6 +1533,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
- sb-index
- sb-pool-index
@@ -1542,6 +1552,7 @@ operations:
attributes: &sb-tc-pool-bind-id-attrs
- bus-name
- dev-name
+ - index
- port-index
- sb-index
- sb-pool-type
@@ -1567,6 +1578,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
- sb-index
- sb-pool-index
@@ -1588,6 +1600,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- sb-index
-
@@ -1603,6 +1616,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- sb-index
-
@@ -1621,6 +1635,7 @@ operations:
attributes: &eswitch-attrs
- bus-name
- dev-name
+ - index
- eswitch-mode
- eswitch-inline-mode
- eswitch-encap-mode
@@ -1649,12 +1664,14 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- dpipe-table-name
reply:
value: 31
attributes:
- bus-name
- dev-name
+ - index
- dpipe-tables
-
@@ -1669,11 +1686,13 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- dpipe-table-name
reply:
attributes:
- bus-name
- dev-name
+ - index
- dpipe-entries
-
@@ -1688,10 +1707,12 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
reply:
attributes:
- bus-name
- dev-name
+ - index
- dpipe-headers
-
@@ -1707,6 +1728,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- dpipe-table-name
- dpipe-table-counters-enabled
@@ -1723,6 +1745,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- resource-id
- resource-size
@@ -1738,11 +1761,13 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
reply:
value: 36
attributes:
- bus-name
- dev-name
+ - index
- resource-list
-
@@ -1758,6 +1783,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- reload-action
- reload-limits
- netns-pid
@@ -1767,6 +1793,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- reload-actions-performed
-
@@ -1781,6 +1808,7 @@ operations:
attributes: ¶m-id-attrs
- bus-name
- dev-name
+ - index
- param-name
reply: ¶m-get-reply
attributes: *param-id-attrs
@@ -1802,6 +1830,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- param-name
- param-type
# param-value-data is missing here as the type is variable
@@ -1821,6 +1850,7 @@ operations:
attributes: ®ion-id-attrs
- bus-name
- dev-name
+ - index
- port-index
- region-name
reply: ®ion-get-reply
@@ -1845,6 +1875,7 @@ operations:
attributes: ®ion-snapshot-id-attrs
- bus-name
- dev-name
+ - index
- port-index
- region-name
- region-snapshot-id
@@ -1875,6 +1906,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
- region-name
- region-snapshot-id
@@ -1886,6 +1918,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
- region-name
@@ -1935,6 +1968,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- info-driver-name
- info-serial-number
- info-version-fixed
@@ -1956,6 +1990,7 @@ operations:
attributes: &health-reporter-id-attrs
- bus-name
- dev-name
+ - index
- port-index
- health-reporter-name
reply: &health-reporter-get-reply
@@ -1978,6 +2013,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
- health-reporter-name
- health-reporter-graceful-period
@@ -2048,6 +2084,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- flash-update-file-name
- flash-update-component
- flash-update-overwrite-mask
@@ -2065,6 +2102,7 @@ operations:
attributes: &trap-id-attrs
- bus-name
- dev-name
+ - index
- trap-name
reply: &trap-get-reply
value: 63
@@ -2087,6 +2125,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- trap-name
- trap-action
@@ -2103,6 +2142,7 @@ operations:
attributes: &trap-group-id-attrs
- bus-name
- dev-name
+ - index
- trap-group-name
reply: &trap-group-get-reply
value: 67
@@ -2125,6 +2165,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- trap-group-name
- trap-action
- trap-policer-id
@@ -2142,6 +2183,7 @@ operations:
attributes: &trap-policer-id-attrs
- bus-name
- dev-name
+ - index
- trap-policer-id
reply: &trap-policer-get-reply
value: 71
@@ -2164,6 +2206,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- trap-policer-id
- trap-policer-rate
- trap-policer-burst
@@ -2194,6 +2237,7 @@ operations:
attributes: &rate-id-attrs
- bus-name
- dev-name
+ - index
- port-index
- rate-node-name
reply: &rate-get-reply
@@ -2217,6 +2261,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- rate-node-name
- rate-tx-share
- rate-tx-max
@@ -2238,6 +2283,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- rate-node-name
- rate-tx-share
- rate-tx-max
@@ -2259,6 +2305,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- rate-node-name
-
@@ -2274,6 +2321,7 @@ operations:
attributes: &linecard-id-attrs
- bus-name
- dev-name
+ - index
- linecard-index
reply: &linecard-get-reply
value: 80
@@ -2296,6 +2344,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- linecard-index
- linecard-type
@@ -2329,6 +2378,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- selftests
-
@@ -2340,4 +2390,5 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- port-index
diff --git a/net/devlink/core.c b/net/devlink/core.c
index 49f856beb8b2..2dd6d45bec18 100644
--- a/net/devlink/core.c
+++ b/net/devlink/core.c
@@ -333,13 +333,15 @@ void devlink_put(struct devlink *devlink)
queue_rcu_work(system_percpu_wq, &devlink->rwork);
}
-struct devlink *devlinks_xa_find_get(struct net *net, unsigned long *indexp)
+static struct devlink *__devlinks_xa_find_get(struct net *net,
+ unsigned long *indexp,
+ unsigned long end)
{
struct devlink *devlink = NULL;
rcu_read_lock();
retry:
- devlink = xa_find(&devlinks, indexp, ULONG_MAX, DEVLINK_REGISTERED);
+ devlink = xa_find(&devlinks, indexp, end, DEVLINK_REGISTERED);
if (!devlink)
goto unlock;
@@ -358,6 +360,16 @@ struct devlink *devlinks_xa_find_get(struct net *net, unsigned long *indexp)
goto retry;
}
+struct devlink *devlinks_xa_find_get(struct net *net, unsigned long *indexp)
+{
+ return __devlinks_xa_find_get(net, indexp, ULONG_MAX);
+}
+
+struct devlink *devlinks_xa_lookup_get(struct net *net, unsigned long index)
+{
+ return __devlinks_xa_find_get(net, &index, index);
+}
+
/**
* devl_register - Register devlink instance
* @devlink: devlink
diff --git a/net/devlink/devl_internal.h b/net/devlink/devl_internal.h
index d43819c6b452..67425c5d8cfc 100644
--- a/net/devlink/devl_internal.h
+++ b/net/devlink/devl_internal.h
@@ -92,6 +92,7 @@ extern struct genl_family devlink_nl_family;
for (index = 0; (devlink = devlinks_xa_find_get(net, &index)); index++)
struct devlink *devlinks_xa_find_get(struct net *net, unsigned long *indexp);
+struct devlink *devlinks_xa_lookup_get(struct net *net, unsigned long index);
static inline bool __devl_is_registered(struct devlink *devlink)
{
diff --git a/net/devlink/netlink.c b/net/devlink/netlink.c
index a517d42c7b96..b73cec31089f 100644
--- a/net/devlink/netlink.c
+++ b/net/devlink/netlink.c
@@ -186,6 +186,14 @@ devlink_get_from_attrs_lock(struct net *net, struct nlattr **attrs,
char *busname;
char *devname;
+ if (attrs[DEVLINK_ATTR_INDEX]) {
+ index = nla_get_uint(attrs[DEVLINK_ATTR_INDEX]);
+ devlink = devlinks_xa_lookup_get(net, index);
+ if (!devlink)
+ return ERR_PTR(-ENODEV);
+ goto found;
+ }
+
if (!attrs[DEVLINK_ATTR_BUS_NAME] || !attrs[DEVLINK_ATTR_DEV_NAME])
return ERR_PTR(-EINVAL);
@@ -356,7 +364,8 @@ int devlink_nl_dumpit(struct sk_buff *msg, struct netlink_callback *cb,
int flags = NLM_F_MULTI;
if (attrs &&
- (attrs[DEVLINK_ATTR_BUS_NAME] || attrs[DEVLINK_ATTR_DEV_NAME]))
+ (attrs[DEVLINK_ATTR_BUS_NAME] || attrs[DEVLINK_ATTR_DEV_NAME] ||
+ attrs[DEVLINK_ATTR_INDEX]))
return devlink_nl_inst_single_dumpit(msg, cb, flags, dump_one,
attrs);
else
diff --git a/net/devlink/netlink_gen.c b/net/devlink/netlink_gen.c
index f4c61c2b4f22..9812897c5524 100644
--- a/net/devlink/netlink_gen.c
+++ b/net/devlink/netlink_gen.c
@@ -56,37 +56,42 @@ const struct nla_policy devlink_dl_selftest_id_nl_policy[DEVLINK_ATTR_SELFTEST_I
};
/* DEVLINK_CMD_GET - do */
-static const struct nla_policy devlink_get_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_PORT_GET - do */
-static const struct nla_policy devlink_port_get_do_nl_policy[DEVLINK_ATTR_PORT_INDEX + 1] = {
+static const struct nla_policy devlink_port_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_PORT_GET - dump */
-static const struct nla_policy devlink_port_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_port_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_PORT_SET - do */
-static const struct nla_policy devlink_port_set_nl_policy[DEVLINK_ATTR_PORT_FUNCTION + 1] = {
+static const struct nla_policy devlink_port_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_PORT_TYPE] = NLA_POLICY_MAX(NLA_U16, 3),
[DEVLINK_ATTR_PORT_FUNCTION] = NLA_POLICY_NESTED(devlink_dl_port_function_nl_policy),
};
/* DEVLINK_CMD_PORT_NEW - do */
-static const struct nla_policy devlink_port_new_nl_policy[DEVLINK_ATTR_PORT_PCI_SF_NUMBER + 1] = {
+static const struct nla_policy devlink_port_new_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_PORT_FLAVOUR] = NLA_POLICY_MAX(NLA_U16, 7),
[DEVLINK_ATTR_PORT_PCI_PF_NUMBER] = { .type = NLA_U16, },
@@ -95,58 +100,66 @@ static const struct nla_policy devlink_port_new_nl_policy[DEVLINK_ATTR_PORT_PCI_
};
/* DEVLINK_CMD_PORT_DEL - do */
-static const struct nla_policy devlink_port_del_nl_policy[DEVLINK_ATTR_PORT_INDEX + 1] = {
+static const struct nla_policy devlink_port_del_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_PORT_SPLIT - do */
-static const struct nla_policy devlink_port_split_nl_policy[DEVLINK_ATTR_PORT_SPLIT_COUNT + 1] = {
+static const struct nla_policy devlink_port_split_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_PORT_SPLIT_COUNT] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_PORT_UNSPLIT - do */
-static const struct nla_policy devlink_port_unsplit_nl_policy[DEVLINK_ATTR_PORT_INDEX + 1] = {
+static const struct nla_policy devlink_port_unsplit_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_SB_GET - do */
-static const struct nla_policy devlink_sb_get_do_nl_policy[DEVLINK_ATTR_SB_INDEX + 1] = {
+static const struct nla_policy devlink_sb_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_SB_GET - dump */
-static const struct nla_policy devlink_sb_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_sb_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_SB_POOL_GET - do */
-static const struct nla_policy devlink_sb_pool_get_do_nl_policy[DEVLINK_ATTR_SB_POOL_INDEX + 1] = {
+static const struct nla_policy devlink_sb_pool_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_POOL_INDEX] = { .type = NLA_U16, },
};
/* DEVLINK_CMD_SB_POOL_GET - dump */
-static const struct nla_policy devlink_sb_pool_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_sb_pool_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_SB_POOL_SET - do */
-static const struct nla_policy devlink_sb_pool_set_nl_policy[DEVLINK_ATTR_SB_POOL_THRESHOLD_TYPE + 1] = {
+static const struct nla_policy devlink_sb_pool_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_POOL_INDEX] = { .type = NLA_U16, },
[DEVLINK_ATTR_SB_POOL_THRESHOLD_TYPE] = NLA_POLICY_MAX(NLA_U8, 1),
@@ -154,24 +167,27 @@ static const struct nla_policy devlink_sb_pool_set_nl_policy[DEVLINK_ATTR_SB_POO
};
/* DEVLINK_CMD_SB_PORT_POOL_GET - do */
-static const struct nla_policy devlink_sb_port_pool_get_do_nl_policy[DEVLINK_ATTR_SB_POOL_INDEX + 1] = {
+static const struct nla_policy devlink_sb_port_pool_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_POOL_INDEX] = { .type = NLA_U16, },
};
/* DEVLINK_CMD_SB_PORT_POOL_GET - dump */
-static const struct nla_policy devlink_sb_port_pool_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_sb_port_pool_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_SB_PORT_POOL_SET - do */
-static const struct nla_policy devlink_sb_port_pool_set_nl_policy[DEVLINK_ATTR_SB_THRESHOLD + 1] = {
+static const struct nla_policy devlink_sb_port_pool_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_POOL_INDEX] = { .type = NLA_U16, },
@@ -179,9 +195,10 @@ static const struct nla_policy devlink_sb_port_pool_set_nl_policy[DEVLINK_ATTR_S
};
/* DEVLINK_CMD_SB_TC_POOL_BIND_GET - do */
-static const struct nla_policy devlink_sb_tc_pool_bind_get_do_nl_policy[DEVLINK_ATTR_SB_TC_INDEX + 1] = {
+static const struct nla_policy devlink_sb_tc_pool_bind_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_POOL_TYPE] = NLA_POLICY_MAX(NLA_U8, 1),
@@ -189,15 +206,17 @@ static const struct nla_policy devlink_sb_tc_pool_bind_get_do_nl_policy[DEVLINK_
};
/* DEVLINK_CMD_SB_TC_POOL_BIND_GET - dump */
-static const struct nla_policy devlink_sb_tc_pool_bind_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_sb_tc_pool_bind_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_SB_TC_POOL_BIND_SET - do */
-static const struct nla_policy devlink_sb_tc_pool_bind_set_nl_policy[DEVLINK_ATTR_SB_TC_INDEX + 1] = {
+static const struct nla_policy devlink_sb_tc_pool_bind_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_SB_POOL_INDEX] = { .type = NLA_U16, },
@@ -207,80 +226,91 @@ static const struct nla_policy devlink_sb_tc_pool_bind_set_nl_policy[DEVLINK_ATT
};
/* DEVLINK_CMD_SB_OCC_SNAPSHOT - do */
-static const struct nla_policy devlink_sb_occ_snapshot_nl_policy[DEVLINK_ATTR_SB_INDEX + 1] = {
+static const struct nla_policy devlink_sb_occ_snapshot_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_SB_OCC_MAX_CLEAR - do */
-static const struct nla_policy devlink_sb_occ_max_clear_nl_policy[DEVLINK_ATTR_SB_INDEX + 1] = {
+static const struct nla_policy devlink_sb_occ_max_clear_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_SB_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_ESWITCH_GET - do */
-static const struct nla_policy devlink_eswitch_get_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_eswitch_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_ESWITCH_SET - do */
-static const struct nla_policy devlink_eswitch_set_nl_policy[DEVLINK_ATTR_ESWITCH_ENCAP_MODE + 1] = {
+static const struct nla_policy devlink_eswitch_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_ESWITCH_MODE] = NLA_POLICY_MAX(NLA_U16, 2),
[DEVLINK_ATTR_ESWITCH_INLINE_MODE] = NLA_POLICY_MAX(NLA_U8, 3),
[DEVLINK_ATTR_ESWITCH_ENCAP_MODE] = NLA_POLICY_MAX(NLA_U8, 1),
};
/* DEVLINK_CMD_DPIPE_TABLE_GET - do */
-static const struct nla_policy devlink_dpipe_table_get_nl_policy[DEVLINK_ATTR_DPIPE_TABLE_NAME + 1] = {
+static const struct nla_policy devlink_dpipe_table_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_DPIPE_TABLE_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_DPIPE_ENTRIES_GET - do */
-static const struct nla_policy devlink_dpipe_entries_get_nl_policy[DEVLINK_ATTR_DPIPE_TABLE_NAME + 1] = {
+static const struct nla_policy devlink_dpipe_entries_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_DPIPE_TABLE_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_DPIPE_HEADERS_GET - do */
-static const struct nla_policy devlink_dpipe_headers_get_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_dpipe_headers_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_DPIPE_TABLE_COUNTERS_SET - do */
-static const struct nla_policy devlink_dpipe_table_counters_set_nl_policy[DEVLINK_ATTR_DPIPE_TABLE_COUNTERS_ENABLED + 1] = {
+static const struct nla_policy devlink_dpipe_table_counters_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_DPIPE_TABLE_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DPIPE_TABLE_COUNTERS_ENABLED] = { .type = NLA_U8, },
};
/* DEVLINK_CMD_RESOURCE_SET - do */
-static const struct nla_policy devlink_resource_set_nl_policy[DEVLINK_ATTR_RESOURCE_SIZE + 1] = {
+static const struct nla_policy devlink_resource_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_RESOURCE_ID] = { .type = NLA_U64, },
[DEVLINK_ATTR_RESOURCE_SIZE] = { .type = NLA_U64, },
};
/* DEVLINK_CMD_RESOURCE_DUMP - do */
-static const struct nla_policy devlink_resource_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_resource_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_RELOAD - do */
-static const struct nla_policy devlink_reload_nl_policy[DEVLINK_ATTR_RELOAD_LIMITS + 1] = {
+static const struct nla_policy devlink_reload_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_RELOAD_ACTION] = NLA_POLICY_RANGE(NLA_U8, 1, 2),
[DEVLINK_ATTR_RELOAD_LIMITS] = NLA_POLICY_BITFIELD32(6),
[DEVLINK_ATTR_NETNS_PID] = { .type = NLA_U32, },
@@ -289,22 +319,25 @@ static const struct nla_policy devlink_reload_nl_policy[DEVLINK_ATTR_RELOAD_LIMI
};
/* DEVLINK_CMD_PARAM_GET - do */
-static const struct nla_policy devlink_param_get_do_nl_policy[DEVLINK_ATTR_PARAM_NAME + 1] = {
+static const struct nla_policy devlink_param_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PARAM_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_PARAM_GET - dump */
-static const struct nla_policy devlink_param_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_param_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_PARAM_SET - do */
-static const struct nla_policy devlink_param_set_nl_policy[DEVLINK_ATTR_PARAM_RESET_DEFAULT + 1] = {
+static const struct nla_policy devlink_param_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PARAM_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_PARAM_TYPE] = NLA_POLICY_VALIDATE_FN(NLA_U8, &devlink_attr_param_type_validate),
[DEVLINK_ATTR_PARAM_VALUE_CMODE] = NLA_POLICY_MAX(NLA_U8, 2),
@@ -312,41 +345,46 @@ static const struct nla_policy devlink_param_set_nl_policy[DEVLINK_ATTR_PARAM_RE
};
/* DEVLINK_CMD_REGION_GET - do */
-static const struct nla_policy devlink_region_get_do_nl_policy[DEVLINK_ATTR_REGION_NAME + 1] = {
+static const struct nla_policy devlink_region_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_REGION_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_REGION_GET - dump */
-static const struct nla_policy devlink_region_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_region_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_REGION_NEW - do */
-static const struct nla_policy devlink_region_new_nl_policy[DEVLINK_ATTR_REGION_SNAPSHOT_ID + 1] = {
+static const struct nla_policy devlink_region_new_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_REGION_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_REGION_SNAPSHOT_ID] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_REGION_DEL - do */
-static const struct nla_policy devlink_region_del_nl_policy[DEVLINK_ATTR_REGION_SNAPSHOT_ID + 1] = {
+static const struct nla_policy devlink_region_del_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_REGION_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_REGION_SNAPSHOT_ID] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_REGION_READ - dump */
-static const struct nla_policy devlink_region_read_nl_policy[DEVLINK_ATTR_REGION_DIRECT + 1] = {
+static const struct nla_policy devlink_region_read_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_REGION_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_REGION_SNAPSHOT_ID] = { .type = NLA_U32, },
@@ -356,44 +394,50 @@ static const struct nla_policy devlink_region_read_nl_policy[DEVLINK_ATTR_REGION
};
/* DEVLINK_CMD_PORT_PARAM_GET - do */
-static const struct nla_policy devlink_port_param_get_nl_policy[DEVLINK_ATTR_PORT_INDEX + 1] = {
+static const struct nla_policy devlink_port_param_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_PORT_PARAM_SET - do */
-static const struct nla_policy devlink_port_param_set_nl_policy[DEVLINK_ATTR_PORT_INDEX + 1] = {
+static const struct nla_policy devlink_port_param_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_INFO_GET - do */
-static const struct nla_policy devlink_info_get_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_info_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_HEALTH_REPORTER_GET - do */
-static const struct nla_policy devlink_health_reporter_get_do_nl_policy[DEVLINK_ATTR_HEALTH_REPORTER_NAME + 1] = {
+static const struct nla_policy devlink_health_reporter_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_HEALTH_REPORTER_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_HEALTH_REPORTER_GET - dump */
-static const struct nla_policy devlink_health_reporter_get_dump_nl_policy[DEVLINK_ATTR_PORT_INDEX + 1] = {
+static const struct nla_policy devlink_health_reporter_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_HEALTH_REPORTER_SET - do */
-static const struct nla_policy devlink_health_reporter_set_nl_policy[DEVLINK_ATTR_HEALTH_REPORTER_BURST_PERIOD + 1] = {
+static const struct nla_policy devlink_health_reporter_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_HEALTH_REPORTER_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_HEALTH_REPORTER_GRACEFUL_PERIOD] = { .type = NLA_U64, },
@@ -403,137 +447,155 @@ static const struct nla_policy devlink_health_reporter_set_nl_policy[DEVLINK_ATT
};
/* DEVLINK_CMD_HEALTH_REPORTER_RECOVER - do */
-static const struct nla_policy devlink_health_reporter_recover_nl_policy[DEVLINK_ATTR_HEALTH_REPORTER_NAME + 1] = {
+static const struct nla_policy devlink_health_reporter_recover_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_HEALTH_REPORTER_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_HEALTH_REPORTER_DIAGNOSE - do */
-static const struct nla_policy devlink_health_reporter_diagnose_nl_policy[DEVLINK_ATTR_HEALTH_REPORTER_NAME + 1] = {
+static const struct nla_policy devlink_health_reporter_diagnose_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_HEALTH_REPORTER_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_HEALTH_REPORTER_DUMP_GET - dump */
-static const struct nla_policy devlink_health_reporter_dump_get_nl_policy[DEVLINK_ATTR_HEALTH_REPORTER_NAME + 1] = {
+static const struct nla_policy devlink_health_reporter_dump_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_HEALTH_REPORTER_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_HEALTH_REPORTER_DUMP_CLEAR - do */
-static const struct nla_policy devlink_health_reporter_dump_clear_nl_policy[DEVLINK_ATTR_HEALTH_REPORTER_NAME + 1] = {
+static const struct nla_policy devlink_health_reporter_dump_clear_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_HEALTH_REPORTER_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_FLASH_UPDATE - do */
-static const struct nla_policy devlink_flash_update_nl_policy[DEVLINK_ATTR_FLASH_UPDATE_OVERWRITE_MASK + 1] = {
+static const struct nla_policy devlink_flash_update_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_FLASH_UPDATE_FILE_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_FLASH_UPDATE_COMPONENT] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_FLASH_UPDATE_OVERWRITE_MASK] = NLA_POLICY_BITFIELD32(3),
};
/* DEVLINK_CMD_TRAP_GET - do */
-static const struct nla_policy devlink_trap_get_do_nl_policy[DEVLINK_ATTR_TRAP_NAME + 1] = {
+static const struct nla_policy devlink_trap_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_TRAP_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_TRAP_GET - dump */
-static const struct nla_policy devlink_trap_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_trap_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_TRAP_SET - do */
-static const struct nla_policy devlink_trap_set_nl_policy[DEVLINK_ATTR_TRAP_ACTION + 1] = {
+static const struct nla_policy devlink_trap_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_TRAP_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_TRAP_ACTION] = NLA_POLICY_MAX(NLA_U8, 2),
};
/* DEVLINK_CMD_TRAP_GROUP_GET - do */
-static const struct nla_policy devlink_trap_group_get_do_nl_policy[DEVLINK_ATTR_TRAP_GROUP_NAME + 1] = {
+static const struct nla_policy devlink_trap_group_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_TRAP_GROUP_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_TRAP_GROUP_GET - dump */
-static const struct nla_policy devlink_trap_group_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_trap_group_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_TRAP_GROUP_SET - do */
-static const struct nla_policy devlink_trap_group_set_nl_policy[DEVLINK_ATTR_TRAP_POLICER_ID + 1] = {
+static const struct nla_policy devlink_trap_group_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_TRAP_GROUP_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_TRAP_ACTION] = NLA_POLICY_MAX(NLA_U8, 2),
[DEVLINK_ATTR_TRAP_POLICER_ID] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_TRAP_POLICER_GET - do */
-static const struct nla_policy devlink_trap_policer_get_do_nl_policy[DEVLINK_ATTR_TRAP_POLICER_ID + 1] = {
+static const struct nla_policy devlink_trap_policer_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_TRAP_POLICER_ID] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_TRAP_POLICER_GET - dump */
-static const struct nla_policy devlink_trap_policer_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_trap_policer_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_TRAP_POLICER_SET - do */
-static const struct nla_policy devlink_trap_policer_set_nl_policy[DEVLINK_ATTR_TRAP_POLICER_BURST + 1] = {
+static const struct nla_policy devlink_trap_policer_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_TRAP_POLICER_ID] = { .type = NLA_U32, },
[DEVLINK_ATTR_TRAP_POLICER_RATE] = { .type = NLA_U64, },
[DEVLINK_ATTR_TRAP_POLICER_BURST] = { .type = NLA_U64, },
};
/* DEVLINK_CMD_HEALTH_REPORTER_TEST - do */
-static const struct nla_policy devlink_health_reporter_test_nl_policy[DEVLINK_ATTR_HEALTH_REPORTER_NAME + 1] = {
+static const struct nla_policy devlink_health_reporter_test_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_HEALTH_REPORTER_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_RATE_GET - do */
-static const struct nla_policy devlink_rate_get_do_nl_policy[DEVLINK_ATTR_RATE_NODE_NAME + 1] = {
+static const struct nla_policy devlink_rate_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_RATE_NODE_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_RATE_GET - dump */
-static const struct nla_policy devlink_rate_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_rate_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_RATE_SET - do */
-static const struct nla_policy devlink_rate_set_nl_policy[DEVLINK_ATTR_RATE_TC_BWS + 1] = {
+static const struct nla_policy devlink_rate_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_RATE_NODE_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_RATE_TX_SHARE] = { .type = NLA_U64, },
[DEVLINK_ATTR_RATE_TX_MAX] = { .type = NLA_U64, },
@@ -544,9 +606,10 @@ static const struct nla_policy devlink_rate_set_nl_policy[DEVLINK_ATTR_RATE_TC_B
};
/* DEVLINK_CMD_RATE_NEW - do */
-static const struct nla_policy devlink_rate_new_nl_policy[DEVLINK_ATTR_RATE_TC_BWS + 1] = {
+static const struct nla_policy devlink_rate_new_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_RATE_NODE_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_RATE_TX_SHARE] = { .type = NLA_U64, },
[DEVLINK_ATTR_RATE_TX_MAX] = { .type = NLA_U64, },
@@ -557,50 +620,57 @@ static const struct nla_policy devlink_rate_new_nl_policy[DEVLINK_ATTR_RATE_TC_B
};
/* DEVLINK_CMD_RATE_DEL - do */
-static const struct nla_policy devlink_rate_del_nl_policy[DEVLINK_ATTR_RATE_NODE_NAME + 1] = {
+static const struct nla_policy devlink_rate_del_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_RATE_NODE_NAME] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_LINECARD_GET - do */
-static const struct nla_policy devlink_linecard_get_do_nl_policy[DEVLINK_ATTR_LINECARD_INDEX + 1] = {
+static const struct nla_policy devlink_linecard_get_do_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_LINECARD_INDEX] = { .type = NLA_U32, },
};
/* DEVLINK_CMD_LINECARD_GET - dump */
-static const struct nla_policy devlink_linecard_get_dump_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_linecard_get_dump_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_LINECARD_SET - do */
-static const struct nla_policy devlink_linecard_set_nl_policy[DEVLINK_ATTR_LINECARD_TYPE + 1] = {
+static const struct nla_policy devlink_linecard_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_LINECARD_INDEX] = { .type = NLA_U32, },
[DEVLINK_ATTR_LINECARD_TYPE] = { .type = NLA_NUL_STRING, },
};
/* DEVLINK_CMD_SELFTESTS_GET - do */
-static const struct nla_policy devlink_selftests_get_nl_policy[DEVLINK_ATTR_DEV_NAME + 1] = {
+static const struct nla_policy devlink_selftests_get_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
};
/* DEVLINK_CMD_SELFTESTS_RUN - do */
-static const struct nla_policy devlink_selftests_run_nl_policy[DEVLINK_ATTR_SELFTESTS + 1] = {
+static const struct nla_policy devlink_selftests_run_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_SELFTESTS] = NLA_POLICY_NESTED(devlink_dl_selftest_id_nl_policy),
};
/* DEVLINK_CMD_NOTIFY_FILTER_SET - do */
-static const struct nla_policy devlink_notify_filter_set_nl_policy[DEVLINK_ATTR_PORT_INDEX + 1] = {
+static const struct nla_policy devlink_notify_filter_set_nl_policy[DEVLINK_ATTR_INDEX + 1] = {
[DEVLINK_ATTR_BUS_NAME] = { .type = NLA_NUL_STRING, },
[DEVLINK_ATTR_DEV_NAME] = { .type = NLA_NUL_STRING, },
+ [DEVLINK_ATTR_INDEX] = { .type = NLA_UINT, },
[DEVLINK_ATTR_PORT_INDEX] = { .type = NLA_U32, },
};
@@ -613,7 +683,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_get_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
@@ -629,14 +699,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_port_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_port_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_PORT_GET,
.dumpit = devlink_nl_port_get_dumpit,
.policy = devlink_port_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -646,7 +716,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_port_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_port_set_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_FUNCTION,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -656,7 +726,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_port_new_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_port_new_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_PCI_SF_NUMBER,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -666,7 +736,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_port_del_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_port_del_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -676,7 +746,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_port_split_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_port_split_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_SPLIT_COUNT,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -686,7 +756,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_port_unsplit_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_port_unsplit_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -696,14 +766,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_SB_GET,
.dumpit = devlink_nl_sb_get_dumpit,
.policy = devlink_sb_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -713,14 +783,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_pool_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_pool_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_POOL_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_SB_POOL_GET,
.dumpit = devlink_nl_sb_pool_get_dumpit,
.policy = devlink_sb_pool_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -730,7 +800,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_pool_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_pool_set_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_POOL_THRESHOLD_TYPE,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -740,14 +810,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_port_pool_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_port_pool_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_POOL_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_SB_PORT_POOL_GET,
.dumpit = devlink_nl_sb_port_pool_get_dumpit,
.policy = devlink_sb_port_pool_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -757,7 +827,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_port_pool_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_port_pool_set_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_THRESHOLD,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -767,14 +837,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_tc_pool_bind_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_tc_pool_bind_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_TC_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_SB_TC_POOL_BIND_GET,
.dumpit = devlink_nl_sb_tc_pool_bind_get_dumpit,
.policy = devlink_sb_tc_pool_bind_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -784,7 +854,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_tc_pool_bind_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_tc_pool_bind_set_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_TC_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -794,7 +864,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_occ_snapshot_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_occ_snapshot_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -804,7 +874,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_sb_occ_max_clear_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_sb_occ_max_clear_nl_policy,
- .maxattr = DEVLINK_ATTR_SB_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -814,7 +884,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_eswitch_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_eswitch_get_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -824,7 +894,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_eswitch_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_eswitch_set_nl_policy,
- .maxattr = DEVLINK_ATTR_ESWITCH_ENCAP_MODE,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -834,7 +904,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_dpipe_table_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_dpipe_table_get_nl_policy,
- .maxattr = DEVLINK_ATTR_DPIPE_TABLE_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
@@ -844,7 +914,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_dpipe_entries_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_dpipe_entries_get_nl_policy,
- .maxattr = DEVLINK_ATTR_DPIPE_TABLE_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
@@ -854,7 +924,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_dpipe_headers_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_dpipe_headers_get_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
@@ -864,7 +934,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_dpipe_table_counters_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_dpipe_table_counters_set_nl_policy,
- .maxattr = DEVLINK_ATTR_DPIPE_TABLE_COUNTERS_ENABLED,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -874,7 +944,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_resource_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_resource_set_nl_policy,
- .maxattr = DEVLINK_ATTR_RESOURCE_SIZE,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -884,7 +954,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_resource_dump_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_resource_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
@@ -894,7 +964,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_reload_doit,
.post_doit = devlink_nl_post_doit_dev_lock,
.policy = devlink_reload_nl_policy,
- .maxattr = DEVLINK_ATTR_RELOAD_LIMITS,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -904,14 +974,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_param_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_param_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_PARAM_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_PARAM_GET,
.dumpit = devlink_nl_param_get_dumpit,
.policy = devlink_param_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -921,7 +991,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_param_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_param_set_nl_policy,
- .maxattr = DEVLINK_ATTR_PARAM_RESET_DEFAULT,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -931,14 +1001,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_region_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_region_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_REGION_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_REGION_GET,
.dumpit = devlink_nl_region_get_dumpit,
.policy = devlink_region_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -948,7 +1018,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_region_new_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_region_new_nl_policy,
- .maxattr = DEVLINK_ATTR_REGION_SNAPSHOT_ID,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -958,7 +1028,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_region_del_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_region_del_nl_policy,
- .maxattr = DEVLINK_ATTR_REGION_SNAPSHOT_ID,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -966,7 +1036,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.validate = GENL_DONT_VALIDATE_DUMP_STRICT,
.dumpit = devlink_nl_region_read_dumpit,
.policy = devlink_region_read_nl_policy,
- .maxattr = DEVLINK_ATTR_REGION_DIRECT,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DUMP,
},
{
@@ -976,7 +1046,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_port_param_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_port_param_get_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
@@ -992,7 +1062,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_port_param_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_port_param_set_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1002,7 +1072,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_info_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_info_get_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
@@ -1018,14 +1088,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_health_reporter_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_health_reporter_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_HEALTH_REPORTER_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_HEALTH_REPORTER_GET,
.dumpit = devlink_nl_health_reporter_get_dumpit,
.policy = devlink_health_reporter_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -1035,7 +1105,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_health_reporter_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_health_reporter_set_nl_policy,
- .maxattr = DEVLINK_ATTR_HEALTH_REPORTER_BURST_PERIOD,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1045,7 +1115,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_health_reporter_recover_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_health_reporter_recover_nl_policy,
- .maxattr = DEVLINK_ATTR_HEALTH_REPORTER_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1055,7 +1125,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_health_reporter_diagnose_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_health_reporter_diagnose_nl_policy,
- .maxattr = DEVLINK_ATTR_HEALTH_REPORTER_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1063,7 +1133,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.validate = GENL_DONT_VALIDATE_DUMP_STRICT,
.dumpit = devlink_nl_health_reporter_dump_get_dumpit,
.policy = devlink_health_reporter_dump_get_nl_policy,
- .maxattr = DEVLINK_ATTR_HEALTH_REPORTER_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DUMP,
},
{
@@ -1073,7 +1143,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_health_reporter_dump_clear_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_health_reporter_dump_clear_nl_policy,
- .maxattr = DEVLINK_ATTR_HEALTH_REPORTER_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1083,7 +1153,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_flash_update_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_flash_update_nl_policy,
- .maxattr = DEVLINK_ATTR_FLASH_UPDATE_OVERWRITE_MASK,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1093,14 +1163,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_trap_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_trap_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_TRAP_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_TRAP_GET,
.dumpit = devlink_nl_trap_get_dumpit,
.policy = devlink_trap_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -1110,7 +1180,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_trap_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_trap_set_nl_policy,
- .maxattr = DEVLINK_ATTR_TRAP_ACTION,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1120,14 +1190,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_trap_group_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_trap_group_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_TRAP_GROUP_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_TRAP_GROUP_GET,
.dumpit = devlink_nl_trap_group_get_dumpit,
.policy = devlink_trap_group_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -1137,7 +1207,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_trap_group_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_trap_group_set_nl_policy,
- .maxattr = DEVLINK_ATTR_TRAP_POLICER_ID,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1147,14 +1217,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_trap_policer_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_trap_policer_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_TRAP_POLICER_ID,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_TRAP_POLICER_GET,
.dumpit = devlink_nl_trap_policer_get_dumpit,
.policy = devlink_trap_policer_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -1164,7 +1234,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_trap_policer_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_trap_policer_set_nl_policy,
- .maxattr = DEVLINK_ATTR_TRAP_POLICER_BURST,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1174,7 +1244,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_health_reporter_test_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_health_reporter_test_nl_policy,
- .maxattr = DEVLINK_ATTR_HEALTH_REPORTER_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1184,14 +1254,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_rate_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_rate_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_RATE_NODE_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_RATE_GET,
.dumpit = devlink_nl_rate_get_dumpit,
.policy = devlink_rate_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -1201,7 +1271,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_rate_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_rate_set_nl_policy,
- .maxattr = DEVLINK_ATTR_RATE_TC_BWS,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1211,7 +1281,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_rate_new_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_rate_new_nl_policy,
- .maxattr = DEVLINK_ATTR_RATE_TC_BWS,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1221,7 +1291,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_rate_del_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_rate_del_nl_policy,
- .maxattr = DEVLINK_ATTR_RATE_NODE_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1231,14 +1301,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_linecard_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_linecard_get_do_nl_policy,
- .maxattr = DEVLINK_ATTR_LINECARD_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_LINECARD_GET,
.dumpit = devlink_nl_linecard_get_dumpit,
.policy = devlink_linecard_get_dump_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DUMP,
},
{
@@ -1248,7 +1318,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_linecard_set_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_linecard_set_nl_policy,
- .maxattr = DEVLINK_ATTR_LINECARD_TYPE,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
@@ -1258,7 +1328,7 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_selftests_get_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_selftests_get_nl_policy,
- .maxattr = DEVLINK_ATTR_DEV_NAME,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
{
@@ -1274,14 +1344,14 @@ const struct genl_split_ops devlink_nl_ops[74] = {
.doit = devlink_nl_selftests_run_doit,
.post_doit = devlink_nl_post_doit,
.policy = devlink_selftests_run_nl_policy,
- .maxattr = DEVLINK_ATTR_SELFTESTS,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO,
},
{
.cmd = DEVLINK_CMD_NOTIFY_FILTER_SET,
.doit = devlink_nl_notify_filter_set_doit,
.policy = devlink_notify_filter_set_nl_policy,
- .maxattr = DEVLINK_ATTR_PORT_INDEX,
+ .maxattr = DEVLINK_ATTR_INDEX,
.flags = GENL_CMD_CAP_DO,
},
};
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 03/10] devlink: avoid extra iterations when found devlink is not registered
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Since the one found is not registered, very unlikely another one with
the same bus_name/dev_name is going to be found. Stop right away and
prepare common "found" path for the follow-up patch.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
net/devlink/netlink.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/net/devlink/netlink.c b/net/devlink/netlink.c
index 3f73ced2d879..a517d42c7b96 100644
--- a/net/devlink/netlink.c
+++ b/net/devlink/netlink.c
@@ -194,16 +194,20 @@ devlink_get_from_attrs_lock(struct net *net, struct nlattr **attrs,
devlinks_xa_for_each_registered_get(net, index, devlink) {
if (strcmp(devlink->bus_name, busname) == 0 &&
- strcmp(devlink->dev_name, devname) == 0) {
- devl_dev_lock(devlink, dev_lock);
- if (devl_is_registered(devlink))
- return devlink;
- devl_dev_unlock(devlink, dev_lock);
- }
+ strcmp(devlink->dev_name, devname) == 0)
+ goto found;
devlink_put(devlink);
}
return ERR_PTR(-ENODEV);
+
+found:
+ devl_dev_lock(devlink, dev_lock);
+ if (devl_is_registered(devlink))
+ return devlink;
+ devl_dev_unlock(devlink, dev_lock);
+ devlink_put(devlink);
+ return ERR_PTR(-ENODEV);
}
static int __devlink_nl_pre_doit(struct sk_buff *skb, struct genl_info *info,
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 02/10] devlink: store bus_name and dev_name pointers in struct devlink
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Currently, bus_name and dev_name are obtained from the parent dev
instance every time they are needed.
In preparation to dev-less devlinks, store bus_name and dev_name
pointers in struct devlink during devlink_alloc_ns() and use them
throughout.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
include/net/devlink.h | 2 ++
include/trace/events/devlink.h | 24 ++++++++++++------------
net/devlink/core.c | 14 ++++++++++++++
net/devlink/devl_internal.h | 10 ++++++----
net/devlink/netlink.c | 4 ++--
net/devlink/port.c | 4 ++--
6 files changed, 38 insertions(+), 20 deletions(-)
diff --git a/include/net/devlink.h b/include/net/devlink.h
index cb839e0435a1..0afb0958b910 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -1611,6 +1611,8 @@ struct devlink_ops {
void *devlink_priv(struct devlink *devlink);
struct devlink *priv_to_devlink(void *priv);
struct device *devlink_to_dev(const struct devlink *devlink);
+const char *devlink_bus_name(const struct devlink *devlink);
+const char *devlink_dev_name(const struct devlink *devlink);
/* Devlink instance explicit locking */
void devl_lock(struct devlink *devlink);
diff --git a/include/trace/events/devlink.h b/include/trace/events/devlink.h
index f241e204fe6b..32304ce9ad15 100644
--- a/include/trace/events/devlink.h
+++ b/include/trace/events/devlink.h
@@ -21,8 +21,8 @@ TRACE_EVENT(devlink_hwmsg,
TP_ARGS(devlink, incoming, type, buf, len),
TP_STRUCT__entry(
- __string(bus_name, devlink_to_dev(devlink)->bus->name)
- __string(dev_name, dev_name(devlink_to_dev(devlink)))
+ __string(bus_name, devlink_bus_name(devlink))
+ __string(dev_name, devlink_dev_name(devlink))
__string(driver_name, devlink_to_dev(devlink)->driver->name)
__field(bool, incoming)
__field(unsigned long, type)
@@ -55,8 +55,8 @@ TRACE_EVENT(devlink_hwerr,
TP_ARGS(devlink, err, msg),
TP_STRUCT__entry(
- __string(bus_name, devlink_to_dev(devlink)->bus->name)
- __string(dev_name, dev_name(devlink_to_dev(devlink)))
+ __string(bus_name, devlink_bus_name(devlink))
+ __string(dev_name, devlink_dev_name(devlink))
__string(driver_name, devlink_to_dev(devlink)->driver->name)
__field(int, err)
__string(msg, msg)
@@ -85,8 +85,8 @@ TRACE_EVENT(devlink_health_report,
TP_ARGS(devlink, reporter_name, msg),
TP_STRUCT__entry(
- __string(bus_name, devlink_to_dev(devlink)->bus->name)
- __string(dev_name, dev_name(devlink_to_dev(devlink)))
+ __string(bus_name, devlink_bus_name(devlink))
+ __string(dev_name, devlink_dev_name(devlink))
__string(driver_name, devlink_to_dev(devlink)->driver->name)
__string(reporter_name, reporter_name)
__string(msg, msg)
@@ -116,8 +116,8 @@ TRACE_EVENT(devlink_health_recover_aborted,
TP_ARGS(devlink, reporter_name, health_state, time_since_last_recover),
TP_STRUCT__entry(
- __string(bus_name, devlink_to_dev(devlink)->bus->name)
- __string(dev_name, dev_name(devlink_to_dev(devlink)))
+ __string(bus_name, devlink_bus_name(devlink))
+ __string(dev_name, devlink_dev_name(devlink))
__string(driver_name, devlink_to_dev(devlink)->driver->name)
__string(reporter_name, reporter_name)
__field(bool, health_state)
@@ -150,8 +150,8 @@ TRACE_EVENT(devlink_health_reporter_state_update,
TP_ARGS(devlink, reporter_name, new_state),
TP_STRUCT__entry(
- __string(bus_name, devlink_to_dev(devlink)->bus->name)
- __string(dev_name, dev_name(devlink_to_dev(devlink)))
+ __string(bus_name, devlink_bus_name(devlink))
+ __string(dev_name, devlink_dev_name(devlink))
__string(driver_name, devlink_to_dev(devlink)->driver->name)
__string(reporter_name, reporter_name)
__field(u8, new_state)
@@ -181,8 +181,8 @@ TRACE_EVENT(devlink_trap_report,
TP_ARGS(devlink, skb, metadata),
TP_STRUCT__entry(
- __string(bus_name, devlink_to_dev(devlink)->bus->name)
- __string(dev_name, dev_name(devlink_to_dev(devlink)))
+ __string(bus_name, devlink_bus_name(devlink))
+ __string(dev_name, devlink_dev_name(devlink))
__string(driver_name, devlink_to_dev(devlink)->driver->name)
__string(trap_name, metadata->trap_name)
__string(trap_group_name, metadata->trap_group_name)
diff --git a/net/devlink/core.c b/net/devlink/core.c
index da56e2b8afc1..49f856beb8b2 100644
--- a/net/devlink/core.c
+++ b/net/devlink/core.c
@@ -248,6 +248,18 @@ struct device *devlink_to_dev(const struct devlink *devlink)
}
EXPORT_SYMBOL_GPL(devlink_to_dev);
+const char *devlink_bus_name(const struct devlink *devlink)
+{
+ return devlink->bus_name;
+}
+EXPORT_SYMBOL_GPL(devlink_bus_name);
+
+const char *devlink_dev_name(const struct devlink *devlink)
+{
+ return devlink->dev_name;
+}
+EXPORT_SYMBOL_GPL(devlink_dev_name);
+
struct net *devlink_net(const struct devlink *devlink)
{
return read_pnet(&devlink->_net);
@@ -428,6 +440,8 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops,
goto err_xa_alloc;
devlink->dev = get_device(dev);
+ devlink->bus_name = dev->bus->name;
+ devlink->dev_name = dev_name(dev);
devlink->ops = ops;
xa_init_flags(&devlink->ports, XA_FLAGS_ALLOC);
xa_init_flags(&devlink->params, XA_FLAGS_ALLOC);
diff --git a/net/devlink/devl_internal.h b/net/devlink/devl_internal.h
index 31fa98af418e..d43819c6b452 100644
--- a/net/devlink/devl_internal.h
+++ b/net/devlink/devl_internal.h
@@ -32,6 +32,8 @@ struct devlink_dev_stats {
struct devlink {
u32 index;
+ const char *bus_name;
+ const char *dev_name;
struct xarray ports;
struct list_head rate_list;
struct list_head sb_list;
@@ -174,9 +176,9 @@ devlink_dump_state(struct netlink_callback *cb)
static inline int
devlink_nl_put_handle(struct sk_buff *msg, struct devlink *devlink)
{
- if (nla_put_string(msg, DEVLINK_ATTR_BUS_NAME, devlink->dev->bus->name))
+ if (nla_put_string(msg, DEVLINK_ATTR_BUS_NAME, devlink->bus_name))
return -EMSGSIZE;
- if (nla_put_string(msg, DEVLINK_ATTR_DEV_NAME, dev_name(devlink->dev)))
+ if (nla_put_string(msg, DEVLINK_ATTR_DEV_NAME, devlink->dev_name))
return -EMSGSIZE;
if (nla_put_uint(msg, DEVLINK_ATTR_INDEX, devlink->index))
return -EMSGSIZE;
@@ -211,8 +213,8 @@ static inline void devlink_nl_obj_desc_init(struct devlink_obj_desc *desc,
struct devlink *devlink)
{
memset(desc, 0, sizeof(*desc));
- desc->bus_name = devlink->dev->bus->name;
- desc->dev_name = dev_name(devlink->dev);
+ desc->bus_name = devlink->bus_name;
+ desc->dev_name = devlink->dev_name;
}
static inline void devlink_nl_obj_desc_port_set(struct devlink_obj_desc *desc,
diff --git a/net/devlink/netlink.c b/net/devlink/netlink.c
index 593605c1b1ef..3f73ced2d879 100644
--- a/net/devlink/netlink.c
+++ b/net/devlink/netlink.c
@@ -193,8 +193,8 @@ devlink_get_from_attrs_lock(struct net *net, struct nlattr **attrs,
devname = nla_data(attrs[DEVLINK_ATTR_DEV_NAME]);
devlinks_xa_for_each_registered_get(net, index, devlink) {
- if (strcmp(devlink->dev->bus->name, busname) == 0 &&
- strcmp(dev_name(devlink->dev), devname) == 0) {
+ if (strcmp(devlink->bus_name, busname) == 0 &&
+ strcmp(devlink->dev_name, devname) == 0) {
devl_dev_lock(devlink, dev_lock);
if (devl_is_registered(devlink))
return devlink;
diff --git a/net/devlink/port.c b/net/devlink/port.c
index 1ff609571ea4..1d4a79c6d4d3 100644
--- a/net/devlink/port.c
+++ b/net/devlink/port.c
@@ -220,8 +220,8 @@ size_t devlink_nl_port_handle_size(struct devlink_port *devlink_port)
{
struct devlink *devlink = devlink_port->devlink;
- return nla_total_size(strlen(devlink->dev->bus->name) + 1) /* DEVLINK_ATTR_BUS_NAME */
- + nla_total_size(strlen(dev_name(devlink->dev)) + 1) /* DEVLINK_ATTR_DEV_NAME */
+ return nla_total_size(strlen(devlink->bus_name) + 1) /* DEVLINK_ATTR_BUS_NAME */
+ + nla_total_size(strlen(devlink->dev_name) + 1) /* DEVLINK_ATTR_DEV_NAME */
+ nla_total_size(8) /* DEVLINK_ATTR_INDEX */
+ nla_total_size(4); /* DEVLINK_ATTR_PORT_INDEX */
}
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 01/10] devlink: expose devlink instance index over netlink
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
In-Reply-To: <20260225133422.290965-1-jiri@resnulli.us>
From: Jiri Pirko <jiri@nvidia.com>
Each devlink instance has an internally assigned index used for xarray
storage. Expose it as a new DEVLINK_ATTR_INDEX uint attribute alongside
the existing bus_name and dev_name handle.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
---
Documentation/netlink/specs/devlink.yaml | 5 +++++
include/uapi/linux/devlink.h | 2 ++
net/devlink/devl_internal.h | 2 ++
net/devlink/port.c | 1 +
4 files changed, 10 insertions(+)
diff --git a/Documentation/netlink/specs/devlink.yaml b/Documentation/netlink/specs/devlink.yaml
index 837112da6738..1bed67a0eefb 100644
--- a/Documentation/netlink/specs/devlink.yaml
+++ b/Documentation/netlink/specs/devlink.yaml
@@ -867,6 +867,10 @@ attribute-sets:
type: flag
doc: Request restoring parameter to its default value.
value: 183
+ -
+ name: index
+ type: uint
+ doc: Unique devlink instance index.
-
name: dl-dev-stats
subset-of: devlink
@@ -1311,6 +1315,7 @@ operations:
attributes:
- bus-name
- dev-name
+ - index
- reload-failed
- dev-stats
dump:
diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h
index e7d6b6d13470..1ba3436db4ae 100644
--- a/include/uapi/linux/devlink.h
+++ b/include/uapi/linux/devlink.h
@@ -642,6 +642,8 @@ enum devlink_attr {
DEVLINK_ATTR_PARAM_VALUE_DEFAULT, /* dynamic */
DEVLINK_ATTR_PARAM_RESET_DEFAULT, /* flag */
+ DEVLINK_ATTR_INDEX, /* uint */
+
/* Add new attributes above here, update the spec in
* Documentation/netlink/specs/devlink.yaml and re-generate
* net/devlink/netlink_gen.c.
diff --git a/net/devlink/devl_internal.h b/net/devlink/devl_internal.h
index 1377864383bc..31fa98af418e 100644
--- a/net/devlink/devl_internal.h
+++ b/net/devlink/devl_internal.h
@@ -178,6 +178,8 @@ devlink_nl_put_handle(struct sk_buff *msg, struct devlink *devlink)
return -EMSGSIZE;
if (nla_put_string(msg, DEVLINK_ATTR_DEV_NAME, dev_name(devlink->dev)))
return -EMSGSIZE;
+ if (nla_put_uint(msg, DEVLINK_ATTR_INDEX, devlink->index))
+ return -EMSGSIZE;
return 0;
}
diff --git a/net/devlink/port.c b/net/devlink/port.c
index 93d8a25bb920..1ff609571ea4 100644
--- a/net/devlink/port.c
+++ b/net/devlink/port.c
@@ -222,6 +222,7 @@ size_t devlink_nl_port_handle_size(struct devlink_port *devlink_port)
return nla_total_size(strlen(devlink->dev->bus->name) + 1) /* DEVLINK_ATTR_BUS_NAME */
+ nla_total_size(strlen(dev_name(devlink->dev)) + 1) /* DEVLINK_ATTR_DEV_NAME */
+ + nla_total_size(8) /* DEVLINK_ATTR_INDEX */
+ nla_total_size(4); /* DEVLINK_ATTR_PORT_INDEX */
}
--
2.51.1
^ permalink raw reply related
* [PATCH net-next v2 00/10] devlink: introduce shared devlink instance for PFs on same chip
From: Jiri Pirko @ 2026-02-25 13:34 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, horms, donald.hunter, corbet,
skhan, saeedm, leon, tariqt, mbloch, przemyslaw.kitszel, mschmidt,
andrew+netdev, rostedt, mhiramat, mathieu.desnoyers, chuck.lever,
matttbe, cjubran, daniel.zahka, linux-doc, linux-rdma,
linux-trace-kernel
From: Jiri Pirko <jiri@nvidia.com>
Multiple PFs on a network adapter often reside on the same physical
chip, running a single firmware. Some resources and configurations
are inherently shared among these PFs - PTP clocks, VF group rates,
firmware parameters, and others. Today there is no good object in
the devlink model to attach these chip-wide configuration knobs to.
Drivers resort to workarounds like pinning shared state to PF0 or
maintaining ad-hoc internal structures (e.g., ice_adapter) that are
invisible to userspace.
This problem was discussed extensively starting with Przemek Kitszel's
"whole device devlink instance" RFC for the ice driver [1]. Several
approaches for representing the parent instance were considered:
using a partial PCI BDF as the dev_name (breaks when PFs have different
BDFs in VMs), creating a per-driver bus, using auxiliary devices, or
using faux devices. All of these required a backing struct device for
the parent devlink instance, which does not naturally exist - there is
no PCI device that represents the chip as a whole.
This patchset takes a different approach: allow devlink instances to
exist without any backing struct device. The instance is identified
purely by its internal index, exposed over devlin netlink. This avoids
fabricating fake devices and keeps the devlink handle semantics clean.
The first seven patches prepare the devlink core for device-less
instances by decoupling the handle from the parent device. The last
three introduce the shared devlink infrastructure and its first user
in the mlx5 driver.
Example output showing the shared instance and nesting:
pci/0000:08:00.0: index 0
nested_devlink:
auxiliary/mlx5_core.eth.0
devlink_index/1: index 1
nested_devlink:
pci/0000:08:00.0
pci/0000:08:00.1
auxiliary/mlx5_core.eth.0: index 2
pci/0000:08:00.1: index 3
nested_devlink:
auxiliary/mlx5_core.eth.1
auxiliary/mlx5_core.eth.1: index 4
[1] https://lore.kernel.org/netdev/20250219164410.35665-1-przemyslaw.kitszel@intel.com/
---
Decoupled from "devlink and mlx5: Support cross-function rate scheduling"
patchset to maintain 15-patches limit.
See individual patches for changelog.
Jiri Pirko (10):
devlink: expose devlink instance index over netlink
devlink: store bus_name and dev_name pointers in struct devlink
devlink: avoid extra iterations when found devlink is not registered
devlink: allow to use devlink index as a command handle
devlink: support index-based lookup via bus_name/dev_name handle
devlink: add devlink_dev_driver_name() helper and use it in trace
events
devlink: allow devlink instance allocation without a backing device
devlink: introduce shared devlink instance for PFs on same chip
documentation: networking: add shared devlink documentation
net/mlx5: Add a shared devlink instance for PFs on same chip
Documentation/netlink/specs/devlink.yaml | 56 +++
.../networking/devlink/devlink-shared.rst | 89 +++++
Documentation/networking/devlink/index.rst | 1 +
.../net/ethernet/mellanox/mlx5/core/Makefile | 5 +-
.../net/ethernet/mellanox/mlx5/core/main.c | 17 +
.../ethernet/mellanox/mlx5/core/sh_devlink.c | 62 ++++
.../ethernet/mellanox/mlx5/core/sh_devlink.h | 12 +
include/linux/mlx5/driver.h | 1 +
include/net/devlink.h | 9 +
include/trace/events/devlink.h | 36 +-
include/uapi/linux/devlink.h | 4 +
net/devlink/Makefile | 2 +-
net/devlink/core.c | 59 ++-
net/devlink/dev.c | 11 +-
net/devlink/devl_internal.h | 17 +-
net/devlink/netlink.c | 38 +-
net/devlink/netlink_gen.c | 350 +++++++++++-------
net/devlink/port.c | 19 +-
net/devlink/sh_dev.c | 142 +++++++
19 files changed, 738 insertions(+), 192 deletions(-)
create mode 100644 Documentation/networking/devlink/devlink-shared.rst
create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h
create mode 100644 net/devlink/sh_dev.c
--
2.51.1
^ permalink raw reply
* [PATCH] tracing: Fix use-after-free race in copy_trace_marker on instance removal
From: Sasha Levin @ 2026-02-25 13:31 UTC (permalink / raw)
To: rostedt, mhiramat
Cc: mathieu.desnoyers, linux-kernel, linux-trace-kernel, Sasha Levin
When a trace instance with copy_trace_marker enabled is removed,
__remove_instance() first iterates ZEROED_TRACE_FLAGS (which includes
COPY_MARKER), calling set_tracer_flag() -> update_marker_trace(tr, 0).
This removes the instance from the marker_copies RCU list via
list_del_init() and returns immediately.
The subsequent explicit update_marker_trace(tr, 0) call then finds
list_empty(&tr->marker_list) is true and returns false, causing
synchronize_rcu() to be skipped. The ring buffer and trace_array are
then freed while a concurrent writer in tracing_mark_write() may still
hold an RCU-protected reference, leading to use-after-free.
BUG: KASAN: slab-use-after-free in write_marker_to_buffer+0x1e7/0x610 kernel/trace/trace.c:6527
Write of size 4054 at addr ffff888103af7058 by task syz.0.277/5019
CPU: 3 UID: 0 PID: 5019 Comm: syz.0.277 Tainted: G N 7.0.0-rc1-00001-gc5447a46efed #51 PREEMPT(full)
Tainted: [N]=TEST
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1 04/01/2014
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0xba/0x110 lib/dump_stack.c:120
print_address_description mm/kasan/report.c:378 [inline]
print_report+0x156/0x4d9 mm/kasan/report.c:482
kasan_report+0xf6/0x1f0 mm/kasan/report.c:595
check_region_inline mm/kasan/generic.c:186 [inline]
kasan_check_range+0x125/0x200 mm/kasan/generic.c:200
__asan_memcpy+0x3c/0x60 mm/kasan/shadow.c:106
write_marker_to_buffer+0x1e7/0x610 kernel/trace/trace.c:6527
tracing_mark_write+0x218/0x3f0 kernel/trace/trace.c:6875
vfs_write+0x2b7/0x1070 fs/read_write.c:686
ksys_write+0x1f8/0x250 fs/read_write.c:740
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0xf3/0x700 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fdb7eb9df29
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48
c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fdb7fa81008 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00007fdb7ee15fa0 RCX: 00007fdb7eb9df29
RDX: 0000000000001000 RSI: 0000200000000300 RDI: 0000000000000003
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffec21bfd06 R14: 00007fdb7fa81ce4 R15: 00007fdb7fa61000
</TASK>
The buggy address belongs to the physical page:
page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffffffffffffffff pfn:0x103af7
flags: 0x200000000000000(node=0|zone=2)
raw: 0200000000000000 0000000000000000 dead000000000122 0000000000000000
raw: ffffffffffffffff 0000000000000000 00000001ffffffff 0000000000000000
page dumped because: kasan: bad access detected
Memory state around the buggy address:
ffff888103af7f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ffff888103af7f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>ffff888103af8000: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff888103af8080: fb fb fb fb fb fb fb fb fb fb fb fb fb fc fc fc
ffff888103af8100: fc fc fc fc fc fc fc fc fa fb fb fb fb fb fb fb
Fix this by:
1. Removing TRACE_ITER(COPY_MARKER) from ZEROED_TRACE_FLAGS so the flag
loop doesn't pre-clear it. The explicit update_marker_trace(tr, 0) +
synchronize_rcu() then correctly waits for RCU readers to finish
before freeing.
2. Replacing list_del_init() with list_del_rcu() in update_marker_trace()
for proper RCU list removal semantics. list_del_init() overwrites
entry->next to point to itself, which can cause concurrent RCU readers
to loop infinitely. list_del_rcu() preserves entry->next so readers
can safely finish their traversal. The duplicate-operation guards are
changed from list_empty() to trace_flags bit checks accordingly, since
list_del_rcu() does not reinitialize the list head.
Fixes: 7b382efd5e8a ("tracing: Allow the top level trace_marker to write into another instances")
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
kernel/trace/trace.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 23de3719f4952..fa413214da764 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -523,8 +523,7 @@ EXPORT_SYMBOL_GPL(unregister_ftrace_export);
/* trace_flags that are default zero for instances */
#define ZEROED_TRACE_FLAGS \
- (TRACE_ITER(EVENT_FORK) | TRACE_ITER(FUNC_FORK) | TRACE_ITER(TRACE_PRINTK) | \
- TRACE_ITER(COPY_MARKER))
+ (TRACE_ITER(EVENT_FORK) | TRACE_ITER(FUNC_FORK) | TRACE_ITER(TRACE_PRINTK))
/*
* The global_trace is the descriptor that holds the top-level tracing
@@ -555,7 +554,7 @@ static bool update_marker_trace(struct trace_array *tr, int enabled)
lockdep_assert_held(&event_mutex);
if (enabled) {
- if (!list_empty(&tr->marker_list))
+ if (tr->trace_flags & TRACE_ITER(COPY_MARKER))
return false;
list_add_rcu(&tr->marker_list, &marker_copies);
@@ -563,10 +562,10 @@ static bool update_marker_trace(struct trace_array *tr, int enabled)
return true;
}
- if (list_empty(&tr->marker_list))
+ if (!(tr->trace_flags & TRACE_ITER(COPY_MARKER)))
return false;
- list_del_init(&tr->marker_list);
+ list_del_rcu(&tr->marker_list);
tr->trace_flags &= ~TRACE_ITER(COPY_MARKER);
return true;
}
--
2.51.0
^ permalink raw reply related
* Re: [LSF/MM/BPF TOPIC][RFC PATCH v4 00/27] Private Memory Nodes (w/ Compressed RAM)
From: Alejandro Lucero Palau @ 2026-02-25 12:40 UTC (permalink / raw)
To: Gregory Price, lsf-pc
Cc: linux-kernel, linux-cxl, cgroups, linux-mm, linux-trace-kernel,
damon, kernel-team, gregkh, rafael, dakr, dave, jonathan.cameron,
dave.jiang, alison.schofield, vishal.l.verma, ira.weiny,
dan.j.williams, longman, akpm, david, lorenzo.stoakes,
Liam.Howlett, vbabka, rppt, surenb, mhocko, osalvador, ziy,
matthew.brost, joshua.hahnjy, rakie.kim, byungchul, ying.huang,
apopple, axelrasmussen, yuanchu, weixugc, yury.norov, linux,
mhiramat, mathieu.desnoyers, tj, hannes, mkoutny, jackmanb, sj,
baolin.wang, npache, ryan.roberts, dev.jain, baohua, lance.yang,
muchun.song, xu.xin16, chengming.zhou, jannh, linmiaohe,
nao.horiguchi, pfalcato, rientjes, shakeel.butt, riel, harry.yoo,
cl, roman.gushchin, chrisl, kasong, shikemeng, nphamcs, bhe,
zhengqi.arch, terry.bowman
In-Reply-To: <20260222084842.1824063-1-gourry@gourry.net>
On 2/22/26 08:48, Gregory Price wrote:
> Topic type: MM
>
> Presenter: Gregory Price <gourry@gourry.net>
>
> This series introduces N_MEMORY_PRIVATE, a NUMA node state for memory
> managed by the buddy allocator but excluded from normal allocations.
>
> I present it with an end-to-end Compressed RAM service (mm/cram.c)
> that would otherwise not be possible (or would be considerably more
> difficult, be device-specific, and add to the ZONE_DEVICE boondoggle).
>
>
> TL;DR
> ===
>
> N_MEMORY_PRIVATE is all about isolating NUMA nodes and then punching
> explicit holes in that isolation to do useful things we couldn't do
> before without re-implementing entire portions of mm/ in a driver.
>
>
> /* This is my memory. There are many like it, but this one is mine. */
> rc = add_private_memory_driver_managed(nid, start, size, name, flags,
> online_type, private_context);
>
> page = alloc_pages_node(nid, __GFP_PRIVATE, 0);
Hi Gregory,
I can see the nid param is just a "preferred nid" with alloc pages.
Using __GFP_PRIVATE will restrict the allocation to private nodes but I
think the idea here is:
1) I own this node
2) Do not give me memory from another private node but from mine.
Should not this be ensure somehow?
> /* Ok but I want to do something useful with it */
> static const struct node_private_ops ops = {
> .migrate_to = my_migrate_to,
> .folio_migrate = my_folio_migrate,
> .flags = NP_OPS_MIGRATION | NP_OPS_MEMPOLICY,
> };
> node_private_set_ops(nid, &ops);
>
> /* And now I can use mempolicy with my memory */
> buf = mmap(...);
> mbind(buf, len, mode, private_node, ...);
> buf[0] = 0xdeadbeef; /* Faults onto private node */
>
> /* And to be clear, no one else gets my memory */
> buf2 = malloc(4096); /* Standard allocation */
> buf2[0] = 0xdeadbeef; /* Can never land on private node */
>
> /* But i can choose to migrate it to the private node */
> move_pages(0, 1, &buf, &private_node, NULL, ...);
>
> /* And more fun things like this */
>
>
> Patchwork
> ===
> A fully working branch based on cxl/next can be found here:
> https://github.com/gourryinverse/linux/tree/private_compression
>
> A QEMU device which can inject high/low interrupts can be found here:
> https://github.com/gourryinverse/qemu/tree/compressed_cxl_clean
>
> The additional patches on these branches are CXL and DAX driver
> housecleaning only tangentially relevant to this RFC, so i've
> omitted them for the sake of trying to keep it somewhat clean
> here. Those patches should (hopefully) be going upstream anyway.
>
> Patches 1-22: Core Private Node Infrastructure
>
> Patch 1: Introduce N_MEMORY_PRIVATE scaffolding
> Patch 2: Introduce __GFP_PRIVATE
> Patch 3: Apply allocation isolation mechanisms
> Patch 4: Add N_MEMORY nodes to private fallback lists
> Patches 5-9: Filter operations not yet supported
> Patch 10: free_folio callback
> Patch 11: split_folio callback
> Patches 12-20: mm/ service opt-ins:
> Migration, Mempolicy, Demotion, Write Protect,
> Reclaim, OOM, NUMA Balancing, Compaction,
> LongTerm Pinning
> Patch 21: memory_failure callback
> Patch 22: Memory hotplug plumbing for private nodes
>
> Patch 23: mm/cram -- Compressed RAM Management
>
> Patches 24-27: CXL Driver examples
> Sysram Regions with Private node support
> Basic Driver Example: (MIGRATION | MEMPOLICY)
> Compression Driver Example (Generic)
>
>
> Background
> ===
>
> Today, drivers that want mm-like services on non-general-purpose
> memory either use ZONE_DEVICE (self-managed memory) or hotplug into
> N_MEMORY and accept the risk of uncontrolled allocation.
>
> Neither option provides what we really want - the ability to:
> 1) selectively participate in mm/ subsystems, while
> 2) isolating that memory from general purpose use.
>
> Some device-attached memory cannot be managed as fully general-purpose
> system RAM. CXL devices with inline compression, for example, may
> corrupt data or crash the machine if the compression ratio drops
> below a threshold -- we simply run out of physical memory.
>
> This is a hard problem to solve: how does an operating system deal
> with a device that basically lies about how much capacity it has?
>
> (We'll discuss that in the CRAM section)
>
>
> Core Proposal: N_MEMORY_PRIVATE
> ===
>
> Introduce N_MEMORY_PRIVATE, a NUMA node state for memory managed by
> the buddy allocator, but excluded from normal allocation paths.
>
> Private nodes:
>
> - Are filtered from zonelist fallback: all existing callers to
> get_page_from_freelist cannot reach these nodes through any
> normal fallback mechanism.
>
> - Filter allocation requests on __GFP_PRIVATE
> numa_zone_allowed() excludes them otherwise.
>
> Applies to systems with and without cpusets.
>
> GFP_PRIVATE is (__GFP_PRIVATE | __GFP_THISNODE).
>
> Services use it when they need to allocate specifically from
> a private node (e.g., CRAM allocating a destination folio).
>
> No existing allocator path sets __GFP_PRIVATE, so private nodes
> are unreachable by default.
>
> - Use standard struct page / folio. No ZONE_DEVICE, no pgmap,
> no struct page metadata limitations.
>
> - Use a node-scoped metadata structure to accomplish filtering
> and callback support.
>
> - May participate in the buddy allocator, reclaim, compaction,
> and LRU like normal memory, gated by an opt-in set of flags.
>
> The key abstraction is node_private_ops: a per-node callback table
> registered by a driver or service.
>
> Each callback is individually gated by an NP_OPS_* capability flag.
>
> A driver opts in only to the mm/ operations it needs.
>
> It is similar to ZONE_DEVICE's pgmap at a node granularity.
>
> In fact...
>
>
> Re-use of ZONE_DEVICE Hooks
> ===
>
> The callback insertion points deliberately mirror existing ZONE_DEVICE
> hooks to minimize the surface area of the mechanism.
>
> I believe this could subsume most DEVICE_COHERENT users, and greatly
> simplify the device-managed memory development process (no more
> per-driver allocator and migration code).
>
> (Also it's just "So Fresh, So Clean").
>
> The base set of callbacks introduced include:
>
> free_folio - mirrors ZONE_DEVICE's
> free_zone_device_page() hook in
> __folio_put() / folios_put_refs()
>
> folio_split - mirrors ZONE_DEVICE's
> called when a huge page is split up
>
> migrate_to - demote_folio_list() custom demotion (same
> site as ZONE_DEVICE demotion rejection)
>
> folio_migrate - called when private node folio is moved to
> another location (e.g. compaction)
>
> handle_fault - mirrors the ZONE_DEVICE fault dispatch in
> handle_pte_fault() (do_wp_page path)
>
> reclaim_policy - called by reclaim to let a driver own the
> boost lifecycle (driver can driver node reclaim)
>
> memory_failure - parallels memory_failure_dev_pagemap(),
> but for online pages that enter the normal
> hwpoison path
>
> At skip sites (mlock, madvise, KSM, user migration), a unified
> folio_is_private_managed() predicate covers both ZONE_DEVICE and
> N_MEMORY_PRIVATE folios, consolidating existing zone_device checks
> with private node checks rather than adding new ones.
>
> static inline bool folio_is_private_managed(struct folio *folio)
> {
> return folio_is_zone_device(folio) ||
> folio_is_private_node(folio);
> }
>
> Most integration points become a one-line swap:
>
> - if (folio_is_zone_device(folio))
> + if (unlikely(folio_is_private_managed(folio)))
>
>
> Where a one-line integration is insufficient, the integration is
> kept as clean as possible with zone_device, rather than simply
> adding more call-sites on top of it:
>
> static inline bool folio_managed_handle_fault(struct folio *folio,
> struct vm_fault *vmf, vm_fault_t *ret)
> {
> /* Zone device pages use swap entries; handled in do_swap_page */
> if (folio_is_zone_device(folio))
> return false;
>
> if (folio_is_private_node(folio)) {
> const struct node_private_ops *ops = folio_node_private_ops(folio);
>
> if (ops && ops->handle_fault) {
> *ret = ops->handle_fault(vmf);
> return true;
> }
> }
> return false;
> }
>
>
>
> Flag-gated behavior (NP_OPS_*) controls:
> ===
>
> We use OPS flags to denote what mm/ services we want to allow on our
> private node. I've plumbed these through so far:
>
> NP_OPS_MIGRATION - Node supports migration
> NP_OPS_MEMPOLICY - Node supports mempolicy actions
> NP_OPS_DEMOTION - Node appears in demotion target lists
> NP_OPS_PROTECT_WRITE - Node memory is read-only (wrprotect)
> NP_OPS_RECLAIM - Node supports reclaim
> NP_OPS_NUMA_BALANCING - Node supports numa balancing
> NP_OPS_COMPACTION - Node supports compaction
> NP_OPS_LONGTERM_PIN - Node supports longterm pinning
> NP_OPS_OOM_ELIGIBLE - (MIGRATION | DEMOTION), node is reachable
> as normal system ram storage, so it should
> be considered in OOM pressure calculations.
>
> I wasn't quite sure how to classify ksm, khugepaged, madvise, and
> mlock - so i have omitted those for now.
>
> Most hooks are straightforward.
>
> Including a node as a demotion-eligible target was as simple as:
>
> static void establish_demotion_targets(void)
> {
> ..... snip .....
> /*
> * Include private nodes that have opted in to demotion
> * via NP_OPS_DEMOTION. A node might have custom migrate
> */
> all_memory = node_states[N_MEMORY];
> for_each_node_state(node, N_MEMORY_PRIVATE) {
> if (node_private_has_flag(node, NP_OPS_DEMOTION))
> node_set(node, all_memory);
> }
> ..... snip .....
> }
>
> The Migration and Mempolicy support are the two most complex pieces,
> and most useful things are built on top of Migration (meaning the
> remaining implementations are usually simple).
>
>
> Private Node Hotplug Lifecycle
> ===
>
> Registration follows a strict order enforced by
> add_private_memory_driver_managed():
>
> 1. Driver calls add_private_memory_driver_managed(nid, start,
> size, resource_name, mhp_flags, online_type, &np).
>
> 2. node_private_register(nid, &np) stores the driver's
> node_private in pgdat and sets pgdat->private. N_MEMORY and
> N_MEMORY_PRIVATE are mutually exclusive -- registration fails
> with -EBUSY if the node already has N_MEMORY set.
>
> Only one driver may register per private node.
>
> 3. Memory is hotplugged via __add_memory_driver_managed().
>
> When online_pages() runs, it checks pgdat->private and sets
> N_MEMORY_PRIVATE instead of N_MEMORY.
>
> Zonelist construction gives private nodes a self-only NOFALLBACK
> list and an N_MEMORY fallback list (so kernel/slab allocations on
> behalf of private node work can fall back to DRAM).
>
> 4. kswapd and kcompactd are NOT started for private nodes. The
> owning service is responsible for driving reclaim if needed
> (e.g., CRAM uses watermark_boost to wake kswapd on demand).
>
> Teardown is the reverse:
>
> 1. Driver calls offline_and_remove_private_memory(nid, start,
> size).
>
> 2. offline_pages() offlines the memory. When the last block is
> offlined, N_MEMORY_PRIVATE is cleared automatically.
>
> 3. node_private_unregister() clears pgdat->node_private and
> drops the refcount. It refuses to unregister (-EBUSY) if
> N_MEMORY_PRIVATE is still set (other memory ranges remain).
>
> The driver is responsible for ensuring memory is hot-unpluggable
> before teardown. The service must ensure all memory is cleaned
> up before hot-unplug - or the service must support migration (so
> memory_hotplug.c can evacuate the memory itself).
>
> In the CRAM example, the service supports migration, so memory
> hot-unplug can remove memory without any special infrastructure.
>
>
> Application: Compressed RAM (mm/cram)
> ===
>
> Compressed RAM has a serious design issue: Its capacity a lie.
>
> A compression device reports more capacity than it physically has.
> If workloads write faster than the OS can reclaim from the device,
> we run out of real backing store and corrupt data or crash.
>
> I call this problem: "Trying to Out Run A Bear"
>
> I.e. This is only stable as long as we stay ahead of the pressure.
>
> We don't want to design a system where stability depends on outrunning
> a bear - I am slow and do not know where to acquire bear spray.
>
> Fun fact: Grizzly bears have a top-speed of 56-64 km/h.
> Unfun Fact: Humans typically top out at ~24 km/h.
>
> This MVP takes a conservative position:
>
> all compressed memory is mapped read-only.
>
> - Folios reach the private node only via reclaim (demotion)
> - migrate_to implements custom demotion with backpressure.
> - fixup_migration_pte write-protects PTEs on arrival.
> - wrprotect hooks prevent silent upgrades
> - handle_fault promotes folios back to DRAM on write.
> - free_folio scrubs stale data before buddy free.
>
> Because pages are read-only, writes can never cause runaway
> compression ratio loss behind the allocator's back. Every write
> goes through handle_fault, which promotes the folio to DRAM first.
>
> The device only ever sees net compression (demotion in) and explicit
> decompression (promotion out via fault or reclaim), and has a much
> wider timeframe to respond to poor compression scenarios.
>
> That means there's no bear to out run. The bears are safely asleep in
> their bear den, and even if they show up we have a bear-proof cage.
>
> The backpressure system is our bear-proof cage: the driver reports real
> device utilization (generalized via watermark_boost on the private
> node's zone), and CRAM throttles demotion when capacity is tight.
>
> If compression ratios are bad, we stop demoting pages and start
> evicting pages aggressively.
>
> The service as designed is ~350 functional lines of code because it
> re-uses mm/ services:
>
> - Existing reclaim/vmscan code handles demotion.
> - Existing migration code handles migration to/from.
> - Existing page fault handling dispatches faults.
>
> The driver contains all the CXL nastiness core developers don't want
> anything to do with - No vendor logic touches mm/ internals.
>
>
>
> Future CRAM : Loosening the read-only constraint
> ===
>
> The read-only model is safe but conservative. For workloads where
> compressed pages are occasionally written, the promotion fault adds
> latency. A future optimization could allow a tunable fraction of
> compressed pages to be mapped writable, accepting some risk of
> write-driven decompression in exchange for lower overhead.
>
> The private node ops make this straightforward:
>
> - Adjust fixup_migration_pte to selectively skip
> write-protection.
> - Use the backpressure system to either revoke writable mappings,
> deny additional demotions, or evict when device pressure rises.
>
> This comes at a mild memory overhead: 32MB of DRAM per 1TB of CRAM.
> (1 bit per 4KB page).
>
> This is not proposed here, but it should be somewhat trivial.
>
>
> Discussion Topics
> ===
> 0. Obviously I've included the set as an RFC, please rip it apart.
>
> 1. Is N_MEMORY_PRIVATE the right isolation abstraction, or should
> this extend ZONE_DEVICE? Prior feedback pushed away from new
> ZONE logic, but this will likely be debated further.
>
> My comments on this:
>
> ZONE_DEVICE requires re-implementing every service you want to
> provide to your device memory, including basic allocation.
>
> Private nodes use real struct pages with no metadata
> limitations, participate in the buddy allocator, and get NUMA
> topology for free.
>
> 2. Can this subsume ZONE_DEVICE COHERENT users? The architecture
> was designed with this in mind, but it is only a thought experiment.
>
> 3. Is a dedicated mm/ service (cram) the right place for compressed
> memory management, or should this be purely driver-side until
> more devices exist?
>
> I wrote it this way because I forsee more "innovation" in the
> compressed RAM space given current... uh... "Market Conditions".
>
> I don't see CRAM being CXL-specific, though the only solutions I've
> seen have been CXL. Nothing is stopping someone from soldering such
> memory directly to a PCB.
>
> 5. Where is your hardware-backed data that shows this works?
>
> I should have some by conference time.
>
> Thanks for reading
> Gregory (Gourry)
>
>
> Gregory Price (27):
> numa: introduce N_MEMORY_PRIVATE node state
> mm,cpuset: gate allocations from N_MEMORY_PRIVATE behind __GFP_PRIVATE
> mm/page_alloc: add numa_zone_allowed() and wire it up
> mm/page_alloc: Add private node handling to build_zonelists
> mm: introduce folio_is_private_managed() unified predicate
> mm/mlock: skip mlock for managed-memory folios
> mm/madvise: skip madvise for managed-memory folios
> mm/ksm: skip KSM for managed-memory folios
> mm/khugepaged: skip private node folios when trying to collapse.
> mm/swap: add free_folio callback for folio release cleanup
> mm/huge_memory.c: add private node folio split notification callback
> mm/migrate: NP_OPS_MIGRATION - support private node user migration
> mm/mempolicy: NP_OPS_MEMPOLICY - support private node mempolicy
> mm/memory-tiers: NP_OPS_DEMOTION - support private node demotion
> mm/mprotect: NP_OPS_PROTECT_WRITE - gate PTE/PMD write-upgrades
> mm: NP_OPS_RECLAIM - private node reclaim participation
> mm/oom: NP_OPS_OOM_ELIGIBLE - private node OOM participation
> mm/memory: NP_OPS_NUMA_BALANCING - private node NUMA balancing
> mm/compaction: NP_OPS_COMPACTION - private node compaction support
> mm/gup: NP_OPS_LONGTERM_PIN - private node longterm pin support
> mm/memory-failure: add memory_failure callback to node_private_ops
> mm/memory_hotplug: add add_private_memory_driver_managed()
> mm/cram: add compressed ram memory management subsystem
> cxl/core: Add cxl_sysram region type
> cxl/core: Add private node support to cxl_sysram
> cxl: add cxl_mempolicy sample PCI driver
> cxl: add cxl_compression PCI driver
>
> drivers/base/node.c | 250 +++-
> drivers/cxl/Kconfig | 2 +
> drivers/cxl/Makefile | 2 +
> drivers/cxl/core/Makefile | 1 +
> drivers/cxl/core/core.h | 4 +
> drivers/cxl/core/port.c | 2 +
> drivers/cxl/core/region_sysram.c | 381 ++++++
> drivers/cxl/cxl.h | 53 +
> drivers/cxl/type3_drivers/Kconfig | 3 +
> drivers/cxl/type3_drivers/Makefile | 3 +
> .../cxl/type3_drivers/cxl_compression/Kconfig | 20 +
> .../type3_drivers/cxl_compression/Makefile | 4 +
> .../cxl_compression/compression.c | 1025 +++++++++++++++++
> .../cxl/type3_drivers/cxl_mempolicy/Kconfig | 16 +
> .../cxl/type3_drivers/cxl_mempolicy/Makefile | 4 +
> .../type3_drivers/cxl_mempolicy/mempolicy.c | 297 +++++
> include/linux/cpuset.h | 9 -
> include/linux/cram.h | 66 ++
> include/linux/gfp_types.h | 15 +-
> include/linux/memory-tiers.h | 9 +
> include/linux/memory_hotplug.h | 11 +
> include/linux/migrate.h | 17 +-
> include/linux/mm.h | 22 +
> include/linux/mmzone.h | 16 +
> include/linux/node_private.h | 532 +++++++++
> include/linux/nodemask.h | 1 +
> include/trace/events/mmflags.h | 4 +-
> include/uapi/linux/mempolicy.h | 1 +
> kernel/cgroup/cpuset.c | 49 +-
> mm/Kconfig | 10 +
> mm/Makefile | 1 +
> mm/compaction.c | 32 +-
> mm/cram.c | 508 ++++++++
> mm/damon/paddr.c | 3 +
> mm/huge_memory.c | 23 +-
> mm/hugetlb.c | 2 +-
> mm/internal.h | 226 +++-
> mm/khugepaged.c | 7 +-
> mm/ksm.c | 9 +-
> mm/madvise.c | 5 +-
> mm/memory-failure.c | 15 +
> mm/memory-tiers.c | 46 +-
> mm/memory.c | 26 +
> mm/memory_hotplug.c | 122 +-
> mm/mempolicy.c | 69 +-
> mm/migrate.c | 63 +-
> mm/mlock.c | 5 +-
> mm/mprotect.c | 4 +-
> mm/oom_kill.c | 52 +-
> mm/page_alloc.c | 79 +-
> mm/rmap.c | 4 +-
> mm/slub.c | 3 +-
> mm/swap.c | 21 +-
> mm/vmscan.c | 55 +-
> 54 files changed, 4057 insertions(+), 152 deletions(-)
> create mode 100644 drivers/cxl/core/region_sysram.c
> create mode 100644 drivers/cxl/type3_drivers/Kconfig
> create mode 100644 drivers/cxl/type3_drivers/Makefile
> create mode 100644 drivers/cxl/type3_drivers/cxl_compression/Kconfig
> create mode 100644 drivers/cxl/type3_drivers/cxl_compression/Makefile
> create mode 100644 drivers/cxl/type3_drivers/cxl_compression/compression.c
> create mode 100644 drivers/cxl/type3_drivers/cxl_mempolicy/Kconfig
> create mode 100644 drivers/cxl/type3_drivers/cxl_mempolicy/Makefile
> create mode 100644 drivers/cxl/type3_drivers/cxl_mempolicy/mempolicy.c
> create mode 100644 include/linux/cram.h
> create mode 100644 include/linux/node_private.h
> create mode 100644 mm/cram.c
>
^ permalink raw reply
* Re: [RFC PATCH bpf-next 2/3] ftrace: Use kallsyms binary search for single-symbol lookup
From: Steven Rostedt @ 2026-02-25 11:47 UTC (permalink / raw)
To: Andrey Grodzovsky
Cc: bpf, ast, daniel, andrii, jolsa, linux-trace-kernel,
linux-open-source
In-Reply-To: <20260223215113.924599-3-andrey.grodzovsky@crowdstrike.com>
On Mon, 23 Feb 2026 16:51:12 -0500
Andrey Grodzovsky <andrey.grodzovsky@crowdstrike.com> wrote:
> When ftrace_lookup_symbols() is called with a single symbol (cnt == 1),
> use kallsyms_lookup_name() for O(log N) binary search instead of the
> full linear scan via kallsyms_on_each_symbol().
So this patch looks like it should go through the tracing tree, not bpf.
>
> ftrace_lookup_symbols() was designed for batch resolution of many
> symbols in a single pass. For large cnt this is efficient: a single
> O(N) walk over all symbols with O(log cnt) binary search into the
> sorted input array. But for cnt == 1 it still decompresses all ~200K
> kernel symbols only to match one.
>
> kallsyms_lookup_name() uses the sorted kallsyms index and needs only
> ~17 decompressions for a single lookup.
>
> This is the common path for kprobe.session with exact function names,
> where libbpf sends one symbol per BPF_LINK_CREATE syscall.
>
> If binary lookup fails (duplicate symbol names where the first match
> is not ftrace-instrumented, or module symbols), the function falls
> through to the existing linear scan path.
>
> Before (cnt=1, 50 kprobe.session programs):
> Attach: 858 ms (kallsyms_expand_symbol 25% of CPU)
>
> After:
> Attach: 52 ms (16x faster)
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@crowdstrike.com>
> ---
> kernel/trace/ftrace.c | 28 ++++++++++++++++++++++++++++
> 1 file changed, 28 insertions(+)
>
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index 827fb9a0bf0d..bfd7670669c2 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -9263,6 +9263,19 @@ static int kallsyms_callback(void *data, const char *name, unsigned long addr)
> * @addrs array, which needs to be big enough to store at least @cnt
> * addresses.
> *
> + * For a single symbol (cnt == 1), uses kallsyms_lookup_name() which
> + * performs an O(log N) binary search via the sorted kallsyms index.
> + * This avoids the full O(N) linear scan over all kernel symbols that
> + * the multi-symbol path requires.
> + *
> + * For multiple symbols, uses a single-pass linear scan via
> + * kallsyms_on_each_symbol() with binary search into the sorted input
> + * array.
The above is fine.
> While individual lookups are O(log N), doing K lookups
> + * totals O(K * log N) which loses to a single sequential O(N) pass
> + * at scale due to cache-friendly memory access patterns of the linear
> + * walk. Empirical testing shows the linear scan is faster for batch
> + * lookups even well below 10K symbols.
The above is unneeded for a comment in the code and just belongs in the
change log.
-- Steve
> + *
> * Returns: 0 if all provided symbols are found, -ESRCH otherwise.
> */
> int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *addrs)
> @@ -9270,6 +9283,21 @@ int ftrace_lookup_symbols(const char **sorted_syms, size_t cnt, unsigned long *a
> struct kallsyms_data args;
> int found_all;
>
> + /* Fast path: single symbol uses O(log N) binary search */
> + if (cnt == 1) {
> + addrs[0] = kallsyms_lookup_name(sorted_syms[0]);
> + if (addrs[0])
> + addrs[0] = ftrace_location(addrs[0]);
> + if (addrs[0])
> + return 0;
> + /*
> + * Binary lookup can fail for duplicate symbol names
> + * where the first match is not ftrace-instrumented,
> + * or for module symbols. Retry with linear scan.
> + */
> + }
> +
> + /* Batch path: single-pass O(N) linear scan */
> memset(addrs, 0, sizeof(*addrs) * cnt);
> args.addrs = addrs;
> args.syms = sorted_syms;
^ permalink raw reply
* Re: [RFC PATCH v2 09/37] KVM: guest_memfd: Add support for KVM_SET_MEMORY_ATTRIBUTES2
From: Fuad Tabba @ 2026-02-25 11:00 UTC (permalink / raw)
To: Ackerley Tng
Cc: kvm, linux-doc, linux-kernel, linux-kselftest, linux-trace-kernel,
x86, aik, andrew.jones, binbin.wu, bp, brauner, chao.p.peng,
chao.p.peng, chenhuacai, corbet, dave.hansen, david, hpa,
ira.weiny, jgg, jmattson, jroedel, jthoughton, maobibo,
mathieu.desnoyers, maz, mhiramat, michael.roth, mingo, mlevitsk,
oupton, pankaj.gupta, pbonzini, prsampat, qperret, ricarkol,
rick.p.edgecombe, rientjes, rostedt, seanjc, shivankg, shuah,
steven.price, tglx, vannapurve, vbabka, willy, wyihan, yan.y.zhao
In-Reply-To: <CAEvNRgFBLgvYoR_XTH-LiN1Q00R9u1HGC5URbstLPxYtedS0MA@mail.gmail.com>
Hi Ackerley,
Here are my thoughts, at least when it comes to pKVM.
On Tue, 24 Feb 2026 at 10:14, Ackerley Tng <ackerleytng@google.com> wrote:
>
> Ackerley Tng <ackerleytng@google.com> writes:
>
> > Ackerley Tng <ackerleytng@google.com> writes:
> >
> >>
> >> [...snip...]
> >>
> > Before this lands, Sean wants, at the very minimum, an in-principle
> > agreement on guest_memfd behavior with respect to whether or not memory
> > should be preserved on conversion.
> >>
> >> [...snip...]
> >>
>
> Here's what I've come up with, following up from last guest_memfd
> biweekly.
>
> Every KVM_SET_MEMORY_ATTRIBUTES2 request will be accompanied by an
> enum set_memory_attributes_content_policy:
>
> enum set_memory_attributes_content_policy {
> SET_MEMORY_ATTRIBUTES_CONTENT_ZERO,
> SET_MEMORY_ATTRIBUTES_CONTENT_READABLE,
> SET_MEMORY_ATTRIBUTES_CONTENT_ENCRYPTED,
> }
>
> Within guest_memfd's KVM_SET_MEMORY_ATTRIBUTES2 handler, guest_memfd
> will make an arch call
>
> kvm_gmem_arch_content_policy_supported(kvm, policy, gfn, nr_pages)
>
> where every arch will get to return some error if the requested policy
> is not supported for the given range.
This hook provides the validation mechanism pKVM requires.
> ZERO is the simplest of the above, it means that after the conversion
> the memory will be zeroed for the next reader.
>
> + TDX and SNP today will support ZERO since the firmware handles
> zeroing.
> + pKVM and SW_PROTECTED_VM will apply software zeroing.
> + Purpose: having this policy in the API allows userspace to be sure
> that the memory is zeroed after the conversion - there is no need to
> zero again in userspace (addresses concern that Sean pointed out)
>
> READABLE means that after the conversion, the memory is readable by
> userspace (if converting to shared) or readable by the guest (if
> converting to private).
>
> + TDX and SNP (today) can't support this, so return -EOPNOTSUPP
> + SW_PROTECTED_VM will support this and do nothing extra on
> conversion, since there is no encryption anyway and all content
> remains readable.
> + pKVM will make use of the arch function above.
>
> Here's where I need input: (David's questions during the call about
> the full flow beginning with the guest prompted this).
>
> Since pKVM doesn't encrypt the memory contents, there must be some way
> that pKVM can say no when userspace requests to convert and retain
> READABLE contents? I think pKVM's arch function can be used to check
> if the guest previously made a conversion request. Fuad, to check that
> the guest made a conversion request, what's other parameters are
> needed other than gfn and nr_pages?
The gfn and nr_pages parameters are enough I think.
To clarify how pKVM would use this hook: all memory sharing and
unsharing must be initiated by the guest via a hypercall. When the
guest issues this hypercall, the pKVM hypervisor (EL2) exits to the
host kernel (EL1). The host kernel records the exit reason (share or
unshare) along with the specific memory address in the kvm_run
structure before exiting to userspace.
We do not track this pending conversion state in the hypervisor. If a
compromised host kernel wants to lie and corrupt the state, it can
crash the system or the guest (which is an accepted DOS risk), but it
cannot compromise guest confidentiality because EL2 still strictly
enforces Stage-2 permissions. Our primary goal here is to prevent a
malicious or buggy userspace VMM from crashing the system.
When the VMM subsequently issues the KVM_SET_MEMORY_ATTRIBUTES2 ioctl
with the READABLE policy, we will use the
kvm_gmem_arch_content_policy_supported() hook in EL1 to validate the
ioctl. We will cross-reference the requested gfn and nr_pages against
the pending exit reason stored in kvm_run.
If the VMM attempts an unsolicited conversion (i.e., there is no
matching exit request in kvm_run, or the addresses do not match), our
current plan is to reject the request and return an error. In the
future, rather than outright rejecting an unsolicited conversion, we
might evolve this to treat it as a host-initiated destructive reclaim,
forcing an unshare and zeroing the memory. For the time being,
explicit rejection is the simplest and safest path.
> ENCRYPTED means that after the conversion, the memory contents are
> retained as-is, with no decryption.
>
> + TDX and SNP (today) can't support this, so return -EOPNOTSUPP
> + pKVM and SW_PROTECTED_VM can do nothing, but doing nothing retains
> READABLE content, not ENCRYPTED content, hence SW_PROTECTED_VM
> should return -EOPNOTSUPP.
> + Michael, you mentioned during the call that SNP is planning to
> introduce a policy that retains the ENCRYPTED version for a special
> GHCB call. ENCRYPTED is meant for that use case. Does it work? I'm
> assuming that SNP should only support this policy given some
> conditions, so would the arch call as described above work?
> + If this policy is specified on conversion from shared to private,
> always return -EOPNOTSUPP.
> + When this first lands, ENCRYPTED will not be a valid option, but I'm
> listing it here so we have line of sight to having this support.
>
> READABLE and ENCRYPTED defines the state after conversion clearly
> (instead of DONT_CARE or similar).
>
> DESTROY could be another policy, which means that after the
> conversion, the memory is unreadable. This is the option to address
> what David brought up during the call, for cases where userspace knows
> it is going to free the memory already and doesn't care about the
> state as long as nobody gets to read it. This will not implemented
> when feature first lands, but is presented here just to show how this
> can be extended in future.
>
> Right now, I'm thinking that one of the above policies MUST be
> specified (not specifying a policy will result in -EINVAL).
>
> How does this sound?
I don't think that returning -EINVAL is the right thing to do here. If
userspace omits the policy, the API should default to
SET_MEMORY_ATTRIBUTES_CONTENT_ZERO and proceed with the conversion. I
believe that, in Linux APIs in general, omitting an optional behavior
flag results in the safest, most standard default action.
Also, returning -EINVAL when no policy is specified makes the policy
parameter strictly mandatory. This makes it difficult for userspace's
to seamlessly request clean-slate, destructive conversions. Software
zeroing ensures deterministic behavior across pKVM, TDX, and SNP,
isolating the KVM uAPI from micro-architectural data destruction
nuances.
Cheers,
/fuad
^ permalink raw reply
* [PATCH v6 16/16] rv: Add dl_server specific monitors
From: Gabriele Monaco @ 2026-02-25 9:51 UTC (permalink / raw)
To: linux-kernel, Steven Rostedt, Nam Cao, Juri Lelli,
Gabriele Monaco, Jonathan Corbet, Masami Hiramatsu,
linux-trace-kernel, linux-doc
Cc: Peter Zijlstra, Tomas Glozar, Clark Williams, John Kacur
In-Reply-To: <20260225095122.80683-1-gmonaco@redhat.com>
Add monitors to validate the behaviour of the deadline server.
The currently implemented monitors are:
* boost
fair tasks run either independently or boosted
* laxity
deferrable servers wait for zero-laxity and run
Cc: Peter Zijlstra <peterz@infradead.org>
Reviewed-by: Nam Cao <namcao@linutronix.de>
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
---
Notes:
V6:
* Add support for ext server in monitors
* Add missing transition running -> zero_laxity_wait
* Handle dl_update event in laxity to cover missing update without
enqueue nor replenish when multiple server types are active
* Handle events only for supported deadline servers (future proof)
* Cleanup and sort constraints as generated
V4
* Rely on enqueue/dequeue tracepoints instead of syscalls
* Improve timing conditions in laxity and handle resume action
* Remove fragile Stopping state from boost
Documentation/trace/rv/monitor_deadline.rst | 120 ++++++++
kernel/trace/rv/Kconfig | 2 +
kernel/trace/rv/Makefile | 2 +
kernel/trace/rv/monitors/boost/Kconfig | 15 +
kernel/trace/rv/monitors/boost/boost.c | 258 ++++++++++++++++
kernel/trace/rv/monitors/boost/boost.h | 146 +++++++++
kernel/trace/rv/monitors/boost/boost_trace.h | 19 ++
kernel/trace/rv/monitors/laxity/Kconfig | 14 +
kernel/trace/rv/monitors/laxity/laxity.c | 279 ++++++++++++++++++
kernel/trace/rv/monitors/laxity/laxity.h | 140 +++++++++
.../trace/rv/monitors/laxity/laxity_trace.h | 19 ++
kernel/trace/rv/rv_trace.h | 2 +
tools/verification/models/deadline/boost.dot | 48 +++
tools/verification/models/deadline/laxity.dot | 37 +++
14 files changed, 1101 insertions(+)
create mode 100644 kernel/trace/rv/monitors/boost/Kconfig
create mode 100644 kernel/trace/rv/monitors/boost/boost.c
create mode 100644 kernel/trace/rv/monitors/boost/boost.h
create mode 100644 kernel/trace/rv/monitors/boost/boost_trace.h
create mode 100644 kernel/trace/rv/monitors/laxity/Kconfig
create mode 100644 kernel/trace/rv/monitors/laxity/laxity.c
create mode 100644 kernel/trace/rv/monitors/laxity/laxity.h
create mode 100644 kernel/trace/rv/monitors/laxity/laxity_trace.h
create mode 100644 tools/verification/models/deadline/boost.dot
create mode 100644 tools/verification/models/deadline/laxity.dot
diff --git a/Documentation/trace/rv/monitor_deadline.rst b/Documentation/trace/rv/monitor_deadline.rst
index 566bce4ff582..4e2c4ebc5687 100644
--- a/Documentation/trace/rv/monitor_deadline.rst
+++ b/Documentation/trace/rv/monitor_deadline.rst
@@ -156,3 +156,123 @@ server can run from sleeping without being ready::
+--------------+ <---------+ ^
| |
+------ dl_throttle;is_constr_dl == 1 || is_defer == 1 ------+
+
+Monitor boost
+~~~~~~~~~~~~~
+
+The boost monitor ensures tasks associated to a server (e.g. fair tasks) run
+either independently or boosted in a timely manner.
+Unlike other models, the ``running`` state (and the ``switch_in/out`` events)
+indicates that any fair task is running, this needs to happen within a
+threshold that depends on server deadline and remaining runtime, whenever a
+task is ready.
+
+The following chart is simplified to avoid confusion, several less important
+self-loops on states have been removed and event names have been simplified:
+
+* ``idle`` (``dl_server_idle``) occurs when the CPU runs the idle task.
+* ``start/stop`` (``dl_server_start/stop``) start and stop the server.
+* ``switch`` (``sched_switch_in/out``) represented as a double arrow to
+ indicate both edges are present: ``ready -- switch_in -> running`` and
+ ``running -- switch_out -> ready``. As stated above this fires when any fair
+ task starts or stops to running.
+* ``resume/resume_throttle``: a fair task woke up, potentially when the server
+ is throttled (no runtime left), this event is especially frequent on self
+ loops (no state change during a wakeup) but is removed here for clarity.
+* arrows merge with an ``x`` sign to indicate they are the same event going to
+ the same state (but with different origins, e.g. ``{idle/throttled} -- stop
+ -> stopped``). The ``+`` sign indicates standard crossings or corners.
+
+Refer to the dot file for the full specification::
+
+ |
+ v
+ #===============# stop;reset(clk)
+ H H <---------------+
+ +------------>H stopped H |
+ | H H |
+ | #===============# |
+ | ^ | |
+ | | | | replenish;reset(clk)
+ | stop | | +--+
+ | | start;reset(clk) +-----------------+ | |
+ | | v | | v
+ | +---------------+ <---------- switch --------> +---------+
+ | +- resume -> | ready | | |
+ | | | | -replenish;reset(clk) | running |
+ | | +- idle - | clk < thesh() | | | |
+ | | | +---------------+ <-+ +---------------- +---------+
+ | | | | ^ | ^ |
+ | | | | | throttle | |
+ | | | | |replenish;reset(clk) | | |
+ | | | throttle | | replenish;reset(clk) |
+ | | | | | | | |
+ | | | v | v | |
+ | | | +---------+ switch +-------------------+ | |
+ x---+--+-- | | <----------> | throttled_running | --------+ |
+ | | | |throttled| +-------------------+ |
+ | | | | | -----+ | |
+ | | | +---------+ | | |
+ | | | ^ | | |
+ | | | resume_throttle | | |
+ stop | | | | | |
+ | | v | | | |
+ | +---------+ <-----------x--- idle ---x-----------------------------+
+ | | |
+ +-- | idle | <--+
+ | | | replenish;reset(clk)
+ +---------+ ---+
+
+Monitor laxity
+~~~~~~~~~~~~~~
+
+The laxity monitor ensure deferrable servers go to a zero-laxity wait unless
+already running and run in starvation cases. The model can stay in the
+zero-laxity wait only for up to a period, then the server either prepares to
+stop (after ``idle_wait``) or prepares to boost a task (``running``). Boosting
+(``sched_switch_in``) is only allowed in the ``running`` state.
+``dl_replenish_running`` should not be allowed in ``running``, but can happen
+as soon as the server started, the model allows this only within a short
+threshold::
+
+ |
+ +---- dl_server_stop -----+ |
+ | v v
+ | #=======================================#
+ | +------- H stopped H
+ | | #=======================================#
+ | | | ^
+ | | dl_server_start_running; dl_server_stop
+ | | reset(clk) |
+ | | v | dl_replenish_running;
+ | | +-------------------------------------+ -----------clk < REPLENISH_NS
+ | | | | |
+ | | | running | <------------+
+ | | | |
+ | | +-------------------------------------+ ------------------+
+ | | | ^ ^ |
+ | | dl_throttle dl_replenish_running | dl_update
+ | | v | | dl_replenish;reset(clk)
+ | | +-------------------+ | dl_replenish_idle;reset(clk)
+ | | | replenish_wait | | |
+ | | | clk < period_ns() | ----------------+---------------------+--------+
+ | | +-------------------+ | | |
+ | | | | | |
+ | | dl_update | | |
+ | | dl_replenish;reset(clk) dl_replenish_running | |
+ | | v | | |
+ | | +--------------------------+ | |
+ | dl_server_start; | | <----------------+ |
+ | reset(clk) | zero_laxity_wait | |
+ | | | clk < period_ns() | ------+ dl_replenish; |
+ | +---------------> | | | reset(clk) |
+ | +--------------------------+ <-----+ dl_update |
+ | | ^ |
+ | dl_replenish_idle;reset(clk) | dl_replenish;reset(clk) |
+ | v dl_update |
+ | +------------------------+ | |
+ +----------------- | idle_wait | -+ |
+ | clk < period_ns() | |
+ +------------------------+ <-- dl_replenish_idle;reset(clk)
+ ^ |
+ +------dl_replenish_idle;reset(clk)
diff --git a/kernel/trace/rv/Kconfig b/kernel/trace/rv/Kconfig
index 719cdcfb6d41..139443e0e51c 100644
--- a/kernel/trace/rv/Kconfig
+++ b/kernel/trace/rv/Kconfig
@@ -82,6 +82,8 @@ source "kernel/trace/rv/monitors/stall/Kconfig"
source "kernel/trace/rv/monitors/deadline/Kconfig"
source "kernel/trace/rv/monitors/nomiss/Kconfig"
source "kernel/trace/rv/monitors/throttle/Kconfig"
+source "kernel/trace/rv/monitors/boost/Kconfig"
+source "kernel/trace/rv/monitors/laxity/Kconfig"
# Add new deadline monitors here
# Add new monitors here
diff --git a/kernel/trace/rv/Makefile b/kernel/trace/rv/Makefile
index 15a1edc8bd0f..4cf15c189a96 100644
--- a/kernel/trace/rv/Makefile
+++ b/kernel/trace/rv/Makefile
@@ -21,6 +21,8 @@ obj-$(CONFIG_RV_MON_STALL) += monitors/stall/stall.o
obj-$(CONFIG_RV_MON_DEADLINE) += monitors/deadline/deadline.o
obj-$(CONFIG_RV_MON_NOMISS) += monitors/nomiss/nomiss.o
obj-$(CONFIG_RV_MON_THROTTLE) += monitors/throttle/throttle.o
+obj-$(CONFIG_RV_MON_BOOST) += monitors/boost/boost.o
+obj-$(CONFIG_RV_MON_LAXITY) += monitors/laxity/laxity.o
# Add new monitors here
obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
diff --git a/kernel/trace/rv/monitors/boost/Kconfig b/kernel/trace/rv/monitors/boost/Kconfig
new file mode 100644
index 000000000000..3fa121f77729
--- /dev/null
+++ b/kernel/trace/rv/monitors/boost/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_BOOST
+ depends on RV
+ depends on RV_MON_DEADLINE
+ default y
+ select HA_MON_EVENTS_ID
+ bool "boost monitor"
+ help
+ Monitor to ensure tasks associated to a server (e.g. fair tasks) run
+ either independently or boosted in a timely manner.
+ This monitor is part of the deadline monitors collection.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/boost/boost.c b/kernel/trace/rv/monitors/boost/boost.c
new file mode 100644
index 000000000000..85b3b97ca6ef
--- /dev/null
+++ b/kernel/trace/rv/monitors/boost/boost.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+
+#define MODULE_NAME "boost"
+
+#include <trace/events/sched.h>
+#include <rv_trace.h>
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define DA_SKIP_AUTO_ALLOC
+#define HA_TIMER_TYPE HA_TIMER_WHEEL
+typedef struct sched_dl_entity *monitor_target;
+#include "boost.h"
+#include <rv/ha_monitor.h>
+#include <monitors/deadline/deadline.h>
+
+static inline u64 server_threshold_ns(struct ha_monitor *ha_mon)
+{
+ struct sched_dl_entity *dl_se = ha_get_target(ha_mon);
+
+ return dl_se->dl_deadline + TICK_NSEC - dl_se->runtime;
+}
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_boost env, u64 time_ns)
+{
+ if (env == clk_boost)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ return ENV_INVALID_VALUE;
+}
+
+static void ha_reset_env(struct ha_monitor *ha_mon, enum envs_boost env, u64 time_ns)
+{
+ if (env == clk_boost)
+ ha_reset_clk_ns(ha_mon, env, time_ns);
+}
+
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == ready_boost)
+ return ha_check_invariant_ns(ha_mon, clk_boost, time_ns);
+ return true;
+}
+
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ bool res = true;
+
+ if (curr_state == stopped_boost && event == dl_server_start_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == idle_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == ready_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == running_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == throttled_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ else if (curr_state == throttled_running_boost && event == dl_replenish_boost)
+ ha_reset_env(ha_mon, clk_boost, time_ns);
+ return res;
+}
+
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == curr_state && event != dl_replenish_boost)
+ return;
+ if (next_state == ready_boost)
+ ha_start_timer_ns(ha_mon, clk_boost, server_threshold_ns(ha_mon), time_ns);
+ else if (curr_state == ready_boost)
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+static void handle_dl_replenish(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_server_type(type))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_replenish_boost);
+}
+
+static inline void handle_server_switch(struct task_struct *next,
+ struct task_struct *prev, int cpu,
+ u8 type)
+{
+ struct sched_dl_entity *dl_se = get_server(next, type);
+
+ if (!dl_se)
+ return;
+ if (is_idle_task(next))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_server_idle_boost);
+ else if (get_server_type(next) == type && !rt_or_dl_task(next))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), sched_switch_in_boost);
+ else if (get_server_type(prev) == type && !is_idle_task(prev))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), sched_switch_out_boost);
+}
+
+static void handle_sched_switch(void *data, bool preempt,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ int cpu = task_cpu(next);
+
+ /*
+ * The server is available in next only if the next task is boosted,
+ * otherwise we need to retrieve it.
+ * This monitor considers switch in/out whenever a task related to the
+ * server (i.e. fair) is scheduled in or out, boosted or not.
+ * Any switch to the same policy is ignored.
+ * PI boosted tasks are not considered fair.
+ */
+ if (get_server_type(next) == get_server_type(prev) &&
+ !is_idle_task(next) && !is_idle_task(prev))
+ return;
+ handle_server_switch(next, prev, cpu, DL_SERVER_FAIR);
+ if (IS_ENABLED(CONFIG_SCHED_CLASS_EXT))
+ handle_server_switch(next, prev, cpu, DL_SERVER_EXT);
+}
+
+static void handle_sched_enqueue(void *data, struct task_struct *tsk, int cpu)
+{
+ struct sched_dl_entity *dl_se = NULL;
+ uint8_t type = get_server_type(tsk);
+
+ if (is_server_type(type))
+ dl_se = get_server(tsk, type);
+ if (dl_se) {
+ da_handle_event(EXPAND_ID(dl_se, cpu, type),
+ dl_se->runtime > 0 ?
+ dl_server_resume_boost :
+ dl_server_resume_throttled_boost);
+ }
+}
+
+static void handle_sched_dequeue(void *data, struct task_struct *tsk, int cpu)
+{
+ struct sched_dl_entity *dl_se = NULL;
+ uint8_t type = get_server_type(tsk);
+
+ if (is_server_type(type))
+ dl_se = get_server(tsk, type);
+ /*
+ * A dequeue is counted as switching out only in case of a change in
+ * scheduler where the task is moved to another scheduler's runqueue.
+ */
+ if (dl_se && task_is_running(tsk) && sched_task_on_rq(tsk))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), sched_switch_out_boost);
+}
+
+static void handle_dl_server_start(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_server_type(type))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_server_start_boost);
+}
+
+static void handle_dl_server_stop(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_server_type(type))
+ da_handle_start_event(EXPAND_ID(dl_se, cpu, type), dl_server_stop_boost);
+}
+
+static void handle_dl_throttle(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_server_type(type))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_throttle_boost);
+}
+
+static int enable_boost(void)
+{
+ int retval;
+
+ retval = da_monitor_init();
+ if (retval)
+ return retval;
+
+ retval = init_storage(true);
+ if (retval)
+ return retval;
+ rv_attach_trace_probe("boost", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_attach_trace_probe("boost", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_attach_trace_probe("boost", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_attach_trace_probe("boost", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_attach_trace_probe("boost", sched_enqueue_tp, handle_sched_enqueue);
+ rv_attach_trace_probe("boost", sched_dequeue_tp, handle_sched_dequeue);
+ rv_attach_trace_probe("boost", sched_switch, handle_sched_switch);
+
+ return 0;
+}
+
+static void disable_boost(void)
+{
+ rv_this.enabled = 0;
+
+ rv_detach_trace_probe("boost", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_detach_trace_probe("boost", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_detach_trace_probe("boost", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_detach_trace_probe("boost", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_detach_trace_probe("boost", sched_enqueue_tp, handle_sched_enqueue);
+ rv_detach_trace_probe("boost", sched_dequeue_tp, handle_sched_dequeue);
+ rv_detach_trace_probe("boost", sched_switch, handle_sched_switch);
+
+ da_monitor_destroy();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "boost",
+ .description = "fair tasks run either independently or boosted.",
+ .enable = enable_boost,
+ .disable = disable_boost,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_boost(void)
+{
+ return rv_register_monitor(&rv_this, &rv_deadline);
+}
+
+static void __exit unregister_boost(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_boost);
+module_exit(unregister_boost);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("boost: fair tasks run either independently or boosted.");
diff --git a/kernel/trace/rv/monitors/boost/boost.h b/kernel/trace/rv/monitors/boost/boost.h
new file mode 100644
index 000000000000..70757f25a90d
--- /dev/null
+++ b/kernel/trace/rv/monitors/boost/boost.h
@@ -0,0 +1,146 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Automatically generated C representation of boost automaton
+ * For further information about this format, see kernel documentation:
+ * Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#define MONITOR_NAME boost
+
+enum states_boost {
+ stopped_boost,
+ idle_boost,
+ ready_boost,
+ running_boost,
+ throttled_boost,
+ throttled_running_boost,
+ state_max_boost,
+};
+
+#define INVALID_STATE state_max_boost
+
+enum events_boost {
+ dl_replenish_boost,
+ dl_server_idle_boost,
+ dl_server_resume_boost,
+ dl_server_resume_throttled_boost,
+ dl_server_start_boost,
+ dl_server_stop_boost,
+ dl_throttle_boost,
+ sched_switch_in_boost,
+ sched_switch_out_boost,
+ event_max_boost,
+};
+
+enum envs_boost {
+ clk_boost,
+ env_max_boost,
+ env_max_stored_boost = env_max_boost,
+};
+
+_Static_assert(env_max_stored_boost <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_boost {
+ char *state_names[state_max_boost];
+ char *event_names[event_max_boost];
+ char *env_names[env_max_boost];
+ unsigned char function[state_max_boost][event_max_boost];
+ unsigned char initial_state;
+ bool final_states[state_max_boost];
+};
+
+static const struct automaton_boost automaton_boost = {
+ .state_names = {
+ "stopped",
+ "idle",
+ "ready",
+ "running",
+ "throttled",
+ "throttled_running",
+ },
+ .event_names = {
+ "dl_replenish",
+ "dl_server_idle",
+ "dl_server_resume",
+ "dl_server_resume_throttled",
+ "dl_server_start",
+ "dl_server_stop",
+ "dl_throttle",
+ "sched_switch_in",
+ "sched_switch_out",
+ },
+ .env_names = {
+ "clk",
+ },
+ .function = {
+ {
+ INVALID_STATE,
+ stopped_boost,
+ stopped_boost,
+ stopped_boost,
+ ready_boost,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ stopped_boost,
+ },
+ {
+ idle_boost,
+ idle_boost,
+ ready_boost,
+ throttled_boost,
+ INVALID_STATE,
+ stopped_boost,
+ idle_boost,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ {
+ ready_boost,
+ idle_boost,
+ ready_boost,
+ ready_boost,
+ INVALID_STATE,
+ stopped_boost,
+ throttled_boost,
+ running_boost,
+ ready_boost,
+ },
+ {
+ running_boost,
+ idle_boost,
+ running_boost,
+ running_boost,
+ INVALID_STATE,
+ stopped_boost,
+ throttled_running_boost,
+ INVALID_STATE,
+ ready_boost,
+ },
+ {
+ ready_boost,
+ idle_boost,
+ INVALID_STATE,
+ throttled_boost,
+ INVALID_STATE,
+ stopped_boost,
+ throttled_boost,
+ throttled_running_boost,
+ INVALID_STATE,
+ },
+ {
+ running_boost,
+ idle_boost,
+ INVALID_STATE,
+ throttled_running_boost,
+ INVALID_STATE,
+ INVALID_STATE,
+ throttled_running_boost,
+ INVALID_STATE,
+ throttled_boost,
+ },
+ },
+ .initial_state = stopped_boost,
+ .final_states = { 1, 0, 0, 0, 0, 0 },
+};
diff --git a/kernel/trace/rv/monitors/boost/boost_trace.h b/kernel/trace/rv/monitors/boost/boost_trace.h
new file mode 100644
index 000000000000..7e422b0e586d
--- /dev/null
+++ b/kernel/trace/rv/monitors/boost/boost_trace.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_BOOST
+DEFINE_EVENT(event_da_monitor_id, event_boost,
+ TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_boost,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_boost,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+#endif /* CONFIG_RV_MON_BOOST */
diff --git a/kernel/trace/rv/monitors/laxity/Kconfig b/kernel/trace/rv/monitors/laxity/Kconfig
new file mode 100644
index 000000000000..7ba69405d09b
--- /dev/null
+++ b/kernel/trace/rv/monitors/laxity/Kconfig
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_LAXITY
+ depends on RV
+ depends on RV_MON_DEADLINE
+ default y
+ select HA_MON_EVENTS_ID
+ bool "laxity monitor"
+ help
+ Monitor to ensure deferrable servers go to a zero-laxity wait unless
+ already running and run in starvation cases.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/laxity/laxity.c b/kernel/trace/rv/monitors/laxity/laxity.c
new file mode 100644
index 000000000000..336e07f59256
--- /dev/null
+++ b/kernel/trace/rv/monitors/laxity/laxity.c
@@ -0,0 +1,279 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+
+#define MODULE_NAME "laxity"
+
+#include <trace/events/sched.h>
+#include <rv_trace.h>
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define HA_TIMER_TYPE HA_TIMER_WHEEL
+/* The start condition is on server_stop, allocation likely fails on PREEMPT_RT */
+#define DA_SKIP_AUTO_ALLOC
+typedef struct sched_dl_entity *monitor_target;
+#include "laxity.h"
+#include <rv/ha_monitor.h>
+#include <monitors/deadline/deadline.h>
+
+/* allow replenish when running only right after server start */
+#define REPLENISH_NS TICK_NSEC
+
+static inline u64 period_ns(struct ha_monitor *ha_mon)
+{
+ return ha_get_target(ha_mon)->dl_period + TICK_NSEC;
+}
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_laxity env, u64 time_ns)
+{
+ if (env == clk_laxity)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ return ENV_INVALID_VALUE;
+}
+
+static void ha_reset_env(struct ha_monitor *ha_mon, enum envs_laxity env, u64 time_ns)
+{
+ if (env == clk_laxity)
+ ha_reset_clk_ns(ha_mon, env, time_ns);
+}
+
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == idle_wait_laxity)
+ return ha_check_invariant_ns(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == replenish_wait_laxity)
+ return ha_check_invariant_ns(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == zero_laxity_wait_laxity)
+ return ha_check_invariant_ns(ha_mon, clk_laxity, time_ns);
+ return true;
+}
+
+static inline void ha_convert_inv_guard(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == next_state)
+ return;
+ if (curr_state == zero_laxity_wait_laxity)
+ ha_inv_to_guard(ha_mon, clk_laxity, period_ns(ha_mon), time_ns);
+}
+
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ bool res = true;
+
+ if (curr_state == stopped_laxity && event == dl_server_start_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == stopped_laxity && event == dl_server_start_running_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == idle_wait_laxity && event == dl_replenish_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == idle_wait_laxity && event == dl_replenish_idle_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == replenish_wait_laxity && event == dl_replenish_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == replenish_wait_laxity && event == dl_replenish_idle_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == replenish_wait_laxity && event == dl_replenish_running_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == running_laxity && event == dl_replenish_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == running_laxity && event == dl_replenish_idle_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == running_laxity && event == dl_replenish_running_laxity)
+ res = ha_monitor_env_invalid(ha_mon, clk_laxity) ||
+ ha_get_env(ha_mon, clk_laxity, time_ns) < REPLENISH_NS;
+ else if (curr_state == running_laxity && event == dl_throttle_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == zero_laxity_wait_laxity && event == dl_replenish_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == zero_laxity_wait_laxity && event == dl_replenish_idle_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ else if (curr_state == zero_laxity_wait_laxity && event == dl_replenish_running_laxity)
+ ha_reset_env(ha_mon, clk_laxity, time_ns);
+ return res;
+}
+
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == curr_state && event != dl_replenish_laxity &&
+ event != dl_replenish_idle_laxity)
+ return;
+ if (next_state == idle_wait_laxity)
+ ha_start_timer_ns(ha_mon, clk_laxity, period_ns(ha_mon), time_ns);
+ else if (next_state == replenish_wait_laxity)
+ ha_start_timer_ns(ha_mon, clk_laxity, period_ns(ha_mon), time_ns);
+ else if (next_state == zero_laxity_wait_laxity)
+ ha_start_timer_ns(ha_mon, clk_laxity, period_ns(ha_mon), time_ns);
+ else if (curr_state == idle_wait_laxity)
+ ha_cancel_timer(ha_mon);
+ else if (curr_state == replenish_wait_laxity)
+ ha_cancel_timer(ha_mon);
+ else if (curr_state == zero_laxity_wait_laxity)
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_convert_inv_guard(ha_mon, curr_state, event, next_state, time_ns);
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+static void handle_dl_replenish(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (!is_server_type(type))
+ return;
+ /* Special replenish happening after throttle, ignore it */
+ if (dl_se->dl_defer_running && dl_se->dl_throttled)
+ return;
+ if (dl_se->dl_defer_running)
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_replenish_running_laxity);
+ else if (idle_cpu(cpu))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_replenish_idle_laxity);
+ else
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_replenish_laxity);
+}
+
+static void handle_dl_server_start(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (!is_server_type(type))
+ return;
+ if (dl_se->dl_defer_running)
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_server_start_running_laxity);
+ else
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_server_start_laxity);
+}
+
+static void handle_dl_server_stop(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_server_type(type))
+ da_handle_start_event(EXPAND_ID(dl_se, cpu, type), dl_server_stop_laxity);
+}
+
+static void handle_dl_throttle(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_server_type(type))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_throttle_laxity);
+}
+
+static void handle_dl_update(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (!is_server_type(type) || idle_cpu(cpu) || dl_se->dl_defer_running)
+ return;
+ /* The idle flag can be cleared without passing from an actual replenish */
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_update_laxity);
+}
+
+
+static void handle_sched_switch(void *data, bool preempt,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ if (!next->dl_server)
+ return;
+ da_handle_event(EXPAND_ID(next->dl_server, task_cpu(next),
+ get_server_type(next)),
+ sched_switch_in_laxity);
+}
+
+static void handle_sched_enqueue(void *data, struct task_struct *tsk, int cpu)
+{
+ struct sched_dl_entity *dl_se = NULL;
+ uint8_t type = get_server_type(tsk);
+
+ if (is_server_type(type))
+ dl_se = get_server(tsk, type);
+ if (dl_se)
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_server_resume_laxity);
+}
+
+static int enable_laxity(void)
+{
+ int retval;
+
+ retval = da_monitor_init();
+ if (retval)
+ return retval;
+
+ retval = init_storage(true);
+ if (retval)
+ return retval;
+ rv_attach_trace_probe("laxity", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_attach_trace_probe("laxity", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_attach_trace_probe("laxity", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_attach_trace_probe("laxity", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_attach_trace_probe("laxity", sched_dl_update_tp, handle_dl_update);
+ rv_attach_trace_probe("laxity", sched_switch, handle_sched_switch);
+ rv_attach_trace_probe("laxity", sched_enqueue_tp, handle_sched_enqueue);
+
+ return 0;
+}
+
+static void disable_laxity(void)
+{
+ rv_this.enabled = 0;
+
+ rv_detach_trace_probe("laxity", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_detach_trace_probe("laxity", sched_dl_server_start_tp, handle_dl_server_start);
+ rv_detach_trace_probe("laxity", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_detach_trace_probe("laxity", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_detach_trace_probe("laxity", sched_dl_update_tp, handle_dl_update);
+ rv_detach_trace_probe("laxity", sched_switch, handle_sched_switch);
+ rv_detach_trace_probe("laxity", sched_enqueue_tp, handle_sched_enqueue);
+
+ da_monitor_destroy();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "laxity",
+ .description = "deferrable servers wait for zero-laxity and run.",
+ .enable = enable_laxity,
+ .disable = disable_laxity,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_laxity(void)
+{
+ return rv_register_monitor(&rv_this, &rv_deadline);
+}
+
+static void __exit unregister_laxity(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_laxity);
+module_exit(unregister_laxity);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("laxity: deferrable servers wait for zero-laxity and run.");
diff --git a/kernel/trace/rv/monitors/laxity/laxity.h b/kernel/trace/rv/monitors/laxity/laxity.h
new file mode 100644
index 000000000000..8ea45c4207e9
--- /dev/null
+++ b/kernel/trace/rv/monitors/laxity/laxity.h
@@ -0,0 +1,140 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Automatically generated C representation of laxity automaton
+ * For further information about this format, see kernel documentation:
+ * Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#define MONITOR_NAME laxity
+
+enum states_laxity {
+ stopped_laxity,
+ idle_wait_laxity,
+ replenish_wait_laxity,
+ running_laxity,
+ zero_laxity_wait_laxity,
+ state_max_laxity,
+};
+
+#define INVALID_STATE state_max_laxity
+
+enum events_laxity {
+ dl_replenish_laxity,
+ dl_replenish_idle_laxity,
+ dl_replenish_running_laxity,
+ dl_server_resume_laxity,
+ dl_server_start_laxity,
+ dl_server_start_running_laxity,
+ dl_server_stop_laxity,
+ dl_throttle_laxity,
+ dl_update_laxity,
+ sched_switch_in_laxity,
+ event_max_laxity,
+};
+
+enum envs_laxity {
+ clk_laxity,
+ env_max_laxity,
+ env_max_stored_laxity = env_max_laxity,
+};
+
+_Static_assert(env_max_stored_laxity <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_laxity {
+ char *state_names[state_max_laxity];
+ char *event_names[event_max_laxity];
+ char *env_names[env_max_laxity];
+ unsigned char function[state_max_laxity][event_max_laxity];
+ unsigned char initial_state;
+ bool final_states[state_max_laxity];
+};
+
+static const struct automaton_laxity automaton_laxity = {
+ .state_names = {
+ "stopped",
+ "idle_wait",
+ "replenish_wait",
+ "running",
+ "zero_laxity_wait",
+ },
+ .event_names = {
+ "dl_replenish",
+ "dl_replenish_idle",
+ "dl_replenish_running",
+ "dl_server_resume",
+ "dl_server_start",
+ "dl_server_start_running",
+ "dl_server_stop",
+ "dl_throttle",
+ "dl_update",
+ "sched_switch_in",
+ },
+ .env_names = {
+ "clk",
+ },
+ .function = {
+ {
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ stopped_laxity,
+ zero_laxity_wait_laxity,
+ running_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ {
+ zero_laxity_wait_laxity,
+ idle_wait_laxity,
+ INVALID_STATE,
+ zero_laxity_wait_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ stopped_laxity,
+ INVALID_STATE,
+ zero_laxity_wait_laxity,
+ INVALID_STATE,
+ },
+ {
+ zero_laxity_wait_laxity,
+ idle_wait_laxity,
+ running_laxity,
+ replenish_wait_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ zero_laxity_wait_laxity,
+ INVALID_STATE,
+ },
+ {
+ zero_laxity_wait_laxity,
+ zero_laxity_wait_laxity,
+ running_laxity,
+ running_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ stopped_laxity,
+ replenish_wait_laxity,
+ zero_laxity_wait_laxity,
+ running_laxity,
+ },
+ {
+ zero_laxity_wait_laxity,
+ idle_wait_laxity,
+ running_laxity,
+ zero_laxity_wait_laxity,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ INVALID_STATE,
+ zero_laxity_wait_laxity,
+ INVALID_STATE,
+ },
+ },
+ .initial_state = stopped_laxity,
+ .final_states = { 1, 0, 0, 0, 0 },
+};
diff --git a/kernel/trace/rv/monitors/laxity/laxity_trace.h b/kernel/trace/rv/monitors/laxity/laxity_trace.h
new file mode 100644
index 000000000000..32580dba8f42
--- /dev/null
+++ b/kernel/trace/rv/monitors/laxity/laxity_trace.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_LAXITY
+DEFINE_EVENT(event_da_monitor_id, event_laxity,
+ TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_laxity,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_laxity,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+#endif /* CONFIG_RV_MON_LAXITY */
diff --git a/kernel/trace/rv/rv_trace.h b/kernel/trace/rv/rv_trace.h
index 1bf0f3666ee4..f1d55c39dc48 100644
--- a/kernel/trace/rv/rv_trace.h
+++ b/kernel/trace/rv/rv_trace.h
@@ -190,6 +190,8 @@ DECLARE_EVENT_CLASS(error_env_da_monitor_id,
#include <monitors/stall/stall_trace.h>
#include <monitors/nomiss/nomiss_trace.h>
#include <monitors/throttle/throttle_trace.h>
+#include <monitors/boost/boost_trace.h>
+#include <monitors/laxity/laxity_trace.h>
// Add new monitors based on CONFIG_HA_MON_EVENTS_ID here
#endif
diff --git a/tools/verification/models/deadline/boost.dot b/tools/verification/models/deadline/boost.dot
new file mode 100644
index 000000000000..aaab7d08fae6
--- /dev/null
+++ b/tools/verification/models/deadline/boost.dot
@@ -0,0 +1,48 @@
+digraph state_automaton {
+ center = true;
+ size = "7,11";
+ {node [shape = circle] "idle"};
+ {node [shape = circle] "ready"};
+ {node [shape = circle] "running"};
+ {node [shape = plaintext, style=invis, label=""] "__init_stopped"};
+ {node [shape = doublecircle] "stopped"};
+ {node [shape = circle] "stopped"};
+ {node [shape = circle] "throttled"};
+ {node [shape = circle] "throttled_running"};
+ "__init_stopped" -> "stopped";
+ "idle" [label = "idle"];
+ "idle" -> "idle" [ label = "dl_server_idle\ndl_replenish;reset(clk)\ndl_throttle" ];
+ "idle" -> "ready" [ label = "dl_server_resume" ];
+ "idle" -> "stopped" [ label = "dl_server_stop" ];
+ "idle" -> "throttled" [ label = "dl_server_resume_throttled" ];
+ "ready" [label = "ready\nclk < server_threshold_ns()"];
+ "ready" -> "idle" [ label = "dl_server_idle" ];
+ "ready" -> "ready" [ label = "sched_switch_out\ndl_server_resume_throttled\ndl_server_resume\ndl_replenish;reset(clk)" ];
+ "ready" -> "running" [ label = "sched_switch_in" ];
+ "ready" -> "stopped" [ label = "dl_server_stop" ];
+ "ready" -> "throttled" [ label = "dl_throttle" ];
+ "running" [label = "running"];
+ "running" -> "idle" [ label = "dl_server_idle" ];
+ "running" -> "ready" [ label = "sched_switch_out" ];
+ "running" -> "running" [ label = "dl_server_resume_throttled\ndl_server_resume\ndl_replenish;reset(clk)" ];
+ "running" -> "stopped" [ label = "dl_server_stop" ];
+ "running" -> "throttled_running" [ label = "dl_throttle" ];
+ "stopped" [label = "stopped", color = green3];
+ "stopped" -> "ready" [ label = "dl_server_start;reset(clk)" ];
+ "stopped" -> "stopped" [ label = "dl_server_idle\nsched_switch_out\ndl_server_resume\ndl_server_resume_throttled" ];
+ "throttled" [label = "throttled"];
+ "throttled" -> "idle" [ label = "dl_server_idle" ];
+ "throttled" -> "ready" [ label = "dl_replenish;reset(clk)" ];
+ "throttled" -> "stopped" [ label = "dl_server_stop" ];
+ "throttled" -> "throttled" [ label = "dl_throttle\ndl_server_resume_throttled" ];
+ "throttled" -> "throttled_running" [ label = "sched_switch_in" ];
+ "throttled_running" [label = "throttled_running"];
+ "throttled_running" -> "idle" [ label = "dl_server_idle" ];
+ "throttled_running" -> "running" [ label = "dl_replenish;reset(clk)" ];
+ "throttled_running" -> "throttled" [ label = "sched_switch_out" ];
+ "throttled_running" -> "throttled_running" [ label = "dl_throttle\ndl_server_resume_throttled" ];
+ { rank = min ;
+ "__init_stopped";
+ "stopped";
+ }
+}
diff --git a/tools/verification/models/deadline/laxity.dot b/tools/verification/models/deadline/laxity.dot
new file mode 100644
index 000000000000..acece40c7971
--- /dev/null
+++ b/tools/verification/models/deadline/laxity.dot
@@ -0,0 +1,37 @@
+digraph state_automaton {
+ center = true;
+ size = "7,11";
+ {node [shape = circle] "idle_wait"};
+ {node [shape = circle] "replenish_wait"};
+ {node [shape = circle] "running"};
+ {node [shape = plaintext, style=invis, label=""] "__init_stopped"};
+ {node [shape = doublecircle] "stopped"};
+ {node [shape = circle] "stopped"};
+ {node [shape = circle] "zero_laxity_wait"};
+ "__init_stopped" -> "stopped";
+ "idle_wait" [label = "idle_wait\nclk < period_ns()"];
+ "idle_wait" -> "idle_wait" [ label = "dl_replenish_idle;reset(clk)" ];
+ "idle_wait" -> "stopped" [ label = "dl_server_stop" ];
+ "idle_wait" -> "zero_laxity_wait" [ label = "dl_replenish;reset(clk)\ndl_server_resume" ];
+ "replenish_wait" [label = "replenish_wait\nclk < period_ns()"];
+ "replenish_wait" -> "idle_wait" [ label = "dl_replenish_idle;reset(clk)" ];
+ "replenish_wait" -> "replenish_wait" [ label = "dl_server_resume" ];
+ "replenish_wait" -> "running" [ label = "dl_replenish_running;reset(clk)" ];
+ "replenish_wait" -> "zero_laxity_wait" [ label = "dl_replenish;reset(clk)" ];
+ "running" [label = "running"];
+ "running" -> "replenish_wait" [ label = "dl_throttle;reset(clk)" ];
+ "running" -> "running" [ label = "sched_switch_in\ndl_server_resume\ndl_replenish_running;clk < REPLENISH_NS" ];
+ "running" -> "stopped" [ label = "dl_server_stop" ];
+ "stopped" [label = "stopped", color = green3];
+ "stopped" -> "running" [ label = "dl_server_start_running;reset(clk)" ];
+ "stopped" -> "stopped" [ label = "dl_server_resume" ];
+ "stopped" -> "zero_laxity_wait" [ label = "dl_server_start;reset(clk)" ];
+ "zero_laxity_wait" [label = "zero_laxity_wait\nclk < period_ns()"];
+ "zero_laxity_wait" -> "idle_wait" [ label = "dl_replenish_idle;reset(clk)" ];
+ "zero_laxity_wait" -> "running" [ label = "dl_replenish_running;reset(clk)" ];
+ "zero_laxity_wait" -> "zero_laxity_wait" [ label = "dl_replenish;reset(clk)\ndl_server_resume" ];
+ { rank = min ;
+ "__init_stopped";
+ "stopped";
+ }
+}
--
2.53.0
^ permalink raw reply related
* [PATCH v6 15/16] rv: Add deadline monitors
From: Gabriele Monaco @ 2026-02-25 9:51 UTC (permalink / raw)
To: linux-kernel, Steven Rostedt, Nam Cao, Juri Lelli,
Gabriele Monaco, Jonathan Corbet, Masami Hiramatsu,
linux-trace-kernel, linux-doc
Cc: Peter Zijlstra, Tomas Glozar, Clark Williams, John Kacur
In-Reply-To: <20260225095122.80683-1-gmonaco@redhat.com>
Add the deadline monitors collection to validate the deadline scheduler,
both for deadline tasks and servers.
The currently implemented monitors are:
* throttle:
validate dl entities are throttled when they use up their runtime
* nomiss:
validate dl entities run to completion before their deadiline
Cc: Peter Zijlstra <peterz@infradead.org>
Reviewed-by: Nam Cao <namcao@linutronix.de>
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
---
Notes:
V6:
* Add support for ext server in monitors
* Handle events only for supported deadline entities (future proof)
* Remove mostly unused dl_server_start from nomiss monitor
* Allow to skip runtime constraint on throttle monitor
* Consider also BATCH and IDLE as fair policies
* Cleanup and sort constraints as generated
V5:
* Do not use boosted dl_se in monitors
V4:
* Rename handle_syscall as it collides with some UM function
* Simplify idle handling on nomiss and throttle from sleeping
* Improve switch_out for servers in throttle
V3:
* Adapt models to new dl server behaviour
* Rename dl argument to dl_se in tracepoints
* Use __COUNTER__ in dl monitor syscall helpers
Documentation/trace/rv/index.rst | 1 +
Documentation/trace/rv/monitor_deadline.rst | 158 ++++++++++
kernel/trace/rv/Kconfig | 5 +
kernel/trace/rv/Makefile | 3 +
kernel/trace/rv/monitors/deadline/Kconfig | 10 +
kernel/trace/rv/monitors/deadline/deadline.c | 35 +++
kernel/trace/rv/monitors/deadline/deadline.h | 206 +++++++++++++
kernel/trace/rv/monitors/nomiss/Kconfig | 15 +
kernel/trace/rv/monitors/nomiss/nomiss.c | 287 ++++++++++++++++++
kernel/trace/rv/monitors/nomiss/nomiss.h | 123 ++++++++
.../trace/rv/monitors/nomiss/nomiss_trace.h | 19 ++
kernel/trace/rv/monitors/throttle/Kconfig | 15 +
kernel/trace/rv/monitors/throttle/throttle.c | 279 +++++++++++++++++
kernel/trace/rv/monitors/throttle/throttle.h | 116 +++++++
.../rv/monitors/throttle/throttle_trace.h | 19 ++
kernel/trace/rv/rv_trace.h | 2 +
tools/verification/models/deadline/nomiss.dot | 41 +++
.../verification/models/deadline/throttle.dot | 44 +++
18 files changed, 1378 insertions(+)
create mode 100644 Documentation/trace/rv/monitor_deadline.rst
create mode 100644 kernel/trace/rv/monitors/deadline/Kconfig
create mode 100644 kernel/trace/rv/monitors/deadline/deadline.c
create mode 100644 kernel/trace/rv/monitors/deadline/deadline.h
create mode 100644 kernel/trace/rv/monitors/nomiss/Kconfig
create mode 100644 kernel/trace/rv/monitors/nomiss/nomiss.c
create mode 100644 kernel/trace/rv/monitors/nomiss/nomiss.h
create mode 100644 kernel/trace/rv/monitors/nomiss/nomiss_trace.h
create mode 100644 kernel/trace/rv/monitors/throttle/Kconfig
create mode 100644 kernel/trace/rv/monitors/throttle/throttle.c
create mode 100644 kernel/trace/rv/monitors/throttle/throttle.h
create mode 100644 kernel/trace/rv/monitors/throttle/throttle_trace.h
create mode 100644 tools/verification/models/deadline/nomiss.dot
create mode 100644 tools/verification/models/deadline/throttle.dot
diff --git a/Documentation/trace/rv/index.rst b/Documentation/trace/rv/index.rst
index bf9962f49959..29769f06bb0f 100644
--- a/Documentation/trace/rv/index.rst
+++ b/Documentation/trace/rv/index.rst
@@ -17,3 +17,4 @@ Runtime Verification
monitor_sched.rst
monitor_rtapp.rst
monitor_stall.rst
+ monitor_deadline.rst
diff --git a/Documentation/trace/rv/monitor_deadline.rst b/Documentation/trace/rv/monitor_deadline.rst
new file mode 100644
index 000000000000..566bce4ff582
--- /dev/null
+++ b/Documentation/trace/rv/monitor_deadline.rst
@@ -0,0 +1,158 @@
+Scheduler monitors
+==================
+
+- Name: deadline
+- Type: container for multiple monitors
+- Author: Gabriele Monaco <gmonaco@redhat.com>
+
+Description
+-----------
+
+The deadline monitor is a set of specifications to describe the deadline
+scheduler behaviour. It includes monitors per scheduling entity (deadline tasks
+and servers) that work independently to verify different specifications the
+deadline scheduler should follow.
+
+Specifications
+--------------
+
+Monitor throttle
+~~~~~~~~~~~~~~~~
+
+The throttle monitor ensures deadline entities are throttled when they use up
+their runtime. Deadline tasks can be only ``running``, ``preempted`` and
+``throttled``, the runtime is enforced only in ``running`` based on an internal
+clock and the runtime value in the deadline entity.
+On systems with CPU frequency scaling or turbo boost, deadline tasks can run
+longer than their runtime as this is scaled according to the frequency. In this
+scenario, the monitor allows to skip the runtime check with the module
+parameter ``throttle.skip_runtime_check``.
+
+Servers can be also in the ``armed`` state, which represents when the
+server is consuming bandwidth in background (e.g. idle or normal tasks are
+running without any boost). From this state the server can be throttled but it
+can also use more runtime than available. A server is considered ``running``
+when it's actively boosting a task, only there the runtime is enforced. The
+server is preempted if the running task is not in the server's runqueue (e.g. a
+FIFO task for the fair server).
+Events like ``dl_armed`` and ``sched_switch_in`` can occur sequentially for
+servers since they are related to the current task (e.g. a 2 fair tasks can be
+switched in sequentially, that corresponds to multiple ``dl_armed``).
+
+Any task or server in the ``throttled`` state must leave it shortly, e.g.
+become ``preempted``::
+
+ |
+ |
+ dl_replenish;reset(clk) v
+ sched_switch_in #=========================# sched_switch_in;
+ +--------------- H H reset(clk)
+ | H H <----------------+
+ +--------------> H running H |
+ dl_throttle;reset(clk) H clk < runtime_left_ns() H |
+ +--------------------------- H H sched_switch_out |
+ | +------------------> H H -------------+ |
+ | dl_replenish;reset(clk) #=========================# | |
+ | | | ^ | |
+ v | dl_defer_arm | | |
+ +-------------------------+ | | | |
+ | throttled | | sched_switch_in;reset(clk) | |
+ | clk < THROTTLED_TIME_NS | v | | |
+ +-------------------------+ +----------------+ | |
+ | | | | sched_switch_out | |
+ | | +---------- | | -------------+ | |
+ | | dl_replenish | armed | | | |
+ | | dl_defer_arm | | <--------+ | | |
+ | | +---------> | | dl_defer_arm | | |
+ | | +----------------+ | | | |
+ | | | ^ | | | |
+ | | dl_throttle dl_replenish | | | |
+ | | v | | | | |
+ | | dl_defer_arm +-------------------+ | v v |
+ | | +---------- | | +--------------+
+ | | | | | | |
+ | | +---------> | armed_throttled | | preempted |
+ | | | | | |
+ | +-----------------> | | +--------------+
+ | dl_defer_arm +-------------------+ sched_switch_out ^ | ^
+ | | ^ dl_replenish | | |
+ | sched_switch_out dl_defer_arm +----+ | |
+ | v | | |
+ | sched_switch_out +-----------------------+ | |
+ | +-------------- | | dl_throttle; | |
+ | | | | is_constr_dl==1 | |
+ | +-------------> | preempted_throttled | <-----------------+ |
+ | | | |
+ +-----------------------> | | -- dl_replenish -----+
+ sched_switch_out +-----------------------+
+
+The value of ``runtime_left_ns()`` is directly read from the deadline entity
+and updated as the task runs. It is increased by 1 tick to account for the
+maximum delay to throttle (not valid if ``sched_feat(HRTICK_DL)`` is active).
+
+Monitor nomiss
+~~~~~~~~~~~~~~
+
+The nomiss monitor ensures dl entities get to run *and* run to completion
+before their deadline, although deferrable servers may not run. An entity is
+considered done if ``throttled``, either because it yielded or used up its
+runtime, or when it voluntarily starts ``sleeping``.
+The monitor includes a user configurable deadline threshold. If the total
+utilisation of deadline tasks is larger than 1, they are only guaranteed
+bounded tardiness. See Documentation/scheduler/sched-deadline.rst for more
+details. The threshold (module parameter ``nomiss.deadline_thresh``) can be
+configured to avoid the monitor to fail based on the acceptable tardiness in
+the system. Since ``dl_throttle`` is a valid outcome for the entity to be done,
+the minimum tardiness needs be 1 tick to consider the throttle delay, unless
+the ``HRTICK_DL`` scheduler feature is active.
+
+Servers have also an intermediate ``idle`` state, occurring as soon as no
+runnable task is available from ready or running where no timing constraint
+is applied. A server goes to sleep by stopping, there is no wakeup equivalent
+as the order of a server starting and replenishing is not defined, hence a
+server can run from sleeping without being ready::
+
+ |
+ sched_wakeup v
+ dl_replenish;reset(clk) -- #=========================#
+ | H H dl_replenish;reset(clk)
+ +-----------> H H <--------------------+
+ H H |
+ +- dl_server_stop ---- H ready H |
+ | +-----------------> H clk < DEADLINE_NS() H dl_throttle; |
+ | | H H is_defer == 1 |
+ | | sched_switch_in - H H -----------------+ |
+ | | | #=========================# | |
+ | | | | ^ | |
+ | | | dl_server_idle dl_replenish;reset(clk) | |
+ | | | v | | |
+ | | | +--------------+ | |
+ | | | +------ | | | |
+ | | | dl_server_idle | | dl_throttle | |
+ | | | | | idle | -----------------+ | |
+ | | | +-----> | | | | |
+ | | | | | | | |
+ | | | | | | | |
+ +--+--+---+--- dl_server_stop -- +--------------+ | | |
+ | | | | | ^ | | |
+ | | | | sched_switch_in dl_server_idle | | |
+ | | | | v | | | |
+ | | | | +---------- +---------------------+ | | |
+ | | | | sched_switch_in | | | | |
+ | | | | sched_wakeup | | | | |
+ | | | | dl_replenish; | running | -------+ | | |
+ | | | | reset(clk) | clk < DEADLINE_NS() | | | | |
+ | | | | +---------> | | dl_throttle | | |
+ | | | +----------------> | | | | | |
+ | | | +---------------------+ | | | |
+ | | sched_wakeup ^ sched_switch_suspend | | | |
+ v v dl_replenish;reset(clk) | dl_server_stop | | | |
+ +--------------+ | | v v v |
+ | | - sched_switch_in + | +---------------+
+ | | <---------------------+ dl_throttle +-- | |
+ | sleeping | sched_wakeup | | throttled |
+ | | -- dl_server_stop dl_server_idle +-> | |
+ | | dl_server_idle sched_switch_suspend +---------------+
+ +--------------+ <---------+ ^
+ | |
+ +------ dl_throttle;is_constr_dl == 1 || is_defer == 1 ------+
diff --git a/kernel/trace/rv/Kconfig b/kernel/trace/rv/Kconfig
index 720fbe4935f8..719cdcfb6d41 100644
--- a/kernel/trace/rv/Kconfig
+++ b/kernel/trace/rv/Kconfig
@@ -79,6 +79,11 @@ source "kernel/trace/rv/monitors/sleep/Kconfig"
# Add new rtapp monitors here
source "kernel/trace/rv/monitors/stall/Kconfig"
+source "kernel/trace/rv/monitors/deadline/Kconfig"
+source "kernel/trace/rv/monitors/nomiss/Kconfig"
+source "kernel/trace/rv/monitors/throttle/Kconfig"
+# Add new deadline monitors here
+
# Add new monitors here
config RV_REACTORS
diff --git a/kernel/trace/rv/Makefile b/kernel/trace/rv/Makefile
index 51c95e2d2da6..15a1edc8bd0f 100644
--- a/kernel/trace/rv/Makefile
+++ b/kernel/trace/rv/Makefile
@@ -18,6 +18,9 @@ obj-$(CONFIG_RV_MON_NRP) += monitors/nrp/nrp.o
obj-$(CONFIG_RV_MON_SSSW) += monitors/sssw/sssw.o
obj-$(CONFIG_RV_MON_OPID) += monitors/opid/opid.o
obj-$(CONFIG_RV_MON_STALL) += monitors/stall/stall.o
+obj-$(CONFIG_RV_MON_DEADLINE) += monitors/deadline/deadline.o
+obj-$(CONFIG_RV_MON_NOMISS) += monitors/nomiss/nomiss.o
+obj-$(CONFIG_RV_MON_THROTTLE) += monitors/throttle/throttle.o
# Add new monitors here
obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
diff --git a/kernel/trace/rv/monitors/deadline/Kconfig b/kernel/trace/rv/monitors/deadline/Kconfig
new file mode 100644
index 000000000000..38804a6ad91d
--- /dev/null
+++ b/kernel/trace/rv/monitors/deadline/Kconfig
@@ -0,0 +1,10 @@
+config RV_MON_DEADLINE
+ depends on RV
+ bool "deadline monitor"
+ help
+ Collection of monitors to check the deadline scheduler and server
+ behave according to specifications. Enable this to enable all
+ scheduler specification supported by the current kernel.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/deadline/deadline.c b/kernel/trace/rv/monitors/deadline/deadline.c
new file mode 100644
index 000000000000..45aed62c1371
--- /dev/null
+++ b/kernel/trace/rv/monitors/deadline/deadline.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+
+#define MODULE_NAME "deadline"
+
+#include "deadline.h"
+
+struct rv_monitor rv_deadline = {
+ .name = "deadline",
+ .description = "container for several deadline scheduler specifications.",
+ .enable = NULL,
+ .disable = NULL,
+ .reset = NULL,
+ .enabled = 0,
+};
+
+static int __init register_deadline(void)
+{
+ return rv_register_monitor(&rv_deadline, NULL);
+}
+
+static void __exit unregister_deadline(void)
+{
+ rv_unregister_monitor(&rv_deadline);
+}
+
+module_init(register_deadline);
+module_exit(unregister_deadline);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("deadline: container for several deadline scheduler specifications.");
diff --git a/kernel/trace/rv/monitors/deadline/deadline.h b/kernel/trace/rv/monitors/deadline/deadline.h
new file mode 100644
index 000000000000..be449c5d18c4
--- /dev/null
+++ b/kernel/trace/rv/monitors/deadline/deadline.h
@@ -0,0 +1,206 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#include <linux/kernel.h>
+#include <linux/uaccess.h>
+#include <linux/sched/deadline.h>
+#include <linux/sched/ext.h>
+#include <asm/syscall.h>
+#include <uapi/linux/sched/types.h>
+#include <trace/events/sched.h>
+
+/*
+ * Dummy values if not available
+ */
+#ifndef __NR_sched_setscheduler
+#define __NR_sched_setscheduler -__COUNTER__
+#endif
+#ifndef __NR_sched_setattr
+#define __NR_sched_setattr -__COUNTER__
+#endif
+
+/*
+ * If both have dummy values, the syscalls are not supported and we don't even
+ * need to register the handler.
+ */
+static inline bool should_skip_syscall_handle(void)
+{
+ return __NR_sched_setattr < 0 && __NR_sched_setscheduler < 0;
+}
+
+/*
+ * is_supported_type - return true if @type is supported by the deadline monitors
+ */
+static inline bool is_supported_type(u8 type)
+{
+ return type == DL_TASK || type == DL_SERVER_FAIR || type == DL_SERVER_EXT;
+}
+
+/*
+ * is_server_type - return true if @type is a supported server
+ */
+static inline bool is_server_type(u8 type)
+{
+ return is_supported_type(type) && type != DL_TASK;
+}
+
+/*
+ * Use negative numbers for the server.
+ * Currently only one fair server per CPU, may change in the future.
+ */
+#define fair_server_id(cpu) (-cpu)
+#define ext_server_id(cpu) (-cpu - num_possible_cpus())
+#define NO_SERVER_ID (-2 * num_possible_cpus())
+/*
+ * Get a unique id used for dl entities
+ *
+ * The cpu is not required for tasks as the pid is used there, if this function
+ * is called on a dl_se that for sure corresponds to a task, DL_TASK can be
+ * used in place of cpu.
+ * We need the cpu for servers as it is provided in the tracepoint and we
+ * cannot easily retrieve it from the dl_se (requires the struct rq definition).
+ */
+static inline int get_entity_id(struct sched_dl_entity *dl_se, int cpu, u8 type)
+{
+ if (dl_server(dl_se) && type != DL_TASK) {
+ if (type == DL_SERVER_FAIR)
+ return fair_server_id(cpu);
+ if (type == DL_SERVER_EXT)
+ return ext_server_id(cpu);
+ return NO_SERVER_ID;
+ }
+ return dl_task_of(dl_se)->pid;
+}
+
+/* Expand id and target as arguments for da functions */
+#define EXPAND_ID(dl_se, cpu, type) get_entity_id(dl_se, cpu, type), dl_se
+#define EXPAND_ID_TASK(tsk) get_entity_id(&tsk->dl, task_cpu(tsk), DL_TASK), &tsk->dl
+
+static inline uint8_t get_server_type(struct task_struct *tsk)
+{
+ if (tsk->policy == SCHED_NORMAL || tsk->policy == SCHED_EXT ||
+ tsk->policy == SCHED_BATCH || tsk->policy == SCHED_IDLE)
+ return task_is_scx_enabled(tsk) ? DL_SERVER_EXT : DL_SERVER_FAIR;
+ return DL_OTHER;
+}
+
+static inline int extract_params(struct pt_regs *regs, long id, struct task_struct **p)
+{
+ size_t size = offsetof(struct sched_attr, sched_nice);
+ struct sched_attr __user *uattr, attr;
+ int new_policy = -1, ret;
+ unsigned long args[6];
+ pid_t pid;
+
+ switch (id) {
+ case __NR_sched_setscheduler:
+ syscall_get_arguments(current, regs, args);
+ pid = args[0];
+ new_policy = args[1];
+ break;
+ case __NR_sched_setattr:
+ syscall_get_arguments(current, regs, args);
+ pid = args[0];
+ uattr = (void *)args[1];
+ /*
+ * Just copy up to sched_flags, we are not interested after that
+ */
+ ret = copy_struct_from_user(&attr, size, uattr, size);
+ if (ret)
+ return ret;
+ if (attr.sched_flags & SCHED_FLAG_KEEP_POLICY)
+ return -EINVAL;
+ new_policy = attr.sched_policy;
+ break;
+ default:
+ return -EINVAL;
+ }
+ if (!pid)
+ *p = current;
+ else {
+ /*
+ * Required for find_task_by_vpid, make sure the caller doesn't
+ * need to get_task_struct().
+ */
+ guard(rcu)();
+ *p = find_task_by_vpid(pid);
+ if (unlikely(!*p))
+ return -EINVAL;
+ }
+
+ return new_policy & ~SCHED_RESET_ON_FORK;
+}
+
+/* Helper functions requiring DA/HA utilities */
+#ifdef RV_MON_TYPE
+
+/*
+ * get_fair_server - get the fair server associated to a task
+ *
+ * If the task is a boosted task, the server is available in the task_struct,
+ * otherwise grab the dl entity saved for the CPU where the task is enqueued.
+ * This function assumes the task is enqueued somewhere.
+ */
+static inline struct sched_dl_entity *get_server(struct task_struct *tsk, u8 type)
+{
+ if (tsk->dl_server && get_server_type(tsk) == type)
+ return tsk->dl_server;
+ if (type == DL_SERVER_FAIR)
+ return da_get_target_by_id(fair_server_id(task_cpu(tsk)));
+ if (type == DL_SERVER_EXT)
+ return da_get_target_by_id(ext_server_id(task_cpu(tsk)));
+ return NULL;
+}
+
+/*
+ * Initialise monitors for all tasks and pre-allocate the storage for servers.
+ * This is necessary since we don't have access to the servers here and
+ * allocation can cause deadlocks from their tracepoints. We can only fill
+ * pre-initialised storage from there.
+ */
+static inline int init_storage(bool skip_tasks)
+{
+ struct task_struct *g, *p;
+ int cpu;
+
+ for_each_possible_cpu(cpu) {
+ if (!da_create_empty_storage(fair_server_id(cpu)))
+ goto fail;
+ if (IS_ENABLED(CONFIG_SCHED_CLASS_EXT) &&
+ !da_create_empty_storage(ext_server_id(cpu)))
+ goto fail;
+ }
+
+ if (skip_tasks)
+ return 0;
+
+ read_lock(&tasklist_lock);
+ for_each_process_thread(g, p) {
+ if (p->policy == SCHED_DEADLINE) {
+ if (!da_create_storage(EXPAND_ID_TASK(p), NULL)) {
+ read_unlock(&tasklist_lock);
+ goto fail;
+ }
+ }
+ }
+ read_unlock(&tasklist_lock);
+ return 0;
+
+fail:
+ da_monitor_destroy();
+ return -ENOMEM;
+}
+
+static void __maybe_unused handle_newtask(void *data, struct task_struct *task, u64 flags)
+{
+ /* Might be superfluous as tasks are not started with this policy.. */
+ if (task->policy == SCHED_DEADLINE)
+ da_create_storage(EXPAND_ID_TASK(task), NULL);
+}
+
+static void __maybe_unused handle_exit(void *data, struct task_struct *p, bool group_dead)
+{
+ if (p->policy == SCHED_DEADLINE)
+ da_destroy_storage(get_entity_id(&p->dl, DL_TASK, DL_TASK));
+}
+
+#endif
diff --git a/kernel/trace/rv/monitors/nomiss/Kconfig b/kernel/trace/rv/monitors/nomiss/Kconfig
new file mode 100644
index 000000000000..e1886c3a0dd9
--- /dev/null
+++ b/kernel/trace/rv/monitors/nomiss/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_NOMISS
+ depends on RV
+ depends on HAVE_SYSCALL_TRACEPOINTS
+ depends on RV_MON_DEADLINE
+ default y
+ select HA_MON_EVENTS_ID
+ bool "nomiss monitor"
+ help
+ Monitor to ensure dl entities run to completion before their deadiline.
+ This monitor is part of the deadline monitors collection.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/nomiss/nomiss.c b/kernel/trace/rv/monitors/nomiss/nomiss.c
new file mode 100644
index 000000000000..bb69c574c5b2
--- /dev/null
+++ b/kernel/trace/rv/monitors/nomiss/nomiss.c
@@ -0,0 +1,287 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+
+#define MODULE_NAME "nomiss"
+
+#include <uapi/linux/sched/types.h>
+#include <trace/events/syscalls.h>
+#include <trace/events/sched.h>
+#include <trace/events/task.h>
+#include <rv_trace.h>
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define HA_TIMER_TYPE HA_TIMER_WHEEL
+/* The start condition is on sched_switch, it's dangerous to allocate there */
+#define DA_SKIP_AUTO_ALLOC
+typedef struct sched_dl_entity *monitor_target;
+#include "nomiss.h"
+#include <rv/ha_monitor.h>
+#include <monitors/deadline/deadline.h>
+
+/*
+ * User configurable deadline threshold. If the total utilisation of deadline
+ * tasks is larger than 1, they are only guaranteed bounded tardiness. See
+ * Documentation/scheduler/sched-deadline.rst for more details.
+ * The minimum tardiness without sched_feat(HRTICK_DL) is 1 tick to accommodate
+ * for throttle enforced on the next tick.
+ */
+static u64 deadline_thresh = TICK_NSEC;
+module_param(deadline_thresh, ullong, 0644);
+#define DEADLINE_NS(ha_mon) (ha_get_target(ha_mon)->dl_deadline + deadline_thresh)
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_nomiss env, u64 time_ns)
+{
+ if (env == clk_nomiss)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ else if (env == is_constr_dl_nomiss)
+ return !dl_is_implicit(ha_get_target(ha_mon));
+ else if (env == is_defer_nomiss)
+ return ha_get_target(ha_mon)->dl_defer;
+ return ENV_INVALID_VALUE;
+}
+
+static void ha_reset_env(struct ha_monitor *ha_mon, enum envs_nomiss env, u64 time_ns)
+{
+ if (env == clk_nomiss)
+ ha_reset_clk_ns(ha_mon, env, time_ns);
+}
+
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == ready_nomiss)
+ return ha_check_invariant_ns(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == running_nomiss)
+ return ha_check_invariant_ns(ha_mon, clk_nomiss, time_ns);
+ return true;
+}
+
+static inline void ha_convert_inv_guard(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == next_state)
+ return;
+ if (curr_state == ready_nomiss)
+ ha_inv_to_guard(ha_mon, clk_nomiss, DEADLINE_NS(ha_mon), time_ns);
+ else if (curr_state == running_nomiss)
+ ha_inv_to_guard(ha_mon, clk_nomiss, DEADLINE_NS(ha_mon), time_ns);
+}
+
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ bool res = true;
+
+ if (curr_state == ready_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == ready_nomiss && event == dl_throttle_nomiss)
+ res = ha_get_env(ha_mon, is_defer_nomiss, time_ns) == 1ull;
+ else if (curr_state == idle_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == running_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == sleeping_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ else if (curr_state == sleeping_nomiss && event == dl_throttle_nomiss)
+ res = ha_get_env(ha_mon, is_constr_dl_nomiss, time_ns) == 1ull ||
+ ha_get_env(ha_mon, is_defer_nomiss, time_ns) == 1ull;
+ else if (curr_state == throttled_nomiss && event == dl_replenish_nomiss)
+ ha_reset_env(ha_mon, clk_nomiss, time_ns);
+ return res;
+}
+
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == curr_state && event != dl_replenish_nomiss)
+ return;
+ if (next_state == ready_nomiss)
+ ha_start_timer_ns(ha_mon, clk_nomiss, DEADLINE_NS(ha_mon), time_ns);
+ else if (next_state == running_nomiss)
+ ha_start_timer_ns(ha_mon, clk_nomiss, DEADLINE_NS(ha_mon), time_ns);
+ else if (curr_state == ready_nomiss)
+ ha_cancel_timer(ha_mon);
+ else if (curr_state == running_nomiss)
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_convert_inv_guard(ha_mon, curr_state, event, next_state, time_ns);
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+static void handle_dl_replenish(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_supported_type(type))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_replenish_nomiss);
+}
+
+static void handle_dl_throttle(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_supported_type(type))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_throttle_nomiss);
+}
+
+static void handle_dl_server_stop(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ /*
+ * This isn't the standard use of da_handle_start_run_event since this
+ * event cannot only occur from the initial state.
+ * It is fine to use here because it always brings to a known state and
+ * the fact we "pretend" the transition starts from the initial state
+ * has no side effect.
+ */
+ if (is_supported_type(type))
+ da_handle_start_run_event(EXPAND_ID(dl_se, cpu, type), dl_server_stop_nomiss);
+}
+
+static inline void handle_server_switch(struct task_struct *next, int cpu, u8 type)
+{
+ struct sched_dl_entity *dl_se = get_server(next, type);
+
+ if (dl_se && is_idle_task(next))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_server_idle_nomiss);
+}
+
+static void handle_sched_switch(void *data, bool preempt,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ int cpu = task_cpu(next);
+
+ if (prev_state != TASK_RUNNING && !preempt && prev->policy == SCHED_DEADLINE)
+ da_handle_event(EXPAND_ID_TASK(prev), sched_switch_suspend_nomiss);
+ if (next->policy == SCHED_DEADLINE)
+ da_handle_start_run_event(EXPAND_ID_TASK(next), sched_switch_in_nomiss);
+
+ /*
+ * The server is available in next only if the next task is boosted,
+ * otherwise we need to retrieve it.
+ * Here the server continues in the state running/armed until actually
+ * stopped, this works since we continue expecting a throttle.
+ */
+ if (next->dl_server)
+ da_handle_start_event(EXPAND_ID(next->dl_server, cpu,
+ get_server_type(next)),
+ sched_switch_in_nomiss);
+ else {
+ handle_server_switch(next, cpu, DL_SERVER_FAIR);
+ if (IS_ENABLED(CONFIG_SCHED_CLASS_EXT))
+ handle_server_switch(next, cpu, DL_SERVER_EXT);
+ }
+}
+
+static void handle_sys_enter(void *data, struct pt_regs *regs, long id)
+{
+ struct task_struct *p;
+ int new_policy = -1;
+
+ new_policy = extract_params(regs, id, &p);
+ if (new_policy < 0 || new_policy == p->policy)
+ return;
+ if (p->policy == SCHED_DEADLINE)
+ da_reset(EXPAND_ID_TASK(p));
+ else if (new_policy == SCHED_DEADLINE)
+ da_create_or_get(EXPAND_ID_TASK(p));
+}
+
+static void handle_sched_wakeup(void *data, struct task_struct *tsk)
+{
+ if (tsk->policy == SCHED_DEADLINE)
+ da_handle_event(EXPAND_ID_TASK(tsk), sched_wakeup_nomiss);
+}
+
+static int enable_nomiss(void)
+{
+ int retval;
+
+ retval = da_monitor_init();
+ if (retval)
+ return retval;
+
+ retval = init_storage(false);
+ if (retval)
+ return retval;
+ rv_attach_trace_probe("nomiss", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_attach_trace_probe("nomiss", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_attach_trace_probe("nomiss", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_attach_trace_probe("nomiss", sched_switch, handle_sched_switch);
+ rv_attach_trace_probe("nomiss", sched_wakeup, handle_sched_wakeup);
+ if (!should_skip_syscall_handle())
+ rv_attach_trace_probe("nomiss", sys_enter, handle_sys_enter);
+ rv_attach_trace_probe("nomiss", task_newtask, handle_newtask);
+ rv_attach_trace_probe("nomiss", sched_process_exit, handle_exit);
+
+ return 0;
+}
+
+static void disable_nomiss(void)
+{
+ rv_this.enabled = 0;
+
+ /* Those are RCU writers, detach earlier hoping to close a bit faster */
+ rv_detach_trace_probe("nomiss", task_newtask, handle_newtask);
+ rv_detach_trace_probe("nomiss", sched_process_exit, handle_exit);
+ if (!should_skip_syscall_handle())
+ rv_detach_trace_probe("nomiss", sys_enter, handle_sys_enter);
+
+ rv_detach_trace_probe("nomiss", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_detach_trace_probe("nomiss", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_detach_trace_probe("nomiss", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_detach_trace_probe("nomiss", sched_switch, handle_sched_switch);
+ rv_detach_trace_probe("nomiss", sched_wakeup, handle_sched_wakeup);
+
+ da_monitor_destroy();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "nomiss",
+ .description = "dl entities run to completion before their deadiline.",
+ .enable = enable_nomiss,
+ .disable = disable_nomiss,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_nomiss(void)
+{
+ return rv_register_monitor(&rv_this, &rv_deadline);
+}
+
+static void __exit unregister_nomiss(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_nomiss);
+module_exit(unregister_nomiss);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("nomiss: dl entities run to completion before their deadiline.");
diff --git a/kernel/trace/rv/monitors/nomiss/nomiss.h b/kernel/trace/rv/monitors/nomiss/nomiss.h
new file mode 100644
index 000000000000..3d1b436194d7
--- /dev/null
+++ b/kernel/trace/rv/monitors/nomiss/nomiss.h
@@ -0,0 +1,123 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Automatically generated C representation of nomiss automaton
+ * For further information about this format, see kernel documentation:
+ * Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#define MONITOR_NAME nomiss
+
+enum states_nomiss {
+ ready_nomiss,
+ idle_nomiss,
+ running_nomiss,
+ sleeping_nomiss,
+ throttled_nomiss,
+ state_max_nomiss,
+};
+
+#define INVALID_STATE state_max_nomiss
+
+enum events_nomiss {
+ dl_replenish_nomiss,
+ dl_server_idle_nomiss,
+ dl_server_stop_nomiss,
+ dl_throttle_nomiss,
+ sched_switch_in_nomiss,
+ sched_switch_suspend_nomiss,
+ sched_wakeup_nomiss,
+ event_max_nomiss,
+};
+
+enum envs_nomiss {
+ clk_nomiss,
+ is_constr_dl_nomiss,
+ is_defer_nomiss,
+ env_max_nomiss,
+ env_max_stored_nomiss = is_constr_dl_nomiss,
+};
+
+_Static_assert(env_max_stored_nomiss <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_nomiss {
+ char *state_names[state_max_nomiss];
+ char *event_names[event_max_nomiss];
+ char *env_names[env_max_nomiss];
+ unsigned char function[state_max_nomiss][event_max_nomiss];
+ unsigned char initial_state;
+ bool final_states[state_max_nomiss];
+};
+
+static const struct automaton_nomiss automaton_nomiss = {
+ .state_names = {
+ "ready",
+ "idle",
+ "running",
+ "sleeping",
+ "throttled",
+ },
+ .event_names = {
+ "dl_replenish",
+ "dl_server_idle",
+ "dl_server_stop",
+ "dl_throttle",
+ "sched_switch_in",
+ "sched_switch_suspend",
+ "sched_wakeup",
+ },
+ .env_names = {
+ "clk",
+ "is_constr_dl",
+ "is_defer",
+ },
+ .function = {
+ {
+ ready_nomiss,
+ idle_nomiss,
+ sleeping_nomiss,
+ throttled_nomiss,
+ running_nomiss,
+ INVALID_STATE,
+ ready_nomiss,
+ },
+ {
+ ready_nomiss,
+ idle_nomiss,
+ sleeping_nomiss,
+ throttled_nomiss,
+ running_nomiss,
+ INVALID_STATE,
+ INVALID_STATE,
+ },
+ {
+ running_nomiss,
+ idle_nomiss,
+ sleeping_nomiss,
+ throttled_nomiss,
+ running_nomiss,
+ sleeping_nomiss,
+ running_nomiss,
+ },
+ {
+ ready_nomiss,
+ sleeping_nomiss,
+ sleeping_nomiss,
+ throttled_nomiss,
+ running_nomiss,
+ INVALID_STATE,
+ ready_nomiss,
+ },
+ {
+ ready_nomiss,
+ throttled_nomiss,
+ INVALID_STATE,
+ throttled_nomiss,
+ INVALID_STATE,
+ throttled_nomiss,
+ throttled_nomiss,
+ },
+ },
+ .initial_state = ready_nomiss,
+ .final_states = { 1, 0, 0, 0, 0 },
+};
diff --git a/kernel/trace/rv/monitors/nomiss/nomiss_trace.h b/kernel/trace/rv/monitors/nomiss/nomiss_trace.h
new file mode 100644
index 000000000000..42e7efaca4e7
--- /dev/null
+++ b/kernel/trace/rv/monitors/nomiss/nomiss_trace.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_NOMISS
+DEFINE_EVENT(event_da_monitor_id, event_nomiss,
+ TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_nomiss,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_nomiss,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+#endif /* CONFIG_RV_MON_NOMISS */
diff --git a/kernel/trace/rv/monitors/throttle/Kconfig b/kernel/trace/rv/monitors/throttle/Kconfig
new file mode 100644
index 000000000000..d9bd2dc903cd
--- /dev/null
+++ b/kernel/trace/rv/monitors/throttle/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_THROTTLE
+ depends on RV
+ depends on HAVE_SYSCALL_TRACEPOINTS
+ depends on RV_MON_DEADLINE
+ default y
+ select HA_MON_EVENTS_ID
+ bool "throttle monitor"
+ help
+ Monitor to ensure dl entities are throttled when they use up their runtime.
+ This monitor is part of the deadline monitors collection.
+
+ For further information, see:
+ Documentation/trace/rv/monitor_deadline.rst
diff --git a/kernel/trace/rv/monitors/throttle/throttle.c b/kernel/trace/rv/monitors/throttle/throttle.c
new file mode 100644
index 000000000000..08169686d1ae
--- /dev/null
+++ b/kernel/trace/rv/monitors/throttle/throttle.c
@@ -0,0 +1,279 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/ftrace.h>
+#include <linux/tracepoint.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/rv.h>
+#include <rv/instrumentation.h>
+
+#define MODULE_NAME "throttle"
+
+#include <uapi/linux/sched/types.h>
+#include <trace/events/syscalls.h>
+#include <trace/events/sched.h>
+#include <trace/events/task.h>
+#include <rv_trace.h>
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define HA_TIMER_TYPE HA_TIMER_WHEEL
+/* The start condition is on sched_switch, it's dangerous to allocate there */
+#define DA_SKIP_AUTO_ALLOC
+typedef struct sched_dl_entity *monitor_target;
+#include "throttle.h"
+#include <rv/ha_monitor.h>
+#include <monitors/deadline/deadline.h>
+
+#define THROTTLED_TIME_NS TICK_NSEC
+/* with sched_feat(HRTICK_DL) the threshold can be lower */
+#define RUNTIME_THRESH TICK_NSEC
+/*
+ * On systems with CPU frequency scaling or turbo boost, deadline tasks can run
+ * longer than their runtime as this is scaled according to the frequency. As a
+ * result, this constraint cannot work.
+ */
+static bool skip_runtime_check;
+module_param(skip_runtime_check, bool, 0644);
+
+static inline u64 runtime_left_ns(struct ha_monitor *ha_mon)
+{
+ return ha_get_target(ha_mon)->runtime + RUNTIME_THRESH;
+}
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_throttle env, u64 time_ns)
+{
+ if (env == clk_throttle)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ else if (env == is_constr_dl_throttle)
+ return !dl_is_implicit(ha_get_target(ha_mon));
+ return ENV_INVALID_VALUE;
+}
+
+static void ha_reset_env(struct ha_monitor *ha_mon, enum envs_throttle env, u64 time_ns)
+{
+ if (env == clk_throttle)
+ ha_reset_clk_ns(ha_mon, env, time_ns);
+}
+
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == running_throttle && !skip_runtime_check)
+ return ha_check_invariant_ns(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == throttled_throttle)
+ return ha_check_invariant_ns(ha_mon, clk_throttle, time_ns);
+ return true;
+}
+
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ bool res = true;
+
+ if (curr_state == running_throttle && event == dl_replenish_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == running_throttle && event == dl_throttle_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == armed_throttle && event == sched_switch_in_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == preempted_throttle && event == dl_throttle_throttle)
+ res = ha_get_env(ha_mon, is_constr_dl_throttle, time_ns) == 1ull;
+ else if (curr_state == preempted_throttle && event == sched_switch_in_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ else if (curr_state == throttled_throttle && event == dl_replenish_throttle)
+ ha_reset_env(ha_mon, clk_throttle, time_ns);
+ return res;
+}
+
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == curr_state && event != dl_replenish_throttle)
+ return;
+ if (next_state == running_throttle && !skip_runtime_check)
+ ha_start_timer_ns(ha_mon, clk_throttle, runtime_left_ns(ha_mon), time_ns);
+ else if (next_state == throttled_throttle)
+ ha_start_timer_ns(ha_mon, clk_throttle, THROTTLED_TIME_NS, time_ns);
+ else if (curr_state == running_throttle)
+ ha_cancel_timer(ha_mon);
+ else if (curr_state == throttled_throttle)
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+static void handle_dl_replenish(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_supported_type(type))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_replenish_throttle);
+}
+
+static void handle_dl_throttle(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_supported_type(type))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_throttle_throttle);
+}
+
+static void handle_dl_server_stop(void *data, struct sched_dl_entity *dl_se,
+ int cpu, uint8_t type)
+{
+ if (is_supported_type(type))
+ da_handle_start_run_event(EXPAND_ID(dl_se, cpu, type), sched_switch_out_throttle);
+}
+
+static inline void handle_server_switch(struct task_struct *next, int cpu, u8 type)
+{
+ struct sched_dl_entity *dl_se = get_server(next, type);
+
+ if (!dl_se)
+ return;
+ if (get_server_type(next) == type || is_idle_task(next))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_defer_arm_throttle);
+ else
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), sched_switch_out_throttle);
+}
+
+static void handle_sched_switch(void *data, bool preempt,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ int cpu = task_cpu(next);
+
+ if (prev->policy == SCHED_DEADLINE)
+ da_handle_event(EXPAND_ID_TASK(prev), sched_switch_out_throttle);
+ if (next->policy == SCHED_DEADLINE)
+ da_handle_start_event(EXPAND_ID_TASK(next), sched_switch_in_throttle);
+
+ /*
+ * The server is available in next only if the next task is boosted,
+ * otherwise we need to retrieve it.
+ * Here the server continues in the state running/armed until actually
+ * stopped, this works since we continue expecting a throttle.
+ */
+ if (next->dl_server) {
+ da_handle_start_event(EXPAND_ID(next->dl_server, cpu,
+ get_server_type(next)),
+ sched_switch_in_throttle);
+ } else {
+ handle_server_switch(next, cpu, DL_SERVER_FAIR);
+ if (IS_ENABLED(CONFIG_SCHED_CLASS_EXT))
+ handle_server_switch(next, cpu, DL_SERVER_EXT);
+ }
+}
+
+static void handle_sched_enqueue(void *data, struct task_struct *tsk, int cpu)
+{
+ struct sched_dl_entity *dl_se = NULL;
+ uint8_t type = get_server_type(tsk);
+
+ if (is_server_type(type))
+ dl_se = get_server(tsk, type);
+ /*
+ * An enqueue is counted as server arming only in case of a change in
+ * scheduler where the task is moved to another scheduler's runqueue.
+ */
+ if (dl_se && task_is_running(tsk) && sched_task_on_rq(tsk))
+ da_handle_event(EXPAND_ID(dl_se, cpu, type), dl_defer_arm_throttle);
+}
+
+static void handle_sys_enter(void *data, struct pt_regs *regs, long id)
+{
+ struct task_struct *p;
+ int new_policy = -1;
+
+ new_policy = extract_params(regs, id, &p);
+ if (new_policy < 0 || new_policy == p->policy)
+ return;
+ if (p->policy == SCHED_DEADLINE)
+ da_reset(EXPAND_ID_TASK(p));
+ else if (new_policy == SCHED_DEADLINE)
+ da_create_or_get(EXPAND_ID_TASK(p));
+}
+
+static int enable_throttle(void)
+{
+ int retval;
+
+ retval = da_monitor_init();
+ if (retval)
+ return retval;
+
+ retval = init_storage(false);
+ if (retval)
+ return retval;
+ rv_attach_trace_probe("throttle", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_attach_trace_probe("throttle", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_attach_trace_probe("throttle", sched_enqueue_tp, handle_sched_enqueue);
+ rv_attach_trace_probe("throttle", sched_switch, handle_sched_switch);
+ if (!should_skip_syscall_handle())
+ rv_attach_trace_probe("throttle", sys_enter, handle_sys_enter);
+ rv_attach_trace_probe("throttle", task_newtask, handle_newtask);
+ rv_attach_trace_probe("throttle", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_attach_trace_probe("throttle", sched_process_exit, handle_exit);
+
+ return 0;
+}
+
+static void disable_throttle(void)
+{
+ rv_this.enabled = 0;
+
+ /* Those are RCU writers, detach earlier hoping to close a bit faster */
+ rv_detach_trace_probe("throttle", task_newtask, handle_newtask);
+ rv_detach_trace_probe("throttle", sched_process_exit, handle_exit);
+ if (!should_skip_syscall_handle())
+ rv_detach_trace_probe("throttle", sys_enter, handle_sys_enter);
+
+ rv_detach_trace_probe("throttle", sched_dl_replenish_tp, handle_dl_replenish);
+ rv_detach_trace_probe("throttle", sched_dl_throttle_tp, handle_dl_throttle);
+ rv_detach_trace_probe("throttle", sched_dl_server_stop_tp, handle_dl_server_stop);
+ rv_detach_trace_probe("throttle", sched_enqueue_tp, handle_sched_enqueue);
+ rv_detach_trace_probe("throttle", sched_switch, handle_sched_switch);
+
+ da_monitor_destroy();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "throttle",
+ .description = "throttle dl entities when they use up their runtime.",
+ .enable = enable_throttle,
+ .disable = disable_throttle,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_throttle(void)
+{
+ return rv_register_monitor(&rv_this, &rv_deadline);
+}
+
+static void __exit unregister_throttle(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_throttle);
+module_exit(unregister_throttle);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Gabriele Monaco <gmonaco@redhat.com>");
+MODULE_DESCRIPTION("throttle: throttle dl entities when they use up their runtime.");
diff --git a/kernel/trace/rv/monitors/throttle/throttle.h b/kernel/trace/rv/monitors/throttle/throttle.h
new file mode 100644
index 000000000000..3ab6d73280d2
--- /dev/null
+++ b/kernel/trace/rv/monitors/throttle/throttle.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Automatically generated C representation of throttle automaton
+ * For further information about this format, see kernel documentation:
+ * Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#define MONITOR_NAME throttle
+
+enum states_throttle {
+ running_throttle,
+ armed_throttle,
+ armed_throttled_throttle,
+ preempted_throttle,
+ preempted_throttled_throttle,
+ throttled_throttle,
+ state_max_throttle,
+};
+
+#define INVALID_STATE state_max_throttle
+
+enum events_throttle {
+ dl_defer_arm_throttle,
+ dl_replenish_throttle,
+ dl_throttle_throttle,
+ sched_switch_in_throttle,
+ sched_switch_out_throttle,
+ event_max_throttle,
+};
+
+enum envs_throttle {
+ clk_throttle,
+ is_constr_dl_throttle,
+ env_max_throttle,
+ env_max_stored_throttle = is_constr_dl_throttle,
+};
+
+_Static_assert(env_max_stored_throttle <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_throttle {
+ char *state_names[state_max_throttle];
+ char *event_names[event_max_throttle];
+ char *env_names[env_max_throttle];
+ unsigned char function[state_max_throttle][event_max_throttle];
+ unsigned char initial_state;
+ bool final_states[state_max_throttle];
+};
+
+static const struct automaton_throttle automaton_throttle = {
+ .state_names = {
+ "running",
+ "armed",
+ "armed_throttled",
+ "preempted",
+ "preempted_throttled",
+ "throttled",
+ },
+ .event_names = {
+ "dl_defer_arm",
+ "dl_replenish",
+ "dl_throttle",
+ "sched_switch_in",
+ "sched_switch_out",
+ },
+ .env_names = {
+ "clk",
+ "is_constr_dl",
+ },
+ .function = {
+ {
+ armed_throttle,
+ running_throttle,
+ throttled_throttle,
+ running_throttle,
+ preempted_throttle,
+ },
+ {
+ armed_throttle,
+ armed_throttle,
+ armed_throttled_throttle,
+ running_throttle,
+ preempted_throttle,
+ },
+ {
+ armed_throttled_throttle,
+ armed_throttle,
+ INVALID_STATE,
+ INVALID_STATE,
+ preempted_throttled_throttle,
+ },
+ {
+ armed_throttle,
+ preempted_throttle,
+ preempted_throttled_throttle,
+ running_throttle,
+ preempted_throttle,
+ },
+ {
+ armed_throttled_throttle,
+ preempted_throttle,
+ INVALID_STATE,
+ INVALID_STATE,
+ preempted_throttled_throttle,
+ },
+ {
+ armed_throttled_throttle,
+ running_throttle,
+ INVALID_STATE,
+ INVALID_STATE,
+ preempted_throttled_throttle,
+ },
+ },
+ .initial_state = running_throttle,
+ .final_states = { 1, 0, 0, 0, 0, 0 },
+};
diff --git a/kernel/trace/rv/monitors/throttle/throttle_trace.h b/kernel/trace/rv/monitors/throttle/throttle_trace.h
new file mode 100644
index 000000000000..7e376d3aec60
--- /dev/null
+++ b/kernel/trace/rv/monitors/throttle/throttle_trace.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_THROTTLE
+DEFINE_EVENT(event_da_monitor_id, event_throttle,
+ TP_PROTO(int id, char *state, char *event, char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_throttle,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_throttle,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+#endif /* CONFIG_RV_MON_THROTTLE */
diff --git a/kernel/trace/rv/rv_trace.h b/kernel/trace/rv/rv_trace.h
index 9e8072d863a2..1bf0f3666ee4 100644
--- a/kernel/trace/rv/rv_trace.h
+++ b/kernel/trace/rv/rv_trace.h
@@ -188,6 +188,8 @@ DECLARE_EVENT_CLASS(error_env_da_monitor_id,
);
#include <monitors/stall/stall_trace.h>
+#include <monitors/nomiss/nomiss_trace.h>
+#include <monitors/throttle/throttle_trace.h>
// Add new monitors based on CONFIG_HA_MON_EVENTS_ID here
#endif
diff --git a/tools/verification/models/deadline/nomiss.dot b/tools/verification/models/deadline/nomiss.dot
new file mode 100644
index 000000000000..fd1ea6bf2509
--- /dev/null
+++ b/tools/verification/models/deadline/nomiss.dot
@@ -0,0 +1,41 @@
+digraph state_automaton {
+ center = true;
+ size = "7,11";
+ {node [shape = circle] "idle"};
+ {node [shape = plaintext, style=invis, label=""] "__init_ready"};
+ {node [shape = doublecircle] "ready"};
+ {node [shape = circle] "ready"};
+ {node [shape = circle] "running"};
+ {node [shape = circle] "sleeping"};
+ {node [shape = circle] "throttled"};
+ "__init_ready" -> "ready";
+ "idle" [label = "idle"];
+ "idle" -> "idle" [ label = "dl_server_idle" ];
+ "idle" -> "ready" [ label = "dl_replenish;reset(clk)" ];
+ "idle" -> "running" [ label = "sched_switch_in" ];
+ "idle" -> "sleeping" [ label = "dl_server_stop" ];
+ "idle" -> "throttled" [ label = "dl_throttle" ];
+ "ready" [label = "ready\nclk < DEADLINE_NS()", color = green3];
+ "ready" -> "idle" [ label = "dl_server_idle" ];
+ "ready" -> "ready" [ label = "sched_wakeup\ndl_replenish;reset(clk)" ];
+ "ready" -> "running" [ label = "sched_switch_in" ];
+ "ready" -> "sleeping" [ label = "dl_server_stop" ];
+ "ready" -> "throttled" [ label = "dl_throttle;is_defer == 1" ];
+ "running" [label = "running\nclk < DEADLINE_NS()"];
+ "running" -> "idle" [ label = "dl_server_idle" ];
+ "running" -> "running" [ label = "dl_replenish;reset(clk)\nsched_switch_in\nsched_wakeup" ];
+ "running" -> "sleeping" [ label = "sched_switch_suspend\ndl_server_stop" ];
+ "running" -> "throttled" [ label = "dl_throttle" ];
+ "sleeping" [label = "sleeping"];
+ "sleeping" -> "ready" [ label = "sched_wakeup\ndl_replenish;reset(clk)" ];
+ "sleeping" -> "running" [ label = "sched_switch_in" ];
+ "sleeping" -> "sleeping" [ label = "dl_server_stop\ndl_server_idle" ];
+ "sleeping" -> "throttled" [ label = "dl_throttle;is_constr_dl == 1 || is_defer == 1" ];
+ "throttled" [label = "throttled"];
+ "throttled" -> "ready" [ label = "dl_replenish;reset(clk)" ];
+ "throttled" -> "throttled" [ label = "sched_switch_suspend\nsched_wakeup\ndl_server_idle\ndl_throttle" ];
+ { rank = min ;
+ "__init_ready";
+ "ready";
+ }
+}
diff --git a/tools/verification/models/deadline/throttle.dot b/tools/verification/models/deadline/throttle.dot
new file mode 100644
index 000000000000..c24fc3f291a9
--- /dev/null
+++ b/tools/verification/models/deadline/throttle.dot
@@ -0,0 +1,44 @@
+digraph state_automaton {
+ center = true;
+ size = "7,11";
+ {node [shape = circle] "armed"};
+ {node [shape = circle] "armed_throttled"};
+ {node [shape = circle] "preempted"};
+ {node [shape = circle] "preempted_throttled"};
+ {node [shape = plaintext, style=invis, label=""] "__init_running"};
+ {node [shape = doublecircle] "running"};
+ {node [shape = circle] "running"};
+ {node [shape = circle] "throttled"};
+ "__init_running" -> "running";
+ "armed" [label = "armed"];
+ "armed" -> "armed" [ label = "dl_replenish\ndl_defer_arm" ];
+ "armed" -> "armed_throttled" [ label = "dl_throttle" ];
+ "armed" -> "preempted" [ label = "sched_switch_out" ];
+ "armed" -> "running" [ label = "sched_switch_in;reset(clk)" ];
+ "armed_throttled" [label = "armed_throttled"];
+ "armed_throttled" -> "armed" [ label = "dl_replenish" ];
+ "armed_throttled" -> "armed_throttled" [ label = "dl_defer_arm" ];
+ "armed_throttled" -> "preempted_throttled" [ label = "sched_switch_out" ];
+ "preempted" [label = "preempted"];
+ "preempted" -> "armed" [ label = "dl_defer_arm" ];
+ "preempted" -> "preempted" [ label = "dl_replenish\nsched_switch_out" ];
+ "preempted" -> "preempted_throttled" [ label = "dl_throttle;is_constr_dl == 1" ];
+ "preempted" -> "running" [ label = "sched_switch_in;reset(clk)" ];
+ "preempted_throttled" [label = "preempted_throttled"];
+ "preempted_throttled" -> "armed_throttled" [ label = "dl_defer_arm" ];
+ "preempted_throttled" -> "preempted" [ label = "dl_replenish" ];
+ "preempted_throttled" -> "preempted_throttled" [ label = "sched_switch_out" ];
+ "running" [label = "running\nclk < runtime_left_ns()", color = green3];
+ "running" -> "armed" [ label = "dl_defer_arm" ];
+ "running" -> "preempted" [ label = "sched_switch_out" ];
+ "running" -> "running" [ label = "dl_replenish;reset(clk)\nsched_switch_in" ];
+ "running" -> "throttled" [ label = "dl_throttle;reset(clk)" ];
+ "throttled" [label = "throttled\nclk < THROTTLED_TIME_NS"];
+ "throttled" -> "armed_throttled" [ label = "dl_defer_arm" ];
+ "throttled" -> "preempted_throttled" [ label = "sched_switch_out" ];
+ "throttled" -> "running" [ label = "dl_replenish;reset(clk)" ];
+ { rank = min ;
+ "__init_running";
+ "running";
+ }
+}
--
2.53.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox