From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B19B822F767 for ; Sat, 17 Oct 2026 13:50:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792245007; cv=none; b=mQ+26MQIcNjeLkUOja8T84n96QK+GCc3VJjNOM7HUMy9aadvlFz1dDsdW/D/P+MysIyKx8Wyr/jJQTAWj00xePwCl6k6V3lPjVXo3CJdV+F/wfpRqfrCuo77buiKL+SZhu8av+d5p/BP6YaQQ14N2TBV+7SMs4AkO0Nxh2Px1Kg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792245007; c=relaxed/simple; bh=ppqkr0bsf0K0V04GxiQDG4ok2jrlvyIwjZ48w69zt0w=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=X73SiUGSMr0juw2BO2Oe/s0xRcTgtkD2fNCh8seQTmLp889nQGiHsE36j05/2/NdX3YmgLgRnbpT8x+JYUuCKaGvvf3XTppl+BEcpGVEKKZgz0iGCzdhlwz3+8eWtoDMhKu4X2RLLY2YXkrejF/nJ/yaXm+5PrsRtfxkWMUf5Zg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=tHFn/Mfr; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tHFn/Mfr" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-487049569b6so394250f8f.1 for ; Sat, 17 Oct 2026 06:50:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1792245004; x=1792849804; darn=vger.kernel.org; h=cc:to:message-id:content-transfer-encoding:content-type :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to:content-type; bh=x/7sTjM+j/xtn89k/aOm3H9ejU9nyAE+tKqITVTZM6c=; b=tHFn/Mfr8BwuOSfFmtY8rw1SmsVvBkcLon/qJPN5QS268rx/RUOrN9BeAeYzJE6QR+ dIjf8oWyMZwPsmiG828x03pEd12r2rayfZf+4Vby1COldkl7Oy1BvA/MQ5BEzPzm3SrS yE10SUuvRtDWJO6TjV2JDLQi+RugeuICSgOHf9cuXHYoD07213J5vpibVCQ6SN3YrNZ6 ffCskmXK+GXO3j8L2WXNGRNPv0Eshh9N5XCQhCOmBjnQmr9x/+1TH8poWMyND4X31Lz1 iEDbpIg+1/j88qQ5FLvJ5XFbiPCVGtRGtfDe2GD9DrtrQh0fTgLgIUUyW/FxgVklmRxK A6bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792245004; x=1792849804; h=cc:to:message-id:content-transfer-encoding:content-type :mime-version:subject:date:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to:content-type; bh=x/7sTjM+j/xtn89k/aOm3H9ejU9nyAE+tKqITVTZM6c=; b=whcKDuo9yJ2TfRxjYVwOSz5lK4n9BhuRsyi/j2SpZECw8mbP0YPGuodFonWdmAjVWx CZKZMNkqHedlAm5YFD93XUxOcfYZrcd/XJ3qFyqKze9VsBxXXDi4+hHY45bpcv70w68Y PhnHMLjo07oIMCdzPFXXR7kfBLY0cYJ8OXrB6ibHPMvqCu7l/fXjMFnGxwHrcmbQT+vH 8KhEV9etdwPMvf9CnLVHDTReYE1KaF13+d0sRjLY8U1FK1tMZzWvC+dh+kR/0Sct7wJy n1CKfWrv2Jr/B5/L5cfH5W1ZuIA1nXTdyNJGeguVsZ13BpDYJYhTID8OHnDCNEdazJiP RxrQ== X-Forwarded-Encrypted: i=1; AKwUvBxP7VF4xW+TCMU3lVno/HCPKZopV6nycubfpa0tlEA1JC0gNSdJEirpWQDhXpZinu73pd6hXeVWcTrRk4jU24jOs5A=@vger.kernel.org X-Gm-Message-State: AFq9FYLpsw/IFMseK7W7Wu8NTFgCLxQuMxF7joKRIFCJ1g2qimm06zH1 UTsykAVv6KQaAw+u4SKHWfUSOkke4WBBwozGope4T9mtPC0GQWyPsEd8 X-Gm-Gg: AYBFou11uTqo6eIVlN1X0tv27rZ1B32gzEAf3vEdoMqY7UviLC1bIJxflJz9vkqEmYo 3tkU03MX4DCLz5z+ta9/pl96mTg3D1NQIz9JHP0oKdLs/xifmgi4FbZhueTvtOVRJ1KTzLnHQw1 4bEW4EWbfR4VvFzaZ5YMU/Al5hP3TI1HRgeqVvyUKcie//v8ZOhOZSe4EYOeMklc0IVhJBLV4hc XmYIv+Za6CaS6Vu+cIxZhi7B+Q6RIVPyx3LkOx0j1M8WeMPhXTcF3o6uwx5UR98uV4sJjMoCEUv hZ+5IZngFxQKjWyJlCV79NLRMA5UWdacjAScJMKS11bpbemFKI9+oQ9KBDwplfRPM6g4Imz5rHw jTf2SiH2AibfOOp7a6YspgjVXaWIKYHWS1Yj941zw+C3yY3ZKPuZ6ZdV5w/uoqGoP+OtgQLTeXb 5DxJ6bqz6NhmX1V7n+O47bbIFaRSMil6XTtgj515HJVRkDKW92mROz2R8wkEK5MqFmT9CCGJToP GEH+7qcQ8hVXPlPU5gcXZbpgmws08RtXe9/70T6mwY= X-Received: by 2002:a05:6000:60b:b0:487:1662:cf4c with SMTP id ffacd0b85a97d-48e0797ffddmr5174562f8f.7.1792245003468; Sat, 17 Oct 2026 06:50:03 -0700 (PDT) Received: from [127.0.1.1] (89-65-152-218.dynamic.play.pl. [89.65.152.218]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-48dfd4c0894sm10294478f8f.12.2026.10.17.06.50.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 17 Oct 2026 06:50:02 -0700 (PDT) From: Roman 'Hedin' Storozhenko Date: Sat, 17 Oct 2026 15:49:45 +0200 Subject: [PATCH v5] riscv: mm: Trace TLB flush requests Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261017-tlb_tracepoint-v5-1-256fb5393a7f@gmail.com> X-B4-Tracking: v=1; b=H4sIAAAAAAAC/33PTQrCMBCG4atI1kaSSWKjK+8hIvmZtAFtS1qKU np3UxdSLLr8Bp4XZiQdpogdOW5GknCIXWzqPNR2Q1xl6hJp9HkTYLBnGg60v9lrn4zDtol1T7W UnCljFTsAyahNGOLjHTxf8q5i1zfp+e4PfL7+TA2ccuqDN+ixwELyU3k38bZzzZ3MqQEWXLAVh 8xROK0dSB+U/ebiwznjasXFzDGI/I2wdi++ufzPZeYBhDZgvQ4FLvk0TS/8CM1aaQEAAA== X-Change-ID: 20260829-tlb_tracepoint-844105ab5092 To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers Cc: Jisheng Zhang , =?utf-8?q?Radim_Kr=C4=8Dm=C3=A1=C5=99?= , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Roman 'Hedin' Storozhenko X-Mailer: b4 0.14.3 Make RISC-V TLB flush requests observable together with the architecture-specific context available when the request reaches the RISC-V implementation. The generic tlb:tlb_flush event provides reason and page-count information. At the RISC-V implementation level, additional context is available, including the invalidation range and stride, ASID, target CPU mask, and invalidation scope. This context is useful for debugging and workload analysis. The target CPU mask shows the address-space CPU footprint involved in an invalidation, while range and stride distinguish requests at different mapping granularities. The ASID and scope help correlate requests with MM, scheduler, IPI, and firmware activity. Add riscv_tlb:riscv_tlb_flush_request in flush_tlb_all() and __flush_tlb_range(). Record start, size, stride, ASID (or the no-ASID sentinel), whether an mm is associated with the request, the target CPU mask and its weight, and the derived scope. Record the complete target mask because CPU identity cannot be reconstructed from a count. Calculate its weight from the recorded bitmap so both fields agree even if mm_cpumask() changes during recording. The event describes an invalidation request before dispatch. It does not report whether a remote flush uses SBI RFENCE or Linux cross-CPU coordination, since that choice is effectively established during initialization rather than made for each request. The address field (start) may contain kernel virtual addresses for flushes involving kernel mappings. Require perf tracepoint permissions even for counting-only events, since perf filters could otherwise reveal address information through event counts. Tested on QEMU virt with OpenSBI using local and shared-mm mprotect()/munmap() workloads, exercising single, range, and address-space invalidations. A MADV_PAGEOUT reclaim workload exercised global flushes. All four scope values were observed. Confirmed that trace-cmd records the event and that perf can count it with sufficient privileges. With perf_event_paranoid=0, unprivileged attempts to count the event using perf were denied, as intended. Built with and without CONFIG_PERF_EVENTS. Signed-off-by: Roman 'Hedin' Storozhenko --- Add a RISC-V tracepoint for observing TLB invalidation requests before they are dispatched by the architecture-specific implementation. The generic tlb:tlb_flush event describes TLB flush activity using reason and page-count information. The RISC-V event records additional context, including the invalidation range and stride, ASID, target CPU mask and its weight, and derived request scope. This context helps correlate TLB invalidation activity with MM, scheduler, IPI, and firmware activity. The event describes the request rather than the execution backend. SBI RFENCE versus Linux cross-CPU coordination is effectively determined at initialization rather than selected independently for each request. The target CPU mask weight is derived from the recorded bitmap to keep both fields consistent under concurrent mm_cpumask() updates. Since the address field (start) may contain kernel virtual addresses, the event requires perf tracepoint permissions even for counting-only access. Tested on QEMU virt with OpenSBI using local and shared-mm mprotect()/munmap() workloads. A MADV_PAGEOUT reclaim workload on QEMU virt with APLIC+IMSIC exercised global flushes. All four scope values -- single, range, address-space, and all -- were observed. Confirmed trace-cmd recording and privileged perf counting. With perf_event_paranoid=0, unprivileged attempts to count the event using perf were denied, as intended. The kernel was built with and without CONFIG_PERF_EVENTS. --- Changes in v5: - Move the architecture-specific trace event definition under arch/riscv/, as requested by Steven Rostedt. - Calculate target_mask_weight from the recorded CPU mask to keep both fields consistent under concurrent updates. - Require perf tracepoint permissions for counting-only access, since the event may expose kernel virtual addresses. - Link to v4: https://lore.kernel.org/r/20261015-tlb_tracepoint-v4-1-f238a2bd8f7e@gmail.com Changes in v4: - Drop unrelated whitespace-only changes in arch/riscv/mm/tlbflush.c. - Link to v3: https://lore.kernel.org/all/20261015-tlb_tracepoint-v3-1-eef305a3bb63@gmail.com/ Changes in v3: - Rebase on current RISC-V for-next. - Thanks to Jisheng Zhang for pointing out that SBI RFENCE versus Linux cross-CPU coordination is effectively a system-wide choice established during initialization rather than a per-request decision. - Drop the local/SBI RFENCE/cross-CPU path field. - Rename the event from riscv_tlb_flush_path to riscv_tlb_flush_request and refocus it on the semantic TLB invalidation request. - Emit one request event independently of the mechanism subsequently used to perform the invalidation. - Address checkpatch.pl --strict alignment issues. - Link to v2: https://lore.kernel.org/r/20260830-tlb_tracepoint-v2-1-e3c88c24df5b@gmail.com Changes in v2: - Use trace_call__riscv_tlb_flush_path() after the explicit trace_riscv_tlb_flush_path_enabled() check to avoid a second tracepoint static-key test, as suggested by Steven Rostedt. - Link to v1: https://lore.kernel.org/r/20260829-tlb_tracepoint-v1-1-dfdaede7e741@gmail.com --- arch/riscv/include/asm/riscv_tlb.h | 103 +++++++++++++++++++++++++++++++++++++ arch/riscv/mm/tlbflush.c | 45 ++++++++++++++-- 2 files changed, 144 insertions(+), 4 deletions(-) diff --git a/arch/riscv/include/asm/riscv_tlb.h b/arch/riscv/include/asm/riscv_tlb.h new file mode 100644 index 000000000000..2c438f6bc0d4 --- /dev/null +++ b/arch/riscv/include/asm/riscv_tlb.h @@ -0,0 +1,103 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM riscv_tlb + +#if !defined(_TRACE_RISCV_TLB_H) || defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_RISCV_TLB_H + +#include +#include +#include + +#ifndef _TRACE_RISCV_TLB_ENUMS +#define _TRACE_RISCV_TLB_ENUMS + +enum riscv_tlb_flush_scope { + RISCV_TLB_FLUSH_SCOPE_SINGLE, + RISCV_TLB_FLUSH_SCOPE_RANGE, + RISCV_TLB_FLUSH_SCOPE_ADDRESS_SPACE, + RISCV_TLB_FLUSH_SCOPE_ALL, +}; + +#endif /* _TRACE_RISCV_TLB_ENUMS */ + +TRACE_DEFINE_ENUM(RISCV_TLB_FLUSH_SCOPE_SINGLE); +TRACE_DEFINE_ENUM(RISCV_TLB_FLUSH_SCOPE_RANGE); +TRACE_DEFINE_ENUM(RISCV_TLB_FLUSH_SCOPE_ADDRESS_SPACE); +TRACE_DEFINE_ENUM(RISCV_TLB_FLUSH_SCOPE_ALL); + +#define show_riscv_tlb_flush_scope(scope) \ + __print_symbolic(scope, \ + { RISCV_TLB_FLUSH_SCOPE_SINGLE, "single" }, \ + { RISCV_TLB_FLUSH_SCOPE_RANGE, "range" }, \ + { RISCV_TLB_FLUSH_SCOPE_ADDRESS_SPACE, "address-space" }, \ + { RISCV_TLB_FLUSH_SCOPE_ALL, "all" }) + +/* + * Record the invalidation request received by the RISC-V architecture code + * + * The target CPU mask represents the CPUs Linux intends to cover for the + * request. It can be correlated with per-CPU activity, but does not describe + * which harts ultimately performed an invalidation. + * + * The ASID is hardware-visible and may be reused. It must not be treated as a + * persistent identifier for an mm. + * + * The stride describes the invalidation granularity supplied to the RISC-V + * implementation. + * + * The event records the invalidation request before it is dispatched. It does + * not describe the mechanism used to execute the request. + * + */ +TRACE_EVENT(riscv_tlb_flush_request, + TP_PROTO(unsigned long start, unsigned long size, + unsigned long stride, unsigned long asid, bool has_mm, + const struct cpumask *cmask, + enum riscv_tlb_flush_scope scope), + + TP_ARGS(start, size, stride, asid, has_mm, cmask, scope), + + TP_STRUCT__entry(__field(unsigned long, start) + __field(unsigned long, size) + __field(unsigned long, stride) + __field(unsigned long, asid) + __field(bool, has_mm) + __field(unsigned int, target_mask_weight) + __cpumask(target_cpus) + __field(u8, scope) + ), + + TP_fast_assign(__entry->start = start; + __entry->size = size; + __entry->stride = stride; + __entry->asid = asid; + __entry->has_mm = has_mm; + __assign_cpumask(target_cpus, cpumask_bits(cmask)); + __entry->target_mask_weight = cpumask_weight((const struct cpumask *) + __get_cpumask(target_cpus)); + __entry->scope = scope; + ), + + TP_printk("start=%#lx size=%#lx stride=%#lx asid=%#lx has_mm=%d target_mask_weight=%u target_cpus=%s scope=%s", + __entry->start, __entry->size, __entry->stride, + __entry->asid, __entry->has_mm, + __entry->target_mask_weight, __get_cpumask(target_cpus), + show_riscv_tlb_flush_scope(__entry->scope)) +); + +#ifdef CONFIG_PERF_EVENTS +TRACE_EVENT_PERF_PERM(riscv_tlb_flush_request, + perf_allow_tracepoint()); +#endif + +#endif /* _TRACE_RISCV_TLB_H */ + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE + +#define TRACE_INCLUDE_PATH asm +#define TRACE_INCLUDE_FILE riscv_tlb + +/* This part must be outside protection. */ +#include diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 962db300a166..f66348a9b999 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -9,6 +9,9 @@ #include #include +#define CREATE_TRACE_POINTS +#include + #define has_svinval() riscv_has_extension_unlikely(RISCV_ISA_EXT_SVINVAL) /* @@ -63,6 +66,32 @@ void local_flush_tlb_kernel_range(unsigned long start, unsigned long end) local_flush_tlb_range_asid(start, end - start, PAGE_SIZE, FLUSH_TLB_NO_ASID); } +static enum riscv_tlb_flush_scope +riscv_tlb_get_flush_scope(unsigned long size, unsigned long stride, bool has_mm) +{ + if (size == FLUSH_TLB_MAX_SIZE) + return has_mm ? RISCV_TLB_FLUSH_SCOPE_ADDRESS_SPACE : + RISCV_TLB_FLUSH_SCOPE_ALL; + + return size <= stride ? RISCV_TLB_FLUSH_SCOPE_SINGLE : + RISCV_TLB_FLUSH_SCOPE_RANGE; +} + +static __always_inline void +riscv_tlb_trace_flush_request(const struct cpumask *cmask, unsigned long start, + unsigned long size, unsigned long stride, + unsigned long asid, bool has_mm) +{ + enum riscv_tlb_flush_scope scope; + + if (!trace_riscv_tlb_flush_request_enabled()) + return; + + scope = riscv_tlb_get_flush_scope(size, stride, has_mm); + trace_call__riscv_tlb_flush_request(start, size, stride, asid, has_mm, + cmask, scope); +} + static void __ipi_flush_tlb_all(void *info) { local_flush_tlb_all(); @@ -70,12 +99,18 @@ static void __ipi_flush_tlb_all(void *info) void flush_tlb_all(void) { - if (num_online_cpus() < 2) + riscv_tlb_trace_flush_request(cpu_online_mask, 0, + FLUSH_TLB_MAX_SIZE, 0, + FLUSH_TLB_NO_ASID, false); + + if (num_online_cpus() < 2) { local_flush_tlb_all(); - else if (riscv_use_sbi_for_rfence()) - sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); - else + } else if (riscv_use_sbi_for_rfence()) { + sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, + FLUSH_TLB_NO_ASID); + } else { on_each_cpu(__ipi_flush_tlb_all, NULL, 1); + } } struct flush_tlb_range_data { @@ -105,6 +140,8 @@ static void __flush_tlb_range(struct mm_struct *mm, cpu = get_cpu(); + riscv_tlb_trace_flush_request(cmask, start, size, stride, asid, !!mm); + /* Check if the TLB flush needs to be sent to other CPUs. */ if (cpumask_any_but(cmask, cpu) >= nr_cpu_ids) { local_flush_tlb_range_asid(start, size, stride, asid); --- base-commit: a5ae1bf65cee84bcc121d659e7e22a767853f5f5 change-id: 20260829-tlb_tracepoint-844105ab5092 Best regards, -- Roman 'Hedin' Storozhenko